Minutes for the TERAPIX meeting 10th January 2014 Present: Patrick, Henry, Olivier, Nawfel, Fred.
Review CASU "nebulosity filtering" (Nawfel, Olivier, Patrick) : http://casu.ast.cam.ac.uk/publications/nebulosity-filter
Continue/Finish Pagani processing (testing of new methods) - Nawfel
CFHTLS-Deep - New background subtraction for De Lapparent : put the data on the new cluster and remove background from individual images. (Nawfel, Patrick)
Test condor_transfer (HJMCC)
Mount fcix11 in the rack (Fred, Patrick)
Mount the SSD disk in youpix (Fred)
Prepare the migrations of the computers in the new cluster with the computer Team (Patrick)
Cleaning old mixes/fcixes (Patrick)
Organize data for the table backup (CFHTLS, etc).
Ask for redmine TERAPIX pages.
Actions for last Terameeting :
Henry finds out what the CASU algorithm is to do sky subtraction for nebula
Nawfel will update the SR82 doc with the correct zero-point.
Patrick will copy VIKING data to TERAPIX machines so that Nawfel can start processing them.
Patrick will send a mail to teralist listing what has been backuped
Patrick will investigate what data can be removed from the machines for accounts which are no longer active. (all) old machines will be isolated, backed-up and turned off.
Fred will send a mail to the IAP concerning the old TERAPIX machines and asking if anyone is interested in them (after any useful hardware has been removed).
Fred will back up CFHTLS releases, PI data on external 4TB disks.
PI projects :
Karun 13AD92 The release mail has been resent. Apparently Karun did not get the first mail sent by Nawfel.
Pagani Several new methods are beeing tested to correct for the large scale variations on the stack :
compute the stripes on OFF image and apply on ON images (not working).
Try a new scaling of the sky variations with the sky mean value (ongoing)
redo the nebulosity mask / try CASU nebulosity filtering.
deLapparent Not much progress since 2 months. Processing was stopped due to hardware problems. The data should be copied to the new cluster and processed these.
UltraVista Nothing to been done now. The new images are still at ESO ongoing the validation process.
VIKING The download at CASU was impossible due to maintenance of the release server. This should be fixed in the coming week(s). The central data of W1 should be downloaded (on the new cluster) as soon as the CASU server is up again.
Strip82 Everything has been delivered. The data were used on a public website for a classification by non scientist geeks. Henry complained that TERAPIX was not credited, this should be fixed.
Scamp has been tested on all machines. a speedup factor of two has been estimated comparing to mix31.
MPI code was run during several hours using all available cores.
condor was tested (successfully) on all nodes (without condor_transfer though).
Migration of computers from TERAPIX to TERAPIX2 The baseline for the migration should be :
for the fcixes : make a backup on fcix11 through the 10GB network, unload the raid modules before installing the system.
for the mixes : physically unplug the disks from the backplane.