Teraminutes for meeting of Friday April 23rd 2004

Present: Chiara, Gerard, Yannick, Mireille, Henry, Fred, Emmanuel, Laurent, Jean-Christophe.

Chiara Marmo was introduced to us this week. Chiara is a postdoc at Saclay who will be here for one month, in order to learn how things work at Terapix. She will focus on cluster analysis and detection the W1/D1 fields. She will start working on D1 with Henry.

Scamp (Emmanuel):

 Photometric calibration step demonstrated, appears to work. For the moment, the photometric solution is computed assuming that all CCDs are locked together. The photometric part more or less works (photometric images are identified by flag T/F in the header).

 Emmanuel will add an option to set the airmass in the output image to 0 and magnitude to 30. Several questions raised by Mireille, Laurent, Henry to clarify the way SCAMP will be used. The astrometric solution still has a problem, although the linear astrometric solution seems to be okay.

 Waiting now for the remaining bugs in the astrometric part to be finished. Emmanuel will try to improve the stability of the astrometric solution before he leaves for Naples. A prototype should be delivered next Monday to Mireille and Henry for local tests, and next Tuesday to Laurent for more global robustness tests in the pipeline context.

Skywatcher (JC):

 Can now plot CCDs on the sky based on images extracted from a local database. Demo shown, which can plot the CRVALS on the sky of all objects on the sky.

 Important milestone towards the the first prototype delivery to Terapix users.

QualityFITS (Fred):

 Some minor corrections made in the code. Some problems found in some images, but when QualityFITS was run again on these images the problems disappear (this must come from a non-initialised variable somewhere).

 More seriously, there are 50-80 images which have been graded 'C' because of a problem in the CRVAL for one of the master files, which makes it impossible to Swarp the the weight maps for the image display (However, Fred notes that the actual weight maps created are fine).

 These "C" graded images will be processed once all images put in cluster disks be QualityFITS-ed.

Quality assessment tools (Laurent):

 Completeness tool...completed, based on a script by supplied Henry. This works now. Should take around  15 minutes per image. Will run on each of the images, as well as on each of the 1 deg2 sections from the wide survey.
 The other primary quality tools will use the flag produced by QualityFITS to separate galaxies, compact objects and saturated objects.

Sending new data to CADC (Laurent, Fred):

 All output from QualityFITS will be sent to CADC, with the exception of postscript plots. Before this can be done, the images with CRVAL problems will be re-processed, and the output filenames will be changed to conform with what's written in the ICD.

DBclient2 (Fred, Gerard)

 Testix fixed (it was really a problem with the network cable) and ready for tests on dbclient2.

Panier (Gerard).

 In two weeks it should be ready. It will allow the users to stack data using the web interface (no work carried out this for several months now).

Since we need to pipeline process all 03A and 03B megacam data, the panier is now an urgent goal.

Avocat (Gerard):

 XML output demonstrated. Not tested yet on single Megacam CCD nor big MEF files; currently only tested on very small images.

 UCDs not installed yet. Avocat will be delivered to CADC for validation.

Data processing (Henry, Mireille, Fred):

 D1 (Henry) For the images 0.9'' 84x520, around 12hrs of integration in total, Henry works on that.

 Wide survey (Mireille). It's not clear that astrometrix works for the wide pre-survey data, in many cases the -linfix solution is actually better than the -global solution. In some cases where there is no overlap at all the astrometric solution is actually better! Some tests will be made again, leaving out problem images, in the hope of making a photometric solution, which has not been tested yet. Too much background processes going on going on pix9 and mix2.

 It is important to make mix1 avaliable to Mireille. Fred will clone the mix2 configuration on mix1 next week.

Defectix (Anthony):

 Can detect haloes, real problem right now is that we detect too many features, for example big bright galaxies or stars. Will try to add satellite trail detection. Anthony comes back from Italy on the 7th May.

Panorapix (Fred): no progress.

Hardware (Fred):

Science cluster, cosmix (Fred:)

 All nodes constructed, added to cosmix, raid-5 fixed, passwords are back. Replication of nodes is completely unstable, don't know what the problem is.

 Ganglia: still not working fully on the cluster.

 Will buy a 24 port gigabit switch (1000eur). Still discussion about exchanging with the gigabit switch from the iap, but this could take too long.

 Buying new machine for the cluster: we don't have enough disk space or processing power available. Need more capacity in the short term, and a solution which minimises the amount of system adminstration: not ready for an Opteron quad-pro solution yet, because not all software is ready yet. Price/performance ratio is better for opterons with respect to 32 bit Athlons, but in 64 bit mode the opterons are difficult to integrate into the cluster.

 One possible short term solution: buy an opteron and install a 32 bit o/s. Has to be installed in the cluster. Fred will also make some tests to see if kernel 2.6 works can be installed on some of the older nodes (potentially big gains in i/o if this works). Athlon solution costs,  8000eur, Opteron 9000eur. Summary: have to decide based on what will work rapidly, with the minimum of work. Delay if possible buying 'heavy duty' opteron solution for the summer.

Top priority for Fred: clone mix2 on mix1.

 /raid1 on mix2: doesn't work. Cable changed, but still problem. Now suspect it is a problem with a disk? Will buy new cables and possibly try changing the disk with a spare.

Next meeting: Friday 30th April 2004.