Dear All, This is the summary of Terapix activities over the period Dec. 12, 2008 to Jan. 11, 2009. Please let me know if points are missing or need clarifications. I wish you a very happy new year! A galette is planned after the meeting. Amicalement, Yannick {{0. Specific Yuliana training and setup: Yuliana}} Welcome, Yuliana! MegaCam PI data processing: - Training QualityFITS and Condor on Hudson's data set. - Developing `fcard' wrapper for getting FITS header information and storing the output in a single table. - Problems starting Condor from mix15, running QualityFITS directly. Mix15 failure during data processing, 90% of the jobs successfully done. Data transfered to mix17 and the remaining tasks completed. No data lost. - Writing and testing script for monitoring data processing progress. Encountered mismatch between images in the input directory and in the `goqfits08AC24-Hudson-octo08.condor' script (to be clarified with Yannick). WIRCam data processing (with Patrick): - developing procedure for getting masks out of saturated pixels and pixels affected by crowded-field sky subtraction problems. - 191 (MEF) masks produced. Other: - computer setup, software installation - getting started with MissFITS, WeightWatcher & Python {{I. T0005 and T0006: Yannick, Mathias M.}} T0005 data processing with YOUPI: Most of the 12167 images have been processed by QualityFITS using YOUPI's QualityFITS-in plugin. More than 99% of T0005 data have been processed successfully. All results are available at http://clix.iap.fr/youpi/results/ T0005 masking: all masks delivered for T0005 were in WCS but the polygons of the 4 boudaries. Yannick transform all deep+wide accordingly. T0006: Yannick compiled all images that will be part of CFHTLS Wide T0006. The compilation includes the survey+pre-survey and 237 short photometric validated exposures. The list have to be archived and considered as the baseline for T0006. All survey+presurvey images have been graded by Yannick already and should be added to the YOUPI data base. {{II. Processing Megacam PI: Patrick, Yannick, Mathias S.}} - Further investigations by Mathias S. of R with PI data -as I already said for large datasets R is not very suitable as its very slow in reading in data. - Together with L. Girardi, Mathias S. put it the MEGACAM filter curves in the Padua isochrones with the goal to be able to calculate for any stellar population with a certain age and metallicity the MEGACAM colours. The same will be done for SDSS too -so that one can also can get synthetical transformations between SDSS-MEGACAM. However, first we have to check the theoretical isochrones with different data sets. {{III. Processing WIRCam : Patrick, Henry}} - Download of data from run 07BC23 by Henry (PI Mohaddesseh Azimlu). We were contacted by the PI to process his WIRCAM run. Unfortunately, the run was removed by CFHT as it's more than three months since it was processed. However, I was able to successfully download this data from CADC and it is currently in /data/fcix4/raid/hjmcc/07BC23 . There are 134 p and 134 s image cubes. I note that the CADC interface is a bit of a pain to use because there is no way to select just the p and the s images, one must also select the g and the o ones too (and then delete them afterwards). - WIRCam operation (Patrick) HUDSON : - Re-stacking the hudon fields with the new global (WIRCam+MEGACAM) astrometry. - Check of the astrometry - Release of the data (both MEGACAM and WIRCAM) on ftpix. FLAGUEY : - New PI request. M16 famous fields of the pillars of creation. The problem is that the OFF field that was observed to create the sky background is a completely crowded field. The PI said he tried to find an empty field !!! - QFITS on all images. - Creation of the sky background on the crowded fields. Modification and recompilation of swarp and sextractor to handle weight maps of very crowded fields. - SCAMP results are fine on the skysubstacted images. - Saturation masks created with Yuliana's script. QFITS has to be rerun on the data using those masks. BARMBY : - Test of a modification in the skysubstraction script (remove entire chips from the sky computation). It worked well on one field and has to be applied on the rest of the data. BOUVIER : - Reduction resumed after the script creating the saturation mask has been tested. The saturated stars are now rejected from the catalog which should improve the scamp astrometry. - SCAMP has improved but on one field, a star is so bright that nothing can be done (one chip is completely saturated and the reflections mask all objects on the other chips). - Stack is ongoing for the other fields. {{IV. YOUPI development and misc: Mathias M., Greg.}} Greg and Mathias (M.) have adopted two new popular Javascript libraries (both released under the GPL) that extend current browsers' Javascript implementation, allowing us to write cleaner and more maintainable code. Those libraries are named 'prototypejs' and 'scriptaculous' and are both available at http://www.prototypejs.org/ and http://script.aculo.us/ respectively. Most of Mathias's work for the last 2 weeks has consisted in code refactoring and optimization. YOUPI works faster, uses less memory and is better designed in some parts. Many bugs have been fixed too. In order to ease navigation between thumbnail images on the results page, users can click on any image in the list to open some kind of easy-to-navigate slideshow previewer. For example, go to the 'Processing Results' page at http://clix.iap.fr:8000/youpi/results/ then select an item on the left (in the 'Processing History' left panel). This will load the processing results on the right side of the screen. Next, click a thumbnail image in the list to see the slideshow effect. Once opened, you can navigate through the image list with usual keyboard keys (left and right arrows for previous or next image respectively) or by moving the mouse cursor on either side of the border. This is a quite useful feature to quickly view generated bitmaps for a processing. Finally, the SWARP plugin is beeing worked on. The user interface is almost set (http://clix.iap.fr:8000/youpi/processing/swarp/) and a live demo should be possible for the next Tera-meeting. A new user account has been created by Mathias (M.) - and sent by email - for Yuliana so that she can log into YOUPI. {{V. Developpement processing: Patrick, Fred }} - Modification of the script used to correct for the bleeding (vertical pattern) of very bright sources : masking of the sourced before applying the median. - Sky background median level added to the header (usefull to estimation the actual saturation level on skysubstracted images) through the MINIBACKGROUND checkimage. - QFITS has been modified to include saturation masks. The objects will have a flag of 32 in the final catalogs. - Fred modified QFITS to avoid some bugs met by Yuliana {{VI. Hardware/system/cluster Fred:}} - HARDWARE: - last CPUs from agorus changed to B3 version (mix17, mix18). - the new machines have been installed in the rack with the help of Patrick, and wired. - mix15 is back up, crippled (with only 4GB of memory, 2 DIMMs)... Its motherboard and 6 DIMMs probably need to be changed. No news yet from agorus/unika. -SYSTEM: - investigation on the attack on the cluster: an escalation to root privilege was possible, and has been awkwardly tried, without evidence of success, but confirming the need for complete (from scratch) re-installation - investigation of scientific linux as replacement of gentoo for the cluster {{VII. Divers (All):}} - The VISTA telescope has been further delayed, which means that it is unlikely that significant amounts of Ultravista data will arrive at TERAPIX before the end of 2009. [Precision from Yannick: ESO did not accept the VISTA telescope during the early commisionning steps which then delayed the plans of all VISTA surveys.] - Yannick sent the proposal for Terapix funds at CNRS before Xmas. We (CSAA of CNRS/INSU) met and discussed them last week (all CNRS/INSU proposals). Terapix got 100% of its requests (33kEuros in top priority). Yannick sent also the proposals for Terapix to PNCG. PNCG did not meet yet.