Terapix Star Formation Region IC 1396, © 2001 CFHT
End-to-end operation for PI data processing
Article
by YME - Updated June 5th, 2007

The following article summarises a meeting between Mireille, Henry and Yannick (Jan. 24, 2007). It describes all PI data processing steps that should be listed in the PI operation control boxes to for each PI data set queued at Terapix.

-  0. Request from a PI sent by email to Terapix. Can be received through any channel (terapix or personal email to any of us)

-  1. Email forwarded to teralist with subject label "[PI]" (made by Henry or the person at Terapix who received the request. If Henry not here, Yannick will do it).

-  2. Henry sends an email to the PI, to Kanoa Withington (kanoa@cfht.hawaii.edu) at CFHT and to CADC (cadchd2@nrc-cnrc.gc.ca ; jj.kavelaars@nrc-cnrc.gc.ca ; david.schade@nrc.ca) : (i) ask the PI the RUNID, (ii) ask the PI to send an email to CFHT/CADC in order to give terapix authorisation to get its data (*p.fits, masterfiles and log, if any), (iii) ask the PI a description of the processing and data products needed, spectifications (seeing limits, etc..) and whether it is urgent. More information here

-  3. Henry and Yannick examine the amount of data, the urgency, the potential technical difficulties and the date of request, then set priority in the queue. Fill in the RUNID PI list set by Laurent.

-  4. Transfer the data from CFHT/CADC to Terapix-in disks. (Laurent)

-  5. Transfer the data to SPICA disks, run QFITS-in (Laurent)

-  6. Control and Grade data from QFITS-in (Mireille)

-  7. Select the option: processed by "Mireille" or by "SPICA-PI" and continue (the following steps assume option "Mireille")

-  8. Selecting and collecting images, weightmaps, ldac files. Sorting data into field (RA, DEC or field name needed) and filters. Uncompressing weightmaps (Mireille). Overall, these tasks are time-consuming. Would be useful to have an interface to select, sort and distribute the data.

-  9. Scamp-ing + Q-control (Mireille)

-  10. Swarp-ing, QFITS-out, visual inspection of stacks (Mireille: control done even if data processed via "SPICA-PI" instead of "Mireille"). If the final stacks are not acceptable, Mireille/Henry may have to ask the PI to change some target specifications (e.g. crowded fields or very extended objects may mean the mesh background size will have be to modified).

-  11. Compute the mask in DS9 compliant format (Yannick)

-  12. IF data are produced in several bands, then: production of color-images, Chi-2 images (Mireille)

-  -12-a. IF more than 2 filters avaiable: catalogue production in double-pass mode using the Chi-2 image, merged catalogue production using the mask and gal/star separation, color-color plots (Mireille). Would be useful to implement the double-image detection tool developed by Henry at this stage (which functions on SExtractor catalogues generated by PSFex).

-  -12-b IF only two filters available: catalogue production in double-pass mode using one filter as a reference image, instead of Chi-2. No merged catalogue produced (Mireille).

-  13. IF only one filter available: no Chi-2 image, production of a simple catalogue without using the mask (Mireille).

-  14. For all options ("Mireille" or "SPICA-PI", one or several filters): all data produced. Mireille sends an email to Henry/Yannick . Henry/Yannick do a quality control check of final data products.

-  15. Summary and README files for the PI (Mireille)

-  16. Data compression, products copied to ftpix (Mireille)

-  17. Restricted access directory created with username/password and an email sent to the PI (Fred).


Site Map  -   -  Contact
© Terapix 2003-2011