Minutes for the additional Terameeting of Jan 8th 2005

Present: Henry, Yannick, Fred, Laurent, Jean-Christophe, Mireille, Emmanuel (for part of the meeting)

Spica: discussion of operational modes

 Yannick asks if Laurent needs help, for example if Fred could help with development and future release dates. Laurent is initially reluctant, mainly because of the growing complexity of SPICA, due to past developments and all modifications requested over the past months. However, it seems that Fred could provide some useful help in re-writing parts of the SPICA code-base and implementing extra functionality (see below).

Monitoring of operations

 Some discussion about how the production system can be monitored remotely. Yannick proposes that Henry, Gilles, Yannick and JC follow the production (and be available in case of problems). There should be a clearly defined protocol in case of problems (i.e., who does what), probably described on a web-page. These people will have to follow the Qfits/SPICA processing. In case of problems, they will have to find out its origin and, if it is in the problem-list, to eventually repair. Emails should be sent to all monitors prior to taking any action in order to avoid duplication that may produce conflicting operations which could lead to corruption of DBTERAPIX. All emails should be CC'd to teralist.

 Yannick asks Laurent to provide a recipe for what to do in case of problems with SPICA production, and that the method to restart the stacks is clearly described. In this case, if there is a crash with the machines (i.e., NOT a software problem) we should know how to restart the stacks.

 Fred proposes to install some programs (already in use at the IAP which can monitor the status of daemons (like yp and so forth) on the system to restart them if there are problem. If this restart fails then an email should be sent directly to the people in charge of the production and cc'ed to teralist.

Production of stacks

 Next followed a discussion about the automatic production of stacks: Fred will try to implement a "daemon" which can automatically launch the stacks if none is currently running.

 Laurent will implement a multiple photometric rescaling based on 3 photometric calibration files.

Current problems

 RA/DEC problem (pix1,pix2,pix5,mix3). These images were not loaded into dbterapix. Laurent will investigate.

Hardware problems

Automount disk problems. Why do sometimes (usually at 2AM!) the cross-mounted node disks fall off the network?

 There seems to be a problem linked to the interoperation of YP service, automount, nfs, ip. Fred proposes several upgrades (for instance to install NSF v4.0), but it is probably not a good idea to update these services while we are in the middle of production.

Current status

 Qualityfits on mix3 and pix9 should be finished in three days, we should be ready to start spica on Friday. Tests are in progress with the stacking mode.