From jle@mit.edu Mon Oct 23 08:49:53 2000 Date: Sun, 22 Oct 2000 12:41:04 -0400 From: Jim Elliot To: "Rayner, John" Cc: denault@jeans.ifa.hawaii.edu, onaka@jeans.ifa.hawaii.edu, "Bosh, Amanda" , "Tokunaga, Alan" Subject: SPeX occultation report [The following text is in the "iso-8859-1" character set] [Your display is set for the "US-ASCII" character set] [Some characters may be displayed incorrectly] Second transmission, since an error was reported on the first attempt. ---------------------------------------------------------- John: Here's a first report on the occultation "engineering run" that we carried out with SPeX by observing an occultation by Uranus and its rings. First of all, we want to thank you, Tony, Paul, and the rest of the IRTF staff for all the great support that contributed to the success of the run. SPeX is a pleasure to use and has great potential for occultation work. You and your team are commended for producing such a fine instrument. Our observing report will discuss the specifics of the run. Our reference point for the comments below is the occultation performance of NSFCAM, which, in conjunction with our own high-speed CCD photometer (PCCD), has served us well over the years for observing occultations with the IRTF. SPeX, in addition to the obvious advantages of spectrometric data, has much higher data throughput and an improved system for inserting GPS time into the data stream. It will be a great instrument for observing occultations. Suggestions for improvement of SPeX for occultations are divided into four categories: timing integrity, content of the data files, the user interface, and checkout procedures. These are discussed in detail below. Jim Elliot and Amanda Bosh ------------------------------------------------------------------------- Timing Integrity ---------------- For occultation work, verifiably accurate time-tagging of the data stream is critical, and during our run in October 2000 (program 079) we had two problems along these lines: (i) the GPS clock was not "locked" to the satellite signals, and (ii) the "bigdog" clock was not synchronized to the GPS clock. For our October data, problem (i) limits the inherent timing accuracy of our data to about 0.3 seconds, which is over two orders of magnitude worse than the time errors in the best occultation data. Problem (ii) can be overcome, to a certain extent, with our LED calibration procedures. With these we can relate the bigdog clock with the guidedog clock, which appeared to be synchronized to the GPS receiver. The effectiveness of this approach for calibrating the bigdog clock, however, depends critically on the stability of the oscillator on the DSP board. According to Peter Onaka, this oscillator should be very stable, and we should be able to test this with data in hand. These two problems with the timing could have been avoided if someone could have checked out the synchronization and the bigdog clock synchronization prior to our run. Now that we have become aware of these issues, we will certainly perform these checks upon arrival. We recommend that the GPS status (locked/not locked) be included in the user display for SPeX when it is in movie mode, perhaps a steady green field when in lock, or a flashing red otherwise. Also, the GPS lock status should be included in the FITS header for the data file (as we do for our PCCD). We also recommend the same procedures for the displaying and recording the synchronization status for the bigdog and guidedog clocks. Content of the Data Files ------------------------- [cut out all the zero pad] The data files that are output when using multiple subframes include padding to fill the space between the n subframes. This was done to retain the relative spacing between the n subframes. However, a more space-efficient method would be to pack the n subframes as closely as possible (may still need some 0 padding if the subframes are different sizes) and record the source locations of the subframes in header keywords. We do this with our PCCD data files, and an example of a PCCD header is included at the end of this message. The impact of this can be great; our data files were 4.9 MB each in movie mode, whereas the movie mode buffer is only 4MB. A savings of 0.9 MB per file adds up quickly. During our occultation observation we recorded 896 files while in movie mode; this adds up to approx. 806 MB of zeros saved during this 2.5 hour run. [pixel-map timing] Although it was thought that the pixel timing maps output by the SpeX acquisition program (the seqtbl.spex files) were useful only for engineering or debugging, we find that they are useful for understanding the exact timing of each pixel in the frame. However, with every mode change the previous file is overwritten. It would be useful to have some way to automatically save these files, and to record the file name in the header. [frame times in the header] In movie mode, a *.time file is recorded along with the *.movie.fit file. This file includes the timing information for each time step in the *.movie.fit file. One suggestion would be to include the hours along with minutes, seconds, and fractional seconds in this *.time file. For an observation that spans an hour change, it is difficult to quickly determine the timing of a particular frame. Also, along with this suggestion, a header keyword giving the name of the first file in the sequence would be helpful when writing a program to figure out the time of each frame within each file. It would also ease the ambiguity of TIME_OBS, which seems to refer to the approx. start time of the first file in the sequence. The explanation of this keyword should also be changed from "UT TIME OF ACQISTION" which implies it is the time for the current file, to something like "UT TIME OF ACQISTION OF FIRST FRAME". A more elegant method of addressing the timing keywords issue would be to drop the *.time file and include the end time of each frame as a keyword: TIME0001="yyyy mm dd hh mm ss.ssssss" TIME0002= TIMEnnnn= / up to NAXIS3 value Having 4 spaces for an index would cover the most ambitious use of this. (A 32 by 32 subframe x 2 bytes per pixel would create about 2048 frames per file.) --------------------------------------------------------------------------- SUMMARY OF SUGGESTED IMPROVEMENTS TO SPEX FOR OCCULTATION WORK 1) User interface Display status of GPS lock Display status of bigdog and guidedog clock synchronization Allow the option of a second keyboard and mouse so that each camera could be operated by a different person Have the GPS clock located in the user area for the LED calibration 2) FITS file Eliminate the large zero pads in the data when two subframes are used Add keywords that state the absolute coordinates of the subframes Add keywords that state the lock status for the GPS Add keywords that state the synchronization status for the two clocks Add the hours to the frame times Add the frame times to the FITS file (keywords may be the best approach) Add a reference to the file name for the pixel timing map 3) Procedures for the day crew Check the lock of the GPS clock at regular intervals Check the synchronization of the bigdog and guidedog clocks ================================================================== SUMMARY OF FITS CONSIDERATIONS In the following discussion we must deal with two meanings of the word "pixel." In the FITS literature, "pixel" refers to a datum or picture element, while "pixel" also refers to a specific location on a CCD chip where the light was converted into electrons. So when referring to "pixel" in the sense used in the FITS literature, we shall surround it with quotes; when referring to pixel as our unit of measurement on the CCD chip, we shall not quote it. The purpose of the following group of keywords, all having C as their first letter, is to relate the values in the data array to coordinate axes. In the md header we can identify the integer data indices i, j, and k with the three axes as follows: i = 1, NAXIS1 j = 1, NAXIS2 k = 1, NAXIS3 Suppose that we want to assign the coordinate axes x, y, and z to the three data index axes through linear relations. To do this we define a data index coordinate for each of the three axes so that the integer values correspond to the (1 based) index in the data array. Next we specify "CDELT" as the increment on the coordinate axis that would correspond to an increment of 1 in the data index. Finally we specify one point of correspondence between data index coordinates ("CRPIX") and the coordinate axis ("CRVAL"). This correspondence is set up so that the data value is tagged to a coordinate at the center of its integration interval (i.e. to give a concrete example: "1" on the column axis corresponds to the center of the first column). The transformation equations would be: x = CRVAL1 + CDELT1 * (i - CRPIX1) y = CRVAL2 + CDELT2 * (j - CRPIX2) t = CRVAL3 + CDELT3 * (k - CRPIX3) Note that "CDELT" and "CRVAL" have units of the coordinates, while "CRPIX" has units of the data index. "CRPIX" need not be an integer, but can assume fractional and negative values. A diagram illustrating this type of transformation between the data index, i, and the x coordinate is given below. (Here we have let i assume continuous values instead of introducing a continuous variable that would equal i at its integer values.) i = CRPIX1 data index (i) values: 1 2 3 | 4 data index (i) axis: -------|-----------|-----------|--------|--|----> integration boundaries: ^ ^ ^ ^ | <---CDELT1--> | coordinate (x) axis: --|-----|-----|-----|-----|-----|-----|---|-|-----> coordinate (x) values: 0 1 2 3 4 5 6 | 7 x = CRVAL1 CRPIX1 = 1 / double; (index of reference column pixel, see above) CDELT1 = 1.00 / double; increment in pixels on the CCD for one "pixel". In other words, this is the on-chip binning factor. This could become another number if the original pixels on the chip were integrated before recording, such as a 2x2 square, as is possible on some systems. We always treat a pixel as the unit on the chip, not on the recorded data. CRVAL1 = / double; reference value in pixels on the CCD chip of the first "pixel" in this subframe CFINT1 = 1.00 / double; the fraction of CDELT1 that the data are are integrated over. This is 1.00 for abutting data, such as pixels, but may be less if there is some "deadtime" in reading out data. CLABL1 = Column Number / Column Number on the CCD chip CUNIT1 = pixel / 1 pixel (= 22.3 um for TI4849; 15 um for Ford) CDSCR1 = on the CCD chip / further description of axis #1 The keyword "CLABL1" is intended to be the label of a the axis of a graph that could be used in an automatic plotting program, "CUNIT" gives the units that would be used to label the axis, and "CDSCR1" gives a further explanation of axis #1 (such as might be included in a figure caption). CRPIX2 = 1 / double; (index of reference column pixel, see above) CDELT2 = 1.00 / increment of pixels; binning factor CRVAL2 = / integer; row number of first pixel in this subframe For the SNAPSHOT, this refers to the row number (1 is the first row) on the chip for regular frames, but always starts at 1 for strip scans. CFINT2 = 1.00 / double; the fraction of CDELT2 that the data are are integrated over. This is 1.00 for abutting data, such as pixels. CLABL2 = Row Number / Row Number on the CCD chip CUNIT2 = pixel / 1 pixel (= 22.3 um for TI4849; 15 um for Ford) CDSCR2 = on the CCD chip / further description of axis #2 CRPIX3 = / Reference pixel at start of obs. (one base) CDELT3 = / double; integration duration in seconds CRVAL3 = / Value added to start time (in seconds) to get the midtime of the reference pixel. CFINT3 = / double; the fraction of CDELT3 that the data are integrated over. CLABL3 = Time / label for this axis CUNIT3 = seconds / This axis is time in seconds CDSCR3 = / description of axis ("from GC clock" for the SNAPSHOT) This comment discusses how the above keywords, C*3, are to be interpreted depending on sequencer program number (SEQ-PRGM). The Dots mode is a special case and is not included in the table below. For the other modes, the following table converts from the mt header to the md header keywords. Note that DURINT and SHUTTER in the mt header are in 10's of microseconds and SHUTTER in this header is in seconds. Following the table is a description of the timing in each operating mode. Keyword Series(1) Spacewatch(3) Fast Series(4) CRPIX3 1 1 1 CDELT3 DURINT DURINT DURINT CRVAL3 SHUTTER/2 DURINT/2 DURINT/2 CFINT3 SHUTTER/DURINT 1 1 ************************************************************* And this is how it's used in the PCCD data headers: ************************************************************* SIMPLE = T / Standard FITS file flag BITPIX = 16 / number of bits per pixel NAXIS = 3 / number of axes: columns, rows (and time) NAXIS1 = 000033 / number of columns in image NAXIS2 = 00033 / number of rows in an image NAXIS3 = 13500 / number of images in time series CTYPE = 'short ' / declaration type for C (short, etc.) NUMSUBF = 1 / integer; number of subframes (1-5) TOP1 = 289 / Top row of subframe 1 BOTTOM1 = 354 / Bottom row of subframe 1 LEFT1 = 184 / Left column of subframe 1 RIGHT1 = 249 / Right column of subframe 1 TOP2 = 000 / Top row of subframe 2 BOTTOM2 = 000 / Bottom row of subframe 2 LEFT2 = 000 / Left column of subframe 2 RIGHT2 = 000 / Right column of subframe 2 TOP3 = 000 / Top row of subframe 3 BOTTOM3 = 000 / Bottom row of subframe 3 LEFT3 = 000 / Left column of subframe 3 RIGHT3 = 000 / Right column of subframe 3 TOP4 = 000 / Top row of subframe 4 BOTTOM4 = 000 / Bottom row of subframe 4 LEFT4 = 000 / Left column of subframe 4 RIGHT4 = 000 / Right column of subframe 4 TOP5 = 000 / Top row of subframe 5 BOTTOM5 = 000 / Bottom row of subframe 5 LEFT5 = 000 / Left column of subframe 5 RIGHT5 = 000 / Right column of subframe 5 FILETYPE= 'pd ' / pd is the filetype for all PCCD data FLTPVERS= 2.0 / version number of the filetype VERSDATE= '1994/06/17' / date that this version was created VERSAUTH= 'mwb,efy,ewd' / authors of header on 'push along stack' BZERO = 32768.0 / Real = File*BSCALE + BZERO BSCALE = 1.0 / 1.0 if BUNIT is ADU BUNIT = 'ADU ' / Units of brightness of Real (above) CRPIX1 = 1 / index of reference column pixel for NAXIS1 CDELT1 = 2 / binning factor in NAXIS1 direction CRVAL1 = 001 / ref value in pixels on the CCD (NAXIS1) CFINT1 = 1.00 / frac of CDELT1 data are integrated over. CLABL1 = 'Column Number' / Description of NAXIS1 CUNIT1 = 'pixel ' / Units of NAXIS1 pixel (=23 um for Th7883) CDSCR1 = 'on the CCD chip' / further description of NAXIS1 CRPIX2 = 1 / index of reference column pixel for NAXIS2 CDELT2 = 2 / binning factor in NAXIS2 direction CRVAL2 = 001 / ref value in pixels on the CCD (NAXIS2) CFINT2 = 1.00 / frac of CDELT2 data are integrated over. CLABL2 = 'Row Number' / Description of NAXIS2 CUNIT2 = 'pixel ' / Units of NAXIS2 pixel (=23 um for Th7883) CDSCR2 = 'on the CCD chip' / further description of NAXIS2 CRPIX3 = 1 / Reference pixel in NAXIS3 (one base) CDELT3 = 0.200000 / time interval (sec) between NAXIS3 elements CRVAL3 = 0.100000 / time offset (midtime-start) of ref pixel CFINT3 = 1.0 / frac of CDELT3 data are integrated over. CLABL3 = 'Time ' / Description of NAXIS3 CUNIT3 = 'seconds ' / Units of NAXIS3 CDSCR3 = ' ' / description of axis ************************************************************* Info from the Definition of the FITS Standard (http://fits.gsfc.nasa.gov/). ************************************************************* 5.4.2.5 Array Keywords These keywords are used to describe the contents of an array, either alone or in a series of random groups (§7). They are optional, but if they appear in the header describing an array or groups, they must be used as defined in this section of this standard. They shall not be used in headers describing other structures unless the meaning is the same as that for a primary or groups array. BSCALE Keyword This keyword shall be used, along with the BZERO keyword, when the array pixel values are not the true physical values, to transform the primary data array values to the true physical values they represent, using Eq. 5.3. The value field shall contain a floating point number representing the coefficient of the linear term in the scaling equation, the ratio of physical value to array value at zero offset. The default value for this keyword is 1.0. BZERO Keyword This keyword shall be used, along with the BSCALE keyword, when the array pixel values are not the true physical values, to transform the primary data array values to the true values. The value field shall contain a floating point number representing the physical value corresponding to an array value of zero. The default value for this keyword is 0.0. The transformation equation is as follows: physical_values = BZERO + BSCALE * array_value (5.3) BUNIT Keyword The value field shall contain a character string , describing the physical units in which the quantities in the array, after application of BSCALE and BZERO, are expressed. These units must follow the prescriptions of §5.3. BLANK Keyword This keyword shall be used only in headers with positive values of BITPIX (i.e., in arrays with integer data). Columns 1-8 contain the string, ``BLANK'' (ASCII blanks in columns 6-8). The value field shall contain an integer that specifies the representation of array values whose physical values are undefined. CTYPEn Keywords The value field shall contain a character string, giving the name of the coordinate represented by axis n. CRPIXn Keywords The value field shall contain a floating point number , identifying the location of a reference point along axis n, in units of the axis index. This value is based upon a counter that runs from 1 to NAXISn with an increment of 1 per pixel. The reference point value need not be that for the center of a pixel nor lie within the actual data array. Use comments to indicate the location of the index point relative to the pixel. CRVALn Keywords The value field shall contain a floating point number , giving the value of the coordinate specified by the CTYPEn keyword at the reference point CRPIXn. Units must follow the prescriptions of §5.3. CDELTn Keywords The value field shall contain a floating point number giving the partial derivative of the coordinate specified by the CTYPEn keywords with respect to the pixel index, evaluated at the reference point CRPIXn, in units of the coordinate specified by the CTYPEn keyword. These units must follow the prescriptions of §5.3. CROTAn Keywords This keyword is used to indicate a rotation from a standard coordinate system described by the CTYPEn to a different coordinate system in which the values in the array are actually expressed. Rules for such rotations are not further specified in this standard; the rotation should be explained in comments. The value field shall contain a floating point number giving the rotation angle in degrees between axis n and the direction implied by the coordinate system defined by CTYPEn. DATAMAX Keyword The value field shall always contain a floating point number, regardless of the value of BITPIX. This number shall give the maximum valid physical value represented by the array, exclusive of any special values. DATAMIN Keyword The value field shall always contain a floating point number, regardless of the value of BITPIX. This number shall give the minimum valid physical value represented by the array, exclusive of any special values. [Part 2, Text/HTML (charset: ISO-8859-1 "Latin 1") 835 lines] [Unable to print this part]