Quicklook Data Reduction

For observers to assess the quality of their SpeX or iSHELL data in (near-)real-time and make better informed decisions, the IRTF implemented "quicklook" data reduction software. During the observations, the software reads the FITS file headers and determines if sufficient data are available to run a scripted version of Spextool. The extracted spectra are then sent to DV so the observer can inspect them. All this is automated.

Starting Quicklook Data Reduction

Quicklook runs 24/7, without observer interaction. The program will recognize when an observer logs into the Bigdog (SpeX) or Cartman (iSHELL) XUI user interface. It will automatically change the raw data directory when needed. If Quicklook is not running for some reason during observations (after a computer reboot), the operator will need to restart it:

  • On the Bigdog or Cartman VNC, open an xterm.
  • On the xterm, type ssh quicklook@cybele
  • Enter the project password
  • On the Cybele prompt, type quicklook_spex or quicklook_ishell
  • Do not kill the xterm. It will display messages with the data reduction progress. See screenshot:
  • Image1

    When Quicklook is Running

    When the Quicklook software is running, its progress can be monitored as follows:

  • Messages on the xterm will show which FITS files are found, and which are being reduced.
  • Reduced spectra are automatically loaded into buffers F and G of DV. To see them, make sure the DV display is in "quicklook" mode. See screenshot.
  • One can load and plot any reduced spectrum using the "Open" button in DV. In the file selection window, enter the path of the reduced data (/home/quicklook/Data/2019AXXX/YYMMDD_reduced/ for SpeX, and /home/quicklook/Data_ishell/2019AXXX/YYMMDD_reduced/ for iSHELL), select the appropriate buffer, and click on the "ReadQuickLook" button on the left.
  • FITS files of the reduced spectra will be stored in /home/quicklook/Data/2019AXXX/YYMMDD_reduced/ (SpeX) and /home/quicklook/Data_ishell/2019AXXX/YYMMDD_reduced/ (iSHELL). The observer may copy them to their home computer (scp) within 2 weeks of the observations, after which the files will be deleted..
  • A copy of the xterm messages for this observing session is saved in /home/quicklook/Data/2019AXXX/YYMMDD/irtfquicklook_MM_DD_YY_HHhMMmSSs.log (SpeX) or /home/quicklook/Data_ishell/2019AXXX/YYMMDD/irtfquicklook_MM_DD_YY_HHhMMmSSs.log (iSHELL).
  • Troubleshooting

    If the observations are ongoing, but no new spectra appear in DV buffers F and G, check the messages in the xterm in which Quicklook was started:
  • There could a backlog. Each reduction takes about 30 seconds.
  • There may be insufficient calibration files (flats and arcs) for the reduction to proceed. The software will select flats and arcs taken in the same night. When those are not taken yet, the most recently reduced calibration files will be selected from the library at /home/quicklook/Data/Cal_library/ (SpeX) or /home/quicklook/Data_ishell/Cal_library/ (iSHELL).
  • The automated reduction does not work for every observing mode yet. See the section on "Known Issues and Limitations" below.
  • There may be observing situations that are not yet covered by Quicklook. For example, as a result of an "abort" after clicking "Go" in the Bigdog or Cartman user interface, after an observatory systems crash, or after network interupts. We are checking the reduction logs every morning and will address issues accordingly.
  • Known Issues and Limitations

    The development of Quicklook data reduction is ongoing. We are aware of the following problems and limitations:

  • The software works only for spectroscopy, not imaging.
  • For SpeX, only observations taken with Beam Pattern AB will be reduced. This excludes slit scans.
  • The extracted spectra are not divided over a standard star spectrum. The primary goal is to determine the S/N of the observations. Standard star division will be implemented later.
  • If one quits Quicklook (by Control-C, for example), and then restarts it with the same raw data input directory, the reduction will start from scratch, ignoring previous reductions.
  • If the user logs out of the observing session on Bigdog or Cartman, and then logs back in on the same date with the same account, the permissions have changed for the previously taken raw FITS files and they are no longer readable by the quicklook account. In this case the observer could start Quicklook using their user account after logging in to cybele (see above). The reduced data will then be saved in their home directory.
  • Occasionally, the wavelength calibration for the SpeX LXD mode is very wrong (by a few tenths to several micron).
  • For SpeX, the reduction of spectra obtained in off-slit nodding mode currently gives incorrect results.
  • For non-sidereal rates faster than ~0.10 arcsec/sec the software currently has problems recognizing if nodding is done on- or off-slit. Until a better algorithm is implemented, on-slit nodding is typically assumed. If not, a warning is given in the xterm that the extraction apertures may be incorrect.
  • Development History

  • 04/26/19: first night-time use with SpeX
  • 04/27-05/30/19: multiple upgrades and bug-fixes
  • 01/15/20: first night-time use with iSHELL

  • Contact Adwin Boogert for further information and assistance.