|
Quicklook Data Reduction
|
For observers to assess the quality of their SpeX or iSHELL data in
(near-)real-time and make better informed decisions, the IRTF
implemented "quicklook" data reduction software. During the
observations, the software reads the FITS file headers and determines
if sufficient data are available to run a scripted version of
Spextool. The extracted spectra are then sent to DV so the observer
can inspect them. All this is automated.
Starting Quicklook Data Reduction
Quicklook runs 24/7, without observer interaction. The program will
recognize when an observer logs into the Bigdog (SpeX) or Cartman
(iSHELL) XUI user interface. It will automatically change the raw data
directory when needed. If Quicklook is not running for some reason
during observations (after a computer reboot), the operator
will need to restart it:
On the Bigdog or Cartman VNC, open an xterm.
On the xterm, type ssh quicklook@cybele
Enter the project password
On the Cybele prompt, type quicklook_spex
or quicklook_ishell
Do not kill the xterm. It will display messages with the data
reduction progress. See screenshot:
When Quicklook is Running
When the Quicklook software is running, its progress can be
monitored as follows:
Messages on the xterm will show which FITS files are found, and which
are being reduced.
Reduced spectra are automatically loaded into buffers F
and G of DV. To see them, make sure the DV display is in
"quicklook" mode. See screenshot.
-
Buffer F will contain the most recently reduced spectrum (a
single nod position).
-
Buffer G will contain the latest average over a contiguous
collection of spectra, sharing the same position on the sky,
instrument settings, itime, and coadds. It will be updated when a new
nod pair becomes available.
One can load and plot any reduced spectrum using the "Open" button in
DV. In the file selection window, enter the path of the reduced data
(/home/quicklook/Data/2019AXXX/YYMMDD_reduced/ for SpeX, and
/home/quicklook/Data_ishell/2019AXXX/YYMMDD_reduced/ for iSHELL),
select the appropriate buffer, and click on the "ReadQuickLook" button
on the left.
FITS files of the reduced spectra will be stored in
/home/quicklook/Data/2019AXXX/YYMMDD_reduced/ (SpeX) and
/home/quicklook/Data_ishell/2019AXXX/YYMMDD_reduced/ (iSHELL). The
observer may copy them to their home computer (scp) within 2 weeks of
the observations, after which the files will be deleted..
A copy of the xterm messages for this observing session is saved in
/home/quicklook/Data/2019AXXX/YYMMDD/irtfquicklook_MM_DD_YY_HHhMMmSSs.log
(SpeX) or
/home/quicklook/Data_ishell/2019AXXX/YYMMDD/irtfquicklook_MM_DD_YY_HHhMMmSSs.log
(iSHELL).
Troubleshooting
If the observations are ongoing, but no new spectra appear in DV
buffers F and G, check the messages in the xterm in which
Quicklook was started:
There could a backlog. Each reduction takes about 30 seconds.
There may be insufficient calibration files (flats and arcs) for the
reduction to proceed. The software will select flats and arcs taken in
the same night. When those are not taken yet, the most recently
reduced calibration files will be selected from the library at
/home/quicklook/Data/Cal_library/ (SpeX) or
/home/quicklook/Data_ishell/Cal_library/ (iSHELL).
The automated reduction does not work for every observing mode
yet. See the section on "Known Issues and Limitations" below.
There may be observing situations that are not yet covered by
Quicklook. For example, as a result of an "abort" after clicking
"Go" in the Bigdog or Cartman user interface, after an observatory
systems crash, or after network interupts. We are checking the
reduction logs every morning and will address issues accordingly.
Known Issues and Limitations
The development of Quicklook data reduction is ongoing. We are aware
of the following problems and limitations:
The software works only for spectroscopy, not imaging.
For SpeX, only observations taken with Beam Pattern AB will be
reduced. This excludes slit scans.
The extracted spectra are not divided over a standard star
spectrum. The primary goal is to determine the S/N of the
observations. Standard star division will be implemented later.
If one quits Quicklook (by Control-C, for example), and then
restarts it with the same raw data input directory, the reduction will
start from scratch, ignoring previous reductions.
If the user logs out of the observing session on Bigdog or Cartman,
and then logs back in on the same date with the same account, the
permissions have changed for the previously taken raw FITS files and
they are no longer readable by the quicklook account. In this case the
observer could start Quicklook using their user account after
logging in to cybele (see above). The reduced data will then be saved
in their home directory.
Occasionally, the wavelength calibration for the SpeX LXD mode is very
wrong (by a few tenths to several micron).
For SpeX, the reduction of spectra obtained in off-slit nodding mode
currently gives incorrect results.
For non-sidereal rates faster than ~0.10 arcsec/sec the software
currently has problems recognizing if nodding is done on- or off-slit.
Until a better algorithm is implemented, on-slit nodding is typically
assumed. If not, a warning is given in the xterm that the extraction
apertures may be incorrect.
Development History
04/26/19: first night-time use with SpeX
04/27-05/30/19: multiple upgrades and bug-fixes
01/15/20: first night-time use with iSHELL
Contact Adwin Boogert for further information and assistance.