CookBook2dfdr < Main

Tags: view all tags
---++!! !CookBook for AAOmega data reduction with 2dfdr

AAOmega data is processed using a revised version of the 2dfdr software package.  2dfdr currently has modes which reduce data for a number of instruments including AAOmega with both the 2df and Spiral IFU top ends, 6dF, FMOS and the older 2dF and Spiral.  In upgrading the software to work with AAOmega data a number of fundamental changes to the software were required.  At this time, the full 2dfdr manual 
([[http://www.aao.gov.au/AAO/2df/manual/UsersManual.doc][Word]],
[[http://www.aao.gov.au/AAO/2df/manual/UsersManual.pdf][pdf]])
is out of date and for the most part does not cover the modifications and improvements.  This !CookBook is aimed at providing a basic introduction to the reduction process.  An updated manual will be available shortly.  In the mean time, please contact your support astronomer with any questions about the software or problems with data reduction.

%TOC%

---+++ Download the software
2dfdr is rapidly evolving as more is learned about AAOmega data.  The high quality of AAOmega spectra, in comparison to what was achievable with the older 2dF spectrograph (which was, it must be stressed, principally designed to record low resolution redshifts of bright galaxies) means that there are many improvements required in the reduction system to do justice to the data recorded with AAOmega.  The user should check the ftp site below to ensure they have the every latest version of the software before starting a campaign of data reduction.  This rate of evolution should deminish once a small number of outstanding issues are addressed.

The software is only available as a set of binary executables and support files. The system is created with the Starlink software library and cannot be built without a Starlink installation.  While this software is not proprietary, we cannot support external builds at this time. With the demise of the Starlink project in 2005, it may become necessary in the future to completely remove the Starlink components.

The executables are currently only available for the !RedHat Enterprise3 Linux.  This system has been tested and also works well under current implementations of !RedHat Enterprise4 and most installations of Fedora, Scientific Linux and other popular Linux flavors.  Limited support can be offered for other operating systems.  A Solaris version can be made available on request, but due to speed problems is unlikely to receive much use.  A version for Mac OSX is not available at this time due to complications with running elements of the Starlink libraries on the Mac.

Download the latest tar file from the [[ftp://ftp.aao.gov.au/pub/2df/][FTP site]]

The file will have a name something like: 2dfdr-RH3E_060610.tgz

Unpack the tar file and to extract the software to your chosen working directory

<verbatim>
tar xvzf 2dfdr-RH3E_060610.tgz
</verbatim>

---+++ Running 2dfdr for AAOmega
First we need to set an environment variable and run the setup script

<verbatim>
    tcsh
    setenv DRCONTROL_DIR /path/to/software/2dfdr-linux_x86
    source $DRCONTROL_DIR/2dfdr_setup
</verbatim>

2dfdr should be run in a separate working directory for each set of observations with a particular field plate.  A meaningful directory structure for your observing run can save a lot of heartache later on.  An example directory structure might be:

<verbatim>
    Observing_june05/
        night1/
            field1/
            field1b/
            field2/
            field3/
        night2/
            field1/
            field4/
</verbatim>

Note that due to the way the flat and arc frames are used, each independent observation (i.e. with a different configuration of the fibres on the field plate) will require a new directory, even if all you have done is tweak the positions of fibre on a previously observed configuration.  Once the AAOmega slit wheel is moved with a change of field plate, a new set of flats and arcs are required for the reduction.  Data from multiple repeats of the same field, or for fields that contain some repeat observations can be automatically combined, but this is done after the full reduction of data for each field.

Data from the blue and red arms can be reduced in the same directory, but this is often not easy to work with and so most users create separate ccd_1 and ccd_2 sub directories with blue and red data, respectively.

Move to your working directory of choice and then the software can be started with the command

<verbatim>
    drcontrol aaomega.idx &
</verbatim>

which selects defaults for AAOmega, or

<verbatim>
    drcontrol aaomega###.idx &
</verbatim>

where ### is the grating name (e.g. 385R, 580V or 1700D etc.) with which the data were taken.  The .idx files are all stored at

<verbatim>
    /path/to/software/2dfdr-linux_x86/*.idx
</verbatim>

Users, if required, can make a copy of these instrument (.idx) files in the local directory and modify them to set their own reduction preferences.  Not all grating configurations currently have corresponding .idx files.

These commands should bring up the main 2dfdr control window shown here.

<img src="%ATTACHURLPATH%/2dfdr_main_gui.jpg" alt="2dfdr Main GUI" width='1009' height='579' />

---+++ Starting a reduction

The basic files needed to reduce AAOmega data are:

   * a multi-fibre flat field exposure (class MFFFF) These exposures are made with a quartz lamp that provides a uniform spectrum. They are used to calibrate the detector, and to find the centre and profile of each spectra.

   * an arc exposure (class MFARC) These exposures are made with lamps having various known strong lines. They are used to calibrate the central wavelength and dispersion.

   * one or more science frames (class MFOBJECT)

Additional frames of various types may be needed, depending on the nature of your data.

To start the automatic reduction process, hit the *Setup* button in the upper right hand section of the GUI.  The window shown here will appear.

<center>
<img src="%ATTACHURLPATH%/setup_gui.jpg" alt="Setup GUI" width='530' height='189' />
</center>

Automatic reduction depends on the use of a file naming convention in which the name has a root that is the same for all files. The root name is followed by a four-digit integer run number. Raw data from the AAT conforms to this convention with names of the form 13apr0001.fits, 13apr0002.fits etc . Data from the archive also conforms to the convention though the names are changed to run0001.fts, etc.

It is possible to reduce individual files that do not conform to the naming convention. They can be loaded into the system using the *Reduce* item in the *Commands* menu. However such files cannot form part of the automatic reduction of a sequence of files, unless they are renamed to conform to the other files in the sequence.

This *Setup Automatic Reduction* window is asking for the root name which prefixes the data to be reduced.  In this example the prefix is 28jun2 which indicates the software has found data taken on June 28th and for ccd2, the red arm.  If one wants to reduce data from the blue and red arms in the same directory then the root name can be altered to 28jun1 to pick up ccd1 data.

When the *Ok* button on the *Setup Automatic Reduction* window is pressed the file information becomes available in the left *Data* tab section of the main GUI as shown here.

<center>
<img src="%ATTACHURLPATH%/file_info.jpg" alt="File info GUI.jpg" width='468' height='358' />
</center>

Here we see the the first file, Run 1, is file 28jun20001.fits (which is run 1 for ccd2 from 28th June).  The file  is a Multi-Fibre Fibre Flat Field (class MFFFF) frame (refer to the full manual for an explanation of this insane classification convention).  The file is *Not Reduced* and so there is no *Reduced File*.  We can use the up and down spinner buttons to right of *Run* field, or select a file from the *File* chooser to cycle through the currently available data to view the other files with the same root name in the local directory.

If we hit the *Plot* button below the file information, we can see the 2D image shown here. This is useful to check that everything looks okay.  Note that the full CCD is 2kx4k and so many of the displays you will see during reduction are heavily aliased and will often show strange artifacts which are simply not in the data.

<img src="%ATTACHURLPATH%/a_flat.jpg" alt="Reduced flat with tramline overlaid" width='1025' height='693' />

---+++ Find Fibres

For older 2dF data it was neccessary to run the Find Fibres command from the main GUI commands menu.  This allowed the user to asign a fibre identifiaction to the flat field and prevented the extraction loosing fibres later on.  For AAOmega data this step is not required (and infact will not work and will crash the reduction if you try it).  AAOmega has eight well spaced gaps in the data, at the positions of guide fibres bundles on the field plates.  These gaps allow the accurate first order fit to the fibre positions and so the fibres are found automaticaly.

---+++ Reduce the data

The user should now be able to simply hit the *Start* button, on the upper right of the main GUI, to reduce the data in the current working directory.

The process runs as follows:

   * Reduce any and all multi-fibre flat field frames
   * Reduce any and all arc frames
   * Re-reduce the flat field frames using the accurate wavelength solution obtained from the arc frame reduction to compute a better average illumination correction
   * Reduce any and all science frames
   * Combine the science frames

When the process is complete your working directory will contain a set of *red.fits files which are the reduced data, and the combined data will be in the combined_frames.fits file.

---+++ Note on extraction methods

There are currently four extraction options for AAOmega data, the option can be set under the Extract tab on the main GUI window:

   1. TRAM - a simple summation of the data either side of the tram line fit.  This is quick, but a little noisy
   1. GAUSS - a weighted summation of the data based on the predicted Gaussian PSF either side of the tram line fit
   1. FIT - a full PSF fitting of the data, which takes into account fibre-to-fibre cross talk fully.
   1. OPTEX - optimal extraction (has been implemented but has not been fully tested).

Ultimately FIT extraction will give the best results, although the fitting process is rather slow.  At this time however, the FIT routine has not been correctly implemented and we suggest you use TRAM extraction for a quick look reduction and GAUSS extraction for science.

---+++ Known problems

1. The software will not run correctly if the path to the executables is too long (greater than 40 characters). The error means that 2dfdr will appear to start correctly, but on attempting to plot data the system will report that it cannot find the xwindow to display to. This bug stems from the use of the STARLINK libraries which are currently under review.

Note for users using Linux: You may have problems trying to reduce data on SUN/Solaris disks running 2dfdr under Linux: 2dfdr may simply fail giving an error message like this: "Task drexec2 has crashed while executing an action" This seems to be a problem related to accessing data over NFS between SUN and Linux machines. We don't have a solution for this, but the workaround is to copy the data to a local Linux disk, and reduce from there.

-- Main.RonHeald - 05 Feb 2007
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
jpg	2dfdr_gui.jpg	r1	manage	83.6 K	2007-01-11 - 03:53	RonHeald	2dfdr Main GUI
jpg	2dfdr_main_gui.jpg	r1	manage	67.0 K	2007-01-11 - 04:31	RonHeald	2dfdr Main GUI
jpg	a_flat.jpg	r1	manage	64.1 K	2007-01-11 - 03:57	RonHeald	flat
jpg	file_info.jpg	r1	manage	24.9 K	2007-01-11 - 03:56	RonHeald	data tab file info
jpg	setup_gui.jpg	r1	manage	14.2 K	2007-01-11 - 03:55	RonHeald	setup GUI
Topic revision: r8 - 2007-02-05 - RonHeald
Main Web