Difference: CookBook2dfdr (7 vs. 8)

Revision 82007-02-05 - RonHeald

Line: 1 to 1
 

CookBook for AAOmega data reduction with 2dfdr

Changed:
<
<		AAOmega data is processed using a revised version of the 2dfdr software package. 2dfdr currently has modes which reduce data for a number of instruments including AAOmega with both the 2df and Spiral IFU top ends, 2dF, 6dF, FMOS and the older Spiral. In upgrading the software to work with AAOmega data a number of fundamental changes to the software have been required. At this time, the full 2dfdr manual
>
>		AAOmega data is processed using a revised version of the 2dfdr software package. 2dfdr currently has modes which reduce data for a number of instruments including AAOmega with both the 2df and Spiral IFU top ends, 6dF, FMOS and the older 2dF and Spiral. In upgrading the software to work with AAOmega data a number of fundamental changes to the software were required. At this time, the full 2dfdr manual
 (Word, pdf)
Changed:
<
<		is out of date and does not cover the modifications and improvements. This CookBook is aimed at providing a basic introduction to the reduction process. An updated manual will be available shortly. In the mean time, please contact your support astronomer with any questions about the software or problems with data reduction.
>
>		is out of date and for the most part does not cover the modifications and improvements. This CookBook is aimed at providing a basic introduction to the reduction process. An updated manual will be available shortly. In the mean time, please contact your support astronomer with any questions about the software or problems with data reduction.
 

Download the software

Changed:
<
<		2dfdr is rapidly evolving as more is learned about AAOmega data. The high quality of AAOmega spectra, in comparison to what was achievable with the older 2dF spectrograph (which was, it must be stressed, principally designed to record low resolution redshifts of bright galaxies) means that there are many improvements required in the reduction system to do justice to the data recorded with AAOmega. The user should check the ftp site below to ensure they have the every latest version of the software before starting a campaign of data reduction. This rate of evolution will deminish once a small number of outstanding issues are addressed.
>
>		2dfdr is rapidly evolving as more is learned about AAOmega data. The high quality of AAOmega spectra, in comparison to what was achievable with the older 2dF spectrograph (which was, it must be stressed, principally designed to record low resolution redshifts of bright galaxies) means that there are many improvements required in the reduction system to do justice to the data recorded with AAOmega. The user should check the ftp site below to ensure they have the every latest version of the software before starting a campaign of data reduction. This rate of evolution should deminish once a small number of outstanding issues are addressed.
 
Changed:
<
<		The software is only available as a set of binary executables and support files. The system is created with the Starlink software library and cannot be built without a Starlink installation. While this software is not proprietary, we cannot support external builds at this time. With the demise of the Starlink project in 2005, it may become necessary to completely remove the Starlink components in the future.
>
>		The software is only available as a set of binary executables and support files. The system is created with the Starlink software library and cannot be built without a Starlink installation. While this software is not proprietary, we cannot support external builds at this time. With the demise of the Starlink project in 2005, it may become necessary in the future to completely remove the Starlink components.
 
Changed:
<
<		The executables are currently only available for the RedHat Enterprise3 Linux. This system has been tested and works well under current implementations of RedHat Enterprise4 and most installations of Fedora, Scientific Linux and other popular flavors. Some problems have been tracked to older Tcl libraries being used. Limited support can be offered for other operating systems. A Solaris version can be made available on request, but due to speed problems is unlikely to receive much use. A version for Mac OSX is not available at this time due to complications with running elements of the Starlink libraries on the Mac.
>
>		The executables are currently only available for the RedHat Enterprise3 Linux. This system has been tested and also works well under current implementations of RedHat Enterprise4 and most installations of Fedora, Scientific Linux and other popular Linux flavors. Limited support can be offered for other operating systems. A Solaris version can be made available on request, but due to speed problems is unlikely to receive much use. A version for Mac OSX is not available at this time due to complications with running elements of the Starlink libraries on the Mac.
  Download the latest tar file from the FTP site
Changed:
<
<		The file will look something like: 2dfdr-RH3E_060610.tgz
>
>		The file will have a name something like: 2dfdr-RH3E_060610.tgz
  Unpack the tar file and to extract the software to your chosen working directory
Line: 48 to 48
  field4/
Changed:
<
<		Note that due to the way the flat and arc frames are used, each independent observation (i.e. with a different configuration of the fibres on the field plate) will require a new directory, even if all you have done is tweak the positions of fibre on a previously observed configuration. Once the AAOmega slit wheel is moved, a new set of flats and arcs are required for the reduction. Data from multiple repeats of the same field, or for fields that contain some repeat observations can be automatically combined, but this is done after the full reduction of data for each field.
>
>		Note that due to the way the flat and arc frames are used, each independent observation (i.e. with a different configuration of the fibres on the field plate) will require a new directory, even if all you have done is tweak the positions of fibre on a previously observed configuration. Once the AAOmega slit wheel is moved with a change of field plate, a new set of flats and arcs are required for the reduction. Data from multiple repeats of the same field, or for fields that contain some repeat observations can be automatically combined, but this is done after the full reduction of data for each field.
  Data from the blue and red arms can be reduced in the same directory, but this is often not easy to work with and so most users create separate ccd_1 and ccd_2 sub directories with blue and red data, respectively.
Line: 64 to 64
  drcontrol aaomega###.idx &
Changed:
<
<		where ### is the grating name (e.g. 385R, 580V or 1700D etc.) with which the data was taken. The .idx files are all stored at
>
>		where ### is the grating name (e.g. 385R, 580V or 1700D etc.) with which the data were taken. The .idx files are all stored at
 
Added:
>
>
 /path/to/software/2dfdr-linux_x86/*.idx
Added:
>
>
 
Changed:
<
<		Users, if required, can make a copy of these instrument (.idx) files in the local directory and modify them to set their own reduction preferences. Not all grating configurations currently have .idx files created.
>
>		Users, if required, can make a copy of these instrument (.idx) files in the local directory and modify them to set their own reduction preferences. Not all grating configurations currently have corresponding .idx files.
  These commands should bring up the main 2dfdr control window shown here.
Line: 82 to 84
 
  • an arc exposure (class MFARC) These exposures are made with lamps having various known strong lines. They are used to calibrate the central wavelength and dispersion.
Changed:
<
<
  • one or more science frames
>
>
  • one or more science frames (class MFOBJECT)

Additional frames of various types may be needed, depending on the nature of your data.

 
Changed:
<
<		Additional frames of various types may be needed, depending on the nature of your data. To start the automatic reduction process, hit the Setup button in the upper right hand section of the GUI. The window shown here will appear.
>
>		To start the automatic reduction process, hit the Setup button in the upper right hand section of the GUI. The window shown here will appear.
 
Setup GUI
Changed:
<
<		The automatic data reduction depends on the use of a file naming convention in which the name has a root that is the same for all files. The root name is followed by a four-digit integer run number. Raw data from the AAT conforms to this convention with names of the form 13apr0001.fits, 13apr0002.fits etc . Data from the archive also conforms to the convention though the names are changed to run0001.fts, etc.
>
>		Automatic reduction depends on the use of a file naming convention in which the name has a root that is the same for all files. The root name is followed by a four-digit integer run number. Raw data from the AAT conforms to this convention with names of the form 13apr0001.fits, 13apr0002.fits etc . Data from the archive also conforms to the convention though the names are changed to run0001.fts, etc.
 
Changed:
<
<		It is possible to reduce individual files that do not conform to the naming convention. They can be loaded into the system using the Reduce item in the Commands menu. However such files cannot form part of the automatic reduction of a sequence of files, unless they are renamed to conform to the other files in the sequence.
>
>		It is possible to reduce individual files that do not conform to the naming convention. They can be loaded into the system using the Reduce item in the Commands menu. However such files cannot form part of the automatic reduction of a sequence of files, unless they are renamed to conform to the other files in the sequence.
 
Changed:
<
<		This Setup Automatic Reduction window is asking for the root name which prefixes the data to be reduced. In this example the prefix is 28jun2 which indicates the software has found data taken on June 28th and for ccd2, the red arm. If one wants to reduce data from the blue and red arms in the same directory then the root name can be altered to 28jun1 to pick up ccd1 data.
>
>		This Setup Automatic Reduction window is asking for the root name which prefixes the data to be reduced. In this example the prefix is 28jun2 which indicates the software has found data taken on June 28th and for ccd2, the red arm. If one wants to reduce data from the blue and red arms in the same directory then the root name can be altered to 28jun1 to pick up ccd1 data.
 
Changed:
<
<		Once Setup has run, the file information will be available in the left Data tab section of the main GUI as shown here.
>
>		When the Ok button on the Setup Automatic Reduction window is pressed the file information becomes available in the left Data tab section of the main GUI as shown here.
 
File info GUI.jpg
Changed:
<
<		Here we see the the first file, Run 1, is file 28jun20001.fits (which is run 1 for ccd2 from 28th June). The file is a Multi-Fibre Fibre Flat Field (class MFFFF, please refer to the full manual for an explanation of this insane classification convention). The file is Not Reduced and so there is no Reduced File. We can use the up and down spinner buttons to right of Run. or select a file from the File chooser to cycle through the current available data to view the other files with the same root name in the local directory.
>
>		Here we see the the first file, Run 1, is file 28jun20001.fits (which is run 1 for ccd2 from 28th June). The file is a Multi-Fibre Fibre Flat Field (class MFFFF) frame (refer to the full manual for an explanation of this insane classification convention). The file is Not Reduced and so there is no Reduced File. We can use the up and down spinner buttons to right of Run field, or select a file from the File chooser to cycle through the currently available data to view the other files with the same root name in the local directory.
 
Changed:
<
<		If we hit the Plot button below the file information, we can see the 2D image shown here. It's useful to check that everything looks okay. Note that the full CCD is 2kx4k and so many of the displays you will see during reduction are heavily aliased and will often show strange artifacts which are simply not in the data.
>
>		If we hit the Plot button below the file information, we can see the 2D image shown here. This is useful to check that everything looks okay. Note that the full CCD is 2kx4k and so many of the displays you will see during reduction are heavily aliased and will often show strange artifacts which are simply not in the data.
  Reduced flat with tramline overlaid
Line: 124 to 128
 
  • Reduce any and all science frames
  • Combine the science frames
Changed:
<
<		When the process is complete your working directory should contain a set of *red.fits files which are the reduced data, and the combined data will be in the combined_frames.fits file.
>
>		When the process is complete your working directory will contain a set of *red.fits files which are the reduced data, and the combined data will be in the combined_frames.fits file.
 

Note on extraction methods

Changed:
<
<		There are currently three extraction options for AAOmega data, the option can be set under the Extract tab on the main GUI window:
>
>		There are currently four extraction options for AAOmega data, the option can be set under the Extract tab on the main GUI window:
 
  1. TRAM - a simple summation of the data either side of the tram line fit. This is quick, but a little noisy
  2. GAUSS - a weighted summation of the data based on the predicted Gaussian PSF either side of the tram line fit
  3. FIT - a full PSF fitting of the data, which takes into account fibre-to-fibre cross talk fully.
Added:
>
>
  1. OPTEX - optimal extraction (has been implemented but has not been fully tested).
 
Changed:
<
<		Ultimately Fit extraction will give the best results, although the fitting process is rather slow. At this time however, the FIT routine has not been correctly implemented and we suggest you use Tram extraction for a quick look reduction and Gauss extraction for science.
>
>		Ultimately FIT extraction will give the best results, although the fitting process is rather slow. At this time however, the FIT routine has not been correctly implemented and we suggest you use TRAM extraction for a quick look reduction and GAUSS extraction for science.
 

Known problems

Changed:
<
<		1. The software will not run correctly if the path to the executables is to long (greater than 40 characters). The error means that 2dfdr will appear to start okay, but on attempting to plot data the system will report that it cannot find the xwindow to display to. This bug stems from the use of the STARLINK libraries which are currently under review.
>
>		1. The software will not run correctly if the path to the executables is too long (greater than 40 characters). The error means that 2dfdr will appear to start correctly, but on attempting to plot data the system will report that it cannot find the xwindow to display to. This bug stems from the use of the STARLINK libraries which are currently under review.
  Note for users using Linux: You may have problems trying to reduce data on SUN/Solaris disks running 2dfdr under Linux: 2dfdr may simply fail giving an error message like this: "Task drexec2 has crashed while executing an action" This seems to be a problem related to accessing data over NFS between SUN and Linux machines. We don't have a solution for this, but the workaround is to copy the data to a local Linux disk, and reduce from there.
Changed:
<
<		-- RonHeald - 30 Jan 2007
>
>		-- RonHeald - 05 Feb 2007
 
META FILEATTACHMENT attr="" autoattached="1" comment="setup GUI" date="1168487738" name="setup_gui.jpg" path="setup_gui.jpg" size="14515" user="Main.RonHeald" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="2dfdr Main GUI" date="1168487622" name="2dfdr_gui.jpg" path="2dfdr_gui.jpg" size="85650" user="Main.RonHeald" version="1"
View topic | History: r8 < r7 < r6 < r5 | More topic actions...
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback