Exercise: GMOS Imaging Reduction - NGC 6872 - Which band?

IMPORTANT: Edits are required for the tables below, and in STEP 1 and 2. After that, it should just work.

Datasets:

Science observation UT date(s)	2010 Nov 01
Data filename prefix(es)	S20101101S

Target Frames

Science	S20101101S	eg. format: 1-3,5-7	binning 2x2
Processed Bias	for 20101101	S20000101S0001_bias.fits
Processed Twilight Flat	for 20101101	S20000101S0001_flat_band.fits
Processed Fringe Frame	N/A	N/A

Fringe Frames

N/A

Twilight Flats

The twilight flat that matches the science target observations.

In the template, the name of the processed flat is the name of the first raw flat in the list followed with the suffix "_flat_band", eg. "_flat_r". However, the name could be anything, just define it later in Step 1, and above in the Target Frame table.

Processed: S20000101S0001_flat_band.fits
Twilight Frames	S20000101S	1-3	binning 2x2
Processed Bias	for 20000101	S20000101S0000_bias.fits

Biases

The bias(es) that match the science observation and the twilight flat.

The Gemini package default name of the processed bias is the name of the first raw bias in the list followed with the suffix "_bias". (You could override that if it really bothers you.)

If combining biases from multiple nights:

Processed: S20000101S0001_bias.fits
Biases	S20000101S	1-3	binning 2x2
Biases	S20000102S	1-3	binning 2x2

If combining biases from a single night:

Processed: S20000101S0001_bias.fits
Biases	S20000101S	1-3	binning 2x2

STEP 0: Directory structure used by this Notebook

The directory paths used in this notebook are assumed to match this structure:

rootdir/
    calib/
        bias/
        flat/
        fringe/
    raw/
    redux/

STEP 1: Set up the Notebook and the PyRAF session

The data processing will happen in the Notebook. It is also convenient to have a PyRAF interactive session ready to use if needed, eg. to imexam. In this step, we will configure the Notebook and the PyRAF session.

This step needs to be repeated everytime the Notebook and/or the PyRAF session is restarted.

STEP 1.1: Set up the Notebook

STEP 1.1.1: Set up the variables.



In [ ]:

    
import os
import os.path
import shutil

# WARNING: Make sure that the directory structure given in Step 0 
#          is already in place.

# EDIT HERE - Directories
rootdir = '/data/workspace/DRWorkshopAAO'

# EDIT HERE - File numbers and file names
biases = {
    # When combining biases from multiple nights.
    'bias_20000101' : {
        'procbias' : 'S20000101S0000_bias.fits',
        'frames'   : [('S20000101S', '1-3'),
                      ('S20000102S', '1-3'),
                     ]
    },
    # When combining biases from a single night.
    'bias_20000101' : {
        'procbias'   : 'S20000101S0000_bias.fits',
        'frames'     : [('S20000101S', '1-3')],
    }
}

flats = {
    'flat_20000101' : {
        'procflat'   : 'S20000101S0001_flat_band.fits',
        'frames'     : [('S20000101S', '1-3')],
        'flatbias'   : biases['bias_20000101']['procbias'],
    }
}

science = {
    'procsci'   : 'ngc6872_band.fits',
    'sciroot'   : 'S20101101S',
    'sciframes' : '1-3',
    'scibias'   : biases['bias_20000101']['procbias'],
    'sciflat'   : flats['flat_20000101']['procflat'],
}

logfile = 'NGC6872_band.log'

# Automatically set directories to match structure from Step 0
raw_path = os.path.join(rootdir, 'raw') + '/'
redux_path = os.path.join(rootdir, 'redux') + '/'
procbias_path = os.path.join(rootdir, 'calib', 'bias') + '/'
procflat_path = os.path.join(rootdir, 'calib', 'flat') + '/'
products_path = os.path.join(rootdir, 'products') + '/'

# Change to work directory
os.chdir(redux_path)

Step 1.1.2: Load and initialize the package

---> LAUNCH DS9 before proceeding further

Load the package required for the Notebook session and reset tasks to the default parameters.



In [ ]:

    
from pyraf import iraf
iraf.gemini()
iraf.gmos()

iraf.unlearn(iraf.gemini, iraf.gmos, iraf.gemtools)

iraf.gmos.logfile = logfile

iraf.set(stdimage='imtgmos')

STEP 1.2: Set up the PyRAF session

The commands shown here need to be run from the shell.

Open an xterm or a Terminal, then:

cd /data/workspace/DRWorkshopAAO/redux
pyraf

Then in the PyRAF session:

gemini
gmos
unlearn gemini
unlearn gmos
unlearn gemtools

Get your PyRAF configured. In the PyRAF session:

iraf.gmos.logfile = "NGC6872_band.log"
set stdimage=imtgmos

STEP 2: Create the file lists



In [ ]:

    
os.chdir(redux_path)

# delete any pre-existing lists
iraf.delete('*.lis', verify='no')

# EDIT HERE - the bias_??? and flat_??? need editing to match your observations.

# biases
#  bias that matches the science
for (root, frames) in biases['bias_20000101']['frames']:
    iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
iraf.concat('tmp_*.lis', 'bias_20000101.lis')
iraf.delete('tmp_*.lis', verify='no')

# if another bias is needed, eg. for the twilight flat
#for (root, frames) in biases['bias_20000101']['frames']:
#    iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
#iraf.concat('tmp_*.lis', 'bias_20000101.lis')
#iraf.delete('tmp_*.lis', verify='no')


# flats
for (root, frames) in flats['flat_20000101']['frames']:
    iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
iraf.concat('tmp_*.lis', 'flat_20000101.lis')
iraf.delete('tmp_*.lis', verify='no')


# science
iraf.gemlist(science['sciroot'], science['sciframes'], \
             Stdout='sci.lis')

# Get a full list
iraf.concat('*.lis', 'all.lis')

STEP 3: Visually inspect all the data



In [ ]:

    
all_files = open('all.lis', 'r')
for line in all_files:
    image = line.strip() + '[1]'
    print image
    iraf.gdisplay(raw_path + image, 1, fl_paste='no')
    iraf.sleep(5)
    
all_files.close()

STEP 4: Process the biases

This will produce the bias(es) defined above. That/Those bias(es) will be used later to process the science frames and the twilight flats.

The raw biases go through basic processing then are combined to produce the master bias.



In [ ]:

    
os.chdir(redux_path)

for biaskey in biases.keys():
    iraf.imdelete(biases[biaskey]['procbias'], verify='no')
    iraf.imdelete('g//@'+biaskey+'.lis', verify='no')

    print 'Creating bias ', biases[biaskey]['procbias']
    iraf.gbias('@'+biaskey+'.lis', outbias=biases[biaskey]['procbias'], \
               rawpath=raw_path, fl_over='yes', fl_trim='yes', \
               fl_vardq='yes')
    shutil.copy(biases[biaskey]['procbias'], procbias_path)

Check that the biases have been copied over to the "calib" directory



In [ ]:

    
# The processed biases are:
for biaskey in biases.keys():
    print biases[biaskey]['procbias']

print 'Content of "calib/bias/"'
os.listdir(procbias_path)

Visually inspect the processed biases



In [ ]:

    
# WARNING: interactive.  It launches imexam.  Type 'q' to quit.
for biaskey in biases.keys():
    iraf.gdisplay(os.path.join(procbias_path, biases[biaskey]['procbias']), 1, fl_paste='yes')

STEP 5: Process the twilight flats

This will produce the twilight flat defined above. That flat will be used later to process the science frames.

STEP 5.1: Basic processing of the raw flats



In [ ]:

    
os.chdir(redux_path)

for flatkey in flats.keys():
    iraf.imdelete('g//@'+flatkey+'.lis', verify='no')
    iraf.imdelete('rg//@'+flatkey+'.lis', verify='no')
    
    flatlist = flatkey+'.lis'
    procbias = os.path.join(procbias_path, flats[flatkey]['flatbias'])
    
    print 'Processing raw twilight flats'
    
    # gprepare and bias correct the raw flats
    iraf.gireduce('@'+flatlist, rawpath=raw_path, \
                  fl_over='yes', fl_trim='yes', \
                  fl_bias='yes', bias=procbias, fl_flat='no', \
                  fl_vardq='yes')

STEP 5.2: Create the processed flat field



In [ ]:

    
os.chdir(redux_path)

for flatkey in flats.keys():
    iraf.imdelete(flats[flatkey]['procflat'], verify='no')
    
    flatlist = flatkey+'.lis'
    procflat = flats[flatkey]['procflat']
    
    print 'Creating twilight flat ', procflat
    
    # Create the processed flat
    iraf.giflat('rg//@'+flatlist, outflat=procflat, fl_vardq='yes')
    
    # Copy the processed flat to the calibration directory
    shutil.copy(procflat, procflat_path)

Check that the flat has been copied over to the "calib" directory



In [ ]:

    
# The processed flats are:
for flatkey in flats.keys():
    print flats[flatkey]['procflat']

print 'Content of "calib/flat/"'
os.listdir(procflat_path)

Visually inspect the processed twilight flats



In [ ]:

    
for flatkey in flats.keys():
    iraf.gdisplay(os.path.join(procflat_path, flats[flatkey]['procflat']), 1, fl_paste='yes')

STEP 6: Process the fringe frame

STEP 7: Process the target

There is no significant fringing in the r-band and g-band. No extra fringing correction will be necessary. If you do want to learn about how to correct for fringing, see the i-band notebook.

All we need to do here is the straighforward processing, i.e. the bias and flat correction. (And overscan and trim.)

STEP 7.1: Bias and flat fielding of the target



In [ ]:

    
os.chdir(redux_path)

iraf.imdelete('g//@sci.lis', verify='no')
iraf.imdelete('rg//@sci.lis', verify='no')

iraf.gireduce('@sci.lis', rawpath=raw_path, fl_over='yes', fl_trim='yes', \
              fl_bias='yes', bias=science['scibias'], \
              fl_flat='yes', flat1=science['sciflat'], \
              fl_vardq='yes')

Visually inspect the reduce science frames



In [ ]:

    
sciframes = open('sci.lis', 'r')
for line in sciframes:
    frame = line.strip()
    iraf.gdisplay('rg'+frame, 1, fl_paste='yes')
sciframes.close()

STEP 7.2: Fringe removal

STEP 8: Mosaic the reduced target frames

GMOS uses 3 CCDs. Depending on the generation of CCDs, they could be read with 1, 2, or 4 amplifiers. Each amplifier creates one MEF extension. GMOS data can have 3, 6, or 12 science extension. The mosaic step tile and align them all correctly into one single extension.

Here we are using GMOS-S 2010 data. The CCDs were read with only one amplifier, thus only three science extensions, one per CCD need to be mosaic. (Nothing special needs to be done, gmosaic recognizes the data and do the appropriate thing.)



In [ ]:

    
iraf.imdelete('mrg//@sci.lis', verify='no')

iraf.gmosaic('rg//@sci.lis', fl_vardq='yes')

Visually inspect the mosaiced frames



In [ ]:

    
sciframes = open('sci.lis', 'r')
for line in sciframes:
    frame = line.strip()
    iraf.gdisplay('mrg'+frame, 1, z2=20000, fl_paste='yes')
sciframes.close()

STEP 9: Align and stack the mosaic target frames

The observations were dithered. This allows for easy removal of cosmic rays and the elimination of the CCD gaps.



In [ ]:

    
iraf.imdelete(science['procsci'], verify='no')

iraf.imcoadd('mrg//@sci.lis', outimage=science['procsci'], logfile=logfile)

STEP 10: Inspect and save product



In [ ]:

    
iraf.display(science['procsci']+'[sci,1]', 1, \
             zrange='no', zscale='no', z1=1000, z2=20000)

shutil.copy(science['procsci'], products_path)

# Final reduced image
print 'Final image: ', science['procsci']
print 'Content of the "products" directory:'
os.listdir(products_path)



In [ ]: