| Science observation UT date(s) | 2010 Nov 01 |
| Data filename prefix(es) | S20101101S |
| Science | S20101101S | 83-86 | binning 2x2 |
| Processed Bias | for 20101101 | S20101101S0169_bias.fits | |
| Processed Twilight Flat | for 20101101 | S20101102S0100_flat_i.fits | |
| Processed Fringe Frame | for 20101101 | S20101111S0028_fringe_i.fits |
The fringe frame that matches the science target observations.
| Processed: S20101111S0028_fringe_i.fits | |||
| Fringe Frames | S20101111S | 28-30,40-43 | binning 2x2 |
| Processed Bias | for 20101111 | S20101111S0061_bias.fits | |
| Processed Twilight Flat | for 20101111 | S20101102S0100_flat_i.fits | |
The twilight flat that matches the science target observations and the fringe frames.
| Processed: S20101102S0100_flat_i.fits | |||
| Twilight Frames | S20101102S | 100-103,106-115 | binning 2x2 |
| Processed Bias | for 20101102 | S20101101S0169_bias.fits | |
The biases that match the science target observations, the twilight flats, and the fringe frames.
| Processed: S20101101S0169_bias.fits | |||
| Biases | S20101101S | 169-172 | binning 2x2 |
| S20101102S | 131-135 | binning 2x2 | |
| Processed: S20101111S0061_bias.fits | |||
| Biases | S20101111S | 61-65 | binning 2x2 |
The directory paths used in this notebook are assumed to match this structure:
rootdir/
calib/
bias/
flat/
fringe/
raw/
redux/The data processing will happen in the Notebook. It is also convenient to have a PyRAF interactive session ready to use if needed, eg. to imexam. In this step, we will configure the Notebook and the PyRAF session.
This step needs to be repeated everytime the Notebook and/or the PyRAF session is restarted.
In [ ]:
import os
import os.path
import shutil
# WARNING: Make sure that the directory structure given in Step 0
# is already in place.
# EDIT HERE - Directories
rootdir = '/data/workspace/DRWorkshopAAO/NGC6872'
# EDIT HERE - File numbers and file names
biases = {
'bias_20101101' : {
'procbias' : 'S20101101S0169_bias.fits',
'frames' : [('S20101101S', '169-172'),
('S20101102S', '131-135'),
]
},
'bias_20101111' : {
'procbias' : 'S20101111S0061_bias.fits',
'frames' : [('S20101111S', '61-65')],
}
}
flats = {
'flat_20101102' : {
'procflat' : 'S20101102S0100_flat_i.fits',
'frames' : [('S20101102S', '100-103,106-115')],
'flatbias' : biases['bias_20101101']['procbias'],
}
}
fringes = {
'fringe_20101111' : {
'procfringe' : 'S20101111S0028_fringe_i.fits',
'frames' : [('S20101111S', '28-30,40-43')],
'fringebias' : biases['bias_20101111']['procbias'],
'fringeflat' : flats['flat_20101102']['procflat'],
}
}
science = {
'procsci' : 'ngc6872_i.fits',
'sciroot' : 'S20101101S',
'sciframes' : '83-86',
'scibias' : biases['bias_20101101']['procbias'],
'sciflat' : flats['flat_20101102']['procflat'],
'scifringe' : fringes['fringe_20101111']['procfringe'],
}
logfile = 'NGC6872_i.log'
# Automatically set directories to match structure from Step 0
raw_path = os.path.join(rootdir, 'raw') + '/'
redux_path = os.path.join(rootdir, 'redux') + '/'
procbias_path = os.path.join(rootdir, 'calib', 'bias') + '/'
procflat_path = os.path.join(rootdir, 'calib', 'flat') + '/'
procfringe_path = os.path.join(rootdir, 'calib', 'fringe') + '/'
products_path = os.path.join(rootdir, 'products') + '/'
# Change to work directory
os.chdir(redux_path)
---> LAUNCH DS9 before proceeding further
Load the package required for the Notebook session and reset tasks to the default parameters.
In [ ]:
from pyraf import iraf
iraf.gemini()
iraf.gmos()
iraf.unlearn(iraf.gemini, iraf.gmos, iraf.gemtools)
iraf.gmos.logfile = logfile
iraf.set(stdimage='imtgmos')
The commands shown here need to be run from the shell.
Open an xterm or a Terminal, then:
cd /data/workspace/DRWorkshopAAO/NGC6872/redux
pyraf
Then in the PyRAF session:
gemini
gmos
unlearn gemini
unlearn gmos
unlearn gemtools
Get your PyRAF configured. In the PyRAF session:
iraf.gmos.logfile = "NGC6872_i.log"
set stdimage=imtgmos
In [ ]:
os.chdir(redux_path)
# delete any pre-existing lists
iraf.delete('*.lis', verify='no')
# biases
for (root, frames) in biases['bias_20101101']['frames']:
iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
iraf.concat('tmp_*.lis', 'bias_20101101.lis')
iraf.delete('tmp_*.lis', verify='no')
for (root, frames) in biases['bias_20101111']['frames']:
iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
iraf.concat('tmp_*.lis', 'bias_20101111.lis')
iraf.delete('tmp_*.lis', verify='no')
# flats
for (root, frames) in flats['flat_20101102']['frames']:
iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
iraf.concat('tmp_*.lis', 'flat_20101102.lis')
iraf.delete('tmp_*.lis', verify='no')
# fringes
for (root, frames) in fringes['fringe_20101111']['frames']:
iraf.gemlist(root, frames, Stdout='tmp_'+root+'.lis')
iraf.concat('tmp_*.lis', 'fringe_20101111.lis')
iraf.delete('tmp_*.lis', verify='no')
# science
iraf.gemlist(science['sciroot'], science['sciframes'], \
Stdout='sci.lis')
# Get a full list
iraf.concat('*.lis', 'all.lis')
In [ ]:
all_files = open('all.lis', 'r')
for line in all_files:
image = line.strip() + '[1]'
print image
iraf.gdisplay(raw_path + image, 1, fl_paste='no')
iraf.sleep(5)
all_files.close()
This will produce the two biases defined above. Those two biases will be used later to process the science frames, the twilight flats, and the fringe frames.
In [ ]:
os.chdir(redux_path)
for biaskey in biases.keys():
iraf.imdelete(biases[biaskey]['procbias'], verify='no')
iraf.imdelete('g//@'+biaskey+'.lis', verify='no')
print 'Creating bias ', biases[biaskey]['procbias']
iraf.gbias('@'+biaskey+'.lis', outbias=biases[biaskey]['procbias'], \
rawpath=raw_path, fl_over='yes', fl_trim='yes', \
fl_vardq='yes')
shutil.copy(biases[biaskey]['procbias'], procbias_path)
# The processed biases are:
os.listdir(procbias_path)
In [ ]:
# WARNING: interactive. It launches imexam. Type 'q' to quit.
for biaskey in biases.keys():
iraf.gdisplay(biases[biaskey]['procbias'], 1, fl_paste='yes')
This will produce the one twilight flat defined above. That flat will be used later to process the science frames and the fringe frames.
In [ ]:
os.chdir(redux_path)
for flatkey in flats.keys():
iraf.imdelete('g//@'+flatkey+'.lis', verify='no')
iraf.imdelete('rg//@'+flatkey+'.lis', verify='no')
flatlist = flatkey+'.lis'
procbias = os.path.join(procbias_path, flats[flatkey]['flatbias'])
print 'Processing raw twilight flats'
# gprepare and bias correct the raw flats
# The output files have the 'rg' prefix.
iraf.gireduce('@'+flatlist, rawpath=raw_path, \
fl_over='yes', fl_trim='yes', \
fl_bias='yes', bias=procbias, fl_flat='no', \
fl_vardq='yes')
In [ ]:
os.chdir(redux_path)
for flatkey in flats.keys():
iraf.imdelete(flats[flatkey]['procflat'], verify='no')
flatlist = flatkey+'.lis'
procflat = flats[flatkey]['procflat']
print 'Creating twilight flat ', procflat
# Create the processed flat
iraf.giflat('rg//@'+flatlist, outflat=procflat, fl_vardq='yes')
# Copy the processed flat to the calibration directory
shutil.copy(procflat, procflat_path)
# The processed flats are:
os.listdir(procflat_path)
In [ ]:
for flatkey in flats.keys():
iraf.gdisplay(flats[flatkey]['procflat'], 1, fl_paste='yes')
In [ ]:
os.chdir(redux_path)
for fringekey in fringes.keys():
iraf.imdelete('g//@'+fringekey+'.lis', verify='no')
iraf.imdelete('rg//@'+fringekey+'.lis', verify='no')
fringelist = fringekey+'.lis'
procflat = os.path.join(procflat_path, fringes[fringekey]['fringeflat'])
procbias = os.path.join(procbias_path, fringes[fringekey]['fringebias'])
print 'Processing raw fringe frames'
# gprepare, bias correct, and flat field the raw fringe frames
iraf.gireduce('@'+fringelist, rawpath=raw_path, \
fl_over='yes', fl_trim='yes', \
fl_bias='yes', bias=procbias, \
fl_flat='yes', flat1=procflat, \
fl_vardq='yes')
In [ ]:
os.chdir(redux_path)
for fringekey in fringes.keys():
iraf.imdelete(fringes[fringekey]['procfringe'], verify='no')
fringelist = fringekey+'.lis'
procfringe = fringes[fringekey]['procfringe']
print 'Creating fringe frame ', procfringe
# Create the processed fringe
iraf.gifringe('rg@'+fringelist, outimage=procfringe, fl_vardq='yes')
# Copy the processed fringe to the calibration directory
shutil.copy(procfringe, procfringe_path)
# The processed fringes are:
os.listdir(procfringe_path)
In [ ]:
for fringekey in fringes.keys():
iraf.gdisplay(fringes[fringekey]['procfringe'], 1, fl_paste='yes')
In the redder bands, for some CCDs, the fringing can be important. With the pre-Hamamatsu CCDs, that is the case for the i-band and the z-band. We are here reducing i-band data, so we will have to correct for the fringing.
Also, because our target is extended, it is not possible to use the on-target frames to produce the fringe frame. We have to use 'blank field' observations taken a few nights later to create a fringe frame. Because the fringe data was obtained at another time, we will need to measure the appropriate scale factor and scale it accordingly prior to removing it from the science observations.
Here, we first do the straighforward processing, i.e. the bias and flat correction. Then we deal with the fringe correction.
In [ ]:
os.chdir(redux_path)
iraf.imdelete('g//@sci.lis', verify='no')
iraf.imdelete('rg//@sci.lis', verify='no')
iraf.gireduce('@sci.lis', rawpath=raw_path, fl_over='yes', fl_trim='yes', \
fl_bias='yes', bias=science['scibias'], \
fl_flat='yes', flat1=science['sciflat'], \
fl_vardq='yes')
In [ ]:
sciframes = open('sci.lis', 'r')
for line in sciframes:
frame = line.strip()
iraf.gdisplay('rg'+frame, 1, fl_paste='yes')
sciframes.close()
As explained in the section's introduction, the fringe frames were obtained on another night. Therefore, there will certainly be a need to scale the processed fringe frame to the science observations for proper fringe pattern removal.
girmfringe can try to calculate a scale factor for you based on image statistics. For fine-grain adjustment, one can also feed girmfringe manually measured scale factor. Here, we will let girmfringe figure it out for us.
However, it will need some help. The default section used for the statistics is the full center CCD. Well, that's where our big and bright galaxy is. We won't get good sky statistics from that. So we need to inspect the image and select a good section to calculate statistics on.
In [ ]:
# get the file name for the first science frame
# (we just need to inspect one of them)
# Here's a little useful trick.
first_sci = iraf.head('sci.lis', nlines=1, Stdout=1)[0].strip()
iraf.display('rg'+first_sci+'[sci,2]', 1)
# Select a good section to represent the sky.
# I recommend [50:350,450:750]
In [ ]:
skysection = '[SCI,2][50:350,450:750]'
iraf.imdelete('frg//@sci.lis', verify='no')
iraf.girmfringe('rg//@sci.lis', science['scifringe'], \
fl_statscale='yes', statsec=skysection, \
fl_propfvardq='yes')
In [ ]:
sciframes = open('sci.lis', 'r')
for line in sciframes:
frame = line.strip()
iraf.gdisplay('frg'+frame, 1, fl_paste='yes')
sciframes.close()
In [ ]:
first_sci = iraf.head('sci.lis', nlines=1, Stdout=1)[0].strip()
iraf.gdisplay('frg'+frame, 1, z2=20000, fl_paste='yes')
In [ ]:
iraf.imdelete('mfrg//@sci.lis', verify='no')
iraf.gmosaic('frg//@sci.lis', fl_vardq='yes')
In [ ]:
sciframes = open('sci.lis', 'r')
for line in sciframes:
frame = line.strip()
iraf.gdisplay('mfrg'+frame, 1, z2=20000, fl_paste='yes')
sciframes.close()
In [ ]:
iraf.imdelete(science['procsci'], verify='no')
iraf.imcoadd('mfrg//@sci.lis', outimage=science['procsci'], logfile=logfile)
In [ ]:
iraf.display(science['procsci']+'[sci,1]', 1, \
zrange='no', zscale='no', z1=6000, z2=20000)
shutil.copy(science['procsci'], products_path)
# Final reduced image
os.listdir(products_path)
In [ ]: