Template for how the legacypipe/Tractor pipeline is setup and run on DECam data

Modify accordingly for PTF, ZTF, etc.

Python Stack

  • At NERSC do the following. This uses Ted Kisner's conda build for DESI imaging code then copies everything to your scratch so you can do "conda install ..." for any extra packages you need

In [ ]:
module use /global/common/$NERSC_HOST/contrib/desi/modulefiles
module load desiconda/20170719-1.1.9-imaging
conda create --prefix $CSCRATCH/conda-envs/20170719-1.1.9-imaging --file $DESICONDA/pkg_list.txt
source activate $CSCRATCH/conda-envs/20170719-1.1.9-imaging
  • Make sure it all works by running a test case

In [ ]:
# SF98 dust maps
export MYDIR=$CSCRATCH/repos
mkdir -p $MYDIR/dust/maps                                                           
cd $MYDIR/dust/maps
wget -c http://portal.nersc.gov/project/cosmo/temp/dstn/travis-ci/maps/SFD_dust_4096_ngp.fits                                                    
wget -c http://portal.nersc.gov/project/cosmo/temp/dstn/travis-ci/maps/SFD_dust_4096_sgp.fits                                                    
export DUST_DIR=$MYREPO/dust
# imaging pipeline
cd $MYDIR
git clone https://github.com/legacysurvey/legacypipe.git
cd legacypipe
git checkout f4fc46ea0

In [ ]:
cd py/test
wget https://raw.githubusercontent.com/legacysurvey/obiwan/master/py/obiwan/test/end_to_end/test_decam_rex.p
cd ../
python py/test_decam_rex.py

Setup that went into test/test_decam_rex.py

  • Environment vars
    • export DUST_DIR=$MYREPO/dust
    • export LEGACY_SURVEY_DIR=$MYREPO/legacypipe/py/test/testcase6
    • ignore WISE forced photometry for now
  • Input Files (stored in legacy_survey_dir)
    1. fits tables for the 0.25x0.25 deg2 chunks of sky, "bricks", to process (survey-bricks.fits.gz), bright star locations to avoid (tycho2.fits.gz), and photometric and astrometric statistics for each CCD (survey-ccds-1.fits.gz)
    2. directories containing the image (images/) and calibration (calib/sextractor, calib/psfex) files
  • Create the calibration files BEFORE running main() below

What this command does:


In [ ]:
main(args=['--brick', '1102p240', '--zoom', '500', '600', '650', '750',            
           '--force-all', '--no-write', '--no-wise',                                                   
           '--survey-dir', surveydir,                                              
           '--outdir', outdir])
  • "brick 1102p240" is in the survey-bricks.fits.gz table.
  • Legacypipe interpolates 0.25x0.25 deg2 to a 3600x3600 pixel grid
    • "zoom 500 600 ..." analyses only that slice of the full 3600x3600 region
  • "force-all, no-write, no-wise" run everything, dont be verbose, skip WISE forced photometry
  • "survey-dir, outdir" legacy_survey_dir and where to write all legacypipe/tractor outputs

The main() above does the following:

  • call the run_brick() function in legacypipe/py/legacypipe/runbrick.py
  • which runs the 6 "stages" of the legacypipe/Tractor pipeline
    1. stage_tims: read in all fits tables, images, and calibration data from legacy_survey_dir, save as Tractor Image Objects (tims)
    2. stage_image_coadds
    3. stage_srcs: detect sources at SN >= 6 using five matched filters
    4. stage_fitblobs: finds the best fitting point-source or galaxy model for each source. Each detection is pixel dilated to a group of contiguous pixels called a "blob", blobs are processed in parallel, each blob can have many sources
    5. stage_coadds: make coadded image, invvar, model, chi2, and depth images
    6. stage_wise_forced: use best fit models to do forced photometry on WISE imaging
    7. stage_writecat: write Tractor catalogues

REMEMBER: create the calibration files BEFORE running legacypipe

Code you'll need to write

  • decam.py

In [ ]:
# https://github.com/legacysurvey/legacypipe/blob/master/py/legacypipe/decam.py
# Edited to be barebones
from __future__ import print_function
import os
import numpy as np
import fitsio
from legacypipe.image import CalibMixin
from legacypipe.cpimage import CPImage
from legacypipe.survey import LegacySurveyData

'''
Code specific to images from the Dark Energy Camera (DECam).
'''

class DecamImage(CPImage, CalibMixin):
    '''
    A LegacySurveyImage subclass to handle images from the Dark Energy
    Camera, DECam, on the Blanco telescope.
    '''
    # background subtraction fitting spline to every 512 pixels, interpolating between
    splinesky_boxsize = 512

    def __init__(self, survey, t):
        super(DecamImage, self).__init__(survey, t)
        # Check that these are set properly
        #self.imgfn # image relative path starting from legacy_survey_dir/images/
        #self.dqfn # bad pixel image
        #self.wtfn # invvar image
        
        # zeropoint units in ADU/sec
        self.ccdzpt += 2.5 * np.log10(self.exptime)
        
    def __str__(self):
        return 'DECam ' + self.name

    @classmethod
    def nominal_zeropoints(self):
        # Units ADU/sec for a good night 2 years ago
        return dict(g = 25.08,
                    r = 25.29,
                    z = 24.92,)
    
    @classmethod
    def photometric_ccds(self, survey, ccds):
        """Remove exposures from survey-ccds-1.fits.gz if image quality not 
        good enough
        
        Args:
          ccds: the survey-ccds-1.fits.gz table
        """
        # Nominal zeropoints (DECam)
        z0 = self.nominal_zeropoints()
        z0 = np.array([z0[f[0]] for f in ccds.filter])
        # You can skipping removing any of them with:
        good = np.ones(len(ccds), bool)
        return np.flatnonzero(good)

    @classmethod
    def ccd_cuts(self, survey, ccds):
        return np.zeros(len(ccds), np.int32)

    def get_good_image_subregion(self):
        """Optional"""
        x0,x1,y0,y1 = None,None,None,None
        # x0,x1,y0,y1 = 100,1023,100,self.height-100
        return x0,x1,y0,y1

    def read_dq(self, header=False, **kwargs):
        """read bad pixel image and possibly its header"""
        print('Reading data quality from', self.dqfn, 'hdu', self.hdu)
        dq,hdr = self._read_fits(self.dqfn, self.hdu, header=True, **kwargs)
        if header:
            return dq,hdr
        else:
            return dq

    def read_invvar(self, clip=True, clipThresh=0.2, **kwargs):
        print('Reading inverse-variance from', self.wtfn, 'hdu', self.hdu)
        invvar = self._read_fits(self.wtfn, self.hdu, **kwargs)
        return invvar

    def run_calibs(self, psfex=True, sky=True, se=False,
                   funpack=False, fcopy=False, use_mask=True,
                   force=False, just_check=False, git_version=None,
                   splinesky=False):
        """Read psfex PSF model and splinesky model"""
        self.read_psf_model(0, 0, pixPsf=True, hybridPsf=True)
        self.read_sky_model(splinesky=splinesky)
  • WCS
    • write TPV formation keys to the header of each image, something like this
CTYPE2 = 'DEC--TPV' / WCS projection type for this axis CTYPE1 = 'RA---TPV' / WCS projection type for this axis CRVAL1 = 121.7780402905 / World coordinate on this axis CRVAL2 = 9.777517812217 / World coordinate on this axis CRPIX1 = 13422.2 / Reference pixel on this axis CRPIX2 = 6306.333 / Reference pixel on this axis CD1_1 = 4.088761751834E-08 / Linear projection matrix CD1_2 = 7.28654335298E-05 / Linear projection matrix CD2_1 = -7.286249453005E-05 / Linear projection matrix CD2_2 = 4.829789501779E-09 / Linear projection matrix PV1_7 = 0.0001292520097439 / Projection distortion parameter PV2_8 = -0.01085639046593 / Projection distortion parameter PV2_9 = 0.006447478695312 / Projection distortion parameter PV2_0 = -0.005865638033892 / Projection distortion parameter PV2_1 = 1.022934665377 / Projection distortion parameter PV2_2 = -0.009988184329701 / Projection distortion parameter PV2_3 = 0.0 / PV distortion coefficient PV2_4 = -0.02553759230415 / Projection distortion parameter PV2_5 = 0.02039038294948 / Projection distortion parameter PV2_6 = -0.009420461615663 / Projection distortion parameter PV2_7 = 0.009366132237292 / Projection distortion parameter PV1_6 = 0.01073408153216 / Projection distortion parameter PV2_10 = -0.002902607846633 / Projection distortion parameter PV1_4 = 0.007984725876406 / Projection distortion parameter PV1_3 = 0.0 / PV distortion coefficient PV1_2 = -0.01061382938526 / Projection distortion parameter PV1_1 = 1.015157192369 / Projection distortion parameter PV1_0 = 0.004301039435068 / Projection distortion parameter PV1_9 = 0.007785488836336 / Projection distortion parameter PV1_8 = -0.007287409639159 / Projection distortion parameter PV1_5 = -0.02011363647483 / Projection distortion parameter PV1_10 = -0.003686927828818 / Projection distortion parameter

That should be 99% of getting it running


In [ ]: