First, ensure that your environment matches a standard DESI environment. For example:
module unload desimodules
source /project/projectdirs/desi/software/desi_environment.sh 18.7desitarget relies on desiutil and desimodel, so you may also need to set up a wider DESI environment, as detailed at:
https://desi.lbl.gov/trac/wiki/Pipeline/GettingStarted/Laptop/JuneMeeting
It may also be useful to set up some additional environment variables that are used in some versions of the desitarget code (you could also place these in your .bash_profile.ext file):
export DESIMODEL=$HOME/git/desimodel
export DUST_DIR=/project/projectdirs/desi/software/edison/dust/v0_1/maps
export GAIA_DIR=/project/projectdirs/desi/target/gaia_dr2Here, I've set DESIMODEL to a reasonable location. For a more detailed description of checking out the desimodel data files from svn see:
https://desi.lbl.gov/trac/wiki/Pipeline/GettingStarted/Laptop/JuneMeeting#Datafilesfordesimodel
The critical values that select_targets produces are the DESI_TARGET, BGS_TARGET and MWS_TARGET bit masks, which contain the target bits for the DESI main (or "dark time") survey and the Bright Galaxy Survey and Milky Way Survey respectively. Let's examine the masks that correspond to these surveys.
In [91]:
from desitarget.targets import desi_mask, bgs_mask, mws_mask
print(desi_mask)
The mask contains the name of the target bit (e.g. ELG) the bit value to which that name corresponds (e.g. 1, meaning 2-to-the-power-1), a description of the target (e.g. "ELG") and a dictionary of values that contain information for fiber assignment, such as the observing conditions allowed for the target, the initial priority with which the target class should be observed, and the initial number of observations for the target class. Note that these bits of information can be accessed individually in a number of ways:
In [92]:
desi_mask["QSO"], desi_mask.QSO # ADM different ways of accessing the bit values.
Out[92]:
In [93]:
desi_mask.names() # ADM the names of each target type.
Out[93]:
In [94]:
desi_mask.names(7) # ADM the names of target classes that correspond to an integer value of 5.
# ADM note that 7 is 2**0 + 2**1 + 2**2.
Out[94]:
In [95]:
desi_mask.bitnum("SKY") # ADM the integer value that corresponds to the "SKY" bit.
Out[95]:
In [96]:
names = desi_mask.names()
bitnums = [desi_mask.bitnum(name) for name in names]
bitvals = [desi_mask[name] for name in names]
list(zip(names,bitnums,bitvals)) # ADM the bit and integer value for each defined name.
Out[96]:
In [97]:
desi_mask["LRG"].priorities # ADM a dictionary of initial priorities for the LRG target class.
Out[97]:
In [98]:
desi_mask["LRG"].obsconditions, desi_mask["LRG"].numobs, desi_mask["LRG"].priorities["MORE_ZGOOD"]
Out[98]:
There are corresponding masks for the BGS and MWS, which can be accessed in the same way, e.g.:
In [99]:
bgs_mask.names()
Out[99]:
In [100]:
mws_mask.names()
Out[100]:
In addition to the DESI Main Survey, desitarget produces targets for Commissioning ("CMX") and Survey Validation ("SV"). The CMX and SV bitmasks can be obtained and examined as follows (other manipulation of these masks is similar to the previous sub-section, above):
In [101]:
from desitarget.cmx.cmx_targetmask import cmx_mask
print(cmx_mask)
In [102]:
from desitarget.sv1.sv1_targetmask import desi_mask, bgs_mask, mws_mask
desi_mask.names()
Out[102]:
Target classes have evolved throughout the history of the desitarget code, and the bits that correspond to those targets have thus occasionally changed. It is therefore critical that you use the same version of desitarget when working with bits in a target file as was used to create that target file!
For example, say you are working with commissioning targets that were created with version 0.X.X of desitarget. The correct version of Git to use to study this file can be obtained via:
git checkout 0.X.Xand the corresponding .yaml file online on GitHub would be:
https://github.com/desihub/desitarget/blob/0.X.X/py/desitarget/cmx/data/cmx_targetmask.yaml
For example, for version 0.31.1 of desitarget issue:
git checkout 0.31.1or look at:
https://github.com/desihub/desitarget/blob/0.31.1/py/desitarget/cmx/data/cmx_targetmask.yaml
Equivalently, for version 0.31.1 of desitarget for SV or the Main Survey:
https://github.com/desihub/desitarget/blob/0.31.1/py/desitarget/sv1/data/sv1_targetmask.yaml
https://github.com/desihub/desitarget/blob/0.31.1/py/desitarget/data/targetmask.yaml
Commissioning targeting bits are expected to be final as of version 0.32.0 of desitarget. SV and Main Survey bits may not yet be final.
The target files produced by select_targets contain many quantities from the Legacy Surveys data model sweeps files at, e.g.:
http://www.legacysurvey.org/dr7/files/#sweep-7-0-sweep-brickmin-brickmax-fits
The main columns added by select_targets are DESI_TARGET, BGS_TARGET and MWS_TARGET, which contain the output bitmasks from target selection. Let's take a closer look at how these columns can be used in conjunction with the bitmasks.
First, enter the Python prompt. Now, let's read in a file of targets. I'll assume you're working at NERSC, but set targdir, below, to wherever you have a targets- file.
In [125]:
import os
from glob import glob
from astropy.io.fits import getdata
import numpy as np
# ADM replace this with any directory you know of that holds targets.
targdir = "/project/projectdirs/desi/target/catalogs/examples"
# ADM replace this with the name of any target file.
targfile = 'targets.fits'
targfile = os.path.join(targdir, targfile)
targs = getdata(targfile)
Note that if you took the file from my the examples directory, then you're using an example file that only contains a subset of columns.
In [126]:
print(targs.dtype)
Let's consider the value of DESI_TARGET for the forty-second target:
In [127]:
targ = targs[41]
print(targ["DESI_TARGET"])
What does this number mean? Well, let's see which target classes are defined by this integer:
In [128]:
from desitarget.targets import desi_mask
desi_mask.names(targ["DESI_TARGET"])
Out[128]:
Now let's see what target classes are include for the first 10 targets:
In [129]:
bitnames = np.array(desi_mask.names()) # ADM note the array conversion to help manipulation.
bitvals = [desi_mask[name] for name in bitnames]
for targ in targs[:10]:
w = np.where( (targ["DESI_TARGET"] & bitvals) != 0)[0]
print(targ["DESI_TARGET"], bitnames[w])
So far, we've looked at the target class for each target. Now, let's just extract target classes that correspond to a certain bit. For example, which of the first 10 targets have the 'BGS_ANY' bit set?
In [130]:
np.where((targs[:10]["DESI_TARGET"] & desi_mask["BGS_ANY"]) != 0)[0]
Out[130]:
Which of all of the targets are both ELG and quasar targets?
In [131]:
isELG = (targs["DESI_TARGET"] & desi_mask["ELG"]) != 0
isQSO = (targs["DESI_TARGET"] & desi_mask["QSO"]) != 0
np.where(isELG & isQSO)[0]
Out[131]:
Alternatively, more compactly"
In [132]:
bitvalboth = desi_mask["ELG"] + desi_mask["QSO"]
np.where(targs["DESI_TARGET"] & bitvalboth == bitvalboth)[0]
Out[132]:
You should note that the forty-second target studied above pops up in these lists!
Note that desi_mask contains a couple of special bits that simply denote whether a target is a BGS or MWS target. These are called BGS_ANY and MWS_ANY. For example:
In [133]:
print((targs[:10]["DESI_TARGET"] & desi_mask["BGS_ANY"]) != 0)
print(targs[:10]["BGS_TARGET"] != 0)
Bits representing targets for the Bright Galaxy Survey and Milky Way Survey can be manipulated in the same way as previous examples in this section. The relevant columns and masks are BGS_TARGET and bgs_mask, and MWS_TARGET and mws_mask respectively. For example:
In [134]:
from desitarget.targets import bgs_mask, mws_mask
bitnames = np.array(bgs_mask.names()) # ADM note the array conversion to help manipulation.
bitvals = [bgs_mask[name] for name in bitnames]
for targ in targs[:10]:
w = np.where( (targ["BGS_TARGET"] & bitvals) != 0)[0]
print(targ["BGS_TARGET"], bitnames[w])
As noted in the previous section, Commissioning and SV have different bitmasks. Conveniently, commissioning and SV also have different _TARGET column names, allowing a user to easily distinguish which "flavor" of file they are using:
Main Survey files have the columns DESI_TARGET, BGS_TARGET and MWS_TARGET.
Commissioning files have the column CMX_TARGET.
SV files have the columns SV1_DESI_TARGET, SV1_BGS_TARGET and SV1_MWS_TARGET.
A convenient utility is desitarget.targets.main_cmx_or_sv which will use the differing column names to load the appropriate mask or masks. For example, using our Main Survey example file:
In [147]:
import os, fitsio
# ADM replace this with any directory you know of that holds targets.
targdir = "/project/projectdirs/desi/target/catalogs/examples"
# ADM replace this with the name of any target file.
targfile = 'targets.fits'
targfile = os.path.join(targdir, targfile)
targs = fitsio.read(targfile)
# ADM load the convenient utility and use it.
from desitarget.targets import main_cmx_or_sv
[desi_target, bgs_target, mws_target], [desi_mask, bgs_mask, mws_mask], surv = main_cmx_or_sv(targs)
print(desi_target, mws_target)
In [148]:
print(surv)
In [149]:
print(bgs_mask)
Let's see what would happen if our targets file was actually an SV file:
In [150]:
import numpy.lib.recfunctions as rfn
for col in [desi_target, bgs_target, mws_target]:
sv1_targs = rfn.rename_fields(targs, {col: 'SV1_'+col})
[desi_target, bgs_target, mws_target], [desi_mask, bgs_mask, mws_mask], surv = main_cmx_or_sv(sv1_targs)
print(bgs_target, mws_target)
In [151]:
print(surv)
In [152]:
print(desi_mask)
In [153]:
from desitarget.targets import desi_mask
isLRG = (targs["DESI_TARGET"] & desi_mask["LRG"]) != 0
isQSO = (targs["DESI_TARGET"] & desi_mask["QSO"]) != 0
lrgs, qsos = targs[isLRG], targs[isQSO]
In [154]:
# ADM a sanity check.
for qso in qsos[:10]:
print(desi_mask.names(qso["DESI_TARGET"]))
We'll need the astropy spatial matching functions:
In [155]:
from astropy.coordinates import SkyCoord
from astropy import units as u
Convert the lrgs and quasars to SkyCoord objects:
In [156]:
clrgs = SkyCoord(lrgs["RA"], lrgs["DEC"], unit='degree')
cqsos = SkyCoord(qsos["RA"], qsos["DEC"], unit='degree')
Perform the match. Let's choose a radius of 1 arcminute:
In [157]:
matchrad = 20*u.arcsec
idlrgs, idqsos, sep, _ = cqsos.search_around_sky(clrgs, matchrad)
Finally, write out the matching lrgs and quasars, and the distance between them:
In [158]:
lrgmatch, qsomatch = lrgs[idlrgs], qsos[idqsos]
for i in range(len(lrgmatch)):
print("LRG coordinates: {:.4f} deg, {:.4f} deg".format(lrgmatch[i]["RA"], lrgmatch[i]["DEC"]))
print("QSO coordinates: {:.4f} deg, {:.4f} deg".format(qsomatch[i]["RA"], qsomatch[i]["DEC"]))
print("Angular separation: {:.4f} arcsec".format(sep.value[i]*3600))
So, there are a couple of LRG targets within 20 arcseconds of a quasar target.