Gallery for DR6

The purpose of this notebook is to build the gallery for the sixth Legacy Survey data release, DR6. The theme of this gallery is...the NGC catalog!

For future reference: The notebook must be run from https://jupyter-dev.nersc.gov with the following (approximate) activation script:

#!/bin/bash                                                                                                           
version=$1                                                                                                            
connection_file=$2                                                                                                    

desiconda_version=20170818-1.1.12-img                                                                                 
module use /global/common/${NERSC_HOST}/contrib/desi/desiconda/$desiconda_version/modulefiles                         
module load desiconda                                                                                                 

export LEGACY_SURVEY_DIR=/global/cscratch1/sd/dstn/dr6plus                                                            
export LEGACYPIPE_DIR=$SCRATCH/repos/legacypipe                                                                       

export PATH=$LEGACYPIPE_DIR/bin:${PATH}                                                                               
export PATH=$SCRATCH//repos/build/bin:$PATH                                                                           
export PYTHONPATH=$LEGACYPIPE_DIR/py:${PYTHONPATH}                                                                    
export PYTHONPATH=$SCRATCH/repos/build/lib/python3.5/site-packages:$PYTHONPATH                                        

module use $LEGACYPIPE_DIR/bin/modulefiles/cori                                                                       
module load dust                                                                                                      

exec python -m ipykernel -f $connection_file

Imports and paths


In [1]:
import os, sys
import shutil, time, warnings
from contextlib import redirect_stdout
import numpy as np
import numpy.ma as ma
import matplotlib.pyplot as plt

In [2]:
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.table import Table, Column, vstack
from astropy.io import ascii
from PIL import Image, ImageDraw, ImageFont
#from astrometry.util.starutil_numpy import hmsstring2ra

In [3]:
from astrometry.util.util import Tan
from astrometry.util.fits import merge_tables
from legacypipe.survey import LegacySurveyData
from legacypipe.runbrick import run_brick

In [4]:
import multiprocessing
nproc = multiprocessing.cpu_count() // 2

In [5]:
%matplotlib inline

Preliminaries

Define the data release and the various output directories.


In [6]:
dr = 'dr6'
PIXSCALE = 0.262

In [7]:
gallerydir = os.path.join( os.getenv('SCRATCH'), dr, 'gallery' )
galleryfile = os.path.join(gallerydir, 'gallery-{}.fits'.format(dr))

In [8]:
jpgdir = os.path.join(gallerydir, 'jpg')
if not os.path.isdir(jpgdir):
    os.mkdir(jpgdir)

In [9]:
pngdir = os.path.join(gallerydir, 'png')
if not os.path.isdir(pngdir):
    os.mkdir(pngdir)

Read the Open NGC catalog created by Mattia Verga:

https://github.com/mattiaverga/OpenNGC

wget https://raw.githubusercontent.com/mattiaverga/OpenNGC/master/NGC.csv
  • Name: Object name composed by catalog + number NGC: New General Catalogue IC: Index Catalogue

  • Type: Object type

    • *: Star
    • **: Double star
    • *Ass: Association of stars
    • OCl: Open Cluster
    • GCl: Globular Cluster
    • Cl+N: Star cluster + Nebula
    • G: Galaxy
    • GPair: Galaxy Pair
    • GTrpl: Galaxy Triplet
    • GGroup: Group of galaxies
    • PN: Planetary Nebula
    • HII: HII Ionized region
    • DrkN: Dark Nebula
    • EmN: Emission Nebula
    • Neb: Nebula
    • RfN: Reflection Nebula
    • SNR: Supernova remnant
    • Nova: Nova star
    • NonEx: Nonexistent object
  • RA: Right Ascension in J2000 Epoch (HH:MM:SS.SS)

  • Dec: Declination in J2000 Epoch (+/-DD:MM:SS.SS)

  • Const: Constellation where the object is located

  • MajAx: Major axis, expressed in arcmin

  • MinAx: Minor axis, expressed in arcmin

  • PosAng: Major axis position angle (North Eastwards)

  • B-Mag: Apparent total magnitude in B filter

  • V-Mag: Apparent total magnitude in V filter

  • J-Mag: Apparent total magnitude in J filter

  • H-Mag: Apparent total magnitude in H filter

  • K-Mag: Apparent total magnitude in K filter

  • SurfBr (only Galaxies): Mean surface brigthness within 25 mag isophot (B-band), expressed in mag/arcsec2

  • Hubble (only Galaxies): Morphological type (for galaxies)

  • Cstar U-Mag (only Planetary Nebulae): Apparent magnitude of central star in U filter

  • Cstar B-Mag (only Planetary Nebulae): Apparent magnitude of central star in B filter

  • Cstar V-Mag (only Planetary Nebulae): Apparent magnitude of central star in V filter

  • M: cross reference Messier number

  • NGC: other NGC identification, if the object is listed twice in the catalog

  • IC: cross reference IC number, if the object is also listed with that identification

  • Cstar Names (only Planetary Nebulae): central star identifications

  • Identifiers: cross reference with other catalogs

  • Common names: Common names of the object if any

  • NED Notes: notes about object exported from NED

  • OpenNGC Notes: notes about the object data from OpenNGC catalog


In [10]:
names = ('name', 'type', 'ra_hms', 'dec_dms', 'const', 'majax', 'minax', 
         'pa', 'bmag', 'vmag', 'jmag', 'hmag', 'kmag', 'sbrightn', 'hubble', 
         'cstarumag', 'cstarbmag', 'cstarvmag', 'messier', 'ngc', 'ic', 
         'cstarnames', 'identifiers', 'commonnames', 'nednotes', 'ongcnotes')

In [11]:
NGC = ascii.read(os.path.join(gallerydir, 'NGC.csv'), delimiter=';', names=names)
NGC.write(os.path.join(gallerydir, 'NGC.fits'), overwrite=True)
NGC


Out[11]:
<Table masked=True length=13953>
nametypera_hmsdec_dmsconstmajaxminaxpabmagvmagjmaghmagkmagsbrightnhubblecstarumagcstarbmagcstarvmagmessierngciccstarnamesidentifierscommonnamesnednotesongcnotes
str13str6str11str11str3float64float64int64float64float64float64float64float64float64str4float64float64float64int64str10str14str51str211str56str80str97
IC0002G00:11:00.88-12:49:22.3Cet0.980.3214215.46--12.2611.4811.1723.45Sb--------------2MASX J00110081-1249206,IRAS 00084-1306,MCG -02-01-031,PGC 000778----B-Mag taken from LEDA.
IC0003G00:12:06.09-00:24:54.8Psc0.930.675315.1--11.5310.7910.5423.5E--------------2MASX J00120604-0024543,MCG +00-01-038,PGC 000836,SDSS J001206.08-002454.7,SDSS J001206.09-002454.7,SDSS J001206.09-002454.8,SDSS J001206.10-002454.8------
IC0004G00:13:26.94+17:29:11.2Peg1.170.841214.2--11.5110.6510.523.01Sc--------------2MASX J00132695+1729111,IRAS 00108+1712,MCG +03-01-029,PGC 000897,UGC 00123------
IC0005G00:17:34.93-09:32:36.1Cet0.990.66914.57--11.510.8510.523.4E--------------2MASX J00173495-0932364,MCG -02-01-047,PGC 001145,SDSS J001734.93-093236.0,SDSS J001734.93-093236.1----B-Mag taken from LEDA.
IC0006G00:18:55.04-03:16:33.9Psc1.231.0814614.5--11.0310.3210.0823.89E--------------2MASX J00185505-0316339,MCG -01-01-075,PGC 001228------
IC0007G00:18:53.16+10:35:40.9Psc0.90.6317414.7--11.3310.5710.2623.22S0--------------2MASX J00185316+1035410,PGC 001216------
IC0008G00:19:02.72-03:13:19.5Psc0.820.3412915.16--12.712.0812.0823.4E?--------------2MASX J00190272-0313196,MCG -01-01-076,PGC 001234----B-Mag taken from LEDA.
IC0009G00:19:43.98-14:07:18.8Cet0.590.4612215.41--12.3811.7111.2822.88Sa--------------2MASX J00194400-1407184,MCG -02-02-001,PGC 001271----B-Mag taken from LEDA.
IC0010G00:20:17.34+59:18:13.6Cas6.766.0312913.69.57.236.346.0124.53IB--------------2MASX J00201733+5918136,IRAS 00175+5902,IRAS 00177+5900,MCG +10-01-001,PGC 001305,UGC 00192--The 2MASX position refers to the center of the IR isophotes.--
IC0011Dup00:52:59.35+56:37:18.8Cas----------------------------0281------------
..............................................................................
NGC7831G00:07:19.53+32:36:33.3And1.80.394013.4--10.9510.169.8822.59Sb----------1530--2MASX J00071951+3236334,IRAS 00047+3219,MCG +05-01-032,PGC 000569,UGC 00060------
NGC7832G00:06:28.46-03:42:58.1Psc1.680.382313.0--10.359.699.4523.68E----------5386--2MASX J00062844-0342581,MCG -01-01-033,PGC 000485------
NGC7833Other00:06:31.46+27:38:22.8Peg--------------------------------------Five Galactic stars.--
NGC7834G00:06:37.82+08:22:04.4Psc0.930.831615.4--12.7412.2411.8523.94Sc--------------2MASX J00063780+0822045,MCG +01-01-030,PGC 000504,UGC 00049------
NGC7835G00:06:46.77+08:25:33.4Psc0.560.2415015.5--12.2811.511.21--Sb--------------2MASX J00064677+0825335,MCG +01-01-031,PGC 000505------
NGC7836G00:08:01.64+33:04:14.8And0.90.3913313.8--11.3210.6410.2822.8E?--------------2MASX J00080163+3304148,IRAS 00054+3247,PGC 000608,UGC 00065------
NGC7837G00:06:51.39+08:21:05.0Psc0.670.3816916.0--12.3911.8911.4624.01Sb--------------2MASX J00065141+0821045,MCG +01-01-035,PGC 000516------
NGC7838G00:06:53.95+08:21:03.1Psc1.30.579315.3--11.4810.8110.4824.37S0-a--------------2MASX J00065396+0821027,MCG +01-01-036,PGC 000525------
NGC7839**00:07:00.63+27:38:06.8Peg------------------------------------------
NGC7840G00:07:08.80+08:23:00.6Psc0.680.4612716.47--------24.36----------------2MASX J00070878+0822598,PGC 1345780------

Select the desired object types.

Here we choose ...


In [12]:
majax = ma.getdata(NGC['majax']) # arcmin
objtype = np.char.strip(ma.getdata(NGC['type']))

In [13]:
keeptype = ('G', 'PN', 'OCl', 'GCl', 'Cl+N') # Cl gives us GCl, OCl, and Cl+N
#keeptype = ('G', 'GPair', 'GTrpl', 'GGroup', 'PN', 'Cl') # Cl gives us GCl, OCl, and Cl+N
keep = np.zeros(len(NGC), dtype=bool)
for otype in keeptype:
    print('Working on {}'.format(otype))
    ww = [otype == tt for tt in objtype]
    keep = np.logical_or(keep, ww)


Working on G
Working on PN
Working on OCl
Working on GCl
Working on Cl+N

Require "big" objects, particularly the galaxies (to cut down the sample size).


In [14]:
galtoss = (objtype == 'G') * (majax < 3)
keep = np.logical_and(keep, (majax > 0.3) * (majax < 20))
keep = np.logical_and(keep, ~galtoss)

In [15]:
nobj = np.count_nonzero(keep)
print('Keeping {} / {} objects'.format(nobj, len(NGC)))
cat = NGC[keep]
cat


Keeping 1676 / 13953 objects
Out[15]:
<Table masked=True length=1676>
nametypera_hmsdec_dmsconstmajaxminaxpabmagvmagjmaghmagkmagsbrightnhubblecstarumagcstarbmagcstarvmagmessierngciccstarnamesidentifierscommonnamesnednotesongcnotes
str13str6str11str11str3float64float64int64float64float64float64float64float64float64str4float64float64float64int64str10str14str51str211str56str80str97
IC0010G00:20:17.34+59:18:13.6Cas6.766.0312913.69.57.236.346.0124.53IB--------------2MASX J00201733+5918136,IRAS 00175+5902,IRAS 00177+5900,MCG +10-01-001,PGC 001305,UGC 00192--The 2MASX position refers to the center of the IR isophotes.--
IC0166OCl01:52:23.82+61:51:09.4Cas7.5----13.011.7------------------------MWSC 0146------
IC0239G02:36:27.88+38:58:11.7And4.254.017512.1--9.549.098.7823.72SABc--------------2MASX J02362783+3858085,IRAS 02333+3845,MCG +06-06-065,PGC 009899,UGC 02080------
IC0284G03:06:09.91+42:22:18.9Per4.292.151413.8--10.039.819.2823.82Sd--------------2MASX J03060989+4222189,IRAS 03029+4211,MCG +07-07-023,PGC 011643,UGC 02531------
IC0289PN03:10:19.26+61:19:00.4Cas0.58----16.8--15.5915.6215.14------15.115.9--------IRAS 03062+6107,PK 138+02 1,PN G138.8+02.8------
IC0342G03:46:48.50+68:05:46.9Cam19.7718.79010.5--5.665.014.5624.85SABc--------------2MASX J03464851+6805459,C5,IRAS 03419+6756,MCG +11-05-003,PGC 013826,UGC 02847------
IC0348Cl+N03:44:34.19+32:09:46.2Per10.010.0--------------------------1985--BD +31 0643,HIP 017465,LBN 758,MWSC 0301omi Per Cloud----
IC0356G04:07:46.91+69:48:44.8Cam4.023.739013.3--7.166.336.0423.27Sb--------------2MASX J04074690+6948447,IRAS 04025+6940,MCG +12-04-011,PGC 014508,UGC 02953------
IC0361OCl04:18:50.70+58:14:58.2Cam7.2----12.9211.7------------------------MWSC 0362------
IC0446Cl+N06:31:06.18+10:27:33.4Mon5.04.0--------------------------2167--LBN 898, MWSC 0890------
..............................................................................
NGC7755G23:47:51.76-30:31:19.3Scl3.72.272712.111.369.929.298.9723.63Sc--------------2MASX J23475176-3031195,ESO 471-020,ESO-LV 471-0200,IRAS 23452-3048,MCG -05-56-014,PGC 072444,UGCA 443------
NGC7762OCl23:50:01.75+68:02:16.7Cep6.5--------------------------------MWSC 3784------
NGC7788OCl23:56:45.58+61:23:59.7Cas2.4------9.4------------------------MWSC 3777------
NGC7789OCl23:57:24.06+56:42:29.8Cas14.4----7.686.7------------------------MWSC 3779------
NGC7790OCl23:58:24.26+61:12:29.9Cas3.6----9.188.5------------------------MWSC 3781------
NGC7793G23:57:49.83-32:35:27.7Scl10.426.05899.749.287.567.026.8623.05Scd--------------2MASX J23574982-3235277,ESO 349-012,ESO-LV 349-0120,IRAS 23552-3252,MCG -06-01-009,PGC 073049--Extended HIPASS source--
NGC7795OCl23:58:37.44+60:02:05.9Cas2.22--------------------------------MWSC 3782------
NGC7801OCl00:00:21.44+50:44:42.0Cas4.2--------------------------------MWSC 0002------
NGC7814G00:03:14.89+16:08:43.5Peg4.371.8813512.0--8.097.367.0823.03Sab--------------2MASX J00031494+1608428,C43,MCG +03-01-020,PGC 000218,SDSS J000315.04+160844.7,SDSS J000315.04+160844.8,UGC 00008--Multiple SDSS entries describe this object.--
NGC7817G00:03:58.91+20:45:08.4Peg3.290.734612.8611.949.498.738.4223.0Sbc--------------2MASX J00035889+2045084,IRAS 00014+2028,MCG +03-01-021,PGC 000279,UGC 00019------

In [16]:
print(np.unique(ma.getdata(cat['type'])))
print(np.unique(ma.getdata(cat['hubble'])))


['Cl+N' 'G' 'GCl' 'OCl' 'PN']
['0' 'E' 'E-S0' 'I' 'IAB' 'IB' 'S0' 'S0-a' 'S?' 'SABa' 'SABb' 'SABc' 'SABd'
 'SABm' 'SBa' 'SBab' 'SBb' 'SBbc' 'SBc' 'SBcd' 'SBd' 'SBm' 'Sa' 'Sab' 'Sb'
 'Sbc' 'Sc' 'Scd' 'Sd' 'Sm']

In [17]:
ww = (cat['type'] == 'G')
_ = plt.hist(cat['majax'][ww], bins=100)


Convert coordinates in decimal degrees.


In [18]:
coord = SkyCoord(ra=cat['ra_hms'], dec=cat['dec_dms'], unit=(u.hourangle, u.deg))
cat.add_column(Column(name='ra', unit='deg', length=nobj))
cat.add_column(Column(name='dec', unit='deg', length=nobj))
cat['ra'] = coord.ra.value
cat['dec'] = coord.dec.value

Generate (find) the sample of objects in the DR6 footprint.


In [19]:
survey = LegacySurveyData()
survey.output_dir = gallerydir

In [20]:
def get_name(cat, nice=False):
    name = np.atleast_1d(ma.getdata(cat['name']))
    mess = np.atleast_1d(ma.getdata(cat['messier']))
    comm = np.atleast_1d(ma.getdata(cat['commonnames']))

    outname = []
    if nice:
        hubble_type = np.empty_like(name)
        for nn, mm, cc in zip(name, mess, comm):
            oo = nn.strip().replace('NED01', '').upper()
            if mm != 0:
                oo = '{} = M{}'.format(oo, mm)
            if cc != 0:
                oo = '{} = {}'.format(oo, str(cc).replace(',', ' = '))
            outname.append(oo)
    else:
        for nn in name:
            outname.append(nn.strip().replace(' ', '_').lower())
            
    if len(outname) == 1:
        outname = outname[0]
    return outname

In [21]:
def simple_wcs(obj, diam):
    """Build a simple WCS object for a single object."""
    size = np.rint(diam / PIXSCALE).astype('int') # [pixels]
    wcs = Tan(obj['ra'], obj['dec'], size/2+0.5, size/2+0.5,
                 -PIXSCALE/3600.0, 0.0, 0.0, PIXSCALE/3600.0, 
                 float(size), float(size))
    return wcs

In [22]:
def _build_sample_one(args):
    """Wrapper function for the multiprocessing."""
    return build_sample_one(*args)

In [23]:
def build_sample_one(obj, factor=0.5, verbose=False):
    """Wrapper function to find overlapping grz CCDs for a given object.
    
    """
    name = get_name(obj)
    print('Working on {}...'.format(name))
    diam = factor * ma.getdata(obj['majax']) * 60.0 # diameter [arcsec]
    wcs = simple_wcs(obj, diam)
    try:
        ccds = survey.ccds_touching_wcs(wcs) # , ccdrad=2*diam/3600)
    except:
        return None
    
    if ccds:
        # Is there 3-band coverage?
        if 'g' in ccds.filter and 'r' in ccds.filter and 'z' in ccds.filter:
            if verbose:
                print('For {} (type={}) found {} CCDs, RA = {:.5f}, Dec = {:.5f}, Diameter={:.4f} arcmin'.format(
                        obj['name'], obj['type'], len(ccds), obj['ra'], obj['dec'], obj['majax']))
            return obj
    return None

In [24]:
def build_sample(cat, factor=1.0):
    """Build the full sample with grz coverage in DR6."""

    sampleargs = list()
    for cc in cat:
        sampleargs.append( (cc, factor, True) ) # the False refers to verbose=False

    if nproc > 1:
        p = multiprocessing.Pool(nproc)
        result = p.map(_build_sample_one, sampleargs)
        p.close()
    else:
        result = list()
        for args in sampleargs:
            result.append(_build_sample_one(args))

    # Remove non-matching objects and write out the sample
    outcat = vstack(list(filter(None, result)))
    print('Found {}/{} objects in the DR6 footprint.'.format(len(outcat), len(cat)))
    
    return outcat

In [25]:
samplelogfile = os.path.join(gallerydir, 'build-sample.log')
print('Building the sample.')
print('Logging to {}'.format(samplelogfile))
t0 = time.time()
with open(samplelogfile, 'w') as log:
    with redirect_stdout(log):
        sample = build_sample(cat)
print('Found {}/{} objects in the DR6 footprint.'.format(len(sample), len(cat)))
print('Total time = {:.3f} seconds.'.format(time.time() - t0))


Building the sample.
Logging to /global/cscratch1/sd/ioannis/dr6/gallery/build-sample.log
Found 120/1676 objects in the DR6 footprint.
Total time = 4.715 seconds.

In [26]:
print('Writing {}'.format(galleryfile))
sample.write(galleryfile, overwrite=True)


Writing /global/cscratch1/sd/ioannis/dr6/gallery/gallery-dr6.fits

In [27]:
sample


Out[27]:
<Table masked=True length=120>
nametypera_hmsdec_dmsconstmajaxminaxpabmagvmagjmaghmagkmagsbrightnhubblecstarumagcstarbmagcstarvmagmessierngciccstarnamesidentifierscommonnamesnednotesongcnotesradec
str52str24str44str44str12float64float64int64float64float64float64float64float64float64str16float64float64float64int64str40str56str204str844str224str320str388float64float64
IC0712G11:34:49.31+49:04:39.7UMa3.020.899114.8--10.8410.179.8925.42E--------------2MASX J11344932+4904388,MCG +08-21-063,PGC 035785,SDSS J113449.29+490439.4,SDSS J113449.29+490439.5,SDSS J113449.29+490439.7,SDSS J113449.30+490439.5------173.70545833349.0776944444
IC3687G12:42:15.10+38:30:12.0CVn3.343.08015.5--------25.04IAB--------------MCG +07-26-039,PGC 042656,UGC 07866------190.56291666738.5033333333
IC4182G13:05:49.54+37:36:17.6CVn6.015.52011.86--------25.09Sm--------------MCG +06-29-031,PGC 045314,SDSS J130548.70+373613.0,UGC 08188------196.45641666737.6048888889
NGC2146G06:18:37.71+78:21:25.3Cam5.314.3212311.3810.598.237.427.0622.76SBab--------------2MASX J06183771+7821252,IRAS 06106+7822,MCG +13-05-022,PGC 018797,UGC 03429------94.65712578.3570277778
NGC2281OCl06:48:17.84+41:04:43.9Aur10.8----6.055.4------------------------MWSC 0989------102.07433333341.0788611111
NGC2340G07:11:10.83+50:10:29.1Lyn3.161.647113.9--9.959.268.8824.06E--------------2MASX J07111080+5010288,MCG +08-13-096,PGC 020338,UGC 03720------107.79512550.17475
NGC2419GCl07:38:07.95+38:52:47.9Lyn4.5------10.05------------------------2MASX J07380795+3852479,C25,MWSC 1290------114.53312538.8799722222
NGC2424G07:40:39.29+39:13:59.9Lyn3.370.528213.9--10.389.629.3223.77Sb--------------2MASX J07403931+3914003,IRAS 07372+3920,MCG +07-16-009,PGC 021558,SDSS J074039.28+391359.8,SDSS J074039.28+391359.9,SDSS J074039.29+391359.9,UGC 03959------115.16370833339.2333055556
NGC2541G08:14:40.12+49:03:42.2Lyn3.021.617012.2611.810.8710.210.0922.85SABc--------------2MASX J08144007+4903411,IRAS 08110+4912,MCG +08-15-054,PGC 023110,SDSS J081440.11+490342.1,UGC 04284------123.66716666749.0617222222
NGC2681G08:53:32.74+51:18:49.2UMa3.973.977011.0910.298.417.717.4522.88S0-a--------------2MASX J08533273+5118493,IRAS 08500+5130,MCG +09-15-041,PGC 024961,UGC 04645------133.38641666751.3136666667
....................................................................................
NGC6015G15:51:25.23+62:18:36.1Dra5.812.572811.6911.149.318.718.4723.48Sc--------------2MASX J15512523+6218361,IRAS 15506+6227,MCG +10-23-003,PGC 056219,UGC 10075------237.85512562.3100277778
NGC6058PN16:04:26.54+40:40:59.0Her0.38----13.3------------12.3713.5513.91------UCAC2 45993562IRAS 16027+4049,PN G064.6+48.2,SDSS J160426.54+404058.9------241.11058333340.6830555556
NGC6205GCl16:41:41.63+36:27:40.7Her16.5------5.84.453.943.85----------13------2MASX J16414163+3627407,MWSC 2445Hercules Globular Cluster----250.42345833336.4613055556
NGC6207G16:43:03.75+36:49:56.7Her3.461.611712.1911.6510.089.429.1222.66Sc--------------2MASX J16430375+3649567,IRAS 16412+3655,MCG +06-37-007,PGC 058827,SDSS J164303.66+364955.7,UGC 10521------250.76562536.8324166667
NGC6223G16:43:04.31+61:34:44.1Dra3.162.339013.1--10.099.349.1124.18E-S0--------------2MASX J16430433+6134441,MCG +10-24-040,PGC 058828,UGC 10527------250.76795833361.5789166667
NGC6229GCl16:46:58.86+47:31:40.1Her4.8----9.389.867.987.437.33------------------MWSC 2460------251.7452547.5278055556
NGC6255G16:54:47.95+36:30:04.0Her3.11.379513.4--13.2312.6412.8524.17SBc--------------2MASX J16544796+3630031,MCG +06-37-014,PGC 059244,UGC 10606------253.69979166736.5011111111
NGC6341GCl17:17:07.27+43:08:11.5Her14.4------6.52----------------92------MWSC 2557------259.28029166743.1365277778
NGC6742PN18:59:19.90+48:27:55.0Dra0.45----15.0--------------------------PN G078.5+18.7------284.83291666748.4652777778
NGC6832OCl19:48:15.26+59:25:16.2Cyg3.3--------------------------------MWSC 3183------297.06358333359.4211666667

In [28]:
print(np.unique(sample['type']).data)


['G' 'GCl' 'OCl' 'PN']

In [29]:
def qa_sample():
    fig, ax = plt.subplots()
    ax.scatter(cat['ra'], cat['dec'], alpha=0.5, s=10, label='Trimmed NGC Catalog')
    ax.scatter(sample['ra'], sample['dec'], s=20, label='Objects in DR6 Footprint')
    ax.set_xlabel('RA')
    ax.set_ylabel('Dec')
    ax.legend(loc='lower right')

In [30]:
qa_sample()


Generate the color mosaics for each object.


In [31]:
def custom_brickname(obj, prefix='custom-'): 
    brickname = 'custom-{:06d}{}{:05d}'.format(
        int(1000*obj['ra']), 'm' if obj['dec'] < 0 else 'p', 
        int(1000*np.abs(obj['dec'])))
    return brickname

In [32]:
def get_factor(objtype):
    """Scale factors for the mosaics."""
    ref = dict(
        G = 2,
        GCl = 2,
        OCl = 2,
        PN = 4,
    )
    return ref[objtype]

In [33]:
def make_coadds_one(obj, scale=PIXSCALE, clobber=False):
    name = get_name(obj)
    jpgfile = os.path.join(jpgdir, '{}.jpg'.format(name))
    if os.path.isfile(jpgfile) and not clobber:
        print('File {} exists...skipping.'.format(jpgfile))
    else:
        factor = get_factor(obj['type'])
        diam = factor * ma.getdata(obj['majax']) * 60.0 # diameter [arcsec]
        
        size = np.rint(diam / scale).astype('int') # [pixels]
        print('Generating mosaic for {} (type={}) with width={} pixels.'.format(name, obj['type'], size))
        
        with warnings.catch_warnings():
            warnings.simplefilter("ignore")
            run_brick(None, survey, radec=(obj['ra'], obj['dec']), pixscale=scale, 
                      width=size, height=size, stages=['image_coadds'], splinesky=True,
                      early_coadds=True, pixPsf=True, hybridPsf=True, normalizePsf=True,
                      write_pickles=False, depth_cut=False, apodize=True, threads=nproc,
                      do_calibs=False, ceres=False)
            
        sys.stdout.flush()    
        brickname = custom_brickname(obj, prefix='custom-')
        _jpgfile = os.path.join(survey.output_dir, 'coadd', 'cus', brickname, 
                               'legacysurvey-{}-image.jpg'.format(brickname))
        shutil.copy(_jpgfile, jpgfile)
        shutil.rmtree(os.path.join(survey.output_dir, 'coadd'))

In [34]:
def make_coadds(sample, clobber=False):
    for obj in sample:
        make_coadds_one(obj, clobber=clobber)

In [35]:
#make_coadds_one(sample[111], clobber=True)

In [36]:
coaddslogfile = os.path.join(gallerydir, 'make-coadds.log')
print('Generating the coadds.')
print('Logging to {}'.format(coaddslogfile))
t0 = time.time()
with open(coaddslogfile, 'w') as log:
    with redirect_stdout(log):
        make_coadds(sample, clobber=False)
print('Total time = {:.3f} minutes.'.format((time.time() - t0) / 60))


Generating the coadds.
Logging to /global/cscratch1/sd/ioannis/dr6/gallery/make-coadds.log
Total time = 5.100 minutes.

Add labels and a scale bar.


In [37]:
barlen = np.round(60.0 / PIXSCALE).astype('int')
fonttype = os.path.join(gallerydir, 'Georgia.ttf')

In [38]:
def _add_labels_one(args):
    """Wrapper function for the multiprocessing."""
    return add_labels_one(*args)

In [39]:
def add_labels_one(obj, verbose=False):
    name = get_name(obj)
    nicename = get_name(obj, nice=True)
    
    jpgfile = os.path.join(jpgdir, '{}.jpg'.format(name))
    pngfile = os.path.join(pngdir, '{}.png'.format(name))
    thumbfile = os.path.join(pngdir, 'thumb-{}.png'.format(name))
        
    im = Image.open(jpgfile)
    sz = im.size
    fntsize = np.round(sz[0]/28).astype('int')
    width = np.round(sz[0]/175).astype('int')
    font = ImageFont.truetype(fonttype, size=fntsize)
    draw = ImageDraw.Draw(im)
    # Label the object name--
    draw.text((0+fntsize*2, 0+fntsize*2), nicename, font=font)
    # Add a scale bar--
    x0, x1, yy = sz[1]-fntsize*2-barlen, sz[1]-fntsize*2, sz[0]-fntsize*2
    draw.line((x0, yy, x1, yy), fill='white', width=width)
    im.save(pngfile)    
        
    # Generate a thumbnail
    cmd = '/usr/bin/convert -thumbnail 300x300 {} {}'.format(pngfile, thumbfile)
    os.system(cmd)

In [40]:
def add_labels(sample):
    labelargs = list()
    for obj in sample:
        labelargs.append((obj, False))

    if nproc > 1:
        p = multiprocessing.Pool(nproc)
        res = p.map(_add_labels_one, labelargs)
        p.close()
    else:
        for args in labelargs:
            res = _add_labels_one(args)

In [41]:
%time add_labels(sample)


CPU times: user 319 ms, sys: 1.08 s, total: 1.4 s
Wall time: 32.6 s

To test the webpage before release, do

  • rsync -auvP /global/cscratch1/sd/ioannis/dr6/gallery/png /global/project/projectdirs/cosmo/www/temp/ioannis/dr6/gallery/
  • rsync -auvP /global/cscratch1/sd/ioannis/dr6/gallery/*.html /global/project/projectdirs/cosmo/www/temp/ioannis/dr6/gallery/

In [42]:
def get_type(hubble):
    """Convert Hubble type to numerical type, for sorting purposes."""
    numtype = {
        'E': 0,
        'E-S0': 1,
        
        'S0': 2,
        'S0-a': 3,
        
        'Sa': 4,
        'SBa': 4,
        'SABa': 4,

        'Sab': 5,
        'SBab': 5,
        
        'Sb': 6,
        'SABb': 6,
        'SBb': 6,

        'Sbc': 7,
        
        'Sc': 8,
        'SABc': 8,
        'SBc': 8,
        
        'Scd': 9,
        'SBcd': 9,
        
        'Sd': 10,
        
        'Sm': 11,
        'SBm': 11,
        
        'I': 12,
        'IAB': 12,
        'IB': 12,
        
        '0': -1
    }
    return np.array([numtype[hh] for hh in hubble])

In [43]:
reject = ['ngc3587', 'ngc6832', 'ngc5982', 'ngc2832', 'ngc2340', 'ngc5195',
          'ngc5308', 'ngc4346', 'ngc4036', 'ngc2681', 'ngc3718', 'ngc5377',
          'ngc2146', 'ngc3126', 'ngc2841', 'ngc2683', 'ngc4217', 'ngc4357',
          'ngc5055', 'ngc4100', 'ngc5879', 'ngc5297', 'ngc4605', 'ngc6015',
          'ngc4144', 'ngc3733', 'ngc3079', 'ngc3198', 'ngc3430', 'ngc3877',
          'ngc4062', 'ngc4631', 'ngc4656_ned01', 'ngc4395']
toss = np.zeros(len(sample), dtype=bool)
name = get_name(sample)
for ii, nn in enumerate(name):
    for rej in np.atleast_1d(reject):
        toss[ii] = rej in nn.lower()
        if toss[ii]:
            break
print('Rejecting {} objects.'.format(np.sum(toss)))
pngkeep = sample[~toss]
if np.sum(toss) > 0:
    pngrej = sample[toss]
else:
    pngrej = []


Rejecting 34 objects.

In [44]:
htmlfile = os.path.join(gallerydir, 'index.html')
htmlfile_reject = os.path.join(gallerydir, 'index-reject.html')
baseurl = 'http://legacysurvey.org/viewer-dev'

In [59]:
def html_rows(pngkeep, nperrow=4):
    nrow = np.ceil(len(pngkeep) / nperrow).astype('int')
    pngsplit = list()
    for ii in range(nrow):
        i1 = nperrow*ii
        i2 = nperrow*(ii+1)
        if i2 > len(pngkeep):
            i2 = len(pngkeep)
        pngsplit.append(pngkeep[i1:i2])
    #pngsplit = np.array_split(pngkeep, nrow)
    print('Splitting the sample into {} rows with {} mosaics per row.'.format(nrow, nperrow))

    html.write('<table class="ls-gallery">\n')
    html.write('<tbody>\n')
    for pngrow in pngsplit:
        html.write('<tr>\n')
        for obj in pngrow:
            name = get_name(obj)
            nicename = get_name(obj, nice=True)
            pngfile = os.path.join('png', '{}.png'.format(name))
            thumbfile = os.path.join('png', 'thumb-{}.png'.format(name))
            img = 'src="{}" alt="{}"'.format(thumbfile, nicename)
            #img = 'class="ls-gallery" src="{}" alt="{}"'.format(thumbfile, nicename)
            html.write('<td><a href="{}"><img {}></a></td>\n'.format(pngfile, img))
        html.write('</tr>\n')
        html.write('<tr>\n')
        for obj in pngrow:
            nicename = get_name(obj, nice=True)
            href = '{}/?layer=decals-{}&ra={:.8f}&dec={:.8f}&zoom=12'.format(baseurl, dr, obj['ra'], obj['dec'])
            html.write('<td><a href="{}" target="_blank">{}</a></td>\n'.format(href, nicename))
        html.write('</tr>\n')
    html.write('</tbody>\n')            
    html.write('</table>\n')

In [60]:
objtype = ma.getdata(pngkeep['type'])
hubbletype = get_type(ma.getdata(pngkeep['hubble']))

In [63]:
with open(htmlfile, 'w') as html:
    html.write('<html><head>\n')
    html.write('<style type="text/css">\n')
    html.write('table.ls-gallery {width: 90%;}\n')
    #html.write('img.ls-gallery {display: block;}\n')
    #html.write('td.ls-gallery {width: 100%; height: auto}\n')
    #html.write('td.ls-gallery {width: 100%; word-wrap: break-word;}\n')
    html.write('p.ls-gallery {width: 80%;}\n')
    html.write('</style>\n')
    html.write('</head><body>\n')
    html.write('<h1>DR6 Image Gallery</h1>\n')
    html.write("""<p class="ls-gallery">This gallery highlights the exquisite image quality and diversity 
    of objects observed by the Legacy Survey, including planetary nebulae, globular clusters, and 
    large, nearby galaxies.  Each thumbnail links to a larger image while the object name below each 
    thumbnail links to the 
    <a href="http://legacysurvey.org/viewer">Sky Viewer</a>.  For reference, the horizontal white bar in 
    the lower-right corner of each image represents one arcminute.</p>\n""")
    html.write("""<p>We gratefully acknowledge the <a href="https://github.com/mattiaverga/OpenNGC" target="_blank">
    OpenNGC</a> catalog created by Mattia Verga, which was used to generate this sample.</p>\n""")
    html.write("""<p>For more eye candy, please visit the gallery of galaxy groups highlighted in the 
    <a href="http://portal.nersc.gov/project/cosmo/data/legacysurvey/dr5/gallery/">DR5 Gallery.</a></p>\n""")
    
    # Split by object type
    
    html.write('<h2>Planetary Nebulae, Open Clusters, and Globular Clusters</h2>\n')
    these = np.logical_or( np.logical_or(objtype == 'PN', objtype == 'OCl'), objtype == 'GCl' )
    srt = np.argsort(objtype[these])[::-1]
    html_rows(pngkeep[these][srt])
    html.write('<br />\n')
    
    html.write('<h2>Spheroidal & Elliptical Galaxies</h2>\n')
    these = (objtype == 'G') * (hubbletype <= 2)
    srt = np.argsort(hubbletype[these])
    html_rows(pngkeep[these][srt])

    html.write('<h2>Early-Type Disk Galaxies</h2>\n')
    these = (objtype == 'G') * (hubbletype >= 3) * (hubbletype <= 6)
    srt = np.argsort(hubbletype[these])
    html_rows(pngkeep[these][srt])

    html.write('<h2>Late-Type Disk Galaxies</h2>\n')
    these = (objtype == 'G') * (hubbletype >= 7) * (hubbletype <= 10)
    srt = np.argsort(hubbletype[these])
    html_rows(pngkeep[these][srt])
    
    html.write('<h2>Irregular Galaxies</h2>\n')
    these = (objtype == 'G') * (hubbletype >= 11)
    srt = np.argsort(hubbletype[these])
    html_rows(pngkeep[these][srt])
    
    html.write('</body></html>\n')


Splitting the sample into 2 rows with 4 mosaics per row.
Splitting the sample into 3 rows with 4 mosaics per row.
Splitting the sample into 7 rows with 4 mosaics per row.
Splitting the sample into 9 rows with 4 mosaics per row.
Splitting the sample into 3 rows with 4 mosaics per row.

In [48]:
if len(pngrej) > 0:
    with open(htmlfile_reject, 'w') as html:
        html.write('<html><head>\n')
        html.write('<style type="text/css">\n')
        html.write('img.ls-gallery {display: block;}\n')
        html.write('td.ls-gallery {width: 20%; word-wrap: break-word;}\n')
        html.write('</style>\n')
        html.write('</head><body>\n')
        html.write('<h1>DR6 Image Gallery - Rejected</h1>\n')
        html_rows(pngrej)
        html.write('</body></html>\n')


Splitting the sample into 7 rows with 5 mosaics per row.