Pre-selecting Extended UVX Objects from SDSS

We want to look for lenses by colour and spatial extent, in SDSS DR10 + WISE. We can grab this data from the SDSS skyserver SQL server from python, using the mechanize web browser, and then manipulate the catalog with pandas.


In [1]:
# Querying skyserver, using code from Eduardo Martin's blog post at
# http://balbuceosastropy.blogspot.com/2013/10/an-easy-way-to-make-sql-queries-from.html

import pandas as pd

def SDSS_select(sql):
    from StringIO import StringIO # To read a string like a file
    import mechanize
    url = "http://skyserver.sdss3.org/dr10/en/tools/search/sql.aspx"
    br = mechanize.Browser()
    br.open(url)
    br.select_form(name="sql")
    br['cmd'] = sql
    br['format']=['csv']
    response = br.submit()
    file_like = StringIO(response.get_data())
    return pd.read_csv(file_like,  skiprows=1)

We want a sample of galaxies and their 5-band photometry, positions and sizes. Let's focus on the photometric catalog, which goes deeper than SQLS.


In [4]:
galaxies = "SELECT top 1000 \
objid, ra, dec, \
dered_u AS mag_u, \
dered_g AS mag_g, \
dered_r AS mag_r, \
dered_i AS mag_i, \
dered_z AS mag_z \
FROM Galaxy"
print galaxies


SELECT top 1000 objid, ra, dec, dered_u AS mag_u, dered_g AS mag_g, dered_r AS mag_r, dered_i AS mag_i, dered_z AS mag_z FROM Galaxy

This SQL checks out at the SkyServer SQL box, which has a syntax checking option.


In [5]:
data = SDSS_select(galaxies)
data


Out[5]:
objid ra dec mag_u mag_g mag_r mag_i mag_z
0 1237665226235052837 194.024970 31.686003 25.18058 21.84216 20.29285 19.40382 18.89390
1 1237665226235052838 194.024636 31.684690 24.20213 22.70980 21.28123 20.62695 20.25502
2 1237665226235052839 194.027520 31.685358 24.95906 24.72264 21.78952 20.98459 20.45373
3 1237665226235052843 194.023841 31.656035 23.73315 21.28691 19.72269 19.07679 18.69604
4 1237665226235052844 194.021893 31.657103 25.07309 23.67389 21.27428 20.90533 22.26461
5 1237665226235052845 194.030627 31.740684 24.66286 22.50499 21.18971 20.67201 20.15721
6 1237665226235052846 194.027253 31.695203 24.08194 22.85443 23.24907 22.59280 21.88211
7 1237665226235052847 194.017713 31.594269 23.14602 22.95777 22.06895 22.16009 21.87406
8 1237665226235052848 194.017383 31.579548 23.15300 22.12232 20.46155 19.86591 19.29375
9 1237665226235052849 194.027470 31.677284 24.54190 22.79082 21.41718 20.58021 20.02970
10 1237665226235052850 194.019503 31.612184 24.96838 21.88818 21.89388 21.62789 21.47078
11 1237665226235052853 194.025642 31.680781 22.67123 23.39714 22.09009 21.94214 20.96629
12 1237665226235052855 194.035718 31.751381 25.22009 21.72452 20.41283 19.69813 19.34769
13 1237665226235052856 194.037521 31.752107 24.49104 25.40699 21.90702 21.16683 20.49273
14 1237665226235052857 194.018894 31.582640 24.21153 22.54806 22.27833 22.79782 22.54676
15 1237665226235052858 194.021851 31.608799 24.03981 22.41500 21.26492 20.77686 20.27316
16 1237665226235052859 194.025148 31.638349 27.38907 21.18638 21.14197 20.61774 24.80168
17 1237651752400322868 181.432961 1.370713 21.21119 23.40105 23.96064 24.67674 24.10459
18 1237651752400322870 181.436062 1.343471 20.21576 19.29533 18.51414 18.20516 18.02176
19 1237651752400322872 181.439637 1.421739 21.56520 20.39297 19.36605 18.90998 18.63198
20 1237651752400322874 181.453501 1.283399 20.96577 20.26348 19.88073 19.72432 19.54157
21 1237651752400322886 181.326811 1.268460 24.13008 22.08476 22.05123 21.39130 22.20289
22 1237651752410088429 203.676363 1.312397 22.88667 23.31611 22.76672 21.49211 21.57060
23 1237651752410088430 203.679431 1.302271 24.24175 23.52463 23.42250 21.31718 21.52153
24 1237651752410088433 203.719458 1.272547 24.68199 23.41098 23.29813 21.81727 23.15776
25 1237669519051785082 358.481615 8.430975 21.72086 22.15085 21.24714 20.44395 20.05983
26 1237669519051785083 358.484198 8.381472 26.13955 22.39513 20.89672 20.15686 19.73814
27 1237669519051785084 358.486270 8.302037 25.22889 23.76700 21.72180 20.42291 19.60994
28 1237669519051785085 358.486046 8.393166 24.59352 22.38468 21.53897 21.54985 22.85229
29 1237669519051785087 358.490229 8.325225 22.93756 22.34213 21.38956 20.81841 19.96773
... ... ... ... ... ... ... ... ...
970 1237657065255470112 16.565096 -3.377318 24.61093 24.27891 24.42641 22.02286 21.03405
971 1237657065255470113 16.570456 -3.480576 23.11643 22.66311 22.33107 21.31942 21.21999
972 1237657065255470114 16.570490 -3.484354 24.68205 25.01176 23.89510 22.01940 21.87421
973 1237657065255470115 16.599498 -3.314405 24.58594 23.70745 23.45836 21.96740 22.55458
974 1237657065255470116 16.604870 -3.375788 22.86257 23.49984 23.35161 22.05020 21.78262
975 1237657065255470118 16.629703 -3.396937 23.76175 24.00378 23.60205 21.45797 21.53632
976 1237657065255470119 16.680357 -3.380023 23.31317 23.14161 22.64831 21.74959 21.81305
977 1237657065255534805 16.725113 -3.301741 21.83973 25.49855 24.92111 22.74767 23.49829
978 1237657065255534809 16.730886 -3.340860 20.78091 19.75992 18.79554 18.37605 18.08337
979 1237657065255534812 16.739284 -3.442137 21.71655 27.29066 26.07592 25.38439 23.42066
980 1237657065255534813 16.739510 -3.417136 21.45792 20.79914 19.97359 19.61915 19.53302
981 1237657065255534815 16.747775 -3.322600 19.93794 18.90537 18.51718 18.30110 18.26095
982 1237657065255534816 16.748541 -3.323885 21.45211 19.84721 19.52238 19.63063 19.82398
983 1237657065255534817 16.742522 -3.321233 20.74352 19.88968 19.65660 19.50786 19.30767
984 1237657065255534825 16.755278 -3.374596 26.57001 24.28088 22.94062 21.77206 22.25745
985 1237657065255534999 16.749675 -3.308435 25.34271 22.69684 21.27935 20.77277 20.29755
986 1237657065255535002 16.750822 -3.378098 23.41228 22.45292 22.55329 22.78794 21.84584
987 1237657065255535003 16.749848 -3.377044 25.86052 22.86983 22.16364 22.10801 20.96760
988 1237657065255535004 16.751881 -3.358916 25.80055 22.79446 21.39692 20.74694 20.42576
989 1237657065255535006 16.755417 -3.449156 23.05181 22.10694 20.95480 20.58037 20.23356
990 1237657065255535008 16.755179 -3.364619 22.52571 22.20003 21.38236 21.02765 20.97431
991 1237657065255535009 16.756586 -3.474249 23.45675 22.68132 22.01774 21.84199 21.40762
992 1237657065255535010 16.756705 -3.407395 22.14469 20.87489 19.82617 19.40565 18.92782
993 1237657065255535011 16.757701 -3.441783 22.03507 22.14923 22.04599 21.23111 22.10757
994 1237657065255535014 16.760057 -3.423351 22.90638 22.30566 21.46668 20.96068 20.57510
995 1237657065255535016 16.759534 -3.361439 23.03814 24.55969 23.75516 21.51139 21.14187
996 1237657065255535017 16.760641 -3.361363 25.69914 22.16454 21.81608 23.41236 21.97759
997 1237657065255535018 16.760373 -3.331375 22.41648 22.81698 23.12515 21.86695 21.35169
998 1237657065255404473 16.513527 -3.413423 25.03164 23.70479 23.10860 21.79836 22.48013
999 1237657065255404475 16.514340 -3.416357 23.76511 22.72585 22.69493 21.79688 21.63132

1000 rows × 8 columns

Visualizing the galaxies

Let's look for some structure in the color distribution. Initial plotting code from Josh Bloom's AstroHackWeek 2014 lecture.


In [6]:
# For pretty plotting
!pip install --upgrade seaborn


Downloading/unpacking seaborn from https://pypi.python.org/packages/source/s/seaborn/seaborn-0.5.1.tar.gz#md5=2ce6ea7d3c67858c0b1f5793fa2043b8
  Downloading seaborn-0.5.1.tar.gz (104kB): 104kB downloaded
  Running setup.py (path:/private/var/folders/w3/x34d7vb51pd4bp9lgzxb19540000gp/T/pip_build_pjm/seaborn/setup.py) egg_info for package seaborn
    
Installing collected packages: seaborn
  Found existing installation: seaborn 0.4.0
    Uninstalling seaborn:
      Successfully uninstalled seaborn
  Running setup.py install for seaborn
    
Successfully installed seaborn
Cleaning up...

In [7]:
%pylab inline
import seaborn
seaborn.set()
import copy


Populating the interactive namespace from numpy and matplotlib

First let's see the i-band magnitude distribution.


In [9]:
bins =  hist(data['mag_i'].values,bins=100) ; xlabel("redshift") ; ylabel("N")


Out[9]:
<matplotlib.text.Text at 0x1093ef9d0>

Now let's look at everything else, colored by magnitude.


In [11]:
import matplotlib as mpl
import matplotlib.cm as cm

data = data[(data['mag_r'] > -9999)]

## truncate the color at 22 just to keep some contrast.
norm = mpl.colors.Normalize(vmin=min(data['mag_i'].values), vmax=22)
cmap = cm.jet_r
m = cm.ScalarMappable(norm=norm, cmap=cmap)
rez = pd.scatter_matrix(data[0:2000], alpha=0.2,figsize=[15,15],color=m.to_rgba(data['mag_i'].values))


Save this for future use.


In [8]:
data.to_csv('galaxies.clean.csv')

In [ ]: