In [1]:
import warnings
warnings.simplefilter('ignore')
This tutorial goes through a few basics of how to perform queries on the MaNGA dataset using the Marvin Query tool. Please see the Marvin Query page for more details on how to use Queries. This tutorial covers the basics of:
First let's import some basics
In [4]:
# we should be using DR15 MaNGA data
from marvin import config
config.release
# import the Query tool
from marvin.tools.query import Query
Let's go through some Query basics of how to do a query on metadata. The two main keyword arguments to Query are search_filter and return_params. search_filter is a string representing the SQL where condition you'd like to filter on. This tutorial assumes a basic familiarity with the SQL boolean syntax needed to construct Marvin Queries. Please see the tutorial on SQL Boolean syntax to learn more. return_params is a list of parameters you want to return in the query in addition to those used in the SQL filter condition.
Let's search for all galaxies with a redshift less than 0.1. To specify our search parameter, redshift, we must know the database table and name of the parameter. In this case, MaNGA uses the NASA-Sloan Atlas (NSA) for redshift information. In the NSA catalog, the redshift is the z parameter of the nsa table, so our search parameter will be nsa.z. Generically, all search parameters will take the form table.parameter.
In [5]:
# filter for galaxies with a redshift < 0.1
my_filter = 'nsa.z < 0.1'
In [7]:
# construct the query
q = Query(search_filter=my_filter)
q
Out[7]:
The Query tool works with a local or remote manga database. Without a database, the Query tool submits your inputs to the remote server using the API, rather than doing anything locally. We run the query with the run method. Your inputs are sent to the server where the query is built dynamically and run. The results of the query are returned via the API, parsed and converted into a Marvin Results object.
In [8]:
# run the query
r = q.run()
# print some results information
print(r)
print('number of results:', r.totalcount)
After constructing queries, we can run them with q.run(). This returns a Marvin Results object. Let's take a look. This query returned 4275 objects. For queries with large results, the results are automatically paginated in sets of 100 objects. Default parameters returned in queries always include the mangaid and plateifu. Marvin Queries will also return any parameters used in the definition of your filter condition. Since we filtered on redshift, the redshift is automatically included.
In [12]:
# look at the current page of results (subset of 10)
print('number in current set:', len(r.results))
print(r.results[0:10])
We can use the Query datamodel to look up all the available parameters one can use in the search_filter or in the return_params keyword arguments to Query. The Query datamodel contains many parameters, which we bundle into groups for easier navigation. We currently offer four groups of parameters, with remaining parametrs captured in an Other group. See the Query Datamodel in the docs for more detailed info.
In [10]:
# look up the available query datamodel groups
q.datamodel.groups
Out[10]:
Each group contains a list of QueryParameters. A QueryParameter contains the designation and syntax used for specifying parameters in your query. The full attribute indicates the table.parameter name needed as input into the search_filter or return_params keyword arguments to the Marvin Query. The redshift is a parameter from the NSA catalog. There are 158 parameters in the NSA Catalog group.
In [9]:
# select and print the NSA parameter group
nsa = q.datamodel.groups['nsa']
nsa.parameters
Out[9]:
We can easily combine query filter conditions by constructing a boolean string using AND. Let's search for galaxies with a redshift < 0.1 and log M$_\star$ < 10. The NSA catalog contains the Sersic profile determination for stellar mass, which is the sersic_mass or sersic_logmass parameter of the nsa table, so our search parameter will be nsa.sersic_logmass.
Let's also return the object RA and Dec as well using the return_params keyword. This accepts a list of string parameters. Object RA and Dec are included in the cube table so the parameter names are cube.ra and cube.dec.
In [13]:
my_filter = 'nsa.z < 0.1 and nsa.sersic_logmass < 10'
q = Query(search_filter=my_filter, return_params=['cube.ra', 'cube.dec'])
r = q.run()
print(r)
print('Number of objects:', r.totalcount)
This query return 1932 objects and now includes the RA, Dec, redshift and log Sersic stellar mass parameters.
In [14]:
# print the first 10 rows
r.results[0:10]
Out[14]:
Cone searches can be performed with Marvin Queries using a special functional syntax in your SQL string. Cone searches can be performed using the special radial string function. The syntax for a cone search query is radial(RA, Dec, radius). Let's search for all galaxies within 0.5 degrees centered on RA, Dec = 232.5447, 48.6902. The RA and Dec must be in decimal degrees and the radius is in units of degrees.
In [15]:
# build the radial filter condition
my_filter = 'radial(232.5447, 48.6902, 0.5)'
q = Query(search_filter=my_filter)
r = q.run()
print(r)
print(r.results)
MaNGA provides derived analysis properties in its dapall summary file. Marvin allows for queries on any of the parameters in the file. The table name for these parameters is dapall. Let's find all galaxies that have a total measure star-formation rate > 5 M$_\odot$/year. The total SFR parameter in the DAPall table is sfr_tot.
In [17]:
my_filter = 'dapall.sfr_tot > 5'
q = Query(search_filter=my_filter)
r = q.run()
print(r)
print(r.results)
The query returns 6 results, but looking at the plateifu, we see there are only 3 unique targets. This is because the DAPall file provides measurements for multiple bintypes and by default will return entries for all bintypes. We can select those out using the bintype.name parameter. Let's filter on only the HYB10 bintype.
In [18]:
my_filter = 'dapall.sfr_tot > 5 and bintype.name==HYB10'
q = Query(search_filter=my_filter)
r = q.run()
print(r)
print(r.results)
Marvin includes the ability to perform queries using quality or target flag information. These work using the special quality and targets keyword arguments. These keywords accept a list of flag maskbit labels provided by the Maskbit Datamodel. These keywords are inclusive, meaning they will only filter on objects satisfying those labels.
Let's find all galaxies that are in the MaNGA MAIN target selection sample. Targets in the MAIN sample are a part of the PRIMARY, SECONDARY and COLOR-ENHANCED samples. These are the primary, secondary, and color-enhanced flag labels. The targets keywords accepts all labels from the MANGA_TARGET1, MANGA_TARGET2, or MANGA_TARGET3 maskbit schema.
In [19]:
# create the targets list of labels
targets = ['primary', 'secondary', 'color-enhanced']
q = Query(targets=targets)
r = q.run()
print(r)
print('There are {0} galaxies in the main sample'.format(r.totalcount))
print(r.results[0:5])
The targets keyword is equivalent to the cube.manga_targetX search parameter, where X is 1, 2, or 3. The bits for the primary, secondary, and color-enhanced samples are 10, 11, and 12, respectively. These combine into the value 7168. The above query is equivalent to the filter condition cube.manga_target1 & 7168
In [20]:
value = 1<<10 | 1<<11 | 1<<12
my_filter = 'cube.manga_target1 & {0}'.format(value)
q = Query(search_filter=my_filter)
r = q.run()
print(r)
Let's search only for galaxies that are Milky Way Analogs or Dwarfs ancillary targets.
In [21]:
targets = ['mwa', 'dwarf']
q = Query(targets=targets)
r = q.run()
print(r)
print('There are {0} galaxies from the Milky Way Analogs and Dwarfs ancillary target catalogs'.format(r.totalcount))
print(r.results)
In [22]:
quality = ['BADFLUX']
q = Query(quality=quality)
r = q.run()
print(r)
print('There are {0} galaxies with bad flux calibration'.format(r.totalcount))
print(r.results[0:10])
The quality keyword is equivalent to the search parameters cube.quality for DRP flags or the file.quality for DAP flags. The above query is equivalent to cube.quality & 256. You can also perform a NOT bitmask selection using the ~ symbol. To perform a NOT selection we can only use the cube.quality parameter. Let's select all galaxies that do not have bad flux calibration.
In [23]:
# the above query as a filter condition
q = Query(search_filter='cube.quality & 256')
r = q.run()
print('Objects with bad flux calibration:', r.totalcount)
# objects with bad quality other than bad flux calibration
q = Query(search_filter='cube.quality & ~256')
r = q.run()
print('Bad objects with no bad flux calibration:', r.totalcount)
To find exactly objects with good quality and no bad flags set, use cube.quality == 0.
In [24]:
q = Query(search_filter='cube.quality == 0')
r = q.run()
print(r)
print('Objects with good quality:', r.totalcount)
In [ ]: