Query CSW to find all COAWST WMS services

Find all the COAWST (ocean model) datasets that have WMS services by using the CSW queryables apiso:anyText and apiso:ServiceType on different CSW endpoints.



In [1]:

    
from owslib.csw import CatalogueServiceWeb
from owslib import fes
import numpy as np



In [2]:

    
endpoint = 'http://geoport.whoi.edu/csw'
#endpoint = 'http://catalog.data.gov/csw-all'
#endpoint = 'http://www.ngdc.noaa.gov/geoportal/csw'
#endpoint = 'http://www.nodc.noaa.gov/geoportal/csw'
csw = CatalogueServiceWeb(endpoint,timeout=60)
print csw.version



In [3]:

    
csw.get_operation_by_name('GetRecords').constraints









    Out[3]:





[Constraint: SupportedISOQueryables - ['apiso:DistanceValue', 'apiso:Abstract', 'apiso:RevisionDate', 'apiso:Subject', 'apiso:KeywordType', 'apiso:Title', 'apiso:CRS', 'apiso:PublicationDate', 'apiso:Type', 'apiso:AlternateTitle', 'apiso:BoundingBox', 'apiso:AnyText', 'apiso:ParentIdentifier', 'apiso:Modified', 'apiso:Operation', 'apiso:Format', 'apiso:TempExtent_end', 'apiso:DistanceUOM', 'apiso:OrganisationName', 'apiso:ServiceType', 'apiso:TempExtent_begin', 'apiso:ResourceLanguage', 'apiso:ServiceTypeVersion', 'apiso:OperatesOn', 'apiso:Denominator', 'apiso:HasSecurityConstraints', 'apiso:OperatesOnIdentifier', 'apiso:GeographicDescriptionCode', 'apiso:Language', 'apiso:Identifier', 'apiso:OperatesOnName', 'apiso:TopicCategory', 'apiso:CreationDate', 'apiso:CouplingType'],
 Constraint: AdditionalQueryables - ['apiso:Lineage', 'apiso:Classification', 'apiso:Creator', 'apiso:Relation', 'apiso:OtherConstraints', 'apiso:SpecificationTitle', 'apiso:ResponsiblePartyRole', 'apiso:SpecificationDateType', 'apiso:Degree', 'apiso:Contributor', 'apiso:ConditionApplyingToAccessAndUse', 'apiso:SpecificationDate', 'apiso:AccessConstraints', 'apiso:Publisher'],
 Constraint: SupportedDublinCoreQueryables - ['dc:contributor', 'dc:source', 'dc:language', 'dc:title', 'dc:subject', 'dc:creator', 'dc:type', 'ows:BoundingBox', 'dct:modified', 'dct:abstract', 'dc:relation', 'dc:date', 'dc:identifier', 'dc:publisher', 'dc:format', 'csw:AnyText', 'dc:rights']]



In [4]:

    
try:
    csw.get_operation_by_name('GetDomain')
    csw.getdomain('apiso:ServiceType', 'property')
    print(csw.results['values'])
except:
    print('GetDomain not supported')









    



['THREDDS OPeNDAP,Open Geospatial Consortium Web Map Service (WMS),THREDDS HTTP Service', 'THREDDS OPeNDAP,Open Geospatial Consortium Web Map Service (WMS),THREDDS NetCDF Subset Service', 'THREDDS OPeNDAP,Open Geospatial Consortium Web Map Service (WMS),THREDDS NetCDF Subset Service,THREDDS HTTP Service']

Query for all COAWST datasets



In [5]:

    
val = 'COAWST'
filter1 = fes.PropertyIsLike(propertyname='apiso:AnyText',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?')
filter_list = [ filter1 ]



In [6]:

    
csw.getrecords2(constraints=filter_list,maxrecords=100,esn='full')
print len(csw.records.keys())
for rec in list(csw.records.keys()):
    print csw.records[rec].title









    



7
COAWST Hindcast:Barnegat Bay:ADCIRC tides,Real rivers,Plume,Lowpass Espresso bdry,NAM,new bathy
COAWST Forecast System : USGS : US East Coast and Gulf of Mexico (Experimental)
COAWST Hindcast:MVCO/CBlast 2007:ripples with SWAN-40m res
USGS-CMG-COAWST Model: Hurricane Sandy, NYB05 700m Nest
USGS-CMG-COAWST Model: Hurricane Sandy, SHF05 2km Nest
USGS-CMG-COAWST Model: Hurricane Sandy, USE27 6km Nest
ASGS NCFS Sandy Advisory 27



In [7]:

    
choice=np.random.choice(list(csw.records.keys()))
print(csw.records[choice].title)
csw.records[choice].references









    



USGS-CMG-COAWST Model: Hurricane Sandy, USE27 6km Nest






    Out[7]:





[{'scheme': 'WWW:LINK',
  'url': 'http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_USE27.ncml.html'},
 {'scheme': 'WWW:LINK',
  'url': 'http://www.ncdc.noaa.gov/oa/wct/wct-jnlp-beta.php?singlefile=http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_USE27.ncml'},
 {'scheme': 'OPeNDAP:OPeNDAP',
  'url': 'http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_USE27.ncml'},
 {'scheme': 'OGC:WMS',
  'url': 'http://sci-wms.whoi.edu/wms/datasets/Sandy_triple_nest_00_dir_USE27?REQUEST=GetCapabilities'},
 {'scheme': 'UNIDATA:NCSS',
  'url': 'http://geoport-dev.whoi.edu/thredds/ncss/grid/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_USE27.ncml/dataset.html'},
 {'scheme': 'file',
  'url': 'http://geoport-dev.whoi.edu/thredds/fileServer/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_USE27.ncml'}]

Query for all COAWST datsets that also contain WMS endpoints

Since all COAWST datasets contain WMS endpoints, this should return the same number of dataset records



In [8]:

    
val = 'COAWST'
filter1 = fes.PropertyIsLike(propertyname='apiso:AnyText',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?',matchCase=True)
val = 'WMS'
filter2 = fes.PropertyIsLike(propertyname='apiso:ServiceType',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?',matchCase=False)
filter_list = [ [filter1, filter2] ]
csw.getrecords2(constraints=filter_list, maxrecords=1000)
print(len(csw.records.keys()))



In [9]:

    
for rec in list(csw.records.keys()):
    print('title:'+csw.records[rec].title) 
    print('identifier:'+csw.records[rec].identifier)
    print('modified:'+csw.records[rec].modified)









    



title:COAWST Hindcast:Barnegat Bay:ADCIRC tides,Real rivers,Plume,Lowpass Espresso bdry,NAM,new bathy
identifier:gov.usgs.cmgp:COAWST.Barnegat_Bay.spring2012
modified:2015-08-25
title:COAWST Forecast System : USGS : US East Coast and Gulf of Mexico (Experimental)
identifier:gov.usgs.cmgp:COAWST.USEAST.Forecast
modified:2015-08-25
title:COAWST Hindcast:MVCO/CBlast 2007:ripples with SWAN-40m res
identifier:gov.usgs.cmgp:COAWST.MVCO.CBLAST.spatial_7_ar0fd
modified:2015-08-25
title:USGS-CMG-COAWST Model: Hurricane Sandy, NYB05 700m Nest
identifier:gov.usgs.cmg:USGS_COAWST_Sandy_NYB05_sim6
modified:2015-08-25
title:USGS-CMG-COAWST Model: Hurricane Sandy, SHF05 2km Nest
identifier:gov.usgs.cmg:USGS_COAWST_Sandy_SHF05_sim6
modified:2015-08-25
title:USGS-CMG-COAWST Model: Hurricane Sandy, USE27 6km Nest
identifier:gov.usgs.cmg:USGS_COAWST_Sandy_USE27_sim6
modified:2015-08-25
title:ASGS NCFS Sandy Advisory 27
identifier:RENCI_ADCIRC.ASGS.NCFS.Sandy.27
modified:2015-05-05



In [11]:

    
val = 'wms'
filter2 = fes.PropertyIsLike(propertyname='apiso:ServiceType',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?',matchCase=True)
filter_list = [  filter2 ]
csw.getrecords2(constraints=filter_list, maxrecords=1000)
print(len(csw.records.keys()))



In [10]: