The Herschel Science Archive has a I/O interface called HAIO. How to use it is explained in some details in this url http://archives.esac.esa.int/hsa/aio/doc/howto.html
In order to access Herschel data, one can construct an HAIO HTTP request and submit it to the archive.
In the following example, we download SPIRE FTS level-2 data as a tar file, extract the relevant fits file and read it. Then we reorganise the spectral data in a dictionary.
In [1]:
#
# import some necessary packages
#
import matplotlib.pyplot as plt
%matplotlib inline
from astropy.io import fits
import tarfile
import requests
import tempfile
import os, shutil
In [2]:
def getSpireFtsLevel2(obsid, what='spss', archive='hsa', saveTar=True):
"""
PURPOSE:
Using the HTTP access to HAIO, retrieve all level-2 products in a tar file
and extract only the requested fits file (spss in the name) at then put the data in
a dictionary
INPUTS:
obsid - the OBSID of the observation
what - can be 'spss' for point source calibrated spectra or 'sds' for extended source
calibrated data. Note that only non-apodized spectra are searched for. You can change this
behaviour in the function below, by replacing '_spg_' with '_spgApod_'.
archive - can be 'hsa' or 'hsaint' (for the integration archive).
saveTar - whether to save the downloaded level-2 tar file in the current folder. The tar file name
will be OBSID_level2.tar. If False, the file will be deleted after the spss FITS file is extracted.
OUTPUTS:
A nested dictionary with the following structure:
spec[det]['wave'],
spec[det]['flux'],
spec[det]['fluxErr'],
where det is the name of the detector.
The FITS header of the primary extension of the fits file.
EXAMPLE:
spss, header = getSpireFtsLevel2(1342259588)
spss.keys()
spss["SSWD4"].keys()
header
TODO:
* Decide what to extract from the tar file when the observation is in H+LR mode.
HISTORY:
Created 16 Mar 2016 (IV), version 1.0
"""
# the name of the saved tar file.
tarFile = "%i_level2.tar"%obsid
archiveUrl = "http://archives.esac.esa.int/%s/aio/jsp/product.jsp"%archive
haioRequest = "%s?PROTOCOL=HTTP&OBSERVATION_ID=%i&PRODUCT_LEVEL=Level2"%(archiveUrl,obsid)
print ("Downloading level-2 data from the Herschel Science Archive. May take a while... be patient")
#
# submit the HAIO request to the server
#
r = requests.get(haioRequest)
#
# save the result in a .tar file
#
with open(tarFile, "wb") as tmp:
tmp.write(r.content)
#
# now read the downloaded tar file
#
xx = None
try:
tar = tarfile.open(tarFile,'r')
for member in tar.getnames():
tmpdir = tempfile.mkdtemp()
if ((what in member) and ('_spg_' in member)):
f=tar.extract(member,tmpdir)
xx = fits.open(os.path.join(tmpdir,member))
break
# clean up the temprary folder with the extracted file
shutil.rmtree(tmpdir)
# remove the tar file if asked for.
if (not saveTar): os.remove(tarFile)
except:
print ("*** OBSID %i is not available or it is proprietory."%obsid)
return None, None
finally:
tar.close()
if (xx == None):
print ("*** OBSID %i is not an FTS sparse mode. Return."%obsid)
return None, None
spec = {}
# get the primary header
header = xx[0].header
for k in xx:
extname = k.name
if ('S' in extname):
spec[k.name] = {}
spec[k.name]['wave'] = k.data["wave"]
spec[k.name]['flux'] = k.data["flux"]
spec[k.name]['fluxErr'] = k.data["error"]
return spec, header
Now, let's try it with one observation.
In [3]:
# try a proprietory data
#spss2, hh2 = getSpireFtsLevel2(1342246269)
#
# now a normal one
spss, hh = getSpireFtsLevel2(1342259588)
print (spss.keys())
print (spss["SSWD4"].keys())
hh
# now not a spec one
#spss2, hh2 = getSpireFtsLevel2(1342195774)
Out[3]:
Let's try to plot all spectra, the central detectors with one colour and the off-axis ones in another.
In [4]:
def plotSpireSparseSpec(specIn, fitsHeader=None, onlyCentral=True):
"""
"""
central = ['SSWD4','SLWC3']
fig = plt.figure(figsize=(8,5))
for det in central:
plt.plot(specIn[det]['wave'],specIn[det]['flux'],'k-')
if (not onlyCentral):
for det in specIn.keys():
if (not det in central):
plt.plot(specIn[det]['wave'],specIn[det]['flux'],'c-')
plt.xlabel('Frequency (GHz)')
plt.ylabel('Flux density (Jy)')
plt.grid(True)
if (fitsHeader):
titleText = 'Target: %s, OD: %i, OBSID: %i'%(fitsHeader['OBJECT'],fitsHeader['ODNUMBER'],fitsHeader['OBS_ID'])
fig.suptitle(titleText, fontsize=14, fontweight='bold')
#plt.title(titleText)
#
plotSpireSparseSpec(spss, fitsHeader=hh)
#
# add the off axis detectors to the plot
plotSpireSparseSpec(spss, fitsHeader=hh, onlyCentral=False)
Use the provided defs and implement:
Short-term implementations:
Long-term ideas:
Ivan Valtchanov, Herschel Science Centre, March 2016
In [ ]: