All CMEMS in situ data products can be found and downloaded after registration via CMEMS catalogue.
Such channel is advisable just for sporadic netCDF donwloading because when operational, interaction with the web user interface is not practical. In this context though, the use of scripts for ftp file transference is is a much more advisable approach.
As long as every line of such files contains information about the netCDFs contained within the different directories see at tips why, it is posible for users to loop over its lines to download only those that matches a number of specifications such as spatial coverage, time coverage, provider, data_mode, parameters or file_name related (region, data type, TS or PF, platform code, or/and platform category, timestamp).
i.e:
In [285]:
user = #type CMEMS user name within colons
password = #type CMEMS password within colons
product_name = 'INSITU_BAL_NRT_OBSERVATIONS_013_032' #type aimed CMEMS in situ product
distribution_unit = 'cmems.smhi.se' #type aimed hosting institution
index_file = 'index_latest.txt' #type aimed index file name
In [269]:
import ftplib
In [286]:
ftp=ftplib.FTP(distribution_unit,user,password)
ftp.cwd("Core")
ftp.cwd(product_name)
remote_filename= index_file
local_filename = remote_filename
local_file = open(local_filename, 'wb')
ftp.retrbinary('RETR ' + remote_filename, local_file.write)
local_file.close()
ftp.quit()
#ready when 221 Goodbye.!
Out[286]:
In [287]:
import numpy as np
import pandas as pd
from random import randint
In [288]:
index = np.genfromtxt(index_file, skip_header=6, unpack=False, delimiter=',', dtype=None,
names=['catalog_id', 'file_name', 'geospatial_lat_min', 'geospatial_lat_max',
'geospatial_lon_min', 'geospatial_lon_max',
'time_coverage_start', 'time_coverage_end',
'provider', 'date_update', 'data_mode', 'parameters'])
In [289]:
dataset = randint(0,len(index)) #ramdom line of the index file
In [290]:
values = [index[dataset]['catalog_id'], '<a href='+index[dataset]['file_name']+'>'+index[dataset]['file_name']+'</a>', index[dataset]['geospatial_lat_min'], index[dataset]['geospatial_lat_max'],
index[dataset]['geospatial_lon_min'], index[dataset]['geospatial_lon_max'], index[dataset]['time_coverage_start'],
index[dataset]['time_coverage_end'], index[dataset]['provider'], index[dataset]['date_update'], index[dataset]['data_mode'],
index[dataset]['parameters']]
headers = ['catalog_id', 'file_name', 'geospatial_lat_min', 'geospatial_lat_max',
'geospatial_lon_min', 'geospatial_lon_max',
'time_coverage_start', 'time_coverage_end',
'provider', 'date_update', 'data_mode', 'parameters']
df = pd.DataFrame(values, index=headers, columns=[dataset])
df.style
Out[290]:
In [275]:
from shapely.geometry import box, multipoint
import shapely
Regarding the above glimpse, it is posible to filter by 12 criteria. As example we will setup next a filter to only download those files that contains data within a defined boundingbox.
1. Aimed boundingbox
In [276]:
targeted_geospatial_lat_min = 55.0 # enter min latitude of your bounding box
targeted_geospatial_lat_max = 70.0 # enter max latitude of your bounding box
targeted_geospatial_lon_min = 12.0 # enter min longitude of your bounding box
targeted_geospatial_lon_max = 26.00 # enter max longitude of your bounding box
targeted_bounding_box = box(targeted_geospatial_lon_min, targeted_geospatial_lat_min, targeted_geospatial_lon_max, targeted_geospatial_lat_max)
2. netCDF filtering/selection
In [291]:
selected_netCDFs = [];
for netCDF in index:
file_name = netCDF['file_name']
geospatial_lat_min = float(netCDF['geospatial_lat_min'])
geospatial_lat_max = float(netCDF['geospatial_lat_max'])
geospatial_lon_min = float(netCDF['geospatial_lon_min'])
geospatial_lon_max = float(netCDF['geospatial_lon_max'])
bounding_box = shapely.geometry.box(geospatial_lon_min, geospatial_lat_min, geospatial_lon_max, geospatial_lat_max)
bounding_box_centroid = bounding_box.centroid
if (targeted_bounding_box.contains(bounding_box_centroid)):
selected_netCDFs.append(file_name)
print("total: " +str(len(selected_netCDFs)))
In [293]:
for nc in selected_netCDFs:
last_idx_slash = nc.rfind('/')
ncdf_file_name = nc[last_idx_slash+1:]
folders = nc.split('/')[3:len(nc.split('/'))-1]
host = nc.split('/')[2] #or distribution unit
ftp=ftplib.FTP(host,user,password)
for folder in folders:
ftp.cwd(folder)
local_file = open(ncdf_file_name, 'wb')
ftp.retrbinary('RETR '+ncdf_file_name, local_file.write)
local_file.close()
ftp.quit()