2D Preprocessing the GlobColour Dataset

  • prepare dataset for LDS-fitting:
    • load all 5 variables and merge (interpolate) with the float dataset
    • load the distance to coast and merge (interpolate) with the float dataset
    • output the data on disk
    • (plan)if needed, split Nov-Dec, encoding the weekly number

In [1]:
import xarray as xr
import numpy as np
import pandas as pd
%matplotlib inline
from matplotlib import pyplot as plt
from dask.diagnostics import ProgressBar
import seaborn as sns
from matplotlib.colors import LogNorm

In [27]:
# resampling frequency in number of days
freq=3

In [3]:
''' SST is not from GlobColor
from tools.load_GlobColor_dataset import load_dataset
import importlib
importlib.reload(load_dataset)
############### T865
ds_daily = load_dataset.load_t865()
'''


Out[3]:
' SST is not from GlobColor\nfrom tools.load_GlobColor_dataset import load_dataset\nimport importlib\nimportlib.reload(load_dataset)\n############### T865\nds_daily = load_dataset.load_t865()\n'

Load data from disk

We already downloaded a subsetted MODIS-Aqua dataset for the Arabian Sea. We can read all the netcdf files into one xarray Dataset using the open_mfsdataset function. Note that this does not load the data into memory yet. That only happens when we try to access the values.


In [4]:
#ds_8day = xr.open_mfdataset('./data_collector_modisa_chla9km/ModisA_Arabian_Sea_chlor_a_9km_*_8D.nc')
ds_daily = xr.open_mfdataset('./data_collector_modisa_NSST4mu_4km_tomatchHermes/ModisA_Arabian_Sea_SST4_sst4_4km_*D.nc')
#both_datasets = [ds_8day, ds_daily]

In [5]:
ds_daily = ds_daily.drop(['palette','qual_sst4'])

In [6]:
ds_daily.var


Out[6]:
<bound method ImplementsDatasetReduce._reduce_method.<locals>.wrapped_func of <xarray.Dataset>
Dimensions:  (lat: 552, lon: 720, time: 5539)
Coordinates:
  * lat      (lat) float64 27.98 27.94 27.9 27.85 27.81 27.77 27.73 27.69 ...
  * lon      (lon) float64 45.02 45.06 45.1 45.15 45.19 45.23 45.27 45.31 ...
  * time     (time) datetime64[ns] 2002-07-04 2002-07-05 2002-07-06 ...
Data variables:
    sst4     (time, lat, lon) float64 nan nan nan nan nan nan nan nan nan ...
Attributes:
    product_name:                      A2002185.L3m_DAY_SST4_sst4_4km.nc
    instrument:                        MODIS
    title:                             HMODISA Level-3 Standard Mapped Image
    project:                           Ocean Biology Processing Group (NASA/G...
    platform:                          Aqua
    temporal_range:                    15-hour
    processing_version:                2014.0.1
    date_created:                      2017-05-08T12:15:57.000Z
    history:                           l3mapgen par=A2002185.L3m_DAY_SST4_sst...
    l2_flag_names:                     LAND,~HISOLZEN
    time_coverage_start:               2002-07-04T00:00:15.000Z
    time_coverage_end:                 2002-07-04T14:55:04.000Z
    start_orbit_number:                885
    end_orbit_number:                  894
    map_projection:                    Equidistant Cylindrical
    latitude_units:                    degrees_north
    longitude_units:                   degrees_east
    northernmost_latitude:             90.0
    southernmost_latitude:             -90.0
    westernmost_longitude:             -180.0
    easternmost_longitude:             180.0
    geospatial_lat_max:                90.0
    geospatial_lat_min:                -90.0
    geospatial_lon_max:                180.0
    geospatial_lon_min:                -180.0
    grid_mapping_name:                 latitude_longitude
    latitude_step:                     0.0416667
    longitude_step:                    0.0416667
    sw_point_latitude:                 -89.9792
    sw_point_longitude:                -179.979
    geospatial_lon_resolution:         4.63831
    geospatial_lat_resolution:         4.63831
    geospatial_lat_units:              degrees_north
    geospatial_lon_units:              degrees_east
    spatialResolution:                 4.64 km
    number_of_lines:                   4320
    number_of_columns:                 8640
    measure:                           Mean
    suggested_image_scaling_minimum:   -2.0
    suggested_image_scaling_maximum:   45.0
    suggested_image_scaling_type:      LINEAR
    suggested_image_scaling_applied:   No
    _lastModified:                     2017-05-08T12:15:57.000Z
    Conventions:                       CF-1.6
    institution:                       NASA Goddard Space Flight Center, Ocea...
    standard_name_vocabulary:          NetCDF Climate and Forecast (CF) Metad...
    Metadata_Conventions:              Unidata Dataset Discovery v1.0
    naming_authority:                  gov.nasa.gsfc.sci.oceandata
    id:                                A2002185.L3b_DAY_SST4.nc/L3/A2002185.L...
    license:                           http://science.nasa.gov/earth-science/...
    creator_name:                      NASA/GSFC/OBPG
    publisher_name:                    NASA/GSFC/OBPG
    creator_email:                     data@oceancolor.gsfc.nasa.gov
    publisher_email:                   data@oceancolor.gsfc.nasa.gov
    creator_url:                       http://oceandata.sci.gsfc.nasa.gov
    publisher_url:                     http://oceandata.sci.gsfc.nasa.gov
    processing_level:                  L3 Mapped
    cdm_data_type:                     grid
    identifier_product_doi_authority:  http://dx.doi.org
    identifier_product_doi:            10.5067/AQUA/MODIS_OC.2014.0
    keywords:                          Oceans > Ocean Temperature > Sea Surfa...
    keywords_vocabulary:               NASA Global Change Master Directory (G...
    data_bins:                         3246245
    data_minimum:                      -1.635
    data_maximum:                      31.465>

In [7]:
ds_daily.sst4.sel(time='2002-07-28').plot()


Out[7]:
<matplotlib.collections.QuadMesh at 0x115e45278>

In [9]:
freq_resample = str(8) + 'D'
ds_8day = ds_daily.resample(freq_resample, dim='time')  # see the above for doc, test case, & default behavior
ds_8day


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[9]:
<xarray.Dataset>
Dimensions:  (lat: 552, lon: 720, time: 693)
Coordinates:
  * lat      (lat) float64 27.98 27.94 27.9 27.85 27.81 27.77 27.73 27.69 ...
  * lon      (lon) float64 45.02 45.06 45.1 45.15 45.19 45.23 45.27 45.31 ...
  * time     (time) datetime64[ns] 2002-07-04 2002-07-12 2002-07-20 ...
Data variables:
    sst4     (time, lat, lon) float64 nan nan nan nan nan nan nan nan nan ...

In [10]:
# check data quality
both_datasets = [ds_8day, ds_daily]
print([(ds.nbytes / 1e6) for ds in both_datasets])


[2203.42308, 17611.415768]

In [11]:
'''
# no need for termperature, sst can be <0
def fix_bad_data(ds):
    # for some reason, the cloud / land mask is backwards on some data
    # this is obvious because there are t865 values less than zero
    bad_data = ds.t865.groupby('time').min() < 0
    # loop through and fix
    for n in np.nonzero(bad_data.values)[0]:
        data = ds.t865[n].values 
        ds.t865.values[n] = np.ma.masked_less(data, 0).filled(np.nan)
[fix_bad_data(ds) for ds in both_datasets]
'''


Out[11]:
"\n# no need for termperature, sst can be <0\ndef fix_bad_data(ds):\n    # for some reason, the cloud / land mask is backwards on some data\n    # this is obvious because there are t865 values less than zero\n    bad_data = ds.t865.groupby('time').min() < 0\n    # loop through and fix\n    for n in np.nonzero(bad_data.values)[0]:\n        data = ds.t865[n].values \n        ds.t865.values[n] = np.ma.masked_less(data, 0).filled(np.nan)\n[fix_bad_data(ds) for ds in both_datasets]\n"

In [12]:
# Count the number of ocean data points
(~ds_8day.sst4.isnull()).sum(dim='time').plot()


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[12]:
<matplotlib.collections.QuadMesh at 0x11b2d0c88>

In [13]:
#  find a mask for the land
ocean_mask = ((~ds_8day.sst4.isnull()).sum(dim='time'))>1
num_ocean_points = ocean_mask.sum().values
ocean_mask.plot()
plt.title('%g total ocean points' % num_ocean_points)


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[13]:
<matplotlib.text.Text at 0x11b400588>

In [14]:
plt.figure(figsize=(8,6))
ds_daily.sst4.sel(time='2002-11-18',method='nearest').plot(norm=LogNorm())


Out[14]:
<matplotlib.collections.QuadMesh at 0x11b55c470>
/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/matplotlib/colors.py:1022: RuntimeWarning: invalid value encountered in less_equal
  mask |= resdat <= 0

In [15]:
(~ds_daily.sst4.isnull()).sum(['lat','lon']) # information from original data


Out[15]:
<xarray.DataArray 'sst4' (time: 5539)>
array([     0,  93356, 111044, ...,  53633,  70480,  72927])
Coordinates:
  * time     (time) datetime64[ns] 2002-07-04 2002-07-05 2002-07-06 ...

In [16]:
count_8day,count_daily = [(~ds.sst4.isnull()).sum(['lat','lon'])/float(num_ocean_points)
                            for ds in (ds_8day,ds_daily)]

plt.figure(figsize=(12,4))
count_8day.plot(color='k')
count_daily.plot(color='r')

plt.legend(['8 day','daily'])    # there is one day in the datset have SST values on the land !


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[16]:
<matplotlib.legend.Legend at 0x11b2c6128>

In [17]:
count_8day,count_daily = [ds.sst4.groupby('time').count()/float(num_ocean_points)
                            for ds in (ds_8day,ds_daily)]

plt.figure(figsize=(12,4))
count_8day.plot(color='k')
count_daily.plot(color='r')

plt.legend(['8 day','daily'])    # there is one day in the datset have SST values on the land !


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[17]:
<matplotlib.legend.Legend at 0x1163910f0>

In [18]:
# Maps of individual days
target_date = '2003-02-15'
plt.figure(figsize=(8,6))
ds_8day.sst4.sel(time=target_date, method='nearest').plot()


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[18]:
<matplotlib.collections.QuadMesh at 0x11ef5fd30>

In [19]:
plt.figure(figsize=(8,6))
ds_daily.sst4.sel(time=target_date, method='nearest').plot()


Out[19]:
<matplotlib.collections.QuadMesh at 0x11f2fa588>

In [28]:
freq


Out[28]:
3

In [29]:
# next carry out interpolation starts here
freq_resample = str(freq) + 'D'
ds_resample = ds_daily.resample(freq_resample, dim='time')  # see the above for doc, test case, & default behavior
ds_resample


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[29]:
<xarray.Dataset>
Dimensions:  (lat: 552, lon: 720, time: 1847)
Coordinates:
  * lat      (lat) float64 27.98 27.94 27.9 27.85 27.81 27.77 27.73 27.69 ...
  * lon      (lon) float64 45.02 45.06 45.1 45.15 45.19 45.23 45.27 45.31 ...
  * time     (time) datetime64[ns] 2002-07-04 2002-07-07 2002-07-10 ...
Data variables:
    sst4     (time, lat, lon) float64 nan nan nan nan nan nan nan nan nan ...

In [30]:
plt.figure(figsize=(8,6))
ds_resample.sst4.sel(time=target_date, method='nearest').plot()


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
Out[30]:
<matplotlib.collections.QuadMesh at 0x1193ffb70>

In [31]:
# check the range for the longitude
print(ds_resample.lon.min(),'\n' ,ds_resample.lat.min())


<xarray.DataArray 'lon' ()>
array(45.02083206176758) 
 <xarray.DataArray 'lat' ()>
array(5.0208306312561035)

++++++++++++++++++++++++++++++++++++++++++++++

All GDP Floats

Load the float data

Map a (time, lon, lat) to a value on the cholorphlly value


In [32]:
# load preprocessed float data, and start the interpolation right here!!!!
var5 = "par"
var4 = "t865"
var3 = "kd490"
var2 = "cdm"
var1 = "chl"
vardist = "dist"

indir_prefix = "./data_globcolour/output.data.interpolate/" + "df_Globcolor_"
indir = indir_prefix + var1 + vardist + var2 + var3 + var4 + var5 + "_" + str(freq) + "d.csv"

floatDF_tmp = pd.read_csv(indir,index_col=0)
print(floatDF_tmp)


              id        time        lat        lon       temp         ve  \
1827       10206  2002-07-04  16.208333  66.375833        NaN  10.941333   
3654       10208  2002-07-04  13.816917  69.589833        NaN   9.899750   
5481       11089  2002-07-04  16.331167  64.731500  27.917667  12.239583   
7308       15703  2002-07-04  13.829750  69.617667  28.555167   9.378000   
10962      27069  2002-07-04  20.177000  68.844083  28.973000  26.284000   
14616      28842  2002-07-04  18.852000  60.748083  27.663833  11.585000   
16443      34159  2002-07-04  12.568250  59.009333        NaN  25.174250   
20097      34210  2002-07-04   6.409500  56.899000  26.715250  -9.563583   
21924      34211  2002-07-04   8.539083  68.015250  28.316167  20.941750   
23751      34212  2002-07-04   6.327167  64.844583  28.476000  23.924750   
38367      34708  2002-07-04  10.175333  59.870667  27.167000  42.975000   
42021      34710  2002-07-04  13.062167  49.858667  30.956917 -17.011917   
43848      34714  2002-07-04  13.643167  63.802250  27.707000  37.529500   
45675      34716  2002-07-04   7.507417  65.514500  28.814583  36.070250   
47502      34718  2002-07-04  16.206417  72.491917  29.149750  22.008917   
49329      34719  2002-07-04  17.720833  71.027333  28.921667  19.046167   
51156      34720  2002-07-04  14.669917  69.224833  28.653417   9.947083   
52983      34721  2002-07-04  17.159250  65.406167  27.919250   9.293667   
54810      34722  2002-07-04  11.702833  70.516500  28.712917   9.067667   
56637      34723  2002-07-04  16.687667  66.267000  28.457167   3.445750   
429345   2134712  2002-07-04   9.580250  63.608250  27.994167   9.460417   
1828       10206  2002-07-07  16.176417  66.548583        NaN   4.699917   
3655       10208  2002-07-07  13.513167  69.884583        NaN  15.487750   
5482       11089  2002-07-07  16.166750  65.045000  27.767250  14.413917   
7309       15703  2002-07-07  13.540000  69.895583  28.572750  14.464250   
10963      27069  2002-07-07  20.085167  69.475000  28.954250  22.634333   
14617      28842  2002-07-07  18.616833  60.873833  27.666833  -0.396667   
16444      34159  2002-07-07  12.787583  59.661250        NaN  33.664417   
20098      34210  2002-07-07   5.958667  56.622000  26.709667 -12.825083   
21925      34211  2002-07-07   8.209583  68.556000  28.406333  26.996250   
...          ...         ...        ...        ...        ...        ...   
566366  64115560  2017-06-24  16.696833  72.861833  29.779167  16.026167   
568193  64117500  2017-06-24   6.750167  70.993833  29.461833   8.437000   
422034    147140  2017-06-27  11.302333  60.222250  28.488917  47.011583   
425688    147144  2017-06-27  14.403833  70.675333  29.359333  14.331000   
496941  62321990  2017-06-27  13.098333  61.296500  28.240500  25.029583   
502422  63157510  2017-06-27   7.939917  60.210917  28.621750   6.026917   
506076  63158530  2017-06-27   7.948083  57.405500  28.807083  18.645167   
513384  63255180  2017-06-27   6.407917  63.229083  29.639917  12.110333   
522519  63258900  2017-06-27  10.770167  67.567333  29.242833  11.566750   
526173  63259180  2017-06-27   9.948250  70.782083  29.604833  16.286583   
542616  63348720  2017-06-27   7.019250  59.563125  28.293500  16.261714   
560886  64111550  2017-06-27  15.961500  67.288333  29.196417  13.215667   
562713  64113560  2017-06-27  14.065333  62.669417  28.106583   3.358250   
566367  64115560  2017-06-27  16.668667  73.197167  28.904083   9.514500   
568194  64117500  2017-06-27   6.571667  71.304333  29.409333   8.921333   
422035    147140  2017-06-30  11.557667  61.136333  28.039000  83.892000   
425689    147144  2017-06-30  14.291000  70.923667  29.260333  17.542500   
496942  62321990  2017-06-30  12.863000  61.727800  27.980800  24.212500   
502423  63157510  2017-06-30   8.170833  60.425000  28.591667  25.888400   
506077  63158530  2017-06-30   7.562400  57.591200  28.432600   2.102750   
513385  63255180  2017-06-30   6.457000  63.455750  29.492250  26.794667   
522520  63258900  2017-06-30  10.611667  67.725333  29.285333  11.653500   
526174  63259180  2017-06-30   9.708250  71.031750  29.490500  17.048000   
560887  64111550  2017-06-30  15.298917  67.618667  28.750917  14.127583   
562714  64113560  2017-06-30  13.832750  62.741000  27.985750   0.487667   
566368  64115560  2017-06-30  16.871250  73.274250  28.602000  -0.189000   
568195  64117500  2017-06-30   6.172583  71.411500  29.291417   3.511583   
560888  64111550  2017-07-03  15.032000  67.807000  28.605000        NaN   
562715  64113560  2017-07-03  13.760000  62.716000  27.871000        NaN   
568196  64117500  2017-07-03   5.869000  71.419000  29.169000        NaN   

               vn        spd   var_lat   var_lon      var_tmp   chlor_a  \
1827    -6.362417  12.924000  0.001675  0.006395  1000.000000       NaN   
3654   -19.104583  21.864250  0.000053  0.000096  1000.000000       NaN   
5481    -7.016583  14.670833  0.000076  0.000148     0.003607       NaN   
7308   -18.706167  21.204917  0.000054  0.000099     0.077642       NaN   
10962    4.231083  27.516167  0.000058  0.000107     0.001681       NaN   
14616   -5.116833  24.501167  0.000106  0.000225     0.003285       NaN   
16443    5.826667  26.245667  0.000058  0.000109  1000.000000       NaN   
20097  -16.234583  19.873667  0.000072  0.000146     0.003603  0.269992   
21924  -15.920167  26.681000  0.000054  0.000098     0.003496  0.035356   
23751   18.941000  32.034083  0.000053  0.000096     0.003571  0.072778   
38367    2.843583  43.217000  0.000050  0.000093     0.001796  0.321224   
42021   28.993750  47.145833  0.000037  0.000066     0.001769  0.127973   
43848   11.571500  39.495917  0.000058  0.000110     0.001840       NaN   
45675    3.266917  36.961917  0.000057  0.000105     0.001765  0.068843   
47502  -29.327667  37.194417  0.000046  0.000082     0.001739       NaN   
49329  -10.221667  22.969250  0.000049  0.000088     0.001578       NaN   
51156  -36.747667  38.318250  0.000063  0.000118     0.001779       NaN   
52983   -9.409917  14.014583  0.000063  0.000120     0.001746       NaN   
54810   -5.827833  12.391667  0.000066  0.000125     0.001827  0.213574   
56637  -14.846000  16.573750  0.000082  0.000173     0.001780       NaN   
429345 -38.704000  42.767250  0.000070  0.000136     0.001893  0.239311   
1828     2.534583   5.653083  0.001890  0.007368  1000.000000       NaN   
3655    -5.675667  17.390500  0.000073  0.000146  1000.000000       NaN   
5482   -10.390667  18.944083  0.000052  0.000096     0.003695       NaN   
7309    -4.531583  16.451750  0.000056  0.000104     0.091701       NaN   
10963   -9.058500  24.616667  0.000050  0.000091     0.001732       NaN   
14617   -7.914750  13.265917  0.000077  0.000152     0.003384       NaN   
16444   13.599417  36.723417  0.000058  0.000109  1000.000000       NaN   
20098  -14.977750  23.167000  0.000069  0.000136     0.003747  0.312435   
21925  -10.213500  30.061583  0.000060  0.000113     0.003481  0.096390   
...           ...        ...       ...       ...          ...       ...   
566366  -8.852083  19.160417  0.000003  0.000005     0.001698       NaN   
568193  -0.344250  11.212667  0.000003  0.000005     0.001684       NaN   
422034  27.559667  56.111500  0.000033  0.000015     0.001586       NaN   
425688  -5.099833  16.134500  0.000287  0.000143     0.001931       NaN   
496941 -14.966083  29.203583  0.008870  0.008387     0.001871       NaN   
502422  17.840000  19.168333  0.000009  0.000008     0.001769       NaN   
506076  -9.715833  26.787500  0.000240  0.000126     0.001786       NaN   
513384   1.742667  14.765333  0.000123  0.000058     0.001983       NaN   
522519 -20.607583  25.702500  0.000560  0.000320     0.001880       NaN   
526173 -18.976250  25.353250  0.001676  0.001119     0.001843       NaN   
542616   6.473714  17.652143  0.018954  0.019842     0.002010       NaN   
560886 -31.782083  35.294083  0.000009  0.000007     0.001684       NaN   
562713 -10.653917  11.876667  0.000130  0.000063     0.001684       NaN   
566367   7.251250  17.765000  0.000003  0.000005     0.001684       NaN   
568194 -16.165833  20.011250  0.000003  0.000005     0.001684  0.120794   
422035 -12.843000  85.019000  0.000032  0.000014     0.001585       NaN   
425689 -15.481500  23.507500  0.000129  0.000055     0.001716       NaN   
496942 -13.379500  27.666750  0.008339  0.007574     0.002022       NaN   
502423   5.273200  27.029800  0.000015  0.000010     0.001986       NaN   
506077 -33.419500  33.622250  0.000004  0.000006     0.001870       NaN   
513385  -4.388000  27.262333  0.000476  0.000268     0.002092       NaN   
522520  -5.334000  12.912000  0.000794  0.000473     0.002321       NaN   
526174 -12.979667  21.434667  0.000538  0.000273     0.002068       NaN   
560887 -23.375000  27.887500  0.000003  0.000005     0.001684       NaN   
562714  -8.357417  10.514917  0.000106  0.000049     0.001684       NaN   
566368  16.893333  17.142667  0.000003  0.000005     0.001754       NaN   
568195 -21.195083  22.225833  0.000003  0.000005     0.001684  0.130224   
560888        NaN        NaN  0.000003  0.000005     0.001963       NaN   
562715        NaN        NaN  0.000120  0.000049     0.001963       NaN   
568196        NaN        NaN  0.000003  0.000005     0.001963  0.143746   

               dist       cdm     kd490      t865        par  
1827     667.953438       NaN       NaN       NaN  49.034976  
3654     373.137389       NaN       NaN       NaN  53.496844  
5481     765.238337       NaN       NaN       NaN  53.454139  
7308     371.794920       NaN       NaN       NaN  53.520393  
10962    165.604551       NaN       NaN       NaN  46.622451  
14616    259.549813       NaN       NaN       NaN  55.028858  
16443    485.465494       NaN       NaN  0.503937  51.507130  
20097    722.709253  0.028208  0.073228  0.188034  52.464594  
21924    496.722689  0.013793  0.040269  0.245733  52.961485  
23751    869.183456  0.011631  0.046718  0.177406  51.805411  
38367    637.726441  0.044554  0.087768  0.281767  53.926559  
42021    150.041321  0.023710  0.057218  0.229587  54.856869  
43848    869.875997       NaN       NaN       NaN  52.382753  
45675    794.904527  0.013106  0.045550  0.174962  52.394974  
47502     94.399598       NaN       NaN       NaN  53.371143  
49329    209.207758       NaN       NaN       NaN  52.110463  
51156    466.775024       NaN       NaN       NaN  50.977288  
52983    658.601932       NaN       NaN       NaN  52.810799  
54810    181.318093  0.014866  0.070182  0.151630  53.497975  
56637    633.601722       NaN       NaN       NaN  47.800933  
429345   945.579543  0.021446  0.071863  0.184988  54.035277  
1828     659.605429       NaN       NaN       NaN  48.483505  
3655     327.080358       NaN       NaN       NaN  53.416297  
5482     763.443591       NaN       NaN       NaN  51.568421  
7309     328.082706       NaN       NaN       NaN  54.044723  
10963    128.044009       NaN       NaN       NaN  43.254285  
14617    286.213707       NaN       NaN       NaN  54.845130  
16444    556.657247       NaN       NaN       NaN  52.655368  
20098    744.690630  0.030674  0.079035  0.240987  48.682185  
21925    456.319394  0.009839  0.046514  0.144738  47.418040  
...             ...       ...       ...       ...        ...  
566366    47.894783       NaN       NaN       NaN  44.835579  
568193   192.580141       NaN       NaN       NaN  35.771104  
422034   633.789905       NaN       NaN       NaN  49.239842  
425688   347.961449       NaN       NaN       NaN  48.995252  
496941   720.307881       NaN       NaN       NaN  48.310914  
502422   802.690780       NaN       NaN       NaN  49.893464  
506076   596.807468       NaN       NaN       NaN  51.616781  
513384  1047.788375       NaN       NaN       NaN  44.533565  
522519   496.504659       NaN       NaN       NaN  51.876145  
526173   164.145250       NaN       NaN       NaN  45.877881  
542616   822.155576       NaN       NaN       NaN  50.951352  
560886   635.710919       NaN       NaN       NaN  37.295285  
562713   750.685783       NaN       NaN       NaN  46.043139  
566367    13.393062       NaN       NaN       NaN  40.387566  
568194   155.536670  0.013346  0.053254  0.137075  43.179400  
422035   726.254721       NaN       NaN       NaN  48.465288  
425689   326.533641       NaN       NaN       NaN  42.478397  
496942   772.348381       NaN       NaN       NaN  40.060113  
502423   805.220865       NaN       NaN       NaN  47.462840  
506077   643.899278       NaN       NaN       NaN  43.820887  
513385  1022.681205       NaN       NaN       NaN  44.700691  
522520   481.403226       NaN       NaN       NaN  41.214599  
526174   141.373115       NaN       NaN       NaN  25.991692  
560887   624.744140       NaN       NaN       NaN  47.919789  
562714   774.936277       NaN       NaN       NaN  47.710578  
566368     2.169704       NaN       NaN       NaN  44.126968  
568195   144.895234  0.013506  0.050810  0.140181  36.505060  
560888   607.056887       NaN       NaN       NaN  51.403744  
562715   779.169800       NaN       NaN       NaN  48.201698  
568196   153.656073  0.013860  0.050002  0.253969  43.950959  

[11247 rows x 17 columns]

In [33]:
import importlib
from tools.time_lat_lon_interpolate import interpolate
importlib.reload(interpolate)

result_out5 = interpolate.sel_points_multilinear_time_lat_lon(ds_resample, floatDF_tmp, dims = 'points', col_name ='sst4')
print('\n *** after the interpolation *** \n', result_out5)
# important: keep the id, since the dataframe has been modified in a bound-aware way in the function
print('\n *** this two length should be equal *** %d >= %d?' %(len(floatDF_tmp.index), len(result_out5.index) ) )


/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/dask/array/numpy_compat.py:45: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
 *** after the interpolation *** 
               id        time        lat        lon       temp         ve  \
1827       10206  2002-07-04  16.208333  66.375833        NaN  10.941333   
3654       10208  2002-07-04  13.816917  69.589833        NaN   9.899750   
5481       11089  2002-07-04  16.331167  64.731500  27.917667  12.239583   
7308       15703  2002-07-04  13.829750  69.617667  28.555167   9.378000   
10962      27069  2002-07-04  20.177000  68.844083  28.973000  26.284000   
14616      28842  2002-07-04  18.852000  60.748083  27.663833  11.585000   
16443      34159  2002-07-04  12.568250  59.009333        NaN  25.174250   
20097      34210  2002-07-04   6.409500  56.899000  26.715250  -9.563583   
21924      34211  2002-07-04   8.539083  68.015250  28.316167  20.941750   
23751      34212  2002-07-04   6.327167  64.844583  28.476000  23.924750   
38367      34708  2002-07-04  10.175333  59.870667  27.167000  42.975000   
42021      34710  2002-07-04  13.062167  49.858667  30.956917 -17.011917   
43848      34714  2002-07-04  13.643167  63.802250  27.707000  37.529500   
45675      34716  2002-07-04   7.507417  65.514500  28.814583  36.070250   
47502      34718  2002-07-04  16.206417  72.491917  29.149750  22.008917   
49329      34719  2002-07-04  17.720833  71.027333  28.921667  19.046167   
51156      34720  2002-07-04  14.669917  69.224833  28.653417   9.947083   
52983      34721  2002-07-04  17.159250  65.406167  27.919250   9.293667   
54810      34722  2002-07-04  11.702833  70.516500  28.712917   9.067667   
56637      34723  2002-07-04  16.687667  66.267000  28.457167   3.445750   
429345   2134712  2002-07-04   9.580250  63.608250  27.994167   9.460417   
1828       10206  2002-07-07  16.176417  66.548583        NaN   4.699917   
3655       10208  2002-07-07  13.513167  69.884583        NaN  15.487750   
5482       11089  2002-07-07  16.166750  65.045000  27.767250  14.413917   
7309       15703  2002-07-07  13.540000  69.895583  28.572750  14.464250   
10963      27069  2002-07-07  20.085167  69.475000  28.954250  22.634333   
14617      28842  2002-07-07  18.616833  60.873833  27.666833  -0.396667   
16444      34159  2002-07-07  12.787583  59.661250        NaN  33.664417   
20098      34210  2002-07-07   5.958667  56.622000  26.709667 -12.825083   
21925      34211  2002-07-07   8.209583  68.556000  28.406333  26.996250   
...          ...         ...        ...        ...        ...        ...   
566366  64115560  2017-06-24  16.696833  72.861833  29.779167  16.026167   
568193  64117500  2017-06-24   6.750167  70.993833  29.461833   8.437000   
422034    147140  2017-06-27  11.302333  60.222250  28.488917  47.011583   
425688    147144  2017-06-27  14.403833  70.675333  29.359333  14.331000   
496941  62321990  2017-06-27  13.098333  61.296500  28.240500  25.029583   
502422  63157510  2017-06-27   7.939917  60.210917  28.621750   6.026917   
506076  63158530  2017-06-27   7.948083  57.405500  28.807083  18.645167   
513384  63255180  2017-06-27   6.407917  63.229083  29.639917  12.110333   
522519  63258900  2017-06-27  10.770167  67.567333  29.242833  11.566750   
526173  63259180  2017-06-27   9.948250  70.782083  29.604833  16.286583   
542616  63348720  2017-06-27   7.019250  59.563125  28.293500  16.261714   
560886  64111550  2017-06-27  15.961500  67.288333  29.196417  13.215667   
562713  64113560  2017-06-27  14.065333  62.669417  28.106583   3.358250   
566367  64115560  2017-06-27  16.668667  73.197167  28.904083   9.514500   
568194  64117500  2017-06-27   6.571667  71.304333  29.409333   8.921333   
422035    147140  2017-06-30  11.557667  61.136333  28.039000  83.892000   
425689    147144  2017-06-30  14.291000  70.923667  29.260333  17.542500   
496942  62321990  2017-06-30  12.863000  61.727800  27.980800  24.212500   
502423  63157510  2017-06-30   8.170833  60.425000  28.591667  25.888400   
506077  63158530  2017-06-30   7.562400  57.591200  28.432600   2.102750   
513385  63255180  2017-06-30   6.457000  63.455750  29.492250  26.794667   
522520  63258900  2017-06-30  10.611667  67.725333  29.285333  11.653500   
526174  63259180  2017-06-30   9.708250  71.031750  29.490500  17.048000   
560887  64111550  2017-06-30  15.298917  67.618667  28.750917  14.127583   
562714  64113560  2017-06-30  13.832750  62.741000  27.985750   0.487667   
566368  64115560  2017-06-30  16.871250  73.274250  28.602000  -0.189000   
568195  64117500  2017-06-30   6.172583  71.411500  29.291417   3.511583   
560888  64111550  2017-07-03  15.032000  67.807000  28.605000        NaN   
562715  64113560  2017-07-03  13.760000  62.716000  27.871000        NaN   
568196  64117500  2017-07-03   5.869000  71.419000  29.169000        NaN   

               vn        spd   var_lat   var_lon      var_tmp   chlor_a  \
1827    -6.362417  12.924000  0.001675  0.006395  1000.000000       NaN   
3654   -19.104583  21.864250  0.000053  0.000096  1000.000000       NaN   
5481    -7.016583  14.670833  0.000076  0.000148     0.003607       NaN   
7308   -18.706167  21.204917  0.000054  0.000099     0.077642       NaN   
10962    4.231083  27.516167  0.000058  0.000107     0.001681       NaN   
14616   -5.116833  24.501167  0.000106  0.000225     0.003285       NaN   
16443    5.826667  26.245667  0.000058  0.000109  1000.000000       NaN   
20097  -16.234583  19.873667  0.000072  0.000146     0.003603  0.269992   
21924  -15.920167  26.681000  0.000054  0.000098     0.003496  0.035356   
23751   18.941000  32.034083  0.000053  0.000096     0.003571  0.072778   
38367    2.843583  43.217000  0.000050  0.000093     0.001796  0.321224   
42021   28.993750  47.145833  0.000037  0.000066     0.001769  0.127973   
43848   11.571500  39.495917  0.000058  0.000110     0.001840       NaN   
45675    3.266917  36.961917  0.000057  0.000105     0.001765  0.068843   
47502  -29.327667  37.194417  0.000046  0.000082     0.001739       NaN   
49329  -10.221667  22.969250  0.000049  0.000088     0.001578       NaN   
51156  -36.747667  38.318250  0.000063  0.000118     0.001779       NaN   
52983   -9.409917  14.014583  0.000063  0.000120     0.001746       NaN   
54810   -5.827833  12.391667  0.000066  0.000125     0.001827  0.213574   
56637  -14.846000  16.573750  0.000082  0.000173     0.001780       NaN   
429345 -38.704000  42.767250  0.000070  0.000136     0.001893  0.239311   
1828     2.534583   5.653083  0.001890  0.007368  1000.000000       NaN   
3655    -5.675667  17.390500  0.000073  0.000146  1000.000000       NaN   
5482   -10.390667  18.944083  0.000052  0.000096     0.003695       NaN   
7309    -4.531583  16.451750  0.000056  0.000104     0.091701       NaN   
10963   -9.058500  24.616667  0.000050  0.000091     0.001732       NaN   
14617   -7.914750  13.265917  0.000077  0.000152     0.003384       NaN   
16444   13.599417  36.723417  0.000058  0.000109  1000.000000       NaN   
20098  -14.977750  23.167000  0.000069  0.000136     0.003747  0.312435   
21925  -10.213500  30.061583  0.000060  0.000113     0.003481  0.096390   
...           ...        ...       ...       ...          ...       ...   
566366  -8.852083  19.160417  0.000003  0.000005     0.001698       NaN   
568193  -0.344250  11.212667  0.000003  0.000005     0.001684       NaN   
422034  27.559667  56.111500  0.000033  0.000015     0.001586       NaN   
425688  -5.099833  16.134500  0.000287  0.000143     0.001931       NaN   
496941 -14.966083  29.203583  0.008870  0.008387     0.001871       NaN   
502422  17.840000  19.168333  0.000009  0.000008     0.001769       NaN   
506076  -9.715833  26.787500  0.000240  0.000126     0.001786       NaN   
513384   1.742667  14.765333  0.000123  0.000058     0.001983       NaN   
522519 -20.607583  25.702500  0.000560  0.000320     0.001880       NaN   
526173 -18.976250  25.353250  0.001676  0.001119     0.001843       NaN   
542616   6.473714  17.652143  0.018954  0.019842     0.002010       NaN   
560886 -31.782083  35.294083  0.000009  0.000007     0.001684       NaN   
562713 -10.653917  11.876667  0.000130  0.000063     0.001684       NaN   
566367   7.251250  17.765000  0.000003  0.000005     0.001684       NaN   
568194 -16.165833  20.011250  0.000003  0.000005     0.001684  0.120794   
422035 -12.843000  85.019000  0.000032  0.000014     0.001585       NaN   
425689 -15.481500  23.507500  0.000129  0.000055     0.001716       NaN   
496942 -13.379500  27.666750  0.008339  0.007574     0.002022       NaN   
502423   5.273200  27.029800  0.000015  0.000010     0.001986       NaN   
506077 -33.419500  33.622250  0.000004  0.000006     0.001870       NaN   
513385  -4.388000  27.262333  0.000476  0.000268     0.002092       NaN   
522520  -5.334000  12.912000  0.000794  0.000473     0.002321       NaN   
526174 -12.979667  21.434667  0.000538  0.000273     0.002068       NaN   
560887 -23.375000  27.887500  0.000003  0.000005     0.001684       NaN   
562714  -8.357417  10.514917  0.000106  0.000049     0.001684       NaN   
566368  16.893333  17.142667  0.000003  0.000005     0.001754       NaN   
568195 -21.195083  22.225833  0.000003  0.000005     0.001684  0.130224   
560888        NaN        NaN  0.000003  0.000005     0.001963       NaN   
562715        NaN        NaN  0.000120  0.000049     0.001963       NaN   
568196        NaN        NaN  0.000003  0.000005     0.001963  0.143746   

               dist       cdm     kd490      t865        par       sst4  
1827     667.953438       NaN       NaN       NaN  49.034976  25.553169  
3654     373.137389       NaN       NaN       NaN  53.496844  27.911874  
5481     765.238337       NaN       NaN       NaN  53.454139        NaN  
7308     371.794920       NaN       NaN       NaN  53.520393  27.142097  
10962    165.604551       NaN       NaN       NaN  46.622451  26.487629  
14616    259.549813       NaN       NaN       NaN  55.028858  26.033749  
16443    485.465494       NaN       NaN  0.503937  51.507130  24.487131  
20097    722.709253  0.028208  0.073228  0.188034  52.464594  26.426437  
21924    496.722689  0.013793  0.040269  0.245733  52.961485  27.202769  
23751    869.183456  0.011631  0.046718  0.177406  51.805411        NaN  
38367    637.726441  0.044554  0.087768  0.281767  53.926559  25.107149  
42021    150.041321  0.023710  0.057218  0.229587  54.856869  30.800005  
43848    869.875997       NaN       NaN       NaN  52.382753  25.219260  
45675    794.904527  0.013106  0.045550  0.174962  52.394974  28.003749  
47502     94.399598       NaN       NaN       NaN  53.371143  27.423657  
49329    209.207758       NaN       NaN       NaN  52.110463  27.858124  
51156    466.775024       NaN       NaN       NaN  50.977288  27.628333  
52983    658.601932       NaN       NaN       NaN  52.810799  27.252755  
54810    181.318093  0.014866  0.070182  0.151630  53.497975  27.108313  
56637    633.601722       NaN       NaN       NaN  47.800933  27.529999  
429345   945.579543  0.021446  0.071863  0.184988  54.035277  26.691717  
1828     659.605429       NaN       NaN       NaN  48.483505  26.326233  
3655     327.080358       NaN       NaN       NaN  53.416297  27.885891  
5482     763.443591       NaN       NaN       NaN  51.568421  25.351550  
7309     328.082706       NaN       NaN       NaN  54.044723  27.738617  
10963    128.044009       NaN       NaN       NaN  43.254285  26.464999  
14617    286.213707       NaN       NaN       NaN  54.845130  24.313336  
16444    556.657247       NaN       NaN       NaN  52.655368  25.433203  
20098    744.690630  0.030674  0.079035  0.240987  48.682185  27.176635  
21925    456.319394  0.009839  0.046514  0.144738  47.418040  28.130644  
...             ...       ...       ...       ...        ...        ...  
566366    47.894783       NaN       NaN       NaN  44.835579  28.964999  
568193   192.580141       NaN       NaN       NaN  35.771104  29.208749  
422034   633.789905       NaN       NaN       NaN  49.239842  26.602507  
425688   347.961449       NaN       NaN       NaN  48.995252  28.354770  
496941   720.307881       NaN       NaN       NaN  48.310914  25.410887  
502422   802.690780       NaN       NaN       NaN  49.893464  25.815120  
506076   596.807468       NaN       NaN       NaN  51.616781  27.640820  
513384  1047.788375       NaN       NaN       NaN  44.533565  27.861277  
522519   496.504659       NaN       NaN       NaN  51.876145  28.018564  
526173   164.145250       NaN       NaN       NaN  45.877881  28.483235  
542616   822.155576       NaN       NaN       NaN  50.951352  27.285356  
560886   635.710919       NaN       NaN       NaN  37.295285        NaN  
562713   750.685783       NaN       NaN       NaN  46.043139  26.751249  
566367    13.393062       NaN       NaN       NaN  40.387566        NaN  
568194   155.536670  0.013346  0.053254  0.137075  43.179400  29.145761  
422035   726.254721       NaN       NaN       NaN  48.465288  27.437627  
425689   326.533641       NaN       NaN       NaN  42.478397  26.739999  
496942   772.348381       NaN       NaN       NaN  40.060113  27.024438  
502423   805.220865       NaN       NaN       NaN  47.462840  27.475581  
506077   643.899278       NaN       NaN       NaN  43.820887  27.456354  
513385  1022.681205       NaN       NaN       NaN  44.700691        NaN  
522520   481.403226       NaN       NaN       NaN  41.214599  27.512719  
526174   141.373115       NaN       NaN       NaN  25.991692  27.910672  
560887   624.744140       NaN       NaN       NaN  47.919789        NaN  
562714   774.936277       NaN       NaN       NaN  47.710578  26.259374  
566368     2.169704       NaN       NaN       NaN  44.126968        NaN  
568195   144.895234  0.013506  0.050810  0.140181  36.505060  28.834985  
560888   607.056887       NaN       NaN       NaN  51.403744  26.074999  
562715   779.169800       NaN       NaN       NaN  48.201698  24.818800  
568196   153.656073  0.013860  0.050002  0.253969  43.950959  28.652562  

[11247 rows x 18 columns]

 *** this two length should be equal *** 11247 >= 11247?
/Users/vyan2000/local/miniconda3/envs/condapython3/lib/python3.5/site-packages/numpy/lib/nanfunctions.py:703: RuntimeWarning: Mean of empty slice
  warnings.warn("Mean of empty slice", RuntimeWarning)

In [34]:
# output the dataframe result_out4
var6 = "sst4"
outdir_prefix = "./data_globcolour/output.data.interpolate/" + "df_Globcolor_"
outdir = outdir_prefix + var1 + vardist + var2 + var3 + var4 + var5 + var6 +"_" + str(freq) + "d.csv"
result_out5.to_csv(outdir)
print(pd.read_csv(outdir,index_col=0))

### plot for id 125776, which will be fit by LDS
plt.figure(figsize=(8,6))
result_out5[result_out5.id == 135776].plot(x='time', y ='sst4', title=('id - %d' % 135776) )
plt.show();
plt.close("all")


              id        time        lat        lon       temp         ve  \
1827       10206  2002-07-04  16.208333  66.375833        NaN  10.941333   
3654       10208  2002-07-04  13.816917  69.589833        NaN   9.899750   
5481       11089  2002-07-04  16.331167  64.731500  27.917667  12.239583   
7308       15703  2002-07-04  13.829750  69.617667  28.555167   9.378000   
10962      27069  2002-07-04  20.177000  68.844083  28.973000  26.284000   
14616      28842  2002-07-04  18.852000  60.748083  27.663833  11.585000   
16443      34159  2002-07-04  12.568250  59.009333        NaN  25.174250   
20097      34210  2002-07-04   6.409500  56.899000  26.715250  -9.563583   
21924      34211  2002-07-04   8.539083  68.015250  28.316167  20.941750   
23751      34212  2002-07-04   6.327167  64.844583  28.476000  23.924750   
38367      34708  2002-07-04  10.175333  59.870667  27.167000  42.975000   
42021      34710  2002-07-04  13.062167  49.858667  30.956917 -17.011917   
43848      34714  2002-07-04  13.643167  63.802250  27.707000  37.529500   
45675      34716  2002-07-04   7.507417  65.514500  28.814583  36.070250   
47502      34718  2002-07-04  16.206417  72.491917  29.149750  22.008917   
49329      34719  2002-07-04  17.720833  71.027333  28.921667  19.046167   
51156      34720  2002-07-04  14.669917  69.224833  28.653417   9.947083   
52983      34721  2002-07-04  17.159250  65.406167  27.919250   9.293667   
54810      34722  2002-07-04  11.702833  70.516500  28.712917   9.067667   
56637      34723  2002-07-04  16.687667  66.267000  28.457167   3.445750   
429345   2134712  2002-07-04   9.580250  63.608250  27.994167   9.460417   
1828       10206  2002-07-07  16.176417  66.548583        NaN   4.699917   
3655       10208  2002-07-07  13.513167  69.884583        NaN  15.487750   
5482       11089  2002-07-07  16.166750  65.045000  27.767250  14.413917   
7309       15703  2002-07-07  13.540000  69.895583  28.572750  14.464250   
10963      27069  2002-07-07  20.085167  69.475000  28.954250  22.634333   
14617      28842  2002-07-07  18.616833  60.873833  27.666833  -0.396667   
16444      34159  2002-07-07  12.787583  59.661250        NaN  33.664417   
20098      34210  2002-07-07   5.958667  56.622000  26.709667 -12.825083   
21925      34211  2002-07-07   8.209583  68.556000  28.406333  26.996250   
...          ...         ...        ...        ...        ...        ...   
566366  64115560  2017-06-24  16.696833  72.861833  29.779167  16.026167   
568193  64117500  2017-06-24   6.750167  70.993833  29.461833   8.437000   
422034    147140  2017-06-27  11.302333  60.222250  28.488917  47.011583   
425688    147144  2017-06-27  14.403833  70.675333  29.359333  14.331000   
496941  62321990  2017-06-27  13.098333  61.296500  28.240500  25.029583   
502422  63157510  2017-06-27   7.939917  60.210917  28.621750   6.026917   
506076  63158530  2017-06-27   7.948083  57.405500  28.807083  18.645167   
513384  63255180  2017-06-27   6.407917  63.229083  29.639917  12.110333   
522519  63258900  2017-06-27  10.770167  67.567333  29.242833  11.566750   
526173  63259180  2017-06-27   9.948250  70.782083  29.604833  16.286583   
542616  63348720  2017-06-27   7.019250  59.563125  28.293500  16.261714   
560886  64111550  2017-06-27  15.961500  67.288333  29.196417  13.215667   
562713  64113560  2017-06-27  14.065333  62.669417  28.106583   3.358250   
566367  64115560  2017-06-27  16.668667  73.197167  28.904083   9.514500   
568194  64117500  2017-06-27   6.571667  71.304333  29.409333   8.921333   
422035    147140  2017-06-30  11.557667  61.136333  28.039000  83.892000   
425689    147144  2017-06-30  14.291000  70.923667  29.260333  17.542500   
496942  62321990  2017-06-30  12.863000  61.727800  27.980800  24.212500   
502423  63157510  2017-06-30   8.170833  60.425000  28.591667  25.888400   
506077  63158530  2017-06-30   7.562400  57.591200  28.432600   2.102750   
513385  63255180  2017-06-30   6.457000  63.455750  29.492250  26.794667   
522520  63258900  2017-06-30  10.611667  67.725333  29.285333  11.653500   
526174  63259180  2017-06-30   9.708250  71.031750  29.490500  17.048000   
560887  64111550  2017-06-30  15.298917  67.618667  28.750917  14.127583   
562714  64113560  2017-06-30  13.832750  62.741000  27.985750   0.487667   
566368  64115560  2017-06-30  16.871250  73.274250  28.602000  -0.189000   
568195  64117500  2017-06-30   6.172583  71.411500  29.291417   3.511583   
560888  64111550  2017-07-03  15.032000  67.807000  28.605000        NaN   
562715  64113560  2017-07-03  13.760000  62.716000  27.871000        NaN   
568196  64117500  2017-07-03   5.869000  71.419000  29.169000        NaN   

               vn        spd   var_lat   var_lon      var_tmp   chlor_a  \
1827    -6.362417  12.924000  0.001675  0.006395  1000.000000       NaN   
3654   -19.104583  21.864250  0.000053  0.000096  1000.000000       NaN   
5481    -7.016583  14.670833  0.000076  0.000148     0.003607       NaN   
7308   -18.706167  21.204917  0.000054  0.000099     0.077642       NaN   
10962    4.231083  27.516167  0.000058  0.000107     0.001681       NaN   
14616   -5.116833  24.501167  0.000106  0.000225     0.003285       NaN   
16443    5.826667  26.245667  0.000058  0.000109  1000.000000       NaN   
20097  -16.234583  19.873667  0.000072  0.000146     0.003603  0.269992   
21924  -15.920167  26.681000  0.000054  0.000098     0.003496  0.035356   
23751   18.941000  32.034083  0.000053  0.000096     0.003571  0.072778   
38367    2.843583  43.217000  0.000050  0.000093     0.001796  0.321224   
42021   28.993750  47.145833  0.000037  0.000066     0.001769  0.127973   
43848   11.571500  39.495917  0.000058  0.000110     0.001840       NaN   
45675    3.266917  36.961917  0.000057  0.000105     0.001765  0.068843   
47502  -29.327667  37.194417  0.000046  0.000082     0.001739       NaN   
49329  -10.221667  22.969250  0.000049  0.000088     0.001578       NaN   
51156  -36.747667  38.318250  0.000063  0.000118     0.001779       NaN   
52983   -9.409917  14.014583  0.000063  0.000120     0.001746       NaN   
54810   -5.827833  12.391667  0.000066  0.000125     0.001827  0.213574   
56637  -14.846000  16.573750  0.000082  0.000173     0.001780       NaN   
429345 -38.704000  42.767250  0.000070  0.000136     0.001893  0.239311   
1828     2.534583   5.653083  0.001890  0.007368  1000.000000       NaN   
3655    -5.675667  17.390500  0.000073  0.000146  1000.000000       NaN   
5482   -10.390667  18.944083  0.000052  0.000096     0.003695       NaN   
7309    -4.531583  16.451750  0.000056  0.000104     0.091701       NaN   
10963   -9.058500  24.616667  0.000050  0.000091     0.001732       NaN   
14617   -7.914750  13.265917  0.000077  0.000152     0.003384       NaN   
16444   13.599417  36.723417  0.000058  0.000109  1000.000000       NaN   
20098  -14.977750  23.167000  0.000069  0.000136     0.003747  0.312435   
21925  -10.213500  30.061583  0.000060  0.000113     0.003481  0.096390   
...           ...        ...       ...       ...          ...       ...   
566366  -8.852083  19.160417  0.000003  0.000005     0.001698       NaN   
568193  -0.344250  11.212667  0.000003  0.000005     0.001684       NaN   
422034  27.559667  56.111500  0.000033  0.000015     0.001586       NaN   
425688  -5.099833  16.134500  0.000287  0.000143     0.001931       NaN   
496941 -14.966083  29.203583  0.008870  0.008387     0.001871       NaN   
502422  17.840000  19.168333  0.000009  0.000008     0.001769       NaN   
506076  -9.715833  26.787500  0.000240  0.000126     0.001786       NaN   
513384   1.742667  14.765333  0.000123  0.000058     0.001983       NaN   
522519 -20.607583  25.702500  0.000560  0.000320     0.001880       NaN   
526173 -18.976250  25.353250  0.001676  0.001119     0.001843       NaN   
542616   6.473714  17.652143  0.018954  0.019842     0.002010       NaN   
560886 -31.782083  35.294083  0.000009  0.000007     0.001684       NaN   
562713 -10.653917  11.876667  0.000130  0.000063     0.001684       NaN   
566367   7.251250  17.765000  0.000003  0.000005     0.001684       NaN   
568194 -16.165833  20.011250  0.000003  0.000005     0.001684  0.120794   
422035 -12.843000  85.019000  0.000032  0.000014     0.001585       NaN   
425689 -15.481500  23.507500  0.000129  0.000055     0.001716       NaN   
496942 -13.379500  27.666750  0.008339  0.007574     0.002022       NaN   
502423   5.273200  27.029800  0.000015  0.000010     0.001986       NaN   
506077 -33.419500  33.622250  0.000004  0.000006     0.001870       NaN   
513385  -4.388000  27.262333  0.000476  0.000268     0.002092       NaN   
522520  -5.334000  12.912000  0.000794  0.000473     0.002321       NaN   
526174 -12.979667  21.434667  0.000538  0.000273     0.002068       NaN   
560887 -23.375000  27.887500  0.000003  0.000005     0.001684       NaN   
562714  -8.357417  10.514917  0.000106  0.000049     0.001684       NaN   
566368  16.893333  17.142667  0.000003  0.000005     0.001754       NaN   
568195 -21.195083  22.225833  0.000003  0.000005     0.001684  0.130224   
560888        NaN        NaN  0.000003  0.000005     0.001963       NaN   
562715        NaN        NaN  0.000120  0.000049     0.001963       NaN   
568196        NaN        NaN  0.000003  0.000005     0.001963  0.143746   

               dist       cdm     kd490      t865        par       sst4  
1827     667.953438       NaN       NaN       NaN  49.034976  25.553169  
3654     373.137389       NaN       NaN       NaN  53.496844  27.911874  
5481     765.238337       NaN       NaN       NaN  53.454139        NaN  
7308     371.794920       NaN       NaN       NaN  53.520393  27.142097  
10962    165.604551       NaN       NaN       NaN  46.622451  26.487629  
14616    259.549813       NaN       NaN       NaN  55.028858  26.033749  
16443    485.465494       NaN       NaN  0.503937  51.507130  24.487131  
20097    722.709253  0.028208  0.073228  0.188034  52.464594  26.426437  
21924    496.722689  0.013793  0.040269  0.245733  52.961485  27.202769  
23751    869.183456  0.011631  0.046718  0.177406  51.805411        NaN  
38367    637.726441  0.044554  0.087768  0.281767  53.926559  25.107149  
42021    150.041321  0.023710  0.057218  0.229587  54.856869  30.800005  
43848    869.875997       NaN       NaN       NaN  52.382753  25.219260  
45675    794.904527  0.013106  0.045550  0.174962  52.394974  28.003749  
47502     94.399598       NaN       NaN       NaN  53.371143  27.423657  
49329    209.207758       NaN       NaN       NaN  52.110463  27.858124  
51156    466.775024       NaN       NaN       NaN  50.977288  27.628333  
52983    658.601932       NaN       NaN       NaN  52.810799  27.252755  
54810    181.318093  0.014866  0.070182  0.151630  53.497975  27.108313  
56637    633.601722       NaN       NaN       NaN  47.800933  27.529999  
429345   945.579543  0.021446  0.071863  0.184988  54.035277  26.691717  
1828     659.605429       NaN       NaN       NaN  48.483505  26.326233  
3655     327.080358       NaN       NaN       NaN  53.416297  27.885891  
5482     763.443591       NaN       NaN       NaN  51.568421  25.351550  
7309     328.082706       NaN       NaN       NaN  54.044723  27.738617  
10963    128.044009       NaN       NaN       NaN  43.254285  26.464999  
14617    286.213707       NaN       NaN       NaN  54.845130  24.313336  
16444    556.657247       NaN       NaN       NaN  52.655368  25.433203  
20098    744.690630  0.030674  0.079035  0.240987  48.682185  27.176635  
21925    456.319394  0.009839  0.046514  0.144738  47.418040  28.130644  
...             ...       ...       ...       ...        ...        ...  
566366    47.894783       NaN       NaN       NaN  44.835579  28.964999  
568193   192.580141       NaN       NaN       NaN  35.771104  29.208749  
422034   633.789905       NaN       NaN       NaN  49.239842  26.602507  
425688   347.961449       NaN       NaN       NaN  48.995252  28.354770  
496941   720.307881       NaN       NaN       NaN  48.310914  25.410887  
502422   802.690780       NaN       NaN       NaN  49.893464  25.815120  
506076   596.807468       NaN       NaN       NaN  51.616781  27.640820  
513384  1047.788375       NaN       NaN       NaN  44.533565  27.861277  
522519   496.504659       NaN       NaN       NaN  51.876145  28.018564  
526173   164.145250       NaN       NaN       NaN  45.877881  28.483235  
542616   822.155576       NaN       NaN       NaN  50.951352  27.285356  
560886   635.710919       NaN       NaN       NaN  37.295285        NaN  
562713   750.685783       NaN       NaN       NaN  46.043139  26.751249  
566367    13.393062       NaN       NaN       NaN  40.387566        NaN  
568194   155.536670  0.013346  0.053254  0.137075  43.179400  29.145761  
422035   726.254721       NaN       NaN       NaN  48.465288  27.437627  
425689   326.533641       NaN       NaN       NaN  42.478397  26.739999  
496942   772.348381       NaN       NaN       NaN  40.060113  27.024438  
502423   805.220865       NaN       NaN       NaN  47.462840  27.475581  
506077   643.899278       NaN       NaN       NaN  43.820887  27.456354  
513385  1022.681205       NaN       NaN       NaN  44.700691        NaN  
522520   481.403226       NaN       NaN       NaN  41.214599  27.512719  
526174   141.373115       NaN       NaN       NaN  25.991692  27.910672  
560887   624.744140       NaN       NaN       NaN  47.919789        NaN  
562714   774.936277       NaN       NaN       NaN  47.710578  26.259374  
566368     2.169704       NaN       NaN       NaN  44.126968        NaN  
568195   144.895234  0.013506  0.050810  0.140181  36.505060  28.834985  
560888   607.056887       NaN       NaN       NaN  51.403744  26.074999  
562715   779.169800       NaN       NaN       NaN  48.201698  24.818800  
568196   153.656073  0.013860  0.050002  0.253969  43.950959  28.652562  

[11247 rows x 18 columns]
<matplotlib.figure.Figure at 0x1193b8c18>

In [ ]:


In [ ]:


In [ ]:


In [ ]: