notebook.community

Edit and run 





In [1]:



    


import pymicra as pm
import pandas as pd
from glob import glob

fconfig = pm.fileConfig('tij_pr_qc.config')
fnames = sorted(glob('mydata/*.out'))

After the preamble is loaded, the simplest way to obtain fluxes and relevant data scales is

  • Quality control
  • Obtain fluctuations
  • Calculate fluxes and covariances The script below does this in the simplest way possible.

First, start with the quality control, which is separated into two functions:





In [2]:



    


pm.util.qc_replace(fnames, fconfig,
    file_lines=36000,
    lower_limits=dict(theta_v=10, mrho_h2o=0, mrho_co2=0),
    upper_limits=dict(theta_v=45),
    spikes_test=True,
    max_replacement_count=360, # replacement count test
    chunk_size=1200,
    outdir='out1',
    replaced_report='rrep.txt')

fnames2 = sorted(glob('out1/*.out'))
pm.util.qc_discard(fnames2, fconfig,
    std_limits = dict(u=0.03, v=0.03, w=0.01, theta_v=0.02),
    dif_limits = dict(u=4.0, v=4.0, w=1.0, theta_v=2.0),
    chunk_size=1200,
    outdir='out2',
    summary_file='discard_summary.csv',
    full_report='frep.txt')



    


















    






20110224-1220.out
Passed all tests
Re-writing mydata/20110224-1220.out

20110224-1250.out
Passed all tests
Re-writing mydata/20110224-1250.out

20110224-1340.out
Passed all tests
Re-writing mydata/20110224-1340.out

20110224-1410.out
Passed all tests
Re-writing mydata/20110224-1410.out

20110224-1440.out
Passed all tests
Re-writing mydata/20110224-1440.out

20110224-1510.out
Passed all tests
Re-writing mydata/20110224-1510.out

20110224-1610.out
Passed all tests
Re-writing mydata/20110224-1610.out

20110224-1640.out
Passed all tests
Re-writing mydata/20110224-1640.out

                           control  percent
total                            8    100.0
failed lines test                0      0.0
failed replacement test          0      0.0
passed all tests                 8    100.0
Runs with replaced nans          0      0.0
Runs with replaced bound         0      0.0
Runs with replaced spikes        8    100.0
20110224-1220.out
20110224-1220.out : !FAILED failed maxdif test test!

Failed variable(s): theta_v, u 

20110224-1250.out
20110224-1250.out : !FAILED failed maxdif test test!

Failed variable(s): theta_v 

20110224-1340.out
Passed all tests
Re-writing out1/20110224-1340.out

20110224-1410.out
20110224-1410.out : !FAILED failed maxdif test test!

Failed variable(s): theta_v 

20110224-1440.out
Passed all tests
Re-writing out1/20110224-1440.out

20110224-1510.out
20110224-1510.out : !FAILED failed maxdif test test!

Failed variable(s): theta_v 

20110224-1610.out
20110224-1610.out : !FAILED failed maxdif test test!

Failed variable(s): theta_v 

20110224-1640.out
Passed all tests
Re-writing out1/20110224-1640.out

                    control  percent
total                     8    100.0
failed STD test           0      0.0
failed maxdif test        5     62.5
passed all tests          3     37.5










    
Out[2]:











  
    

      
      passed all tests
      total
      failed STD test
      failed maxdif test
    

  
  
    

      0
      NaN
      20110224-1220.out
      NaN
      20110224-1220.out
    

    

      1
      NaN
      20110224-1250.out
      NaN
      20110224-1250.out
    

    

      2
      20110224-1340.out
      20110224-1340.out
      NaN
      NaN
    

    

      3
      NaN
      20110224-1410.out
      NaN
      20110224-1410.out
    

    

      4
      20110224-1440.out
      20110224-1440.out
      NaN
      NaN
    

    

      5
      NaN
      20110224-1510.out
      NaN
      20110224-1510.out
    

    

      6
      NaN
      20110224-1610.out
      NaN
      20110224-1610.out
    

    

      7
      20110224-1640.out
      20110224-1640.out
      NaN
      NaN

Now we calculate the fluxes and scales only with the quality-controled data:





In [5]:



    


fconfig = pm.fileConfig('tij_pr.config')
sconfig = pm.siteConfig('tij_pr.site')
fnames = sorted(glob('out2/*out'))
allresults=[]
for fname in fnames:
    data, units = pm.timeSeries(fname, fconfig, parse_dates=False)
    data = pm.micro.preProcess(data, units, solutes=['co2'])
    fulldata = data.detrend(units=units, ignore=['p'], join_data=True)

    results = pm.micro.eddyCovariance(fulldata, units, site_config=sconfig, solutes=['co2'])
    allresults.append(results)

allresults = pd.concat(allresults, ignore_index=True)



    


















    






Beginning of pre-processing ...
Rotating data with 2d method ... Done!
Converting theta_v to kelvin ... Done!
Didn't locate mass density of h2o. Trying to calculate it ... Done!
Moist air density not present in dataset
Calculating rho_air = p/(Rdry * theta_v) ... Done!
Calculating dry_air mass_density = rho_air - rho_h2o ... Done!
Dry air molar density not in dataset
Calculating dry_air molar_density = rho_dry / dry_air_molar_mass ... Done!
Calculating specific humidity = rho_h2o / rho_air ... Done!
Calculating h2o mass mixing ratio = rho_h2o / rho_dry ... Done!
Calculating h2o molar mixing ratio = rho_h2o / rho_dry ... Done!
Thermodynamic temperature not found ... trying to calculate it with theta_v ~ theta (1 + 0.61 q) relation ... done!
Didn't locate mass density of co2. Trying to calculate it ... Done!
Calculating co2 mass concentration (g/g) = rho_co2 / rho_air ... Done!
Calculating co2 mass mixing ratio = rho_co2 / rho_dry ... Done!
Calculating co2 molar mixing ratio = mrho_co2 / mrho_dry ... Done!
Pre-processing complete.

Beginning Eddy Covariance method...
Fluctuations of theta not found. Will try to calculate it with theta' = (theta_v' - 0.61 theta_mean q')/(1 + 0.61 q_mean ... done!
Calculating fluxes from covariances ... done!
Applying WPL correction for water vapor flux ... done!
Applying WPL correction for latent heat flux using result for water vapor flux ... done!
Re-calculating cov(mrho_h2o', w') according to WPL correction ... done!
Applying WPL correction for F_co2 ... done!
Re-calculating cov(mrho_co2', w') according to WPL correction ... done!
Beginning to extract turbulent scales...
Data seems to be covariances. Will it use as covariances ...
Calculating the turbulent scales of wind, temperature and humidity ... done!
Calculating the turbulent scale of co2 ... done!
Calculating Obukhov length and stability parameter ... done!
Calculating turbulent scales of mass concentration ... done!
Done with Eddy Covariance.

Beginning of pre-processing ...
Rotating data with 2d method ... Done!
Converting theta_v to kelvin ... Done!
Didn't locate mass density of h2o. Trying to calculate it ... Done!
Moist air density not present in dataset
Calculating rho_air = p/(Rdry * theta_v) ... Done!
Calculating dry_air mass_density = rho_air - rho_h2o ... Done!
Dry air molar density not in dataset
Calculating dry_air molar_density = rho_dry / dry_air_molar_mass ... Done!
Calculating specific humidity = rho_h2o / rho_air ... Done!
Calculating h2o mass mixing ratio = rho_h2o / rho_dry ... Done!
Calculating h2o molar mixing ratio = rho_h2o / rho_dry ... Done!
Thermodynamic temperature not found ... trying to calculate it with theta_v ~ theta (1 + 0.61 q) relation ... done!
Didn't locate mass density of co2. Trying to calculate it ... Done!
Calculating co2 mass concentration (g/g) = rho_co2 / rho_air ... Done!
Calculating co2 mass mixing ratio = rho_co2 / rho_dry ... Done!
Calculating co2 molar mixing ratio = mrho_co2 / mrho_dry ... Done!
Pre-processing complete.

Beginning Eddy Covariance method...
Fluctuations of theta not found. Will try to calculate it with theta' = (theta_v' - 0.61 theta_mean q')/(1 + 0.61 q_mean ... done!
Calculating fluxes from covariances ... done!
Applying WPL correction for water vapor flux ... done!
Applying WPL correction for latent heat flux using result for water vapor flux ... done!
Re-calculating cov(mrho_h2o', w') according to WPL correction ... done!
Applying WPL correction for F_co2 ... done!
Re-calculating cov(mrho_co2', w') according to WPL correction ... done!
Beginning to extract turbulent scales...
Data seems to be covariances. Will it use as covariances ...
Calculating the turbulent scales of wind, temperature and humidity ... done!
Calculating the turbulent scale of co2 ... done!
Calculating Obukhov length and stability parameter ... done!
Calculating turbulent scales of mass concentration ... done!
Done with Eddy Covariance.

Beginning of pre-processing ...
Rotating data with 2d method ... Done!
Converting theta_v to kelvin ... Done!
Didn't locate mass density of h2o. Trying to calculate it ... Done!
Moist air density not present in dataset
Calculating rho_air = p/(Rdry * theta_v) ... Done!
Calculating dry_air mass_density = rho_air - rho_h2o ... Done!
Dry air molar density not in dataset
Calculating dry_air molar_density = rho_dry / dry_air_molar_mass ... Done!
Calculating specific humidity = rho_h2o / rho_air ... Done!
Calculating h2o mass mixing ratio = rho_h2o / rho_dry ... Done!
Calculating h2o molar mixing ratio = rho_h2o / rho_dry ... Done!
Thermodynamic temperature not found ... trying to calculate it with theta_v ~ theta (1 + 0.61 q) relation ... done!
Didn't locate mass density of co2. Trying to calculate it ... Done!
Calculating co2 mass concentration (g/g) = rho_co2 / rho_air ... Done!
Calculating co2 mass mixing ratio = rho_co2 / rho_dry ... Done!
Calculating co2 molar mixing ratio = mrho_co2 / mrho_dry ... Done!
Pre-processing complete.

Beginning Eddy Covariance method...
Fluctuations of theta not found. Will try to calculate it with theta' = (theta_v' - 0.61 theta_mean q')/(1 + 0.61 q_mean ... done!
Calculating fluxes from covariances ... done!
Applying WPL correction for water vapor flux ... done!
Applying WPL correction for latent heat flux using result for water vapor flux ... done!
Re-calculating cov(mrho_h2o', w') according to WPL correction ... done!
Applying WPL correction for F_co2 ... done!
Re-calculating cov(mrho_co2', w') according to WPL correction ... done!
Beginning to extract turbulent scales...
Data seems to be covariances. Will it use as covariances ...
Calculating the turbulent scales of wind, temperature and humidity ... done!
Calculating the turbulent scale of co2 ... done!
Calculating Obukhov length and stability parameter ... done!
Calculating turbulent scales of mass concentration ... done!
Done with Eddy Covariance.


















In [4]:



    


print(allresults.with_units(units))



    


















    






                               tau                   H                  Hv  \
  <kilogram / meter / second ** 2> <watt / meter ** 2> <watt / meter ** 2>   
0                         0.034587          -18.135628           -5.744854   
1                         0.016308          -15.999976           -2.961901   
2                         0.144688          -15.763729           -2.274071   

                                  E                  LE  \
  <millimole / meter ** 2 / second> <watt / meter ** 2>   
0                          3.930086          173.166173   
1                          4.089761          179.763498   
2                          4.315018          190.512702   

                              F_co2           u_star theta_v_star theta_star  \
  <millimole / meter ** 2 / second> <meter / second>     <kelvin>   <kelvin>   
0                         -0.012771         0.180323    -0.029847  -0.094221   
1                         -0.012538         0.124400    -0.022516  -0.121627   
2                         -0.013377         0.368191    -0.005767  -0.039974   

             mrho_h2o_star            mrho_co2_star           Lo            zeta  \
  <millimole / meter ** 3> <millimole / meter ** 3>      <meter> <dimensionless>   
0                21.794729                -0.070824    83.302161        0.022208   
1                32.875863                -0.100788    53.002880        0.034904   
2                11.719512                -0.036331  1787.779908        0.001035   

           q_star   conc_co2_star  
  <dimensionless> <dimensionless>  
0        0.000369       -0.000003  
1        0.000562       -0.000004  
2        0.000198       -0.000001  

















In [ ]:

Content source: tomchor/pymicra

Similar notebooks:

notebook.community | gallery | about