Reading the Docklight CO2 instrument text file

In [1]:

    

'''
Try to read the Docklight CO2 instrument text file:

#Reading the Docklight CO2 instrument file

Originally it looked like this:

Docklight Log File (ASCII) - Started 6/14/2013 11:34:17.033 
 RH0000   P1040X<CR><LF>
2013/06/04 19:42:23 - G 498.00 T26.78    HT0000 RH0000   P1040X<CR><LF>
2013/06/04 19:42:25 - G 498.00 T26.78    HT0000 RH0000   P1040X<CR><LF>
2013/06/04 19:42:27 - G 498.00 T26.78    HT0000 RH0000   P1040X<CR><LF>
2013/06/04 19:42:29 - G 498.00 T26.79    HT0000 RH0000   P1040X<CR><LF>
2013/06/04 19:42:31 - G 498.00 T26.80    HT0000 RH0000   P1042X<CR><LF>

and also had junk in the middle and end of the file.

So we cleaned it to look like this using notepad++:

Docklight Log File (ASCII) - Started 6/14/2013 11:34:17.033 
 RH0000   P1040X<CR><LF>
2013/06/04 19:42:23  498.00 26.78    0000 0000   1040
2013/06/04 19:42:25  498.00 26.78    0000 0000   1040
2013/06/04 19:42:27  498.00 26.78    0000 0000   1040
2013/06/04 19:42:29  498.00 26.79    0000 0000   1040
2013/06/04 19:42:31  498.00 26.80    0000 0000   1042

'''


    

In [25]:

    

import pandas as pd
file='./data/ICO2sensordata_asc.txt'


    

In [26]:

    

df = pd.read_csv(file,skiprows = [0,1],parse_dates = [[0,1]], index_col = 0, sep = r"\s*", 
    names=['date','time','CO2','Temperature','Humidity','Relative Humidity','Pressure'],header = None)


    

In [27]:

    

df.head()


    

    
Out[27]:






  

    

      

      
CO2
      
Temperature
      
Humidity
      
Relative Humidity
      
Pressure
    
    

      
date_time
      

      

      

      

      

    
  
  

    

      
2013-06-04 19:42:23
      
 498
      
 26.78
      
 0
      
 0
      
 1040
    
    

      
2013-06-04 19:42:25
      
 498
      
 26.78
      
 0
      
 0
      
 1040
    
    

      
2013-06-04 19:42:27
      
 498
      
 26.78
      
 0
      
 0
      
 1040
    
    

      
2013-06-04 19:42:29
      
 498
      
 26.79
      
 0
      
 0
      
 1040
    
    

      
2013-06-04 19:42:31
      
 498
      
 26.80
      
 0
      
 0
      
 1042
    
  

In [28]:

    

df['CO2'].plot(figsize=(12,6))


    

    
Out[28]:





<matplotlib.axes.AxesSubplot at 0x118246d0>






    

In [29]:

    

df_5min=df.resample('5min',how = 'mean')


    

In [39]:

    

df_5min['Temperature'][0:20]


    

    
Out[39]:





date_time
2013-06-04 20:35:00    28.801429
2013-06-04 20:40:00    29.013267
2013-06-04 20:45:00    29.330200
2013-06-04 20:50:00    29.563600
2013-06-04 20:55:00    29.785733
2013-06-04 21:00:00    29.965400
2013-06-04 21:05:00    30.098667
2013-06-04 21:10:00    30.078000
2013-06-04 21:15:00    29.631400
2013-06-04 21:20:00    29.010533
2013-06-04 21:25:00    28.612267
2013-06-04 21:30:00    28.468000
2013-06-04 21:35:00    28.440267
2013-06-04 21:40:00    28.449333
2013-06-04 21:45:00    28.499067
2013-06-04 21:50:00    28.554000
2013-06-04 21:55:00    28.497000
2013-06-04 22:00:00    28.248000
2013-06-04 22:05:00    27.980533
2013-06-04 22:10:00    27.826400
Freq: 5T, Name: Temperature, dtype: float64

In [31]:

    

df_5min['Temperature']['2013-06-11 12:00:00':'2013-06-11 18:00:00']


    

    
Out[31]:





date_time
2013-06-11 12:00:00    24.309733
2013-06-11 12:05:00    24.347000
2013-06-11 12:10:00    24.381133
2013-06-11 12:15:00    24.431333
2013-06-11 12:20:00    24.486600
2013-06-11 12:25:00    24.562800
2013-06-11 12:30:00    24.635800
2013-06-11 12:35:00    24.711800
2013-06-11 12:40:00    24.784000
2013-06-11 12:45:00    24.848733
2013-06-11 12:50:00    24.905400
2013-06-11 12:55:00    24.976733
2013-06-11 13:00:00    25.056267
2013-06-11 13:05:00    25.145200
2013-06-11 13:10:00    25.214200
...
2013-06-11 16:50:00    25.417733
2013-06-11 16:55:00    25.456533
2013-06-11 17:00:00    25.470533
2013-06-11 17:05:00    25.500467
2013-06-11 17:10:00    25.520400
2013-06-11 17:15:00    25.476970
2013-06-11 17:20:00          NaN
2013-06-11 17:25:00          NaN
2013-06-11 17:30:00          NaN
2013-06-11 17:35:00          NaN
2013-06-11 17:40:00          NaN
2013-06-11 17:45:00          NaN
2013-06-11 17:50:00          NaN
2013-06-11 17:55:00          NaN
2013-06-11 18:00:00          NaN
Freq: 5T, Name: Temperature, Length: 73, dtype: float64

In [32]:

    

# clip to time when instrument was in water
df_5min=df_5min['2013-06-04 20:35:00':'2013-06-11 17:15:00']


    

In [33]:

    

df_5min[['CO2','Temperature']].plot(figsize=(12,4),secondary_y='Temperature');


    

In [34]:

    

df_5min[['Humidity','Relative Humidity']].plot(figsize=(12,4));


    

In [35]:

    

df_5min['Pressure'].plot(figsize=(12,4));


    

In [36]:

    

# look at correlation between Temp and CO2
corrcoef(df_5min['Temperature'],df_5min['CO2'])


    

    
Out[36]:





array([[ 1.        , -0.32966428],
       [-0.32966428,  1.        ]])

In [37]:

    

plot(df_5min['Temperature'],df_5min['CO2'],'go');
grid();


    

In [ ]: