Simple quick update latest weather
In [318]:
# Tell matplotlib to plot in line
%matplotlib inline
import datetime
# import pandas
import pandas
# seaborn magically adds a layer of goodness on top of Matplotlib
# mostly this is just changing matplotlib defaults, but it does also
# provide some higher level plotting methods.
import seaborn
# Tell seaborn to set things up
seaborn.set()
def smooth(data, thresh=None):
""" This smoothing function is meant to spot and eliminate bad readings.
It has some issues ;)
Also, I really need to find out what is causing the bad readings.
"""
means = data.mean()
if thresh is None:
sds = data.std()
else:
sds = thresh
delta = data - data.shift()
good = delta[abs(delta) < sds]
#print(good.describe())
return delta.where(good, 0.0)
In [319]:
infile = "../files/kittycam_weather.csv"
!scp 192.168.0.128:Adafruit_Python_BMP/weather.csv $infile
In [320]:
!wc ../files/weather.csv
In [321]:
""" assume it is csv and let pandas do magic
index_col tells it to use the 'date' column in the data
as the row index, plotting picks up on this and uses the
date on the x-axis
The *parse_dates* bit just tells it to try and figure out
the date/time in the columne labeled 'date'.
"""
data = pandas.read_csv(infile, index_col='date', parse_dates=['date'])
In [322]:
data.plot()
Out[322]:
In [323]:
# smooth the data to filter out bad temps and pressures
#data.altitude = (smooth(data.altitude, 5.0).cumsum() + data.altitude[0])
#data.temp = (smooth(data.temp, 5.0).cumsum() + data.temp[0])
data.altitude = data.altitude.clip(-100, 100)
In [324]:
data.altitude.plot()
Out[324]:
In [325]:
data.tail()
Out[325]:
In [326]:
data.describe()
Out[326]:
The warmer air is, the more moisture it can hold. The dew point is the temperature at which air would be totally saturated if it had as much moisture as it currently does.
Given the temperature and humidity the dew point can be calculated, the actual formula is pretty complex.
It is explained in more detail here: http://iridl.ldeo.columbia.edu/dochelp/QA/Basic/dewpoint.html
$$Td = T - ((100 - RH)/5.)$$If you are interested in a simpler calculation that gives an approximation of dew point temperature if you know >the observed temperature and relative humidity, the following formula was proposed in a 2005 article by Mark G. >Lawrence in the Bulletin of the American Meteorological Society:
In [327]:
data['dewpoint'] = data.temp - ((100. - data.humidity)/5.)
In [328]:
print(datetime.datetime.now())
print(data.ix[-1:])
In [329]:
# reading is once a minute, so take last 24 * 60 readings
def plotem(data, n=-60, start=None):
if n < 0:
start = n
end = len(data)
else:
start = start or 0
end = start + n
data[['temp', 'altitude', 'humidity', 'dewpoint']][start:end].plot(subplots=True)
plotem(data, -24*60)
This plot is interesting, the spike just after 10am (BDA - 3 hours) in temperature, just as there is a drop in pressure.
This coincided with the start of heavy rain.
The noise in the humidity coincides with the rain too. The increase in humidity around 2pm looks to be at least part (mostly?) due to the temperature dropping. The dewpoint remained steadier, this is a measure of the amount of moisture in the air.
Now the good news is the pressure looks to be rising a bit.
In [330]:
plotem(data, 24 * 60, 8440)
In [331]:
# reading is once a minute, so take last 7 * 24 * 60 readings
plotem(data, -7*24*60)
In [332]:
plotem(data)
In [333]:
plotem(data, -5)
In [334]:
data.describe()
Out[334]:
I currently have two temperature sensors:
The plot below shows the two temperature plots.
Both these sensors are currently in my study. For temperature and humidity I would like to have some readings from outside. If I can solder them to a phone jack then I can just run phone cable to where they need to be.
Below plots the current values from these sensors. This is handy for calibration.
In [335]:
data[['temp', 'temp_dht']].plot()
Out[335]:
In [336]:
data[['temp', 'dewpoint', 'humidity']].plot()
Out[336]:
In [337]:
data[['temp', 'dewpoint', 'humidity']].plot(subplots=True)
Out[337]:
In [338]:
data[['temp', 'dewpoint']].plot()
Out[338]:
In [339]:
from numpy import fft
In [340]:
data['fft_altitude'] = fft.fft(data.altitude)
data['fft_alt_real'] = data.fft_altitude.real
data['fft_alt_imag'] = data.fft_altitude.imag
In [341]:
data['alt_power'] = ((data.fft_alt_real ** 2.0) + (data.fft_alt_imag ** 2.0)) ** 0.5
In [342]:
import numpy
N = len(data)
TWELVE = N//(12*60)
print(N/(12*60.), TWELVE)
data['pos'] = numpy.arange(N)
data[['fft_alt_imag', 'fft_alt_real', 'alt_power']][TWELVE//3:2*TWELVE].plot()
Out[342]:
In [343]:
data[TWELVE//3:2*TWELVE].plot(x='pos', y='alt_power')
Out[343]:
In [344]:
xx = data.fft_altitude.copy()
xx[TWELVE:-TWELVE] = 0.0
data['smooth_alt'] = [x.real for x in fft.ifft(xx)]
In [345]:
data[['smooth_alt', 'altitude']].plot()
Out[345]:
In [346]:
data.smooth_alt.plot()
Out[346]:
In [347]:
def power(x, y):
return ((x ** 2.0) + (y ** 2.0)) ** 0.5
In [348]:
P = TWELVE
xx = data.fft_altitude.copy()
xx[:] = 0.0
xx[P:-P] = data.fft_altitude[P:-P]
data['twelve'] = [x.real for x in fft.ifft(xx)]
data['itwelve'] = [x.imag for x in fft.ifft(xx)]
data['twelve_pow'] = power(data.twelve, data.itwelve)
data['smoothed'] = data.altitude - data.twelve
data[['twelve', 'altitude', 'smoothed', 'smooth_alt']].plot(subplots=True)
Out[348]: