Reproducible LMA research with the IPython notebook and brawl4d

This notebook demonstrates how to display data included in the lmatools repository.

If you haven't yet, process and view sample data included with lmatools

python ~/code/lmatools/testing/test_sklearn.py /path/to/output/files/

Then, edit the data_path in the second cell to include /path/to/output/files/ as defined above. Run all the cells prior to the "Charge Analysis" and try interacting with the plot. You should see some data.


In [1]:
import matplotlib
%matplotlib qt4
# import matplotlib
# matplotlib.use('nbagg')
#import matplotlib.pyplot as plt
from brawl4d.brawl4d import B4D_startup, redraw
import os

In [2]:
data_path = '/data/DC3/flash_sort/'
lma_file = os.path.join(data_path, 'h5_files/2012/Jun/02/LYLOUT_120602_211001_0600.dat.flash.h5')

In the cell below, note that the basedate has been set to match the dataset we specified above.

If you are not using data from the WTLMA, then you'll also need to pass ctr_lon=value and ctr_lat=value to B4D_startup.


In [3]:
from datetime import datetime
panels = B4D_startup(basedate=datetime(2012,6,2), ctr_lat=40.4463980, ctr_lon=-104.6368130)


/Users/ebruning/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py:1279: UserWarning: findfont: Font family [u'Helvetica'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))

In [4]:
import matplotlib.pyplot as plt; plt.show()

Below, set a valid path to lma_file. IPython will tab-complete paths, like the shell.


In [5]:
from brawl4d.LMA.controller import LMAController
lma_ctrl = LMAController()
d, post_filter_brancher, scatter_ctrl, charge_lasso = lma_ctrl.load_hdf5_to_panels(panels, lma_file)


found flash data

Zoom to overview of first minute


In [7]:
h,m,s,dt = 21, 10, 0, 60
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((-150, 150, -150, 150))


Out[7]:
(-150, 150, -150, 150)

In [6]:
from brawl4d.LMA.widgets import LMAwidgetController
from IPython.display import display
from brawl4d.LMA.controller import LMAController

lma_tools = LMAwidgetController(panels, lma_ctrl, scatter_ctrl, charge_lasso, d)
display(lma_tools.tools_popup)

The cell above contains and combines all functions of Brawl4d into a single .py file. This allows all tools to be activiated simultaneously, while also enabling a centralized container for all active widgets for convienence.

LMA Tools Contained:

Number of Stations: Specify accordingly with data file (min=1; max=11)

Max Chi2: Values for chi2 obtained from the data file (min=0.0; max=1.0)

Charge Selection: Selection for Negative (-1), Neutral (0), and Positive (1) charge for charge selection and analyzation in the browser. The draw button activates the lasso tool enabling charge selection; re-clicking the draw button is necessary upon each selection made.

Color By: Allows the display of LMA data by chi2, time, or charge; selecting one will redraw the plot in the browser.

Animation Time: Allows for animation of the LMA data in the browser for charge polarity determination made by Charge Selection. The slider allows the user to select a desired time for total animation duration (min=1s, max=30s). Clicking Animate will then animate the data after a desired time has been selected.

Hereford storm - first 5 min


In [7]:
h,m,s,dt = 21, 15, 0, 60*5
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((25, 45, 45, 65))


Out[7]:
(25, 45, 45, 65)

First few flashes


In [8]:
h,m,s,dt = 21, 15, 50, 50
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((29, 36, 49, 56))


Out[8]:
(29, 36, 49, 56)

Most flashes are small clusters with no discernible positive leaders; what's observed is most likely negative leader. Flash at 2116:17.7 s shows negative leaders at 7.8 km, and downward propagation at the end of the flash to 6.5 km. Other flashes are small <1 km$^2$ balls of sources, but which show some propagation on occasion. Most of the small clusters have altitudes of about 8.8 km, excep for the flash after 2116:30, which matches the altitude of the flash at 2116:17.7, but with a slight westward offset.

First two minutes - two flashes with easily discernible charge structure.


In [9]:
h,m,s,dt = 21, 15, 50, 110
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((27, 36, 48, 57))


Out[9]:
(27, 36, 48, 57)

See some evidence of a several distinct regions of flashing.

  • All small flashes at x=34 km, at the hightest altitude.
  • Another region to west, x=30 to 33 km, medium and small flashes.
  • Region west of x=30 km, largest flashes

Zoom in on the activity at 2117:00, which is actually two flashes. Finally some good evidence of -ICs. Eastern flash is in the middle region.

Next minute


In [23]:
h,m,s,dt = 21, 17, 41, 60
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((27, 36, 48, 57))


Out[23]:
(27, 36, 48, 57)

Elevated eastern flashes, lower western flashes.

  • Elevated flashes a bit larger,
  • Western flashes still have two levels of negative leaders.

Two flashes in the eastern cluster show a tilted, -/+/- charge structure, sloping down and to southwest

  • 2117:53 is a -IC, with descending negative retrograde breakdown
  • 2117:23 is a +IC, with ascending negative retrograde breakdown

Summary of first three minutes


In [24]:
h,m,s,dt = 21, 15, 40, 60*3
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((27, 36, 48, 57))


Out[24]:
(27, 36, 48, 57)

Overall, the initial charge structure is complicated, even in this small cell which is less than 10 km across. But there are distinct regions with elevated flashes tending to be small, and larger flashes down lower. This is consistent with the idea that updrafts have small flashes and precipitating regions tend toward larger, organized charge regions.

This is common for storms with a strong updraft, in contrast to mountainous or weak multicellular ("airmass") storms as are common in the subtropics.

Adding the 80 seconds in this file


In [27]:
h,m,s,dt = 21, 15, 40, 180+60+20
panels.panels['tz'].axis((h*3600 + m*60 + s, h*3600 + m*60 + s+dt, 1, 18))
panels.panels['xy'].axis((27, 39, 48, 60))


Out[27]:
(27, 39, 48, 60)

Flashes in the eastern half of the storm remain elvated, but tend a bit larger toward the latter half of this file.

In the next notebook, we will jump ahead to 2121-2123, and see this pattern consolidate.

Flash statistics

If the LMA controller found flash data, then it's possible to get a live update of flashes in the current view. current_events_flashes is an analysis pipeline branchpoint, which will send events and flashes to another analysis pipeline segment that can be specified with current_events_flashes.targets.add(target). Behind the scenes, it's hooked up to an segment that receives the events and flashes, and prints the average flash area of all flashes that have more than a threshold number of points.

Change the view a few times and you'll see updated flash stats below.


In [8]:
current_events_flashes = lma_ctrl.flash_stats_for_dataset(d, scatter_ctrl.branchpoint)

In [8]:


In [13]:
@coroutine
def events_flashes_printer():
    while True:
        evs, fls = (yield)
        print evs['lon'], evs['lat']
info_printer = events_flashes_printer()
current_events_flashes.targets.add(info_printer)

In [24]:
panels.bounds.limits()


2 of 4 flashes have > 10 points. Their average area =   2.7 km^2
Out[24]:
[('x', (32.791941358574263, 38.72589515320481)),
 ('y', (51.260562726344133, 57.157117652521123)),
 ('z', (0.45241156355291423, 13.523410086875302)),
 ('time', (77249.540000192297, 77251.49070000503))]

In [ ]: