This notebook shows how BigBang can help you explore a mailing list archive.
First, use this IPython magic to tell the notebook to display matplotlib graphics inline. This is a nice way to display results.
In [1]:
%matplotlib inline
Import the BigBang modules as needed. These should be in your Python environment if you've installed BigBang correctly.
In [2]:
import bigbang.mailman as mailman
import bigbang.graph as graph
import bigbang.process as process
from bigbang.parse import get_date
#from bigbang.functions import *
from bigbang.archive import Archive
Also, let's import a number of other dependencies we'll use later.
In [3]:
import pandas as pd
import datetime
import matplotlib.pyplot as plt
import numpy as np
import math
import pytz
import pickle
import os
pd.options.display.mpl_style = 'default' # pandas has a set of preferred graph formatting options
Now let's load the data for analysis.
In [4]:
urls = ["ipython-dev",
"ipython-user"]
archives = [Archive(url,archive_dir="../archives",mbox=True) for url in urls]
activities = [arx.get_activity(resolved=False) for arx in archives]
/home/sb/projects/bigbang/bigbang/mailman.py:105: UserWarning: No mailing list name found at ipython-dev
warnings.warn("No mailing list name found at %s" % url)
/home/sb/projects/bigbang/bigbang/mailman.py:105: UserWarning: No mailing list name found at ipython-user
warnings.warn("No mailing list name found at %s" % url)
Opening 144 archive files
Opening 144 archive files
In [5]:
archives[0].data
Out[5]:
From
Subject
Date
In-Reply-To
References
Body
Message-ID
<3E9DE124.8080309@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] Mailing lists indexed at gmane
2003-04-16 23:03:00
None
None
Hi all,\n\nafter a suggestion by Jacek Generow...
<3E9DE124.8080309@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] Mailing lists indexed at gmane
2003-04-16 23:03:00
None
None
Hi all,\n\nafter a suggestion by Jacek Generow...
<3E9E4094.7030802@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] Re: Refactoring of bdist_wininst...
2003-04-17 05:50:12
<003d01c28a9a$3dcb8560$e301340a@cyberhigh.fcoe...
<003d01c28a9a$3dcb8560$e301340a@cyberhigh.fcoe...
Hi Cory,\n\n> Done. install command will now ...
<3E9E4094.7030802@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] Re: Refactoring of bdist_wininst...
2003-04-17 05:50:12
<003d01c28a9a$3dcb8560$e301340a@cyberhigh.fcoe...
<003d01c28a9a$3dcb8560$e301340a@cyberhigh.fcoe...
Hi Cory,\n\n> Done. install command will now ...
<000c01c304ee$3cb79e60$e901340a@cyberhigh.fcoe.k12.ca.us>
cdodt@fcoe.k12.ca.us (Cory Dodt)
[IPython-dev] RE: Refactoring of bdist_wininst...
2003-04-17 14:32:56
<3E9E4094.7030802@colorado.edu>
None
Distutils 1.0.3 is not included with Python 2....
<000c01c304ee$3cb79e60$e901340a@cyberhigh.fcoe.k12.ca.us>
cdodt at fcoe.k12.ca.us (Cory Dodt)
[IPython-dev] RE: Refactoring of bdist_wininst...
2003-04-17 14:32:56
<3E9E4094.7030802@colorado.edu>
None
Distutils 1.0.3 is not included with Python 2....
<3E9EC1CA.3060800@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] RE: Refactoring of bdist_wininst...
2003-04-17 15:01:30
<000c01c304ee$3cb79e60$e901340a@cyberhigh.fcoe...
<000c01c304ee$3cb79e60$e901340a@cyberhigh.fcoe...
Cory Dodt wrote:\n> Distutils 1.0.3 is not inc...
<3E9EC1CA.3060800@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] RE: Refactoring of bdist_wininst...
2003-04-17 15:01:30
<000c01c304ee$3cb79e60$e901340a@cyberhigh.fcoe...
<000c01c304ee$3cb79e60$e901340a@cyberhigh.fcoe...
Cory Dodt wrote:\n> Distutils 1.0.3 is not inc...
<3E9EF5E3.8080100@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] [Fwd: [ANN] A new IPython is out...
2003-04-17 18:43:47
None
None
Hi all,\n\nI've just put out a new pre-release...
<3E9EF5E3.8080100@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] [Fwd: [ANN] A new IPython is out...
2003-04-17 18:43:47
None
None
Hi all,\n\nI've just put out a new pre-release...
<3E9EFC95.7040309@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] ToDo for 0.4.0
2003-04-17 19:12:21
None
None
Hi all,\n\nI'd like to put out a list of thing...
<3E9EFC95.7040309@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] ToDo for 0.4.0
2003-04-17 19:12:21
None
None
Hi all,\n\nI'd like to put out a list of thing...
<3E9F3B79.7070005@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] New bug tracker for IPython
2003-04-17 23:40:41
None
None
Hi all,\n\nI just wanted to let you know that,...
<3E9F3B79.7070005@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] New bug tracker for IPython
2003-04-17 23:40:41
None
None
Hi all,\n\nI just wanted to let you know that,...
<3E9F3D9B.8040807@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] Re: iPython on Windows
2003-04-17 23:49:47
<GCEDKONBLEFPPADDJCOECEOIIPAA.whisper@oz.net>
<GCEDKONBLEFPPADDJCOECEOIIPAA.whisper@oz.net>
Hi David,\n\nmy apologies for the long delay i...
<3E9F3D9B.8040807@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] Re: iPython on Windows
2003-04-17 23:49:47
<GCEDKONBLEFPPADDJCOECEOIIPAA.whisper@oz.net>
<GCEDKONBLEFPPADDJCOECEOIIPAA.whisper@oz.net>
Hi David,\n\nmy apologies for the long delay i...
<200304291817.05898.Kasper.Souren@ircam.fr>
Kasper.Souren at ircam.fr (Kasper Souren)
[IPython-dev] possible feature request: auto-run
2003-04-29 18:17:05
None
None
Hi!\n\nI just had a little idea for a new IPyt...
<200304291817.05898.Kasper.Souren@ircam.fr>
Kasper.Souren@ircam.fr (Kasper Souren)
[IPython-dev] possible feature request: auto-run
2003-04-29 18:17:05
None
None
Hi!\n\nI just had a little idea for a new IPyt...
<3EAEF194.5030709@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] possible feature request: auto-run
2003-04-29 21:41:40
<200304291817.05898.Kasper.Souren@ircam.fr>
<200304291817.05898.Kasper.Souren@ircam.fr>
Kasper Souren wrote:\n> Hi!\n> \n> I just had ...
<3EAEF194.5030709@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] possible feature request: auto-run
2003-04-29 21:41:40
<200304291817.05898.Kasper.Souren@ircam.fr>
<200304291817.05898.Kasper.Souren@ircam.fr>
Kasper Souren wrote:\n> Hi!\n> \n> I just had ...
<200304292248.10994.Kasper.Souren@ircam.fr>
Kasper.Souren@ircam.fr (Kasper Souren)
[IPython-dev] possible feature request: auto-run
2003-04-29 22:48:10
<3EAEF194.5030709@colorado.edu>
<200304291817.05898.Kasper.Souren@ircam.fr> <3...
> It's rather complicated to get it right, and...
<200304292248.10994.Kasper.Souren@ircam.fr>
Kasper.Souren at ircam.fr (Kasper Souren)
[IPython-dev] possible feature request: auto-run
2003-04-29 22:48:10
<3EAEF194.5030709@colorado.edu>
<200304291817.05898.Kasper.Souren@ircam.fr> <3...
> It's rather complicated to get it right, and...
<CB0365D517B7D611B5E100508B9498B6022A9B50@erlh904a.med.siemens.de>
christopher.drexler@siemens.com (Drexler Chris...
[IPython-dev] RE: [Fwd: [IPython-user] re: Fwd...
2003-05-12 07:28:55
None
None
Dear List,\n\nI'm working with IPython since a...
<CB0365D517B7D611B5E100508B9498B6022A9B50@erlh904a.med.siemens.de>
christopher.drexler at siemens.com (Drexler Ch...
[IPython-dev] RE: [Fwd: [IPython-user] re: Fwd...
2003-05-12 07:28:55
None
None
Dear List,\n\nI'm working with IPython since a...
<200305121234.h4CCYmXo027167@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] RE: [Fwd: [IPython-user] re: Fwd...
2003-05-12 08:34:48
None
None
Thanks Chris,\n\nWith that hint and some googl...
<200305121234.h4CCYmXo027167@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] RE: [Fwd: [IPython-user] re: Fwd...
2003-05-12 08:34:48
None
None
Thanks Chris,\n\nWith that hint and some googl...
<3EC143D7.8050907@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] Re: IPython Crash Report
2003-05-13 19:13:27
<200305131849.h4DInjXo018909@wren.cs.unc.edu>
<200305131849.h4DInjXo018909@wren.cs.unc.edu>
Hi Gary,\n\n> The idea is simple. I assume tha...
<3EC143D7.8050907@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] Re: IPython Crash Report
2003-05-13 19:13:27
<200305131849.h4DInjXo018909@wren.cs.unc.edu>
<200305131849.h4DInjXo018909@wren.cs.unc.edu>
Hi Gary,\n\n> The idea is simple. I assume tha...
<200305171149.h4HBneXo024735@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] re: 0.4.0 ready for Monday
2003-05-17 07:49:39
None
None
It still says it is 0.2.15.pre5, I guess that ...
<200305171149.h4HBneXo024735@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] re: 0.4.0 ready for Monday
2003-05-17 07:49:39
None
None
It still says it is 0.2.15.pre5, I guess that ...
<3ECAA865.9090109@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] [ANN] IPython 0.4.0 has been rel...
2003-05-20 22:12:53
None
None
Hi all,\n\nI've just released IPython 0.4.0. ...
<3ECAA865.9090109@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] [ANN] IPython 0.4.0 has been rel...
2003-05-20 22:12:53
None
None
Hi all,\n\nI've just released IPython 0.4.0. ...
<200305221924.h4MJOEXo018537@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] Small change to MagicCompleter f...
2003-05-22 15:24:14
None
None
Now that I've got my Python readline starting ...
<200305221924.h4MJOEXo018537@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] Small change to MagicCompleter f...
2003-05-22 15:24:14
None
None
Now that I've got my Python readline starting ...
<200305221927.h4MJR0Xo018707@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] Small change to MagicCompleter f...
2003-05-22 15:27:00
None
None
Ignore that previous patch. That code should g...
<200305221927.h4MJR0Xo018707@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] Small change to MagicCompleter f...
2003-05-22 15:27:00
None
None
Ignore that previous patch. That code should g...
<200305221955.h4MJteXo020281@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] A patch to fix filename completi...
2003-05-22 15:55:40
None
None
OK, sorry for the first one. Here is another t...
<200305221955.h4MJteXo020281@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] A patch to fix filename completi...
2003-05-22 15:55:40
None
None
OK, sorry for the first one. Here is another t...
<3ECD3E3F.8010205@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] A patch to fix filename completi...
2003-05-22 21:16:47
<200305221955.h4MJteXo020281@wren.cs.unc.edu>
<200305221955.h4MJteXo020281@wren.cs.unc.edu>
Gary Bishop wrote:\n> OK, sorry for the first ...
<3ECD3E3F.8010205@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] A patch to fix filename completi...
2003-05-22 21:16:47
<200305221955.h4MJteXo020281@wren.cs.unc.edu>
<200305221955.h4MJteXo020281@wren.cs.unc.edu>
Gary Bishop wrote:\n> OK, sorry for the first ...
<200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] IPython bug?
2003-05-23 16:45:08
None
None
With readline mark-directories set to "on" (th...
<200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] IPython bug?
2003-05-23 16:45:08
None
None
With readline mark-directories set to "on" (th...
<200305232058.h4NKwHXo019746@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] IPython with Color on windows
2003-05-23 16:58:17
None
None
I *can* make color work on Windows with my Pyt...
<200305232058.h4NKwHXo019746@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] IPython with Color on windows
2003-05-23 16:58:17
None
None
I *can* make color work on Windows with my Pyt...
<200305232102.h4NL2LXo019939@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] What readline features do people...
2003-05-23 17:02:21
None
None
I've got a Python implementation of GNU readli...
<200305232102.h4NL2LXo019939@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] What readline features do people...
2003-05-23 17:02:21
None
None
I've got a Python implementation of GNU readli...
<16078.27403.669231.313029@monster.linux.in>
prabhu at aero.iitm.ernet.in (Prabhu Ramachand...
[IPython-dev] Making docs and installing from ...
2003-05-23 18:40:11
None
None
Hi,\n\nI just got IPython off CVS and when ins...
<16078.27403.669231.313029@monster.linux.in>
prabhu@aero.iitm.ernet.in (Prabhu Ramachandran)
[IPython-dev] Making docs and installing from ...
2003-05-23 18:40:11
None
None
Hi,\n\nI just got IPython off CVS and when ins...
<3ECE998E.4030107@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] Making docs and installing from ...
2003-05-23 21:58:38
<16078.27403.669231.313029@monster.linux.in>
<16078.27403.669231.313029@monster.linux.in>
Prabhu Ramachandran wrote:\n> Hi,\n> \n> I jus...
<3ECE998E.4030107@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] Making docs and installing from ...
2003-05-23 21:58:38
<16078.27403.669231.313029@monster.linux.in>
<16078.27403.669231.313029@monster.linux.in>
Prabhu Ramachandran wrote:\n> Hi,\n> \n> I jus...
<200305240204.h4O24nXo029923@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] IPython bug?
2003-05-23 22:04:49
None
None
On Fri, 23 May 2003 20:01:54 -0600 "Fernando P...
<200305240204.h4O24nXo029923@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] IPython bug?
2003-05-23 22:04:49
None
None
On Fri, 23 May 2003 20:01:54 -0600 "Fernando P...
<200305240229.h4O2TqXo000725@wren.cs.unc.edu>
gb at cs.unc.edu (Gary Bishop)
[IPython-dev] IPython bug?
2003-05-23 22:29:52
None
None
On Fri, 23 May 2003 20:12:18 -0600 "Fernando P...
<200305240229.h4O2TqXo000725@wren.cs.unc.edu>
gb@cs.unc.edu (Gary Bishop)
[IPython-dev] IPython bug?
2003-05-23 22:29:52
None
None
On Fri, 23 May 2003 20:12:18 -0600 "Fernando P...
<3ECED168.9000603@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] IPython with Color on windows
2003-05-24 01:56:56
<200305232058.h4NKwHXo019746@wren.cs.unc.edu>
<200305232058.h4NKwHXo019746@wren.cs.unc.edu>
Gary Bishop wrote:\n> I *can* make color work ...
<3ECED168.9000603@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] IPython with Color on windows
2003-05-24 01:56:56
<200305232058.h4NKwHXo019746@wren.cs.unc.edu>
<200305232058.h4NKwHXo019746@wren.cs.unc.edu>
Gary Bishop wrote:\n> I *can* make color work ...
<3ECED1CD.7020109@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] What readline features do people...
2003-05-24 01:58:37
<200305232102.h4NL2LXo019939@wren.cs.unc.edu>
<200305232102.h4NL2LXo019939@wren.cs.unc.edu>
Gary Bishop wrote:\n> I've got a Python implem...
<3ECED1CD.7020109@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] What readline features do people...
2003-05-24 01:58:37
<200305232102.h4NL2LXo019939@wren.cs.unc.edu>
<200305232102.h4NL2LXo019939@wren.cs.unc.edu>
Gary Bishop wrote:\n> I've got a Python implem...
<3ECED292.1050307@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-dev] IPython bug?
2003-05-24 02:01:54
<200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
<200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
Gary Bishop wrote:\n> With readline mark-direc...
<3ECED292.1050307@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-dev] IPython bug?
2003-05-24 02:01:54
<200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
<200305232045.h4NKj9Xo019149@wren.cs.unc.edu>
Gary Bishop wrote:\n> With readline mark-direc...
...
...
...
...
...
...
15485 rows × 6 columns
This variable is for the range of days used in computing rolling averages.
In [6]:
window = 100
For each of the mailing lists we are looking at, plot the rolling average of number of emails sent per day.
In [7]:
plt.figure(figsize=(12.5, 7.5))
for i, activity in enumerate(activities):
colors = 'rgbkm'
ta = activity.sum(1)
rmta = pd.rolling_mean(ta,window)
rmtadna = rmta.dropna()
plt.plot_date(rmtadna.index,
rmtadna.values,
colors[i],
label=mailman.get_list_name(urls[i]) + ' activity',xdate=True)
plt.legend()
plt.savefig("activites-marked.png")
plt.show()
/home/sb/anaconda/envs/bigbang/lib/python2.7/site-packages/matplotlib/font_manager.py:1236: UserWarning: findfont: Font family ['monospace'] not found. Falling back to Bitstream Vera Sans
(prop.get_family(), self.defaultFamily[fontext]))
In [8]:
arx.data
Out[8]:
From
Subject
Date
In-Reply-To
References
Body
Message-ID
<3271DBB88437ED41A0AB239E6C2554A401117873@ussunm001.palmsource.com>
Robin.Siebler at palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 20:27:08
None
None
I installed IPython-0.2.15pre3, played with it...
<3271DBB88437ED41A0AB239E6C2554A401117873@ussunm001.palmsource.com>
Robin.Siebler@palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 20:27:08
None
None
I installed IPython-0.2.15pre3, played with it...
<3E8364F0.2000107@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 20:54:08
<3271DBB88437ED41A0AB239E6C2554A401117873@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117873@ussu...
Robin Siebler wrote:\n> I installed IPython-0....
<3E8364F0.2000107@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 20:54:08
<3271DBB88437ED41A0AB239E6C2554A401117873@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117873@ussu...
Robin Siebler wrote:\n> I installed IPython-0....
<1048798697.25990.6.camel@localhost.localdomain>
jives at gorge.net (Jason Ives)
[IPython-user] IPython under Jython?
2003-03-27 20:58:15
None
None
Hi,\n\nI'm wondering if anyone's had success r...
<1048798697.25990.6.camel@localhost.localdomain>
jives@gorge.net (Jason Ives)
[IPython-user] IPython under Jython?
2003-03-27 20:58:15
None
None
Hi,\n\nI'm wondering if anyone's had success r...
<3E836641.8000008@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] IPython under Jython?
2003-03-27 20:59:45
<1048798697.25990.6.camel@localhost.localdomain>
<1048798697.25990.6.camel@localhost.localdomain>
Jason Ives wrote:\n\n> I'm wondering if anyone...
<3E836641.8000008@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] IPython under Jython?
2003-03-27 20:59:45
<1048798697.25990.6.camel@localhost.localdomain>
<1048798697.25990.6.camel@localhost.localdomain>
Jason Ives wrote:\n\n> I'm wondering if anyone...
<3271DBB88437ED41A0AB239E6C2554A401117875@ussunm001.palmsource.com>
Robin.Siebler@palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 21:13:13
None
None
I searched but couldn't find any such file.\n\...
<3271DBB88437ED41A0AB239E6C2554A401117875@ussunm001.palmsource.com>
Robin.Siebler at palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 21:13:13
None
None
I searched but couldn't find any such file.\n\...
<3E836B5E.4020702@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 21:21:34
<3271DBB88437ED41A0AB239E6C2554A401117875@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117875@ussu...
Robin Siebler wrote:\n> I searched but couldn'...
<3E836B5E.4020702@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 21:21:34
<3271DBB88437ED41A0AB239E6C2554A401117875@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117875@ussu...
Robin Siebler wrote:\n> I searched but couldn'...
<1048802417.25990.16.camel@localhost.localdomain>
jives at gorge.net (Jason Ives)
[IPython-user] IPython under Jython?
2003-03-27 22:00:14
<3E836641.8000008@colorado.edu>
<1048798697.25990.6.camel@localhost.localdomai...
Hi,\n\nFernando Perez wrote:\n\n Insofar as...
<1048802417.25990.16.camel@localhost.localdomain>
jives@gorge.net (Jason Ives)
[IPython-user] IPython under Jython?
2003-03-27 22:00:14
<3E836641.8000008@colorado.edu>
<1048798697.25990.6.camel@localhost.localdomai...
Hi,\n\nFernando Perez wrote:\n\n Insofar as...
<3271DBB88437ED41A0AB239E6C2554A401117878@ussunm001.palmsource.com>
Robin.Siebler@palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 22:01:57
None
None
I just searched for 'ipyt*'. I didn't get a hi...
<3271DBB88437ED41A0AB239E6C2554A401117878@ussunm001.palmsource.com>
Robin.Siebler at palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 22:01:57
None
None
I just searched for 'ipyt*'. I didn't get a hi...
<3E8376BF.5060203@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 22:10:07
<3271DBB88437ED41A0AB239E6C2554A401117878@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117878@ussu...
Robin Siebler wrote:\n> I just searched for 'i...
<3E8376BF.5060203@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 22:10:07
<3271DBB88437ED41A0AB239E6C2554A401117878@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117878@ussu...
Robin Siebler wrote:\n> I just searched for 'i...
<3E837EE7.5050302@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 22:44:55
<3271DBB88437ED41A0AB239E6C2554A43F5C1F@ussunm...
<3271DBB88437ED41A0AB239E6C2554A43F5C1F@ussunm...
Ah! I see now, the problem is with the curses...
<3E837EE7.5050302@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 22:44:55
<3271DBB88437ED41A0AB239E6C2554A43F5C1F@ussunm...
<3271DBB88437ED41A0AB239E6C2554A43F5C1F@ussunm...
Ah! I see now, the problem is with the curses...
<3271DBB88437ED41A0AB239E6C2554A40111787D@ussunm001.palmsource.com>
Robin.Siebler at palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 22:49:25
None
None
There might be one, but I don't have it instal...
<3271DBB88437ED41A0AB239E6C2554A40111787D@ussunm001.palmsource.com>
Robin.Siebler@palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 22:49:25
None
None
There might be one, but I don't have it instal...
<3E83832A.4020506@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 23:03:06
<3271DBB88437ED41A0AB239E6C2554A40111787D@ussu...
<3271DBB88437ED41A0AB239E6C2554A40111787D@ussu...
Ok, there's something seriously strange here g...
<3E83832A.4020506@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 23:03:06
<3271DBB88437ED41A0AB239E6C2554A40111787D@ussu...
<3271DBB88437ED41A0AB239E6C2554A40111787D@ussu...
Ok, there's something seriously strange here g...
<3271DBB88437ED41A0AB239E6C2554A401117882@ussunm001.palmsource.com>
Robin.Siebler at palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 23:14:45
None
None
>My python2.2 installation doesn't even have t...
<3271DBB88437ED41A0AB239E6C2554A401117882@ussunm001.palmsource.com>
Robin.Siebler@palmsource.com (Robin Siebler)
[IPython-user] Crash
2003-03-27 23:14:45
None
None
>My python2.2 installation doesn't even have t...
<3E838774.9020006@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 23:21:24
<3271DBB88437ED41A0AB239E6C2554A401117882@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117882@ussu...
Robin Siebler wrote:\n>>My python2.2 installat...
<3E838774.9020006@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-03-27 23:21:24
<3271DBB88437ED41A0AB239E6C2554A401117882@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117882@ussu...
Robin Siebler wrote:\n>>My python2.2 installat...
<20030329055915.GL21370@i.cantcode.com>
jack@xiph.org (Jack Moffitt)
[IPython-user] ipython -p numeric problem
2003-03-29 05:59:15
None
None
I'm sure I'm missing something, but it's not o...
<20030329055915.GL21370@i.cantcode.com>
jack at xiph.org (Jack Moffitt)
[IPython-user] ipython -p numeric problem
2003-03-29 05:59:15
None
None
I'm sure I'm missing something, but it's not o...
<20030329061136.GM21370@i.cantcode.com>
jack@xiph.org (Jack Moffitt)
[IPython-user] ipython -p numeric problem
2003-03-29 06:11:36
<20030329055915.GL21370@i.cantcode.com>
<20030329055915.GL21370@i.cantcode.com>
> ImportError: No module named kinds\n> WARNIN...
<20030329061136.GM21370@i.cantcode.com>
jack at xiph.org (Jack Moffitt)
[IPython-user] ipython -p numeric problem
2003-03-29 06:11:36
<20030329055915.GL21370@i.cantcode.com>
<20030329055915.GL21370@i.cantcode.com>
> ImportError: No module named kinds\n> WARNIN...
<3E85F4C6.7000803@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] ipython -p numeric problem
2003-03-29 19:32:22
None
None
Quoting Jack Moffitt <jack@xiph.org>:\n\n>> Im...
<3E85F4C6.7000803@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] ipython -p numeric problem
2003-03-29 19:32:22
None
None
Quoting Jack Moffitt <jack at xiph.org>:\n\n>>...
<20030330001652.GO21370@i.cantcode.com>
jack@xiph.org (Jack Moffitt)
[IPython-user] ipython -p numeric problem
2003-03-30 00:16:52
<3E85F4C6.7000803@colorado.edu>
<3E85F4C6.7000803@colorado.edu>
> It is caused by the fact that I'm trying to ...
<20030330001652.GO21370@i.cantcode.com>
jack at xiph.org (Jack Moffitt)
[IPython-user] ipython -p numeric problem
2003-03-30 00:16:52
<3E85F4C6.7000803@colorado.edu>
<3E85F4C6.7000803@colorado.edu>
> It is caused by the fact that I'm trying to ...
<1049000506.3e867a3ac807c@webmail.colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] ipython -p numeric problem
2003-03-30 05:01:46
<20030330001652.GO21370@i.cantcode.com>
<3E85F4C6.7000803@colorado.edu> <2003033000165...
Quoting Jack Moffitt <jack at xiph.org>:\n\n> ...
<1049000506.3e867a3ac807c@webmail.colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] ipython -p numeric problem
2003-03-30 05:01:46
<20030330001652.GO21370@i.cantcode.com>
<3E85F4C6.7000803@colorado.edu> <2003033000165...
Quoting Jack Moffitt <jack@xiph.org>:\n\n> > I...
<200303311503.38505.Kasper.Souren@ircam.fr>
Kasper.Souren@ircam.fr (Kasper Souren)
[IPython-user] readline problem
2003-03-31 15:03:38
None
None
Hi!\n\nI have experienced this many times, but...
<200303311503.38505.Kasper.Souren@ircam.fr>
Kasper.Souren at ircam.fr (Kasper Souren)
[IPython-user] readline problem
2003-03-31 15:03:38
None
None
Hi!\n\nI have experienced this many times, but...
<65A5DB3E-63CC-11D7-9FDE-00039311EA24@insightscientific.com>
devlists@insightscientific.com (Andrew Straw)
[IPython-user] ipython -p numeric problem
2003-03-31 22:59:16
<1049000506.3e867a3ac807c@webmail.colorado.edu>
None
I'd better speak up as I'm one of the supposed...
<65A5DB3E-63CC-11D7-9FDE-00039311EA24@insightscientific.com>
devlists at insightscientific.com (Andrew Straw)
[IPython-user] ipython -p numeric problem
2003-03-31 22:59:16
<1049000506.3e867a3ac807c@webmail.colorado.edu>
None
I'd better speak up as I'm one of the supposed...
<3E89DB96.7070606@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] readline problem
2003-04-01 18:33:58
<200303311503.38505.Kasper.Souren@ircam.fr>
<200303311503.38505.Kasper.Souren@ircam.fr>
> I have experienced this many times, but neve...
<3E89DB96.7070606@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] readline problem
2003-04-01 18:33:58
<200303311503.38505.Kasper.Souren@ircam.fr>
<200303311503.38505.Kasper.Souren@ircam.fr>
> I have experienced this many times, but neve...
<3E8DFD44.5040603@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Re: modifying class instance.me...
2003-04-04 21:46:44
<3E8DE467.1080500@brainbot.com>
<3E8DE467.1080500@brainbot.com>
Hi heiko,\n\n> hi i am a great fan of ipython,...
<3E8DFD44.5040603@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Re: modifying class instance.me...
2003-04-04 21:46:44
<3E8DE467.1080500@brainbot.com>
<3E8DE467.1080500@brainbot.com>
Hi heiko,\n\n> hi i am a great fan of ipython,...
<3E92E809.1010509@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Re: Sorcerer spell
2003-04-08 15:17:29
<27DADDF47F50084597AF5C2C58B540CA69D549@MIECOE...
<27DADDF47F50084597AF5C2C58B540CA69D549@MIECOE...
Low Kian Seong (LOWKS) wrote:\n> Dear Frenando...
<3E92E809.1010509@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Re: Sorcerer spell
2003-04-08 15:17:29
<27DADDF47F50084597AF5C2C58B540CA69D549@MIECOE...
<27DADDF47F50084597AF5C2C58B540CA69D549@MIECOE...
Low Kian Seong (LOWKS) wrote:\n> Dear Frenando...
<3E945F1D.8080208@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-04-09 17:57:49
<3271DBB88437ED41A0AB239E6C2554A401117947@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117947@ussu...
Hi Robin,\n\nRobin Siebler wrote:\n> Did you e...
<3E945F1D.8080208@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] Crash
2003-04-09 17:57:49
<3271DBB88437ED41A0AB239E6C2554A401117947@ussu...
<3271DBB88437ED41A0AB239E6C2554A401117947@ussu...
Hi Robin,\n\nRobin Siebler wrote:\n> Did you e...
<Pine.LNX.4.44.0304110130370.18157-100000@mail.allcaps.org>
bsder@allcaps.org (Andrew P. Lentvorski, Jr.)
[IPython-user] ipython and pygtk?
2003-04-11 08:36:50
None
None
Has anybody attempted to get ipython working w...
<Pine.LNX.4.44.0304110130370.18157-100000@mail.allcaps.org>
bsder at allcaps.org (Andrew P. Lentvorski, Jr.)
[IPython-user] ipython and pygtk?
2003-04-11 08:36:50
None
None
Has anybody attempted to get ipython working w...
<3E96EA0B.4030905@colorado.edu>
fperez@colorado.edu (Fernando Perez)
[IPython-user] ipython and pygtk?
2003-04-11 16:15:07
<Pine.LNX.4.44.0304110130370.18157-100000@mail...
<Pine.LNX.4.44.0304110130370.18157-100000@mail...
Andrew P. Lentvorski, Jr. wrote:\n> Has anybod...
<3E96EA0B.4030905@colorado.edu>
fperez at colorado.edu (Fernando Perez)
[IPython-user] ipython and pygtk?
2003-04-11 16:15:07
<Pine.LNX.4.44.0304110130370.18157-100000@mail...
<Pine.LNX.4.44.0304110130370.18157-100000@mail...
Andrew P. Lentvorski, Jr. wrote:\n> Has anybod...
<1050079080.13210.96.camel@localhost.localdomain>
beach at verinet.com (David J. C. Beach)
[IPython-user] ipython and pygtk?
2003-04-11 16:37:58
<3E96EA0B.4030905@colorado.edu>
<Pine.LNX.4.44.0304110130370.18157-100000@mail...
On Fri, 2003-04-11 at 12:15, Fernando Perez wr...
<1050079080.13210.96.camel@localhost.localdomain>
beach@verinet.com (David J. C. Beach)
[IPython-user] ipython and pygtk?
2003-04-11 16:37:58
<3E96EA0B.4030905@colorado.edu>
<Pine.LNX.4.44.0304110130370.18157-100000@mail...
On Fri, 2003-04-11 at 12:15, Fernando Perez wr...
<Pine.LNX.4.44.0304111616260.21396-100000@mail.allcaps.org>
bsder at allcaps.org (Andrew P. Lentvorski, Jr.)
[IPython-user] ipython and pygtk?
2003-04-11 23:21:40
<1050079080.13210.96.camel@localhost.localdomain>
None
On 11 Apr 2003, David J. C. Beach wrote:\n\n> ...
<Pine.LNX.4.44.0304111616260.21396-100000@mail.allcaps.org>
bsder@allcaps.org (Andrew P. Lentvorski, Jr.)
[IPython-user] ipython and pygtk?
2003-04-11 23:21:40
<1050079080.13210.96.camel@localhost.localdomain>
None
On 11 Apr 2003, David J. C. Beach wrote:\n\n> ...
<5.1.0.14.0.20030413225152.03f8fdc0@smtp.sbcglobal.net>
tony@tcapp.com (Tony Cappellini)
[IPython-user] IPython - Runtime Errors on Win...
2003-04-14 05:54:52
None
None
Are there any Windows users here who have succ...
<5.1.0.14.0.20030413225152.03f8fdc0@smtp.sbcglobal.net>
tony at tcapp.com (Tony Cappellini)
[IPython-user] IPython - Runtime Errors on Win...
2003-04-14 05:54:52
None
None
Are there any Windows users here who have succ...
...
...
...
...
...
...
13149 rows × 6 columns
Now, let's see: who are the authors of the most messages to one particular list?
In [9]:
a = activities[0] # for the first mailing list
ta = a.sum(0) # sum along the first axis
ta.sort()
ta[-10:].plot(kind='barh')
Out[9]:
<matplotlib.axes.AxesSubplot at 0x7fbf425510d0>
This might be useful for seeing the distribution (does the top message sender dominate?) or for identifying key participants to talk to.
Many mailing lists will have some duplicate senders: individuals who use multiple email addresses or are recorded as different senders when using the same email address. We want to identify those potential duplicates in order to get a more accurate representation of the distribution of senders.
To begin with, let's do a naive calculation of the similarity of the From strings, based on the Levenshtein distance.
This can take a long time for a large matrix, so we will truncate it for purposes of demonstration.
In [10]:
import Levenshtein
distancedf = process.matricize(a.columns[:100], process.from_header_distance) # calculate the edit distance between the two From titles
df = distancedf.astype(int) # specify that the values in the matrix are integers
In [11]:
fig = plt.figure(figsize=(18, 18))
plt.imshow(df)
#plt.yticks(np.arange(0.5, len(df.index), 1), df.index) # these lines would show labels, but that gets messy
#plt.xticks(np.arange(0.5, len(df.columns), 1), df.columns)
Out[11]:
<matplotlib.image.AxesImage at 0x7fbf421cc0d0>