Run this from the _talks/ directory, which contains .md files of all your talks. This scrapes the location YAML field from each .md file, geolocates it with geopy/Nominatim, and uses the getorg library to output data, HTML, and Javascript for a standalone cluster map.
In [1]:
!pip install getorg --upgrade
!pip install ipyleaflet --upgrade
import glob
import getorg
from geopy import Nominatim
Requirement already up-to-date: getorg in /home/adriano/anaconda3/lib/python3.6/site-packages
Requirement already up-to-date: retrying in /home/adriano/anaconda3/lib/python3.6/site-packages (from getorg)
Requirement already up-to-date: geopy in /home/adriano/anaconda3/lib/python3.6/site-packages (from getorg)
Requirement already up-to-date: pygithub in /home/adriano/anaconda3/lib/python3.6/site-packages (from getorg)
Requirement already up-to-date: six>=1.7.0 in /home/adriano/anaconda3/lib/python3.6/site-packages (from retrying->getorg)
Requirement already up-to-date: pyjwt in /home/adriano/anaconda3/lib/python3.6/site-packages (from pygithub->getorg)
Requirement already up-to-date: ipyleaflet in /home/adriano/anaconda3/lib/python3.6/site-packages
Requirement already up-to-date: ipywidgets~=7.0.0 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipyleaflet)
Requirement already up-to-date: traitlets>=4.3.1 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: ipykernel>=4.5.1 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: widgetsnbextension~=3.0.0 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: ipython>=4.0.0; python_version >= "3.3" in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: nbformat>=4.2.0 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: ipython_genutils in /home/adriano/anaconda3/lib/python3.6/site-packages (from traitlets>=4.3.1->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: six in /home/adriano/anaconda3/lib/python3.6/site-packages (from traitlets>=4.3.1->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: decorator in /home/adriano/anaconda3/lib/python3.6/site-packages (from traitlets>=4.3.1->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: jupyter_client in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipykernel>=4.5.1->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: tornado>=4.0 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipykernel>=4.5.1->ipywidgets~=7.0.0->ipyleaflet)
Collecting notebook>=4.4.1 (from widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Downloading notebook-5.2.2-py2.py3-none-any.whl (8.0MB)
100% |████████████████████████████████| 8.0MB 175kB/s ta 0:00:01
Requirement already up-to-date: pygments in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: simplegeneric>0.8 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: jedi>=0.10 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: pexpect; sys_platform != "win32" in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Collecting setuptools>=18.5 (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Downloading setuptools-38.2.3-py2.py3-none-any.whl (489kB)
100% |████████████████████████████████| 491kB 1.2MB/s ta 0:00:01
Requirement already up-to-date: prompt-toolkit<2.0.0,>=1.0.4 in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: pickleshare in /home/adriano/anaconda3/lib/python3.6/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: jsonschema!=2.5.0,>=2.4 in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbformat>=4.2.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: jupyter_core in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbformat>=4.2.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: pyzmq>=13 in /home/adriano/anaconda3/lib/python3.6/site-packages (from jupyter_client->ipykernel>=4.5.1->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: python-dateutil>=2.1 in /home/adriano/anaconda3/lib/python3.6/site-packages (from jupyter_client->ipykernel>=4.5.1->ipywidgets~=7.0.0->ipyleaflet)
Collecting terminado>=0.3.3; sys_platform != "win32" (from notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Downloading terminado-0.8.1-py2.py3-none-any.whl
Requirement already up-to-date: nbconvert in /home/adriano/anaconda3/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: jinja2 in /home/adriano/anaconda3/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: parso==0.1.0 in /home/adriano/anaconda3/lib/python3.6/site-packages (from jedi>=0.10->ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: ptyprocess>=0.5 in /home/adriano/anaconda3/lib/python3.6/site-packages (from pexpect; sys_platform != "win32"->ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: wcwidth in /home/adriano/anaconda3/lib/python3.6/site-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython>=4.0.0; python_version >= "3.3"->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: mistune>=0.7.4 in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: entrypoints>=0.2.2 in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: bleach in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: pandocfilters>=1.4.1 in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: testpath in /home/adriano/anaconda3/lib/python3.6/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: MarkupSafe>=0.23 in /home/adriano/anaconda3/lib/python3.6/site-packages (from jinja2->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre in /home/adriano/anaconda3/lib/python3.6/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Requirement already up-to-date: webencodings in /home/adriano/anaconda3/lib/python3.6/site-packages (from html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre->bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.0.0->ipywidgets~=7.0.0->ipyleaflet)
Installing collected packages: terminado, notebook, setuptools
Found existing installation: terminado 0.7
Uninstalling terminado-0.7:
Successfully uninstalled terminado-0.7
Found existing installation: notebook 5.2.1
Uninstalling notebook-5.2.1:
Successfully uninstalled notebook-5.2.1
Found existing installation: setuptools 38.2.1
Uninstalling setuptools-38.2.1:
Successfully uninstalled setuptools-38.2.1
Successfully installed notebook-5.2.2 setuptools-38.2.3 terminado-0.8.1
IPywidgets and ipyleaflet support enabled.
In [2]:
g = glob.glob('_talks/*.md')
In [3]:
geocoder = Nominatim()
location_dict = {}
location = ""
permalink = ""
title = ""
In [4]:
for file in g:
with open(file, 'r') as f:
lines = f.read()
if lines.find('location: "') > 1:
loc_start = lines.find('location: "') + 11
lines_trim = lines[loc_start:]
loc_end = lines_trim.find('"')
location = lines_trim[:loc_end]
location_dict[location] = geocoder.geocode(location)
print(location, "\n", location_dict[location])
Mar Del Plata, Argentina
Mar del Plata, Partido de General Pueyrredón, Bs. As., Argentina
Bad Honnef, Germany
Bad Honnef, Rhein-Sieg-Kreis, Regierungsbezirk Köln, Nordrhein-Westfalen, 53604, Deutschland
São Paulo, Brazil
SP, Microrregião de São Paulo, RMSP, Mesorregião Metropolitana de São Paulo, SP, Região Sudeste, Brasil
Valparaíso, Chile
Valparaíso, Provincia de Valparaíso, V Región de Valparaíso, Chile
In [11]:
m = getorg.orgmap.create_map_obj()
getorg.orgmap.output_html_cluster_map(location_dict, folder_name="talkmap", hashed_usernames=False)
Out[11]:
'Written map to talkmap/'
In [ ]:
Content source: abbarreto/abbarreto.github.io
Similar notebooks: