Leaflet cluster map of talk locations

Run this from the _talks/ directory, which contains .md files of all your talks. This scrapes the location YAML field from each .md file, geolocates it with geopy/Nominatim, and uses the getorg library to output data, HTML, and Javascript for a standalone cluster map.


In [12]:
!pip install getorg --upgrade
import glob
import getorg
from geopy import Nominatim


Requirement already up-to-date: getorg in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (0.3.1)
Requirement not upgraded as not directly required: pygithub in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from getorg) (1.43.2)
Requirement not upgraded as not directly required: geopy in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from getorg) (1.17.0)
Requirement not upgraded as not directly required: retrying in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from getorg) (1.3.3)
Requirement not upgraded as not directly required: pyjwt in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from pygithub->getorg) (1.6.4)
Requirement not upgraded as not directly required: Deprecated in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from pygithub->getorg) (1.2.3)
Requirement not upgraded as not directly required: requests>=2.14.0 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from pygithub->getorg) (2.18.4)
Requirement not upgraded as not directly required: geographiclib<2,>=1.49 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from geopy->getorg) (1.49)
Requirement not upgraded as not directly required: six>=1.7.0 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from retrying->getorg) (1.11.0)
Requirement not upgraded as not directly required: wrapt<2,>=1 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from Deprecated->pygithub->getorg) (1.10.11)
Requirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (3.0.4)
Requirement not upgraded as not directly required: idna<2.7,>=2.5 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (2.6)
Requirement not upgraded as not directly required: urllib3<1.23,>=1.21.1 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (1.22)
Requirement not upgraded as not directly required: certifi>=2017.4.17 in /Users/mattyboy/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (2018.4.16)
distributed 1.21.8 requires msgpack, which is not installed.
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

In [13]:
g = glob.glob("*.md")

In [14]:
geocoder = Nominatim()
location_dict = {}
location = ""
permalink = ""
title = ""


/Users/mattyboy/anaconda3/lib/python3.6/site-packages/geopy/geocoders/osm.py:143: UserWarning: Using Nominatim with the default "geopy/1.17.0" `user_agent` is strongly discouraged, as it violates Nominatim's ToS https://operations.osmfoundation.org/policies/nominatim/ and may possibly cause 403 and 429 HTTP errors. Please specify a custom `user_agent` with `Nominatim(user_agent="my-application")` or by overriding the default `user_agent`: `geopy.geocoders.options.default_user_agent = "my-application"`. In geopy 2.0 this will become an exception.
  UserWarning

In [15]:
for file in g:
    with open(file, 'r') as f:
        lines = f.read()
        if lines.find('location: "') > 1:
            loc_start = lines.find('location: "') + 11
            lines_trim = lines[loc_start:]
            loc_end = lines_trim.find('"')
            location = lines_trim[:loc_end]
                            
           
        location_dict[location] = geocoder.geocode(location)
        print(location, "\n", location_dict[location])


 
 None
 
 None
 
 None

In [16]:
m = getorg.orgmap.create_map_obj()
getorg.orgmap.output_html_cluster_map(location_dict, folder_name="../talkmap", hashed_usernames=False)


Out[16]:
'Written map to ../talkmap/'

In [ ]:


In [ ]: