Leaflet cluster map of talk locations

Run this from the _talks/ directory, which contains .md files of all your talks. This scrapes the location YAML field from each .md file, geolocates it with geopy/Nominatim, and uses the getorg library to output data, HTML, and Javascript for a standalone cluster map.


In [1]:
!pip install getorg --upgrade
import glob
import getorg
from geopy import Nominatim
import os


Requirement already up-to-date: getorg in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (0.3.1)
Requirement already satisfied, skipping upgrade: retrying in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from getorg) (1.3.3)
Requirement already satisfied, skipping upgrade: pygithub in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from getorg) (1.43.2)
Requirement already satisfied, skipping upgrade: geopy in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from getorg) (1.17.0)
Requirement already satisfied, skipping upgrade: six>=1.7.0 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from retrying->getorg) (1.12.0)
Requirement already satisfied, skipping upgrade: pyjwt in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from pygithub->getorg) (1.6.4)
Requirement already satisfied, skipping upgrade: Deprecated in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from pygithub->getorg) (1.2.4)
Requirement already satisfied, skipping upgrade: requests>=2.14.0 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from pygithub->getorg) (2.21.0)
Requirement already satisfied, skipping upgrade: geographiclib<2,>=1.49 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from geopy->getorg) (1.49)
Requirement already satisfied, skipping upgrade: wrapt<2,>=1 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from Deprecated->pygithub->getorg) (1.10.11)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (1.24.1)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (2.8)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/aclarkdata/anaconda3/lib/python3.6/site-packages (from requests>=2.14.0->pygithub->getorg) (2019.3.9)
IPywidgets and ipyleaflet support enabled.

In [2]:
# change directory to talks
os.chdir('_talks')
os.getcwd()


Out[2]:
'/home/aclarkdata/repos/aclarkData.github.io/_talks'

In [3]:
g = glob.glob("*.md")

In [4]:
geocoder = Nominatim()
location_dict = {}
location = ""
permalink = ""
title = ""


/home/aclarkdata/anaconda3/lib/python3.6/site-packages/geopy/geocoders/osm.py:143: UserWarning: Using Nominatim with the default "geopy/1.17.0" `user_agent` is strongly discouraged, as it violates Nominatim's ToS https://operations.osmfoundation.org/policies/nominatim/ and may possibly cause 403 and 429 HTTP errors. Please specify a custom `user_agent` with `Nominatim(user_agent="my-application")` or by overriding the default `user_agent`: `geopy.geocoders.options.default_user_agent = "my-application"`. In geopy 2.0 this will become an exception.
  UserWarning

In [6]:
for file in g:
    with open(file, 'r') as f:
        lines = f.read()
        if lines.find('location: "') > 1:
            loc_start = lines.find('location: "') + 11
            lines_trim = lines[loc_start:]
            loc_end = lines_trim.find('"')
            location = lines_trim[:loc_end]
                            
           
        location_dict[location] = geocoder.geocode(location)
        print(location, "\n", location_dict[location])


Richmond, Virginia 
 Richmond, Richmond City, Virginia, 23298, USA
Chicago, Illinois 
 Chicago, Cook County, Illinois, USA
Las Vegas, Nevada 
 Las Vegas, Clark County, Nevada, USA
Austin, Texas 
 Austin, Travis County, Texas, United States of America
Tulsa, Oklahoma 
 Tulsa, Tulsa County, Oklahoma, USA
Chicago, Illinois 
 Chicago, Cook County, Illinois, USA
Anaheim, California 
 Anaheim, Orange County, California, USA
Las Vegas, Nevada 
 Las Vegas, Clark County, Nevada, USA
Dallas, Texas 
 Dallas, Dallas County, Texas, United States of America
Chicago, Illinois 
 Chicago, Cook County, Illinois, USA
Arlington, Virginia 
 Arlington, Arlington County, Virginia, USA
New York City, New York 
 NYC, New York, United States of America
New Orleans, Louisiana 
 New Orleans, Orleans Parish, Louisiana, USA
Richmond, Virginia 
 Richmond, Richmond City, Virginia, 23298, USA

In [7]:
m = getorg.orgmap.create_map_obj()
getorg.orgmap.output_html_cluster_map(location_dict, folder_name="../talkmap", hashed_usernames=False)


Out[7]:
'Written map to ../talkmap/'