In [1]:
# packages to get JSON data from an HTTP API
using Requests
using JSON
In [2]:
# no, I'm not providing my Meetup API key -- get your own!
apikey = open(readchomp, "apikey");
apikey[1:5]
Out[2]:
In [3]:
# ask the Meetup API for deets about a group
function getGroupInfo(apikey, urlname)
request = "https://api.meetup.com/2/groups?key=$apikey&sign=true&group_urlname=$urlname"
ret = get(request)
dat = JSON.parse(ret.data)
dat["results"][1]
end
gi = getGroupInfo(apikey, "stats-prog-DC")
Out[3]:
In [4]:
# ask the Meetup API for Member IDs from a group
# requires chunked requests (a better way would be to use the "next" field in the response)
function getMembers(apikey, group_id, memberCt; verbose=true)
chunksize = 200
memberIds = Array(Int,0)
if verbose print(group_id) end
for page in 0:ifloor(memberCt/chunksize)
request = "https://api.meetup.com/2/members?key=$apikey&sign=true&group_id=$group_id&page=$chunksize&offset=$page&only=id"
ret = get(request)
dat = JSON.parse(ret.data)
if verbose print('.') end
for x in dat["results"]
push!(memberIds, x["id"])
end
end
if verbose println() end
memberIds
end
member_ids = getMembers(apikey, gi["id"], gi["members"])
member_ids[1:20]
Out[4]:
In [5]:
# great, seems to work, now get all the members for relevant Meetups, storing as a dict of sets
group_names = ["stats-prog-dc", "data-science-dc",
"DC-Hack-and-Tell", "code-for-dc", "hack-edu"]
group_members_struct = Dict()
for grname in group_names
gi = getGroupInfo(apikey, grname)
group_members_struct[grname] =
Set(getMembers(apikey, gi["id"], gi["members"])...)
end
group_members_struct
# takes a couple minutes -- jump to the end!
Out[5]:
In [6]:
# then convert that dict of sets to a bool matrix
everyone = union([v for (k,v) in group_members_struct]...)
memb_group = [in(memb, group_members_struct[group])
for memb in everyone, group in group_names]
Out[6]:
In [7]:
# and now we're good to make a VennEuler diagram!
using VennEuler
In [9]:
eo = make_euler_object(group_names,
memb_group,
EulerSpec(:rectangle), # rectangles > circles!
sizesum=.4) # scaling in unit square
(minf,minx,ret) = optimize_iteratively( # greedy meta-optimization algorithm
eo, # problem we're trying to solve
random_state(eo), # where to start
ftol=-1, xtol=0.0025, maxtime=5, pop=100) # quick 'n dirty
(minf,minx,ret) = optimize(eo, # global optimization
minx, # start where we left off
ftol=.0001, xtol=0.0025, maxtime=20, pop=200) # more horsepower this time...
println("FINALLY:\ngot $minf at $minx (returned $ret)")
In [10]:
render("lkjh.svg", eo, minx)
Due to Wilkinson (2012).
Previous implementations in Java with R wrapper.
My work: Julia. More shapes.
Nota Bene: If using circles, can't generally do this perfectly if more than 2 sets! (With 3 sets, power set is $2^n = 8$, with $3 \cdot (n-1) = 6$ degrees of freedom.)
But, we can approximate and minimize a cost function!
Cool bit: This works with any shape we can plot. No need for a fancy-pants geometric overlap algorithm.
Less cool bits: There's some error if the virtual screen's too small. I use 200px square, which is good enough to be a smaller issue than the inadequate DoF problem.
NLopt, an open-source nonlinear optimization package out of MIT, with a Julia wrapper.