JSON and APIs

JSON

What is JSON? From JSON.org:

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

But that isn't exactly helpful is it? JSON is a string format that allows you to store dictionaries, lists, strings, and numbers in a way that you can pass it from one source to another. You can take a Python dictionary and pass it to a perl program by printing it in JSON format and loading it or you can pull data from the web and create a python dictionary or list from that. Even if you don't understand now, after you use it, JSON will become more clear.

You have in this folder a json file called shapes.json. Take a look at it and then we can talk about JSON format.

JSON Format

JSON is a subset of python. You have a top level object that is either a list or a dictionary. You can have values and keys in the top level object be any of the following: strings, floats, ints, lists, boolean, null, or dictonary. To see how to represent these refer to the documentation www.json.org

JSON in Python

To use json in python we use the module json. It is part of the standard library so you don't need to install anything.

import json



In [ ]:

    
import json

Loading data

You can load data from json format into python from either a string using the loads method or a file handle using the load method.

my_list = json.loads('[1, 2, 3]')
with open(my_file.json) as fh:
    my_dict = json.load(fh)



In [ ]:

    
# Load from file
with open('shapes.json') as fh:
    shapes = json.load(fh)
print(shapes)



In [ ]:

    
# Load from string
complex_shapes_string = '["pentagon", "spiral", "double helix"]'
complex_shapes = json.loads(complex_shapes_string)
print(complex_shapes)

TRY IT

Create a string called three_d_json which cotains the string '["cube", "sphere"]' and then load that data into a python list using json.load.



In [ ]:

Using JSON data

Once you load data from python format, you can now use the data like you would any other python dictionary or list.



In [ ]:

    
for shape in shapes:
    title_shape = shape.title()
    area_formula = shapes[shape]['area']
    print("{}'s area can be calculated using {}".format(title_shape, area_formula))

TRY IT

for each shape in complex_shapes print "shape is hard to find the area of".



In [ ]:

Dumping JSON Data

If you want to store data from your python program into JSON format, it is as simple as loading it. To dump to a string use json.dumps and to dump to a file use json.dump. Make sure that you are using only valid json values in your list or dictionary.

json_string = json.dumps(my_list)
with open('json_file.json', 'w') as fh:
    json.dump(my_dict, fh)



In [ ]:

    
# Dumping to string
favorite_shapes = ['hexagon', 'heart']
fav_shapes_json = json.dumps(favorite_shapes)
print(fav_shapes_json)



In [ ]:

    
# Dumping to a file
with open('fav_shapes.json', 'w') as fh:
    json.dump(favorite_shapes, fh)

TRY IT

create a list of 4 sided shapes and store in a variable called quads, dump quad to json and store the result in a variable called quads_json.



In [ ]:

Web APIs

Web APIs are a way to retreive and send data to and from a url. The urls have a pattern so that you can retreive data programtically. With REST APIs specifically, you build a url putting data in the correct places to retreive the data you need. Many Web APIs (the best ones) return their data in JSON format.

There are many free api's available, most require that you sign up to recieve an API key. You will need to read the API docs for any specific api to figure out how to get the data you want.

Here are some fun APIs to try out:

Dropbox: https://www.dropbox.com/developers
Google Maps: https://developers.google.com/maps/web/
Twitter: https://dev.twitter.com/docs
YouTube: https://developers.google.com/youtube/v3/getting-started
Soundcloud: http://developers.soundcloud.com/docs/api/guide#playing
Stripe: https://stripe.com/docs/tutorials/checkout
Instagram: http://instagram.com/developer/
Twilio: https://www.twilio.com/docs
Yelp: http://www.yelp.com/developers/getting_started
Facebook: https://developers.facebook.com/docs/facebook-login/login-flow-for-web
Etsy: https://www.etsy.com/developers/documentation

We are going to use the steam api because certain endpoints don't require an app id (and who has time to sign up for one when there is python to learn?)

The endpoint we will use is one that will get us metadata info about a specific game:

http://store.steampowered.com/api/appdetails?appids=<id number>

If the game doesn't exist it returns json that looks like this:

    {"1":{"success":false}}

If the game does exist it returns json that looks like this:

"100":{  
"success":true,
"data":{  
  "type":"game",
  "name":"Counter-Strike: Condition Zero",
  "steam_appid":80,
  "required_age":0,
  "is_free":false,
  "detailed_description":"With its extensive Tour of Duty campaign, a near-limitless number of skirmish modes, updates and new content for Counter-Strike's award-winning multiplayer game play, plus over 12 bonus single player missions, Counter-Strike: Condition Zero is a tremendous offering of single and multiplayer content.",
  "about_the_game":"With its extensive Tour of Duty campaign, a near-limitless number of skirmish modes, updates and new content for Counter-Strike's award-winning multiplayer game play, plus over 12 bonus single player missions, Counter-Strike: Condition Zero is a tremendous offering of single and multiplayer content.",
  "supported_languages":"English, French, German, Italian, Spanish, Simplified Chinese, Traditional Chinese, Korean",
  "header_image":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/header.jpg?t=1447889920",
  "website":null,
  "pc_requirements":{  
    "minimum":"\r\n\t\t\t<p><strong>Minimum:<\/strong> 500 mhz processor, 96mb ram, 16mb video card, Windows XP, Mouse, Keyboard, Internet Connection<br \/><\/p>\r\n\t\t\t<p><strong>Recommended:<\/strong> 800 mhz processor, 128mb ram, 32mb+ video card, Windows XP, Mouse, Keyboard, Internet Connection<br \/><\/p>\r\n\t\t\t"
  },
  "mac_requirements":[  

  ],
  "linux_requirements":[  

  ],
  "developers":[  
    "Valve"
  ],
  "publishers":[  
    "Valve"
  ],
  "price_overview":{  
    "currency":"USD",
    "initial":999,
    "final":999,
    "discount_percent":0
  },
  "packages":[  
    7
  ],
  "package_groups":[  
    {  
      "name":"default",
      "title":"Buy Counter-Strike: Condition Zero",
      "description":"",
      "selection_text":"Select a purchase option",
      "save_text":"",
      "display_type":0,
      "is_recurring_subscription":"false",
      "subs":[  
        {  
          "packageid":7,
          "percent_savings_text":"",
          "percent_savings":0,
          "option_text":"Counter-Strike: Condition Zero $9.99",
          "option_description":"",
          "can_get_free_license":"0",
          "is_free_license":false,
          "price_in_cents_with_discount":999
        }
      ]
    }
  ],
  "platforms":{  
    "windows":true,
    "mac":true,
    "linux":true
  },
  "metacritic":{  
    "score":65,
    "url":"http:\/\/www.metacritic.com\/game\/pc\/counter-strike-condition-zero?ftag=MCD-06-10aaa1f"
  },
  "categories":[  
    {  
      "id":2,
      "description":"Single-player"
    },
    {  
      "id":1,
      "description":"Multi-player"
    },
    {  
      "id":8,
      "description":"Valve Anti-Cheat enabled"
    }
  ],
  "genres":[  
    {  
      "id":"1",
      "description":"Action"
    }
  ],
  "screenshots":[  
    {  
      "id":0,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002528.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002528.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":1,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002529.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002529.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":2,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002530.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002530.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":3,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002531.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002531.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":4,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002532.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002532.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":5,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002533.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002533.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":6,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002534.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002534.1920x1080.jpg?t=1447889920"
    },
    {  
      "id":7,
      "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002535.600x338.jpg?t=1447889920",
      "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002535.1920x1080.jpg?t=1447889920"
    }
  ],
  "recommendations":{  
    "total":6647
  },
  "release_date":{  
    "coming_soon":false,
    "date":"Mar 1, 2004"
  },
  "support_info":{  
    "url":"http:\/\/steamcommunity.com\/app\/80",
    "email":""
  },
  "background":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/page_bg_generated_v6b.jpg?t=1447889920"
}

} }

You can actually use the url in a browser. Try that and see if you hit on any interesting games by entering an id number

Accessing API data with python

There are many options for getting data from a url with python: httplib, urllib, urllib2, requests. This isn't limited to JSON data from a web api, you can get the raw html from any website. We are going to use urllib2 because it is part of the standard library and it is easy to use.

First, as with any library, we import it

import urllib2

Then you open a url using the method urlopen

connection = urllib2.urlopen('url')

Then you can read the data

data = connection.read()



In [ ]:

    
import urllib.request, urllib.error, urllib.parse



In [ ]:

    
game_id = str(251990)
connection = urllib.request.urlopen('http://store.steampowered.com/api/appdetails?appids=' + game_id)
data = connection.read()
type(data)

Now the result is a string, but it is valid json and we know how to turn a json string into a python dictionary: json.loads()



In [ ]:

    
game_data = json.loads(data)
print(type(game_data))

Finally you can use this data just like you would any python dictionary.



In [ ]:

    
print(game_data[game_id]['data']['name'])
print(game_data[game_id]['data']['about_the_game'])
print(game_data[game_id]['data']['price_overview']['final'])

TRY IT

Retreive the game data for the game with the id of 212680, parse the json and print out the game's name.



In [ ]:

Project

You will need to sign up for a open weather api key here (FREE tier) http://openweathermap.org/price WARNING The free tier only let you make 60 requests an hour, so be conservitave when testing your code.
After you sign up for an api key and then go look at the documentation for current weather: http://openweathermap.org/current
Pick a list of three cities [London, Paris, New York] for example and use a for loop to get the current weather for each one. Use urllib2 to fetch the data from the url and the json library to decode the data into a python dictionary.
Print the weather under the key 'weather' and then under 'description'. HINT Look at the result of the json in your web browser before you try to use it in your code. http://api.openweathermap.org/data/2.5/weather?q={city}&appid={appid}



In [ ]: