Getting Data from the Web

var is either tas or pr ext is usually CSV iso3 is the ISO standard 3 letter code for the country of interest (capitals) look up country codes online


In [2]:
# Python request library lets us get data straight from a URL 
import requests

In [9]:
url = "http://climatedataapi.worldbank.org/climateweb/rest/v1/country/cru/tas/year/GBR.csv"
response = requests.get(url ) # gets the get function from the request library to find the url and put it in a loop etc 

if response.status_code != 200:
    print ('Failed to get data: ', response.status_code)
else: 
    print ('First 100 charecters of data are: ')
    print (response.text[:100])


First 100 charecters of data are: 
year,data
1901,8.16360092163086
1902,7.798388481140137
1903,8.02857780456543
1904,8.073899269104004

To do:

Get temperature for Guatemala

Fetch rainfall for Afghanistan between 1980 and 1999


In [8]:
url = 'http://climatedataapi.worldbank.org/climateweb/rest/v1/country/cru/tas/GTM.csv'
response = requests.get(url ) # gets the get function from the request library to find the url and put it in a loop etc 

if response.status_code != 200:
    print ('Failed to get data: ', response.status_code)
else: 
    print ('First 100 charecters of data are: ')
    print (response.text[:100])


First 100 charecters of data are: 
year,data
1901,23.428869247436523
1902,23.555654525756836
1903,23.56220245361328
1904,23.44627952575

In [13]:
url = 'http://climatedataapi.worldbank.org/climateweb/rest/v1/country/annualavg/pr/1980/1999/AFG.csv'
response = requests.get(url ) # gets the get function from the request library to find the url and put it in a loop etc 

if response.status_code != 200:
    print ('Failed to get data: ', response.status_code)
else: 
    print ('First 100 charecters of data are: ')
    print (response.text[:100])


First 100 charecters of data are: 
GCM,var,from_year,to_year,annual
bccr_bcm2_0,pr,1980,1999,429.92613067639326
cccma_cgcm3_1,pr,1980,1

In [14]:
# Create a csv file: test01.csv

1901, 12.3
1902, 45.6
1903, 78.9

In [15]:
with open('test01.csv', 'r') as reader: 
    for line in reader: 
        print (len(line))


11
11
10

In [17]:
with open ('test01.csv', 'r') as reader: 
    for line in reader: 
        fields = line.split (',')
        print (fields)


['1901', ' 12.3\n']
['1902', ' 45.6\n']
['1903', ' 78.9']

In [18]:
# We need to get rid of the hidden newline \n
with open ('test01.csv', 'r') as reader: 
    for line in reader: 
        fields = line.strip().split (',')
        print (fields)


['1901', ' 12.3']
['1902', ' 45.6']
['1903', ' 78.9']

Using the csv library instead


In [19]:
import csv

In [23]:
with open ('test01.csv', 'r') as rawdata: 
    csvdata = csv.reader(rawdata)
    for record in csvdata: 
            print (record)


['1901', ' 12.3']
['1902', ' 45.6']
['1903', ' 78.9']

In [31]:
url = 'http://climatedataapi.worldbank.org/climateweb/rest/v1/country/cru/tas/year/GTM.csv'
response = requests.get(url ) # gets the get function from the request library to find the url and put it in a loop etc 

if response.status_code != 200:
    print ('Failed to get data: ', response.status_code)
else: 
    wrapper = csv.reader(response.text.strip().split('\n'))
    for record in wrapper: 
        if record[0] != 'year' : 
            year = int(record[0])
            value = float(record[1])
            print (year, value)


1901 23.428869247436523
1902 23.555654525756836
1903 23.56220245361328
1904 23.446279525756836
1905 23.513988494873047
1906 23.24672508239746
1907 23.337797164916992
1908 23.33348274230957
1909 23.466964721679688
1910 22.739879608154297
1911 23.36443519592285
1912 23.389881134033203
1913 23.087499618530273
1914 23.094493865966797
1915 23.254465103149414
1916 23.149404525756836
1917 23.135862350463867
1918 23.501785278320312
1919 23.32782745361328
1920 23.355655670166016
1921 23.19746971130371
1922 23.65372085571289
1923 23.454910278320312
1924 23.340625762939453
1925 23.29895782470703
1926 23.459524154663086
1927 23.64389991760254
1928 23.569196701049805
1929 23.687053680419922
1930 23.259374618530273
1931 23.489582061767578
1932 23.52336311340332
1933 23.42202377319336
1934 23.356548309326172
1935 23.28928565979004
1936 23.855953216552734
1937 23.561161041259766
1938 22.84910774230957
1939 23.209224700927734
1940 23.152530670166016
1941 23.23169708251953
1942 23.082887649536133
1943 22.462053298950195
1944 22.91979217529297
1945 23.282737731933594
1946 23.284822463989258
1947 23.362350463867188
1948 23.163244247436523
1949 22.975446701049805
1950 22.568599700927734
1951 23.19851303100586
1952 23.199405670166016
1953 23.68764877319336
1954 22.980060577392578
1955 23.047321319580078
1956 22.873214721679688
1957 23.506250381469727
1958 23.937053680419922
1959 23.73675537109375
1960 23.483482360839844
1961 23.428274154663086
1962 23.12172508239746
1963 23.147768020629883
1964 23.33839225769043
1965 23.23675537109375
1966 23.100297927856445
1967 23.074108123779297
1968 22.9449405670166
1969 23.83616065979004
1970 23.637052536010742
1971 23.55253028869629
1972 23.98705291748047
1973 23.762798309326172
1974 23.228572845458984
1975 23.59702491760254
1976 23.209524154663086
1977 23.620237350463867
1978 23.48660659790039
1979 23.438392639160156
1980 23.73110008239746
1981 23.444643020629883
1982 23.705059051513672
1983 23.909076690673828
1984 23.17693519592285
1985 23.457590103149414
1986 23.511159896850586
1987 23.860267639160156
1988 23.779762268066406
1989 23.601041793823242
1990 23.645387649536133
1991 23.99776840209961
1992 23.934673309326172
1993 23.94240951538086
1994 24.121578216552734
1995 23.937053680419922
1996 23.544940948486328
1997 24.376041412353516
1998 24.522619247436523
1999 23.534822463989258
2000 23.979167938232422
2001 24.295833587646484
2002 24.442262649536133
2003 24.412202835083008
2004 24.534671783447266
2005 24.540922164916992
2006 24.38273811340332
2007 24.204612731933594
2008 24.20639991760254
2009 24.54940414428711
2010 24.18541717529297
2011 24.21080780029297
2012 24.315753936767578

In [ ]: