Uber rides value estimation


Objectives: The main objective of this work is to collect data from Uber's Open API for a week requesting information about rides from a unique starting point and finalizing in a one point of all neighborhoods of Natal, capital of Rio Grande do Norte - Brazil.

Explaining video: https://youtu.be/3paclf2j-BQ

Group components:

  • Marco Olimpio - marco.olimpio at gmail
  • Rebecca Betwel - bekbetwel at gmail

The Open Uber API (https://developer.uber.com/) has a limited number of requests per hour and his limited to 2000 requests. The citie of Natal has 36 neighborhoods so according to this we have to make 36 requests per set of neighborhoods, so we could do

$$ {2000}/{36} = 55.55$$

55 set of requests but we limited to only 1440 requests because we configured the data collector to execute this set of requests for every 90 seconds.

The data started to be collected from 29/11/2017 until 08/11/2017 in every 90 seconds. Generating about 34Mb of data and 235779 requests. With this configuration of the data collector if we we could be collect about: $$ 10 days * 24 hours * 1440 requests = 345600 lines$$ or $235779/345600 = 68.22\%$ of data collecting efficiency. Unfortunatelly the computer collecting the data was my laptop.

How do we organized everything? We created a simple python collector that utilizes the Uber rides library. To install simply do

!pip install uber-rides

after that all you have to do is to create an user and configure a new application in Uber's development website. More information abour the Uber Rides could be found here: https://developer.uber.com/docs/riders/ride-requests/tutorials/api/python

Importing the Uber Rides

from uber_rides.session import Session
from uber_rides.client import UberRidesClient

After this we collected some data to know how the API works and persisted the results in a comma separetad file (CSV). This CSV is loaded in this kernel and we start to play! :] You could check the imagem below

Below we do have the code that collected all the data.


In [76]:
from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter
import IPython

with open('../consumer.py') as f:
    code = f.read()

formatter = HtmlFormatter()
IPython.display.HTML('<style type="text/css">{}</style>{}'.format(
    formatter.get_style_defs('.highlight'),
    highlight(code, PythonLexer(), formatter)))


Out[76]:
import time
import sys
import csv
import os
import json
import datetime

from uber_rides.session import Session
from uber_rides.client import UberRidesClient

point_imd = [-5.8323, -35.2054]
points_neighborhood = [["Pitimbu",[-5.876271, -35.224500]],
			["Planalto",[-5.858102, -35.251586]],
			["Ponta Negra",[-5.877522, -35.176073]],
			["Neópolis",[-5.870513, -35.208176]],
			["Capim Macio",[-5.862753, -35.195597]],
			["Lagoa Azul",[-5.734177, -35.253802]],
			["Pajuçara",[-5.751781, -35.234665]],
			["Lagoa Seca",[-5.809191, -35.209374]],
			["Barro Vermelho",[-5.800741, -35.211056]],
			["Candelária",[-5.841737, -35.210463]],
			["Praia do Meio",[-5.779198, -35.197163]],
			["Rocas",[-5.771832, -35.203091]],
			["Santos Reis",[-5.763226, -35.196786]],
			["Redinha",[-5.742772, -35.205806]],
			["Salinas",[-5.763117, -35.247973]],
			["Igapó",[-5.769009, -35.254755]],
			["Nossa Senhora da Apresentação",[-5.763654, -35.282543]],
			["Potengi",[-5.758634, -35.247010]],
			["Ribeira",[-5.774943, -35.205578]],
			["Cidade Alta",[-5.785291, -35.206464]],
			["Alecrim",[-5.796158, -35.216566]],
			["Nordeste",[-5.796215, -35.245141]],
			["Quintas",[-5.797290, -35.226006]],
			["Bom Pastor",[-5.813764, -35.240721]],
			["Dix-Sept Rosado",[-5.812036, -35.223915]],
			["Nossa Senhora de Nazaré",[-5.815939, -35.229249]],
			["Lagoa Nova",[-5.819743, -35.212920]],
			["Mãe Luiza",[-5.794771, -35.188619 ]],
			["Nova Descoberta",[-5.824830, -35.200026]],
			["Tirol",[-5.791699, -35.197358]],
			["Petrópolis",[-5.782001, -35.195196]],
			["Areia Preta",[-5.789477, -35.188454]],
			["Cidade Nova",[-5.834856, -35.242523]],
			["Cidade da Esperança",[-5.825376, -35.242751]],
			["Felipe Camarão",[-5.824374, -35.250070]],
			["Guarapes",[-5.841580, -35.274691]]]

counter = 0
fieldnamesEstimates = ['id', 'timestamp', 'periodOfDay', 'neighborhood', 'start_latitude', 'start_longitude', 'finish_latitude', 'finish_longitude', 'currency_code', 'distance', 'duration','high_estimate', 'low_estimate', 'product_id']

def startCsvFile(text,csvfile):
	with open(csvfile, 'w') as csvfilewriter:
		writer = csv.DictWriter(csvfilewriter, fieldnames = fieldnamesEstimates)
		writer.writeheader() 

def consumer(seconds,csvFile):
	while True:
		global counter 
		counter = counter + 1
		print('Request '+ str(counter), end='')
		print(datetime.datetime.now())
		uberRequestTimes(counter, csvFile)
		time.sleep(seconds)	
			
def uberRequestTimes(counter, csvfile):
	
	session = Session(server_token='ZtSVybSe5Ma41cDP49hWRHL_qmS11nugdEr16Por')
	client = UberRidesClient(session)
	internalCounter = 1
	try:
		#iterate throgh all choosen detination points
		for rows in points_neighborhood:
			pontoA = rows[1][0]
			pontoB = rows[1][1]
			response = client.get_price_estimates(start_latitude = point_imd[0], start_longitude = point_imd[1], end_latitude = pontoA, end_longitude = pontoB, seat_count=2)
			requestTimeStamp = datetime.datetime.now()
			print('Request '+str(internalCounter)+ ' ', end='')
			print(' Timestamp: ',end='')
			print(requestTimeStamp)

			if requestTimeStamp.hour < 12:
				periodOfDay = 'morning'
			elif 12 <= requestTimeStamp.hour < 18:
				periodOfDay = 'afternoon'
			else:
				periodOfDay = 'evening'

			for returnLine in response.json.get('prices'):
				with open(csvfile, 'a') as csvfilewriter2:
					writer = csv.DictWriter(csvfilewriter2, fieldnames=fieldnamesEstimates)
					writer.writerow({'id':counter, 'timestamp':requestTimeStamp, 'periodOfDay': periodOfDay, 'neighborhood':rows[0], 'start_latitude':point_imd[0], 'start_longitude':point_imd[1], 'finish_latitude':rows[1][0], 'finish_longitude':rows[1][1], 'currency_code': returnLine['currency_code'], 'distance':returnLine['distance'],'duration':returnLine['duration'],'high_estimate':returnLine['high_estimate'],'low_estimate':returnLine['low_estimate'],'product_id':returnLine['product_id']})
			internalCounter = internalCounter + 1
	except:
		pass

		
def main():
	
	#Current file path for csv file
	currentPath = os.getcwd()
	csvFile = currentPath + '/uberData/uberRidesRequests.csv'
	executionCounter = 0
	fileExists = os.path.exists(csvFile)
	if not fileExists:
		startCsvFile('', csvFile)
	
	#Start UberRIdes configuration
	#main loop, it executes every x seconds
	consumer(90,csvFile)
			

if __name__ == "__main__":
	main()
	

The begining


Below we have the very beginning of the kernel itself. Firtly we load all necessary libraries and the data collected and after start analysing it.


In [2]:
#Jupyter Magic word to inline matplotlib plots
%matplotlib inline

#System libraries
import os
import sys

#Basic libraries for data analysis
import numpy as np
from numpy import random
import pandas as pd

#Choropleth necessary libraries
##GeoJson data
import json
##Necessary to create shapes in folium
from shapely.geometry import Polygon
from shapely.geometry import Point
##Choropleth itself
import folium
##Colormap
from branca.colormap import linear

#Plot
import matplotlib
import matplotlib.pyplot as plt

Data: Loading and adjustments


Importing data from Natal's neighborhoods in GeoJson format


In [4]:
# import geojson file about natal neighborhood
natal_neigh = os.path.join('geojson', 'natal.geojson')

# load the data and use 'UTF-8'encoding
geo_json_natal = json.load(open(natal_neigh,encoding='UTF-8'))

In [5]:
neighborhood = []
# list all neighborhoods
for neigh in geo_json_natal['features']:
        neighborhood.append(neigh['properties']['name'])

Loading uber collected data. The dava was stored in a csv format


In [3]:
uberJsonRequests = pd.read_csv('../uberData/uberRidesRequests.csv')

In [6]:
uberJsonRequests.head(3)


Out[6]:
id timestamp periodOfDay neighborhood start_latitude start_longitude finish_latitude finish_longitude currency_code distance duration high_estimate low_estimate product_id
0 1 2017-10-29 13:39:29.122787 afternoon Pitimbu -5.8323 -35.2054 -5.876271 -35.224500 BRL 4.44 660 16.0 12.0 65cb1829-9761-40f8-acc6-92d700fe2924
1 1 2017-10-29 13:39:29.122787 afternoon Pitimbu -5.8323 -35.2054 -5.876271 -35.224500 BRL 4.44 660 19.0 14.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138
2 1 2017-10-29 13:39:30.552931 afternoon Planalto -5.8323 -35.2054 -5.858102 -35.251586 BRL 6.50 1020 22.0 17.0 65cb1829-9761-40f8-acc6-92d700fe2924

The data returned from Uber API there is no mean estimative so we created a mean estimation based in arithmetic median $$ Estimate_{Mean} = \dfrac{Estimate_{High}+Estimate_{Low}}{2} $$


In [7]:
uberJsonRequests['mean_estimate'] = (uberJsonRequests['low_estimate']+uberJsonRequests['high_estimate'])/2

Another information derived from timestamp feature is the period of the day that was divided in morning, afternoon and night. The aim of this data is to see how the prices of the rides are related with the period of time.


In [9]:
uberJsonRequests['periodOfDay'].unique()


Out[9]:
array(['afternoon', 'evening', 'morning'], dtype=object)

Adjusting types of columns


In [11]:
uberJsonRequests['timestamp'] = pd.to_datetime(uberJsonRequests['timestamp'])

Seting up administrative regions/regions

According to wikipedia (https://pt.wikipedia.org/wiki/Lista_de_bairros_de_Natal_(Rio_Grande_do_Norte)) Natal is divided in four administrative regions: South, East, West and North.

  • East - Alecrim
  • East - Areia Preta
  • East - Barro Vermelho
  • East - Cidade Alta
  • East - Lagoa Seca
  • East - Mãe Luíza
  • East - Petrópolis
  • East - Praia do Meio
  • East - Ribeira
  • East - Rocas
  • East - Santos Reis
  • East - Tirol
  • North - Igapó
  • North - Lagoa Azul
  • North - Nossa Senhora da Apresentação
  • North - Pajuçara
  • North - Potengi
  • North - Redinha
  • North - Salinas
  • South - Candelária
  • South - Capim Macio
  • South - Lagoa Nova
  • South - Neópolis
  • South - Nova Descoberta
  • South - Pitimbu
  • South - Ponta Negra
  • West - Bom Pastor
  • West - Cidade da Esperança
  • West - Cidade Nova
  • West - Dix-Sept Rosado
  • West - Felipe Camarão
  • West - Guarapes
  • West - Nordeste
  • West - Nossa Senhora de Nazaré
  • West - Planalto
  • West - Quintas

In [14]:
def label_region (row):
   if row['neighborhood'] == 'Alecrim' or row['neighborhood'] == 'Areia Preta'  or row['neighborhood'] == 'Barro Vermelho' or row['neighborhood'] == 'Cidade Alta' or row['neighborhood'] == 'Lagoa Seca'  or row['neighborhood'] == 'Mãe Luiza'  or row['neighborhood'] == 'Petrópolis' or row['neighborhood'] == 'Praia do Meio' or row['neighborhood'] == 'Ribeira' or row['neighborhood'] == 'Rocas' or row['neighborhood'] == 'Santos Reis' or row['neighborhood'] == 'Tirol':
      return 'East'
   if row['neighborhood'] =='Igapó' or row['neighborhood']=='Lagoa Azul' or row['neighborhood']=='Nossa Senhora da Apresentação' or row['neighborhood']=='Pajuçara' or row['neighborhood']=='Potengi' or row['neighborhood']=='Redinha' or row['neighborhood']=='Salinas':
      return 'North'
   if row['neighborhood'] =='Candelária' or row['neighborhood']=='Capim Macio' or row['neighborhood']=='Lagoa Nova' or row['neighborhood']=='Neópolis' or row['neighborhood']=='Nova Descoberta' or row['neighborhood']=='Pitimbu' or row['neighborhood']=='Ponta Negra':
      return 'South'
   if row['neighborhood'] =='Bom Pastor' or row['neighborhood']=='Cidade da Esperança' or row['neighborhood']=='Cidade Nova' or row['neighborhood']=='Dix-Sept Rosado' or row['neighborhood']=='Felipe Camarão' or row['neighborhood']=='Guarapes' or row['neighborhood']=='Nordeste' or row['neighborhood']=='Nossa Senhora de Nazaré' or row['neighborhood']=='Planalto' or row['neighborhood']=='Quintas':
      return 'West'
   return ''

In [15]:
uberJsonRequests['region'] = uberJsonRequests.apply(label_region, axis=1)

In [16]:
uberJsonRequests


Out[16]:
id timestamp periodOfDay neighborhood start_latitude start_longitude finish_latitude finish_longitude currency_code distance duration high_estimate low_estimate product_id mean_estimate region
0 1 2017-10-29 13:39:29.122787 afternoon Pitimbu -5.8323 -35.2054 -5.876271 -35.224500 BRL 4.44 660 16.0 12.0 65cb1829-9761-40f8-acc6-92d700fe2924 14.0 South
1 1 2017-10-29 13:39:29.122787 afternoon Pitimbu -5.8323 -35.2054 -5.876271 -35.224500 BRL 4.44 660 19.0 14.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 16.5 South
2 1 2017-10-29 13:39:30.552931 afternoon Planalto -5.8323 -35.2054 -5.858102 -35.251586 BRL 6.50 1020 22.0 17.0 65cb1829-9761-40f8-acc6-92d700fe2924 19.5 West
3 1 2017-10-29 13:39:30.552931 afternoon Planalto -5.8323 -35.2054 -5.858102 -35.251586 BRL 6.50 1020 26.0 20.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 23.0 West
4 1 2017-10-29 13:39:31.783796 afternoon Ponta Negra -5.8323 -35.2054 -5.877522 -35.176073 BRL 4.72 660 17.0 13.0 65cb1829-9761-40f8-acc6-92d700fe2924 15.0 South
5 1 2017-10-29 13:39:31.783796 afternoon Ponta Negra -5.8323 -35.2054 -5.877522 -35.176073 BRL 4.72 660 20.0 15.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 17.5 South
6 1 2017-10-29 13:39:33.112852 afternoon Neópolis -5.8323 -35.2054 -5.870513 -35.208176 BRL 4.14 660 15.0 11.0 65cb1829-9761-40f8-acc6-92d700fe2924 13.0 South
7 1 2017-10-29 13:39:33.112852 afternoon Neópolis -5.8323 -35.2054 -5.870513 -35.208176 BRL 4.14 660 18.0 14.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 16.0 South
8 1 2017-10-29 13:39:34.648221 afternoon Capim Macio -5.8323 -35.2054 -5.862753 -35.195597 BRL 3.17 540 13.0 9.0 65cb1829-9761-40f8-acc6-92d700fe2924 11.0 South
9 1 2017-10-29 13:39:34.648221 afternoon Capim Macio -5.8323 -35.2054 -5.862753 -35.195597 BRL 3.17 540 15.0 11.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 13.0 South
10 1 2017-10-29 13:43:00.377291 afternoon Pitimbu -5.8323 -35.2054 -5.876271 -35.224500 BRL 4.44 660 16.0 12.0 65cb1829-9761-40f8-acc6-92d700fe2924 14.0 South
11 1 2017-10-29 13:43:00.377291 afternoon Pitimbu -5.8323 -35.2054 -5.876271 -35.224500 BRL 4.44 660 19.0 14.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 16.5 South
12 1 2017-10-29 13:43:01.361263 afternoon Planalto -5.8323 -35.2054 -5.858102 -35.251586 BRL 6.50 1020 22.0 17.0 65cb1829-9761-40f8-acc6-92d700fe2924 19.5 West
13 1 2017-10-29 13:43:01.361263 afternoon Planalto -5.8323 -35.2054 -5.858102 -35.251586 BRL 6.50 1020 26.0 20.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 23.0 West
14 1 2017-10-29 13:43:02.629724 afternoon Ponta Negra -5.8323 -35.2054 -5.877522 -35.176073 BRL 4.72 600 17.0 13.0 65cb1829-9761-40f8-acc6-92d700fe2924 15.0 South
15 1 2017-10-29 13:43:02.629724 afternoon Ponta Negra -5.8323 -35.2054 -5.877522 -35.176073 BRL 4.72 600 20.0 15.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 17.5 South
16 1 2017-10-29 13:43:04.064956 afternoon Neópolis -5.8323 -35.2054 -5.870513 -35.208176 BRL 4.14 660 15.0 11.0 65cb1829-9761-40f8-acc6-92d700fe2924 13.0 South
17 1 2017-10-29 13:43:04.064956 afternoon Neópolis -5.8323 -35.2054 -5.870513 -35.208176 BRL 4.14 660 18.0 14.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 16.0 South
18 1 2017-10-29 13:43:05.292747 afternoon Capim Macio -5.8323 -35.2054 -5.862753 -35.195597 BRL 3.17 480 13.0 9.0 65cb1829-9761-40f8-acc6-92d700fe2924 11.0 South
19 1 2017-10-29 13:43:05.292747 afternoon Capim Macio -5.8323 -35.2054 -5.862753 -35.195597 BRL 3.17 480 15.0 11.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 13.0 South
20 1 2017-10-29 13:43:06.418879 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 10.80 1680 32.0 26.0 65cb1829-9761-40f8-acc6-92d700fe2924 29.0 North
21 1 2017-10-29 13:43:06.418879 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 10.80 1680 38.0 31.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 34.5 North
22 1 2017-10-29 13:43:07.647532 afternoon Pajuçara -5.8323 -35.2054 -5.751781 -35.234665 BRL 9.68 1440 30.0 24.0 65cb1829-9761-40f8-acc6-92d700fe2924 27.0 North
23 1 2017-10-29 13:43:07.647532 afternoon Pajuçara -5.8323 -35.2054 -5.751781 -35.234665 BRL 9.68 1440 36.0 28.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 32.0 North
24 1 2017-10-29 13:43:10.108148 afternoon Lagoa Seca -5.8323 -35.2054 -5.809191 -35.209374 BRL 3.00 540 12.0 9.0 65cb1829-9761-40f8-acc6-92d700fe2924 10.5 East
25 1 2017-10-29 13:43:10.108148 afternoon Lagoa Seca -5.8323 -35.2054 -5.809191 -35.209374 BRL 3.00 540 15.0 11.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 13.0 East
26 1 2017-10-29 13:43:16.475345 afternoon Barro Vermelho -5.8323 -35.2054 -5.800741 -35.211056 BRL 3.68 660 14.0 11.0 65cb1829-9761-40f8-acc6-92d700fe2924 12.5 East
27 1 2017-10-29 13:43:16.475345 afternoon Barro Vermelho -5.8323 -35.2054 -5.800741 -35.211056 BRL 3.68 660 17.0 13.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 15.0 East
28 1 2017-10-29 13:43:29.357096 afternoon Candelária -5.8323 -35.2054 -5.841737 -35.210463 BRL 1.75 360 9.0 7.0 65cb1829-9761-40f8-acc6-92d700fe2924 8.0 South
29 1 2017-10-29 13:43:29.357096 afternoon Candelária -5.8323 -35.2054 -5.841737 -35.210463 BRL 1.75 360 11.0 8.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 9.5 South
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
235748 968 2017-11-09 00:03:17.016062 morning Nordeste -5.8323 -35.2054 -5.796215 -35.245141 BRL 5.61 1200 20.0 15.0 65cb1829-9761-40f8-acc6-92d700fe2924 17.5 West
235749 968 2017-11-09 00:03:17.016062 morning Nordeste -5.8323 -35.2054 -5.796215 -35.245141 BRL 5.61 1200 24.0 18.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 21.0 West
235750 968 2017-11-09 00:03:18.223128 morning Quintas -5.8323 -35.2054 -5.797290 -35.226006 BRL 4.58 960 17.0 13.0 65cb1829-9761-40f8-acc6-92d700fe2924 15.0 West
235751 968 2017-11-09 00:03:18.223128 morning Quintas -5.8323 -35.2054 -5.797290 -35.226006 BRL 4.58 960 20.0 16.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 18.0 West
235752 968 2017-11-09 00:03:19.681715 morning Bom Pastor -5.8323 -35.2054 -5.813764 -35.240721 BRL 3.75 720 15.0 11.0 65cb1829-9761-40f8-acc6-92d700fe2924 13.0 West
235753 968 2017-11-09 00:03:19.681715 morning Bom Pastor -5.8323 -35.2054 -5.813764 -35.240721 BRL 3.75 720 17.0 13.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 15.0 West
235754 968 2017-11-09 00:03:20.969915 morning Dix-Sept Rosado -5.8323 -35.2054 -5.812036 -35.223915 BRL 3.23 540 13.0 10.0 65cb1829-9761-40f8-acc6-92d700fe2924 11.5 West
235755 968 2017-11-09 00:03:20.969915 morning Dix-Sept Rosado -5.8323 -35.2054 -5.812036 -35.223915 BRL 3.23 540 15.0 11.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 13.0 West
235756 968 2017-11-09 00:03:22.142218 morning Nossa Senhora de Nazaré -5.8323 -35.2054 -5.815939 -35.229249 BRL 3.11 540 13.0 9.0 65cb1829-9761-40f8-acc6-92d700fe2924 11.0 West
235757 968 2017-11-09 00:03:22.142218 morning Nossa Senhora de Nazaré -5.8323 -35.2054 -5.815939 -35.229249 BRL 3.11 540 15.0 11.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 13.0 West
235758 968 2017-11-09 00:03:23.307961 morning Lagoa Nova -5.8323 -35.2054 -5.819743 -35.212920 BRL 2.11 360 10.0 7.0 65cb1829-9761-40f8-acc6-92d700fe2924 8.5 South
235759 968 2017-11-09 00:03:23.307961 morning Lagoa Nova -5.8323 -35.2054 -5.819743 -35.212920 BRL 2.11 360 12.0 9.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 10.5 South
235760 968 2017-11-09 00:03:24.498919 morning Mãe Luiza -5.8323 -35.2054 -5.794771 -35.188619 BRL 5.14 960 19.0 14.0 65cb1829-9761-40f8-acc6-92d700fe2924 16.5 East
235761 968 2017-11-09 00:03:24.498919 morning Mãe Luiza -5.8323 -35.2054 -5.794771 -35.188619 BRL 5.14 960 22.0 17.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 19.5 East
235762 968 2017-11-09 00:03:25.719111 morning Nova Descoberta -5.8323 -35.2054 -5.824830 -35.200026 BRL 1.23 180 8.0 6.0 65cb1829-9761-40f8-acc6-92d700fe2924 7.0 South
235763 968 2017-11-09 00:03:25.719111 morning Nova Descoberta -5.8323 -35.2054 -5.824830 -35.200026 BRL 1.23 180 9.0 6.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 7.5 South
235764 968 2017-11-09 00:03:26.902923 morning Tirol -5.8323 -35.2054 -5.791699 -35.197358 BRL 4.88 840 18.0 13.0 65cb1829-9761-40f8-acc6-92d700fe2924 15.5 East
235765 968 2017-11-09 00:03:26.902923 morning Tirol -5.8323 -35.2054 -5.791699 -35.197358 BRL 4.88 840 21.0 16.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 18.5 East
235766 968 2017-11-09 00:03:27.972029 morning Petrópolis -5.8323 -35.2054 -5.782001 -35.195196 BRL 5.00 840 18.0 14.0 65cb1829-9761-40f8-acc6-92d700fe2924 16.0 East
235767 968 2017-11-09 00:03:27.972029 morning Petrópolis -5.8323 -35.2054 -5.782001 -35.195196 BRL 5.00 840 21.0 17.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 19.0 East
235768 968 2017-11-09 00:03:29.036227 morning Areia Preta -5.8323 -35.2054 -5.789477 -35.188454 BRL 5.25 960 19.0 15.0 65cb1829-9761-40f8-acc6-92d700fe2924 17.0 East
235769 968 2017-11-09 00:03:29.036227 morning Areia Preta -5.8323 -35.2054 -5.789477 -35.188454 BRL 5.25 960 22.0 17.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 19.5 East
235770 968 2017-11-09 00:03:30.148921 morning Cidade Nova -5.8323 -35.2054 -5.834856 -35.242523 BRL 4.83 900 18.0 13.0 65cb1829-9761-40f8-acc6-92d700fe2924 15.5 West
235771 968 2017-11-09 00:03:30.148921 morning Cidade Nova -5.8323 -35.2054 -5.834856 -35.242523 BRL 4.83 900 21.0 16.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 18.5 West
235772 968 2017-11-09 00:03:31.350074 morning Cidade da Esperança -5.8323 -35.2054 -5.825376 -35.242751 BRL 4.25 720 16.0 12.0 65cb1829-9761-40f8-acc6-92d700fe2924 14.0 West
235773 968 2017-11-09 00:03:31.350074 morning Cidade da Esperança -5.8323 -35.2054 -5.825376 -35.242751 BRL 4.25 720 18.0 14.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 16.0 West
235774 968 2017-11-09 00:03:32.477488 morning Felipe Camarão -5.8323 -35.2054 -5.824374 -35.250070 BRL 5.18 900 18.0 14.0 65cb1829-9761-40f8-acc6-92d700fe2924 16.0 West
235775 968 2017-11-09 00:03:32.477488 morning Felipe Camarão -5.8323 -35.2054 -5.824374 -35.250070 BRL 5.18 900 21.0 17.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 19.0 West
235776 968 2017-11-09 00:03:33.604179 morning Guarapes -5.8323 -35.2054 -5.841580 -35.274691 BRL 8.66 1500 29.0 23.0 65cb1829-9761-40f8-acc6-92d700fe2924 26.0 West
235777 968 2017-11-09 00:03:33.604179 morning Guarapes -5.8323 -35.2054 -5.841580 -35.274691 BRL 8.66 1500 35.0 28.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 31.5 West

235778 rows × 16 columns

Now we can comment about every feature of the dataset.

  • timestamp:It records the time of the request return.
  • periodOfDay:It records the period of the day (morning/afternoon/nigh) of the request return.
  • neighborhood:It records the neighborhood of the requested final point.
  • start_latitude:The latitude of IMD building, the initial point
  • start_longitude:The longitude of IMD building, the initial point
  • finish_latitude:The latitude of the final point, one per neighborhood
  • finish_longitude:The longitude of the final point, one per neighborhood
  • currency_code:The currency utilized in the return of the estimative
  • distance:The distance from starting to finishing points
  • duration:The time estimation for the ride
  • high_estimate:The high ride price estimative
  • low_estimate:The low ride price estimative
  • product_id:The product Uber Select or Uber X
  • mean_estimate:The mean ride price estimative
  • region:The administrative region from where the neighborhood belongs

In [27]:
uberJsonRequests.info()


<class 'pandas.core.frame.DataFrame'>
RangeIndex: 235778 entries, 0 to 235777
Data columns (total 16 columns):
id                  235778 non-null int64
timestamp           235778 non-null datetime64[ns]
periodOfDay         235778 non-null object
neighborhood        235778 non-null object
start_latitude      235778 non-null float64
start_longitude     235778 non-null float64
finish_latitude     235778 non-null float64
finish_longitude    235778 non-null float64
currency_code       235778 non-null object
distance            235778 non-null float64
duration            235778 non-null int64
high_estimate       235778 non-null float64
low_estimate        235778 non-null float64
product_id          235778 non-null object
mean_estimate       235778 non-null float64
region              235778 non-null object
dtypes: datetime64[ns](1), float64(8), int64(2), object(5)
memory usage: 28.8+ MB

In [71]:
uberNeigh = uberJsonRequests.pivot_table(index=['neighborhood'], values='mean_estimate', aggfunc=np.mean)
uberNeigh


Out[71]:
neighborhood
Alecrim                          19.200336
Areia Preta                      21.860483
Barro Vermelho                   16.953020
Bom Pastor                       17.391122
Candelária                       10.376106
Capim Macio                      15.076524
Cidade Alta                      21.293704
Cidade Nova                      21.008410
Cidade da Esperança              18.834939
Dix-Sept Rosado                  15.318307
Felipe Camarão                   21.790520
Guarapes                         33.553058
Igapó                            30.369426
Lagoa Azul                       41.541934
Lagoa Nova                       11.393108
Lagoa Seca                       14.411836
Mãe Luiza                        21.643108
Neópolis                         18.088845
Nordeste                         24.784994
Nossa Senhora da Apresentação    35.551084
Nossa Senhora de Nazaré          14.928561
Nova Descoberta                   8.854600
Pajuçara                         36.488331
Petrópolis                       21.165648
Pitimbu                          19.250305
Planalto                         26.150823
Ponta Negra                      19.894576
Potengi                          34.510541
Praia do Meio                    21.631523
Quintas                          20.716611
Redinha                          32.176686
Ribeira                          23.987011
Rocas                            24.125801
Salinas                          34.020611
Santos Reis                      25.200412
Tirol                            20.511308
Name: mean_estimate, dtype: float64

In [30]:
uberProducts = uberJsonRequests.pivot_table(index=['product_id', 'neighborhood'], values='mean_estimate', aggfunc=np.mean)
uberProducts


Out[30]:
product_id                            neighborhood                 
65cb1829-9761-40f8-acc6-92d700fe2924  Alecrim                          15.579462
                                      Areia Preta                      17.666412
                                      Barro Vermelho                   13.720409
                                      Bom Pastor                       14.040342
                                      Candelária                        8.400366
                                      Capim Macio                      12.220274
                                      Cidade Alta                      17.175275
                                      Cidade Nova                      16.955810
                                      Cidade da Esperança              15.164526
                                      Dix-Sept Rosado                  12.402812
                                      Felipe Camarão                   17.573547
                                      Guarapes                         26.977982
                                      Igapó                            24.342547
                                      Lagoa Azul                       33.206160
                                      Lagoa Nova                        9.215312
                                      Lagoa Seca                       11.624466
                                      Mãe Luiza                        17.485330
                                      Neópolis                         14.557147
                                      Nordeste                         19.961491
                                      Nossa Senhora da Apresentação    28.487172
                                      Nossa Senhora de Nazaré          12.098258
                                      Nova Descoberta                   7.506724
                                      Pajuçara                         29.280201
                                      Petrópolis                       17.049969
                                      Pitimbu                          15.578763
                                      Planalto                         21.085771
                                      Ponta Negra                      16.055454
                                      Potengi                          27.581118
                                      Praia do Meio                    17.456210
                                      Quintas                          16.757641
                                                                         ...    
bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138  Cidade Alta                      25.412133
                                      Cidade Nova                      25.061009
                                      Cidade da Esperança              22.505352
                                      Dix-Sept Rosado                  18.233802
                                      Felipe Camarão                   26.007492
                                      Guarapes                         40.128135
                                      Igapó                            36.396304
                                      Lagoa Azul                       49.877707
                                      Lagoa Nova                       13.570905
                                      Lagoa Seca                       17.199207
                                      Mãe Luiza                        25.800886
                                      Neópolis                         21.620543
                                      Nordeste                         29.608496
                                      Nossa Senhora da Apresentação    42.614997
                                      Nossa Senhora de Nazaré          17.758863
                                      Nova Descoberta                  10.202476
                                      Pajuçara                         43.696461
                                      Petrópolis                       25.281326
                                      Pitimbu                          22.921846
                                      Planalto                         31.215874
                                      Ponta Negra                      23.733699
                                      Potengi                          41.439963
                                      Praia do Meio                    25.806836
                                      Quintas                          24.675581
                                      Redinha                          38.470400
                                      Ribeira                          28.636919
                                      Rocas                            28.812328
                                      Salinas                          40.859847
                                      Santos Reis                      30.112450
                                      Tirol                            24.461797
Name: mean_estimate, dtype: float64

In [31]:
uberPeriodOfDay = uberJsonRequests.pivot_table(index=['periodOfDay', 'neighborhood'], values='mean_estimate', aggfunc=np.mean)
uberPeriodOfDay


Out[31]:
periodOfDay  neighborhood                 
afternoon    Alecrim                          18.618224
             Areia Preta                      21.047196
             Barro Vermelho                   16.265858
             Bom Pastor                       16.900000
             Candelária                       10.016325
             Capim Macio                      14.471601
             Cidade Alta                      20.663084
             Cidade Nova                      20.383677
             Cidade da Esperança              18.275797
             Dix-Sept Rosado                  14.810280
             Felipe Camarão                   21.026735
             Guarapes                         31.643996
             Igapó                            30.080690
             Lagoa Azul                       40.923974
             Lagoa Nova                       10.897664
             Lagoa Seca                       13.825560
             Mãe Luiza                        20.728505
             Neópolis                         17.577747
             Nordeste                         24.430374
             Nossa Senhora da Apresentação    34.552705
             Nossa Senhora de Nazaré          14.347378
             Nova Descoberta                   8.442523
             Pajuçara                         35.132929
             Petrópolis                       20.468224
             Pitimbu                          18.204376
             Planalto                         24.466480
             Ponta Negra                      19.032588
             Potengi                          34.379907
             Praia do Meio                    20.926772
             Quintas                          19.712150
                                                ...    
morning      Cidade Alta                      18.648129
             Cidade Nova                      18.330298
             Cidade da Esperança              16.456373
             Dix-Sept Rosado                  13.425303
             Felipe Camarão                   19.080804
             Guarapes                         29.186014
             Igapó                            26.397928
             Lagoa Azul                       35.584846
             Lagoa Nova                       10.025544
             Lagoa Seca                       12.649849
             Mãe Luiza                        19.030349
             Neópolis                         15.638553
             Nordeste                         21.722573
             Nossa Senhora da Apresentação    30.821501
             Nossa Senhora de Nazaré          13.078994
             Nova Descoberta                   7.855589
             Pajuçara                         31.681509
             Petrópolis                       18.584851
             Pitimbu                          16.961316
             Planalto                         22.969254
             Ponta Negra                      17.274824
             Potengi                          29.555963
             Praia do Meio                    18.989021
             Quintas                          18.416456
             Redinha                          28.240914
             Ribeira                          21.036148
             Rocas                            21.135664
             Salinas                          29.224495
             Santos Reis                      22.062342
             Tirol                            18.004426
Name: mean_estimate, dtype: float64

Finally we show the plot of all the data collected showing the mean_estimate values by timestamp. In this plot is clear in which days the collector did not received data. Another thing that we noticed is how expensive a trip could be.


In [58]:
plt.figure(figsize=(20,10))
plt.plot(uberJsonRequests['timestamp'],uberJsonRequests['mean_estimate'])


Out[58]:
[<matplotlib.lines.Line2D at 0x114dfe940>]

In [60]:
plt.plot(uberJsonRequests[uberJsonRequests['high_estimate']>100]['timestamp'],uberJsonRequests[uberJsonRequests['high_estimate']>100]['mean_estimate'])


Out[60]:
[<matplotlib.lines.Line2D at 0x115305780>]

Trips


In [68]:
uberJsonRequests[uberJsonRequests['high_estimate']>200]


Out[68]:
id timestamp periodOfDay neighborhood start_latitude start_longitude finish_latitude finish_longitude currency_code distance duration high_estimate low_estimate product_id mean_estimate region
99817 369 2017-11-01 12:52:51.860501 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 206.0 167.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 186.5 North
99889 370 2017-11-01 12:55:12.018632 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 206.0 167.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 186.5 North
99961 371 2017-11-01 12:57:31.907471 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 210.0 171.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 190.5 North
100033 372 2017-11-01 12:59:52.505610 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 210.0 171.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 190.5 North
100105 373 2017-11-01 13:02:15.151039 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 204.0 166.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 185.0 North
108961 496 2017-11-01 17:57:05.452988 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2580 209.0 170.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 189.5 North
109033 497 2017-11-01 17:59:30.426915 afternoon Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2580 214.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 194.0 North
109105 498 2017-11-01 18:02:13.694415 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.04 2580 213.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.5 North
109177 499 2017-11-01 18:04:53.295493 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 216.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.5 North
109249 500 2017-11-01 18:07:26.490731 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 215.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.0 North
109321 501 2017-11-01 18:10:37.064569 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 215.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.0 North
109393 502 2017-11-01 18:13:12.681008 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.04 2580 213.0 173.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.0 North
109465 503 2017-11-01 18:15:45.438437 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 218.0 177.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 197.5 North
109537 504 2017-11-01 18:18:23.827898 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 207.0 169.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 188.0 North
109609 505 2017-11-01 18:21:16.986703 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 219.0 178.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 198.5 North
109681 506 2017-11-01 18:23:50.696097 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 219.0 178.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 198.5 North
109753 507 2017-11-01 18:26:27.677365 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.04 2580 220.0 179.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 199.5 North
109825 508 2017-11-01 18:29:10.507036 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 220.0 179.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 199.5 North
109897 509 2017-11-01 18:31:48.832010 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 221.0 180.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 200.5 North
109969 510 2017-11-01 18:34:24.312194 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 217.0 177.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 197.0 North
110041 511 2017-11-01 18:37:11.511977 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 221.0 180.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 200.5 North
110113 512 2017-11-01 18:40:24.032707 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 221.0 180.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 200.5 North
110185 513 2017-11-01 18:43:17.892861 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 220.0 179.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 199.5 North
110257 514 2017-11-01 18:45:48.369259 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 220.0 179.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 199.5 North
110329 515 2017-11-01 18:48:30.879965 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 220.0 179.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 199.5 North
110401 516 2017-11-01 18:51:53.031907 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 226.0 184.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 205.0 North
110473 517 2017-11-01 18:54:44.374608 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 226.0 184.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 205.0 North
110545 518 2017-11-01 18:57:47.644335 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 226.0 184.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 205.0 North
110617 519 2017-11-01 19:00:31.166237 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2700 226.0 184.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 205.0 North
110689 520 2017-11-01 19:03:07.947661 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 223.0 182.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 202.5 North
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
111481 531 2017-11-01 19:36:00.083900 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 227.0 185.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 206.0 North
111499 531 2017-11-01 19:36:27.821087 evening Salinas -5.8323 -35.2054 -5.763117 -35.247973 BRL 11.59 2100 207.0 168.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 187.5 North
111553 532 2017-11-01 19:39:50.972802 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 2160 213.0 173.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.0 North
114937 579 2017-11-01 22:08:36.919976 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 206.0 168.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 187.0 North
115009 580 2017-11-01 22:11:30.247592 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 210.0 171.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 190.5 North
115081 581 2017-11-01 22:14:20.764238 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 214.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 194.0 North
115153 582 2017-11-01 22:17:07.835770 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 214.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 194.0 North
115225 583 2017-11-01 22:19:59.965700 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 214.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 194.0 North
115297 584 2017-11-01 22:22:39.349734 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 10.80 1920 215.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.0 North
115369 585 2017-11-01 22:25:24.236570 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 215.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.0 North
115441 586 2017-11-01 22:27:54.648536 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 215.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.0 North
115513 587 2017-11-01 22:30:22.179242 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 215.0 175.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 195.0 North
115585 588 2017-11-01 22:32:40.323404 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 213.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.5 North
115657 589 2017-11-01 22:35:11.994823 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 213.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.5 North
115729 590 2017-11-01 22:37:40.268255 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 213.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.5 North
115801 591 2017-11-01 22:40:00.656350 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 213.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 193.5 North
115873 592 2017-11-01 22:42:17.464960 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 214.0 174.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 194.0 North
116017 594 2017-11-01 22:46:55.187212 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 12.07 1920 201.0 164.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 182.5 North
226789 844 2017-11-08 18:53:35.166078 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2760 210.0 171.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 190.5 North
226807 844 2017-11-08 18:53:51.657050 evening Salinas -5.8323 -35.2054 -5.763117 -35.247973 BRL 11.59 2400 202.0 164.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 183.0 North
226861 845 2017-11-08 18:55:59.038268 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2760 210.0 171.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 190.5 North
226879 845 2017-11-08 18:56:09.892976 evening Salinas -5.8323 -35.2054 -5.763117 -35.247973 BRL 11.59 2400 201.0 164.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 182.5 North
226933 846 2017-11-08 18:58:17.137254 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2760 210.0 171.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 190.5 North
226951 846 2017-11-08 18:58:28.774410 evening Salinas -5.8323 -35.2054 -5.763117 -35.247973 BRL 11.59 2400 202.0 164.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 183.0 North
227005 847 2017-11-08 19:00:38.084948 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2760 209.0 170.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 189.5 North
227077 848 2017-11-08 19:02:58.864120 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2400 204.0 166.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 185.0 North
227149 849 2017-11-08 19:05:19.505464 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2400 207.0 169.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 188.0 North
227221 850 2017-11-08 19:07:49.611879 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2400 207.0 169.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 188.0 North
227293 851 2017-11-08 19:10:16.253237 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2400 207.0 168.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 187.5 North
227437 853 2017-11-08 19:15:13.879536 evening Lagoa Azul -5.8323 -35.2054 -5.734177 -35.253802 BRL 11.72 2400 207.0 168.0 bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138 187.5 North

78 rows × 16 columns

Choropleth


After adjusting our data we now are prepared to prepare for our Choropleth.

Color map and Threshold


As we noticed we could make a set of values baxed on Uber products, Uber X and Uber Select, and another data based on the period of time, morning/afternoon/evening. For each Uber product or period of time we could use a colormap to see how the trips are distributed in terms of price.


In [10]:
colormap = linear.GnBu.scale(uberJsonRequests['low_estimate'].mean(), uberJsonRequests['high_estimate'].mean())
colormap


Out[10]:
19.85419335137290325.37250718896589
Uber Products

In [39]:
colorscaleUberX = linear.BuPu.scale(uberProducts['65cb1829-9761-40f8-acc6-92d700fe2924'].min(), uberProducts['65cb1829-9761-40f8-acc6-92d700fe2924'].max())
colorscaleUberX


Out[39]:
7.50672371638141833.206160414760596

In [40]:
colorscaleUberSelect = linear.BuPu.scale(uberProducts['bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138'].min(), uberProducts['bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138'].max())
colorscaleUberSelect


Out[40]:
10.2024755501222549.877706617871304
Period of day

In [50]:
colorscaleMorning = linear.BuGn.scale(uberPeriodOfDay['morning'].min(), uberPeriodOfDay['morning'].max())
colorscaleMorning


Out[50]:
7.85558927668184135.58484619263742

In [51]:
colorscaleAfternoon = linear.OrRd.scale(uberPeriodOfDay['afternoon'].min(), uberPeriodOfDay['afternoon'].max())
colorscaleAfternoon


Out[51]:
8.4425233644859840.92397388059702

In [52]:
colorscaleEvening = linear.PuBu.scale(uberPeriodOfDay['evening'].min(), uberPeriodOfDay['evening'].max())
colorscaleEvening


Out[52]:
11.74342105263157957.52105263157895

Observe the different bounds of the period of day. The high estimate in the period of evening (that goes from 6PM to 6AM) is about

61.86%

more expensive than the period of morning (6AM to noon). $$ \big({\dfrac{Max_{Morning}}{Max_{Evening}}}\big)*100 $$


In [46]:
# Create a map object
uberMap = folium.Map(
    location=[-5.826592, -35.212558],
    zoom_start=12,
    tiles='cartodbpositron', 
    height=800
)

# UberX
folium.GeoJson(
    geo_json_natal,
    name='UberX Price Estimates',
    style_function=lambda feature: {
        'fillColor': colorscaleUberX(uberProducts['65cb1829-9761-40f8-acc6-92d700fe2924'][feature['properties']['name']]),
        'color': 'blue',
        'weight': 0.5,
        'dashArray': '5, 5',
        'fillOpacity': 0.8,
        'name': feature['properties']['name']
    }
).add_to(uberMap)

# UberSelect
folium.GeoJson(
    geo_json_natal,
    name='UberSelect Price Estimates',
    style_function=lambda feature: {
        'fillColor': colorscaleUberSelect(uberProducts['bf8f99ca-f5f2-40d4-8ffc-52f1e2b17138'][feature['properties']['name']]),
        'color': 'blue',
        'weight': 0.5,
        'dashArray': '5, 5',
        'fillOpacity': 0.8,
        'name': feature['properties']['name']
    }
).add_to(uberMap)

colorscaleUberX.caption = 'UberX Price Estimatives'
colorscaleUberX.add_to(uberMap)
colorscaleUberSelect.caption = 'UberSelect Price Estimatives'
colorscaleUberSelect.add_to(uberMap)

folium.LayerControl().add_to(uberMap)

uberMap


Out[46]:

In [54]:
# Create a map object
uberMapPeriod = folium.Map(
    location=[-5.826592, -35.212558],
    zoom_start=12,
    tiles='cartodbpositron'
)

# Morning
folium.GeoJson(
    geo_json_natal,
    name='Morning Price Estimates',
    style_function=lambda feature: {
        'fillColor': colorscaleMorning(uberPeriodOfDay['morning'][feature['properties']['name']]),
        'color': 'green',
        'weight': 0.5,
        'dashArray': '1,1',
        'fillOpacity': 0.8,
        'name': feature['properties']['name']
    }
).add_to(uberMapPeriod)

# Afternoon
folium.GeoJson(
    geo_json_natal,
    name='Afternoon Price Estimates',
    style_function=lambda feature: {
        'fillColor': colorscaleAfternoon(uberPeriodOfDay['afternoon'][feature['properties']['name']]),
        'color': 'orange',
        'weight': 0.5,
        'dashArray': '1, 1',
        'fillOpacity': 0.8,
        'name': feature['properties']['name']
    }
).add_to(uberMapPeriod)

# Evening
folium.GeoJson(
    geo_json_natal,
    name='Afternoon Price Estimates',
    style_function=lambda feature: {
        'fillColor': colorscaleEvening(uberPeriodOfDay['evening'][feature['properties']['name']]),
        'color': 'black',
        'weight': 0.5,
        'dashArray': '1, 1',
        'fillOpacity': 0.8,
        'name': feature['properties']['name']
    }
).add_to(uberMapPeriod)

colorscaleMorning.caption = 'Morning Price Estimatives'
colorscaleMorning.add_to(uberMapPeriod)

colorscaleAfternoon.caption = 'Afternoon Price Estimatives'
colorscaleAfternoon.add_to(uberMapPeriod)

colorscaleEvening.caption = 'Evening Price Estimatives'
colorscaleEvening.add_to(uberMapPeriod)

folium.LayerControl().add_to(uberMapPeriod)

uberMapPeriod


Out[54]:

In [74]:
# Create a map object
uberMapGeneral = folium.Map(
    location=[-5.826592, -35.212558],
    zoom_start=12,
    tiles='cartodbpositron', 
    height=800
)


folium.GeoJson(
    geo_json_natal,
    name='UberX Price Estimates',
    style_function=lambda feature: {
        'fillColor': colormap(uberNeigh[feature['properties']['name']]),
        'color': 'white',
        'weight': 0.5,
        'dashArray': '1, 1',
        'fillOpacity': 0.8,
        'name': feature['properties']['name']
    }
).add_to(uberMapGeneral)

colormap.caption = 'General Price Estimatives'
colormap.add_to(uberMapGeneral)

folium.LayerControl().add_to(uberMapGeneral)

for destinationPoint in uberNeigh:
    folium.Marker()
folium.Marker([-5.796158, -35.216566], popup='Alecrim, Valor médio: 21.177857').add_to(uberMapGeneral)
folium.Marker([-5.789477, -35.188454], popup='Areia Preta, Valor médio: 23.977857').add_to(uberMapGeneral)
folium.Marker([-5.800741, -35.211056], popup='Barro Vermelho, Valor médio: 18.666429').add_to(uberMapGeneral)
folium.Marker([-5.813764, -35.240721], popup='Bom Pastor, Valor médio: 19.228393').add_to(uberMapGeneral)
folium.Marker([-5.841737, -35.210463], popup='Candelária, Valor médio: 11.591687').add_to(uberMapGeneral)
folium.Marker([-5.862753, -35.195597], popup='Capim Macio, Valor médio: 16.732264').add_to(uberMapGeneral)
folium.Marker([-5.785291, -35.206464], popup='Cidade Alta, Valor médio: 23.384464').add_to(uberMapGeneral)
folium.Marker([-5.825376, -35.242751], popup='Cidade da Esperança, Valor médio: 23.109185').add_to(uberMapGeneral)
folium.Marker([-5.834856, -35.242523], popup='Cidade Nova, Valor médio: 20.746426').add_to(uberMapGeneral)
folium.Marker([-5.812036, -35.223915], popup='Dix-Sept Rosado, Valor médio: 16.956429').add_to(uberMapGeneral)
folium.Marker([-5.824374, -35.250070], popup='Felipe Camarão, Valor médio: 23.978914').add_to(uberMapGeneral)
folium.Marker([-5.841580, -35.274691], popup='Guarapes, Valor médio: 36.411187').add_to(uberMapGeneral)
folium.Marker([-5.769009, -35.254755], popup='Igapó, Valor médio: 33.120807').add_to(uberMapGeneral)
folium.Marker([-5.734177, -35.253802], popup='Lagoa Azul, Valor médio: 45.140514').add_to(uberMapGeneral)
folium.Marker([-5.819743, -35.212920], popup='Lagoa Nova, Valor médio: 12.775357').add_to(uberMapGeneral)
folium.Marker([-5.809191, -35.209374], popup='Lagoa Seca, Valor médio: 15.975205').add_to(uberMapGeneral)
folium.Marker([-5.794771, -35.188619], popup='Mãe Luiza, Valor médio: 23.746786').add_to(uberMapGeneral)
folium.Marker([-5.870513, -35.208176], popup='Neópolis, Valor médio: 19.888275').add_to(uberMapGeneral)
folium.Marker([-5.796215, -35.245141], popup='Nordeste, Valor médio: 27.215000').add_to(uberMapGeneral)
folium.Marker([-5.763654, -35.282543], popup='Nossa Senhora da Apresentação, Valor médio: 38.594575').add_to(uberMapGeneral)
folium.Marker([-5.815939, -35.229249], popup='Nossa Senhora de Nazaré, Valor médio: 16.518571').add_to(uberMapGeneral)
folium.Marker([-5.824830, -35.200026], popup='Nova Descoberta, Valor médio: 9.996964').add_to(uberMapGeneral)
folium.Marker([-5.751781, -35.234665], popup='Pajuçara, Valor médio: 39.676061').add_to(uberMapGeneral)
folium.Marker([-5.782001, -35.195196], popup='Petrópolis, Valor médio: 23.243929').add_to(uberMapGeneral)
folium.Marker([-5.876271, -35.224500], popup='Pitimbu, Valor médio: 21.161262').add_to(uberMapGeneral)
folium.Marker([-5.858102, -35.251586], popup='Planalto, Valor médio: 28.573770').add_to(uberMapGeneral)
folium.Marker([-5.877522, -35.176073], popup='Ponta Negra, Valor médio: 21.856557').add_to(uberMapGeneral)
folium.Marker([-5.758634, -35.247010], popup='Potengi, Valor médio: 37.519636').add_to(uberMapGeneral)
folium.Marker([-5.779198, -35.197163], popup='Praia do Meio, Valor médio: 23.750624').add_to(uberMapGeneral)
folium.Marker([-5.797290, -35.226006], popup='Quintas, Valor médio: 22.788750').add_to(uberMapGeneral)
folium.Marker([-5.742772, -35.205806], popup='Redinha, Valor médio: 35.059579').add_to(uberMapGeneral)
folium.Marker([-5.774943, -35.205578], popup='Ribeira, Valor médio: 26.279643').add_to(uberMapGeneral)
folium.Marker([-5.771832, -35.203091], popup='Rocas, Valor médio: 26.428826').add_to(uberMapGeneral)
folium.Marker([-5.763117, -35.247973], popup='Salinas, Valor médio: 37.037460').add_to(uberMapGeneral)
folium.Marker([-5.763226, -35.196786], popup='Santos Reis, Valor médio: 27.564395').add_to(uberMapGeneral)
folium.Marker([-5.791699, -35.197358], popup='Tirol, Valor médio: 22.539286').add_to(uberMapGeneral)

uberMapGeneral


Out[74]: