In [279]:
MovieTextFile = open("tmdb_5000_movies.csv")
# for line in MovieTextFile:
# print(line) # not quite right
# type(MovieTextFile)
In [180]:
import csv
with open("tmdb_5000_movies.csv",encoding="utf8") as f:
reader = csv.reader(f)
MovieList = list(reader)
MovieList[:5]
Out[180]:
[['budget',
'genres',
'homepage',
'id',
'keywords',
'original_language',
'original_title',
'overview',
'popularity',
'production_companies',
'production_countries',
'release_date',
'revenue',
'runtime',
'spoken_languages',
'status',
'tagline',
'title',
'vote_average',
'vote_count'],
['237000000',
'[{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 14, "name": "Fantasy"}, {"id": 878, "name": "Science Fiction"}]',
'http://www.avatarmovie.com/',
'19995',
'[{"id": 1463, "name": "culture clash"}, {"id": 2964, "name": "future"}, {"id": 3386, "name": "space war"}, {"id": 3388, "name": "space colony"}, {"id": 3679, "name": "society"}, {"id": 3801, "name": "space travel"}, {"id": 9685, "name": "futuristic"}, {"id": 9840, "name": "romance"}, {"id": 9882, "name": "space"}, {"id": 9951, "name": "alien"}, {"id": 10148, "name": "tribe"}, {"id": 10158, "name": "alien planet"}, {"id": 10987, "name": "cgi"}, {"id": 11399, "name": "marine"}, {"id": 13065, "name": "soldier"}, {"id": 14643, "name": "battle"}, {"id": 14720, "name": "love affair"}, {"id": 165431, "name": "anti war"}, {"id": 193554, "name": "power relations"}, {"id": 206690, "name": "mind and soul"}, {"id": 209714, "name": "3d"}]',
'en',
'Avatar',
'In the 22nd century, a paraplegic Marine is dispatched to the moon Pandora on a unique mission, but becomes torn between following orders and protecting an alien civilization.',
'150.437577',
'[{"name": "Ingenious Film Partners", "id": 289}, {"name": "Twentieth Century Fox Film Corporation", "id": 306}, {"name": "Dune Entertainment", "id": 444}, {"name": "Lightstorm Entertainment", "id": 574}]',
'[{"iso_3166_1": "US", "name": "United States of America"}, {"iso_3166_1": "GB", "name": "United Kingdom"}]',
'2009-12-10',
'2787965087',
'162',
'[{"iso_639_1": "en", "name": "English"}, {"iso_639_1": "es", "name": "Espa\\u00f1ol"}]',
'Released',
'Enter the World of Pandora.',
'Avatar',
'7.2',
'11800'],
['300000000',
'[{"id": 12, "name": "Adventure"}, {"id": 14, "name": "Fantasy"}, {"id": 28, "name": "Action"}]',
'http://disney.go.com/disneypictures/pirates/',
'285',
'[{"id": 270, "name": "ocean"}, {"id": 726, "name": "drug abuse"}, {"id": 911, "name": "exotic island"}, {"id": 1319, "name": "east india trading company"}, {"id": 2038, "name": "love of one\'s life"}, {"id": 2052, "name": "traitor"}, {"id": 2580, "name": "shipwreck"}, {"id": 2660, "name": "strong woman"}, {"id": 3799, "name": "ship"}, {"id": 5740, "name": "alliance"}, {"id": 5941, "name": "calypso"}, {"id": 6155, "name": "afterlife"}, {"id": 6211, "name": "fighter"}, {"id": 12988, "name": "pirate"}, {"id": 157186, "name": "swashbuckler"}, {"id": 179430, "name": "aftercreditsstinger"}]',
'en',
"Pirates of the Caribbean: At World's End",
'Captain Barbossa, long believed to be dead, has come back to life and is headed to the edge of the Earth with Will Turner and Elizabeth Swann. But nothing is quite as it seems.',
'139.082615',
'[{"name": "Walt Disney Pictures", "id": 2}, {"name": "Jerry Bruckheimer Films", "id": 130}, {"name": "Second Mate Productions", "id": 19936}]',
'[{"iso_3166_1": "US", "name": "United States of America"}]',
'2007-05-19',
'961000000',
'169',
'[{"iso_639_1": "en", "name": "English"}]',
'Released',
'At the end of the world, the adventure begins.',
"Pirates of the Caribbean: At World's End",
'6.9',
'4500'],
['245000000',
'[{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 80, "name": "Crime"}]',
'http://www.sonypictures.com/movies/spectre/',
'206647',
'[{"id": 470, "name": "spy"}, {"id": 818, "name": "based on novel"}, {"id": 4289, "name": "secret agent"}, {"id": 9663, "name": "sequel"}, {"id": 14555, "name": "mi6"}, {"id": 156095, "name": "british secret service"}, {"id": 158431, "name": "united kingdom"}]',
'en',
'Spectre',
'A cryptic message from Bond’s past sends him on a trail to uncover a sinister organization. While M battles political forces to keep the secret service alive, Bond peels back the layers of deceit to reveal the terrible truth behind SPECTRE.',
'107.376788',
'[{"name": "Columbia Pictures", "id": 5}, {"name": "Danjaq", "id": 10761}, {"name": "B24", "id": 69434}]',
'[{"iso_3166_1": "GB", "name": "United Kingdom"}, {"iso_3166_1": "US", "name": "United States of America"}]',
'2015-10-26',
'880674609',
'148',
'[{"iso_639_1": "fr", "name": "Fran\\u00e7ais"}, {"iso_639_1": "en", "name": "English"}, {"iso_639_1": "es", "name": "Espa\\u00f1ol"}, {"iso_639_1": "it", "name": "Italiano"}, {"iso_639_1": "de", "name": "Deutsch"}]',
'Released',
'A Plan No One Escapes',
'Spectre',
'6.3',
'4466'],
['250000000',
'[{"id": 28, "name": "Action"}, {"id": 80, "name": "Crime"}, {"id": 18, "name": "Drama"}, {"id": 53, "name": "Thriller"}]',
'http://www.thedarkknightrises.com/',
'49026',
'[{"id": 849, "name": "dc comics"}, {"id": 853, "name": "crime fighter"}, {"id": 949, "name": "terrorist"}, {"id": 1308, "name": "secret identity"}, {"id": 1437, "name": "burglar"}, {"id": 3051, "name": "hostage drama"}, {"id": 3562, "name": "time bomb"}, {"id": 6969, "name": "gotham city"}, {"id": 7002, "name": "vigilante"}, {"id": 9665, "name": "cover-up"}, {"id": 9715, "name": "superhero"}, {"id": 9990, "name": "villainess"}, {"id": 10044, "name": "tragic hero"}, {"id": 13015, "name": "terrorism"}, {"id": 14796, "name": "destruction"}, {"id": 18933, "name": "catwoman"}, {"id": 156082, "name": "cat burglar"}, {"id": 156395, "name": "imax"}, {"id": 173272, "name": "flood"}, {"id": 179093, "name": "criminal underworld"}, {"id": 230775, "name": "batman"}]',
'en',
'The Dark Knight Rises',
"Following the death of District Attorney Harvey Dent, Batman assumes responsibility for Dent's crimes to protect the late attorney's reputation and is subsequently hunted by the Gotham City Police Department. Eight years later, Batman encounters the mysterious Selina Kyle and the villainous Bane, a new terrorist leader who overwhelms Gotham's finest. The Dark Knight resurfaces to protect a city that has branded him an enemy.",
'112.31295',
'[{"name": "Legendary Pictures", "id": 923}, {"name": "Warner Bros.", "id": 6194}, {"name": "DC Entertainment", "id": 9993}, {"name": "Syncopy", "id": 9996}]',
'[{"iso_3166_1": "US", "name": "United States of America"}]',
'2012-07-16',
'1084939099',
'165',
'[{"iso_639_1": "en", "name": "English"}]',
'Released',
'The Legend Ends',
'The Dark Knight Rises',
'7.6',
'9106']]
In [173]:
import pandas as pd
movies = pd.read_csv("tmdb_5000_movies.csv")
movieFrame = pd.DataFrame(movies)
movieFrame[:5]
Out[173]:
budget
genres
homepage
id
keywords
original_language
original_title
overview
popularity
production_companies
production_countries
release_date
revenue
runtime
spoken_languages
status
tagline
title
vote_average
vote_count
0
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
[{"iso_3166_1": "US", "name": "United States o...
2009-12-10
2787965087
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
1
300000000
[{"id": 12, "name": "Adventure"}, {"id": 14, "...
http://disney.go.com/disneypictures/pirates/
285
[{"id": 270, "name": "ocean"}, {"id": 726, "na...
en
Pirates of the Caribbean: At World's End
Captain Barbossa, long believed to be dead, ha...
139.082615
[{"name": "Walt Disney Pictures", "id": 2}, {"...
[{"iso_3166_1": "US", "name": "United States o...
2007-05-19
961000000
169.0
[{"iso_639_1": "en", "name": "English"}]
Released
At the end of the world, the adventure begins.
Pirates of the Caribbean: At World's End
6.9
4500
2
245000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.sonypictures.com/movies/spectre/
206647
[{"id": 470, "name": "spy"}, {"id": 818, "name...
en
Spectre
A cryptic message from Bond’s past sends him o...
107.376788
[{"name": "Columbia Pictures", "id": 5}, {"nam...
[{"iso_3166_1": "GB", "name": "United Kingdom"...
2015-10-26
880674609
148.0
[{"iso_639_1": "fr", "name": "Fran\u00e7ais"},...
Released
A Plan No One Escapes
Spectre
6.3
4466
3
250000000
[{"id": 28, "name": "Action"}, {"id": 80, "nam...
http://www.thedarkknightrises.com/
49026
[{"id": 849, "name": "dc comics"}, {"id": 853,...
en
The Dark Knight Rises
Following the death of District Attorney Harve...
112.312950
[{"name": "Legendary Pictures", "id": 923}, {"...
[{"iso_3166_1": "US", "name": "United States o...
2012-07-16
1084939099
165.0
[{"iso_639_1": "en", "name": "English"}]
Released
The Legend Ends
The Dark Knight Rises
7.6
9106
4
260000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://movies.disney.com/john-carter
49529
[{"id": 818, "name": "based on novel"}, {"id":...
en
John Carter
John Carter is a war-weary, former military ca...
43.926995
[{"name": "Walt Disney Pictures", "id": 2}]
[{"iso_3166_1": "US", "name": "United States o...
2012-03-07
284139100
132.0
[{"iso_639_1": "en", "name": "English"}]
Released
Lost in our world, found in another.
John Carter
6.1
2124
In [174]:
#pull out genres array of JSON strings from data frame
genres = movieFrame['genres']
# genresFrame = pd.DataFrame(genres)
genres[:5]
Out[174]:
0 [{"id": 28, "name": "Action"}, {"id": 12, "nam...
1 [{"id": 12, "name": "Adventure"}, {"id": 14, "...
2 [{"id": 28, "name": "Action"}, {"id": 12, "nam...
3 [{"id": 28, "name": "Action"}, {"id": 80, "nam...
4 [{"id": 28, "name": "Action"}, {"id": 12, "nam...
Name: genres, dtype: object
In [175]:
# Pull out list of names for each row of the data frame
# Start with testing first row and iterating through JSON string
import json
genreList = []
genre = json.loads(genres[0])
for i,val in enumerate(genre):
genreList.append(genre[i]['name'])
genreList
Out[175]:
['Action', 'Adventure', 'Fantasy', 'Science Fiction']
In [197]:
# Iterate through indices of genre array to create a list of lists of genre names
import json
genresAll = []
for k,x in enumerate(genres):
genreList = []
genre = json.loads(genres[k])
for i,val in enumerate(genre):
genreList.append(genre[i]['name'])
genresAll.append(genreList)
genresAll[:10]
Out[197]:
[['Action', 'Adventure', 'Fantasy', 'Science Fiction'],
['Adventure', 'Fantasy', 'Action'],
['Action', 'Adventure', 'Crime'],
['Action', 'Crime', 'Drama', 'Thriller'],
['Action', 'Adventure', 'Science Fiction'],
['Fantasy', 'Action', 'Adventure'],
['Animation', 'Family'],
['Action', 'Adventure', 'Science Fiction'],
['Adventure', 'Fantasy', 'Family'],
['Action', 'Adventure', 'Fantasy']]
In [241]:
genreSeries['W'] = pd.Series(genresAll,index=movieFrame.index)
genreFrame = pd.DataFrame(genreSeries['W'],columns=['GenreList'])
genreFrame[:5]
Out[241]:
GenreList
0
[Action, Adventure, Fantasy, Science Fiction]
1
[Adventure, Fantasy, Action]
2
[Action, Adventure, Crime]
3
[Action, Crime, Drama, Thriller]
4
[Action, Adventure, Science Fiction]
In [244]:
genreDummies = genreFrame.GenreList.astype(str).str.strip('[]').str.get_dummies(', ')
genreDummies[:10]
Out[244]:
'Action'
'Adventure'
'Animation'
'Comedy'
'Crime'
'Documentary'
'Drama'
'Family'
'Fantasy'
'Foreign'
'History'
'Horror'
'Music'
'Mystery'
'Romance'
'Science Fiction'
'TV Movie'
'Thriller'
'War'
'Western'
0
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
1
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
2
1
1
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3
1
0
0
0
1
0
1
0
0
0
0
0
0
0
0
0
0
1
0
0
4
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
5
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
6
0
0
1
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
7
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
8
0
1
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
0
9
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
In [248]:
# append lists as a column at end of dataframe
movieGenreFrame = pd.merge(movieFrame,genreFrame,how='inner',left_index=True, right_index=True)
movieGenreFrame[:5]
Out[248]:
budget
genres
homepage
id
keywords
original_language
original_title
overview
popularity
production_companies
...
release_date
revenue
runtime
spoken_languages
status
tagline
title
vote_average
vote_count
GenreList
0
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
2009-12-10
2787965087
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
1
300000000
[{"id": 12, "name": "Adventure"}, {"id": 14, "...
http://disney.go.com/disneypictures/pirates/
285
[{"id": 270, "name": "ocean"}, {"id": 726, "na...
en
Pirates of the Caribbean: At World's End
Captain Barbossa, long believed to be dead, ha...
139.082615
[{"name": "Walt Disney Pictures", "id": 2}, {"...
...
2007-05-19
961000000
169.0
[{"iso_639_1": "en", "name": "English"}]
Released
At the end of the world, the adventure begins.
Pirates of the Caribbean: At World's End
6.9
4500
[Adventure, Fantasy, Action]
2
245000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.sonypictures.com/movies/spectre/
206647
[{"id": 470, "name": "spy"}, {"id": 818, "name...
en
Spectre
A cryptic message from Bond’s past sends him o...
107.376788
[{"name": "Columbia Pictures", "id": 5}, {"nam...
...
2015-10-26
880674609
148.0
[{"iso_639_1": "fr", "name": "Fran\u00e7ais"},...
Released
A Plan No One Escapes
Spectre
6.3
4466
[Action, Adventure, Crime]
3
250000000
[{"id": 28, "name": "Action"}, {"id": 80, "nam...
http://www.thedarkknightrises.com/
49026
[{"id": 849, "name": "dc comics"}, {"id": 853,...
en
The Dark Knight Rises
Following the death of District Attorney Harve...
112.312950
[{"name": "Legendary Pictures", "id": 923}, {"...
...
2012-07-16
1084939099
165.0
[{"iso_639_1": "en", "name": "English"}]
Released
The Legend Ends
The Dark Knight Rises
7.6
9106
[Action, Crime, Drama, Thriller]
4
260000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://movies.disney.com/john-carter
49529
[{"id": 818, "name": "based on novel"}, {"id":...
en
John Carter
John Carter is a war-weary, former military ca...
43.926995
[{"name": "Walt Disney Pictures", "id": 2}]
...
2012-03-07
284139100
132.0
[{"iso_639_1": "en", "name": "English"}]
Released
Lost in our world, found in another.
John Carter
6.1
2124
[Action, Adventure, Science Fiction]
5 rows × 21 columns
In [249]:
wideMovieFrame = pd.merge(movieGenreFrame,genreDummies,how='inner',left_index=True,right_index=True)
wideMovieFrame[:5]
Out[249]:
budget
genres
homepage
id
keywords
original_language
original_title
overview
popularity
production_companies
...
'History'
'Horror'
'Music'
'Mystery'
'Romance'
'Science Fiction'
'TV Movie'
'Thriller'
'War'
'Western'
0
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
0
0
0
0
0
1
0
0
0
0
1
300000000
[{"id": 12, "name": "Adventure"}, {"id": 14, "...
http://disney.go.com/disneypictures/pirates/
285
[{"id": 270, "name": "ocean"}, {"id": 726, "na...
en
Pirates of the Caribbean: At World's End
Captain Barbossa, long believed to be dead, ha...
139.082615
[{"name": "Walt Disney Pictures", "id": 2}, {"...
...
0
0
0
0
0
0
0
0
0
0
2
245000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.sonypictures.com/movies/spectre/
206647
[{"id": 470, "name": "spy"}, {"id": 818, "name...
en
Spectre
A cryptic message from Bond’s past sends him o...
107.376788
[{"name": "Columbia Pictures", "id": 5}, {"nam...
...
0
0
0
0
0
0
0
0
0
0
3
250000000
[{"id": 28, "name": "Action"}, {"id": 80, "nam...
http://www.thedarkknightrises.com/
49026
[{"id": 849, "name": "dc comics"}, {"id": 853,...
en
The Dark Knight Rises
Following the death of District Attorney Harve...
112.312950
[{"name": "Legendary Pictures", "id": 923}, {"...
...
0
0
0
0
0
0
0
1
0
0
4
260000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://movies.disney.com/john-carter
49529
[{"id": 818, "name": "based on novel"}, {"id":...
en
John Carter
John Carter is a war-weary, former military ca...
43.926995
[{"name": "Walt Disney Pictures", "id": 2}]
...
0
0
0
0
0
1
0
0
0
0
5 rows × 41 columns
In [276]:
longMovieFrame = pd.melt(wideMovieFrame, id_vars=movieGenreFrame.columns, value_vars=genreDummies.columns,
var_name='Genre',value_name="genre_present")
longMovieFrame[:10]
Out[276]:
budget
genres
homepage
id
keywords
original_language
original_title
overview
popularity
production_companies
...
runtime
spoken_languages
status
tagline
title
vote_average
vote_count
GenreList
Genre
genre_present
0
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Action'
1
1
300000000
[{"id": 12, "name": "Adventure"}, {"id": 14, "...
http://disney.go.com/disneypictures/pirates/
285
[{"id": 270, "name": "ocean"}, {"id": 726, "na...
en
Pirates of the Caribbean: At World's End
Captain Barbossa, long believed to be dead, ha...
139.082615
[{"name": "Walt Disney Pictures", "id": 2}, {"...
...
169.0
[{"iso_639_1": "en", "name": "English"}]
Released
At the end of the world, the adventure begins.
Pirates of the Caribbean: At World's End
6.9
4500
[Adventure, Fantasy, Action]
'Action'
1
2
245000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.sonypictures.com/movies/spectre/
206647
[{"id": 470, "name": "spy"}, {"id": 818, "name...
en
Spectre
A cryptic message from Bond’s past sends him o...
107.376788
[{"name": "Columbia Pictures", "id": 5}, {"nam...
...
148.0
[{"iso_639_1": "fr", "name": "Fran\u00e7ais"},...
Released
A Plan No One Escapes
Spectre
6.3
4466
[Action, Adventure, Crime]
'Action'
1
3
250000000
[{"id": 28, "name": "Action"}, {"id": 80, "nam...
http://www.thedarkknightrises.com/
49026
[{"id": 849, "name": "dc comics"}, {"id": 853,...
en
The Dark Knight Rises
Following the death of District Attorney Harve...
112.312950
[{"name": "Legendary Pictures", "id": 923}, {"...
...
165.0
[{"iso_639_1": "en", "name": "English"}]
Released
The Legend Ends
The Dark Knight Rises
7.6
9106
[Action, Crime, Drama, Thriller]
'Action'
1
4
260000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://movies.disney.com/john-carter
49529
[{"id": 818, "name": "based on novel"}, {"id":...
en
John Carter
John Carter is a war-weary, former military ca...
43.926995
[{"name": "Walt Disney Pictures", "id": 2}]
...
132.0
[{"iso_639_1": "en", "name": "English"}]
Released
Lost in our world, found in another.
John Carter
6.1
2124
[Action, Adventure, Science Fiction]
'Action'
1
5
258000000
[{"id": 14, "name": "Fantasy"}, {"id": 28, "na...
http://www.sonypictures.com/movies/spider-man3/
559
[{"id": 851, "name": "dual identity"}, {"id": ...
en
Spider-Man 3
The seemingly invincible Spider-Man goes up ag...
115.699814
[{"name": "Columbia Pictures", "id": 5}, {"nam...
...
139.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
The battle within.
Spider-Man 3
5.9
3576
[Fantasy, Action, Adventure]
'Action'
1
6
260000000
[{"id": 16, "name": "Animation"}, {"id": 10751...
http://disney.go.com/disneypictures/tangled/
38757
[{"id": 1562, "name": "hostage"}, {"id": 2343,...
en
Tangled
When the kingdom's most wanted-and most charmi...
48.681969
[{"name": "Walt Disney Pictures", "id": 2}, {"...
...
100.0
[{"iso_639_1": "en", "name": "English"}]
Released
They're taking adventure to new lengths.
Tangled
7.4
3330
[Animation, Family]
'Action'
0
7
280000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://marvel.com/movies/movie/193/avengers_ag...
99861
[{"id": 8828, "name": "marvel comic"}, {"id": ...
en
Avengers: Age of Ultron
When Tony Stark tries to jumpstart a dormant p...
134.279229
[{"name": "Marvel Studios", "id": 420}, {"name...
...
141.0
[{"iso_639_1": "en", "name": "English"}]
Released
A New Age Has Come.
Avengers: Age of Ultron
7.3
6767
[Action, Adventure, Science Fiction]
'Action'
1
8
250000000
[{"id": 12, "name": "Adventure"}, {"id": 14, "...
http://harrypotter.warnerbros.com/harrypottera...
767
[{"id": 616, "name": "witch"}, {"id": 2343, "n...
en
Harry Potter and the Half-Blood Prince
As Harry begins his sixth year at Hogwarts, he...
98.885637
[{"name": "Warner Bros.", "id": 6194}, {"name"...
...
153.0
[{"iso_639_1": "en", "name": "English"}]
Released
Dark Secrets Revealed
Harry Potter and the Half-Blood Prince
7.4
5293
[Adventure, Fantasy, Family]
'Action'
0
9
250000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.batmanvsupermandawnofjustice.com/
209112
[{"id": 849, "name": "dc comics"}, {"id": 7002...
en
Batman v Superman: Dawn of Justice
Fearing the actions of a god-like Super Hero l...
155.790452
[{"name": "DC Comics", "id": 429}, {"name": "A...
...
151.0
[{"iso_639_1": "en", "name": "English"}]
Released
Justice or revenge
Batman v Superman: Dawn of Justice
5.7
7004
[Action, Adventure, Fantasy]
'Action'
1
10 rows × 23 columns
In [277]:
# test results with 'Avatar' example
longMovieFrame[longMovieFrame['title']=='Avatar']
Out[277]:
budget
genres
homepage
id
keywords
original_language
original_title
overview
popularity
production_companies
...
runtime
spoken_languages
status
tagline
title
vote_average
vote_count
GenreList
Genre
genre_present
0
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Action'
1
4803
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Adventure'
1
9606
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Animation'
0
14409
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Comedy'
0
19212
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Crime'
0
24015
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Documentary'
0
28818
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Drama'
0
33621
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Family'
0
38424
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Fantasy'
1
43227
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Foreign'
0
48030
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'History'
0
52833
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Horror'
0
57636
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Music'
0
62439
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Mystery'
0
67242
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Romance'
0
72045
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Science Fiction'
1
76848
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'TV Movie'
0
81651
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Thriller'
0
86454
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'War'
0
91257
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Western'
0
20 rows × 23 columns
In [278]:
# If only retaining "true" genres
longMovieFrameTrimmed = longMovieFrame[longMovieFrame['genre_present']==1]
longMovieFrameTrimmed[:5]
Out[278]:
budget
genres
homepage
id
keywords
original_language
original_title
overview
popularity
production_companies
...
runtime
spoken_languages
status
tagline
title
vote_average
vote_count
GenreList
Genre
genre_present
0
237000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.avatarmovie.com/
19995
[{"id": 1463, "name": "culture clash"}, {"id":...
en
Avatar
In the 22nd century, a paraplegic Marine is di...
150.437577
[{"name": "Ingenious Film Partners", "id": 289...
...
162.0
[{"iso_639_1": "en", "name": "English"}, {"iso...
Released
Enter the World of Pandora.
Avatar
7.2
11800
[Action, Adventure, Fantasy, Science Fiction]
'Action'
1
1
300000000
[{"id": 12, "name": "Adventure"}, {"id": 14, "...
http://disney.go.com/disneypictures/pirates/
285
[{"id": 270, "name": "ocean"}, {"id": 726, "na...
en
Pirates of the Caribbean: At World's End
Captain Barbossa, long believed to be dead, ha...
139.082615
[{"name": "Walt Disney Pictures", "id": 2}, {"...
...
169.0
[{"iso_639_1": "en", "name": "English"}]
Released
At the end of the world, the adventure begins.
Pirates of the Caribbean: At World's End
6.9
4500
[Adventure, Fantasy, Action]
'Action'
1
2
245000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://www.sonypictures.com/movies/spectre/
206647
[{"id": 470, "name": "spy"}, {"id": 818, "name...
en
Spectre
A cryptic message from Bond’s past sends him o...
107.376788
[{"name": "Columbia Pictures", "id": 5}, {"nam...
...
148.0
[{"iso_639_1": "fr", "name": "Fran\u00e7ais"},...
Released
A Plan No One Escapes
Spectre
6.3
4466
[Action, Adventure, Crime]
'Action'
1
3
250000000
[{"id": 28, "name": "Action"}, {"id": 80, "nam...
http://www.thedarkknightrises.com/
49026
[{"id": 849, "name": "dc comics"}, {"id": 853,...
en
The Dark Knight Rises
Following the death of District Attorney Harve...
112.312950
[{"name": "Legendary Pictures", "id": 923}, {"...
...
165.0
[{"iso_639_1": "en", "name": "English"}]
Released
The Legend Ends
The Dark Knight Rises
7.6
9106
[Action, Crime, Drama, Thriller]
'Action'
1
4
260000000
[{"id": 28, "name": "Action"}, {"id": 12, "nam...
http://movies.disney.com/john-carter
49529
[{"id": 818, "name": "based on novel"}, {"id":...
en
John Carter
John Carter is a war-weary, former military ca...
43.926995
[{"name": "Walt Disney Pictures", "id": 2}]
...
132.0
[{"iso_639_1": "en", "name": "English"}]
Released
Lost in our world, found in another.
John Carter
6.1
2124
[Action, Adventure, Science Fiction]
'Action'
1
5 rows × 23 columns
I think the melted data is duplicating in the context of data storage, as much more storage is required to create a separate row for each possible genre of the movies data.
However, there is not data duplication in the context of the table design for analysis, because each row has a unique combination of attributes.
Content source: Fifth-Cohort-Awesome/NightThree
Similar notebooks: