Prepare to submit a job to Mechanical Turk

Used Google Sheets to list the attributes (100 of them) and movies (top 250 popular ones from IMDb), copy those to csv files manually, and use this script to generate a file with inputs for Mechanical Turk

script by Stephen Becker, June 9--12 2017


In [2]:
import csv

Read in the list of 100 questions, as a list of 25 sub-lists, 4 questions per sub-list


In [80]:
# string info: http://www.openbookproject.net/books/bpp4awd/ch03.html
# [lists] are mutable, (tuples) and 'strings' are not

# Read in the list of 100 questions, putting it into 25 groups of 4
questions = []
rowQuestions= []
with open('questions.csv', 'rb') as csvfile:
    myreader = csv.reader(csvfile)
    for index,row in enumerate(myreader):
        rowQuestions.append( row[0].rstrip() )
        if index%4 is 3:
            #print index, ' '.join(row)
            #print index, rowQuestions
            questions.append( rowQuestions )
            rowQuestions = []
len(questions)


Out[80]:
25

Read in all 250 movies


In [36]:
# Read in the list of 250 movies, making sure to remove commas from their names
# (actually, if it has commas, it will be read in as different fields)
movies = []
with open('movies.csv','rb') as csvfile:
    myreader = csv.reader(csvfile)
    for index, row in enumerate(myreader):
        movies.append( ' '.join(row) ) # the join() call merges all fields

Write an output file to be used as the input file for Amazon Mechanical Turk. Each row will be one HIT


In [61]:
N = len(movies)
with open('input.csv', 'wb') as csvfile:
    mywriter = csv.writer(csvfile)
    mywriter.writerow( ['MOVIE','QUESTION1','QUESTION2','QUESTION3','QUESTION4'])
    for i in range(5):
        for q in questions:
            mywriter.writerow( [movies[i], q[0], q[1], q[2], q[3] ])
            #mywriter.writerow( [movies[i]+','+','.join(q)] ) # has extra " "

After submitting to Mechanical Turk...

Read in the results. Note that the order of the questions is the same as the input file, so we can use that to simplify recording the answers. Let's encode 1 = Yes, 2 = No, 0 = Unsure


In [98]:
with open('Batch_2832525_batch_results.csv', 'rb') as csvfile:
    myreader = csv.DictReader(csvfile)
    #myreader = csv.reader(csvfile)
    # see dir(myreader) to list available methods
    for row in myreader:
        #print row
        print row['Input.MOVIE'] +": " + row['Input.QUESTION1'] , row['Answer.MovieAnswer1']
        print '               ' + row['Input.QUESTION2'] , row['Answer.MovieAnswer2']
        print '               ' + row['Input.QUESTION3'] , row['Answer.MovieAnswer3']
        print '               ' + row['Input.QUESTION4'] , row['Answer.MovieAnswer4']


The Shawshank Redemption (1994): Are there criminals in it? Yes
               Are witches part of the story? No
               Did it inspire a line of toys? No
               Did it make you cry? No
The Shawshank Redemption (1994): Did it make you jump? No
               Did it spawn a catch phrase? No
               Did it stir up controversy? No
               Did it win a Best Picture award? Unsure
The Shawshank Redemption (1994): Did you like the ending? Yes
               Do you think it's funny? No
               Does it deal with racism? Yes
               Does it explore fringe culture? Yes
The Shawshank Redemption (1994): Does it feature a determined female character? No
               Does it feature dancing? No
               Does it feature memorable music? No
               Does it have a cult following? No
The Shawshank Redemption (1994): Does it have a famous cast? Yes
               Does it have a famous director? No
               Does it have a happy ending? Yes
               Does it have a lot of nudity? No
The Shawshank Redemption (1994): Does it have a memorable quote? Unsure
               Does it have a popular soundtrack? Unsure
               Does it have a sequel? No
               Does it have an award winning character? Yes
The Shawshank Redemption (1994): Does it have Mob connections? No
               Does it have wacky characters? Yes
               Does it involve a murder? Yes
               Does it involve advanced technology? No
The Shawshank Redemption (1994): Does it involve aliens? No
               Does it involve animals? No
               Does it involve college students? No
               Does it involve imprisonment? Yes
The Shawshank Redemption (1994): Does it involve magic? No
               Does it make you think? Yes
               Does it remind you of your childhood? No
               Does it take place in North America? Yes
The Shawshank Redemption (1994): Does someone die? Yes
               Does someone get betrayed? Yes
               Does the title have a person's name in it? No
               Has it won an Oscar? Yes
The Shawshank Redemption (1994): Is it a bloody affair? No
               Is it a chick flick? No
               Is it a classic? Yes
               Is it a comedy? No
The Shawshank Redemption (1994): Is it a costume drama? No
               Is it a drama? Yes
               Is it a good family movie? No
               Is it a guy flick? Yes
The Shawshank Redemption (1994): Is it a love story? No
               Is it a musical? No
               Is it a story within a story? No
               Is it a teen movie? No
The Shawshank Redemption (1994): Is it a western? No
               Is it about a girl? No
               Is it about a journey? Yes
               Is it about family life? No
The Shawshank Redemption (1994): Is it about friends? Yes
               Is it about sports? No
               Is it about starting a new life? Yes
               Is it about teenagers? No
The Shawshank Redemption (1994): Is it about the future? No
               Is it about the present? Yes
               Is it about the sea? No
               Is it action-adventure? No
The Shawshank Redemption (1994): Is it based on a book? Yes
               Is it based on a true story? Yes
               Is it based on a video game? No
               Is it based on historic events? No
The Shawshank Redemption (1994): Is it comedic? No
               Is it confusing? No
               Is it inspirational? Yes
               Is it mainly about one character? Yes
The Shawshank Redemption (1994): Is it optimistic? Yes
               Is it part of a trilogy? No
               Is it political? Yes
               Is it popular with a younger audience? No
The Shawshank Redemption (1994): Is it popular with children? No
               Is it realistic? Yes
               Is it really long? Yes
               Is it romantic? No
The Shawshank Redemption (1994): Is it scary? No
               Is it science fiction? No
               Is it set during a war? No
               Is it set in America? Yes
The Shawshank Redemption (1994): Is it set in an exotic place? No
               Is it set in modern times? Yes
               Is it set in the mountains? No
               Is it set mainly in the outdoors? No
The Shawshank Redemption (1994): Is it suspenseful? Yes
               Is it violent? Yes
               Is it your favorite movie? No
               Was it a hit? Yes
The Shawshank Redemption (1994): Was it filmed in Hollywood? Unsure
               Was it made after the year 2000? No
               Was it nominated for an award? Yes
               Was it popular in the '70s? Yes
The Shawshank Redemption (1994): Was it popular in the '80s? Yes
               Was it popular in the '90s? Yes
               Would teenagers love it? Yes
               Would your grandmother like it? Yes
The Godfather (1972): Are there criminals in it? Yes
               Are witches part of the story? No
               Did it inspire a line of toys? Unsure
               Did it make you cry? Yes
The Godfather (1972): Did it make you jump? No
               Did it spawn a catch phrase? Yes
               Did it stir up controversy? Yes
               Did it win a Best Picture award? Unsure
The Godfather (1972): Did you like the ending? No
               Do you think it's funny? Yes
               Does it deal with racism? No
               Does it explore fringe culture? Yes
The Godfather (1972): Does it feature a determined female character? No
               Does it feature dancing? No
               Does it feature memorable music? No
               Does it have a cult following? Yes
The Godfather (1972): Does it have a famous cast? Yes
               Does it have a famous director? Yes
               Does it have a happy ending? No
               Does it have a lot of nudity? Yes
The Godfather (1972): Does it have a memorable quote? Yes
               Does it have a popular soundtrack? Yes
               Does it have a sequel? Yes
               Does it have an award winning character? Yes
The Godfather (1972): Does it have Mob connections? Yes
               Does it have wacky characters? Yes
               Does it involve a murder? Yes
               Does it involve advanced technology? Unsure
The Godfather (1972): Does it involve aliens? No
               Does it involve animals? No
               Does it involve college students? No
               Does it involve imprisonment? No
The Godfather (1972): Does it involve magic? No
               Does it make you think? Yes
               Does it remind you of your childhood? No
               Does it take place in North America? No
The Godfather (1972): Does someone die? Yes
               Does someone get betrayed? Yes
               Does the title have a person's name in it? Yes
               Has it won an Oscar? Yes
The Godfather (1972): Is it a bloody affair? Yes
               Is it a chick flick? No
               Is it a classic? Yes
               Is it a comedy? No
The Godfather (1972): Is it a costume drama? Yes
               Is it a drama? Yes
               Is it a good family movie? Yes
               Is it a guy flick? Yes
The Godfather (1972): Is it a love story? No
               Is it a musical? No
               Is it a story within a story? No
               Is it a teen movie? No
The Godfather (1972): Is it a western? No
               Is it about a girl? No
               Is it about a journey? No
               Is it about family life? No
The Godfather (1972): Is it about friends? No
               Is it about sports? No
               Is it about starting a new life? No
               Is it about teenagers? No
The Godfather (1972): Is it about the future? No
               Is it about the present? No
               Is it about the sea? No
               Is it action-adventure? No
The Godfather (1972): Is it based on a book? Yes
               Is it based on a true story? No
               Is it based on a video game? No
               Is it based on historic events? No
The Godfather (1972): Is it comedic? No
               Is it confusing? No
               Is it inspirational? No
               Is it mainly about one character? No
The Godfather (1972): Is it optimistic? Yes
               Is it part of a trilogy? Yes
               Is it political? No
               Is it popular with a younger audience? No
The Godfather (1972): Is it popular with children? No
               Is it realistic? Yes
               Is it really long? Yes
               Is it romantic? No
The Godfather (1972): Is it scary? No
               Is it science fiction? No
               Is it set during a war? No
               Is it set in America? Yes
The Godfather (1972): Is it set in an exotic place? No
               Is it set in modern times? No
               Is it set in the mountains? No
               Is it set mainly in the outdoors? No
The Godfather (1972): Is it suspenseful? Yes
               Is it violent? Yes
               Is it your favorite movie? No
               Was it a hit? Yes
The Godfather (1972): Was it filmed in Hollywood? Yes
               Was it made after the year 2000? No
               Was it nominated for an award? Yes
               Was it popular in the '70s? Yes
The Godfather (1972): Was it popular in the '80s? No
               Was it popular in the '90s? No
               Would teenagers love it? Yes
               Would your grandmother like it? Yes
The Godfather: Part II (1974): Are there criminals in it? Yes
               Are witches part of the story? No
               Did it inspire a line of toys? No
               Did it make you cry? No
The Godfather: Part II (1974): Did it make you jump? No
               Did it spawn a catch phrase? Yes
               Did it stir up controversy? Yes
               Did it win a Best Picture award? Unsure
The Godfather: Part II (1974): Did you like the ending? Yes
               Do you think it's funny? No
               Does it deal with racism? No
               Does it explore fringe culture? No
The Godfather: Part II (1974): Does it feature a determined female character? No
               Does it feature dancing? No
               Does it feature memorable music? Yes
               Does it have a cult following? Yes
The Godfather: Part II (1974): Does it have a famous cast? Yes
               Does it have a famous director? Yes
               Does it have a happy ending? No
               Does it have a lot of nudity? Unsure
The Godfather: Part II (1974): Does it have a memorable quote? Yes
               Does it have a popular soundtrack? Yes
               Does it have a sequel? Yes
               Does it have an award winning character? Yes
The Godfather: Part II (1974): Does it have Mob connections? No
               Does it have wacky characters? No
               Does it involve a murder? No
               Does it involve advanced technology? No
The Godfather: Part II (1974): Does it involve aliens? No
               Does it involve animals? No
               Does it involve college students? No
               Does it involve imprisonment? No
The Godfather: Part II (1974): Does it involve magic? No
               Does it make you think? Yes
               Does it remind you of your childhood? No
               Does it take place in North America? Yes
The Godfather: Part II (1974): Does someone die? Yes
               Does someone get betrayed? Yes
               Does the title have a person's name in it? Yes
               Has it won an Oscar? Yes
The Godfather: Part II (1974): Is it a bloody affair? Yes
               Is it a chick flick? No
               Is it a classic? Yes
               Is it a comedy? No
The Godfather: Part II (1974): Is it a costume drama? No
               Is it a drama? Yes
               Is it a good family movie? No
               Is it a guy flick? Yes
The Godfather: Part II (1974): Is it a love story? No
               Is it a musical? No
               Is it a story within a story? Yes
               Is it a teen movie? No
The Godfather: Part II (1974): Is it a western? No
               Is it about a girl? No
               Is it about a journey? Yes
               Is it about family life? Yes
The Godfather: Part II (1974): Is it about friends? No
               Is it about sports? No
               Is it about starting a new life? No
               Is it about teenagers? No
The Godfather: Part II (1974): Is it about the future? No
               Is it about the present? No
               Is it about the sea? No
               Is it action-adventure? No
The Godfather: Part II (1974): Is it based on a book? Yes
               Is it based on a true story? No
               Is it based on a video game? No
               Is it based on historic events? No
The Godfather: Part II (1974): Is it comedic? No
               Is it confusing? No
               Is it inspirational? No
               Is it mainly about one character? Yes
The Godfather: Part II (1974): Is it optimistic? No
               Is it part of a trilogy? Yes
               Is it political? Yes
               Is it popular with a younger audience? Yes
The Godfather: Part II (1974): Is it popular with children? No
               Is it realistic? Yes
               Is it really long? Yes
               Is it romantic? No
The Godfather: Part II (1974): Is it scary? Yes
               Is it science fiction? No
               Is it set during a war? No
               Is it set in America? Yes
The Godfather: Part II (1974): Is it set in an exotic place? No
               Is it set in modern times? No
               Is it set in the mountains? No
               Is it set mainly in the outdoors? Unsure
The Godfather: Part II (1974): Is it suspenseful? Yes
               Is it violent? Yes
               Is it your favorite movie? No
               Was it a hit? Yes
The Godfather: Part II (1974): Was it filmed in Hollywood? Yes
               Was it made after the year 2000? No
               Was it nominated for an award? Yes
               Was it popular in the '70s? Yes
The Godfather: Part II (1974): Was it popular in the '80s? Yes
               Was it popular in the '90s? Yes
               Would teenagers love it? Unsure
               Would your grandmother like it? Yes
The Dark Knight (2008): Are there criminals in it? Yes
               Are witches part of the story? No
               Did it inspire a line of toys? Yes
               Did it make you cry? No
The Dark Knight (2008): Did it make you jump? Yes
               Did it spawn a catch phrase? Yes
               Did it stir up controversy? Yes
               Did it win a Best Picture award? Unsure
The Dark Knight (2008): Did you like the ending? Yes
               Do you think it's funny? Yes
               Does it deal with racism? No
               Does it explore fringe culture? No
The Dark Knight (2008): Does it feature a determined female character? No
               Does it feature dancing? No
               Does it feature memorable music? Yes
               Does it have a cult following? Yes
The Dark Knight (2008): Does it have a famous cast? Yes
               Does it have a famous director? Yes
               Does it have a happy ending? No
               Does it have a lot of nudity? No
The Dark Knight (2008): Does it have a memorable quote? Yes
               Does it have a popular soundtrack? Yes
               Does it have a sequel? Yes
               Does it have an award winning character? Yes
The Dark Knight (2008): Does it have Mob connections? Yes
               Does it have wacky characters? Yes
               Does it involve a murder? Yes
               Does it involve advanced technology? Yes
The Dark Knight (2008): Does it involve aliens? No
               Does it involve animals? No
               Does it involve college students? No
               Does it involve imprisonment? Yes
The Dark Knight (2008): Does it involve magic? Yes
               Does it make you think? No
               Does it remind you of your childhood? No
               Does it take place in North America? Yes
The Dark Knight (2008): Does someone die? Yes
               Does someone get betrayed? Yes
               Does the title have a person's name in it? No
               Has it won an Oscar? Yes
The Dark Knight (2008): Is it a bloody affair? Yes
               Is it a chick flick? No
               Is it a classic? Yes
               Is it a comedy? No
The Dark Knight (2008): Is it a costume drama? No
               Is it a drama? No
               Is it a good family movie? No
               Is it a guy flick? Yes
The Dark Knight (2008): Is it a love story? No
               Is it a musical? No
               Is it a story within a story? No
               Is it a teen movie? No
The Dark Knight (2008): Is it a western? No
               Is it about a girl? No
               Is it about a journey? Yes
               Is it about family life? No
The Dark Knight (2008): Is it about friends? No
               Is it about sports? No
               Is it about starting a new life? No
               Is it about teenagers? No
The Dark Knight (2008): Is it about the future? No
               Is it about the present? No
               Is it about the sea? No
               Is it action-adventure? No
The Dark Knight (2008): Is it based on a book? Yes
               Is it based on a true story? No
               Is it based on a video game? No
               Is it based on historic events? No
The Dark Knight (2008): Is it comedic? No
               Is it confusing? Yes
               Is it inspirational? No
               Is it mainly about one character? Yes
The Dark Knight (2008): Is it optimistic? No
               Is it part of a trilogy? Yes
               Is it political? No
               Is it popular with a younger audience? Yes
The Dark Knight (2008): Is it popular with children? Yes
               Is it realistic? No
               Is it really long? No
               Is it romantic? No
The Dark Knight (2008): Is it scary? No
               Is it science fiction? No
               Is it set during a war? Yes
               Is it set in America? No
The Dark Knight (2008): Is it set in an exotic place? No
               Is it set in modern times? Yes
               Is it set in the mountains? No
               Is it set mainly in the outdoors? No
The Dark Knight (2008): Is it suspenseful? Yes
               Is it violent? Yes
               Is it your favorite movie? No
               Was it a hit? Yes
The Dark Knight (2008): Was it filmed in Hollywood? Yes
               Was it made after the year 2000? Yes
               Was it nominated for an award? Yes
               Was it popular in the '70s? No
The Dark Knight (2008): Was it popular in the '80s? No
               Was it popular in the '90s? No
               Would teenagers love it? Yes
               Would your grandmother like it? No
12 Angry Men (1957): Are there criminals in it? Yes
               Are witches part of the story? No
               Did it inspire a line of toys? No
               Did it make you cry? No
12 Angry Men (1957): Did it make you jump? No
               Did it spawn a catch phrase? No
               Did it stir up controversy? Yes
               Did it win a Best Picture award? Yes
12 Angry Men (1957): Did you like the ending? No
               Do you think it's funny? No
               Does it deal with racism? No
               Does it explore fringe culture? Yes
12 Angry Men (1957): Does it feature a determined female character? Yes
               Does it feature dancing? No
               Does it feature memorable music? Yes
               Does it have a cult following? No
12 Angry Men (1957): Does it have a famous cast? Yes
               Does it have a famous director? No
               Does it have a happy ending? No
               Does it have a lot of nudity? No
12 Angry Men (1957): Does it have a memorable quote? Yes
               Does it have a popular soundtrack? No
               Does it have a sequel? No
               Does it have an award winning character? No
12 Angry Men (1957): Does it have Mob connections? Unsure
               Does it have wacky characters? Unsure
               Does it involve a murder? Unsure
               Does it involve advanced technology? Unsure
12 Angry Men (1957): Does it involve aliens? No
               Does it involve animals? No
               Does it involve college students? No
               Does it involve imprisonment? Yes
12 Angry Men (1957): Does it involve magic? Unsure
               Does it make you think? Unsure
               Does it remind you of your childhood? No
               Does it take place in North America? Yes
12 Angry Men (1957): Does someone die? Unsure
               Does someone get betrayed? No
               Does the title have a person's name in it? No
               Has it won an Oscar? Yes
12 Angry Men (1957): Is it a bloody affair? No
               Is it a chick flick? No
               Is it a classic? Yes
               Is it a comedy? No
12 Angry Men (1957): Is it a costume drama? No
               Is it a drama? Yes
               Is it a good family movie? No
               Is it a guy flick? Yes
12 Angry Men (1957): Is it a love story? No
               Is it a musical? No
               Is it a story within a story? No
               Is it a teen movie? No
12 Angry Men (1957): Is it a western? No
               Is it about a girl? No
               Is it about a journey? Unsure
               Is it about family life? Unsure
12 Angry Men (1957): Is it about friends? No
               Is it about sports? No
               Is it about starting a new life? No
               Is it about teenagers? No
12 Angry Men (1957): Is it about the future? No
               Is it about the present? Yes
               Is it about the sea? No
               Is it action-adventure? Yes
12 Angry Men (1957): Is it based on a book? No
               Is it based on a true story? No
               Is it based on a video game? No
               Is it based on historic events? No
12 Angry Men (1957): Is it comedic? No
               Is it confusing? No
               Is it inspirational? Yes
               Is it mainly about one character? No
12 Angry Men (1957): Is it optimistic? Yes
               Is it part of a trilogy? No
               Is it political? Yes
               Is it popular with a younger audience? No
12 Angry Men (1957): Is it popular with children? No
               Is it realistic? Yes
               Is it really long? No
               Is it romantic? No
12 Angry Men (1957): Is it scary? No
               Is it science fiction? No
               Is it set during a war? Unsure
               Is it set in America? Yes
12 Angry Men (1957): Is it set in an exotic place? No
               Is it set in modern times? No
               Is it set in the mountains? No
               Is it set mainly in the outdoors? No
12 Angry Men (1957): Is it suspenseful? Unsure
               Is it violent? Unsure
               Is it your favorite movie? Unsure
               Was it a hit? Unsure
12 Angry Men (1957): Was it filmed in Hollywood? Yes
               Was it made after the year 2000? No
               Was it nominated for an award? Yes
               Was it popular in the '70s? Yes
12 Angry Men (1957): Was it popular in the '80s? No
               Was it popular in the '90s? No
               Would teenagers love it? No
               Would your grandmother like it? Yes

In [99]:
#import os
cwd = os.getcwd()
print cwd
#dir(myreader)
myreader.line_num
#row
#row['Input.QUESTION1']


/Users/srbecker/Documents/python_and_julia/HighSchoolOutreach
Out[99]:
126L

In [ ]: