Rand 2011 Cooperation Study

This notebook outlines how to recreate the analysis of the Rand et al. 2011 study "Dynamic social networks promote cooperation in experiments with humans" Link to Paper

This workbook focuses on the re-wire analysis workflow portion of the study

Run the cooperation analysis first for a step by step description of interacting with Bedrock, this workflow uses those concepts to complete the rewire study



In [ ]:

    
from bedrock.client.client import BedrockAPI
import requests
import pandas
import pprint
SERVER = "http://localhost:81/"
api = BedrockAPI(SERVER)

Check for csv file locally

The following code opens the file and prints out the first part. The file must be a csv file with a header that has labels for each column. The file is comma delimited csv.



In [ ]:

    
filepath = 'Rand2011PNAS_rewire_data.csv'
datafile = pandas.read_csv('Rand2011PNAS_rewire_data.csv')
datafile.head(10)



In [ ]:

    
null_data = datafile[datafile.isnull().any(axis=1)]
null_data

Now Upload the source file to the Bedrock Server

This code block uses the Spreadsheet ingest module to upload the source file to Bedrock. Note: This simply copies the file to the server, but does not create a Bedrock Matrix format

If the following fails to upload. Check that the csv file is in the correct comma delimited format with headers.



In [ ]:

    
ingest_id = 'opals.spreadsheet.Spreadsheet.Spreadsheet'
resp = api.put_source('Rand2011_rewire', ingest_id, 'default', {'file': open(filepath, "rb")})

if resp.status_code == 201:
    source_id = resp.json()['src_id']
    print('Source {0} successfully uploaded'.format(filepath))
else:
    try:
        print("Error in Upload: {}".format(resp.json()['msg']))
    except Exception:
        pass
    
    try:
        source_id = resp.json()['src_id']
        print("Using existing source.  If this is not the desired behavior, upload with a different name.")
    except Exception:
        print("No existing source id provided")

Create a Bedrock Matrix from the CSV Source

In order to use the data, the data source must be converted to a Bedrock matrix. The following code steps through that process. Here we are doing a simple transform of csv to matrix. There are options to apply filters (like renaming columns, excluding colum



In [ ]:

    
resp = api.create_matrix(source_id, 'rand_mtx')
base_mtx = resp[0]
matrix_id = base_mtx['id']
base_mtx

Look at basic statistics on the source data

Here we can see that Bedrock has computed some basic statistics on the source data.

For numeric data

The quartiles, max, mean, min, and standard deviation are provided

For non-numeric data

The label values and counts for each label are provided.

For both types

The proposed tags and data type that Bedrock is suggesting are provided



In [ ]:

    
analytic_id = "opals.summarize.Summarize.Summarize"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = []

summary_mtx = api.run_analytic(analytic_id, base_mtx, 'rand_mtx_summary', input_data=inputData, parameter_data=paramsData)
output = api.download_results_matrix(summary_mtx['src_id'], summary_mtx['id'], 'matrix.csv')
output

Create a filtered matrix where previouslytie==0



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByCondition.SelectByCondition"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = [
    {"attrname":"colname","value":"previouslytie"},
    {"attrname":"comparator","value":"=="},
    {"attrname":"value","value":"0"}
]

filtered_mtx = api.run_analytic(analytic_id, base_mtx, 'prevtie0', input_data=inputData, parameter_data=paramsData)
f = api.download_results_matrix(filtered_mtx['src_id'], filtered_mtx['id'], 'matrix.csv', remote_header_file='features.txt')
f.head(10)

Look at otherd effect on nowtie

Note we have to remove rows that contain missing values for either our exogenous or endogenous factors or else clustered standard errors will fail



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByCondition.SelectByCondition"
inputData = {
    'matrix.csv': filtered_mtx,
    'features.txt': filtered_mtx
}

paramsData = [
    {"attrname":"colname","value":"otherD"},
    {"attrname":"comparator","value":"notnull"},
    {"attrname":"value","value":""}
]

otherd_mtx = api.run_analytic(analytic_id, filtered_mtx, 'otherD', input_data=inputData, parameter_data=paramsData)



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': otherd_mtx,
    'features.txt': otherd_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(nowtie) ~ C(otherD)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, otherd_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Look at otherd and roundnum effect on nowtie

Note we have to remove rows that contain missing values for either our exogenous or endogenous factors or else clustered standard errors will fail



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': otherd_mtx,
    'features.txt': otherd_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(nowtie) ~ C(otherD) + C(roundnum)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, otherd_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Previouslytie == 1



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByCondition.SelectByCondition"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = [
    {"attrname":"colname","value":"previouslytie"},
    {"attrname":"comparator","value":"=="},
    {"attrname":"value","value":"1"}
]

filtered_mtx = api.run_analytic(analytic_id, base_mtx, 'prevtie1', input_data=inputData, parameter_data=paramsData)
f = api.download_results_matrix(filtered_mtx['src_id'], filtered_mtx['id'], 'matrix.csv', remote_header_file='features.txt')
f.head(10)

Look at otherd effect on nowtie

Note we have to remove rows that contain missing values for either our exogenous or endogenous factors or else clustered standard errors will fail



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByCondition.SelectByCondition"
inputData = {
    'matrix.csv': filtered_mtx,
    'features.txt': filtered_mtx
}

paramsData = [
    {"attrname":"colname","value":"otherD"},
    {"attrname":"comparator","value":"notnull"},
    {"attrname":"value","value":""}
]

otherd_mtx = api.run_analytic(analytic_id, filtered_mtx, 'otherD', input_data=inputData, parameter_data=paramsData)



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': otherd_mtx,
    'features.txt': otherd_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(nowtie) ~ C(otherD)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, otherd_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Look at otherd and roundnum effect on nowtie

Note we have to remove rows that contain missing values for either our exogenous or endogenous factors or else clustered standard errors will fail



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': otherd_mtx,
    'features.txt': otherd_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(nowtie) ~ C(otherD) + C(roundnum)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, otherd_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Filter where previouslytie == otherD



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByComplexCondition.SelectByComplexCondition"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = [
    {"attrname":"condition","value":"(previouslytie == otherD)"}
]

filtered_mtx = api.run_analytic(analytic_id, base_mtx, 'prevtie1', input_data=inputData, parameter_data=paramsData)
f = api.download_results_matrix(filtered_mtx['src_id'], filtered_mtx['id'], 'matrix.csv', remote_header_file='features.txt')
f.head(10)

Look at otherd effect on act when prevtie == otherD



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByCondition.SelectByCondition"
inputData = {
    'matrix.csv': filtered_mtx,
    'features.txt': filtered_mtx
}

paramsData = [
    {"attrname":"colname","value":"otherD"},
    {"attrname":"comparator","value":"notnull"},
    {"attrname":"value","value":""}
]

otherd_mtx = api.run_analytic(analytic_id, filtered_mtx, 'otherD', input_data=inputData, parameter_data=paramsData)



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': otherd_mtx,
    'features.txt': otherd_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(act) ~ C(otherD)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":""}
]

result_mtx = api.run_analytic(analytic_id, otherd_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Look at otherD and roundnum effect on act when prevtie == otherD



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': otherd_mtx,
    'features.txt': otherd_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(act) ~ C(otherD) + C(roundnum)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":""}
]

result_mtx = api.run_analytic(analytic_id, otherd_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Look at CC v CD/DC



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByComplexCondition.SelectByComplexCondition"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = [
    {"attrname":"condition","value":'(previouslytie == 1) & ((state=="CC") | (state=="CD") | (state=="DC"))'}
]

filtered_mtx = api.run_analytic(analytic_id, base_mtx, 'prevtie1', input_data=inputData, parameter_data=paramsData)
f = api.download_results_matrix(filtered_mtx['src_id'], filtered_mtx['id'], 'matrix.csv', remote_header_file='features.txt')
f.head(10)



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': filtered_mtx,
    'features.txt': filtered_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(break_) ~ C(CC)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, filtered_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Look at CC v DD



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByComplexCondition.SelectByComplexCondition"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = [
    {"attrname":"condition","value":'(previouslytie == 1) & ((state=="CC") | (state=="DD"))'}
]

filtered_mtx = api.run_analytic(analytic_id, base_mtx, 'prevtie1', input_data=inputData, parameter_data=paramsData)
f = api.download_results_matrix(filtered_mtx['src_id'], filtered_mtx['id'], 'matrix.csv', remote_header_file='features.txt')
f.head(10)



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': filtered_mtx,
    'features.txt': filtered_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(break_) ~ C(CC)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, filtered_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table

Look at DD v CD/DC



In [ ]:

    
analytic_id = "opals.select-from-dataframe.SelectByComplexCondition.SelectByComplexCondition"
inputData = {
    'matrix.csv': base_mtx,
    'features.txt': base_mtx
}

paramsData = [
    {"attrname":"condition","value":'(previouslytie == 1) & ((state=="DD") | (state=="CD") | (state=="DC"))'}
]

filtered_mtx = api.run_analytic(analytic_id, base_mtx, 'prevtie1', input_data=inputData, parameter_data=paramsData)
f = api.download_results_matrix(filtered_mtx['src_id'], filtered_mtx['id'], 'matrix.csv', remote_header_file='features.txt')
f.head(10)



In [ ]:

    
analytic_id = "opals.logit2.Logit2.Logit2"
inputData = {
    'matrix.csv': filtered_mtx,
    'features.txt': filtered_mtx
}

paramsData = [
    {"attrname":"formula","value":"C(break_) ~ C(DD)"},
    {"attrname":"family","value":"binomial"},
    {"attrname":"clustered_rse","value":"sessionnum,playerid"}
]

result_mtx = api.run_analytic(analytic_id, filtered_mtx, 'rewire_step1', input_data=inputData, parameter_data=paramsData)
coef_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'matrix.csv')
coef_table



In [ ]:

    
summary_table = api.download_results_matrix(result_mtx['src_id'], result_mtx['id'], 'summary.csv')
summary_table