prelim_month_human - reliability
2017.10.26 - work log - prelim_month_human - Reliability_Names reliability2017.10.25-work_log-prelim_month_human-Reliability_Names_reliability.ipynbLooks at measures of agreement of coding between corrected (coder 1 - ground truth) and uncorrected (coder 2) human coding.
In [1]:
import datetime
import six
print( "packages imported at " + str( datetime.datetime.now() ) )
If you are using a virtualenv, make sure that you:
Since I use a virtualenv, need to get that activated somehow inside this notebook. One option is to run ../dev/wsgi.py in this notebook, to configure the python environment manually as if you had activated the sourcenet virtualenv. To do this, you'd make a code cell that contains:
%run ../dev/wsgi.py
This is sketchy, however, because of the changes it makes to your Python environment within the context of whatever your current kernel is. I'd worry about collisions with the actual Python 3 kernel. Better, one can install their virtualenv as a separate kernel. Steps:
activate your virtualenv:
workon sourcenetin your virtualenv, install the package ipykernel.
pip install ipykerneluse the ipykernel python program to install the current environment as a kernel:
python -m ipykernel install --user --name <env_name> --display-name "<display_name>"
sourcenet example:
python -m ipykernel install --user --name sourcenet --display-name "sourcenet (Python 3)"More details: http://ipython.readthedocs.io/en/stable/install/kernel_install.html
In [2]:
%pwd
Out[2]:
First, initialize my dev django project, so I can run code in this notebook that references my django models and can talk to the database using my project's settings.
In [3]:
%run ../django_init.py
To allow Python to talk to R, at R prompt:
/* install packages */
install.packages( "Rserve" )
install.packages( "irr" )
/* load Rserve */
library( Rserve )
/* start server */
Rserve( args="--no-save" )Also need to either pass database connection information to names analyzer below, or store database configuration in Django_Config:
# database connection information - 2 options... Enter it here:
#my_analysis_instance.db_username = ""
#my_analysis_instance.db_password = ""
#my_analysis_instance.db_host = "localhost"
#my_analysis_instance.db_name = "sourcenet"
# Or set up the following properties in Django_Config, inside the django admins.
# All have application of: "sourcenet-db-admin":
# - db_username
# - db_password
# - db_host
# - db_port
# - db_nameprelim_month_humanGenerate reliability analysis for label "prelim_month_human".
In [4]:
# start to support python 3:
from __future__ import unicode_literals
from __future__ import division
#==============================================================================#
# ! imports
#==============================================================================#
# grouped by functional area, then alphabetical order by package, then
# alphabetical order by name of thing being imported.
# context_analysis imports
from context_analysis.reliability.reliability_names_analyzer import ReliabilityNamesAnalyzer
#==============================================================================#
# ! logic
#==============================================================================#
# declare variables
my_analysis_instance = None
label = ""
indices_to_process = -1
result_status = ""
# make reliability instance
my_analysis_instance = ReliabilityNamesAnalyzer()
# database connection information - 2 options... Enter it here:
#my_analysis_instance.db_username = ""
#my_analysis_instance.db_password = ""
#my_analysis_instance.db_host = "localhost"
#my_analysis_instance.db_name = "sourcenet"
# Or set up the following properties in Django_Config, inside the django admins.
# All have application of: "sourcenet-db-admin":
# - db_username
# - db_password
# - db_host
# - db_port
# - db_name
# run the analyze method
label = "prelim_month_human"
indices_to_process = 2
result_status = my_analysis_instance.analyze_reliability_names( label, indices_to_process )
In [5]:
print( "result status: {status_string}".format( status_string = result_status ) )
Dropbox/academia/MSU/program_stuff/prelim_paper/analysis/reliability/2016-data/prelim_month_human-reliability_results.pdf.