A Closer Look At Recidivism

Data Bootcamp
Spring 2017
Alexandra Steckmest </center>

An Introduction to Rehabilitative Justice

       The debate of what makes the best criminal justice system has always been one that is riddled with controversies. One particularly interesting comparison is that of punitive justice and rehabilitative justice. While some argue that inmates should be treated in inhumane ways, as a sort of revenge for their wrongdoings, others believe that the purpose of inprisonment is to somehow "heal" the inmates to prevent them from engaging in criminal activities once they are released back into society. One way to argue whether a justice system is successful is to consider recidivism rates – the tendency of a convicted felon to reoffend.

Norway is a country that is known for their uniquely rehabilitative focused justice system, which has seen one of the lowest rates of recidivism worldwide. Kriminalomsorgen (the Norwegian Correctional Service) have since the late 20th century invested a lot of money in improving prison conditions in an effort to improve inmates' lives and hence reduce recidivism. For instance, approximately $1 million was spent on art work at the maximum-security prison Halden to "ease the psychological burdens of imprisonment," according to Time Magazine. But does it work?

In 2008, the Norwegian Storting (supreme legislature) released a white paper to reduce crime and enforce "punishment that works." The goal was that after their sentence, prisoners will live crime-less lives. The purpose of this project is to evaluate whether this has indeed had an effect on recidivism rates.

Looking at the data

Data sources

The data used for this project is retrieved entirely from Statistics Norway and from 3 different data sets:

  • "Expenses" is the first data set, which shows the expenses of the Norwegian Correctional Services between 2005 and 2012. The data set can be retrieved from a statistics bank from which expenditure in different categories across different years can be retrieved.
  • "Years" is significantly larger and describes the raw numbers of people who, by categories of sex and age, were re-convicted after their release from a previous crime. The data set describes this number for each year after the prisoner was released - i.e. how many were charged with a new crime after 1, 2, 3, 4 or years. This dataset can be retrieved from another statistics bank.
  • "Offenses" shows recidivism, in raw numbers and percent, of persons charged by the type of principal offense. The data will allow me to see not only which category of criminal offense shows the highest and lowest rates of recidivism, but also whether or not people who get re-convicted are charged with similar or more severe crimes. This data set can be retrieved from yet another statistics bank and, just like the "Years" data set, covers information from 2002 to 2010.

For the purpose of uploading the data to this Jupyter Notebook, the 3 different data sets were imported to Excel and put into 3 separate sheets in the excel document. This excel document was then uploaded to a github page.


In [1]:
# Importing packages

import matplotlib as mpl
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
import sys
import datetime as dt
from pandas_datareader import wb, data as web 

# plotly imports
from plotly.offline import iplot, iplot_mpl  # plotting functions
import plotly.graph_objs as go               # ditto
import plotly                                # just to print version and init notebook
import cufflinks as cf                       # gives us df.iplot that feels like df.plot
cf.set_config_file(offline=True, offline_show_link=False)

# for graphics
%matplotlib inline


A breakdown of expenditure in Norwegian prisons

This data will reveal whether or not the 2008 paper had a direct effect on the spending, which will be important for any potential conclusions I will be able to draw. Presumably, we will see an increase in spending in 2008 that either stays high or increases as time passes.

Load data


In [2]:
# importing from github
url = 'https://raw.githubusercontent.com/asteckmest/data-bootcamp-project/master/SSB_recidivism.xlsx'

# letting "expenses" be a data set, retrieved from the file, and setting header and index
expenses = pd.read_excel(url, header=[2], index_col=[0], sheetname='Expenses')
expenses


Out[2]:
2005 2006 2007 2008 2009 2010 2011 2012
Operating expenditure (NOK million) 2244.7 2387.9 2651.9 2869.8 3557.3 3859.7 4002.7 4104.9
Own production (NOK million) 2208.5 2351.4 2611.2 2844.3 3526.4 3805.6 3931.8 4044.0
Compensation of employees (NOK million) 1641.0 1726.5 1906.5 2099.7 2294.4 2429.9 2567.5 2695.1
Use of goods and services (NOK million) 567.5 624.8 704.7 744.6 1232.0 1375.8 1364.3 1348.9
Transfers (NOK million) 36.2 36.5 40.7 25.5 30.9 54.0 70.8 60.9
Capital formation (NOK million) 86.3 116.6 127.9 121.7 84.6 20.9 22.6 14.6
Total expenditure (NOK million) 2331.1 2504.5 2779.8 2991.5 3641.8 3880.6 4025.3 4119.5
Compensation of employees in per cent of own production (per cent) 74.3 73.4 73.0 73.8 65.1 63.8 65.3 66.6
Use of goods and services in per cent of own production (per cent) 25.7 26.6 27.0 26.2 34.9 36.2 34.7 33.4
Estimated contributions to the Norwegian Public Service Pension Fund (NOK mill.) 198.9 212.1 234.0 256.8 280.9 297.0 314.5 330.6

Clean data


In [3]:
# renaming the index
expenses.index.name = 'Years'
expenses


Out[3]:
2005 2006 2007 2008 2009 2010 2011 2012
Years
Operating expenditure (NOK million) 2244.7 2387.9 2651.9 2869.8 3557.3 3859.7 4002.7 4104.9
Own production (NOK million) 2208.5 2351.4 2611.2 2844.3 3526.4 3805.6 3931.8 4044.0
Compensation of employees (NOK million) 1641.0 1726.5 1906.5 2099.7 2294.4 2429.9 2567.5 2695.1
Use of goods and services (NOK million) 567.5 624.8 704.7 744.6 1232.0 1375.8 1364.3 1348.9
Transfers (NOK million) 36.2 36.5 40.7 25.5 30.9 54.0 70.8 60.9
Capital formation (NOK million) 86.3 116.6 127.9 121.7 84.6 20.9 22.6 14.6
Total expenditure (NOK million) 2331.1 2504.5 2779.8 2991.5 3641.8 3880.6 4025.3 4119.5
Compensation of employees in per cent of own production (per cent) 74.3 73.4 73.0 73.8 65.1 63.8 65.3 66.6
Use of goods and services in per cent of own production (per cent) 25.7 26.6 27.0 26.2 34.9 36.2 34.7 33.4
Estimated contributions to the Norwegian Public Service Pension Fund (NOK mill.) 198.9 212.1 234.0 256.8 280.9 297.0 314.5 330.6

Plot data


In [4]:
# creating a layout to be used for the pyplot
layout = dict(width=950, height=600,                                    # plot width/height
              yaxis={"title": "Spending (million NOK)"},                # yaxis label
              title="Norwegian Correctional Service Expenditure",       # title
              xaxis={"title": "Years"}                                  # xaxis label
             )

In [5]:
# looking only at the 3 most significant variables for the purpose of this research topic
expenses.T[['Operating expenditure (NOK million)',                       
            'Own production (NOK million)', 
            'Total expenditure (NOK million)']].iplot(layout=layout)


What to make of this data

From this data we see that expenditure has increased since 2005, and it is clear that the white paper had a direct effect in 2008. This can be seen by the sudden sharp increase in total spending, operating expenses and own production. This means that we can expect there to be direct effects on recidivism as well.


In [6]:
# adding a vertical line for the year 2008
expenses.T[['Operating expenditure (NOK million)',                       
            'Own production (NOK million)', 
            'Total expenditure (NOK million)']].iplot(hspan={'x0':'2008','x1':'2009','color':'rgba(30,30,30,0.3)','fill':True,'opacity':.4})


Recidivism across time, sex and age

This dataframe is larger and slightly more complex because there are 3 variables, namely the number of years that have passed since the "base" year (year that prisoner was released from first sentence), the sex of the offender and their age. It will be interesting to see how these variables changed over time, especially after 2008.

Load data


In [7]:
# retrieving the data set, setting a multi-index for Base Year, Sex and Age, choosing header as the years
years = pd.read_excel(url, header=[2], index_col=[0,1,2], sheetname='By_Years_Sex_Age')
years


Out[7]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
NaN NaN NaN Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged
All years (0-5) Total Total 69617 72030 78185 78908 79445 82234 79021 75463 75787
All ages 5 years or older 69617 72030 78185 78908 79445 82234 79021 75463 75787
5-14 years 3385 3243 3621 3413 3596 4098 3989 3783 3234
15-17 years 6389 6247 7229 7228 6868 7505 7076 6264 5771
18-20 years 9873 10182 10722 10355 10569 11168 10599 10299 10353
21-24 years 9639 9787 10437 10562 10637 10957 10160 9752 9966
25-29 years 9391 9298 9824 9846 9654 10057 9578 9193 9452
30-39 years 15039 15995 17157 17209 17069 16963 15889 14839 14889
40-49 years 8956 9543 10545 11318 11674 11916 11842 11884 12080
50-59 years 4262 4834 5373 5585 5773 5740 6054 5756 6074
60 years or older 2683 2901 3277 3392 3605 3830 3834 3693 3968
Unknown age 0 0 0 0 0 0 0 0 0
Both sexes Total 69617 72030 78185 78908 79445 82234 79021 75463 75787
All ages 5 years or older 69617 72030 78185 78908 79445 82234 79021 75463 75787
5-14 years 3385 3243 3621 3413 3596 4098 3989 3783 3234
15-17 years 6389 6247 7229 7228 6868 7505 7076 6264 5771
18-20 years 9873 10182 10722 10355 10569 11168 10599 10299 10353
21-24 years 9639 9787 10437 10562 10637 10957 10160 9752 9966
25-29 years 9391 9298 9824 9846 9654 10057 9578 9193 9452
30-39 years 15039 15995 17157 17209 17069 16963 15889 14839 14889
40-49 years 8956 9543 10545 11318 11674 11916 11842 11884 12080
50-59 years 4262 4834 5373 5585 5773 5740 6054 5756 6074
60 years or older 2683 2901 3277 3392 3605 3830 3834 3693 3968
Unknown age 0 0 0 0 0 0 0 0 0
Males Total 58327 60208 64741 65430 65649 67769 65092 62205 62657
All ages 5 years or older 58327 60208 64741 65430 65649 67769 65092 62205 62657
5-14 years 2516 2292 2449 2325 2535 2804 2841 2641 2342
15-17 years 5121 4995 5605 5440 5111 5587 5254 4621 4326
18-20 years 8382 8653 9127 8801 8856 9387 8816 8566 8609
... ... ... ... ... ... ... ... ... ... ... ...
5 years with recidivism Males 40-49 years 134 127 141 140 138 141 137 149 144
50-59 years 11 11 14 11 14 17 24 37 31
60 years or older 1 2 1 2 4 3 4 2 5
Unknown age 0 0 0 0 0 0 0 0 0
Females Total 132 158 151 132 142 130 125 134 155
All ages 5 years or older 132 158 151 132 142 130 125 134 155
5-14 years 3 4 3 6 6 4 7 2 4
15-17 years 6 5 7 9 12 12 8 10 9
18-20 years 15 23 19 18 19 18 24 17 18
21-24 years 21 26 28 25 22 15 14 19 28
25-29 years 19 27 24 19 21 20 17 15 18
30-39 years 49 52 50 36 38 35 33 43 45
40-49 years 15 20 18 17 19 21 17 23 26
50-59 years 4 1 1 2 5 5 4 4 6
60 years or older 0 0 1 0 0 0 1 1 1
Unknown age 0 0 0 0 0 0 0 0 0
Unknown sex Total 0 0 0 0 0 0 0 0 0
All ages 5 years or older 0 0 0 0 0 0 0 0 0
5-14 years 0 0 0 0 0 0 0 0 0
15-17 years 0 0 0 0 0 0 0 0 0
18-20 years 0 0 0 0 0 0 0 0 0
21-24 years 0 0 0 0 0 0 0 0 0
25-29 years 0 0 0 0 0 0 0 0 0
30-39 years 0 0 0 0 0 0 0 0 0
40-49 years 0 0 0 0 0 0 0 0 0
50-59 years 0 0 0 0 0 0 0 0 0
60 years or older 0 0 0 0 0 0 0 0 0
Unknown age 0 0 0 0 0 0 0 0 0
Unknown age NaN NaN NaN NaN NaN NaN NaN NaN NaN
Persons charged only covers charged persons with a valid Norwegian personal identification number, and at least one stipulated time of crime in the base year for the principal offence. A charged person is considered to have recidivism if they have a new charge brought against them in the subsequent five-year period, and if at least one of the principal offences is committed after the principal offence in the base year. Unknown sex Unknown age NaN NaN NaN NaN NaN NaN NaN NaN NaN

483 rows × 9 columns

Clean data


In [8]:
# renaming the index
years.index.names = ['Base Year', 'Sex', 'Age']

# show the first 5 rows
years.head()


Out[8]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
NaN NaN NaN Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged Persons charged
All years (0-5) Total Total 69617 72030 78185 78908 79445 82234 79021 75463 75787
All ages 5 years or older 69617 72030 78185 78908 79445 82234 79021 75463 75787
5-14 years 3385 3243 3621 3413 3596 4098 3989 3783 3234
15-17 years 6389 6247 7229 7228 6868 7505 7076 6264 5771

In [9]:
# dropping the rows with NaN and disclaimer information in index
years = years.loc[~(years.index.labels[0] == -1)]
years = years.drop('Persons charged only covers charged persons with a valid Norwegian personal identification number, and at least one stipulated time of crime in the base year for the principal offence. A charged person is considered to have recidivism if they have a new charge brought against them in the subsequent five-year period, and if at least one of the principal offences is committed after the principal offence in the base year.', level='Base Year')
years = years.drop('Unknown sex', level = 'Sex')
years = years.drop('Unknown age', level = 'Age')
years


Out[9]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Total Total 69617 72030 78185 78908 79445 82234 79021 75463 75787
All ages 5 years or older 69617 72030 78185 78908 79445 82234 79021 75463 75787
5-14 years 3385 3243 3621 3413 3596 4098 3989 3783 3234
15-17 years 6389 6247 7229 7228 6868 7505 7076 6264 5771
18-20 years 9873 10182 10722 10355 10569 11168 10599 10299 10353
21-24 years 9639 9787 10437 10562 10637 10957 10160 9752 9966
25-29 years 9391 9298 9824 9846 9654 10057 9578 9193 9452
30-39 years 15039 15995 17157 17209 17069 16963 15889 14839 14889
40-49 years 8956 9543 10545 11318 11674 11916 11842 11884 12080
50-59 years 4262 4834 5373 5585 5773 5740 6054 5756 6074
60 years or older 2683 2901 3277 3392 3605 3830 3834 3693 3968
Both sexes Total 69617 72030 78185 78908 79445 82234 79021 75463 75787
All ages 5 years or older 69617 72030 78185 78908 79445 82234 79021 75463 75787
5-14 years 3385 3243 3621 3413 3596 4098 3989 3783 3234
15-17 years 6389 6247 7229 7228 6868 7505 7076 6264 5771
18-20 years 9873 10182 10722 10355 10569 11168 10599 10299 10353
21-24 years 9639 9787 10437 10562 10637 10957 10160 9752 9966
25-29 years 9391 9298 9824 9846 9654 10057 9578 9193 9452
30-39 years 15039 15995 17157 17209 17069 16963 15889 14839 14889
40-49 years 8956 9543 10545 11318 11674 11916 11842 11884 12080
50-59 years 4262 4834 5373 5585 5773 5740 6054 5756 6074
60 years or older 2683 2901 3277 3392 3605 3830 3834 3693 3968
Males Total 58327 60208 64741 65430 65649 67769 65092 62205 62657
All ages 5 years or older 58327 60208 64741 65430 65649 67769 65092 62205 62657
5-14 years 2516 2292 2449 2325 2535 2804 2841 2641 2342
15-17 years 5121 4995 5605 5440 5111 5587 5254 4621 4326
18-20 years 8382 8653 9127 8801 8856 9387 8816 8566 8609
21-24 years 8398 8459 8969 9065 9173 9426 8732 8409 8607
25-29 years 8093 8084 8464 8490 8306 8622 8182 7896 8118
30-39 years 12671 13445 14412 14530 14294 14218 13333 12431 12513
... ... ... ... ... ... ... ... ... ... ... ...
5 years with recidivism Both sexes 15-17 years 231 206 235 228 239 228 198 155 151
18-20 years 279 283 271 257 230 240 226 217 207
21-24 years 247 224 259 245 243 224 199 197 219
25-29 years 249 230 209 212 212 232 225 219 211
30-39 years 436 439 386 387 365 334 328 309 305
40-49 years 149 147 159 157 157 162 154 172 170
50-59 years 15 12 15 13 19 22 28 41 37
60 years or older 1 2 2 2 4 3 5 3 6
Males Total 1567 1472 1470 1460 1420 1394 1313 1243 1195
All ages 5 years or older 1567 1472 1470 1460 1420 1394 1313 1243 1195
5-14 years 89 83 82 85 87 75 68 62 40
15-17 years 225 201 228 219 227 216 190 145 142
18-20 years 264 260 252 239 211 222 202 200 189
21-24 years 226 198 231 220 221 209 185 178 191
25-29 years 230 203 185 193 191 212 208 204 193
30-39 years 387 387 336 351 327 299 295 266 260
40-49 years 134 127 141 140 138 141 137 149 144
50-59 years 11 11 14 11 14 17 24 37 31
60 years or older 1 2 1 2 4 3 4 2 5
Females Total 132 158 151 132 142 130 125 134 155
All ages 5 years or older 132 158 151 132 142 130 125 134 155
5-14 years 3 4 3 6 6 4 7 2 4
15-17 years 6 5 7 9 12 12 8 10 9
18-20 years 15 23 19 18 19 18 24 17 18
21-24 years 21 26 28 25 22 15 14 19 28
25-29 years 19 27 24 19 21 20 17 15 18
30-39 years 49 52 50 36 38 35 33 43 45
40-49 years 15 20 18 17 19 21 17 23 26
50-59 years 4 1 1 2 5 5 4 4 6
60 years or older 0 0 1 0 0 0 1 1 1

352 rows × 9 columns

Across time

Let's first consider, across all ages and sexes, how recidivism changes with years after release of the first sentence. For example, it will be interesting to see whether recidivism is highest after 1 year, implying that the released criminals were not "rehabilitated" during their prison time, or whether it is highest after 3 or 4 years, which would then imply that the issue lies with integrating back into society.


In [10]:
# retrieving desired data from 'years' by selecting the total values for all sexes and ages
years_total = years.iloc[years.index.get_level_values('Age') == 'Total']
years_total = years_total.iloc[years_total.index.get_level_values('Sex') == 'Total']
years_total


Out[10]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Total Total 69617 72030 78185 78908 79445 82234 79021 75463 75787
All years with recidivism Total Total 34545 35441 37674 37590 36930 37600 35646 34006 33705
0 years with recidivism Total Total 35072 36589 40511 41318 42515 44634 43375 41457 42082
1 year with recidivism Total Total 14442 15245 16617 16658 16525 17099 16306 15470 15274
2 years with recidivism Total Total 8765 8994 9530 9526 9320 9515 8932 8625 8657
3 years with recidivism Total Total 5792 5890 6117 6097 5858 5930 5571 5330 5317
4 years with recidivism Total Total 3847 3682 3789 3717 3665 3532 3399 3204 3107
5 years with recidivism Total Total 1699 1630 1621 1592 1562 1524 1438 1377 1350

Comment: This dataframe shows that there were 69,617 people who were released in 2002, and of those people 34,545 were arrested again within 5 years and 35,072 were not. This gives a 5-year recidivism rate of approximately 50% for the year 2002. It will be more interesting to convert this table into percent values before plotting it, so let's do that.


In [11]:
# group by Base Year (level 0), then apply a function that, for every column,
# represents the value of each row as a percentage of the first row

years_total = years_total.apply(lambda x: x.div(x.iloc[0]).mul(100))
years_total


Out[11]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Total Total 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000
All years with recidivism Total Total 49.621500 49.203110 48.185713 47.637755 46.484990 45.723180 45.109528 45.063144 44.473327
0 years with recidivism Total Total 50.378500 50.796890 51.814287 52.362245 53.515010 54.276820 54.890472 54.936856 55.526673
1 year with recidivism Total Total 20.744933 21.164792 21.253437 21.110661 20.800554 20.793103 20.635021 20.500113 20.153852
2 years with recidivism Total Total 12.590316 12.486464 12.189039 12.072287 11.731386 11.570640 11.303324 11.429442 11.422803
3 years with recidivism Total Total 8.319807 8.177148 7.823751 7.726720 7.373655 7.211129 7.050025 7.063064 7.015715
4 years with recidivism Total Total 5.525949 5.111759 4.846198 4.710549 4.613254 4.295060 4.301388 4.245789 4.099648
5 years with recidivism Total Total 2.440496 2.262946 2.073288 2.017539 1.966140 1.853248 1.819769 1.824735 1.781308

In [12]:
# dropping "all years" for the plot 
years_total = years_total.drop('All years (0-5)', level = 'Base Year')
years_total


Out[12]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years with recidivism Total Total 49.621500 49.203110 48.185713 47.637755 46.484990 45.723180 45.109528 45.063144 44.473327
0 years with recidivism Total Total 50.378500 50.796890 51.814287 52.362245 53.515010 54.276820 54.890472 54.936856 55.526673
1 year with recidivism Total Total 20.744933 21.164792 21.253437 21.110661 20.800554 20.793103 20.635021 20.500113 20.153852
2 years with recidivism Total Total 12.590316 12.486464 12.189039 12.072287 11.731386 11.570640 11.303324 11.429442 11.422803
3 years with recidivism Total Total 8.319807 8.177148 7.823751 7.726720 7.373655 7.211129 7.050025 7.063064 7.015715
4 years with recidivism Total Total 5.525949 5.111759 4.846198 4.710549 4.613254 4.295060 4.301388 4.245789 4.099648
5 years with recidivism Total Total 2.440496 2.262946 2.073288 2.017539 1.966140 1.853248 1.819769 1.824735 1.781308

Plot data


In [13]:
layout_total = dict(width=950, height=600,                                    # plot width/height
              yaxis={"title": "Recidivism (as % of released offenders)"},     # yaxis label
              title="Recidivism Rates Across Years 0-5",                      # title
              xaxis={"title": "Years"}                                        # xaxis label
             )

In [14]:
years_total.T.iplot(layout=layout_total)


What to make of this data

First of all, it is not surprising that the recidivism rates decrease as years from the release go by. This shows that it is the period immediately after being released from prison (especially the first year, which has a significantly greater rate than the rest) that is the most challenging for past offenders. This, in turn, implies that efforts should be spent on integrating offenders back into society.

There are a couple of interesting things to note here. We see that the 5-year recidivism rate has steadily decreased from 49.6% in 2002 to 44.5% in 2010, which in turn means that the percentage of people released that go back to society and do not get convicted again after 5 years has increased from 50.4% to 55.5%. That said, although there is a slight decrease in recidivism rates between 2008 and 2010, it does not appear as if the 2008 white paper has had a significant effect yet. Of course, 2 years is a short period to be expecting to see great results.


In [15]:
years_total.T.iplot(vline=['2008'], hspan={'y0':'44.473','y1':'55.53','color':'rgba(30,30,30,0.3)','fill':True,'opacity':.4})


Across sexes

Now let's take a look to see whether women were more affected by the increased funding for prison rehabilitation than men.


In [16]:
years_women = years.iloc[years.index.get_level_values('Sex') == 'Females']
years_women


Out[16]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Females Total 11290 11822 13444 13478 13796 14465 13929 13258 13130
All ages 5 years or older 11290 11822 13444 13478 13796 14465 13929 13258 13130
5-14 years 869 951 1172 1088 1061 1294 1148 1142 892
15-17 years 1268 1252 1624 1788 1757 1918 1822 1643 1445
18-20 years 1491 1529 1595 1554 1713 1781 1783 1733 1744
21-24 years 1241 1328 1468 1497 1464 1531 1428 1343 1359
25-29 years 1298 1214 1360 1356 1348 1435 1396 1297 1334
30-39 years 2368 2550 2745 2679 2775 2745 2556 2408 2376
40-49 years 1475 1636 1875 1887 2029 2061 1933 2022 2104
50-59 years 698 765 872 893 901 906 1032 962 1074
60 years or older 582 597 733 736 748 794 831 708 802
All years with recidivism Females Total 3815 4004 4310 4335 4389 4455 4199 4019 3935
All ages 5 years or older 3815 4004 4310 4335 4389 4455 4199 4019 3935
5-14 years 246 271 330 332 326 348 295 272 192
15-17 years 418 413 529 576 571 596 555 507 435
18-20 years 525 556 527 527 563 575 598 535 551
21-24 years 459 507 522 539 500 539 503 465 465
25-29 years 502 494 513 513 505 528 455 441 456
30-39 years 988 1049 1061 1004 992 922 900 907 855
40-49 years 489 526 592 594 681 683 604 611 689
50-59 years 129 140 169 177 164 174 211 203 223
60 years or older 59 48 67 73 87 90 78 78 69
0 years with recidivism Females Total 7475 7818 9134 9143 9407 10010 9730 9239 9195
All ages 5 years or older 7475 7818 9134 9143 9407 10010 9730 9239 9195
5-14 years 623 680 842 756 735 946 853 870 700
15-17 years 850 839 1095 1212 1186 1322 1267 1136 1010
18-20 years 966 973 1068 1027 1150 1206 1185 1198 1193
21-24 years 782 821 946 958 964 992 925 878 894
25-29 years 796 720 847 843 843 907 941 856 878
30-39 years 1380 1501 1684 1675 1783 1823 1656 1501 1521
... ... ... ... ... ... ... ... ... ... ... ...
3 years with recidivism Females 15-17 years 51 57 65 78 85 75 67 62 57
18-20 years 87 75 73 73 71 95 78 76 72
21-24 years 62 63 75 68 93 80 76 75 65
25-29 years 103 82 82 91 73 83 71 63 72
30-39 years 178 190 172 153 137 137 139 133 136
40-49 years 84 80 76 107 85 102 94 90 98
50-59 years 6 11 12 14 12 19 21 18 24
60 years or older 3 1 5 0 6 8 5 4 5
4 years with recidivism Females Total 366 335 373 340 353 362 319 317 299
All ages 5 years or older 366 335 373 340 353 362 319 317 299
5-14 years 14 15 17 14 16 21 11 7 6
15-17 years 24 26 33 35 49 39 32 34 28
18-20 years 50 37 34 39 37 42 49 44 46
21-24 years 46 50 56 45 45 42 43 45 36
25-29 years 65 53 65 61 53 49 48 36 34
30-39 years 119 108 96 92 96 109 92 103 83
40-49 years 37 40 62 46 52 53 32 35 54
50-59 years 7 5 9 6 5 5 11 12 11
60 years or older 4 1 1 2 0 2 1 1 1
5 years with recidivism Females Total 132 158 151 132 142 130 125 134 155
All ages 5 years or older 132 158 151 132 142 130 125 134 155
5-14 years 3 4 3 6 6 4 7 2 4
15-17 years 6 5 7 9 12 12 8 10 9
18-20 years 15 23 19 18 19 18 24 17 18
21-24 years 21 26 28 25 22 15 14 19 28
25-29 years 19 27 24 19 21 20 17 15 18
30-39 years 49 52 50 36 38 35 33 43 45
40-49 years 15 20 18 17 19 21 17 23 26
50-59 years 4 1 1 2 5 5 4 4 6
60 years or older 0 0 1 0 0 0 1 1 1

88 rows × 9 columns


In [17]:
# looking at recidivism across years for females
years_women = years_women.iloc[years_women.index.get_level_values('Age') == 'Total']
years_women


Out[17]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Females Total 11290 11822 13444 13478 13796 14465 13929 13258 13130
All years with recidivism Females Total 3815 4004 4310 4335 4389 4455 4199 4019 3935
0 years with recidivism Females Total 7475 7818 9134 9143 9407 10010 9730 9239 9195
1 year with recidivism Females Total 1790 1962 2193 2221 2253 2271 2216 2016 1954
2 years with recidivism Females Total 929 966 998 1022 1045 1068 958 999 980
3 years with recidivism Females Total 598 583 595 620 596 624 581 553 547
4 years with recidivism Females Total 366 335 373 340 353 362 319 317 299
5 years with recidivism Females Total 132 158 151 132 142 130 125 134 155

In [18]:
# group by Base Year (level 0), then apply a function that, for every column,
# represents the value of each row as a percentage of the first row
years_women = years_women.apply(lambda x: x.div(x.iloc[0]).mul(100))
years_women


Out[18]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Females Total 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000
All years with recidivism Females Total 33.790965 33.869058 32.058911 32.163526 31.813569 30.798479 30.145739 30.313773 29.969535
0 years with recidivism Females Total 66.209035 66.130942 67.941089 67.836474 68.186431 69.201521 69.854261 69.686227 70.030465
1 year with recidivism Females Total 15.854739 16.596177 16.312109 16.478706 16.330821 15.699965 15.909254 15.205913 14.881950
2 years with recidivism Females Total 8.228521 8.171206 7.423386 7.582727 7.574659 7.383339 6.877737 7.535073 7.463823
3 years with recidivism Females Total 5.296723 4.931484 4.425766 4.600089 4.320093 4.313861 4.171154 4.171067 4.166032
4 years with recidivism Females Total 3.241807 2.833700 2.774472 2.522629 2.558713 2.502592 2.290186 2.391009 2.277228
5 years with recidivism Females Total 1.169176 1.336491 1.123178 0.979374 1.029284 0.898721 0.897408 1.010711 1.180503

In [19]:
years_women = years_women.drop('All years (0-5)', level = 'Base Year')
years_women


Out[19]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years with recidivism Females Total 33.790965 33.869058 32.058911 32.163526 31.813569 30.798479 30.145739 30.313773 29.969535
0 years with recidivism Females Total 66.209035 66.130942 67.941089 67.836474 68.186431 69.201521 69.854261 69.686227 70.030465
1 year with recidivism Females Total 15.854739 16.596177 16.312109 16.478706 16.330821 15.699965 15.909254 15.205913 14.881950
2 years with recidivism Females Total 8.228521 8.171206 7.423386 7.582727 7.574659 7.383339 6.877737 7.535073 7.463823
3 years with recidivism Females Total 5.296723 4.931484 4.425766 4.600089 4.320093 4.313861 4.171154 4.171067 4.166032
4 years with recidivism Females Total 3.241807 2.833700 2.774472 2.522629 2.558713 2.502592 2.290186 2.391009 2.277228
5 years with recidivism Females Total 1.169176 1.336491 1.123178 0.979374 1.029284 0.898721 0.897408 1.010711 1.180503

In [20]:
# And now to the men
years_men = years.iloc[years.index.get_level_values('Sex') == 'Males']
years_men


Out[20]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Males Total 58327 60208 64741 65430 65649 67769 65092 62205 62657
All ages 5 years or older 58327 60208 64741 65430 65649 67769 65092 62205 62657
5-14 years 2516 2292 2449 2325 2535 2804 2841 2641 2342
15-17 years 5121 4995 5605 5440 5111 5587 5254 4621 4326
18-20 years 8382 8653 9127 8801 8856 9387 8816 8566 8609
21-24 years 8398 8459 8969 9065 9173 9426 8732 8409 8607
25-29 years 8093 8084 8464 8490 8306 8622 8182 7896 8118
30-39 years 12671 13445 14412 14530 14294 14218 13333 12431 12513
40-49 years 7481 7907 8670 9431 9645 9855 9909 9862 9976
50-59 years 3564 4069 4501 4692 4872 4834 5022 4794 5000
60 years or older 2101 2304 2544 2656 2857 3036 3003 2985 3166
All years with recidivism Males Total 30730 31437 33364 33255 32541 33145 31447 29987 29770
All ages 5 years or older 30730 31437 33364 33255 32541 33145 31447 29987 29770
5-14 years 1487 1353 1399 1359 1408 1452 1404 1308 1107
15-17 years 3489 3422 3778 3630 3437 3598 3362 2906 2749
18-20 years 5144 5341 5593 5260 5248 5460 5037 4844 4841
21-24 years 4869 4898 5148 5189 5150 5091 4775 4574 4660
25-29 years 4401 4395 4542 4499 4291 4423 4176 4217 4259
30-39 years 6786 6974 7378 7301 6975 7030 6392 5991 5909
40-49 years 3266 3538 3793 4173 4119 4218 4253 4183 4166
50-59 years 972 1153 1341 1409 1482 1412 1599 1503 1581
60 years or older 316 363 392 435 431 461 449 461 498
0 years with recidivism Males Total 27597 28771 31377 32175 33108 34624 33645 32218 32887
All ages 5 years or older 27597 28771 31377 32175 33108 34624 33645 32218 32887
5-14 years 1029 939 1050 966 1127 1352 1437 1333 1235
15-17 years 1632 1573 1827 1810 1674 1989 1892 1715 1577
18-20 years 3238 3312 3534 3541 3608 3927 3779 3722 3768
21-24 years 3529 3561 3821 3876 4023 4335 3957 3835 3947
25-29 years 3692 3689 3922 3991 4015 4199 4006 3679 3859
30-39 years 5885 6471 7034 7229 7319 7188 6941 6440 6604
... ... ... ... ... ... ... ... ... ... ... ...
3 years with recidivism Males 15-17 years 673 672 715 688 655 674 651 554 531
18-20 years 938 941 960 968 916 940 845 840 838
21-24 years 849 885 898 856 881 853 796 757 770
25-29 years 741 733 739 741 672 726 686 705 690
30-39 years 1139 1179 1274 1209 1120 1127 985 899 903
40-49 years 466 528 556 601 613 566 596 624 657
50-59 years 71 111 119 148 133 139 175 162 172
60 years or older 25 18 17 28 27 32 29 33 32
4 years with recidivism Males Total 3481 3347 3416 3377 3312 3170 3080 2887 2808
All ages 5 years or older 3481 3347 3416 3377 3312 3170 3080 2887 2808
5-14 years 202 176 182 144 156 160 129 120 107
15-17 years 506 459 470 489 476 459 417 364 310
18-20 years 574 583 622 573 570 533 510 507 479
21-24 years 565 535 525 583 547 482 493 450 451
25-29 years 488 446 474 437 444 436 443 422 448
30-39 years 790 772 773 737 702 666 623 606 594
40-49 years 315 324 305 342 340 363 379 355 327
50-59 years 38 46 58 60 68 65 78 56 86
60 years or older 3 6 7 12 9 6 8 7 6
5 years with recidivism Males Total 1567 1472 1470 1460 1420 1394 1313 1243 1195
All ages 5 years or older 1567 1472 1470 1460 1420 1394 1313 1243 1195
5-14 years 89 83 82 85 87 75 68 62 40
15-17 years 225 201 228 219 227 216 190 145 142
18-20 years 264 260 252 239 211 222 202 200 189
21-24 years 226 198 231 220 221 209 185 178 191
25-29 years 230 203 185 193 191 212 208 204 193
30-39 years 387 387 336 351 327 299 295 266 260
40-49 years 134 127 141 140 138 141 137 149 144
50-59 years 11 11 14 11 14 17 24 37 31
60 years or older 1 2 1 2 4 3 4 2 5

88 rows × 9 columns


In [21]:
# looking at recidivism across years for males
years_men = years_men.iloc[years_men.index.get_level_values('Age') == 'Total']
years_men


Out[21]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Males Total 58327 60208 64741 65430 65649 67769 65092 62205 62657
All years with recidivism Males Total 30730 31437 33364 33255 32541 33145 31447 29987 29770
0 years with recidivism Males Total 27597 28771 31377 32175 33108 34624 33645 32218 32887
1 year with recidivism Males Total 12652 13283 14424 14437 14272 14828 14090 13454 13320
2 years with recidivism Males Total 7836 8028 8532 8504 8275 8447 7974 7626 7677
3 years with recidivism Males Total 5194 5307 5522 5477 5262 5306 4990 4777 4770
4 years with recidivism Males Total 3481 3347 3416 3377 3312 3170 3080 2887 2808
5 years with recidivism Males Total 1567 1472 1470 1460 1420 1394 1313 1243 1195

In [22]:
years_men = years_men.apply(lambda x: x.div(x.iloc[0]).mul(100))
years_men


Out[22]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Males Total 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000
All years with recidivism Males Total 52.685720 52.213991 51.534576 50.825309 49.568158 48.908793 48.311620 48.206736 47.512648
0 years with recidivism Males Total 47.314280 47.786009 48.465424 49.174691 50.431842 51.091207 51.688380 51.793264 52.487352
1 year with recidivism Males Total 21.691498 22.061852 22.279545 22.064802 21.739859 21.880211 21.646285 21.628486 21.258598
2 years with recidivism Males Total 13.434601 13.333776 13.178666 12.997096 12.604914 12.464401 12.250353 12.259465 12.252422
3 years with recidivism Males Total 8.904967 8.814443 8.529371 8.370778 8.015354 7.829539 7.666073 7.679447 7.612876
4 years with recidivism Males Total 5.968077 5.559062 5.276409 5.161241 5.045012 4.677655 4.731764 4.641106 4.481542
5 years with recidivism Males Total 2.686577 2.444858 2.270586 2.231392 2.163018 2.056988 2.017145 1.998232 1.907209

In [23]:
# repeating cleaning 
years_men = years_men.drop('All years (0-5)', level='Base Year')
years_men


Out[23]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years with recidivism Males Total 52.685720 52.213991 51.534576 50.825309 49.568158 48.908793 48.311620 48.206736 47.512648
0 years with recidivism Males Total 47.314280 47.786009 48.465424 49.174691 50.431842 51.091207 51.688380 51.793264 52.487352
1 year with recidivism Males Total 21.691498 22.061852 22.279545 22.064802 21.739859 21.880211 21.646285 21.628486 21.258598
2 years with recidivism Males Total 13.434601 13.333776 13.178666 12.997096 12.604914 12.464401 12.250353 12.259465 12.252422
3 years with recidivism Males Total 8.904967 8.814443 8.529371 8.370778 8.015354 7.829539 7.666073 7.679447 7.612876
4 years with recidivism Males Total 5.968077 5.559062 5.276409 5.161241 5.045012 4.677655 4.731764 4.641106 4.481542
5 years with recidivism Males Total 2.686577 2.444858 2.270586 2.231392 2.163018 2.056988 2.017145 1.998232 1.907209

In [38]:
# Plotting the data together to compare

fig, ax = plt.subplots(1, 2, figsize=(12, 5), sharex=True)

years_women.T.plot(ax=ax[0], kind='line', legend=False)
years_men.T.plot(ax=ax[1], kind='line')
ax[0].set_title('Women with Recidivism')
ax[0].set_xlabel('Years')
ax[0].set_ylabel('Recidivism Rates (% of total released)')
ax[0].set_xticklabels(range(2002, 2011, 1), rotation='horizontal')

ax[1].set_title('Men with Recidivism')
ax[1].set_xlabel('Years')
ax[1].set_ylabel('Recidivism Rates (% of total released)')
ax[1].legend(bbox_to_anchor=(1, 1), loc='best', ncol=1)


Out[38]:
<matplotlib.legend.Legend at 0x11e9df1d0>

What to make of this data

From this data we note some striking things. First of all, in both cases total recidivism rates have decreased - from 34% to 30% for women and from 53% to 48% for men. The vast difference between female and male recidivism rates is also interesting - a difference of almost 20%.

Across Ages

From this data set I will look at the group 30-39, which has the largest sizes and may therefore be the more interesting to consider.


In [86]:
years_30 = years.iloc[years.index.get_level_values('Age') == '30-39 years']
years_30


Out[86]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Total 30-39 years 15039 15995 17157 17209 17069 16963 15889 14839 14889
Both sexes 30-39 years 15039 15995 17157 17209 17069 16963 15889 14839 14889
Males 30-39 years 12671 13445 14412 14530 14294 14218 13333 12431 12513
Females 30-39 years 2368 2550 2745 2679 2775 2745 2556 2408 2376
All years with recidivism Total 30-39 years 7774 8023 8439 8305 7967 7952 7292 6898 6764
Both sexes 30-39 years 7774 8023 8439 8305 7967 7952 7292 6898 6764
Males 30-39 years 6786 6974 7378 7301 6975 7030 6392 5991 5909
Females 30-39 years 988 1049 1061 1004 992 922 900 907 855
0 years with recidivism Total 30-39 years 7265 7972 8718 8904 9102 9011 8597 7941 8125
Both sexes 30-39 years 7265 7972 8718 8904 9102 9011 8597 7941 8125
Males 30-39 years 5885 6471 7034 7229 7319 7188 6941 6440 6604
Females 30-39 years 1380 1501 1684 1675 1783 1823 1656 1501 1521
1 year with recidivism Total 30-39 years 3176 3345 3575 3638 3555 3545 3300 3093 3004
Both sexes 30-39 years 3176 3345 3575 3638 3555 3545 3300 3093 3004
Males 30-39 years 2791 2904 3104 3179 3079 3126 2896 2682 2630
Females 30-39 years 385 441 471 459 476 419 404 411 374
2 years with recidivism Total 30-39 years 1936 1990 2163 2089 1992 2034 1825 1755 1739
Both sexes 30-39 years 1936 1990 2163 2089 1992 2034 1825 1755 1739
Males 30-39 years 1679 1732 1891 1825 1747 1812 1593 1538 1522
Females 30-39 years 257 258 272 264 245 222 232 217 217
3 years with recidivism Total 30-39 years 1317 1369 1446 1362 1257 1264 1124 1032 1039
Both sexes 30-39 years 1317 1369 1446 1362 1257 1264 1124 1032 1039
Males 30-39 years 1139 1179 1274 1209 1120 1127 985 899 903
Females 30-39 years 178 190 172 153 137 137 139 133 136
4 years with recidivism Total 30-39 years 909 880 869 829 798 775 715 709 677
Both sexes 30-39 years 909 880 869 829 798 775 715 709 677
Males 30-39 years 790 772 773 737 702 666 623 606 594
Females 30-39 years 119 108 96 92 96 109 92 103 83
5 years with recidivism Total 30-39 years 436 439 386 387 365 334 328 309 305
Both sexes 30-39 years 436 439 386 387 365 334 328 309 305
Males 30-39 years 387 387 336 351 327 299 295 266 260
Females 30-39 years 49 52 50 36 38 35 33 43 45

In [65]:
years_18 = years_18.apply(lambda x: x.div(x.iloc[0]).mul(100))
years_18


Out[65]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Total 18-20 years 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000
Both sexes 18-20 years 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000 100.000000
Males 18-20 years 84.898207 84.983304 85.124044 84.992757 83.792223 84.052650 83.177658 83.173124 83.154641
Females 18-20 years 15.101793 15.016696 14.875956 15.007243 16.207777 15.947350 16.822342 16.826876 16.845359
All years with recidivism Total 18-20 years 57.419224 57.915930 57.078903 55.886045 54.981550 54.038324 53.165393 52.228372 52.081522
Both sexes 18-20 years 57.419224 57.915930 57.078903 55.886045 54.981550 54.038324 53.165393 52.228372 52.081522
Males 18-20 years 52.101691 52.455313 52.163775 50.796717 49.654650 48.889685 47.523351 47.033693 46.759393
Females 18-20 years 5.317533 5.460617 4.915128 5.089329 5.326899 5.148639 5.642042 5.194679 5.322129
0 years with recidivism Total 18-20 years 42.580776 42.084070 42.921097 44.113955 45.018450 45.961676 46.834607 47.771628 47.918478
Both sexes 18-20 years 42.580776 42.084070 42.921097 44.113955 45.018450 45.961676 46.834607 47.771628 47.918478
Males 18-20 years 32.796516 32.527991 32.960269 34.196041 34.137572 35.162966 35.654307 36.139431 36.395248
Females 18-20 years 9.784260 9.556079 9.960828 9.917914 10.880878 10.798711 11.180300 11.632197 11.523230
1 year with recidivism Total 18-20 years 23.022384 23.708505 24.277187 22.549493 23.928470 23.862822 22.888952 21.934168 22.756689
Both sexes 18-20 years 23.022384 23.708505 24.277187 22.549493 23.928470 23.862822 22.888952 21.934168 22.756689
Males 18-20 years 20.662413 20.948733 21.441895 19.942057 20.929132 21.257163 19.964148 19.506748 19.897614
Females 18-20 years 2.359972 2.759772 2.835292 2.607436 2.999338 2.605659 2.924804 2.427420 2.859075
2 years with recidivism Total 18-20 years 14.868834 15.360440 14.521544 14.891357 13.795061 13.610315 14.161713 13.943101 13.464696
Both sexes 18-20 years 14.868834 15.360440 14.521544 14.891357 13.795061 13.610315 14.161713 13.943101 13.464696
Males 18-20 years 13.450825 13.985465 13.616863 13.664896 12.669127 12.455229 12.869139 12.506069 12.315271
Females 18-20 years 1.418009 1.374975 0.904682 1.226461 1.125934 1.155086 1.292575 1.437033 1.149425
3 years with recidivism Total 18-20 years 10.381849 9.978393 9.634397 10.053114 9.338632 9.267550 8.708369 8.894067 8.789723
Both sexes 18-20 years 10.381849 9.978393 9.634397 10.053114 9.338632 9.267550 8.708369 8.894067 8.789723
Males 18-20 years 9.500658 9.241799 8.953553 9.348141 8.666856 8.416905 7.972450 8.156132 8.094272
Females 18-20 years 0.881191 0.736594 0.680843 0.704973 0.671776 0.850645 0.735918 0.737936 0.695451
4 years with recidivism Total 18-20 years 6.320267 6.089177 6.118262 5.910188 5.743211 5.148639 5.274082 5.350034 5.070994
Both sexes 18-20 years 6.320267 6.089177 6.118262 5.910188 5.743211 5.148639 5.274082 5.350034 5.070994
Males 18-20 years 5.813836 5.725791 5.801157 5.533559 5.393131 4.772564 4.811775 4.922808 4.626678
Females 18-20 years 0.506432 0.363386 0.317105 0.376630 0.350080 0.376074 0.462308 0.427226 0.444316
5 years with recidivism Total 18-20 years 2.825889 2.779415 2.527514 2.481893 2.176176 2.148997 2.132277 2.107001 1.999420
Both sexes 18-20 years 2.825889 2.779415 2.527514 2.481893 2.176176 2.148997 2.132277 2.107001 1.999420
Males 18-20 years 2.673959 2.553526 2.350308 2.308064 1.996405 1.987822 1.905840 1.941936 1.825558
Females 18-20 years 0.151930 0.225889 0.177206 0.173829 0.179771 0.161175 0.226436 0.165065 0.173863

In [87]:
# dropping both sexes, because it is equal to total
years_30 = years_30.drop('Both sexes', level = 'Sex')
years_30 = years_30.drop('Total', level = 'Sex')
years_30


Out[87]:
2002 2003 2004 2005 2006 2007 2008 2009 2010
Base Year Sex Age
All years (0-5) Males 30-39 years 12671 13445 14412 14530 14294 14218 13333 12431 12513
Females 30-39 years 2368 2550 2745 2679 2775 2745 2556 2408 2376
All years with recidivism Males 30-39 years 6786 6974 7378 7301 6975 7030 6392 5991 5909
Females 30-39 years 988 1049 1061 1004 992 922 900 907 855
0 years with recidivism Males 30-39 years 5885 6471 7034 7229 7319 7188 6941 6440 6604
Females 30-39 years 1380 1501 1684 1675 1783 1823 1656 1501 1521
1 year with recidivism Males 30-39 years 2791 2904 3104 3179 3079 3126 2896 2682 2630
Females 30-39 years 385 441 471 459 476 419 404 411 374
2 years with recidivism Males 30-39 years 1679 1732 1891 1825 1747 1812 1593 1538 1522
Females 30-39 years 257 258 272 264 245 222 232 217 217
3 years with recidivism Males 30-39 years 1139 1179 1274 1209 1120 1127 985 899 903
Females 30-39 years 178 190 172 153 137 137 139 133 136
4 years with recidivism Males 30-39 years 790 772 773 737 702 666 623 606 594
Females 30-39 years 119 108 96 92 96 109 92 103 83
5 years with recidivism Males 30-39 years 387 387 336 351 327 299 295 266 260
Females 30-39 years 49 52 50 36 38 35 33 43 45

In [92]:
# plotting the data

layout_30 = dict(width=950, height=600,                                # plot width/height
              yaxis={"title": "Recidivism for 30-39"},                 # yaxis label
              title="Recidivism Rates for Ages 30-39",                 # title
              xaxis={"title": "Years"}                                 # xaxis label
             )

years_30.T.iplot(layout=layout_30)


From this data we actually see a decrease across all years after 2007, though it seems unlikely that this is due to the white paper.

Persons Charged by Criminal Offense

This data set is interesting because it gives a view into what groups of offenders have higher recidivism, and also provides some insight into whether people who were arrested for one type of offence are more likely to be arrested for the same type or for a more or less severe crime. One would perhaps assume that an investments in rehabilitation would decrease recidivism among violent crimes more than it would economic crimes, for instance.

Look at data


In [40]:
offences = pd.read_excel(url, header=[2], index_col=[0,1], sheetname='By_Principal_Offense')
offences


Out[40]:
2002 Unnamed: 1 2003 Unnamed: 3 2004 Unnamed: 5 2005 Unnamed: 7 2006 Unnamed: 9 2007 Unnamed: 11 2008 Unnamed: 13 2009 Unnamed: 15 2010 Unnamed: 17
NaN NaN Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent)
All groups of offences Total 69617 100 72030 100 78185 100 78908 100 79445 100 82234 100 79021 100 75463 100 .. ..
All groups of offences at recidivism 34545 49.6 35441 49.2 37674 48.2 37590 47.6 36930 46.5 37600 45.7 35646 45.1 34006 45.1 .. ..
Economic offences at recidivism 873 1.3 818 1.1 805 1 814 1 845 1.1 847 1 735 0.9 660 0.9 .. ..
Other offences for profit at recidivism 7734 11.1 7498 10.4 7488 9.6 7034 8.9 6780 8.5 6549 8 6128 7.8 5942 7.9 .. ..
Offences of violence at recidivism 3929 5.6 3977 5.5 4251 5.4 4366 5.5 4303 5.4 4369 5.3 4212 5.3 3992 5.3 .. ..
Sexual offences at recidivism 288 0.4 308 0.4 329 0.4 329 0.4 385 0.5 406 0.5 399 0.5 354 0.5 .. ..
Offences of narcotics at recidivism 6769 9.7 6945 9.6 7433 9.5 7586 9.6 7768 9.8 8260 10 8001 10.1 7996 10.6 .. ..
Damage to property at recidivism 696 1 715 1 702 0.9 730 0.9 686 0.9 683 0.8 645 0.8 595 0.8 .. ..
Environment offences at recidivism 143 0.2 143 0.2 164 0.2 143 0.2 157 0.2 150 0.2 149 0.2 159 0.2 .. ..
Traffic offences at recidivism 9130 13.1 9880 13.7 10808 13.8 10966 13.9 10462 13.2 10636 12.9 10000 12.7 9172 12.2 .. ..
Other offences (incl. Work environment) at recidivism 4983 7.2 5157 7.2 5694 7.3 5622 7.1 5544 7 5700 6.9 5377 6.8 5136 6.8 .. ..
No group of offences (without recidivism) 35072 50.4 36589 50.8 40511 51.8 41318 52.4 42515 53.5 44634 54.3 43375 54.9 41457 54.9 .. ..
Economic offences Total 1869 100 2074 100 2550 100 2552 100 2393 100 2166 100 2005 100 1918 100 .. ..
All groups of offences at recidivism 771 41.3 933 45 1077 42.2 1032 40.4 1018 42.5 876 40.4 795 39.7 754 39.3 .. ..
Economic offences at recidivism 133 7.1 180 8.7 183 7.2 182 7.1 198 8.3 174 8 159 7.9 147 7.7 .. ..
Other offences for profit at recidivism 166 8.9 171 8.2 185 7.3 172 6.7 163 6.8 134 6.2 105 5.2 131 6.8 .. ..
Offences of violence at recidivism 52 2.8 51 2.5 57 2.2 59 2.3 56 2.3 45 2.1 47 2.3 45 2.3 .. ..
Sexual offences at recidivism 10 0.5 9 0.4 1 0 10 0.4 6 0.3 8 0.4 10 0.5 8 0.4 .. ..
Offences of narcotics at recidivism 74 4 87 4.2 120 4.7 100 3.9 119 5 92 4.2 99 4.9 73 3.8 .. ..
Damage to property at recidivism 4 0.2 6 0.3 7 0.3 8 0.3 10 0.4 6 0.3 4 0.2 3 0.2 .. ..
Environment offences at recidivism 5 0.3 6 0.3 5 0.2 6 0.2 10 0.4 2 0.1 5 0.2 2 0.1 .. ..
Traffic offences at recidivism 215 11.5 307 14.8 375 14.7 352 13.8 328 13.7 297 13.7 242 12.1 242 12.6 .. ..
Other offences (incl. Work environment) at recidivism 112 6 116 5.6 144 5.6 143 5.6 128 5.3 118 5.4 124 6.2 103 5.4 .. ..
No group of offences (without recidivism) 1098 58.7 1141 55 1473 57.8 1520 59.6 1375 57.5 1290 59.6 1210 60.3 1164 60.7 .. ..
Other offences for profit Total 14621 100 14656 100 16668 100 15951 100 15131 100 15275 100 13947 100 13827 100 .. ..
All groups of offences at recidivism 8561 58.6 8678 59.2 9693 58.2 9016 56.5 8352 55.2 8417 55.1 7445 53.4 7236 52.3 .. ..
Economic offences at recidivism 177 1.2 171 1.2 160 1 158 1 149 1 148 1 108 0.8 87 0.6 .. ..
Other offences for profit at recidivism 3423 23.4 3428 23.4 3516 21.1 3202 20.1 2935 19.4 2861 18.7 2570 18.4 2515 18.2 .. ..
Offences of violence at recidivism 886 6.1 842 5.7 947 5.7 867 5.4 861 5.7 898 5.9 777 5.6 753 5.4 .. ..
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
Environment offences Environment offences at recidivism 34 4.8 32 4.3 30 4.1 29 4 29 3.8 27 4.1 25 4.1 33 5.2 .. ..
Traffic offences at recidivism 63 8.8 58 7.8 61 8.2 54 7.5 64 8.3 52 8 36 5.9 50 7.9 .. ..
Other offences (incl. Work environment) at recidivism 28 3.9 34 4.5 22 3 21 2.9 32 4.2 26 4 23 3.8 26 4.1 .. ..
No group of offences (without recidivism) 552 77.5 582 77.8 593 80.1 581 80.2 607 79.1 521 79.8 489 80.7 496 78.2 .. ..
Traffic offences Total 23243 100 25403 100 26780 100 26935 100 27402 100 29266 100 29644 100 28221 100 .. ..
All groups of offences at recidivism 8859 38.1 9568 37.7 9781 36.5 9985 37.1 9638 35.2 10097 34.5 10092 34 9608 34 .. ..
Economic offences at recidivism 243 1 210 0.8 203 0.8 232 0.9 247 0.9 274 0.9 247 0.8 236 0.8 .. ..
Other offences for profit at recidivism 1069 4.6 1105 4.3 1041 3.9 957 3.6 965 3.5 916 3.1 963 3.2 934 3.3 .. ..
Offences of violence at recidivism 768 3.3 788 3.1 785 2.9 832 3.1 761 2.8 823 2.8 825 2.8 782 2.8 .. ..
Sexual offences at recidivism 72 0.3 74 0.3 65 0.2 87 0.3 93 0.3 99 0.3 123 0.4 78 0.3 .. ..
Offences of narcotics at recidivism 997 4.3 1068 4.2 1056 3.9 1065 4 1105 4 1119 3.8 1224 4.1 1305 4.6 .. ..
Damage to property at recidivism 137 0.6 144 0.6 113 0.4 133 0.5 109 0.4 112 0.4 123 0.4 105 0.4 .. ..
Environment offences at recidivism 51 0.2 47 0.2 49 0.2 44 0.2 58 0.2 60 0.2 61 0.2 64 0.2 .. ..
Traffic offences at recidivism 4292 18.5 4793 18.9 5042 18.8 5344 19.8 4967 18.1 5257 18 5168 17.4 4751 16.8 .. ..
Other offences (incl. Work environment) at recidivism 1230 5.3 1339 5.3 1427 5.3 1291 4.8 1333 4.9 1437 4.9 1358 4.6 1353 4.8 .. ..
No group of offences (without recidivism) 14384 61.9 15835 62.3 16999 63.5 16950 62.9 17764 64.8 19169 65.5 19552 66 18613 66 .. ..
Other offences (incl. Work environment) Total 10683 100 10864 100 11742 100 12775 100 13275 100 13741 100 12462 100 10874 100 .. ..
All groups of offences at recidivism 4710 44.1 4760 43.8 5049 43 5476 42.9 5517 41.6 5638 41 5200 41.7 4586 42.2 .. ..
Economic offences at recidivism 145 1.4 99 0.9 105 0.9 99 0.8 133 1 108 0.8 102 0.8 78 0.7 .. ..
Other offences for profit at recidivism 701 6.6 613 5.6 653 5.6 661 5.2 649 4.9 696 5.1 634 5.1 580 5.3 .. ..
Offences of violence at recidivism 611 5.7 667 6.1 663 5.6 768 6 710 5.3 785 5.7 713 5.7 654 6 .. ..
Sexual offences at recidivism 41 0.4 47 0.4 59 0.5 49 0.4 58 0.4 66 0.5 53 0.4 51 0.5 .. ..
Offences of narcotics at recidivism 592 5.5 645 5.9 699 6 810 6.3 883 6.7 923 6.7 883 7.1 818 7.5 .. ..
Damage to property at recidivism 129 1.2 106 1 134 1.1 138 1.1 134 1 137 1 120 1 98 0.9 .. ..
Environment offences at recidivism 24 0.2 25 0.2 30 0.3 28 0.2 28 0.2 27 0.2 29 0.2 27 0.2 .. ..
Traffic offences at recidivism 1240 11.6 1275 11.7 1395 11.9 1469 11.5 1482 11.2 1453 10.6 1327 10.6 1108 10.2 .. ..
Other offences (incl. Work environment) at recidivism 1227 11.5 1283 11.8 1311 11.2 1454 11.4 1440 10.8 1443 10.5 1339 10.7 1172 10.8 .. ..
No group of offences (without recidivism) 5973 55.9 6104 56.2 6693 57 7299 57.1 7758 58.4 8103 59 7262 58.3 6288 57.8 .. ..
No group of offences (without recidivism) NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Persons charged only covers charged persons with a valid Norwegian personal identification number, and at least one stipulated time of crime in the base year for the principal offence. A charged person is considered to have recidivism if they have a new charge brought against them in the subsequent five-year period, and if at least one of the principal offences is committed after the principal offence in the base year. No group of offences (without recidivism) NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

123 rows × 18 columns

Clean data


In [43]:
# renaming the index
offences.index.names = ['Principal Offence', 'Recidivism Offence']

# show the first 5 rows
offences.head()


Out[43]:
2002 Unnamed: 1 2003 Unnamed: 3 2004 Unnamed: 5 2005 Unnamed: 7 2006 Unnamed: 9 2007 Unnamed: 11 2008 Unnamed: 13 2009 Unnamed: 15 2010 Unnamed: 17
Principal Offence Recidivism Offence
NaN NaN Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent) Persons charged Persons charged (per cent)
All groups of offences Total 69617 100 72030 100 78185 100 78908 100 79445 100 82234 100 79021 100 75463 100 .. ..
All groups of offences at recidivism 34545 49.6 35441 49.2 37674 48.2 37590 47.6 36930 46.5 37600 45.7 35646 45.1 34006 45.1 .. ..
Economic offences at recidivism 873 1.3 818 1.1 805 1 814 1 845 1.1 847 1 735 0.9 660 0.9 .. ..
Other offences for profit at recidivism 7734 11.1 7498 10.4 7488 9.6 7034 8.9 6780 8.5 6549 8 6128 7.8 5942 7.9 .. ..

In [48]:
# dropping columns that have raw numbers, keeping those with percentages
offences = offences.drop(offences.columns[[0, 2, 4, 6, 8, 10, 12, 14, 16, 17]], axis=1)
offences


Out[48]:
Unnamed: 1 Unnamed: 3 Unnamed: 5 Unnamed: 7 Unnamed: 9 Unnamed: 11 Unnamed: 13 Unnamed: 15
Principal Offence Recidivism Offence
NaN NaN Persons charged (per cent) Persons charged (per cent) Persons charged (per cent) Persons charged (per cent) Persons charged (per cent) Persons charged (per cent) Persons charged (per cent) Persons charged (per cent)
All groups of offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 49.6 49.2 48.2 47.6 46.5 45.7 45.1 45.1
Economic offences at recidivism 1.3 1.1 1 1 1.1 1 0.9 0.9
Other offences for profit at recidivism 11.1 10.4 9.6 8.9 8.5 8 7.8 7.9
Offences of violence at recidivism 5.6 5.5 5.4 5.5 5.4 5.3 5.3 5.3
Sexual offences at recidivism 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5
Offences of narcotics at recidivism 9.7 9.6 9.5 9.6 9.8 10 10.1 10.6
Damage to property at recidivism 1 1 0.9 0.9 0.9 0.8 0.8 0.8
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 13.1 13.7 13.8 13.9 13.2 12.9 12.7 12.2
Other offences (incl. Work environment) at recidivism 7.2 7.2 7.3 7.1 7 6.9 6.8 6.8
No group of offences (without recidivism) 50.4 50.8 51.8 52.4 53.5 54.3 54.9 54.9
Economic offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 41.3 45 42.2 40.4 42.5 40.4 39.7 39.3
Economic offences at recidivism 7.1 8.7 7.2 7.1 8.3 8 7.9 7.7
Other offences for profit at recidivism 8.9 8.2 7.3 6.7 6.8 6.2 5.2 6.8
Offences of violence at recidivism 2.8 2.5 2.2 2.3 2.3 2.1 2.3 2.3
Sexual offences at recidivism 0.5 0.4 0 0.4 0.3 0.4 0.5 0.4
Offences of narcotics at recidivism 4 4.2 4.7 3.9 5 4.2 4.9 3.8
Damage to property at recidivism 0.2 0.3 0.3 0.3 0.4 0.3 0.2 0.2
Environment offences at recidivism 0.3 0.3 0.2 0.2 0.4 0.1 0.2 0.1
Traffic offences at recidivism 11.5 14.8 14.7 13.8 13.7 13.7 12.1 12.6
Other offences (incl. Work environment) at recidivism 6 5.6 5.6 5.6 5.3 5.4 6.2 5.4
No group of offences (without recidivism) 58.7 55 57.8 59.6 57.5 59.6 60.3 60.7
Other offences for profit Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 58.6 59.2 58.2 56.5 55.2 55.1 53.4 52.3
Economic offences at recidivism 1.2 1.2 1 1 1 1 0.8 0.6
Other offences for profit at recidivism 23.4 23.4 21.1 20.1 19.4 18.7 18.4 18.2
Offences of violence at recidivism 6.1 5.7 5.7 5.4 5.7 5.9 5.6 5.4
... ... ... ... ... ... ... ... ... ...
Environment offences Environment offences at recidivism 4.8 4.3 4.1 4 3.8 4.1 4.1 5.2
Traffic offences at recidivism 8.8 7.8 8.2 7.5 8.3 8 5.9 7.9
Other offences (incl. Work environment) at recidivism 3.9 4.5 3 2.9 4.2 4 3.8 4.1
No group of offences (without recidivism) 77.5 77.8 80.1 80.2 79.1 79.8 80.7 78.2
Traffic offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 38.1 37.7 36.5 37.1 35.2 34.5 34 34
Economic offences at recidivism 1 0.8 0.8 0.9 0.9 0.9 0.8 0.8
Other offences for profit at recidivism 4.6 4.3 3.9 3.6 3.5 3.1 3.2 3.3
Offences of violence at recidivism 3.3 3.1 2.9 3.1 2.8 2.8 2.8 2.8
Sexual offences at recidivism 0.3 0.3 0.2 0.3 0.3 0.3 0.4 0.3
Offences of narcotics at recidivism 4.3 4.2 3.9 4 4 3.8 4.1 4.6
Damage to property at recidivism 0.6 0.6 0.4 0.5 0.4 0.4 0.4 0.4
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 18.5 18.9 18.8 19.8 18.1 18 17.4 16.8
Other offences (incl. Work environment) at recidivism 5.3 5.3 5.3 4.8 4.9 4.9 4.6 4.8
No group of offences (without recidivism) 61.9 62.3 63.5 62.9 64.8 65.5 66 66
Other offences (incl. Work environment) Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 44.1 43.8 43 42.9 41.6 41 41.7 42.2
Economic offences at recidivism 1.4 0.9 0.9 0.8 1 0.8 0.8 0.7
Other offences for profit at recidivism 6.6 5.6 5.6 5.2 4.9 5.1 5.1 5.3
Offences of violence at recidivism 5.7 6.1 5.6 6 5.3 5.7 5.7 6
Sexual offences at recidivism 0.4 0.4 0.5 0.4 0.4 0.5 0.4 0.5
Offences of narcotics at recidivism 5.5 5.9 6 6.3 6.7 6.7 7.1 7.5
Damage to property at recidivism 1.2 1 1.1 1.1 1 1 1 0.9
Environment offences at recidivism 0.2 0.2 0.3 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 11.6 11.7 11.9 11.5 11.2 10.6 10.6 10.2
Other offences (incl. Work environment) at recidivism 11.5 11.8 11.2 11.4 10.8 10.5 10.7 10.8
No group of offences (without recidivism) 55.9 56.2 57 57.1 58.4 59 58.3 57.8
No group of offences (without recidivism) NaN NaN NaN NaN NaN NaN NaN NaN
Persons charged only covers charged persons with a valid Norwegian personal identification number, and at least one stipulated time of crime in the base year for the principal offence. A charged person is considered to have recidivism if they have a new charge brought against them in the subsequent five-year period, and if at least one of the principal offences is committed after the principal offence in the base year. No group of offences (without recidivism) NaN NaN NaN NaN NaN NaN NaN NaN

123 rows × 8 columns


In [51]:
# dropping NaN
offences = offences.loc[~(offences.index.labels[0] == -1)]
offences = offences.drop('Persons charged only covers charged persons with a valid Norwegian personal identification number, and at least one stipulated time of crime in the base year for the principal offence. A charged person is considered to have recidivism if they have a new charge brought against them in the subsequent five-year period, and if at least one of the principal offences is committed after the principal offence in the base year.', level='Principal Offence')
offences


Out[51]:
Unnamed: 1 Unnamed: 3 Unnamed: 5 Unnamed: 7 Unnamed: 9 Unnamed: 11 Unnamed: 13 Unnamed: 15
Principal Offence Recidivism Offence
All groups of offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 49.6 49.2 48.2 47.6 46.5 45.7 45.1 45.1
Economic offences at recidivism 1.3 1.1 1 1 1.1 1 0.9 0.9
Other offences for profit at recidivism 11.1 10.4 9.6 8.9 8.5 8 7.8 7.9
Offences of violence at recidivism 5.6 5.5 5.4 5.5 5.4 5.3 5.3 5.3
Sexual offences at recidivism 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5
Offences of narcotics at recidivism 9.7 9.6 9.5 9.6 9.8 10 10.1 10.6
Damage to property at recidivism 1 1 0.9 0.9 0.9 0.8 0.8 0.8
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 13.1 13.7 13.8 13.9 13.2 12.9 12.7 12.2
Other offences (incl. Work environment) at recidivism 7.2 7.2 7.3 7.1 7 6.9 6.8 6.8
No group of offences (without recidivism) 50.4 50.8 51.8 52.4 53.5 54.3 54.9 54.9
Economic offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 41.3 45 42.2 40.4 42.5 40.4 39.7 39.3
Economic offences at recidivism 7.1 8.7 7.2 7.1 8.3 8 7.9 7.7
Other offences for profit at recidivism 8.9 8.2 7.3 6.7 6.8 6.2 5.2 6.8
Offences of violence at recidivism 2.8 2.5 2.2 2.3 2.3 2.1 2.3 2.3
Sexual offences at recidivism 0.5 0.4 0 0.4 0.3 0.4 0.5 0.4
Offences of narcotics at recidivism 4 4.2 4.7 3.9 5 4.2 4.9 3.8
Damage to property at recidivism 0.2 0.3 0.3 0.3 0.4 0.3 0.2 0.2
Environment offences at recidivism 0.3 0.3 0.2 0.2 0.4 0.1 0.2 0.1
Traffic offences at recidivism 11.5 14.8 14.7 13.8 13.7 13.7 12.1 12.6
Other offences (incl. Work environment) at recidivism 6 5.6 5.6 5.6 5.3 5.4 6.2 5.4
No group of offences (without recidivism) 58.7 55 57.8 59.6 57.5 59.6 60.3 60.7
Other offences for profit Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 58.6 59.2 58.2 56.5 55.2 55.1 53.4 52.3
Economic offences at recidivism 1.2 1.2 1 1 1 1 0.8 0.6
Other offences for profit at recidivism 23.4 23.4 21.1 20.1 19.4 18.7 18.4 18.2
Offences of violence at recidivism 6.1 5.7 5.7 5.4 5.7 5.9 5.6 5.4
Sexual offences at recidivism 0.4 0.4 0.4 0.3 0.5 0.5 0.4 0.4
... ... ... ... ... ... ... ... ... ...
Environment offences Damage to property at recidivism .. 0.4 .. 0.1 .. 0.5 .. ..
Environment offences at recidivism 4.8 4.3 4.1 4 3.8 4.1 4.1 5.2
Traffic offences at recidivism 8.8 7.8 8.2 7.5 8.3 8 5.9 7.9
Other offences (incl. Work environment) at recidivism 3.9 4.5 3 2.9 4.2 4 3.8 4.1
No group of offences (without recidivism) 77.5 77.8 80.1 80.2 79.1 79.8 80.7 78.2
Traffic offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 38.1 37.7 36.5 37.1 35.2 34.5 34 34
Economic offences at recidivism 1 0.8 0.8 0.9 0.9 0.9 0.8 0.8
Other offences for profit at recidivism 4.6 4.3 3.9 3.6 3.5 3.1 3.2 3.3
Offences of violence at recidivism 3.3 3.1 2.9 3.1 2.8 2.8 2.8 2.8
Sexual offences at recidivism 0.3 0.3 0.2 0.3 0.3 0.3 0.4 0.3
Offences of narcotics at recidivism 4.3 4.2 3.9 4 4 3.8 4.1 4.6
Damage to property at recidivism 0.6 0.6 0.4 0.5 0.4 0.4 0.4 0.4
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 18.5 18.9 18.8 19.8 18.1 18 17.4 16.8
Other offences (incl. Work environment) at recidivism 5.3 5.3 5.3 4.8 4.9 4.9 4.6 4.8
No group of offences (without recidivism) 61.9 62.3 63.5 62.9 64.8 65.5 66 66
Other offences (incl. Work environment) Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 44.1 43.8 43 42.9 41.6 41 41.7 42.2
Economic offences at recidivism 1.4 0.9 0.9 0.8 1 0.8 0.8 0.7
Other offences for profit at recidivism 6.6 5.6 5.6 5.2 4.9 5.1 5.1 5.3
Offences of violence at recidivism 5.7 6.1 5.6 6 5.3 5.7 5.7 6
Sexual offences at recidivism 0.4 0.4 0.5 0.4 0.4 0.5 0.4 0.5
Offences of narcotics at recidivism 5.5 5.9 6 6.3 6.7 6.7 7.1 7.5
Damage to property at recidivism 1.2 1 1.1 1.1 1 1 1 0.9
Environment offences at recidivism 0.2 0.2 0.3 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 11.6 11.7 11.9 11.5 11.2 10.6 10.6 10.2
Other offences (incl. Work environment) at recidivism 11.5 11.8 11.2 11.4 10.8 10.5 10.7 10.8
No group of offences (without recidivism) 55.9 56.2 57 57.1 58.4 59 58.3 57.8
No group of offences (without recidivism) NaN NaN NaN NaN NaN NaN NaN NaN

121 rows × 8 columns


In [52]:
# renaming the columns

offences = offences.rename(columns={'Unnamed: 1': '2002',
                         'Unnamed: 3': '2003',
                         'Unnamed: 5': '2004',
                         'Unnamed: 7': '2005',
                         'Unnamed: 9': '2006',
                         'Unnamed: 11': '2007',
                         'Unnamed: 13': '2008',
                         'Unnamed: 15': '2009'})

offences


Out[52]:
2002 2003 2004 2005 2006 2007 2008 2009
Principal Offence Recidivism Offence
All groups of offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 49.6 49.2 48.2 47.6 46.5 45.7 45.1 45.1
Economic offences at recidivism 1.3 1.1 1 1 1.1 1 0.9 0.9
Other offences for profit at recidivism 11.1 10.4 9.6 8.9 8.5 8 7.8 7.9
Offences of violence at recidivism 5.6 5.5 5.4 5.5 5.4 5.3 5.3 5.3
Sexual offences at recidivism 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5
Offences of narcotics at recidivism 9.7 9.6 9.5 9.6 9.8 10 10.1 10.6
Damage to property at recidivism 1 1 0.9 0.9 0.9 0.8 0.8 0.8
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 13.1 13.7 13.8 13.9 13.2 12.9 12.7 12.2
Other offences (incl. Work environment) at recidivism 7.2 7.2 7.3 7.1 7 6.9 6.8 6.8
No group of offences (without recidivism) 50.4 50.8 51.8 52.4 53.5 54.3 54.9 54.9
Economic offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 41.3 45 42.2 40.4 42.5 40.4 39.7 39.3
Economic offences at recidivism 7.1 8.7 7.2 7.1 8.3 8 7.9 7.7
Other offences for profit at recidivism 8.9 8.2 7.3 6.7 6.8 6.2 5.2 6.8
Offences of violence at recidivism 2.8 2.5 2.2 2.3 2.3 2.1 2.3 2.3
Sexual offences at recidivism 0.5 0.4 0 0.4 0.3 0.4 0.5 0.4
Offences of narcotics at recidivism 4 4.2 4.7 3.9 5 4.2 4.9 3.8
Damage to property at recidivism 0.2 0.3 0.3 0.3 0.4 0.3 0.2 0.2
Environment offences at recidivism 0.3 0.3 0.2 0.2 0.4 0.1 0.2 0.1
Traffic offences at recidivism 11.5 14.8 14.7 13.8 13.7 13.7 12.1 12.6
Other offences (incl. Work environment) at recidivism 6 5.6 5.6 5.6 5.3 5.4 6.2 5.4
No group of offences (without recidivism) 58.7 55 57.8 59.6 57.5 59.6 60.3 60.7
Other offences for profit Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 58.6 59.2 58.2 56.5 55.2 55.1 53.4 52.3
Economic offences at recidivism 1.2 1.2 1 1 1 1 0.8 0.6
Other offences for profit at recidivism 23.4 23.4 21.1 20.1 19.4 18.7 18.4 18.2
Offences of violence at recidivism 6.1 5.7 5.7 5.4 5.7 5.9 5.6 5.4
Sexual offences at recidivism 0.4 0.4 0.4 0.3 0.5 0.5 0.4 0.4
... ... ... ... ... ... ... ... ... ...
Environment offences Damage to property at recidivism .. 0.4 .. 0.1 .. 0.5 .. ..
Environment offences at recidivism 4.8 4.3 4.1 4 3.8 4.1 4.1 5.2
Traffic offences at recidivism 8.8 7.8 8.2 7.5 8.3 8 5.9 7.9
Other offences (incl. Work environment) at recidivism 3.9 4.5 3 2.9 4.2 4 3.8 4.1
No group of offences (without recidivism) 77.5 77.8 80.1 80.2 79.1 79.8 80.7 78.2
Traffic offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 38.1 37.7 36.5 37.1 35.2 34.5 34 34
Economic offences at recidivism 1 0.8 0.8 0.9 0.9 0.9 0.8 0.8
Other offences for profit at recidivism 4.6 4.3 3.9 3.6 3.5 3.1 3.2 3.3
Offences of violence at recidivism 3.3 3.1 2.9 3.1 2.8 2.8 2.8 2.8
Sexual offences at recidivism 0.3 0.3 0.2 0.3 0.3 0.3 0.4 0.3
Offences of narcotics at recidivism 4.3 4.2 3.9 4 4 3.8 4.1 4.6
Damage to property at recidivism 0.6 0.6 0.4 0.5 0.4 0.4 0.4 0.4
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 18.5 18.9 18.8 19.8 18.1 18 17.4 16.8
Other offences (incl. Work environment) at recidivism 5.3 5.3 5.3 4.8 4.9 4.9 4.6 4.8
No group of offences (without recidivism) 61.9 62.3 63.5 62.9 64.8 65.5 66 66
Other offences (incl. Work environment) Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 44.1 43.8 43 42.9 41.6 41 41.7 42.2
Economic offences at recidivism 1.4 0.9 0.9 0.8 1 0.8 0.8 0.7
Other offences for profit at recidivism 6.6 5.6 5.6 5.2 4.9 5.1 5.1 5.3
Offences of violence at recidivism 5.7 6.1 5.6 6 5.3 5.7 5.7 6
Sexual offences at recidivism 0.4 0.4 0.5 0.4 0.4 0.5 0.4 0.5
Offences of narcotics at recidivism 5.5 5.9 6 6.3 6.7 6.7 7.1 7.5
Damage to property at recidivism 1.2 1 1.1 1.1 1 1 1 0.9
Environment offences at recidivism 0.2 0.2 0.3 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 11.6 11.7 11.9 11.5 11.2 10.6 10.6 10.2
Other offences (incl. Work environment) at recidivism 11.5 11.8 11.2 11.4 10.8 10.5 10.7 10.8
No group of offences (without recidivism) 55.9 56.2 57 57.1 58.4 59 58.3 57.8
No group of offences (without recidivism) NaN NaN NaN NaN NaN NaN NaN NaN

121 rows × 8 columns

What groups see the highest and lowest recidivism

To get a sense of what groups have higher recidivism across the board, let's consider first the recidivism rates for all groups of offences.


In [53]:
offences_total = offences.iloc[offences.index.get_level_values('Principal Offence') == 'All groups of offences']
offences_total


Out[53]:
2002 2003 2004 2005 2006 2007 2008 2009
Principal Offence Recidivism Offence
All groups of offences Total 100 100 100 100 100 100 100 100
All groups of offences at recidivism 49.6 49.2 48.2 47.6 46.5 45.7 45.1 45.1
Economic offences at recidivism 1.3 1.1 1 1 1.1 1 0.9 0.9
Other offences for profit at recidivism 11.1 10.4 9.6 8.9 8.5 8 7.8 7.9
Offences of violence at recidivism 5.6 5.5 5.4 5.5 5.4 5.3 5.3 5.3
Sexual offences at recidivism 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5
Offences of narcotics at recidivism 9.7 9.6 9.5 9.6 9.8 10 10.1 10.6
Damage to property at recidivism 1 1 0.9 0.9 0.9 0.8 0.8 0.8
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 13.1 13.7 13.8 13.9 13.2 12.9 12.7 12.2
Other offences (incl. Work environment) at recidivism 7.2 7.2 7.3 7.1 7 6.9 6.8 6.8
No group of offences (without recidivism) 50.4 50.8 51.8 52.4 53.5 54.3 54.9 54.9

In [54]:
# dropping total

offences_total = offences_total.drop('Total', level='Recidivism Offence')
offences_total


Out[54]:
2002 2003 2004 2005 2006 2007 2008 2009
Principal Offence Recidivism Offence
All groups of offences All groups of offences at recidivism 49.6 49.2 48.2 47.6 46.5 45.7 45.1 45.1
Economic offences at recidivism 1.3 1.1 1 1 1.1 1 0.9 0.9
Other offences for profit at recidivism 11.1 10.4 9.6 8.9 8.5 8 7.8 7.9
Offences of violence at recidivism 5.6 5.5 5.4 5.5 5.4 5.3 5.3 5.3
Sexual offences at recidivism 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5
Offences of narcotics at recidivism 9.7 9.6 9.5 9.6 9.8 10 10.1 10.6
Damage to property at recidivism 1 1 0.9 0.9 0.9 0.8 0.8 0.8
Environment offences at recidivism 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Traffic offences at recidivism 13.1 13.7 13.8 13.9 13.2 12.9 12.7 12.2
Other offences (incl. Work environment) at recidivism 7.2 7.2 7.3 7.1 7 6.9 6.8 6.8
No group of offences (without recidivism) 50.4 50.8 51.8 52.4 53.5 54.3 54.9 54.9

In [85]:
# plotting the data
layout_offences = dict(width=850, height=600,                           # plot width/height
              yaxis={"title": "Recidivism for groups"},                 # yaxis label
              title="Recidivism Rates for Types of Offences",           # title
              xaxis={"title": "Years"}                                  # xaxis label
             )

offences_total.T.iplot(layout=layout_offences)



In [89]:
offences_total.T.iplot(legend=False)


What to make of this data

Interestingly, economic offences have the highest recidivism rates and has even increased over time. Second highest is traffic offences. This implies perhaps that rehabilitation for more "hardened" criminals is not as important as for-profit offenders.

Conclusion

       From the three datasets explored above, we can note the following key things:

  • The White Paper from 2008 had a direct effect on spending in Norwegian prisons. Spending increased significantly, especially operating expenditure and own production. With more resources available to boost rehabilitation in prisons, it is expected that results in the form of reduced recidivism would follow.
  • It is unclear whether the increased resources had an effect on recidivism, either across groups of offences, time, sexes or ages. That said, there are some interesting things to note about each of these categories that could help decide where the increased resources should be allocated to have a greater effect.
  • Although there do not seem to be any immediate results on recidivism rates in Norway after the 2008 white paper, the data does not show more than one or two years later than 2008. Therefore, it may be that in a few years time we will see some more dramatic results.