Below we walk through an example of how to setup and use pygrade to automatically grade student assignments using GitHub and Python unittests.
1. Create assignment skeletons.
First, we will create a GitHub repository containing starter code for each assignment.
In this tutorial, we will use: https://github.com/tapilab/pygrade-example-assignment
2. Create a unittest.
test_asg0.py contains the unit tests for this assignment, as well as indications of the points to be deducted for each failing test case.
3. Create students.tsv
Next, we create a tab-separated file containing student information. The only required fields are:
github_repo: The full URL of the student's GitHub repository, e.g., https://github.com/tapilab/pygrade-example-studentgithub_id: The GitHub user name of this student, e.g., aronwc4. Initialize repositories
Using students.tsv, we will create private repositories for each student and initialize them with the starter code from https://github.com/tapilab/pygrade-example-assignment . (Note, I've made this repo public for demonstration purposes).
$ pygrade init -h
Initialize student repositories. Create one repo per student. Also create one team per student consisting of that student. Each repo is made private to that team.
usage:
pygrade init --org <name> --user <username> --pass <passwd> --remote <uri> [--students <file>] [--workdir <file>]
Options
-h, --help
-o, --org <string> Name of the GitHub Organization for the course.
-p, --pass <file> GitHub password
-r, --remote <uri> URL of remote github repo used for starter code.
-s, --students <file> Students JSON file [default: students.tsv]
-u, --user <file> GitHub username
-w, --workdir <file> Temporary directory for storing assignments [default: students]
$ pygrade init --org tapilab --user aronwc --pass XXX --remote https://github.com/tapilab/pygrade-example-assignment --students students.tsv --workdir students
found org https://api.github.com/orgs/tapilab
initializing repo https://github.com/tapilab/pygrade-example-student for aronwc
found existing team pygrade-example-student
found existing remote repo pygrade-example-student
cloning https://github.com/tapilab/pygrade-example-student to students/pygrade-example-student ...
pushed Info.md
pushed template from remote https://github.com/tapilab/pygrade-example-assignment
A new student repo has been created and cloned to the local working directory (students by default).
$ ls students/*/*
students/pygrade-example-student/Info.md students/pygrade-example-student/README.md
students/pygrade-example-student/asg0:
asg0.py
5. Grade assignments
After the students have pushed their solutions to their private repositories, we can then grade the assignments using pygrade grade:
$ pygrade grade -h
Grade a Python assignment.
usage:
pygrade grade --test <file> [--students <file>] [--output <file>] [--workdir <file>]
Options
-h, --help
-o, --output <file> Output file [default: grades.json]
-s, --students <file> Students JSON file [default: students.tsv]
-t, --test <file> File containing python tests for grading
-w, --workdir <file> Temporary directory for storing assignments [default: students]
$ pygrade grade -t test_asg0.py -s students.tsv
working directory=students
read 1 students
saved results in grades.json
$ cat grades.json | json
{
"student": {
"github_repo": "https://github.com/tapilab/pygrade-example-student",
"github_id": "aronwc",
"Name": "Test Student",
"Student ID": "1234567"
},
"possible_points": 20,
"grade": 8,
"assignment": [
"asg0/asg0.py"
],
"deductions": [
{
"summary": "test_add: @points=10 ",
"points": 10,
"trace": "Traceback (most recent call last):\n File \"test_asg0.py\", line 39, in test_add\n self.assertEqual(add(2, 2), 4)\nAssertionError: 0 != 4\n\nsource:\n def test_add(self):\n \"\"\" @points=10 \"\"\"\n self.assertEqual(add(2, 2), 4)\n"
},
{
"summary": "test_hard: @points=2 ",
"points": 2,
"trace": "Traceback (most recent call last):\n File \"test_asg0.py\", line 35, in test_hard\n self.assertTrue(is_mammal('dolphin'))\nAssertionError: False is not true\n\nsource:\n def test_hard(self):\n \"\"\" @points=2 \"\"\"\n self.assertTrue(is_mammal('dolphin'))\n"
}
],
"time_graded": "Mon Jan 18 10:28:27 2016"
}
grades.json has one entry per student. In addition to storing the student info, the most important keys are "grade" (the final grade), and "deductions" (a list of test failures and the points deducted for each).
6. Push the grades to each students private repo.
Finally, we will create a grade.txt file for each student and push it to their private repo. This file will list the deductions for the assignment.
$ pygrade push -h
Push grades to student repositories.
usage:
pygrade push [--grades <file>] [--workdir <file>]
Options
-h, --help
-g, --grades <file> JSON grades output by the grade command [default: grades.json]
-w, --workdir <file> Temporary directory for storing assignments [default: students]
$ pygrade push -h
Push grades to student repositories.
usage:
pygrade push [--grades <file>] [--workdir <file>]
Options
-h, --help
-g, --grades <file> JSON grades output by the grade command [default: grades.json]
-w, --workdir <file> Temporary directory for storing assignments [default: students]
bronze:tmp awculott$ pygrade push
pushing 1 grades
pushing grade students/pygrade-example-student/asg0/grade.txt to https://github.com/tapilab/pygrade-example-student
This file is now here: https://github.com/tapilab/pygrade-example-student/blob/master/asg0/grade.txt:
Grade: 8.00/20.00
2 Deduction(s):
--------------
#1: 10.00 points
Failing test: test_add: @points=10
Traceback (most recent call last):
File "test_asg0.py", line 39, in test_add
self.assertEqual(add(2, 2), 4)
AssertionError: 0 != 4
source:
def test_add(self):
""" @points=10 """
self.assertEqual(add(2, 2), 4)
--------------
--------------
#2: 2.00 points
Failing test: test_hard: @points=2
Traceback (most recent call last):
File "test_asg0.py", line 35, in test_hard
self.assertTrue(is_mammal('dolphin'))
AssertionError: False is not true
source:
def test_hard(self):
""" @points=2 """
self.assertTrue(is_mammal('dolphin'))
--------------That's it!