Grading Logistics

Figuring out how to efficiently grade students' assignments is a non-trivial task. Grading can be made more efficient by automatic output checking but that doesn't leave room for quality assessment and feedback. To deal with the logistics of grading a set of scripts and grading practices has been developed. They are outlined here.

Scripts

• cloneRepos.sh - uses a text file with student info to clone repositories from github.
• updateRepos.sh - uses the text file with student info to pull updates of all repositories from github.
• makeGradeNotebook.sh - uses the text file with student info to create a blank grading notebook for a given assignment. Each student has two notebook cells created for them - one for recording scores and comments that are saved to a text file, the other for reviewing the git log history to determine the date/time the assignment was turned in. The text files are named according to LastName_AssignmentName.txt and saved in the student's repository directory.
• makeProjectGradebook.sh - similar to makeGradeNotebook.sh but specialized for the project grading with rubric categories and descriptions.
• emailGrades.sh - uses the text file with student info to generate personalized emails to each student for a given assignment with the grade text file created with the Grade Notebook appended to the email. This is the method of communicating scores and comments back to the students. The script must be executed on a machine with an email server.
• printGrades.sh - extracts the total score for each student from the grade files for review and/or transfer to the gradebook.

The text file with student info has columns to represent Section #, Last Name, First Name, email alias, Github user account, repository name, major, class standing. For example:

Homework Submission

Students are required to "turn in" assignments by committing their changes to their local git repository and then pushing them to Github by the assignment deadline.

Basic Rubric

The rubrics for grading can be found in the syllabus.

Grader Instructions

These are some general instructions on the workflow for grading.

1. Use the script to clone/update student repositories.
2. Create Grade Notebook using script.
3. Open student's assignment notebook and evaluate based on criteria set for that assignment.
4. Include comments and score for each element in the file magic code cell of the grade notebook, formatted according to assignment grading instructions.
5. Check the git log for the assignment in the git code cell to verify that they turned it in by the deadline, typically 11:59pm on the due date. If they submitted it within 4 hours after the due date, accept it without penalty. If it is later than that, indicate in the text file with a -1 point penalty per day for the assignment. (e.g., if an assignment is worth 5 points or 20 points because it has 4 questions, just deduct 1 point per day late overall)
6. Look for suspicious solutions that might have been copied verbatim from an external source. In general, solutions to simple problems are likely to be similar. Keep an eye out for any that have telling details, such as exact same wording, structure, comments, variable names, etc. that seem too similar to be a coincidence. Keep track of these and flag as potential violations of the academic honesty guidelines. Follow up with student to correct behavior and/or take approriate action. See point 7 for reasonable exceptions.
7. Some assignments are done as "pair-programming" exercises where one person "drives" while the other "navigates" and they solve the problem together. In these instances, it is mandatory that they indicate the pairing in their solution with a markdown cell indicating who is the driver and who is the navigator. In that case, code similarities are to be expected and these need not be flagged for review.
8. Use the printGrades script to record the grades in the grading spreadsheet.

Project Euler Grading Guidelines

For Project Euler assignments, the grading is fairly straightforward. For full credit (5/5), they must have all of the following:

1. Header Cell with Problem Title
2. Markdown Cell with Problem Description
3. Solution of the test case in the Problem Description (if provided)
4. Solution of the problem

Some example grades for specific cases, which can be indicated in the comments to the student:

• Correct solution but missing 1 and/or 2 : 4/5
• Correct solution but missing 3 : 3/5
• Correct solution but missing 1 and/or 2 and 3 : 2/5
• Incorrect solution but solid attempt (regardless of 1 or 2): 2/5
• Incorrect solution because very little attempt (regardless of 1 or 2): 1/5
• Did 1 and/or 2 but otherwise no solution : 1/5
• No solution submitted : 0/5

Include any comments about why their code might not be working. E.g. if they left it because they got some kind of error and couldn't figure it out, offer a suggestion for what they did wrong. If their code runs but returns the wrong solution, try to figure out why and indicate what the problem is. If it is too much work to figure out what they did wrong, just indicate that in the comments.

NumPy Grading Guidelines

Guidelines provided to a grader for the NumPy exercises - a basic grading rubric.

There are four questions, but some of them have subparts. To reinforce Learn by Doing, they get full credit for doing the parts that don't require additional input beyond correctly typing in the code and executing it.

Question 1.

Give 5 points for array 1 and 5 points for array 2. Maximum credit if they were able to achieve the result in just a few lines, similar to my examples.

• 3/5 for brute force method but otherwise correct
• 4/5 for getting the type of the array elements incorrect
• 2/5 for multiple problems, such as brute force and a type error
• 1/5 for a weak attempt
• 0/5 for no attempt

Question 2.

There are a couple of ways to accomplish this one. I gave two examples in my solutions. Full credit if their solution involves using the NumPy tile method.

• 2/5 for brute force, not using np.tile

Question 3.

• (a) 2/2 for executing the code I gave them
• (b) 2/2 for executing the code I gave them
• (c) 2/2 for executing the code I gave them

• (d) 3/3 for correct implementation of np.nonzero

• (e) This one is where they had to do some work:

  • 5/5 for using NumPy arrays to create the sieve and print the primes with np.nonzero
  • 3/5 for using Python lists instead of NumPy arrays
  • Deduct a point if they didn't do both cases (10 and 100)
  • For incorrect programs, use your discretion to decide whether their work merits an "A-" (4/5), a "B" (3/5), a "C" (2/5) or a "D" (1/5).
  • Reserve 0/5 for those cases where there was no attempt.

Question 4.

• (a) 2/2 for executing the code I gave them
• (b) 2/2 for executing the code I gave them
• (c) 2/2 for executing the code I gave them

• (d) 2/2 for executing the code I gave them

• (e) This one is where they had to do some work:

  • 5/5 Correct result

  • 4/5 A common mistake may be using integers in the mask expression that produces very blocky output instead of a smooth curve.

  • Same approximate rubric as question 3(e) for other types of incorrect answers.

Exercise Solutions

These are "available by request" to authorized educators with a valid .edu email address. Contact the repository owner to request them.

Project Philosophy and Assessment

I purposely choose projects that I have not personally coded myself that form a basis for answering real research questions. There are several reasons I do this:

• I find it much more interesting to learn something new through the students from a course such as this. I would be bored otherwise.
• Having the students work on a novel project is similar to how I work with students in research mentoring. My interactions with them are much more like a real research environment. I can offer guidance and suggestions but my not "knowing" the answer to every problem or roadblock they encounter means I won't have to resist the temptation to give them the quick fixes.
• These projects could easily be extended into senior projects or research internship opportunities, giving the students the motivation to keep working on the problem after the course is over.

For these reasons, the project assessment is built less on "correctness" than on their formulation of the solution, documentation of the results, and their attempt to answer the assigned "research question". The rubric is set up so that they can get most of the points for developing an organized, complete project with documentation, even if their results are incorrect.

See the syllabus for project grading guidelines.

All content is under a modified MIT License, and can be freely used and adapted. See the full license text here.