DLKit Overview

DLKit is a Python implementation of the OSIDs (www.osid.org). DLKit specific documentation is available at http://dlkit-doc.readthedocs.io

DLKit exposes Python API bindings of the OSID service contracts, which are organized into a number of service packages. Service packages describe a complete set of functionality for a particular service domain, like assessment, or logging. These service packages define APIs that describe all the entities, or objects related to a particular service domain, as well as the various actions that a programmer can take on these objects. Each service defines a catalog object, which houses / contains the additional objects. In addition to aiding organization, these catalogs also help control authorization to the contained objects, and can be arranged hierarchically (with authorizations flowing down).

         MIT
        /   \
       /     \
   Physics   Math  

Someone with access to all of MIT could see both Math and Physics materials, but folks in Physics may not be able to see Math materials.

Let's start with a simple example and dig more in depth.

Note that all data generated with this tutorial is saved in JSON files to your harddrive, in a folder called tutorial-data, sibling to this .ipynb file. If you ever want to start over, just delete that folder and all its contents.

Assessment Service

In the Assessment Service, the catalog is called a Bank. Banks contain other objects, like Items, Assessments, AssessmentsOffered, AssessmentsTaken, etc. We'll start with Item and Assessment.

To being with, you access the functionality of each service package through a Manager. For example, an AssessmentManager gives you access to the various methods available for accessing Banks and Items, and the other objects defined for assessment.

In order to get a Manager, we go through the runtime -- in this tutorial, the dlkit_runtime runtime. We will simulate a user and a test web request, to pass along to the service manager when its instantiated. This username is automatically available to many kinds of underlying actions and objects, such as when taking assessments.

Banks

Now that we have an AssessmentManager, we can see what Banks exist in the system. Calling for "lists" of things returns a "thing list", like a BankList (a Python generator), and we can check the number of results with .available().

No banks!! Okay, let's create one. In DLKit, CrUD operations are done with forms. So let's get a form to create a new assessment bank, and assign it a displayName and description that we'll recognize later.

Notice that in the get_bank_form_for_create() method, we pass an empty list as an argument. This can be used to extend the base functionality of the bank, via record extensions. This is a more advanced feature we'll touch on in a bit.

You can see that our new Bank has some attributes -- some that we assigned (display_name and description, and others that were created by DLKit, like ident). display_name and description return DisplayText objects that include the text strings we created, but can also contain language, format, and script data, which is why we call display_name.text and description.text above.

Items

Now that we have a Bank, we can create assessment Items in it.

An Item is what you might think of as a basic assessment question with the associated answers (right or wrong). There are many types of Items, including multiple choice, fill in the blank, short answer, etc.

Again, we can inspect to see if any exist.

To create a new item, we'll grab a form. In the OSIDS, since Items are not defined beyond a question and answer, we'll need to pass along a list of record extensions, so give the item some functionality. This requires some internal knowledge about DLKit, so for now well just use a simple multiple choice record plus accomodation for wrong-answers.

So we have now created an item, but where's the actual question that we want a student to respond to?? We'll create that separately, with its own records. Recall:

    Item
      |--Question
      |--Answers

Since Questions are distinct from Items, students can be sent only the Question object, without any danger of them seeing any answers . Note that the Question form requires an extra itemId argument. This attaches the question to the Item created previously.

The Item has a method that allows us to get access to the Question.

What?!?! Oh, wait ... because the Item object we have was initialized before the Question was actually created, we need to re-grab that Item to get it in the "newest" state. Note that this is a due to the particular service implementation we are using. Some implementations might actually keep objects up-to-date with the underlying persistence (like database or filespace) and never let an object get stale. you can check if an object is known to be up-to-date with underlying data by calling it's is_current() method. If it is not, you can refresh the object, in this case our Item by calling the get_item(item_id) method of the Bank.

We can also set up wrong / right Answers, to be used in evaluating the "correctness" of the response. We indicate the type of Answer with the genusTypeId property.

Now we can find the Item Answers (if you run the above block multiple times, you'll get a varying number of Answer elements.

Wait ... where are all the wrong Answers? There should be 2x the number of times you ran the block, Answers ... and by looking at the genusTypeId attributes, it seems like all the Answers from get_answers() are only "right-answer" types.

This is because DLKit defines the get_answers() method to only return the "right" answers, so the "wrong" answers need to be retrieved with a different method, defined by the Item record extension for wrong answers.

There they are!

That's a basic overview of Items. They are very powerful, and to learn more, you can inspect the records directory that came along with this tutorial.

Assessments

Conceptually, Items are organized into Assessments -- which are then offered to students. An Assessment in a classroom might be a homework, a quiz, or an exam ... a tool that evaluates students' knowledge of a topic or outcome.

Let's see what Assessments exist in our Bank.

If we're starting with a clean slate, we have no Assessments. Let's get a form, and create one. Note that with the current DLKit configuration, we need to include at least the simple-child-sequencing record for each Assessment.

Now that we have our Assessment, we can add Items to it.

This is great for authoring Assessments, but DLKit defines another set of methods to offer the Assessment in "student mode". This is through the AssessmentSession methods which define actions for taking assessments -- but first, we need to create an AssessmentOffered. AssessmentsOffered wrap some other information around the canonical Assessment, like the start_time, deadline, duration, etc. -- all of these typically are optional. Leaving them out "opens" the AssessmentOffered to students immediately, and it never "closes".

Again, we need a form for this, and that method requires the Assessment ID.

Once we have an AssessmentOffered, we can provide a set of methods for students to take the assessment. Each "taker" generates an AssessmentTaken, which links a taker's user ID (which was provided through the Proxy when the AssessmentManager was initially set up) to the AssessmentOffered ID. The AssessmentTaken also links to one or more AssessmentSections, which maintains a list of questions takers have seen and thier responses.

There are several ways a taker can get the Assessment questions ... they are the same for our simple Assessment because there is only one question, but you can easily imagine how the behavior might change when multiple questions are present. For sequential Assessments, each question must be answered in order. Hence, you iterate through the questions using methods like get_first_question() & get_next_question() or get_first_unanswered_question() & get_next_unanswered_question(). Alternatively, if the Assessment does not specify that the questions must be answered sequentially, you can get all the questions in bulk, via get_questions(). In more complex and adaptive scenarios, even this method might be dynamically updated, as students navigate through the Assessment.

Assessments can be divided into multiple AssessmentSections, for example to provide UI separation. In our simple Assessment, there is only one AssessmentSection, and we'll use that ID to grab the questions.

Let's demonstrate the methods.

Now, it's interesting to note that this question ID is different than the original Item ID.

This is because questions, when put into an AssessmentSection, generate new, unique IDs. This helps us manage more advanced Item and Question records that manipulate the ID attribute to provide adaptability or consistent randomization -- which is beyond the scope of this basic tutorial.

However, we will use this new question ID to submit a Response. Again, we'll use a form, and we'll supply it with a choice ID from earlier.

Note that this last method, is_correct(), is a non-OSID convenience method, that currently only works with multiple choice questions, but can easily be integrated into any record extension.

We can try submitting again, if the AssessmentOffered allows us to.

Note that both submissions are stored in the database, but currently only the most recent one can be retrieved. Time data in DLKit is stored as UTC, so the information printed above may differ from your local time.

At any time, instructors can get a record of all student Responses and questions.

At the moment, the results only include the Responses without regard to AssessmentSections, but you can easily link in the actual questions as well. Sample code is provided below.

Note that we add back in the itemId attribute so that we can map the questions that students see (now with all unique IDs) back to the original Items.

But, um...why is the Responses list empty? Because OsidLists are exhaustive -- they are Python generators. So they can only be iterated through once. We can solve that either by converting to a list, or calling the get_assessment_taken_responses() method again.

There you have the basics of the Assessment service! Much of the additional complexity appears in the record extensions for various Item types and Assessment / AssessmentOffered / AssessmentTaken settings.

For example, we have randomized multiple choice questions where the choices appear in different orders to each student, but when students return to the AssessmentTaken, they see the exact same order they've seen before -- randomized per student, not per view of the question.

Another example of a complex assessment is adaptive behavior, where the "next question" a student sees depends on their response to the previous question. So each students gets a different set of questions, depending on their knowledge and performance.

Object Patterns in DLKit

Now that we've gone through the Assessment service, you've hopefully learned the basics about catalogs, objects, forms, sessions, and managers. Luckily, these patterns appear across all the DLKit services. So you can easily pick up how to use the other services. http://osid.org/ is a good reference, and we've included a simple table below that reflects the services available in this DLKit build:

-------------------------------------------------------------------------------------------------------------
|   Service     |      Catalog     |                            Objects                                     |
-------------------------------------------------------------------------------------------------------------
| Assessment    | Bank             | Item, Assessment, AssessmentOffered, AssessmentTaken, AssessmentPart   |
| Authorization | Vault            | Authorization                                                          |
| Commenting    | Book             | Comment                                                                |
| Grading       | Gradebook        | GradeSystem, GradeEntry, GradebookColumn                               |
| Logging       | Log              | LogEntry                                                               |
| Repository    | Repository       | Asset, AssetContent, Composition                                       |
| Resource      | Bin              | Resource                                                               |
-------------------------------------------------------------------------------------------------------------

Repository Service Example

Assets

Assets are digital "things", like a video, an HTML page, an image, etc. It includes things like copyright, license, etc., but the actual digital bits are stored in Asset Contents.

For example, an asset might represent a video, but it might include assetContents for a transcript, an .mp4 version, a thumbnail, etc.

    Asset
      |--AssetContents

To make this work with and save files to your filesystem (instead of GridFS), you need to include an assetContent record type.

Compositions

Compositions are organizational "buckets" that allow you to organize assets into things like "chapters" or "sections". They can have children compositions or include assets.

    Composition
          |--Composition
          |     |--Asset
          |--Asset

For example, if you represent this like a book, it might be:

    Chapter
          |--Section
          |      |--Example problem
          |--Video

Resource Service

In the Resource Service, the catalogs are called bins and the objects are called resources. While this sounds very generic, one specific use case is for representing users. Resources have a displayName and description, and also an avatar -- for a profile picture or something. We'll re-use the green-dot asset from above, so make sure you run those cells in the Repository Service first.