Data Analyse

If you want to do your own analyse of the data on db.sqlite3 and are going to use Python you can take advantage of some Django code. This Jupyter Notebook will help you to enable the Django code.

Setup and run

To setup your environment to run this Jupyter notebook you need to install some packages. Our suggestion is to run

$ python -m pip install -r requirements.txt
$ python -m pip install -r requirements-jupyter.txt

from your terminal.

To start Jupyter server, run

$ python manage.py shell_plus --notebook

Basic (Django Part)

You can use all power of Django on the notebook. For example, to gain access to the models you can use



In [ ]:

    
import lowfat.models as models

To select all the fellows you can use



In [ ]:

    
fellows = models.Claimant.objects.filter(fellow=True)
fellows

Remember that the Claimant table can have entries that aren't fellows and because of it we need to use .filter(selected=True).

Basic (Pandas Part)

You can use Pandas with Django.



In [ ]:

    
from django_pandas.io import read_frame

fellows = read_frame(fellows.values())
fellows

When converting a Django QuerySet into a Pandas DataFrame you will need to as the previous example because so far Pandas can't process Django QuerySets by default.



In [ ]:

    
expenses = read_frame(Expense.objects.all())
expenses



In [ ]:

    
expenses.sum()



In [ ]:

    
expenses["amount_authorized_for_payment"].sum()

Pandas table as CSV and as Data URIs

For the report, we need to Pandas table as CSV encoded inside data URIs so users can download the CSV file without querying the server.



In [ ]:

    
from base64 import b64encode

csv = fellows.to_csv(
    header=True,
    index=False
)

b64encode(csv.encode())

The output of b64encode can be included in

<a download="fellows.csv" href="data:application/octet-stream;charset=utf-16le;base64,{{ b64encode_output | safe }}">Download the data as CSV.</a>

so that user can download the data.

Basic (Tagulous)

We use Tagulous as a tag library.



In [ ]:

    
funds = models.Fund.objects.all()
read_frame(funds)

Get a list of all tags:



In [ ]:

    
funds[0].activity.all()

You can loop over each tag:



In [ ]:

    
for tag in funds[0].activity.all():
    print(tag.name)

Filter for a specific tag:



In [ ]:

    
models.Fund.objects.filter(activity="ssi2/fellowship")

You can query for part of the name of the tag:



In [ ]:

    
models.Fund.objects.filter(activity__name__contains="fellowship")



In [ ]:

    
for fund in models.Fund.objects.filter(activity__name__contains="fellowship"):
    print("{} - {}".format(fund, fund.activity.all()))