In [ ]:
from __future__ import print_function, division, absolute_import
The notebook contains problems for documenting code.
These probablems assume you've done the "code repo" problem set, or otherwise are familiar with code repositories and packaging enough to have a functioning test repository with a Python package.
E Tollerud, B Sipocz
One of Python's most powerful documentation tools are docstrings. These are basically just little strings you put at the top of a class, function, or similar, which then gets bound as sort of a glorified comment. But with internal consistency and tenacity, these can do most of the work you need to document your code.
Note that all of this problem can be done in the notebook, and it is shown that way to make the notebook internally consistent. But you might find it more useful to use a function from your code repository, as that makes it clearer why docstrings are useful (i.e., the code is not immediately visible, as it is in the notebook).
Make a function (or just use one from your pre-existing repo) and give it a docstring. The docstring can be anything in principal, but the example case below shows one of the most standard conventions used for scientific Python code. (The format originated in numpy, although with some "flavors" like that in astropy.)
Hint: you might want to keep the code part of your function secret from your neighbor, as it will reduce the work for 1c if you do so
In [ ]:
def do_something(arg1, arg2):
"""
A short sentence describing what this function does.
More description
Parameters
----------
arg1 : type1
Description of the parameter ``arg1``
arg2 : type2
Description of the parameter ``arg2``
Returns
-------
type of return value (e.g. int, float, string, etc.)
A description of the thing the function returns (if anything)
""" # complete
# complete
It turns out that docstrings are more than just strings sitting un-used inside a function: they are associated with and carried around as a part of the function. This means you can access them in useful ways even if you don't have the code right in front of you. Explore some of the ways available to look at your function's docstring. Try the various ways below and consider which seems most useful in various contexts.
In [ ]:
do_something.__doc__
In [ ]:
help(do_something)
In [ ]:
do_something?
In [ ]:
# this one does more than just show the docstring, but is useful to know nonetheless
do_something??
Using either the function you just wrote or a new one (if you've already shown the function to your neighbor), try to communicate the essentials of your function enough for your neighbor to use the function. That is, have your neighbor run your function (and do something with the output) using only the information in the docstring.
No peeking at the code! One way to ensure this is to put the function in your github repo and have your neighbor pull down the updates and import the code directly. Or you can just type your function higher in the notebook and scroll down without your neighbor looking.
Rinse and repeat for you looking at your neighbor's code.
In [ ]:
from your_neighbors_package import your_neighbors_code # complete
your_neighbors_code? # complete
In [ ]:
... = your_neighbors_code(...)
... # complete
Try making a docstring for a class. This is quite similar to a function, but with some subtle difference (as detailed in the template below).
In [ ]:
class MyClass: # if you're using Py2, you'll want to do "MyClass(object)"
"""
A short description of the class.
Possibly some extended description, notes on how to sub-class, etc.
Parameters
----------
arg1 : type
Describe the first argument of the initializer
arg2 : type
Describe the second argument of the initializer
"""
def __init__(self, arg1 arg2):
# note that the initializer gets *no* docstring, because it's in the class docs
#complete
def some_method(self, method_arg):
"""
A short description of the method.
Possibly extended description.
Parameters
----------
method_arg : type
A description of the method's first (non-self) argument.
Returns
-------
return type
Description of the return value (if any)
"""
#complete
Add a docstring to the module and packages in your repository. This is usually just free-form text (not as much structure as a function or class), although you might include some structure like section headings or bullet-pointed lists. Once you've done that (and reloaded or restarted the kernel), the commands below should pop up your documentation.
Hint: remember that a package's __init__.py file acts sort of like the "package.py" file for the package
In [ ]:
import <mypackage> #complete
<mypackage>? #complete
In [ ]:
from <mypackage> import <mymodule> #complete
<mymodule>? #complete
Intro
You may have sphinx installed already, but if not, you'll want to install it. The invocation below is appropriate for the Anaconda Python Distribution (but change it from conda to pip to install from pip).
In [ ]:
!conda install sphinx
It's good practice to keep the narrative documentation (i.e., the non-docstring part) and the code in separate places. To do that you'll need to create a new directory for the docs.
In [ ]:
%cd <yourpackage> #complete
In [ ]:
!mkdir docs
%cd docs
Sphinx has a standard layout of files that it uses, and even provides a tool for doing this called sphinx-quickstart. Use that command to create a sphinx repo.
The invocation of sphinx-quickstart below just gives you the defaults for everything without prompting. If you want to see all the options, you can run this tool in a terminal inside your docs directory - by default it prompts you for lots of information, though, and we can't respond to those in the notebook. If you do that, be sure to answer "yes" to the question about "autodoc" (more on that in 2g).
In [ ]:
!ls
# should be empty...
In [ ]:
!sphinx-quickstart -a "<yourname>" -p <yourpackagename> -v <version> --ext-autodoc -q #complete
In [ ]:
!ls
You should see various files have appeared in your docs directory, most critically a conf.py and index.rst
The index.rst file is the root for all of your documentation. You'll see it's already been pre-populated with some boilerplate structure. None of this is specific to your package, however. Open this file in an editor, and add some documentation for your package. You'll likely want to put it after the "Welcome to
It's up to you to decide what should go in this front page for your package, although a simple idea is given below.
Another important thing to keep in mind is the format for these packes. The markup language is Restructured Text (reST), which is roughly driven by the philosophy of "readable as plain text, but with extra bits to make it prettier in doc pages". Sphinx provides a great reST primer, which you can reference to build your docs.
This package does some neat stuff! It's features include:
* A cool thing
* Another thing
* Something else that's not quite so useful but I like.
Citing this code
----------------
There's no way to cite this code right now. But it would be great if you acknowledge <your name> if you use it. Someday I hope to put it up on `Zenodo <https://zenodo.org/>`_, though...Now try actually building the docs with sphinx. The command below should be all that's required. Once it's finished, have a look at the page it generates (start from the index.html).
In [ ]:
!make html
If you haven't been doing so, now's a good time to add all the new content of the docs directory to git and push it up to github.
Important gotcha to watch out for: if you just git add docs, you'll probably get both your docs and the stuff in the _build directory. In general you never want to include generated files in your github repository, because it confuses users (and is nearly-impossible to keep up-to-date, anyway). To keep yourself from getting confused, you can create a .gitignore file that is aware of all the generated-file directories. That prevents them from getting added by git accidentally. An example is shown below that should be appropriate for your repo if you've followed the other notebooks.
In [ ]:
%cd .. #or whatever you need to do to get back to the base of your repository
In [ ]:
%%file .gitignore
docs/_build/*
build
dist
In [ ]:
!git add .gitignore docs
!git commit -m #complete
Read the Docs is an online service that automatically builds documentation for public projects. In this problem, we will set up the repo that you just got sphinx working in to generate its documentation on RTD.
You'll need an RTD account to be able to do anything with it, of course. Go to the Read the Docs front page, and you should see a "sign up" button on the upper right. Use that to create an account.
RTD can automatically read your github repos if you authorize it to connect to Github, and is often smart enough to do the right thing in one click. You may need to go to your account settings->connected services page and "Connect to Github" to get this to work. Once you've done that, go to your RTD dashboard (click on your username in the upper-right) and "Import a Project". If the github sync worked, you should see your project.
Alternatively, you can choose the "Import Manually" option, where you have to manually provide the name and github repo URL.
RTD takes a bit of time to build. You can watch the progress by going to the project you just created in your dashboard and hitting the "builds" button. But all you really can do about it is wait until it finishes.
Hint: Maybe you'd like some refreshing tea in the meantime? I believe there's some in the back.
Once the build finishes, have a look at your doc page and see if it looks like you think it should. RTD will sometimes succeed in building even though something went wrong on a page, so it's usually worth a look at any significant changes from the last build.
RTD is at its most powerful when you have it run automatically. To do this, you need to set up a "web hook" that tells RTD when a new commit is sent up to github. You may already be seeing a message in your dashboard warning you that "This repository doesn't have a valid webhook set up". If so, try clicking the link there that is supposed to set the hook up automatically. Sometimes this doesn't work, though. In that case you can follow RTD's instructions for doing it manually.
Now try making a change to your documentation and pushing it up to github. You should see RTD spring into action, and after a bit, your is automatically appears on your RTD docs site!
This problem extends your documentation to use some of the features that Sphinx offers that makes it powerful for documenting code.
Note that many of these are in the form of extensions - pieces of Sphinx that are not part of the core functionality. Some of these are built-in (and used below) and others that are downloaded separately. Here we won't be using any third-party extensions, but you can check some out if you're interested.
One of Sphinx's most important features is that it understands how to link things across documents. So lets try creating another page in your docs. The example below will do the trick. After you've made that file (or something similar), you'll need to add the text second_doc into the ..toctree:: section of the index.rst. That will add the new document to your table of contents. Once you've done all this, build the doc again and have a look at your handiwork.
Hint: like Python, sphinx uses indentation for contextual meaning. So be sure you have consistent indentation in an .rst file, just like in a .py file
In [ ]:
%%file second_doc.rst
A Document title goes here
--------------------------
More information. Here's a link back to the index page: :doc:`index`.
Another important feature of Sphinx is the ability to include docstrings in the documentation. The example below shows some of that functionality. Add a file like that, add it to your table of contents (just like for second_doc), and then re-build and examine your docs.
Hint: you need to have your package accessible from python for this to work. So if you haven't done a python setup.py install on your package yet, you'll need to do that. If this bothers you, the challenge problem shows how to set up machinery that does not require installing the package to generate the docs.
In [ ]:
%%file api_docs.rst
API Documentation
=================
This package has two modules, detailed below.
Also, after doing this, you can mention some of the functions from *anywhere* in the docs by doing :func:`<yourpackage>.<module>.<function_you_want_documented>`.
<yourpackagename>
-----------------
.. automodule:: <yourpackage>
<your modulename>
-----------------
.. automodule:: <yourpackage>.<module>
.. autofunction:: <yourpackage>.<module>.<function_you_want_documented>
.. autofunction:: <yourpackage>.<module>.<function_you_want_documented2>
You may not need multiple automodule calls if uo on how you structured things. Also you might be able to avoid the autofunction directives depending on how you laid things out, by adding :members: undert he automodule directive (properly indented. The above explicit approach is what Sphinx recommends, but see the challenge problem for a tool that makes this all quite a bit simpler.
As an example of some of the extensions that Sphinx provides, there's a neat tool to generate class "inheritance diagrams" - basically these are diagrams that show which classes are subclass of what other classes. For a complex example of a diagram that justifies this feature, see here.
For this problem you should add the set of (do-nothing) classes shown below. Then, you'll need to add the string 'sphinx.ext.inheritance_diagram' to the extensions list in your conf.py. Then somewhere in your .rst file, add .. inheritance-diagram:: yourmodule.class_heirarchy. Then once you build it with sphinx, you should see the diagram at that point in the docs.
In [ ]:
%%file <yourpackage>/class_heirarchy.py #complete
class A(): # this needs to be "A(object)" in py 2.x
pass
class B(A):
pass
class C(A):
pass
class D(B,C):
pass
The Astropy package template (discussed in a previous challenge problem) contains all the machinery necessary to build documentation just like Astropy. One particularly useful tool is the automodapi machinery, which lets you do:
.. automodapi:: mypackagein the sphinx, and if you have docstrings (in the format expected), it will automatically generated pages like this one.
For this problem try to adapt the affiliated package template to your package, and use it to generate an API section. Note that you can also try using automodapi on its own (it can be used independently by installing the stand-alone version), but it's probably easier to just use it from the template.