Open Software Engineering

Overview

The foundation of IPython's software engineering practices is openness. Openness is built into the project at a deep level. This attribute builds a dedicated and invested community of users and developers that have good feelings about the project. Openness also builds trust, accountabilty and integrity. It blurs the line betweeen users and developers by providing a single community that spans all project activities.

Open source

The IPython source code is released under the terms of the BSD license. This license allows any individual or legal entity to use the software, without restrictions, for any purpose, commercial or otherwise. Companies are building commercial products based on IPython:

• Wakari by Continuum Analytics
• Anaconda by Continuum Analytics
• Canopy by Enthought
• PiCloud

A wide range of open-source projects and scientific collaborations also build on top of IPython.

IPython also uses a large number of open source projects itself:

• Python - Programming language
• ZeroMQ/PyZMQ - High performance networking and messaging
• Tornado - Async web server
• jQuery - JavaScript DOM manipulation and events
• Bootstrap - Web UI
• Less - Dynamic CSS stylesheets
• Backbone.js - MVC for JavaScript
• MathJax - LaTeX in the browser
• CodeMirror - Code editor in the browser
• require.js - JavaScript module and dependency loading
• reveal.js - HTML5 slideshows
• LaTeX - Mathematical document typesetting
• Pandoc - Document conversions
• Jinja - Templating library
• And others...

We work closely with the developers of other open source projects that are building tools for scientific computing and data science:

• NumPy - Multidimensional numerical arrays
• SciPy - Numerical algorithms (ODE, linear algebra,...)
• SymPy - Symbolic mathematics
• Matplotlib - Plotting
• Pandas - Columnar data structures and data analysis
• scikit-learn - Machine learning
• scikit-image - Image processing
• Statsmodels - Statistical modeling
• Cython - Optimized code generator
• PyMC - Bayesian modeling, Markov chain Monte Carlo
• And others...

There is a vibrant ecosystem of tools and an active community of users and developers around those tools. All of this is enabled by open source.

Open process

The entire development process of IPython is carried out in the open:

• All contributions undergo public code review on GitHub using Pull Requests.
• All Issues are reported and discussed publicly through GitHub.
• All real time discussions are done using public HipChat rooms (Help, Development).
• We maintain a public Roadmap that describes where the project is headed.
• We hold weekly "lab meetings" on Google Hangouts to discuss the project with core developers, users and guests. The minutes and agenda for these meetings are maintained on a public Hackpad. These meetings are broadcast publicly and recorded.
• Our test suite is run automatically on all Pull Requests and master and posted publicly.

Open standards

It is really easy, and sometimes tempting, to invent your own:

• Serialization format
• Domain specific language
• Network protocols
• Document formats

As much as possible, we try to use established, open standards:

• Notebooks documents are just JSON data and can be read in any language with a JSON parser.
• The actual data stored in the JSON document is also in open standards (HTML, Markdown, PNG, JPEG).
• The Notebook is styled using CSS
• Our message architecture is built on top of JSON, ZeroMQ and WebSockets.

In a few cases we have had to invent our own standards on top of these base standards:

• Notebook format (how exactly we store data in the JSON).
• Kernel message architecture.

We are attempting to make these standards documented, tested, versioned, etc. The broad adoption of the IPython Notebook by other language communities would not be possible without this openness.

Styling