Collaborative software development in Python.
In the next two hours, we are going to build our own Python package. This package will be installable via the standard installation mechanism (pip
), and will be tested and documented. We will host the source code for our project online, on GitHub, and use the git
tool to track changes to our package as it evolves. We will also see how to make modifications to others’ software, as well as how to have them contribute to ours.
Each lesson is broken up into a short lecture, followed by hands-on exercise time.
The goal of today’s workshop is for you to leave with practices skills, not to cover as much ground as possible—so, we may not get through the entire schedule. If you already understand what is being taught, please take a walk around and help others complete the exercises.
Make sure to setup your environment for the workshop by following these instructions
Demonstration: Building a Python package
Minimal package structure:
example-package
├── README.md
├── LICENSE.txt
├── requirements.txt
├── setup.py
├── darkbox
└── └── energy.py
See templates/*-example-package-*
.
Exercise: Construct your own package, and add a submodule called energy
. Inside that submodule, add a function abs_area
, that takes the absolute value of an input array, and calculates its area using the trapezoidal rule.
If you get stuck on the implementation, use the following (incorrect) implementation for the time being:
Make sure your package can be installed with pip
:
If you want to install your package while developing it, and without reinstalling after each change, use:
Launch a Python terminal (IPython, Python, or Jupyter) and import your new package. Execute your submodule function, and make sure it performs according to your expectation.
Demonstration: Writing a test suite
Exercise: Add tests for each function in your package. You may use all the assert_
functions inside numpy.testing
. Run your test suite to ensure that it passes:
Now, introduce a breaking change in one of the functions, and run your test suite again: it should fail. If it does not, go back, improve your tests, and repeat the exercise.
Demonstration: Documenting with Sphinx
Exercise: Set up Sphinx documentation for your test package using sphinx-quickstart
. In the generated docs/conf.py
, ensure you have the following extensions enabled:
Optionally, you can enable Markdown rendering:
from recommonmark.parser import CommonMarkParser
source_parsers = {
'.md': CommonMarkParser,
}
source_suffix = ['.rst', '.md']
Ensure one or more of your functions have numpydoc
-compatible docstrings. Execute the following command to generate API docs:
Compile your documentation using make html
.
Demonstration: git: feature branches
git init
, to start tracking current directory.git add
, git add -p
)For collaborative purposes, it is important to know how branching works.
Exercise: From the master
(default) branch, make two branches: docs
and add-func
using git checkout -b <branch_name>
. In docs
, update the README.md
, with a short description of the package and commit. In add-func
, add a new function to one of the submodules, commit. Switch back to master
, and merge the docs
branch, and then the add-func
branch.
Demonstration: GitHub: starting a project
Exercise: Make a new GitHub repository for your package. Upload your package.
Demonstration: How to set up ReadTheDocs and Travis-CI for your repo Slides are here
Demonstration: GitHub: Making and reviewing a Pull Request
One example of a good quality PR.
Exercise: Talk to one of your neighbors, and get their repo location on GitHub. Fork their repository, and git clone
it locally. Make a feature branch add-hello
. Add a new function to their energy
submodule (or anywhere else, if that doesn’t exist): def norm(x): ...
, that calculates the Euclidean norm, ∥x∥2 := sqrt(x12+⋯+xn2). Commit, and make a pull request.
Exercise: Wait for your neighbor to review your pull request. Modify your code in response to their feedback. Push the changes back to GitHub. Work with them until they are happy to merge your change.
PyPi (Python Packaging Index) publication
Follow the instructions at https://packaging.python.org/guides/using-testpypi/ and upload your library to the test packaging index.
Check whether your package can be installed, using: