Science in the Sands

My open science notebook and blog

The top 10 signs that you are running the Hejaz 50 Ultramarathon

Last Saturday, the ultramarathon scene finally, quietly, reached the Kingdom of Saudi Arabia. There was no fanfare and certainly no media coverage at this first organized footrace of more than marathon length. The course involved 50 kilometers of dirt, sand, gravel, and rocks in the hills northea... Read more

Step 7 - Automate the tests

Welcome to step 7 in your training as a scientific Python code ninja: test automation. Tests are extremely useful, as long as you run them. Unfortunately, a common pattern in many projects is this: Tests are written and run by the person who wrote them. Others, and even the test-writer at a futu... Read more

Step 6 - Keep track of issues

Welcome to step 6 in your training as a scientific Python code ninja: issue tracking. Look at you! You’ve got your nice code with some docstrings and tests, and you’re humming along analyzing data, simulating widgets, and whatnot. In the middle of it all, you realize that your regression analysis... Read more

Step 5 - write tests

This is step 5 in your journey toward rock-solid scientific Python code: write tests. Tests? Ugh. I hear you. Writing tests is not an inherently fun process. However, the alternative is debugging, staring at code and thinking real hard about what could be wrong, and more debugging. I’ll take writ... Read more


Step 6 - Keep track of issues

16 May 2015

Welcome to step 6 in your training as a scientific Python code ninja: issue tracking. Look at you! You’ve got your nice code with some docstrings and tests, and you’re humming along analyzing data, simulating widgets, and whatnot. In the middle of it all, you realize that your regression analysis function breaks if you feed it more than 57 data points. And wouldn’t it be nice if it made a plot ...

Read more



Step 5 - write tests

15 May 2015

This is step 5 in your journey toward rock-solid scientific Python code: write tests. Tests? Ugh. I hear you. Writing tests is not an inherently fun process. However, the alternative is debugging, staring at code and thinking real hard about what could be wrong, and more debugging. I’ll take writing tests any day of the week. You may still not be convinced. Fortunately, I don’t have to convince...

Read more



Step 4 - write docstrings

including examples!

14 May 2015

This is step 4 in your journey toward rock-solid scientific Python code. Steps 1-3 were language-agnostic, but now I’m going to assume you’re using Python. The Python language has a built-in feature for documenting functions, classes, and modules; it is the docstring. A docstring for a very simple function looks like this: def square(x): """ Takes a number x and returns x...

Read more



Step 3 - Add a README and a License

13 May 2015

This is step 3 in your journey toward rock-solid scientific Python code. So your code is under version control and it’s floating up there in the cloud. In principle, someone else could use it. But how will they know it’s there? How will they know what it does, and how to install and use it? You need a README file. A README is minimalist documentation – just what is absolutely necessary for some...

Read more



Step 2 - Code in the open

by putting your code in the cloud

12 May 2015

This is step 2 in your journey toward rock-solid scientific Python code. So you have your code under version control. Nice. But it’s sitting there on your computer. How do you share it with somebody else? And if they improve it, how do you incorporate their changes? Version control is your first tool in this regard, too. Your second tool is Github (or Bitbucket, Google Code, etc.). Github is a ...

Read more



Step 1 - Use version control

11 May 2015

Welcome to step 1 in your training as a scientific Python code ninja: version control. This is not going to be an in-depth course, so if you’re already using distributed version control (like Git or Mercurial), feel free to skip ahead to step 2. Life before version control Programs change. That script I wrote last week to process my data needs to be adapted to a new file format or to produce a ...

Read more



12 steps toward rock-solid scientific Python code

10 May 2015

Much of my research is based on computer code. My code has bugs. Does that mean that my research has errors? Quite possibly. Are you, my fellow scientist, in the same boat? Probably. What can we do about it? We must adopt practices that are known to lead to more correct code. We must be willing to devote the time and energy that those practices require. Most of the time, this is neither fun nor...

Read more



Testing whether your code produces the correct plots -- in Python and on Travis

How to use matplotlib's image_comparison decorator in your own project

13 Jan 2015

A few years ago I was introduced to the idea of testing the scientific code I write, and now I am a huge fan of this approach. It saves huge amounts of time and vastly improves the quality of my research by helping me catch bugs early and often. I use nose for automatically finding and running all the tests. I use Travis to automatically run the tests every time I update the code – or issue a p...

Read more



Reviewing my marathon training using MapMyFitness and Pandas

Some fun with data

31 Dec 2014

I’m training for a marathon and I use MapMyFitness (MMF) on my iPhone to track my mileage and pace for each workout. MMF has a public API and Jason Sanford has written a Python front end for it. Which means that I can easily get hold of all my data in Python and explore it with Pandas! To run this notebook, you’ll need: A scientific Python stack (I recommend Anaconda) A MapMyFitness account an...

Read more