How to get Fit(bit) with Python

Fitbit trackers are used by millions of people to track personal metrics such as number of steps, floors climbed, heart rate and quality of sleep. Their products include wristbands and smart watches which sync the data collected to the Fitbit cloud. By the end of 2019, Fitbit had sold over 100 million devices^[1]. This data is displayed in graphs and tables through Fitbit’s own excellent web and mobile apps. But what if you want to further analyse the raw data and create your own charts?

Alongside the apps, Fitbit also makes your data available through an Application Programming Interface (API). Using this API, it is possible to query for certain metrics and answer questions such as: what was my step count for each day over the last week; or how much deep sleep did I get last night? In this way you can analyse and plot the data using your own tools to uncover trends that the Fitbit generated charts may not show.

The guide below will firstly show how to register an “app” with Fitbit in order to use the API. The next step will be to authorise the newly registered app to allow use of your data. After this is done, a “token” is provided to make it possible to run queries against the API. This token expires after eight hours, but the app will automatically fetch a new token using the “refresh token”. This authorisation flow used by the Fitbit API is called OAuth 2.0.

Fitbit OAuth Background

OAuth is an open standard for access delegation, commonly used as a way for internet users to grant websites or applications access to their information on other websites but without giving them the passwords.

OAuth wikipedia ^[2]

In this case:

• internet users is you
• applications is the small Python application we will create
• their information is your Fitbit data
• other websites is the Fitbit platform

Access tokens are issued to third-party clients by an authorization server with the approval of the resource owner.

The OAuth 2.0 Authorization Framework ^[3]

In this guide, we will obtain access tokens from Fitbit with your approval to access the data. Figure 1, taken from the Fitbit developer page^[4], shows the process. In this case the “Client Application” will be a local running Python application which will start up in order for us to obtain the access token.

In this guide, we make use of the unofficial Python Fitbit client (https://github.com/orcasgit/python-fitbit)^[5] which will hide much of the lower-level details of OAuth, as it is built on top of Python OAuth libraries.

Steps to Authorisation

The steps to initially authorise Fitbit are the most complicated for newcomers. Once the access tokens are obtained then it is easier to explore the API as each query either gets a request data or a clear error message. The OAuth 2.0 process is tricky due to security concerns with giving access to your data.

Register a new app

Open a browser and log in to your fitbit account. Once logged in, register a new app by going to the developer pages at https://dev.fitbit.com/apps/new. You should see a screen similar to Figure 2. Fill in the form details. As this is “app” for personal data use, ensure that the “Personal” radio button is clicked, and that the “http://127.0.0.1:8080/” is the callback URL. The callback URL is needed for later in the process. The other fields are not as important as the app is only for your personal data.

Once you have filled in the details and clicked “Register”, a screen similar to Figure 3 is shown. This means that your app has been successfully registered and you have been assigned a “OAuth 2.0 Client ID” and a “Client Secret”. These details are unique to your app. Leave this browser page open as the ID and secret will be needed in the next step.

Clone fitbit-collect Github repository

In order to simplify the initial retrieval and also the refreshing of tokens, I have created a small Github repo (https://github.com/davgibbs/fitbit-collect) to help with the process. The purpose of the repo is to simplify Fitbit data collection by providing Python 3 tools to authorise your app and store the tokens. Clone this repo onto your computer and installing the dependencies:

 ~$ git clone https://github.com/davgibbs/fitbit-collect
 ~$ cd fitbit-collect
 ~/fitbit-collect$ virtualenv --python=/usr/bin/python3.6 venv
 ~/fitbit-collect$ source venv/bin/activate
 (venv) ~/fitbit-collect$ pip install -r requirements.txt

Run gather_keys Python script

The last step in the authorisation phase is to run the gather_keys_oauth2.py. This script opens up a connection to Fitbit to allow you to authorise the client. In parallel, it also starts up a CherryPy Web application on 127.0.0.1:8080 to receive the tokens from Fitbit once you have done the authorisation. The script uses the client details shown from above as arguments to the script in the order <client_id> <client_secret>. In my case, this means running the command below, but you should add in your own client ID and secret:

(venv) ~/fitbit-collect$ python gather_keys_oauth2.py 22BPKX 04a77d071c5c71ce155c459841ed66ab

Once this script starts, the screen in Figure 5 should appear open in your browser. Click the “Allow All” checkbox and then “Allow”.

After allowing access, this script finishes successfully and your token details have been obtained. They are printed in the console and have been written to two new files in your directory: client_details.json and user_details.json. Have a look inside user_details.json to see what the token details look like. Keep these files safe on your computer and do not share as they give access to your personal Fitbit data. Once you do not lose these token details, you will never need to look at this authorisation step again 🙂

Note: If the browser does not open up the correct URL, copy the URL (shown in the script output) into your browser manually. Then click “Allow” access as above.

Query your Fitbit Data

Great! You are authorised to access your data for “Personal Data” app, and have received the access token and refresh token. Next the token is used to get the real data you want from the Fitbit platform. The script used for this is run_collect.py. If you look inside the file, on line 7 of the snippet below, you can see that it calls the Fitbit() class passing the relevant parameters of client details and tokens and the makes the request:

def run():
    # read from client and user details json files
    client_id, client_secret = _get_client_details()
    access_token, refresh_token, expires_at = _get_user_details()

    # Create Fitbit OAuth 2.0 session
    auth2_client = Fitbit(client_id, client_secret, oauth2=True,
                          access_token=access_token,
                          refresh_token=refresh_token,
                          expires_at=expires_at,
                          refresh_cb=refresh_callback)

    # Get the API URL
    fitbit_url = FITBIT_API + _get_api_call()
    request_result = auth2_client.make_request(fitbit_url)
    # Write result to local file 
    _write_results(request_result)

The “refresh_callback” in line 11 is a function that is only called when the token has expired and a new token has been issued. It updates the user_detail.json file with the new user token details.

The _get_api_call() function from line 14 is shown in the snippet below. Query what you would like to know from the Fitbit data you have available. Some examples are shown below. If what you need is not included below, the Fitbit API tool contains many options for obtaining what you need: https://dev.fitbit.com/build/reference/web-api/explore/

def _get_api_call():
    """
    Date Api in this format:
    GET /<api-version>/user/<user-id>/<resource-path>/date/<base-date>/<end-date>.<response-format>
    <user-id> can be left as - to get the current user
    """
    # Steps in the last 7 days
    steps_last_seven_days = '/1/user/-/activities/log/steps/date/today/7d.json'
    # Steps between two dates
    steps_dates = '/1/user/-/activities/log/steps/date/2020-04-12/2020-04-18.json'
    # Calories last 7 days
    calories_last_seven_days = '/1/user/-/activities/log/calories/date/today/7d.json'
    # Profile info
    profile_info = '/1/user/-/profile.json'
    # Sleep between two dates
    sleep_dates = '/1.2/user/-/sleep/date/2020-04-13/2020-04-17.json'

    return steps_last_seven_days

In my case, I want to know how many steps I have taken each day over the last week. Running the script makes a call to Fitbit API and writes the results in a fitbit_data.json file.

(venv) davidg:~/fitbit-collect$ python run_collect.py 
Successfully written result data to file fitbit_data.json
(venv) davidg:~/fitbit_collect$ cat fitbit_data.json 
{"activities-log-steps": [{"dateTime": "2020-04-12", "value": "5007"}, {"dateTime": "2020-04-13", "value": "5555"}, {"dateTime": "2020-04-14", "value": "10854"}, {"dateTime": "2020-04-15", "value": "4564"}, {"dateTime": "2020-04-16", "value": "3404"}, {"dateTime": "2020-04-17", "value": "3141"}, {"dateTime": "2020-04-18", "value": "12558"}]}

Plotting the data

Now that we have the data in a usable JSON format, plotting libraries such as matplotlib can be used to create graphs. Included in the Github repository is a plot.py file to get you started. But many different charts are possible. In Figure 6, I have charted my steps over a week:

In Figure 7 I have charted my sleep type over four days. Fitbit breaks down the sleeping hours into “Awake”, “REM” (Rapid Eye Movement), “Light” and “Deep”. This can help you to assess your sleep quality each night.

Conclusion

This guide showed you how to obtain your raw Fitbit data using Python. OAuth 2.0 is a secure means of allowing your “app” to access the data. Fitbit maintains an API to allow you to access your data in this way.

The initial authentication step is a little complicated. It involves creating the app, then authenticating that this app can access your data. Access is granted through a valid token.

Once the token is obtained, then it is easy to make API queries against all types of your Fitbit data. The full range of Python analysis and plotting tools, such as matplotlib then become available to you to explore your data.

References

1. Statistics on Fitbit: https://www.statista.com/statistics/472591/fitbit-devices-sold/
2. OAuth wikipedia page: https://en.wikipedia.org/wiki/OAuth
3. The OAuth 2.0 Authorization Framework: https://tools.ietf.org/html/rfc6749
4. Fitbit OAuth 2.0 documentation: https://dev.fitbit.com/build/reference/web-api/oauth2/
5. Unofficial Python Fitbit Client: https://python-fitbit.readthedocs.io/en/latest/#