• The dashboard shows a lot of information and allows a variety of actions – this made it crucial to design the application focusing on the user experience.  I worked with various aspects of a front-end framework (Bootstrap here) and came across the ease of basic styling of HTML elements such as forms, tables, buttons, icons, etc. and the most useful – Bootstrap modal. Bootstrap provided with a consistent theme for the dashboard with good documentation. It now keeps the scope to leverage the grid system to allow development of a mobile-first application.
• Running codebase in different environments is always a great learning. Working with the file-download feature I learned that a given request on the web worker may only last 30 seconds on Heroku (our production environment) and will be killed after that period. Since creating a zip file can easily take longer, we did some brainstorming and decided to keep the network calls out of the request-response cycle by creating a celery worker task to do the downloading job. Working with celery worker tasks and Redis broker was a new and enriching experience for me.
• To allow OH-run version and allow developers to run a version themselves as well, we weighed in two options to store the downloaded files – AWS S3 and transfer.sh wherein we settled on the first option given the 10GB limit on the latter.
• Since the downloading of files happened as a background job, the user could be notified of the completion of file download either through a dashboard notification or via an email. We decided to go ahead by emailing the user (easy peasy) by setting up a configurable SMTP server in Django.

The work done so far has been rewarding in terms of experience with Django, Bootstrap and various other modules. More development calls for more feedback and hence, more iterations. Therefore, I’ll be working on making some modifications to the UI and features incorporating the feedback.

More Tango with Django ahead 😀 Cheers!

We are happy that Open Humans will have four Outreachy interns this summer. Our interns are working on their own Open Humans related projects and will regularly blog about their internship experience. Read Manaswini’s post about working with the Github API for a new Open Humans project:

I have been working with Github API all this while. I had come across some really cool visualizations with Github API but hadn’t had the chance to work on it. Thanks to the project ‘Adding data sources’ that I was motivated to add Github as a data source and guess what, I discovered that it will work in principle! The online development tool Github enables developers to contribute and discover open source projects, thus realizing their aspirations.

Github provides REST API access to get variety of the data of your projects from Github, be it repositories, issues or pull requests and lots more! I chose Github API since the documentation is meticulous and easy to comprehend for anyone who is looking forward to getting started. Also this provides an excellent source for creating data explorations.

The outputs generated are available in various formats including JSON and other compatible formats depending upon the data to be extracted. One can view public data without authentication but in case one wants to store private data, then authentication is a must. Both single and two-factor authentications are supported. Two-factor authentications, as the name suggests, is more secure as this doesn’t include sharing your passwords as in basic authentication.

This internship period has been truly rewarding. I came across some really cool stuffs such as rate limiting and requests respectful during this period which I was unaware of until now. Diving deeper, I also came across the types of rate limiting.i.e. user rate limiting, geographic rate limiting and server rate limiting. All thanks to my mentor, Mike Escalante for giving me invaluable insight regarding the above terms and supervising me in each and every step.

The Github API provides rate limits of 5000 requests per hour. Till now, I have been editing the demo template and resolving issues simultaneously. I was experimenting with the output JSON and investigating various ways in which the output JSON will look better.

Cheers!

We are happy that Open Humans will have four Outreachy interns this summer. Our interns are working on their own Open Humans related projects and will regularly blog about their internship experience. Read Tarannum’s post about working on creating packages  for the Open Humans API:

The Open Humans API provides command line tools to interact with Open Humans. From where, I started working with Open Humans API to where it is now, the transformation has been just magical.

With the help of Open Humans organisation mentors,  lot of features were added like messaging and uploading large files with the help of AWS. From learning the proper way to name the functions to testing the functions, documenting the API using Sphinx, adding more command line tools, every smaller piece of learning added to create an avalanche of wisdom.

For legacy code, test is a necessity. The code is full of logic, it’s alive and it’s core to any application. We need to keep going back to the code to update it, so tests are crucial.

I mocked the API test using VCR.py . It’s just like Thor’s stormbreaker (Marvel fan will understand) in the field of unit testing. It’s apt for those applications which make http calls to external services(in my case Open Humans).

Why VCR.py

The external service should not be tightly coupled with the application. If the external services fail, the application should still run correctly. One absolute novice testing way is to hard code error prone request and code the necessary error handling routines. But, this is highly not recommended because once you update even a few line of code, you have to manually test it again. So, we hop on to the more sophisticated way of testing: Unittesting.

Running tests which call the external service every time will be very slow. Apart from it, the tests won’t work offline and sending too many requests to the external service can be a problem too. So, here steps the protagonist, VCR.py. 🙂

You just need to run the test online once when the cassette file which stores all the important information related to the requests response has not been created. Once, it is done, the response of requests will just be compared with the cassette file and yes , it’s done 🙂 . You have became successful in making your application more sturdy.

How to use VCR.py

For each function serving a particular feature, a class for testing is created. For testing different lines of code of a particular feature, different tests(functions with assertion) is created in the test class. Tests for valid response, invalid responses are written.

For the first time, when the test is run, the request really hits the external service and a cassette file is formed. Once the cassette is formed which contain status codes, status, etc, the response of tests is just compared with the cassette file when the tests are run again. It really speeds up the testing process and saves us from sending requests to the external service again and again. Whether you are offline at your home or 40,000 feet above the ground anytime you can test your code, once the cassettes are formed.

Want to try a hand on this cool stuff?  You can follow this https://github.com/OpenHumans/open-humans-api/blob/master/ohapi/tests/test_api.py. The link contains a lot of examples to write tests using vcr.py. You can see the bigger picture by observing any function in the https://github.com/OpenHumans/open-humans-api/blob/master/ohapi/api.py and how for each functions tests and cassettes are formed.

I would like to thank Mad Price Ball for mentoring me in the best way possible. The meetings with them are fun where I learn a lot of new stuff and a new way to see things, to tackle an issue with a fresh perspective.

Apart from this, I also worked on testing the public functions of Open Humans API. Presently, I am working on creating a reusable django app for Open Humans. A lot of brainstorming goes while you are in the process of designing an application, it’s use cases, what kind of audience it will cater.

Reusable application are the application which will be used by other applications. After all, reusability is the way of life in Python. I will cover the reusable app topic in my next blog. Till then, stay tuned. 🙂