Monthly Archives: December 2017

Meet the TwArχiv

This is a cross-post from my personal blog, where I also introduced the new Open Humans project I created.

Would you like to analyze your Twitter history? Exactly 5 years ago Twitter started offering the option for users to download their full archive of personal tweets. The archive gives you a change to quickly browse through your personal history and find those funny cat pictures you once posted. But there is additional value in the archive, transcending the trips down to memory lane. For example, by looking into a full Twitter archive one can investigate longitudinal trends in interaction behaviour or geotag-based movement patterns. While Twitter archives come with their own user interface, they are not really designed for such deeper dives into the data. Which is why I have been working on a small tool called TwArχiv that tries to allow for such insights.

Screen Shot 2017-12-21 at 14.08.09

The plot above is a static example of what TwArχiv does. It gives you an idea of how my personal reply behaviour has changed over time. Using a gender-guessing Python library it classifies the users I replied to based on their first names. You can quickly see that I did have an extreme male-bias starting in early 2009 that becomes less pronounced in more recent years. What happened in 2009? I became an active member of the overwhelmingly male German Pirate Party. Go figure.

The plots that the TwArχiv generates include further interesting data. You can see a full, interactive demo done on my personal Twitter archive. The TwArχiv also looks into how your tweet behaviour changes with respect to whether you use Twitter for replying, retweeting or making original posts on your own; at which times of the day you tweet; and even how you move across the globe while tweeting. The gif below gives you a small glimpse into parts of my 2016 movements based on tweets.

If you want to analyse your own Twitter archive please give TwArχiv a try. As it uses Open Humans for the archive storage you can optionally also choose to make that data publicly available. If you have an unprotected Twitter account this data is already public but harder to systematically access (at least for anyone but Twitter itself). So why not help out social media researchers everywhere by giving your data?

If you have ideas for some data analyses that are missing so far: The code of TwArχiv is open source and I’d love to get your contributions and suggestions. And if you have a larger feature/analysis idea that’s not in TwArχiv yet, Open Humans is giving out grants of up to $5,000 for projects that help to grow their eco-system. Improving on TwArχiv is a perfect match for this.

An interview with project grant awardee Benjamin Carr


Today we’re interviewing Benjamin Carr. He is not only a long-term member of the Open Humans community, but also the recipient of two(!) of our Project Grants.

His two – closely related – projects are the integration of both Google Fit and Microsoft Healthvault into the Open Humans Network.

Benjamin, please give our blog readers a quick introduction about who you are!

I originally became aware of the Personal Genome Project, way way back at its dawn in 2005 while I was working at Woods Hole Oceanographic Institution in Physical Oceanography. The idea to a young scientist that PGP was basically providing seed money to make human DNA sequencing affordable was amazing, and I was quick to sign up, and encourage my whole family to join as well. For those that don’t know, becoming a PGP volunteer was a multi-year process requiring a multitude of questionnaires. I didn’t receive my tubes to spit into until 2012! I attended a couple of the GET conferences in Cambridge, MA when I was across the river working on my PhD in Biology at Boston University. My science and technology passions found an excellent intersection with Open Humans.

You have been a part of the Open Humans community for a long time. How did you initially come to Open Humans?

I’m not sure of the exact date, but it was shortly after launch in 2015 that I joined Open Humans. Most likely due to an email blast from Madeleine Ball to the PGP group. I quickly linked up as much data as I had available at the time, and made things like Wildlife of Our Homes public. Furthermore, I am one of just 26 People to admit that we owned a “Jawbone Fitness tracker,” an early step counter and Fitbit competitor.

Between trying to spur communications on the forum and Slack channel, submitting issues and pull requests for the website itself and individual projects, I’ve banged my head against most things Open Humans, save the legal bits.  

You volunteer for Open Humans, what are your responsibilities and how did you end up in that role?

Like all of us I volunteer data. But I also help to edit mailings and postings that go out in email blasts and on the blog. I report issues on Github and Slack. I try to help new users and developers when I can and do  a bit of coding for different parts of the system myself, and keeping an eye out for bugs and vulnerabilities. I also started managing the Facebook Page in March.

Open Humans is looking for more than just data, programmers, and scientists, the idea is to make lots of information “open”. Having had some experience and the resources to manage a social media system and strategy my offer to try to make the Facebook Community more vibrant was welcomed with open arms. Hopefully you’ve noticed the steady stream of what I hope are interesting links and stories on our Facebook Page.

If you have any ideas for content or want to host something like a Reddit AMA on our Facebook drop me a line! I am always looking for new exciting things to share. Being a scientist myself, I am actually running a Facebook experiment. I started posting over the weekend, and Sunday posts seem like a winner for interaction with our Facebook fans! I’m also trying to see if changing the timing of the posts inspires more people to like, share or comment. We’ll see!

Open Humans could always use help making things more accessible. We try our best to make each piece understandable to a wide audience, but just having volunteers review the documentation and explanations of projects and goals would be welcomed!.

You did not only get one Open Humans project grant, but two. What are these two projects you have planned?

The first as you mentioned is to incorporate Google Fit data into the Open Humans platform. Google Fit ships with every Android phone that uses the Google Apps, which means as of this interview there are more than 2.6 Billion active Android phones. While Fit natively tracks things like steps and stairs, there is an entire ecosystem of products that plug into it, like Running, Biking, and even Push Up Apps! As well as many devices from glucose meters for diabetics to blood pressure cuffs, and even CPAP machines for people with sleep apnea! The second project has some overlap with the devices supported, but MS HealthVault is really a repository for millions of people to keep track of medical records, MRI images, XRays, Prescriptions, and even directly link Electronic Health Records!

Your two projects sound a bit related. How did you get the idea for them? Are you using these services yourself?

The two are related, honestly the Google Fit seemed like a logical integration I just hadn’t had time to work on it, the Healthvault occurred to be during a discussion with you on Slack about openSNP and pulling that data into Open Humans. By my using the APIs (Application programming interfaces) provided by Google and Microsoft I can allow participants to share data collected in these two “silos” with the Open Humans community and researchers. By adding these two integrations I hope to increase the appeal for researchers to look at and use the Open Humans datasets, as we have years, and in some cases in MS HealthVault, decades of data that can be accessed along with genetic sequences from those that have provided it it from services using microarrays or chips like 23andMe and Ancestry, or full genome sequences!

When I’m done with the grant work, the code should be self updating, or offer the ability for users to request data be pulled in at the push of a button. I do use both MS HealthVault and Google Fit, both directly and with attached devices and services. I started using MS HealthVault when Google Health closed, and I migrated all of the data I had put into Google Health for PGP from Google to the Microsoft platform. I’m also a long time Google Phone user, these two facts makes coding for both APIs easier as I have a ton of data in each of the respective warehouses.

Is there anything important that we didn’t cover so far that you’d like to add?

I have seen how genomics can play a huge part in one’s life. I am very open with my doctors about being lucky enough to have a sequenced genome. By being able to do a little bit of my own research and ‘checking my genome for Z’ based on doctor’s suggestions, symptoms that previously eluded diagnosis started to line up. I also lost my first wife to Ovarian Cancer that was accelerated – if not actually caused – by a family history of Prostate Cancer, something we wouldn’t know until after she had passed.

On the lighter side I’m an avid photographer, and when I was doing my PhD work in Boston went to the woods nearly every weekend, to clear my head, rejuvenate my soul, throw a tennis ball for my labrador retriever, and rack up a very high shutter count on my old DSLR. I took this picture of the first of three upcoming “Supermoons” a couple weekends ago.


I am also an environmentalist. My background spans fisheries and oceanography, but also the hereditary traits passed on from mother to offspring of Walleye fish, the biocomplexity and interconnectedness of the Great Lakes.I was one of the four man crew who did the 9/11 impact assessment of the Hudson River Estuary from the George Washington Bridge to the Battery! I’ve been Chief Scientist on multiple cruises including working from small boats to large 115 ft, 200 ton vessels! Even though my first cruise in 1999 was one of the worst weather-wise (it was so bad the glass in the windows shattered), I still have an undying love for the oceans, being on the water or just being near it.

Meet the first recipients of the Open Humans Project Grants

Open Humans is a collaborative endeavour that would be impossible without the individual support of each and every user. The whole idea behind Open Humans relies on a healthy ecosystem that flourishes with each contribution made by individuals. Our project grants are thus designed to support individual ideas that have the potential to help grow the whole Open Humans ecosystem. For example, such projects can provide new ways of analyzing and visualizing existing data sources. Or they can connect new data sources that were so far not represented in Open Humans. This open-endedness has inspired many of you. Since the initial announcement of the project grants in July 2017 we got many great submissions. And as the grants are on a rolling schedule there is no deadline to apply: You can go ahead right now!

After carefully going through the applications we got so far we are happy to now announce the first three projects. Each of them will be funded with $5,000 to support them in realizing their vision for how Open Humans should grow.

MyFitnessPal Miner

Millions of users all over the globe already use MyFitnessPal. This thriving community makes MyFitnessPal (MFP) one of the most successful apps to track both your calorie intake and your exercise. Anh Nguyet Vu’s goal is to bring this community to Open Humans. Her MyFitnessPal Miner will not only enable the import of public MFP data into Open Humans, but also generate insightful visualizations out of the MFP data. Long-term goals for the MFP Miner are the inclusion of genetic data from 23andMe and the inclusion of private data.

You can find an early prototype of the MyFitnessPal Miner on Anh Nguyet’s website, Bring Your Own Biology.

Open Humans Google Fit Integration

Google Fit allows Android users to merge activity data from different sensors, devices and apps into a single data stream. In that sense it is Google’s health-tracking answer to Apple’s HealthKit. But while Open Humans could already import data from HealthKit, there was so far no easy way to do the same for Google Fit data. Thanks to Benjamin Carr this is about to change. His Open Humans Google Fit Integration will allow both services to communicate to each other. With over 80% of all smartphones running Android this will allow a large community to put their data into Open Humans!

You can find Benjamin on Twitter and his first stab at the Google Fit Integration can already be found on GitHub.

Microsoft Healthvault Integration

Microsoft as a third big player, alongside Google and Apple, when it comes to health-related data about individuals. Microsoft’s Healthvault is a web-based personal health record that has been around for around 10 years by now. It not only takes in data from personal fitness devices, but Healthvault can also aggregate medical records and prescription fillings. This makes it go a step further than HealthKit and Google Fit and offers great potential for a connection with Open Humans. Luckily API-programming nerd Benjamin Carr – yes, the same who will work on the Google Fit Integration – did have some more time on his hands. This is why he proposed a second project that will bring together Microsoft Healthvault and Open Humans.

Find Benjamin’s current progress on this on GitHub!

What’s next?

In the next weeks we will release some short interviews our grantees so that we all can get to know them (and their projects) better. Did seeing these projects inspire you to run your own? Apply with your own project idea right now!

What is Open Humans to me?

I’m Steph, I’ve just started as a software developer at Open Humans, and in this post I want to describe what the organisation means to me.


I feel like the value of Open Humans can be split into three main categories, perhaps of increasing fuzziness in terms of concrete assets, but also, in my opinion, of increasing importance and rarity. Open Humans is a technological platform; it’s a vibrant community; and it’s a paradigm shift.


At its very core, Open Humans is a technological platform. People are increasingly finding themselves in possession of their own personal data. Whether this be from fitness tracking devices; commercial genome sequencing services; or internet search history, we are, somewhat inadvertently, gathering more and more data about ourselves.

Fig1: people are gathering data about themselves


The Open Humans platform allows members to upload and store these data privately, to choose whether to share some publicly, and to use their data to contribute to research projects and learn more about themselves.

Fig2: people can upload their data easily using data tools


For researchers and citizen scientists, the platform enables painless and efficient data collection from engaged research participants. It is a seamless pipeline for human subjects research, which puts the individual participants in charge of how their data is used, avoiding a one-size-fits-all ethics approach which is common in traditional research protocols.

Fig3: data can be used to further human subjects research


Open Humans is defined by its vibrant community. In recent years there has been a sharp rise in production and use of personal tracking systems: wearable devices; smart scales; lifestyle logging apps (including diet, exercise, and sleep); and commercial genetic and ancestry tools. People are intrigued by their own data. For this reason, there is no single user profile: we are researchers; patients; data scientists; citizen scientists; any and all people who want to learn more about themselves. The Open Humans community have written 19 data transfer tools enabling data from external projects to be added to Open Humans by users at the touch of a button. They have contributed 9 projects to the site, where research can be done on participants’ shared data. And they have continued to be enthusiastic, motivated, and truly engaged in the work of the organisation.


Open Humans is a paradigm shift: a totally new way to do humans subjects research. For me this is the most exciting way to look at Open Humans. Personal data can be sensitive: sharing can cause embarrassment, expose health concerns leading to discrimination, or lead to identity theft. Historically in the medical world, this has been handled by keeping health and human subjects research data anonymous. However as data becomes richer and more descriptive (for example, a genome, or internet search history), it is becoming easier to identify the original subject. So now we have more data than ever, and keeping it private when using it in research is becoming harder than ever.

Fig4: the traditional human subjects research pipeline: data is handed over to scientists and generally not returned, subjects do not learn about the results and don’t get a say in how they are shared


Open Humans turns the traditional research pipeline on its head. It puts individual subjects at the centre of the sharing process, and in full control of how their own data is used in research. People are unique, and each will have their own reasons for wanting to keep some data private. These diverse sharing preferences call for a new system for human subjects research, that focuses on the subjects themselves, and meets their own personal privacy requirements.

Fig5: the Open Humans research pipeline: participants have autonomy over their data and can choose which studies they share their data with


Giving research subjects the autonomy they deserve and at the same time increasing the efficiency of the research pipeline seems like a great idea, but the project is ambitious. Large scale open projects do have the ability to change the world (think Wikipedia), but changing the status quo in a system that has been around for a long time will always come with a lot of friction. However the vast amount of data being generated these days means that we are in new territory. This is a great time for change. Making sure that people are empowered to make their own decisions about how their data is used is an important endeavor. Working closely with our community, we hope to reach a critical mass of membership such that personal data sharing in this way becomes the standard approach. I am excited and optimistic about how Open Humans can revolutionise human subjects research, and I’m very grateful to be a part of this exciting movement.