h-index i-index g-index … these are used for academic promotions, award selections, while flawed, these are indexes that are used to measure our academic contributions.
Part of advancing open science is developing a metric that can quickly convey how researchers participate in open science. I was looking over: https://www.open-science-conference.eu/wp-content/uploads/2016/02/Open-Science-Conf_Panel_full-set.pdf
I see 4 areas that could be included… data, software, publishing, engagement, but maybe there are more that are web-scrapable?
Data - author of any open datasets at public repositories?
Journals - % publications open access
Software - github id? active? open license?
Engagement - medium / twitter / discourse posts?
I know something like this would be very difficult to design, but it seems like many of the right minds are in the room to start a discussion, perhaps bring in the authors of this talk, and discuss how something might be formulated?
1 Like
Just on this, I recently stumbled on the Orbit Model which is a framework for measuring the community engagement aspect across different platforms like GitHub, Twitter, Discourse, Slack, etc. Not sure if this is what you’re looking for, but I thought I’d share the concept with the wider Pangeo community.
I’ve had discussions related to managing a community with @JessicaS11 in the ICESat-2/icepyx community which is also working on promoting Open Science, and we do keep track of metrics like GitHub traffic and PyPI downloads for reporting to funders. But that is on a ‘project’ level rather than a level for an ‘individual’ scientist, and as you said, it is indeed difficult to design a single, web-scrapable metric that combines all the things you mentioned across so many different open platforms.
Just to be a bit more meta and fuzzy, perhaps we should be moving away from single metrics in general, traditional or otherwise. Who’s to say that you can label Researcher X (who published lots of open access papers) as more ‘open’ than Researcher Y (who does lots of open science outreach). There are so many different dimensions and layers of openness, and I think it’s more important that we simply acknowledge and recognize all the diverse ways people can do Open Science. We’ll know that we’ve reached the goal of 100% Open Science when people just call it ‘Science’ 
Other references:
Just came across Open Science Badges from Center for Open Science which is a good concept to digitally encourage and engage researchers to embrace the open science culture.