6 posts tagged with "impact data science"

The voting results for Optimism's Retro Funding Round 4 (RF4) were tallied last week and shared with the community.

This is the last in a series of posts on RF4, analyzing the ballot data from different angles. First, we cover high-level trends among voters. Then, we compare voters’ expressed preferences (from a pre-round survey) against their revealed preferences (from the voting data). Finally, we perform some clustering analysis on the votes and identify three distinct “blocs” of voters.

Retro Funding aims for iteration and improvement. We hope these insights can inform both the evolution of impact metrics and governance discussions around impact, badgeholder composition, and round design.

You can find links to our work here.

This post is a brief exploration of the before-after impact of Optimism’s Retro Funding (RF) on open source software (OSS) projects. For context, see some of our previous work on the Optimism ecosystem and especially this one from the start of RF3 in November 2023.

We explore:

1. Cohort analysis. Most RF3 projects were also in RF2. However, most projects in RF4 are new to the game.
2. Trends in developer activity before/after RF3. Builder numbers are up across the board since RF3, even when compared to a baseline cohort of other projects in the crypto ecosystem that have never received RF.
3. Onchain activity before/after RF3. Activity is increasing for most onchain projects, especially returning ones. However, RF impact is hard to isolate because L2 activity is rising everywhere.
4. Open source incentives. Over 50 projects turned their GitHubs public to apply for RF4. Will building in public become the norm or were they just trying to get into the round?

As always, we've included source code for all our analysis (and even CSV dumps of the underlying data), so you can check our work and draw your own conclusions.

Voting for Optimism’s fourth round of Retroactive Public Goods Funding (“Retro Funding”) opened on June 27 and will run until July 11, 2024. You can check out the voting interface here.

As discussed in our companion post, Impact Metrics for Optimism Retro Funding 4, the round is a significant departure from the previous three rounds. This round, voters will be comparing just 16 metrics – and using their ballots to construct a weighting function that can be applied consistently to the roughly 200 projects in the round.

This post is a deeper dive on the work we did at Open Source Observer to help organize data about projects and prepare Optimism badgeholders for voting.

Open Source Observer is working with the Optimism Collective and its badgeholder community to develop a suite of impact metrics for assessing projects applying for Retro Funding 4.

Introduction​

Retro Funding 4 is the Optimism Collective’s first experiment with Metrics-based Evaluation. The hypothesis is that by leveraging quantitative metrics, citizens are able to more accurately express their preferences for the types of impact they want to reward, as well as make more accurate judgements of the impact delivered by individual projects.

In contrast to other Retro Funding experiments, badgeholders will not vote on individual projects but will rather vote via selecting and weighting a number of metrics which measure different types of impact.

The Optimism Foundation has published high level guidance on the types of impact that will be rewarded:

• Demand generated for Optimism blockspace
• Interactions from repeat Optimism users
• Interactions from Optimism users with high trust scores / onchain reputations
• Interactions of new Optimism users
• Open source license of contract code

The round is expected to receive applications from hundreds of projects building on six Superchain networks (OP Mainnet, Base, Frax, Metal, Mode, and Zora). Details for the round can be found here.

At Open Source Observer, our objective is to help the Optimism community arrive at up to 20 credible impact metrics that can be applied to projects with contracts on the Superchain.

This page explains where the metrics come from and includes a working list of all metrics under consideration for badgeholders. We will update it regularly, at least until the start of voting (June 23), to reflect the evolution of metrics. The first version metrics was released on 2024-05-16 and the most recent version (below) was released on 2024-06-24.

Over the past few months, we've been hard at work creating the infrastructure to collect and analyze impact metrics for open source projects. We're excited to announce that we're ready to start doing some new analysis on data from open source projects ... and we need your help!

This post includes some of the domains we're interested in exploring as well as an initial list of impact metrics we'd like to collect. We're looking for feedback on these metrics and suggestions for additional metrics we should consider. We're also looking for contributors to help us apply these impact metrics.

Get Involved​

If you'd like to get involved, here's what to do:

1. Apply to join the Data Collective. It only takes a few minutes. We'll review, and reach out to schedule an onboarding call.
2. Join our Discord server and say hello.
3. Get inspiration from our Colab directory of starter notebooks for impact metrics. We also have of them in our Insights repo if you prefer to run them locally.
4. Consult our docs and especially our impact metric spec as you prepare your analysis.

The rest of this post includes more details on the types of impact metrics we're interested in.

One of our primary goals at Kariba (the team behind Open Source Observer) is to build a network of Impact Data Scientists. However, “Impact Data Scientist” isn’t a career path that currently exists. It’s not even a job description that currently exists.

This post is our first step in trying to change that. In it, we discuss:

1. Why we think the Impact Data Scientist is an important job of the future

2. The characteristics and job spec of an Impact Data Scientist

3. Ways to get involved if you are an aspiring Impact Data Scientist

   Spoiler alert: join this groupchat and apply for data access here

One important caveat. This post is focused on building a network of Impact Data Scientists that serve crypto open source software ecosystems. In the long run, we hope to see Impact Data Scientists work in all sorts of domains. We are starting in crypto because there is already a strong culture around supporting open source software and decentralizing grantmaking decisions. We hope this culture of building in public and experimenting crosses over to non-crypto grantmaking ecosystems. When it does, we’d love to help build a network of Impact Data Scientists in those places too!