Audit of RF4 Impact Metric Calculations
Objective
Conduct a code and data audit of the 16 impact metrics implemented by Open Source Observer for Optimism Retro Funding 4.
The intended business logic of impact metrics will not change. The purpose of this audit is to identify any bugs/issues within the code. If any critical issues are found, then a fix will be pushed to the query logic before voting closes. Changes made in this way will not impact voting ballots, as badgeholders are voting on impact metrics not individual projects.
Context
A full write-up on the process OSO employed is available here and in a more distilled format here. All SQL models are available in interactive format here and from our GitHub.
The downstream models most relevant to this audit are available in interactive format here and in our GitHub here
Auditor requirements
The auditor will need a strong command of SQL and blockchain concepts. Experience with dbt and working in a BigQuery environment will be helpful.
Important: If you are interested in participating in the bounty, please join our Discord and provide a short introduction, including a link to your GitHub profile. We will review your profile and let you know if you are eligible to participate.
Scope of audit
The auditor is expected to audit the following components of the impact metrics model:
-
Unified events model
-
Audit the primary event models (eg, Events Daily to Project, 4337 Events), upstream source data (eg, Base Transactions, Base Traces), and the dbt macros used for intermediate processing (eg, contract_invocation_events, filtered_blockchain_events).
-
Comment on the following aspects:
- Completeness and accuracy of underlying transaction and trace data for the six relevant chains
- Correct and consistent implementation of the RF4 transaction window (2023-10-01 to 2024-06-01)
- Correct and consistent implementation of the relevant RF4 onchain event types (eg, gas fees, contract interactions, etc.)
- Implementation of “special cases” for 4337 interactions (see here) and EOA bridges (see here)
-
-
Contract discovery and attribution model
-
Audit the contract discovery logic (eg, Derived Contracts, Contracts by Project) and upstream logic for identifying factories, deployers, and deterministic deployments
-
Comment on the following aspects:
- Completeness and accuracy of the Contract by Project database, including efforts to de-dupe contracts deployed by creator factories such as Zora’s and pool factories such as Aerodrome’s
- Correct attribution of all contract artifacts included in approved project’s applications (see here)
-
-
Trusted user model
-
Audit the trusted user model logic (see here) and upstream source data (from Farcaster and reputation data providers)
-
Comment on the following aspects:
- Completeness and accuracy of underlying source data
- Correct implementation of the trusted user model heuristics
-
-
Impact metric implementation
-
Audit the logic of the Impact Metrics by Project model and the 13 upstream models that power individual metrics
-
Comment on the following aspects:
- Correct and consistent implementation of all 13 underlying metrics models and 3 log-transformed metrics models
- Correct and consistent implementation of the summary model that aggregates all 16 metrics and links them with project applications
-
-
Open Source labeling