Contribute Data

We're always looking for new data sources to integrate with OSO. Here are the current patterns for integrating new data sources:

• 💻 Get started - Setup your development environment with Dagster
• 🗂️ BigQuery Public Datasets - Preferred and easiest route for sharing a dataset
• 🗄️ Database Replication - Provide access to your database for replication as an OSO dataset
• 📈 GraphQL API Crawler - Automatically crawl any GraphQL API
• 🌐 REST API Crawler - Automatically crawl any REST API
• 📁 Files into Google Cloud Storage (GCS) - Drop Parquet/CSV files in our GCS bucket for loading into OSO
• ⚙️ Custom Dagster Assets - Write a custom Dagster asset for unique data sources
• 📜 Static Files - Coordinate hand-off for high-quality data via static files. This path is predominantly used for grant funding data.

Reach out to us on Discord for help.

Deprecated

• 🔁 BigQuery Data Transfer Service - Makes it easy to transfer from S3 to GCS into BigQuery
• ✈️ Airbyte - A modern ELT tool