Google required a reliable analytics dataset for Bitcoin to replace an abandoned open‑source Blockchain ETL project. The need was not for a one‑time experiment, but for a continuously supported, production‑grade deployment that internal teams could trust.
Main pain points / challenges
Aging and unsupported open‑source ETL project
Inconsistent dataset freshness and correctness
Lack of reproducible deployment and monitoring
Issues we solved
Task
Dataset built on abandoned community components
No stable operations or supportability on GCP
Temporary inconsistencies due to blockchain reorganizations
No defined monitoring, alerting, or runbooks
No responsible team for long‑term maintenance
Results
Operationalized Bitcoin ETL for long‑term production use
Balanced real‑time streaming with daily reconciliation for correctness
Integrated reorg protection mechanisms by design
Full observability stack with tuned alerts and custom dashboards
Reproducible IaC + GitOps deployment on GCP
Continuous support and improvements from Dysnix team
Case summary
Dysnix turned an abandoned blockchain ETL pipeline into a stable Google service. It streams Bitcoin data into BigQuery with daily reconciliation for dataset consistency. The historic backfill of the entire Bitcoin blockchain was done in under 24 hours.
Delivered as Infrastructure as Code with Terraform and Flux CD, it offers reproducibility, observability, and resilience against reorgs—maintained continuously to ensure dataset reliability and currency.
Solutions implemented by Dysnix
Secure internal Bitcoin node cluster (GKE)
Private RPC endpoints serving only the ETL flow.
Hybrid batch + streaming ingestion
Pub/Sub + Dataflow pipelines with Airflow daily reconciliation.
IaC + GitOps
Terraform / Terragrunt + Helm + Flux CD for reproducible deployments.
Reliability-verified design
Block delay strategy with correction jobs for consistent dataset states.
Observability and runbooks
VictoriaMetrics, Grafana, and tuned alerts with clear failure recovery guides.
Continuous support and upgrades
Ensuring long-term Bitcoin dataset stability and quick client-oriented adjustments.
Daniel Yavorovych
Co-Founder & CTO
Meet the Dysnix engineers trusted by Google and discover what they can offer for your project.
.svg){kind=icon}
.svg){kind=icon}
.svg){kind=icon}
-min-p-800.webp)
-min-p-800.webp)
-min-p-800.webp)
-min-p-800.webp)
-min-p-800.webp)
