Request a Free AuditD5 is a group of data scientists and engineers structured as a decentralized autonomous organization (DAO), a form of organization partly inspired by the blockchain infrastructure. They help startups, corporations, and organizations accelerate their data science & engineering initiatives. The project Blockchain-ETL, completed by D5, facilitates data science on blockchain data. It's currently available in Google BigQuery.
Building a high-scalable infrastructure for the Blockchain-ETL (available in Google BigQuery)
D5 Business Task
The blockchain infrastructure requirements of the platform were:
- to automate the deployment and maintenance processes
- to make the infrastructure scalable and fault-tolerant, even in the most unpredictable situations on the go (within a few minutes)
-
The Dysnix team's preparation and action list
After detailed clarification, we came to an agreement with the team participating on the project regarding the needed blockchain infrastructure components and their realization, related processes to be set up, and so on. Next, we created a to-do list of the project's goals:
- Optimize the current project infrastructure, create and implement recommendations to achieve the highest level of fault-tolerance;
- Describe the infrastructure as code;
- Optimize the project's total cost of ownership (TCO);
- Make the fault-tolerant system even more secure;
- Ensure early warning of possible technical problems and establish fast troubleshooting.
The infrastructure was optimized following our recommendations. We described the infrastructure applying Terraform and Helmfile that allowed editing and adding new components to it by easily using the inheritance system, as well as quickly deploying new environments from scratch and effectively providing maintenance.
-
Solutions we provided and implemented
Our team of three specialists passed through the all stages of renovation of the requested scalable infrastructure:
- Built a high-available, secure, multizone, cost-effective, and scalable Kubernetes-based server infrastructure in GCP with private topology;
- Fully implemented the Infrastructure as Code using Terraform;
- Implemented of useful monitoring system;
- Created a handy user interface for logs aggregation system.
Our client didn't waste time on choosing a blockchain infrastructure provider or any other planning: D5 just checked how the Dysnix team was doing their job due to the action plan, closing issues one by one.
-
Profits and values delivered by Dysnix
This project was a fascinating experience for our team. We got a lot of positive feedback and visible results at each stage of the project. All in all, we met the expectations of D5 and achieved the following goals:
- A highly-available, secure, multi-zone, and scalable server infrastructure for a Blockchain-ETL platform;
- All the development stages were shown as demos weekly so that customers could see progress on the work done;
- 70% savings in terms of the project's TCO;
- Full integration of the Dysnix team into the project and the company not as an outsourcer, but a single in-house team, which is essential at the stage of implementation of the solutions;
- Trained the in-house team on how to use the developed solutions;
- Now, the Dysnix team continues to support the project and regularly improve it.
That's one of our blockchain infrastructure projects we'd like to share with you. Let us know if you have any complicated tasks. We can solve it for you with our experience and skills.