DaivioMaking complex datawork harder.
The backend behind dataset ingestion, transformation, and AI training workflows.
A little background.
At Daivio, I work on a data-analytics and AI-training platform in the same problem space as Kaggle. The platform brings together dataset ingestion, processing, and visualisation. My role is to build the backend systems that support those workflows, working with data scientists and frontend engineers.
Where it gets interesting.
Data-heavy operations do not fit neatly inside a short HTTP request. Dataset transformations and training triggers can run for a long time, while users still need responsive APIs and a structured way to interact with their data.
The work behind the work.
APIs that connect the workflow
Built REST APIs for dataset ingestion, transformation, and visualisation workflows, giving frontend and data teams a shared interface to the platform.
Long-running work, off the request path
Implemented Celery-based asynchronous pipelines for ETL operations and model-training triggers, separating background processing from interactive requests.
A data layer built with intent
Optimised PostgreSQL schemas and queries for large structured datasets and contributed to broader system-architecture decisions.
Follow the flow.
A simplified, conceptual workflow based on the résumé. It is not a production architecture diagram and does not disclose proprietary details.
Beyond the implementation.
The interesting architectural boundary is between accepting work and completing it. A robust data platform makes that boundary explicit: the API handles interaction, workers handle processing, and the data model carries enough structure to support both. This is a design reading of the work—not a claim about undisclosed implementation details.
This story focuses on my documented backend contributions. Diagrams simplify the workflow, forward-looking ideas are clearly labelled, and proprietary implementation details stay private.