At AI-DAPT, we are building a comprehensive platform to support the design, execution, and evolution of trustworthy AI pipelines. While previous posts have introduced the overall architecture and user journeys of the platform (https://www.ai-dapt.eu/ai-dapt-platform-architecting-trustworthy-ai-with-seamless-user-journeys/), we are now launching a new series of blog posts dedicated to exploring some of its core components.
This new series aims to open the “black box” of the platform and give you a closer look at the building blocks that make AI-DAPT work in practice. From data foundations to adaptive AI services, each post will highlight how specific components contribute to making AI pipelines more reliable, transparent, and easier to operate in real-world environments.
The Pipeline Manager (DAVE)
We begin this journey with one of the most central components of the platform: the Pipeline Manager, also known as DAVE (Data Analytics and Visualization Environment).
👉 Prefer to see it immediately in action? Jump directly to the demo video below.
DAVE is designed to simplify the way AI pipelines are created and managed. Instead of relying solely on complex code-based workflows, it offers a visual environment where users can design, configure, and execute data and AI pipelines in an intuitive and interactive way. By combining usability with flexibility, DAVE enables both technical and domain experts to actively participate in the development of AI solutions.
At its core, DAVE is the pipeline “control centre” of the AI-DAPT platform. It allows users to:
- Connect different data sources and processing steps
- Orchestrate complex workflows across the AI lifecycle
- Experiment with different configurations and models
- Monitor and refine pipelines as they evolve over time
This is particularly important in environments where AI is not static. Pipelines must adapt to new data, changing conditions, and evolving requirements. DAVE provides the foundation to manage this complexity in a structured and transparent way. Under the hood, DAVE leverages established tools such as Apache Airflow for workflow orchestration and MLflow for model lifecycle management.
A Visual and Collaborative Approach
DAVE is not only about managing pipelines, but also about making them accessible and collaborative across different roles and expertise levels. Through its visual interface, users can clearly understand how data flows, how models are applied, and where key decisions take place within the pipeline. This reduces the dependency on purely code-based interactions and enables a wider group of stakeholders to engage with AI development. This approach helps bridge the gap between:
- Data scientists and domain experts
- Technical development and operational use
- Automated processes and human oversight
Pipelines are represented as modular workflows composed of interconnected operators, making their structure explicit and easier to reason about. This shared representation allows teams to align on data transformations, modelling strategies, and validation steps, fostering collaboration throughout the lifecycle. As a result, DAVE supports a more inclusive approach to AI development, where technical and domain expertise can be combined more effectively.
Enabling Human-in-the-Loop AI
A key feature of DAVE is its support for human-in-the-loop operation, which is essential for building trustworthy and adaptable AI systems. Rather than fully automating every step, DAVE allows users to remain actively involved in the pipeline. This is achieved through the concept of operators, which can be either:
- Internal operators, representing automated processing steps within the pipeline
- External operators, where human input, validation, or decision-making is required
This approach makes it possible to combine automation with expert knowledge at critical points of the workflow. For example, domain experts can validate data, adjust model parameters, or approve results before moving to the next stage. By embedding human intervention directly into the pipeline, DAVE ensures that AI systems remain transparent, controllable, and aligned with real-world requirements.
Making Pipelines Transparent
A key capability of DAVE is the ability to visualise samples of the data as they flow through the pipeline. After each operator execution, users can inspect intermediate results, understand how data is being transformed, and quickly identify anomalies or unexpected behaviours.
This step-by-step visibility turns pipelines into transparent processes rather than black boxes. It enables users to trace how inputs evolve into outputs, making it easier to debug workflows, validate transformations, and and assess the impact of each processing step. Because this inspection is integrated within DAVE’s execution environment, users can interact with pipelines as they are being designed and tested, observing how data evolves after each operator and refining workflows in an iterative manner. Combined with human-in-the-loop capabilities, this approach enables pipelines that are not only functional, but also transparent, auditable, and continuously improvable.
In parallel, the AI-DAPT platform includes a dedicated Pipeline Execution Engine, responsible for more advanced runtime scenarios, including resource-aware execution, scheduling across computing nodes, and continuous monitoring of distributed pipelines. This separation ensures that interactive development and large-scale execution are handled by complementary components.
Watch It in Action 🎥
To bring the Pipeline Manager component to life, we are releasing a new video that showcases DAVE in action. The component is currently being integrated as a core element of the upcoming AI-DAPT platform alpha release, providing a first glimpse into how pipelines can be designed, explored, and refined in practice. Note: The UI might change in future AI-DAPT platform relases.
Stay tuned for future videos of our compoments!