In many AI projects, access to suitable data is one of the main barriers to experimentation, validation and deployment. Real datasets may be limited, sensitive, imbalanced, incomplete, or difficult to share across teams and environments. Synthetic data offers a way to address some of these limitations.
By generating artificial datasets that preserve relevant characteristics, patterns and constraints of the original problem, synthetic data can support model development, testing, validation and experimentation when real data is scarce, restricted or not sufficiently representative. Synthetic data is not intended to blindly replace real data. Its value depends on whether it is useful, representative and appropriate for the intended task.
This post introduces Synthetic Data Generation Engine, the second component presented from the AI-DAPT Data Services area following the Data Features Toolkit. The engine is designed to support the creation and assessment of synthetic datasets for AI and analytics operations.
The Synthetic Data Generation Engine
The Synthetic Data Generation Engine provides an environment to configure, generate and evaluate synthetic datasets. It supports users in creating synthetic data from scratch or based on existing datasets, depending on the needs of the use case and the availability of reference data.
The component supports different types of generation tasks, including tabular data, time-series data, and the creation of synthetic samples to help address dataset imbalance. This flexibility is important because it supports early experimentation, extends limited datasets, enables safer data sharing, and improves the representativeness of data used for model development.
Beyond data generation itself, the component also supports the assessment of generated outputs, to analyse if the generated data is sufficiently representative, coherent and suitable for the intended downstream task.
👉 Prefer to see it immediately in action? Jump directly to the demo video below.
Supporting Different Synthetic Data Generation Scenarios
Synthetic data generation can serve different purposes depending on the context of the AI task. In some cases, users may need to generate data from scratch to simulate a plausible dataset structure. In other cases, they may want to generate synthetic data from an existing dataset, preserving relevant patterns while creating additional artificial records. The Synthetic Data Generation Engine supports both scenarios, allowing users to define the generation mode according to the available data and the intended use of the output.
The project creation process therefore starts with two important decisions: the generation mode and the synthetic dataset type. The generation mode defines whether the dataset will be created from scratch or based on an existing asset. The synthetic dataset type defines the target scenario, such as tabular data, time-series data, or data generation for imbalance handling.
This first configuration step is important because it frames the generation process before any model is selected. A synthetic dataset created to address class imbalance, for example, has a different objective from a dataset generated to simulate time-series behaviour or to create a tabular dataset for experimentation. By making these options explicit, the engine helps users align the generation process with the problem they are trying to solve.
Once the project is created, users need to configure the main elements required to generate the synthetic dataset. These include the project instructions, the original dataset when generation is based on existing data, the columns and metadata, the generation model (see Figure 3), and the final information describing the synthetic dataset.
The configuration stage also allows users to define how the output dataset should be described. This includes metadata such as the name and description of the generated dataset, helping ensure that synthetic data is not treated as an anonymous file, but as a documented data asset within the platform.
Evaluating Synthetic Data Utility
As introduced before, generating synthetic data is only useful if users can understand whether the resulting dataset is appropriate for the intended task. For this reason, evaluation is a central part of the Synthetic Data Generation Engine. The component does not stop at producing artificial records. It also provides a diagnostic environment to compare the synthetic output with the original source dataset.
The evaluation workspace allows users to inspect the generated data from different perspectives. A first step is the data preview, where original and synthetic datasets can be viewed side by side, providing users the means to inspect whether the generated records appear coherent and whether the main structure of the data has been preserved.
Beyond visual inspection, the engine also provides dataset-level metrics, which help to assess the statistical similarity between original and synthetic data, as well as provide high-level indicators of synthetic data quality. This is useful because synthetic data may look plausible in individual rows while still failing to reproduce relevant distributions or relationships at dataset level.
The evaluation process also includes column-level metrics for both numerical and categorical variables that help users understand where the synthetic data performs well and where it may require further attention. For numerical columns, the engine can assess aspects such as similarity of summary statistics, distribution shape, boundary adherence and range coverage. For categorical columns, it can assess whether synthetic values respect the categories present in the real data and whether the generated dataset covers the expected category values.
This evaluation stage is essential for trustworthy use of synthetic data. It helps users avoid treating synthetic data as automatically valid simply because it has been generated. Instead, the engine supports a more evidence-based approach, where synthetic datasets are assessed before being reused in AI workflows, analytics tasks or model development. By combining dataset previews, overall quality indicators and column-level metrics, the Synthetic Data Generation Engine gives users a clearer understanding of the strengths and limitations of the generated dataset. This makes it easier to decide whether the output is suitable for downstream use, whether the configuration should be adjusted, or whether additional generation experiments are needed.
Watch It in Action 🎥
To bring the Synthetic Data Generation Engine component to life, we are releasing a new video that shows it in action. The video demonstrates how users can create a synthetic data generation project, select the generation mode and dataset type, configure the required metadata and generation settings, execute the process, and evaluate the resulting synthetic dataset.
Stay tuned for future videos of our compoments!