Training for the long tail: synthetic data for rare events

The events a model most needs to get right are often the ones it sees least: fraudulent transactions, equipment failures, rare diagnoses, unusual road conditions. Synthetic data can help close that gap, but only if it is generated and evaluated with the rare cases in mind. This article compares the main approaches and covers how to use them without fooling yourself.

Why rare events are hard

A model trained on heavily imbalanced data learns that predicting the majority class is almost always right. Overall accuracy looks excellent while recall on the class you care about is poor. The underlying problems are:

Too few examples to learn the shape of the minority class, especially when it has several distinct sub-types.
Skewed loss, where the majority class dominates the gradient.
Unrepresentative history, where the rare events you have seen are not the ones you will see next.
Misleading metrics, where accuracy and even ROC AUC hide weak performance on the minority class.

Synthetic data addresses the first problem directly and can help with the third. It does not fix the second or fourth on its own, so it should sit alongside sensible loss weighting and evaluation, not replace them.

Get the evaluation right first

Before changing the training data, make sure you can measure improvement on rare events. Use metrics that focus on the minority class: precision, recall, and F1 per class, precision-recall curves and average precision, and recall at a fixed false-positive rate that reflects operational cost. Keep a real holdout set with as many genuine rare events as you can spare, and never put synthetic examples into it. If the holdout contains only a handful of positives, report confidence intervals, because small changes will be noise.

The options, from simplest to most flexible

Reweighting and resampling

Class weights in the loss function and random oversampling of minority examples are the baseline. They are cheap and often effective. Their limit is that they add no new variation: oversampling repeats the same points, which can lead the model to overfit to those specific examples.

SMOTE and its variants

SMOTE creates new minority examples by interpolating between a minority point and one of its nearest minority neighbours. Variants such as Borderline-SMOTE and ADASYN focus generation near the decision boundary, and SMOTE-NC handles mixed numeric and categorical features. Interpolation adds some variation and works well in low-dimensional, mostly continuous feature spaces. It has known weaknesses:

Interpolated points can fall in regions that belong to the majority class, adding noise to the boundary.
It relies on distance metrics that become less meaningful in high dimensions.
It cannot extrapolate beyond the convex hull of existing minority examples.
It handles categorical data, constraints, and sequences poorly.

Conditional generative models

A generative model trained on the full dataset, conditioned on the class label or on specific attributes, can produce minority examples that reflect the joint structure of the data rather than straight-line interpolations. Because the model learns shared structure from the majority class, it can produce realistic minority records even when the minority sample is modest. Conditioning also lets you target specific sub-types, such as a particular fraud pattern or a failure mode on a specific machine type, rather than the minority class as a whole.

The cost is complexity. Generative models need more tuning, more validation, and careful checks that the generated minority examples are actually distinct from the majority and from each other.

Simulation and scenario generation

When the rare events you care about have not happened yet, or have happened too rarely to learn from, rule-based simulation and scenario design are often the only option. A simulator of a payment network can inject fraud patterns described by investigators. A rendered driving scene can place a pedestrian in conditions absent from collected footage. This is most valuable for test coverage and for teaching a model about conditions that domain experts can describe but data cannot yet show.

A generator can make rare events more common in your training set. It cannot make them more informative than the examples and domain knowledge it was built from.

How much to generate

Balancing to a 50/50 split is rarely the right target. It distorts the base rate the model learns, which hurts calibration and can increase false positives in production. Treat the amount of synthetic minority data as a hyperparameter:

Train a baseline on real data with class weighting.
Add synthetic minority examples at several ratios.
Evaluate each on the real holdout using minority-focused metrics.
Choose the smallest amount that captures most of the gain.
Recalibrate predicted probabilities on real data if the model will be used for scoring or thresholding.

Checks specific to rare-class generation

General fidelity metrics are dominated by the majority class, so a dataset can score well overall while the minority class is poorly modelled. Evaluate the minority class on its own:

Per-class fidelity: compare distributions and correlations within the minority class, not only across the whole dataset.
Sub-type coverage: if the rare class has known sub-types, confirm each is represented in roughly the intended proportion and none has collapsed.
Class separation: check that generated minority examples do not drift into regions occupied by the majority class, which would blur the boundary you are trying to sharpen.
Memorization: with few real minority examples, a generator is more likely to reproduce them closely. Measure distance to the closest training record for the minority class specifically.
Domain review: ask someone who handles these cases, such as a fraud analyst or reliability engineer, to review a sample. Implausible examples are often obvious to an expert and invisible to a metric.

When synthetic data is the wrong fix

Some rare-event problems are not data-volume problems. If the features do not carry the signal needed to separate the classes, more minority examples will not help; better features will. If labels for the rare class are noisy or inconsistent, a generator will reproduce that noise at scale. If the rare events are changing quickly, as with adversarial fraud, a generator trained on last quarter's patterns may teach the model to recognize attacks that have already moved on. In each case, fix the underlying issue first, then consider whether synthetic data still adds value.

Putting it together

A sound approach to rare events combines several pieces: minority-focused evaluation on real holdout data, sensible loss weighting, targeted synthetic generation for under-represented sub-types, and per-class checks on the generated data. Synthetic data is one part of that system, and it is most effective when its contribution is measured rather than assumed.

SETC's generation and rebalancing controls are built for this kind of targeted work; the product page covers how conditioning and per-class evaluation fit together. If you have a rare-event problem you would like to test, book a demo and we'll walk through it on your data.