External Feature Covariates — Technical Test Guide

ForecastAI 2026.01 · May 2026

1. Overview

The External Feature (Covariate) system allows ForecastAI to incorporate time-series signals beyond historical demand when generating forecasts. These signals — called covariates — can be future-available (known in advance, e.g. planned holidays, weather forecasts) or history-only (observed in the past, e.g. customer backlog).

Future-available covariates are injected directly into NeuralForecast models as futr_exog inputs. History-only covariates are used as lagged features in the ML feature matrix.

All covariate data is stored in ClickHouse, partitioned by (pipeline_id, scenario_id), so each forecast run uses its own isolated dataset. A reliability gate prevents low-quality or uncorrelated features from entering the models.

2. Inputs

2.1 PostgreSQL Feature Definitions

Two PostgreSQL tables in the scenario schema define available features and their activation configuration per pipeline.

Table	Key Columns
scenario.feature_definitions	id, name, type (global / item-specific), future_available (bool), source_table, source_column, description
scenario.feature_parameter_sets	id, pipeline_id, name, params (JSONB with activated_features and min_reliability)

Example params JSONB for feature_parameter_sets:

{
  "activated_features": [
    {"feature_id": 1, "enabled": true, "weight": 1.0},
    {"feature_id": 2, "enabled": true, "weight": 0.8}
  ],
  "min_reliability": 0.4
}

2.2 ClickHouse Feature Values Table

Property	Value
Table Name	PIPE_feature_values
Engine	ReplacingMergeTree(loaded_at)
Partition Key	(pipeline_id, scenario_id)

Column	Type	Description
pipeline_id	Int64	Pipeline identifier
scenario_id	Int64	Scenario identifier
feature_id	Int64	References feature_definitions.id
feature_name	String	Denormalized name for fast queries
feature_type	LowCardinality(String)	global or item-specific
future_available	UInt8	1 = futr_exog, 0 = hist_exog
date	Date	ISO week start date (Monday)
unique_id	String	Series identifier (item_id_site_id)
value	Float64	Covariate signal value
loaded_at	DateTime	Upsert timestamp for ReplacingMergeTree

2.3 Demo Features (Seeded via API)

Feature Name	Type	future_available	Behaviour
Holiday Index	global	true	1.0 normal weeks; 2.5 on bank holiday weeks (25 Dec, 1 Jan, Easter Friday)
Weather Index	global	true	Sinusoidal seasonal: 0.7 in winter, 1.3 in summer (northern hemisphere)
Customer Backlog	item-specific	false	Random walk bounded [0.5, 2.0] per unique_id — history-only lag feature

3. Data Flow and Computation

3.1 FeatureLoader (files/forecasting/feature_loader.py)

On first call per Dask worker batch (lazy init), loads all rows for (pipeline_id, scenario_id) from ClickHouse into a pandas DataFrame in memory.

get_series_features(unique_id, hist_dates, future_dates) returns:

• hist_df — history-only signals aligned to hist_dates
• futr_df — future-available signals aligned to future_dates
• futr_exog_list — column names for NeuralForecast constructors
• hist_exog_list — column names for ML feature matrix

Missing values: forward-fill within range; flat extrapolation beyond loaded window.

3.2 NeuralForecast Integration (files/forecasting/neural_models.py)

After the calendar feature block, if a FeatureLoader is attached:

1. Calls get_series_features() to retrieve futr_df and futr_exog_list
2. Merges futr_exog columns into the series DataFrame passed to NeuralForecast
3. Passes futr_exog=futr_exog_list to each model constructor

Models benefiting from future-available covariates: NHITS, NBEATS, PatchTST, TFT, DeepAR.

3.3 ML Model Integration (files/forecasting/ml_models.py)

In _forecast_method():

1. Calls get_series_features() to retrieve hist_df and hist_exog_list
2. Merges hist_exog columns into the lag feature matrix
3. For each forecast horizon step h, substitutes futr_exog values from futr_df at the target date

3.4 Reliability Gating (feature-reliability pipeline step)

For each (feature, unique_id) pair, a composite reliability score in [0, 1] is computed:

Component	Weight	Formula
Pearson correlation	50%	Correlation between feature values and demand_corrected values
Naive MAPE improvement	30%	MAPE improvement vs. no feature
Bias contribution inverted	20%	Lower bias = higher score

Pairs with a composite score below the configured min_reliability threshold are excluded for that series. The gating result is persisted in scenario.feature_reliability_scores.

Score Range	Signal Quality	Action
≥ 0.7	Strong signal	Feature injected for this series
0.4 – 0.7	Moderate signal	Injected if min_reliability ≤ 0.4 (default)
< 0.4	Weak / uncorrelated	Excluded for this series

4. Database Tables

Table Name	Location	Purpose	Key Columns
feature_definitions	PostgreSQL scenario	Defines available external signals	id, name, type, future_available
feature_parameter_sets	PostgreSQL scenario	Per-pipeline activation config	id, pipeline_id, name, params (JSONB)
feature_reliability_scores	PostgreSQL scenario	Per-(feature, series) reliability result	feature_id, unique_id, pipeline_id, score, gated_in
PIPE_feature_values	ClickHouse	Actual covariate values per run	pipeline_id, scenario_id, feature_id, date, value
demand_corrected	PostgreSQL scenario	Outlier-corrected demand for correlation	item_id, site_id, date, corrected_value, pipeline_id
forecast_results	PostgreSQL scenario	Forecast outputs (covariate-enhanced)	unique_id, method, point_forecast, pipeline_id

5. User Configuration (Step-by-Step)

Step 1 — Select Your Pipeline

In the left toolbar, choose the pipeline (e.g. "Bicycle — Base Pipeline"). All configuration is scoped to this pipeline.

Step 2 — Open the Feature Params Tab

Navigate to Processes (left menu) and click the Feature Params tab (puzzle-piece icon). Three panels appear: Feature Coverage, Feature Activation, Data Management.

Step 3 — Seed Demo Data

In the Data Management panel, click "Seed Demo Data". This calls POST /api/features/seed-demo and loads the three demo features into ClickHouse.

Step 4 — Configure Activation

Toggle each feature on or off, adjust weights (0.0 to 1.0), and set the Min Reliability threshold (default 0.4). Changes are saved automatically when you click outside the field.

Step 5 — Run the Pipeline

In the Run Pipeline panel, ensure both feature-reliability and forecast steps are selected. Click Run. The feature-reliability step must precede the forecast step to gate low-quality signals.

6. Observing Results

6.1 Feature Coverage Table

After seeding, the Feature Coverage panel shows one row per feature with: Feature Name, Type, Future Available, Row Count, Date Range, Unique Series Count. A green tick indicates sufficient data coverage; a warning icon appears when fewer than 52 rows are present for a feature.

6.2 Reliability Score Interpretation

See Section 3.4 above for the score ranges and what they mean.

6.3 Forecast Quality Comparison

Compare backtest metrics (MAE, MAPE, SMAPE) between a run without features and a run with features. Navigate to a series in the Dashboard, open the Series Viewer, and compare the backtest metric columns across methods.

7. Testable SKUs

The following SKUs from the Bicycle tenant are recommended for testing covariate integration:

unique_id	Item Name	Site	Why Test	Expected Benefit
10_301	Mountain Bike	Paris	High volume, strong seasonality. Holiday and weather signals correlate with spring/summer peaks.	NHITS MAPE reduction 15–25%
18_305	Wheelset	Bordeaux	Assembly component — tracks finished goods. Customer backlog adds lag predictive power.	ML feature importance increase
5_320	Road Bike	Lyon	Strong summer seasonality, intermittent winters. Weather Index alignment expected.	Croston less competitive vs. NHITS
12_310	E-Bike	Marseille	Growing trend. Holiday Index spikes (Black Friday, Christmas) drive clear signal.	MASE improvement on TFT
3_335	Frame Set	Verona	Low ADI, sporadic demand. Features expected below reliability threshold — tests gating.	Reliability score < 0.4; features excluded

8. API Reference

Method	Endpoint	Description
GET	/api/features/pipeline/{pipeline_id}/scenario/{scenario_id}/summary	Feature coverage summary from ClickHouse
GET	/api/features/pipeline/{pipeline_id}/scenario/{scenario_id}/list	Feature definitions with activation status
POST	/api/features/pipeline/{pipeline_id}/scenario/{scenario_id}/upload	Upload CSV (columns: feature_name, date, unique_id, value)
POST	/api/features/pipeline/{pipeline_id}/scenario/{scenario_id}/push	JSON array push to ClickHouse
DELETE	/api/features/pipeline/{pipeline_id}/scenario/{scenario_id}	Drop ClickHouse partition for this run
POST	/api/features/seed-demo	Seed demo features (query params: pipeline_id, scenario_id)
GET	/api/features/scenarios/{pipeline_id}/parameter-sets	Get activation parameter sets for pipeline
POST	/api/features/scenarios/{pipeline_id}/parameter-sets	Create activation parameter set

9. Troubleshooting

Issue	Resolution
Coverage table shows 0 rows after seeding	Check ClickHouse connection in config.yaml. Verify ensure_ch_feature_tables() ran on startup. Re-run seed-demo with correct pipeline_id/scenario_id.
All reliability scores below threshold	demand_corrected may be empty. Run the forecast step first (which embeds outlier detection) to populate demand_corrected before re-running feature-reliability.
NeuralForecast models ignore features	Check that pipeline_id > 0 is passed to NeuralForecaster. FeatureLoader is only instantiated when pipeline_id > 0. Verify futr_exog_list is non-empty after get_series_features().
ML models show no improvement	hist_exog needs at least 52 weeks of history. Verify Customer Backlog covers the full demand_actuals date range for the series under test.
Login fails with 'database unavailable'	HTTP 503 is expected when the PostgreSQL pool is exhausted. Wait 30 seconds or restart the API. This is not related to the feature system.