ForecastAI 2026.01 - Architectural Audit Report¶
Date: 2026-04-19
Principle: Parameters are stored at scenario level; process outputs (forecast results, supply projections, etc.) are tagged with both pipeline_id AND scenario_id.
Executive Summary¶
| Domain | Status | Issues | Critical |
|---|---|---|---|
| Forecast Results | WARN | Missing pipeline_id in ForecastResult.to_dict() | YES |
| Backtest Metrics | FAIL | No scenario_id/pipeline_id in EvaluationMetrics | YES |
| Best Methods | FAIL | No scenario_id/pipeline_id in MethodSelector output | YES |
| Fitted Distributions | FAIL | No scenario_id/pipeline_id in FittedDistribution | YES |
| Supply Orders | PASS | Both columns present in INSERT statements | - |
| Supply Inventory | PASS | Both columns present in INSERT statements | - |
| Supply Exceptions | PASS | Both columns present in INSERT statements | - |
| Parameter Storage | PASS | Parameters keyed correctly by level | - |
| Frontend API Calls | PASS | Supply calls pass both pipeline_id and scenario_id | - |
Critical Issues Found (4)¶
1. Forecast Results Missing Pipeline Context (WARN)¶
Location: /files/forecasting/statistical_models.py lines 95-104
Problem: ForecastResult.to_dict() returns dictionary without scenario_id or pipeline_id: - Missing columns cause default values (scenario_id=1, pipeline_id=0) on insertion - Prevents pipeline-specific isolation of forecast results - Multiple concurrent pipelines overwrite each other's forecasts
Files Affected:
- /files/forecasting/statistical_models.py
- /files/forecasting/ml_models.py
- /files/forecasting/neural_models.py
- /files/forecasting/foundation_models.py
2. Backtest Metrics Missing Pipeline Context (FAIL)¶
Location: /files/evaluation/metrics.py lines 33-68, 926
Problem: EvaluationMetrics dataclass lacks scenario_id and pipeline_id fields. Results inserted with defaults.
Symptom: Cannot isolate backtest results by pipeline; rolling window metrics from different runs corrupt each other.
Fix Required: - Add fields to EvaluationMetrics dataclass - Pass context through backtest_series() call chain - Populate before bulk_insert
3. Best Methods Missing Pipeline Context (FAIL)¶
Location: /files/selection/best_method.py lines 228-379, 420
Problem: _rank_methods_for_series() output dict missing scenario_id and pipeline_id
Symptom: Best method selections unscoped from pipeline. Concurrent runs corrupt method locks and selections.
Fix Required: - Thread pipeline_id/scenario_id through select_best_methods() call chain - Add to output dict - Update helper methods (_default_result, _empty_result)
4. Fitted Distributions Missing Pipeline Context (FAIL)¶
Location: /files/distribution/fitting.py lines 34-63, 679
Problem: FittedDistribution dataclass lacks scenario_id and pipeline_id fields
Symptom: Cannot scope distribution fits to a pipeline. MEIO optimization cannot reference correct distribution set for a run.
Fix Required: - Add fields to dataclass - Pass context from orchestration layer - Populate before bulk_insert
Fully Compliant Domains¶
Supply Operations (PASS)¶
All supply output tables properly include both pipeline_id and scenario_id:
- supply_run: lines 2998-3007 in /files/supply_runner.py
- supply_order: lines 3152-3161
- supply_inventory_projection: lines 3278-3293
- supply_exception: lines 3314-3321
Frontend (PASS)¶
All API calls from frontend properly pass both parameters: - DetailSupplyTab.jsx lines 1393-1398 - supply_router.py properly filters by both columns
Parameter Storage (PASS)¶
Parameters correctly stored per scenario level: - forecast_parameter_set: keyed by pipeline_id (per-pipeline parameters) - supply_scenario: reusable across pipelines (scenario-level parameters)
MEIO Status (PARTIAL AUDIT)¶
MEIO subprocess at /files/api/main.py line 17073 receives:
- --scenario-ids
- --pipeline-id (optional)
- --account-id
Action Required: Audit Rust/ClickHouse schema to verify result tables include both columns.
Remediation Roadmap¶
Effort Estimate: 15-22 hours total
- Forecast Results (2 hours)
- Add scenario_id, pipeline_id to ForecastResult dataclass
-
Update 4 forecasting modules to populate fields
-
Backtest Metrics (2 hours)
- Add fields to EvaluationMetrics
-
Thread context through backtest_series()
-
Best Methods (2 hours)
- Thread context through select_best_methods()
-
Add to output dict and helper stubs
-
Fitted Distributions (2 hours)
- Add fields to FittedDistribution
-
Update fitting.py to populate
-
Pipeline Orchestration (3 hours)
-
Update run_pipeline.py to pass context to all modules
-
MEIO Audit (4-8 hours)
- Review Rust/ClickHouse schema
- Verify and fix if needed
Risk Level: CRITICAL¶
Process outputs are not isolated by pipeline. Multiple concurrent pipeline runs will: - Overwrite each other's backtest metrics - Corrupt best method selections - Mix distribution fits from different parameter sets
Immediate action required before multi-pipeline deployments.
Report Generated: 2026-04-19
Recommended Review: Upon completion of all fixes