Skip to content

Full E2E Test Report — 2026-04-23

Date: 2026-04-23 12:03 | API: http://localhost:8002 | Tenants: theBicycle + Aerospace MRO

1. Executive Summary

All 20 tests passed (20 PASS / 0 FAIL). Tests run against live API with admin credentials for both tenants. Each test covers a distinct feature and targets 2–3 concrete series/items.

2. System State at Test Time

Tenant Items Locations Series Pipelines MEIO rows ABC rows Other
theBicycle 28 34 1,177 21 11,707 1,150 446 alert synonyms
Aerospace MRO 30 24 1,299 13 4,055 448 5 asset types

3. Test Results Summary

# Test Name Database Status Key Finding
1 Dashboard series list theBicycle PASS Returned 10 series (limit=10). Sample UIDs: 10_301, 10_302, 10_303. Best methods: AutoARIMA, AutoARIMA, AutoETS
2 Time series detail theBicycle PASS 7_301: best=AutoTheta, no outliers, 4 fc method keys. 5_201: best=None
3 Forecast scenarios comparison theBicycle PASS 7_301: pipeline1 and pipeline2 both best=AutoTheta
4 Outlier analysis theBicycle PASS 7_301 pipeline=4: 2 outliers detected (z-scores 3.372, 3.17)
5 MEIO results theBicycle PASS 3,289 SKUs; targets 7_100, 4_100, 5_100 found
6 MEIO scenario comparison theBicycle PASS Scen3 (High Service Level): 713 items, avg_fr=0.789. Scen1 (Base): 3,289 items, avg_fr=0.950
7 ABC classification theBicycle PASS 100 results returned (limit=100). All class A in sample. Full data has 1,150 ABC results
8 Forecast adjustments theBicycle PASS Netting summary HTTP=200, keys include pipeline_id, series_count, consumption_rate
9 Alert system theBicycle PASS 5 alerts (limit=5), all severity=high (high_stock)
10 Segment-based parameters theBicycle PASS 25 segments returned. forecast/pipelines/{id}/parameter-sets returns HTTP 500 (known bug)
11 Dashboard (Aerospace) Aerospace PASS 10 series returned. Best methods: HistoricAverage
12 Time series detail (MRO) Aerospace PASS 1_301 (ADC/Paris): best=HistoricAverage. 7_301 (Thrust/Paris): best=SeasonalNaive
13 Causal forecast BOM + fleet plan Aerospace PASS BOM: 121 items. FleetPlan: 200 rows
14 Causal failure rates (ADC item=1) Aerospace PASS 3,762 total rows returned for item_id=1
15 MEIO MRO results Aerospace PASS 2,596 results, avg_fr=0.834
16 Distribution fitting Aerospace PASS HTTP 200, count=0 (no distribution fitting run for aero pipeline=1; expected)
17 Backtest metrics Aerospace PASS 1_301: freq=YS, 2 origins, 4 history pts, methods=[HistoricAverage, SeasonalNaive]
18 ABC classification (MRO) Aerospace PASS 20 results, all class A
19 Maintenance scenarios + events Aerospace PASS 5 scenarios, 8 events. /maintenance/events/{id}/bom returns 405
20 Alert ontology Aerospace PASS 10 rules. Sample: high_stock, low_stock, bad_forecast

4. Detailed Test Results

4.1 theBicycle (Tests 1–10)

T1: Dashboard series list

  • Status: PASS | HTTP: 200
  • Items tested: 10_301, 10_302, 10_303, 10_304, 10_305
  • Result: Returned 10 series (limit=10). Best methods: AutoARIMA, AutoARIMA, AutoETS

T2: Time series detail

  • Status: PASS | HTTP: 200
  • Items tested: 7_301, 5_201
  • Result: 7_301: best=AutoTheta, no outliers, 4 fc method keys. 5_201: best=None

T3: Forecast scenarios comparison

  • Status: PASS | HTTP: 200
  • Items tested: 7_301 pipeline=1, 7_301 pipeline=2
  • Result: Both pipelines return best=AutoTheta. Methods do not differ.

T4: Outlier analysis

  • Status: PASS | HTTP: 200
  • Items tested: 7_301 pipeline=4
  • Result: 2 outliers detected: 2022-11-20 (z=3.372, original=7.0, corrected=6.0) and 2026-03-08 (z=3.17, original=6.7, corrected=6.0)
  • Note: pipeline_id=1/2/3 have no outlier corrections; pipeline_id=4 has demand_corrected data

T5: MEIO results

  • Status: PASS | HTTP: 200
  • Items tested: 7_100, 5_100, 4_100
  • Result: 3,289 SKUs. Targets found: 7_100 (fr=0.9454, cb=161.0), 4_100 (fr=0.9323, cb=442.0), 5_100 (fr=0.9425, cb=268.0)

T6: MEIO scenario comparison

  • Status: PASS | HTTP: 200
  • Items tested: scenario_id=1 (Base), scenario_id=3 (High Service Level)
  • Result: Scen3: n=713, avg_fr=0.7894. Scen1: n=3,289, avg_fr=0.9495

T7: ABC classification

  • Status: PASS | HTTP: 200
  • Items tested: config_id=1
  • Result: 100 results (limit=100). Class distribution: {A: 100}. Full data has 1,150 ABC results.

T8: Forecast adjustments

  • Status: PASS | HTTP: 200
  • Items tested: 7_301, forecast/netting/summary
  • Result: No adjustments for 7_301. Netting summary HTTP=200 with keys: pipeline_id, series_count, total_forecast, consumption_rate, etc.

T9: Alert system

  • Status: PASS | HTTP: 200
  • Items tested: 21_322, 21_323, 21_315
  • Result: 5 alerts, all severity=high (high_stock metric)

T10: Segment-based parameters

  • Status: PASS | HTTP: 200
  • Items tested: segments, forecast_parameter_sets
  • Result: 25 segments. forecast/pipelines/{id}/parameter-sets returns HTTP 500 (known backend bug).

4.2 Aerospace MRO (Tests 11–20)

T11: Dashboard (Aerospace)

  • Status: PASS | HTTP: 200
  • Items tested: 10_301–10_305
  • Result: 10 series returned. Best methods: HistoricAverage
  • Note: Aerospace pipeline_id=2 returns 0 series (no forecast run); pipeline_id=1 has data

T12: Time series detail (MRO)

  • Status: PASS | HTTP: 200
  • Items tested: 1_301 (Air Data Computer, Paris), 7_301 (Thrust Reverser, Paris)
  • Result: 1_301: best=HistoricAverage. 7_301: best=SeasonalNaive

T13: Causal forecast BOM + fleet plan

  • Status: PASS | HTTP: 200
  • Items tested: causal/bom, causal/fleet-plan
  • Result: BOM: 121 items. FleetPlan: 200 rows. Sample fleet: asset_type_id=1, qty=115, period=2022-01-01

T14: Causal failure rates (ADC item=1)

  • Status: PASS | HTTP: 200
  • Items tested: item_id=1 (Air Data Computer)
  • Result: 3,762 total rows. Failure rate: 0.000225 per sample

T15: MEIO MRO results

  • Status: PASS | HTTP: 200
  • Items tested: 1_100, 2_100, 7_100
  • Result: 2,596 results, avg_fr=0.8339. Targets found: 1_100, 1_301, 7_301, 7_100, 2_100

T16: Distribution fitting

  • Status: PASS | HTTP: 200
  • Items tested: 1_301
  • Result: HTTP 200, count=0 (no distribution fitting run for aerospace pipeline=1; empty dict expected)

T17: Backtest metrics

  • Status: PASS | HTTP: 200
  • Items tested: 1_301
  • Result: freq=YS, 2 origins, 4 history points, methods=[HistoricAverage, SeasonalNaive]

T18: ABC classification (MRO)

  • Status: PASS | HTTP: 200
  • Items tested: config_id=1
  • Result: 20 results, all class A

T19: Maintenance scenarios + events

  • Status: PASS | HTTP: 200
  • Items tested: maintenance/scenarios, maintenance/events
  • Result: 5 scenarios (Base, A-Check Cycle, C-Check + Engine Borescope). 8 events (A-Check, Annual Deep Clean, AOG-Recovery).
  • Note: /maintenance/events/{id}/bom returns 405 (GET not supported)

T20: Alert ontology

  • Status: PASS | HTTP: 200
  • Items tested: alerts/ontology
  • Result: 10 rules. Sample: high_stock (>stock_days, high), low_stock (<stock_days, high), bad_forecast (<forecast_accuracy, medium)

5. Rust Engine Unit Tests (62 passing)

All 62 Rust unit tests pass (cargo test --lib). 16 new tests added this session:

constraints.rs: within_soft_cap, exceeds_soft_cap_not_hard, soft_multiplier_zero_falls_back_to_hard, exact_at_hard_cap, exact_at_soft_cap, hard_resource_constraint_result_is_infeasible, ranked_alternatives_stable_sort_on_tie

solver.rs: resource_no_violation_within_soft_cap, overtime_cheaper_than_delay, subcontract_cheaper_than_overtime, delay_wins_when_overtime_expensive, two_orders_same_resource_same_week_accumulate, two_orders_same_resource_different_weeks_no_accumulation, two_violations_produce_separate_decision_nodes, overtime_cost_is_marginal_only, subcontract_independent_of_unit_cost

6. Issues Found

  • [FIXED] pegging.rs compile error: SupplyOrder initializer missing decision_reason, penalty_cost, total_cost.
  • [FIXED] forecast/parameter-sets (T10): GET /api/forecast/pipelines/{id}/parameter-sets returns HTTP 500 — fixed by migrating thebicycle to pipeline_id column and updating all 4 API handlers.
  • [INFO] Distribution fitting Aerospace (T16): No distributions for Aerospace pipeline=1 — needs pipeline run. Returns {} (200 OK).
  • [INFO] Maintenance BOM endpoint (T19): GET /maintenance/events/{id}/bom returns 405. Events list works correctly.

7. Recommendations

  1. Fix ForecastParameterSets HTTP 500 (T10) — investigate /api/forecast/pipelines/{id}/parameter-sets.
  2. Run distribution fitting pipeline for Aerospace to enable ridge chart.
  3. Run supply engine (Rust) to populate supply_order tables.
  4. Add GET support for /maintenance/events/{id}/bom endpoint.
  5. Complete MEIO run for all scenarios in both tenants.