MEIO V3 — Empirical Distributions, Monte-Carlo Digital Twin & RL Policy Layer¶

Chapter summary

This chapter documents the V3 enhancement of the MEIO engine: a strict superset of V2 that replaces parametric-only demand distributions with empirical demand-over-lead-time modeling, adds a Monte-Carlo simulation engine (the digital twin), and introduces a Reinforcement Learning policy layer that optimizes service targets, risk multipliers, and sourcing preferences — never order quantities.

V2 remains frozen for demos; V3 is opt-in per pipeline via meio_optimizer_version = 'v3'.

For the V1/V2 algorithm (greedy marginal value, golden formula, fitted distributions), see MEIO Theory. For the distribution families (Normal, Gamma, LogNormal, Weibull, NegativeBinomial, Poisson), see Distribution Fitting.

1. Why V3? — The principles¶

Principle 1 — Inventory protects against more than demand variability¶

Traditional safety stock formulas assume inventory protects against demand variability alone:

\[ SS = z_\alpha \cdot \sigma_{\text{demand}} \cdot \sqrt{L} \]

In reality, inventory protects against a compound uncertainty:

Source of uncertainty	V2 model	V3 model
Demand variability	\(\sigma_d\) from `dmd_stddev`	Empirical demand-over-LT samples
Forecast error	Not modeled	Empirical forecast-error distribution
Forecast bias	Not modeled	Mean of forecast-error distribution
Lead-time variability	Scalar `lt_stddev`	Empirical lead-time distribution
Supplier unreliability	Not modeled	Supplier reliability = \(P(\text{actual} \leq \text{promised})\)
Execution delays	Not modeled	Actual-vs-promised lead-time delta
Capacity shortages	Not modeled	(Phase 12 risk indicator)
Allocation decisions	Not modeled	(Phase 12 risk indicator)

The primary uncertainty V3 models is therefore demand occurring during the actual lead time — not simple daily demand distributions.

Principle 2 — Do not assume normality¶

V3 uses observed reality instead of theoretical distributions whenever possible:

Empirical distributions — sorted sample arrays with CDF/quantile computed via linear interpolation
Bootstrap resampling — resample observed realizations with replacement
Historical samples — actual demand-during-actual-LT observations
Quantile estimation — direct from data, no parametric assumption
Kernel density estimation (optional, Phase 1+)

When empirical samples are too few (n < empirical_shrinkage_n_min, default 30), V3 shrinks toward the parametric fitted distribution (E3 bias correction) rather than reverting entirely.

Principle 3 — RL optimizes policies, not quantities¶

The Reinforcement Learning layer does not decide:

Order quantities
Safety stock levels
Inventory targets

MEIO remains responsible for these. RL decides policy parameters that MEIO consumes:

Target fill rates
Risk multipliers
Inventory budgets
Expedite thresholds
Sourcing preferences

2. Architecture — the V3 stack¶

graph TD
    HIST[HIST_backtest_forecast<br/>HIST_forecast_point_values]
    DA[master_demand_actuals<br/>with received_date]
    PR[master_purchase_receipt<br/>NEW: supplier receipt history]
    RT[master_route / master_item<br/>master_item_location]

    HIST --> P1
    DA --> P1
    PR --> P3
    RT --> P3

    P1[Phase 1: Demand-over-LT<br/>empirical distribution]
    P2[Phase 2: Forecast-error<br/>distribution]
    P3[Phase 3: Lead-time<br/>distribution]

    P1 --> MC
    P2 --> MC
    P3 --> MC

    MC[Phase 4: Monte-Carlo<br/>simulation engine<br/>RUST DIGITAL TWIN]
    MC --> MEIO[V3 MEIO optimizer<br/>greedy marginal value<br/>with empirical dists]

    MEIO --> IT[Inventory targets<br/>PIPE_meio_results]
    IT --> SPO[Supply planning<br/>optimizer]

    SPO --> RL[Phase 15: RL policy layer<br/>PPO / SAC]
    MC --> RL
    RL -->|target_fill_rate<br/>risk_multiplier<br/>inventory_budget<br/>expedite_threshold<br/>supplier_preferences| MEIO

    style MC fill:#fff3e0,stroke:#e65100,stroke-width:3px
    style RL fill:#e8f5e9,stroke:#2e7d32,stroke-width:3px
    style MEIO fill:#e3f2fd,stroke:#1565c0,stroke-width:3px

V2↔V3 coexistence¶

V3 is a separate Rust crate (files/MEIO_v3/) that builds a separate PyO3 extension meio_optimizer_v3. It coexists with V1 (meio_optimizer) and V2 (meio_optimizer_v2) exactly as V1 and V2 coexist today:

Setting	Behavior
`meio_optimizer_version = 'v3'` (default)	V3 runs. V2 is the explicit fallback only.
`meio_optimizer_version = 'v2'` + V3 unavailable	V2 runs (explicit fallback).
`meio_optimizer_version = 'v3'` + V3 NOT available	Falls back to V2 with a warning.

When use_empirical_distributions = false (the default), V3 produces byte-identical results to V2 — this is the Phase-0 A/B gate.

3. Phase 1 — Demand-over-lead-time empirical modeling¶

What it replaces¶

V2 uses a parametric demand distribution (Normal, Gamma, LogNormal, Weibull, or NegativeBinomial) fitted from forecast quantiles. The fill-rate integral is computed analytically:

\[ FR(b) = 1 - \frac{1}{E[D_{LT}]} \int_b^{\infty} (x - b) \, f(x) \, dx \]

where \(f(x)\) is the fitted parametric density.

What V3 does instead¶

Figure: empirical-distribution sampling — historical demand-during-actual-LT observations are sorted into a sample array; the MC engine bootstraps (draws with replacement) and the E3 shrinkage blends with the parametric fit when n < 30.

flowchart LR
  DA["master_demand_actuals<br/>(date, qty, received_date)"] --> OBS["Observations<br/>demand during actual LT"]
  OBS --> SORT["Sorted sample array<br/>EmpiricalDistribution"]
  SORT --> NCHECK{"n_observations<br/>at least 30?"}
  NCHECK -->|"yes"| EMP["Use empirical quantiles<br/>cdf / inv_cdf / bootstrap"]
  NCHECK -->|"no"| BLEND["E3 shrinkage<br/>w = n / n_min"]
  PARA["Parametric fitted dist<br/>Gamma / LogNormal / etc."] --> BLEND
  BLEND --> QBLEND["Blended quantiles<br/>w·q_emp + (1-w)·q_param"]
  EMP --> MC["BootstrapMC<br/>compound_sample"]
  QBLEND --> MC

For every SKU-location combination, V3 collects historical realizations of demand occurring during the actual lead time:

Observation	Source
Order date	`master_demand_actuals.date`
Actual lead time	`master_demand_actuals.received_date - date`
Demand during actual LT	`SUM(qty)` over `[date, date + actual_lt_days)` for the same (item, site)

Example realizations:

Order date	Actual LT (days)	Demand during LT
Jan 1	12	520
Jan 15	15	690
Feb 1	10	410
Feb 20	14	580
Mar 5	11	445

These become the sorted sample array of an EmpiricalDistribution.

The EmpiricalDistribution struct (Rust)¶

pub struct EmpiricalDistribution {
    pub samples: Vec<f64>,      // sorted ascending
    pub n_observations: usize,  // original observation count
}

Key methods:

Method	Complexity	Description
`cdf(x)`	\(O(\log n)\)	Binary search: fraction of samples \(\leq x\)
`inv_cdf(p)`	\(O(1)\)	Linear interpolation between order statistics
`sample(rng)`	\(O(1)\)	Uniform random draw from observed samples
`bootstrap(n, rng)`	\(O(n)\)	Draw `n` values with replacement
`mean()`	\(O(n)\)	Sample mean
`variance()`	\(O(n)\)	Population variance (ddof=0)
`stddev()`	\(O(n)\)	\(\sqrt{\text{variance}}\)
`skewness()`	\(O(n)\)	Fisher-Pearson skewness
`kurtosis()`	\(O(n)\)	Excess kurtosis (normal → 0)
`quantiles()`	\(O(1)\)	Tuple (p10, p25, p50, p75, p90, p95, p99)
`shrink_toward(param_q, n_min)`	\(O(1)\)	E3 bias correction

E3 — Bias-corrected shrinkage¶

When n_observations < empirical_shrinkage_n_min (default 30), empirical quantiles are blended with the parametric fitted distribution's quantiles:

\[ q_{\text{blended}} = w \cdot q_{\text{empirical}} + (1 - w) \cdot q_{\text{parametric}} \]

where \(w = n / n_{\min}\). This prevents noisy quantile tails from inflating safety stock on SKUs with sparse history.

graph LR
    subgraph "n >= 30"
        A1[Empirical samples] --> A2[Use empirical<br/>quantiles directly]
    end
    subgraph "n < 30"
        B1[Empirical samples] --> B3[Blend]
        B2[Parametric fitted<br/>distribution] --> B3
        B3 --> B4[Blended quantiles<br/>w = n/n_min]
    end

Storage¶

Empirical distribution statistics are persisted to the ClickHouse table PIPE_uncertainty_distributions:

Column	Description
`dist_type`	`'demand_over_lt'`
`mean`, `variance`, `stddev`	Moment estimates
`skewness`, `kurtosis`	Shape descriptors
`p10` … `p99`	Empirical quantiles
`n_samples`	Observation count
`sample_blob`	JSON-encoded sorted sample array (for Rust)
`source`	`'demand_actuals'`

Performance¶

The Python builder (_build_and_attach_empirical_distributions in meio_runner.py) uses:

Pre-sorted numpy arrays per (item, site)
np.searchsorted binary search for range-sum queries — \(O(\log N)\) per row, \(O(N \log N)\) total per SKU (vs \(O(N^2)\) for a naive inner loop)
Vectorized moment/quantile computation via numpy

Typical timing: 10K SKUs × 104 weekly rows → ~3-8 seconds.

4. Phase 2 — Forecast error during lead time¶

What it models¶

For every historical forecast observation, V3 pairs:

Forecast at order date — from HIST_backtest_forecast (forecast_origin = order date, horizon_step = round(actual_lt_weeks))
Actual demand during realized lead time — from master_demand_actuals

The forecast error is:

\[ \text{forecast\_error} = \text{actual\_demand\_during\_lt} - \text{forecast\_during\_lt} \]

The ForecastErrorDistribution¶

Statistic	Description
`bias`	Mean of forecast errors (positive = under-forecast)
`MAE`	Mean absolute error
`MAPE`	Mean absolute percentage error
`RMSE`	Root mean square error
`quantiles`	p10 … p99 of the error distribution
`skewness`	Error distribution shape
`kurtosis`	Error distribution tail weight

Segmentation¶

Forecast errors are segmented by:

SKU (item_id)
Family (item_type_id)
Location (site_id)
Supplier (route.supplier_id)
Market (channel)

How the MC engine uses it¶

The Monte-Carlo engine (Phase 4) adds a sampled forecast error to each demand sample before computing demand-during-actual-LT:

\[ D_{\text{compound}} = D_{\text{empirical}} + \epsilon_{\text{forecast}} \]

This propagates forecast bias (systematic under/over-forecasting) into the inventory trajectory — a critical input that V2 cannot model.

Fallback source¶

When HIST_backtest_forecast coverage is thin for an (item_id, site_id) horizon pair, V3 falls back to HIST_forecast_point_values (production forecasts keyed by archive_date).

Empirical-mean cycle-stock baseline¶

When an empirical demand-over-LT distribution is active, compute_mv_recursive (files/MEIO_v3/src/marginal.rs) sets effective_total_lt_fcst = dist.empirical_mean() (files/MEIO_v3/src/distributions.rs), overriding demand_rate × planned_lead_time. The empirical mean captures demand during the actual replenishment lead time; using the planned-LT baseline alongside an empirical distribution built over actual LT creates a consistency break (distribution says "need 72 for 95 %" while baseline says "cycle stock = 4"), driving marginal value ≈ 0 and rop ≈ 0. This override is what makes empirical safety stock non-degenerate.

5. Phase 3 — Lead-time variability modeling¶

The problem with V2¶

V2 uses a scalar lt_stddev from master_item.lt_stddev — an item-level constant that:

Is the same for all sites (ignores per-site supplier differences)
Is the same for all suppliers (ignores supplier reliability)
Cannot capture skewness or tail behavior

What V3 does¶

V3 builds an empirical supply LeadTimeDistribution per (item, site) — the replenishment lead-time marginal used by the Monte-Carlo convolution. The distribution is sourced in priority order; the first tier yielding enough observations wins, pooled across all suppliers/routes for the SKU (the MC convolution is per-SKU, so the supplier dimension is aggregated, with the dominant supplier recorded for traceability):

Priority	Tier	`lt_source`	Source	Weighting
1	PO receipts	`purchase_receipt`	`master.master_purchase_receipt.actual_lead_time_days`	volume-weighted by `received_qty`
2	CH supply actuals	`supply_actuals`	`PIPE_lead_time_actuals.actual_lead_time`	equal-weighted
3	Route-synthesized	`route_synthesized`	lognormal samples from `master.route` (primary route, total LT days = mean, `lead_time_cv` = CV)	equal-weighted
4	V2 scalar	(unset)	optimizer falls back to `lt_stddev` / `leg_lead_time`	—

Tier 3 only fires when the primary route carries a positive lead_time_cv (a CV of 0 means the route is deterministic and contributes no LT uncertainty, so the V2 scalar path is appropriate). The synthesized lognormal uses \(\sigma = \sqrt{\ln(1 + cv^2)}\), \(\mu = \ln(\text{mean}) - \sigma^2/2\), clamped to \(\geq 1\) day.

Important: master.demand_actuals.received_date is not used. That column measures customer demand-fulfillment lead time (order → ship-to-customer), not supply/replenishment lead time (supplier PO → warehouse receipt). Using it for safety-stock computation over/under-states replenishment uncertainty and was the source of a bug; the supply-side tiers above are the correct sources.

The distribution stores:

Statistic	Description
`mean_lt_days`	Average actual lead time (weighted)
`lt_stddev`	Lead-time standard deviation (weighted)
`lt_cv`	Coefficient of variation = \(\sigma / \mu\)
`p10` … `p99`	Lead-time quantiles (weighted interpolation)
`supplier_reliability`	On-time probability (1.0 placeholder until receipt promise data is wired)
`n_obs`	Number of observations (or synthesized sample count for tier 3)
`lt_source`	One of `purchase_receipt` / `supply_actuals` / `route_synthesized`

Dedicated lead-time stats table¶

V3 introduces PIPE_lead_time_stats — a dedicated, queryable lead-time table (not folded into the generic uncertainty blob):

Column	Grain
`item_id, site_id`	SKU-location
`supplier_id`	Supplier
`transport_mode`	Transport mode
`archive_date`	Planning week
`mean_lt_days, lt_stddev, lt_cv`	Computed stats
`p10_lt_days` … `p99_lt_days`	Empirical quantiles
`supplier_reliability`	On-time probability
`promised_vs_actual_mean/stddev`	Supplier deviation stats

V2 write-back hook (frozen V2 benefits too)¶

After Phase 3 builds PIPE_lead_time_stats, a runner hook computes the per-(item, site) rollup of lt_stddev and writes it to a new master_item_location.lt_stddev column.

The frozen V2 optimizer reads it via a one-line COALESCE in _load_sku_records:

COALESCE(isit.lt_stddev, i.lt_stddev, 0.0) AS lt_stddev

This means V2 demos gain better per-site lead-time deviation without any V2 code changes. A per-pipeline opt-out flag (writeback_lt_stddev = false) lets demo pipelines freeze their V2 numerics.

Fallback for warehouse nodes¶

Per (item, site) without replenishment receipts or a route carrying LT variability (e.g. warehouse nodes with transfer routes, or tenants that have not populated lead_time_cv), V3 leaves lead_time_distribution unset. The optimizer then falls back to the V2 scalar lt_stddev / leg_lead_time path (meio_runner.py::_load_sku_records). This is tier 4 in the sourcing table above.

6. Phase 4 — Monte-Carlo simulation engine (the digital twin)¶

This is the most important V3 addition

The Monte-Carlo engine is the digital twin — a simulation of the inventory system that samples from empirical distributions to produce service-level, inventory, cost, and shortage distributions (not point estimates). It becomes the training environment for the RL policy layer (Phase 14).

What it simulates¶

For every simulation iteration, the MC engine:

Samples a lead time from the SKU's LeadTimeDistribution
Samples a forecast error from the ForecastErrorDistribution
Samples demand-over-lead-time from the DemandOverLeadTimeDistribution
Simulates the inventory trajectory over the replenishment cycle

The compound sample¶

The DemandOverLeadTimeDistribution is the composite uncertainty model (Principle 1 + enhancement E2). It holds three empirical distributions:

pub struct DemandOverLeadTimeDistribution {
    pub demand: EmpiricalDistribution,           // Phase 1
    pub forecast_error: Option<EmpiricalDistribution>,  // Phase 2
    pub lead_time: Option<EmpiricalDistribution>,       // Phase 3
}

Each iteration draws one triple:

fn compound_sample(&self, rng: &mut R) -> (f64, Option<f64>) {
    let mut demand = self.demand.sample(rng);        // draw demand
    if let Some(ref fe) = self.forecast_error {
        demand += fe.sample(rng);                     // add forecast error
    }
    let lt = self.lead_time.as_ref().map(|lt| lt.sample(rng));  // draw LT
    (demand, lt)
}

The BootstrapMC engine¶

pub struct BootstrapMC {
    rng: SmallRng,
}

Method	Description
`BootstrapMC::new()`	Nondeterministic seed
`BootstrapMC::with_seed(seed)`	Deterministic (for replay)
`run(dist, n_iterations)`	Returns `Vec<(demand, lt)>`
`run_summary(dist, n_iterations)`	Returns aggregated `MCSummary`

What the MC produces¶

Running 1000+ iterations per SKU produces distributions of outcomes:

graph TD
    subgraph "Per iteration"
        S1[Sample LT] --> S2[Sample forecast error]
        S2 --> S3[Sample demand-over-LT]
        S3 --> S4[Simulate inventory trajectory]
        S4 --> S5[service_level, backorders,<br/>lost_sales, expedites,<br/>inventory_value, working_capital]
    end

    S5 --> AGG[Aggregate over<br/>1000+ iterations]

    AGG --> D1[Service distribution]
    AGG --> D2[Inventory distribution]
    AGG --> D3[Cost distribution]
    AGG --> D4[Shortage distribution]

MCSummary output¶

Field	Description
`mean`	Mean demand-during-LT across iterations
`variance`, `stddev`	Spread of demand-during-LT
`skewness`, `kurtosis`	Shape of the demand-during-LT distribution
`p10` … `p99`	Quantiles of demand-during-LT
`n_samples`	Number of MC iterations

MC configuration parameters¶

These are V3 config keys (all optional, backward-compatible defaults):

Parameter	Default	Description
`mc_iterations`	`1000`	Number of Monte-Carlo iterations per SKU
`mc_seed`	`0`	RNG seed. 0 = nondeterministic; nonzero = reproducible
`bootstrap_samples`	`1000`	Bootstrap resample count
`empirical_shrinkage_n_min`	`30`	Shrinkage threshold (E3)
`use_empirical_distributions`	`false`	Master on/off switch
`rl_episode_mode`	`false`	RL digital-twin entry-point switch

Determinism for replay

Set mc_seed to a nonzero value for replay determinism. This ensures the same replay week produces identical MC outputs, which is essential for A/B comparison between V2 and V3.

Performance characteristics¶

The Rust MC engine uses:

SmallRng — a fast non-cryptographic PRNG (Xoshiro256++)
Rayon parallelism — iterations are embarrassingly parallel
Pre-sorted sample arrays — O(1) sampling via index lookup

Typical throughput: 50,000+ SKU-iterations/second on the dev box (reuses the existing compute_causal_uncertainty MC pattern).

Storage¶

MC summary statistics are persisted to:

PIPE_uncertainty_distributions (per-SKU distribution stats)
PIPE_simulation_summary (per-SKU MC summary, Phase 4+)
PIPE_simulation_trajectories (sampled trajectories, not all 1000, to control ClickHouse volume)

7. The Empirical and CompoundPoissonEmpirical distribution variants¶

V3 adds two new variants to the existing DistributionType enum:

`DistributionType::Empirical`¶

{
    "type": "empirical",
    "samples": [10.5, 12.3, 15.0, 18.2, 22.1, ...]
}

The samples array is sorted ascending
CDF uses binary search: \(O(\log n)\)
Fill-rate integration uses the same EOQ-step trapezoidal loop as the parametric distributions, but with the empirical CDF substituted

`DistributionType::CompoundPoissonEmpirical`¶

{
    "type": "compound_poisson_empirical",
    "demand_samples": [100, 150, 200, ...],
    "lt_samples": [7, 10, 14, 21, ...]
}

Used when both demand and lead-time empirical distributions are available
The MC engine samples (demand, LT) pairs and computes demand-during-actual-LT
For fill-rate computation, the CDF uses the demand samples directly (the LT variability is baked into the demand-over-LT observations)

Distribution resolution priority¶

graph TD
    A[use_empirical_distributions = true?] -->|Yes| B{demand_over_lt_distribution<br/>present?}
    A -->|No| C{fitted_distribution<br/>present?}
    B -->|Yes| D[Use Empirical /<br/>CompoundPoissonEmpirical]
    B -->|No| E[Warn + fall through]
    E --> C
    C -->|Yes| F[Use fitted<br/>Gamma/LogNormal/etc.]
    C -->|No| G{sku_count > threshold?}
    G -->|Yes| H[Use Normal<br/>golden formula]
    G -->|No| I[Use Poisson]

8. RL policy parameters — the bridge to MEIO¶

Figure: the RL policy loop — the agent observes the digital-twin state (service/inventory/cost distributions), chooses policy parameters (never order quantities), MEIO + supply act on them, and the reward signal (service-level + working-capital + shortage) closes the loop.

flowchart LR
  STATE["State<br/>MC service / inventory / cost<br/>distributions + risk indicators"] --> AGENT["RL Agent<br/>PPO / SAC"]
  AGENT -->|"action"| POLICY["Policy parameters<br/>target_fill_rate, risk_multiplier,<br/>inventory_budget, expedite_threshold,<br/>supplier_preferences"]
  POLICY --> MEIO["MEIO optimizer<br/>(sets committed_buffer / EOQ)"]
  MEIO --> SUPPLY["Supply planning<br/>(generates orders)"]
  SUPPLY --> TWIN["Monte-Carlo digital twin<br/>simulates trajectory"]
  TWIN --> REWARD["Reward<br/>service_level − working_capital<br/>− shortage / expedite penalties"]
  REWARD --> AGENT
  TWIN --> STATE

V3 extends the config_meio_parameter_set with new policy keys that the RL layer sets and MEIO consumes:

Parameter	Default	Set by RL	Description
`risk_multiplier`	`1.0`	Yes	Scales safety-stock buffer post-MEIO
`expedite_threshold`	`0.0` (off)	Yes	Inventory level triggering expedite
`supplier_preference_weights`	`[]` (route priority)	Yes	Weights for alternative suppliers
`inventory_budget_multiplier`	`1.0`	Yes	Scales `max_budget`
`target_fill_rate`	(per-segment)	Yes	Override segment fill-rate target

These flow through the existing per-segment parameter-set model — no new override mechanism is invented.

The `SkuRecord` V3 fields¶

// V3 additions on SkuRecord (all optional, serde-default):
pub demand_over_lt_distribution: Option<DistributionType>,
pub forecast_error_distribution: Option<DistributionType>,
pub lead_time_distribution: Option<LeadTimeDistribution>,
pub risk_multiplier: f64,              // default 1.0
pub expedite_threshold: f64,           // default 0.0
pub supplier_preference_weights: Vec<f64>,  // default []

9. Data flow — from observations to inventory targets¶

sequenceDiagram
    participant DB as PostgreSQL
    participant CH as ClickHouse
    participant PY as Python (meio_runner)
    participant RUST as Rust (meio_optimizer_v3)

    Note over DB: master_demand_actuals<br/>(date, qty, received_date)
    Note over DB: master_purchase_receipt<br/>(release_dt, actual_receipt_dt)
    Note over CH: HIST_backtest_forecast<br/>(forecast_origin, actual_value)

    PY->>DB: Query demand rows with received_date
    PY->>DB: Query purchase receipts
    PY->>CH: Query backtest forecasts

    PY->>PY: Build EmpiricalDistribution per (item, site)
    PY->>PY: Build ForecastErrorDistribution per (item, site)
    PY->>PY: Build LeadTimeDistribution per (item, site, supplier)

    PY->>CH: Persist to PIPE_uncertainty_distributions
    PY->>CH: Persist to PIPE_lead_time_stats
    PY->>DB: Write-back lt_stddev to master_item_location

    PY->>RUST: run_optimization(skus_json, config_json, targets)
    Note over RUST: use_empirical_distributions=true<br/>→ Empirical CDF in fill_rate_for_rop<br/>→ BootstrapMC samples for RL

    RUST->>RUST: Greedy marginal value loop<br/>(same as V2, but with empirical CDF)
    RUST-->>PY: sku_results, group_results

    PY->>CH: Persist to PIPE_meio_results
    PY->>CH: Persist to PIPE_meio_group_results

10. New database tables (Phase 0 scaffolding)¶

PostgreSQL (master / scenario)¶

Table	Purpose
`master_purchase_receipt`	Supplier receipt history (promised vs actual)
`master_forecast_version`	Forecast-at-order-date registry
`master_item_location.lt_stddev`	Per-site LT deviation (new column)
`scen_causal_graph`	Versioned causal graph (Phase 7)
`scen_causal_graph_edge`	Causal graph edges as rows
`config_corrective_action_kb`	Corrective action knowledge base (Phase 11)
`config_rl_policy`	RL policy parameter sets (Phase 15)

ClickHouse (analytics)¶

Table	Purpose
`PIPE_uncertainty_distributions`	Phase 1-3 empirical dist stats
`PIPE_lead_time_stats`	Phase 3 dedicated LT + LT-deviation
`PIPE_shortage_events`	Phase 5 shortage event dataset
`PIPE_shortage_root_causes`	Phase 8 root cause attribution
`PIPE_shortage_counterfactuals`	Phase 9 counterfactual analysis
`PIPE_preventability_scores`	Phase 10 preventability scoring
`PIPE_risk_indicators`	Phase 12 early-warning indicators
`PIPE_predictive_shortage`	Phase 13 baseline model predictions
`PIPE_rl_policy_outputs`	Phase 15 RL policy outputs

All tables carry pipeline_id + archive_date for partition isolation and cascade-delete compatibility.

11. API endpoints¶

Endpoint	Method	Description
`/api/admin/preserve-firm-orders`	POST	Snapshot `import.firm_supply_orders` into `master_purchase_receipt`
`/api/admin/seed-v3-data`	POST	Seed V3 data (received_date, receipts, lt_stddev) — see §17
`/api/admin/populate-received-dates`	POST	Populate `demand_actuals.received_date` (legacy, admin-only)
`/api/meio/distributions/build`	POST	Build empirical demand-over-LT distributions for a pipeline
`/api/meio/distributions`	GET	Fetch distribution statistics from CH

Example: building distributions¶

POST /api/meio/distributions/build?pipeline_id=42

Response:

{
    "pipeline_id": 42,
    "total_skus": 10000,
    "distributions_built": 7234,
    "message": "Built 7234 empirical demand-over-LT distributions"
}

Example: fetching distributions¶

GET /api/meio/distributions?pipeline_id=42&dist_type=demand_over_lt

Response:

href="#__codelineno-11-1">{ "pipeline_id": 42, "dist_type": "demand_over_lt", "distributions": [ { "item_id": 1, "site_id": 10, "dist_type": "demand_over_lt", "mean": 520.0, "variance": 18400.0, "stddev": 135.6, "skewness": 0.42, "kurtosis": 0.18, "p10": 320.0, "p25": 410.0, "p50": 520.0, "p75": 630.0, "p90": 720.0, "p95": 780.0, "p99": 890.0, "n_samples": 52, "source": "demand_actuals" } ] }

12. Configuration reference (V3 additions)¶

All V3 keys live in the 'meio' parameter set and are optional with backward-compatible defaults.

Parameter	Default	Effect
`meio_optimizer_version`	`'v3'`	`'v1'`, `'v2'`, or `'v3'` (V3 default; V2 explicit fallback)
`use_empirical_distributions`	`false`	Master switch for empirical dists
`bootstrap_samples`	`1000`	Bootstrap resample count
`empirical_shrinkage_n_min`	`30`	E3 shrinkage threshold
`mc_iterations`	`1000`	MC simulation iterations
`mc_seed`	`0`	MC seed (0 = nondeterministic)
`rl_episode_mode`	`false`	RL digital-twin switch
`risk_multiplier`	`1.0`	RL: scales safety stock
`expedite_threshold`	`0.0`	RL: expedite trigger level
`supplier_preference_weights`	`[]`	RL: supplier weights
`inventory_budget_multiplier`	`1.0`	RL: scales max_budget
`writeback_lt_stddev`	`true`	Write per-site LT to master_item_location

13. Enabling V3 on a pipeline¶

To opt a pipeline into V3:

Build the V3 crate:
```
cd files/MEIO_v3
cargo build --release
```

Run the DDL migrations:

psql -d <tenant_db> -v schema=scenario -f files/DDL/v3_phase0_pg.sql
clickhouse-client --database <tenant> --multiquery < files/DDL/v3_phase0_ch.sql
clickhouse-client --database <tenant> --multiquery < files/DDL/v3_phase0_hist.sql

Set the MEIO parameter:

UPDATE scenario.config_parameters
SET parameters_set = jsonb_set(
    parameters_set,
    '{meio_optimizer_version}',
    '"v3"'
)
WHERE parameter_type = 'meio' AND is_default = true;

Enable empirical distributions (optional):

UPDATE scenario.config_parameters
SET parameters_set = jsonb_set(
    parameters_set,
    '{use_empirical_distributions}',
    'true'
)
WHERE parameter_type = 'meio' AND is_default = true;

Preserve firm orders before each ERP reload:
```
POST /api/admin/preserve-firm-orders
```

Build distributions before the first V3 MEIO run:

POST /api/meio/distributions/build?pipeline_id=<your_pipeline>

14. Validation — the A/B gate¶

V3 is only allowed to diverge from V2 when use_empirical_distributions = true. The Phase-0 A/B gate asserts:

V2 and V3 produce identical PIPE_meio_results on the same replay week with use_empirical_distributions = false.

This is verified by running both optimizers on the same replay week and comparing committed_buffer, fill_rate, and marginal_value for every SKU.

When use_empirical_distributions = true, V3 is expected to diverge — and the Phase 1 gate requires safety stock to differ by ≥ 5% on at least 20% of SKUs with n_samples >= 50 (proof that the empirical path is actually engaged).

15. Roadmap — phases beyond this implementation¶

Phase	Status	Description
Phase 0	✅ Complete	Scaffolding, DDL, V3 crate, preservation endpoint
Phase 1	✅ Complete	Demand-over-LT empirical distributions
Phase 2	🔲 Planned	Forecast-error distributions
Phase 3	🔲 Planned	Lead-time variability + write-back hook
Phase 4	🔲 Planned	Monte-Carlo simulation engine (digital twin)
Phase 5	🔲 Planned	Shortage event dataset
Phase 6-11	🔲 Planned	Causal shortage intelligence
Phase 12	🔲 Planned	Early-warning risk indicators
Phase 13	🔲 Planned	Predictive baselines (XGBoost/LightGBM/CatBoost)
Phase 14-19	🔲 Planned	RL policy layer (PPO/SAC + Gymnasium)

See the full implementation plan in .kilo/plans/meio-causal-rl-enhancement.md for details on all phases.

16. Key files¶

File	Description
`files/MEIO_v3/Cargo.toml`	V3 crate manifest
`files/MEIO_v3/src/lib.rs`	PyO3 module entry point
`files/MEIO_v3/src/uncertainty.rs`	EmpiricalDistribution, BootstrapMC, DemandOverLeadTimeDistribution
`files/MEIO_v3/src/distributions.rs`	Empirical + CompoundPoissonEmpirical variants
`files/MEIO_v3/src/monte_carlo.rs`	MC simulation engine (Phase 4 scaffold)
`files/MEIO_v3/src/config.rs`	V3 config keys
`files/MEIO_v3/src/sku.rs`	V3 SkuRecord fields
`files/meio_runner.py`	Python builder + V3 version selection
`files/api/main.py`	API endpoints
`files/DDL/v3_phase0_pg.sql`	PostgreSQL DDL
`files/DDL/v3_phase0_ch.sql`	ClickHouse DDL
`files/DDL/v3_phase0_hist.sql`	HIST mirror DDL
`files/seed_v3_meio_data.py`	V3 data seeder (received_date, receipts, lt_stddev)
`.kilo/plans/meio-causal-rl-enhancement.md`	Full implementation plan

17. Seeding V3 data — filling the gaps¶

Prerequisite

The V3 empirical-distribution engine requires three data sources that may not exist in a fresh demo database. The seed_v3_meio_data.py script fills these gaps idempotently — it only writes where data is missing.

What needs to be seeded¶

Data source	Table	Purpose	When missing
Demand fulfillment dates	`master_demand_actuals.received_date`	Phase 1: demand-over-LT + Phase 3 fallback LT	Generated lognormal around route LT
Supplier receipt history	`master_purchase_receipt`	Phase 3: replenishment LT + supplier reliability	Synthetic receipts per BUY route
Per-site LT deviation	`master_item_location.lt_stddev`	V2 write-back + V3 golden-formula fallback	Computed from receipts or demand

The lognormal lead-time model¶

Both received_date and synthetic receipts use a lognormal distribution to generate realistic lead-time variability:

\[ L_{\text{actual}} \sim \text{Lognormal}(\mu, \sigma) \]

where:

\(\mu = \ln(L_{\text{expected}}) - \sigma^2 / 2\) (so \(E[L] = L_{\text{expected}}\))
\(\sigma = 0.40\) (default) → coefficient of variation \(\approx 0.41\)
\(L_{\text{expected}}\) = route (lead_time + transit_time + pick_pack_time) × 7 days

graph LR
    R[master.route<br/>lead_time + transit_time + lead_time_cv] --> SYN[Route-synthesized lognormal<br/>tier 3]
    PR[master_purchase_receipt<br/>actual_lead_time_days] --> R1[Tier 1: PO receipts<br/>volume-weighted]
    CH[PIPE_lead_time_actuals<br/>actual_lead_time] --> R2[Tier 2: CH supply actuals<br/>equal-weighted]
    R1 --> STD[lt_stddev / LeadTimeDistribution<br/>per item-site, aggregated]
    R2 --> STD
    SYN --> STD
    STD --> V2F[Tier 4: V2 scalar fallback<br/>when no supply LT data]
    STD --> V3[V3 reads empirical<br/>LeadTimeDistribution]
    V2F --> V2[Frozen V2 reads<br/>COALESCEisit.lt_stddev, i.lt_stddev]

Running the seeder¶

Option A — API endpoint¶

POST /api/admin/seed-v3-data?only=all&seed=42&receipts_per_sku=52&sigma=0.40

Parameters:

Parameter	Default	Description
`only`	`all`	`all`, `received_date`, `receipts`, or `lt_stddev`
`seed`	`42`	RNG seed for reproducibility
`receipts_per_sku`	`52`	Synthetic receipts per (item, site) with a BUY route
`sigma`	`0.40`	Lognormal shape (CV ≈ 0.41)
`dry_run`	`false`	Print what would be done without writing

Option B — Standalone script¶

cd files
python seed_v3_meio_data.py                          # all three steps
python seed_v3_meio_data.py --only received_date     # just step 1
python seed_v3_meio_data.py --only receipts          # just step 2
python seed_v3_meio_data.py --only lt_stddev         # just step 3
python seed_v3_meio_data.py --dry-run                # preview only
python seed_v3_meio_data.py --seed 123 --sigma 0.30  # custom params

Step 1 — Demand fulfillment dates¶

Populates received_date on master_demand_actuals rows where it is NULL.

Source of expected LT: master.route (lead_time + transit_time + pick_pack_time) × 7 days
Plus MEIO wait_time from PIPE_meio_results (ClickHouse) when available
Distribution: Lognormal(\(\mu\), \(\sigma\)) with \(\sigma = 0.40\)
Result: received_date = date + actual_lt_days

Step 2 — Synthetic purchase receipts¶

Generates master_purchase_receipt rows for every (item, site) with a BUY route.

Per receipt:

Field	Value
`release_dt`	Random date in the past 2 years
`promised_dt`	`release_dt + expected_lt_days`
`actual_receipt_dt`	`release_dt + lognormal(actual_lt)`
`received_qty`	4 weeks of avg demand (rounded to route `mult_qty`)
`erp_status`	90% `closed` (full receipt), 10% `partial`
`transport_mode`	Random: TRUCK / RAIL / AIR / SEA
`supplier_id`	From `master.route.supplier_id`

This produces supplier reliability = \(P(\text{actual} \leq \text{promised})\) and lead-time variability stats per supplier.

Real ERP data preferred

Synthetic receipts are a fallback for demo/seed environments. In production, run POST /api/admin/preserve-firm-orders before each ERP reload to snapshot real import.firm_supply_orders into master_purchase_receipt.

Step 3 — Per-site lead-time deviation¶

Computes lt_stddev per (item, site) and writes to master_item_location:

Preferred source: master_purchase_receipt.actual_lead_time_days (replenishment LT)
Fallback: master_demand_actuals.received_date - date (fulfillment LT)
Computes: population standard deviation of LT observations (in weeks)
Writes: master_item_location.lt_stddev

The frozen V2 optimizer picks this up via:

COALESCE(isit.lt_stddev, i.lt_stddev, 0.0) AS lt_stddev

So V2 demos gain better per-site LT deviation without any V2 code changes.

Complete setup sequence¶

# 1. Apply DDL migrations
psql -d <tenant_db> -v schema=scenario -f files/DDL/v3_phase0_pg.sql
clickhouse-client --database <tenant> --multiquery < files/DDL/v3_phase0_ch.sql
clickhouse-client --database <tenant> --multiquery < files/DDL/v3_phase0_hist.sql

# 2. Build the V3 crate
cd files/MEIO_v3 && cargo build --release && cd ../..

# 3. Seed V3 data (received_date + receipts + lt_stddev)
cd files && python seed_v3_meio_data.py && cd ..

# 4. Build empirical distributions
POST /api/meio/distributions/build?pipeline_id=<N>

# 5. Enable V3 on a pipeline
#    Set meio_optimizer_version='v3' in the meio param set
#    Optionally set use_empirical_distributions=true

MEIO V3 — Empirical Distributions, Monte-Carlo Digital Twin & RL Policy Layer¶

1. Why V3? — The principles¶

Principle 1 — Inventory protects against more than demand variability¶

Principle 2 — Do not assume normality¶

Principle 3 — RL optimizes policies, not quantities¶

2. Architecture — the V3 stack¶

V2↔V3 coexistence¶

3. Phase 1 — Demand-over-lead-time empirical modeling¶

What it replaces¶

What V3 does instead¶

The EmpiricalDistribution struct (Rust)¶

E3 — Bias-corrected shrinkage¶

Storage¶

Performance¶

4. Phase 2 — Forecast error during lead time¶

What it models¶

The ForecastErrorDistribution¶

Segmentation¶

How the MC engine uses it¶

Fallback source¶

Empirical-mean cycle-stock baseline¶

5. Phase 3 — Lead-time variability modeling¶

The problem with V2¶

What V3 does¶

Dedicated lead-time stats table¶

V2 write-back hook (frozen V2 benefits too)¶

Fallback for warehouse nodes¶

6. Phase 4 — Monte-Carlo simulation engine (the digital twin)¶

What it simulates¶

The compound sample¶

The BootstrapMC engine¶

What the MC produces¶

MCSummary output¶

MC configuration parameters¶

Performance characteristics¶

Storage¶

7. The Empirical and CompoundPoissonEmpirical distribution variants¶

DistributionType::Empirical¶

DistributionType::CompoundPoissonEmpirical¶

Distribution resolution priority¶

8. RL policy parameters — the bridge to MEIO¶

The SkuRecord V3 fields¶

9. Data flow — from observations to inventory targets¶

10. New database tables (Phase 0 scaffolding)¶

PostgreSQL (master / scenario)¶

ClickHouse (analytics)¶

11. API endpoints¶

Example: building distributions¶

Example: fetching distributions¶

12. Configuration reference (V3 additions)¶

13. Enabling V3 on a pipeline¶

14. Validation — the A/B gate¶

15. Roadmap — phases beyond this implementation¶

16. Key files¶

17. Seeding V3 data — filling the gaps¶

What needs to be seeded¶

The lognormal lead-time model¶

Running the seeder¶

Option A — API endpoint¶

Option B — Standalone script¶

Step 1 — Demand fulfillment dates¶

Step 2 — Synthetic purchase receipts¶

Step 3 — Per-site lead-time deviation¶

Complete setup sequence¶

`DistributionType::Empirical`¶

`DistributionType::CompoundPoissonEmpirical`¶

The `SkuRecord` V3 fields¶