Pipeline Configuration

A pipeline is a named workspace that bundles all the scenario IDs needed for a complete supply chain plan.

---

What Is a Pipeline?

Figure: pipeline composition — six scenario IDs are bundled into one pipeline; a run produces ClickHouse PIPE outputs partitioned by pipeline_id.

flowchart LR
    subgraph Scenarios
      S1["series_scenario_id<br/>(forecast)"]
      S2["causal_scenario_id"]
      S3["maintenance_scenario_id"]
      S4["inventory_scenario_id<br/>(EOQ)"]
      S5["meio_scenario_id"]
      S6["supply_scenario_id"]
    end
    P["config_scenario_pipeline<br/>(pipeline_id, name)"]
    S1 --> P
    S2 --> P
    S3 --> P
    S4 --> P
    S5 --> P
    S6 --> P
    P --> R["Pipeline run<br/>(POST /api/pipeline/run-all)"]
    R --> O["ClickHouse PIPE outputs<br/>(partitioned by pipeline_id)"]

scenario.config_scenario_pipeline contains:

Field	Description
pipeline_id	Auto-increment primary key
name	Display name in toolbar dropdown
series_scenario_id	Forecast scenario
causal_scenario_id	Causal/asset scenario
maintenance_scenario_id	Maintenance event scenario
inventory_scenario_id	EOQ / inventory scenario
meio_scenario_id	MEIO safety-stock scenario
supply_scenario_id	Supply planning scenario
data_source_pipeline_id	Source pipeline for copying corrections
data_source_type	pipeline or baseline
parent_pipeline_id	Branch parent (for what-if branches)
freeze_on_run	Auto-freeze on next run
freeze_period	Weeks of frozen horizon

Note: The old scenario.scenario_pipeline table has been removed. All pipelines are now in scenario.config_scenario_pipeline.

---

The Pipelines Page

Route: /pipelines

Tabs (left to right):

Tab	Content
Classification	ABC/XYZ classification configs — run, create, edit; combined results table with per-config class filters
Forecast	Forecast scenario management and per-pipeline forecast parameter sets
EOQ	EOQ / K-Curve scenario management
MEIO	MEIO scenario management and parameter sets
Supply Planning	Supply scenario management
Exceptions	Exception scenario parameter sets
Pipeline	Pipeline CRUD — create, edit, run, view status

---

The Left Toolbar Dropdown

Selecting a pipeline:

1. Calls setActivePipelineId(id) in PipelineContext
2. Persists in localStorage key mirabelle_pipeline
3. All components using usePipeline() re-render
4. Supply API calls auto-pick up pipeline_id + supply_scenario_id via useSupplyParams()

---

Immutable vs Pipeline-Scoped Data

Immutable (master schema — no pipeline tag)

• item, location, item_location, demand_actuals, route, bill_of_material, on_hand
• causal_*, maintenance_*, segment, segment_membership

Pipeline-Scoped (require pipeline_id + scenario_id)

Stored in ClickHouse PIPE_* tables, always filter by pipeline_id first:

• PIPE_forecast_point_values, PIPE_series_characteristics, PIPE_series_best_methods
• PIPE_series_backtest_metrics, PIPE_fitted_distributions
• PIPE_meio_results, PIPE_supply_orders, PIPE_supply_inventory_projection
• PIPE_abc_results, PIPE_segment_membership, PIPE_feature_reliability

---

Creating a Pipeline

Via the UI

1. Go to Pipelines → Pipeline tab.
2. Click New pipeline.
3. Enter a name, select scenario IDs for each domain.
4. Optionally link segments to scope the pipeline.
5. Save.

Via the API

POST /api/scenario-pipeline
Content-Type: application/json

{
  "name": "Q2 2026 Plan",
  "series_scenario_id": 2,
  "meio_scenario_id": 3,
  "supply_scenario_id": 4
}

---

How Workflows Reference Pipelines

Process runs launched from Processes (/processes) always use the pipeline ID explicitly passed in the API body — not the sidebar selection. This allows running processes for any pipeline without changing the active view.

{
  "pipeline_id": 42,
  "scenario_id": 2,
  "segment_id": null
}

---

Pipeline API Reference

Figure: pipeline run / job lifecycle — queued to running, then completed, failed, or killed.

stateDiagram-v2
    [*] --> queued: "POST /api/pipeline/run/{step}"
    queued --> running: "Subprocess started"
    running --> completed: "All steps finish"
    running --> failed: "Error / exception"
    queued --> killed: "POST .../jobs/{id}/kill"
    running --> killed: "POST .../jobs/{id}/kill"
    failed --> [*]
    completed --> [*]
    killed --> [*]

Method	Path	Description
GET	/api/scenario-pipeline	List all pipelines
POST	/api/scenario-pipeline	Create a pipeline
PUT	/api/scenario-pipeline/{id}	Update a pipeline
DELETE	/api/scenario-pipeline/{id}	Delete a pipeline
GET	/api/pipeline/steps	List available pipeline steps
POST	/api/pipeline/run/{step}	Launch a step as background job
POST	/api/pipeline/run-all	Launch the complete pipeline
GET	/api/pipeline/jobs	List all jobs
GET	/api/pipeline/jobs/{job_id}	Job status
GET	/api/pipeline/jobs/{job_id}/stream	SSE log stream