Non-Regression Test Suite¶
File: files/tests/test_non_regression.py | 120 tests | 0.24s runtime
Generated from 50+ development sessions. Pure-logic tests (no live DB/CH/HTTP). Each test maps to a specific bug or edge case found during development, ensuring it never reoccurs.
Coverage by Functional Area¶
1. MEIO / Inventory Optimization (11 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_demand_weighted_fill_rate_matches_weighted_formula |
IO fill rate exceeded target | avg(fill_rate) inflated by saturated low-demand SKUs |
Demand-weighted formula sum(fr*dr)/sum(dr) matches expected value |
test_demand_weighted_fill_rate_nan_fallback_when_zero_demand |
IO fill rate exceeded target | Division by zero when all direct_demand_rate=0 |
Falls back to simple average |
test_optimizer_endgame_closes_gap |
IO fill rate exceeded target | Greedy loop stalled with groups below target | Gap > 0 and group not completed → endgame should fire |
test_endgame_skips_zero_demand_rate_skus |
IO fill rate exceeded target | Endgame invested budget in WH SKUs that don't advance group metric | marginal_group_gain == 0 for zero-demand SKUs |
test_diagnostic_display_consistency_field_exists |
IO fill rate exceeded target | No way to compare display vs optimizer FR | display_consistency dict has required keys |
test_lead_time_label_matches_route_type[BUY/BUILD/TRANSFER] |
Series 10_201 lead time | Hardcoded "Procurement LT" for all routes | Dynamic label matches route type |
test_wait_time_weeks_to_days_conversion |
Series 10_201 lead time | wait_time displayed in weeks with "d" suffix |
weeks * 7 = days |
test_heapq_tiebreaker_prevents_dict_comparison |
V3 MEIO validation | heapq TypeError on tied direct_demand_rate (dict comparison) |
Integer tiebreaker prevents dict comparison |
test_archive_date_as_date_not_string |
V3 MEIO validation | archive_date.isoformat() passed to CH instead of date object |
date constructor rejects strings |
2. Parameter Resolution (10 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_require_pipeline_id_raises_on_none |
Tab netting 500, EOQ failure | pipeline_id=0 silently skips preprocessed path |
ValueError on None |
test_require_pipeline_id_raises_on_zero |
Tab netting 500, EOQ failure | pipeline_id or 0 defaults to 0 |
ValueError on 0 |
test_require_pipeline_id_passes_on_valid |
Tab netting 500 | Valid ID passes through | Returns int(pipeline_id) |
test_require_pipeline_id_context_in_message |
Tab netting 500 | Error message lacks context | Context string appears in ValueError |
test_discover_only_existing_columns |
Tab netting 500 | ParameterResolver SELECTs columns that don't exist |
Only existing columns selected |
test_dead_columns_not_in_build_sql |
Audit/standardize params | Dead columns (evaluation_parameter_id, etc.) still in build SQL |
Dead columns not in current set |
test_pipeline_id_check_constraint_rejects_zero |
Making overrides pipeline-driven | No pipeline_id column on override tables |
CHECK (pipeline_id > 0) rejects 0 |
test_pipeline_id_check_constraint_rejects_none |
Making overrides pipeline-driven | NULL pipeline_id allowed | Rejected |
test_two_pipelines_can_have_same_item_site |
Making overrides pipeline-driven | Same (item_id, site_id) across pipelines should coexist | Composite key includes pipeline_id |
test_watchdog_timeout_default_is_600 |
Tab netting 500 | 1800 vs 600 divergence | Default is 600 |
3. Forecasting / Backtesting (20 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_empty_method_selection_does_not_collapse_to_historic_average |
Forecast selection | Empty method_selection → all series get ['HistoricAverage'] |
Per-group fallback has multiple methods |
test_best_fit_empty_method_selection_not_blocked_by_all_empty_guard |
Method selection impact | all_empty guard prevented best_fit_methods migration |
best_fit_methods used when available |
test_attribute_error_is_caught_by_broad_exception |
NeuralForecast compat | AttributeError from pytorch_lightning 2.x not caught by except ImportError |
except Exception catches it |
test_dormant_confidence_intervals_key_removed |
Confidence levels in forecast params | confidence_intervals in FPS was dormant (editable but never merged) |
Key stripped from persisted params |
test_is_monotonic_identity_check_fragility |
Confidence levels | np.False_ is False evaluates to False |
not ds.is_monotonic_increasing works |
test_is_monotonic_with_numpy_bool |
Confidence levels | Same — numpy bool identity check | Robust check returns True for non-monotonic |
test_nan_coverage_fillna_zero_is_neutral |
Backtest performance | fillna(1.0) penalizes NaN as worst; fillna(0.0) neutral |
Different fillna values produce different scores |
test_batch_size_default_enables_parallelism |
Forecast cluster offline | batch_size=100 prevented Dask for ≤100 series |
25 series with batch_size=20 → multiple batches |
test_batch_size_zero_clamped |
Forecast cluster offline | batch_size=0 → ZeroDivisionError |
Clamped to ≥1 |
test_batch_size_division_by_zero_guarded |
Forecast cluster offline | Same | No division by zero |
test_uids_pre_derived_from_characteristics_df |
_uids_pre undefined | _uids_pre referenced but never defined |
Derived from characteristics_df |
test_empty_uids_pre_guards_delete |
_uids_pre undefined | Empty _uids_pre → unique_id IN () (invalid CH SQL) |
Delete guarded on non-empty list |
test_to_numeric_coerce_handles_non_numeric |
Netting characterization, Method selection | astype(int, errors='ignore') crashes on 'INDTKITP' in pandas 2.x |
pd.to_numeric(errors='coerce') handles gracefully |
test_astype_int_errors_ignore_raises_in_pandas2 |
Same | Same | errors='raise' throws; errors='ignore' may not |
test_empty_df_has_required_columns |
Method selection | Empty fallback DataFrame has no columns → KeyError |
Correct column schema on empty DF |
test_clickhouse_max_empty_set_returns_epoch |
Forecast selection | CH max() over empty set returns '1970-01-01' instead of NULL |
Guard: dates before 2000 → None |
4. Allocation (5 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_archive_date_uses_planning_monday_not_today |
Supply without allocation | allocation_run.archive_date = planning_today() ≠ CH planning_monday() |
planning_monday always returns a Monday |
test_concurrent_run_returns_409 |
Supply without allocation | Two concurrent allocation runs race on same partition | Second Lock.acquire(blocking=False) fails |
test_dict_key_collision_keeps_last_value |
Supply without allocation | {sid: scenario_id, sid: site_id} keeps site_id |
Separate param names sid/site |
test_firm_meta_reduce_customer_from_highest_priority |
Pipeline step Seg+ABC | Enrichment vs engine had divergent tie-break logic | Customer from non-null, priority from highest rank |
test_convention_independent_week_index |
Time phase allocation | Monday-snap vs Sunday-start week mismatch | (date - planning_today).days // 7 is convention-independent |
5. Supply / Indirect Demand (4 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_capacity_persist_guard_only_checks_rr_map |
Network resources missing | if _rr_map and capacity: — capacity loaded from empty CH |
if _rr_map: persists even with empty capacity |
test_writer_adapts_to_schema_version |
PIPE_indirect_demand refactor | Writer fails on pre-migration table (no source_item_id) |
Conditionally include columns |
test_breakdown_query_includes_pipeline_id |
PIPE_indirect_demand refactor | Breakdown queries omit pipeline_id → cross-partition scan |
pipeline_id in WHERE clause |
test_parse_canonical_uid / test_parse_multi_digit_ids |
Supply network hover | _uids_to_pairs can't resolve canonical uids from BFS-keyed dict |
Fallback parses (item_id, site_id) from string |
6. K-Curve (2 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_cache_size_cap |
Refactoring kcurve to Rust | _KCURVE_ITEMS_CACHE unbounded memory growth |
OrderedDict with 16-entry cap, LRU eviction |
test_threshold_binning_reduces_count |
K-curve rename, exchange stalls | O(16,000) thresholds → exchange curve hangs | Binned to ≤300 |
7. Auto-Approval Engine (10 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_deterministic_bucket_consistent_across_calls |
APS Auto-Approval | hash(v) % 1000 non-deterministic (PYTHONHASHSEED) |
md5-based bucket is deterministic |
test_deterministic_bucket_different_inputs |
APS Auto-Approval | Same | Different inputs → different buckets |
test_existing_active_rules_preserved_after_migration |
APS Auto-Approval | DEFAULT 'draft' deactivates all existing rules |
is_active=TRUE → governance_state='active' |
test_none_segment_out_of_scope_when_rule_restricts |
APS Auto-Approval | seg is None short-circuits scope filter |
None segment treated as out-of-scope |
test_matching_segment_in_scope |
APS Auto-Approval | Same | Matching segment passes scope |
test_path_traversal_rejected |
APS Auto-Approval | model_uri loaded from DB without path confinement |
../../etc/passwd raises ValueError |
test_valid_model_uri_accepted |
APS Auto-Approval | Same | Valid URI inside artifacts dir accepted |
test_collect_tuples_avoids_stale_var |
APS Auto-Approval | CH mirror list comprehension reuses stale loop variable | Collect tuples inside loop, not after |
test_or_true_is_always_true |
APS Auto-Approval | AND ... OR TRUE bypasses item/site filter |
OR TRUE removed, replaced with meaningful condition |
test_rule_auto_approve_short_circuits_model |
APS Auto-Approval | Confidence policy: rule must short-circuit model | AUTO_APPROVE rule → decision from rule source |
test_mandatory_review_forces_manual |
APS Auto-Approval | Model confidence shouldn't override mandatory_review |
MANDATORY_REVIEW → manual_review even with p=0.999 |
test_imports_reference_same_module |
APS Auto-Approval | rows_to_matrix/safe_proba duplicated in trainer & inference |
Both import from shared preprocess.py |
8. Audit / Auth (5 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_resolve_actor_uses_sub_when_no_email |
Audit log events | user.get("email") returns None for dicts without email |
Falls through to name or sub |
test_resolve_actor_falls_to_sub |
Audit log events | Same | sub used when no email/name |
test_resolve_actor_system_default |
Audit log events | Same | Empty dict → "system" |
test_available_cols_adapts_to_schema |
Audit log events | Hardcoded ip, user_agent columns → 500 on unmigrated DB |
SELECT only existing columns |
test_all_failure_branches_produce_audit_row |
Audit log events | 3 early-return branches skip _audit_workflow_lifecycle |
All branches produce audit outcome |
9. Mirabelle (4 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_select_gets_two_columns |
Remove unique_id | _fix_sql_builtin_names expanded unique_id to 2 columns in WHERE (invalid SQL) |
SELECT gets item_id, item_name; WHERE gets item_id only |
test_where_gets_single_column |
Remove unique_id | Same | No comma in WHERE expansion |
test_no_unique_id_in_output |
Remove unique_id | Same | No unique_id in fixed SQL |
test_name_columns_inserted_after_id |
Replace IDs with names | LLM SQL returns bare IDs | item_name inserted right after item_id |
10. Database / Schema Migration (7 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_migrations_must_be_sequential |
Remove unique_id | CH/HIST migration gap → column-count mismatch | ch_migration_v6 before hist_migration |
test_split_by_string_empty_guard[5_/__201/_/empty] |
Remove unique_id | splitByString('_', '5_') → toInt64('') throws |
Empty-part guard rejects malformed uids |
test_dead_columns_not_in_current_set |
Pipeline step Seg+ABC | Dead columns still referenced in build SQL | Disjoint from current columns |
test_backtesting_renamed_from_best_method |
Audit/standardize params | best_method_parameter_id not renamed |
backtesting_parameter_id in current set |
test_ch_alter_add_column_if_not_exists |
Audit/standardize params | CH PIPE_preprocessed missing ADD COLUMN migration |
New columns added to existing set |
test_dynamic_column_detection_handles_both_schemas |
Audit/standardize params | scenario_id → pipeline_id column mismatch |
Detection picks whichever exists |
11. ETL / Import (3 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_truncate_replace_falls_back_to_upsert |
Making overrides pipeline-driven | step[f"sql_{mode}"] KeyErrors for truncate_replace |
Falls back to sql_upsert |
test_normal_mode_works |
Making overrides pipeline-driven | Same | Normal mode still works |
test_route_and_bom_are_separate_entries |
Making overrides pipeline-driven | "route" entry ran BOM SQL (copy-paste error) | Separate entries with correct targets |
12. Planning Date (3 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_planning_monday_is_monday |
Multiple | date.today() used instead of planning_today() |
planning_monday always returns weekday 0 |
test_planning_today_env_var_overrides |
Multiple | REPLAY_PLANNING_DATE not respected | Env var parsed and overrides |
test_date_today_violation_detection |
Multiple | Business logic using date.today() |
Pattern detection catches violations |
13. Pipeline Job Management (5 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_404_marks_job_as_interrupted |
API error explanation | 404 swallowed → infinite polling loop | 404 sets status='interrupted' |
test_is_live_job_returns_false_for_interrupted |
API error explanation | 8s window perpetuates loop after 404 fix | interrupted status → isLiveJob=false |
test_jobs_filtered_by_pipeline |
Method selection impact | Stale job cards from other pipelines | Jobs filtered by activePipelineId |
test_override_job_restored |
Process pipeline override | Overridden jobs not restored after reload | Pinned override pipeline accepted |
14. Feature Reliability / NaN Handling (3 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_clean_float_converts_nan |
Process pipeline override | NaN in JSON insert → PostgreSQL rejects | _clean_float converts NaN/inf → None |
test_json_dumps_nan_is_invalid_pg |
Process pipeline override | Same | json.dumps(NaN) produces NaN token (not valid JSON for PG) |
test_none_results_skipped |
Process pipeline override | _log_achievement_validation crashes on None results |
if result is None: continue |
15. Dask / Autoscaler (3 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_scale_floor_prevents_destructive_retire |
NeuralForecast compat | Stale 0-worker reading → cluster.scale(2) retires 12 workers |
Scale floor substitutes stale 0 |
test_real_nonzero_reading_trusted |
NeuralForecast compat | Same | Real non-zero reading overrides floor |
test_logger_available_before_sync |
Forecast cluster offline | self.logger used before assignment in __init__ |
Logger assigned before sync block |
16. IO Network / Distributions (3 tests)¶
| Test | Session | Bug | Regression Check |
|---|---|---|---|
test_cache_key_includes_pipeline_id |
IO tab demand discrepancy | Cache key omits pipeline_id → cross-pipeline collisions |
pipeline_id in cache key |
test_outer_select_cannot_reference_subquery_alias |
Supply network hover | SELECT f.item_id FROM (subquery AS f) — alias out of scope |
f. prefix removed on outer level |
test_ch_only_columns |
Supply network hover | CH SELECT queries ks_statistic/service_level_quantiles (PG-only) |
CH columns are subset of PG columns |
17–27. Misc (14 tests)¶
Covers: demand horizon limits, cascade delete, allocation view mode, preprocessed parameter table, seed safety, Mirabelle nav targets, concurrent index build, AbortController cleanup, reaper transaction commit, _pname unbound local, savepoint for seed transactions.
Running¶
Adding New Tests¶
- Find the relevant test class by functional area (or create a new one)
- Name the test method after the specific bug/edge case
- Include a docstring referencing the session title
- Keep tests pure-logic (no live DB/CH/HTTP); mark integration tests with
@pytest.mark.integration