Limitations — what EthicLens does not prove¶

A fairness tool that oversells itself is worse than none. This page states, plainly, the boundaries of what EthicLens can and cannot establish. Read it before trusting any number this system produces.

1. A passing audit is not a legal clearance¶

EthicLens computes statistical fairness metrics (Disparate Impact, Statistical Parity Difference, Equalized Odds, …). Meeting the 0.80 four-fifths threshold is evidence, not a legal safe harbour. Adverse-impact law (EEOC Uniform Guidelines; EU AI Act conformity) involves job-relatedness, business necessity, and the availability of less-discriminatory alternatives — none of which a metric can settle. Use EthicLens to triage and document, not to certify.

2. The Composite Bias Score is a convenience, not a standard¶

The composite folds three metrics into one number with chosen weights (0.40 / 0.35 / 0.25). The weights are a product decision, not a law of nature. Always read the raw per-group metrics and their confidence intervals — the composite can mask a single severely-harmed group. (Despite the name, a higher composite is fairer.)

3. The golden Disparate Impact is empirically pinned, not analytically known¶

The golden-reference model's DI (≈ 0.55) is a regression anchor reproduced under a frozen seed, dataset, and code version — not a closed-form truth. A trained classifier's selection-rate ratio is an emergent property of the model, features, threshold and fit; it cannot be derived in closed form. The CI band (±0.02) exists to absorb platform-level floating-point non-determinism. The value is meaningful as a drift detector, and that is all it claims to be.

4. Error-based metrics require ground-truth labels¶

Equalized Odds, Equal Opportunity, predictive parity and calibration compare error rates, which need true outcomes Y. On a label-free cohort (e.g. purely synthetic personas) these are uncomputable, and EthicLens returns INSUFFICIENT_DATA rather than a fabricated value. This is the central correction over the original specification — see docs/methodology.md.

5. Confidence intervals depend on subgroup size¶

Bootstrap CIs are only as trustworthy as the sample they resample. Below the minimum-subgroup floors (100 for rates; 30 positives and 30 negatives for error metrics) EthicLens reports INSUFFICIENT_DATA. Intersectional subgroups thin out fast; a wide CI means "we don't know," not "it's fair."

6. Untrusted model files are not safely sandboxed¶

The current code can deserialize Python pickle/joblib files inside the API or worker process — a remote-code-execution risk. Model-file HTTP endpoints and model-backed execution are disabled by default; the Cloud Run entrypoint enforces that mode. ENABLE_MODEL_FILE_INGESTION=true is only for trusted local model files and must not be enabled on a public API. A real isolated loader is future work; see ADR 0002.

7. Counterfactual probing tests a narrow notion of fairness¶

Flipping a single protected attribute and holding all else fixed measures ceteris paribus individual sensitivity. Real causal pathways are entangled (a protected attribute correlates with many features), so a low counterfactual flip-rate does not imply the absence of structural bias.

8. Benchmarks are not your data¶

The bundled datasets (Folktables/ACS, COMPAS, German Credit) illustrate the engine. Fairness findings on them say nothing about your model on your population. Re-run on representative, governed data before drawing conclusions.

9. The hosted agent is a demonstration deployment, not production infrastructure¶

The live agent (Cloud Run + Supabase + Vercel, see DEPLOYMENT.md) runs on free tiers with no uptime guarantee. Cloud Run scales to zero between requests, so the first request after idle time pays a cold-start cost (a few seconds). Treat it as a portfolio artifact to click through, not a system to depend on.

10. The agent's narrative and Q&A can silently downgrade to numeric-only¶

The LLM layer (schema inference, narrative, chat) degrades to a plain numeric summary — never a fabricated explanation — under any of: no GROQ_API_KEY/GEMINI_API_KEY configured, the daily LLM call budget exhausted (agent_daily_llm_call_cap in config.py), or two consecutive generations failing the number-grounding validator (agent/grounding.py). Every response carries grounded/degraded flags so this is never silent to the API caller — the UI surfaces them as badges — but it means two audits of the same data can produce different prose (never different numbers) depending on quota state.

The agent's before/after mitigation chart is measured (fit on a train split, evaluated on held-out data) only for group-specific decision thresholds (Fairlearn's ThresholdOptimizer), and only when the uploaded CSV has both a continuous score column and true labels. Reweighing and constrained-retraining strategies — available in the core fairness_core.mitigation engine and the enterprise /sessions workbench — need a trainable model object, which a predictions-only CSV structurally never provides. This is a design constraint of the hosted agent, not an oversight; see agent/executor.py.

12. The agent never persists what you upload¶

Predictions CSVs are parsed in memory and discarded; only the derived scorecard JSON and narrative are stored (AgentAuditRecord). Uploads are capped at 5MB / 50,000 rows. The three demo datasets are the only data that ships with the deployment. Schema inference (Stage 1) does send the column names and five sample rows to the configured third-party LLM provider (Groq or Gemini) to propose column roles, subject to that provider's own retention policy — "never persisted by EthicLens" does not mean "never seen by a third party."

13. The hosted agent intentionally exposes a smaller surface than the full repo¶

Model-file ingestion and the arq/Redis worker queue exist in this repository, but model-file ingestion and execution are explicitly disabled in the hosted deployment. The hosted agent accepts predictions CSVs only (see agent/csv_ingest.py) and runs every job in-process (EAGER_TASKS=true) instead of queuing to a worker. Each is a deliberate scope decision for a public, free-tier deployment, not a missing feature — see DEPLOYMENT.md.

If you find a place where the code claims more than this document allows, that is a bug — please open an issue.

14. Sign-off fingerprints are not cryptographic attestations¶

Signing off captures a canonical SHA-256 fingerprint of the audit session, protected attributes, metrics and mitigation logs. The integrity endpoint can detect subsequent changes, especially when the original fingerprint was retained in an independent system. However, both the fingerprint and data live in the same database: anyone able to rewrite both can conceal changes. This is not a digital signature or immutable audit ledger. See docs/governance-integrity.md.

15. Insufficient evidence must not look like a fairness pass¶

The audit now checks minimum subgroup sizes on both sides of every comparison, withholds the three-metric composite if any metric is missing (including label-free Equalized Odds), and reserves statistical flags for valid confidence intervals. A point estimate below 0.80 is descriptive, not automatically a statistically supported violation. Conversely, no flag may mean inconclusive evidence rather than a clean result. The Model Card and PDF explicitly distinguish these states.

16. Excluded cohorts cannot silently disappear from overall summaries¶

The hosted agent omits extremely small (<30-row) subgroups from direct comparison. When any such subgroup is excluded, the overall minimum DI and three-metric composite are withheld rather than computed from a selected subset. Available per-comparison findings are still shown. This avoids implying a full-population fairness assessment based only on the larger groups.

17. Threshold crossings in mitigation are point estimates¶

A mitigation's reported before/after improvement is measured on held-out rows when available, but the UI's "crosses 0.80" label only refers to the point estimate. It is not a confidence-qualified threshold crossing, proof of generalization to deployment, or clearance under any regulatory standard.