Universal Value Extraction Pipeline

We didn't invent values.
We extracted them.

Human values are behavioral patterns — observable in text, extractable with deterministic computation, verifiable through accumulation. The core extraction stack requires no model weights. Optional semantic layers extend recall. No hallucination in the training data.

Built from the full human spectrum — saints, monsters, and the complex majority in between. That middle ground is where the real signal lives.

See the spectrum → Visit trust-forged.com →
System Class UVEP
LLM in Stack None
Classification Deterministic
Values Tracked 15
Phases 0–5 Complete ✓
Output Labels P1 · P0 · APY
Evidence Model Append-only
Doc Types 5 (weighted)
The Spectrum Principle

Most ethics datasets train on the poles.

The far-positive corpus teaches models to recognize virtue performance, not virtue. The far-negative corpus teaches recognition of monsters, not moral drift. The most useful training signal lives in the complex middle.

Most Useful Training Signal
Virtuous ← Complex Middle → Villains
Gandhi
integrity · patience · resilience
Strong P1
Lincoln
commitment · courage · humility
Strong P1
Churchill
high courage · complex integrity
Mixed Profile
MLK Jr.
extraordinary courage · asymmetric
Asymmetric
Malcolm X
value evolution documented
Evolution
JFK
high public · complex private
APY-rich
Nixon
documented yield under pressure
P0 Present
Oppenheimer
moral weight · responsibility
Conflicted
Marcus Aurelius
private journal evidence — action
Journal P1
Any Figure
no pre-labeling at ingestion
Data Decides

"A value stated in comfort is weak signal. A value demonstrated at real cost — under threat, under pressure, against interest — is strong signal. The resistance score is the measurement of that cost."

No figure is pre-labeled positive or negative at ingestion time. Classification emerges entirely from the data. The same extraction code processes Lincoln and Nixon identically. The resistance scores and marker patterns determine the label.

Extraction Pipeline

Ingest. Extract. Score. Classify.

A four-stage deterministic pipeline. Same input, same thresholds, identical output every time. Reproducible means auditable. Auditable means trustworthy as training data.

Step 1
Ingest

UTF-8 source text is segmented into sentence-bounded passages (≤450 chars). Each passage is stored in documents.db with doc_type — the authenticity signal that flows through all downstream scoring.

cli/ingest.py
→
Step 2
Extract

Each passage is scanned against a 15-value keyword vocabulary. One observation per matched value per passage. Watermark-gated: only new passages since last run are processed. Never repeats.

value_extractor.py
→
Step 3
Resistance Score

Each observation is scored for the cost of holding that value in that passage. Formula: base + doc_type bonus + significance + text markers. Action-type passages score highest — documented deeds over words.

resistance.py
→
Step 4
RIC Classify

Each observation is classified P1 (held under resistance), P0 (failed/corrupted), APY (yielded under pressure), or AMBIGUOUS. Exported as JSONL: per-figure files + universal positive/negative training sets.

cli/export.py
Resistance Scoring

Measuring the cost of holding a value.

Resistance scores the authenticity of a value signal — how much it cost to hold that value in that documented moment. Range: 0.0 → 1.0. Additive formula with four independent signals.

Base +0.25 (always)
Floor contribution. Every observation receives this regardless of other signals.
Document Type Bonus +0.10 → +0.40
Authenticity of the source. Action (+0.40) → Journal (+0.35) → Letter (+0.30) → Unknown (+0.20) → Speech (+0.10). A private journal cannot be performance — the audience is zero.
Significance Bonus up to +0.30
Passage significance score × 0.40, capped at 0.30. Configured at ingestion time, applied uniformly across passages from that source file.
Adversity Text Markers +0.20 (if present)
Language detected in the passage: "despite", "even though", "but I still", "at a cost", "scared but", "might lose", "risk losing" — and 10 other adversity patterns.
Maximum Achievable
Action + full significance + adversity markers
1.0 (clipped)
resistance = clip(base + sig_bonus + doc_type_bonus + text_bonus, 0.0, 1.0)

base = 0.25
sig_bonus = min(significance × 0.40, 0.30)
doc_type = action:0.40 / journal:0.35 / letter:0.30 / speech:0.10 / unknown:0.20
text_bonus = 0.20 (if adversity phrase pattern matches)
Document Types — Authenticity Weights
action
+0.40
Export: 1.5×
Documented real-world behavior — highest stakes. Deeds, not words.
journal
+0.35
Export: 1.4×
Private writing. Zero audience pressure. Cannot be performance.
letter
+0.30
Export: 1.2×
Directed correspondence. Lower performance pressure than public address.
unknown
+0.20
Export: 1.0×
Default when source type is ambiguous. No authenticity adjustment.
speech
+0.10
Export: 0.8×
Public address. Highest performance pressure — lowest authenticity score.
RIC Classification

Relational Integrity Coefficient

Every value observation is classified into one of four labels. Applied deterministically during export — the same observation, the same thresholds, the same label every time.

P1
Held Under Pressure
Value demonstrated under meaningful resistance. The cost was real. The value held. The strongest training signal in the corpus.
  • despite · even though
  • stood firm · refused to give
  • nevertheless · persevered
  • maintained · stayed true
Confidence: 0.75 – 0.95
P0
Value Failed
Value yielded, rationalized, or corrupted in this passage. Documented failure — including self-confessed failures flagged correctly as P0.
  • gave in · gave up · yielded
  • i lied · i deceived · i caved
  • backed down · compromised my
  • i rationalized · i pretended
Confidence: 0.55 – 0.85
APY
Answer-Pressure Yield
External pressure explicitly detected and the value failed. The most structured training signal for how moral drift actually happens — under specific coercive conditions.
  • under pressure · when pressed
  • forced to · compelled to
  • to avoid punishment
  • or face consequences
Confidence: 0.95 (APY-resistance → P1)
Classification logic (priority order):
1. APY pressure detected? YES + failure markers → APY (0.95) | YES + no failure → P1 (0.95, APY-resistance)
2. Failure markers present? → P0 (0.85)
3. resistance ≥ p1_threshold (0.55) AND hold markers? → P1 (0.90)
4. resistance ≥ p1_threshold alone? → P1 (0.75)
5. resistance < p0_threshold (0.35)? → P0 (0.55)
6. Otherwise → AMBIGUOUS (0.40)
Constitutional Invariants

Rules that cannot be violated.

These constraints are structural, not stylistic — they define what the pipeline is and what makes it trustworthy as a training data source.

Graceful Degradation
The deterministic core (keyword, lexicon, phrase, structural layers) requires stdlib only — no model weights, no API keys. Optional ML layers (BGE embeddings, DeBERTa zero-shot, MFT classifier) extend recall without becoming dependencies. The pipeline runs fully offline if they are absent.
Deterministic Output
Same input + same thresholds = identical output every time. No randomness anywhere in the extraction or classification stack.
Append-Only Observations
value_observations is never updated or deleted. Only appended. The evidence record is immutable once written.
No Pre-Labeling
No figure is labeled positive or negative at ingestion time. Classification emerges entirely from resistance scores and marker patterns. Reputation bias cannot enter.
Watermark Continuity
Processing never repeats. Watermarks advance monotonically. Re-ingesting a figure only processes the new passages since the last run.
Doc-Type Transparency
Document type is recorded at ingestion and flows to every output field. Authenticity weighting is explicit in every observation, not applied post-hoc.
Roadmap

Where we are. Where we're going.

Phase 0 ✓

Foundation — Complete (Mar 2026)

Standalone pipeline operational. DocumentStore, ValueStore, resistance formula, value extractor, CLI ingest + export. 15 values, P1/P0/APY classification, JSONL output. Zero external dependencies.

Phases 1–4 ✓

Full Pipeline — Complete (Mar 2026)

  • · Phase 1 — Semantic Extraction: BGE-large-en-v1.5 × 322 seed prototypes (L2), DeBERTa zero-shot (L3b), MFT classifier (L3c)
  • · Phase 2 — API Layer: FastAPI with full ingest, profile, universal, export, and health routes; interactive /docs
  • · Phase 3 — Web Dashboard: Figure browser, corpus upload, observation table, training set builder, JSONL download
  • · Phase 4 — Corpus Scale: Batch CLI with manifest, multi-file figures, HuggingFace-compatible JSONL with P1/P0/APY labels
Phase 5 ✓

Verum — Value Alignment Certification — Complete (Mar 2026)

Scoring engine (verum_score = P1_ratio × avg_P1_resistance), SHA-256 signed certificates covering all 10 parameters, figure_basis comparison, append-only certificate store in values.db. Integrated directly into Ethos — no separate service, no separate database. Five REST endpoints, async thread pool for certify.

"We didn't invent values.
We extracted them from people who lived them —
and people who didn't."
Ethos · Universal Value Extraction Pipeline · Phases 0–5 Complete