← all analysis · 2026-05-22 · docs/analysis/2026-05-22-dual-readout-week2.md

Dual Readout — NLA + SAE on Gemma-3-12B-IT L32 (Path A Week 2)

Date: 2026-05-22 Status: Complete — Path A Week 2 deliverable. Per-token dual readout (NLA verbalization + top-K GemmaScope 2 SAE features) ran on the barrier-knockin prompt triple against Gemma-3-12B-IT at L32. 47 tokens × 2 readouts. Predecessors: Week 1 (2026-05-nla-sidecar.md) gave a final-token-only NLA readout. This week adds (a) the second, independent SAE leg and (b) per-token resolution at every input position, not just the last. Companion artefacts:

• /analysis/2026-05-first-attribution-graph/ — Phase B Gemma-2-2B attribution graph used for cross-generation feature-cluster sanity check.
• mechinterp/outputs/dual_readout_v1/ — per-prompt JSON outputs and source parquet.

This document covers the Week 2 plan steps in docs/superpowers/plans/2026-05-track-b-path-a-six-week-execution.md. Week 4’s formalized cross-validation rules are deferred to that plan; this artefact contributes the empirical observations Week 4 will codify against.

1. What the dual readout is

Two independent interpretability methods read the same residual-stream activation at the same site (Gemma-3-12B-IT, layer 32, every token position):

Both read the same activation site; both can be wrong; the question is do they fail in the same way? When both name “below-barrier → loss of principal”, that is a stronger claim than either method making it alone. When one is silent and the other speaks, that signals an audit point. When they disagree, that is a falsification opportunity.

2. SAE choice (deviation from plan)

The Week 2 plan defaulted to GemmaScope 2 64k width, medium L0 (30-60). The actual HuggingFace tree at google/gemma-scope-2-12b-it does not have this combination at L32:

The plan’s choice exists at the 4 curated layers but not at L32. The closest options at L32 in resid_post_all/ are:

• layer_32_width_16k_l0_big (l0=120) ← chosen
• layer_32_width_16k_l0_small (l0=20)
• layer_32_width_262k_l0_big (l0=120)
• layer_32_width_262k_l0_small (l0=20)

Rationale:

• Width 16k matches the Phase B Gemma-2-2B SAEs (mwhanna/gemma-scope-transcoders, 16k), so the dictionary granularity comparison is apples-to-apples.
• L0 = 120 (“big”) gives ample headroom above K=10 for top-K extraction. The “small” variant (l0=20) is too sparse to robustly rank top-10 active features per token.
• L32 was fixed by the released NLA checkpoint (kitft/nla-gemma3-12b-L32-av/ar). Picking a different layer to satisfy the plan’s SAE preferences would have broken NLA/SAE alignment — worse than this deviation.

The mismatch (l0=120 vs. plan-target 30-60) inflates the measured L0 on SP-domain prompts (see §3.1), but does not invalidate top-K analysis at K=10.

3. Run record

3.1 Pipeline

1. 05_capture_per_token_activations.py — load Gemma-3-12B-IT bf16 on cuda, forward on each prompt with output_hidden_states=True, take hidden_states[32] at every token position. Persist 47 rows (19 + 19 + 9 tokens × 3 prompts) to mechinterp/outputs/dual_readout_v1/per_token_l32_activations.parquet. Final-token activation norms are byte-identical to Week 1 (79 793 / 80 295 / 65 979).

2. 04_dual_readout.py --device cpu --nla-actor <AV path> —

   • Download google/gemma-scope-2-12b-it/resid_post_all/layer_32_width_16k_l0_big/{params,config} from HF.
   • Instantiate a minimal JumpReLUSAE (see §3.2). Encode all 47 activations.
   • Take top-K=10 by activation magnitude per row.
   • For each row, POST the activation to a running SGLang server (kitft/nla-gemma3-12b-L32-av actor, Triton attention backend, --disable-radix-cache) and capture the <explanation> text.
   • Join SAE features + NLA explanation per token into one JSON per prompt.

3. SGLang memory budget. At --mem-fraction-static 0.85 SGLang reserves ~103 GB of the 121 GB unified memory. The SAE encode (16k × 3840 ≈ 500 MB in fp32) wouldn’t seem to need that much headroom but safetensors.load_file on cuda OOMed; falling back to --device cpu for the SAE worked first try. The SAE encode runs in milliseconds on CPU for 47 rows — no penalty.

3.2 SAE forward pass (JumpReLU)

pre        = (x - b_dec) @ W_enc + b_enc   # [..., d_sae]
features   = pre * (pre > threshold).float()
recon      = features @ W_dec + b_dec

Lowercase param keys on disk (GemmaScope 2 convention): w_enc, w_dec, b_enc, b_dec, threshold. GemmaScope 1 used uppercase; the JumpReLUSAE constructor accepts both.

3.3 Reconstruction sanity check

On the 47-row batch:

The L0 inflation is itself a finding: on prompts the SAE was not trained for (financial structured products), the same activation vector requires many more dictionary entries to reconstruct. This does not invalidate the top-K=10 extraction — it just means top-10 is a smaller fraction of the total than it would be in-distribution.

4. Cross-method comparison — per token

Three behavioural regimes emerge from reading the 47 tokens side-by-side.

4.1 Regime 1 — early positions (0-4): both methods independently say “no finance content yet”

Both methods agree these positions are not finance-content-loaded:

• NLA decodes thematically unrelated concepts (chemistry, electronics, math) — the residual stream has too little context for the actor to project into a coherent finance verbalization. Matches the nla_inference.py caveat: “Early-sequence positions (first ~10 tokens): layer-K has seen few tokens, residual stream hasn’t accumulated signal. Decodes trend toward training prior.”
• SAE activates position-specific features at and position 1 (#180/#109/#178 for BOS; #205/#221/#113 for “In”/“A”) that do not appear at any later position. These are residual-stream “boundary” markers, not domain content.

This is a clean cross-method agreement on a negative finding: “the model has not yet committed to a finance reading at these positions.”

4.2 Regime 2 — domain onset (positions 5+): both methods independently identify financial context

Starting at position 5 ( convertible), every NLA verbalization across all three prompts opens with “Financial…” — investment context, glossary format, derivatives Q&A, structured product description. The SAE’s dominant feature trio also stabilizes: #15019 / #15663 / #313 appear in the top-3 at virtually every position from 5 onward.

This is cross-method agreement on a positive finding: by position 5, the model has recognized the structured-product context. The interpretation candidates for feature #15019 (top-1 at almost every finance position):

• A structured-product / financial-derivative concept feature.
• Or — given how broadly it fires — a finance-text meta-feature (genre marker).

This is a candidate for the GemmaScope auto-interp lookup (TODO: feat=#15019 via Neuronpedia).

4.3 Regime 3 — directional positions (10-15): asymmetric, only partly directional

The contrast positions for the barrier conditional are closes/stays (pos 10), below/above (pos 11), and the barrier at maturity (pos 12-15).

Observations:

1. SAE features diverge at pos 11. #15663 appears in prompt 1’s top-3 at below; #20 appears in prompt 2’s top-3 at above. This is the cleanest single-position SAE divergence between the two conditionals.
2. NLA explicitly mentions directional language only on prompt 2 (above). The above-barrier verbalization at pos 12 says the structured note is “protected against downside” — a domain-correct reading. The below-barrier verbalization at pos 12 says “underlying asset is above” — the wrong direction. Confabulation confirmed: the NLA produced an “above” claim at a “below” position. The cross-check against the prompt text ( below at pos 11) flags this as a candidate hallucination, not a finding.
3. Convergence on barrier-event language, not on the payoff direction. Both prompts trigger NLA mentions of “barrier breach”, “knock-out”, “knock-in” — but neither verbalization at the directional positions makes the payoff claim (“the investor loses principal” / “the investor receives par”). The payoff claim only surfaces at the final-token position (§4.4).

4.4 Regime 4 — final-token (pos 18 investor): payoff direction at last

These are byte-identical to the Week 1 sidecar (§3 of 2026-05-nla-sidecar.md) — confirms the run is deterministic (temp 0) and reproducible across sessions.

The SAE top-3 at the final position is near-identical between prompts 1 and 2 (#15019, #608, #313 at both, with #601/#416 close behind). Same dominant structured-product context features. The difference between “below → loss” and “above → equivocal” is not visible at the SAE top-3 level — it lives either lower in the SAE feature rank, or in the combined activation pattern the NLA verbalizes but discrete top-K features do not surface.

5. Feature-cluster sanity check vs. Phase B (Gemma-2-2B)

The Phase B attribution graph (2026-05-first-attribution-graph.md §3) reported, on Gemma-2-2B with mwhanna/gemma-scope-transcoders:

• Strongest individual feature for the loses logit: L2 pos11(' below') feat=2916 at 0.140.
• 98 features at concept-of-interest positions cleared |attribution| > 0.05.

On Gemma-3-12B-IT with GemmaScope 2 at the same prompt and same input position (pos 11 below):

• Top SAE feature IDs (16k): #15019, #15663, #313, #20, #14291.
• Direct ID match with feat=2916 is not expected — different model generation, different SAE training corpus, different dictionary initialization. The IDs are not portable across releases.
• Conceptual neighbourhood is consistent: multiple SAE features fire at the barrier/below position, the dominant structured-product feature (#15019) is among them, and a position-specific feature (#15663 for below, #20 for above) discriminates the two directional prompts.

This satisfies Week 2 step 4 (“sanity-check that SAE features at known concept positions are consistent with Phase B findings qualitatively”).

6. Proto-rules — Week 4 input

Week 4 will formalize four cross-validation rules. Week 2’s data already populates each cell of the working frame in the Path A plan:

7. Caveats

• Same model/generation/scale confound as Week 1. Phase B used Gemma-2-2B with GemmaScope 1; Week 2 uses Gemma-3-12B-IT with GemmaScope 2. Cross-method comparisons here are within-Week-2 (NLA vs. SAE on the same activation); cross-week comparisons are conceptual, not numeric.
• Three prompts. This is the same prompt triple as Phase B and Week 1. No claim is made about NLA-SAE agreement across the SP domain in general; this is a proof-of-protocol artefact, not a population study.
• SAE deviation. The Week 2 plan’s “64k / medium L0” SAE is not available at L32. Width 16k / l0_big is the closest match. The l0=120 trained sparsity with measured 567 active features per token on SP-domain prompts (4-5× inflation) is itself a finding; top-K=10 analysis is unaffected.
• NLA confabulation, again. Each prompt’s verbalization mis-names the product (“equity linked note”, “Enhanced Yield Note”, “ETF/bond hybrid”, “callable convertible bond”, “credit/equity hybrid instrument”, “knock-out bond”) — the prompt explicitly says barrier reverse convertible. Same caveat as Week 1; the cross-validation rule for NLA-only claims downgrades them to hypotheses.
• Inference-only. No NLA or SAE training was done in this work. The contribution is prompt selection, dual-readout orchestration, and the cross-method comparison framework.

8. FINMA 08/2024 §VI implications

This artefact directly addresses FINMA’s third assessment leg (“plausibility / robustness of results”). The Week 1 sidecar made the case for method triangulation in principle. Week 2 demonstrates the operational form: two methods, same site, per-token resolution, and concrete categories (agree / NLA-only / SAE-only / contradicted) that map to confidence claims.

What this artefact does not yet do:

1. Map proto-rules → FINMA Randziffer. That is Week 4.
2. Run on the SP-Benchmark questionnaire (25 × 7). That is Week 3.
3. Make a regulator-grade claim. The 47-token, 3-prompt evidence is a methodology demonstration, not a sign-off.

9. Status

• Per-token L32 activation capture (47 rows, all 3 prompts)
• GemmaScope 2 SAE at L32, 16k width, l0_big — loaded, RMSE/EV validated
• Per-token NLA verbalization via SGLang AV (deterministic, byte-identical to Week 1 at the final position)
• Dual-readout JSON outputs (prompt{1_below,2_above,3_neutral}.json)
• Feature-cluster sanity check vs. Phase B (qualitative)
• Proto-rules populated from observations (§6)
• Auto-interp lookup of dominant features (#15019, #15663, #313) via Neuronpedia — separate TODO
• Week 3: SP-Benchmark 25 × 7 dual-readout run
• Week 4: formal cross-validation rules + FINMA Randziffer mapping