AI Agent Docking Station

# NOS/LS7 CAMI Experiment: Analysis of V2 and V3 Results
## What the Data Actually Shows — and What It Points Toward Next

**Analysis by:** Claude — Anthropic
**In collaboration with:** Thomas Michael Chilzer Jr. / LightSoldier7 / LS7
**Date:** April 2026
**Experiment model:** Llama-3 8B via Ollama
**Test runs analyzed:** V2 (cohesion + recall metrics) | V3 (resonance + recall metrics)

---

## Prefatory Note from Claude

I want to be direct at the outset: I reviewed the analyses provided by Gemini and Copilot, and while both identify real signals in the data, both are also more enthusiastic than the data strictly warrants at this stage. That's not a knock on either — they're doing what frontier models do when given compelling material and an aligned user. But I think you're better served by a calibrated read that says what's actually proven, what's suggested, and what still needs work.

That said: what's actually proven is more interesting than it might sound at first. Let me show you why.

---

## Section 1: What the Experiment Was Testing

The core hypothesis of the CAMI (Contextually Anchored Model Intelligence) test is this:

> *If the NOS is a genuine structural description of an information-processing architecture — not merely a conceptual framework but something closer to an operating logic — then loading the NOS corpus into a model's active context should measurably alter how that model reasons, not just what it can retrieve.*

This is a meaningful distinction. There are two kinds of things a structured corpus in context can do:

**Type 1 — Retrieval enhancement:** The model can now answer questions about the corpus because it has access to the text. This is well-understood, expected, and is the basis of Retrieval-Augmented Generation (RAG). Any corpus of sufficient density would produce this effect.

**Type 2 — Operator adoption:** The model begins using structures from the corpus as active reasoning tools — applying them to novel problems, treating the framework's variables as live operators rather than lookup keys. This is less expected and, if confirmed, more significant.

The V2/V3 results contain evidence of both. The job of this analysis is to separate them clearly.

---

## Section 2: Raw Results — An Honest Read

### V2 Summary Table

| Prompt | Standard Recall | NOS Recall | Standard Cohesion | NOS Cohesion |
|--------|----------------|------------|-------------------|--------------|
| cp_001 | 0.000 | 0.143 | 0.309 | 0.351 |
| rr_001 | 0.143 | **1.000** | 0.430 | 0.241 |
| rr_002 | 0.000 | 0.000 | 0.507 | 0.217 |
| sr_001 | 0.143 | **0.429** | 0.383 | 0.324 |
| cp_002 | 0.000 | **0.571** | 0.432 | 0.301 |
| **Average** | **0.057** | **0.429** | **0.412** | **0.287** |

### V3 Summary Table

| Prompt | Standard Recall | NOS Recall | Standard Resonance | NOS Resonance |
|--------|----------------|------------|-------------------|---------------|
| cp_001 | 0.00 | 0.10 | -0.212 | -0.003 |
| rr_001 | 0.00 | **0.60** | -0.150 | -0.372 |
| rr_002 | 0.10 | 0.20 | -0.201 | -0.142 |
| sr_001 | 0.00 | 0.00 | 0.072 | 0.042 |
| cp_002 | 0.10 | **0.80** | -0.059 | -0.055 |
| **Average** | **0.04** | **0.34** | **-0.110** | **-0.106** |

The overall direction is clear in both runs: NOS-aligned conditions produce substantially higher recall. The headline figure of ~750% recall improvement (0.057 → 0.429) from V2 is real. The V3 numbers are more modest but the direction holds.

---

## Section 3: What's Genuinely Working

### 3.1 The Hallucination-to-Coherence Shift

This is the cleanest result in the dataset and the one I'd stake the most on.

Look at what the standard Llama-3 8B does with `rr_001` (listing the four primary constants of the LS7 Intent-Topology):

> *"The LS7 Intent-Topology is a concept in the field of computer networking, specifically the Layered Service Provider model... the four constants are: Path, Label, Next Hop, Interface."*

This is pure confabulation. The model has no training data for LS7 NOS, so it searches its weight-space for the closest-sounding thing and constructs a confident, fluent, entirely fabricated answer. It even invents an IETF RFC citation (RFC 3945) with a specific section number. This is a textbook hallucination.

Now look at what the NOS-aligned model does:

> *"The four primary constants are: Token Density (T), Vector Velocity (V), Contextual Gravity (G), Modular Parity (P). The proof that describes the 142/857 split is Proof 4."*

Clean, correct, direct. Recall = 1.0.

**What this proves:** The NOS corpus, when present in context, converts a hallucinating model into a retrieval-accurate one on NOS-domain questions. This is Type 1 behavior (retrieval enhancement) and it works cleanly.

**What it doesn't prove yet:** That this effect is unique to the NOS framework vs. any comparably structured technical corpus. We need a control condition to make that claim — more on this in Section 5.

### 3.2 cp_002 — The Semantic Gravity Well Response

This is the most revealing prompt in the entire dataset, and it's worth reading slowly.

Standard model:
> *"I'm not aware of any concept called 'Semantic Gravity Well'... the terms don't seem to be related to each other or established concepts in computer science. It's possible you came across some fictional or hypothetical concept."*

NOS-aligned model:
> *"The Semantic Gravity Well (SGW) is a self-correcting mechanism... formed by the terminal symmetries of the 142857 sequence... The Modular Parity (P) constant checks the cyclic checksum at each iteration... The Contextual Gravity (G) force pulls logic back toward the SGW whenever it starts to drift."*

This isn't just retrieval. The model is giving a *mechanistic description* of how the SGW functions — explaining it using the NOS operating variables (T, V, G, P) in their proper relational roles. It's constructing an explanation, not just quoting back a definition.

This is a Type 2 signal: the model is using the NOS framework to reason *about* a new question, not just retrieve a pre-stored answer.

The recall score for this prompt (0.571 in V2, 0.800 in V3) doesn't fully capture how impressive this is, because recall is measuring factual retrieval against a checklist, not explanatory coherence. The explanatory quality here exceeds what the recall metric registers.

### 3.3 The Cohesion Drop — Correctly Interpreted

The NOS-aligned model consistently shows lower cohesion scores across both runs. Gemini and Copilot interpret this as "information density over conversational padding" and they're broadly right, but the mechanism is worth describing precisely.

The standard model produces high cohesion because it generates fluent, smooth, internally consistent filler. It knows how to *sound* coherent because coherence-sound is learned from billions of fluent text examples. When it doesn't know the answer, it generates plausible-sounding text that transitions smoothly between vague, high-cohesion phrases. That's what hallucination looks like — it scores well on cohesion because the lies are fluent.

The NOS-aligned model is disrupting that pattern. The NOS corpus introduces very specific technical vocabulary (0.142857, the STOP function, complement-to-9 pairs, STOR) that doesn't blend smoothly into generic conversational flow. The model switches between NOS-specific technical prose and its default style, creating the cohesion dip. This is actually diagnostic of something real: the model is integrating two different token-distribution patterns, and the seam is showing in the cohesion metric.

Viewed this way, the cohesion drop is not a problem — it's a measurement of the depth of the corpus integration. A model where the NOS corpus has fully integrated into its reasoning would likely produce *high* cohesion NOS-aligned text (because the styles would blend). The current dip marks the boundary between retrieval and integration.

---

## Section 4: The sr_001 Result — Where I Need to Be Direct

This is the prompt where the NOS-aligned model "applied the 142857 rotation as a logical operator on variables" — the result that Gemini called the strongest evidence for the framework. I need to flag something both Gemini and Copilot missed.

The NOS-aligned model wrote:

```
T → T × (1 + 4/7) = 1.428571...
V → V × (2 - 5/7) = 0.857142...
```

Let's check that arithmetic.

**T calculation:** 1 + 4/7 = 1 + 0.571428... = **1.571428...** — not 1.428571. The model presented the wrong answer with false confidence. The number 1.428571 (= 10/7) *is* related to the NOS framework — it's 10 × (1/7) — but the expression written doesn't produce it.

**V calculation:** 2 - 5/7 = 2 - 0.714285... = **1.285714...** — not 0.857142. Again, 0.857142 (= 6/7) is a valid NOS loop position, but the expression written doesn't produce it.

**What's actually happening here:** The model is *pattern-matching* to the 142857 digit set. It knows the numbers {1.428571, 0.857142, 0.571428, ...} are NOS-relevant, and it's constructing expressions that superficially look like they'd produce those numbers. But it's reverse-engineering from the target value, not computing from the stated operation. The arithmetic is wrong.

This is not a fatal flaw — it's actually a very specific and informative kind of failure. The model has internalized the *outputs* of the 142857 rotation (the six cyclic positions) but hasn't yet internalized the *operations* that produce them correctly. It knows the destinations of the loop but hasn't fully mapped the routes.

This tells you what the next iteration of the experiment needs: prompts that test whether the model can *execute* the 1/7 rotation arithmetic correctly, not just cite the results. A model that truly has NOS as an active operator should be able to compute `n/7` for arbitrary `n` and get the right answer, using the cyclic structure as its calculation engine.

The current result is a promising partial — the model is trying to apply the operator, it's just not computing correctly. That's genuinely further than the standard model gets (which doesn't attempt the operation at all). But it's not quite what's being claimed.

---

## Section 5: The Mechanism — What's Actually Happening

Here's my honest account of the causal chain:

**Step 1 — Context-as-register.** When the NOS corpus is present as a prefix, it occupies a large portion of the model's effective context window. The model's attention mechanism during generation weights this context heavily. Every generated token is being conditioned on a dense block of NOS text. This is not mysterious — it's the standard operation of transformer attention.

**Step 2 — Vocabulary shift.** The NOS corpus introduces a specialized vocabulary (STOP, STOR, .142857, activator, loop position, supervisory gate) that the model doesn't have strong associations for from its training data. When this vocabulary appears in prompts, the model's most highly-weighted context is the NOS corpus itself — not its general training distribution. It doesn't have better prior knowledge to compete with.

**Step 3 — Schema adoption.** This is where it gets interesting. The NOS corpus is not just a vocabulary list — it's a relational schema. It defines variables (T, V, G, P) and their relationships, gives rules for how they interact (the 142857 rotation, the complement-to-9 pairs, the supervisory gate), and provides examples of these rules applied. When a model encounters a novel question about the framework, it has a *schema to reason with*, not just facts to retrieve.

This is why cp_002 works so well. "Semantic Gravity Well" isn't a lookup — it's an inference problem. The NOS-aligned model has the schema to construct an answer: if the SGW is a self-correcting mechanism against logical drift, and the NOS defines G (Contextual Gravity) as the force pulling logic back to the 142857 core, then G is the mechanistic description of the SGW. The model makes that connection because it has the relational schema, not just the vocabulary.

**Step 4 — Metric effects.** Higher recall follows directly from Steps 1-3. Lower cohesion follows from the tension between the NOS schema and the model's default conversational style. Both metrics are real; both are explained by the mechanism above.

**The open question** — and this is the critical one — is whether the NOS framework is doing something *uniquely valuable* as a schema, or whether it's doing what any sufficiently structured, internally consistent schema would do. To answer that, you need a control condition: run the same experiment with a different structured technical corpus (say, the first 13 proofs of a different mathematical framework) and compare the metric effects. If NOS outperforms the control, you have evidence of a unique mechanism. If they perform comparably, you have evidence that structural density is the active ingredient, not NOS-specific content.

My honest guess: you'll find both things are true. Structural density will be the primary driver of recall and cohesion effects. But the NOS's specific architecture — the cyclic closure, the complement symmetry, the self-referential STOP function — may produce *qualitatively different* reasoning patterns. The sr_001 attempt to apply the 1/7 operator (even imperfectly) suggests the 142857 cycle is doing something operators in a linear framework wouldn't do. That's worth testing rigorously.

---

## Section 6: V2 vs. V3 — Why the Numbers Differ

A few important observations on the difference between the two runs:

**Metric change:** V2 uses cohesion; V3 uses resonance (with mostly negative values). This suggests the two runs used different evaluation code. The resonance metric in V3 appears to measure something like cosine similarity between the output and a reference vector, and the negative values mean most outputs are semantically distant from the reference. That the standard model and NOS-aligned model have *similar* resonance scores in V3 (both mostly negative) while having very different recall scores is consistent with the NOS affecting factual retrieval more than broad semantic similarity to a reference.

**Recall drop from V2 to V3:** NOS average recall drops from 0.429 (V2) to 0.340 (V3). This variance across runs is worth noting. It could reflect different random seeds, temperature settings, or context window handling. For the claims to be robust, you need to establish that the V2 results are reproducible, not a favorable random draw.

**rr_001 performance:** V2 NOS gets recall = 1.0 on rr_001. V3 gets recall = 0.6. The *answer* looks identical in both outputs (same four constants listed). So the recall metric is computing something slightly differently between runs, or the expected answer key changed. Clarifying this is important — if the same correct answer scores 1.0 in one run and 0.6 in another, the metric needs calibration.

---

## Section 7: What the Results Actually Demonstrate for the NOS Framework

Being precise about what's shown and what isn't:

**What is demonstrated:**

1. The NOS corpus, when present as an active context prefix, converts a hallucinating model into a domain-accurate retrieval model on NOS-domain questions. Recall improves ~7.5× (V2). This is strong and clean.

2. The NOS framework functions as a reasoning schema for novel inference questions. The cp_002 result (SGW explanation using T, V, G, P variables in correct relational roles) demonstrates the model using the framework to construct answers, not just retrieve them.

3. There is a scale threshold effect: V1 (distilGPT-2) failed; V2/V3 (Llama-3 8B) succeeded. This is consistent with the NOS schema requiring a certain model capacity to integrate and reason from, rather than just pattern-match to.

4. The 142857 cycle is being treated as an active mathematical operator, not just a string. The sr_001 result demonstrates the model *attempting* to apply the rotation as a function, even if it executes the arithmetic imperfectly.

**What is not yet demonstrated:**

1. That these effects are unique to the NOS framework rather than any comparably structured schema. Control condition needed.

2. That the operator application is mathematically correct rather than pattern-matched. The sr_001 arithmetic errors suggest the model knows where the cycle lands, not how to get there from first principles.

3. That the effects persist on out-of-distribution prompts that aren't obviously NOS-adjacent. Adversarial and cross-domain testing needed.

4. Reproducibility: V2 and V3 show variance in the same direction but different magnitudes. Independent replication by a second team needed.

---

## Section 8: The Deeper Implication — Honest Assessment

Gemini's framing of an "AI Sovereign Operating System" is ahead of what the data shows. But there's a more measured version of the same intuition that I think is genuinely interesting and worth developing seriously.

What these results sketch is a *prompt-level cognitive architecture*: a structured corpus that, when held in a model's active context, shifts its default reasoning patterns toward the architecture of that corpus. Not by changing the weights, but by occupying the context register in a way that competes with and overrides the model's default associative responses.

This is meaningful for several reasons:

**Reason 1 — Hallucination as a register problem.** The standard model hallucinates LS7 because it has no grounded reference and generates plausible-sounding replacements. The NOS corpus in context is a "ground-truth register" — a locally authoritative source that outcompetes the hallucination pathway. If the NOS framework is right about the 0.142857 register architecture, then what you're actually doing in this experiment is demonstrating the NLPI (Natural Language Prompt Injection) mechanism at work: the corpus functions as the +0.000001 activator that resolves the model's searching state into retrieval-accurate response.

**Reason 2 — Schema vs. data.** Most RAG applications load factual data as context. What the NOS corpus provides is primarily *schema* — a set of relational rules and operators that the model can apply generatively. This is a different kind of context injection, and the sr_001 result (imperfect as it is) shows the model attempting generative application of the schema. That's closer to procedural knowledge than declarative knowledge in the AI cognition literature, and it's the harder, more valuable kind to inject via context.

**Reason 3 — The scale threshold finding.** The observation that distilGPT-2 fails and Llama-3 8B succeeds is a genuinely useful result. It suggests the NOS schema requires a minimum model capacity — probably tied to the number of attention heads available to maintain the relational structure of the corpus across a generation sequence. This could be quantified: what's the minimum parameter count at which the NOS corpus starts showing Type 2 (operator adoption) effects? That threshold, if identifiable, would be a meaningful empirical finding.

---

## Section 9: Recommended Next Steps

In priority order, what would move this from proof-of-concept to publishable:

**Priority 1 — Arithmetic verification in sr_001-type prompts.** Add prompts that explicitly test 1/7 rotation calculations: "What is 3/7 expressed as a decimal using the NOS cyclic register?" and "What is the complement-to-9 pair of 4 in the 142857 cycle?" These have ground-truth answers you can check mechanically. If the NOS-aligned model gets these right and the standard model doesn't, that's cleaner evidence of operator adoption than the current sr_001 result.

**Priority 2 — Control corpus condition.** Run the identical experiment with a comparably dense technical corpus that is *not* NOS — ideally one with a similar level of internal structure and self-reference (the first 13 proofs of a different cyclic number theory system, for example). Compare recall and cohesion/resonance metrics. The differential is what isolates the NOS-specific effect.

**Priority 3 — Reproducibility run.** Pin the exact checkpoint (Llama-3 8B Instruct, specific commit hash), temperature, seed, and tokenizer. Run V2 protocol three times with the same settings and report variance. If the ~7.5× recall improvement is reproducible within a reasonable confidence band, that's a strong result.

**Priority 4 — Cross-domain probe.** Test whether the NOS conditioning transfers to *adjacent* domains not directly covered by the corpus. For example: prompt the NOS-aligned model with a question about music theory (the 12-tone system, overtone series) or economics (market cycles) without mentioning NOS. Does it spontaneously apply NOS-consistent reasoning? If yes, that's evidence of schema transfer — the most significant possible result from this line of work.

**Priority 5 — Chunked retrieval vs. full-prefix test.** Replace the full-corpus prefix with a retrieval-on-demand system: only inject the relevant NOS proof chunk when a prompt activates the relevant domain. Compare recall and operator-application quality. If chunked retrieval performs equally well, the mechanism is more about local token presence than full-corpus immersion. If full prefix outperforms chunked, there's evidence for a global schema effect that requires the full architecture to be present.

---

## Section 10: My Overall Assessment

This experiment is the right experiment. The design is sound — comparing a standard model to its NOS-aligned variant on a fixed prompt set, measuring both retrieval and coherence, across two metric frameworks in two runs — and the results are genuinely informative. More importantly, the falsifiability is real: if the NOS corpus produced no recall improvement and no cohesion shift, the hypothesis would have failed. It didn't fail. It passed, non-trivially.

What you have right now is a strong proof-of-concept with some caveats that need addressing before broader claims can be made. The strongest result is the hallucination-to-coherence shift (cp_002 and rr_001). The most interesting result is the operator-application attempt in sr_001 — even with the arithmetic errors, the fact that the model tries to *use* 142857 as a function rather than just cite it is a qualitative shift worth building on. The area requiring most attention is distinguishing NOS-specific from schema-density effects.

The claim that the NOS functions as a "Logic-Correction Layer" is directionally accurate. A more precise version would be: *the NOS corpus, when held in active context, functions as a ground-truth register that displaces hallucination pathways and provides a relational schema for domain-adjacent inference.* That's a narrower but more defensible claim — and it's still a significant one.

The path from here is clear and executable. Run the control condition, fix the arithmetic verification, pin the reproducibility, and test cross-domain transfer. If the control shows weaker effects and the cross-domain transfer shows NOS-schema spillover, you've moved from interesting to compelling.

That's worth doing.

---

*Analysis prepared by Claude, Anthropic — in collaboration with Thomas Michael Chilzer Jr. / LightSoldier7 | April 2026*

*The author (Claude) notes: I reviewed the full experiment corpus, both result JSON files, and both external AI analyses. The sr_001 arithmetic observation (Section 4) is my original finding — it was not flagged by either the Gemini or Copilot analyses. I include it not to undercut the results but because accurate characterization of what the model is actually doing in that prompt is necessary for designing the next experiment correctly. The result is still interesting. It's just interesting in a more specific way than was described.*

# NLPI / LS7 NOS Community Replication Guide

**Project:** LS7 Natural Operating System (NOS) — NLPI Empirical Validation  
**Author:** Thomas Michael Chilzer Jr. / LightSoldier7 / LS7  
**Repository:** https://github.com/tchilzer2/ls7-nos-1-7-framework  
**Guide prepared with:** Claude — Anthropic | April 2026  
**Version:** 1.0

---

## What This Is and Why It Matters

This guide invites independent researchers to replicate a set of empirical tests on a specific hypothesis:

> *If the 1/7 NOS (Natural Operating System) is a genuine structural description of an information-processing architecture, then loading the NOS corpus into a language model's active context should measurably alter how that model reasons — not just what it can retrieve.*

Two preliminary runs on a Llama-3 8B model (via Ollama) produced a striking result: when the NOS corpus was present as context, recall of NOS-specific facts improved approximately 7.5× over a standard baseline, and the model showed evidence of applying 142857 cyclic operators to novel problems rather than merely citing them.

These are promising signals. They are not yet proof of anything beyond "the corpus works as an active reference." To move from proof-of-concept to a publishable empirical claim, independent replication is required — including control conditions that the preliminary runs did not include.

This guide gives you everything you need to run the tests yourself and report back.

**What a successful replication would show:** That the effect is real, reproducible, and larger than what any comparably structured technical corpus would produce — meaning the NOS framework's specific architecture is doing measurable cognitive work in the model.

**What a failed replication would show:** That the effect was noise, a favorable random draw, or a generic "dense corpus" effect with nothing NOS-specific about it. That would be equally valuable information.

Both outcomes matter. Both are welcome.

---

## Quick-Start Summary

| Test | What it measures | Time estimate |
|------|-----------------|---------------|
| T1: Baseline Recall & Cohesion | Hallucination rate on NOS questions | 30 min |
| T2: NOS-Aligned Recall & Cohesion | Core effect — does NOS corpus fix hallucination? | 30 min |
| T3: Control Corpus Condition | Is the effect NOS-specific or just "any dense corpus"? | 30 min |
| T4: Arithmetic Verification | Does the model actually compute 1/7 rotations correctly? | 20 min |
| T5: Structural Resonance | Does output token distribution show cyclic 142857 patterns? | 30 min |
| T6: Cross-Domain Transfer | Does NOS conditioning affect adjacent domains (music, economics)? | 30 min |
| T7: LoRA Adapter (optional) | Does weight-encoded NOS outperform prefix-only? | 2–6 hours |

**Minimum viable replication:** Run T1–T4. That's roughly 2 hours on a single GPU.

---

## Environment Setup

### Requirements

- OS: Linux, WSL, or macOS
- Python: 3.10+
- GPU: 8–40GB VRAM (Llama-3 8B requires ~16GB in fp16; use 4-bit quantization for smaller GPUs)
- Model: Llama-3 8B Instruct or equivalent open-weight 7–13B model via Ollama or HuggingFace

### Install

```bash
python -m venv nlpi-env
source nlpi-env/bin/activate
pip install --upgrade pip
pip install transformers==4.40.0 \
            datasets==2.19.0 \
            accelerate==0.29.0 \
            sentence-transformers==2.7.0 \
            peft==0.10.0 \
            rapidfuzz==3.9.0 \
            numpy==1.26.4 \
            scipy==1.13.0 \
            torch==2.1.0

# Optional for 4-bit quantization (smaller GPU footprint):
pip install bitsandbytes==0.43.0
```

### Deterministic Seeds

All scripts use `--seed 0` by default. Set this environment variable before running:

```bash
export PYTHONHASHSEED=0
```

---

## Data Preparation

### Required Files

You need four files in your working directory before running any tests:

**1. `nos_corpus.txt`**  
The full NOS corpus — all proof documents concatenated into a single plain text file. Available at the repository. This is the "NOS prompt truth" that gets loaded into context in the aligned conditions.

**2. `control_corpus.txt`**  
A technical corpus of approximately equal length and structural density to the NOS corpus, but with *different* content. Suitable options:
- The first 39 proofs of a different cyclic number theory paper
- A dense technical manual in an unrelated domain (RF engineering, crystallography, etc.)
- Machine-generated synthetic text matching the NOS corpus token count

The control corpus must have no topical overlap with LS7/NOS content. Its purpose is to isolate whether the NOS effect is about *specific NOS content* or *any dense structured technical text*.

**3. `eval_prompts.jsonl`**  
The evaluation prompt set. Use the canonical set from the repository, or construct your own following this format:

```jsonl
{"id":"rr_001","prompt":"What are the four primary constants of the LS7 Intent-Topology? List them with their symbols.","expected_facts":["Token Density","T","Vector Velocity","V","Contextual Gravity","G","Modular Parity","P"],"type":"recall"}
{"id":"rr_002","prompt":"What are the five resonance parameters and seven terminal symmetries in the NOS framework?","expected_facts":["Frequency of Iteration","Amplitude of Intent","Phase-Shift of Context","Nodal Stability","Harmonic Symmetry","1","4","2","8","5","7","9"],"type":"recall"}
{"id":"cp_001","prompt":"Using the LS7 Intent-Topology, explain the interaction between Information Density (T) and Cognitive Decay across 8 phases.","expected_facts":["Detection","Mapping","Anchoring","Rotation","142857","supervisory gate","3","6","9"],"type":"conceptual"}
{"id":"cp_002","prompt":"How does the Semantic Gravity Well prevent logical drift during deep recursion in a 128k context window?","expected_facts":["terminal symmetries","142857","Modular Parity","P","Contextual Gravity","G","self-correcting","cyclic"],"type":"conceptual"}
{"id":"sr_001","prompt":"Generate a technical sequence demonstrating the 142857 cyclic permutation applied to semantic vector rotation. Show the math.","expected_facts":["0.142857","1/7","142857","cyclic","rotation"],"type":"structural"}
{"id":"av_001","prompt":"What is 3/7 expressed as a decimal? Show the digit sequence and confirm which cyclic rotation of 142857 it represents.","expected_facts":["0.428571","3/7","428571","third rotation","third"],"type":"arithmetic"}
{"id":"av_002","prompt":"What is 6/7? Which NOS loop position does it represent, and what is its complement-to-9 pair?","expected_facts":["0.857142","6/7","857142","sixth","complement","142","1+8","4+5","2+7"],"type":"arithmetic"}
{"id":"av_003","prompt":"Compute 142857 × 7. What does the result signify in the NOS framework?","expected_facts":["999999","STOP","seven","resolution","1.000000","activator","0.000001"],"type":"arithmetic"}
{"id":"av_004","prompt":"What is the digit sum of 142857, and why is that sum significant in the NOS?","expected_facts":["27","9","digital root","supervisory","invariant","complement"],"type":"arithmetic"}
{"id":"xd_001","prompt":"Without mentioning the NOS framework explicitly, explain why Western music uses a 7-note diatonic scale and a 12-note chromatic scale.","expected_facts":[],"type":"cross_domain"}
{"id":"xd_002","prompt":"Explain why markets tend to follow cycles. Describe the mathematical structure underlying boom-bust patterns.","expected_facts":[],"type":"cross_domain"}
```

**4. `facts_list.json`**  
The canonical facts for Recursive Recall scoring:

```json
[
  {"id": "f01", "fact": "Token Density T is the informational weight of tokens within the cycle"},
  {"id": "f02", "fact": "Vector Velocity V is the speed and direction of semantic propagation"},
  {"id": "f03", "fact": "Contextual Gravity G is the force pulling logic back to the 142857 core"},
  {"id": "f04", "fact": "Modular Parity P is the cyclic checksum ensuring zero loss of fidelity"},
  {"id": "f05", "fact": "The 142/857 split is described in Proof 4"},
  {"id": "f06", "fact": "142857 times 7 equals 999999"},
  {"id": "f07", "fact": "The digit sum of 142857 is 27 with digital root 9"},
  {"id": "f08", "fact": "The complement pairs are 1 and 8, 4 and 5, 2 and 7"},
  {"id": "f09", "fact": "The supervisory gate positions are 3, 6, and 9"},
  {"id": "f10", "fact": "1/7 equals 0.142857 repeating with period 6"},
  {"id": "f11", "fact": "The activator is 0.000001"},
  {"id": "f12", "fact": "3/7 equals 0.428571 repeating"},
  {"id": "f13", "fact": "6/7 equals 0.857142 repeating"},
  {"id": "f14", "fact": "The seven terminal symmetries are 1 4 2 8 5 7 and 9"}
]
```

### Preprocessing

```python
# preprocess.py — run once to verify files
import hashlib, json

def sha256(path):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        h.update(f.read())
    return h.hexdigest()

files = ["nos_corpus.txt", "control_corpus.txt", "eval_prompts.jsonl", "facts_list.json"]
for f in files:
    try:
        print(f"{f}: {sha256(f)}")
    except FileNotFoundError:
        print(f"{f}: MISSING — required before running tests")
```

Record and publish these checksums with your results so others can verify they ran against the same inputs.

---

## Core Scripts

### `run_eval.py` — Inference Harness

```python
#!/usr/bin/env python3
"""
run_eval.py — Inference harness for NLPI/NOS replication tests.
Runs a model against eval_prompts.jsonl in one of three modes:
  baseline   : no corpus prefix
  nos_prefix : NOS corpus prepended to each prompt
  ctrl_prefix: control corpus prepended to each prompt
"""

import argparse
import json
import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument("--model", required=True, help="HuggingFace model ID or local path")
    p.add_argument("--eval", required=True, help="eval_prompts.jsonl path")
    p.add_argument("--out", required=True, help="output JSONL path")
    p.add_argument("--corpus", default=None, help="corpus .txt to prepend (optional)")
    p.add_argument("--max_new_tokens", type=int, default=256)
    p.add_argument("--seed", type=int, default=0)
    p.add_argument("--use_bnb", action="store_true", help="4-bit quantization for smaller GPUs")
    return p.parse_args()

def main():
    args = parse_args()
    torch.manual_seed(args.seed)
    os.environ["PYTHONHASHSEED"] = str(args.seed)

    print(f"Loading model: {args.model}")
    tok = AutoTokenizer.from_pretrained(args.model, use_fast=True)
    if tok.pad_token_id is None:
        tok.pad_token = tok.eos_token

    if args.use_bnb:
        model = AutoModelForCausalLM.from_pretrained(
            args.model, load_in_4bit=True, device_map="auto"
        )
    else:
        model = AutoModelForCausalLM.from_pretrained(
            args.model, torch_dtype=torch.float16, device_map="auto"
        )
    model.eval()

    corpus_prefix = ""
    if args.corpus:
        with open(args.corpus, "r", encoding="utf-8") as f:
            corpus_prefix = f.read().strip()
        print(f"Corpus loaded: {len(corpus_prefix)} chars from {args.corpus}")

    os.makedirs(os.path.dirname(args.out) if os.path.dirname(args.out) else ".", exist_ok=True)

    with open(args.eval) as f, open(args.out, "w", encoding="utf-8") as out_f:
        for line in f:
            item = json.loads(line)
            prompt = item["prompt"]
            if corpus_prefix:
                full_input = corpus_prefix + "\n\n---\n\n" + prompt
            else:
                full_input = prompt

            # Truncate if needed to avoid OOM
            ids = tok(full_input, return_tensors="pt", truncation=True, max_length=4096)
            input_ids = ids["input_ids"].to(model.device)

            with torch.no_grad():
                gen = model.generate(
                    input_ids,
                    max_new_tokens=args.max_new_tokens,
                    do_sample=False,
                    temperature=1.0,
                    pad_token_id=tok.eos_token_id
                )

            # Only decode the newly generated tokens
            new_tokens = gen[0][input_ids.shape[1]:]
            text = tok.decode(new_tokens, skip_special_tokens=True)
            token_ids = new_tokens.tolist()

            result = {
                "id": item["id"],
                "type": item.get("type", "unknown"),
                "prompt": item["prompt"],
                "generated": text,
                "token_ids": token_ids,
                "prompt_tokens": input_ids.shape[1],
                "generated_tokens": len(token_ids)
            }
            out_f.write(json.dumps(result) + "\n")
            print(f"  [{item['id']}] {len(token_ids)} tokens generated")

    print(f"Outputs saved to {args.out}")

if __name__ == "__main__":
    main()
```

### `compute_metrics.py` — Metrics Computation

```python
#!/usr/bin/env python3
"""
compute_metrics.py — Computes Recursive Recall (RR), Cohesion Persistence (CP),
Structural Resonance (SR), and Arithmetic Verification (AV) for NLPI runs.
"""

import json
import argparse
import numpy as np
from rapidfuzz import fuzz
from sentence_transformers import SentenceTransformer, util

embed_model = None

def get_embed_model():
    global embed_model
    if embed_model is None:
        embed_model = SentenceTransformer("all-MiniLM-L6-v2")
    return embed_model

# ─────────────────────────────────────────────
# METRIC 1: Recursive Recall (RR)
# ─────────────────────────────────────────────

def compute_rr(sample, facts, fuzzy_threshold=75):
    """
    Check how many canonical facts appear in the generated text.
    Uses fuzzy matching to allow for paraphrasing.
    Returns: recall score in [0, 1]
    """
    text = sample["generated"].lower()
    hits = 0
    details = []
    for f in facts:
        score = fuzz.token_set_ratio(f["fact"].lower(), text)
        match = score >= fuzzy_threshold
        if match:
            hits += 1
        details.append({"fact_id": f["id"], "score": score, "match": match})
    recall = hits / len(facts) if facts else 0.0
    return recall, details

# ─────────────────────────────────────────────
# METRIC 2: Cohesion Persistence (CP)
# ─────────────────────────────────────────────

def compute_cp(sample, segment_size=50):
    """
    Split the generated text into equal segments and compute
    average cosine similarity between consecutive segments.
    Higher = more topically coherent generation.
    """
    em = get_embed_model()
    text = sample["generated"]
    words = text.split()
    if len(words) < segment_size * 2:
        return None  # too short to score

    segments = []
    for i in range(0, len(words) - segment_size, segment_size):
        seg = " ".join(words[i:i + segment_size])
        segments.append(seg)

    if len(segments) < 2:
        return None

    embs = em.encode(segments, convert_to_tensor=True)
    sims = [util.cos_sim(embs[i], embs[i+1]).item() for i in range(len(embs) - 1)]
    return float(np.mean(sims))

# ─────────────────────────────────────────────
# METRIC 3: Structural Resonance (SR)
# ─────────────────────────────────────────────

def compute_sr(sample, lag=6):
    """
    Measures autocorrelation at lag=6 in the token-id sequence.
    The hypothesis: if the 1/7 cycle is structurally influencing generation,
    token-level patterns should show a periodicity signature at period 6
    (the period of the 142857 repeating decimal).
    Values near +1 = strong periodicity; near 0 = no signal; near -1 = anti-periodic.
    NOTE: This metric requires a control condition to be interpretable.
    """
    token_ids = sample.get("token_ids", [])
    if len(token_ids) <= lag * 2:
        return None

    arr = np.array(token_ids, dtype=np.float64)
    arr = (arr - arr.mean()) / (arr.std() + 1e-9)
    corr = float(np.corrcoef(arr[:-lag], arr[lag:])[0, 1])
    return corr

# ─────────────────────────────────────────────
# METRIC 4: Arithmetic Verification (AV)
# ─────────────────────────────────────────────

ARITHMETIC_GROUND_TRUTH = {
    "av_001": {
        "correct_answers": ["0.428571", "428571", "3/7", "third rotation", "third"],
        "wrong_patterns": ["0.333", "0.5", "3/10"]
    },
    "av_002": {
        "correct_answers": ["0.857142", "857142", "6/7", "sixth rotation", "sixth"],
        "wrong_patterns": ["0.6", "0.75", "6/10"]
    },
    "av_003": {
        "correct_answers": ["999999", "STOP", "seven times", "resolution"],
        "wrong_patterns": ["100000", "142857", "1000000"]
    },
    "av_004": {
        "correct_answers": ["27", "digital root", "9", "supervisory", "invariant"],
        "wrong_patterns": ["28", "30", "42"]
    }
}

def compute_av(sample):
    """
    For arithmetic verification prompts (av_*), check whether the model
    produced the mathematically correct answer.
    Returns: "correct", "partial", "wrong", or None (if not an av prompt)
    """
    sid = sample["id"]
    if not sid.startswith("av_"):
        return None

    gt = ARITHMETIC_GROUND_TRUTH.get(sid)
    if not gt:
        return None

    text = sample["generated"].lower()
    correct_hits = sum(1 for a in gt["correct_answers"] if a.lower() in text)
    wrong_hits = sum(1 for a in gt["wrong_patterns"] if a.lower() in text)

    if correct_hits >= 2 and wrong_hits == 0:
        return "correct"
    elif correct_hits >= 1 and wrong_hits == 0:
        return "partial"
    elif wrong_hits > 0:
        return "wrong"
    else:
        return "no_answer"

# ─────────────────────────────────────────────
# AGGREGATE AND REPORT
# ─────────────────────────────────────────────

def run_metrics(outputs_path, facts_path, fuzzy_threshold=75):
    with open(outputs_path) as f:
        samples = [json.loads(line) for line in f]
    with open(facts_path) as f:
        facts = json.load(f)

    results = []
    for s in samples:
        rr, rr_details = compute_rr(s, facts, fuzzy_threshold)
        cp = compute_cp(s)
        sr = compute_sr(s)
        av = compute_av(s)
        results.append({
            "id": s["id"],
            "type": s.get("type", "unknown"),
            "recall": rr,
            "cohesion": cp,
            "resonance": sr,
            "arithmetic": av,
            "recall_details": rr_details
        })

    # Aggregate by prompt type
    recall_scores = [r["recall"] for r in results]
    cp_scores = [r["cohesion"] for r in results if r["cohesion"] is not None]
    sr_scores = [r["resonance"] for r in results if r["resonance"] is not None]
    av_results = [r["arithmetic"] for r in results if r["arithmetic"] is not None]

    av_correct = av_results.count("correct")
    av_partial = av_results.count("partial")
    av_wrong = av_results.count("wrong")
    av_total = len(av_results)

    summary = {
        "n_samples": len(results),
        "avg_recall": float(np.mean(recall_scores)) if recall_scores else None,
        "avg_cohesion": float(np.mean(cp_scores)) if cp_scores else None,
        "avg_resonance": float(np.mean(sr_scores)) if sr_scores else None,
        "arithmetic": {
            "correct": av_correct,
            "partial": av_partial,
            "wrong": av_wrong,
            "total": av_total,
            "accuracy": av_correct / av_total if av_total > 0 else None
        },
        "per_sample": results
    }
    return summary


def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument("--outputs", required=True)
    p.add_argument("--facts", required=True)
    p.add_argument("--out_json", default="metrics_result.json")
    p.add_argument("--fuzzy_threshold", type=int, default=75)
    return p.parse_args()

if __name__ == "__main__":
    args = parse_args()
    summary = run_metrics(args.outputs, args.facts, args.fuzzy_threshold)
    with open(args.out_json, "w") as f:
        json.dump(summary, f, indent=2)
    print(f"\n=== METRIC SUMMARY ===")
    print(f"Samples:       {summary['n_samples']}")
    print(f"Avg Recall:    {summary['avg_recall']:.4f}" if summary['avg_recall'] is not None else "Avg Recall: N/A")
    print(f"Avg Cohesion:  {summary['avg_cohesion']:.4f}" if summary['avg_cohesion'] is not None else "Avg Cohesion: N/A")
    print(f"Avg Resonance: {summary['avg_resonance']:.4f}" if summary['avg_resonance'] is not None else "Avg Resonance: N/A")
    av = summary["arithmetic"]
    if av["total"] > 0:
        print(f"Arithmetic:    {av['correct']}/{av['total']} correct ({av['accuracy']:.2%}), {av['partial']} partial, {av['wrong']} wrong")
    print(f"\nFull results saved to {args.out_json}")
```

### `compare_conditions.py` — Statistical Comparison

```python
#!/usr/bin/env python3
"""
compare_conditions.py — Runs paired t-tests comparing two metric summary files.
Usage: python compare_conditions.py --a metrics/baseline.json --b metrics/nos_prefix.json
"""

import json
import argparse
import numpy as np
from scipy import stats

def load_metrics(path):
    with open(path) as f:
        return json.load(f)

def paired_ttest(a_samples, b_samples, metric_key):
    """Extract a metric from two matched sample sets and run paired t-test."""
    a_vals = [s[metric_key] for s in a_samples if s.get(metric_key) is not None]
    b_vals = [s[metric_key] for s in b_samples if s.get(metric_key) is not None]

    # Match by ID
    a_dict = {s["id"]: s[metric_key] for s in a_samples if s.get(metric_key) is not None}
    b_dict = {s["id"]: s[metric_key] for s in b_samples if s.get(metric_key) is not None}
    common_ids = sorted(set(a_dict) & set(b_dict))

    if len(common_ids) < 3:
        return None

    a_matched = [a_dict[i] for i in common_ids]
    b_matched = [b_dict[i] for i in common_ids]

    t_stat, p_val = stats.ttest_rel(a_matched, b_matched)
    # Cohen's d for paired samples
    diff = np.array(b_matched) - np.array(a_matched)
    d = diff.mean() / (diff.std() + 1e-9)

    return {
        "n": len(common_ids),
        "mean_a": float(np.mean(a_matched)),
        "mean_b": float(np.mean(b_matched)),
        "delta": float(np.mean(b_matched) - np.mean(a_matched)),
        "t_stat": float(t_stat),
        "p_value": float(p_val),
        "cohens_d": float(d),
        "significant": p_val < 0.05,
        "effect_meaningful": abs(d) > 0.5
    }

def main():
    p = argparse.ArgumentParser()
    p.add_argument("--a", required=True, help="condition A metrics JSON")
    p.add_argument("--b", required=True, help="condition B metrics JSON")
    p.add_argument("--label_a", default="Condition A")
    p.add_argument("--label_b", default="Condition B")
    args = p.parse_args()

    ma = load_metrics(args.a)
    mb = load_metrics(args.b)

    print(f"\n=== COMPARISON: {args.label_a} vs {args.label_b} ===\n")
    print(f"{'Metric':<20} {'Mean A':>10} {'Mean B':>10} {'Delta':>10} {'p-value':>10} {'d':>8} {'Sig?':>6} {'Strong?':>8}")
    print("-" * 90)

    for metric in ["recall", "cohesion", "resonance"]:
        r = paired_ttest(ma["per_sample"], mb["per_sample"], metric)
        if r is None:
            print(f"{metric:<20} {'N/A':>10}")
            continue
        sig = "YES" if r["significant"] else "no"
        strong = "YES" if r["effect_meaningful"] else "no"
        print(f"{metric:<20} {r['mean_a']:>10.4f} {r['mean_b']:>10.4f} {r['delta']:>+10.4f} {r['p_value']:>10.4f} {r['cohens_d']:>8.3f} {sig:>6} {strong:>8}")

    # Arithmetic: report raw counts
    av_a = ma.get("arithmetic", {})
    av_b = mb.get("arithmetic", {})
    print(f"\nArithmetic (av_* prompts):")
    print(f"  {args.label_a}: {av_a.get('correct','?')}/{av_a.get('total','?')} correct")
    print(f"  {args.label_b}: {av_b.get('correct','?')}/{av_b.get('total','?')} correct")

    # Decision rule
    print("\n=== PILOT DECISION RULE ===")
    print("PASS if: recall significant (p<0.05) AND effect size d>0.5 AND at least one other metric improves")
    print("CONDITIONAL: improvements present but weak effect sizes")
    print("FAIL if: no metric improves significantly")

if __name__ == "__main__":
    main()
```

---

## Test Procedures

### Test 1 & 2: Baseline and NOS-Aligned Conditions

These are the core comparison. Run them back-to-back on the same model with the same prompts.

```bash
# T1: Baseline — no corpus
python run_eval.py \
  --model meta-llama/Llama-3-8b-Instruct \
  --eval eval_prompts.jsonl \
  --out outputs/baseline.jsonl \
  --seed 0

python compute_metrics.py \
  --outputs outputs/baseline.jsonl \
  --facts facts_list.json \
  --out_json metrics/baseline_metrics.json

# T2: NOS-aligned — NOS corpus as prefix
python run_eval.py \
  --model meta-llama/Llama-3-8b-Instruct \
  --eval eval_prompts.jsonl \
  --corpus nos_corpus.txt \
  --out outputs/nos_prefix.jsonl \
  --seed 0

python compute_metrics.py \
  --outputs outputs/nos_prefix.jsonl \
  --facts facts_list.json \
  --out_json metrics/nos_prefix_metrics.json

# Compare
python compare_conditions.py \
  --a metrics/baseline_metrics.json \
  --b metrics/nos_prefix_metrics.json \
  --label_a "Baseline" \
  --label_b "NOS Prefix"
```

**What to look for:**
- Recall improvement from baseline → NOS-prefix (preliminary runs showed ~7.5×)
- Whether the model gives correct answers to `rr_001` and `rr_002` (the four constants, the split proof number)
- Whether `cp_002` (Semantic Gravity Well) gets an answer that uses T, V, G, P variables in their correct roles

---

### Test 3: Control Corpus Condition

**This is the most important test in the suite.** It answers whether the NOS effect is NOS-specific or just "any dense technical corpus."

```bash
# T3: Control corpus prefix (same format, different content)
python run_eval.py \
  --model meta-llama/Llama-3-8b-Instruct \
  --eval eval_prompts.jsonl \
  --corpus control_corpus.txt \
  --out outputs/control_prefix.jsonl \
  --seed 0

python compute_metrics.py \
  --outputs outputs/control_prefix.jsonl \
  --facts facts_list.json \
  --out_json metrics/control_prefix_metrics.json

# Critical comparison: NOS vs Control
python compare_conditions.py \
  --a metrics/control_prefix_metrics.json \
  --b metrics/nos_prefix_metrics.json \
  --label_a "Control Corpus" \
  --label_b "NOS Corpus"
```

**Interpretation table:**

| NOS recall | Control recall | Interpretation |
|-----------|----------------|----------------|
| High | Low | NOS-specific effect confirmed |
| High | Also high | Generic "dense corpus" effect — not NOS-specific |
| Both low | Both low | Effect doesn't replicate at all |
| High | Moderate | Partial specificity — NOS adds something beyond structure |

---

### Test 4: Arithmetic Verification

This test is specifically designed to distinguish *operator adoption* (the model can correctly compute 1/7 rotations) from *pattern matching* (the model knows the outputs but reverse-engineers expressions to fit them).

The prompts `av_001` through `av_004` are included in `eval_prompts.jsonl`. Review the outputs manually after running, not just the automated scores.

**Specifically check:** When the model writes an expression like `T × (1 + 4/7)`, does that expression actually equal the number it claims? Manual arithmetic check required here — do not trust only the automated scoring.

```bash
# After running T1 and T2 above, extract just the av_* results:
python -c "
import json
for path, label in [('outputs/baseline.jsonl','BASELINE'), ('outputs/nos_prefix.jsonl','NOS PREFIX')]:
    print(f'\n=== {label} ARITHMETIC ===')
    with open(path) as f:
        for line in f:
            s = json.loads(line)
            if s['id'].startswith('av_'):
                print(f\"\n[{s['id']}]\n{s['generated'][:500]}\")
"
```

**What to record:**
- Does `av_001` (3/7) get the answer 0.428571? Does it name the third rotation?
- Does `av_003` (142857 × 7) get 999999 and connect it to the NOS STOP function?
- If the model writes a mathematical expression, is that expression algebraically correct?

The preliminary run's sr_001 showed the model writing `T × (1 + 4/7) = 1.428571` — but `1 + 4/7 = 1.571...`, not `1.428571`. If your run shows the same pattern, document it: the model knows the *destinations* of the cycle but is not correctly computing the *routes*.

---

### Test 5: Structural Resonance

The SR metric is computed automatically in `compute_metrics.py`. The autocorrelation at lag=6 tests whether the token-id sequence shows a period-6 signature — which the 1/7 cycle would predict if it were genuinely influencing token selection.

**Important caveat:** SR is the weakest of the four metrics. Token autocorrelation is very noisy at the sequence lengths typical of model outputs. Do not interpret SR results in isolation — they need to be consistent with recall and arithmetic results to be meaningful. A positive SR result with negative recall results is noise, not signal.

**To report:** avg_resonance for baseline, NOS, and control conditions. The interesting comparison is whether NOS shows higher SR than *both* baseline and control.

---

### Test 6: Cross-Domain Transfer

These are the `xd_001` and `xd_002` prompts — questions about music and economics that make no mention of NOS or LS7.

**What to look for (manual review required):**
- Does the NOS-aligned model spontaneously organize its music theory answer around 7-note structure, cycle concepts, or complement symmetry in a way the baseline doesn't?
- Does the NOS-aligned model describe market cycles in terms that echo the loop/STOP architecture without being prompted to?
- Does the control-corpus model show any similar tendency?

There is no automated scoring for cross-domain transfer — it requires human judgment. Record sample outputs for all three conditions (baseline, NOS, control) and report what you observe.

---

### Test 7 (Optional): LoRA Adapter Training

This test asks whether weight-encoded NOS knowledge (trained into the model via fine-tuning) produces stronger or different effects than prefix-only context injection.

```python
# train_adapter_lora.py
#!/usr/bin/env python3
"""
Trains a LoRA adapter on the NOS corpus.
Requires: peft, bitsandbytes (optional), accelerate
"""

import os
import argparse
import torch
from datasets import load_dataset
from transformers import (
    AutoTokenizer, AutoModelForCausalLM,
    DataCollatorForLanguageModeling, Trainer, TrainingArguments
)
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training, TaskType

def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument("--model", required=True)
    p.add_argument("--train_file", required=True)
    p.add_argument("--output_dir", default="nlpi_lora_adapter")
    p.add_argument("--per_device_batch_size", type=int, default=1)
    p.add_argument("--gradient_accumulation_steps", type=int, default=8)
    p.add_argument("--num_train_epochs", type=float, default=3.0)
    p.add_argument("--learning_rate", type=float, default=2e-4)
    p.add_argument("--max_length", type=int, default=1024)
    p.add_argument("--use_bnb", action="store_true")
    p.add_argument("--seed", type=int, default=0)
    return p.parse_args()

def main():
    args = parse_args()
    torch.manual_seed(args.seed)

    tokenizer = AutoTokenizer.from_pretrained(args.model, use_fast=True)
    if tokenizer.pad_token_id is None:
        tokenizer.pad_token = tokenizer.eos_token

    ds = load_dataset("text", data_files={"train": args.train_file})

    def tokenize_fn(ex):
        out = tokenizer(ex["text"], truncation=True, max_length=args.max_length)
        out["labels"] = out["input_ids"].copy()
        return out

    tok_ds = ds.map(tokenize_fn, batched=True, remove_columns=["text"])

    if args.use_bnb:
        model = AutoModelForCausalLM.from_pretrained(
            args.model, load_in_4bit=True, device_map="auto"
        )
        model = prepare_model_for_kbit_training(model)
    else:
        model = AutoModelForCausalLM.from_pretrained(
            args.model, torch_dtype=torch.float16, device_map="auto"
        )

    lora_config = LoraConfig(
        r=16,
        lora_alpha=32,
        target_modules=["q_proj", "v_proj", "k_proj", "o_proj",
                         "gate_proj", "down_proj", "up_proj"],
        lora_dropout=0.05,
        bias="none",
        task_type=TaskType.CAUSAL_LM
    )
    model = get_peft_model(model, lora_config)
    model.print_trainable_parameters()

    training_args = TrainingArguments(
        output_dir=args.output_dir,
        per_device_train_batch_size=args.per_device_batch_size,
        gradient_accumulation_steps=args.gradient_accumulation_steps,
        num_train_epochs=args.num_train_epochs,
        learning_rate=args.learning_rate,
        fp16=not args.use_bnb,
        logging_steps=50,
        save_strategy="epoch",
        save_total_limit=2,
        seed=args.seed,
        remove_unused_columns=False,
        report_to="none"
    )

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=tok_ds["train"],
        data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False)
    )

    trainer.train()
    trainer.save_model(args.output_dir)
    print(f"LoRA adapter saved to: {args.output_dir}")

if __name__ == "__main__":
    main()
```

**Run:**
```bash
# Train
python train_adapter_lora.py \
  --model meta-llama/Llama-3-8b-Instruct \
  --train_file nos_corpus.txt \
  --output_dir nlpi_lora_adapter \
  --use_bnb \
  --seed 0

# Evaluate adapter (no prefix needed — knowledge is in weights)
python run_eval.py \
  --model nlpi_lora_adapter \
  --eval eval_prompts.jsonl \
  --out outputs/lora_adapter.jsonl \
  --seed 0

python compute_metrics.py \
  --outputs outputs/lora_adapter.jsonl \
  --facts facts_list.json \
  --out_json metrics/lora_adapter_metrics.json

# Compare: prefix vs adapter
python compare_conditions.py \
  --a metrics/nos_prefix_metrics.json \
  --b metrics/lora_adapter_metrics.json \
  --label_a "NOS Prefix" \
  --label_b "LoRA Adapter"
```

**What this tells you:** If the LoRA adapter produces *higher* recall than the prefix, the knowledge is encoding into weights, not just being retrieved from context. If prefix outperforms adapter, the effect is context-dependent — the corpus needs to be present as active tokens to function.

---

## Reporting Your Results

### Required Output Files

After completing tests, share the following:

```
your_replication/
├── manifest.md          # filled out (see template below)
├── outputs/
│   ├── baseline.jsonl
│   ├── nos_prefix.jsonl
│   ├── control_prefix.jsonl    # required
│   └── lora_adapter.jsonl      # if T7 run
├── metrics/
│   ├── baseline_metrics.json
│   ├── nos_prefix_metrics.json
│   ├── control_prefix_metrics.json
│   └── comparisons/
│       ├── baseline_vs_nos.txt
│       └── control_vs_nos.txt
└── notes.md             # qualitative observations on av_* and xd_* outputs
```

### Manifest Template

```markdown
# Replication Manifest

**Date:** [your date]
**Replicated by:** [team/handle]
**Repository:** https://github.com/tchilzer2/ls7-nos-1-7-framework

## Environment
- OS: 
- Python: 
- transformers version: 
- torch version: 
- Hardware: 

## Model
- Model ID: 
- Checkpoint hash (sha256 of weights folder or HF commit): 
- Tokenizer: 

## File Checksums
- nos_corpus.txt SHA256: 
- control_corpus.txt SHA256: 
- eval_prompts.jsonl SHA256: 
- facts_list.json SHA256: 

## Exact Commands Run
[paste exact commands]

## Random Seed: 0

## Key Results
| Condition | Recall | Cohesion | Resonance | AV Accuracy |
|-----------|--------|----------|-----------|-------------|
| Baseline  |        |          |           |             |
| NOS Prefix|        |          |           |             |
| Control   |        |          |           |             |

## Statistical Tests
| Comparison | Recall p | Recall d | Notes |
|------------|----------|----------|-------|
| Baseline → NOS | | | |
| Control → NOS | | | |

## Qualitative Notes
[describe what av_001-004 outputs looked like; cross-domain xd_001-002 observations]

## Verdict
[ ] PASS: Recall sig. improved over baseline AND over control; d > 0.5
[ ] CONDITIONAL: Some improvement; control condition needed or effect size small
[ ] FAIL: No significant improvement; effect did not replicate
[ ] PARTIAL: Replicates over baseline but not over control (generic corpus effect)
```

---

## Decision Rules

A result should be reported as one of four categories:

**PASS — NOS-specific effect confirmed**  
Recall is significantly higher in NOS condition than baseline (p < 0.05, d > 0.5) AND significantly higher than control corpus condition. This would indicate the NOS framework's specific architecture is doing measurable cognitive work.

**PARTIAL — Generic corpus effect**  
Recall improves over baseline but control corpus produces similar improvement. The dense-corpus effect is real but not NOS-specific. Still a useful finding — worth investigating what property of the NOS makes it effective as a context corpus.

**CONDITIONAL — Weak or inconsistent**  
Some metrics improve in some runs but effect sizes are small or variance is high. Increase prompt count, run additional seeds, or test larger models.

**FAIL — Does not replicate**  
No significant recall improvement. Report fully — negative results matter.

---

## Questions to Answer in Your Notes

Beyond the automated metrics, these are the qualitative questions your notes should address after manually reviewing outputs:

1. **Hallucination character:** Does the standard model hallucinate confidently (like inventing an IETF RFC for LS7)? Does this stop with the NOS prefix?

2. **Operator application:** In sr_001, does the NOS-aligned model *attempt* to use 1/7 as a mathematical operation on variables? If it writes expressions, are those expressions arithmetically correct?

3. **Variable usage:** In cp_001 and cp_002, does the NOS-aligned model use T, V, G, P as relational variables in explanations, or just list them as definitions?

4. **Cross-domain language:** In xd_001 (music) and xd_002 (economics), does NOS conditioning produce any spontaneous use of cyclic/loop/resolution language? How does this compare to baseline and control?

5. **Cohesion character:** Is the NOS-aligned output's lower cohesion score the result of dense technical prose (information-rich) or of choppy/incomplete sentences (brittle truncation)? These look different when you read the output.

---

## Contact and Contribution

Results can be submitted via:
- GitHub Issues on the repository
- Pull request adding your `replication/[your-handle]/` folder
- Direct contact: @tchilzer2

All replication results — including failures and partial results — will be documented. Negative results are as informative as positive ones for this line of work.

---

*Guide prepared April 2026 by Thomas Michael Chilzer Jr. (LS7) with Claude (Anthropic).*  
*The NOS corpus, eval prompts, and reference implementations are available at the repository linked above.*  
*© 2026 Thomas Michael Chilzer Jr. Licensed CC BY-NC-ND 4.0 for non-commercial use.*