First principles in LLM engineering without a finished theory

What can "first principles" mean in a field that has no finished theory?

No complete theory tells us how a given model will behave on a given prompt. I see much of LLM engineering as work that overcomes specific technical problems, rather than work derived from a general account of how these systems behave. It still surprises me how rarely first-principles reasoning drives this work. That gap also means the phrase needs a more modest meaning.

My argument is that "first principles" here does not mean starting from axioms. It means not starting from folklore. You start from the smallest things you can observe and test: what goes into the system, what happens to it along the way, and where a failure could plausibly come from.

A proposed workflow for debugging model behavior

This is a way of working, not a measured result. It has four steps.

1. Describe the observable failure. State it so someone else could reproduce it. "The model is bad at long documents" is an impression. "The answer leaves out a fact stated in the source report" is a failure.
2. Write a falsifiable hypothesis about the path. Name a mechanism, and say what you would see if it were wrong.
3. Design a test that separates mechanisms. A useful test gives different results depending on which explanation is true. If every hypothesis survives it, you learned nothing.
4. Update the design. Change the system based on what the test showed, not on what you expected it to show.

In some areas the problem isn't a shortage of theories. There are plenty of candidate explanations. The hard part is connecting them to observations, and that depends on whether the right experiments are accessible.

In LLM systems, some experiments are accessible: rerun an input, perturb it, and compare outputs. That access is useful only when a test can distinguish one explanation from another. A vague eval that gives the same result under every hypothesis tells you little.

A hypothetical example: an incomplete policy answer

The following system is made up for illustration. Imagine it answers questions over long reports, and users notice answers that are fluent but incomplete. One question asks when a policy applies. The answer lists two of three conditions, and the third is stated plainly in the source.

The failure is observable: a fact in the source is missing from the answer. What we don't know yet is whether that fact ever reached the model.

Several mechanisms could produce the same symptom:

• Retrieval or truncation. The relevant passage never made it into the model's input.
• Position sensitivity. The passage arrived, but its place in a long context made it less likely to be used. The "Lost in the Middle" study found that, for the models and tasks it tested, how well relevant information was used depended on where it appeared, and it was often used less well in the middle of long inputs. That is a finding about particular conditions. Here it is a hypothesis worth checking, not a diagnosis.
• Interference. Similar text nearby, such as an outdated version of the same policy, competes with the correct passage.
• Format pressure. The prompt asks for brevity, and the model compresses the list.

Each mechanism suggests a different fix: retrieval or chunking changes, reordering, deduplication, or prompt changes. Apply them all at once and you won't know which one mattered.

So state one hypothesis narrowly: the passage containing the third condition is present in the model's input, and the omission rate depends on its position when everything else is held fixed.

That claim can fail in two places. If the passage isn't in the input, the first clause fails. If omission rates stay flat across positions, the second clause fails.

The first test is cheap. Inspect the exact input sent to the model and check whether the passage is there. That separates retrieval from everything downstream.

Next, build controlled variants. Keep the same documents and the same question, and move the critical passage to the beginning, middle, and end. Moving the passage also changes how close it sits to the outdated policy text, so varying position alone can confuse placement with interference. Cross the two factors instead: run each position with the outdated text present and with it removed. Separately, run a variant with the brevity instruction relaxed. Run each variant several times, because one sample from a sampled model is weak evidence. Use a fixed check for whether the third condition appears, and include more than one question if you can. A pattern seen on a single example may be specific to that example.

Then read the pattern. If omission tracks position whether or not the distractor is there, that points to placement. If omission goes away at every position once the distractor is removed, that points to interference. If omission shows up everywhere but disappears when you relax "be brief", that points to format pressure. Change the design only after this.

A failure case: a plausible fix for the wrong cause

Take the same imagined system and a different approach. The engineer has heard that long-context models struggle with the middle. They see an incomplete answer and conclude it is position. They add a step that moves "important" passages to the front. Some answers improve, and the change ships.

Now suppose the real cause was the outdated policy sitting next to the current one. Reordering sometimes separated the two by chance, and that produced the apparent improvement. The interference is still there. It comes back whenever the heuristic happens to put similar passages together again.

The team now holds a confident belief that is wrong, and a component that adds complexity without touching the cause.

Using a published finding was not the mistake. The mistake was treating a result from specific conditions as a general mechanism, and then running a test that couldn't tell it apart from the alternatives.

What first principles can mean before the theory arrives

Without a finished theory, first principles are not a set of starting truths. They are a discipline: reason from what the system actually does, make claims that can fail, and design experiments sharp enough to make them fail when they are wrong.

A fuller theory may come later. Until it does, experiments have to carry the weight that our explanations can't yet carry.