Intent Compilation and Behavioral Caching

“To this day, I still believe we were right to focus on the execution efficiency of the target program rather than on language design… The next programming revolution will happen only when two conditions are met: 1. a new programming language far more powerful than those we have today is developed; and 2. a way is found to execute it at a cost not much higher than that of today's programs.” — John Backus (creator of FORTRAN, 1977 Turing Award recipient, History of Programming Languages conference, 1978)

1. In brief

Intent compilation is the process in which people give AI instructions in natural language, and AI understands the intent, plans the task, calls tools, integrates results, and ultimately delivers the task. In this process, natural language, as the carrier of intent, can be viewed as a source language above programming languages; the AI's execution process can be viewed as the compilation of natural language.

Behavioral caching consists of reusable tool-call results or action sequences constructed during intent compilation. When model inference costs more than deterministic computation, reusing behavioral caches resembles a just-in-time compiler's (JIT) hot-code compilation: compiling and optimizing frequently repeated blocks of code, while retaining model inference only where something actually changes. If an environment change invalidates the cache, a fallback mechanism safely returns the task to the model for replanning.

Intent compilation makes deep long-tail needs that standardized software could not cover achievable. Developing for these intents creates strong user retention; behavioral caching reduces the ongoing cost of inference and maintains a marginal-cost advantage.

The following sections discuss how this paradigm emerged, the difficulties facing its compiler and existing work on them, how behavioral caches operate as reusable middleware, how an edge–cloud architecture can address cross-application long-tail needs, and how to understand the boundary between improving model capabilities and external caches.

2. Why now?

Treating natural language as a programming language is not a new idea. When Grace Hopper designed FLOW-MATIC and COBOL in the 1950s, she sought to bring code closer to English sentence structures. In the 1970s, with the rise of early symbolic AI, academia and industry saw another wave of interest in “automatic programming in natural language,” including MIT's SHRDLU, IBM's conversational programming system NLPQ, and Stanford's PSI program-synthesis system. The basic approach was straightforward: manually add grammatical rules, match English against those rules, and parse it into program instructions.

These attempts ultimately failed to become mainstream. Their foundations were still rigid rule matching, without human common sense or the ability to fill in implicit assumptions. Manually introducing grammatical rules to formalize natural language in an intermediate layer leads to a combinatorial explosion caused by ambiguity. Writing that layer becomes harder than writing a programming language, and errors abound. As Turing Award recipient Dijkstra argued in his 1978 essay “On the Foolishness of ‘Natural Language Programming’”, attempting to turn natural language into a programming language gives up the concision and precision of formal notation while producing something that “looks like English but breaks when a preposition is wrong.”

The arrival of large language models (LLMs) changed this. LLMs brought at least two foundational capabilities that had never previously been available together: a deep ability to understand the human meaning expressed in natural language, and a strong ability to generate formal code and actions. On top of these came:

1. Multi-step planning and control: Advances in post-training moved models beyond single-turn answers. They can now break down tasks, call external tools, and dynamically revise their next steps based on environmental feedback.
2. Tool calling and harnesses: MCP (Model Context Protocol) standardizes tool calling, while harness engineering enables LLMs to act in operating-system environments rather than remaining confined to a single cloud process.
3. Most importantly, token costs continue to fall. LLMs are no longer toys available only to a handful of AI laboratories.

This work may be called agent workflows, vibe coding, or loop engineering. Taken together, however, it amounts to this: people express intent in natural language; a large model, as the compiler's core process, performs understanding, planning, and tool orchestration, ultimately producing computer state transitions. As Andrej Karpathy put it, “the hottest new programming language is English”. Natural language is replacing traditional programming languages as the top-level source language, while traditional languages become downstream compilation artifacts, much like assembly or bytecode.

A natural question follows: How should we build this language's compiler, and how should we optimize it?

Engineering practices emerging over the past two years have already provided the initial structure of this “natural-language compiler.” Two core challenges remain.

3. Two problems

Building and optimizing a natural-language compiler requires addressing at least two problems:

1. Nondeterminism in the source language and the compiler's core process: Natural-language semantics are ambiguous and informal, while LLM outputs are probabilistic and can drift—even setting temperature to zero does not make them completely reproducible.
2. Inference cost: Compiling human intent requires ongoing model inference each time a task runs. Inference costs more than computation, and inference resources are scarcer than computational resources.

3.1 An ambiguous source language and a probabilistic compilation process

Programming languages trade expressive freedom for decidability: syntax is constrained, types can be checked, and side effects are explicit. A compiler reports syntax errors. Natural language, by contrast, is often full of ambiguity, implicit assumptions, unclear references, and unconstrained types. A person's real intent is not always fully planned in advance and compiled in one pass; it often evolves during execution, particularly in long task chains. Meanwhile, the large model at the compiler's core performs conditional probabilistic sampling “one token at a time,” introducing its own random variation.

There are three main approaches to addressing this problem.

The first constrains outputs through syntax and type contracts, primarily to address uncertainty in LLM outputs:

• OpenAI Structured Outputs masks tokens that violate a JSON Schema during sampling, guaranteeing syntactically compliant output.
• Microsoft TypeChat uses TypeScript type definitions as prompts, translating natural language into strongly typed structures and validating them.
• Pydantic / Instructor declares output contracts with Python type annotations. When parsing fails, errors are automatically returned to the model for correction and retry.
• BAML (BoundaryML) allows the model to produce output freely while reasoning, then uses an external fault-tolerant parser written in Rust to reconcile loosely structured text with strongly typed data in milliseconds.

The second refines ambiguous intent through specification-driven development (SDD), primarily to address ambiguity in natural language:

• GitHub Spec Kit uses a four-step template—constitution, specification, plan, and tasks—to break vague requirements into reviewable intermediate documents.
• AWS Kiro requires prompts to become requirements and design documents in the IDE, and introduces property-based tests to catch boundary cases.
• OpenSpec puts documentation before development and maintains a “living specification” in the repository that evolves with each change.
• Tessl advocates “specifications as source”: code is generated from specifications and not edited manually, with scripts mechanically checking links and attribution.

The third adds independent external verification and trace observation, primarily to address behavioral drift when LLMs call tools:

• The LLM-Modulo framework asks the large model only to generate candidates, while external compilers, solvers, or test scripts act as deterministic judges. Failed candidates are sent back with errors for another attempt.
• Guardrails AI / NeMo Guardrails places independent rule and code-validation pipelines between models and tools to intercept out-of-bounds parameters and high-risk operations.
• OpenTelemetry GenAI and trace-observation platforms (Langfuse, Arize Phoenix, AgentOps, and Braintrust) record each reasoning step, MCP tool call, and returned result as a structured call tree (trace). This supports “time-travel” replay and attribution, and turns production failure traces into CI/CD regression gates.

By 2026, engineering optimization for LLM task execution has entered a period of large-scale development. These efforts also introduce substantial burdens.

First, they insert a heavy, artificially constructed intermediate representation between natural-language intent and the result. Software engineering researchers Martin Fowler and Birgitta Böckeler have warned that this risks repeating the failure of formal specification languages in the previous century. When specifications become as cumbersome as code in an effort to remove ambiguity, writing them and maintaining synchronization can become more burdensome than writing the code itself. As model capabilities improve, we may actually need less of the heavy harness approach exemplified by the original OpenSpec.

The response is that the burden of specification does not disappear; it can only be transferred. Since natural language's “low-cost ambiguity” is the main advantage it offers people when expressing intent, large models should do the work of completing and refining ambiguous intent into specifications. People should confirm and audit key outcomes.

Second, rigid formatting constraints and heavy orchestration frameworks can overburden AI. Overly strict formatting instructions during generation consume attention bandwidth and can reduce reasoning accuracy by 10% to 30% (Tam et al., 2024). Constraints added to prevent careless mistakes end up occupying much of the model's effort.

One response can be drawn from MLIR (Multi-Level Intermediate Representation), a modern compiler infrastructure: establish lightweight domain-specific intermediate representations, then let deterministic compilation systems beneath them handle conversion to low-level execution. Once a domain has a deterministic compiler for its intermediate layer, the model can move up one level of abstraction and stop reasoning further down (“Compiled Cognition”).

Work on compilation ambiguity continues, and no fully mature approach has emerged. Even if a single task can be steered correctly, reliability across long task chains remains a problem. This is good news: there are still engineering optimization opportunities for us.

3.2 Inference cost

After traditional software has been developed and compiled, its runtime consumption consists of very inexpensive hardware computation. Intent compilation is different: large-model inference cannot yet be as cheap and widely available as conventional computation. If every task requires a large model to act as an interpreter and reason through everything from the beginning, expensive “interpretive overhead” continues indefinitely. It is slow, and each run consumes tokens and compute to make the same decisions again. Turning all computation into LLM work is neither sensible nor economical.

It helps to distinguish “inference” from “computation” in this discussion. A Transformer is itself a process running on GPU clusters and ultimately a set of matrix operations. Here, however, computation means applying definite rules to definite inputs, whether those rules live in software, a system, or hardware. Inference means using a parameterized model to predict probabilities over symbol sequences. Asking a hardware adder to solve "2+2=" produces "4" after a circuit delay of nanoseconds to microseconds. Encoding "2+2=" as tokens, passing it through a Transformer, and using many neural-network layers to predict and output the next token "4" with 99.999% probability consumes milliseconds or even seconds of compute and costly memory bandwidth.

Infrastructure work has already introduced many low-level optimizations to reduce inference costs, including:

• KV caches and prompt caching, such as vLLM PagedAttention, Anthropic, and OpenAI Prompt Caching: reuse attention states for identical context prefixes to avoid repeated computation.
• Speculative decoding: a lightweight draft model quickly guesses tokens, and a larger model verifies them in parallel in a single pass.
• Semantic caching, such as GPTCache and Redis LangCache: use vector similarity to return historical answer text directly for similar questions.

Optimizing AI infrastructure will remain an important objective for a long time. Token costs have fallen sharply over the past three years, but not nearly enough. Lower unit inference costs have stimulated longer chains of thought and an explosion of more complex cross-system tasks—the Jevons paradox. Demand and invocation volumes are growing much faster than prices are falling. Inference resources are costly; computational resources are inexpensive. Inference resources are scarce; computational resources are abundant. This imbalance will persist, and it is the premise underlying the engineering optimizations discussed below.

4. Behavioral caching: reusable middleware along state-transition paths

4.1 Core definition: reusable middleware along state-transition paths

My simple definition of behavioral caching is this: reusable tool-call results or action sequences constructed during intent compilation, converted into faster, lower-cost deterministic computation.

The large model's core task in intent compilation can be understood as finding a path through an unknown state space, from the initial intent (state S₀) to the final delivered result (state S_n). Behavioral caching, meanwhile, identifies heavily traveled paths and builds expressways.

[Early stage: end-to-end reasoning and path finding (intent compilation)]
Human intent (S0) --(model inference)--> State A --(model inference)--> State B --(model inference)--> Final result (Sn)

[Mature stage: computational shortcuts between fixed nodes (behavioral caching)]
Human intent (S0) --(model inference)--> State A ==(behavioral cache: deterministic computation/replay)==> State B --(model inference)--> Final result (Sn)
                                          |                                                              ^
                                          +---- (environment guard fails: fall back to the model) --------+

The core idea is to find stable, reusable deterministic middleware during intent compilation—the process of finding a path.

If every task requires a large model to act as an interpreter and find the entire route from S₀ to S_n again, we remain trapped by the two problems in Section 3: probabilistic drift along long task chains, and expensive inference repeated on every run. Drawing on JIT hot-code compilation, not every segment of the path from intent to result changes each time. If the transition between two intermediate nodes, A and B, is shown to be frequent and stable, inexpensive, deterministic “computation and storage” can replace costly, probabilistic “model inference.”

The entire task need not be 100% rigid and unchanging. Nor does this exclude a large model's flexible understanding at the beginning and end. If the transition between any pair of intermediate nodes can be made deterministic, it constitutes a behavioral cache:

• Results as state (caching node artifacts): Often, the objective of moving from node A to node B is simply to produce a definite new state—for example, parsing a document into structured Markdown, producing a cleaned table, or obtaining the return value of a read-only tool call. Saving this intermediate artifact and reading it directly when the same conditions recur takes execution to node B in one step. This is the most direct form of behavioral caching.
• Paths as programs (caching action sequences): When input parameters differ on each transition from A to B, such as processing business data for different dates, or when execution must actually change the environment, such as modifying a local file or clicking a software interface, a static file cannot simply be reused. Instead, the tool-call order, code script, or action sequence from A to B is fixed into an executable path. New parameters are supplied and deterministic computation runs the path, again avoiding the model inference in between.

4.2 Evolution: from manual anticipation before execution (AOT) to dynamic compilation at runtime (JIT)

We have already observed several lines of exploration:

• MCP and Skills: MCP fixes interface definitions and transport protocols for common tool calls. Skills fixes operating instructions for the model and saves common actions as local scripts. They establish the “signposts and interfaces” first, but each execution still requires the model to read instructions and reason about orchestration in real time.
• Tool Result Caching: retain return values from recurring tool calls. Alternatively, Cornell's TVCache organizes multi-step calls into a stateful “tool invocation graph” and reuses results and snapshots when a sandbox prefix matches. A later request can then receive the result without executing the tool again.
• Primitive services that compress multiple steps (Firecrawl / Context7): compress and package frequently used, general-purpose tasks with many steps and long chains. Firecrawl, for example, turns “launch a browser, render, extract the main content, perform OCR, and generate a Markdown file” into a single API service. Creating the Markdown file on the first request can be viewed as interpreted execution; returning that file for later requests to the same page can be viewed as compiled execution. Context7 compresses version matching, chunking, and retrieval across code libraries into a documentation-query service. When APIs or syntax change, the model can ask Context7 directly instead of grepping the entire repository each time.

Comparison:

Existing work already achieves this: developers anticipate high-frequency needs shared across an industry before execution (AOT), design compressed paths in advance, and expose them as APIs. The direction we hope to advance is from “manual anticipation before execution” to “dynamic compilation at runtime (JIT)”—finding hot paths as intent compilation runs, and automatically turning them into behavioral caches.

4.3 Core mechanisms of a behavioral cache layer

Building a runtime behavioral cache layer revolves around four core engineering problems.

First, observability of intermediate processes and automatic hot-path discovery. By instrumenting an agent's execution flow as structured state–action call trees, a system can perform process mining and common-subgraph extraction across scattered traces. Frequently successful intermediate subpaths then emerge automatically.

Second, distinguish “result replay” from “action replay.” Single-turn answers, semantic caches, and public-web scraping are fundamentally stateless and read-only; the cached object is a historical return value. Real complex tasks, however, often include writes or side effects that change the external world—clicking a UI, modifying a file, submitting a form, or sending a request. Later outputs may also depend strongly on environment changes caused by earlier steps. In one sentence: read operations can optimistically return historical results directly, while write operations must execute action sequences again in the real environment.

Third, compile tool combinations and control flow together. Even when individual tool-call nodes are fixed, a large model still repeatedly intervenes between every pair of calls to “observe, think, and decide the next step.” Compiling combinations of tools, loops, and conditional branches into deterministic control-flow code—and retaining model calls only at nodes that genuinely need semantic judgment—is what reduces latency further.

Fourth, dynamic validation, eviction, and a safety boundary for caches. Trace-level behavioral caching, the deepest layer in this progression, offers the most aggressive gains and the highest engineering risks. Blind replay without environment validation can click the wrong button in the wrong place when an interface is redesigned or an API field drifts. If a cache only grows and never shrinks, a bloated library slows retrieval. Cache lifecycle management is therefore essential. A complete behavioral-cache component must include:

1. Parameterized trajectories: record the intermediate state results produced between nodes A and B, or the executed tool sequence, code skeleton, and slots for dynamic parameters.
2. Environment guards: record the environment-state assumptions on which the trajectory depends, such as page DOM structure, visual screen features, API versions, or hashes of data definitions.
3. Reuse validation and fallback: compare the current environment with the guards before and during replay. A match allows millisecond-scale deterministic computation; a mismatch immediately interrupts execution and safely falls back to the large model for replanning.
4. Verify-before-store and eviction: recording does not establish usability. A new trajectory must pass independent consistency tests on a held-out set before admission. After admission, Redis-style TTL expiration, LFU/LRU capacity eviction based on frequency and recency, and periodic probe sampling automatically remove infrequent or invalid caches.

5. Development paradigm and commercialization

Having introduced intent compilation and behavioral caching, the next questions are: How will these mechanisms reshape software development? As entrepreneurs who are neither foundation-model vendors nor super-app giants, how should we build defensible advantages?

5.1 From a one-time development cost to ongoing inference costs

In the traditional software-development paradigm, the business model assumes “high fixed cost and zero marginal cost.” Product managers identify the broadest common requirements. Engineering teams spend months or years coding and deploying a product. Once that substantial one-time development cost has been paid, the software company can sell licenses to thousands or millions of users at nearly zero marginal cost. Users, meanwhile, must force their intentions into fixed menus and buttons. In this old paradigm, many infrequent, cross-application, personalized deep long-tail needs remain unexplored: real intent is hidden behind fragmented software interactions, and the high initial development cost cannot be recovered.

Intent compilation frees users from the predefined functional boundaries of a single application. Cross-application, long-chain, small-volume yet persistent long-tail needs can, for the first time, be expressed directly in natural language and realized by AI. At the same time, the system's cost is no longer paid once during development. It becomes an inference cost incurred continuously, every time a task executes at runtime.

The central economic significance of a behavioral cache layer is that it uses state-result reuse and action-trace replay between nodes to compress ongoing “variable inference costs” back toward traditional software's “near-zero marginal cost.” Intent compilation raises the ceiling on what deep long-tail tasks can achieve; behavioral caching provides a cost floor that makes them practical. Use tokens to establish a capability, then repeat that capability with as few tokens as possible.

5.2 Architecture: edge–cloud collaboration

Express intent on the device—compile in the cloud—cache shared paths—continuously inspect in the cloud.

Cloud-based personal agents can replace people in “production,” but they can never replace people in “consumption.” This is an unavoidable limitation of purely cloud-based agents. Phones and computers are not used only for reports and code; people also use them to watch movies, play games, read, and socialize. As long as people's consumption and experiences happen on personal devices, attachment to those devices will persist. Frequent use and local context—screens, local files, login state, and privacy permissions—mean that the human “intent layer” remains on personal devices rather than in cloud inference.

+-------------------------------------------------------------------------------------------+
|                                      Cloud                                                |
| 1. Path-finding compiler (large model + Harness) --> 2. Shared-path clustering / Verify Gate |
|                                                        |                                  |
|                 +--------------------------------------+---------------------+            |
|                 v (read-only segment: state/file replay)                     v            |
| [Cloud behavioral cache gateway: CDN-like shared results]   [3. Continuous probes/updates] |
+-----------------^-----------------------------------------------------------+------------+
                  |                                                           |
                  | Return state artifacts without inference                  | Deliver guarded action artifacts
                  |                                                           v
+-----------------+-----------------------------------------------------------+------------+
|                                   Personal device (Edge)                                  |
| Natural-language intent / local screens and files --> Local executor (guards + replay)    |
|                                                              |                            |
|                                  (Guard mismatch: report breakpoint, fall back to cloud)  |
+-------------------------------------------------------------------------------------------+

1. Express intent on-device; find and compile paths in the cloud: Users express natural-language intent on personal devices, where full screen, file, and permission context is available. For a previously unseen task—or when a local guard fails—the device invokes a cloud model. The cloud model calls tools and a runtime harness to explore the first execution path.
2. Extract shared paths across users and distribute by read/write behavior: Across thousands of users' intent-compilation processes, apparently very different long-tail tasks often share highly overlapping common paths between intermediate nodes. Examples include extracting structured data from a particular website or application, parsing a document type into an intermediate file format, and performing standard format conversion or reconciliation across systems. The cloud identifies and clusters these frequent shared paths, then compiles them into behavioral-cache components after admission checks.
3. Continuous cloud inspection and live updates: External environments—page DOM, third-party APIs, and software UI versions—continually change. The cloud uses test sets and sampled device-side Guard Miss signals to inspect the health of stored behavioral caches. When a shared path's failure rate rises or guards raise an alert, it invokes the large model to explore and recompile, then smoothly pushes updated cache artifacts to devices: “fix once, apply everywhere.”

5.3 Core assets

Under this paradigm and edge–cloud architecture, code shifts from a core asset to a downstream artifact that a compiler can regenerate at any time. The core assets that are difficult to copy become retention at the intent interface, verification suites, private context, and the behavioral-cache library.

Our opportunity is to address deep long-tail needs involving cross-application, long-chain tasks that existing apps do not cover, and use behavioral caching to build a structural cost advantage.

6. The bitter lesson

“We must learn the painful lesson that building the way we think into artificial-intelligence systems does not work in the long run. This lesson rests on historical observations: 1) AI researchers often try to build knowledge into their agents; 2) this always helps in the short term and brings researchers personal satisfaction; but 3) in the long run, the approach plateaus and can obstruct further progress; and 4) breakthroughs ultimately come from the opposing approach, which scales computation through search and learning. The eventual success is often bitter and difficult to fully absorb because it defeats the human-centered approach people prefer.” — Rich Sutton, “The Bitter Lesson”

Engineering optimization around AI frequently meets this objection: If AI capabilities are still improving, will today's tools and frameworks remain necessary? Will better models simply absorb today's harnesses and specification layers? Why not let an LLM handle everything end to end instead of optimizing intermediate paths?

More seriously, overreliance on external tools and rigid human constraints may slow the development of the model's own intelligence. This is the strongest argument from proponents of pure reinforcement learning: models must encounter real environments and receive feedback in order to improve. Surrounding tools and frameworks can produce a shielding effect (Hinton & Nowlan, Shielding Effect), weakening the possibility of intelligence itself evolving. If an individual can easily solve survival problems with external aids, the evolutionary pressure of natural selection on genes—or a model's underlying parameters—is shielded. Similarly, because our ancestors had abundant citrus fruit and could obtain vitamin C externally, genes responsible for synthesizing it degraded and were lost during evolution.

I must emphasize that every theory has boundaries. In domains with abundant data and almost zero feedback cost, we should certainly take a reinforcement-learning approach, scale the data, and increase compute. But not every real-world situation fits that description.

First, being able to internalize something does not mean doing so is efficient. By counting one at a time, people could answer “How many primes below 2⁶⁴ end in 1?” A sufficiently intelligent approach, however, is to write a Python script. A powerful LLM could internalize an adder, a multiplier, or even a complete CPU in its parameters, reason through a Linux system inside its neurons (Qwenputer), and run C and Python programs precisely without any external tools. But would a sufficiently intelligent model choose to do this? Using expensive neural-network inference to execute deterministic instructions can be orders of magnitude less efficient.

Second, intelligence and the tools it uses need not be seen as entirely separate (The Extended Mind). When solving arithmetic, people can make notes either in their heads or on paper. Should we ban scratch paper and calculators to maintain evolutionary selection pressure? Faith in “general model capabilities” should not lead us to abandon optimization of external tools and caches. If a robotics company wants a robot to travel quickly over long distances, the intelligent choice is to teach it to drive a car, rather than strengthen its legs until it can run at 120 kilometers per hour.

There is an even more direct example: why does the Transformer architecture itself run on specialized GPUs, rather than waiting for general-purpose CPUs to become powerful enough to outperform them? Every operator optimization assumes that compute cannot be expanded without limit. That assumption is valid.

The point of engineering optimization is therefore not to assume that model capabilities will stop improving. It is to acknowledge that models' behavioral patterns have boundaries: the principles underlying LLMs determine their ambiguity and inference costs. Let models use slow reasoning to explore unknown paths, then let systems automatically compile the discovered hot paths into symbolic caches. The stronger the model, the deeper the long-tail chains it can explore—and the richer the behavioral caches it can accumulate automatically.

7. Conclusion

The practice of intelligence engineering is only beginning. What matters is solving problems through action.

Wan Yanjie 2026.11.02

References

1. Compiler History and Foundational Theory:
   • Edsger W. Dijkstra, On the Foolishness of "Natural Language Programming" (EWD 667), 1978.
   • John Backus, The History of FORTRAN I, II, and III, ACM SIGPLAN Notices, 1978.
   • Chris Lattner et al., MLIR: Scaling Compiler Infrastructure for Domain Specific Computation, CGO 2021.
   • Hongzheng Chen, Compiled Cognition: When Agents Should Stop Thinking, 2026-07.
   • Andy Clark & David Chalmers, The Extended Mind, Analysis, 1998.
   • Rich Sutton, The Bitter Lesson, 2019; Geoffrey E. Hinton & Steven J. Nowlan, How Learning Can Guide Evolution (Shielding Effect), Complex Systems, 1987; Divyansh Agrawal, Qwenputer: a computer compiled into the weights of a pretrained language model, 2026-10.
   • Subbarao Kambhampati et al., LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks, ICML 2024.
   • Chao Li, The Engineering Philosophy of LLM Applications (The Engineering Philosophy of LLM Applications), 2026.
2. Natural Language as Source Code, Type Contracts, Specification-Driven Development (SDD), and Trace Observability:
   • Andrej Karpathy, The hottest new programming language is English, 2023; Sean Grove, The New Code, 2025.
   • Types and parsing: OpenAI Structured Outputs, Microsoft TypeChat, Pydantic, Instructor, BAML (BoundaryML).
   • Specifications: GitHub Spec Kit, AWS Kiro, OpenSpec, Tessl; empirical study Spec-Driven Development as Prescribed and as Practiced (arXiv:2608.25202)，2026-08.
   • Verification and trace observability: Guardrails AI, NeMo Guardrails, OpenTelemetry GenAI, Langfuse, Arize Phoenix, AgentOps, Braintrust.
   • Engineering reflections: Birgitta Böckeler & Martin Fowler, Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl, 2025-10; Tam et al., Let Me Speak Freely?, EMNLP 2024.
3. Inference Economics, Infrastructure Optimization, and Tool Primitives:
   • Cost trends: Epoch AI, The Plunging Price of Thought, 2026-09; Du, Tiered Super-Moore's Law (arXiv:2603.28576); The Price of Progress (arXiv:2511.23455).
   • Inference and caching optimization: KV Cache / vLLM PagedAttention, Anthropic Prompt Caching, OpenAI Prompt Caching, Speculative Decoding, semantic caching: GPTCache / Redis LangCache.
   • Protocols, primitives, and tool caching: MCP, Anthropic Skills, Tool Result Caching, Firecrawl, Context7, stateful tool-value caching: TVCache (2026-02).