LLM Orchestration, Context Engineering & Agentic AI

TwoTwo people can type the same three-word question into the same chat window and get very different quality of answer, and the difference is rarely luck. It is usually that one of them is working with the model's raw defaults and the other is, deliberately or not, doing context engineering: shaping everything the model sees before it generates a single token. Understanding that distinction — and the further distinction between a single well-shaped prompt and an orchestrated, multi-step, tool-using system — is the difference between using an AI tool and being used by its defaults.

Context Engineering Is Not Prompt Writing

AA large language model has no persistent memory of you, your task, or your organisation between calls. Everything it "knows" about the current situation has to be present in the context window at the moment it generates a response, and that window is assembled from several distinct ingredients, not just the sentence you typed. A typical modern AI assistant call combines a system prompt (instructions set by the tool's designer, usually invisible to the user, defining role, tone, and constraints), few-shot examples (sample input/output pairs that demonstrate the desired pattern), any retrieved context pulled in from documents or a knowledge base for the specific query, tool or action definitions (descriptions of what external capabilities the model is allowed to invoke and how), a running conversation history or longer-term memory, and finally the user's own message. Context engineering is the discipline of deliberately designing and controlling all of these, not just the last one — a "custom GPT," a configured enterprise assistant, or a web-based agent builder is, under the hood, mostly a pre-packaged bundle of exactly these choices, made once by a designer so an end user doesn't have to remake them every time.

This matters practically because the single biggest quality lever most users never touch is the system prompt and the examples, not the wording of their own question. Two assistants built on the identical underlying model can behave completely differently in tone, accuracy, and usefulness purely because of how their context was engineered — which is also why the same tool can feel excellent for one task and unreliable for another: it was context-engineered for the first, not the second.

Prompting Techniques With Real Evidence Behind Them

TwoTwo of the best-evidenced techniques for shaping a model's output are worth knowing by name, because they explain why certain habits — that otherwise look like folklore — reliably work. Few-shot prompting — showing the model a small number of worked examples of the input/output pattern you want, rather than only describing it — was shown at scale to sharply improve a large language model's performance across a wide range of tasks without any retraining, simply by including examples directly in the prompt [1]. Chain-of-thought prompting — asking a model to work through a problem in visible intermediate steps, rather than jumping straight to a final answer — was shown to substantially improve performance on tasks that require multi-step reasoning, arithmetic, or logical deduction, precisely because it gives the model room to build up an answer incrementally instead of having to produce a correct result in one uninterruptible leap [2]. Neither of these requires any technical skill to use: "walk through this step by step before giving your answer" and "here are two examples of the kind of answer I want" are prompts anyone can type, and both are now standard advice precisely because the underlying research measured a real, repeatable effect rather than folk intuition.

Orchestration: Chaining Calls Toward a Fit-for-Purpose Output

AA single prompt asks a model to do everything at once: understand the task, gather what it needs, reason about it, and produce a polished final answer, all in one pass. Orchestration is the alternative: breaking a task into a sequence of smaller AI calls (and, often, non-AI steps such as a search or a database lookup), where the output of one step becomes the input to the next, and each step is scoped narrowly enough to do its one job well. A briefing document produced by orchestration might involve a retrieval step that gathers relevant source material, a drafting step that turns that material into prose, a critique step that checks the draft against the sources, and a formatting step that reshapes the result for its intended audience — four narrow, checkable jobs instead of one large, opaque one. This is also why retrieval-augmented generation is best understood as a special case of orchestration rather than a separate technology: it is exactly the "gather relevant material, then generate" pattern, wired together as a pipeline. The practical payoff of decomposing a task this way is that each step's output can be inspected and corrected independently, which is far harder to do with one large prompt whose reasoning is invisible from the outside.

Agentic AI: Planning, Acting, and Observing

OrchestrationOrchestration becomes agentic once the sequence of steps is no longer fixed in advance by a human designer, but is instead decided dynamically by the model itself, in response to what it observes at each step. The clearest formal statement of this pattern is ReAct ("reasoning and acting"), which interleaves three kinds of output from the model in a loop: a thought (reasoning in natural language about what to do next), an action (invoking a tool — a web search, a calculator, a database query, a piece of software), and an observation (the real-world result the tool returns), which then feeds into the next thought [3]. This loop is what separates an agentic system from a single-shot chat response: a chatbot answering a question produces one output and stops; an agent working the same question might search, read what it found, realise the first source was insufficient, search again with a better query, and only then produce an answer — deciding the number and order of steps for itself based on what actually happens along the way, not on a script written in advance.

Tool use itself needed to be demonstrated as something a language model could be taught to do reliably rather than assumed: Toolformer showed that a model could learn, from its own generated examples, when and how to call external tools such as a calculator, a search engine, or a calendar, deciding for itself which tool call would actually improve its answer to a given piece of text [4]. In a modern web-based agent builder, none of this mechanism is hidden from the person configuring it in principle — connecting an assistant to "actions" (a search tool, a calendar, a document store, a form-filler) and giving it a goal is, structurally, wiring up exactly the thought→action→observation loop above, without writing a line of code. Recognising that structure is what lets a non-programmer configure an agent sensibly — choosing which tools it genuinely needs, writing tool descriptions precise enough for the model's "thought" step to use them correctly, and anticipating where an unattended multi-step loop might wander off course — rather than treating the agent builder as an unaccountable black box.

Why This Matters for Fit-for-Purpose Outputs

FitFit for purpose" is doing real work in that phrase: the same underlying model, prompted well or badly, orchestrated or not, produces wildly different outputs for the same nominal task. A single naive prompt for "a report on X" and a context-engineered, orchestrated, tool-using agent set the same goal will diverge enormously in reliability, traceability, and quality — not because one used a "better" model, but because one deliberately engineered the context, decomposed the task, and gave the system a way to check its own intermediate work against reality, and the other did not. Treating prompting, orchestration, and agentic design as three points on the same spectrum — rather than three unrelated topics — is what makes it possible to choose, deliberately, how much structure a given task actually needs.

References

1. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., ... Amodei, D. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
2. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35, 24824–24837.
3. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. Proceedings of the 11th International Conference on Learning Representations (ICLR 2023).
4. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. Advances in Neural Information Processing Systems, 36.