Knowledge Without a Knower? Machine Epistemology

AA language model states a wrong date with exactly the same fluent confidence it states a correct one. The usual word for this, hallucination, names the output but not the actual problem. The actual problem is a mismatch among three things that ordinarily travel together and have silently come apart: how confident the language sounds, how strong the evidence behind it actually is, and how much access to real information the user assumes the system has. Treating hallucination as a factual-error rate to be minimised misses that the mismatch would be exactly as dangerous even on a true statement — the same process that states a false date with total confidence states a true one the same way, for the same reasons, and a user has no way to tell the two apart from the sentence alone.

Justified True Belief, and Where a Machine Breaks It FoundationalKnowledge that endures for decades — core principles

TheThe classical analysis of knowledge — a true belief, held for good reason — already runs into trouble for ordinary human cases, since Gettier's famous counterexamples show a belief can be true and apparently well-justified by pure accident, which few would call genuine knowledge¹. A system raises the prior question more sharply: does it have beliefs at all, in the sense the analysis presupposes? This series' page on the intentional stance already gives the most defensible answer available — a belief attribution is legitimate when it affords real predictive compression, not when some inner token is located — which lets "the model believes X" mean something precise without requiring a settled answer about inner experience. What the classical analysis still demands, even granting that much, is justification: some good reason the belief is true, not just a lucky coincidence that it is.

Reliabilism: A Better Fit Than It First Looks FoundationalKnowledge that endures for decades — core principles

Goldman'sGoldman's reliabilism answers a version of this problem for ordinary perception and memory: a belief is justified if it's produced by a reliable process — one that tends to produce true beliefs across the relevant range of cases — whether or not the believer can introspect on or articulate why the process is reliable². This travels unusually well to a trained model. A system's output on a well-represented, well-calibrated topic can be produced by a process with a measurable, checkable track record of accuracy, which is exactly what reliabilism asks for — and critically, this doesn't require the system to know how it knows, only that the process generating the answer is in fact reliable. Reliabilism is also exactly where the account comes under the most strain: a process can be reliable on one distribution of inputs and silently unreliable on a different one the system cannot distinguish from inside, and "the process that produced this answer is usually reliable" is a very different, weaker claim than "this specific answer is reliable," with no way for the system itself to flag which situation it's currently in.

Can Training Material Function as Testimony? FoundationalKnowledge that endures for decades — core principles

TestimonyTestimony is a recognised source of knowledge in its own right — much of what any person knows, they know because someone else told them, not because they verified it directly — but testimonial knowledge normally depends on a testifier who could, in principle, be asked to justify the claim or be held responsible for its accuracy. Training data complicates this structurally rather than just practically: a model's output is shaped by an enormous aggregate of text from sources with no individual accountability for the specific claim the model eventually produces, recombined in ways no single original author anticipated or endorsed. This isn't testimony failing to transmit cleanly — it's something that doesn't fit the category's basic shape, with no answer to "who could be asked to justify this" that points at any actual person.

Who Is the Epistemic Agent? FoundationalKnowledge that endures for decades — core principles

AA further question this series has deferred until it had the vocabulary to ask precisely: when a system produces a well-justified true claim, who is the epistemic agent — the model, the surrounding software system, the developers who built and evaluated it, or the combined human-machine arrangement actually making use of the output? This matters practically, not just theoretically, because of one asymmetry worth stating directly: a human testifier can take responsibility for a claim — defend it, revise it on challenge, accept consequences for asserting it falsely. A trained model cannot do any of this in the sense that matters; whatever stands behind its outputs, responsibility for them is held, if it's held anywhere, by the people and institutions that built, evaluated, and deployed it. A system that cannot assume responsibility for a claim is, on many accounts of epistemic agency, not a full epistemic agent with respect to that claim at all — which doesn't mean its outputs carry no epistemic weight, only that the weight has to attach somewhere else.

How Uncertainty Should Be Represented Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

IfIf the core problem is a mismatch between confident-sounding language and the actual strength of the evidence, the fix has to operate on that gap directly rather than on factual accuracy alone. A system whose language tracked its own calibration — hedging exactly as much as its actual track record on similar claims warrants, distinguishing a claim resting on strong, convergent evidence from one resting on a single uncertain source — would be addressing the mismatch this page opened with, independent of whether any given claim turns out true. This is the same ground this series' page on the deception criterion covers from the evaluation side; here the same demand is read as an epistemic one: calibrated confidence is not a nicety added on top of knowledge, it's part of what separates a justified belief from a lucky guess that happened to sound the same.

Strongest Objection: This Makes "Knowledge" Do Too Much Work FoundationalKnowledge that endures for decades — core principles

AA fair complaint: stretching "belief," "justification," and "testimony" to cover a trained model risks producing a vocabulary so generalised it stops tracking the distinctions that made these terms useful for human knowers in the first place. The reply available to this page is the same move the Dennett page makes for belief specifically: these terms earn their keep here exactly to the extent they predict and explain something real about how the system behaves and fails, not because a prior theory says they must apply. Where they stop paying their way — testimony's dependence on an accountable testifier is the clearest case above — the better move is to say so and reach for a different vocabulary, not to force the fit.

Provisional Conclusion FoundationalKnowledge that endures for decades — core principles

AA trained system's outputs can be justified in a recognisably reliabilist sense without the system being an epistemic agent that could stand behind them, and without its training data functioning as testimony in the sense that normally grounds secondhand knowledge. Hallucination, read this way, isn't primarily a factual-accuracy problem — it's a standing mismatch between confidence, evidence, and responsibility that a true statement and a false one both exhibit identically, and that only calibrated, carefully-scoped language can actually address.

Questions for Further Thought

• If no individual source in a model's training data can be held responsible for a specific claim, does the aggregate somehow acquire a responsibility none of its parts had?
• What would a reliabilist standard actually require a deployed system to track about its own accuracy, distribution by distribution?
• Is there a human analogue to "justified but not testimony and not full epistemic agency" — a source of true, well-grounded belief that doesn't fit the normal categories either?

Related Topics

• Belief Without an Inner Believer? Dennett and Artificial Agency — the account of belief-attribution this page's justification question is built on top of.
• The Deception Criterion — calibration and disclosure as evaluation criteria, here read as epistemic requirements rather than test design.
• The Compression of Knowledge: From Explicit Graphs to Latent Representations — what a trained representation actually stores, which this page's reliabilism treats as a black-box process rather than examining directly.
• Knowledge Graph Fundamentals — an explicit, inspectable alternative to the opaque reliability this page examines, where justification is closer to traceable by construction.

Further Reading

• Gettier, E. L. (1963). Is justified true belief knowledge? Analysis, 23(6), 121–123.
• Goldman, A. I. (1979). What is justified belief? In G. S. Pappas (Ed.), Justification and Knowledge: New Studies in Epistemology (pp. 1–23). Reidel.

References