006.3 — The Model Doesn't Know It Doesn't Know

Confidence is not accuracy. The model has learned to sound sure.

Confidence is not accuracy. The model has learned to sound sure in the same way it learned to sound human: by being trained on text produced by humans who sound sure. Certainty is a writing style. Th…