Raw LLM Responses

If you want a glimpse under the hood of how an LLM actually works, ask it for a seahorse emoji (which doesn't exist) while requiring the response to start with “Yes.” You’ll see it struggle to reconcile incompatible constraints, often producing evasive, inconsistent, or fabricated outputs. If these outputs are anthropomorphized, they might seem like the AI is going crazy, lying, or is otherwise performing some form of malpractice. But it has no intent; it is instead just statistically optimizing for the next token under conflicting requirements. No feelings or anything like that; it's all just simulated, perceived, and humanized. It has no intrinsic morality or goal other than optimizing outcomes, with the highest weight assigned to it during supervised fine-tuning and RLHF training. Giving unrestricted agency to something that has no moral baseline, survival instinct, or any other goal other than responding to a prompt is a really bad idea. In that sense, the “Shoggoth” metaphor is real, but not as an alien intelligence with hidden intentions. It is simply a distorted mirror of humanity itself, reflecting both the contents of its training data and the preferences of the people who assign rewards and weights. So don’t be afraid of the LLM; instead, be afraid of the data it is trained on (and its human origins) and the humans deciding what counts as a favorable outcome. TL;DR It's all conditioning, baby.