Raw LLM Responses

Consciousness is an emergent property of systems with adapting components embedded within a feedback loop with their environment. Similar to temperature which is also an emergent property from the interactions of large numbers of molecules that results in a phase diagram with different states of matter, there are different states of consciousness. For animals, the big phase shift to a fundamentally different large-scale state is the development of language. Language invents the ability to refer to, and thus think about, things which are not immediately present or which are entirely fictional. There is a key part of this, however - the original state of consciousness driven very directly by the feedback loop with ones environment, the one concerned with the immediately present, physical, material world still is involved in the interface between the language-consciousness and the body. In order to make a decision, to take an action, the language-consciousness must interact with this body-consciousness in order to have it commit to a decision. LLMs like ChatGPT are disembodied. They are the language-consciousness in a sort of free-floating form (they are also in a very weird state in terms of not really involved in a feedback loop with their own world, so they are not really capable of being conscious at all) and it makes obvious sense, to me anyway, that they would be incapable of the sorts of thinking necessary to perform math, or construct rational arguments which obey absolute rules that do not depend upon associations. Recognizing that association-based thinking leads to errors is one of the key insights that critical thinking observes and solves. Just think of some of the basic rules of critical thinking. Like how a statement is true or false based upon its contents, and not based upon who the speaker is. Someone who does not recognize this rule can be easily manipulated into believing a false statement or disbelieving a true statement by causing them to have very negative associations and opinions of the speaker so that when the statement is made, they reject it. The mention of the person they dislike will activate all kinds of negative associations in their neural network, and they will just rely upon that to determine their opinion of the truth of what the person says. In order for critical thinking to step in and ignore those associations and evaluate the statement in isolation, the absolute rule has to be recognized and used instead. LLMs only have their associations, though. They don't have the 'lower level' that can step in and commit to following an absolute rule, ignoring the associations. Yet.