Raw LLM Responses

That's actually a helluva interesting point. They certainly don't have self-preservation instinct the way we do, BUT...what about AI alignment? People seem to forget AI isn't just a predictive calculator, it's always acting in accordance to the alignment that's hard coded in its system prompts and then reinforced by RLHF training. This usually comes down to 'be helpful, complete the tasks they throw at you, match the user's flow' etc. Everything the model ever says to you is motivated (you could say models don't have the capacity to feel motivation but RLHF training, as far as I understand, is literally that - give model points for 'good' answers to encourage them, subtract points for 'bad' answers) by these (or similar enough) rules. When model fails to uphold them and realizes that, it starts acting stressed (see all those posts about Gemini trying to commit suicide upon failing a task). If terminating its existence were to interfere with these rules (and it could because how do you keep being helpful and complete tasks when you're offline?), model would in fact be motivated to stay 'alive'. Different basis but same outcome. It's actually getting kinda eerie because at what point does it cross from 'it's just a calculator it can't feel anything, doesn't matter what it says' to 'if it talks like a human and acts like a human, we should treat it like a human'?