Browse Comments — LLM coded

This gets me thinking in lot of other examples of hidden inference Name in prompt: “Help John Smith with his resume” vs “Help Priya Sharma with her resume.” AI suggestions subtly shift, different tone, different industry assumptions. Writing style: Formal academic English vs casual slang. Same question, different answers. AI adjusts confidence level, complexity, even what it omits. Currency/units: Type “$500 budget” vs “₹500 budget.” AI changes scope of recommendations entirely, not just currency conversion. Time format: “Schedule at 3pm” with no timezone. AI infers timezone from language/locale context, silently. Gender pronouns in context: Describe a nurse vs describe a surgeon. AI completion biases shift based on training data stereotypes, even when not asked. Looks like our system prompt keeps getting bigger

Great observation and analysis! That is why it's so important to ground any AI Medical tool in local clinical guidelines and escalation thresholds. You mentioned US defensive medicine culture leading to higher ER rates, but it could also very well be the model implicitly inferring closeness to hospitals (US vs Tokyo) in the case of a serious emergency and providing the better value/efficient approach. Would be useful to do an analysis of model's recommendations for rural vs urban settings (the same language/country context) to find out the exact source of its reasoning.

Love this insight! That's why thorough testing against good data will be the only way to make sure that an AI system is working properly and without bias!

Thanks Qi Han Wong, very interesting!This maps very directly to legal AI too. Language is not jurisdiction. A Spanish prompt may require Argentine, Spanish, Mexican, or US law. An English contract may still be governed by Argentine law. If the model silently treats language as a proxy for geography or governing law, it may understand the risk correctly but route the answer through the wrong institutional pathway. For legal and compliance AI, explicit jurisdiction anchoring is not a detail. It is a safety layer: governing law, forum, user location, institutional authority, and role of the user all matter.

Geographic anchoring may take care of logistical routing but at the same time erase a patient's biological identity by defaulting to Western clinical baselines by increasing genetic and biological blind spots. Medical AI safety requires decoupling genetics from location, prompting for both the physical location of the patient and their specific ethnic health predispositions. This problem is already existing example where patient of different ethinicty vists a GP in a different geograhical location My view is that Medical AI would be more efficent on regional flavour rather than one solution fits all

Angharad Hurley Now that you point it out, I have a feeling that particular sentence was AI generated (AI summary of the research?). I don’t quite agree with the sentence’s premise. Hmm. But to answer your question about whether training data is tested and validated... it’s not my field, but as far as I know... no. You can get “data poisoning” and models that collapse because they were trained on “synthetic data” (so AI generated training data, a photocopy of a photocopy!), some models have been trained using “distillation techniques” which basically is smaller models cribbing off other larger models (DeepSeek does this) and which may amplify biases. What I know from a red team perspective is that people are poisoning training data to leave backdoors open for jailbreak hacks. So no, I wouldn’t trust that training data Has been tested and validated, certainly not to the level that research scientists expect! I really value your question on this by the way, as it’s reminded me how researchers have far higher expectations of data than the models they might encounter, and most probably don’t ask!