Raw LLM Responses

We're going to see both sides of this coin again and again: a. Bad data / design leads to biased algorithms. b. Good data / design leads to correctly biased algorithms. I really worry that we don't have the appropriate nuance to have this discussion. Diagnosing which it is will be impossible for non-experts, and it follows almost along a reverse discrimination lawsuit: If we go the opposite way and say the model has to be, say, gender neutral, and a man loses out on, say, 10 points that by the math he should have gotten were the math done with fidelity, don't we then have the same problem we tried to correct?