Oh God, I never really thought about that in terms of AI for medical use but that actually makes AI sycophancy so much more terrifying.
Especially thinking in terms of how it could be used to dissuade people from seeking preventative care. (Which lets be honest, is probably exactly what is intended by at least some of the billionaires promoting AI in medicine as a convenient way to avoid having to go to the doctor for “every little thing.”)
On the contrary I’ve used it a bit for benign ouchies and it is very conventional and will always always refer you to an actual doctor.
But it might be another sycophancy aspect, and I wouldn’t be surprised if it went off the rail when you prime it with stuff like “give me a natural holistic ancestral remedy for XYZ”
Well I wouldn’t say it has discouraged care. Frankly it has been more accurate than I would have expected. It’s more that whenever you give it an update it goes wow that’s fantastic news! But then if you read the actual details it’s not so fantastic lol. So it will maintain this sort of fake positivity even as it accurately explains some of the bad things that could happen.
Oh God, I never really thought about that in terms of AI for medical use but that actually makes AI sycophancy so much more terrifying.
Especially thinking in terms of how it could be used to dissuade people from seeking preventative care. (Which lets be honest, is probably exactly what is intended by at least some of the billionaires promoting AI in medicine as a convenient way to avoid having to go to the doctor for “every little thing.”)
On the contrary I’ve used it a bit for benign ouchies and it is very conventional and will always always refer you to an actual doctor.
But it might be another sycophancy aspect, and I wouldn’t be surprised if it went off the rail when you prime it with stuff like “give me a natural holistic ancestral remedy for XYZ”
Well I wouldn’t say it has discouraged care. Frankly it has been more accurate than I would have expected. It’s more that whenever you give it an update it goes wow that’s fantastic news! But then if you read the actual details it’s not so fantastic lol. So it will maintain this sort of fake positivity even as it accurately explains some of the bad things that could happen.