Study reveals AI chatbots can detect race, but racial bias reduces response empathy

Submitted ⁨⁨1⁩ ⁨day⁩ ago⁩ by ⁨Joker@sh.itjust.works⁩ to ⁨technology@lemmy.zip⁩

https://news.mit.edu/2024/study-reveals-ai-chatbots-can-detect-race-but-racial-bias-reduces-response-empathy-1216

Researchers at MIT, NYU, and UCLA develop an approach to help evaluate whether large language models like GPT-4 are equitable enough to be clinically viable for mental health support.

source

Comments

Sort:hotnew top

PhilipTheBucket@ponder.cat ⁨1⁩ ⁨day⁩ ago

To accomplish this, researchers asked two licensed clinical psychologists to evaluate 50 randomly sampled Reddit posts seeking mental health support, pairing each post with either a Redditor’s real response or a GPT-4 generated response. Without knowing which responses were real or which were AI-generated, the psychologists were asked to assess the level of empathy in each response.

I am doubtful that this framework is going to accurately detect anything at all about the usefulness of chatbots in this context, whether about race or anything else.

I don’t think using chatbots for psychology is a good idea, but this study isn’t the way to study and make that determination.

source
- rumba@lemmy.zip ⁨1⁩ ⁨day⁩ ago
  The problem with using GPT as it is currently, you can ask it the same question 27 tomes and get 18 different answers. One of them a hallucination.
  
  source
AllNewTypeFace@leminal.space ⁨1⁩ ⁨day⁩ ago
As we speak, Elon Musk has his best engineers working on developing artificial racism

source
- possiblylinux127@lemmy.zip ⁨1⁩ ⁨day⁩ ago
  Chat bots are already racist. They just have to let it run wild.
  
  source
  - Free_Opinions@feddit.uk ⁨1⁩ ⁨day⁩ ago
    It’s physically impossible for an LLM to hold prejudice.
    
    source
    -> View More Comments