This is really bad. It shows that LLMs cannot be trusted, since there is no corrective instance above them. According to this video (German), Microsoft, who has been questioned about this incident, claims that the Copilot feature is to be seen as entertainment rather than a search engine or anything what is considered as a serious source. Although search results regarding the victim have been deleted after they had reached out to Microsoft, the same wrong search results re-appeared a few days later.

With the raise of fake news over the last decade, combined with a growing lack of media literacy, such feature can destroy lives, especially when people tend to ignore the sources. A victim barely has any possibility to proof that these facts are wrong and a result of a hallucinating LLM. And even if - the internet doesn’t forget. Fake news will be circleing around as well as legit news.