Police deployment of AI for report writing and facial recognition is accompanied by errors with serious consequences for those arrested
Police in the USA, Canada and UK are deploying AI (ChatGPT for writing reports, facial recognition for identifying people) without regulation. Errors led to an innocent woman spending five months in custody and a teenager nearly being shot because of a bag of chips.
Police forces in the USA, Canada and Great Britain have begun deploying artificial intelligence tools – from generative tools for writing police reports to facial recognition systems and surveillance technologies – without consistent regulation or effective oversight mechanisms. According to the article, the Royal Canadian Mounted Police (RCMP) completed a pilot project as part of its body camera programme, using a modified version of ChatGPT to write police reports. In Utah, a report generated this way incorrectly stated that an officer had “turned into a frog” during an intervention – this was a misinterpretation of audio from the Disney film The Princess and the Frog playing in the background of the body camera recording. It was only this incident that prompted a police administrator to admit that he had realised the importance of reviewing AI-generated reports.
The article also states that facial recognition used to identify people makes errors more frequently for women and people with darker skin. In North Dakota, a woman from Tennessee was detained based on an incorrect AI identification and spent more than five months in custody, even though she had not committed the crime in question. In Maryland late last year, a police officer pointed a gun at a high school student after an AI surveillance system mistakenly identified a bag of potato chips as a possible weapon. In Britain, according to the article, a detective also improperly used AI to create evidence in several cases, and police in the USA and Canada published AI-generated images of seized drugs.
The authors of the article (The Conversation) point out that members of the public cannot avoid interactions with the police, yet have not consented to their image, voice and personal data from body cameras being used to train AI models. It is not known whether this data will be used for the further development of AI tools for police use. According to the article, there is no uniform regulatory framework for AI deployment across Canadian police forces, and some forces declined to comment on their AI policies. Based on the cases described, the authors argue that the use of AI by Canadian police should be banned because it fundamentally undermines procedural justice and the legitimacy of police work.
Why it matters
The text presents specific cases in which unchecked AI deployment in police work led to real harm – months in custody for an innocent person and a threat to a teenager's life – and highlights the lack of regulation, checks on the accuracy of AI-generated evidence and reports, and uncertainty around the use of body camera data to train models. For sectors using AI in decision-making with legal consequences, this is a documented example of the risks that arise when human review of outputs is absent.
Relevant practical impact
What this means
For a business
Cases from police work show that deploying generative AI and facial recognition in decision-making processes with serious consequences without rigorous human oversight leads to errors, legal liability and a loss of public trust — a relevant lesson for companies supplying or operating AI in public administration, security and other high-risk applications.
Risks and complianceCheck the original
Event sources
only one source so far · 1 publisher, 1 independent. We count feeds from the same owner only once.