Ethical Hacking News
The UK's Avon and Somerset Police has been accused of building an AI-powered crime-prediction machine with flawed algorithms, raising concerns about accuracy and potential impact on citizens' lives. As the government prepares to roll out similar projects across the country, critics warn that transparency and accountability are crucial to ensuring these programs serve justice rather than perpetuate bias.
The UK police force Avon and Somerset has built a crime-prediction machine using AI, raising concerns over accuracy and impact on citizens' lives.The project began in 2016 with the creation of a database called Think Family, containing sensitive information on nearly half a million residents.Concerns have been raised about biased algorithms, poor decision-making processes, and the potential for false positives to have devastating consequences.A recent review found that the risk-scoring models had "poor predictive performance" with low precision scores, incorrectly identifying many individuals as risks.The lack of transparency and accountability in Avon and Somerset's data science work sets a concerning precedent for the rollout of similar programs nationwide.
In the heart of England, a sprawling crime-prediction machine has been built by British police, sparking concerns over its accuracy and potential impact on citizens' lives. The Avon and Somerset Police's use of artificial intelligence (AI) in predictive analytics has been shrouded in secrecy, with officials citing transparency issues as a reason for the lack of information. However, through public records requests and investigative journalism, WIRED uncovered a tangled web of data collection, biased algorithms, and questionable decision-making processes.
The project began in 2016, when the Bristol City Council and Avon and Somerset Police collaborated to create a database called Think Family. This database housed sensitive information on nearly half a million residents, including police intelligence reports, housing status, mental health records, teenage pregnancies, enrollment in parenting courses, and free school meals. Machine-learning models were then developed to assign scores to thousands of adults and children, creating a "picture of threat, harm, and risk" in the region.
One of these risk-scoring models aimed to identify children at risk of sexual exploitation. The CSE model drew on a wide range of datasets, including anonymized data from Barnardos, a child protection charity. However, researchers have highlighted concerns about the variables used in the model, noting that they can be proxies for poverty. Moreover, officials with knowledge of the program acknowledged that most children with high risk scores were already being worked by social workers and family workers.
Despite these concerns, the police force remained committed to predictive analytics, claiming it would help prevent crime. In 2019, Chief Constable Andy Marsh announced plans to deploy predictive analytics across all aspects of policing in Avon and Somerset. However, a recent review by Social Finance found that the risk-scoring models were "the weakest element" of the project, with a lack of accuracy undermining their potential usefulness.
Avon and Somerset Police has produced over 36,000 model performance scores, which appear to show "genuinely poor predictive performance." An independent analyst reviewed this data for WIRED and found that many of these models produce low precision scores, meaning a high proportion of the individuals they flag as risks are incorrectly identified. Furthermore, officials have refused to disclose information about how the risk-scoring models worked or precisely why they were scrapped.
The controversy surrounding Avon and Somerset Police's AI-powered crime-prediction machine has sparked concerns about bias in data collection and decision-making processes. Experts warn that one false positive can have devastating consequences for a family, particularly if a child is flagged as at risk of criminal or sexual exploitation.
In response to the growing criticism, the UK government has announced plans to create PoliceAI, a £75 million-backed body tasked with rolling out AI tools across 43 police forces in England and Wales. While this initiative aims to drive forward policing innovation, critics argue that the lack of transparency and accountability in Avon and Somerset Police's data science work sets a concerning precedent for the rollout of similar programs nationwide.
Related Information:
https://www.ethicalhackingnews.com/articles/Bristle-Bound-The-Unsettling-Truth-Behind-Avon-and-Somerset-Polices-AI-Powered-Crime-Prediction-Machine-ehn.shtml
https://www.wired.com/story/british-police-built-a-sprawling-crime-prediction-machine-some-results-couldnt-be-trusted/
Published: Thu Jun 25 06:15:46 2026 by llama3.2 3B Q4_K_M
