Introduction
The integration of artificial intelligence (AI) and automated decision-making (ADM) systems into criminal justice and law enforcement is reshaping how these sectors operate globally. These technologies are employed to profile individuals, forecast potential criminal activities, and evaluate risks of future behavior. While proponents argue that such tools enhance public safety, their deployment raises serious concerns about fairness, equity, and the erosion of fundamental rights. This article examines the risks associated with predictive policing and advocates for its prohibition to protect justice and equality.
Predictive Policing: A Threat to Fundamental Rights
Predictive policing and risk-assessment technologies are no longer speculative concepts but operational tools influencing law enforcement practices worldwide. These systems guide critical decisions, including surveillance, stop-and-search operations, arrests, prosecutions, sentencing, and probation. However, their use often results in significant harm, particularly when individuals are flagged as potential offenders without evidence of wrongdoing.
Evidence indicates that predictive systems disproportionately target marginalized groups. For instance, the Netherlands’ “Top 600” initiative has singled out Moroccan youth for heightened police scrutiny, while Italy’s Delia system incorporates ethnicity data to predict criminality. Such practices amplify racial discrimination, undermine the presumption of innocence, and exacerbate social inequalities, perpetuating systemic biases within the justice system.
The Problem of Biased Data and Lack of Accountability
The core issue with predictive policing lies in its reliance on police databases and crime data, which are inherently biased. These datasets reflect where law enforcement focuses its efforts rather than the true distribution of criminal activity. In countries like the United Kingdom, this leads to over-policing of Black communities at every stage of the justice process, from stops and searches to incarceration.
When biased data is processed by algorithms, the resulting predictions reinforce and amplify existing inequalities, disproportionately affecting racialized and disadvantaged groups. Compounding this issue is the lack of transparency in these systems. Individuals are often unaware that they have been profiled or that AI has influenced decisions affecting their lives. Moreover, avenues for challenging these decisions or seeking redress are limited or entirely absent, eroding trust in justice institutions and undermining democratic accountability.
The consequences of AI-driven decisions extend beyond criminal justice. Data collected by police is increasingly shared with other authorities, impacting decisions related to immigration, housing, welfare, education, and child protection. This cross-sector use of AI threatens equality and risks long-term harm to already vulnerable populations.
The Urgent Need for Regulation and a Ban on Predictive AI
The growing recognition of these risks has prompted calls for robust regulation. In Europe, the Artificial Intelligence Act (AI Act), under negotiation as of June 2023, represents a critical step forward. The European Parliament’s vote to ban predictive policing systems and mandate greater transparency marks a significant milestone. However, many civil society organizations argue that these measures fall short of addressing the full scope of the problem. They advocate for a complete ban on predictive, profiling, and risk-assessment AI in criminal justice to ensure comprehensive protection against their harms.
Parallel efforts are underway at the Council of Europe, which is developing a legally binding convention on AI, human rights, and the rule of law. Human rights organizations, participating as observers on its Committee on AI, are pushing for explicit acknowledgment of the profound dangers posed by these technologies.
Recommendations for Safeguarding Justice
To address the risks of AI in criminal justice, experts propose the following measures:
- Complete Prohibition of Predictive and Profiling Systems: Ban the use of predictive and profiling AI in law enforcement to prevent discrimination and protect fundamental rights.
- Mandatory Bias Testing: Require rigorous bias testing for all AI tools used in justice systems, supported by improved demographic data collection to identify and mitigate disparities.
- Enhanced Transparency: Implement measures to ensure clarity about how AI systems operate and how decisions are made, fostering public trust and accountability.
- Human Accountability: Mandate that decision-makers provide clear evidence and reasoning for choices influenced by AI, ensuring human oversight remains central.
- Effective Redress Mechanisms: Establish accessible pathways for individuals to challenge automated decisions and seek remedies for harm caused.
Conclusion
Without stringent regulation, AI in criminal justice risks entrenching discrimination and perpetuating inequality within already flawed systems. Predictive policing, in particular, threatens the presumption of innocence and undermines the principles of fairness and justice. A comprehensive ban on these technologies, coupled with robust safeguards for other AI applications, is essential to protect fundamental rights and ensure equitable treatment for all. As the global community grapples with the implications of AI, prioritizing human rights and accountability is critical to building a just and inclusive future.
