Artificial intelligence (AI) is playing an expanding — and often invisible — role as machine learning tools are being deployed by police and prosecutors to identify faces, weapons, license plates and objects at crime scenes, survey live feeds for suspicious behavior, enhance DNA analysis, direct police to gunshots, determine how likely a defendant is to skip bail, forecast crime and process evidence, according to the National Institute of Justice. Trade secrets laws are blocking public scrutiny of how these tools work, creating a “black box” in the criminal justice system, with no guardrails for how AI can be used and when it must be disclosed. While AI tools are being used to inform criminal investigations, there is often no way for defendants to challenge their digital accuser or even know what role it played in the case, the Hill reports. “Under current law in most jurisdictions, [prosecutors] don’t have to disclose artificial intelligence use to the judge or defense counsel,” said Rebecca Wexler, professor of law at the University of California, Berkeley.
Concerns about AI in the criminal justice system are compounded by research showing how tools like facial recognition are prone to bias — for example, misidentifying people of color because it was trained on mostly white faces. For the past three congresses, Rep. Mark Takano (D-CA), joined twice by Rep. Dwight Evans (D-PA), has introduced legislation that addresses issues of testing and transparency in criminal justice, so far failing to garner enough traction to pass the bill. “Nobody had really addressed this particular issue of black box technologies that are being marketed to prosecutors, police and law enforcement folks on the basis of their alleged efficacy,” Takano said. “Every American wants to feel that they can get a fair trial if they are accused of something wrong — that’s one of the hallmarks of being an American,” he added. “But what do you do when the witness and evidence brought against you is a machine protected as a trade secret, how do you contend with that?”