Can AI predict which criminals may break the law again?

Computer algorithms can outperform people in predicting which criminals will be arrested again, according to a new study. Risk-assessment algorithms that predict future crimes often help judges and parole boards decide who stays behind bars, but these systems have been criticized for exhibiting racial bias, and some research has given reason to doubt whether that algorithms are better at predicting arrests than humans. A 2018 study pitting human volunteers against the COMPAS risk assessment tool found that people predicted criminal recidivism just as well as the software.

The new set of experiments confirms that humans predict repeat offenders, as well as algorithms, when people receive immediate feedback on the accuracy of their predictions and are shown limited information about each offender; but people are worse than computers when people don’t get feedback or are shown more detailed criminal profiles.

In reality, judges and parole boards also don’t get instant feedback and usually have a lot of information to work with to make their decisions; So, the study findings suggest that, under realistic prediction conditions, algorithms outperform people in predicting recidivism.

Computational social scientists began by mimicking the setup of the 2018 study. Online volunteers read brief descriptions of 50 offenders, including characteristics such as gender, age, and number of prior arrests, and guessed whether each person was likely to be arrested for another crime within two years; after each round, volunteers were told whether they had guessed correctly. As seen in 2018, people rivaled the performance of COMPAS – accurate about 65 percent of the time.

But in a slightly different version of this human-computer competition, the team of scientists found that COMPAS had an advantage over people who didn’t get feedback. With feedback, humans performed this task with 83 percent accuracy, close to 89 percent for COMPASS, but without feedback, human accuracy dropped to about 60 percent; that’s because people overestimate the risk of criminals committing violent crime, despite being told that only 11 percent of criminals in the data set fell into this camp, the researchers say.