The Algorithm Game

Jane Bambauer* & Tal Zarsky**

Most of the discourse on algorithmic decisionmaking, whether it comes in the form of praise or warning, assumes that algorithms apply to a static world. But automated decisionmaking is a dynamic process. Algorithms attempt to estimate some difficult-to-measure quality about a subject using proxies, and the subjects in turn change their behavior in order to game the system and get a better treatment for themselves (or, in some cases, to protest the system.) These behavioral changes can then prompt the algorithm to make corrections. The moves and countermoves create a dance that has great import to the fairness and efficiency of a decision-making process. And this dance can be structured through law. Yet existing law lacks a clear policy vision or even a coherent language to foster productive debate.

This Article provides the foundation. We describe gaming and countergaming strategies using credit scoring, employment markets, criminal investigation, and corporate reputation management as key examples. We then show how the law implicitly promotes or discourages these behaviors, with mixed effects on accuracy, distributional fairness, efficiency, and autonomy.

Continue reading in the print edition . . .

© 2018 Jane Bambauer & Tal Zarsky. Individuals and nonprofit institutions may reproduce and distribute copies of this Article in any format at or below cost, for educational purposes, so long as each copy identifies the authors, provides a citation to the Notre Dame Law Review, and includes this provision in the copyright notice.

*Professor of Law, University of Arizona James E. Rogers College of Law.

**Vice Dean and Professor, University of Haifa—Faculty of Law. We thank Solon Barocas, Gaia Bernstein, Kiel Brennan-Marquez, Gordon Hull, Yafit Lev-Aretz, Karen Levy, Helen Nissenbaum, Amit Elazari, Michael Veale, Felix Wu, David Lehr, Joseph Turow, Ignacio Cofone, Seda Guerses, Katherine Strandburg, Ian Kerr, Elana Zeide, Gaia Bernstein, Matthew Kugler, Kirsten Martin, BJ Ard, Emily Schlesinger, David Heyd, David Blankfein-Tabachnick, Adam Candeub, James M. Chen, Matthew Fletcher, Catherine Grosso, Mae Kuykendall, Michael Sant’Ambrogio, Larry Ponoroff, and the participants of our workshop at the Privacy Law Scholars Conference (U.C. Berkeley, California), participants of the Algorithmic States to Algorithmic Brains Workshop (NUI Galway, Ireland), participants of the ISF Workshop on User Modeling and Recommendation Systems (University of Haifa, Israel), participants of the Privacy, Cyber and Technology Workshop (Tel Aviv University, Israel), and participants of the workshop at Michigan State University College of Law. We are also grateful for excellent research assistance from Mack Thompson (University of Arizona, Candidate for Juris Doctor 2019).