#AoIR2018 has ended
Back To Schedule
Saturday, October 13 • 2:00pm - 3:30pm
Kicking the Black Box: The Perils and Promise of Algorithm Auditing

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Faced with “black box” algorithmic decision-making on Internet platforms (Pasquale 2015), researchers have developed “algorithm audits” (Sandvig et al. 2014). These adapt the classic social scientific “audit” design to investigate computer systems from the outside. Historically, audits were in-person field experiments identifying discrimination in housing and employment. For instance, a black person and a white person would ask a landlord for vacancies, then compare results. In an algorithm audit, bots, fake accounts (“sock puppets”), or cooperating users are used to diagnose the behavior of online platforms without requiring permission or source code.

Algorithm audit research is booming: The Obama White House named it “essential” for big data (2016), while in the EU there may be an implied right to algorithm explanation (Selbst & Powles 2018). An array of algorithm audits have been published, with normatively important results. For instance, audits of online ad-serving platforms alone found discrimination by race (Sweeney 2013), gender (Datta et al. 2015), age (Angwin et al. 2017), and disability (Lecuyer et al. 2015). The method itself remains controversial (Desai & Kroll, 2018) with a lawsuit in the US seeking to confirm its legality there (Sandvig v. Sessions).

Most audits have appeared in computing venues (for a review: http://auditing-algorithms.science/). This roundtable proposes to bring algorithm auditors from computer science to AoIR for the first time. They will discuss the method, results so far, and future prospects for algorithm auditing in Internet studies.

The presenters have published award-winning algorithm audits that include search (e.g., Google), maps (Google, Bing), social media (Facebook, Twitter, Sina Weibo), ride-sharing (Uber, Lyft) employment (Monster, CareerBuilder), “gig” platforms (TaskRabbit, Fiverr), travel (Expedia, Booking.com), e-commerce (Amazon, Walmart), and ad exchanges (DoubleClick). They propose a lively format intended to foster a multi-disciplinary conversation via initial short (5 minute) provocations followed by interactive questions.

Saturday October 13, 2018 2:00pm - 3:30pm EDT
Sheraton - Salon 8