AI & SOCIETY (2022) 37:215–230 https://doi.org/10.1007/s00146-021-01154-8
OPEN FORUM
The ethics of algorithms: key problems and solutions
1 · 1,3 · Josh Cowls1,2 · 1 · Huw Roberts1 · 1,2 · 1,2
Copyright By PowCoder代写 加微信 powcoder
Received: 27 July 2020 / Accepted: 22 January 2021 / Published online: 20 February 2021 © The Author(s) 2021
Research on the ethics of algorithms has grown substantially over the past decade. Alongside the exponential development and application of machine learning algorithms, new ethical problems and solutions relating to their ubiquitous use in soci- ety have been proposed. This article builds on a review of the ethics of algorithms published in 2016 (Mittelstadt et al. Big Data Soc 3(2), 2016). The goals are to contribute to the debate on the identification and analysis of the ethical implications of algorithms, to provide an updated analysis of epistemic and normative concerns, and to offer actionable guidance for the governance of the design, development and deployment of algorithms.
Keywords Algorithm · Artificial intelligence · Autonomy · Digital ethics · Explainability · Fairness · Machine learning · Privacy · Responsibility · Transparency · Trust
1 Introduction
Algorithms have become a key element underpinning cru- cial services and infrastructures of information societies. Individuals interact with recommender systems—algorith- mic systems that make suggestions about what a user may like—on a daily basis, be it to choose a song, a movie, a product or even a friend (Paraschakis 2017; Perra and Rocha 2019; Milano et al. 2020). At the same time, schools and hospitals (Obermeyer et al. 2019; Zhou et al. 2019; Mor- ley et al. 2019a, b), financial institutions (Lee and Floridi 2020; Aggarwal 2020) courts (Green and Chen 2019; Yu and Du 2019), local governmental bodies (Eubanks 2017; Lewis 2019), and national governments (Labati et al. 2016; Hauer 2019; Taddeo and Floridi 2018a; Taddeo et al. 2019;
Roberts et al. 2019), all increasingly rely on algorithms to make significant decisions.
The potential for algorithms to improve individual and social welfare comes with significant ethical risks (Floridi and Taddeo 2016). Algorithms are not ethically neutral. Consider, for example, how the outputs of translation and search engine algorithms are largely perceived as objective, yet frequently encode language in gendered ways (Larson 2017; Prates et al. 2019). Bias has also been reported in algorithmic advertisement, with opportunities for higher- paying jobs and jobs within the field of science and tech- nology advertised to men more often than to women (Datta et al. 2015; Lambrecht and Tucker 2019). Likewise, pre- diction algorithms used to manage the health data of mil- lions of patients in the United States exacerbate existing problems, with white patients given measurably better care than comparably similar, black patients (Obermeyer et al. 2019). While solutions to these issues are being discussed and designed, the number of algorithmic systems exhibiting ethical problems continues to grow.
Since 2012, artificial intelligence (AI) has been experienc- ing a new ‘summer’, both in terms of the technical advances being made and the attention that the field has received from academics, policy makers, technologists, and investors
Oxford OX1 3JS, UK
Oxford Internet Institute, University of Oxford, 1 St Giles’,
London NW1 2DB, UK
Alan Turing Institute, British Library, 96 Euston Rd,
St. Cross Road, Oxford OX1 3UL, UK
Faculty of Law, University of Oxford, St. Cross Building,
Vol.:(01233456789)
216 AI & SOCIETY (2022) 37:215–230
Fig. 1 Six types of ethical concerns raised by algorithms (Mittelstadt et al. 2016, 4)
(Perrault et al. 2019). Within this, there has been a growing body of research on the ethical implications of algorithms, particularly in relation to fairness, accountability, and transpar- ency (Lee 2018; Hoffmann et al. 2018; Shin and Park 2019). In 2016, our research group at the Digital Ethics Lab published a comprehensive study that sought to map these ethical concerns (Mittelstadt et al. 2016). However, this is a fast-changing field and both novel ethical problems and ways to address them have emerged, making it necessary to improve and update that study.
In particular, work on the ethics of algorithms has increased significantly since 2016, when national governments, non- governmental organisations, and private companies started to take a prominent role in the conversation on “fair” and “ethical” AI and algorithms (Sandvig et al. 2016; Binns 2018; Selbst et al. 2019; Wong 2019; Ochigame 2019). Both the quantity and the quality of the research available on the topic have expanded enormously. Given these changes, this article updates our previous work in light of new insights into the ethics of algorithms, updates the initial analysis, includes refer- ences to the literature that were missed by the original review, and extends the analysed topics, including for example work on AI for social good (see the Sect. 9). At the same time, the conceptual map proposed in 2016 (see Fig. 1) remains a fruit- ful framework for reviewing the current debate on the ethics of algorithms, identifying the ethical problems that algorithms give rise to, and the solutions that have been proposed in recent relevant literature. Specifically, in Sect. 2, we summarise the conceptual map and detail our methodology for the literature review. In Sects. 3–8, we offer systematic search and review (in the methodological sense specified by Grant and Booth 2009) on the ethics of algorithms and draw links with the types of ethical concerns previously identified. Section 9 concludes the article with an overview.
2 Map of the ethics of algorithms
There is little agreement in the relevant literature on the definition of an algorithm. The term is often used to indi- cate both the formal definition of an algorithm as a math- ematical construct, with ‘‘a finite, abstract, effective, com- pound control structure, imperatively given, accomplishing a given purpose under given provisions” (Hill 2016, 47), as well as domain-specific understandings which focus on the implementation of these mathematical constructs into a technology configured for a specific task. In this article, we decided to maintain the same approach adopted in the 2016 article and to focus on the ethical issues posed by algo- rithms as mathematical constructs, their implementations as programs and configurations (applications), and the ways in which these can be addressed. We consider algorithms that are used to (1) turn data into evidence for a given outcome, which is used to, (2) trigger and motivate an action that may have ethical consequences. Actions (1) and (2) may be per- formed by (semi-)autonomous algorithms—such as machine learning (ML) algorithms—and this complicates, (3) the attribution of responsibility for the effects of actions that an algorithm may trigger. Here, ML is of particular interest, as a field which includes deep learning architectures. Com- puter systems deploying ML algorithms may be described as “autonomous” or “semi-autonomous”, to the extent that their outputs are induced from data and thus, non-deterministic.
Based on this approach, we used the conceptual map shown in Fig. 1 to identify the ethical issues that algorithms pose. The map identifies six ethical concerns, which define the conceptual space of the ethics of algorithms as a field of research. Three of the ethical concerns refer to epistemic fac- tors, specifically: inconclusive, inscrutable, and misguided evidence. Two are explicitly normative: unfair outcomes and transformative effects; while one—traceability—is relevant both for epistemic and normative purposes.
The epistemic factors in the map highlight the relevance of the quality and accuracy of the data for the justifiability of the conclusions that algorithms reach and which, in turn, may shape morally-loaded decisions affecting indi- viduals, societies, and the environment. The normative concerns identified in the map refer explicitly to the ethi- cal impact of algorithmically-driven actions and decisions, including lack of transparency (opacity) of algorithmic processes, unfair outcomes, and unintended consequences. Epistemic and normative concerns, together with the dis- tribution of the design, development, and deployment of algorithms make it hard to trace the chain of events and factors leading to a given outcome, thus, hindering the possibility of identifying its cause, and of attributing moral responsibility for it. This is what the sixth ethical concern, traceability, refers to.
AI & SOCIETY (2022) 37:215–230
Table 1 Systematic literature search results
Database Scopus
Web of Science
Philpapers Google Scholar
Algorithm* AND ethics AND traceability
AND fairness
AND autonomy
AND (accountability OR responsibility)
AND (transparency OR scrutability OR opacity) AND discrimination
Algorithm* AND ethics
AND traceability
AND fairness
AND autonomy
AND (accountability OR responsibility)
AND (transparency OR scrutability OR opacity) AND discrimination
Algorithm* AND ethics
Algorithm* AND ethics
It is important to stress that this conceptual map can be interpreted at both a micro- and macro-ethical level. At the micro-ethical level, it sheds light on the ethical problems that particular algorithms may pose. By highlighting how these issues are inseparable from those related to data and responsibilities, it shows the need to take a macro-ethical approach to addressing the ethics of algorithms as part of a wider conceptual space, namely, digital ethics (Floridi and Taddeo 2016). As Floridi and Taddeo argue:
“While they are distinct lines of research, the ethics of data, algorithms and practices are obviously inter- twined … [Digital] ethics must address the whole conceptual space and hence all three axes of research together, even if with different priorities and focus” (Floridi and Taddeo 2016, 4).
In the remainder of this article, we address each of these six ethical concerns in turn, offering an updated analysis of the ethics of algorithms literature (at a micro level), with the goal of contributing to the debate on digital ethics (at a macro level).
A systematic literature search was performed via keyword queries on four widely used reference repositories to identify and analyse the literature on the ethics of algorithms (seeTa- ble 1). Four keywords were used to describe an algorithm: ‘algorithm’, ‘machine learning’, ‘software’ and ‘computer program’.1 The search was limited to publications made available between November 2016 and March 2020.
1 The literature search was limited to English language articles in peer-reviewed journals and conference proceedings.
The search identified 4891 unique papers for review.2 After initial review of title/abstract, 180 papers were selected for a full review. Of these, 62 were rejected as off-topic, leaving 118 articles for a full review. There are all listed in the reference list of the paper. Another 37 articles and books were reviewed and referenced in this paper to provide additional information regarding specific ethical issues and solutions (eg. technical details, examples and tools). These were sourced from the bibliographies of the 118 articles we reviewed as well as provided on an ad-hoc basis when agreed upon by the authors as being helpful for clarification.
3 Inconclusive evidence leading to unjustified actions
Research focusing on inconclusive evidence refers to the way in which non-deterministic, ML algorithms produce outputs that are expressed in probabilistic terms (James et al. 2013; Valiant 1984). These types of algorithms gener- ally identify association and correlation between variables in the underlying data, but not causal connections. As such, they encourage the practice of apophenia: “seeing patterns where none actually exist, simply because massive quantities of data can offer connections that radiate in all directions” (boyd and Crawford 2012, 668). This is highly problem- atic, as patterns identified by algorithms may be the result of inherent properties of the system modelled by the data,
2 Many of which were purely technical in nature, especially for “dis- crimination” and “(transparency OR scrutability OR opacity)”.
aAbout 93 000 returned, first 100 reviewed
AI & SOCIETY (2022) 37:215–230
of the datasets (that is, of the model itself, rather than the underlying system), or of skillful manipulation of datasets (properties neither of the model nor of the system). This is the case, for example, of Simpson’s paradox, when trends that are observed in different groups of data reverse when the data is aggregated (Blyth 1972). In the last two cases, poor quality of the data leads to inconclusive evidence to support human decisions.
Recent research has underlined the concern that inconclu- sive evidence can give rise to serious ethical risks. For exam- ple, focusing on non-causal indicators may distract attention from the underlying causes of a given problem (Floridi et al. 2020). Even with the use of causal methods, the available data may not always contain enough information to justify an action or make a decision fair (Olhede and Wolfe 2018, 7). Data quality—the timeliness, completeness and correctness of a dataset—constrains the questions that can be answered using a given dataset (Olteanu et al. 2016). Additionally, the insights that can be extracted from datasets are fundamentally depend- ent on the assumptions that guided the data collection process itself (Diakopoulos and Koliska 2017). For example, algo- rithms designed to predict patient outcomes in clinical settings rely entirely on data inputs that can be quantified (e.g. vital signs and previous success rates of comparative treatments), whilst ignoring other emotional facts (e.g. the willingness to live) which can have a significant impact on patient outcomes, and thus, undermine the accuracy of the algorithmic prediction (Buhmann, Paßmann, and Fieseler 2019). This example high- lights how insights stemming from algorithmic data processing can be uncertain, incomplete, and time-sensitive (Diakopoulos and Koliska 2017).
One may embrace a naïve, inductivist approach and assume that inconclusive evidence can be avoided if algorithms are fed enough data, even if a causal explanation for these results cannot be established. Yet, recent research rejects this view. In particular, literature focusing on the ethical risks of racial profiling using algorithmic systems has demonstrated the limits of this approach highlighting, among other things, that long-standing structural inequalities are often deeply embed- ded in the algorithms’ datasets and are rarely, if ever, corrected for (Hu 2017; 2018; Noble 2018; Benjamin 2019; Richardson et al. 2019; Abebe et al. 2020). More data by them- selves do not lead to greater accuracy or greater representation. On the contrary, they may exacerbate issues of inconclusive data by enabling correlations to be found where there really are none. As Ruha Benjamin (2020) put it “computational depth without historical or sociological depth is just superficial learn- ing [not deep learning]”. These limitations pose serious con- straints on the justifiability of algorithmic outputs, which could have a negative impact on individuals or an entire population due to suboptimal inferences or, in the case of the physical sciences, even tip the evidence for or against “a specific scien- tific theory” (Ras et al. 2018, 10). This is why it is crucial to
ensure that data fed to algorithms are validated independently, and data retention and reproducibility measures are in place to mitigate inconclusive evidence leading to unjustified actions, along with auditing processes to identify unfair outcomes and unintended consequences (Henderson et al. 2018; Rahwan 2018; Davis and Marcus 2019; Brundage et al. 2020).
The danger arising from inconclusive evidence and erro- neous actionable insights also stems from the perceived mechanistic objectivity associated with computer-generated analytics (Karppi 2018; Lee 2018; Buhmann et al. 2019). This can lead to human decision-makers ignoring their own experienced assessments—so-called ‘automation bias’ (Cummings 2012)—or even shirking part of their respon- sibility for decisions (see Traceability below) (Grote and Berens 2020). As we shall see in Sects. 4 and 8, a lack of understanding of how algorithms generate outputs exacer- bates this problem.
4 Inscrutable evidence leading to opacity
Inscrutable evidence focuses on problems related to the lack of transparency that often characterise algorithms (particularly ML algorithms and models); the socio-technical infrastructure in which they exist; and the decisions they support. Lack of transparency—whether inherent due to the limits of technology or acquired by design decisions and obfuscation of the underly- ing data (Lepri et al. 2018; Dahl 2018; Ananny and Crawford 2018; Weller 2019)—often translates into a lack of scrutiny and/ or accountability (Oswald 2018; Fink 2018; Webb et al. 2019) and leads to a lack of “trustworthiness” (see Al-Hleg 2019).
According to the recent literature, factors contributing to the overall lack of algorithmic transparency include the cognitive impossibility for humans to interpret massive algo- rithmic models and datasets; a lack of appropriate tools to visualise and track large volumes of code and data; code and data that are so poorly structured that they are impossible to read; and ongoing updates and human influence over a model (Diakopoulos and Koliska 2017; Stilgoe 2018; Zerilli et al. 2019; Buhmann et al. 2019). Lack of transparency is also an inherent characteristic of self-learning algorithms, which alter their decision logic (produce new sets of rules) during the learning process, making it difficult for devel- opers to maintain a detailed understanding of why certain changes were made (Burrell 2016; Buhmann et al. 2019). However, this does not necessarily translate into opaque outcomes, as even without understanding each logical step, developers can adjust hyperparameters, the parameters that govern the training process, to test for various outputs. In this respect, Martin (2019) stresses that, while the difficulty of explaining ML algorithms’ outputs is certainly real, it is important not to let this difficulty incentivise organisations to develop complex systems to shirk responsibility.
AI & SOCIETY (2022) 37:215–230
Lack of transparency can also result from the malleability of algorithms, whereby algorithms can be reprogrammed in a continuous, distributed, and dynamic way (Sandvig et al. 2016). Algorithmic malleability allows developers to moni- tor and improve an already-deployed algorithm, but it may also be abused to blur the history of its evolution and leave end-users in a state of confusion about the affordances of a given algorithm (Ananny and Crawford 2018). Consider for example Google’s main search algorithm. Its malleability enables the company to make continuous revisions, sug- gesting a permanent state of destabilisation (Sandvig et al. 2016). This requires those affected by the algorithm to moni- tor it constantly and update their understanding accordingly –an impossible task for most (Ananny and Crawford 2018).
As Floridi and Turilli (2009, 105) note, transparency is not an “ethical principle in itself but a pro-ethical condition for enabling or impairing other ethical practices or princi- ples”. And indeed, complete transparency can itself cause distinct ethical problems (Ananny and Crawford 2018): transparency can provide users with some critical infor- mation about the features and lim
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com