Fairness, Accountability, Principlism
Learning Outcomes Module 4
Copyright By PowCoder代写 加微信 powcoder
At the end of this module, you should be able to:
• Explain framework of Principlism in AI ethics
• Explain the concepts of fairness and accountability in relation to AI
• Intelligently apply the concepts of fairness and accountability to cases involving AI
Student rep. volunteer?
See me during break todayJJ
What questions do you have on anything so far in the subject??
Essay 1 – further tips
• You don’t have data about the exact implications of the AI – think about plausible implications and why they may or are likely to occur
• Make reasonable estimations e.g. AI tool X could cause hundreds to thousands of people to lose a great deal of privacy, which will likely affect them in ways Y and Z, which could also lead to A and B.
• A stronger essay will say something about why the chosen ethical theories (U,D,VE,CE) are potentially strong or have possible weaknesses
• Think carefully about what the theory could say about the case – don’t just skate over the surface
• Each ethical theory could potentially be used for or against a given use of AI – don’t assume the answer is obvious!
• Compare: Is creating autonomous weapons immoral? (U,D,VE,CE)
Criticism of VE (from U and D)
Virtuous people still need to act on/for reasons Reasons include rules, duties, principles,
consequences/utility
VE: Reasons are important but: We understand these reasons through virtuous models
No specific guidance on moral dilemmas
VE: Morality not a precise science (Aristotle) + phronesis + virtuous role models
Relativist?
VE: We can recognise virtue across cultures – courage, honesty, benevolence
Duty or consequences are primary Character only valuable instrumentally
VE: Good character essential for good action (and a good life)
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing 7
Correctional Offender Management Profiling for Alternative Sanctions
Northpointe (now equivant – tagline: “Software for justice”) – for-profit
Recidivism algorithm
• Risk score for reoffending (‘recidivating’) after initial arrest
• Violent crimes, nonviolent crimes
• Blacks and whites involved
• Guides officers in determining bail, parole etc
• May reduce rates of detention (e.g. allow bail)
• 137 questions e.g. ‘A hungry person has a right to steal’
• Factors may include: current charges, pending charges, prior arrest history, previous pretrial failure, residential stability, employment status, community ties, substance abuse, age
• NOT race – ‘protected attribute’ like e.g. gender
• Actual algorithms are trade secrets
• Used actual re-arrest rates to determine actual offending post-COMPAS
• Apparent bias against African Americans
• Black non-offenders: higher risk scores; white non- offenders lower
• Disparate impact
ProPublica
• Equal accuracy (61%) for both whites and blacks – true
• But – the wrong predictions (39%) went wrong in different ways
Not disparate treatment – COMPAS fair!
No explicit use of arbitrary differences (race)
Yes, can be proxies for race (e.g. suburb)…but…
Same accuracy for each race
Reoffending rate for blacks and whites equal at each COMPAS scale (‘calibration’)
E.g. At risk factor 3, blacks and whites had same rate of actual reoffending
Without equal base rates of recidivism, impossible to meet all algorithmic criteria of fairness simultaneously
E.g. can’t achieve calibration and ProPublica’s demands at same time
Other replies
• Better than biased humans
• Historically: inconsistent, gut instinct
• One study (Dressel et al): COMPAS more accurate than individuals lacking criminal justice expertise and slightly less accurate than groups of individuals
• Decreased accuracy: more crime?
• Problem identified by ProPublica is less bad than e.g. poor ‘calibration’ (could violate discrimination laws)
• How do we decide what is fair?
• Is algorithmic (un)fairness purely a technical
Is COMPAS fair?
Should it be used?
How should we decide?
Principlism
• Framework derived from medical ethics principles
• Distill theories into handy principles for AI ethics
• Midlevel principles: b/w theory and detailed rules
• Theory can guide and interpret these broad principles
Principlism:
4 principles +1
1. Non-maleficence – do no harm
• Predict harm, avoid causing harm, minimize harm, short and long term
2. Beneficence – do good
• Anticipate good outcomes, short and long term
3. Respect autonomy – respect people’s values, choices, life plans
• Understand what others’ value, don’t override their choices, be honest
Principlism
4. Justice – fairness
• Distribute benefits and harms fairly, fair processes, don’t unfairly discriminate
4+1. Explicability – transparency and accountability (Floridi*)
• Complements the 4 principles
• Ensure those potentially impacted have sufficient understanding of the AI and that relevant people are held to account
Principles: need to be balanced against one another; all are ‘equal’
*Floridi, Luciano, et al. “AI4People—an ethical framework for a good AI society: opportunities, risks, principles, and recommendations.” Minds and Machines 28.4 (2018): 689-707.
Explicability
• ‘Democratizing Algorithmic Fairness’ (2020) by Pak-Hang Wong • Fairness
• Accountability
Shin, D., & Park, Y. J. (2019). Role of fairness, accountability, and transparency in algorithmic affordance. Computers in Human Behavior, 98, 277-284.
• • • • • • • • •
Justice and Fairness
Justice-fairness often used interchangeably
High stakes decisions – AI Bail
Sentencing
Catching criminals
Job applications
Assigning grades
Censoring or generating misinformation Diagnosing illness
Justice/fairness
Broad definition: ‘Giving each their due’ or ‘what they are owed’. Treat similar cases similarly; ‘blind’ to arbitrary differences
Equal regardless of race, religion, class, sex, gender, sexual orientation, etc.
‘Equal treatment/consideration/respect’
• U – all similar interests are counted equal in calculus
• D – duty of justice
• Kant – Dignity equal; 1st CI: can I rationally will that everyone does this?; 2nd CI: always ends, never merely means
• VE – justice: praiseworthy trait
• CE – justice in care
Distributive justice
Resources, opportunities Need vs. merit
Should everyone be given the same career opportunities regardless of talent?
Same income?
Should we help people suffer bad luck?
Should we favour people who are morally responsible rather than selfish?
Positive/reverse discrimination Affirmative action
More resources/opportunities/advantages to disadvantaged and historically oppressed groups
Procedural justice/fairness
• Fair procedure or process allocate benefits or harms
• E.g. only 10 spaces at uni and 15 equally qualified candidates
• Random, queue
• Social psychologists: many care more about being treated fairly by
institutions than about actual outcomes
• Pure procedural justice: no question of need/merit etc. E.g. one lollipop and two best friends – who gets it?
Retributive justice
Impose penalty due to wrongdoing
U: opposes mere retribution; but: deterrence can by justified
Based on actual guilt and fair procedure (trial)
Reparative justice
Remediation for unfair treatment
Other types
Algorithmic Fairness
ML algorithms can have embedded bias – unfair
E.g. Discriminate against groups unfairly e.g. race, gender
Either explicitly or by proxy
Technical solutions to minimize unfairness E.g. change inputs
E.g. improve processing of dataset
E.g. change weighting of false –ves vs. +ves
Mathematical measures e.g. (Berk et al 2018) 1. overall accuracy equality
2. statistical parity
3. conditional procedure accuracy equality
4. conditional use accuracy equality 5. treatment equality
6. total fairness (1-5 achieved)
Ethical vs. technical problem: Whether mathematical fairness really is fair depends on the standard of fairness adopted. And this standard is disputed.
Hence: ethical question and debate Also…Cannot always have perfectly fair algorithms:
The Impossibility Theorem: “mathematically impossible for an algorithm to simultaneously satisfy different popular fairness measures”
E.g. group parity that unfairly punishes group X who broke the law less often àsame treatment and disparate impact
The Inherent Tradeoff: between fairness and performance E.g. ↑ group fairness — ↓ accuracy recidivism prediction
↓ false +ve (defendants falsely scored as high risk) but ↑ false -ve (miss some high- risk defendants) — social cost!
Ethical theories and fairness/justice
Utilitarianism: Fair/just = maximize net wellbeing (even if some individual’s must be made worse off than others.)
Everyone’s similar interests are still considered equal
Kant’s deontology (absolute): Recognise human dignity, respect autonomy
Treat autonomous agents always as ends, never merely as means.
Deontology (prima facie sort): Rules of e.g. procedural justice; distributive justice; merit; desert; reverse discrimination
Interaction with other rules e.g. be honest; don’t break promises; do good/be beneficent; don’t cause harm
Virtue ethics: What a fair person would do? Think: what are relevant virtues here? How do they apply?
Ethics of care: Consider special relationships, roles, responsibilities, deep human needs, hands-on work
Think: impacts on vulnerable, historical oppression, power Less about detached duties
Our good is often in emotional, compassionate, active, mutual caring relationships
Joan Tronto: attentiveness, responsibility, competence, responsiveness
All these theories recognize basic human equality
They may have different/similar views on algorithms that:
Reinforce existing disadvantage e.g. increasing policing for some groups
Overlook past oppression e.g. that affects algorithmic to compound disadvantage or don’t positively discriminate to help disadvantaged
Ethical theories and fairness
Accountability
Assuming accountability (responsibility) E.g. holding myself to fair procedures
Being held accountable (responsible)
By external pressures and mechanisms
E.g. law, profession, codes of practice, colleagues, elections
Who is accountable? E.g. researchers, engineers, organizations (private and public), deployers, authorities
What mechanisms are the right and fair ones?
Pak-Hang Wong (reading)
• AI may necessarily create winners and losers – harms and benefits for different people.
• Determining what is fair in high stakes AI is not purely a technical task, but an ethical one
• But: how to ensure this determination is itself fair?
• Recall: no consensus about what is fair + perfect algorithmic fairness is impossible
• Wong: procedural justice
• What mechanism is fairest for holding
AI designers and owners accountable?
• E.g. panel of AI ethics experts?
• Political mechanism
Accountability for reasonableness (AFR)
• AFR developed from health ethics
• Wong: “ensure decisions are morally and politically acceptable to those affected by algorithms through inclusion and accommodation of their views and voices”
• AFR assumes no totally final and ‘right’ answer: answers emerge through open, democratic, good-faith dialogue and reason-giving involving stakeholders
• Not just developers and researchers determining what is fair AI
• Four conditions
Four conditions for AFR
1. Publicity condition: Decisions about algorithmic fairness and their rationales must be publicly accessible, transparent, and understandable to non-technical people.
2. Full Acceptability condition: Provide a reasonable explanation of the chosen fairness parameters i.e. give evidence, principles, reasons that fair-minded persons could accept – for all affected stakeholders, especially the vulnerable.
3. Revision and appeals condition: Have ongoing (not one-off) mechanisms for challenge and dispute resolution and revision of policies.
4. Regulative condition: Have ongoing public regulation of process to ensure conditions (1)–(3) are met.
1. Publicity condition: Explain clearly what measures of fairness will be used to predict re-offending
2. Full Acceptability condition: Justify parameters and impacts. Could those impacted accept these reasons? Using education level or victim of crime – that negatively affects certain racial groups more?
3. Revision and appeals condition: Mechanism to contest reasons? e.g. vulnerable groups, representatives of broader society
4. Regulative condition: Media spotlighted COMPAS, but no stronger regulation or enforcement
Other accountability mechanisms?
• Rigorous testing
• Contest individual decisions
• Review committees
• ‘Turing’ stamps – suitable body
• Open-source software – ‘crowd’
• When justifying or critiquing AI, ask: is there sufficient accountability?
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com