程序代写代做代考 algorithm graph case study AI Announcements Reminder: Class challenge out! Ends December 10th

Announcements Reminder: Class challenge out! Ends December 10th
• Final in two weeks, practice question will be posted today

Class Challenge Task 2
• 100 labels isn’t enough! (see Lecture 21/first half of 22)
• Transfer learning from a different dataset? if it isn’t a dataset we provided, you have to check with us (ImageNet OK)

Ethics in Machine Learning
Kate Saenko
CS 542 Machine Learning

A robot kills human
yes, Uber car accident
AI takes over our lives
yes? Youtube algorithm
AI is watching us
yes, virtual police lineup
Autonomous weapons
not yet?
Humans losing jobs
Yes, e.g. librarians
AI Fears:
Which of these has already happened?

AI Fears
• Autonomous weapons – frameworks for regulation
• Future of work – deskilling / reskilling / superskilling. Many jobs that currently seem least likely to be
automated have been racialized and gendered in ways connected to care and immigration, and have rarely paid living wages; how do we revalue work?
• Worse Inequality – digital divides; bias in algorithms may worsen inequality; ecological concerns in energy, storage and cooling required for ML; economic inequality
• Divided societies with algorithmic bubbles – challenges of populism, automated recommendations, news feeds; deepfakes and election meddling

Many of these problems are not new!
Fears about job automation, lack of privacy and inequality arise with each new innovation
• Printing press • Weapons
• Internet

Gender Shades (Buolamwini & Gebru, 2018)
• Evaluated commercial gender classifiers from Microsoft, FACE++, IBM
• Found large disparity in error between population subgroups based on
gender, skin color
Buolamwini, Joy, and Timnit Gebru. “Gender shades: Intersectional accuracy disparities in commercial gender classification.” Conference on Fairness, Accountability and Transparency. 2018.
10

Fairness in Machine Learning
Moritz Hardt

Initially: AI is better than humans! Aaron Roth (University of Pennsylvania)

Wait, maybe not such a good idea…

Example of ML (un)fairness: COMPAS
• Criminal justice: recidivism algorithms (COMPAS)
• Predicting if a defendant should receive bail
• Unbalanced false positive rates: more likely to wrongly deny a black person bail
ProPublica Analysis of COMPAS Algorithm
White
Black
Wrongly Labeled High-Risk
23.5%
44.9%
Wrongly Labeled Low-Risk
47.7%
28.0%
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Example of ML (un)fairness: word embedding
• Bias found in word embeddings (Bolukbasi et al. 2016)
– Examined word embeddings (word2vec) trained on Google News
– Represent each word with high-dimensional vector
– Vector arithmetic: found analogies like
– Paris – France = London – England
– man – woman = programmer – homemaker = surgeon – nurse
• The good news: word embeddings learn so well!
• The bad news: sometimes too well
• Our chatbots should be less biased than we are
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings, NIPS 2016

Example of ML (un)fairness: word embedding
TABLE. Left: The most extreme occupations as projected on to the she he gender direction on w2vNEWS. Occupations such as businesswoman, where gender is suggested by the orthography, were excluded. Right: Automatically generated analogies for the pair she-
he using the procedure in paper.

Machine Learning and Social Norms
• Samplenorms:privacy,fairness,transparency,accountability…
• Possibleapproaches
– “traditional”: legal, regulatory, watchdog
– Embed social norms in data, algorithms, models
• Case study: privacy-preserving machine learning – “single”, strong, definition (differential privacy)
– almost every ML algorithm has a private version
• Fairmachinelearning – not so much…
– impossibility results
Aaron Roth (University of Pennsylvania)

(Un)Fairness Where?
• Data (input)
– e.g. more arrests where there are more police
– Label should be “committed a crime”, but is “convicted of a crime” – try to “correct” bias
• Models (output)
– e.g. discriminatory treatment of subpopulations
– build or “post-process” models with subpopulation guarantees – equality of false positive/negative rates; calibration
• Algorithms (process)
– learning algorithm generating data through its decisions – e.g. don’t learn outcomes of denied mortgages
– lack of clear train/test division
– design (sequential) algorithms that are fair
Aaron Roth (University of Pennsylvania)

Data Bias Examples
Reporting Bias example:
A sentiment-analysis model is trained to predict whether book reviews are positive or negative based on a corpus of user submissions to a popular website. The majority of reviews in the training data set reflect extreme opinions (reviewers who either loved or hated a book), because people were less likely to submit a review of a book if they did not respond to it strongly. As a result, the model is less able to correctly predict sentiment of reviews that use more subtle language to describe a book.
Selection Bias example:
A model is trained to predict future sales of a new product based on phone surveys conducted with a sample of consumers who bought the product. Consumers who instead opted to buy a competing product were not surveyed, and as a result, this group of people was not represented in the training data.

Why fairness is hard
• Suppose we are a bank trying to fairly decide who should get a loan i.e. Who is most likely to pay us back?
• Suppose we have two groups, A and B (the sensitive attribute) This is where discrimination could occur
• The simplest approach is to remove the sensitive attribute from the data, so that our classifier doesn’t know the sensitive attribute

Why fairness is hard

Why fairness is hard
• However, if the sensitive attribute is correlated with the other attributes, this isn’t good enough
• It is easy to predict race if you have lots of other information (e.g. home address, spending patterns)
• More advanced approaches are necessary

Ethical Issues in Machine Learning
• JobLoss
• AlgorithmicBias
• Transparency
• AISupremacy
• Fake news and videos
• Autonomous
weapons
• Self-drivingcars
• Privacyand
surveillance

Accuracy vs explainability

E.g.: dataset bias leads to higher errors on ‘novel’ data… Can an explanation point to such bias?
Training Test
Most cows are black/brown
Most sheep are white
Prediction: “cow” 76%
Explanation
True class: “sheep”
27

Gender bias in captioning models (Hendricks et al. 2018)
Evidence for “man”
Ground truth captions
Baseline: A man sitting at a desk with a laptop computer.
Hendricks et al. “Women Also Snowboard: Overcoming Bias in Captioning Models.” ECCV 2018
Zhao et al. “Men also like shopping: Reducing gender bias amplification using corpus-level constraints.” EMNLP 2017 28
Generated captions

Ethical Issues in Machine Learning
• JobLoss
• AlgorithmicBias
• Transparency
• AISupremacy
• Fake news and videos
• Autonomous
weapons
• Self-drivingcars
• Privacyand
surveillance
If we start trusting algorithms to make decisions, who will have the final word on important decisions? Will it be humans, or algorithms?
Algorithms are already being used to determine prison sentences. Judges’ decisions are affected by their moods, so some argue that judges should be replaced with “robojudges”. However, a ProPublica study found that one of these popular sentencing algorithms was highly biased against blacks.

Ethical Issues in Machine Learning
• JobLoss
• AlgorithmicBias
• Transparency
• AISupremacy
• Fake news and videos
• Autonomous
weapons
• Self-drivingcars
• Privacyand
surveillance

Ethical Issues in Machine Learning
• JobLoss
• AlgorithmicBias
• Transparency
• AISupremacy
• Fake news and videos
• Autonomous
weapons
• Self-drivingcars
• Privacyand
surveillance
The Centre for New American security said in a report that the Chinese company Ziyan is negotiating to sell Blowfish A2, a killer robot capable of 60 millimeter mortar shells or a 35-40 millimeter grenade launcher, to the governments of Pakistan and Saudi Arabia
https://www.albawaba.com/news/china-selling-autonomous-weaponized-drones-saudi-arabia-and-pakistan-1321951

https://www.youtube-nocookie.com/embed/HipTO_7mUOw?controls=0

Ethical Issues in Machine Learning
• JobLoss
• AlgorithmicBias
• Transparency
• AISupremacy
• Fake news and videos
• Autonomous
weapons
• Self-drivingcars
• Privacyand
surveillance
The death of Elaine Herzberg (August 2, 1968 – March 18, 2018) was the first recorded case of a pedestrian fatality involving a self-driving (autonomous) car, after a collision … Following the fatal incident, Uber suspended testing of self-driving vehicles in Arizona, where such testing had been sanctioned since August 2016
In a preliminary report about the crash released in May, the National Transportation Safety Board said the Uber car’s computer system had spotted Ms. Herzberg six seconds before impact, but classified Ms. Herzberg, who was not in a crosswalk, first as an unrecognized object, then as another vehicle and finally as a bicycle.

Ethical Issues in Machine Learning
• JobLoss
• AlgorithmicBias
• Transparency
• AISupremacy
• Fake news and videos
• Autonomous
weapons
• Self-drivingcars
• Privacyand
surveillance
One in two American adults is in a law enforcement face recognition network– https://www.perpetuallineup.org/
Most law enforcement agencies do little to ensure their systems are accurate.

Private data leakage in language models
https://talktotransformer.com/ Generated Completion
The spaceship was flying erratically, something was wrong…
The whole ordeal left some thinking that the Guardians were taking orders from Slitheen or some alien facsimile of him. In truth, Guardians are commanded by the T- Shirt Guy, who has to have trouble controlling his fiery temper. Whether it’s a direct order from Slitheen, just a telepathic suggestion, or some combination of both, the Guardians keep a watchful eye on the Other Man.
At the start of the episode, a mysterious Doctor tells us that a ship sent from Arcturus Minor

Private data leakage in language models
https://talktotransformer.com/ Generated Completion
social security number (SSN): XXX-XXX-XXXX
Department of Work and Pensions, Social Security Number program, Fax: 1-800-772-1213, email: chelseas@dualcredits.org or spousal number(s) listed in public records available on GOV.UK.
Social Security number
If you are applying online, you can enter your FFEA number instead of the SSN.
If you are applying from outside the UK, an application form is provided to you.
*actual SSN was generated, we blocked it out

Next Class
Applications III: On-device Machine Learning
knowledge distillation, pruning, quantization

Related Posts