ANLP Week 10/Unit 1: Coreference
Sharon Goldwater
Video 1: Introduction and basic terminology
Co-reference (Goldwater, ANLP) 2
Today’s lecture
• What is co-reference, what makes co-reference resolution hard, and what sources of information are relevant?
– What is a discourse model and what are discourse entities?
– What are some different kinds of referring expressions and how do these relate to information status?
– What is a Winograd schema and what is it supposed to test?
Co-reference (Goldwater, ANLP) 3
Recap
In thinking about meaning we have discussed:
• Distributional representations for word meaning.
• Symbolic representations for words and how to combine these to form sentence meanings.
– Our meaning representation language used constants to represent entities.
– Same constant (symbol) always refers to same entity. – Does natural language do the same?
Co-reference (Goldwater, ANLP) 4
Co-reference exercise
• How many entities are referred to?
• How many referring expressions (REs) are there?
• Do all references to a particular entity use the same RE?
• Do all identical REs refer to the same entity?
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Co-reference (Goldwater, ANLP) 5
Co-reference exercise
• There are 7 entities and many more REs (aka mentions).
• The list of REs that refer to the same entity constitutes a co-reference chain. Chains for this example:
•
the mention: “The famous magician, Ashwini Noir” is a single mention.
1. {Ashwini Noir, She, Ashwini}
2. {the stage}
3. {the audience}
4. {a volunteer, A woman, her, her, She} 5. {her hand}
6. {a card, one, it}
7. {the deck}
Note: “The famous magician” is an
• aUppnolsiiktievecpohrnaseta(dnestcsribiningMraRthLe,r two REs that look the same
(“She”, “She”) may pick out different entities: more
than introducing new entity). In some
annotation schemes, it’s included in
ambiguity!
Figuring out which REs refer to the same entity (building these chains) is called co-reference resolution.
Co-reference (Goldwater, ANLP) 6
Co-reference exercise
• There are 7 entities and many more REs (aka mentions).
• The list of REs that refer to the same entity constitutes a co-reference chain. Chains for this example:
1. {Ashwini Noir, She, Ashwini}
2. {the stage}
3. {the audience}
4. {a volunteer, A woman, her, her, She} 5. {her hand}
6. {a card, one, it}
7. {the deck}
• Unlike constants in MRL, two REs that look the same (“She”, “She”) may pick out different entities: more ambiguity!
• Figuring out which REs refer to the same entity (building these chains) is called co-reference resolution.
Co-reference (Goldwater, ANLP) 7
Discourse entities vs real-world entities
• Last time, we assumed constants denote entities in the world.
• Here, we are one step removed:
– Assume the listener/system builds a discourse model
while listening/reading.
– This model builds up facts about discourse entities.
– We may later need to map those entities to real-world entities (entity linking), e.g., to unique IDs of individuals.
Co-reference (Goldwater, ANLP) 8
More terminology
• The discourse entity an RE refers to is its referent.
– {Ashwini Noir, She, Ashwini} all have the same referent.
That is, they co-refer.
– An anaphor is a RE that co-refers with an earlier RE: an
antecedent. The act of doing so is anaphora.
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Co-reference (Goldwater, ANLP) 9
More terminology
• The discourse entity an RE refers to is its referent.
– {Ashwini Noir, She, Ashwini} all have the same referent.
That is, they co-refer.
– An anaphor is a RE that co-refers with an earlier RE: an
antecedent. The act of doing so is anaphora.
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Note: In linguistics, the term anaphor is only used for pronouns, but in NLP, it is often used for any RE that co-refers with an earlier RE (as here). The vCiod-ereofesrehnocew(Gso‘lsdhwea’tear,sAaNnLPa) ntecedent for ‘Ashwini’ but I am un1s0ure if that is normally considered right, or if only the first mention (as shown here) is considered the antecedent.
Video 2: Referring expressions and information status
Co-reference (Goldwater, ANLP) 11
Types of REs and information status
• This example included several types of REs:
– indefinite noun phrases (a volunteer, a woman) – definite noun phrases (the stage, the deck)
– names (Ashwini Noir, Ashwini)
– pronouns (She, her, one, it)
• Which type is appropriate depends on the information status of the RE: where does it fall between
– Given: very salient or predictable – New: not salient, unpredictable
Co-reference (Goldwater, ANLP) 12
Types of REs: indefinite noun phrases
• In English, usually an NP with determiner “a”/”an”.
• Normally refers to an entity that is both:
– Discourse-new: not mentioned before, must be added to the discourse model
– Hearer-new: the hearer doesn’t know about it already.
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Co-reference (Goldwater, ANLP) 13
Types of REs: definite noun phrases
• In English, usually an NP with determiner “the” (but also “his”, “her”, “this”, and others)
• May refer to a discourse-old entity. Are these?
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Co-reference (Goldwater, ANLP) 14
Types of REs: definite noun phrases
• In English, usually an NP with determiner “the” (but also “his”, “her”, “this”, and others)
• No. Most are discourse-new and hearer-new, but are inferrable based on world knowledge and the discourse model so far – therefore definite (identifiable).
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
(Can also have something like “the president of the US”: discourse-new
but hearer-old, because the hearer already knows they exist.)
Co-reference (Goldwater, ANLP) 15
Types of REs: names
• May refer to an entity that is either new or old to both discourse and hearer.
– But given/new still matters: should I use full name (Ashwini Noir), shorter version (Ashwini), or some other type of RE?
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Co-reference (Goldwater, ANLP) 16
Types of REs: pronouns
• Normally refer to entities that are discourse-old (and therefore also (hearer-old).
– More specifically, usually refer to entities that are highly salient.
The famous magician, Ashwini Noir, stepped onto the stage. She turned to the audience and asked for a volunteer. A woman raised her hand. Ashwini asked her to step forward and take a card. She pulled one from the deck and gave it to Ashwini.
Co-reference (Goldwater, ANLP) 17
•
Even when unambiguous, it’s weird to use a pronoun if the entity isn’t salient enough. For example:
The famous magician, Ashwin Noir, stepped onto the stage. He turned to the audience and asked for a volunteer. A woman raised her hand. She was tall and looked a bit nervous, but she stepped forward when chosen. He/Ashwin asked her to take a card. She pulled one from the deck and gave it to him/Ashwin.
Co-reference (Goldwater, ANLP) 18
Types of REs: pronouns
•
Types of REs: pronouns
Even when unambiguous, it’s weird to use a pronoun if the entity isn’t salient enough. For example:
The famous magician, Ashwin Noir, stepped onto the stage. He turned to the audience and asked for a volunteer. A woman raised her hand. She was tall and looked a bit nervous, but she stepped forward when chosen. He/Ashwin asked her to take a card. She pulled one from the deck and gave it to him/Ashwin.
“Ashwin” is better in the first case. “He” is harder to understand because at this point the woman is more salient than Ashwin.
“him” is better in the second case because Ashwin is salient again (so repeating the name sounds weird).
Co-reference (Goldwater, ANLP)
19
Zero anaphora
• In other languages, it’s possible to refer to an entity without any surface realization [word] at all.
• E.g., Chinese (ex from JM3):
我 前一会精神上太紧张。 现在比较平静了
I was too nervous a while ago. … I am now calmer.
Co-reference (Goldwater, ANLP) 20
Video 3: Linguistic aspects of coreference resolution
Co-reference (Goldwater, ANLP) 21
Related tasks
• Mention detection: find NPs and decide which are REs. ✓✓✓
She wanted it to rain vs. She wanted it badly ✓
– She wanted it to rain vs. She wanted it badly My dog is a black Labrador
– My dog is a black Labrador
• Coreference resolution: which REs co-refer?
• Entity linking: link discourse entities to real-world entities (unique IDs, e.g., entry in ontology or Wikipedia).
• Getting all this right is hard, especially if working with multiple documents!
Co-reference (Goldwater, ANLP) 22
What went wrong here?
Co-reference (Goldwater, ANLP)
23
•
•
Real example of coreference/entity linking failure – even when names are spelt differently!
(Why? Because names do vary in spelling due to errors, use of initials and other real variants)
Focus here: coreference resolution
• We’ll stick to linguistic aspects of this: what features can help? What makes it hard?
• JM3 also discusses the other tasks (mention detection, entity linking) and technical descriptions of some systems. Feel free to read, but non-examinable.
Co-reference (Goldwater, ANLP) 24
Linguistic agreement
• In English, pronouns agree with their antecedent in: – Number (singular/plural):
Take a card and look at it.
Take your cards and look at them.
– Gender (neuter/feminine/masculine/unspecified):
Take a card and look at it.
A woman raised her hand. A man raised his hand. Someone raised their hand.
Co-reference (Goldwater, ANLP) 25
Linguistic agreement
• In English, pronouns agree with their antecedent in: – Person (1st/2nd/3rd):
Kim and I are leaving. We need to find our car.
Kim and Sandy are leaving. They need to find their car.
• Agreement varies across languages, e.g.,
– May have no gender agreement at all, or gender
agreement marked on full NPs (not just pronouns).
– NPs may also have more than 2-3 “genders” (noun classes).
Co-reference (Goldwater, ANLP) 26
•
Agreement can help resolve ambiguity
Which mentions co-refer in each case?
The secretary read the letter to the workers. It was angry.
The secretary read the letter to the workers. They were angry. The secretary read the letter to the workers. He was angry.
Co-reference (Goldwater, ANLP) 27
•
Agreement can help resolve ambiguity
Which mentions co-refer in each case?
The secretary read the letter to the workers. It was angry.
The secretary read the letter to the workers. They were angry. The secretary read the letter to the workers. He was angry.
Using indexing notation:
•
[The secretary]i read [the letter]j to [the workers]k. [It]j was angry. [The secretary]i read [the letter]j to [the workers]k. [They]k were angry. [The secretary]i read [the letter]j to [the workers]k. [He]i was angry.
Co-reference (Goldwater, ANLP) 28
Reflexives
• Agreement alone can’t always force co-reference: Looking at [the workers]i, [the woman]j saw [them]i critically.
Looking at [the workers]i, [the woman]j saw [her]k critically.
• To co-refer with “woman” requires a reflexive pronoun: Looking at [the workers]i, [the woman]j saw [herself]j critically.
• Use of (non)reflexives is determined by syntactic constraints (ie, relationships in the phrase structure tree).
Co-reference (Goldwater, ANLP) 29
Reflexives
• Use of (non)reflexives is constrained by syntax: S
NP
VP
A node c-commands its siblings and all their children.
NP
Ashwini tapped herself on the head.
Ashwini tapped on the head.
Reflexive NP: antecedent NP must c-command the reflexive.
Co-reference (Goldwater, ANLP) 30
Reflexives
• Use of (non)reflexives is constrained by syntax: S
NP
VP
A node c-commands its siblings and all their children.
NP
Ashwini tapped seherlf on the head. Ashwini tapped on the head.
Pronoun NP: antecedent NP must not c-command the pronoun.
Co-reference (Goldwater, ANLP) 31
Reflexives
• Use of (non)reflexives is constrained by syntax: S
NP
VP
VP
VP
NP V
NP
Ashwini saw the woman and tapped her on the head. Pronoun NP: antecedent NP must
not c-command the pronoun.
(You don’t need to memorize the syntactic details here, but you should be able to recognize examples where syntactic constraints might help resolve the coreference.)
32
Other features
• Since pronouns usually refer to more salient entities, we can use clues to salience. Salience tendencies:
– Recency: mentioned more recently > less recently
– Syntax: subject position > object position > other positions
• Semantics, discourse context and world knowledge…
Co-reference (Goldwater, ANLP) 33
•
Sometimes knowing about individual verbs is enough (selectional restrictions: what possible types of arguments?)
I took my [homework]i to the [café]j and… …worked on [it]i all day.
…sat in [it]j all day.
But sometimes, things are trickier (Winograd, 1972):
Effects of semantics and context
•
The [city council]i denied [the demonstrators]j a permit because… …[they]i feared violence.
…[they]j advocated violence.
Co-reference (Goldwater, ANLP) 34
Winograd schemas
• Co-reference challenge data sets, using sentence pairs:
– Two entities, pronoun can grammatically refer to either.
– Question asks which entity pronoun refers to.
– Changing one word changes the human-preferred answer.
The trophy didn’t fit into the suitcase because it was too [large/small]. Q: What was too [large/small]? A: the trophy/the suitcase.
Bill passed the gameboy to John because his turn was [over/next]. Q: Whose turn was [over/next]? A: Bill/John
• Seem to require “world knowledge”: e.g., about physical world or social conventions.
– Though sufficiently large language models do surprisingly well. Co-reference (Goldwater, ANLP) 35
What is “world knowledge”?
• Humans observe the physical world and how things behave in it.
• NLP systems observe how humans talk about the world. – Our text contains many statistical patterns that reflect our
own world knowledge…
– But also reflects our own biases, and sometimes magnifies them.
Co-reference (Goldwater, ANLP) 36
Example: gender bias
The secretary read the letter to the workers. He was angry. The secretary read the letter to the workers. She was angry.
• In many cultures today, secretaries tend to be female.
• This bias in the data we observe makes it harder for people from those cultures to process the first (anti-stereotypical) example than the second (pro-stereotypical) example.
– Measured, for example, by reading time or eye movements.
• Do NLP systems have similar biases? If so, where do they
come from, what are the effects, and can we overcome them?
– We’ll talk about this next time.
Co-reference (Goldwater, ANLP) 37
Questions for review
• What is co-reference and what can make it hard?
• What sources of information are relevant?
• What is a discourse model and what are discourse entities?
• What are some different kinds of referring expressions and how do these relate to information status?
• What is a Winograd schema and what is it supposed to test?
Co-reference (Goldwater, ANLP) 38