Lexical Semantics
COMP90042
Natural Language Processing Lecture 9
COPYRIGHT 2020, THE UNIVERSITY OF MELBOURNE
1
COMP90042
L9
Sentiment Analysis
• Bagofwords,kNNclassifier.Trainingdata:
“This is a good movie.” → ☺ “This is a great movie.” → ☺ “This is a terrible film.” → ☹
‣ ‣ ‣
•
• Twoproblems:
“This is a wonderful film.” → ?
‣ The model does not know that “movie” and “film” are synonyms. Since “film” appears only in negative examples the model learns that it is a negative word.
‣ “wonderful” is not in the vocabulary (OOV – Out-Of- Vocabulary).
2
COMP90042
L9
Sentiment Analysis Comparing words directly will not work. How to
make sure we compare word meanings instead?
•
•
Solution: add this information explicitly through a
lexical database.
3
COMP90042
L9
•
Lexical semantics (this lecture)
‣ How the meanings of words connect to one another.
‣ Manually constructed resources: lexicons, thesauri, ontologies, etc.
•
Distributional semantics (next)
‣ How words relate to each other in the text.
‣ Automatically created resources from corpora.
Word Semantics
4
COMP90042
L9
What Do Words Mean? Referents in the physical or social world
‣ But not usually useful in text analysis
•
•
Their dictionary definition
‣ But dictionary definitions are necessarily circular ‣ Only useful if meaning is already understood
•
Their relationships with other words ‣ Also circular, but more practical
5
COMP90042
L9
•
A word sense describes one aspect of the meaning of a word
Word Senses
6
COMP90042
L9
•
•
Gloss: textual definition of a sense, given by a dictionary
•
If a word has multiple senses, it is polysemous
7
Word Glosses
Bank:
‣ financial institution that accepts deposits and
channels the money into lending activities
‣ sloping land (especially the slope beside a body of water)
COMP90042
L9
•
•
Another way to define meaning: by looking at how it relates to other words
•
Antonymy: opposite meaning ‣ long vs. short
‣ big vs. little
Meaning Through Relations
Synonymy: near identical meaning ‣ vomit vs. throw up
‣ big vs. large
8
COMP90042
L9
•
Hypernymy: is-a relation ‣ cat is an animal
‣ mango is a fruit
•
Meronymy: part-whole relation ‣ leg is part of a chair
‣ wheel is part of a car
Meaning Through Relations (2)
9
COMP90042
L9
Meaning Through Relations (3)
hypernymy
10
COMP90042
L9
WordNet
A database of lexical relations
English WordNet includes ~120,000 nouns, ~12,000 verbs, ~21,000 adjectives, ~4,000 adverbs
• •
•
• •
On average: noun has 1.23 senses; verbs 2.16 WordNets available in most major languages
(www.globalwordnet.org, https://babelnet.org/)
English version freely available (accessible via NLTK)
11
COMP90042
L9
WordNet Example
J&M3 Ch. 19
12
COMP90042
L9
Synsets
• NodesofWordNetarenotwordsorlemmas,butsenses • Therearerepresentedbysetsofsynonyms,orsynsets • Basssynsets:
‣ {bass1, deep6}
‣ {bass6, bass voice1, basso2} • Anothersynset:
‣ {chump1, fool2, gull1, mark9, patsy1, fall guy1, sucker1, soft touch1, mug2}
‣ Gloss: a person who is gullible and easy to take advantage of
13
COMP90042
L9
Synsets (2) >>> nltk.corpus.wordnet.synsets(‘bank’)
[Synset(‘bank.n.01’), Synset(‘depository_financial_institution.n.01’), Synset(‘bank.n.03’), Synset(‘bank.n.04’), Synset(‘bank.n.05’), Synset(‘bank.n.06’), Synset(‘bank.n.07’), Synset(‘savings_bank.n.02’), Synset(‘bank.n.09’), Synset(‘bank.n.10’), Synset(‘bank.v.01’), Synset(‘bank.v.02’), Synset(‘bank.v.03’), Synset(‘bank.v.04’), Synset(‘bank.v.05’), Synset(‘deposit.v.02’), Synset(‘bank.v.07’), Synset(‘trust.v.01′)]
>>> nltk.corpus.wordnet.synsets(‘bank’)[0].definition()
u’sloping land (especially the slope beside a body of water)‘
>>> nltk.corpus.wordnet.synsets(‘bank’)[1].lemma_names()
[u’depository_financial_institution’, u’bank’, u’banking_concern’, u’banking_company’]
14
COMP90042
L9
Lexical Relations in WordNet
15
COMP90042
L9
Hypernymy Chain
16
COMP90042
L9
Word Similarity
17
COMP90042
L9
•
•
•
•
Synonymy: film vs. movie
What about show vs. film? opera vs. film?
Word Similarity
Unlike synonymy (which is a binary relation), word similarity is a spectrum
We can use lexical database (e.g. WordNet) or thesaurus to estimate word similarity
18
COMP90042
L9
•
• •
•
Given WordNet, find similarity based on path length
Word Similarity with Paths
pathlen(c1,c2) = 1+ edge length in the shortest path between sense c1 and c2
similarity between two senses:
1
‣ wordsim(w1, w2) = max simpath(c1, c2) c1 ∈senses(w1 ),c2∈senses(w2 )
‣ simpath(c1, c2) =
similarity between two words
pathlen(c1, c2)
19
COMP90042
L9
simpath(c1, c2) = simpath(nickel,coin) = 1/2 = 0.5
Examples
1
pathlen(c1, c2)
simpath(nickel,currency)
= 1/4 = 0.25
simpath(nickel,money)
= 1/6 = 0.17
simpath(nickel,Richter scale)
= 1/8 = 0.13
20
COMP90042
L9
Beyond Path Length
• Problem: edges vary widely in actual semantic distance
‣ Much bigger jumps near top of hierarchy
• Solution 1: include depth information (Wu & Palmer) ‣ Use path to find lowest common subsumer (LCS)
‣ Compare using depths
simwup(c1,c2) =
2∗depth(LCS(c1,c2)) depth(c1) + depth(c2)
21
COMP90042
L9
Examples
2∗depth(LCS(c1,c2)) depth(c1) + depth(c2)
simwup(c1,c2) =
simwup(nickel,money) = 2*2/(3+ 6) = 0.44
simwup(nickel,Richter scale) =
2*1/(3 + 6) = 0.22
22
COMP90042
L9
Abstract Nodes
• Butcountofedgesornodedepthisstillpoor semantic distance metric
• Nodeshighinthehierarchyisveryabstract/general
• Howdowemakewordsthatconnectthroughvery
abstract nodes much less similar
‣ simwup(nickel,money) = 0.44
‣ simwup(nickel,
Richter scale) = 0.22
23
COMP90042
L9
Concept Probability
w∈words(c)) count(w) N
P(c) =
• P(c):probabilitythatarandomlyselectedwordina
corpus is an instance of concept c
• words(c):setofallwords
that are children of c
• words(geological-formation)=
{hill, ridge, grotto, coast,
natural elevation, cave, shore}
• words(naturalelevation)=
{hill, ridge}
24
COMP90042
L9
•
Abstract nodes higher in the hierarchy has a higher P(c)
Example
25
COMP90042
L9
Similarity with Information Content
IC(c) = −logP(c)
use IC instead of depth in simwup
simlin(c1, c2) =
simlin(hill, coast) =
2 × IC(LCS(c1, c2)) IC(c1) + IC(c2)
2 × − log P (geological-formation) − log P (hill) − log P (coast)
−2 log 0.00176
− log 0.0000189 − log 0.0000216
=
= 0.587
if LCS node is very high up in the hierarchy (say P(c) = 0.99), then IC will be very low (0.01 in this case)
26
COMP90042
L9
Sentiment Analysis Revisited “This is a great movie.” → ☺
“This is a wonderful film.” → ?
• ComparingwordsusingWordNetpathsworkwellif our classifier is based on word similarities (such as kNN)
• Butwhatifwewantsenseasageneralfeature representation, so we can employ other classifiers?
• Solution:mapwordsintexttosensesinWordNet explicitly.
• •
27
COMP90042
L9
•
•
•
Task: selects the correct sense for words in a sentence
Word Sense Disambiguation
Baseline:
‣ Assume the most popular sense
Good WSD potentially useful for many tasks in NLP
‣ In practice, often ignored because good WSD too hard
‣ Active research area
28
COMP90042
L9
Supervised WSD
Apply standard machine classifiers
Feature vectors typically words and syntax around target
‣ But context is ambiguous too!
‣ How big should context window be? (typically very
small)
•
• •
Requires sense-tagged corpora
‣ E.g. SENSEVAL, SEMCOR (available in NLTK) ‣ Very time consuming to create!
29
COMP90042
L9
Less Supervised Approaches
• Lesk:ChoosesensewhosedictionaryglossfromWordNet most overlaps with the context
• Thebankcanguaranteedepositswilleventuallycover future tuition costs because it invests in adjustable-rate mortgage securities.
• bank1:2overlappingnon-stopwords,depositsandmortgage
• bank2:0
30
COMP90042
L9
•
Based on frame semantics
‣ Mary bought a car from John
‣ John sold a car to Mary
‣ Same situation (semantic frame), just different perspective
•
A lexical database of frames, typically prototypical situations
‣ E.g. “commerce_buy”, “apply_heat”
Other Databases – FrameNet
31
COMP90042
L9
FrameNet
Includes lists of lexical units that evoke the frame
‣ E.g. cook, fry, bake, boil, etc.
Lists of semantic roles or frame elements
‣ E.g. “the cook”, “the food”, “the container”, “the instrument”
•
•
•
Semantic relationships among frames
‣ “apply_heat” is Causative of “absorb_heat”, is Used by “cooking_creation”
32
COMP90042
L9
•
•
•
Manually-tagged lexical resources an important starting point for text analysis
Moving On To The Corpus
But much modern work attempts to derive semantic information directly from corpora, without human intervention
Distributional semantics!
33
COMP90042
L9
•
Reading JM3 Ch 19.1-19.3, 19.4.1, 19.5.1
34