Computational
Linguistics
CSC 485 Summer 2020
3 Reading: Jurafsky & Martin: 19.1–4, 20.8; Bird et al: 2.5
Copyright © 2017 Graeme
Hirst, Suzanne Stevenson and Gerald Penn. All rights reserved.
3. Lexical semantics
Gerald Penn
Department of Computer Science, University of Toronto
Lexical semantics
• Word meanings and their internal structure.
• The structure of the relations among words and meanings.
2
Current CL research
• Current focus in CL on lexical semantics:
• word senses;
• detailed lexical representations/vectors;
• organization of senses, or lexical entries more
generally (like a dictionary entry? Probably not).
3
Knowledge about words
Lexicon with entry for each word (or fixed phrase).
• Senses (meanings). For each: • Surface form:
• Orthography, phonology, … • Syntax:
• Part-of-speech, morphology, subcategorization, …
• Behaviour, usage, …:
• Collocations, register and genre, …
4
Word senses
• How are word senses defined? • Grounded in world knowledge?
• Are they defined and fixed at all?
• Or wholly context-dependent? (See also slide 9)
• Constructional versus differential approaches.
Sense is distinguished from others by a set of (ad hoc) differentia.
Sense is built from elements of a set of universal primitives of meaning.
5
Relations between words and senses
• Synonymy: Two (or more) words (synonyms) having the same meaning.
• Homonymy, polysemy: Two (or more) meanings having the same word (homonym, polyseme).
• Lexical ambiguity
What does this mean?
6
Lexical ambiguity: Homonymy
• Homonymy: meanings are unrelated. [Etymology or history of word is not a deciding factor.]
• Due to same spelling (homography):
• bank for money, bank of river, bank of switches,
…bank → banque or bord or rangée or …? bass: “bȧss” fish, “bāss” guitar;
bow: “bau” to the audience, tie a “bō”. • Due to same sound (homophony):
• wood, would; weather, whether; you, ewe, yew; bough, bow.
7
Lexical ambiguity: Polysemy 1
• Polysemy: meanings are related.
• run: of humans, rivers, buses, bus routes, … line: of people, of type, drawn on paper, transit
route, …
• Often, no clear line between polysemy and homonymy.
8
Lexical ambiguity: Polysemy 2
• Sense modulation by context: • fast train, fast typist, fast road.
• Systematic polysemy or sense extension:
• bank as financial institution and as building; window as hole in wall or what fits in hole; bottle, book, DVD, Toyota, lamb, …
• Applies to most or all senses of certain semantic classes.
9
Relations between senses 1
• Hyponymy, hyperonymy: subtype, supertype:
• sedan is a hyponym of car; car is a hyperonym of sedan.
[hypo- = under; hyper- = over]
• The fundamental relation for creating a taxonomy: a tree-like structure that expresses classes and inheritance of properties.
[Terminology:
• is-a relation in ontologies of (language-independent) concepts;
• hyponymy relation in taxonomies of (language-dependent) senses.]
14
Relations between senses 2
• Meronymy, holonymy: part/whole, or membership:
• leg is a meronym of chair;
chair is a holonym of leg and a meronym
of dining-set.
• Many subtypes of meronym relations. Component-of: kitchen–apartment Member-of: soldier–army Portion-of: slice–pie
Examples of meronymy from Roxana Girju, Adriana Badulescu, and Dan I. Moldovan, “Automatic discovery of part-whole relations”, Computational Linguistics, 32(1), 2006, 83–135, based on relations from Morton E. Winston, Roger Chaffin, and Douglas Herrmann, “A taxonomy of part-whole relations”, Cognitive Science, 11(4), 1987, 417–444.
15
Relations between senses 3
• Entailment, implicature: various kinds:
• snore entails sleep; manage implies try.
16
Lexical acquisition 1
• Problem: Need a complete lexicon for each natural language.
• Dictionary as starting point? Limitations?
• Learner’s dictionary? Limitations?
• Text (corpus) as starting point? Limitations?
• Build by hand (lexicographers) or automatically? Limitations?
18
Lexical acquisition 2
• Corpus-based pattern recognition methods.
• Accurate, representative information.
• Includes statistical information.
• Extraction from online dictionary.
• More knowledge-based.
• Can treat dictionary as highly specialized corpus.
19
WordNet 1
• WordNet: A hierarchical (taxonomic) lexicon and thesaurus of English.
• Developed by lexicographers at Princeton, 1990s to present.
• Graph structure:
• Nodes are synsets (“synonym sets”) (≈ word senses).
http://wordnetweb.princeton.edu/perl/webwn
21
Noun slip
• faux pas#1, gaffe#1, solecism#1, slip#1, gaucherie#2
• slip#2, slip-up#1, miscue#2, parapraxis#1 (a minor inadvertent mistake usually observed in speech or writing or in small accidents or memory lapses etc.)
• slip#3 (potter’s clay that is thinned and used for coating or decorating ceramics)
• cutting#2, slip#4 (a part (sometimes a root or leaf or bud) removed from a plant to propagate a new plant through rooting or grafting)
• slip#5 (a young and slender person) “he’s a mere slip of a lad”
• mooring#1, moorage#2, berth#2, slip#6 (a place where a craft can be
made fast)
• slip#7, trip#3 (an accidental misstep threatening (or causing) a fall) “he
blamed his slip on the ice”; “the jolt caused many slips and a few spills”
• slickness#3, slick#1, slipperiness#1, slip#8 (a slippery smoothness) “he
could feel the slickness of the tiller”
• strip#2, slip#9 (artifact consisting of a narrow flat piece of material)
• slip#10, slip of paper#1 (a small sheet of paper) “a receipt slip”
• chemise#1, shimmy#2, shift#9, slip#11, teddy#2 (a woman’s sleeveless
undergarment)
Synonyms for this sense
(a socially awkward
Gloss
or tactless act)
Example
•…
22
Noun slip: Hyperonyms
• slip#10, slip of paper#1 (a small sheet of paper)
• sheet#2, piece of paper#1, sheet of paper#1 (paper used for writing or printin • paper#1 (a material made of cellulose pulp derived mainly from wood or r
• material#1, stuff#1 (the tangible substance that goes into the makeup of a • substance#1 (the real physical matter of which a person or thing consi
• matter#3 (that which has mass and occupies space)
• physical entity#1 (an entity that has physical existence)
• entity#1 (that which is perceived or known or inferred to have it • part#1, portion#1, component part#1, component#2, constituent#3 (so
• relation#1 (an abstraction belonging to or characteristic of two entit • abstraction#6, abstract entity#1 (a general concept formed by extr
• entity#1 (that which is perceived or known or inferred to have it
23
g a
s
s m
i a
s
Noun slip: Sister terms
• sheet#2, piece of paper#1, sheet of paper#1 (paper used for writing or printin
• slip#10, slip of paper#1 (a small sheet of paper)
• signature#5 (a sheet with several pages printed on it; it folds to page size and is
• leaf#2, folio#2 (a sheet of any written or printed material (especially in a manus
• tear sheet#1 (a sheet that can be easily torn out of a publication)
• foolscap#1 (a size of paper used especially in Britain)
• style sheet#1 (a sheet summarizing the editorial conventions to be followed in p
• worksheet#1 (a sheet of paper with multiple columns; used by an accountant to
• revenue stamp#1, stamp#6 (a small piece of adhesive paper that is put on an ob
• Sister terms belong to synsets
24
g
c
r j
Eight senses of board in WordNet, and their hyperonyms and hyponyms
25
Diagram from Ellen Voorhees 1998
WordNet 2
• Graph structure (cont.):
• Edges from hyponymy relations: near-tree.
• Edges from meronymy relations: network.
• Index maps each word to all of its synsets.
• Separate trees for nouns, verbs, adjectives,
adverbs (with derivational cross-connections).
• Differential approach to meaning:
• The hyponyms of a node are differentiations of its meaning.
26
WordNet 3
• WordNets now available or under construction for many languages.
Afrikaans, Albanian, Arabic, Bantu, Basque, Bengali, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Farsi (Persian), Finnish, French, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Irish, Japanese, Kannada, Korean, Latin, Latvian, Macedonian, Maltese, Marathi, Moldavian, Mongolian, Myanmar, Nepali, Norwegian, Oriya, Polish, Portuguese, Romanian, Russian, Sanskrit, Serbian, Slovenian, Spanish, Swedish, Tamil, Thai, Turkish, Vietnamese
www.globalwordnet.org, July 2013
27
Building and updating WordNets
• Problem: Need a complete lexicon and lexical relations for each natural language.
• Dictionary as starting point? Limitations?
• Another WordNet as starting point? Limitations?
• Build by hand (lexicographers) or automatically? Limitations?
• Text (corpus) as starting point? Limitations?
29
Hearst Discovering lexical relations 1
• • •
Corpus-based method.
Makes “suggestions” for lexicographers.
Scan partially-parsed text looking for instances of patterns:
“such NP1 as {NPi}* {or|and} NPi” → NP1 is a hyperonym of the NPi
Hearst, Marti. Automated discovery of WordNet relations. In: Fellbaum, Christane (editor), WordNet: An electronic lexical database, The MIT Press, 1998, pages 131–151.
30
32
Hearst Discovering lexical relations 2
• Develop patterns
• “by hand”, or
• by scanning for sentences containing known related pairs.
34
Hearst Results (good)
1. Some relations already in WordNet:
• fabric–silk, grain–barley, disorders–epilepsy, …
2. Some relations not already in WordNet (but the words were):
• crops–milo, perishables–fruit, conditions–epilespy, …
3. Some relations with words not yet in WordNet:
• companies–Shell, institutions–Tufts, …
35
Hearst Results (less good)
4. Some too-general relations:
• things–exercise, topics–nutrition, areas– Sacremento
5. Some too-context-specific relations:
• others–Meadowbrook, classics–Gaslight,
categories–drama, …
6. Some really bad relations (usually due to parsing errors, not detecting full NP):
• children–Headstart, jobs–computer, companies– sports
36
Hearst Limitations
• Problems:
• Which word is the hyperonym?
A bearing is a structure that supports a rotating part
of a machine, such as
wheel.
a shaft, axle, spindle, or
• Can’t find good patterns for meronyms.
• How to evaluate method quantitatively?
37
Since Hearst’s paper 1
• Methods that use syntactic (not just lexical) patterns, and which derive the patterns from corpora.
• Methods that use senses, not words.
• Methods for finding coordinate (sister) terms
by distributional similarity in text.
• Methods that combine the evidence from all of these to identify additional hyponym relations.
• SISTER(X,Y) ∧ HYPONYM (Y,Z) ⇒ HYPONYM (X,Z)
38
Since Hearst’s paper 2
• Methods for meronymous relations.
• Each subtype tends to have its own indicators.
• These tend to have much more ambiguous patterns than hyponymy.
• Complex methods for learning additional semantic constraints on the patterns.
• Methods for causal relations.
• Look esp. for verbs such as give rise to, induce, generate, cause, …
39
Since Hearst’s paper 3
• “Learning ontologies from text” as important research topic.
• “Learning commonsense knowledge from text” as new research topic.
• “Learning temporal information” (e.g., learning a timeline of events described in a news story) as a new research topic.
• Learning vector-space embeddings from unannotated text, from which some combination of these relations emerges (more on this later).
40
Properties of verbs Revision
•
Subcategorization of verbs:
• VPs can include more than one NP, can include clauses of various types.
• Can classify verbs by kinds of VPs they permit. Thematic roles of a verb — some common
mappings:
Subject ≈ Agent / Experiencer
Object ≈ Theme
Object of preposition ≈ Goal / Location/
Recipient / Instrument
•
41
Lexical semantics of verbs 1
Verbs are more complex than nouns.
• They are predicates that encode relations between their arguments.
• They place selectional restrictions on their arguments.
• E.g., agent of eat must be animate; theme must be physical, edible.
• Different senses of verb may impose different selectional restrictions.
• So argument types may disambiguate verb-sense.
• There are numerous subregularities in how senses cluster together, in fact. 42
Lexical semantics of verbs 2
• Their taxonomy is more difficult to determine.
• Grouping is not as intuitively clear.
• Differentiating sister nodes is more complex.
43
Lexical semantics of verbs 3
WordNet for verbs is not very useful.
• Only shallow hierarchy of troponymy and
hyperonymy.
• e.g., to saunter is to walk in a certain manner.
• Insufficient information about thematic roles, selectional restrictions, and subcategorization.
• No information about regularity in behaviour of classes of verbs.
44
Verb
• S: (v) spray (be discharged in sprays of liquid) “Water sprayed all over the
floor”
• S: (v) spray (scatter in a mass or jet of droplets) “spray water on someone”;
“spray paint on the wall”
• S: (v) spray (cover by spraying with a liquid) “spray the wall with paint”
Verb
• S: (v) spray (be discharged in sprays of liquid) “Water sprayed all over the
floor”
◦ direct hyperonym / inherited hyperonym / sister term
• S: (v) scatter, sprinkle, dot, dust, disperse (distribute loosely) “He
scattered gun powder under the wagon”
• S: (v) discharge (pour forth or release) “discharge liquids”
• S: (v) spread, distribute (distribute or disperse widely) “The invaders spread their language all over the country”
◦ derivationally related form
◦ sentence frame
• Something —-s
• Something is —-ing PP
45
Levin’s verb classification 1
• Groups (English) verbs by diathesis alternations
— syntactic patterns of argument structure.
• May be subtle semantic differences between alternations.
• Shows mapping between semantics of verbs and their syntactic behaviour / subcategorization.
Levin, Beth. English Verb Classes and Alternations. University of Chicago Press, 1993.
Palmer, Martha; Gildea, Daniel; Xue, Nianwen. Semantic Role Labeling. Synthesis Lectures on Human
Language Technologies #6, Morgan & Claypool, 2010. www.morganclaypool.com/toc/hlt/1/1
46
Examples of verb class behaviour 1
[Verb class 45.1]
break, crack, rip,…
Jay broke Bill’s finger.
*Jay broke Bill on the finger.
Jay broke the vase. Vases break easily.
[Verb class 20]
touch, stroke, tickle, …
Kay touched Bill’s neck.
Kay touched Bill on the neck.
Kay touched the cat. *Cats touch easily.
• Motion/contact required for body-part alternation.
• Change of state required for middle construction.
47
Example of diathesis alternation
[Alternation 2.3.1]
The spray–load alternation Nadia sprayed paint onto the wall. Nadia sprayed the wall with paint. Paint sprayed onto the wall.
*The wall sprayed with paint. *Walls spray easily.
Greater suggestion of ‘completeness’ of action
Other verbs that undergo this alternation:
brush, cram, crowd, dust, jam, load, scatter, splash, …
48
Levin’s verb classification 2
• ~80 alternations, ~190 verb classes, ~3000 English verbs classified.
Subsequently extended by other researchers (Korhonen and Briscoe 2004).
• Different senses of a verb may fall into different classes.
• Used extensively in CL; basis for VerbNet.
Anna Korhonen and Ted Briscoe. Extended lexical-semantic classification of English verbs. HLT–NAACL Workshop on Computational Lexical Semantics, Boston, 2004.
49
VerbNet
• Embeds Levin’s classes in a computational lexicon.
• Adds thematic roles and semantics.
• Uses WordNet senses.
Karin Kipper, Hoa Trang Dang, Martha Palmer. Class-based construction of a verb lexicon. 17th National Conference on Artificial Intelligence, 2000.
Karin Kipper Schuler. VerbNet: A Broad-Coverage Comprehensive Verb Lexicon. PhD thesis, University of Pennsylvania, 2005.
50
Class Spray-9.7
Thematic roles and restrictions on them
Semantic form for the kind of event E the frame represents
http://verbs.colorado.edu/verb- index/vn/spray-9.7.php
51
Class Spray-9.7
Restriction on preposition PREP
Unspecified argument
Thematic roles and restrictions on them
Semantic form for the kind of event E the frame represents
http://verbs.colorado.edu/verb- index/vn/spray-9.7.php
52
WordNet and FrameNet sense numbers
Class Spray-9.7-1
53
Class Spray-9.7-1-1
Class Spray-9.7-2
54
FrameNet
• Semantics-first classification of verbs (and nouns).
• Frame: “A conceptual structure that describes a particular type of situation, object, or event along with its participants and props.”*
• Groups of predicates in same semantic class share case frames.
• Includes both a lexicon and a corpus of anno- tated sentences to illustrate predicate usage.
*Josef Ruppenhofer et al. FrameNet II: Extended theory and practice. June 2010.
55
Example
Frame APPLY-HEAT:
bake, barbecue, blanch, boil, braise, broil, …, poach, roast, saute, scald, simmer, singe, steam, stew, toast
Nadia fried the sliced onions in a skillet.
Cook Food Heating instrument
Frame elements
Josef Ruppenhofer et al. FrameNet II: Extended theory and practice. June 2010.
56
Frame elements of Apply_heat
Core elements
Container
Semantic Type Container Cook
Semantic Type Sentient Food
Semantic Type — Heating_instrument
Semantic Type Physical_entity Temperature_setting Semantic Type Temperature
Non-core elements
Co_participant
Semantic Type — Degree
Semantic Type Degree Duration
Semantic Type Duration Manner
Semantic Type Manner Means
Semantic Type State_of_affairs Medium
Semantic Type —
Place
Semantic Type Locative_relation Purpose
Semantic Type State_of_affairs Time
Semantic Type Time
57
https://framenet.icsi.berkeley.edu/fndrupal/index.php?q=frameIndex
Apply_heat
This frame differs from Cooking_creation in focusing on the process of handling the ingredients, rather than the edible entity that results from the process.
Inherits From: Activity, Intentionally_affect Is Inherited By: —
Is Used By: Cooking_creation
Is Causative of: Absorb_heat
58
https://framenet.icsi.berkeley.edu/fndrupal/index.php?q=frameIndex
Lexical entry for an Apply_heat word: bake
CNI = Constructional null instantiation
INI = Indefinite null instantiation
Grammatical functions: Dependent, External argument, Object
59
https://framenet.icsi.berkeley.edu/fndrupal/index.php?q=frameIndex
Lexical entry for an Apply_heat word: bake Valence patterns
60
https://framenet.icsi.berkeley.edu/fndrupal/index.php?q=frameIndex
Text with FrameNet annotations 1
As capital of Europe’s most explosive economy, Dublin seems to be changing before your very eyes.
Subscripts: Frames
Italics: Unannotated words Yellow: Named entities
61
https://framenet.icsi.berkeley.edu/fndrupal/index.php?q=fulltextIndex The text is from the American National Corpus.
Text with FrameNet annotations 2
As capital of Europe’s most explosive economy, Dublin seems to be changing before your very eyes.
62
https://framenet.icsi.berkeley.edu/fndrupal/index.php?q=fulltextIndex The text is from the American National Corpus.
FrameNet in other languages
• FrameNets now available or under construction for several other languages.
Brazilian Portuguese, Chinese, German, Japanese, Spanish, Swedish
https://framenet.icsi.berkeley.edu/fndrupal/framenets_in_other_languages, June 2014
63
FrameNet vs VerbNet 1
Complementary resources:
• VerbNet:
• Groups by syntactic behaviour (Levin classes).
• Any resultant grouping by meaning is side-effect.
• FrameNet:
• Groups by meaning class (frame).
• Not limited to verbs.
• Any resultant grouping by syntactic behaviour is side-effect.
64
FrameNet vs VerbNet 2
• Combine both with WordNet.
• Algorithmic methods to map VerbNet entries to
FrameNet entries and vice versa.
• Semi-automatic methods to map VerbNet constraints into the WordNet hierarchy.
Lei Shi and Rada Mihalcea. “Putting pieces together: Combining FrameNet, VerbNet and WordNet for robust semantic parsing.” 6th International Conference on Intelligent Text Processing and Computational Linguistics (Springer Lecture Notes in Computer Science 3406), 2005, 100–111.
65