Computational
Linguistics
CSC 485/2501 Fall 2021
2C
2c. Understanding language
Department of Computer Science, University of Toronto (many slides taken or adapted from others)
Copyright © 2021 , and . All rights reserved.
What does it mean to “understand” language?
2
The Turing Test
In the first line of your sonnet which reads “Shall I compare thee to a summer’s day,” would not “a spring day” do as well or better?
It wouldn’t scan.
How about “a winter’s day”? That would scan all right.
Yes, but nobody wants to be compared to a winter’s day.
, “Computing machinery and intelligence”, Mind, 59, 1950, 433–460.
3
The Turing Test
Would you say Mr Pickwick reminded you of Christmas?
In a way.
Yet Christmas is a winter’s day, and I do not think Mr Pickwick would mind the comparison.
I don’t think you’re serious. By a winter’s day one means a typical winter’s day, rather than a special one like Christmas.
, “Computing machinery and intelligence”, Mind, 59, 1950, 433–460.
4
Using language
is not necessarily understanding language
5
A computer psychiatrist
Men are all alike.
In what way?
They’re always bugging us about something or the other.
Can you think of a specific example?
Well, my boyfriend made me come here.
Your boyfriend made you come here. He says I’m depressed.
, Computer Power and Human Reason, W.H. Freeman, 1976.
6
A computer psychiatrist
I’m sorry to hear you are depressed. It’s true. I am unhappy.
Do you think coming here will help you not to be unhappy?
Perhaps I could learn to get along with my mother.
Tell me more about your family.
, Computer Power and Human Reason, W.H. Freeman, 1976.
7
Levels of understanding 1
• 0. Keyword processing:
Limited knowledge of particular words or phrases, or their collocations.
• Chatbots.
• Information retrieval.
• Web searching.
8
Levels of understanding 2
• 1. Limited linguistic ability: Appropriate response to simple, highly constrained sentences.
• Database queries in NL.
“Show all sales staff who exceeded their quota in May.”
• Simple NL interfaces.
“I want to fly from Toronto to Vancouver next Sunday.”
9
Levels of understanding 2
10
Levels of understanding 3
• 2. Full text comprehension: Understanding multi-sentence text and its relation to the “real world”.
• Conversational dialogue.
• Automatic knowledge acquisition
• Machine translation?
• 3. Emotional understanding/generation:
• Responding to literature, poetry, humour
• Story narration.
11
Levels of linguistic structure and analysis 1
• Phonology
• The sound system of a language.
• Morphology
• The minimal meaningful units of language (root of a word; suffixes and prefixes), and how they combine.
• Lexicon
• The semantic and syntactic properties of words.
12
Levels of linguistic structure and analysis 2
• Syntax
• The means of expressing meaning: how words
can combine, and in what order. • Semantics
• The meaning of a sentence (a logical statement?). • Pragmatics
• The use of a sentence: pronominal referents; intentions; multi-sentence structure.
13
“Building blocks” of CL systems 1
• Language interpretation + language generation = machine translation?
• Part-of-speech (PoS) tagging.
• Parsing and grammars.
• Reference resolution.
• Dialogue management.
• These are better thought of as functional units now rather than as modular components of modern NLP architectures.
14
Natural language interpretation
Does Flight 207 serve lunch?
YNQ ( ∃e SERVING(e) ∧ SERVER(e, flight-207) ∧ SERVED(e, lunch) )
15
Natural language generation
(spray-1 (OBJECT paint-1) (PATH (path-1
(DESTINATION wall-1)))) (CAUSER sally-1)
Sally sprayed paint onto the wall.
16
Machine translation
• History lesson: the Vauquois triangle (1968). Language-indep. semantic rep.
Parsing/interp
Generation
Czech string
English string
17
Machine translation
• History lesson: the Vauquois triangle (1968).
• Current systems based purely on statistical associations and lexical semantic embeddings.
• Getting incrementally better as they learn from more data.
• Probably more emergent knowledge of linguistics in there than we give them credit for, but it’s awfully difficult for us to extract it.
18
http://www.duchcov.cz/gymnazium/
19
http://www.duchcov.cz/gymnazium/ Translated by Google Translate, 14 July 2008
20
http://gymdux.sokolici.eu/index.php/informace/historie-koly Translated by Google Translate, 3 August 2010.
22
http://gymdux.sokolici.eu/index.php/informace/historie-koly Translated by Google Translate, 17 June 2013.
24
http://www.gspsd.cz/historie/historie-skoly Translated by Google Translate, 26 May 2014.
25
https://www.gspsd.cz/index.php?type=Post&id=256&ids=249 Translated by Google Translate, 5th September 2019.
26
Information extraction
“Bridgestone Sports Co. said Friday it has set up a joint venture in Taiwan with a local concern and a Japanese trading house to produce golf clubs to be shipped to Japan. The joint venture, Bridgestone Sports Taiwan Co., capitalized at 20 million new Taiwan dollars, will start production in January 1990.”
Tie-up-1: Relation: Tie-up
Entities: Bridgestone Sports Co.
a local concern
a Japanese trading house
Joint venture: Bridgestone Sports Taiwan Co. Activity: Activity-1
Amount: NT $ 20,000,000
Activity-1: Company: Bridgestone Sports Taiwan Co. Product: golf clubs
Start date: January 1990
27
“Building blocks” of CL systems 2
• Information extraction
• Chunking (instead of parsing).
• Template filling.
• Named-entity recognition.
28
“Building blocks” of CL systems 3
• Lexical semantics
• Word sense disambiguation (WSD).
• Taxonomies of word senses.
• Analysis of verbs and other predicates
• Embeddings of words into continuous vector space (word2vec, BERT, XLNet, etc.) .
• Computational morphology
29
Why is understanding hard? 1
• The structures that we are interested in are richer than strings – often hierarchical or scope-bearing.
Nadia knows Ross left.
S
NP VP
Nadia V
knows NP VP
Ross left
KNOWS(Nadia, LEFT(Ross))
S
30
Why is understanding hard? 2
• Mapping from surface-form to meaning is many-to-one: Expressiveness.
Nadia kisses Ross. Ross is kissed by Nadia.
KISS (Nadia, Ross)
Nadia gave Ross a kiss. Nadia gave a kiss to Ross.
31
Why is understanding hard? 3
• Mapping is one-to-many: Ambiguity at all levels.
• Lexical
• Syntactic • Semantic • Pragmatic
32
Lexical ambiguity
The lawyer walked to the bar and addressed the jury. The lawyer walked to the bar and ordered a beer. You held your breath and the door for me. ( )
• Computational issues
• Representing the possible meanings of words,
and their frequencies and their indications.
• Representing semantic relations between words.
• Maintaining adequate context.
33
used to strain microscopic plant life from the zonal distribution of plant life .
close-up studies of plant life and natural too rapid growth of aquatic plant life in water
the proliferation of plant and animal life establishment phase of the plant virus life cycle
that divide life into plant and animal kingdom many dangers to plant and animal life
mammals . Animal and plant life are delicately
automated manufacturing plant in Fremont vast manufacturing plant and distribution
chemical manufacturing plant , producing viscose keep a manufacturing plant profitable without
computer manufacturing plant and adjacent discovered at a St. Louis plant manufacturing
copper manufacturing plant found that they copper wire manufacturing plant , for example
‘s cement manufacturing plant in Alpena
vinyl chloride monomer plant , which is molecules found in plant and animal tissue
Nissan car and truck plant in Japan is
and Golgi apparatus of plant and animal cells
union responses to plant closures . cell types found in the plant kingdom are
company said the plant is still operating Although thousands of plant and animal species
animal rather than plant tissues can be
Decision list for plant
LogL Collocation Sense
8.10 plant life →A 7.58 manufacturing plant →B 7.39 life (within ±2-10 words) →A 7.20 manufacturing (in ±2-10 words) →B 6.27 animal (within ±2-10 words) →A 4.70 equipment (within ±2-10 words) →B 4.39 employee (within ±2-10 words) →B 4.30 assembly plant →B 4.10 plant closure →B 3.52 plant species →A 3.48 automate (within ±2-10 words) →B 3.45 microscopic plant →A
…
Syntactic ambiguity 1
Nadia saw the cop with the binoculars. SS
NP NadiaV
VP
NP
NP
VP NP
V
PP NP
with
the binoculars
saw
NP PP
Nadia
saw thecopP
the copP with
NP
the binoculars
36
Syntactic ambiguity 2
[ ][ ]
Put the book in the box on the table.
[ ][[[[ []]]]
Put the book in the red book box.
Noun phrase Adj Noun
Visiting relatives can be trying.
Verb Noun
Verb phrase
[ [ ]]
37
Syntactic ambiguity 3
• These are absolutely everywhere. Some real headlines:
Juvenile Court to Try Shooting Defendant Teacher Strikes Idle Kids
Stolen Painting Found by Tree
Clinton Wins on Budget, but More Lies Ahead Hospitals are Sued by 7 Foot Doctors
Ban on Nude Dancing on Governor’s Desk
• Usually we don’t even notice – we’re that good at this kind of resolution.
38
Syntactic ambiguity 4
• Most syntactic ambiguity is local — resolved by syntactic or semantic context.
Visiting relatives is trying.
Visiting relatives are trying.
Nadia saw the cop with the gun.
• Sometimes, resolution comes too fast!
[ ][ ][ [????
The cotton clothing is made from comes from Mississippi.
[[ ] [ ]][ [ ]]
“Garden-path” sentences.
39
Syntactic ambiguity 5
• Computational issues
• Representing the possible combinatorial
structure of words.
• Capturing syntactic preferences and frequencies.
• Devising incremental parsing algorithms.
40
Semantic ambiguity
• Sentence can have more than one meaning, even when the words and structure are agreed on.
Nadia wants a dog like Ross’s.
Everyone here speaks two languages. Iraqi Head Seeks Arms.
DCS Undergrads Make Nutritious Snacks.
41
Pragmatic ambiguity
• A sample dialogue
Nadia: Do you know who’s going to the party?
Emily: Who?
Nadia: I don’t know.
Emily: Oh … I think Carol and Amy will be there.
• Computational issues
• Representing intentions and beliefs.
• Planning and plan recognition.
• Inferencing and diagnosis.
42
Need for domain knowledge 1
Derivatization of the carboxyl function of retinoic acid by fluor- escent or electroactive reagents prior to liquid chromatography was studied. Ferrocenylethylamine was synthesized and could be coupled to retinoic acid. The coupling reaction involved activ- ation by diphenylphosphinyl chloride. The reaction was carried out at ambient temperature in 50 min with a yield of ca. 95%. The derivative can be detected by coulometric reduction (+100 mV) after on-line coulometric oxidation (+400 mV). The limit of de- tection was 1 pmol of derivative on-column, injected in a volume of 10μl, but the limit of quantification was 10 pmol of retinoic acid.
S. , M. Tod, M. Leclercq, M. Porthault, J. Chalom, “Precolumn derivatization of retinoic acid for liquid chromatography with fluorescence and coulometric detection.” Acta, 293(3), 29 July 1994, 245–250.
43
Need for domain knowledge 2
In doing sociology, lay and professional, every reference to the “real world”, even where the reference is to physical or biological events, is a reference to the organized activities of everyday life. Thereby, in contrast to certain versions of Durkheim that teach that the objective reality of social facts is sociology’s fundamental principle, the lesson is taken instead, and used as a study policy, that the objective reality of social facts as an ongoing accomp- lishment of the concerted activities of daily life, with the ordinary, artful ways of that accomplishment being by members known, used, and taken for granted is, for members doing sociology, a fun- damental phenomenon.
, Preface, Studies in Ethnomethodology, Prentice-Hall, 1967, page vii.
44