Scheme代写代考

COMP6714: Informa2on Retrieval & Web Search Introduc*on to Informa(on Retrieval Lecture 7: Scoring and results assembly 1 COMP6714: Informa2on Retrieval & Web Search Recap: ;-idf weigh*ng §  The ;-idf weight of a term is the product of its ; weight and its idf weight. §  Best known weigh*ng scheme in informa*on retrieval §  Increases with

COMP6714: Informa2on Retrieval & Web Search  Introduc)on to  Informa(on Retrieval  Lecture 1: Boolean retrieval  COMP6714: Informa2on Retrieval & Web Search  Unstructured data in 1680    Which plays of Shakespeare contain the words Brutus  AND Caesar  but NOT Calpurnia?    One could grep all of Shakespeare’s plays for Brutus  and Caesar, then strip out lines containing Calpurnia?    Why is that not the answer?    Slow (for large corpora)    NOT Calpurnia is non‐trivial    Other opera)ons (e.g., find the word Romans near  countrymen) not feasible    Ranked retrieval (best documents to return)    Later lectures  2  Sec. 1.1 COMP6714: Informa2on Retrieval & Web Search  Term‐document incidence  1 if play contains word, 0 otherwise Brutus AND Caesar BUT NOT Calpurnia Sec. 1.1 COMP6714: Informa2on Retrieval & Web Search  Incidence vectors    So we have a 0/1 vector for each term.    To answer query: take the vectors for Brutus, Caesar  and Calpurnia (complemented)   bitwise AND.    110100 AND 110111 AND 101111 = 100100.   4 

COMP6714:
Informa2on
Retrieval
&
Web
Search
 Introduc)on
to
 Informa(on
Retrieval
 Lecture
6:
Scoring,
Term
Weigh)ng
and
the
 Vector
Space
Model
 1
 COMP6714:
Informa2on
Retrieval
&
Web
Search
 Recap
of
lecture
5
   Collec)on
and
vocabulary
sta)s)cs:
Heaps’
and
Zipf’s
laws
   Dic)onary
compression
for
Boolean
indexes
   Dic)onary
string,
blocks,
front
coding
   Pos)ngs
compression:
Gap
encoding,
prefix‐unique
codes
   Variable‐Byte,
Gamma
codes,
Golomb/Rice
codes
 collection (text, xml markup etc) 3,600.0 collection (text) 960.0 Term-doc incidence matrix 40,000.0 postings, uncompressed (32-bit words) 400.0 postings, uncompressed (20 bits) 250.0 postings, variable byte encoded 116.0 postings, γ-encoded 101.0 MB 2
 COMP6714:
Informa2on
Retrieval
&
Web
Search
 This
lecture;
IIR
Sec)ons
6.2‐6.4.3
   Ranked
retrieval