[1607.04606] Enriching Word Vectors with Subword Information
Global Survey
In just 3 minutes, help us better understand how you perceive arXiv.
Take the survey
TAKE SURVEY
Skip to main content
We gratefully acknowledge support from
the Simons Foundation and member institutions.
arXiv.org > cs > arXiv:1607.04606
Help | Advanced Search
All fields
Title
Author
Abstract
Comments
Journal reference
ACM classification
MSC classification
Report number
arXiv identifier
DOI
ORCID
arXiv author ID
Help pages
Full text
Search
GO
quick links
Login
Help Pages
About
Computer Science > Computation and Language
arXiv:1607.04606 (cs)
[Submitted on 15 Jul 2016 (v1), last revised 19 Jun 2017 (this version, v2)]
Title:Enriching Word Vectors with Subword Information
Authors:Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov
Download PDF
Abstract: Continuous word representations, trained on large unlabeled corpora are
useful for many natural language processing tasks. Popular models that learn
such representations ignore the morphology of words, by assigning a distinct
vector to each word. This is a limitation, especially for languages with large
vocabularies and many rare words. In this paper, we propose a new approach
based on the skipgram model, where each word is represented as a bag of
character $n$-grams. A vector representation is associated to each character
$n$-gram; words being represented as the sum of these representations. Our
method is fast, allowing to train models on large corpora quickly and allows us
to compute word representations for words that did not appear in the training
data. We evaluate our word representations on nine different languages, both on
word similarity and analogy tasks. By comparing to recently proposed
morphological word representations, we show that our vectors achieve
state-of-the-art performance on these tasks.
Comments: Accepted to TACL. The two first authors contributed equally
Subjects:
Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:1607.04606 [cs.CL]
(or
arXiv:1607.04606v2 [cs.CL] for this version)
Submission history
From: Edouard Grave [view email]
[v1]
Fri, 15 Jul 2016 18:27:55 UTC (18 KB)
[v2]
Mon, 19 Jun 2017 17:41:07 UTC (150 KB)
Full-text links:
Download:
PDF
Other formats
(license)
Current browse context: cs.CL
< prev | next >
new
|
recent
|
1607
Change to browse by:
cs
cs.LG
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
6 blog links
(what is this?)
DBLP – CS Bibliography
listing | bibtex
Piotr Bojanowski
Edouard Grave
Armand Joulin
Tomas Mikolov
a
export bibtex citation
Loading…
Bibtex formatted citation
×
loading…
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code & Data
Code and Data Associated with this Article
arXiv Links to Code Toggle
arXiv Links to Code & Data (What is Links to Code & Data?)
Related Papers
Recommenders and Search Tools
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Core recommender toggle
CORE Recommender (What is CORE?)
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv’s community? Learn more about arXivLabs and how to get involved.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)
About
Help
Click here to contact arXiv
Contact
Click here to subscribe
Subscribe
Copyright
Privacy Policy
Web Accessibility Assistance
arXiv Operational Status
Get status notifications via
email
or slack