* Port Japanese mecab tokenizer from v1
This brings the Mecab-based Japanese tokenization introduced in #1246 to
spaCy v2. There isn't a JapaneseTagger implementation yet, but POS tag
information from Mecab is stored in a token extension. A tag map is also
included.
As a reminder, Mecab is required because Universal Dependencies are
based on Unidic tags, and Janome doesn't support Unidic.
Things to check:
1. Is this the right way to use a token extension?
2. What's the right way to implement a JapaneseTagger? The approach in
#1246 relied on `tag_from_strings` which is just gone now. I guess the
best thing is to just try training spaCy's default Tagger?
-POLM
* Add tagging/make_doc and tests
* Fix code sample for `set_extension`
The previous sample code for `set_extension` fails the assertion at the end, because `city_getter` it checked if the whole document text matches any of the city names. Now it checks if any of the city names is contained in the document text.
* Contributor agreement
* Integrate Python kernel via Binder
* Add live model test for languages with examples
* Update docs and code examples
* Adjust margin (if not bootstrapped)
* Add binder version to global config
* Update terminal and executable code mixins
* Pass attributes through infobox and section
* Hide v-cloak
* Fix example
* Take out model comparison for now
* Add meta text for compat
* Remove chart.js dependency
* Tidy up and simplify JS and port big components over to Vue
* Remove chartjs example
* Add Twitter icon
* Add purple stylesheet option
* Add utility for hand cursor (special cases only)
* Add transition classes
* Add small option for section
* Add thumb object for small round thumbnail images
* Allow unset code block language via "none" value
(workaround to still allow unset language to default to DEFAULT_SYNTAX)
* Pass through attributes
* Add syntax highlighting definitions for Julia, R and Docker
* Add website icon
* Remove user survey from navigation
* Don't hide GitHub icon on small screens
* Make top navigation scrollable on small screens
* Remove old resources page and references to it
* Add Universe
* Add helper functions for better page URL and title
* Update site description
* Increment versions
* Update preview images
* Update mentions of resources
* Fix image
* Fix social images
* Fix problem with cover sizing and floats
* Add divider and move badges into heading
* Add docstrings
* Reference converting section
* Add section on converting word vectors
* Move converting section to custom section and fix formatting
* Remove old fastText example
* Move extensions content to own section
Keep weird ID to not break permalinks for now (we don't want to rewrite URLs if not absolutely necessary)
* Use better component example and add factories section
* Add note on larger model
* Use better example for non-vector
* Remove similarity in context section
Only works via small models with tensors so has always been kind of confusing
* Add note on init-model command
* Fix lightning tour examples and make excutable if possible
* Add spacy train CLI section to train
* Fix formatting and add video
* Fix formatting
* Fix textcat example description (resolves#2246)
* Add dummy file to try resolve conflict
* Delete dummy file
* Tidy up [ci skip]
* Ensure sufficient height of loading container
* Add loading animation to universe
* Update Thebelab build and use better startup message
* Fix asset versioning
* Fix typo [ci skip]
* Add note on project idea label
* Remove erroneous lemma lookup år > åra in Swedish
* Add contributors agreement
* Add contrib agreement to correct directory
* Revert change to CONTRIBUTOR_AGREEMENT
* Remove incorrect lemma lookup gäng->gänga
In modern Swedish, "gäng" is mostly associated with "gang" or "group of people". The removed lemma lookup lemmatized it to the verb "thread".
* Add contrib agreement to correct directory
* Revert change to CONTRIBUTOR_AGREEMENT
