Paul O'Leary McCann
bc87b815cc
Add comment clarifying what LANGUAGES does
2017-07-09 16:28:55 +09:00
Paul O'Leary McCann
04e6a65188
Remove Japanese from LANGUAGES
...
LANGUAGES is a list of languages whose tokenizers get run through a
variety of generic tests. Since the generic tests don't check the JA
fixture, it blows up when it can't find janome. -POLM
2017-07-09 16:23:26 +09:00
Paul O'Leary McCann
c336193392
Parametrize and extend Japanese tokenizer tests
2017-06-29 00:09:40 +09:00
Paul O'Leary McCann
30a34ebb6e
Add importorskip for janome
2017-06-29 00:09:20 +09:00
Paul O'Leary McCann
e56fea14eb
Add basic Japanese tokenizer test
2017-06-28 01:24:25 +09:00
Paul O'Leary McCann
84041a2bb5
Make create_tokenizer work with Japanese
2017-06-28 01:18:05 +09:00
Ines Montani
f69ff15089
Update CONTRIBUTORS.md
2017-06-27 14:49:02 +02:00
Ines Montani
d6e08f2bf6
Merge pull request #1142 from garfieldnate/patch-1
...
fix confusing typo
2017-06-26 10:41:47 +02:00
Nathan Glenn
81166c3d56
fix confusing typo
...
This document describes the `Vocab` class, not the `Span` class.
2017-06-21 19:22:30 +02:00
Ines Montani
9335736c20
Merge pull request #1127 from bartbroere/master
...
Fixed a minor typo in the documentation
2017-06-13 13:15:20 +02:00
Bart Broere
e3be243e06
Merge pull request #1 from explosion/master
...
Update
2017-06-12 22:06:59 +02:00
Ines Montani
6b94c3cf00
Merge pull request #1126 from ianmobbs/master
...
Added html5lib==1.0b8 to requirements.txt
2017-06-12 21:18:24 +02:00
Ian Mobbs
d19ce29a23
Create requirements.txt
2017-06-12 13:21:44 -04:00
Bart Broere
e4a45ae55f
Very minor documentation fix
2017-06-12 12:28:51 +02:00
Ines Montani
47aaecd974
Merge pull request #1109 from oroszgy/patch-2
...
Fixed typo in cli/package.py
2017-06-07 16:39:40 +02:00
György Orosz
fa26041da6
Fixed typo in cli/package.py
2017-06-07 16:19:08 +02:00
Ines Montani
3cceabbf32
Update README.rst
2017-06-06 14:39:54 +02:00
ines
6ef04afdc8
Update docs with Spanish model
2017-06-06 12:49:25 +02:00
ines
6071c727d2
Add more env options to gitignore
2017-06-06 12:49:17 +02:00
Ines Montani
99d02b2bb6
Update CONTRIBUTORS.md
2017-06-06 03:20:20 +02:00
Ines Montani
4cbe55622d
Update README.rst
2017-06-05 21:32:36 +02:00
Ines Montani
289650a857
Merge pull request #1103 from kootenpv/patch-1
...
for easy copy & paste
2017-06-05 20:42:40 +02:00
Pascal van Kooten
e66cd9cc70
for easy copy & paste
2017-06-05 20:41:28 +02:00
Ines Montani
e7ef51b382
Update tokenizer_exceptions.py
2017-06-02 19:00:01 +02:00
Ines Montani
83467a00a7
Update README.rst
2017-06-02 12:42:22 +02:00
Ines Montani
c4e62c7651
Update README.rst
2017-06-02 12:39:44 +02:00
Ines Montani
af466496f1
Rename appveyor.yml to .appveyor.yml
2017-06-02 12:33:57 +02:00
Ines Montani
3e16535fef
Update appveyor.yml
2017-06-02 12:31:31 +02:00
Ines Montani
3c2cce8efc
Update appveyor.yml
2017-06-02 12:27:36 +02:00
Ines Montani
0404b5f43b
Update appveyor.yml
2017-06-02 12:18:51 +02:00
Ines Montani
86277d4ef2
Create appveyor.yml
2017-06-02 12:13:59 +02:00
Ines Montani
81918155ef
Merge pull request #1096 from recognai/master
...
Spanish model features
2017-06-02 11:07:27 +02:00
Francisco Aranda
70a2180199
fix(spanish sentence segmentation): remove tokenizer exceptions the break sentence segmentation. Aligned with training corpus
2017-06-02 08:19:57 +02:00
Francisco Aranda
5b385e7d78
feat(spanish model): add the spanish noun chunker
2017-06-02 08:14:06 +02:00
ines
36b20d66bf
Add alpha banner
2017-06-01 18:11:49 +02:00
ines
a6d99f8dab
Add prefix to option IDs to avoid conflicts
2017-06-01 17:56:18 +02:00
ines
1e918b871c
Remove infoboxes
2017-06-01 17:53:47 +02:00
ines
ab83dd5d25
Fix lightning tour example
2017-06-01 17:53:41 +02:00
Ines Montani
88ca82bfa6
Merge pull request #1081 from yuvalpinter/patch-2
...
Fixed link
2017-05-23 16:58:45 +02:00
Ines Montani
e6f4bf7af2
Merge pull request #1082 from yuvalpinter/patch-3
...
Fixed span example error
2017-05-23 16:58:25 +02:00
Yuval Pinter
cb418c7aef
Fixed span example error
...
Span as written gives empty text.
2017-05-23 10:54:13 -04:00
Yuval Pinter
68b387ffc3
Fixed link
...
link to Doc API documentation fixed
2017-05-23 10:46:17 -04:00
Ines Montani
7f6be41f21
Fix typo in English tokenizer exceptions ( resolves #1071 )
2017-05-23 12:18:00 +02:00
Matthew Honnibal
2262211c46
Merge pull request #1070 from raphael0202/issue_1069
...
Fix issue #1069 with custom hook `Doc.sents` definition
2017-05-23 11:07:06 +02:00
Matthew Honnibal
c282167310
Merge pull request #1076 from raphael0202/patch-1
...
Deleting (legacy?) whitespace attribute in doc
2017-05-23 11:03:29 +02:00
Matthew Honnibal
7669b9f923
Merge pull request #1077 from raphael0202/patch-2
...
Add Token.orth and Token.orth_ description in doc
2017-05-23 11:00:27 +02:00
Matthew Honnibal
b885ab405d
Merge pull request #1079 from yuvalpinter/patch-1
...
extend suffixes from first-by-character-order to last
2017-05-23 10:59:49 +02:00
Yuval Pinter
af3d121ec9
extend suffixes from first to last
...
reverse suffix list in `tokenizer_pseudo_code()` so the order of returned tokens matches input order
2017-05-22 10:56:03 -04:00
Raphaël Bournhonesque
a330287304
Add Token.orth and Token.orth_ description in doc
2017-05-19 21:17:31 +02:00
Raphaël Bournhonesque
7e4f31c362
Deleting (legacy?) whitespace attribute
...
token.whitespace raises an AttributeError
2017-05-19 21:12:41 +02:00
