Matthew Honnibal
|
fde53be3b4
|
Move whole token mach inside _split_affixes.
|
2016-12-30 17:11:50 -06:00 |
|
Matthew Honnibal
|
3ba7c167a8
|
Fix URL tests
|
2016-12-30 17:10:08 -06:00 |
|
Matthew Honnibal
|
9936a1b9b5
|
Merge branch 'tokenization_w_exception_patterns' of https://github.com/oroszgy/spaCy.hu into oroszgy-tokenization_w_exception_patterns
|
2016-12-30 14:53:40 -06:00 |
|
Matthew Honnibal
|
3e8d9c772e
|
Test interaction of token_match and punctuation
Check that the new token_match function applies after punctuation is split off.
|
2016-12-31 00:52:17 +11:00 |
|
Matthew Honnibal
|
623d94e14f
|
Whitespace
|
2016-12-31 00:30:28 +11:00 |
|
Petter Hohle
|
f112e7754e
|
Add PART to tag map
16 of the 17 PoS tags in the UD tag set is added; PART is missing.
|
2016-12-28 18:39:01 +01:00 |
|
Matthew Honnibal
|
f62db78dc3
|
Increment version
|
2016-12-27 21:11:22 +01:00 |
|
Matthew Honnibal
|
cade536d1e
|
Merge branch 'master' of ssh://github.com/explosion/spaCy
|
2016-12-27 21:04:10 +01:00 |
|
Matthew Honnibal
|
ce4539dafd
|
Allow the vocabulary to grow to 10,000, to prevent cold-start problem.
|
2016-12-27 21:03:45 +01:00 |
|
Ines Montani
|
ad3669cef5
|
Merge pull request #703 from magnusburton/master
Added Swedish abbreviations
|
2016-12-27 01:01:49 +01:00 |
|
Ines Montani
|
78f754dd9a
|
Merge pull request #705 from oroszgy/hu_tokenizer
Initial support for Hungarian
|
2016-12-27 00:48:13 +01:00 |
|
Ines Montani
|
8785706039
|
Reformat stop words for better readability
|
2016-12-24 00:58:40 +01:00 |
|
Gyorgy Orosz
|
45e045a87b
|
Unicode/UTF8 compatibility for Python2
|
2016-12-24 00:21:00 +01:00 |
|
Gyorgy Orosz
|
72b61b6d03
|
Typo fix.
|
2016-12-24 00:10:29 +01:00 |
|
Gyorgy Orosz
|
3a9be4d485
|
Updated token exception handling mechanism to allow the usage of arbitrary functions as token exception matchers.
|
2016-12-23 23:49:34 +01:00 |
|
Ines Montani
|
1436b9f15a
|
Fix formatting and consistency
|
2016-12-23 21:36:01 +01:00 |
|
Ines Montani
|
1d64527727
|
Update Spanish tokenizer
Remove reflexive pronouns as they're part of an open class, fix
mistakes and add exceptions
|
2016-12-23 21:36:01 +01:00 |
|
Ines Montani
|
7f411fd01c
|
Remove exceptions containing whitespace / no special chars
|
2016-12-23 14:30:06 +01:00 |
|
Magnus Burton
|
fdf4776262
|
Added Swedish abbreviations
|
2016-12-22 22:45:18 +01:00 |
|
Gyorgy Orosz
|
d9c59c4751
|
Maintaining backward compatibility.
|
2016-12-21 23:30:49 +01:00 |
|
Gyorgy Orosz
|
1748549aeb
|
Added exception pattern mechanism to the tokenizer.
|
2016-12-21 23:16:19 +01:00 |
|
Gyorgy Orosz
|
35aa54765d
|
Hungarian module is exposed in spacy.
|
2016-12-21 20:45:36 +01:00 |
|
Gyorgy Orosz
|
ab2f6ea46c
|
Removed data files from tests..
|
2016-12-21 20:22:09 +01:00 |
|
Ines Montani
|
3c87c71d43
|
Add tokenizer exceptions for a.m. and p.m. in Spanish
|
2016-12-21 18:19:10 +01:00 |
|
Ines Montani
|
78e63dc7d0
|
Update tokenizer exceptions for English
|
2016-12-21 18:06:34 +01:00 |
|
Ines Montani
|
702d1eed93
|
Update tokenizer exceptions for German
|
2016-12-21 18:06:27 +01:00 |
|
Ines Montani
|
d60380418e
|
Update tokenizer exceptions for Spanish
|
2016-12-21 18:06:17 +01:00 |
|
Ines Montani
|
920fa0fed2
|
Add DET_LEMMA constant
|
2016-12-21 18:05:41 +01:00 |
|
Ines Montani
|
8978806ea6
|
Allow Vocab to load without serializer_freqs
|
2016-12-21 18:05:23 +01:00 |
|
Ines Montani
|
be8ed811f6
|
Remove trailing whitespace
|
2016-12-21 18:04:41 +01:00 |
|
Ines Montani
|
926e19184a
|
Merge pull request #695 from magnusburton/master
Added Swedish morph rules
|
2016-12-21 01:06:00 +01:00 |
|
Gyorgy Orosz
|
3d5306acb9
|
Added further testcases.
|
2016-12-20 23:49:35 +01:00 |
|
Gyorgy Orosz
|
23956e72ff
|
Improved partial support for tokenzing Hungarian numbers
|
2016-12-20 23:36:59 +01:00 |
|
Gyorgy Orosz
|
6add156075
|
Refactored language data structure
|
2016-12-20 22:28:20 +01:00 |
|
Gyorgy Orosz
|
366b3f8685
|
Merge branch 'master' into hu_tokenizer
|
2016-12-20 20:53:31 +01:00 |
|
Gyorgy Orosz
|
c035928156
|
Partial Hungarian number tokenization is added.
|
2016-12-20 20:46:20 +01:00 |
|
JM
|
70ff0639b5
|
Fixed missing vec_path declaration that was failing if 'add_vectors' was set
Added vec_path variable declaration to avoid accessing it before assignment in case 'add_vectors' is in overrides.
|
2016-12-20 18:21:05 +01:00 |
|
Magnus Burton
|
48dcc9f647
|
Added morph rules
|
2016-12-20 13:18:41 +01:00 |
|
Magnus Burton
|
db5a077d2b
|
Initial commit for Swedish
|
2016-12-20 11:05:06 +01:00 |
|
Matthew Honnibal
|
3f5747a9b2
|
Merge branch 'master' of ssh://github.com/explosion/spaCy
|
2016-12-18 23:44:22 +01:00 |
|
Matthew Honnibal
|
40e71586d6
|
Fix Issue #683: Add 'SP' to tag_map, if it's not there already, within the Morphology class.
|
2016-12-18 23:44:05 +01:00 |
|
Matthew Honnibal
|
fa1d23e10d
|
Merge branch 'master' of https://github.com/explosion/spaCy
|
2016-12-18 23:32:03 +01:00 |
|
Matthew Honnibal
|
f38eb25fe1
|
Fix test for word vector
|
2016-12-18 23:31:55 +01:00 |
|
Matthew Honnibal
|
4e68abebc4
|
Merge branch 'master' of ssh://github.com/explosion/spaCy
|
2016-12-18 23:19:45 +01:00 |
|
Matthew Honnibal
|
5a6328a5a4
|
Increment version
|
2016-12-18 23:19:19 +01:00 |
|
Matthew Honnibal
|
13a0b31279
|
Another tweak to GloVe path hackery.
|
2016-12-18 23:12:49 +01:00 |
|
Matthew Honnibal
|
2c6228565e
|
Fix vector loading re glove hack
|
2016-12-18 23:06:44 +01:00 |
|
Matthew Honnibal
|
618b50a064
|
Fix issue #684: GloVe vectors not loaded in spacy.en.English.
|
2016-12-18 22:46:31 +01:00 |
|
Matthew Honnibal
|
404019ad2f
|
Fix issue #672: ent_iob_ was a string, not unicode, due to missing unicode_literals statement.
|
2016-12-18 22:33:53 +01:00 |
|
Matthew Honnibal
|
2ef9d53117
|
Untested fix for issue #684: GloVe vectors hack should be inserted in English, not in spacy.load.
|
2016-12-18 22:29:31 +01:00 |
|