explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-26 01:46:28 +03:00
Code Issues Packages Projects Releases Wiki Activity
5,329 Commits 70 Branches 177 Tags 279 MiB
ce9234f593
Commit Graph

3029 Commits

Author SHA1 Message Date
ines
12c3d5fbba Fix formatting 2017-05-09 01:15:28 +02:00
ines
2829a024ef Re-add basic like_num check to global lex_attrs 2017-05-09 01:15:23 +02:00
ines
88adeee548 Add English lex_attrs overrides 2017-05-09 01:09:52 +02:00
ines
8f3fbbb147 Fix typos 2017-05-09 01:09:37 +02:00
ines
ea5fa46475 Import LEX_ATTRS from lang.lex_attrs 2017-05-09 00:58:10 +02:00
ines
2216e5f326 Reorganise lex_attrs and add dict 2017-05-09 00:57:54 +02:00
ines
e666f14d20 Add global lex_attrs 2017-05-09 00:41:53 +02:00
ines
41972c43fe Use consistent regex imports 2017-05-09 00:34:31 +02:00
ines
7b83977020 Remove unused munge package 2017-05-09 00:16:16 +02:00
ines
c714841cc8 Move language-specific tests to tests/lang 2017-05-09 00:02:37 +02:00
ines
bd57b611cc Update conftest to lazy load languages 2017-05-09 00:02:21 +02:00
ines
9f0fd5963f Reorganise Hungarian punctuation rules 2017-05-09 00:01:59 +02:00
ines
fc0d793360 Reorganise Bengali punctuation rules 2017-05-09 00:01:52 +02:00
ines
e895d1afd7 Reorganise French punctuation rules 2017-05-09 00:00:54 +02:00
ines
014bda0ae3 Reorganise global punctuation rules 2017-05-09 00:00:46 +02:00
ines
a91278cb32 Rename _URL_PATTERN to URL_PATTERN 2017-05-09 00:00:00 +02:00
ines
604f299cf6 Add char classes to global language data 2017-05-08 23:59:33 +02:00
ines
f6f5d78cb9 Fix formatting 2017-05-08 23:59:17 +02:00
ines
6eb6306843 Fix language data imports 2017-05-08 23:58:31 +02:00
ines
3c0f85de8e Remove imports in /lang/__init__.py 2017-05-08 23:58:07 +02:00
ines
86d9c29f30 Reorder util functions 2017-05-08 23:51:15 +02:00
ines
9a0d2fdef1 Add load_lang_class() util function 2017-05-08 23:50:45 +02:00
ines
614aa09582 Tidy up Bengali tokenizer exceptions 2017-05-08 22:29:49 +02:00
ines
73b577cb01 Fix relative imports 2017-05-08 22:29:04 +02:00
ines
ae99990f63 Fix formatting 2017-05-08 22:23:48 +02:00
ines
f46ffe3e89 Move language data to /lang module 2017-05-08 20:00:40 +02:00
ines
41a322c733 Fix LEMMA in exceptions and morph rules 2017-05-08 19:57:36 +02:00
ines
2edc0aee12 Update warning message 2017-05-08 19:53:36 +02:00
ines
6025cdb992 Fix string interpolation in times 2017-05-08 16:38:16 +02:00
ines
b9ba58ba5c Add function to resolve load name 
Warn if old 'path' keyword argument is used.
 2017-05-08 16:33:37 +02:00
ines
e6f1a5d0a1 Add unicode declaration 2017-05-08 16:22:17 +02:00
ines
be5541bd16 Fix import and tokenizer exceptions 2017-05-08 16:20:14 +02:00
ines
2324788970 Remove bad tests 2017-05-08 16:15:27 +02:00
ines
b88c4193e7 Add missing symbol 2017-05-08 16:15:20 +02:00
ines
9a5b2bdd4c Don't set morph rules without tag map 2017-05-08 16:15:12 +02:00
ines
4930f0fa8f Explicitly import TOKEN_MATCH 2017-05-08 16:11:54 +02:00
ines
50b7ec03ca Fix typo 2017-05-08 16:11:45 +02:00
ines
3ca611fe48 Fix wildcard imports 2017-05-08 15:56:29 +02:00
ines
c2469b8135 Remove __all__ export 2017-05-08 15:56:22 +02:00
ines
14a9c3ee7a Fix wildcard import 2017-05-08 15:56:13 +02:00
ines
deed623864 Remove comment 2017-05-08 15:56:05 +02:00
ines
e7f95c37ee Merge base tokenizer exceptions 2017-05-08 15:55:52 +02:00
ines
24606d364c Remove redundant language_data.py files in languages 
Originally intended to collect all components of a language, but just
made things messy. Now each component is in charge of exporting itself
properly.
 2017-05-08 15:55:29 +02:00
ines
a627d3e3b0 Reorganise Chinese language data 2017-05-08 15:54:36 +02:00
ines
7b86ee093a Reorganise Swedish language data 2017-05-08 15:54:29 +02:00
ines
50510fa947 Reorganise Portuguese language data 2017-05-08 15:52:01 +02:00
ines
279895ea83 Reorganise Dutch language data 2017-05-08 15:51:39 +02:00
ines
04ef5025bd Reorganise Norwegian language data 2017-05-08 15:51:22 +02:00
ines
5edbc725d8 Reorganise Japanese language data 2017-05-08 15:50:46 +02:00
ines
51a389d3bb Reorganise Italian language data 2017-05-08 15:50:17 +02:00
ines
1bbfa14436 Reorganise Hungarian language data 2017-05-08 15:49:56 +02:00
ines
a77c9fc60d Reorganise Hebrew language data 2017-05-08 15:49:28 +02:00
ines
7f05e977fa Reorganise French language data 2017-05-08 15:49:05 +02:00
ines
0207ffdd52 Reorganise Finnish language data 2017-05-08 15:48:31 +02:00
ines
8e483ec950 Reorganise Spanish language data 2017-05-08 15:48:04 +02:00
ines
c7c21b980f Reorganise English language data 2017-05-08 15:47:25 +02:00
ines
1bf9d5ec8b Reorganise German language data 2017-05-08 15:44:26 +02:00
ines
7b3a983f96 Reorganise Bengali language data 2017-05-08 15:43:50 +02:00
ines
607ba458e7 Fix whitespace 2017-05-08 15:42:31 +02:00
ines
60db497525 Add update_exc and expand_exc to util 
Doesn't require separate language data util anymore
 2017-05-08 15:42:12 +02:00
Matthew Honnibal
b44f7e259c Clean up unused parser code 2017-05-08 15:42:04 +02:00
ines
6e5bd4f228 Remove unused functions from deprecated 2017-05-08 15:40:16 +02:00
Matthew Honnibal
17efb1c001 Change width 2017-05-08 08:40:13 -05:00
ines
f68e420bc0 Add PRON_LEMMA and DET_LEMMA to deprecated 
Will be replaced with proper values across the language data later.
 2017-05-08 15:35:30 +02:00
ines
bd6a7cf4f6 Simplify deprecated model downloading 
Only relevant for spaCy < v1.7.0.
 2017-05-08 15:32:10 +02:00
ines
95edd9e896 Let parse_package_meta take full path 2017-05-08 15:30:48 +02:00
ines
326746eb15 Add util function to resolve arg to model path 
1. check if in data dir or shortcut link
2. check if installed as a pip package
3. check if string is path to model
4. check if Path or Path-like object
 2017-05-08 15:29:47 +02:00
Matthew Honnibal
bef89ef23d Mergery 2017-05-08 08:29:36 -05:00
ines
a7801e7342 Update spacy.load() 
path argument is now deprecated and name can either take a model name
or path. Implement lazy loading by importing module and read Language
class name off __all__.
 2017-05-08 15:27:25 +02:00
Matthew Honnibal
50ddc9fc45 Fix infinite loop bug 2017-05-08 07:54:26 -05:00
Matthew Honnibal
94e86ae00a Predict tags with encoder 2017-05-08 07:53:45 -05:00
Matthew Honnibal
56073a11ef Don't use tags when calculating token vectors 2017-05-08 07:52:24 -05:00
Matthew Honnibal
a66a4a4d0f Replace einsums 2017-05-08 14:46:50 +02:00
Matthew Honnibal
8d2eab74da Use PretrainableMaxouts 2017-05-08 14:24:55 +02:00
Matthew Honnibal
807cb2e370 Add PretrainableMaxouts 2017-05-08 14:24:43 +02:00
Matthew Honnibal
2e2268a442 Precomputable hidden now working 2017-05-08 11:36:37 +02:00
ines
94697e9afc Fix typo 2017-05-08 02:00:37 +02:00
ines
0ee2a22b67 Merge branch 'pr/1024' into develop 2017-05-08 01:12:44 +02:00
ines
c4492d260a Fix kwargs 2017-05-08 01:05:24 +02:00
Matthew Honnibal
10682d35ab Get pre-computed version working 2017-05-08 00:38:35 +02:00
ines
b5a726c5cd Tidy up deprecated.py 2017-05-07 23:29:22 +02:00
ines
59c3b9d4dd Tidy up CLI and fix print functions 2017-05-07 23:25:29 +02:00
ines
311704674d Add path2str compat function 2017-05-07 23:24:56 +02:00
ines
e34069db9f Move is_package and get_model_package_path to util 2017-05-07 23:24:51 +02:00
ines
957ba676b4 Add model files base path to about.py 2017-05-07 23:22:35 +02:00
ines
8d8dd9ceb2 Don't set default value for model 2017-05-07 23:22:21 +02:00
Matthew Honnibal
35458987e8 Checkpoint -- nearly finished reimpl 2017-05-07 23:05:01 +02:00
Matthew Honnibal
4441866f55 Checkpoint -- nearly finished reimpl 2017-05-07 22:47:06 +02:00
Matthew Honnibal
6782eedf9b Tmp GPU code 2017-05-07 11:04:24 -05:00
Matthew Honnibal
e420e5a809 Tmp 2017-05-07 07:31:09 -05:00
Matthew Honnibal
12039e80ca Switch to single matmul for state layer 2017-05-07 14:26:34 +02:00
Matthew Honnibal
700979fb3c CPU/GPU compat 2017-05-07 04:01:11 +02:00
Matthew Honnibal
f99f5b75dc working residual net 2017-05-07 03:57:26 +02:00
Matthew Honnibal
bdf2dba9fb WIP on refactor, with hidde pre-computing 2017-05-07 02:02:43 +02:00
Matthew Honnibal
b439e04f8d Learning smoothly 2017-05-06 20:38:12 +02:00
Matthew Honnibal
08bee76790 Learns things 2017-05-06 18:24:38 +02:00
Matthew Honnibal
04ae1c01f1 Learns things 2017-05-06 18:21:02 +02:00
Matthew Honnibal
bcf4cd0a5f Learns things 2017-05-06 17:37:36 +02:00
Matthew Honnibal
8e48b58cd6 Gradients look correct 2017-05-06 16:47:15 +02:00
Matthew Honnibal
7e04260d38 Data running through, likely errors in model 2017-05-06 14:22:20 +02:00
Matthew Honnibal
fa7c1990b6 Restore tok2vec function 2017-05-05 20:12:03 +02:00
Matthew Honnibal
efe9630e1c Bug fixes 2017-05-05 20:09:50 +02:00
Matthew Honnibal
ef4fa594aa Draft of NN parser, to be tested 2017-05-05 19:20:39 +02:00
Matthew Honnibal
7d1df50aec Draft up Parser model 2017-05-04 13:31:40 +02:00
Matthew Honnibal
ccaf26206b Pseudocode for parser 2017-05-04 12:17:59 +02:00
ines
b1f22c5a10 Fix formatting 2017-05-03 20:11:02 +02:00
ines
a04b5be1b2 Add glossary for annotation scheme (closes #1034) 
Can be imported as explain from spacy.glossary, or called as
spacy.explain(term)
 2017-05-03 17:02:17 +02:00
Gregory Howard
929f2792a7 Rennaming cls in module. cls is now a class 2017-05-03 15:41:07 +02:00
Gregory Howard
0e8c41ea4f Adding method lemmatizer for every class 2017-05-03 12:14:42 +02:00
Gregory Howard
32ca07989e adding export japanese 2017-05-03 11:07:29 +02:00
Grégory Howard
f9d7144224 Merge branch 'master' into master 2017-05-03 11:04:51 +02:00
Gregory Howard
f2ab7d77b4 Lazy imports language 2017-05-03 11:01:42 +02:00
Ines Montani
3ea23a3f4d Fix formatting 2017-05-03 09:44:38 +02:00
Ines Montani
d730eb0c0d Raise custom ImportError if importing janome fails 2017-05-03 09:43:29 +02:00
Ines Montani
949ad6594b Add newline 2017-05-03 09:38:43 +02:00
Ines Montani
d12ca587ea Add newline 2017-05-03 09:38:29 +02:00
Ines Montani
8676cd0135 Add newline 2017-05-03 09:38:07 +02:00
Yasuaki Uechi
c8f83aeb87 Add basic japanese support 2017-05-03 13:56:21 +09:00
Gregory Howard
c0afcd22bb Merge remote-tracking branch 'remotes/upstream/master' 2017-04-27 14:42:54 +02:00
Matthew Honnibal
31ec9e1371 Merge branch 'master' of https://github.com/explosion/spaCy 2017-04-27 13:21:39 +02:00
Matthew Honnibal
2da16adcc2 Add dropout optin for parser and NER 
Dropout can now be specified in the `Parser.update()` method via
the `drop` keyword argument, e.g.

    nlp.entity.update(doc, gold, drop=0.4)

This will randomly drop 40% of features, and multiply the value of the
others by 1. / 0.4. This may be useful for generalising from small data
sets.

This commit also patches the examples/training/train_new_entity_type.py
example, to use dropout and fix the output (previously it did not output
the learned entity).
 2017-04-27 13:18:39 +02:00
Gregory Howard
92f368f83b Removing extra spaces 2017-04-27 12:02:14 +02:00
Gregory Howard
13b6957c8e Adding unitest for tokenization in french (with title) 2017-04-27 11:53:44 +02:00
Gregory Howard
8ff4682255 correcting tokenizer exception. 
Adding tests for lemmatization
 2017-04-27 11:52:14 +02:00
Ines Montani
7da9cefd25 Merge pull request #1022 from luvogels/master 
Initial support for Norwegian Bokmål
 2017-04-27 11:16:06 +02:00
Ines Montani
c9e592ae6c Add newline 2017-04-27 11:15:41 +02:00
Ines Montani
5942adccc2 Add newline 2017-04-27 11:15:19 +02:00
Ines Montani
4cd9269aef Add newline 2017-04-27 11:15:04 +02:00
Ines Montani
ccf13ecc21 Add newline 2017-04-27 11:14:42 +02:00
Ines Montani
03d2b0cc05 Add newline 2017-04-27 11:14:26 +02:00
Gregory Howard
44cb486849 Adding unitest for tokenization in french (with title) 2017-04-27 10:59:38 +02:00
Gregory Howard
ad8129cb45 Improvement of rules now title insentive and have same declaration format 2017-04-27 10:23:56 +02:00
luvogels
d12a0b6431 Hooked up tokenizer tests 2017-04-26 23:21:41 +02:00
Matthew Honnibal
f0e1606d27 Increment version 2017-04-26 20:25:41 +02:00
luvogels
b331929a7e Merge branch 'master' of https://github.com/luvogels/spaCy 2017-04-26 19:15:48 +02:00
luvogels
8de59ce3b9 Added tokenizer tests 2017-04-26 19:10:18 +02:00
Matthew Honnibal
4d98511db7 Make Span hashable. Closes #1019 2017-04-26 19:01:05 +02:00
Matthew Honnibal
24c4c51f13 Try to make test999 less flakey 2017-04-26 18:42:06 +02:00
Leif Uwe Vogelsang
460094bf09 Update __init__.py 2017-04-26 18:27:55 +02:00
ines
527d51ac9a Fetch shortcuts from GitHub and improve error handling 2017-04-26 18:00:28 +02:00
Gregory Howard
ed5f094451 Adding insensitive lemmatisation test 2017-04-25 18:07:02 +02:00
ghoward
26e31afc18 renamming tests 2017-04-25 17:46:01 +02:00
ghoward
c085c2d391 Adding some unitests 2017-04-25 17:44:16 +02:00
ghoward
55c6910f90 Look_up table for languages in spacy. 
Need to find an another name for lemmatizerlookup. I was not inspired.
Trying to uses new files in fr language.
 2017-04-24 16:39:00 +02:00
Matthew Honnibal
c4be9c36fe Fix unicode header in tests 2017-04-24 10:09:01 +02:00
Matthew Honnibal
65f10b53e5 Fix test 2017-04-24 00:25:55 +02:00
Matthew Honnibal
70a43858e1 Fix flakey test 2017-04-24 00:06:30 +02:00
Matthew Honnibal
3973af2d15 Make training test less flakey 2017-04-23 22:59:34 +02:00
Matthew Honnibal
4f9657b42b Fix reporting if no dev data with train 2017-04-23 22:27:10 +02:00
Matthew Honnibal
df2ac8b843 Merge branch 'master' of https://github.com/explosion/spaCy 2017-04-23 21:25:07 +02:00