spaCy/spacy
Joshua Smith e8420ab2b7 Added support for serializing overwrite and ent_id_sep (#3918)
* Perserve flags in EntityRuler

The EntityRuler (explosion/spaCy#3526) does not preserve
overwrite flags (or `ent_id_sep`) when serialized.  This
commit adds support for serialization/deserialization preserving
overwrite and ent_id_sep flags.

* add signed contributor agreement

* flake8 cleanup

mostly blank line issues.

* mark test from the issue as needing a model

The test from the issue needs some language model for serialization
but the test wasn't originally marked correctly.

* remove unneeded model loading

The model didn't need to be loaded, and I replaced it with
a change that doesn't require it (using existings fixtures)

* change tempdir handling to be compatible with python 2.7

* Adds code to handle item saved before this change.

This code chanes how the save files are handled and how the bytes
are stored as well.  This code adds check to dispatch correctly
if it encounters bytes or files saved in the old format (and tests
for those cases).

* use util function for tempdir management

Updated after PR comments: this code now uses the make_tempdir function from util
instead of doing it by hand.
2019-07-08 17:28:28 +02:00
..
cli Auto-format [ci skip] 2019-06-20 10:36:38 +02:00
data Make spacy/data a package 2017-03-18 20:04:22 +01:00
displacy Exclude user_data when copying doc in displaCy (closes #3882) 2019-06-26 14:37:05 +02:00
lang Norwegian tweaks (#3894) 2019-07-08 10:28:47 +02:00
matcher Dependency tree pattern matcher (#3465) 2019-06-16 13:25:32 +02:00
pipeline Added support for serializing overwrite and ent_id_sep (#3918) 2019-07-08 17:28:28 +02:00
syntax 💫 Fix class mismap on parser deserializing (closes #3433) (#3470) 2019-03-23 13:46:25 +01:00
tests Added support for serializing overwrite and ent_id_sep (#3918) 2019-07-08 17:28:28 +02:00
tokens 💫 Improve introspection of custom extension attributes (#3729) 2019-05-12 00:53:11 +02:00
__init__.pxd * Seems to be working after refactor. Need to wire up more POS tag features, and wire up save/load of POS tags. 2014-10-24 02:23:42 +11:00
__init__.py Fix formatting (hopefully also restarts build properly) 2019-03-20 09:55:45 +01:00
__main__.py Update __main__.py 2019-03-20 09:43:26 +01:00
_align.pyx Improve alignment around quotes 2018-08-16 01:04:34 +02:00
_ml.py fixed tag_map.py merge conflict 2019-04-04 14:18:27 +08:00
about.py Set version to v2.1.4 2019-05-11 22:57:53 +02:00
attrs.pxd Fix LANG symbol 2018-02-17 18:10:50 +01:00
attrs.pyx Revert "Merge branch 'develop' of https://github.com/explosion/spaCy into develop" 2018-03-27 19:23:02 +02:00
compat.py Fix symlink creation to show error message on failure (#3589) (resolves #3307)) 2019-04-16 11:58:31 +02:00
errors.py Update pretrain docs and add unsupported loss_func error (#3860) 2019-06-20 10:30:44 +02:00
glossary.py Update glossary.py to match information found in documentation (#3704) (closes ##3679) 2019-05-10 14:23:20 +02:00
gold.pxd Add support for sent_start to GoldParse 2017-08-25 20:03:14 -05:00
gold.pyx fix all references to BILUO annotation format (#3797) 2019-05-31 12:19:19 +02:00
kb.pxd entity as one field instead of both ID and name 2019-03-25 18:10:41 +01:00
kb.pyx entity as one field instead of both ID and name 2019-03-25 18:10:41 +01:00
language.py Document Language.evaluate 2019-05-24 14:06:36 +02:00
lemmatizer.py Fix inconsistant lemmatizer issue #3484 (#3646) 2019-05-04 18:16:03 +02:00
lexeme.pxd 💫 Support lexical attributes in retokenizer attrs (closes #2390) (#3325) 2019-02-24 21:13:51 +01:00
lexeme.pyx Tidy up property code style (#3391) 2019-03-11 15:59:09 +01:00
morphology.pxd annotate kb_id through ents in doc 2019-03-22 11:36:44 +01:00
morphology.pyx Fix issue #3551: Upper case lemmas 2019-04-16 12:27:15 +02:00
parts_of_speech.pxd Add support for Universal Dependencies v2.0 2017-03-03 13:17:34 +01:00
parts_of_speech.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
scorer.py Update Scorer and add API docs 2019-05-24 14:06:04 +02:00
strings.pxd Try to fix StringStore clean up (see #1506) 2017-11-11 03:11:27 +03:00
strings.pyx 💫 Make serialization methods consistent (#3385) 2019-03-10 19:16:45 +01:00
structs.pxd annotate kb_id through ents in doc 2019-03-22 11:36:44 +01:00
symbols.pxd Revert "Merge branch 'develop' of https://github.com/explosion/spaCy into develop" 2018-03-27 19:23:02 +02:00
symbols.pyx Revert "Merge branch 'develop' of https://github.com/explosion/spaCy into develop" 2018-03-27 19:23:02 +02:00
tokenizer.pxd Disable tokenizer cache for special-cases. Fixes #1250 2017-10-24 16:08:05 +02:00
tokenizer.pyx DOC: Update tokenizer docs to include default value for batch_size in pipe (#3492) 2019-03-28 12:48:02 +01:00
typedefs.pxd Work on changing StringStore to return hashes. 2017-05-28 12:36:27 +02:00
typedefs.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
util.py Make jsonschema dependency optional (#3784) 2019-05-30 14:34:58 +02:00
vectors.pyx Update Vectors.find docs [ci skip] 2019-03-16 17:10:57 +01:00
vocab.pxd 💫 Small efficiency fixes to tokenizer (#2587) 2018-07-24 23:35:54 +02:00
vocab.pyx Tidy up property code style (#3391) 2019-03-11 15:59:09 +01:00