spaCy/spacy/cli
adrianeboyd 44829950ba Fix Example details for train CLI / pipeline components (#4624)
* Switch to train_dataset() function in train CLI

* Fixes for pipe() methods in pipeline components

* Don't clobber `examples` variable with `as_example` in pipe() methods
* Remove unnecessary traversals of `examples`

* Update Parser.pipe() for Examples

* Add `as_examples` kwarg to `pipe()` with implementation to return
`Example`s

* Accept `Doc` or `Example` in `pipe()` with `_get_doc()` (copied from
`Pipe`)

* Fixes to Example implementation in spacy.gold

* Move `make_projective` from an attribute of Example to an argument of
`Example.get_gold_parses()`

* Head of 0 are not treated as unset

* Unset heads are set to self rather than `None` (which causes problems
while projectivizing)

* Check for `Doc` (not just not `None`) when creating GoldParses for
pre-merged example

* Don't clobber `examples` variable in `iter_gold_docs()`

* Add/modify gold tests for handling projectivity

* In JSON roundtrip compare results from `dev_dataset` rather than
`train_dataset` to avoid projectivization (and other potential
modifications)

* Add test for projective train vs. nonprojective dev versions of the
same `Doc`

* Handle ignore_misaligned as arg rather than attr

Move `ignore_misaligned` from an attribute of `Example` to an argument
to `Example.get_gold_parses()`, which makes it parallel to
`make_projective`.

Add test with old and new align that checks whether `ignore_misaligned`
errors are raised as expected (only for new align).

* Remove unused attrs from gold.pxd

Remove `ignore_misaligned` and `make_projective` from `gold.pxd`

* Refer to Example.goldparse in iter_gold_docs()

Use `Example.goldparse` in `iter_gold_docs()` instead of `Example.gold`
because a `None` `GoldParse` is generated with ignore_misaligned and
generating it on-the-fly can raise an unwanted AlignmentError

* Update test for ignore_misaligned
2019-11-23 14:32:15 +01:00
..
converters Example class for training data (#4543) 2019-11-11 17:35:27 +01:00
__init__.py Move UD scripts to bin 2019-03-20 01:19:34 +01:00
_schemas.py Store JSON schemas in Python and tidy up (#3235) 2019-02-07 19:44:31 +11:00
convert.py Auto-format [ci skip] 2019-10-24 16:21:08 +02:00
debug_data.py Fix minor issues in debug-data (#4636) 2019-11-13 15:25:03 +01:00
download.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
evaluate.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
info.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
init_model.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
link.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
package.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
pretrain.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
profile.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00
train.py Fix Example details for train CLI / pipeline components (#4624) 2019-11-23 14:32:15 +01:00
validate.py Generalize handling of tokenizer special cases (#4259) 2019-11-13 21:24:35 +01:00