Previously the command string was built by joining with a space, which
wouldn't work for anything requiring quotes. The command string built
this way is only used for debugging purposes, but shlex.join will be
always be valid on Linux, and will usually be valid in Windows. It's
unclear how to do better on Windows.
Before this commit the string version of a command was used when running
on Windows. This changes it so the input - string or list of strings -
is passed directly.
In the case the input was a string, nothing changes.
In the case the input was a list, Python will internally build a command
string such that each element of the list is quoted correctly for
Windows exec call. We can't call that method directly because it's
considered an implementation detail, see below.
https://bugs.python.org/issue10838
An example of when this behavior is correct is if we build a command
line list and sys.executable needs to be quoted because it contains
spaces or something. Before this change the arguments would just be
joined with a space, which would not work.
The main change here is the preprocessing of commands run in projects,
before being passed to the actual command running function.
Previously commands were split and the first argument was checked, and
if it was python or pip it was rewritten to use sys.executable. This
change makes it so that preprocessing is not done on Windows, because
there is no reliable way to split the string the same way the Windows
interpreter would.
It is possible to handle a limited set of commands in Windows, like
literal "python" declarations, even without full shell parsing, but then
quoting related to sys.executable has to be handled.
* Add token and span custom attributes to to_json()
* Change logic for to_json
* Add functionality to from_json
* Small adjustments
* Move token/span attributes to new dict key
* Fix test
* Fix the same test but much better
* Add backwards compatibility tests and adjust logic
* Add test to check if attributes not set in underscore are not saved in the json
* Add tests for json compatibility
* Adjust test names
* Fix tests and clean up code
* Fix assert json tests
* small adjustment
* adjust naming and code readability
* Adjust naming, added more tests and changed logic
* Fix typo
* Adjust errors, naming, and small test optimization
* Fix byte tests
* Fix bytes tests
* Change naming and json structure
* update schema
* Update spacy/schemas.py
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
* Update spacy/tokens/doc.pyx
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
* Update spacy/tokens/doc.pyx
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
* Update spacy/schemas.py
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
* Update schema for underscore attributes
* Adjust underscore schema
* adjust schema tests
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
* Clean up automated label-based issue handline
1. upgrade tiangolo/issue-manager to latest
2. move needs-more-info to tiangolo
3. change needs-more-info close time to 7 days
4. delete old needs-more-info config
* Use old, longer message
* Fix label name
* adding spans to doc_annotation in Example.to_dict
* to_dict compatible with from_dict: tuples instead of spans
* use strings for label and kb_id
* Simplify test
* Update data formats docs
Co-authored-by: Stefanie Wolf <stefanie.wolf@vitecsoftware.com>
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
* Update documentation for dependency parser
* Update documentation for trainable_lemmatizer
* Update documentation for entity_linker
* Update documentation for ner
* Update documentation for morphologizer
* Update documentation for senter
* Update documentation for spancat
* Update documentation for tagger
* Update documentation for textcat
* Update documentation for tok2vec
* Run prettier on edited files
* Apply similar changes in transformer docs
* Remove need to say annotated example explicitly
I removed the need to say "Must contain at least one annotated Example"
because it's often a given that Examples will contain some gold-standard
annotation.
* Run prettier on transformer docs
* add additional REL_OP
* change to condition and new rel_op symbols
* add operators to docs
* add the anchor while we're in here
* add tests
Co-authored-by: Peter Baumgartner <5107405+pmbaumgartner@users.noreply.github.com>
After the precomputable affine table of shape [nB, nF, nO, nP] is
computed, padding with shape [1, nF, nO, nP] is assigned to the first
row of the precomputed affine table. However, when we are indexing the
precomputed table, we get a row of shape [nF, nO, nP]. CuPy versions
before 10.0 cannot paper over this shape difference.
This change fixes compatibility with CuPy < 10.0 by squeezing the first
dimension of the padding before assignment.
