.github/FUNDING.yml

@@ -1 +0,0 @@
-custom: [https://explosion.ai/merch, https://explosion.ai/tailored-solutions]

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,18 @@
+<!--- Please provide a summary in the title and describe your issue here.
+Is this a bug or feature request? If a bug, include all the steps that led to the issue.
+
+If you're looking for help with your code, consider posting a question here:
+
+- GitHub Discussions: https://github.com/explosion/spaCy/discussions
+- Stack Overflow: http://stackoverflow.com/questions/tagged/spacy
+-->
+
+## Your Environment
+
+<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type
+`python -m spacy info --markdown` and copy-paste the result here.-->
+
+- Operating System:
+- Python Version Used:
+- spaCy Version Used:
+- Environment Information:

.github/ISSUE_TEMPLATE/01_bugs.md

@@ -1,16 +1,14 @@
 ---
-name: "\U0001F6A8 Submit a Bug Report"
+name: "\U0001F6A8 Bug Report"
-about: Use this template if you came across a bug or unexpected behaviour differing from the docs.
+about: Did you come across a bug or unexpected behaviour differing from the docs?
 
 ---
 
-<!-- NOTE: For questions or install related issues, please open a Discussion instead. -->
-
 ## How to reproduce the behaviour
 <!-- Include a code example or the steps that led to the problem. Please try to be as specific as possible. -->
 
 ## Your Environment
-<!-- Include details of your environment. You can also type `python -m spacy info --markdown` and copy-paste the result here.-->
+<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type `python -m spacy info --markdown` and copy-paste the result here.-->
 * Operating System:
 * Python Version Used:
 * spaCy Version Used:

.github/ISSUE_TEMPLATE/02_docs.md

@@ -1,5 +1,5 @@
 ---
-name: "\U0001F4DA Submit a Documentation Report"
+name: "\U0001F4DA Documentation"
 about: Did you spot a mistake in the docs, is anything unclear or do you have a
   suggestion?
 

.github/ISSUE_TEMPLATE/03_other.md

@@ -0,0 +1,19 @@
+---
+name: "\U0001F4AC Anything else?"
+about: For feature and project ideas, general usage questions or help with your code, please post on the GitHub Discussions board instead.
+---
+
+<!-- Describe your issue here. Please keep in mind that the GitHub issue tracker is mostly intended for reports related to the spaCy code base and source, and for bugs and enhancements. If you're looking for help with your code, consider posting a question here:
+
+- GitHub Discussions: https://github.com/explosion/spaCy/discussions
+- Stack Overflow: http://stackoverflow.com/questions/tagged/spacy
+-->
+
+## Your Environment
+
+<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type `python -m spacy info --markdown` and copy-paste the result here.-->
+
+- Operating System:
+- Python Version Used:
+- spaCy Version Used:
+- Environment Information:

.github/ISSUE_TEMPLATE/config.yml

@@ -1,14 +0,0 @@
-blank_issues_enabled: false
-contact_links:
-  - name: 🗯 Discussions Forum
-    url: https://github.com/explosion/spaCy/discussions
-    about: Install issues, usage questions, general discussion and anything else that isn't a bug report.
-  - name: 📖 spaCy FAQ & Troubleshooting
-    url: https://github.com/explosion/spaCy/discussions/8226
-    about: Before you post, check out the FAQ for answers to common community questions!
-  - name: 💫 spaCy Usage Guides & API reference
-    url: https://spacy.io/usage
-    about: Everything you need to know about spaCy and how to use it.
-  - name: 🛠 Submit a Pull Request
-    url: https://github.com/explosion/spaCy/pulls
-    about: Did you spot a mistake and know how to fix it? Feel free to submit a PR straight away!

.github/PULL_REQUEST_TEMPLATE.md

@@ -14,6 +14,6 @@ or new feature, or a change to the documentation? -->
 ## Checklist
 <!--- Before you submit the PR, go over this checklist and make sure you can
 tick off all the boxes. [] -> [x] -->
-- [ ] I confirm that I have the right to submit this contribution under the project's MIT license.
+- [ ] I have submitted the spaCy Contributor Agreement.
 - [ ] I ran the tests, and all new and existing tests passed.
 - [ ] My changes don't require a change to the documentation, or if they do, I've added all required information.

.github/contributors/AyushExel.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [X] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Ayush Chaurasia      |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2021-03-12           |
-| GitHub username                | AyushExel            |
-| Website (optional)             |                      |

.github/contributors/Jette16.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |  Henriette Behr      |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |  23.09.2021          |
-| GitHub username                |  Jette16             |
-| Website (optional)             |                      |

.github/contributors/KennethEnevoldsen.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [X] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                      |
-|------------------------------- | -------------------------- |
-| Name                           | Kenneth Enevoldsen         |
-| Company name (if applicable)   |                            |
-| Title or role (if applicable)  |                            |
-| Date                           | 2021-07-13                 |
-| GitHub username                | KennethEnevoldsen          |
-| Website (optional)             | www.kennethenevoldsen.com  |

.github/contributors/Lucaterre.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry         |
-|------------------------------- |---------------|
-| Name                           | Lucas Terriel |
-| Company name (if applicable)   |               |
-| Title or role (if applicable)  |               |
-| Date                           | 2022-06-20    |
-| GitHub username                | Lucaterre     |
-| Website (optional)             |               |

.github/contributors/Pantalaymon.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |Valentin-Gabriel Soumah|
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |    2021-11-23        |
-| GitHub username                |     Pantalaymon      |
-| Website (optional)             |                      |

.github/contributors/SamEdwardes.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Sam Edwardes         |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2021-04-02           |
-| GitHub username                | SamEdwardes          |
-| Website (optional)             | samedwardes.com      |

.github/contributors/ZeeD.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Vito De Tullio       |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2021-06-01           |
-| GitHub username                | ZeeD                 |
-| Website (optional)             |                      |

.github/contributors/armsp.md

@@ -98,9 +98,9 @@ mark both statements:
 
 | Field                          | Entry                |
 |------------------------------- | -------------------- |
-| Name                           |  Shantam Raj         |
+| Name                           |  Shantam             |
 | Company name (if applicable)   |                      |
 | Title or role (if applicable)  |                      |
-| Date                           |   10/4/2021          |
+| Date                           |   21/5/2018          |
 | GitHub username                |     armsp            |
-| Website (optional)             |https://shantamraj.com|
+| Website (optional)             |                      |

.github/contributors/avi197.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Son Pham             |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 09/10/2021           |
-| GitHub username                | Avi197               |
-| Website (optional)             |                      |

.github/contributors/bbieniek.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [X] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Baltazar Bieniek     |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2021.08.19           |
-| GitHub username                | bbieniek             |
-| Website (optional)             | https://baltazar.bieniek.org.pl/                     |

.github/contributors/bodak.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Kristian Boda        |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 18.05.2021           |
-| GitHub username                | bodak                |
-| Website (optional)             |                      |

.github/contributors/broaddeep.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Dongjun Park         |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2021-03-06           |
-| GitHub username                | broaddeep            |
-| Website (optional)             |                      |

.github/contributors/bsweileh.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |  Belal               |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |  March 13, 2021      |
-| GitHub username                |  bsweileh            |
-| Website (optional)             |                      |

.github/contributors/connorbrinton.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Connor Brinton       |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | July 20th, 2021      |
-| GitHub username                | connorbrinton        |
-| Website (optional)             |                      |

.github/contributors/ezorita.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Eduard Zorita        |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 06/17/2021           |
-| GitHub username                | ezorita              |
-| Website (optional)             |                      |

.github/contributors/fgaim.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Fitsum Gaim          |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2021-08-07           |
-| GitHub username                | fgaim                |
-| Website (optional)             |                      |

.github/contributors/fonfonx.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Xavier Fontaine      |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2022-04-13           |
-| GitHub username                | fonfonx              |
-| Website (optional)             |                      |

.github/contributors/gtoffoli.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI UG (haftungsbeschränkt)](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your 
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                    |
-|------------------------------- | ------------------------ |
-| Name                           | Giovanni Toffoli         |
-| Company name (if applicable)   |                          |
-| Title or role (if applicable)  |                          |
-| Date                           | 2021-05-12               |
-| GitHub username                | gtoffoli                 |
-| Website (optional)             |                          |

.github/contributors/hlasse.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [X] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                      |
-|------------------------------- | -------------------------- |
-| Name                           | Lasse Hansen               |
-| Company name (if applicable)   |                            |
-| Title or role (if applicable)  |                            |
-| Date                           | 2021-08-11                 |
-| GitHub username                | HLasse                     |
-| Website (optional)             | www.lassehansen.me         |

.github/contributors/jklaise.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |Janis Klaise          |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |26/04/2021            |
-| GitHub username                |jklaise               |
-| Website (optional)             |janisklaise.com       |

.github/contributors/jmyerston.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-      assignment is or becomes invalid, ineffective or unenforceable, you hereby
-      grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-      royalty-free, unrestricted license to exercise all rights under those
-      copyrights. This includes, at our option, the right to sublicense these same
-      rights to third parties through multiple levels of sublicensees or other
-      licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-      contribution as if each of us were the sole owners, and if one of us makes
-      a derivative work of your contribution, the one who makes the derivative
-      work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-      against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-      exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-      consent of, pay or render an accounting to the other for any use or
-      distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-      your contribution in whole or in part, alone or in combination with or
-      included in any product, work or materials arising out of the project to
-      which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-      multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-      or entity, including my employer, has or will have rights with respect to my
-      contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-      actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                         | Entry                               |
-| ----------------------------- | ----------------------------------- |
-| Name                          | Jacobo Myerston                     |
-| Company name (if applicable)  | University of California, San Diego |
-| Title or role (if applicable) | Academic                            |
-| Date                          | 07/05/2021                          |
-| GitHub username               | jmyerston                           |
-| Website (optional)            | diogenet.ucsd.edu                                    |

.github/contributors/julien-talkair.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [ ] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [x] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |  Julien Rossi                    |
-| Company name (if applicable)   |  TalkAir BV                    |
-| Title or role (if applicable)  |  CTO, Partner                    |
-| Date                           |  June 28 2021                    |
-| GitHub username                |  julien-talkair                    |
-| Website (optional)             |                      |

.github/contributors/juliensalinas.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [X] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                         | Entry               |
-| ----------------------------- | ------------------- |
-| Name                          | Julien Salinas      |
-| Company name (if applicable)  | NLP Cloud           |
-| Title or role (if applicable) | Founder and CTO     |
-| Date                          | Mayb 14th 2021      |
-| GitHub username               | juliensalinas       |
-| Website (optional)            | https://nlpcloud.io |

.github/contributors/mariosasko.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [ ] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [x] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Mario Šaško          |
-| Company name (if applicable)   | TakeLab FER          |
-| Title or role (if applicable)  | R&D Intern           |
-| Date                           | 2021-07-12           |
-| GitHub username                | mariosasko           |
-| Website (optional)             |                      |

.github/contributors/meghanabhange.md

@@ -1,107 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI UG (haftungsbeschränkt)](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your 
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                    |
-|------------------------------- | ------------------------ |
-| Name                           | Meghana Bhange            |
-| Company name (if applicable)   | Verloop.io                 |
-| Title or role (if applicable)  | ML Engineer        |
-| Date                           | 2020-04-21               |
-| GitHub username                | meghanbhange                  |
-| Website (optional)             | https://meghana.blog |
-

.github/contributors/narayanacharya6.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Narayan Acharya      |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 29 APR 2021          |
-| GitHub username                | narayanacharya6      |
-| Website (optional)             | narayanacharya.com   |

.github/contributors/nsorros.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Nick Sorros          |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 2/8/2021             |
-| GitHub username                | nsorros              |
-| Website (optional)             |                      |

.github/contributors/philipvollet.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |  Philip Vollet       |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |  22.09.2021          |
-| GitHub username                |  philipvollet        |
-| Website (optional)             |                      |

.github/contributors/plison.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [ ] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Pierre Lison         |
-| Company name (if applicable)   | Norsk Regnesentral   |
-| Title or role (if applicable)  | Senior Researcher    |
-| Date                           | 22.04.2021           |
-| GitHub username                | plison               |
-| Website (optional)             | www.nr.no/~plison    |

.github/contributors/sevdimali.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI UG (haftungsbeschränkt)](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Sevdimali            |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | 10/4/2021            |
-| GitHub username                | sevdimali            |
-| Website (optional)             | https://sevdimali.me |

.github/contributors/shigapov.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                    |
-|------------------------------- | ------------------------ |
-| Name                           | Renat Shigapov           |
-| Company name (if applicable)   |                          |
-| Title or role (if applicable)  |                          |
-| Date                           | 2021-09-09               |
-| GitHub username                | shigapov                 |
-| Website (optional)             |                          |

.github/contributors/swfarnsworth.md

@@ -1,88 +0,0 @@
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |  Steele Farnsworth                    |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |  13 August, 2021                    |
-| GitHub username                |   swfarnsworth                   |
-| Website (optional)             |                      |
-

.github/contributors/syrull.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Dimitar Ganev |
-| Company name (if applicable)   |  |
-| Title or role (if applicable)  |  |
-| Date                           | 2021/8/2 |
-| GitHub username                | syrull |
-| Website (optional)             |                      |

.github/contributors/thomashacker.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |  Edward Schmuhl                    |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                    |
-| Date                           |  09.07.2021                    |
-| GitHub username                |  thomashacker                    |
-| Website (optional)             |                      |

.github/contributors/xadrianzetx.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           |Adrian Zuber          |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           |20-06-2021            |
-| GitHub username                |xadrianzetx           |
-| Website (optional)             |                      |

.github/contributors/yohasebe.md

@@ -1,106 +0,0 @@
-# spaCy contributor agreement
-
-This spaCy Contributor Agreement (**"SCA"**) is based on the
-[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
-The SCA applies to any contribution that you make to any product or project
-managed by us (the **"project"**), and sets out the intellectual property rights
-you grant to us in the contributed materials. The term **"us"** shall mean
-[ExplosionAI GmbH](https://explosion.ai/legal). The term
-**"you"** shall mean the person or entity identified below.
-
-If you agree to be bound by these terms, fill in the information requested
-below and include the filled-in version with your first pull request, under the
-folder [`.github/contributors/`](/.github/contributors/). The name of the file
-should be your GitHub username, with the extension `.md`. For example, the user
-example_user would create the file `.github/contributors/example_user.md`.
-
-Read this agreement carefully before signing. These terms and conditions
-constitute a binding legal agreement.
-
-## Contributor Agreement
-
-1. The term "contribution" or "contributed materials" means any source code,
-object code, patch, tool, sample, graphic, specification, manual,
-documentation, or any other material posted or submitted by you to the project.
-
-2. With respect to any worldwide copyrights, or copyright applications and
-registrations, in your contribution:
-
-    * you hereby assign to us joint ownership, and to the extent that such
-    assignment is or becomes invalid, ineffective or unenforceable, you hereby
-    grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
-    royalty-free, unrestricted license to exercise all rights under those
-    copyrights. This includes, at our option, the right to sublicense these same
-    rights to third parties through multiple levels of sublicensees or other
-    licensing arrangements;
-
-    * you agree that each of us can do all things in relation to your
-    contribution as if each of us were the sole owners, and if one of us makes
-    a derivative work of your contribution, the one who makes the derivative
-    work (or has it made will be the sole owner of that derivative work;
-
-    * you agree that you will not assert any moral rights in your contribution
-    against us, our licensees or transferees;
-
-    * you agree that we may register a copyright in your contribution and
-    exercise all ownership rights associated with it; and
-
-    * you agree that neither of us has any duty to consult with, obtain the
-    consent of, pay or render an accounting to the other for any use or
-    distribution of your contribution.
-
-3. With respect to any patents you own, or that you can license without payment
-to any third party, you hereby grant to us a perpetual, irrevocable,
-non-exclusive, worldwide, no-charge, royalty-free license to:
-
-    * make, have made, use, sell, offer to sell, import, and otherwise transfer
-    your contribution in whole or in part, alone or in combination with or
-    included in any product, work or materials arising out of the project to
-    which your contribution was submitted, and
-
-    * at our option, to sublicense these same rights to third parties through
-    multiple levels of sublicensees or other licensing arrangements.
-
-4. Except as set out above, you keep all right, title, and interest in your
-contribution. The rights that you grant to us under these terms are effective
-on the date you first submitted a contribution to us, even if your submission
-took place before the date you sign these terms.
-
-5. You covenant, represent, warrant and agree that:
-
-    * Each contribution that you submit is and shall be an original work of
-    authorship and you can legally grant the rights set out in this SCA;
-
-    * to the best of your knowledge, each contribution will not violate any
-    third party's copyrights, trademarks, patents, or other intellectual
-    property rights; and
-
-    * each contribution shall be in compliance with U.S. export control laws and
-    other applicable export and import laws. You agree to notify us if you
-    become aware of any circumstance which would make any of the foregoing
-    representations inaccurate in any respect. We may publicly disclose your
-    participation in the project, including the fact that you have signed the SCA.
-
-6. This SCA is governed by the laws of the State of California and applicable
-U.S. Federal law. Any choice of law rules will not apply.
-
-7. Please place an “x” on one of the applicable statement below. Please do NOT
-mark both statements:
-
-    * [x] I am signing on behalf of myself as an individual and no other person
-    or entity, including my employer, has or will have rights with respect to my
-    contributions.
-
-    * [ ] I am signing on behalf of my employer or a legal entity and I have the
-    actual authority to contractually bind that entity.
-
-## Contributor Details
-
-| Field                          | Entry                |
-|------------------------------- | -------------------- |
-| Name                           | Yoichiro Hasebe      |
-| Company name (if applicable)   |                      |
-| Title or role (if applicable)  |                      |
-| Date                           | July 4th, 2021       |
-| GitHub username                | yohasebe             |
-| Website (optional)             | https://yohasebe.com |

.github/lock.yml

@@ -0,0 +1,19 @@
+# Configuration for lock-threads - https://github.com/dessant/lock-threads
+
+# Number of days of inactivity before a closed issue or pull request is locked
+daysUntilLock: 30
+
+# Issues and pull requests with these labels will not be locked. Set to `[]` to disable
+exemptLabels: []
+
+# Label to add before locking, such as `outdated`. Set to `false` to disable
+lockLabel: false
+
+# Comment to post before locking. Set to `false` to disable
+lockComment: >
+  This thread has been automatically locked since there has not been
+  any recent activity after it was closed. Please open a new issue for
+  related bugs.
+
+# Limit to only `issues` or `pulls`
+only: issues

.github/no-response.yml

@@ -0,0 +1,13 @@
+# Configuration for probot-no-response - https://github.com/probot/no-response
+
+# Number of days of inactivity before an Issue is closed for lack of response
+daysUntilClose: 14
+# Label requiring a response
+responseRequiredLabel: more-info-needed
+# Comment to post when closing an Issue for lack of response. Set to `false` to disable
+closeComment: >
+  This issue has been automatically closed because there has been no response
+  to a request for more information from the original author. With only the
+  information that is currently in the issue, there's not enough information
+  to take action. If you're the original author, feel free to reopen the issue
+  if you have or find the answers needed to investigate further.

.github/spacy_universe_alert.py

@@ -1,67 +0,0 @@
-import os
-import sys
-import json
-from datetime import datetime
-
-from slack_sdk.web.client import WebClient
-
-CHANNEL = "#alerts-universe"
-SLACK_TOKEN = os.environ.get("SLACK_BOT_TOKEN", "ENV VAR not available!")
-DATETIME_FORMAT = "%Y-%m-%dT%H:%M:%SZ"
-
-client = WebClient(SLACK_TOKEN)
-github_context = json.loads(sys.argv[1])
-
-event = github_context['event']
-pr_title = event['pull_request']["title"]
-pr_link = event['pull_request']["patch_url"].replace(".patch", "")
-pr_author_url = event['sender']["html_url"]
-pr_author_name = pr_author_url.rsplit('/')[-1]
-pr_created_at_dt = datetime.strptime(
-    event['pull_request']["created_at"],
-    DATETIME_FORMAT
-)
-pr_created_at = pr_created_at_dt.strftime("%c")
-pr_updated_at_dt = datetime.strptime(
-    event['pull_request']["updated_at"],
-    DATETIME_FORMAT
-)
-pr_updated_at = pr_updated_at_dt.strftime("%c")
-
-blocks = [
-    {
-      "type": "section",
-      "text": {
-        "type": "mrkdwn",
-        "text": "📣 New spaCy Universe Project Alert ✨"
-      }
-    },
-    {
-      "type": "section",
-      "fields": [
-        {
-          "type": "mrkdwn",
-          "text": f"*Pull Request:*\n<{pr_link}|{pr_title}>"
-        },
-        {
-          "type": "mrkdwn",
-          "text": f"*Author:*\n<{pr_author_url}|{pr_author_name}>"
-        },
-        {
-          "type": "mrkdwn",
-          "text": f"*Created at:*\n {pr_created_at}"
-        },
-        {
-          "type": "mrkdwn",
-          "text": f"*Last Updated:*\n {pr_updated_at}"
-        }
-      ]
-    }
-  ]
-
-
-client.chat_postMessage(
-    channel=CHANNEL,
-    text="spaCy universe project PR alert",
-    blocks=blocks
-)

.github/validate_universe_json.py

@@ -1,19 +0,0 @@
-import json
-import re
-import sys
-from pathlib import Path
-
-
-def validate_json(document):
-    universe_file = Path(document)
-    with universe_file.open() as f:
-        universe_data = json.load(f)
-        for entry in universe_data["resources"]:
-            if "github" in entry:
-                assert not re.match(
-                    r"^(http:)|^(https:)", entry["github"]
-                ), "Github field should be user/repo, not a url"
-
-
-if __name__ == "__main__":
-    validate_json(str(sys.argv[1]))

.github/workflows/cibuildwheel.yml

@@ -1,99 +0,0 @@
-name: Build
-
-on:
-  push:
-    tags:
-      # ytf did they invent their own syntax that's almost regex?
-      # ** matches 'zero or more of any character'
-      - 'release-v[0-9]+.[0-9]+.[0-9]+**'
-      - 'prerelease-v[0-9]+.[0-9]+.[0-9]+**'
-jobs:
-  build_wheels:
-    name: Build wheels on ${{ matrix.os }}
-    runs-on: ${{ matrix.os }}
-    strategy:
-      matrix:
-        # macos-13 is an intel runner, macos-14 is apple silicon
-        os: [ubuntu-latest, windows-latest, macos-13, macos-14, ubuntu-24.04-arm]
-
-    steps:
-      - uses: actions/checkout@v4
-      # aarch64 (arm) is built via qemu emulation
-      # QEMU is sadly too slow. We need to wait for public ARM support
-      #- name: Set up QEMU
-      #  if: runner.os == 'Linux'
-      #  uses: docker/setup-qemu-action@v3
-      #  with:
-      #    platforms: all
-      - name: Build wheels
-        uses: pypa/cibuildwheel@v2.21.3
-        env:
-          CIBW_ARCHS_LINUX: auto
-        with:
-          package-dir: .
-          output-dir: wheelhouse
-          config-file: "{package}/pyproject.toml"
-      - uses: actions/upload-artifact@v4
-        with:
-          name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
-          path: ./wheelhouse/*.whl
-
-  build_sdist:
-    name: Build source distribution
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Build sdist
-        run: pipx run build --sdist
-      - uses: actions/upload-artifact@v4
-        with:
-          name: cibw-sdist
-          path: dist/*.tar.gz
-  create_release:
-    needs: [build_wheels, build_sdist]
-    runs-on: ubuntu-latest
-    permissions:
-      contents: write
-      checks: write
-      actions: read
-      issues: read
-      packages: write
-      pull-requests: read
-      repository-projects: read
-      statuses: read
-    steps:
-      - name: Get the tag name and determine if it's a prerelease
-        id: get_tag_info
-        run: |
-          FULL_TAG=${GITHUB_REF#refs/tags/}
-          if [[ $FULL_TAG == release-* ]]; then
-            TAG_NAME=${FULL_TAG#release-}
-            IS_PRERELEASE=false
-          elif [[ $FULL_TAG == prerelease-* ]]; then
-            TAG_NAME=${FULL_TAG#prerelease-}
-            IS_PRERELEASE=true
-          else
-            echo "Tag does not match expected patterns" >&2
-            exit 1
-          fi
-          echo "FULL_TAG=$TAG_NAME" >> $GITHUB_ENV
-          echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
-          echo "IS_PRERELEASE=$IS_PRERELEASE" >> $GITHUB_ENV
-      - uses: actions/download-artifact@v4
-        with:
-          # unpacks all CIBW artifacts into dist/
-          pattern: cibw-*
-          path: dist
-          merge-multiple: true
-      - name: Create Draft Release
-        id: create_release
-        uses: softprops/action-gh-release@v2
-        if: startsWith(github.ref, 'refs/tags/')
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        with:
-          name: ${{ env.TAG_NAME }}
-          draft: true
-          prerelease: ${{ env.IS_PRERELEASE }}
-          files: "./dist/*" 

.github/workflows/explosionbot.yml

@@ -1,28 +0,0 @@
-name: Explosion Bot
-
-on:
-  issue_comment:
-    types:
-      - created
-      - edited
-
-jobs:
-  explosion-bot:
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Dump GitHub context
-        env:
-          GITHUB_CONTEXT: ${{ toJson(github) }}
-        run: echo "$GITHUB_CONTEXT"
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v4
-      - name: Install and run explosion-bot
-        run: |
-          pip install git+https://${{ secrets.EXPLOSIONBOT_TOKEN }}@github.com/explosion/explosion-bot
-          python -m explosionbot
-        env:
-          INPUT_TOKEN: ${{ secrets.EXPLOSIONBOT_TOKEN }}
-          INPUT_BK_TOKEN: ${{ secrets.BUILDKITE_SECRET }}
-          ENABLED_COMMANDS: "test_gpu,test_slow,test_slow_gpu"
-          ALLOWED_TEAMS: "spaCy"

.github/workflows/gputests.yml.disabled

@@ -1,22 +0,0 @@
-name: Weekly GPU tests
-
-on:
-  schedule:
-    - cron: '0 1 * * MON'
-
-jobs:
-  weekly-gputests:
-    strategy:
-      fail-fast: false
-      matrix:
-        branch: [master, v4]
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Trigger buildkite build
-        uses: buildkite/trigger-pipeline-action@v1.2.0
-        env:
-          PIPELINE: explosion-ai/spacy-slow-gpu-tests
-          BRANCH: ${{ matrix.branch }}
-          MESSAGE: ":github: Weekly GPU + slow tests - triggered from a GitHub Action"
-          BUILDKITE_API_ACCESS_TOKEN: ${{ secrets.BUILDKITE_SECRET }}

.github/workflows/issue-manager.yml

@@ -13,10 +13,9 @@ on:
 
 jobs:
   issue-manager:
-    if: github.repository_owner == 'explosion'
     runs-on: ubuntu-latest
     steps:
-      - uses: tiangolo/issue-manager@0.4.0
+      - uses: tiangolo/issue-manager@0.2.1
         with:
           token: ${{ secrets.GITHUB_TOKEN }}
           config: >
@@ -26,11 +25,5 @@ jobs:
                 "message": "This issue has been automatically closed because it was answered and there was no follow-up discussion.",
                 "remove_label_on_comment": true,
                 "remove_label_on_close": true
-              },
-              "more-info-needed": {
-                "delay": "P7D",
-                "message": "This issue has been automatically closed because there has been no response to a request for more information from the original author. With only the information that is currently in the issue, there's not enough information to take action. If you're the original author, feel free to reopen the issue if you have or find the answers needed to investigate further.",
-                "remove_label_on_comment": true,
-                "remove_label_on_close": true
               }
             }

.github/workflows/lock.yml

@@ -1,26 +0,0 @@
-name: 'Lock Threads'
-
-on:
-  schedule:
-    - cron: '0 0 * * *'  # check every day
-  workflow_dispatch:
-
-permissions:
-  issues: write
-
-concurrency:
-  group: lock
-
-jobs:
-  action:
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-    steps:
-      - uses: dessant/lock-threads@v5
-        with:
-          process-only: 'issues'
-          issue-inactive-days: '30'
-          issue-comment: >
-            This thread has been automatically locked since there
-            has not been any recent activity after it was closed.
-            Please open a new issue for related bugs.

.github/workflows/publish_pypi.yml

@@ -1,29 +0,0 @@
-# The cibuildwheel action triggers on creation of a release, this
-# triggers on publication.
-# The expected workflow is to create a draft release and let the wheels
-# upload, and then hit 'publish', which uploads to PyPi.
-
-on:
-  release:
-    types:
-      - published
-
-jobs:
-  upload_pypi:
-    runs-on: ubuntu-latest
-    environment:
-      name: pypi
-      url: https://pypi.org/p/spacy
-    permissions:
-      id-token: write
-      contents: read
-    if: github.event_name == 'release' && github.event.action == 'published'
-    # or, alternatively, upload to PyPI on every tag starting with 'v' (remove on: release above to use this)
-    # if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
-    steps:
-      - uses: robinraju/release-downloader@v1
-        with:
-          tag: ${{ github.event.release.tag_name }}
-          fileName: '*'
-          out-file-path: 'dist'
-      - uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/slowtests.yml.disabled

@@ -1,38 +0,0 @@
-name: Daily slow tests
-
-on:
-  schedule:
-    - cron: '0 0 * * *'
-
-jobs:
-  daily-slowtests:
-    strategy:
-      fail-fast: false
-      matrix:
-        branch: [master, v4]
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v4
-        with:
-          ref: ${{ matrix.branch }}
-      - name: Get commits from past 24 hours
-        id: check_commits
-        run: |
-          today=$(date '+%Y-%m-%d %H:%M:%S')
-          yesterday=$(date -d "yesterday" '+%Y-%m-%d %H:%M:%S')
-          if git log --after="$yesterday" --before="$today" | grep commit ; then
-            echo run_tests=true >> $GITHUB_OUTPUT
-          else
-            echo run_tests=false >> $GITHUB_OUTPUT
-          fi
-
-      - name: Trigger buildkite build
-        if: steps.check_commits.outputs.run_tests == 'true'
-        uses: buildkite/trigger-pipeline-action@v1.2.0
-        env:
-          PIPELINE: explosion-ai/spacy-slow-tests
-          BRANCH: ${{ matrix.branch }}
-          MESSAGE: ":github: Daily slow tests - triggered from a GitHub Action"
-          BUILDKITE_API_ACCESS_TOKEN: ${{ secrets.BUILDKITE_SECRET }}

.github/workflows/spacy_universe_alert.yml

@@ -1,33 +0,0 @@
-name: spaCy universe project alert
-
-on:
-  pull_request_target:
-    paths:
-      - "website/meta/universe.json"
-
-jobs:
-  build:
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-
-    steps:
-      - name: Dump GitHub context
-        env:
-          GITHUB_CONTEXT: ${{ toJson(github) }}
-          PR_NUMBER: ${{github.event.number}}
-        run: |
-          echo "$GITHUB_CONTEXT"
-
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v4
-        with:
-          python-version: '3.10'
-      - name: Install Bernadette app dependency and send an alert
-        env:
-          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
-          GITHUB_CONTEXT: ${{ toJson(github) }}
-          CHANNEL: "#alerts-universe"
-        run: |
-          pip install slack-sdk==3.17.2 aiohttp==3.8.1
-          echo "$CHANNEL"
-          python .github/spacy_universe_alert.py "$GITHUB_CONTEXT"

.github/workflows/tests.yml

@@ -1,175 +0,0 @@
-name: tests
-
-on:
-  push:
-    tags-ignore:
-      - '**'
-    branches-ignore:
-      - "spacy.io"
-      - "nightly.spacy.io"
-      - "v2.spacy.io"
-    paths-ignore:
-      - "*.md"
-      - "*.mdx"
-      - "website/**"
-  pull_request:
-    types: [opened, synchronize, reopened, edited]
-    paths-ignore:
-      - "*.md"
-      - "*.mdx"
-      - "website/**"
-
-jobs:
-  validate:
-    name: Validate
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check out repo
-        uses: actions/checkout@v4
-
-      - name: Configure Python version
-        uses: actions/setup-python@v4
-        with:
-          python-version: "3.10"
-
-      - name: black
-        run: |
-          python -m pip install black -c requirements.txt
-          python -m black spacy --check
-      - name: isort
-        run: |
-          python -m pip install isort -c requirements.txt
-          python -m isort spacy --check
-      - name: flake8
-        run: |
-          python -m pip install flake8==5.0.4
-          python -m flake8 spacy --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics
-          # Unfortunately cython-lint isn't working after the shift to Cython 3.
-          #- name: cython-lint
-          #  run: |
-          #    python -m pip install cython-lint -c requirements.txt
-          #    # E501: line too log, W291: trailing whitespace, E266: too many leading '#' for block comment
-          #    cython-lint spacy --ignore E501,W291,E266
-
-  tests:
-    name: Test
-    needs: Validate
-    strategy:
-      fail-fast: true
-      matrix:
-        os: [ubuntu-latest, windows-latest, macos-latest]
-        python_version: ["3.9", "3.12", "3.13"]
-
-    runs-on: ${{ matrix.os }}
-
-    steps:
-      - name: Check out repo
-        uses: actions/checkout@v4
-
-      - name: Configure Python version
-        uses: actions/setup-python@v4
-        with:
-          python-version: ${{ matrix.python_version }}
-
-      - name: Install dependencies
-        run: |
-          python -m pip install -U build pip setuptools
-          python -m pip install -U -r requirements.txt
-
-      - name: Build sdist
-        run: |
-          python -m build --sdist
-
-      - name: Run mypy
-        run: |
-          python -m mypy spacy
-        if: matrix.python_version != '3.7'
-
-      - name: Delete source directory and .egg-info
-        run: |
-          rm -rf spacy *.egg-info
-        shell: bash
-
-      - name: Uninstall all packages
-        run: |
-          python -m pip freeze
-          python -m pip freeze --exclude pywin32 > installed.txt
-          python -m pip uninstall -y -r installed.txt
-
-      - name: Install from sdist
-        run: |
-          SDIST=$(python -c "import os;print(os.listdir('./dist')[-1])" 2>&1)
-          SPACY_NUM_BUILD_JOBS=2 python -m pip install dist/$SDIST
-        shell: bash
-
-      - name: Test import
-        run: python -W error -c "import spacy"
-
-      - name: "Test download CLI"
-        run: |
-          python -m spacy download ca_core_news_sm
-          python -m spacy download ca_core_news_md
-          python -c "import spacy; nlp=spacy.load('ca_core_news_sm'); doc=nlp('test')"
-        if: matrix.python_version == '3.9'
-
-      - name: "Test download_url in info CLI"
-        run: |
-          python -W error -m spacy info ca_core_news_sm | grep -q download_url
-        if: matrix.python_version == '3.9'
-
-      - name: "Test no warnings on load (#11713)"
-        run: |
-          python -W error -c "import ca_core_news_sm; nlp = ca_core_news_sm.load(); doc=nlp('test')"
-        if: matrix.python_version == '3.9'
-
-      - name: "Test convert CLI"
-        run: |
-          python -m spacy convert extra/example_data/ner_example_data/ner-token-per-line-conll2003.json .
-        if: matrix.python_version == '3.9'
-
-      - name: "Test debug config CLI"
-        run: |
-          python -m spacy init config -p ner -l ca ner.cfg
-          python -m spacy debug config ner.cfg --paths.train ner-token-per-line-conll2003.spacy --paths.dev ner-token-per-line-conll2003.spacy
-        if: matrix.python_version == '3.9'
-
-      - name: "Test debug data CLI"
-        run: |
-          # will have errors due to sparse data, check for summary in output
-          python -m spacy debug data ner.cfg --paths.train ner-token-per-line-conll2003.spacy --paths.dev ner-token-per-line-conll2003.spacy | grep -q Summary
-        if: matrix.python_version == '3.9'
-
-      - name: "Test train CLI"
-        run: |
-          python -m spacy train ner.cfg --paths.train ner-token-per-line-conll2003.spacy --paths.dev ner-token-per-line-conll2003.spacy --training.max_steps 10 --gpu-id -1
-        if: matrix.python_version == '3.9'
-
-      - name: "Test assemble CLI"
-        run: |
-          python -c "import spacy; config = spacy.util.load_config('ner.cfg'); config['components']['ner'] = {'source': 'ca_core_news_sm'}; config.to_disk('ner_source_sm.cfg')"
-          python -m spacy assemble ner_source_sm.cfg output_dir
-        env:
-          PYTHONWARNINGS: "error,ignore::DeprecationWarning" 
-        if: matrix.python_version == '3.9'
-
-      - name: "Test assemble CLI vectors warning"
-        run: |
-          python -c "import spacy; config = spacy.util.load_config('ner.cfg'); config['components']['ner'] = {'source': 'ca_core_news_md'}; config.to_disk('ner_source_md.cfg')"
-          python -m spacy assemble ner_source_md.cfg output_dir 2>&1 | grep -q W113
-        if: matrix.python_version == '3.9'
-
-      - name: "Install test requirements"
-        run: |
-          python -m pip install -U -r requirements.txt
-
-      - name: "Run CPU tests"
-        run: |
-          python -m pytest --pyargs spacy -W error
-        if: "!(startsWith(matrix.os, 'macos') && matrix.python_version == '3.11')"
-
-      - name: "Run CPU tests with thinc-apple-ops"
-        run: |
-          python -m pip install 'spacy[apple]'
-          python -m pytest --pyargs spacy
-        if: startsWith(matrix.os, 'macos') && matrix.python_version == '3.11'

.github/workflows/universe_validation.yml

@@ -1,32 +0,0 @@
-name: universe validation
-
-on:
-  push:
-    branches-ignore:
-      - "spacy.io"
-      - "nightly.spacy.io"
-      - "v2.spacy.io"
-    paths:
-      - "website/meta/universe.json"
-  pull_request:
-    types: [opened, synchronize, reopened, edited]
-    paths:
-      - "website/meta/universe.json"
-
-jobs:
-  validate:
-    name: Validate
-    if: github.repository_owner == 'explosion'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check out repo
-        uses: actions/checkout@v4
-
-      - name: Configure Python version
-        uses: actions/setup-python@v4
-        with:
-          python-version: "3.7"
-
-      - name: Validate website/meta/universe.json
-        run: |
-          python .github/validate_universe_json.py website/meta/universe.json

.gitignore

@@ -10,11 +10,20 @@ spacy/tests/package/setup.cfg
 spacy/tests/package/pyproject.toml
 spacy/tests/package/requirements.txt
 
+# Website
+website/.cache/
+website/public/
+website/node_modules
+website/.npm
+website/logs
+*.log
+npm-debug.log*
+quickstart-training-generator.js
+
 # Cython / C extensions
 cythonize.json
 spacy/*.html
 *.cpp
-*.c
 *.so
 
 # Vim / VSCode / editors

.pre-commit-config.yaml

@@ -1,13 +0,0 @@
-repos:
--   repo: https://github.com/ambv/black
-    rev: 22.3.0
-    hooks:
-    - id: black
-      language_version: python3.7
-      additional_dependencies: ['click==8.0.4']
--   repo: https://github.com/pycqa/flake8
-    rev: 5.0.4
-    hooks:
-    - id: flake8
-      args:
-        - "--config=setup.cfg"

CITATION

@@ -0,0 +1,8 @@
+@software{spacy,
+  author = {Honnibal, Matthew and Montani, Ines and Van Landeghem, Sofie and Boyd, Adriane},
+  title = {{spaCy: Industrial-strength Natural Language Processing in Python}},
+  year = 2020,
+  publisher = {Zenodo},
+  doi = {10.5281/zenodo.1212303},
+  url = {https://doi.org/10.5281/zenodo.1212303}
+}

CITATION.cff

@@ -1,16 +0,0 @@
-cff-version: 1.2.0
-preferred-citation:
-  type: article
-  message: "If you use spaCy, please cite it as below."
-  authors:
-  - family-names: "Honnibal"
-    given-names: "Matthew"
-  - family-names: "Montani"
-    given-names: "Ines"
-  - family-names: "Van Landeghem"
-    given-names: "Sofie"
-  - family-names: "Boyd"
-    given-names: "Adriane"
-  title: "spaCy: Industrial-strength Natural Language Processing in Python"
-  doi: "10.5281/zenodo.1212303"
-  year: 2020

CONTRIBUTING.md

@@ -2,7 +2,11 @@
 
 # Contribute to spaCy
 
-Thanks for your interest in contributing to spaCy 🎉 This page will give you a quick
+Thanks for your interest in contributing to spaCy 🎉 The project is maintained
+by **[@honnibal](https://github.com/honnibal)**,
+**[@ines](https://github.com/ines)**, **[@svlandeg](https://github.com/svlandeg)** and
+**[@adrianeboyd](https://github.com/adrianeboyd)**,
+and we'll do our best to help you get started. This page will give you a quick
 overview of how things are organized and most importantly, how to get involved.
 
 ## Table of contents
@@ -35,7 +39,7 @@ so that more people can benefit from it.
 
 When opening an issue, use a **descriptive title** and include your
 **environment** (operating system, Python version, spaCy version). Our
-[issue templates](https://github.com/explosion/spaCy/issues/new/choose) help you
+[issue template](https://github.com/explosion/spaCy/issues/new) helps you
 remember the most important details to include. If you've discovered a bug, you
 can also submit a [regression test](#fixing-bugs) straight away. When you're
 opening an issue to report the bug, simply refer to your pull request in the
@@ -140,28 +144,29 @@ Changes to `.py` files will be effective immediately.
 
 📖 **For more details and instructions, see the documentation on [compiling spaCy from source](https://spacy.io/usage/#source) and the [quickstart widget](https://spacy.io/usage/#section-quickstart) to get the right commands for your platform and Python version.**
 
+### Contributor agreement
+
+If you've made a contribution to spaCy, you should fill in the
+[spaCy contributor agreement](.github/CONTRIBUTOR_AGREEMENT.md) to ensure that
+your contribution can be used across the project. If you agree to be bound by
+the terms of the agreement, fill in the [template](.github/CONTRIBUTOR_AGREEMENT.md)
+and include it with your pull request, or submit it separately to
+[`.github/contributors/`](/.github/contributors). The name of the file should be
+your GitHub username, with the extension `.md`. For example, the user
+example_user would create the file `.github/contributors/example_user.md`.
+
 ### Fixing bugs
 
 When fixing a bug, first create an
-[issue](https://github.com/explosion/spaCy/issues) if one does not already
+[issue](https://github.com/explosion/spaCy/issues) if one does not already exist.
-exist. The description text can be very short – we don't want to make this too
+The description text can be very short – we don't want to make this too
 bureaucratic.
 
-Next, add a test to the relevant file in the
+Next, create a test file named `test_issue[ISSUE NUMBER].py` in the
-[`spacy/tests`](spacy/tests)folder. Then add a [pytest
+[`spacy/tests/regression`](spacy/tests/regression) folder. Test for the bug
-mark](https://docs.pytest.org/en/6.2.x/example/markers.html#working-with-custom-markers),
+you're fixing, and make sure the test fails. Next, add and commit your test file
-`@pytest.mark.issue(NUMBER)`, to reference the issue number.
+referencing the issue number in the commit message. Finally, fix the bug, make
-
+sure your test passes and reference the issue in your commit message.
-```python
-# Assume you're fixing Issue #1234
-@pytest.mark.issue(1234)
-def test_issue1234():
-    ...
-```
-
-Test for the bug you're fixing, and make sure the test fails. Next, add and
-commit your test file. Finally, fix the bug, make sure your test passes and
-reference the issue number in your pull request description.
 
 📖 **For more information on how to add tests, check out the [tests README](spacy/tests/README.md).**
 
@@ -173,22 +178,9 @@ formatting and [`flake8`](http://flake8.pycqa.org/en/latest/) for linting its
 Python modules. If you've built spaCy from source, you'll already have both
 tools installed.
 
-As a general rule of thumb, we use f-strings for any formatting of strings.
-One exception are calls to Python's `logging` functionality.
-To avoid unnecessary string conversions in these cases, we use string formatting
-templates with `%s` and `%d` etc.
-
 **⚠️ Note that formatting and linting is currently only possible for Python
 modules in `.py` files, not Cython modules in `.pyx` and `.pxd` files.**
 
-### Pre-Commit Hooks
-
-After cloning the repo, after installing the packages from `requirements.txt`, enter the repo folder and run `pre-commit install`.
-Each time a `git commit` is initiated, `black` and `flake8` will run automatically on the modified files only.
-
-In case of error, or when `black` modified a file, the modified file needs to be `git add` once again and a new
-`git commit` has to be issued.
-
 ### Code formatting
 
 [`black`](https://github.com/ambv/black) is an opinionated Python code
@@ -238,7 +230,7 @@ also want to keep an eye on unused declared variables or repeated
 (i.e. overwritten) dictionary keys. If your code was formatted with `black`
 (see above), you shouldn't see any formatting-related warnings.
 
-The `flake8` section in [`setup.cfg`](setup.cfg) defines the configuration we use for this
+The [`.flake8`](.flake8) config defines the configuration we use for this
 codebase. For example, we're not super strict about the line length, and we're
 excluding very large files like lemmatization and tokenizer exception tables.
 
@@ -276,8 +268,7 @@ except:  # noqa: E722
 
 ### Python conventions
 
-All Python code must be written **compatible with Python 3.6+**. More detailed
+All Python code must be written **compatible with Python 3.6+**.
-code conventions can be found in the [developer docs](https://github.com/explosion/spaCy/blob/master/extra/DEVELOPER_DOCS/Code%20Conventions.md).
 
 #### I/O and handling paths
 
@@ -418,7 +409,14 @@ all test files and test functions need to be prefixed with `test_`.
 When adding tests, make sure to use descriptive names, keep the code short and
 concise and only test for one behavior at a time. Try to `parametrize` test
 cases wherever possible, use our pre-defined fixtures for spaCy components and
-avoid unnecessary imports. Extensive tests that take a long time should be marked with `@pytest.mark.slow`.
+avoid unnecessary imports.
+
+Extensive tests that take a long time should be marked with `@pytest.mark.slow`.
+Tests that require the model to be loaded should be marked with
+`@pytest.mark.models`. Loading the models is expensive and not necessary if
+you're not actually testing the model performance. If all you need is a `Doc`
+object with annotations like heads, POS tags or the dependency parse, you can
+use the `Doc` constructor to construct it manually.
 
 📖 **For more guidelines and information on how to add tests, check out the [tests README](spacy/tests/README.md).**
 
@@ -435,7 +433,7 @@ simply click on the "Suggest edits" button at the bottom of a page.
 ## Publishing spaCy extensions and plugins
 
 We're very excited about all the new possibilities for **community extensions**
-and plugins in spaCy v3.0, and we can't wait to see what you build with it!
+and plugins in spaCy v2.0, and we can't wait to see what you build with it!
 
 - An extension or plugin should add substantial functionality, be
   **well-documented** and **open-source**. It should be available for users to download
@@ -449,12 +447,13 @@ and plugins in spaCy v3.0, and we can't wait to see what you build with it!
   [`spacy`](https://github.com/topics/spacy?o=desc&s=stars) and
   [`spacy-extensions`](https://github.com/topics/spacy-extension?o=desc&s=stars)
   to make it easier to find. Those are also the topics we're linking to from the
-  spaCy website. If you're sharing your project on X, feel free to tag
+  spaCy website. If you're sharing your project on Twitter, feel free to tag
-  [@spacy_io](https://x.com/spacy_io) so we can check it out.
+  [@spacy_io](https://twitter.com/spacy_io) so we can check it out.
 
-- Once your extension is published, you can open a
+- Once your extension is published, you can open an issue on the
-  [PR](https://github.com/explosion/spaCy/pulls) to suggest it for the
+  [issue tracker](https://github.com/explosion/spacy/issues) to suggest it for the
-  [Universe](https://spacy.io/universe) page.
+  [resources directory](https://spacy.io/usage/resources#extensions) on the
+  website.
 
 📖 **For more tips and best practices, see the [checklist for developing spaCy extensions](https://spacy.io/usage/processing-pipelines#extensions).**
 

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (C) 2016-2024 ExplosionAI GmbH, 2016 spaCy GmbH, 2015 Matthew Honnibal
+Copyright (C) 2016-2021 ExplosionAI GmbH, 2016 spaCy GmbH, 2015 Matthew Honnibal
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

MANIFEST.in

@@ -1,9 +1,9 @@
-recursive-include spacy *.pyi *.pyx *.pxd *.txt *.cfg *.jinja *.toml *.hh
+recursive-include include *.h
+recursive-include spacy *.pyx *.pxd *.txt *.cfg *.jinja
 include LICENSE
 include README.md
 include pyproject.toml
-include spacy/py.typed
+recursive-exclude spacy/lang *.json
-recursive-include spacy/cli *.yml
+recursive-include spacy/lang *.json.gz
-recursive-include spacy/tests *.json
+recursive-include spacy/cli *.json *.yml
 recursive-include licenses *
-recursive-exclude spacy *.cpp

Makefile

@@ -1,11 +1,11 @@
 SHELL := /bin/bash
 
 ifndef SPACY_EXTRAS
-override SPACY_EXTRAS = spacy-lookups-data==1.0.3
+override SPACY_EXTRAS = spacy-lookups-data==1.0.0 jieba spacy-pkuseg==0.0.28 sudachipy sudachidict_core pymorphy2
 endif
 
 ifndef PYVER
-override PYVER = 3.8
+override PYVER = 3.6
 endif
 
 VENV := ./env$(PYVER)

README.md

@@ -6,20 +6,20 @@ spaCy is a library for **advanced Natural Language Processing** in Python and
 Cython. It's built on the very latest research, and was designed from day one to
 be used in real products.
 
-spaCy comes with [pretrained pipelines](https://spacy.io/models) and currently
+spaCy comes with
-supports tokenization and training for **70+ languages**. It features
+[pretrained pipelines](https://spacy.io/models) and
-state-of-the-art speed and **neural network models** for tagging, parsing,
+currently supports tokenization and training for **60+ languages**. It features
-**named entity recognition**, **text classification** and more, multi-task
+state-of-the-art speed and **neural network models** for tagging,
-learning with pretrained **transformers** like BERT, as well as a
+parsing, **named entity recognition**, **text classification** and more,
+multi-task learning with pretrained **transformers** like BERT, as well as a
 production-ready [**training system**](https://spacy.io/usage/training) and easy
 model packaging, deployment and workflow management. spaCy is commercial
-open-source software, released under the
+open-source software, released under the MIT license.
-[MIT license](https://github.com/explosion/spaCy/blob/master/LICENSE).
 
-💫 **Version 3.8 out now!**
+💫 **Version 3.0 out now!**
 [Check out the release notes here.](https://github.com/explosion/spaCy/releases)
 
-[![tests](https://github.com/explosion/spaCy/actions/workflows/tests.yml/badge.svg)](https://github.com/explosion/spaCy/actions/workflows/tests.yml)
+[![Azure Pipelines](https://img.shields.io/azure-devops/build/explosion-ai/public/8/master.svg?logo=azure-pipelines&style=flat-square&label=build)](https://dev.azure.com/explosion-ai/public/_build?definitionId=8)
 [![Current Release Version](https://img.shields.io/github/release/explosion/spacy.svg?style=flat-square&logo=github)](https://github.com/explosion/spaCy/releases)
 [![pypi Version](https://img.shields.io/pypi/v/spacy.svg?style=flat-square&logo=pypi&logoColor=white)](https://pypi.org/project/spacy/)
 [![conda Version](https://img.shields.io/conda/vn/conda-forge/spacy.svg?style=flat-square&logo=conda-forge&logoColor=white)](https://anaconda.org/conda-forge/spacy)
@@ -28,79 +28,66 @@ open-source software, released under the
 <br />
 [![PyPi downloads](https://static.pepy.tech/personalized-badge/spacy?period=total&units=international_system&left_color=grey&right_color=orange&left_text=pip%20downloads)](https://pypi.org/project/spacy/)
 [![Conda downloads](https://img.shields.io/conda/dn/conda-forge/spacy?label=conda%20downloads)](https://anaconda.org/conda-forge/spacy)
+[![spaCy on Twitter](https://img.shields.io/twitter/follow/spacy_io.svg?style=social&label=Follow)](https://twitter.com/spacy_io)
 
 ## 📖 Documentation
 
-| Documentation                                                                                                                                                                                                             |                                                                                                                                                                                                                                                                                                                                              |
+| Documentation              |                                                                |
-| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| -------------------------- | -------------------------------------------------------------- |
-| ⭐️ **[spaCy 101]**                                                                                                                                                                                                       | New to spaCy? Here's everything you need to know!                                                                                                                                                                                                                                                                                            |
+| ⭐️ **[spaCy 101]**        | New to spaCy? Here's everything you need to know!              |
-| 📚 **[Usage Guides]**                                                                                                                                                                                                     | How to use spaCy and its features.                                                                                                                                                                                                                                                                                                           |
+| 📚 **[Usage Guides]**      | How to use spaCy and its features.                             |
-| 🚀 **[New in v3.0]**                                                                                                                                                                                                      | New features, backwards incompatibilities and migration guide.                                                                                                                                                                                                                                                                               |
+| 🚀 **[New in v3.0]**       | New features, backwards incompatibilities and migration guide. |
-| 🪐 **[Project Templates]**                                                                                                                                                                                                | End-to-end workflows you can clone, modify and run.                                                                                                                                                                                                                                                                                          |
+| 🪐 **[Project Templates]** | End-to-end workflows you can clone, modify and run.            |
-| 🎛 **[API Reference]**                                                                                                                                                                                                     | The detailed reference for spaCy's API.                                                                                                                                                                                                                                                                                                      |
+| 🎛 **[API Reference]**      | The detailed reference for spaCy's API.                        |
-| ⏩ **[GPU Processing]**                                                                                                                                                                                                    | Use spaCy with CUDA-compatible GPU processing.                                                                                                                                                                                                                                                                                               |
+| 📦 **[Models]**            | Download trained pipelines for spaCy.                          |
-| 📦 **[Models]**                                                                                                                                                                                                           | Download trained pipelines for spaCy.                                                                                                                                                                                                                                                                                                        |
+| 🌌 **[Universe]**          | Plugins, extensions, demos and books from the spaCy ecosystem. |
-| 🦙 **[Large Language Models]**                                                                                                                                                                                            | Integrate LLMs into spaCy pipelines.                                                                                                                                                                                                                                                                                                        |
+| 👩‍🏫 **[Online Course]**     | Learn spaCy in this free and interactive online course.        |
-| 🌌 **[Universe]**                                                                                                                                                                                                         | Plugins, extensions, demos and books from the spaCy ecosystem.                                                                                                                                                                                                                                                                               |
+| 📺 **[Videos]**            | Our YouTube channel with video tutorials, talks and more.      |
-| ⚙️ **[spaCy VS Code Extension]**                                                                                                                                                                                          | Additional tooling and features for working with spaCy's config files.                                                                                                                                                                                                                                                                       |
+| 🛠 **[Changelog]**          | Changes and version history.                                   |
-| 👩‍🏫 **[Online Course]**                                                                                                                                                                                                    | Learn spaCy in this free and interactive online course.                                                                                                                                                                                                                                                                                      |
+| 💝 **[Contribute]**        | How to contribute to the spaCy project and code base.          |
-| 📰 **[Blog]**                                                                                                                                                                                                             | Read about current spaCy and Prodigy development, releases, talks and more from Explosion.                                                                                                                                                                                                                 |
-| 📺 **[Videos]**                                                                                                                                                                                                           | Our YouTube channel with video tutorials, talks and more.                                                                                                                                                                                                                                                                                    |
-| 🔴 **[Live Stream]**                                                                                                                                                                                                       | Join Matt as he works on spaCy and chat about NLP, live every week.                                                                                                                                                                                                                                                                         |
-| 🛠 **[Changelog]**                                                                                                                                                                                                         | Changes and version history.                                                                                                                                                                                                                                                                                                                 |
-| 💝 **[Contribute]**                                                                                                                                                                                                       | How to contribute to the spaCy project and code base.                                                                                                                                                                                                                                                                                        |
-| 👕 **[Swag]**                                                                                                                                                                                                             | Support us and our work with unique, custom-designed swag!                                                                                                                                                                                                                                                                                   |
-| <a href="https://explosion.ai/tailored-solutions"><img src="https://github.com/explosion/spaCy/assets/13643239/36d2a42e-98c0-4599-90e1-788ef75181be" width="150" alt="Tailored Solutions"/></a> | Custom NLP consulting, implementation and strategic advice by spaCy’s core development team. Streamlined, production-ready, predictable and maintainable. Send us an email or take our 5-minute questionnaire, and well'be in touch! **[Learn more &rarr;](https://explosion.ai/tailored-solutions)**                 |
 
 [spacy 101]: https://spacy.io/usage/spacy-101
 [new in v3.0]: https://spacy.io/usage/v3
 [usage guides]: https://spacy.io/usage/
 [api reference]: https://spacy.io/api/
-[gpu processing]: https://spacy.io/usage#gpu
 [models]: https://spacy.io/models
-[large language models]: https://spacy.io/usage/large-language-models
 [universe]: https://spacy.io/universe
-[spacy vs code extension]: https://github.com/explosion/spacy-vscode
 [videos]: https://www.youtube.com/c/ExplosionAI
-[live stream]: https://www.youtube.com/playlist?list=PLBmcuObd5An5_iAxNYLJa_xWmNzsYce8c
 [online course]: https://course.spacy.io
-[blog]: https://explosion.ai
 [project templates]: https://github.com/explosion/projects
 [changelog]: https://spacy.io/usage#changelog
 [contribute]: https://github.com/explosion/spaCy/blob/master/CONTRIBUTING.md
-[swag]: https://explosion.ai/merch
 
 ## 💬 Where to ask questions
 
-The spaCy project is maintained by the [spaCy team](https://explosion.ai/about).
+The spaCy project is maintained by **[@honnibal](https://github.com/honnibal)**,
-Please understand that we won't be able to provide individual support via email.
+**[@ines](https://github.com/ines)**, **[@svlandeg](https://github.com/svlandeg)** and
-We also believe that help is much more valuable if it's shared publicly, so that
+**[@adrianeboyd](https://github.com/adrianeboyd)**. Please understand that we won't
-more people can benefit from it.
+be able to provide individual support via email. We also believe that help is
+much more valuable if it's shared publicly, so that more people can benefit from
+it.
 
 | Type                            | Platforms                               |
 | ------------------------------- | --------------------------------------- |
 | 🚨 **Bug Reports**              | [GitHub Issue Tracker]                  |
-| 🎁 **Feature Requests & Ideas** | [GitHub Discussions] · [Live Stream]    |
+| 🎁 **Feature Requests & Ideas** | [GitHub Discussions]                    |
 | 👩‍💻 **Usage Questions**          | [GitHub Discussions] · [Stack Overflow] |
-| 🗯 **General Discussion**        | [GitHub Discussions] · [Live Stream]   |
+| 🗯 **General Discussion**        | [GitHub Discussions]                    |
 
 [github issue tracker]: https://github.com/explosion/spaCy/issues
 [github discussions]: https://github.com/explosion/spaCy/discussions
 [stack overflow]: https://stackoverflow.com/questions/tagged/spacy
-[live stream]: https://www.youtube.com/playlist?list=PLBmcuObd5An5_iAxNYLJa_xWmNzsYce8c
 
 ## Features
 
-- Support for **70+ languages**
+- Support for **60+ languages**
 - **Trained pipelines** for different languages and tasks
 - Multi-task learning with pretrained **transformers** like BERT
 - Support for pretrained **word vectors** and embeddings
 - State-of-the-art speed
 - Production-ready **training system**
 - Linguistically-motivated **tokenization**
-- Components for named **entity recognition**, part-of-speech-tagging,
+- Components for named **entity recognition**, part-of-speech-tagging, dependency parsing, sentence segmentation, **text classification**, lemmatization, morphological analysis, entity linking and more
-  dependency parsing, sentence segmentation, **text classification**,
-  lemmatization, morphological analysis, entity linking and more
 - Easily extensible with **custom components** and attributes
 - Support for custom models in **PyTorch**, **TensorFlow** and other frameworks
 - Built in **visualizers** for syntax and NER
@@ -117,7 +104,7 @@ For detailed installation instructions, see the
 
 - **Operating system**: macOS / OS X · Linux · Windows (Cygwin, MinGW, Visual
   Studio)
-- **Python version**: Python >=3.7, <3.13 (only 64 bit)
+- **Python version**: Python 3.6+ (only 64 bit)
 - **Package managers**: [pip] · [conda] (via `conda-forge`)
 
 [pip]: https://pypi.org/project/spacy/
@@ -126,8 +113,8 @@ For detailed installation instructions, see the
 ### pip
 
 Using pip, spaCy releases are available as source packages and binary wheels.
-Before you install spaCy and its dependencies, make sure that your `pip`,
+Before you install spaCy and its dependencies, make sure that
-`setuptools` and `wheel` are up to date.
+your `pip`, `setuptools` and `wheel` are up to date.
 
 ```bash
 pip install -U pip setuptools wheel
@@ -182,9 +169,9 @@ with the new version.
 
 ## 📦 Download model packages
 
-Trained pipelines for spaCy can be installed as **Python packages**. This means
+Trained pipelines for spaCy can be installed as **Python packages**. This
-that they're a component of your application, just like any other module. Models
+means that they're a component of your application, just like any other module.
-can be installed using spaCy's [`download`](https://spacy.io/api/cli#download)
+Models can be installed using spaCy's [`download`](https://spacy.io/api/cli#download)
 command, or manually by pointing pip to a path or URL.
 
 | Documentation              |                                                                  |
@@ -250,7 +237,8 @@ do that depends on your system.
 | **Mac**     | Install a recent version of [XCode](https://developer.apple.com/xcode/), including the so-called "Command Line Tools". macOS and OS X ship with Python and git preinstalled.                                                                                        |
 | **Windows** | Install a version of the [Visual C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/) or [Visual Studio Express](https://visualstudio.microsoft.com/vs/express/) that matches the version that was used to compile your Python interpreter. |
 
-For more details and instructions, see the documentation on
+For more details
+and instructions, see the documentation on
 [compiling spaCy from source](https://spacy.io/usage#source) and the
 [quickstart widget](https://spacy.io/usage#section-quickstart) to get the right
 commands for your platform and Python version.

azure-pipelines.yml

@@ -0,0 +1,114 @@
+trigger:
+  batch: true
+  branches:
+    include:
+      - "*"
+    exclude:
+      - "spacy.io"
+      - "nightly.spacy.io"
+      - "v2.spacy.io"
+  paths:
+    exclude:
+      - "website/*"
+      - "*.md"
+pr:
+  paths:
+    exclude:
+      - "website/*"
+      - "*.md"
+
+jobs:
+  # Perform basic checks for most important errors (syntax etc.) Uses the config
+  # defined in .flake8 and overwrites the selected codes.
+  - job: "Validate"
+    pool:
+      vmImage: "ubuntu-16.04"
+    steps:
+      - task: UsePythonVersion@0
+        inputs:
+          versionSpec: "3.7"
+      - script: |
+          pip install flake8==3.5.0
+          python -m flake8 spacy --count --select=E901,E999,F821,F822,F823 --show-source --statistics
+        displayName: "flake8"
+
+  - job: "Test"
+    dependsOn: "Validate"
+    strategy:
+      matrix:
+        # We're only running one platform per Python version to speed up builds
+        Python36Linux:
+          imageName: "ubuntu-16.04"
+          python.version: "3.6"
+        #        Python36Windows:
+        #          imageName: "vs2017-win2016"
+        #          python.version: "3.6"
+        #        Python36Mac:
+        #          imageName: "macos-10.14"
+        #          python.version: "3.6"
+        #        Python37Linux:
+        #          imageName: "ubuntu-16.04"
+        #          python.version: "3.7"
+        Python37Windows:
+          imageName: "vs2017-win2016"
+          python.version: "3.7"
+        #        Python37Mac:
+        #          imageName: "macos-10.14"
+        #          python.version: "3.7"
+        #        Python38Linux:
+        #          imageName: "ubuntu-16.04"
+        #          python.version: "3.8"
+        #        Python38Windows:
+        #          imageName: "vs2017-win2016"
+        #          python.version: "3.8"
+        Python38Mac:
+          imageName: "macos-10.14"
+          python.version: "3.8"
+        Python39Linux:
+          imageName: "ubuntu-16.04"
+          python.version: "3.9"
+        Python39Windows:
+          imageName: "vs2017-win2016"
+          python.version: "3.9"
+        Python39Mac:
+          imageName: "macos-10.14"
+          python.version: "3.9"
+      maxParallel: 4
+    pool:
+      vmImage: $(imageName)
+
+    steps:
+      - task: UsePythonVersion@0
+        inputs:
+          versionSpec: "$(python.version)"
+          architecture: "x64"
+
+      - script: |
+          python -m pip install -U setuptools
+          pip install -r requirements.txt
+        displayName: "Install dependencies"
+
+      - script: |
+          python setup.py build_ext --inplace
+          python setup.py sdist --formats=gztar
+        displayName: "Compile and build sdist"
+
+      - task: DeleteFiles@1
+        inputs:
+          contents: "spacy"
+        displayName: "Delete source directory"
+
+      - script: |
+          pip freeze > installed.txt
+          pip uninstall -y -r installed.txt
+        displayName: "Uninstall all packages"
+
+      - bash: |
+          SDIST=$(python -c "import os;print(os.listdir('./dist')[-1])" 2>&1)
+          pip install dist/$SDIST
+        displayName: "Install from sdist"
+
+      - script: |
+          pip install -r requirements.txt
+          python -m pytest --pyargs spacy
+        displayName: "Run tests"

bin/release.sh

@@ -1,20 +0,0 @@
-#!/usr/bin/env bash
-
-set -e
-
-# Insist repository is clean
-git diff-index --quiet HEAD
-
-version=$(grep "__version__ = " spacy/about.py)
-version=${version/__version__ = }
-version=${version/\'/}
-version=${version/\'/}
-version=${version/\"/}
-version=${version/\"/}
-
-echo "Pushing release-v"$version
-
-git tag -d release-v$version || true
-git push origin :release-v$version || true
-git tag release-v$version
-git push origin release-v$version

build-constraints.txt

@@ -1,2 +1,5 @@
-# build version constraints for use with wheelwright
+# build version constraints for use with wheelwright + multibuild
-numpy>=2.0.0,<3.0.0
+numpy==1.15.0; python_version<='3.7'
+numpy==1.17.3; python_version=='3.8'
+numpy==1.19.3; python_version=='3.9'
+numpy; python_version>='3.10'

examples/README.md

@@ -1,130 +0,0 @@
-<a href="https://explosion.ai"><img src="https://explosion.ai/assets/img/logo.svg" width="125" height="125" align="right" /></a>
-
-# spaCy examples
-
-For spaCy v3 we've converted many of the [v2 example
-scripts](https://github.com/explosion/spaCy/tree/v2.3.x/examples/) into
-end-to-end [spacy projects](https://spacy.io/usage/projects) workflows. The
-workflows include all the steps to go from data to packaged spaCy models.
-
-## 🪐 Pipeline component demos
-
-The simplest demos for training a single pipeline component are in the
-[`pipelines`](https://github.com/explosion/projects/blob/v3/pipelines) category
-including:
-
-- [`pipelines/ner_demo`](https://github.com/explosion/projects/blob/v3/pipelines/ner_demo):
-  Train a named entity recognizer
-- [`pipelines/textcat_demo`](https://github.com/explosion/projects/blob/v3/pipelines/textcat_demo):
-  Train a text classifier
-- [`pipelines/parser_intent_demo`](https://github.com/explosion/projects/blob/v3/pipelines/parser_intent_demo):
-  Train a dependency parser for custom semantics
-
-## 🪐 Tutorials
-
-The [`tutorials`](https://github.com/explosion/projects/blob/v3/tutorials)
-category includes examples that work through specific NLP use cases end-to-end:
-
-- [`tutorials/textcat_goemotions`](https://github.com/explosion/projects/blob/v3/tutorials/textcat_goemotions):
-  Train a text classifier to categorize emotions in Reddit posts
-- [`tutorials/nel_emerson`](https://github.com/explosion/projects/blob/v3/tutorials/nel_emerson):
-  Use an entity linker to disambiguate mentions of the same name
-
-Check out the [projects documentation](https://spacy.io/usage/projects) and
-browse through the [available
-projects](https://github.com/explosion/projects/)!
-
-## 🚀 Get started with a demo project
-
-The
-[`pipelines/ner_demo`](https://github.com/explosion/projects/blob/v3/pipelines/ner_demo)
-project converts the spaCy v2
-[`train_ner.py`](https://github.com/explosion/spaCy/blob/v2.3.x/examples/training/train_ner.py)
-demo script into a spaCy v3 project.
-
-1. Clone the project:
-
-   ```bash
-   python -m spacy project clone pipelines/ner_demo
-   ```
-
-2. Install requirements and download any data assets:
-
-   ```bash
-   cd ner_demo
-   python -m pip install -r requirements.txt
-   python -m spacy project assets
-   ```
-
-3. Run the default workflow to convert, train and evaluate:
-
-   ```bash
-   python -m spacy project run all
-   ```
-
-   Sample output:
-
-   ```none
-   ℹ Running workflow 'all'
-   
-   ================================== convert ==================================
-   Running command: /home/user/venv/bin/python scripts/convert.py en assets/train.json corpus/train.spacy
-   Running command: /home/user/venv/bin/python scripts/convert.py en assets/dev.json corpus/dev.spacy
-   
-   =============================== create-config ===============================
-   Running command: /home/user/venv/bin/python -m spacy init config --lang en --pipeline ner configs/config.cfg --force
-   ℹ Generated config template specific for your use case
-   - Language: en
-   - Pipeline: ner
-   - Optimize for: efficiency
-   - Hardware: CPU
-   - Transformer: None
-   ✔ Auto-filled config with all values
-   ✔ Saved config
-   configs/config.cfg
-   You can now add your data and train your pipeline:
-   python -m spacy train config.cfg --paths.train ./train.spacy --paths.dev ./dev.spacy
-   
-   =================================== train ===================================
-   Running command: /home/user/venv/bin/python -m spacy train configs/config.cfg --output training/ --paths.train corpus/train.spacy --paths.dev corpus/dev.spacy --training.eval_frequency 10 --training.max_steps 100 --gpu-id -1
-   ℹ Using CPU
-   
-   =========================== Initializing pipeline ===========================
-   [2021-03-11 19:34:59,101] [INFO] Set up nlp object from config
-   [2021-03-11 19:34:59,109] [INFO] Pipeline: ['tok2vec', 'ner']
-   [2021-03-11 19:34:59,113] [INFO] Created vocabulary
-   [2021-03-11 19:34:59,113] [INFO] Finished initializing nlp object
-   [2021-03-11 19:34:59,265] [INFO] Initialized pipeline components: ['tok2vec', 'ner']
-   ✔ Initialized pipeline
-   
-   ============================= Training pipeline =============================
-   ℹ Pipeline: ['tok2vec', 'ner']
-   ℹ Initial learn rate: 0.001
-   E    #       LOSS TOK2VEC  LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE 
-   ---  ------  ------------  --------  ------  ------  ------  ------
-     0       0          0.00      7.90    0.00    0.00    0.00    0.00
-    10      10          0.11     71.07    0.00    0.00    0.00    0.00
-    20      20          0.65     22.44   50.00   50.00   50.00    0.50
-    30      30          0.22      6.38   80.00   66.67  100.00    0.80
-    40      40          0.00      0.00   80.00   66.67  100.00    0.80
-    50      50          0.00      0.00   80.00   66.67  100.00    0.80
-    60      60          0.00      0.00  100.00  100.00  100.00    1.00
-    70      70          0.00      0.00  100.00  100.00  100.00    1.00
-    80      80          0.00      0.00  100.00  100.00  100.00    1.00
-    90      90          0.00      0.00  100.00  100.00  100.00    1.00
-   100     100          0.00      0.00  100.00  100.00  100.00    1.00
-   ✔ Saved pipeline to output directory
-   training/model-last
-   ```
-
-4. Package the model:
-
-   ```bash
-   python -m spacy project run package
-   ```
-
-5. Visualize the model's output with [Streamlit](https://streamlit.io):
-
-   ```bash
-   python -m spacy project run visualize-model
-   ```

examples/training/README.md

@@ -1,5 +0,0 @@
-<a href="https://explosion.ai"><img src="https://explosion.ai/assets/img/logo.svg" width="125" height="125" align="right" /></a>
-
-# spaCy examples
-
-See [examples/README.md](../README.md)

extra/DEVELOPER_DOCS/Code Conventions.md

@@ -1,580 +0,0 @@
-# Code Conventions
-
-For a general overview of code conventions for contributors, see the [section in the contributing guide](https://github.com/explosion/spaCy/blob/master/CONTRIBUTING.md#code-conventions).
-
-1. [Code compatibility](#code-compatibility)
-2. [Auto-formatting](#auto-formatting)
-3. [Linting](#linting)
-4. [Documenting code](#documenting-code)
-5. [Type hints](#type-hints)
-6. [Structuring logic](#structuring-logic)
-7. [Naming](#naming)
-8. [Error handling](#error-handling)
-9. [Writing tests](#writing-tests)
-
-## Code compatibility
-
-spaCy supports **Python 3.6** and above, so all code should be written compatible with 3.6. This means that there are certain new syntax features that we won't be able to use until we drop support for older Python versions. Some newer features provide backports that we can conditionally install for older versions, although we only want to do this if it's absolutely necessary. If we need to use conditional imports based on the Python version or other custom compatibility-specific helpers, those should live in `compat.py`.
-
-## Auto-formatting
-
-spaCy uses `black` for auto-formatting (which is also available as a pre-commit hook). It's recommended to configure your editor to perform this automatically, either triggered manually or whenever you save a file. We also have a GitHub action that regularly formats the code base and submits a PR if changes are available. Note that auto-formatting is currently only available for `.py` (Python) files, not for `.pyx` (Cython).
-
-As a rule of thumb, if the auto-formatting produces output that looks messy, it can often indicate that there's a better way to structure the code to make it more concise.
-
-```diff
-- range_suggester = registry.misc.get("spacy.ngram_range_suggester.v1")(
--     min_size=1, max_size=3
-- )
-+ suggester_factory = registry.misc.get("spacy.ngram_range_suggester.v1")
-+ range_suggester = suggester_factory(min_size=1, max_size=3)
-```
-
-In some specific cases, e.g. in the tests, it can make sense to disable auto-formatting for a specific block. You can do this by wrapping the code in `# fmt: off` and `# fmt: on`:
-
-```diff
-+ # fmt: off
-text = "I look forward to using Thingamajig.  I've been told it will make my life easier..."
-deps = ["nsubj", "ROOT", "advmod", "prep", "pcomp", "dobj", "punct", "",
-        "nsubjpass", "aux", "auxpass", "ROOT", "nsubj", "aux", "ccomp",
-        "poss", "nsubj", "ccomp", "punct"]
-+ # fmt: on
-```
-
-## Linting
-
-[`flake8`](http://flake8.pycqa.org/en/latest/) is a tool for enforcing code style. It scans one or more files and outputs errors and warnings. This feedback can help you stick to general standards and conventions, and can be very useful for spotting potential mistakes and inconsistencies in your code. Code you write should be compatible with our flake8 rules and not cause any warnings.
-
-```bash
-flake8 spacy
-```
-
-The most common problems surfaced by linting are:
-
-- **Trailing or missing whitespace.** This is related to formatting and should be fixed automatically by running `black`.
-- **Unused imports.** Those should be removed if the imports aren't actually used. If they're required, e.g. to expose them so they can be imported from the given module, you can add a comment and `# noqa: F401` exception (see details below).
-- **Unused variables.** This can often indicate bugs, e.g. a variable that's declared and not correctly passed on or returned. To prevent ambiguity here, your code shouldn't contain unused variables. If you're unpacking a list of tuples and end up with variables you don't need, you can call them `_` to indicate that they're unused.
-- **Redefinition of function.** This can also indicate bugs, e.g. a copy-pasted function that you forgot to rename and that now replaces the original function.
-- **Repeated dictionary keys.** This either indicates a bug or unnecessary duplication.
-- **Comparison with `True`, `False`, `None`**. This is mostly a stylistic thing: when checking whether a value is `True`, `False` or `None`, you should be using `is` instead of `==`. For example, `if value is None`.
-
-### Ignoring linter rules for special cases
-
-To ignore a given line, you can add a comment like `# noqa: F401`, specifying the code of the error or warning we want to ignore. It's also possible to ignore several comma-separated codes at once, e.g. `# noqa: E731,E123`. In general, you should always **specify the code(s)** you want to ignore – otherwise, you may end up missing actual problems.
-
-```python
-# The imported class isn't used in this file, but imported here, so it can be
-# imported *from* here by another module.
-from .submodule import SomeClass  # noqa: F401
-
-try:
-    do_something()
-except:  # noqa: E722
-    # This bare except is justified, for some specific reason
-    do_something_else()
-```
-
-## Documenting code
-
-All functions and methods you write should be documented with a docstring inline. The docstring can contain a simple summary, and an overview of the arguments and their (simplified) types. Modern editors will show this information to users when they call the function or method in their code.
-
-If it's part of the public API and there's a documentation section available, we usually add the link as `DOCS:` at the end. This allows us to keep the docstrings simple and concise, while also providing additional information and examples if necessary.
-
-```python
-def has_pipe(self, name: str) -> bool:
-    """Check if a component name is present in the pipeline. Equivalent to
-    `name in nlp.pipe_names`.
-
-    name (str): Name of the component.
-    RETURNS (bool): Whether a component of the name exists in the pipeline.
-
-    DOCS: https://spacy.io/api/language#has_pipe
-    """
-    ...
-```
-
-We specifically chose this approach of maintaining the docstrings and API reference separately, instead of auto-generating the API docs from the docstrings like other packages do. We want to be able to provide extensive explanations and examples in the documentation and use our own custom markup for it that would otherwise clog up the docstrings. We also want to be able to update the documentation independently of the code base. It's slightly more work, but it's absolutely worth it in terms of user and developer experience.
-
-### Inline code comments
-
-We don't expect you to add inline comments for everything you're doing – this should be obvious from reading the code. If it's not, the first thing to check is whether your code can be improved to make it more explicit. That said, if your code includes complex logic or aspects that may be unintuitive at first glance (or even included a subtle bug that you ended up fixing), you should leave a quick comment that provides more context.
-
-```diff
-token_index = indices[value]
-+ # Index describes Token.i of last token but Span indices are inclusive
-span = doc[prev_token_index:token_index + 1]
-```
-
-```diff
-+ # To create the components we need to use the final interpolated config
-+ # so all values are available (if component configs use variables).
-+ # Later we replace the component config with the raw config again.
-interpolated = filled.interpolate() if not filled.is_interpolated else filled
-```
-
-Don't be shy about including comments for tricky parts that _you_ found hard to implement or get right – those may come in handy for the next person working on this code, or even future you!
-
-If your change implements a fix to a specific issue, it can often be helpful to include the issue number in the comment, especially if it's a relatively straightforward adjustment:
-
-```diff
-+ # Ensure object is a Span, not a Doc (#1234)
-if isinstance(obj, Doc):
-    obj = obj[obj.start:obj.end]
-```
-
-### Including TODOs
-
-It's fine to include code comments that indicate future TODOs, using the `TODO:` prefix. Modern editors typically format this in a different color, so it's easy to spot. TODOs don't necessarily have to be things that are absolutely critical to fix fight now – those should already be addressed in your pull request once it's ready for review. But they can include notes about potential future improvements.
-
-```diff
-+ # TODO: this is currently pretty slow
-dir_checksum = hashlib.md5()
-for sub_file in sorted(fp for fp in path.rglob("*") if fp.is_file()):
-    dir_checksum.update(sub_file.read_bytes())
-```
-
-If any of the TODOs you've added are important and should be fixed soon, you should add a task for this on Explosion's internal Ora board or an issue on the public issue tracker to make sure we don't forget to address it.
-
-## Type hints
-
-We use Python type hints across the `.py` files wherever possible. This makes it easy to understand what a function expects and returns, and modern editors will be able to show this information to you when you call an annotated function. Type hints are not currently used in the `.pyx` (Cython) code, except for definitions of registered functions and component factories, where they're used for config validation. Ideally when developing, run `mypy spacy` on the code base to inspect any issues.
-
-If possible, you should always use the more descriptive type hints like `List[str]` or even `List[Any]` instead of only `list`. We also annotate arguments and return types of `Callable` – although, you can simplify this if the type otherwise gets too verbose (e.g. functions that return factories to create callbacks). Remember that `Callable` takes two values: a **list** of the argument type(s) in order, and the return values.
-
-```diff
-- def func(some_arg: dict) -> None:
-+ def func(some_arg: Dict[str, Any]) -> None:
-    ...
-```
-
-```python
-def create_callback(some_arg: bool) -> Callable[[str, int], List[str]]:
-    def callback(arg1: str, arg2: int) -> List[str]:
-        ...
-
-    return callback
-```
-
-For typing variables, we prefer the explicit format.
-
-```diff
-- var = value    # type: Type
-+ var: Type = value
-```
-
-For model architectures, Thinc also provides a collection of [custom types](https://thinc.ai/docs/api-types), including more specific types for arrays and model inputs/outputs. Even outside of static type checking, using these types will make the code a lot easier to read and follow, since it's always clear what array types are expected (and what might go wrong if the output is different from the expected type).
-
-```python
-def build_tagger_model(
-    tok2vec: Model[List[Doc], List[Floats2d]], nO: Optional[int] = None
-) -> Model[List[Doc], List[Floats2d]]:
-    ...
-```
-
-If you need to use a type hint that refers to something later declared in the same module, or the class that a method belongs to, you can use a string value instead:
-
-```python
-class SomeClass:
-    def from_bytes(self, data: bytes) -> "SomeClass":
-        ...
-```
-
-In some cases, you won't be able to import a class from a different module to use it as a type hint because it'd cause circular imports. For instance, `spacy/util.py` includes various helper functions that return an instance of `Language`, but we couldn't import it, because `spacy/language.py` imports `util` itself. In this case, we can provide `"Language"` as a string and make the import conditional on `typing.TYPE_CHECKING` so it only runs when the code is evaluated by a type checker:
-
-```python
-from typing TYPE_CHECKING
-
-if TYPE_CHECKING:
-    from .language import Language
-
-def load_model(name: str) -> "Language":
-    ...
-```
-
-Note that we typically put the `from typing` import statements on the first line(s) of the Python module.
-
-## Structuring logic
-
-### Positional and keyword arguments
-
-We generally try to avoid writing functions and methods with too many arguments, and use keyword-only arguments wherever possible. Python lets you define arguments as keyword-only by separating them with a `, *`. If you're writing functions with additional arguments that customize the behavior, you typically want to make those arguments keyword-only, so their names have to be provided explicitly.
-
-```diff
-- def do_something(name: str, validate: bool = False):
-+ def do_something(name: str, *, validate: bool = False):
-    ...
-
-- do_something("some_name", True)
-+ do_something("some_name", validate=True)
-```
-
-This makes the function calls easier to read, because it's immediately clear what the additional values mean. It also makes it easier to extend arguments or change their order later on, because you don't end up with any function calls that depend on a specific positional order.
-
-### Avoid mutable default arguments
-
-A common Python gotcha are [mutable default arguments](https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments): if your argument defines a mutable default value like `[]` or `{}` and then goes and mutates it, the default value is created _once_ when the function is created and the same object is then mutated every time the function is called. This can be pretty unintuitive when you first encounter it. We therefore avoid writing logic that does this.
-
-If your arguments need to default to an empty list or dict, you can use the `SimpleFrozenList` and `SimpleFrozenDict` helpers provided by spaCy. They are simple frozen implementations that raise an error if they're being mutated to prevent bugs and logic that accidentally mutates default arguments.
-
-```diff
-- def to_bytes(self, *, exclude: List[str] = []):
-+ def to_bytes(self, *, exclude: List[str] = SimpleFrozenList()):
-    ...
-```
-
-```diff
-def do_something(values: List[str] = SimpleFrozenList()):
-    if some_condition:
--         values.append("foo")  # raises an error
-+         values = [*values, "foo"]
-    return values
-```
-
-### Don't use `try`/`except` for control flow
-
-We strongly discourage using `try`/`except` blocks for anything that's not third-party error handling or error handling that we otherwise have little control over. There's typically always a way to anticipate the _actual_ problem and **check for it explicitly**, which makes the code easier to follow and understand, and prevents bugs:
-
-```diff
-- try:
--     token = doc[i]
-- except IndexError:
--     token = doc[-1]
-
-+ if i < len(doc):
-+     token = doc[i]
-+ else:
-+     token = doc[-1]
-```
-
-Even if you end up having to check for multiple conditions explicitly, this is still preferred over a catch-all `try`/`except`. It can be very helpful to think about the exact scenarios you need to cover, and what could go wrong at each step, which often leads to better code and fewer bugs. `try/except` blocks can also easily mask _other_ bugs and problems that raise the same errors you're catching, which is obviously bad.
-
-If you have to use `try`/`except`, make sure to only include what's **absolutely necessary** in the `try` block and define the exception(s) explicitly. Otherwise, you may end up masking very different exceptions caused by other bugs.
-
-```diff
-- try:
--     value1 = get_some_value()
--     value2 = get_some_other_value()
--     score = external_library.compute_some_score(value1, value2)
-- except:
--     score = 0.0
-
-+ value1 = get_some_value()
-+ value2 = get_some_other_value()
-+ try:
-+     score = external_library.compute_some_score(value1, value2)
-+ except ValueError:
-+     score = 0.0
-```
-
-### Avoid lambda functions
-
-`lambda` functions can be useful for defining simple anonymous functions in a single line, but they also introduce problems: for instance, they require [additional logic](https://stackoverflow.com/questions/25348532/can-python-pickle-lambda-functions) in order to be pickled and are pretty ugly to type-annotate. So we typically avoid them in the code base and only use them in the serialization handlers and within tests for simplicity. Instead of `lambda`s, check if your code can be refactored to not need them, or use helper functions instead.
-
-```diff
-- split_string: Callable[[str], List[str]] = lambda value: [v.strip() for v in value.split(",")]
-
-+ def split_string(value: str) -> List[str]:
-+     return [v.strip() for v in value.split(",")]
-```
-
-### Numeric comparisons
-
-For numeric comparisons, as a general rule we always use `<` and `>=` and avoid the usage of `<=` and `>`. This is to ensure we consistently
-apply inclusive lower bounds and exclusive upper bounds, helping to prevent off-by-one errors.
-
-One exception to this rule is the ternary case. With a chain like
-
-```python
-if value >= 0 and value < max:
-    ...
-```
-
-it's fine to rewrite this to the shorter form
-
-```python
-if 0 <= value < max:
-    ...
-```
-
-even though this requires the usage of the `<=` operator.
-
-### Iteration and comprehensions
-
-We generally avoid using built-in functions like `filter` or `map` in favor of list or generator comprehensions.
-
-```diff
-- filtered = filter(lambda x: x in ["foo", "bar"], values)
-+ filtered = (x for x in values if x in ["foo", "bar"])
-- filtered = list(filter(lambda x: x in ["foo", "bar"], values))
-+ filtered = [x for x in values if x in ["foo", "bar"]]
-
-- result = map(lambda x: { x: x in ["foo", "bar"]}, values)
-+ result = ({x: x in ["foo", "bar"]} for x in values)
-- result = list(map(lambda x: { x: x in ["foo", "bar"]}, values))
-+ result = [{x: x in ["foo", "bar"]} for x in values]
-```
-
-If your logic is more complex, it's often better to write a loop instead, even if it adds more lines of code in total. The result will be much easier to follow and understand.
-
-```diff
-- result = [{"key": key, "scores": {f"{i}": score for i, score in enumerate(scores)}} for key, scores in values]
-
-+ result = []
-+ for key, scores in values:
-+     scores_dict = {f"{i}": score for i, score in enumerate(scores)}
-+     result.append({"key": key, "scores": scores_dict})
-```
-
-### Composition vs. inheritance
-
-Although spaCy uses a lot of classes, **inheritance is viewed with some suspicion** — it's seen as a mechanism of last resort. You should discuss plans to extend the class hierarchy before implementing. Unless you're implementing a new data structure or pipeline component, you typically shouldn't have to use classes at all.
-
-### Don't use `print`
-
-The core library never `print`s anything. While we encourage using `print` statements for simple debugging (it's the most straightforward way of looking at what's happening), make sure to clean them up once you're ready to submit your pull request. If you want to output warnings or debugging information for users, use the respective dedicated mechanisms for this instead (see sections on warnings and logging for details).
-
-The only exceptions are the CLI functions, which pretty-print messages for the user, and methods that are explicitly intended for printing things, e.g. `Language.analyze_pipes` with `pretty=True` enabled. For this, we use our lightweight helper library [`wasabi`](https://github.com/ines/wasabi).
-
-## Naming
-
-Naming is hard and often a topic of long internal discussions. We don't expect you to come up with the perfect names for everything you write – finding the right names is often an iterative and collaborative process. That said, we do try to follow some basic conventions.
-
-Consistent with general Python conventions, we use `CamelCase` for class names including dataclasses, `snake_case` for methods, functions and variables, and `UPPER_SNAKE_CASE` for constants, typically defined at the top of a module. We also avoid using variable names that shadow the names of built-in functions, e.g. `input`, `help` or `list`.
-
-### Naming variables
-
-Variable names should always make it clear _what exactly_ the variable is and what it's used for. Instances of common classes should use the same consistent names. For example, you should avoid naming a text string (or anything else that's not a `Doc` object) `doc`. The most common class-to-variable mappings are:
-
-| Class      | Variable              | Example                                     |
-| ---------- | --------------------- | ------------------------------------------- |
-| `Language` | `nlp`                 | `nlp = spacy.blank("en")`                   |
-| `Doc`      | `doc`                 | `doc = nlp("Some text")`                    |
-| `Span`     | `span`, `ent`, `sent` | `span = doc[1:4]`, `ent = doc.ents[0]`      |
-| `Token`    | `token`               | `token = doc[0]`                            |
-| `Lexeme`   | `lexeme`, `lex`       | `lex = nlp.vocab["foo"]`                    |
-| `Vocab`    | `vocab`               | `vocab = Vocab()`                           |
-| `Example`  | `example`, `eg`       | `example = Example.from_dict(doc, gold)`    |
-| `Config`   | `config`, `cfg`       | `config = Config().from_disk("config.cfg")` |
-
-We try to avoid introducing too many temporary variables, as these clutter your namespace. It's okay to re-assign to an existing variable, but only if the value has the same type.
-
-```diff
-ents = get_a_list_of_entities()
-ents = [ent for ent in doc.ents if ent.label_ == "PERSON"]
-- ents = {(ent.start, ent.end): ent.label_ for ent in ents}
-+ ent_mappings = {(ent.start, ent.end): ent.label_ for ent in ents}
-```
-
-### Naming methods and functions
-
-Try choosing short and descriptive names wherever possible and imperative verbs for methods that do something, e.g. `disable_pipes`, `add_patterns` or `get_vector`. Private methods and functions that are not intended to be part of the user-facing API should be prefixed with an underscore `_`. It's often helpful to look at the existing classes for inspiration.
-
-Objects that can be serialized, e.g. data structures and pipeline components, should implement the same consistent methods for serialization. Those usually include at least `to_disk`, `from_disk`, `to_bytes` and `from_bytes`. Some objects can also implement more specific methods like `{to/from}_dict` or `{to/from}_str`.
-
-## Error handling
-
-We always encourage writing helpful and detailed custom error messages for everything we can anticipate going wrong, and including as much detail as possible. spaCy provides a directory of error messages in `errors.py` with unique codes for each message. This allows us to keep the code base more concise and avoids long and nested blocks of texts throughout the code that disrupt the reading flow. The codes make it easy to find references to the same error in different places, and also helps identify problems reported by users (since we can just search for the error code).
-
-Errors can be referenced via their code, e.g. `Errors.E123`. Messages can also include placeholders for values, that can be populated by formatting the string with `.format()`.
-
-```python
-class Errors:
-    E123 = "Something went wrong"
-    E456 = "Unexpected value: {value}"
-```
-
-```diff
-if something_went_wrong:
--    raise ValueError("Something went wrong!")
-+    raise ValueError(Errors.E123)
-
-if not isinstance(value, int):
--    raise ValueError(f"Unexpected value: {value}")
-+    raise ValueError(Errors.E456.format(value=value))
-```
-
-As a general rule of thumb, all error messages raised within the **core library** should be added to `Errors`. The only place where we write errors and messages as strings is `spacy.cli`, since these functions typically pretty-print and generate a lot of output that'd otherwise be very difficult to separate from the actual logic.
-
-### Re-raising exceptions
-
-If we anticipate possible errors in third-party code that we don't control, or our own code in a very different context, we typically try to provide custom and more specific error messages if possible. If we need to re-raise an exception within a `try`/`except` block, we can re-raise a custom exception.
-
-[Re-raising `from`](https://docs.python.org/3/tutorial/errors.html#exception-chaining) the original caught exception lets us chain the exceptions, so the user sees both the original error, as well as the custom message with a note "The above exception was the direct cause of the following exception".
-
-```diff
-try:
-    run_third_party_code_that_might_fail()
-except ValueError as e:
-+    raise ValueError(Errors.E123) from e
-```
-
-In some cases, it makes sense to suppress the original exception, e.g. if we know what it is and know that it's not particularly helpful. In that case, we can raise `from None`. This prevents clogging up the user's terminal with multiple and irrelevant chained exceptions.
-
-```diff
-try:
-    run_our_own_code_that_might_fail_confusingly()
-except ValueError:
-+    raise ValueError(Errors.E123) from None
-```
-
-### Avoid using naked `assert`
-
-During development, it can sometimes be helpful to add `assert` statements throughout your code to make sure that the values you're working with are what you expect. However, as you clean up your code, those should either be removed or replaced by more explicit error handling:
-
-```diff
-- assert score >= 0.0
-+ if score < 0.0:
-+     raise ValueError(Errors.789.format(score=score))
-```
-
-Otherwise, the user will get to see a naked `AssertionError` with no further explanation, which is very unhelpful. Instead of adding an error message to `assert`, it's always better to `raise` more explicit errors for specific conditions. If you're checking for something that _has to be right_ and would otherwise be a bug in spaCy, you can express this in the error message:
-
-```python
-E161 = ("Found an internal inconsistency when predicting entity links. "
-        "This is likely a bug in spaCy, so feel free to open an issue: "
-        "https://github.com/explosion/spaCy/issues")
-```
-
-### Warnings
-
-Instead of raising an error, some parts of the code base can raise warnings to notify the user of a potential problem. This is done using Python's `warnings.warn` and the messages defined in `Warnings` in the `errors.py`. Whether or not warnings are shown can be controlled by the user, including custom filters for disabling specific warnings using a regular expression matching our internal codes, e.g. `W123`.
-
-```diff
-- print("Warning: No examples provided for validation")
-+ warnings.warn(Warnings.W123)
-```
-
-When adding warnings, make sure you're not calling `warnings.warn` repeatedly, e.g. in a loop, which will clog up the terminal output. Instead, you can collect the potential problems first and then raise a single warning. If the problem is critical, consider raising an error instead.
-
-```diff
-+ n_empty = 0
-for spans in lots_of_annotations:
-    if len(spans) == 0:
--       warnings.warn(Warnings.456)
-+       n_empty += 1
-+ warnings.warn(Warnings.456.format(count=n_empty))
-```
-
-### Logging
-
-Log statements can be added via spaCy's `logger`, which uses Python's native `logging` module under the hood. We generally only use logging for debugging information that **the user may choose to see** in debugging mode or that's **relevant during training** but not at runtime.
-
-```diff
-+ logger.info("Set up nlp object from config")
-config = nlp.config.interpolate()
-```
-
-`spacy train` and similar CLI commands will enable all log statements of level `INFO` by default (which is not the case at runtime). This allows outputting specific information within certain parts of the core library during training, without having it shown at runtime. `DEBUG`-level logs are only shown if the user enables `--verbose` logging during training. They can be used to provide more specific and potentially more verbose details, especially in areas that can indicate bugs or problems, or to surface more details about what spaCy does under the hood. You should only use logging statements if absolutely necessary and important.
-
-## Writing tests
-
-spaCy uses the [`pytest`](http://doc.pytest.org/) framework for testing. Tests for spaCy modules and classes live in their own directories of the same name and all test files should be prefixed with `test_`. Tests included in the core library only cover the code and do not depend on any trained pipelines. When implementing a new feature or fixing a bug, it's usually good to start by writing some tests that describe what _should_ happen. As you write your code, you can then keep running the relevant tests until all of them pass.
-
-### Test suite structure
-
-When adding tests, make sure to use descriptive names and only test for one behavior at a time. Tests should be grouped into modules dedicated to the same type of functionality and some test modules are organized as directories of test files related to the same larger area of the library, e.g. `matcher` or `tokenizer`.
-
-Regression tests are tests that refer to bugs reported in specific issues. They should live in the relevant module of the test suite, named according to the issue number (e.g., `test_issue1234.py`), and [marked](https://docs.pytest.org/en/6.2.x/example/markers.html#working-with-custom-markers) appropriately (e.g. `@pytest.mark.issue(1234)`). This system allows us to relate tests for specific bugs back to the original reported issue, which is especially useful if we introduce a regression and a previously passing regression tests suddenly fails again. When fixing a bug, it's often useful to create a regression test for it first.
-
-The test suite also provides [fixtures](https://github.com/explosion/spaCy/blob/master/spacy/tests/conftest.py) for different language tokenizers that can be used as function arguments of the same name and will be passed in automatically. Those should only be used for tests related to those specific languages. We also have [test utility functions](https://github.com/explosion/spaCy/blob/master/spacy/tests/util.py) for common operations, like creating a temporary file.
-
-### Testing Cython Code
-
-If you're developing Cython code (`.pyx` files), those extensions will need to be built before the test runner can test that code - otherwise it's going to run the tests with stale code from the last time the extension was built. You can build the extensions locally with `python setup.py build_ext -i`.
-
-### Constructing objects and state
-
-Test functions usually follow the same simple structure: they set up some state, perform the operation you want to test and `assert` conditions that you expect to be true, usually before and after the operation.
-
-Tests should focus on exactly what they're testing and avoid dependencies on other unrelated library functionality wherever possible. If all your test needs is a `Doc` object with certain annotations set, you should always construct it manually:
-
-```python
-def test_doc_creation_with_pos():
-    doc = Doc(Vocab(), words=["hello", "world"], pos=["NOUN", "VERB"])
-    assert doc[0].pos_ == "NOUN"
-    assert doc[1].pos_ == "VERB"
-```
-
-### Parametrizing tests
-
-If you need to run the same test function over different input examples, you usually want to parametrize the test cases instead of using a loop within your test. This lets you keep a better separation between test cases and test logic, and it'll result in more useful output because `pytest` will be able to tell you which exact test case failed.
-
-The `@pytest.mark.parametrize` decorator takes two arguments: a string defining one or more comma-separated arguments that should be passed to the test function and a list of corresponding test cases (or a list of tuples to provide multiple arguments).
-
-```python
-@pytest.mark.parametrize("words", [["hello", "world"], ["this", "is", "a", "test"]])
-def test_doc_length(words):
-    doc = Doc(Vocab(), words=words)
-    assert len(doc) == len(words)
-```
-
-```python
-@pytest.mark.parametrize("text,expected_len", [("hello world", 2), ("I can't!", 4)])
-def test_token_length(en_tokenizer, text, expected_len):  # en_tokenizer is a fixture
-    doc = en_tokenizer(text)
-    assert len(doc) == expected_len
-```
-
-You can also stack `@pytest.mark.parametrize` decorators, although this is not recommended unless it's absolutely needed or required for the test. When stacking decorators, keep in mind that this will run the test with all possible combinations of the respective parametrized values, which is often not what you want and can slow down the test suite.
-
-### Handling failing tests
-
-`xfail` means that a test **should pass but currently fails**, i.e. is expected to fail. You can mark a test as currently xfailing by adding the `@pytest.mark.xfail` decorator. This should only be used for tests that don't yet work, not for logic that cause errors we raise on purpose (see the section on testing errors for this). It's often very helpful to implement tests for edge cases that we don't yet cover and mark them as `xfail`. You can also provide a `reason` keyword argument to the decorator with an explanation of why the test currently fails.
-
-```diff
-+ @pytest.mark.xfail(reason="Issue #225 - not yet implemented")
-def test_en_tokenizer_splits_em_dash_infix(en_tokenizer):
-    doc = en_tokenizer("Will this road take me to Puddleton?\u2014No.")
-    assert doc[8].text == "\u2014"
-```
-
-When you run the test suite, you may come across tests that are reported as `xpass`. This means that they're marked as `xfail` but didn't actually fail. This is worth looking into: sometimes, it can mean that we have since fixed a bug that caused the test to previously fail, so we can remove the decorator. In other cases, especially when it comes to machine learning model implementations, it can also indicate that the **test is flaky**: it sometimes passes and sometimes fails. This can be caused by a bug, or by constraints being too narrowly defined. If a test shows different behavior depending on whether its run in isolation or not, this can indicate that it reacts to global state set in a previous test, which is unideal and should be avoided.
-
-### Writing slow tests
-
-If a test is useful but potentially quite slow, you can mark it with the `@pytest.mark.slow` decorator. This is a special marker we introduced and tests decorated with it only run if you run the test suite with `--slow`, but not as part of the main CI process. Before introducing a slow test, double-check that there isn't another and more efficient way to test for the behavior. You should also consider adding a simpler test with maybe only a subset of the test cases that can always run, so we at least have some coverage.
-
-### Skipping tests
-
-The `@pytest.mark.skip` decorator lets you skip tests entirely. You only want to do this for failing tests that may be slow to run or cause memory errors or segfaults, which would otherwise terminate the entire process and wouldn't be caught by `xfail`. We also sometimes use the `skip` decorator for old and outdated regression tests that we want to keep around but that don't apply anymore. When using the `skip` decorator, make sure to provide the `reason` keyword argument with a quick explanation of why you chose to skip this test.
-
-### Testing errors and warnings
-
-`pytest` lets you check whether a given error is raised by using the `pytest.raises` contextmanager. This is very useful when implementing custom error handling, so make sure you're not only testing for the correct behavior but also for errors resulting from incorrect inputs. If you're testing errors, you should always check for `pytest.raises` explicitly and not use `xfail`.
-
-```python
-words = ["a", "b", "c", "d", "e"]
-ents = ["Q-PERSON", "I-PERSON", "O", "I-PERSON", "I-GPE"]
-with pytest.raises(ValueError):
-    Doc(Vocab(), words=words, ents=ents)
-```
-
-You can also use the `pytest.warns` contextmanager to check that a given warning type is raised. The first argument is the warning type or `None` (which will capture a list of warnings that you can `assert` is empty).
-
-```python
-def test_phrase_matcher_validation(en_vocab):
-    doc1 = Doc(en_vocab, words=["Test"], deps=["ROOT"])
-    doc2 = Doc(en_vocab, words=["Test"])
-    matcher = PhraseMatcher(en_vocab, validate=True)
-    with pytest.warns(UserWarning):
-        # Warn about unnecessarily parsed document
-        matcher.add("TEST1", [doc1])
-    with pytest.warns(None) as record:
-        matcher.add("TEST2", [docs])
-        assert not record.list
-```
-
-Keep in mind that your tests will fail if you're using the `pytest.warns` contextmanager with a given warning and the warning is _not_ shown. So you should only use it to check that spaCy handles and outputs warnings correctly. If your test outputs a warning that's expected but not relevant to what you're testing, you can use the `@pytest.mark.filterwarnings` decorator and ignore specific warnings starting with a given code:
-
-```python
-@pytest.mark.filterwarnings("ignore:\\[W036")
-def test_matcher_empty(en_vocab):
-    matcher = Matcher(en_vocab)
-    matcher(Doc(en_vocab, words=["test"]))
-```
-
-### Testing trained pipelines
-
-Our regular test suite does not depend on any of the trained pipelines, since their outputs can vary and aren't generally required to test the library functionality. We test pipelines separately using the tests included in the [`spacy-models`](https://github.com/explosion/spacy-models) repository, which run whenever we train a new suite of models. The tests here mostly focus on making sure that the packages can be loaded and that the predictions seam reasonable, and they include checks for common bugs we encountered previously. If your test does not primarily focus on verifying a model's predictions, it should be part of the core library tests and construct the required objects manually, instead of being added to the models tests.
-
-Keep in mind that specific predictions may change, and we can't test for all incorrect predictions reported by users. Different models make different mistakes, so even a model that's significantly more accurate overall may end up making wrong predictions that it previously didn't. However, some surprising incorrect predictions may indicate deeper bugs that we definitely want to investigate.

extra/DEVELOPER_DOCS/ExplosionBot.md

@@ -1,56 +0,0 @@
-# Explosion-bot
-
-Explosion-bot is a robot that can be invoked to help with running particular test commands.
-
-## Permissions
-
-Only maintainers have permissions to summon explosion-bot. Each of the open source repos that use explosion-bot has its own team(s) of maintainers, and only github users who are members of those teams can successfully run bot commands.
-
-## Running robot commands
-
-To summon the robot, write a github comment on the issue/PR you wish to test. The comment must be in the following format:
-
-```
-@explosion-bot please test_gpu
-```
-
-Some things to note:
-
-- The `@explosion-bot please` must be the beginning of the command - you cannot add anything in front of this or else the robot won't know how to parse it. Adding anything at the end aside from the test name will also confuse the robot, so keep it simple!
-- The command name (such as `test_gpu`) must be one of the tests that the bot knows how to run. The available commands are documented in the bot's [workflow config](https://github.com/explosion/spaCy/blob/master/.github/workflows/explosionbot.yml#L26) and must match exactly one of the commands listed there.
-- The robot can't do multiple things at once, so if you want it to run multiple tests, you'll have to summon it with one comment per test.
-
-### Examples
-
-- Execute spaCy slow GPU tests with a custom thinc branch from a spaCy PR:
-
-  ```
-  @explosion-bot please test_slow_gpu --thinc-branch <branch_name>
-  ```
-
-  `branch_name` can either be a named branch, e.g: `develop`, or an unmerged PR, e.g: `refs/pull/<pr_number>/head`.
-
-- Execute spaCy Transformers GPU tests from a spaCy PR:
-
-  ```
-  @explosion-bot please test_gpu --run-on spacy-transformers --run-on-branch master --spacy-branch current_pr
-  ```
-
-  This will launch the GPU pipeline for the `spacy-transformers` repo on its `master` branch, using the current spaCy PR's branch to build spaCy. The name of the repository passed to `--run-on` is case-sensitive, e.g: use `spaCy` instead of `spacy`.
-
-- General info about supported commands.
-
-  ```
-  @explosion-bot please info
-  ```
-
-- Help text for a specific command
-  ```
-  @explosion-bot please <command> --help
-  ```
-
-## Troubleshooting
-
-If the robot isn't responding to commands as expected, you can check its logs in the [Github Action](https://github.com/explosion/spaCy/actions/workflows/explosionbot.yml).
-
-For each command sent to the bot, there should be a run of the `explosion-bot` workflow. In the `Install and run explosion-bot` step, towards the ends of the logs you should see info about the configuration that the bot was run with, as well as any errors that the bot encountered.

extra/DEVELOPER_DOCS/Language.md

@@ -1,150 +0,0 @@
-# Language
-
-> Reference: `spacy/language.py`
-
-1. [Constructing the `nlp` object from a config](#1-constructing-the-nlp-object-from-a-config)
-   - [A. Overview of `Language.from_config`](#1a-overview)
-   - [B. Component factories](#1b-how-pipeline-component-factories-work-in-the-config)
-   - [C. Sourcing a component](#1c-sourcing-a-pipeline-component)
-   - [D. Tracking components as they're modified](#1d-tracking-components-as-theyre-modified)
-   - [E. spaCy's config utility function](#1e-spacys-config-utility-functions)
-2. [Initialization](#initialization)
-   - [A. Initialization for training](#2a-initialization-for-training): `init_nlp`
-   - [B. Initializing the `nlp` object](#2b-initializing-the-nlp-object): `Language.initialize`
-   - [C. Initializing the vocab](#2c-initializing-the-vocab): `init_vocab`
-
-## 1. Constructing the `nlp` object from a config
-
-### 1A. Overview
-
-Most of the functions referenced in the config are regular functions with arbitrary arguments registered via the function registry. However, the pipeline components are a bit special: they don't only receive arguments passed in via the config file, but also the current `nlp` object and the string `name` of the individual component instance (so a user can have multiple components created with the same factory, e.g. `ner_one` and `ner_two`). This name can then be used by the components to add to the losses and scores. This special requirement means that pipeline components can't just be resolved via the config the "normal" way: we need to retrieve the component functions manually and pass them their arguments, plus the `nlp` and `name`.
-
-The `Language.from_config` classmethod takes care of constructing the `nlp` object from a config. It's the single place where this happens and what `spacy.load` delegates to under the hood. Its main responsibilities are:
-
-- **Load and validate the config**, and optionally **auto-fill** all missing values that we either have defaults for in the config template or that registered function arguments define defaults for. This helps ensure backwards-compatibility, because we're able to add a new argument `foo: str = "bar"` to an existing function, without breaking configs that don't specity it.
-- **Execute relevant callbacks** for pipeline creation, e.g. optional functions called before and after creation of the `nlp` object and pipeline.
-- **Initialize language subclass and create tokenizer**. The `from_config` classmethod will always be called on a language subclass, e.g. `English`, not on `Language` directly. Initializing the subclass takes a callback to create the tokenizer.
-- **Set up the pipeline components**. Components can either refer to a component factory or a `source`, i.e. an existing pipeline that's loaded and that the component is then copied from. We also need to ensure that we update the information about which components are disabled.
-- **Manage listeners.** If sourced components "listen" to other components (`tok2vec`, `transformer`), we need to ensure that the references are valid. If the config specifies that listeners should be replaced by copies (e.g. to give the `ner` component its own `tok2vec` model instead of listening to the shared `tok2vec` component in the pipeline), we also need to take care of that.
-
-Note that we only resolve and load **selected sections** in `Language.from_config`, i.e. only the parts that are relevant at runtime, which is `[nlp]` and `[components]`. We don't want to be resolving anything related to training or initialization, since this would mean loading and constructing unnecessary functions, including functions that require information that isn't necessarily available at runtime, like `paths.train`.
-
-### 1B. How pipeline component factories work in the config
-
-As opposed to regular registered functions that refer to a registry and function name (e.g. `"@misc": "foo.v1"`), pipeline components follow a different format and refer to their component `factory` name. This corresponds to the name defined via the `@Language.component` or `@Language.factory` decorator. We need this decorator to define additional meta information for the components, like their default config and score weights.
-
-```ini
-[components.my_component]
-factory = "foo"
-some_arg = "bar"
-other_arg = ${paths.some_path}
-```
-
-This means that we need to create and resolve the `config["components"]` separately from the rest of the config. There are some important considerations and things we need to manage explicitly to avoid unexpected behavior:
-
-#### Variable interpolation
-
-When a config is resolved, references to variables are replaced, so that the functions receive the correct value instead of just the variable name. To interpolate a config, we need it in its entirety: we couldn't just interpolate a subsection that refers to variables defined in a different subsection. So we first interpolate the entire config.
-
-However, the `nlp.config` should include the original config with variables intact – otherwise, loading a pipeline and saving it to disk will destroy all logic implemented via variables and hard-code the values all over the place. This means that when we create the components, we need to keep two versions of the config: the interpolated config with the "real" values and the `raw_config` including the variable references.
-
-#### Factory registry
-
-Component factories are special and use the `@Language.factory` or `@Language.component` decorator to register themselves and their meta. When the decorator runs, it performs some basic validation, stores the meta information for the factory on the `Language` class (default config, scores etc.) and then adds the factory function to `registry.factories`. The `component` decorator can be used for registering simple functions that just take a `Doc` object and return it so in that case, we create the factory for the user automatically.
-
-There's one important detail to note about how factories are registered via entry points: A package that wants to expose spaCy components still needs to register them via the `@Language` decorators so we have the component meta information and can perform required checks. All we care about here is that the decorated function is **loaded and imported**. When it is, the `@Language` decorator takes care of everything, including actually registering the component factory.
-
-Normally, adding to the registry via an entry point will just add the function to the registry under the given name. But for `spacy_factories`, we don't actually want that: all we care about is that the function decorated with `@Language` is imported so the decorator runs. So we only exploit Python's entry point system to automatically import the function, and the `spacy_factories` entry point group actually adds to a **separate registry**, `registry._factories`, under the hood. Its only purpose is that the functions are imported. The decorator then runs, creates the factory if needed and adds it to the `registry.factories` registry.
-
-#### Language-specific factories
-
-spaCy supports registering factories on the `Language` base class, as well as language-specific subclasses like `English` or `German`. This allows providing different factories depending on the language, e.g. a different default lemmatizer. The `Language.get_factory_name` classmethod constructs the factory name as `{lang}.{name}` if a language is available (i.e. if it's a subclass) and falls back to `{name}` otherwise. So `@German.factory("foo")` will add a factory `de.foo` under the hood. If you add `nlp.add_pipe("foo")`, we first check if there's a factory for `{nlp.lang}.foo` and if not, we fall back to checking for a factory `foo`.
-
-#### Creating a pipeline component from a factory
-
-`Language.add_pipe` takes care of adding a pipeline component, given its factory name, its config. If no source pipeline to copy the component from is provided, it delegates to `Language.create_pipe`, which sets up the actual component function.
-
-- Validate the config and make sure that the factory was registered via the decorator and that we have meta for it.
-- Update the component config with any defaults specified by the component's `default_config`, if available. This is done by merging the values we receive into the defaults. It ensures that you can still add a component without having to specify its _entire_ config including more complex settings like `model`. If no `model` is defined, we use the default.
-- Check if we have a language-specific factory for the given `nlp.lang` and if not, fall back to the global factory.
-- Construct the component config, consisting of whatever arguments were provided, plus the current `nlp` object and `name`, which are default expected arguments of all factories. We also add a reference to the `@factories` registry, so we can resolve the config via the registry, like any other config. With the added `nlp` and `name`, it should now include all expected arguments of the given function.
-- Fill the config to make sure all unspecified defaults from the function arguments are added and update the `raw_config` (uninterpolated with variables intact) with that information, so the component config we store in `nlp.config` is up to date. We do this by adding the `raw_config` _into_ the filled config – otherwise, the references to variables would be overwritten.
-- Resolve the config and create all functions it refers to (e.g. `model`). This gives us the actual component function that we can insert into the pipeline.
-
-### 1C. Sourcing a pipeline component
-
-```ini
-[components.ner]
-source = "en_core_web_sm"
-```
-
-spaCy also allows ["sourcing" a component](https://spacy.io/usage/processing-pipelines#sourced-components), which will copy it over from an existing pipeline. In this case, `Language.add_pipe` will delegate to `Language.create_pipe_from_source`. In order to copy a component effectively and validate it, the source pipeline first needs to be loaded. This is done in `Language.from_config`, so a source pipeline only has to be loaded once if multiple components source from it. Sourcing a component will perform the following checks and modifications:
-
-- For each sourced pipeline component loaded in `Language.from_config`, a hash of the vectors data from the source pipeline is stored in the pipeline meta so we're able to check whether the vectors match and warn if not (since different vectors that are used as features in components can lead to degraded performance). Because the vectors are not loaded at the point when components are sourced, the check is postponed to `init_vocab` as part of `Language.initialize`.
-- If the sourced pipeline component is loaded through `Language.add_pipe(source=)`, the vectors are already loaded and can be compared directly. The check compares the shape and keys first and finally falls back to comparing the actual byte representation of the vectors (which is slower).
-- Ensure that the component is available in the pipeline.
-- Interpolate the entire config of the source pipeline so all variables are replaced and the component's config that's copied over doesn't include references to variables that are not available in the destination config.
-- Add the source `vocab.strings` to the destination's `vocab.strings` so we don't end up with unavailable strings in the final pipeline (which would also include labels used by the sourced component).
-
-Note that there may be other incompatibilities that we're currently not checking for and that could cause a sourced component to not work in the destination pipeline. We're interested in adding more checks here but there'll always be a small number of edge cases we'll never be able to catch, including a sourced component depending on other pipeline state that's not available in the destination pipeline.
-
-### 1D. Tracking components as they're modified
-
-The `Language` class implements methods for removing, replacing or renaming pipeline components. Whenever we make these changes, we need to update the information stored on the `Language` object to ensure that it matches the current state of the pipeline. If a user just writes to `nlp.config` manually, we obviously can't ensure that the config matches the reality – but since we offer modification via the pipe methods, it's expected that spaCy keeps the config in sync under the hood. Otherwise, saving a modified pipeline to disk and loading it back wouldn't work. The internal attributes we need to keep in sync here are:
-
-| Attribute                | Type                         | Description                                                                                                                                                     |
-| ------------------------ | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `Language._components`   | `List[Tuple[str, Callable]]` | All pipeline components as `(name, func)` tuples. This is used as the source of truth for `Language.pipeline`, `Language.pipe_names` and `Language.components`. |
-| `Language._pipe_meta`    | `Dict[str, FactoryMeta]`     | The meta information of a component's factory, keyed by component name. This can include multiple components referring to the same factory meta.                |
-| `Language._pipe_configs` | `Dict[str, Config]`          | The component's config, keyed by component name.                                                                                                                |
-| `Language._disabled`     | `Set[str]`                   | Names of components that are currently disabled.                                                                                                                |
-| `Language._config`       | `Config`                     | The underlying config. This is only internals and will be used as the basis for constructing the config in the `Language.config` property.                      |
-
-In addition to the actual component settings in `[components]`, the config also allows specifying component-specific arguments via the `[initialize.components]` block, which are passed to the component's `initialize` method during initialization if it's available. So we also need to keep this in sync in the underlying config.
-
-### 1E. spaCy's config utility functions
-
-When working with configs in spaCy, make sure to use the utility functions provided by spaCy if available, instead of calling the respective `Config` methods. The utilities take care of providing spaCy-specific error messages and ensure a consistent order of config sections by setting the `section_order` argument. This ensures that exported configs always have the same consistent format.
-
-- `util.load_config`: load a config from a file
-- `util.load_config_from_str`: load a confirm from a string representation
-- `util.copy_config`: deepcopy a config
-
-## 2. Initialization
-
-Initialization is a separate step of the [config lifecycle](https://spacy.io/usage/training#config-lifecycle) that's not performed at runtime. It's implemented via the `training.initialize.init_nlp` helper and calls into `Language.initialize` method, which sets up the pipeline and component models before training. The `initialize` method takes a callback that returns a sample of examples, which is used to initialize the component models, add all required labels and perform shape inference if applicable.
-
-Components can also define custom initialization setting via the `[initialize.components]` block, e.g. if they require external data like lookup tables to be loaded in. All config settings defined here will be passed to the component's `initialize` method, if it implements one. Components are expected to handle their own serialization after they're initialized so that any data or settings they require are saved with the pipeline and will be available from disk when the pipeline is loaded back at runtime.
-
-### 2A. Initialization for training
-
-The `init_nlp` function is called before training and returns an initialized `nlp` object that can be updated with the examples. It only needs the config and does the following:
-
-- Load and validate the config. In order to validate certain settings like the `seed`, we also interpolate the config to get the final value (because in theory, a user could provide this via a variable).
-- Set up the GPU allocation, if required.
-- Create the `nlp` object from the raw, uninterpolated config, which delegates to `Language.from_config`. Since this method may modify and auto-fill the config and pipeline component settings, we then use the interpolated version of `nlp.config` going forward, to ensure that what we're training with is up to date.
-- Resolve the `[training]` block of the config and perform validation, e.g. to check that the corpora are available.
-- Determine the components that should be frozen (not updated during training) or resumed (sourced components from a different pipeline that should be updated from the examples and not reset and re-initialized). To resume training, we can call the `nlp.resume_training` method.
-- Initialize the `nlp` object via `nlp.initialize` and pass it a `get_examples` callback that returns the training corpus (used for shape inference, setting up labels etc.). If the training corpus is streamed, we only provide a small sample of the data, which can potentially be infinite. `nlp.initialize` will delegate to the components as well and pass the data sample forward.
-- Check the listeners and warn about components dependencies, e.g. if a frozen component listens to a component that is retrained, or vice versa (which can degrade results).
-
-### 2B. Initializing the `nlp` object
-
-The `Language.initialize` method does the following:
-
-- **Resolve the config** defined in the `[initialize]` block separately (since everything else is already available in the loaded `nlp` object), based on the fully interpolated config.
-- **Execute callbacks**, i.e. `before_init` and `after_init`, if they're defined.
-- **Initialize the vocab**, including vocab data, lookup tables and vectors.
-- **Initialize the tokenizer** if it implements an `initialize` method. This is not the case for the default tokenizers, but it allows custom tokenizers to depend on external data resources that are loaded in on initialization.
-- **Initialize all pipeline components** if they implement an `initialize` method and pass them the `get_examples` callback, the current `nlp` object as well as well additional initialization config settings provided in the component-specific block.
-- **Initialize pretraining** if a `[pretraining]` block is available in the config. This allows loading pretrained tok2vec weights in `spacy pretrain`.
-- **Register listeners** if token-to-vector embedding layers of a component model "listen" to a previous component (`tok2vec`, `transformer`) in the pipeline.
-- **Create an optimizer** on the `Language` class, either by adding the optimizer passed as `sgd` to `initialize`, or by creating the optimizer defined in the config's training settings.
-
-### 2C. Initializing the vocab
-
-Vocab initialization is handled in the `training.initialize.init_vocab` helper. It takes the relevant loaded functions and values from the config and takes care of the following:
-
-- Add lookup tables defined in the config initialization, e.g. custom lemmatization tables. Those will be added to `nlp.vocab.lookups` from where they can be accessed by components.
-- Add JSONL-formatted [vocabulary data](https://spacy.io/api/data-formats#vocab-jsonl) to pre-populate the lexical attributes.
-- Load vectors into the pipeline. Vectors are defined as a name or path to a saved `nlp` object containing the vectors, e.g. `en_vectors_web_lg`. It's loaded and the vectors are ported over, while ensuring that all source strings are available in the destination strings. We also warn if there's a mismatch between sourced vectors, since this can lead to problems.

extra/DEVELOPER_DOCS/Listeners.md

@@ -1,235 +0,0 @@
-# Listeners
-
-- [1. Overview](#1-overview)
-- [2. Initialization](#2-initialization)
-  - [2A. Linking listeners to the embedding component](#2a-linking-listeners-to-the-embedding-component)
-  - [2B. Shape inference](#2b-shape-inference)
-- [3. Internal communication](#3-internal-communication)
-  - [3A. During prediction](#3a-during-prediction)
-  - [3B. During training](#3b-during-training)
-    - [Training with multiple listeners](#training-with-multiple-listeners)
-  - [3C. Frozen components](#3c-frozen-components)
-    - [The Tok2Vec or Transformer is frozen](#the-tok2vec-or-transformer-is-frozen)
-    - [The upstream component is frozen](#the-upstream-component-is-frozen)
-- [4. Replacing listener with standalone](#4-replacing-listener-with-standalone)
-
-## 1. Overview
-
-Trainable spaCy components typically use some sort of `tok2vec` layer as part of the `model` definition.
-This `tok2vec` layer produces embeddings and is either a standard `Tok2Vec` layer, or a Transformer-based one.
-Both versions can be used either inline/standalone, which means that they are defined and used
-by only one specific component (e.g. NER), or
-[shared](https://spacy.io/usage/embeddings-transformers#embedding-layers),
-in which case the embedding functionality becomes a separate component that can
-feed embeddings to multiple components downstream, using a listener-pattern.
-
-| Type          | Usage      | Model Architecture                                                                                 |
-| ------------- | ---------- | -------------------------------------------------------------------------------------------------- |
-| `Tok2Vec`     | standalone | [`spacy.Tok2Vec`](https://spacy.io/api/architectures#Tok2Vec)                                      |
-| `Tok2Vec`     | listener   | [`spacy.Tok2VecListener`](https://spacy.io/api/architectures#Tok2VecListener)                      |
-| `Transformer` | standalone | [`spacy-transformers.Tok2VecTransformer`](https://spacy.io/api/architectures#Tok2VecTransformer)   |
-| `Transformer` | listener   | [`spacy-transformers.TransformerListener`](https://spacy.io/api/architectures#TransformerListener) |
-
-Here we discuss the listener pattern and its implementation in code in more detail.
-
-## 2. Initialization
-
-### 2A. Linking listeners to the embedding component
-
-To allow sharing a `tok2vec` layer, a separate `tok2vec` component needs to be defined in the config:
-
-```
-[components.tok2vec]
-factory = "tok2vec"
-
-[components.tok2vec.model]
-@architectures = "spacy.Tok2Vec.v2"
-```
-
-A listener can then be set up by making sure the correct `upstream` name is defined, referring to the
-name of the `tok2vec` component (which equals the factory name by default), or `*` as a wildcard:
-
-```
-[components.ner.model.tok2vec]
-@architectures = "spacy.Tok2VecListener.v1"
-upstream = "tok2vec"
-```
-
-When an [`nlp`](https://github.com/explosion/spaCy/blob/master/extra/DEVELOPER_DOCS/Language.md) object is
-initialized or deserialized, it will make sure to link each `tok2vec` component to its listeners. This is
-implemented in the method `nlp._link_components()` which loops over each
-component in the pipeline and calls `find_listeners()` on a component if it's defined.
-The [`tok2vec` component](https://github.com/explosion/spaCy/blob/master/spacy/pipeline/tok2vec.py)'s implementation
-of this `find_listener()` method will specifically identify sublayers of a model definition that are of type
-`Tok2VecListener` with a matching upstream name and will then add that listener to the internal `self.listener_map`.
-
-If it's a Transformer-based pipeline, a
-[`transformer` component](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/pipeline_component.py)
-has a similar implementation but its `find_listener()` function will specifically look for `TransformerListener`
-sublayers of downstream components.
-
-### 2B. Shape inference
-
-Typically, the output dimension `nO` of a listener's model equals the `nO` (or `width`) of the upstream embedding layer.
-For a standard `Tok2Vec`-based component, this is typically known up-front and defined as such in the config:
-
-```
-[components.ner.model.tok2vec]
-@architectures = "spacy.Tok2VecListener.v1"
-width = ${components.tok2vec.model.encode.width}
-```
-
-A `transformer` component however only knows its `nO` dimension after the HuggingFace transformer
-is set with the function `model.attrs["set_transformer"]`,
-[implemented](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/layers/transformer_model.py)
-by `set_pytorch_transformer`.
-This is why, upon linking of the transformer listeners, the `transformer` component also makes sure to set
-the listener's output dimension correctly.
-
-This shape inference mechanism also needs to happen with resumed/frozen components, which means that for some CLI
-commands (`assemble` and `train`), we need to call `nlp._link_components` even before initializing the `nlp`
-object. To cover all use-cases and avoid negative side effects, the code base ensures that performing the
-linking twice is not harmful.
-
-## 3. Internal communication
-
-The internal communication between a listener and its downstream components is organized by sending and
-receiving information across the components - either directly or implicitly.
-The details are different depending on whether the pipeline is currently training, or predicting.
-Either way, the `tok2vec` or `transformer` component always needs to run before the listener.
-
-### 3A. During prediction
-
-When the `Tok2Vec` pipeline component is called, its `predict()` method is executed to produce the results,
-which are then stored by `set_annotations()` in the `doc.tensor` field of the document(s).
-Similarly, the `Transformer` component stores the produced embeddings
-in `doc._.trf_data`. Next, the `forward` pass of a
-[`Tok2VecListener`](https://github.com/explosion/spaCy/blob/master/spacy/pipeline/tok2vec.py)
-or a
-[`TransformerListener`](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/layers/listener.py)
-accesses these fields on the `Doc` directly. Both listener implementations have a fallback mechanism for when these
-properties were not set on the `Doc`: in that case an all-zero tensor is produced and returned.
-We need this fallback mechanism to enable shape inference methods in Thinc, but the code
-is slightly risky and at times might hide another bug - so it's a good spot to be aware of.
-
-### 3B. During training
-
-During training, the `update()` methods of the `Tok2Vec` & `Transformer` components don't necessarily set the
-annotations on the `Doc` (though since 3.1 they can if they are part of the `annotating_components` list in the config).
-Instead, we rely on a caching mechanism between the original embedding component and its listener.
-Specifically, the produced embeddings are sent to the listeners by calling `listener.receive()` and uniquely
-identifying the batch of documents with a `batch_id`. This `receive()` call also sends the appropriate `backprop`
-call to ensure that gradients from the downstream component flow back to the trainable `Tok2Vec` or `Transformer`
-network.
-
-We rely on the `nlp` object properly batching the data and sending each batch through the pipeline in sequence,
-which means that only one such batch needs to be kept in memory for each listener.
-When the downstream component runs and the listener should produce embeddings, it accesses the batch in memory,
-runs the backpropagation, and returns the results and the gradients.
-
-There are two ways in which this mechanism can fail, both are detected by `verify_inputs()`:
-
-- `E953` if a different batch is in memory than the requested one - signaling some kind of out-of-sync state of the
-  training pipeline.
-- `E954` if no batch is in memory at all - signaling that the pipeline is probably not set up correctly.
-
-#### Training with multiple listeners
-
-One `Tok2Vec` or `Transformer` component may be listened to by several downstream components, e.g.
-a tagger and a parser could be sharing the same embeddings. In this case, we need to be careful about how we do
-the backpropagation. When the `Tok2Vec` or `Transformer` sends out data to the listener with `receive()`, they will
-send an `accumulate_gradient` function call to all listeners, except the last one. This function will keep track
-of the gradients received so far. Only the final listener in the pipeline will get an actual `backprop` call that
-will initiate the backpropagation of the `tok2vec` or `transformer` model with the accumulated gradients.
-
-### 3C. Frozen components
-
-The listener pattern can get particularly tricky in combination with frozen components. To detect components
-with listeners that are not frozen consistently, `init_nlp()` (which is called by `spacy train`) goes through
-the listeners and their upstream components and warns in two scenarios.
-
-#### The Tok2Vec or Transformer is frozen
-
-If the `Tok2Vec` or `Transformer` was already trained,
-e.g. by [pretraining](https://spacy.io/usage/embeddings-transformers#pretraining),
-it could be a valid use-case to freeze the embedding architecture and only train downstream components such
-as a tagger or a parser. This used to be impossible before 3.1, but has become supported since then by putting the
-embedding component in the [`annotating_components`](https://spacy.io/usage/training#annotating-components)
-list of the config. This works like any other "annotating component" because it relies on the `Doc` attributes.
-
-However, if the `Tok2Vec` or `Transformer` is frozen, and not present in `annotating_components`, and a related
-listener isn't frozen, then a `W086` warning is shown and further training of the pipeline will likely end with `E954`.
-
-#### The upstream component is frozen
-
-If an upstream component is frozen but the underlying `Tok2Vec` or `Transformer` isn't, the performance of
-the upstream component will be degraded after training. In this case, a `W087` warning is shown, explaining
-how to use the `replace_listeners` functionality to prevent this problem.
-
-## 4. Replacing listener with standalone
-
-The [`replace_listeners`](https://spacy.io/api/language#replace_listeners) functionality changes the architecture
-of a downstream component from using a listener pattern to a standalone `tok2vec` or `transformer` layer,
-effectively making the downstream component independent of any other components in the pipeline.
-It is implemented by `nlp.replace_listeners()` and typically executed by `nlp.from_config()`.
-First, it fetches the original `Model` of the original component that creates the embeddings:
-
-```
-tok2vec = self.get_pipe(tok2vec_name)
-tok2vec_model = tok2vec.model
-```
-
-Which is either a [`Tok2Vec` model](https://github.com/explosion/spaCy/blob/master/spacy/ml/models/tok2vec.py) or a
-[`TransformerModel`](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/layers/transformer_model.py).
-
-In the case of the `tok2vec`, this model can be copied as-is into the configuration and architecture of the
-downstream component. However, for the `transformer`, this doesn't work.
-The reason is that the `TransformerListener` architecture chains the listener with
-[`trfs2arrays`](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/layers/trfs2arrays.py):
-
-```
-model = chain(
-    TransformerListener(upstream_name=upstream)
-    trfs2arrays(pooling, grad_factor),
-)
-```
-
-but the standalone `Tok2VecTransformer` has an additional `split_trf_batch` chained inbetween the model
-and `trfs2arrays`:
-
-```
-model = chain(
-    TransformerModel(name, get_spans, tokenizer_config),
-    split_trf_batch(),
-    trfs2arrays(pooling, grad_factor),
-)
-```
-
-So you can't just take the model from the listener, and drop that into the component internally. You need to
-adjust the model and the config. To facilitate this, `nlp.replace_listeners()` will check whether additional
-[functions](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/layers/_util.py) are
-[defined](https://github.com/explosion/spacy-transformers/blob/master/spacy_transformers/layers/transformer_model.py)
-in `model.attrs`, and if so, it will essentially call these to make the appropriate changes:
-
-```
-replace_func = tok2vec_model.attrs["replace_listener_cfg"]
-new_config = replace_func(tok2vec_cfg["model"], pipe_cfg["model"]["tok2vec"])
-...
-new_model = tok2vec_model.attrs["replace_listener"](new_model)
-```
-
-The new config and model are then properly stored on the `nlp` object.
-Note that this functionality (running the replacement for a transformer listener) was broken prior to
-`spacy-transformers` 1.0.5.
-
-In spaCy 3.7, `Language.replace_listeners` was updated to pass the following additional arguments to the `replace_listener` callback:
-the listener to be replaced and the `tok2vec`/`transformer` pipe from which the new model was copied. To maintain backwards-compatiblity,
-the method only passes these extra arguments for callbacks that support them:
-
-```
-def replace_listener_pre_37(copied_tok2vec_model):
-  ...
-
-def replace_listener_post_37(copied_tok2vec_model, replaced_listener, tok2vec_pipe):
-  ...
-```

extra/DEVELOPER_DOCS/README.md

@@ -1,7 +0,0 @@
-<a href="https://explosion.ai"><img src="https://explosion.ai/assets/img/logo.svg" width="125" height="125" align="right" /></a>
-
-# Developer Documentation
-
-This directory includes additional documentation and explanations of spaCy's internals. It's mostly intended for the spaCy core development team and contributors interested in the more complex parts of the library. The documents generally focus on more abstract implementation details and how specific methods and algorithms work, and they assume knowledge of what's already available in the [usage documentation](https://spacy.io/usage) and [API reference](https://spacy.io/api).
-
-If you're looking to contribute to spaCy, make sure to check out the documentation and [contributing guide](https://github.com/explosion/spaCy/blob/master/CONTRIBUTING.md) first.

extra/DEVELOPER_DOCS/Satellite Packages.md

@@ -1,82 +0,0 @@
-# spaCy Satellite Packages
-
-This is a list of all the active repos relevant to spaCy besides the main one, with short descriptions, history, and current status. Archived repos will not be covered.
-
-## Always Included in spaCy
-
-These packages are always pulled in when you install spaCy. Most of them are direct dependencies, but some are transitive dependencies through other packages.
-
-- [spacy-legacy](https://github.com/explosion/spacy-legacy): When an architecture in spaCy changes enough to get a new version, the old version is frozen and moved to spacy-legacy. This allows us to keep the core library slim while also preserving backwards compatability.
-- [thinc](https://github.com/explosion/thinc): Thinc is the machine learning library that powers trainable components in spaCy. It wraps backends like Numpy, PyTorch, and Tensorflow to provide a functional interface for specifying architectures.
-- [catalogue](https://github.com/explosion/catalogue): Small library for adding function registries, like those used for model architectures in spaCy.
-- [confection](https://github.com/explosion/confection): This library contains the functionality for config parsing that was formerly contained directly in Thinc.
-- [spacy-loggers](https://github.com/explosion/spacy-loggers): Contains loggers beyond the default logger available in spaCy's core code base. This includes loggers integrated with third-party services, which may differ in release cadence from spaCy itself.
-- [wasabi](https://github.com/explosion/wasabi): A command line formatting library, used for terminal output in spaCy.
-- [srsly](https://github.com/explosion/srsly): A wrapper that vendors several serialization libraries for spaCy. Includes parsers for JSON, JSONL, MessagePack, (extended) Pickle, and YAML.
-- [preshed](https://github.com/explosion/preshed): A Cython library for low-level data structures like hash maps, used for memory efficient data storage.
-- [cython-blis](https://github.com/explosion/cython-blis): Fast matrix multiplication using BLIS without depending on system libraries. Required by Thinc, rather than spaCy directly.
-- [murmurhash](https://github.com/explosion/murmurhash): A wrapper library for a C++ murmurhash implementation, used for string IDs in spaCy and preshed.
-- [cymem](https://github.com/explosion/cymem): A small library for RAII-style memory management in Cython. 
-
-## Optional Extensions for spaCy
-
-These are repos that can be used by spaCy but aren't part of a default installation. Many of these are wrappers to integrate various kinds of third-party libraries.
-
-- [spacy-transformers](https://github.com/explosion/spacy-transformers): A wrapper for the [HuggingFace Transformers](https://huggingface.co/docs/transformers/index) library, this handles the extensive conversion necessary to coordinate spaCy's powerful `Doc` representation, training pipeline, and the Transformer embeddings. When released, this was known as `spacy-pytorch-transformers`, but it changed to the current name when HuggingFace update the name of their library as well.
-- [spacy-huggingface-hub](https://github.com/explosion/spacy-huggingface-hub): This package has a CLI script for uploading a packaged spaCy pipeline (created with `spacy package`) to the [Hugging Face Hub](https://huggingface.co/models).
-- [spacy-alignments](https://github.com/explosion/spacy-alignments): A wrapper for the tokenizations library (mentioned below) with a modified build system to simplify cross-platform wheel creation. Used in spacy-transformers for aligning spaCy and HuggingFace tokenizations.
-- [spacy-experimental](https://github.com/explosion/spacy-experimental): Experimental components that are not quite ready for inclusion in the main spaCy library. Usually there are unresolved questions around their APIs, so the experimental library allows us to expose them to the community for feedback before fully integrating them. 
-- [spacy-lookups-data](https://github.com/explosion/spacy-lookups-data): A repository of linguistic data, such as lemmas, that takes up a lot of disk space. Originally created to reduce the size of the spaCy core library. This is mainly useful if you want the data included but aren't using a pretrained pipeline; for the affected languages, the relevant data is included in pretrained pipelines directly.
-- [coreferee](https://github.com/explosion/coreferee): Coreference resolution for English, French, German and Polish, optimised for limited training data and easily extensible for further languages. Used as a spaCy pipeline component.
-- [spacy-stanza](https://github.com/explosion/spacy-stanza): This is a wrapper that allows the use of Stanford's Stanza library in spaCy. 
-- [spacy-streamlit](https://github.com/explosion/spacy-streamlit): A wrapper for the Streamlit dashboard building library to help with integrating [displaCy](https://spacy.io/api/top-level/#displacy).
-- [spacymoji](https://github.com/explosion/spacymoji): A library to add extra support for emoji to spaCy, such as including character names.
-- [thinc-apple-ops](https://github.com/explosion/thinc-apple-ops): A special backend for OSX that uses Apple's native libraries for improved performance.
-- [os-signpost](https://github.com/explosion/os-signpost): A Python package that allows you to use the `OSSignposter` API in OSX for performance analysis.
-- [spacy-ray](https://github.com/explosion/spacy-ray): A wrapper to integrate spaCy with Ray, a distributed training framework. Currently a work in progress.
-
-## Prodigy
-
-[Prodigy](https://prodi.gy) is Explosion's easy to use and highly customizable tool for annotating data. Prodigy itself requires a license, but the repos below contain documentation, examples, and editor or notebook integrations.
-
-- [prodigy-recipes](https://github.com/explosion/prodigy-recipes): Sample recipes for Prodigy, along with notebooks and other examples of usage.
-- [vscode-prodigy](https://github.com/explosion/vscode-prodigy): A VS Code extension that lets you run Prodigy inside VS Code.
-- [jupyterlab-prodigy](https://github.com/explosion/jupyterlab-prodigy): An extension for JupyterLab that lets you run Prodigy inside JupyterLab.
-
-## Independent Tools or Projects
-
-These are tools that may be related to or use spaCy, but are functional independent projects in their own right as well.
-
-- [floret](https://github.com/explosion/floret): A modification of fastText to use Bloom Embeddings. Can be used to add vectors with subword features to spaCy, and also works independently in the same manner as fastText.
-- [sense2vec](https://github.com/explosion/sense2vec): A library to make embeddings of noun phrases or words coupled with their part of speech. This library uses spaCy.
-- [spacy-vectors-builder](https://github.com/explosion/spacy-vectors-builder): This is a spaCy project that builds vectors using floret and a lot of input text. It handles downloading the input data as well as the actual building of vectors.
-- [holmes-extractor](https://github.com/explosion/holmes-extractor): Information extraction from English and German texts based on predicate logic. Uses spaCy.
-- [healthsea](https://github.com/explosion/healthsea): Healthsea is a project to extract information from comments about health supplements. Structurally, it's a self-contained, large spaCy project.
-- [spacy-pkuseg](https://github.com/explosion/spacy-pkuseg): A fork of the pkuseg Chinese tokenizer. Used for Chinese support in spaCy, but also works independently.
-- [ml-datasets](https://github.com/explosion/ml-datasets): This repo includes loaders for several standard machine learning datasets, like MNIST or WikiNER, and has historically been used in spaCy example code and documentation.
-
-## Documentation and Informational Repos
-
-These repos are used to support the spaCy docs or otherwise present information about spaCy or other Explosion projects.
-
-- [projects](https://github.com/explosion/projects): The projects repo is used to show detailed examples of spaCy usage. Individual projects can be checked out using the spaCy command line tool, rather than checking out the projects repo directly.
-- [spacy-course](https://github.com/explosion/spacy-course): Home to the interactive spaCy course for learning about how to use the library and some basic NLP principles.
-- [spacy-io-binder](https://github.com/explosion/spacy-io-binder): Home to the notebooks used for interactive examples in the documentation.
-
-## Organizational / Meta
-
-These repos are used for organizing data around spaCy, but are not something an end user would need to install as part of using the library.
-
-- [spacy-models](https://github.com/explosion/spacy-models): This repo contains metadata (but not training data) for all the spaCy models. This includes information about where their training data came from, version compatability, and performance information. It also includes tests for the model packages, and the built models are hosted as releases of this repo.
-- [wheelwright](https://github.com/explosion/wheelwright): A tool for automating our PyPI builds and releases.
-- [ec2buildwheel](https://github.com/explosion/ec2buildwheel): A small project that allows you to build Python packages in the manner of cibuildwheel, but on any EC2 image. Used by wheelwright.
-
-## Other
-
-Repos that don't fit in any of the above categories.
-
-- [blis](https://github.com/explosion/blis): A fork of the official BLIS library. The main branch is not updated, but work continues in various branches. This is used for cython-blis.
-- [tokenizations](https://github.com/explosion/tokenizations): A library originally by Yohei Tamura to align strings with tolerance to some variations in features like case and diacritics, used for aligning tokens and wordpieces. Adopted and maintained by Explosion, but usually spacy-alignments is used instead.
-- [conll-2012](https://github.com/explosion/conll-2012): A repo to hold some slightly cleaned up versions of the official scripts for the CoNLL 2012 shared task involving coreference resolution. Used in the coref project.
-- [fastapi-explosion-extras](https://github.com/explosion/fastapi-explosion-extras): Some small tweaks to FastAPI used at Explosion.
-

extra/DEVELOPER_DOCS/StringStore-Vocab.md

@@ -1,216 +0,0 @@
-# StringStore & Vocab
-
-> Reference: `spacy/strings.pyx`
-> Reference: `spacy/vocab.pyx`
-
-## Overview
-
-spaCy represents mosts strings internally using a `uint64` in Cython which
-corresponds to a hash. The magic required to make this largely transparent is
-handled by the `StringStore`, and is integrated into the pipelines using the
-`Vocab`, which also connects it to some other information. 
-
-These are mostly internal details that average library users should never have
-to think about. On the other hand, when developing a component it's normal to
-interact with the Vocab for lexeme data or word vectors, and it's not unusual
-to add labels to the `StringStore`.
-
-## StringStore
-
-### Overview
-
-The `StringStore` is a `cdef class` that looks a bit like a two-way dictionary,
-though it is not a subclass of anything in particular.
-
-The main functionality of the `StringStore` is that `__getitem__` converts
-hashes into strings or strings into hashes.
-
-The full details of the conversion are complicated. Normally you shouldn't have
-to worry about them, but the first applicable case here is used to get the
-return value:
-
-1. 0 and the empty string are special cased to each other
-2. internal symbols use a lookup table (`SYMBOLS_BY_STR`)
-3. normal strings or bytes are hashed
-4. internal symbol IDs in `SYMBOLS_BY_INT` are handled
-5. anything not yet handled is used as a hash to lookup a string
-
-For the symbol enums, see [`symbols.pxd`](https://github.com/explosion/spaCy/blob/master/spacy/symbols.pxd).
-
-Almost all strings in spaCy are stored in the `StringStore`. This naturally
-includes tokens, but also includes things like labels (not just NER/POS/dep,
-but also categories etc.), lemmas, lowercase forms, word shapes, and so on. One
-of the main results of this is that tokens can be represented by a compact C
-struct ([`LexemeC`](https://spacy.io/api/cython-structs#lexemec)/[`TokenC`](https://github.com/explosion/spaCy/issues/4854)) that mostly consists of string hashes. This also means that converting
-input for the models is straightforward, and there's not a token mapping step
-like in many machine learning frameworks. Additionally, because the token IDs
-in spaCy are based on hashes, they are consistent across environments or
-models.
-
-One pattern you'll see a lot in spaCy APIs is that `something.value` returns an
-`int` and `something.value_` returns a string. That's implemented using the
-`StringStore`. Typically the `int` is stored in a C struct and the string is
-generated via a property that calls into the `StringStore` with the `int`.
-
-Besides `__getitem__`, the `StringStore` has functions to return specifically a
-string or specifically a hash, regardless of whether the input was a string or
-hash to begin with, though these are only used occasionally.
-
-### Implementation Details: Hashes and Allocations
-
-Hashes are 64-bit and are computed using [murmurhash][] on UTF-8 bytes. There is no
-mechanism for detecting and avoiding collisions. To date there has never been a
-reproducible collision or user report about any related issues.
-
-[murmurhash]: https://github.com/explosion/murmurhash
-
-The empty string is not hashed, it's just converted to/from 0. 
-
-A small number of strings use indices into a lookup table (so low integers)
-rather than hashes. This is mostly Universal Dependencies labels or other
-strings considered "core" in spaCy. This was critical in v1, which hadn't
-introduced hashing yet. Since v2 it's important for items in `spacy.attrs`,
-especially lexeme flags, but is otherwise only maintained for backwards
-compatibility.
-
-You can call `strings["mystring"]` with a string the `StringStore` has never seen
-before and it will return a hash. But in order to do the reverse operation, you
-need to call `strings.add("mystring")` first. Without a call to `add` the
-string will not be interned.
-
-Example:
-
-```
-from spacy.strings import StringStore
-
-ss = StringStore()
-hashval = ss["spacy"] # 10639093010105930009
-try:
-    # this won't work
-    ss[hashval]
-except KeyError:
-    print(f"key {hashval} unknown in the StringStore.")
-
-ss.add("spacy")
-assert ss[hashval] == "spacy" # it works now
-
-# There is no `.keys` property, but you can iterate over keys
-# The empty string will never be in the list of keys
-for key in ss:
-    print(key)
-```
-
-In normal use nothing is ever removed from the `StringStore`. In theory this
-means that if you do something like iterate through all hex values of a certain
-length you can have explosive memory usage. In practice this has never been an
-issue. (Note that this is also different from using `sys.intern` to intern
-Python strings, which does not guarantee they won't be garbage collected later.)
-
-Strings are stored in the `StringStore` in a peculiar way: each string uses a
-union that is either an eight-byte `char[]` or a `char*`. Short strings are
-stored directly in the `char[]`, while longer strings are stored in allocated
-memory and prefixed with their length. This is a strategy to reduce indirection
-and memory fragmentation. See  `decode_Utf8Str` and `_allocate` in
-`strings.pyx` for the implementation.
-
-### When to Use the StringStore?
-
-While you can ignore the `StringStore` in many cases, there are situations where
-you should make use of it to avoid errors. 
-
-Any time you introduce a string that may be set on a `Doc` field that has a hash,
-you should add the string to the `StringStore`. This mainly happens when adding
-labels in components, but there are some other cases:
-
-- syntax iterators, mainly `get_noun_chunks`
-- external data used in components, like the `KnowledgeBase` in the `entity_linker`
-- labels used in tests
-
-## Vocab
-
-The `Vocab` is a core component of a `Language` pipeline. Its main function is
-to manage `Lexeme`s, which are structs that contain information about a token
-that depends only on its surface form, without context. `Lexeme`s store much of
-the data associated with `Token`s. As a side effect of this the `Vocab` also
-manages the `StringStore` for a pipeline and a grab-bag of other data.
-
-These are things stored in the vocab:
-
-- `Lexeme`s
-- `StringStore`
-- `Morphology`: manages info used in `MorphAnalysis` objects
-- `vectors`: basically a dict for word vectors
-- `lookups`: language specific data like lemmas
-- `writing_system`: language specific metadata
-- `get_noun_chunks`: a syntax iterator
-- lex attribute getters: functions like `is_punct`, set in language defaults
-- `cfg`: **not** the pipeline config, this is mostly unused
-- `_unused_object`: Formerly an unused object, kept around until v4 for compatability
-
-Some of these, like the Morphology and Vectors, are complex enough that they
-need their own explanations. Here we'll just look at Vocab-specific items.
-
-### Lexemes
-
-A `Lexeme` is a type that mainly wraps a `LexemeC`, a struct consisting of ints
-that identify various context-free token attributes. Lexemes are the core data
-of the `Vocab`, and can be accessed using `__getitem__` on the `Vocab`. The memory
-for storing `LexemeC` objects is managed by a pool that belongs to the `Vocab`.
-
-Note that `__getitem__` on the `Vocab` works much like the `StringStore`, in
-that it accepts a hash or id, with one important difference: if you do a lookup
-using a string, that value is added to the `StringStore` automatically. 
-
-The attributes stored in a `LexemeC` are:
-
-- orth (the raw text)
-- lower
-- norm
-- shape
-- prefix
-- suffix
-
-Most of these are straightforward. All of them can be customized, and (except
-`orth`) probably should be since the defaults are based on English, but in
-practice this is rarely done at present.
-
-### Lookups
-
-This is basically a dict of dicts, implemented using a `Table` for each
-sub-dict, that stores lemmas and other language-specific lookup data. 
-
-A `Table` is a subclass of `OrderedDict` used for string-to-string data. It uses
-Bloom filters to speed up misses and has some extra serialization features.
-Tables are not used outside of the lookups.
-
-### Lex Attribute Getters
-
-Lexical Attribute Getters like `is_punct` are defined on a per-language basis,
-much like lookups, but take the form of functions rather than string-to-string
-dicts, so they're stored separately.
-
-### Writing System
-
-This is a dict with three attributes:
-
-- `direction`: ltr or rtl (default ltr)
-- `has_case`: bool (default `True`)
-- `has_letters`: bool (default `True`, `False` only for CJK for now)
-
-Currently these are not used much - the main use is that `direction` is used in
-visualizers, though `rtl` doesn't quite work (see
-[#4854](https://github.com/explosion/spaCy/issues/4854)). In the future they
-could be used when choosing hyperparameters for subwords, controlling word
-shape generation, and similar tasks.
-
-### Other Vocab Members
-
-The Vocab is kind of the default place to store things from `Language.defaults`
-that don't belong to the Tokenizer. The following properties are in the Vocab
-just because they don't have anywhere else to go.
-
-- `get_noun_chunks`
-- `cfg`: This is a dict that just stores `oov_prob` (hardcoded to `-20`)
-- `_unused_object`: Leftover C member, should be removed in next major version
-
-

extra/example_data/ner_example_data/README.md

@@ -1,25 +1,7 @@
 ## Examples of NER/IOB data that can be converted with `spacy convert`
 
-To convert an IOB file to `.spacy` ([`DocBin`](https://spacy.io/api/docbin))
+spacy JSON training files were generated with:
-for spaCy v3:
 
-```bash
-python -m spacy convert -c iob -s -n 10 -b en_core_web_sm file.iob .
 ```
-
-See all the `spacy convert` options: https://spacy.io/api/cli#convert
-
----
-
-The spaCy v2 JSON training files were generated using **spaCy v2** with:
-
-```bash
 python -m spacy convert -c iob -s -n 10 -b en file.iob
 ```
-
-To convert an existing JSON training file to `.spacy` for spaCy v3, convert
-with **spaCy v3**:
-
-```bash
-python -m spacy convert file.json .
-```

licenses/3rd_party_licenses.txt

@@ -43,8 +43,8 @@ scikit-learn
 
 * Files: scorer.py
 
-The implementation of roc_auc_score() is adapted from scikit-learn, which is
+The following implementation of roc_auc_score() is adapted from
-distributed under the following license:
+scikit-learn, which is distributed under the following license:
 
 New BSD License
 
@@ -77,126 +77,3 @@ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
 DAMAGE.
-
-
-pyvi
-----
-
-* Files: lang/vi/__init__.py
-
-The MIT License (MIT)
-Copyright (c) 2016 Viet-Trung Tran
-
-Permission is hereby granted, free of charge, to any person obtaining a copy of
-this software and associated documentation files (the "Software"), to deal in
-the Software without restriction, including without limitation the rights to
-use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
-of the Software, and to permit persons to whom the Software is furnished to do
-so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
-
-importlib_metadata
-------------------
-
-* Files: util.py
-
-The implementation of packages_distributions() is adapted from
-importlib_metadata, which is distributed under the following license:
-
-Copyright 2017-2019 Jason R. Coombs, Barry Warsaw
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-
-
-polyleven
----------
-
-* Files: spacy/matcher/polyleven.c
-
-MIT License
-
-Copyright (c) 2021 Fujimoto Seiji <fujimoto@ceptord.net>
-Copyright (c) 2021 Max Bachmann <kontakt@maxbachmann.de>
-Copyright (c) 2022 Nick Mazuk
-Copyright (c) 2022 Michael Weiss <code@mweiss.ch>
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
-
-SciPy
------
-
-* Files: scorer.py
-
-The implementation of trapezoid() is adapted from SciPy, which is distributed
-under the following license:
-
-New BSD License
-
-Copyright (c) 2001-2002 Enthought, Inc. 2003-2023, SciPy Developers.
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions
-are met:
-
-1. Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-
-2. Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following
-   disclaimer in the documentation and/or other materials provided
-   with the distribution.
-
-3. Neither the name of the copyright holder nor the names of its
-   contributors may be used to endorse or promote products derived
-   from this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

pyproject.toml

@@ -1,67 +1,13 @@
 [build-system]
 requires = [
     "setuptools",
-    "cython>=3.0,<4.0",
+    "cython>=0.25",
     "cymem>=2.0.2,<2.1.0",
     "preshed>=3.0.2,<3.1.0",
     "murmurhash>=0.28.0,<1.1.0",
-    "thinc>=8.3.4,<8.4.0",
+    "thinc>=8.0.2,<8.1.0",
-    "numpy>=2.0.0,<3.0.0"
+    "blis>=0.4.0,<0.8.0",
+    "pathy",
+    "numpy>=1.15.0",
 ]
 build-backend = "setuptools.build_meta"
-
-[tool.cibuildwheel]
-build = "*"
-skip = "pp* cp36* cp37* cp38* *-win32 *i686*"
-test-skip = ""
-free-threaded-support = false
-
-archs = ["native"]
-
-build-frontend = "default"
-config-settings = {}
-dependency-versions = "pinned"
-environment = { PIP_CONSTRAINT = "build-constraints.txt" }
-
-environment-pass = []
-build-verbosity = 0
-
-before-all = "curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain stable"
-before-build = "pip install -r requirements.txt && python setup.py clean"
-repair-wheel-command = ""
-
-test-command = ""
-before-test = ""
-test-requires = []
-test-extras = []
-
-container-engine = "docker"
-
-manylinux-x86_64-image = "manylinux2014"
-manylinux-i686-image = "manylinux2014"
-manylinux-aarch64-image = "manylinux2014"
-manylinux-ppc64le-image = "manylinux2014"
-manylinux-s390x-image = "manylinux2014"
-manylinux-pypy_x86_64-image = "manylinux2014"
-manylinux-pypy_i686-image = "manylinux2014"
-manylinux-pypy_aarch64-image = "manylinux2014"
-
-musllinux-x86_64-image = "musllinux_1_2"
-musllinux-i686-image = "musllinux_1_2"
-musllinux-aarch64-image = "musllinux_1_2"
-musllinux-ppc64le-image = "musllinux_1_2"
-musllinux-s390x-image = "musllinux_1_2"
-
-[tool.cibuildwheel.linux]
-repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel}"
-
-[tool.cibuildwheel.macos]
-repair-wheel-command = "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"
-
-[tool.cibuildwheel.windows]
-
-[tool.cibuildwheel.pyodide]
-
-
-[tool.isort]
-profile = "black"

requirements.txt

@@ -1,38 +1,31 @@
 # Our libraries
-spacy-legacy>=3.0.11,<3.1.0
+spacy-legacy>=3.0.0,<3.1.0
-spacy-loggers>=1.0.0,<2.0.0
 cymem>=2.0.2,<2.1.0
 preshed>=3.0.2,<3.1.0
-thinc>=8.3.4,<8.4.0
+thinc>=8.0.2,<8.1.0
+blis>=0.4.0,<0.8.0
 ml_datasets>=0.2.0,<0.3.0
 murmurhash>=0.28.0,<1.1.0
-wasabi>=0.9.1,<1.2.0
+wasabi>=0.8.1,<1.1.0
-srsly>=2.4.3,<3.0.0
+srsly>=2.4.0,<3.0.0
-catalogue>=2.0.6,<2.1.0
+catalogue>=2.0.1,<2.1.0
-typer-slim>=0.3.0,<1.0.0
+typer>=0.3.0,<0.4.0
-weasel>=0.1.0,<0.5.0
+pathy>=0.3.5
 # Third party dependencies
-numpy>=2.0.0,<3.0.0
+numpy>=1.15.0
 requests>=2.13.0,<3.0.0
 tqdm>=4.38.0,<5.0.0
-pydantic>=1.7.4,!=1.8,!=1.8.1,<3.0.0
+pydantic>=1.7.1,<1.8.0
 jinja2
 # Official Python utilities
 setuptools
 packaging>=20.0
+importlib_metadata>=0.20; python_version < "3.8"
+typing_extensions>=3.7.4.1,<4.0.0.0; python_version < "3.8"
 # Development dependencies
-pre-commit>=2.13.0
+cython>=0.25
-cython>=3.0,<4.0
+pytest>=5.2.0
-pytest>=5.2.0,!=7.1.0
 pytest-timeout>=1.3.0,<2.0.0
 mock>=2.0.0,<3.0.0
-flake8>=3.8.0,<6.0.0
+flake8>=3.5.0,<3.6.0
 hypothesis>=3.27.0,<7.0.0
-mypy>=1.5.0,<1.6.0; platform_machine != "aarch64" and python_version >= "3.8"
-types-mock>=0.1.1
-types-setuptools>=57.0.0
-types-requests
-types-setuptools>=57.0.0
-black==22.3.0
-cython-lint>=0.15.0
-isort>=5.0,<6.0

setup.cfg

@@ -17,110 +17,86 @@ classifiers =
     Operating System :: Microsoft :: Windows
     Programming Language :: Cython
     Programming Language :: Python :: 3
+    Programming Language :: Python :: 3.6
+    Programming Language :: Python :: 3.7
+    Programming Language :: Python :: 3.8
     Programming Language :: Python :: 3.9
-    Programming Language :: Python :: 3.10
-    Programming Language :: Python :: 3.11
-    Programming Language :: Python :: 3.12
-    Programming Language :: Python :: 3.13
     Topic :: Scientific/Engineering
-project_urls =
-    Release notes = https://github.com/explosion/spaCy/releases
-    Source = https://github.com/explosion/spaCy
 
 [options]
 zip_safe = false
 include_package_data = true
-python_requires = >=3.9,<3.14
+python_requires = >=3.6
-# NOTE: This section is superseded by pyproject.toml and will be removed in
-# spaCy v4
 setup_requires =
-    cython>=3.0,<4.0
+    cython>=0.25
-    numpy>=2.0.0,<3.0.0; python_version < "3.9"
+    numpy>=1.15.0
-    numpy>=2.0.0,<3.0.0; python_version >= "3.9"
     # We also need our Cython packages here to compile against
     cymem>=2.0.2,<2.1.0
     preshed>=3.0.2,<3.1.0
     murmurhash>=0.28.0,<1.1.0
-    thinc>=8.3.4,<8.4.0
+    thinc>=8.0.2,<8.1.0
 install_requires =
     # Our libraries
-    spacy-legacy>=3.0.11,<3.1.0
+    spacy-legacy>=3.0.0,<3.1.0
-    spacy-loggers>=1.0.0,<2.0.0
     murmurhash>=0.28.0,<1.1.0
     cymem>=2.0.2,<2.1.0
     preshed>=3.0.2,<3.1.0
-    thinc>=8.3.4,<8.4.0
+    thinc>=8.0.2,<8.1.0
-    wasabi>=0.9.1,<1.2.0
+    blis>=0.4.0,<0.8.0
-    srsly>=2.4.3,<3.0.0
+    wasabi>=0.8.1,<1.1.0
-    catalogue>=2.0.6,<2.1.0
+    srsly>=2.4.0,<3.0.0
-    weasel>=0.1.0,<0.5.0
+    catalogue>=2.0.1,<2.1.0
+    typer>=0.3.0,<0.4.0
+    pathy>=0.3.5
     # Third-party dependencies
-    typer-slim>=0.3.0,<1.0.0
     tqdm>=4.38.0,<5.0.0
-    numpy>=1.15.0; python_version < "3.9"
+    numpy>=1.15.0
-    numpy>=1.19.0; python_version >= "3.9"
     requests>=2.13.0,<3.0.0
-    pydantic>=1.7.4,!=1.8,!=1.8.1,<3.0.0
+    pydantic>=1.7.1,<1.8.0
     jinja2
     # Official Python utilities
     setuptools
     packaging>=20.0
+    importlib_metadata>=0.20; python_version < "3.8"
+    typing_extensions>=3.7.4,<4.0.0.0; python_version < "3.8"
 
 [options.entry_points]
 console_scripts =
-    spacy = spacy.cli:setup_cli
+    spacy = spacy.cli:app
 
 [options.extras_require]
 lookups =
-    spacy_lookups_data>=1.0.3,<1.1.0
+    spacy_lookups_data>=1.0.0,<1.1.0
 transformers =
-    spacy_transformers>=1.1.2,<1.4.0
+    spacy_transformers>=1.0.1,<1.1.0
+ray =
+    spacy_ray>=0.1.0,<1.0.0
 cuda =
-    cupy>=5.0.0b4,<13.0.0
+    cupy>=5.0.0b4,<9.0.0
 cuda80 =
-    cupy-cuda80>=5.0.0b4,<13.0.0
+    cupy-cuda80>=5.0.0b4,<9.0.0
 cuda90 =
-    cupy-cuda90>=5.0.0b4,<13.0.0
+    cupy-cuda90>=5.0.0b4,<9.0.0
 cuda91 =
-    cupy-cuda91>=5.0.0b4,<13.0.0
+    cupy-cuda91>=5.0.0b4,<9.0.0
 cuda92 =
-    cupy-cuda92>=5.0.0b4,<13.0.0
+    cupy-cuda92>=5.0.0b4,<9.0.0
 cuda100 =
-    cupy-cuda100>=5.0.0b4,<13.0.0
+    cupy-cuda100>=5.0.0b4,<9.0.0
 cuda101 =
-    cupy-cuda101>=5.0.0b4,<13.0.0
+    cupy-cuda101>=5.0.0b4,<9.0.0
 cuda102 =
-    cupy-cuda102>=5.0.0b4,<13.0.0
+    cupy-cuda102>=5.0.0b4,<9.0.0
 cuda110 =
-    cupy-cuda110>=5.0.0b4,<13.0.0
+    cupy-cuda110>=5.0.0b4,<9.0.0
 cuda111 =
-    cupy-cuda111>=5.0.0b4,<13.0.0
+    cupy-cuda111>=5.0.0b4,<9.0.0
-cuda112 =
-    cupy-cuda112>=5.0.0b4,<13.0.0
-cuda113 =
-    cupy-cuda113>=5.0.0b4,<13.0.0
-cuda114 =
-    cupy-cuda114>=5.0.0b4,<13.0.0
-cuda115 =
-    cupy-cuda115>=5.0.0b4,<13.0.0
-cuda116 =
-    cupy-cuda116>=5.0.0b4,<13.0.0
-cuda117 =
-    cupy-cuda117>=5.0.0b4,<13.0.0
-cuda11x =
-    cupy-cuda11x>=11.0.0,<13.0.0
-cuda12x =
-    cupy-cuda12x>=11.5.0,<13.0.0
-cuda-autodetect =
-    cupy-wheel>=11.0.0,<13.0.0
-apple =
-    thinc-apple-ops>=1.0.0,<2.0.0
 # Language tokenizers with external dependencies
 ja =
-    sudachipy>=0.5.2,!=0.6.1
+    sudachipy>=0.4.9
-    sudachidict_core>=20211220
+    sudachidict_core>=20200330
 ko =
-    natto-py>=0.9.0
+    natto-py==0.9.0
 th =
     pythainlp>=2.0
 
@@ -131,7 +107,7 @@ universal = false
 formats = gztar
 
 [flake8]
-ignore = E203, E266, E501, E731, W503, E741, F541
+ignore = E203, E266, E501, E731, W503, E741
 max-line-length = 80
 select = B,C,E,F,W,T4,B9
 exclude =
@@ -142,11 +118,9 @@ exclude =
 
 [tool:pytest]
 markers =
-    slow: mark a test as slow
+    slow
-    issue: reference specific issue
 
 [mypy]
 ignore_missing_imports = True
 no_implicit_optional = True
 plugins = pydantic.mypy, thinc.mypy
-allow_redefinition = True

setup.py

@@ -1,9 +1,10 @@
 #!/usr/bin/env python
 from setuptools import Extension, setup, find_packages
 import sys
+import platform
 import numpy
-from setuptools.command.build_ext import build_ext
+from distutils.command.build_ext import build_ext
-from sysconfig import get_path
+from distutils.sysconfig import get_python_inc
 from pathlib import Path
 import shutil
 from Cython.Build import cythonize
@@ -22,20 +23,16 @@ Options.docstrings = True
 
 PACKAGES = find_packages()
 MOD_NAMES = [
-    "spacy.training.alignment_array",
     "spacy.training.example",
     "spacy.parts_of_speech",
     "spacy.strings",
     "spacy.lexeme",
     "spacy.vocab",
     "spacy.attrs",
-    "spacy.kb.candidate",
+    "spacy.kb",
-    "spacy.kb.kb",
-    "spacy.kb.kb_in_memory",
     "spacy.ml.parser_model",
     "spacy.morphology",
     "spacy.pipeline.dep_parser",
-    "spacy.pipeline._edit_tree_internals.edit_trees",
     "spacy.pipeline.morphologizer",
     "spacy.pipeline.multitask",
     "spacy.pipeline.ner",
@@ -78,7 +75,6 @@ COMPILER_DIRECTIVES = {
     "language_level": -3,
     "embedsignature": True,
     "annotation_typing": False,
-    "profile": sys.version_info < (3, 12),
 }
 # Files to copy into the package that are otherwise not included
 COPY_FILES = {
@@ -88,6 +84,30 @@ COPY_FILES = {
 }
 
 
+def is_new_osx():
+    """Check whether we're on OSX >= 10.7"""
+    if sys.platform != "darwin":
+        return False
+    mac_ver = platform.mac_ver()[0]
+    if mac_ver.startswith("10"):
+        minor_version = int(mac_ver.split(".")[1])
+        if minor_version >= 7:
+            return True
+        else:
+            return False
+    return False
+
+
+if is_new_osx():
+    # On Mac, use libc++ because Apple deprecated use of
+    # libstdc
+    COMPILE_OPTIONS["other"].append("-stdlib=libc++")
+    LINK_OPTIONS["other"].append("-lc++")
+    # g++ (used by unix compiler on mac) links to libstdc++ as a default lib.
+    # See: https://stackoverflow.com/questions/1653047/avoid-linking-to-libstdc
+    LINK_OPTIONS["other"].append("-nodefaultlibs")
+
+
 # By subclassing build_extensions we have the actual compiler that will be used which is really known only after finalize_options
 # http://stackoverflow.com/questions/724664/python-distutils-how-to-get-a-compiler-that-is-going-to-be-used
 class build_ext_options:
@@ -104,8 +124,6 @@ class build_ext_options:
 
 class build_ext_subclass(build_ext, build_ext_options):
     def build_extensions(self):
-        if self.parallel is None and os.environ.get("SPACY_NUM_BUILD_JOBS") is not None:
-            self.parallel = int(os.environ.get("SPACY_NUM_BUILD_JOBS"))
         build_ext_options.build_options(self)
         build_ext.build_extensions(self)
 
@@ -180,28 +198,13 @@ def setup_package():
 
     include_dirs = [
         numpy.get_include(),
-        get_path("include"),
+        get_python_inc(plat_specific=True),
     ]
     ext_modules = []
-    ext_modules.append(
-        Extension(
-            "spacy.matcher.levenshtein",
-            [
-                "spacy/matcher/levenshtein.pyx",
-                "spacy/matcher/polyleven.c",
-            ],
-            language="c",
-            include_dirs=include_dirs,
-        )
-    )
     for name in MOD_NAMES:
         mod_path = name.replace(".", "/") + ".pyx"
         ext = Extension(
-            name,
+            name, [mod_path], language="c++", include_dirs=include_dirs, extra_compile_args=["-std=c++11"]
-            [mod_path],
-            language="c++",
-            include_dirs=include_dirs,
-            extra_compile_args=["-std=c++11"],
         )
         ext_modules.append(ext)
     print("Cythonizing sources")

spacy/__init__.py

@@ -1,25 +1,26 @@
-import sys
+from typing import Union, Iterable, Dict, Any
 from pathlib import Path
-from typing import Any, Dict, Iterable, Union
+import warnings
+import sys
 
-# set library-specific custom warning handling before doing anything else
+warnings.filterwarnings("ignore", message="numpy.dtype size changed")  # noqa
-from .errors import setup_default_warnings
+warnings.filterwarnings("ignore", message="numpy.ufunc size changed")  # noqa
-
-setup_default_warnings()  # noqa: E402
 
 # These are imported as part of the API
-from thinc.api import Config, prefer_gpu, require_cpu, require_gpu  # noqa: F401
+from thinc.api import prefer_gpu, require_gpu, require_cpu  # noqa: F401
+from thinc.api import Config
 
 from . import pipeline  # noqa: F401
-from . import util
-from .about import __version__  # noqa: F401
 from .cli.info import info  # noqa: F401
-from .errors import Errors
 from .glossary import explain  # noqa: F401
+from .about import __version__  # noqa: F401
+from .util import registry, logger  # noqa: F401
+
+from .errors import Errors
 from .language import Language
-from .registrations import REGISTRY_POPULATED, populate_registry
-from .util import logger, registry  # noqa: F401
 from .vocab import Vocab
+from . import util
+
 
 if sys.maxunicode == 65535:
     raise SystemError(Errors.E130)
@@ -27,36 +28,23 @@ if sys.maxunicode == 65535:
 
 def load(
     name: Union[str, Path],
-    *,
+    disable: Iterable[str] = util.SimpleFrozenList(),
-    vocab: Union[Vocab, bool] = True,
+    exclude: Iterable[str] = util.SimpleFrozenList(),
-    disable: Union[str, Iterable[str]] = util._DEFAULT_EMPTY_PIPES,
-    enable: Union[str, Iterable[str]] = util._DEFAULT_EMPTY_PIPES,
-    exclude: Union[str, Iterable[str]] = util._DEFAULT_EMPTY_PIPES,
     config: Union[Dict[str, Any], Config] = util.SimpleFrozenDict(),
 ) -> Language:
     """Load a spaCy model from an installed package or a local path.
 
     name (str): Package name or model path.
-    vocab (Vocab): A Vocab object. If True, a vocab is created.
+    disable (Iterable[str]): Names of pipeline components to disable. Disabled
-    disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled
         pipes will be loaded but they won't be run unless you explicitly
         enable them by calling nlp.enable_pipe.
-    enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
+    exclude (Iterable[str]): Names of pipeline components to exclude. Excluded
-        pipes will be disabled (but can be enabled later using nlp.enable_pipe).
-    exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded
         components won't be loaded.
     config (Dict[str, Any] / Config): Config overrides as nested dict or dict
         keyed by section values in dot notation.
     RETURNS (Language): The loaded nlp object.
     """
-    return util.load_model(
+    return util.load_model(name, disable=disable, exclude=exclude, config=config)
-        name,
-        vocab=vocab,
-        disable=disable,
-        enable=enable,
-        exclude=exclude,
-        config=config,
-    )
 
 
 def blank(
@@ -64,7 +52,7 @@ def blank(
     *,
     vocab: Union[Vocab, bool] = True,
     config: Union[Dict[str, Any], Config] = util.SimpleFrozenDict(),
-    meta: Dict[str, Any] = util.SimpleFrozenDict(),
+    meta: Dict[str, Any] = util.SimpleFrozenDict()
 ) -> Language:
     """Create a blank nlp object for a given language code.
 

spacy/about.py

@@ -1,5 +1,7 @@
 # fmt: off
 __title__ = "spacy"
-__version__ = "3.8.7"
+__version__ = "3.0.5"
 __download_url__ = "https://github.com/explosion/spacy-models/releases/download"
 __compatibility__ = "https://raw.githubusercontent.com/explosion/spacy-models/master/compatibility.json"
+__projects__ = "https://github.com/explosion/projects"
+__projects_branch__ = "v3"

spacy/attrs.pxd

@@ -1,7 +1,6 @@
 # Reserve 64 values for flag features
 from . cimport symbols
 
-
 cdef enum attr_id_t:
     NULL_ATTR
     IS_ALPHA

spacy/attrs.pyx

@@ -1,7 +1,3 @@
-# cython: profile=False
-from .errors import Errors
-
-IOB_STRINGS = ("", "I", "O", "B")
 
 IDS = {
     "": NULL_ATTR,
@@ -68,6 +64,7 @@ IDS = {
     "FLAG61": FLAG61,
     "FLAG62": FLAG62,
     "FLAG63": FLAG63,
+
     "ID": ID,
     "ORTH": ORTH,
     "LOWER": LOWER,
@@ -75,7 +72,9 @@ IDS = {
     "SHAPE": SHAPE,
     "PREFIX": PREFIX,
     "SUFFIX": SUFFIX,
+
     "LENGTH": LENGTH,
+    "CLUSTER": CLUSTER,
     "LEMMA": LEMMA,
     "POS": POS,
     "TAG": TAG,
@@ -86,10 +85,12 @@ IDS = {
     "ENT_KB_ID": ENT_KB_ID,
     "HEAD": HEAD,
     "SENT_START": SENT_START,
+    "SENT_END": SENT_END,
     "SPACY": SPACY,
+    "PROB": PROB,
     "LANG": LANG,
     "MORPH": MORPH,
-    "IDX": IDX,
+    "IDX": IDX
 }
 
 
@@ -111,66 +112,28 @@ def intify_attrs(stringy_attrs, strings_map=None, _do_deprecated=False):
     """
     inty_attrs = {}
     if _do_deprecated:
-        if "F" in stringy_attrs:
+        if 'F' in stringy_attrs:
             stringy_attrs["ORTH"] = stringy_attrs.pop("F")
-        if "L" in stringy_attrs:
+        if 'L' in stringy_attrs:
             stringy_attrs["LEMMA"] = stringy_attrs.pop("L")
-        if "pos" in stringy_attrs:
+        if 'pos' in stringy_attrs:
             stringy_attrs["TAG"] = stringy_attrs.pop("pos")
-        if "morph" in stringy_attrs:
+        if 'morph' in stringy_attrs:
-            morphs = stringy_attrs.pop("morph")  # no-cython-lint
+            morphs = stringy_attrs.pop('morph')
-        if "number" in stringy_attrs:
+        if 'number' in stringy_attrs:
-            stringy_attrs.pop("number")
+            stringy_attrs.pop('number')
-        if "tenspect" in stringy_attrs:
+        if 'tenspect' in stringy_attrs:
-            stringy_attrs.pop("tenspect")
+            stringy_attrs.pop('tenspect')
         morph_keys = [
-            "PunctType",
+            'PunctType', 'PunctSide', 'Other', 'Degree', 'AdvType', 'Number',
-            "PunctSide",
+            'VerbForm', 'PronType', 'Aspect', 'Tense', 'PartType', 'Poss',
-            "Other",
+            'Hyph', 'ConjType', 'NumType', 'Foreign', 'VerbType', 'NounType',
-            "Degree",
+            'Gender', 'Mood', 'Negative', 'Tense', 'Voice', 'Abbr',
-            "AdvType",
+            'Derivation', 'Echo', 'Foreign', 'NameType', 'NounType', 'NumForm',
-            "Number",
+            'NumValue', 'PartType', 'Polite', 'StyleVariant',
-            "VerbForm",
+            'PronType', 'AdjType', 'Person', 'Variant', 'AdpType',
-            "PronType",
+            'Reflex', 'Negative', 'Mood', 'Aspect', 'Case',
-            "Aspect",
+            'Polarity', 'PrepCase', 'Animacy' # U20
-            "Tense",
-            "PartType",
-            "Poss",
-            "Hyph",
-            "ConjType",
-            "NumType",
-            "Foreign",
-            "VerbType",
-            "NounType",
-            "Gender",
-            "Mood",
-            "Negative",
-            "Tense",
-            "Voice",
-            "Abbr",
-            "Derivation",
-            "Echo",
-            "Foreign",
-            "NameType",
-            "NounType",
-            "NumForm",
-            "NumValue",
-            "PartType",
-            "Polite",
-            "StyleVariant",
-            "PronType",
-            "AdjType",
-            "Person",
-            "Variant",
-            "AdpType",
-            "Reflex",
-            "Negative",
-            "Mood",
-            "Aspect",
-            "Case",
-            "Polarity",
-            "PrepCase",
-            "Animacy",  # U20
         ]
         for key in morph_keys:
             if key in stringy_attrs:
@@ -182,13 +145,8 @@ def intify_attrs(stringy_attrs, strings_map=None, _do_deprecated=False):
     for name, value in stringy_attrs.items():
         int_key = intify_attr(name)
         if int_key is not None:
-            if int_key == ENT_IOB:
+            if strings_map is not None and isinstance(value, basestring):
-                if value in IOB_STRINGS:
+                if hasattr(strings_map, 'add'):
-                    value = IOB_STRINGS.index(value)
-                elif isinstance(value, str):
-                    raise ValueError(Errors.E1025.format(value=value))
-            if strings_map is not None and isinstance(value, str):
-                if hasattr(strings_map, "add"):
                     value = strings_map.add(value)
                 else:
                     value = strings_map[value]

spacy/cli/__init__.py

@@ -1,47 +1,37 @@
 from wasabi import msg
 
-# Needed for testing
-from . import download as download_module  # noqa: F401
 from ._util import app, setup_cli  # noqa: F401
-from .apply import apply  # noqa: F401
-from .assemble import assemble_cli  # noqa: F401
 
 # These are the actual functions, NOT the wrapped CLI commands. The CLI commands
 # are registered automatically and won't have to be imported here.
-from .benchmark_speed import benchmark_speed_cli  # noqa: F401
-from .convert import convert  # noqa: F401
-from .debug_config import debug_config  # noqa: F401
-from .debug_data import debug_data  # noqa: F401
-from .debug_diff import debug_diff  # noqa: F401
-from .debug_model import debug_model  # noqa: F401
 from .download import download  # noqa: F401
-from .evaluate import evaluate  # noqa: F401
-from .find_function import find_function  # noqa: F401
-from .find_threshold import find_threshold  # noqa: F401
 from .info import info  # noqa: F401
-from .init_config import fill_config, init_config  # noqa: F401
-from .init_pipeline import init_pipeline_cli  # noqa: F401
 from .package import package  # noqa: F401
-from .pretrain import pretrain  # noqa: F401
 from .profile import profile  # noqa: F401
-from .project.assets import project_assets  # type: ignore[attr-defined]  # noqa: F401
+from .train import train_cli  # noqa: F401
-from .project.clone import project_clone  # type: ignore[attr-defined]  # noqa: F401
+from .pretrain import pretrain  # noqa: F401
-from .project.document import (  # type: ignore[attr-defined]  # noqa: F401
+from .debug_data import debug_data  # noqa: F401
-    project_document,
+from .debug_config import debug_config  # noqa: F401
-)
+from .debug_model import debug_model  # noqa: F401
-from .project.dvc import project_update_dvc  # type: ignore[attr-defined]  # noqa: F401
+from .evaluate import evaluate  # noqa: F401
-from .project.pull import project_pull  # type: ignore[attr-defined]  # noqa: F401
+from .convert import convert  # noqa: F401
-from .project.push import project_push  # type: ignore[attr-defined]  # noqa: F401
+from .init_pipeline import init_pipeline_cli  # noqa: F401
-from .project.run import project_run  # type: ignore[attr-defined]  # noqa: F401
+from .init_config import init_config, fill_config  # noqa: F401
-from .train import train_cli  # type: ignore[attr-defined]  # noqa: F401
+from .validate import validate  # noqa: F401
-from .validate import validate  # type: ignore[attr-defined]  # noqa: F401
+from .project.clone import project_clone  # noqa: F401
+from .project.assets import project_assets  # noqa: F401
+from .project.run import project_run  # noqa: F401
+from .project.dvc import project_update_dvc  # noqa: F401
+from .project.push import project_push  # noqa: F401
+from .project.pull import project_pull  # noqa: F401
+from .project.document import project_document  # noqa: F401
 
 
 @app.command("link", no_args_is_help=True, deprecated=True, hidden=True)
 def link(*args, **kwargs):
-    """As of spaCy v3.0, symlinks like "en" are not supported anymore. You can load trained
+    """As of spaCy v3.0, symlinks like "en" are deprecated. You can load trained
     pipeline packages using their full names or from a directory path."""
     msg.warn(
-        "As of spaCy v3.0, model symlinks are not supported anymore. You can load trained "
+        "As of spaCy v3.0, model symlinks are deprecated. You can load trained "
         "pipeline packages using their full names or from a directory path."
     )

spacy/cli/_util.py

@@ -1,50 +1,34 @@
-import hashlib
+from typing import Dict, Any, Union, List, Optional, Tuple, Iterable, TYPE_CHECKING
-import os
-import shutil
 import sys
-from configparser import InterpolationError
+import shutil
-from contextlib import contextmanager
 from pathlib import Path
-from typing import (
+from wasabi import msg
-    TYPE_CHECKING,
-    Any,
-    Dict,
-    Iterable,
-    List,
-    Optional,
-    Tuple,
-    Union,
-    overload,
-)
-
 import srsly
+import hashlib
 import typer
 from click import NoSuchOption
 from click.parser import split_arg_string
-from thinc.api import Config, ConfigValidationError, require_gpu
-from thinc.util import gpu_is_available
 from typer.main import get_command
-from wasabi import Printer, msg
+from contextlib import contextmanager
-from weasel import app as project_cli
+from thinc.api import Config, ConfigValidationError, require_gpu
+from thinc.util import has_cupy, gpu_is_available
+from configparser import InterpolationError
+import os
 
+from ..schemas import ProjectConfigSchema, validate
+from ..util import import_file, run_command, make_tempdir, registry, logger
+from ..util import is_compatible_version, SimpleFrozenDict, ENV_VARS
 from .. import about
-from ..compat import Literal
+
-from ..schemas import validate
+if TYPE_CHECKING:
-from ..util import (
+    from pathy import Pathy  # noqa: F401
-    ENV_VARS,
+
-    SimpleFrozenDict,
-    import_file,
-    is_compatible_version,
-    logger,
-    make_tempdir,
-    registry,
-    run_command,
-)
 
 SDIST_SUFFIX = ".tar.gz"
 WHEEL_SUFFIX = "-py3-none-any.whl"
 
 PROJECT_FILE = "project.yml"
+PROJECT_LOCK = "project.lock"
 COMMAND = "python -m spacy"
 NAME = "spacy"
 HELP = """spaCy Command-line Interface
@@ -60,7 +44,6 @@ DEBUG_HELP = """Suite of helpful commands for debugging and profiling. Includes
 commands to check and validate your config files, training and evaluation data,
 and custom model implementations.
 """
-BENCHMARK_HELP = """Commands for benchmarking pipelines."""
 INIT_HELP = """Commands for initializing configs and pipeline packages."""
 
 # Wrappers for Typer's annotations. Initially created to set defaults and to
@@ -69,13 +52,12 @@ Arg = typer.Argument
 Opt = typer.Option
 
 app = typer.Typer(name=NAME, help=HELP)
-benchmark_cli = typer.Typer(name="benchmark", help=BENCHMARK_HELP, no_args_is_help=True)
+project_cli = typer.Typer(name="project", help=PROJECT_HELP, no_args_is_help=True)
 debug_cli = typer.Typer(name="debug", help=DEBUG_HELP, no_args_is_help=True)
 init_cli = typer.Typer(name="init", help=INIT_HELP, no_args_is_help=True)
 
-app.add_typer(project_cli, name="project", help=PROJECT_HELP, no_args_is_help=True)
+app.add_typer(project_cli)
 app.add_typer(debug_cli)
-app.add_typer(benchmark_cli)
 app.add_typer(init_cli)
 
 
@@ -103,9 +85,9 @@ def parse_config_overrides(
     cli_overrides = _parse_overrides(args, is_cli=True)
     if cli_overrides:
         keys = [k for k in cli_overrides if k not in env_overrides]
-        logger.debug("Config overrides from CLI: %s", keys)
+        logger.debug(f"Config overrides from CLI: {keys}")
     if env_overrides:
-        logger.debug("Config overrides from env variables: %s", list(env_overrides))
+        logger.debug(f"Config overrides from env variables: {list(env_overrides)}")
     return {**cli_overrides, **env_overrides}
 
 
@@ -148,6 +130,147 @@ def _parse_override(value: Any) -> Any:
         return str(value)
 
 
+def load_project_config(
+    path: Path, interpolate: bool = True, overrides: Dict[str, Any] = SimpleFrozenDict()
+) -> Dict[str, Any]:
+    """Load the project.yml file from a directory and validate it. Also make
+    sure that all directories defined in the config exist.
+
+    path (Path): The path to the project directory.
+    interpolate (bool): Whether to substitute project variables.
+    overrides (Dict[str, Any]): Optional config overrides.
+    RETURNS (Dict[str, Any]): The loaded project.yml.
+    """
+    config_path = path / PROJECT_FILE
+    if not config_path.exists():
+        msg.fail(f"Can't find {PROJECT_FILE}", config_path, exits=1)
+    invalid_err = f"Invalid {PROJECT_FILE}. Double-check that the YAML is correct."
+    try:
+        config = srsly.read_yaml(config_path)
+    except ValueError as e:
+        msg.fail(invalid_err, e, exits=1)
+    errors = validate(ProjectConfigSchema, config)
+    if errors:
+        msg.fail(invalid_err)
+        print("\n".join(errors))
+        sys.exit(1)
+    validate_project_version(config)
+    validate_project_commands(config)
+    # Make sure directories defined in config exist
+    for subdir in config.get("directories", []):
+        dir_path = path / subdir
+        if not dir_path.exists():
+            dir_path.mkdir(parents=True)
+    if interpolate:
+        err = f"{PROJECT_FILE} validation error"
+        with show_validation_error(title=err, hint_fill=False):
+            config = substitute_project_variables(config, overrides)
+    return config
+
+
+def substitute_project_variables(
+    config: Dict[str, Any],
+    overrides: Dict[str, Any] = SimpleFrozenDict(),
+    key: str = "vars",
+    env_key: str = "env",
+) -> Dict[str, Any]:
+    """Interpolate variables in the project file using the config system.
+
+    config (Dict[str, Any]): The project config.
+    overrides (Dict[str, Any]): Optional config overrides.
+    key (str): Key containing variables in project config.
+    env_key (str): Key containing environment variable mapping in project config.
+    RETURNS (Dict[str, Any]): The interpolated project config.
+    """
+    config.setdefault(key, {})
+    config.setdefault(env_key, {})
+    # Substitute references to env vars with their values
+    for config_var, env_var in config[env_key].items():
+        config[env_key][config_var] = _parse_override(os.environ.get(env_var, ""))
+    # Need to put variables in the top scope again so we can have a top-level
+    # section "project" (otherwise, a list of commands in the top scope wouldn't)
+    # be allowed by Thinc's config system
+    cfg = Config({"project": config, key: config[key], env_key: config[env_key]})
+    cfg = Config().from_str(cfg.to_str(), overrides=overrides)
+    interpolated = cfg.interpolate()
+    return dict(interpolated["project"])
+
+
+def validate_project_version(config: Dict[str, Any]) -> None:
+    """If the project defines a compatible spaCy version range, chec that it's
+    compatible with the current version of spaCy.
+
+    config (Dict[str, Any]): The loaded config.
+    """
+    spacy_version = config.get("spacy_version", None)
+    if spacy_version and not is_compatible_version(about.__version__, spacy_version):
+        err = (
+            f"The {PROJECT_FILE} specifies a spaCy version range ({spacy_version}) "
+            f"that's not compatible with the version of spaCy you're running "
+            f"({about.__version__}). You can edit version requirement in the "
+            f"{PROJECT_FILE} to load it, but the project may not run as expected."
+        )
+        msg.fail(err, exits=1)
+
+
+def validate_project_commands(config: Dict[str, Any]) -> None:
+    """Check that project commands and workflows are valid, don't contain
+    duplicates, don't clash  and only refer to commands that exist.
+
+    config (Dict[str, Any]): The loaded config.
+    """
+    command_names = [cmd["name"] for cmd in config.get("commands", [])]
+    workflows = config.get("workflows", {})
+    duplicates = set([cmd for cmd in command_names if command_names.count(cmd) > 1])
+    if duplicates:
+        err = f"Duplicate commands defined in {PROJECT_FILE}: {', '.join(duplicates)}"
+        msg.fail(err, exits=1)
+    for workflow_name, workflow_steps in workflows.items():
+        if workflow_name in command_names:
+            err = f"Can't use workflow name '{workflow_name}': name already exists as a command"
+            msg.fail(err, exits=1)
+        for step in workflow_steps:
+            if step not in command_names:
+                msg.fail(
+                    f"Unknown command specified in workflow '{workflow_name}': {step}",
+                    f"Workflows can only refer to commands defined in the 'commands' "
+                    f"section of the {PROJECT_FILE}.",
+                    exits=1,
+                )
+
+
+def get_hash(data, exclude: Iterable[str] = tuple()) -> str:
+    """Get the hash for a JSON-serializable object.
+
+    data: The data to hash.
+    exclude (Iterable[str]): Top-level keys to exclude if data is a dict.
+    RETURNS (str): The hash.
+    """
+    if isinstance(data, dict):
+        data = {k: v for k, v in data.items() if k not in exclude}
+    data_str = srsly.json_dumps(data, sort_keys=True).encode("utf8")
+    return hashlib.md5(data_str).hexdigest()
+
+
+def get_checksum(path: Union[Path, str]) -> str:
+    """Get the checksum for a file or directory given its file path. If a
+    directory path is provided, this uses all files in that directory.
+
+    path (Union[Path, str]): The file or directory path.
+    RETURNS (str): The checksum.
+    """
+    path = Path(path)
+    if path.is_file():
+        return hashlib.md5(Path(path).read_bytes()).hexdigest()
+    if path.is_dir():
+        # TODO: this is currently pretty slow
+        dir_checksum = hashlib.md5()
+        for sub_file in sorted(fp for fp in path.rglob("*") if fp.is_file()):
+            dir_checksum.update(sub_file.read_bytes())
+        return dir_checksum.hexdigest()
+    msg.fail(f"Can't get checksum for {path}: not a file or directory", exits=1)
+
+
 @contextmanager
 def show_validation_error(
     file_path: Optional[Union[str, Path]] = None,
@@ -205,33 +328,153 @@ def import_code(code_path: Optional[Union[Path, str]]) -> None:
             msg.fail(f"Couldn't load Python code: {code_path}", e, exits=1)
 
 
+def upload_file(src: Path, dest: Union[str, "Pathy"]) -> None:
+    """Upload a file.
+
+    src (Path): The source path.
+    url (str): The destination URL to upload to.
+    """
+    import smart_open
+
+    dest = str(dest)
+    with smart_open.open(dest, mode="wb") as output_file:
+        with src.open(mode="rb") as input_file:
+            output_file.write(input_file.read())
+
+
+def download_file(src: Union[str, "Pathy"], dest: Path, *, force: bool = False) -> None:
+    """Download a file using smart_open.
+
+    url (str): The URL of the file.
+    dest (Path): The destination path.
+    force (bool): Whether to force download even if file exists.
+        If False, the download will be skipped.
+    """
+    import smart_open
+
+    if dest.exists() and not force:
+        return None
+    src = str(src)
+    with smart_open.open(src, mode="rb", ignore_ext=True) as input_file:
+        with dest.open(mode="wb") as output_file:
+            output_file.write(input_file.read())
+
+
+def ensure_pathy(path):
+    """Temporary helper to prevent importing Pathy globally (which can cause
+    slow and annoying Google Cloud warning)."""
+    from pathy import Pathy  # noqa: F811
+
+    return Pathy(path)
+
+
+def git_checkout(
+    repo: str, subpath: str, dest: Path, *, branch: str = "master", sparse: bool = False
+):
+    git_version = get_git_version()
+    if dest.exists():
+        msg.fail("Destination of checkout must not exist", exits=1)
+    if not dest.parent.exists():
+        msg.fail("Parent of destination of checkout must exist", exits=1)
+    if sparse and git_version >= (2, 22):
+        return git_sparse_checkout(repo, subpath, dest, branch)
+    elif sparse:
+        # Only show warnings if the user explicitly wants sparse checkout but
+        # the Git version doesn't support it
+        err_old = (
+            f"You're running an old version of Git (v{git_version[0]}.{git_version[1]}) "
+            f"that doesn't fully support sparse checkout yet."
+        )
+        err_unk = "You're running an unknown version of Git, so sparse checkout has been disabled."
+        msg.warn(
+            f"{err_unk if git_version == (0, 0) else err_old} "
+            f"This means that more files than necessary may be downloaded "
+            f"temporarily. To only download the files needed, make sure "
+            f"you're using Git v2.22 or above."
+        )
+    with make_tempdir() as tmp_dir:
+        cmd = f"git -C {tmp_dir} clone {repo} . -b {branch}"
+        run_command(cmd, capture=True)
+        # We need Path(name) to make sure we also support subdirectories
+        try:
+            shutil.copytree(str(tmp_dir / Path(subpath)), str(dest))
+        except FileNotFoundError:
+            err = f"Can't clone {subpath}. Make sure the directory exists in the repo (branch '{branch}')"
+            msg.fail(err, repo, exits=1)
+
+
+def git_sparse_checkout(repo, subpath, dest, branch):
+    # We're using Git, partial clone and sparse checkout to
+    # only clone the files we need
+    # This ends up being RIDICULOUS. omg.
+    # So, every tutorial and SO post talks about 'sparse checkout'...But they
+    # go and *clone* the whole repo. Worthless. And cloning part of a repo
+    # turns out to be completely broken. The only way to specify a "path" is..
+    # a path *on the server*? The contents of which, specifies the paths. Wat.
+    # Obviously this is hopelessly broken and insecure, because you can query
+    # arbitrary paths on the server! So nobody enables this.
+    # What we have to do is disable *all* files. We could then just checkout
+    # the path, and it'd "work", but be hopelessly slow...Because it goes and
+    # transfers every missing object one-by-one. So the final piece is that we
+    # need to use some weird git internals to fetch the missings in bulk, and
+    # *that* we can do by path.
+    # We're using Git and sparse checkout to only clone the files we need
+    with make_tempdir() as tmp_dir:
+        # This is the "clone, but don't download anything" part.
+        cmd = (
+            f"git clone {repo} {tmp_dir} --no-checkout --depth 1 "
+            f"-b {branch} --filter=blob:none"
+        )
+        run_command(cmd)
+        # Now we need to find the missing filenames for the subpath we want.
+        # Looking for this 'rev-list' command in the git --help? Hah.
+        cmd = f"git -C {tmp_dir} rev-list --objects --all --missing=print -- {subpath}"
+        ret = run_command(cmd, capture=True)
+        git_repo = _http_to_git(repo)
+        # Now pass those missings into another bit of git internals
+        missings = " ".join([x[1:] for x in ret.stdout.split() if x.startswith("?")])
+        if not missings:
+            err = (
+                f"Could not find any relevant files for '{subpath}'. "
+                f"Did you specify a correct and complete path within repo '{repo}' "
+                f"and branch {branch}?"
+            )
+            msg.fail(err, exits=1)
+        cmd = f"git -C {tmp_dir} fetch-pack {git_repo} {missings}"
+        run_command(cmd, capture=True)
+        # And finally, we can checkout our subpath
+        cmd = f"git -C {tmp_dir} checkout {branch} {subpath}"
+        run_command(cmd, capture=True)
+        # We need Path(name) to make sure we also support subdirectories
+        shutil.move(str(tmp_dir / Path(subpath)), str(dest))
+
+
 def get_git_version(
     error: str = "Could not run 'git'. Make sure it's installed and the executable is available.",
 ) -> Tuple[int, int]:
     """Get the version of git and raise an error if calling 'git --version' fails.
+
     error (str): The error message to show.
     RETURNS (Tuple[int, int]): The version as a (major, minor) tuple. Returns
         (0, 0) if the version couldn't be determined.
     """
-    try:
+    ret = run_command("git --version", capture=True)
-        ret = run_command("git --version", capture=True)
-    except:
-        raise RuntimeError(error)
     stdout = ret.stdout.strip()
     if not stdout or not stdout.startswith("git version"):
-        return 0, 0
+        return (0, 0)
     version = stdout[11:].strip().split(".")
-    return int(version[0]), int(version[1])
+    return (int(version[0]), int(version[1]))
 
 
-@overload
+def _http_to_git(repo: str) -> str:
-def string_to_list(value: str, intify: Literal[False] = ...) -> List[str]:
+    if repo.startswith("http://"):
-    ...
+        repo = repo.replace(r"http://", r"https://")
-
+    if repo.startswith(r"https://"):
-
+        repo = repo.replace("https://", "git@").replace("/", ":", 1)
-@overload
+        if repo.endswith("/"):
-def string_to_list(value: str, intify: Literal[True]) -> List[int]:
+            repo = repo[:-1]
-    ...
+        repo = f"{repo}.git"
+    return repo
 
 
 def string_to_list(value: str, intify: bool = False) -> Union[List[str], List[int]]:
@@ -244,7 +487,7 @@ def string_to_list(value: str, intify: bool = False) -> Union[List[str], List[in
     RETURNS (Union[List[str], List[int]]): A list of strings or ints.
     """
     if not value:
-        return []  # type: ignore[return-value]
+        return []
     if value.startswith("[") and value.endswith("]"):
         value = value[1:-1]
     result = []
@@ -256,57 +499,17 @@ def string_to_list(value: str, intify: bool = False) -> Union[List[str], List[in
             p = p[1:-1]
         p = p.strip()
         if intify:
-            p = int(p)  # type: ignore[assignment]
+            p = int(p)
         result.append(p)
     return result
 
 
-def setup_gpu(use_gpu: int, silent=None) -> None:
+def setup_gpu(use_gpu: int) -> None:
     """Configure the GPU and log info."""
-    if silent is None:
-        local_msg = Printer()
-    else:
-        local_msg = Printer(no_print=silent, pretty=not silent)
     if use_gpu >= 0:
-        local_msg.info(f"Using GPU: {use_gpu}")
+        msg.info(f"Using GPU: {use_gpu}")
         require_gpu(use_gpu)
     else:
-        local_msg.info("Using CPU")
+        msg.info("Using CPU")
-        if gpu_is_available():
+        if has_cupy and gpu_is_available():
-            local_msg.info("To switch to GPU 0, use the option: --gpu-id 0")
+            msg.info("To switch to GPU 0, use the option: --gpu-id 0")
-
-
-def walk_directory(path: Path, suffix: Optional[str] = None) -> List[Path]:
-    """Given a directory and a suffix, recursively find all files matching the suffix.
-    Directories or files with names beginning with a . are ignored, but hidden flags on
-    filesystems are not checked.
-    When provided with a suffix `None`, there is no suffix-based filtering."""
-    if not path.is_dir():
-        return [path]
-    paths = [path]
-    locs = []
-    seen = set()
-    for path in paths:
-        if str(path) in seen:
-            continue
-        seen.add(str(path))
-        if path.parts[-1].startswith("."):
-            continue
-        elif path.is_dir():
-            paths.extend(path.iterdir())
-        elif suffix is not None and not path.parts[-1].endswith(suffix):
-            continue
-        else:
-            locs.append(path)
-    # It's good to sort these, in case the ordering messes up cache.
-    locs.sort()
-    return locs
-
-
-def _format_number(number: Union[int, float], ndigits: int = 2) -> str:
-    """Formats a number (float or int) rounding to `ndigits`, without truncating trailing 0s,
-    as happens with `round(number, ndigits)`"""
-    if isinstance(number, float):
-        return f"{number:.{ndigits}f}"
-    else:
-        return str(number)

spacy/cli/apply.py

@@ -1,142 +0,0 @@
-from itertools import chain
-from pathlib import Path
-from typing import Iterable, List, Optional, Union, cast
-
-import srsly
-import tqdm
-from wasabi import msg
-
-from ..tokens import Doc, DocBin
-from ..util import ensure_path, load_model
-from ..vocab import Vocab
-from ._util import Arg, Opt, app, import_code, setup_gpu, walk_directory
-
-path_help = """Location of the documents to predict on.
-Can be a single file in .spacy format or a .jsonl file.
-Files with other extensions are treated as single plain text documents.
-If a directory is provided it is traversed recursively to grab
-all files to be processed.
-The files can be a mixture of .spacy, .jsonl and text files.
-If .jsonl is provided the specified field is going
-to be grabbed ("text" by default)."""
-
-out_help = "Path to save the resulting .spacy file"
-code_help = (
-    "Path to Python file with additional " "code (registered functions) to be imported"
-)
-gold_help = "Use gold preprocessing provided in the .spacy files"
-force_msg = (
-    "The provided output file already exists. "
-    "To force overwriting the output file, set the --force or -F flag."
-)
-
-
-DocOrStrStream = Union[Iterable[str], Iterable[Doc]]
-
-
-def _stream_docbin(path: Path, vocab: Vocab) -> Iterable[Doc]:
-    """
-    Stream Doc objects from DocBin.
-    """
-    docbin = DocBin().from_disk(path)
-    for doc in docbin.get_docs(vocab):
-        yield doc
-
-
-def _stream_jsonl(path: Path, field: str) -> Iterable[str]:
-    """
-    Stream "text" field from JSONL. If the field "text" is
-    not found it raises error.
-    """
-    for entry in srsly.read_jsonl(path):
-        if field not in entry:
-            msg.fail(f"{path} does not contain the required '{field}' field.", exits=1)
-        else:
-            yield entry[field]
-
-
-def _stream_texts(paths: Iterable[Path]) -> Iterable[str]:
-    """
-    Yields strings from text files in paths.
-    """
-    for path in paths:
-        with open(path, "r") as fin:
-            text = fin.read()
-            yield text
-
-
-@app.command("apply")
-def apply_cli(
-    # fmt: off
-    model: str = Arg(..., help="Model name or path"),
-    data_path: Path = Arg(..., help=path_help, exists=True),
-    output_file: Path = Arg(..., help=out_help, dir_okay=False),
-    code_path: Optional[Path] = Opt(None, "--code", "-c", help=code_help),
-    text_key: str = Opt("text", "--text-key", "-tk", help="Key containing text string for JSONL"),
-    force_overwrite: bool = Opt(False, "--force", "-F", help="Force overwriting the output file"),
-    use_gpu: int = Opt(-1, "--gpu-id", "-g", help="GPU ID or -1 for CPU."),
-    batch_size: int = Opt(1, "--batch-size", "-b", help="Batch size."),
-    n_process: int = Opt(1, "--n-process", "-n", help="number of processors to use.")
-):
-    """
-    Apply a trained pipeline to documents to get predictions.
-    Expects a loadable spaCy pipeline and path to the data, which
-    can be a directory or a file.
-    The data files can be provided in multiple formats:
-        1. .spacy files
-        2. .jsonl files with a specified "field" to read the text from.
-        3. Files with any other extension are assumed to be containing
-           a single document.
-    DOCS: https://spacy.io/api/cli#apply
-    """
-    data_path = ensure_path(data_path)
-    output_file = ensure_path(output_file)
-    code_path = ensure_path(code_path)
-    if output_file.exists() and not force_overwrite:
-        msg.fail(force_msg, exits=1)
-    if not data_path.exists():
-        msg.fail(f"Couldn't find data path: {data_path}", exits=1)
-    import_code(code_path)
-    setup_gpu(use_gpu)
-    apply(data_path, output_file, model, text_key, batch_size, n_process)
-
-
-def apply(
-    data_path: Path,
-    output_file: Path,
-    model: str,
-    json_field: str,
-    batch_size: int,
-    n_process: int,
-):
-    docbin = DocBin(store_user_data=True)
-    paths = walk_directory(data_path)
-    if len(paths) == 0:
-        docbin.to_disk(output_file)
-        msg.warn(
-            "Did not find data to process,"
-            f" {data_path} seems to be an empty directory."
-        )
-        return
-    nlp = load_model(model)
-    msg.good(f"Loaded model {model}")
-    vocab = nlp.vocab
-    streams: List[DocOrStrStream] = []
-    text_files = []
-    for path in paths:
-        if path.suffix == ".spacy":
-            streams.append(_stream_docbin(path, vocab))
-        elif path.suffix == ".jsonl":
-            streams.append(_stream_jsonl(path, json_field))
-        else:
-            text_files.append(path)
-    if len(text_files) > 0:
-        streams.append(_stream_texts(text_files))
-    datagen = cast(DocOrStrStream, chain(*streams))
-    for doc in tqdm.tqdm(
-        nlp.pipe(datagen, batch_size=batch_size, n_process=n_process), disable=None
-    ):
-        docbin.add(doc)
-    if output_file.suffix == "":
-        output_file = output_file.with_suffix(".spacy")
-    docbin.to_disk(output_file)

spacy/cli/assemble.py

@@ -1,65 +0,0 @@
-import logging
-from pathlib import Path
-from typing import Optional
-
-import typer
-from wasabi import msg
-
-from .. import util
-from ..util import get_sourced_components, load_model_from_config
-from ._util import (
-    Arg,
-    Opt,
-    app,
-    import_code,
-    parse_config_overrides,
-    show_validation_error,
-)
-
-
-@app.command(
-    "assemble",
-    context_settings={"allow_extra_args": True, "ignore_unknown_options": True},
-)
-def assemble_cli(
-    # fmt: off
-    ctx: typer.Context,  # This is only used to read additional arguments
-    config_path: Path = Arg(..., help="Path to config file", exists=True, allow_dash=True),
-    output_path: Path = Arg(..., help="Output directory to store assembled pipeline in"),
-    code_path: Optional[Path] = Opt(None, "--code", "-c", help="Path to Python file with additional code (registered functions) to be imported"),
-    verbose: bool = Opt(False, "--verbose", "-V", "-VV", help="Display more information for debugging purposes"),
-    # fmt: on
-):
-    """
-    Assemble a spaCy pipeline from a config file. The config file includes
-    all settings for initializing the pipeline. To override settings in the
-    config, e.g. settings that point to local paths or that you want to
-    experiment with, you can override them as command line options. The
-    --code argument lets you pass in a Python file that can be used to
-    register custom functions that are referenced in the config.
-
-    DOCS: https://spacy.io/api/cli#assemble
-    """
-    if verbose:
-        util.logger.setLevel(logging.DEBUG)
-    # Make sure all files and paths exists if they are needed
-    if not config_path or (str(config_path) != "-" and not config_path.exists()):
-        msg.fail("Config file not found", config_path, exits=1)
-    overrides = parse_config_overrides(ctx.args)
-    import_code(code_path)
-    with show_validation_error(config_path):
-        config = util.load_config(config_path, overrides=overrides, interpolate=False)
-    msg.divider("Initializing pipeline")
-    nlp = load_model_from_config(config, auto_fill=True)
-    config = config.interpolate()
-    sourced = get_sourced_components(config)
-    # Make sure that listeners are defined before initializing further
-    nlp._link_components()
-    with nlp.select_pipes(disable=[*sourced]):
-        nlp.initialize()
-    msg.good("Initialized pipeline")
-    msg.divider("Serializing to disk")
-    if output_path is not None and not output_path.exists():
-        output_path.mkdir(parents=True)
-        msg.good(f"Created output directory: {output_path}")
-    nlp.to_disk(output_path)

spacy/cli/benchmark_speed.py

@@ -1,177 +0,0 @@
-import random
-import time
-from itertools import islice
-from pathlib import Path
-from typing import Iterable, List, Optional
-
-import numpy
-import typer
-from tqdm import tqdm
-from wasabi import msg
-
-from .. import util
-from ..language import Language
-from ..tokens import Doc
-from ..training import Corpus
-from ._util import Arg, Opt, benchmark_cli, import_code, setup_gpu
-
-
-@benchmark_cli.command(
-    "speed",
-    context_settings={"allow_extra_args": True, "ignore_unknown_options": True},
-)
-def benchmark_speed_cli(
-    # fmt: off
-    ctx: typer.Context,
-    model: str = Arg(..., help="Model name or path"),
-    data_path: Path = Arg(..., help="Location of binary evaluation data in .spacy format", exists=True),
-    batch_size: Optional[int] = Opt(None, "--batch-size", "-b", min=1, help="Override the pipeline batch size"),
-    no_shuffle: bool = Opt(False, "--no-shuffle", help="Do not shuffle benchmark data"),
-    use_gpu: int = Opt(-1, "--gpu-id", "-g", help="GPU ID or -1 for CPU"),
-    n_batches: int = Opt(50, "--batches", help="Minimum number of batches to benchmark", min=30,),
-    warmup_epochs: int = Opt(3, "--warmup", "-w", min=0, help="Number of iterations over the data for warmup"),
-    code_path: Optional[Path] = Opt(None, "--code", "-c", help="Path to Python file with additional code (registered functions) to be imported"),
-    # fmt: on
-):
-    """
-    Benchmark a pipeline. Expects a loadable spaCy pipeline and benchmark
-    data in the binary .spacy format.
-    """
-    import_code(code_path)
-    setup_gpu(use_gpu=use_gpu, silent=False)
-
-    nlp = util.load_model(model)
-    batch_size = batch_size if batch_size is not None else nlp.batch_size
-    corpus = Corpus(data_path)
-    docs = [eg.predicted for eg in corpus(nlp)]
-
-    if len(docs) == 0:
-        msg.fail("Cannot benchmark speed using an empty corpus.", exits=1)
-
-    print(f"Warming up for {warmup_epochs} epochs...")
-    warmup(nlp, docs, warmup_epochs, batch_size)
-
-    print()
-    print(f"Benchmarking {n_batches} batches...")
-    wps = benchmark(nlp, docs, n_batches, batch_size, not no_shuffle)
-
-    print()
-    print_outliers(wps)
-    print_mean_with_ci(wps)
-
-
-# Lowercased, behaves as a context manager function.
-class time_context:
-    """Register the running time of a context."""
-
-    def __enter__(self):
-        self.start = time.perf_counter()
-        return self
-
-    def __exit__(self, type, value, traceback):
-        self.elapsed = time.perf_counter() - self.start
-
-
-class Quartiles:
-    """Calculate the q1, q2, q3 quartiles and the inter-quartile range (iqr)
-    of a sample."""
-
-    q1: float
-    q2: float
-    q3: float
-    iqr: float
-
-    def __init__(self, sample: numpy.ndarray) -> None:
-        self.q1 = numpy.quantile(sample, 0.25)
-        self.q2 = numpy.quantile(sample, 0.5)
-        self.q3 = numpy.quantile(sample, 0.75)
-        self.iqr = self.q3 - self.q1
-
-
-def annotate(
-    nlp: Language, docs: List[Doc], batch_size: Optional[int]
-) -> numpy.ndarray:
-    docs = nlp.pipe(tqdm(docs, unit="doc", disable=None), batch_size=batch_size)
-    wps = []
-    while True:
-        with time_context() as elapsed:
-            batch_docs = list(
-                islice(docs, batch_size if batch_size else nlp.batch_size)
-            )
-        if len(batch_docs) == 0:
-            break
-        n_tokens = count_tokens(batch_docs)
-        wps.append(n_tokens / elapsed.elapsed)
-
-    return numpy.array(wps)
-
-
-def benchmark(
-    nlp: Language,
-    docs: List[Doc],
-    n_batches: int,
-    batch_size: int,
-    shuffle: bool,
-) -> numpy.ndarray:
-    if shuffle:
-        bench_docs = [
-            nlp.make_doc(random.choice(docs).text)
-            for _ in range(n_batches * batch_size)
-        ]
-    else:
-        bench_docs = [
-            nlp.make_doc(docs[i % len(docs)].text)
-            for i in range(n_batches * batch_size)
-        ]
-
-    return annotate(nlp, bench_docs, batch_size)
-
-
-def bootstrap(x, statistic=numpy.mean, iterations=10000) -> numpy.ndarray:
-    """Apply a statistic to repeated random samples of an array."""
-    return numpy.fromiter(
-        (
-            statistic(numpy.random.choice(x, len(x), replace=True))
-            for _ in range(iterations)
-        ),
-        numpy.float64,
-    )
-
-
-def count_tokens(docs: Iterable[Doc]) -> int:
-    return sum(len(doc) for doc in docs)
-
-
-def print_mean_with_ci(sample: numpy.ndarray):
-    mean = numpy.mean(sample)
-    bootstrap_means = bootstrap(sample)
-    bootstrap_means.sort()
-
-    # 95% confidence interval
-    low = bootstrap_means[int(len(bootstrap_means) * 0.025)]
-    high = bootstrap_means[int(len(bootstrap_means) * 0.975)]
-
-    print(f"Mean: {mean:.1f} words/s (95% CI: {low-mean:.1f} +{high-mean:.1f})")
-
-
-def print_outliers(sample: numpy.ndarray):
-    quartiles = Quartiles(sample)
-
-    n_outliers = numpy.sum(
-        (sample < (quartiles.q1 - 1.5 * quartiles.iqr))
-        | (sample > (quartiles.q3 + 1.5 * quartiles.iqr))
-    )
-    n_extreme_outliers = numpy.sum(
-        (sample < (quartiles.q1 - 3.0 * quartiles.iqr))
-        | (sample > (quartiles.q3 + 3.0 * quartiles.iqr))
-    )
-    print(
-        f"Outliers: {(100 * n_outliers) / len(sample):.1f}%, extreme outliers: {(100 * n_extreme_outliers) / len(sample)}%"
-    )
-
-
-def warmup(
-    nlp: Language, docs: List[Doc], warmup_epochs: int, batch_size: Optional[int]
-) -> numpy.ndarray:
-    docs = [doc.copy() for doc in docs * warmup_epochs]
-    return annotate(nlp, docs, batch_size)

spacy/cli/convert.py

@@ -1,29 +1,25 @@
-import itertools
+from typing import Optional, Any, List, Union
-import re
-import sys
 from enum import Enum
 from pathlib import Path
-from typing import Any, Callable, Iterable, Mapping, Optional, Union
-
-import srsly
 from wasabi import Printer
+import srsly
+import re
+import sys
+import itertools
 
-from ..tokens import Doc, DocBin
+from ._util import app, Arg, Opt
 from ..training import docs_to_json
-from ..training.converters import (
+from ..tokens import DocBin
-    conll_ner_to_docs,
+from ..training.converters import iob_to_docs, conll_ner_to_docs, json_to_docs
-    conllu_to_docs,
+from ..training.converters import conllu_to_docs
-    iob_to_docs,
+
-    json_to_docs,
-)
-from ._util import Arg, Opt, app, walk_directory
 
 # Converters are matched by file extension except for ner/iob, which are
 # matched by file extension and content. To add a converter, add a new
 # entry to this dict with the file extension mapped to the converter function
 # imported from /converters.
 
-CONVERTERS: Mapping[str, Callable[..., Iterable[Doc]]] = {
+CONVERTERS = {
     "conllubio": conllu_to_docs,
     "conllu": conllu_to_docs,
     "conll": conll_ner_to_docs,
@@ -32,8 +28,6 @@ CONVERTERS: Mapping[str, Callable[..., Iterable[Doc]]] = {
     "json": json_to_docs,
 }
 
-AUTO = "auto"
-
 
 # File types that can be written to stdout
 FILE_TYPES_STDOUT = ("json",)
@@ -55,7 +49,7 @@ def convert_cli(
     model: Optional[str] = Opt(None, "--model", "--base", "-b", help="Trained spaCy pipeline for sentence segmentation to use as base (for --seg-sents)"),
     morphology: bool = Opt(False, "--morphology", "-m", help="Enable appending morphology to tags"),
     merge_subtokens: bool = Opt(False, "--merge-subtokens", "-T", help="Merge CoNLL-U subtokens"),
-    converter: str = Opt(AUTO, "--converter", "-c", help=f"Converter: {tuple(CONVERTERS.keys())}"),
+    converter: str = Opt("auto", "--converter", "-c", help=f"Converter: {tuple(CONVERTERS.keys())}"),
     ner_map: Optional[Path] = Opt(None, "--ner-map", "-nm", help="NER tag mapping (as JSON-encoded dict of entity types)", exists=True),
     lang: Optional[str] = Opt(None, "--lang", "-l", help="Language (if tokenizer required)"),
     concatenate: bool = Opt(None, "--concatenate", "-C", help="Concatenate output to a single file"),
@@ -72,16 +66,19 @@ def convert_cli(
 
     DOCS: https://spacy.io/api/cli#convert
     """
+    if isinstance(file_type, FileTypes):
+        # We get an instance of the FileTypes from the CLI so we need its string value
+        file_type = file_type.value
     input_path = Path(input_path)
-    output_dir: Union[str, Path] = "-" if output_dir == Path("-") else output_dir
+    output_dir = "-" if output_dir == Path("-") else output_dir
     silent = output_dir == "-"
     msg = Printer(no_print=silent)
+    verify_cli_args(msg, input_path, output_dir, file_type, converter, ner_map)
     converter = _get_converter(msg, converter, input_path)
-    verify_cli_args(msg, input_path, output_dir, file_type.value, converter, ner_map)
     convert(
         input_path,
         output_dir,
-        file_type=file_type.value,
+        file_type=file_type,
         n_sents=n_sents,
         seg_sents=seg_sents,
         model=model,
@@ -97,7 +94,7 @@ def convert_cli(
 
 
 def convert(
-    input_path: Path,
+    input_path: Union[str, Path],
     output_dir: Union[str, Path],
     *,
     file_type: str = "json",
@@ -106,21 +103,19 @@ def convert(
     model: Optional[str] = None,
     morphology: bool = False,
     merge_subtokens: bool = False,
-    converter: str,
+    converter: str = "auto",
     ner_map: Optional[Path] = None,
     lang: Optional[str] = None,
     concatenate: bool = False,
     silent: bool = True,
-    msg: Optional[Printer] = None,
+    msg: Optional[Printer],
 ) -> None:
-    input_path = Path(input_path)
     if not msg:
         msg = Printer(no_print=silent)
     ner_map = srsly.read_json(ner_map) if ner_map is not None else None
     doc_files = []
-    for input_loc in walk_directory(input_path, converter):
+    for input_loc in walk_directory(Path(input_path), converter):
-        with input_loc.open("r", encoding="utf-8") as infile:
+        input_data = input_loc.open("r", encoding="utf-8").read()
-            input_data = infile.read()
         # Use converter function to convert data
         func = CONVERTERS[converter]
         docs = func(
@@ -145,7 +140,7 @@ def convert(
         else:
             db = DocBin(docs=docs, store_user_data=True)
             len_docs = len(db)
-            data = db.to_bytes()  # type: ignore[assignment]
+            data = db.to_bytes()
         if output_dir == "-":
             _print_docs_to_stdout(data, file_type)
         else:
@@ -195,14 +190,42 @@ def autodetect_ner_format(input_data: str) -> Optional[str]:
     return None
 
 
+def walk_directory(path: Path, converter: str) -> List[Path]:
+    if not path.is_dir():
+        return [path]
+    paths = [path]
+    locs = []
+    seen = set()
+    for path in paths:
+        if str(path) in seen:
+            continue
+        seen.add(str(path))
+        if path.parts[-1].startswith("."):
+            continue
+        elif path.is_dir():
+            paths.extend(path.iterdir())
+        elif converter == "json" and not path.parts[-1].endswith("json"):
+            continue
+        elif converter == "conll" and not path.parts[-1].endswith("conll"):
+            continue
+        elif converter == "iob" and not path.parts[-1].endswith("iob"):
+            continue
+        else:
+            locs.append(path)
+    # It's good to sort these, in case the ordering messes up cache.
+    locs.sort()
+    return locs
+
+
 def verify_cli_args(
     msg: Printer,
-    input_path: Path,
+    input_path: Union[str, Path],
     output_dir: Union[str, Path],
-    file_type: str,
+    file_type: FileTypes,
     converter: str,
     ner_map: Optional[Path],
 ):
+    input_path = Path(input_path)
     if file_type not in FILE_TYPES_STDOUT and output_dir == "-":
         msg.fail(
             f"Can't write .{file_type} data to stdout. Please specify an output directory.",
@@ -218,22 +241,18 @@ def verify_cli_args(
         input_locs = walk_directory(input_path, converter)
         if len(input_locs) == 0:
             msg.fail("No input files in directory", input_path, exits=1)
-    if converter not in CONVERTERS:
+        file_types = list(set([loc.suffix[1:] for loc in input_locs]))
+        if converter == "auto" and len(file_types) >= 2:
+            file_types = ",".join(file_types)
+            msg.fail("All input files must be same type", file_types, exits=1)
+    if converter != "auto" and converter not in CONVERTERS:
         msg.fail(f"Can't find converter for {converter}", exits=1)
 
 
-def _get_converter(msg, converter, input_path: Path):
+def _get_converter(msg, converter, input_path):
     if input_path.is_dir():
-        if converter == AUTO:
+        input_path = walk_directory(input_path, converter)[0]
-            input_locs = walk_directory(input_path, suffix=None)
+    if converter == "auto":
-            file_types = list(set([loc.suffix[1:] for loc in input_locs]))
-            if len(file_types) >= 2:
-                file_types_str = ",".join(file_types)
-                msg.fail("All input files must be same type", file_types_str, exits=1)
-            input_path = input_locs[0]
-        else:
-            input_path = walk_directory(input_path, suffix=converter)[0]
-    if converter == AUTO:
         converter = input_path.suffix[1:]
     if converter == "ner" or converter == "iob":
         with input_path.open(encoding="utf8") as file_:

spacy/cli/debug_config.py

@@ -1,22 +1,15 @@
+from typing import Optional, Dict, Any, Union, List
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Union
+from wasabi import msg, table
-
-import typer
 from thinc.api import Config
 from thinc.config import VARIABLE_RE
-from wasabi import msg, table
+import typer
 
-from .. import util
+from ._util import Arg, Opt, show_validation_error, parse_config_overrides
+from ._util import import_code, debug_cli
 from ..schemas import ConfigSchemaInit, ConfigSchemaTraining
 from ..util import registry
-from ._util import (
+from .. import util
-    Arg,
-    Opt,
-    debug_cli,
-    import_code,
-    parse_config_overrides,
-    show_validation_error,
-)
 
 
 @debug_cli.command(
@@ -27,12 +20,12 @@ def debug_config_cli(
     # fmt: off
     ctx: typer.Context,  # This is only used to read additional arguments
     config_path: Path = Arg(..., help="Path to config file", exists=True, allow_dash=True),
-    code_path: Optional[Path] = Opt(None, "--code-path", "--code", "-c", help="Path to Python file with additional code (registered functions) to be imported"),
+    code_path: Optional[Path] = Opt(None, "--code-path", "-c", help="Path to Python file with additional code (registered functions) to be imported"),
     show_funcs: bool = Opt(False, "--show-functions", "-F", help="Show an overview of all registered functions used in the config and where they come from (modules, files etc.)"),
     show_vars: bool = Opt(False, "--show-variables", "-V", help="Show an overview of all variables referenced in the config and their values. This will also reflect variables overwritten on the CLI.")
     # fmt: on
 ):
-    """Debug a config file and show validation errors. The command will
+    """Debug a config.cfg file and show validation errors. The command will
     create all objects in the tree and validate them. Note that some config
     validation errors are blocking and will prevent the rest of the config from
     being resolved. This means that you may not see all validation errors at

spacy/cli/debug_diff.py

@@ -1,89 +0,0 @@
-from pathlib import Path
-from typing import Optional
-
-import typer
-from thinc.api import Config
-from wasabi import MarkdownRenderer, Printer, diff_strings
-
-from ..util import load_config
-from ._util import Arg, Opt, debug_cli, parse_config_overrides, show_validation_error
-from .init_config import Optimizations, init_config
-
-
-@debug_cli.command(
-    "diff-config",
-    context_settings={"allow_extra_args": True, "ignore_unknown_options": True},
-)
-def debug_diff_cli(
-    # fmt: off
-    ctx: typer.Context,
-    config_path: Path = Arg(..., help="Path to config file", exists=True, allow_dash=True),
-    compare_to: Optional[Path] = Opt(None, help="Path to a config file to diff against, or `None` to compare against default settings", exists=True, allow_dash=True),
-    optimize: Optimizations = Opt(Optimizations.efficiency.value, "--optimize", "-o", help="Whether the user config was optimized for efficiency or accuracy. Only relevant when comparing against the default config."),
-    gpu: bool = Opt(False, "--gpu", "-G", help="Whether the original config can run on a GPU. Only relevant when comparing against the default config."),
-    pretraining: bool = Opt(False, "--pretraining", "--pt", help="Whether to compare on a config with pretraining involved. Only relevant when comparing against the default config."),
-    markdown: bool = Opt(False, "--markdown", "-md", help="Generate Markdown for GitHub issues")
-    # fmt: on
-):
-    """Show a diff of a config file with respect to spaCy's defaults or another config file. If
-    additional settings were used in the creation of the config file, then you
-    must supply these as extra parameters to the command when comparing to the default settings. The generated diff
-    can also be used when posting to the discussion forum to provide more
-    information for the maintainers.
-
-    The `optimize`, `gpu`, and `pretraining` options are only relevant when
-    comparing against the default configuration (or specifically when `compare_to` is None).
-
-    DOCS: https://spacy.io/api/cli#debug-diff
-    """
-    debug_diff(
-        config_path=config_path,
-        compare_to=compare_to,
-        gpu=gpu,
-        optimize=optimize,
-        pretraining=pretraining,
-        markdown=markdown,
-    )
-
-
-def debug_diff(
-    config_path: Path,
-    compare_to: Optional[Path],
-    gpu: bool,
-    optimize: Optimizations,
-    pretraining: bool,
-    markdown: bool,
-):
-    msg = Printer()
-    with show_validation_error(hint_fill=False):
-        user_config = load_config(config_path)
-        if compare_to:
-            other_config = load_config(compare_to)
-        else:
-            # Recreate a default config based from user's config
-            lang = user_config["nlp"]["lang"]
-            pipeline = list(user_config["nlp"]["pipeline"])
-            msg.info(f"Found user-defined language: '{lang}'")
-            msg.info(f"Found user-defined pipelines: {pipeline}")
-            other_config = init_config(
-                lang=lang,
-                pipeline=pipeline,
-                optimize=optimize.value,
-                gpu=gpu,
-                pretraining=pretraining,
-                silent=True,
-            )
-
-    user = user_config.to_str()
-    other = other_config.to_str()
-
-    if user == other:
-        msg.warn("No diff to show: configs are identical")
-    else:
-        diff_text = diff_strings(other, user, add_symbols=markdown)
-        if markdown:
-            md = MarkdownRenderer()
-            md.add(md.code_block(diff_text, "diff"))
-            print(md.text)
-        else:
-            print(diff_text)

spacy/cli/debug_model.py

@@ -1,32 +1,18 @@
-import itertools
+from typing import Dict, Any, Optional, Iterable
 from pathlib import Path
-from typing import Any, Dict, Optional
-
-import typer
-from thinc.api import (
-    Model,
-    data_validation,
-    fix_random_seed,
-    set_dropout_rate,
-    set_gpu_allocator,
-)
-from wasabi import msg
 
 from spacy.training import Example
 from spacy.util import resolve_dot_names
+from wasabi import msg
+from thinc.api import fix_random_seed, set_dropout_rate, Adam
+from thinc.api import Model, data_validation, set_gpu_allocator
+import typer
 
-from .. import util
+from ._util import Arg, Opt, debug_cli, show_validation_error
+from ._util import parse_config_overrides, string_to_list, setup_gpu
 from ..schemas import ConfigSchemaTraining
 from ..util import registry
-from ._util import (
+from .. import util
-    Arg,
-    Opt,
-    debug_cli,
-    parse_config_overrides,
-    setup_gpu,
-    show_validation_error,
-    string_to_list,
-)
 
 
 @debug_cli.command(
@@ -87,24 +73,23 @@ def debug_model_cli(
         msg.info(f"Fixing random seed: {seed}")
         fix_random_seed(seed)
     pipe = nlp.get_pipe(component)
-
+    if not hasattr(pipe, "model"):
-    debug_model(config, T, nlp, pipe, print_settings=print_settings)
+        msg.fail(
+            f"The component '{component}' does not specify an object that holds a Model.",
+            exits=1,
+        )
+    model = pipe.model
+    debug_model(config, T, nlp, model, print_settings=print_settings)
 
 
 def debug_model(
     config,
     resolved_train_config,
     nlp,
-    pipe,
+    model: Model,
     *,
     print_settings: Optional[Dict[str, Any]] = None,
 ):
-    if not hasattr(pipe, "model"):
-        msg.fail(
-            f"The component '{pipe}' does not specify an object that holds a Model.",
-            exits=1,
-        )
-    model = pipe.model
     if not isinstance(model, Model):
         msg.fail(
             f"Requires a Thinc Model to be analysed, but found {type(model)} instead.",
@@ -120,6 +105,8 @@ def debug_model(
         _print_model(model, print_settings)
 
     # STEP 1: Initializing the model and printing again
+    X = _get_docs()
+    # The output vector might differ from the official type of the output layer
     with data_validation(False):
         try:
             dot_names = [resolved_train_config["train_corpus"]]
@@ -127,17 +114,15 @@ def debug_model(
                 (train_corpus,) = resolve_dot_names(config, dot_names)
                 nlp.initialize(lambda: train_corpus(nlp))
             msg.info("Initialized the model with the training corpus.")
-            examples = list(itertools.islice(train_corpus(nlp), 5))
         except ValueError:
             try:
                 _set_output_dim(nO=7, model=model)
                 with show_validation_error():
-                    examples = [Example.from_dict(x, {}) for x in _get_docs()]
+                    nlp.initialize(lambda: [Example.from_dict(x, {}) for x in X])
-                    nlp.initialize(lambda: examples)
                 msg.info("Initialized the model with dummy data.")
             except Exception:
                 msg.fail(
-                    "Could not initialize the model: you'll have to provide a valid 'train_corpus' argument in the config file.",
+                    "Could not initialize the model: you'll have to provide a valid train_corpus argument in the config file.",
                     exits=1,
                 )
 
@@ -146,31 +131,46 @@ def debug_model(
         _print_model(model, print_settings)
 
     # STEP 2: Updating the model and printing again
+    optimizer = Adam(0.001)
     set_dropout_rate(model, 0.2)
-    # ugly hack to deal with Tok2Vec/Transformer listeners
+    # ugly hack to deal with Tok2Vec listeners
-    upstream_component = None
+    tok2vec = None
-    if model.has_ref("tok2vec") and "tok2vec-listener" in model.get_ref("tok2vec").name:
+    if model.has_ref("tok2vec") and model.get_ref("tok2vec").name == "tok2vec-listener":
-        upstream_component = nlp.get_pipe("tok2vec")
+        tok2vec = nlp.get_pipe("tok2vec")
-    if (
+    goldY = None
-        model.has_ref("tok2vec")
-        and "transformer-listener" in model.get_ref("tok2vec").name
-    ):
-        upstream_component = nlp.get_pipe("transformer")
     for e in range(3):
-        if upstream_component:
+        if tok2vec:
-            upstream_component.update(examples)
+            tok2vec.update([Example.from_dict(x, {}) for x in X])
-        pipe.update(examples)
+        Y, get_dX = model.begin_update(X)
+        if goldY is None:
+            goldY = _simulate_gold(Y)
+        dY = get_gradient(goldY, Y, model.ops)
+        get_dX(dY)
+        model.finish_update(optimizer)
     if print_settings.get("print_after_training"):
         msg.divider(f"STEP 2 - after training")
         _print_model(model, print_settings)
 
     # STEP 3: the final prediction
-    prediction = model.predict([ex.predicted for ex in examples])
+    prediction = model.predict(X)
     if print_settings.get("print_prediction"):
         msg.divider(f"STEP 3 - prediction")
         msg.info(str(prediction))
 
-    msg.good(f"Successfully ended analysis - model looks good.")
+    msg.good(f"Succesfully ended analysis - model looks good.")
+
+
+def get_gradient(goldY, Y, ops):
+    return ops.asarray(Y) - ops.asarray(goldY)
+
+
+def _simulate_gold(element, counter=1):
+    if isinstance(element, Iterable):
+        for i in range(len(element)):
+            element[i] = _simulate_gold(element[i], counter + i)
+        return element
+    else:
+        return 1 / counter
 
 
 def _sentences():
@@ -209,7 +209,11 @@ def _print_model(model, print_settings):
 
             if dimensions:
                 for name in node.dim_names:
-                    msg.info(f" - dim {name}: {node.maybe_get_dim(name)}")
+                    if node.has_dim(name):
+                        msg.info(f" - dim {name}: {node.get_dim(name)}")
+                    else:
+                        msg.info(f" - dim {name}: {node.has_dim(name)}")
+
             if parameters:
                 for name in node.param_names:
                     if node.has_param(name):

spacy/cli/download.py

@@ -1,22 +1,13 @@
-import sys
 from typing import Optional, Sequence
-from urllib.parse import urljoin
-
 import requests
-import typer
+import sys
 from wasabi import msg
+import typer
 
+from ._util import app, Arg, Opt, WHEEL_SUFFIX, SDIST_SUFFIX
 from .. import about
+from ..util import is_package, get_base_version, run_command
 from ..errors import OLD_MODEL_SHORTCUTS
-from ..util import (
-    get_minor_version,
-    is_in_interactive,
-    is_in_jupyter,
-    is_package,
-    is_prerelease_version,
-    run_command,
-)
-from ._util import SDIST_SUFFIX, WHEEL_SUFFIX, Arg, Opt, app
 
 
 @app.command(
@@ -28,7 +19,7 @@ def download_cli(
     ctx: typer.Context,
     model: str = Arg(..., help="Name of pipeline package to download"),
     direct: bool = Opt(False, "--direct", "-d", "-D", help="Force direct download of name + version"),
-    sdist: bool = Opt(False, "--sdist", "-S", help="Download sdist (.tar.gz) archive instead of pre-built binary wheel"),
+    sdist: bool = Opt(False, "--sdist", "-S", help="Download sdist (.tar.gz) archive instead of pre-built binary wheel")
     # fmt: on
 ):
     """
@@ -44,12 +35,7 @@ def download_cli(
     download(model, direct, sdist, *ctx.args)
 
 
-def download(
+def download(model: str, direct: bool = False, sdist: bool = False, *pip_args) -> None:
-    model: str,
-    direct: bool = False,
-    sdist: bool = False,
-    *pip_args,
-) -> None:
     if (
         not (is_package("spacy") or is_package("spacy-nightly"))
         and "--no-deps" not in pip_args
@@ -63,17 +49,13 @@ def download(
             "dependencies, you'll have to install them manually."
         )
         pip_args = pip_args + ("--no-deps",)
+    suffix = SDIST_SUFFIX if sdist else WHEEL_SUFFIX
+    dl_tpl = "{m}-{v}/{m}-{v}{s}#egg={m}=={v}"
     if direct:
-        # Reject model names with '/', in order to prevent shenanigans.
-        if "/" in model:
-            msg.fail(
-                title="Model download rejected",
-                text=f"Cannot download model '{model}'. Models are expected to be file names, not URLs or fragments",
-                exits=True,
-            )
         components = model.split("-")
         model_name = "".join(components[:-1])
         version = components[-1]
+        download_model(dl_tpl.format(m=model_name, v=version, s=suffix), pip_args)
     else:
         model_name = model
         if model in OLD_MODEL_SHORTCUTS:
@@ -84,49 +66,15 @@ def download(
             model_name = OLD_MODEL_SHORTCUTS[model]
         compatibility = get_compatibility()
         version = get_version(model_name, compatibility)
-
+        download_model(dl_tpl.format(m=model_name, v=version, s=suffix), pip_args)
-    filename = get_model_filename(model_name, version, sdist)
-
-    download_model(filename, pip_args)
     msg.good(
         "Download and installation successful",
         f"You can now load the package via spacy.load('{model_name}')",
     )
-    if is_in_jupyter():
-        reload_deps_msg = (
-            "If you are in a Jupyter or Colab notebook, you may need to "
-            "restart Python in order to load all the package's dependencies. "
-            "You can do this by selecting the 'Restart kernel' or 'Restart "
-            "runtime' option."
-        )
-        msg.warn(
-            "Restart to reload dependencies",
-            reload_deps_msg,
-        )
-    elif is_in_interactive():
-        reload_deps_msg = (
-            "If you are in an interactive Python session, you may need to "
-            "exit and restart Python to load all the package's dependencies. "
-            "You can exit with Ctrl-D (or Ctrl-Z and Enter on Windows)."
-        )
-        msg.warn(
-            "Restart to reload dependencies",
-            reload_deps_msg,
-        )
-
-
-def get_model_filename(model_name: str, version: str, sdist: bool = False) -> str:
-    dl_tpl = "{m}-{v}/{m}-{v}{s}"
-    suffix = SDIST_SUFFIX if sdist else WHEEL_SUFFIX
-    filename = dl_tpl.format(m=model_name, v=version, s=suffix)
-    return filename
 
 
 def get_compatibility() -> dict:
-    if is_prerelease_version(about.__version__):
+    version = get_base_version(about.__version__)
-        version: Optional[str] = about.__version__
-    else:
-        version = get_minor_version(about.__version__)
     r = requests.get(about.__compatibility__)
     if r.status_code != 200:
         msg.fail(
@@ -153,24 +101,10 @@ def get_version(model: str, comp: dict) -> str:
     return comp[model][0]
 
 
-def get_latest_version(model: str) -> str:
-    comp = get_compatibility()
-    return get_version(model, comp)
-
-
 def download_model(
     filename: str, user_pip_args: Optional[Sequence[str]] = None
 ) -> None:
-    # Construct the download URL carefully. We need to make sure we don't
+    download_url = about.__download_url__ + "/" + filename
-    # allow relative paths or other shenanigans to trick us into download
-    # from outside our own repo.
-    base_url = about.__download_url__
-    # urljoin requires that the path ends with /, or the last path part will be dropped
-    if not base_url.endswith("/"):
-        base_url = about.__download_url__ + "/"
-    download_url = urljoin(base_url, filename)
-    if not download_url.startswith(about.__download_url__):
-        raise ValueError(f"Download from {filename} rejected. Was it a relative path?")
     pip_args = list(user_pip_args) if user_pip_args is not None else []
     cmd = [sys.executable, "-m", "pip", "install"] + pip_args + [download_url]
     run_command(cmd)

spacy/cli/evaluate.py

@@ -1,21 +1,18 @@
-import re
+from typing import Optional, List, Dict
+from wasabi import Printer
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Union
+import re
-
 import srsly
 from thinc.api import fix_random_seed
-from wasabi import Printer
 
-from .. import displacy, util
-from ..scorer import Scorer
-from ..tokens import Doc
 from ..training import Corpus
-from ._util import Arg, Opt, app, benchmark_cli, import_code, setup_gpu
+from ..tokens import Doc
+from ._util import app, Arg, Opt, setup_gpu, import_code
+from ..scorer import Scorer
+from .. import util
+from .. import displacy
 
 
-@benchmark_cli.command(
-    "accuracy",
-)
 @app.command("evaluate")
 def evaluate_cli(
     # fmt: off
@@ -27,8 +24,6 @@ def evaluate_cli(
     gold_preproc: bool = Opt(False, "--gold-preproc", "-G", help="Use gold preprocessing"),
     displacy_path: Optional[Path] = Opt(None, "--displacy-path", "-dp", help="Directory to output rendered parses as HTML", exists=True, file_okay=False),
     displacy_limit: int = Opt(25, "--displacy-limit", "-dl", help="Limit of parses to render as HTML"),
-    per_component: bool = Opt(False, "--per-component", "-P", help="Return scores per component, only applicable when an output JSON file is specified."),
-    spans_key: str = Opt("sc", "--spans-key", "-sk", help="Spans key to use when evaluating Doc.spans"),
     # fmt: on
 ):
     """
@@ -41,7 +36,7 @@ def evaluate_cli(
     dependency parses in a HTML file, set as output directory as the
     displacy_path argument.
 
-    DOCS: https://spacy.io/api/cli#benchmark-accuracy
+    DOCS: https://spacy.io/api/cli#evaluate
     """
     import_code(code_path)
     evaluate(
@@ -52,9 +47,7 @@ def evaluate_cli(
         gold_preproc=gold_preproc,
         displacy_path=displacy_path,
         displacy_limit=displacy_limit,
-        per_component=per_component,
         silent=False,
-        spans_key=spans_key,
     )
 
 
@@ -67,12 +60,10 @@ def evaluate(
     displacy_path: Optional[Path] = None,
     displacy_limit: int = 25,
     silent: bool = True,
-    spans_key: str = "sc",
+) -> Scorer:
-    per_component: bool = False,
-) -> Dict[str, Any]:
     msg = Printer(no_print=silent, pretty=not silent)
     fix_random_seed()
-    setup_gpu(use_gpu, silent=silent)
+    setup_gpu(use_gpu)
     data_path = util.ensure_path(data_path)
     output_path = util.ensure_path(output)
     displacy_path = util.ensure_path(displacy_path)
@@ -83,86 +74,41 @@ def evaluate(
     corpus = Corpus(data_path, gold_preproc=gold_preproc)
     nlp = util.load_model(model)
     dev_dataset = list(corpus(nlp))
-    scores = nlp.evaluate(dev_dataset, per_component=per_component)
+    scores = nlp.evaluate(dev_dataset)
-    if per_component:
+    metrics = {
-        data = scores
+        "TOK": "token_acc",
-        if output is None:
+        "TAG": "tag_acc",
-            msg.warn(
+        "POS": "pos_acc",
-                "The per-component option is enabled but there is no output JSON file provided to save the scores to."
+        "MORPH": "morph_acc",
-            )
+        "LEMMA": "lemma_acc",
-        else:
+        "UAS": "dep_uas",
-            msg.info("Per-component scores will be saved to output JSON file.")
+        "LAS": "dep_las",
-    else:
+        "NER P": "ents_p",
-        metrics = {
+        "NER R": "ents_r",
-            "TOK": "token_acc",
+        "NER F": "ents_f",
-            "TAG": "tag_acc",
+        "TEXTCAT": "cats_score",
-            "POS": "pos_acc",
+        "SENT P": "sents_p",
-            "MORPH": "morph_acc",
+        "SENT R": "sents_r",
-            "LEMMA": "lemma_acc",
+        "SENT F": "sents_f",
-            "UAS": "dep_uas",
+        "SPEED": "speed",
-            "LAS": "dep_las",
+    }
-            "NER P": "ents_p",
+    results = {}
-            "NER R": "ents_r",
+    data = {}
-            "NER F": "ents_f",
+    for metric, key in metrics.items():
-            "TEXTCAT": "cats_score",
+        if key in scores:
-            "SENT P": "sents_p",
+            if key == "cats_score":
-            "SENT R": "sents_r",
+                metric = metric + " (" + scores.get("cats_score_desc", "unk") + ")"
-            "SENT F": "sents_f",
+            if isinstance(scores[key], (int, float)):
-            "SPAN P": f"spans_{spans_key}_p",
+                if key == "speed":
-            "SPAN R": f"spans_{spans_key}_r",
+                    results[metric] = f"{scores[key]:.0f}"
-            "SPAN F": f"spans_{spans_key}_f",
-            "SPEED": "speed",
-        }
-        results = {}
-        data = {}
-        for metric, key in metrics.items():
-            if key in scores:
-                if key == "cats_score":
-                    metric = metric + " (" + scores.get("cats_score_desc", "unk") + ")"
-                if isinstance(scores[key], (int, float)):
-                    if key == "speed":
-                        results[metric] = f"{scores[key]:.0f}"
-                    else:
-                        results[metric] = f"{scores[key]*100:.2f}"
                 else:
-                    results[metric] = "-"
+                    results[metric] = f"{scores[key]*100:.2f}"
-                data[re.sub(r"[\s/]", "_", key.lower())] = scores[key]
+            else:
+                results[metric] = "-"
+            data[re.sub(r"[\s/]", "_", key.lower())] = scores[key]
 
-        msg.table(results, title="Results")
+    msg.table(results, title="Results")
-        data = handle_scores_per_type(scores, data, spans_key=spans_key, silent=silent)
 
-    if displacy_path:
-        factory_names = [nlp.get_pipe_meta(pipe).factory for pipe in nlp.pipe_names]
-        docs = list(nlp.pipe(ex.reference.text for ex in dev_dataset[:displacy_limit]))
-        render_deps = "parser" in factory_names
-        render_ents = "ner" in factory_names
-        render_spans = "spancat" in factory_names
-
-        render_parses(
-            docs,
-            displacy_path,
-            model_name=model,
-            limit=displacy_limit,
-            deps=render_deps,
-            ents=render_ents,
-            spans=render_spans,
-        )
-        msg.good(f"Generated {displacy_limit} parses as HTML", displacy_path)
-
-    if output_path is not None:
-        srsly.write_json(output_path, data)
-        msg.good(f"Saved results to {output_path}")
-    return data
-
-
-def handle_scores_per_type(
-    scores: Dict[str, Any],
-    data: Dict[str, Any] = {},
-    *,
-    spans_key: str = "sc",
-    silent: bool = False,
-) -> Dict[str, Any]:
-    msg = Printer(no_print=silent, pretty=not silent)
     if "morph_per_feat" in scores:
         if scores["morph_per_feat"]:
             print_prf_per_type(msg, scores["morph_per_feat"], "MORPH", "feat")
@@ -175,12 +121,6 @@ def handle_scores_per_type(
         if scores["ents_per_type"]:
             print_prf_per_type(msg, scores["ents_per_type"], "NER", "type")
             data["ents_per_type"] = scores["ents_per_type"]
-    if f"spans_{spans_key}_per_type" in scores:
-        if scores[f"spans_{spans_key}_per_type"]:
-            print_prf_per_type(
-                msg, scores[f"spans_{spans_key}_per_type"], "SPANS", "type"
-            )
-            data[f"spans_{spans_key}_per_type"] = scores[f"spans_{spans_key}_per_type"]
     if "cats_f_per_type" in scores:
         if scores["cats_f_per_type"]:
             print_prf_per_type(msg, scores["cats_f_per_type"], "Textcat F", "label")
@@ -189,7 +129,26 @@ def handle_scores_per_type(
         if scores["cats_auc_per_type"]:
             print_textcats_auc_per_cat(msg, scores["cats_auc_per_type"])
             data["cats_auc_per_type"] = scores["cats_auc_per_type"]
-    return scores
+
+    if displacy_path:
+        factory_names = [nlp.get_pipe_meta(pipe).factory for pipe in nlp.pipe_names]
+        docs = list(nlp.pipe(ex.reference.text for ex in dev_dataset[:displacy_limit]))
+        render_deps = "parser" in factory_names
+        render_ents = "ner" in factory_names
+        render_parses(
+            docs,
+            displacy_path,
+            model_name=model,
+            limit=displacy_limit,
+            deps=render_deps,
+            ents=render_ents,
+        )
+        msg.good(f"Generated {displacy_limit} parses as HTML", displacy_path)
+
+    if output_path is not None:
+        srsly.write_json(output_path, data)
+        msg.good(f"Saved results to {output_path}")
+    return data
 
 
 def render_parses(
@@ -199,7 +158,6 @@ def render_parses(
     limit: int = 250,
     deps: bool = True,
     ents: bool = True,
-    spans: bool = True,
 ):
     docs[0].user_data["title"] = model_name
     if ents:
@@ -213,11 +171,6 @@ def render_parses(
         with (output_path / "parses.html").open("w", encoding="utf8") as file_:
             file_.write(html)
 
-    if spans:
-        html = displacy.render(docs[:limit], style="span", page=True)
-        with (output_path / "spans.html").open("w", encoding="utf8") as file_:
-            file_.write(html)
-
 
 def print_prf_per_type(
     msg: Printer, scores: Dict[str, Dict[str, float]], name: str, type: str