explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-08-04 20:30:24 +03:00
Code Issues Packages Projects Releases Wiki Activity

Fix loss serialization issue (#10600)

Browse Source 
* Fix loss serialization issue

Serialization of a model fails with:

TypeError: array(738.3855, dtype=float32) is not JSON serializable

Fix this using float conversion.

* Disable CI steps that require spacy.TransitionBasedParser.v2

After finishing the refactor, TransitionBasedParser.v2 should be
provided for backwards compat.
This commit is contained in:
Daniël de Kok 2022-04-01 20:57:52 +02:00 committed by GitHub
parent 995af09073
commit 8cc29154ac
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 20 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
.github/azure-steps.yml vendored
View File

@ -64,12 +64,13 @@ steps:
displayName: "Run GPU tests" displayName: "Run GPU tests"
condition: eq(${{ parameters.gpu }}, true) condition: eq(${{ parameters.gpu }}, true)
- script: | # Disabled in parser refactor branch, requires spacy.TransitionBasedParser.v2.
python -m spacy download ca_core_news_sm #- script: |
python -m spacy download ca_core_news_md # python -m spacy download ca_core_news_sm
python -c "import spacy; nlp=spacy.load('ca_core_news_sm'); doc=nlp('test')" # python -m spacy download ca_core_news_md
displayName: 'Test download CLI' # python -c "import spacy; nlp=spacy.load('ca_core_news_sm'); doc=nlp('test')"
condition: eq(variables['python_version'], '3.8') # displayName: 'Test download CLI'
# condition: eq(variables['python_version'], '3.8')
- script: | - script: |
python -m spacy convert extra/example_data/ner_example_data/ner-token-per-line-conll2003.json . python -m spacy convert extra/example_data/ner_example_data/ner-token-per-line-conll2003.json .
@ -93,17 +94,19 @@ steps:
displayName: 'Test train CLI' displayName: 'Test train CLI'
condition: eq(variables['python_version'], '3.8') condition: eq(variables['python_version'], '3.8')
- script: | # Disabled in parser refactor branch, requires spacy.TransitionBasedParser.v2.
python -c "import spacy; config = spacy.util.load_config('ner.cfg'); config['components']['ner'] = {'source': 'ca_core_news_sm'}; config.to_disk('ner_source_sm.cfg')" # - script: |
PYTHONWARNINGS="error,ignore::DeprecationWarning" python -m spacy assemble ner_source_sm.cfg output_dir # python -c "import spacy; config = spacy.util.load_config('ner.cfg'); config['components']['ner'] = {'source': 'ca_core_news_sm'}; config.to_disk('ner_source_sm.cfg')"
displayName: 'Test assemble CLI' # PYTHONWARNINGS="error,ignore::DeprecationWarning" python -m spacy assemble ner_source_sm.cfg output_dir
condition: eq(variables['python_version'], '3.8') # displayName: 'Test assemble CLI'
# condition: eq(variables['python_version'], '3.8')
- script: | # Disabled in parser refactor branch, requires spacy.TransitionBasedParser.v2.
python -c "import spacy; config = spacy.util.load_config('ner.cfg'); config['components']['ner'] = {'source': 'ca_core_news_md'}; config.to_disk('ner_source_md.cfg')" # - script: |
python -m spacy assemble ner_source_md.cfg output_dir 2>&1 | grep -q W113 # python -c "import spacy; config = spacy.util.load_config('ner.cfg'); config['components']['ner'] = {'source': 'ca_core_news_md'}; config.to_disk('ner_source_md.cfg')"
displayName: 'Test assemble CLI vectors warning' # python -m spacy assemble ner_source_md.cfg output_dir 2>&1 | grep -q W113
condition: eq(variables['python_version'], '3.8') # displayName: 'Test assemble CLI vectors warning'
# condition: eq(variables['python_version'], '3.8')
- script: | - script: |
python .github/validate_universe_json.py website/meta/universe.json python .github/validate_universe_json.py website/meta/universe.json

2
spacy/pipeline/transition_parser.pyx
View File

@ -298,7 +298,7 @@ class Parser(TrainablePipe):
backprop_scores((states, d_scores)) backprop_scores((states, d_scores))
if sgd not in (None, False): if sgd not in (None, False):
self.finish_update(sgd) self.finish_update(sgd)
losses[self.name] += (d_scores**2).sum() losses[self.name] += float((d_scores**2).sum())
# Ugh, this is annoying. If we're working on GPU, we want to free the # Ugh, this is annoying. If we're working on GPU, we want to free the
# memory ASAP. It seems that Python doesn't necessarily get around to # memory ASAP. It seems that Python doesn't necessarily get around to
# removing these in time if we don't explicitly delete? It's confusing. # removing these in time if we don't explicitly delete? It's confusing.