explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-26 09:56:28 +03:00
Code Issues Packages Projects Releases Wiki Activity

Remove unnecessary iterator in Language.pipe (#5101)

Browse Source 
Remove iterator over `raw_texts` with `iterator.tee()` in
`Language.pipe` that is never consumed and consumes memory
unnecessarily.
This commit is contained in:
adrianeboyd 2020-03-08 13:22:25 +01:00 committed by GitHub
parent cd79c7bd26
commit 993758c58f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 0 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
spacy/language.py
View File

@ -757,8 +757,6 @@ class Language(object):
DOCS: https://spacy.io/api/language#pipe DOCS: https://spacy.io/api/language#pipe
""" """
# raw_texts will be used later to stop iterator.
texts, raw_texts = itertools.tee(texts)
if is_python2 and n_process != 1: if is_python2 and n_process != 1:
user_warning(Warnings.W023) user_warning(Warnings.W023)
n_process = 1 n_process = 1