explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-25 01:16:28 +03:00
Code Issues Packages Projects Releases Wiki Activity

Stream large assets on download (#10521)

Browse Source 
Stream large assets on download rather than reading the whole file at
once and potentially running into `urllib3` limits on single read sizes.
This commit is contained in:
Adriane Boyd 2022-03-24 11:47:05 +01:00 committed by GitHub
parent e908a67829
commit d85117f88c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
spacy/cli/_util.py
View File

@ -360,7 +360,7 @@ def download_file(src: Union[str, "Pathy"], dest: Path, *, force: bool = False)
src = str(src) src = str(src)
with smart_open.open(src, mode="rb", ignore_ext=True) as input_file: with smart_open.open(src, mode="rb", ignore_ext=True) as input_file:
with dest.open(mode="wb") as output_file: with dest.open(mode="wb") as output_file:
output_file.write(input_file.read()) shutil.copyfileobj(input_file, output_file)
def ensure_pathy(path): def ensure_pathy(path):