spaCy/tests/parser/test_parser_pickle.py

import pytest

import pickle
import cloudpickle

import io


@pytest.mark.models
def test_pickle(EN):
    file_ = io.BytesIO()
    cloudpickle.dump(EN.parser, file_)

    file_.seek(0)

    loaded = pickle.load(file_)
* Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-12 11:33:11 +03:00			`import pytest`

			`import pickle`
			`import cloudpickle`
* Fix StringIO for Python 3 2015-10-13 12:20:10 +03:00
			`import io`
* Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-12 11:33:11 +03:00

			`@pytest.mark.models`
			`def test_pickle(EN):`
* Fix StringIO for Python 3 2015-10-13 12:20:10 +03:00			`file_ = io.BytesIO()`
* Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-12 11:33:11 +03:00			`cloudpickle.dump(EN.parser, file_)`

			`file_.seek(0)`

			`loaded = pickle.load(file_)`