spaCy/tests/test_pickle.py

import pytest
import io
import cloudpickle
import pickle


@pytest.mark.models
def test_pickle_english(EN):
    file_ = io.BytesIO()
    cloudpickle.dump(EN, file_)

    file_.seek(0)

    loaded = pickle.load(file_)
* Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-12 11:33:11 +03:00			`import pytest`
* Fix StringIO for Python 3 2015-10-13 12:20:10 +03:00			`import io`
* Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-12 11:33:11 +03:00			`import cloudpickle`
			`import pickle`


			`@pytest.mark.models`
			`def test_pickle_english(EN):`
* Fix StringIO for Python 3 2015-10-13 12:20:10 +03:00			`file_ = io.BytesIO()`
* Very scrappy, likely buggy first-cut pickle implementation, to work on Issue #125: allow pickle for Apache Spark. The current implementation sends stuff to temp files, and does almost nothing to ensure all modifiable state is actually preserved. The Language() instance is a deep tree of extension objects, and if pickling during training, some of the C-data state is hard to preserve. 2015-10-12 11:33:11 +03:00			`cloudpickle.dump(EN, file_)`

			`file_.seek(0)`

			`loaded = pickle.load(file_)`