Pillow/docs/reference/open_files.rst

103 lines
3.4 KiB
ReStructuredText
Raw Normal View History

.. _file-handling:
2016-11-14 18:00:17 +03:00
File Handling in Pillow
=======================
When opening a file as an image, Pillow requires a filename, ``pathlib.Path``
object, or a file-like object. Pillow uses the filename or ``Path`` to open a
file, so for the rest of this article, they will all be treated as a file-like
object.
2016-11-14 18:00:17 +03:00
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
The following are all equivalent::
2016-11-14 18:00:17 +03:00
from PIL import Image
import io
import pathlib
2018-01-27 09:04:46 +03:00
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
with Image.open('test.jpg') as im:
...
2016-11-14 18:00:17 +03:00
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
with Image.open(pathlib.Path('test.jpg')) as im2:
...
2018-01-27 09:04:46 +03:00
2016-11-14 18:00:17 +03:00
with open('test.jpg', 'rb') as f:
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
im3 = Image.open(f)
...
2016-11-14 18:00:17 +03:00
with open('test.jpg', 'rb') as f:
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
im4 = Image.open(io.BytesIO(f.read()))
...
2016-11-14 18:00:17 +03:00
If a filename or a path-like object is passed to Pillow, then the resulting
2018-12-11 06:39:10 +03:00
file object opened by Pillow may also be closed by Pillow after the
``Image.Image.load()`` method is called, provided the associated image does not
have multiple frames.
2016-11-14 18:00:17 +03:00
Pillow cannot in general close and reopen a file, so any access to
2018-01-27 09:04:46 +03:00
that file needs to be prior to the close.
2016-11-14 18:00:17 +03:00
Image Lifecycle
---------------
* ``Image.open()`` Filenames and ``Path`` objects are opened as a file.
Metadata is read from the open file. The file is left open for further usage.
2016-11-14 18:00:17 +03:00
2018-12-11 06:39:10 +03:00
* ``Image.Image.load()`` When the pixel data from the image is
2016-11-14 18:00:17 +03:00
required, ``load()`` is called. The current frame is read into
memory. The image can now be used independently of the underlying
2018-01-27 09:04:46 +03:00
image file.
2016-11-14 18:00:17 +03:00
If a filename or a ``Path`` object was passed to ``Image.open()``, then the
file object was opened by Pillow and is considered to be used exclusively by
Pillow. So if the image is a single-frame image, the file will be closed in
this method after the frame is read. If the image is a multi-frame image,
(e.g. multipage TIFF and animated GIF) the image file is left open so that
``Image.Image.seek()`` can load the appropriate frame.
2016-11-14 18:00:17 +03:00
* ``Image.Image.close()`` Closes the file and destroys the core image object.
This is used in the Pillow context manager support. e.g.::
2016-11-14 18:00:17 +03:00
with Image.open('test.jpg') as img:
2018-01-27 09:04:46 +03:00
... # image operations here.
2016-11-14 18:00:17 +03:00
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
The lifecycle of a single-frame image is relatively simple. The file must
remain open until the ``load()`` or ``close()`` function is called or the
context manager exits.
2016-11-14 18:00:17 +03:00
Multi-frame images are more complicated. The ``load()`` method is not
2018-12-11 06:39:10 +03:00
a terminal method, so it should not close the underlying file. In general,
Pillow does not know if there are going to be any requests for additional
2018-01-27 09:04:46 +03:00
data until the caller has explicitly closed the image.
2016-11-14 18:00:17 +03:00
Complications
-------------
* ``TiffImagePlugin`` has some code to pass the underlying file descriptor into
libtiff (if working on an actual file). Since libtiff closes the file
descriptor internally, it is duplicated prior to passing it into libtiff.
2016-11-14 18:00:17 +03:00
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
* After a file has been closed, operations that require file access will fail::
2016-11-14 18:00:17 +03:00
with open('test.jpg', 'rb') as f:
im5 = Image.open(f)
im5.load() # FAILS, closed file
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
with Image.open('test.jpg') as im6:
pass
im6.load() # FAILS, closed file
2016-11-14 18:00:17 +03:00
Proposed File Handling
----------------------
* ``Image.Image.load()`` should close the image file, unless there are
multiple frames.
2018-01-27 09:04:46 +03:00
* ``Image.Image.seek()`` should never close the image file.
2016-11-14 18:00:17 +03:00
Improve handling of file resources Follow Python's file object semantics. User code is responsible for closing resources (usually through a context manager) in a deterministic way. To achieve this, remove __del__ functions. These functions used to closed open file handlers in an attempt to silence Python ResourceWarnings. However, using __del__ has the following drawbacks: - __del__ isn't called until the object's reference count reaches 0. Therefore, resource handlers remain open or in use longer than necessary. - The __del__ method isn't guaranteed to execute on system exit. See the Python documentation: https://docs.python.org/3/reference/datamodel.html#object.__del__ > It is not guaranteed that __del__() methods are called for objects > that still exist when the interpreter exits. - Exceptions that occur inside __del__ are ignored instead of raised. This has the potential of hiding bugs. This is also in the Python documentation: > Warning: Due to the precarious circumstances under which __del__() > methods are invoked, exceptions that occur during their execution > are ignored, and a warning is printed to sys.stderr instead. Instead, always close resource handlers when they are no longer in use. This will close the file handler at a specified point in the user's code and not wait until the interpreter chooses to. It is always guaranteed to run. And, if an exception occurs while closing the file handler, the bug will not be ignored. Now, when code receives a ResourceWarning, it will highlight an area that is mishandling resources. It should not simply be silenced, but fixed by closing resources with a context manager. All warnings that were emitted during tests have been cleaned up. To enable warnings, I passed the `-Wa` CLI option to Python. This exposed some mishandling of resources in ImageFile.__init__() and SpiderImagePlugin.loadImageSeries(), they too were fixed.
2019-05-25 19:30:58 +03:00
* Users of the library should use a context manager or call
``Image.Image.close()`` on any image opened with a filename or ``Path``
object to ensure that the underlying file is closed.