python-pillow/Pillow
1
1
Fork 0
mirror of https://github.com/python-pillow/Pillow.git synced 2024-12-25 17:36:18 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into add-cygwin-to-ci

Browse Source
This commit is contained in:
Andrew Murray 2022-01-29 13:59:37 +11:00
commit 179cdd4444
76 changed files with 993 additions and 351 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.ci/after_success.sh
View File

@ -1,7 +1,7 @@
#!/bin/bash
# gather the coverage data
pip3 install codecov
python3 -m pip install codecov
if [[ $MATRIX_DOCKER ]]; then
coverage xml --ignore-errors
else

2
.ci/install.sh
View File

@ -19,7 +19,7 @@ set -e
sudo apt-get -qq install libfreetype6-dev liblcms2-dev python3-tk\
ghostscript libffi-dev libjpeg-turbo-progs libopenjp2-7-dev\
cmake imagemagick libharfbuzz-dev libfribidi-dev
cmake meson imagemagick libharfbuzz-dev libfribidi-dev
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade wheel

1
.github/workflows/test-docker.yml vendored
View File

@ -22,6 +22,7 @@ jobs:
centos-8-amd64,
centos-stream-8-amd64,
debian-10-buster-x86,
debian-11-bullseye-x86,
fedora-34-amd64,
fedora-35-amd64,
ubuntu-18.04-bionic-amd64,

10
.pre-commit-config.yaml
View File

@ -1,6 +1,6 @@
repos:
- repo: https://github.com/psf/black
rev: 911470a610e47d9da5ea938b0887c3df62819b85 # frozen: 21.9b0
rev: f1d4e742c91dd5179d742b0db9293c4472b765f8 # frozen: 21.12b0
hooks:
- id: black
args: ["--target-version", "py37"]
@ -9,12 +9,12 @@ repos:
types: []
- repo: https://github.com/PyCQA/isort
rev: fd5ba70665a37ec301a1f714ed09336048b3be63 # frozen: 5.9.3
rev: c5e8fa75dda5f764d20f66a215d71c21cfa198e1 # frozen: 5.10.1
hooks:
- id: isort
- repo: https://github.com/asottile/yesqa
rev: 644ede78511c02fc6f8e03e014cc1ddcfbf1e1f5 # frozen: v1.2.3
rev: 35cf7dc24fa922927caded7a21b2a8cb04bf8e10 # frozen: v1.3.0
hooks:
- id: yesqa
@ -25,7 +25,7 @@ repos:
exclude: (Makefile$|\.bat$|\.cmake$|\.eps$|\.fits$|\.opt$)
- repo: https://github.com/PyCQA/flake8
rev: dcd740bc0ebaf2b3d43e59a0060d157c97de13f3 # frozen: 3.9.2
rev: cbeb4c9c4137cff1568659fcc48e8b85cddd0c8d # frozen: 4.0.1
hooks:
- id: flake8
additional_dependencies: [flake8-2020, flake8-implicit-str-concat]
@ -37,7 +37,7 @@ repos:
- id: rst-backticks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: 38b88246ccc552bffaaf54259d064beeee434539 # frozen: v4.0.1
rev: 8fe62d14e0b4d7d845a7022c5c2c3ae41bdd3f26 # frozen: v4.1.0
hooks:
- id: check-merge-conflict
- id: check-yaml

65
CHANGES.rst
View File

@ -2,9 +2,72 @@
Changelog (Pillow)
==================
9.0.0 (unreleased)
9.1.0 (unreleased)
------------------
- Raise an error when performing a negative crop #5972
[radarhere, hugovk]
- Deprecated show_file "file" argument in favour of "path" #5959
[radarhere]
- Fixed SPIDER images for use with Bio-formats library #5956
[radarhere]
- Ensure duplicated file pointer is closed #5946
[radarhere]
- Added specific error if ImagePath coordinate type is incorrect #5942
[radarhere]
- Return an empty bytestring from tobytes() for an empty image #5938
[radarhere]
- Remove readonly from Image.__eq__ #5930
[hugovk]
9.0.0 (2022-01-02)
------------------
- Restrict builtins for ImageMath.eval(). CVE-2022-22817 #5923
[radarhere]
- Ensure JpegImagePlugin stops at the end of a truncated file #5921
[radarhere]
- Fixed ImagePath.Path array handling. CVE-2022-22815, CVE-2022-22816 #5920
[radarhere]
- Remove consecutive duplicate tiles that only differ by their offset #5919
[radarhere]
- Improved I;16 operations on big endian #5901
[radarhere]
- Limit quantized palette to number of colors #5879
[radarhere]
- Fixed palette index for zeroed color in FASTOCTREE quantize #5869
[radarhere]
- When saving RGBA to GIF, make use of first transparent palette entry #5859
[radarhere]
- Pass SAMPLEFORMAT to libtiff #5848
[radarhere]
- Added rounding when converting P and PA #5824
[radarhere]
- Improved putdata() documentation and data handling #5910
[radarhere]
- Exclude carriage return in PDF regex to help prevent ReDoS #5912
[hugovk]
- Fixed freeing pointer in ImageDraw.Outline.transform #5909
[radarhere]
- Added ImageShow support for xdg-open #5897
[m-shinder, radarhere]

2
LICENSE
View File

@ -5,7 +5,7 @@ The Python Imaging Library (PIL) is
Pillow is the friendly PIL fork. It is
Copyright © 2010-2021 by Alex Clark and contributors
Copyright © 2010-2022 by Alex Clark and contributors
Like PIL, Pillow is licensed under the open source HPND License:

4
Makefile
View File

@ -105,14 +105,14 @@ test:
.PHONY: valgrind
valgrind:
python3 -c "import pytest_valgrind" || pip3 install pytest-valgrind
python3 -c "import pytest_valgrind" || python3 -m pip install pytest-valgrind
PYTHONMALLOC=malloc valgrind --suppressions=Tests/oss-fuzz/python.supp --leak-check=no \
--log-file=/tmp/valgrind-output \
python3 -m pytest --no-memcheck -vv --valgrind --valgrind-log=/tmp/valgrind-output
.PHONY: readme
readme:
python3 setup.py --long-description | markdown2 > .long-description.html && open .long-description.html
markdown2 README.md > .long-description.html && open .long-description.html
.PHONY: lint

15
README.md
View File

@ -24,16 +24,19 @@ As of 2019, Pillow development is
<tr>
<th>tests</th>
<td>
<a href="https://github.com/python-pillow/Pillow/actions?query=workflow%3ALint"><img
<a href="https://github.com/python-pillow/Pillow/actions/workflows/lint.yml"><img
alt="GitHub Actions build status (Lint)"
src="https://github.com/python-pillow/Pillow/workflows/Lint/badge.svg"></a>
<a href="https://github.com/python-pillow/Pillow/actions?query=workflow%3ATest"><img
<a href="https://github.com/python-pillow/Pillow/actions/workflows/test.yml"><img
alt="GitHub Actions build status (Test Linux and macOS)"
src="https://github.com/python-pillow/Pillow/workflows/Test/badge.svg"></a>
<a href="https://github.com/python-pillow/Pillow/actions?query=workflow%3A%22Test+Windows%22"><img
<a href="https://github.com/python-pillow/Pillow/actions/workflows/test-windows.yml"><img
alt="GitHub Actions build status (Test Windows)"
src="https://github.com/python-pillow/Pillow/workflows/Test%20Windows/badge.svg"></a>
<a href="https://github.com/python-pillow/Pillow/actions?query=workflow%3A%22Test+Docker%22"><img
<a href="https://github.com/python-pillow/Pillow/actions/workflows/test-mingw.yml"><img
alt="GitHub Actions build status (Test MinGW)"
src="https://github.com/python-pillow/Pillow/workflows/Test%20MinGW/badge.svg"></a>
<a href="https://github.com/python-pillow/Pillow/actions/workflows/test-docker.yml"><img
alt="GitHub Actions build status (Test Docker)"
src="https://github.com/python-pillow/Pillow/workflows/Test%20Docker/badge.svg"></a>
<a href="https://ci.appveyor.com/project/python-pillow/Pillow"><img
@ -42,10 +45,10 @@ As of 2019, Pillow development is
<a href="https://github.com/python-pillow/pillow-wheels/actions"><img
alt="GitHub Actions wheels build status (Wheels)"
src="https://github.com/python-pillow/pillow-wheels/workflows/Wheels/badge.svg"></a>
<a href="https://travis-ci.com/github/python-pillow/pillow-wheels"><img
<a href="https://app.travis-ci.com/github/python-pillow/pillow-wheels"><img
alt="Travis CI wheels build status (aarch64)"
src="https://img.shields.io/travis/com/python-pillow/pillow-wheels/main.svg?label=aarch64%20wheels"></a>
<a href="https://codecov.io/gh/python-pillow/Pillow"><img
<a href="https://app.codecov.io/gh/python-pillow/Pillow"><img
alt="Code coverage"
src="https://codecov.io/gh/python-pillow/Pillow/branch/main/graph/badge.svg"></a>
<a href="https://github.com/python-pillow/Pillow/actions/workflows/tidelift.yml"><img

2
Tests/helper.py
View File

@ -30,7 +30,6 @@ if os.environ.get("SHOW_ERRORS", None):
a.show()
b.show()
elif "GITHUB_ACTIONS" in os.environ:
HAS_UPLOADER = True
@ -44,7 +43,6 @@ elif "GITHUB_ACTIONS" in os.environ:
b.save(os.path.join(tmpdir, "b.png"))
return tmpdir
else:
try:
import test_image_results

BIN
Tests/images/pal8_offset.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.0 KiB

BIN
Tests/images/timeout-6646305047838720 Normal file
View File

Binary file not shown.

2
Tests/oss-fuzz/build.sh
View File

@ -22,7 +22,7 @@ for fuzzer in $(find $SRC -name 'fuzz_*.py'); do
fuzzer_basename=$(basename -s .py $fuzzer)
fuzzer_package=${fuzzer_basename}.pkg
pyinstaller \
--add-binary /usr/local/lib/libjpeg.so.9:. \
--add-binary /usr/local/lib/libjpeg.so.62.3.0:. \
--add-binary /usr/local/lib/libfreetype.so.6:. \
--add-binary /usr/local/lib/liblcms2.so.2:. \
--add-binary /usr/local/lib/libopenjp2.so.7:. \

17
Tests/test_decompression_bomb.py
View File

@ -86,21 +86,12 @@ class TestDecompressionCrop:
pytest.warns(Image.DecompressionBombWarning, src.crop, box)
def test_crop_decompression_checks(self):
im = Image.new("RGB", (100, 100))
good_values = ((-9999, -9999, -9990, -9990), (-999, -999, -990, -990))
warning_values = ((-160, -160, 99, 99), (160, 160, -99, -99))
error_values = ((-99909, -99990, 99999, 99999), (99909, 99990, -99999, -99999))
for value in good_values:
for value in ((-9999, -9999, -9990, -9990), (-999, -999, -990, -990)):
assert im.crop(value).size == (9, 9)
for value in warning_values:
pytest.warns(Image.DecompressionBombWarning, im.crop, value)
pytest.warns(Image.DecompressionBombWarning, im.crop, (-160, -160, 99, 99))
for value in error_values:
with pytest.raises(Image.DecompressionBombError):
im.crop(value)
with pytest.raises(Image.DecompressionBombError):
im.crop((-99909, -99990, 99999, 99999))

7
Tests/test_file_bmp.py
View File

@ -123,3 +123,10 @@ def test_rgba_bitfields():
im = Image.merge("RGB", (r, g, b))
assert_image_equal_tofile(im, "Tests/images/bmp/q/rgb32bf-xbgr.bmp")
def test_offset():
# This image has been hexedited
# to exclude the palette size from the pixel data offset
with Image.open("Tests/images/pal8_offset.bmp") as im:
assert_image_equal_tofile(im, "Tests/images/bmp/g/pal8.bmp")

10
Tests/test_file_gif.py
View File

@ -957,3 +957,13 @@ def test_missing_background():
with Image.open("Tests/images/missing_background.gif") as im:
im.seek(1)
assert_image_equal_tofile(im, "Tests/images/missing_background_first_frame.png")
def test_saving_rgba(tmp_path):
out = str(tmp_path / "temp.gif")
with Image.open("Tests/images/transparent.png") as im:
im.save(out)
with Image.open(out) as reloaded:
reloaded_rgba = reloaded.convert("RGBA")
assert reloaded_rgba.load()[0, 0][3] == 0

24
Tests/test_file_jpeg.py
View File

@ -870,6 +870,30 @@ class TestFileJpeg:
with Image.open("Tests/images/hopper.jpg") as im:
assert im.getxmp() == {}
@pytest.mark.timeout(timeout=1)
def test_eof(self):
# Even though this decoder never says that it is finished
# the image should still end when there is no new data
class InfiniteMockPyDecoder(ImageFile.PyDecoder):
def decode(self, buffer):
return 0, 0
decoder = InfiniteMockPyDecoder(None)
def closure(mode, *args):
decoder.__init__(mode, *args)
return decoder
Image.register_decoder("INFINITE", closure)
with Image.open(TEST_FILE) as im:
im.tile = [
("INFINITE", (0, 0, 128, 128), 0, ("RGB", 0, 1)),
]
ImageFile.LOAD_TRUNCATED_IMAGES = True
im.load()
ImageFile.LOAD_TRUNCATED_IMAGES = False
@pytest.mark.skipif(not is_win32(), reason="Windows only")
@skip_unless_feature("jpg")

3
Tests/test_file_jpeg2k.py
View File

@ -10,7 +10,6 @@ from .helper import (
assert_image_equal,
assert_image_similar,
assert_image_similar_tofile,
is_big_endian,
skip_unless_feature,
)
@ -234,13 +233,11 @@ def test_16bit_monochrome_has_correct_mode():
assert jp2.mode == "I;16"
@pytest.mark.xfail(is_big_endian(), reason="Fails on big-endian")
def test_16bit_monochrome_jp2_like_tiff():
with Image.open("Tests/images/16bit.cropped.tif") as tiff_16bit:
assert_image_similar_tofile(tiff_16bit, "Tests/images/16bit.cropped.jp2", 1e-3)
@pytest.mark.xfail(is_big_endian(), reason="Fails on big-endian")
def test_16bit_monochrome_j2k_like_tiff():
with Image.open("Tests/images/16bit.cropped.tif") as tiff_16bit:
assert_image_similar_tofile(tiff_16bit, "Tests/images/16bit.cropped.j2k", 1e-3)

13
Tests/test_file_libtiff.py
View File

@ -9,7 +9,7 @@ from ctypes import c_float
import pytest
from PIL import Image, ImageFilter, TiffImagePlugin, TiffTags, features
from PIL.TiffImagePlugin import STRIPOFFSETS, SUBIFD
from PIL.TiffImagePlugin import SAMPLEFORMAT, STRIPOFFSETS, SUBIFD
from .helper import (
assert_image_equal,
@ -825,6 +825,17 @@ class TestFileLibTiff(LibTiffTestCase):
assert_image_equal_tofile(im, "Tests/images/copyleft.png", mode="RGB")
def test_sampleformat_write(self, tmp_path):
im = Image.new("F", (1, 1))
out = str(tmp_path / "temp.tif")
TiffImagePlugin.WRITE_LIBTIFF = True
im.save(out)
TiffImagePlugin.WRITE_LIBTIFF = False
with Image.open(out) as reloaded:
assert reloaded.mode == "F"
assert reloaded.getexif()[SAMPLEFORMAT] == 3
def test_lzw(self):
with Image.open("Tests/images/hopper_lzw.tif") as im:
assert im.mode == "RGB"

5
Tests/test_file_pdf.py
View File

@ -313,8 +313,9 @@ def test_pdf_append_to_bytesio():
@pytest.mark.timeout(1)
def test_redos():
malicious = b" trailer<<>>" + b"\n" * 3456
@pytest.mark.parametrize("newline", (b"\r", b"\n"))
def test_redos(newline):
malicious = b" trailer<<>>" + newline * 3456
# This particular exception isn't relevant here.
# The important thing is it doesn't timeout, cause a ReDoS (CVE-2021-25292).

2
Tests/test_file_png.py
View File

@ -13,7 +13,6 @@ from .helper import (
assert_image_equal,
assert_image_equal_tofile,
hopper,
is_big_endian,
is_win32,
mark_if_feature_version,
skip_unless_feature,
@ -77,7 +76,6 @@ class TestFilePng:
png.crc(cid, s)
return chunks
@pytest.mark.xfail(is_big_endian(), reason="Fails on big-endian")
def test_sanity(self, tmp_path):
# internal version number

2
Tests/test_file_psd.py
View File

@ -123,7 +123,7 @@ def test_no_icc_profile():
def test_combined_larger_than_size():
# The 'combined' sizes of the individual parts is larger than the
# The combined size of the individual parts is larger than the
# declared 'size' of the extra data field, resulting in a backwards seek.
# If we instead take the 'size' of the extra data field as the source of truth,

10
Tests/test_file_tiff.py
View File

@ -3,7 +3,7 @@ from io import BytesIO
import pytest
from PIL import Image, TiffImagePlugin
from PIL import Image, ImageFile, TiffImagePlugin
from PIL.TiffImagePlugin import RESOLUTION_UNIT, X_RESOLUTION, Y_RESOLUTION
from .helper import (
@ -726,6 +726,14 @@ class TestFileTiff:
with pytest.raises(OSError):
im.load()
@pytest.mark.timeout(6)
@pytest.mark.filterwarnings("ignore:Truncated File Read")
def test_timeout(self):
with Image.open("Tests/images/timeout-6646305047838720") as im:
ImageFile.LOAD_TRUNCATED_IMAGES = True
im.load()
ImageFile.LOAD_TRUNCATED_IMAGES = False
@pytest.mark.skipif(not is_win32(), reason="Windows only")
class TestFileTiffW32:

21
Tests/test_image.py
View File

@ -1,6 +1,7 @@
import io
import os
import shutil
import sys
import tempfile
import pytest
@ -88,6 +89,17 @@ class TestImage:
# with pytest.raises(MemoryError):
# Image.new("L", (1000000, 1000000))
def test_repr_pretty(self):
class Pretty:
def text(self, text):
self.pretty_output = text
im = Image.new("L", (100, 100))
p = Pretty()
im._repr_pretty_(p, None)
assert p.pretty_output == "<PIL.Image.Image image mode=L size=100x100>"
def test_open_formats(self):
PNGFILE = "Tests/images/hopper.png"
JPGFILE = "Tests/images/hopper.jpg"
@ -192,6 +204,10 @@ class TestImage:
assert not im.readonly
@pytest.mark.skipif(is_win32(), reason="Test requires opening tempfile twice")
@pytest.mark.skipif(
sys.platform == "cygwin",
reason="Test requires opening an mmaped file for writing",
)
def test_readonly_save(self, tmp_path):
temp_file = str(tmp_path / "temp.bmp")
shutil.copy("Tests/images/rgb32bf-rgba.bmp", temp_file)
@ -781,6 +797,11 @@ class TestImage:
34665: 196,
}
@pytest.mark.parametrize("size", ((1, 0), (0, 1), (0, 0)))
def test_zero_tobytes(self, size):
im = Image.new("RGB", size)
assert im.tobytes() == b""
def test_categories_deprecation(self):
with pytest.warns(DeprecationWarning):
assert hopper().category == 0

16
Tests/test_image_convert.py
View File

@ -93,7 +93,7 @@ def test_trns_p(tmp_path):
f = str(tmp_path / "temp.png")
im_l = im.convert("L")
assert im_l.info["transparency"] == 0 # undone
assert im_l.info["transparency"] == 1 # undone
im_l.save(f)
im_rgb = im.convert("RGB")
@ -170,6 +170,20 @@ def test_trns_RGB(tmp_path):
im_p.save(f)
@pytest.mark.parametrize("convert_mode", ("L", "LA", "I"))
def test_l_macro_rounding(convert_mode):
for mode in ("P", "PA"):
im = Image.new(mode, (1, 1))
im.palette.getcolor((0, 1, 2))
converted_im = im.convert(convert_mode)
px = converted_im.load()
converted_color = px[0, 0]
if convert_mode == "LA":
converted_color = converted_color[0]
assert converted_color == 1
def test_gif_with_rgba_palette_to_p():
# See https://github.com/python-pillow/Pillow/issues/2433
with Image.open("Tests/images/hopper.gif") as im:

14
Tests/test_image_crop.py
View File

@ -47,16 +47,12 @@ def test_wide_crop():
assert crop(-25, 75, 25, 125) == (1875, 625)
def test_negative_crop():
# Check negative crop size (@PIL171)
@pytest.mark.parametrize("box", ((8, 2, 2, 8), (2, 8, 8, 2), (8, 8, 2, 2)))
def test_negative_crop(box):
im = Image.new("RGB", (10, 10))
im = Image.new("L", (512, 512))
im = im.crop((400, 400, 200, 200))
assert im.size == (0, 0)
assert len(im.getdata()) == 0
with pytest.raises(IndexError):
im.getdata()[0]
with pytest.raises(ValueError):
im.crop(box)
def test_crop_float():

1
Tests/test_image_mode.py
View File

@ -65,6 +65,5 @@ def test_properties():
check("RGB", "RGB", "L", 3, ("R", "G", "B"))
check("RGBA", "RGB", "L", 4, ("R", "G", "B", "A"))
check("RGBX", "RGB", "L", 4, ("R", "G", "B", "X"))
check("RGBX", "RGB", "L", 4, ("R", "G", "B", "X"))
check("CMYK", "RGB", "L", 4, ("C", "M", "Y", "K"))
check("YCbCr", "RGB", "L", 3, ("Y", "Cb", "Cr"))

23
Tests/test_image_putdata.py
View File

@ -1,6 +1,8 @@
import sys
from array import array
import pytest
from PIL import Image
from .helper import assert_image_equal, hopper
@ -47,6 +49,12 @@ def test_pypy_performance():
im.putdata(list(range(256)) * 256)
def test_mode_with_L_with_float():
im = Image.new("L", (1, 1), 0)
im.putdata([2.0])
assert im.getpixel((0, 0)) == 2
def test_mode_i():
src = hopper("L")
data = list(src.getdata())
@ -87,3 +95,18 @@ def test_array_F():
im.putdata(arr)
assert len(im.getdata()) == len(arr)
def test_not_flattened():
im = Image.new("L", (1, 1))
with pytest.raises(TypeError):
im.putdata([[0]])
with pytest.raises(TypeError):
im.putdata([[0]], 2)
with pytest.raises(TypeError):
im = Image.new("I", (1, 1))
im.putdata([[0]])
with pytest.raises(TypeError):
im = Image.new("F", (1, 1))
im.putdata([[0]])

24
Tests/test_image_quantize.py
View File

@ -77,6 +77,13 @@ def test_quantize_dither_diff():
assert dither.tobytes() != nodither.tobytes()
def test_colors():
im = hopper()
colors = 2
converted = im.quantize(colors)
assert len(converted.palette.palette) == colors * len("RGB")
def test_transparent_colors_equal():
im = Image.new("RGBA", (1, 2), (0, 0, 0, 0))
px = im.load()
@ -85,3 +92,20 @@ def test_transparent_colors_equal():
converted = im.quantize()
converted_px = converted.load()
assert converted_px[0, 0] == converted_px[0, 1]
@pytest.mark.parametrize(
"method, color",
(
(Image.MEDIANCUT, (0, 0, 0)),
(Image.MAXCOVERAGE, (0, 0, 0)),
(Image.FASTOCTREE, (0, 0, 0)),
(Image.FASTOCTREE, (0, 0, 0, 0)),
),
)
def test_palette(method, color):
im = Image.new("RGBA" if len(color) == 4 else "RGB", (1, 1), color)
converted = im.quantize(method=method)
converted_px = converted.load()
assert converted_px[0, 0] == converted.palette.colors[color]

17
Tests/test_imagedraw.py
View File

@ -467,6 +467,23 @@ def test_shape2():
assert_image_equal_tofile(im, "Tests/images/imagedraw_shape2.png")
def test_transform():
# Arrange
im = Image.new("RGB", (100, 100), "white")
expected = im.copy()
draw = ImageDraw.Draw(im)
# Act
s = ImageDraw.Outline()
s.line(0, 0)
s.transform((0, 0, 0, 0, 0, 0))
draw.shape(s, fill=1)
# Assert
assert_image_equal(im, expected)
def helper_pieslice(bbox, start, end):
# Arrange
im = Image.new("RGB", (W, H))

13
Tests/test_imagefile.py
View File

@ -82,6 +82,19 @@ class TestImageFile:
p.feed(data)
assert (48, 48) == p.image.size
@skip_unless_feature("webp")
@skip_unless_feature("webp_anim")
def test_incremental_webp(self):
with ImageFile.Parser() as p:
with open("Tests/images/hopper.webp", "rb") as f:
p.feed(f.read(1024))
# Check that insufficient data was given in the first feed
assert not p.image
p.feed(f.read())
assert (128, 128) == p.image.size
@skip_unless_feature("zlib")
def test_safeblock(self):
im1 = hopper()

7
Tests/test_imagemath.py
View File

@ -1,3 +1,5 @@
import pytest
from PIL import Image, ImageMath
@ -50,6 +52,11 @@ def test_ops():
assert pixel(ImageMath.eval("float(B)**33", images)) == "F 8589934592.0"
def test_prevent_exec():
with pytest.raises(ValueError):
ImageMath.eval("exec('pass')")
def test_logical():
assert pixel(ImageMath.eval("not A", images)) == 0
assert pixel(ImageMath.eval("A and B", images)) == "L 2"

6
Tests/test_imagepath.py
View File

@ -70,9 +70,11 @@ def test_invalid_coords():
coords = ["a", "b"]
# Act / Assert
with pytest.raises(SystemError):
with pytest.raises(ValueError) as e:
ImagePath.Path(coords)
assert str(e.value) == "incorrect coordinate type"
def test_path_odd_number_of_coordinates():
# Arrange
@ -90,6 +92,8 @@ def test_path_odd_number_of_coordinates():
[
([0, 1, 2, 3], (0.0, 1.0, 2.0, 3.0)),
([3, 2, 1, 0], (1.0, 0.0, 3.0, 2.0)),
(0, (0.0, 0.0, 0.0, 0.0)),
(1, (0.0, 0.0, 0.0, 0.0)),
],
)
def test_getbbox(coords, expected):

15
Tests/test_imageshow.py
View File

@ -79,3 +79,18 @@ def test_ipythonviewer():
im = hopper()
assert test_viewer.show(im) == 1
@pytest.mark.skipif(
not on_ci() or is_win32(),
reason="Only run on CIs; hangs on Windows CIs",
)
def test_file_deprecated():
for viewer in ImageShow._viewers:
with pytest.warns(DeprecationWarning):
try:
viewer.show_file(file="test.jpg")
except NotImplementedError:
pass
with pytest.raises(TypeError):
viewer.show_file()

4
depends/install_raqm.sh
View File

@ -2,13 +2,13 @@
# install raqm
archive=raqm-0.7.1
archive=libraqm-0.8.0
./download-and-extract.sh $archive https://raw.githubusercontent.com/python-pillow/pillow-depends/main/$archive.tar.gz
pushd $archive
./configure --prefix=/usr && make -j4 && sudo make -j4 install
meson build --prefix=/usr && sudo ninja -C build install
popd

2
depends/install_webp.sh
View File

@ -1,7 +1,7 @@
#!/bin/bash
# install webp
archive=libwebp-1.2.1
archive=libwebp-1.2.2
./download-and-extract.sh $archive https://raw.githubusercontent.com/python-pillow/pillow-depends/main/$archive.tar.gz

2
docs/COPYING
View File

@ -5,7 +5,7 @@ The Python Imaging Library (PIL) is
Pillow is the friendly PIL fork. It is
Copyright © 2010-2021 by Alex Clark and contributors
Copyright © 2010-2022 by Alex Clark and contributors
Like PIL, Pillow is licensed under the open source PIL
Software License:

2
docs/about.rst
View File

@ -12,7 +12,7 @@ The fork author's goal is to foster and support active development of PIL throug
.. _GitHub Actions: https://github.com/python-pillow/Pillow/actions
.. _AppVeyor: https://ci.appveyor.com/project/Python-pillow/pillow
.. _Travis CI: https://travis-ci.com/github/python-pillow/pillow-wheels
.. _Travis CI: https://app.travis-ci.com/github/python-pillow/pillow-wheels
.. _GitHub: https://github.com/python-pillow/Pillow
.. _Python Package Index: https://pypi.org/project/Pillow/

2
docs/conf.py
View File

@ -53,7 +53,7 @@ master_doc = "index"
# General information about the project.
project = "Pillow (PIL Fork)"
copyright = "1995-2011 Fredrik Lundh, 2010-2021 Alex Clark and Contributors"
copyright = "1995-2011 Fredrik Lundh, 2010-2022 Alex Clark and Contributors"
author = "Fredrik Lundh, Alex Clark and Contributors"
# The version info for the project you're documenting, acts as replacement for

13
docs/deprecations.rst
View File

@ -53,6 +53,19 @@ Before Pillow 8.3.0, ``ImagePalette`` required palette data of particular length
default, and the size parameter could be used to override that. Pillow 8.3.0 removed
the default required length, also removing the need for the size parameter.
ImageShow.Viewer.show_file file argument
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 9.1.0
The ``file`` argument in :py:meth:`~PIL.ImageShow.Viewer.show_file()` has been
deprecated and will be removed in Pillow 10.0.0 (2023-07-01). It has been replaced by
``path``.
In effect, ``viewer.show_file("test.jpg")`` will continue to work unchanged.
``viewer.show_file(file="test.jpg")`` will raise a deprecation warning, and suggest
``viewer.show_file(path="test.jpg")`` instead.
Removed features
----------------

16
docs/index.rst
View File

@ -10,21 +10,25 @@ Pillow for enterprise is available via the Tidelift Subscription. `Learn more <h
:alt: Documentation Status
.. image:: https://github.com/python-pillow/Pillow/workflows/Lint/badge.svg
:target: https://github.com/python-pillow/Pillow/actions?query=workflow%3ALint
:target: https://github.com/python-pillow/Pillow/actions/workflows/lint.yml
:alt: GitHub Actions build status (Lint)
.. image:: https://github.com/python-pillow/Pillow/workflows/Test%20Docker/badge.svg
:target: https://github.com/python-pillow/Pillow/actions?query=workflow%3A%22Test+Docker%22
:target: https://github.com/python-pillow/Pillow/actions/workflows/test-docker.yml
:alt: GitHub Actions build status (Test Docker)
.. image:: https://github.com/python-pillow/Pillow/workflows/Test/badge.svg
:target: https://github.com/python-pillow/Pillow/actions?query=workflow%3ATest
:target: https://github.com/python-pillow/Pillow/actions/workflows/test.yml
:alt: GitHub Actions build status (Test Linux and macOS)
.. image:: https://github.com/python-pillow/Pillow/workflows/Test%20Windows/badge.svg
:target: https://github.com/python-pillow/Pillow/actions?query=workflow%3A%22Test+Windows%22
:target: https://github.com/python-pillow/Pillow/actions/workflows/test-windows.yml
:alt: GitHub Actions build status (Test Windows)
.. image:: https://github.com/python-pillow/Pillow/workflows/Test%20MinGW/badge.svg
:target: https://github.com/python-pillow/Pillow/actions/workflows/test-mingw.yml
:alt: GitHub Actions build status (Test MinGW)
.. image:: https://img.shields.io/appveyor/build/python-pillow/Pillow/main.svg?label=Windows%20build
:target: https://ci.appveyor.com/project/python-pillow/Pillow
:alt: AppVeyor CI build status (Windows)
@ -34,11 +38,11 @@ Pillow for enterprise is available via the Tidelift Subscription. `Learn more <h
:alt: GitHub Actions wheels build status (Wheels)
.. image:: https://img.shields.io/travis/com/python-pillow/pillow-wheels/main.svg?label=aarch64%20wheels
:target: https://travis-ci.com/github/python-pillow/pillow-wheels
:target: https://app.travis-ci.com/github/python-pillow/pillow-wheels
:alt: Travis CI wheels build status (aarch64)
.. image:: https://codecov.io/gh/python-pillow/Pillow/branch/main/graph/badge.svg
:target: https://codecov.io/gh/python-pillow/Pillow
:target: https://app.codecov.io/gh/python-pillow/Pillow
:alt: Code coverage
.. image:: https://zenodo.org/badge/17549/python-pillow/Pillow.svg

13
docs/installation.rst
View File

@ -394,7 +394,8 @@ Prerequisites for **Ubuntu 16.04 LTS - 20.04 LTS** are installed with::
libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \
libharfbuzz-dev libfribidi-dev libxcb1-dev
Then see ``depends/install_raqm.sh`` to install libraqm.
To install libraqm, ``sudo apt-get install meson`` and then see
``depends/install_raqm.sh``.
Prerequisites are installed on recent **Red Hat**, **CentOS** or **Fedora** with::
@ -458,6 +459,8 @@ These platforms are built and tested for every change.
+----------------------------------+----------------------------+---------------------+
| Debian 10 Buster | 3.7 | x86 |
+----------------------------------+----------------------------+---------------------+
| Debian 11 Bullseye | 3.9 | x86 |
+----------------------------------+----------------------------+---------------------+
| Fedora 34 | 3.9 | x86-64 |
+----------------------------------+----------------------------+---------------------+
| Fedora 35 | 3.10 | x86-64 |
@ -495,9 +498,13 @@ These platforms have been reported to work at the versions mentioned.
| Operating system | | Tested Python | | Latest tested | | Tested |
| | | versions | | Pillow version | | processors |
+==================================+===========================+==================+==============+
| macOS 11.0 Big Sur | 3.7, 3.8, 3.9, 3.10 | 8.4.0 |arm |
| macOS 12 Big Sur | 3.7, 3.8, 3.9, 3.10 | 9.0.0 |arm |
+----------------------------------+---------------------------+------------------+--------------+
| macOS 11 Big Sur | 3.7, 3.8, 3.9, 3.10 | 8.4.0 |arm |
| +---------------------------+------------------+--------------+
| | 3.6, 3.7, 3.8, 3.9, 3.10 | 8.4.0 |x86-64 |
| | 3.7, 3.8, 3.9, 3.10 | 9.0.0 |x86-64 |
| +---------------------------+------------------+--------------+
| | 3.6 | 8.4.0 |x86-64 |
+----------------------------------+---------------------------+------------------+--------------+
| macOS 10.15 Catalina | 3.6, 3.7, 3.8, 3.9 | 8.3.2 |x86-64 |
| +---------------------------+------------------+ |

39
docs/releasenotes/9.0.0.rst
View File

@ -100,10 +100,37 @@ argument will also now be supported, e.g. ``im.show(title="My Image")`` and
Security
========
TODO
^^^^
Ensure JpegImagePlugin stops at the end of a truncated file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TODO
``JpegImagePlugin`` may append an EOF marker to the end of a truncated file, so that
the last segment of the data will still be processed by the decoder.
If the EOF marker is not detected as such however, this could lead to an infinite
loop where ``JpegImagePlugin`` keeps trying to end the file.
Remove consecutive duplicate tiles that only differ by their offset
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To prevent attempts to slow down loading times for images, if an image has consecutive
duplicate tiles that only differ by their offset, only load the last tile. Credit to
Google's `OSS-Fuzz`_ project for finding this issue.
Restrict builtins available to ImageMath.eval
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
:cve:`CVE-2022-22817`: To limit :py:class:`PIL.ImageMath` to working with images, Pillow
will now restrict the builtins available to :py:meth:`PIL.ImageMath.eval`. This will
help prevent problems arising if users evaluate arbitrary expressions, such as
``ImageMath.eval("exec(exit())")``.
Fixed ImagePath.Path array handling
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
:cve:`CVE-2022-22815` (:cwe:`CWE-126`) and :cve:`CVE-2022-22816` (:cwe:`CWE-665`) were
found when initializing ``ImagePath.Path``.
.. _OSS-Fuzz: https://github.com/google/oss-fuzz
Other Changes
=============
@ -116,6 +143,12 @@ possible for there to be too many colors to fit in a P mode image. To allow for
seeking to any subsequent GIF frame will now convert the image to RGB or RGBA,
depending on whether or not the first frame had transparency.
Switched to libjpeg-turbo in macOS and Linux wheels
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The Pillow wheels from PyPI for macOS and Linux have switched from libjpeg to
libjpeg-turbo. It is a fork of libjpeg, popular for its speed.
Added support for pickling TrueType fonts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2
requirements.txt
View File

@ -12,7 +12,7 @@ pytest-cov
pytest-timeout
sphinx>=2.4
sphinx-copybutton
sphinx-issues
sphinx-issues>=3.0.1
sphinx-removed-in
sphinx-rtd-theme>=1.0
sphinxext-opengraph

4
setup.py
View File

@ -185,7 +185,7 @@ def _find_library_dirs_ldconfig():
return []
[data, _] = p.communicate()
if isinstance(data, bytes):
data = data.decode()
data = data.decode("latin1")
dirs = []
for dll in re.findall(expr, data):
@ -898,7 +898,7 @@ class pil_build_ext(build_ext):
else:
self._remove_extension("PIL._webp")
tk_libs = ["psapi"] if sys.platform == "win32" else []
tk_libs = ["psapi"] if sys.platform in ("win32", "cygwin") else []
self._update_extension("PIL._imagingtk", tk_libs)
build_ext.build_extensions(self)

2
src/PIL/BmpImagePlugin.py
View File

@ -158,6 +158,8 @@ class BmpImageFile(ImageFile.ImageFile):
if file_info.get("colors", 0)
else (1 << file_info["bits"])
)
if offset == 14 + file_info["header_size"] and file_info["bits"] <= 8:
offset += 4 * file_info["colors"]
# ---------------------- Check bit depth for unusual unsupported values
self.mode, raw_mode = BIT2MODE.get(file_info["bits"], (None, None))

8
src/PIL/GifImagePlugin.py
View File

@ -425,7 +425,13 @@ def _normalize_mode(im, initial_call=False):
palette_size = 256
if im.palette:
palette_size = len(im.palette.getdata()[1]) // 3
return im.convert("P", palette=Image.ADAPTIVE, colors=palette_size)
im = im.convert("P", palette=Image.ADAPTIVE, colors=palette_size)
if im.palette.mode == "RGBA":
for rgba in im.palette.colors.keys():
if rgba[3] == 0:
im.info["transparency"] = im.palette.colors[rgba]
break
return im
else:
return im.convert("P")
return im.convert("L")

45
src/PIL/Image.py
View File

@ -138,8 +138,6 @@ def isImageType(t):
#
# Constants
NONE = 0
# transpose
FLIP_LEFT_RIGHT = 0
FLIP_TOP_BOTTOM = 1
@ -629,7 +627,6 @@ class Image:
and self.size == other.size
and self.info == other.info
and self._category == other._category
and self.readonly == other.readonly
and self.getpalette() == other.getpalette()
and self.tobytes() == other.tobytes()
)
@ -644,6 +641,22 @@ class Image:
id(self),
)
def _repr_pretty_(self, p, cycle):
"""IPython plain text display support"""
# Same as __repr__ but without unpredicatable id(self),
# to keep Jupyter notebook `text/plain` output stable.
p.text(
"<%s.%s image mode=%s size=%dx%d>"
% (
self.__class__.__module__,
self.__class__.__name__,
self.mode,
self.size[0],
self.size[1],
)
)
def _repr_png_(self):
"""iPython display hook support
@ -719,6 +732,9 @@ class Image:
self.load()
if self.width == 0 or self.height == 0:
return b""
# unpack data
e = _getencoder(self.mode, encoder_name, args)
e.setimage(self.im)
@ -814,7 +830,7 @@ class Image:
palette_length = self.im.putpalette(mode, arr)
self.palette.dirty = 0
self.palette.rawmode = None
if "transparency" in self.info and mode in ("RGBA", "LA", "PA"):
if "transparency" in self.info and mode in ("LA", "PA"):
if isinstance(self.info["transparency"], int):
self.im.putpalettealpha(self.info["transparency"], 0)
else:
@ -1111,7 +1127,8 @@ class Image:
from . import ImagePalette
mode = im.im.getpalettemode()
im.palette = ImagePalette.ImagePalette(mode, im.im.getpalette(mode, mode))
palette = im.im.getpalette(mode, mode)[: colors * len(mode)]
im.palette = ImagePalette.ImagePalette(mode, palette)
return im
@ -1144,6 +1161,11 @@ class Image:
if box is None:
return self.copy()
if box[2] < box[0]:
raise ValueError("Coordinate 'right' is less than 'left'")
elif box[3] < box[1]:
raise ValueError("Coordinate 'lower' is less than 'upper'")
self.load()
return self._new(self._crop(self.im, box))
@ -1707,13 +1729,14 @@ class Image:
def putdata(self, data, scale=1.0, offset=0.0):
"""
Copies pixel data to this image. This method copies data from a
sequence object into the image, starting at the upper left
corner (0, 0), and continuing until either the image or the
sequence ends. The scale and offset values are used to adjust
the sequence values: **pixel = value*scale + offset**.
Copies pixel data from a flattened sequence object into the image. The
values should start at the upper left corner (0, 0), continue to the
end of the line, followed directly by the first value of the second
line, and so on. Data will be read until either the image or the
sequence ends. The scale and offset values are used to adjust the
sequence values: **pixel = value*scale + offset**.
:param data: A sequence object.
:param data: A flattened sequence object.
:param scale: An optional scale value. The default is 1.0.
:param offset: An optional offset value. The default is 0.0.
"""

8
src/PIL/ImageFile.py
View File

@ -28,6 +28,7 @@
#
import io
import itertools
import struct
import sys
@ -210,6 +211,13 @@ class ImageFile(Image.Image):
except AttributeError:
prefix = b""
# Remove consecutive duplicates that only differ by their offset
self.tile = [
list(tiles)[-1]
for _, tiles in itertools.groupby(
self.tile, lambda tile: (tile[0], tile[1], tile[3])
)
]
for decoder_name, extents, offset, args in self.tile:
decoder = Image._getdecoder(
self.mode, decoder_name, args, self.decoderconfig

15
src/PIL/ImageMath.py
View File

@ -19,8 +19,6 @@ import builtins
from . import Image, _imagingmath
VERBOSE = 0
def _isconstant(v):
return isinstance(v, (int, float))
@ -69,8 +67,6 @@ class _Operand:
im1 = im1.convert("F")
if im2.mode != "F":
im2 = im2.convert("F")
if im1.mode != im2.mode:
raise ValueError("mode mismatch")
if im1.size != im2.size:
# crop both arguments to a common size
size = (min(im1.size[0], im2.size[0]), min(im1.size[1], im2.size[1]))
@ -78,9 +74,7 @@ class _Operand:
im1 = im1.crop((0, 0) + size)
if im2.size != size:
im2 = im2.crop((0, 0) + size)
out = Image.new(mode or im1.mode, size, None)
else:
out = Image.new(mode or im1.mode, im1.size, None)
out = Image.new(mode or im1.mode, im1.size, None)
im1.load()
im2.load()
try:
@ -246,7 +240,12 @@ def eval(expression, _dict={}, **kw):
if hasattr(v, "im"):
args[k] = _Operand(v)
out = builtins.eval(expression, args)
code = compile(expression, "<string>", "eval")
for name in code.co_names:
if name not in args and name != "abs":
raise ValueError(f"'{name}' not allowed")
out = builtins.eval(expression, {"__builtins": {"abs": abs}}, args)
try:
return out.im
except AttributeError:

4
src/PIL/ImageMode.py
View File

@ -52,6 +52,10 @@ def getmode(mode):
"HSV": ("RGB", "L", ("H", "S", "V")),
# extra experimental modes
"RGBa": ("RGB", "L", ("R", "G", "B", "a")),
"BGR;15": ("RGB", "L", ("B", "G", "R")),
"BGR;16": ("RGB", "L", ("B", "G", "R")),
"BGR;24": ("RGB", "L", ("B", "G", "R")),
"BGR;32": ("RGB", "L", ("B", "G", "R")),
"LA": ("L", "L", ("L", "A")),
"La": ("L", "L", ("L", "a")),
"PA": ("RGB", "L", ("P", "A")),

81
src/PIL/ImageShow.py
View File

@ -16,6 +16,7 @@ import shutil
import subprocess
import sys
import tempfile
import warnings
from shlex import quote
from PIL import Image
@ -105,9 +106,25 @@ class Viewer:
"""Display the given image."""
return self.show_file(self.save_image(image), **options)
def show_file(self, file, **options):
"""Display the given file."""
os.system(self.get_command(file, **options))
def show_file(self, path=None, **options):
"""
Display given file.
Before Pillow 9.1.0, the first argument was ``file``. This is now deprecated,
and will be removed in Pillow 10.0.0 (2023-07-01). ``path`` should be used
instead.
"""
if path is None:
if "file" in options:
warnings.warn(
"The 'file' argument is deprecated and will be removed in Pillow "
"10 (2023-07-01). Use 'path' instead.",
DeprecationWarning,
)
path = options.pop("file")
else:
raise TypeError("Missing required argument: 'path'")
os.system(self.get_command(path, **options))
return 1
@ -145,18 +162,34 @@ class MacViewer(Viewer):
command = f"({command} {quote(file)}; sleep 20; rm -f {quote(file)})&"
return command
def show_file(self, file, **options):
"""Display given file"""
fd, path = tempfile.mkstemp()
def show_file(self, path=None, **options):
"""
Display given file.
Before Pillow 9.1.0, the first argument was ``file``. This is now deprecated,
and will be removed in Pillow 10.0.0 (2023-07-01). ``path`` should be used
instead.
"""
if path is None:
if "file" in options:
warnings.warn(
"The 'file' argument is deprecated and will be removed in Pillow "
"10 (2023-07-01). Use 'path' instead.",
DeprecationWarning,
)
path = options.pop("file")
else:
raise TypeError("Missing required argument: 'path'")
fd, temp_path = tempfile.mkstemp()
with os.fdopen(fd, "w") as f:
f.write(file)
with open(path) as f:
f.write(path)
with open(temp_path) as f:
subprocess.Popen(
["im=$(cat); open -a Preview.app $im; sleep 20; rm -f $im"],
shell=True,
stdin=f,
)
os.remove(path)
os.remove(temp_path)
return 1
@ -172,17 +205,33 @@ class UnixViewer(Viewer):
command = self.get_command_ex(file, **options)[0]
return f"({command} {quote(file)}; rm -f {quote(file)})&"
def show_file(self, file, **options):
"""Display given file"""
fd, path = tempfile.mkstemp()
def show_file(self, path=None, **options):
"""
Display given file.
Before Pillow 9.1.0, the first argument was ``file``. This is now deprecated,
and will be removed in Pillow 10.0.0 (2023-07-01). ``path`` should be used
instead.
"""
if path is None:
if "file" in options:
warnings.warn(
"The 'file' argument is deprecated and will be removed in Pillow "
"10 (2023-07-01). Use 'path' instead.",
DeprecationWarning,
)
path = options.pop("file")
else:
raise TypeError("Missing required argument: 'path'")
fd, temp_path = tempfile.mkstemp()
with os.fdopen(fd, "w") as f:
f.write(file)
with open(path) as f:
command = self.get_command_ex(file, **options)[0]
f.write(path)
with open(temp_path) as f:
command = self.get_command_ex(path, **options)[0]
subprocess.Popen(
["im=$(cat);" + command + " $im; rm -f $im"], shell=True, stdin=f
)
os.remove(path)
os.remove(temp_path)
return 1

3
src/PIL/JpegImagePlugin.py
View File

@ -401,9 +401,10 @@ class JpegImageFile(ImageFile.ImageFile):
"""
s = self.fp.read(read_bytes)
if not s and ImageFile.LOAD_TRUNCATED_IMAGES:
if not s and ImageFile.LOAD_TRUNCATED_IMAGES and not hasattr(self, "_ended"):
# Premature EOF.
# Pretend file is finished adding EOI marker
self._ended = True
return b"\xFF\xD9"
return s

3
src/PIL/PdfParser.py
View File

@ -582,7 +582,8 @@ class PdfParser:
whitespace_or_hex = br"[\000\011\012\014\015\0400-9a-fA-F]"
whitespace_optional = whitespace + b"*"
whitespace_mandatory = whitespace + b"+"
whitespace_optional_no_nl = br"[\000\011\014\015\040]*" # no "\012" aka "\n"
# No "\012" aka "\n" or "\015" aka "\r":
whitespace_optional_no_nl = br"[\000\011\014\040]*"
newline_only = br"[\r\n]+"
newline = whitespace_optional_no_nl + newline_only + whitespace_optional_no_nl
re_trailer_end = re.compile(

8
src/PIL/PsdImagePlugin.py
View File

@ -195,7 +195,6 @@ def _layerinfo(fp, ct_bytes):
x1 = i32(read(4))
# image info
info = []
mode = []
ct_types = i16(read(2))
types = list(range(ct_types))
@ -211,8 +210,7 @@ def _layerinfo(fp, ct_bytes):
m = "RGBA"[type]
mode.append(m)
size = i32(read(4))
info.append((m, size))
read(4) # size
# figure out the image mode
mode.sort()
@ -229,26 +227,22 @@ def _layerinfo(fp, ct_bytes):
read(12) # filler
name = ""
size = i32(read(4)) # length of the extra data field
combined = 0
if size:
data_end = fp.tell() + size
length = i32(read(4))
if length:
fp.seek(length - 16, io.SEEK_CUR)
combined += length + 4
length = i32(read(4))
if length:
fp.seek(length, io.SEEK_CUR)
combined += length + 4
length = i8(read(1))
if length:
# Don't know the proper encoding,
# Latin-1 should be a good guess
name = read(length).decode("latin-1", "replace")
combined += length + 1
fp.seek(data_end)
layers.append((name, mode, (x0, y0, x1, y1)))

14
src/PIL/SpiderImagePlugin.py
View File

@ -238,17 +238,18 @@ def makeSpiderHeader(im):
if 1024 % lenbyt != 0:
labrec += 1
labbyt = labrec * lenbyt
hdr = []
nvalues = int(labbyt / 4)
if nvalues < 23:
return []
hdr = []
for i in range(nvalues):
hdr.append(0.0)
if len(hdr) < 23:
return []
# NB these are Fortran indices
hdr[1] = 1.0 # nslice (=1 for an image)
hdr[2] = float(nrow) # number of rows per slice
hdr[3] = float(nrow) # number of records in the image
hdr[5] = 1.0 # iform for 2D image
hdr[12] = float(nsam) # number of pixels per line
hdr[13] = float(labrec) # number of records in file header
@ -259,10 +260,7 @@ def makeSpiderHeader(im):
hdr = hdr[1:]
hdr.append(0.0)
# pack binary data into a string
hdrstr = []
for v in hdr:
hdrstr.append(struct.pack("f", v))
return hdrstr
return [struct.pack("f", v) for v in hdr]
def _save(im, fp, filename):

23
src/PIL/TiffImagePlugin.py
View File

@ -1234,6 +1234,12 @@ class TiffImageFile(ImageFile.ImageFile):
# UNDONE -- so much for that buffer size thing.
n, err = decoder.decode(self.fp.read())
if fp:
try:
os.close(fp)
except OSError:
pass
self.tile = []
self.readonly = 0
@ -1676,8 +1682,6 @@ def _save(im, fp, filename):
# optional types for non core tags
types = {}
# SAMPLEFORMAT is determined by the image format and should not be copied
# from legacy_ifd.
# STRIPOFFSETS and STRIPBYTECOUNTS are added by the library
# based on the data in the strip.
# The other tags expect arrays with a certain length (fixed or depending on
@ -1686,7 +1690,6 @@ def _save(im, fp, filename):
# SUBIFD may also cause a segfault.
blocklist += [
REFERENCEBLACKWHITE,
SAMPLEFORMAT,
STRIPBYTECOUNTS,
STRIPOFFSETS,
TRANSFERFUNCTION,
@ -1702,9 +1705,14 @@ def _save(im, fp, filename):
legacy_ifd = {}
if hasattr(im, "tag"):
legacy_ifd = im.tag.to_v2()
for tag, value in itertools.chain(
ifd.items(), getattr(im, "tag_v2", {}).items(), legacy_ifd.items()
):
# SAMPLEFORMAT is determined by the image format and should not be copied
# from legacy_ifd.
supplied_tags = {**getattr(im, "tag_v2", {}), **legacy_ifd}
if SAMPLEFORMAT in supplied_tags:
del supplied_tags[SAMPLEFORMAT]
for tag, value in itertools.chain(ifd.items(), supplied_tags.items()):
# Libtiff can only process certain core items without adding
# them to the custom dictionary.
# Custom items are supported for int, float, unicode, string and byte
@ -1729,6 +1737,9 @@ def _save(im, fp, filename):
else:
atts[tag] = value
if SAMPLEFORMAT in atts and len(atts[SAMPLEFORMAT]) == 1:
atts[SAMPLEFORMAT] = atts[SAMPLEFORMAT][0]
logger.debug("Converted items: %s" % sorted(atts.items()))
# libtiff always expects the bytes in native order.

2
src/PIL/_version.py
View File

@ -1,2 +1,2 @@
# Master version for Pillow
__version__ = "9.0.0.dev0"
__version__ = "9.1.0.dev0"

6
src/Tk/tkImaging.c
View File

@ -29,7 +29,7 @@
* 1995-09-12 fl Created
* 1996-04-08 fl Ready for release
* 1997-05-09 fl Use command instead of image type
* 2001-03-18 fl Initialize alpha layer pointer (struct changed in 8.3)
* 2001-03-18 fl Initialize alpha layer pointer (struct changed in Tk 8.3)
* 2003-04-23 fl Fixed building for Tk 8.4.1 and later (Jack Jansen)
* 2004-06-24 fl Fixed building for Tk 8.4.6 and later.
*
@ -116,7 +116,7 @@ PyImagingPhotoPut(
block.offset[1] = 1;
block.offset[2] = 2;
if (strcmp(im->mode, "RGBA") == 0) {
block.offset[3] = 3; /* alpha (or reserved, under 8.2) */
block.offset[3] = 3; /* alpha (or reserved, under Tk 8.2) */
} else {
block.offset[3] = 0; /* no alpha */
}
@ -219,7 +219,7 @@ TkImaging_Init(Tcl_Interp *interp) {
#define TKINTER_FINDER "PIL._tkinter_finder"
#if defined(_WIN32) || defined(__WIN32__) || defined(WIN32)
#if defined(_WIN32) || defined(__WIN32__) || defined(WIN32) || defined(__CYGWIN__)
/*
* On Windows, we can't load the tkinter module to get the Tcl or Tk symbols,

28
src/_imaging.c
View File

@ -1494,6 +1494,14 @@ _putdata(ImagingObject *self, PyObject *args) {
return NULL;
}
#define set_value_to_item(seq, i) \
op = PySequence_Fast_GET_ITEM(seq, i); \
if (PySequence_Check(op)) { \
PyErr_SetString(PyExc_TypeError, "sequence must be flattened"); \
return NULL; \
} else { \
value = PyFloat_AsDouble(op); \
}
if (image->image8) {
if (PyBytes_Check(data)) {
unsigned char *p;
@ -1522,11 +1530,12 @@ _putdata(ImagingObject *self, PyObject *args) {
PyErr_SetString(PyExc_TypeError, must_be_sequence);
return NULL;
}
double value;
if (scale == 1.0 && offset == 0.0) {
/* Clipped data */
for (i = x = y = 0; i < n; i++) {
op = PySequence_Fast_GET_ITEM(seq, i);
image->image8[y][x] = (UINT8)CLIP8(PyLong_AsLong(op));
set_value_to_item(seq, i);
image->image8[y][x] = (UINT8)CLIP8(value);
if (++x >= (int)image->xsize) {
x = 0, y++;
}
@ -1535,9 +1544,8 @@ _putdata(ImagingObject *self, PyObject *args) {
} else {
/* Scaled and clipped data */
for (i = x = y = 0; i < n; i++) {
PyObject *op = PySequence_Fast_GET_ITEM(seq, i);
image->image8[y][x] =
CLIP8((int)(PyFloat_AsDouble(op) * scale + offset));
set_value_to_item(seq, i);
image->image8[y][x] = CLIP8(value * scale + offset);
if (++x >= (int)image->xsize) {
x = 0, y++;
}
@ -1555,9 +1563,10 @@ _putdata(ImagingObject *self, PyObject *args) {
switch (image->type) {
case IMAGING_TYPE_INT32:
for (i = x = y = 0; i < n; i++) {
op = PySequence_Fast_GET_ITEM(seq, i);
double value;
set_value_to_item(seq, i);
IMAGING_PIXEL_INT32(image, x, y) =
(INT32)(PyFloat_AsDouble(op) * scale + offset);
(INT32)(value * scale + offset);
if (++x >= (int)image->xsize) {
x = 0, y++;
}
@ -1566,9 +1575,10 @@ _putdata(ImagingObject *self, PyObject *args) {
break;
case IMAGING_TYPE_FLOAT32:
for (i = x = y = 0; i < n; i++) {
op = PySequence_Fast_GET_ITEM(seq, i);
double value;
set_value_to_item(seq, i);
IMAGING_PIXEL_FLOAT32(image, x, y) =
(FLOAT32)(PyFloat_AsDouble(op) * scale + offset);
(FLOAT32)(value * scale + offset);
if (++x >= (int)image->xsize) {
x = 0, y++;
}

2
src/_webp.c
View File

@ -396,7 +396,7 @@ _anim_decoder_new(PyObject *self, PyObject *args) {
}
PyObject_Del(decp);
}
PyErr_SetString(PyExc_RuntimeError, "could not create decoder object");
PyErr_SetString(PyExc_OSError, "could not create decoder object");
return NULL;
}

12
src/libImaging/Convert.c
View File

@ -1013,7 +1013,7 @@ p2l(UINT8 *out, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
/* FIXME: precalculate greyscale palette? */
for (x = 0; x < xsize; x++) {
*out++ = L(&palette[in[x] * 4]) / 1000;
*out++ = L24(&palette[in[x] * 4]) >> 16;
}
}
@ -1022,7 +1022,7 @@ pa2l(UINT8 *out, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
/* FIXME: precalculate greyscale palette? */
for (x = 0; x < xsize; x++, in += 4) {
*out++ = L(&palette[in[0] * 4]) / 1000;
*out++ = L24(&palette[in[0] * 4]) >> 16;
}
}
@ -1044,7 +1044,7 @@ p2la(UINT8 *out, const UINT8 *in, int xsize, const UINT8 *palette) {
/* FIXME: precalculate greyscale palette? */
for (x = 0; x < xsize; x++, out += 4) {
const UINT8 *rgba = &palette[*in++ * 4];
out[0] = out[1] = out[2] = L(rgba) / 1000;
out[0] = out[1] = out[2] = L24(rgba) >> 16;
out[3] = rgba[3];
}
}
@ -1054,7 +1054,7 @@ pa2la(UINT8 *out, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
/* FIXME: precalculate greyscale palette? */
for (x = 0; x < xsize; x++, in += 4, out += 4) {
out[0] = out[1] = out[2] = L(&palette[in[0] * 4]) / 1000;
out[0] = out[1] = out[2] = L24(&palette[in[0] * 4]) >> 16;
out[3] = in[3];
}
}
@ -1063,7 +1063,7 @@ static void
p2i(UINT8 *out_, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
for (x = 0; x < xsize; x++, out_ += 4) {
INT32 v = L(&palette[in[x] * 4]) / 1000;
INT32 v = L24(&palette[in[x] * 4]) >> 16;
memcpy(out_, &v, sizeof(v));
}
}
@ -1073,7 +1073,7 @@ pa2i(UINT8 *out_, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
INT32 *out = (INT32 *)out_;
for (x = 0; x < xsize; x++, in += 4) {
*out++ = L(&palette[in[0] * 4]) / 1000;
*out++ = L24(&palette[in[0] * 4]) >> 16;
}
}

12
src/libImaging/Draw.c
View File

@ -1854,14 +1854,8 @@ ImagingOutlineTransform(ImagingOutline outline, double a[6]) {
eIn = outline->edges;
n = outline->count;
/* FIXME: ugly! */
outline->edges = NULL;
outline->count = outline->size = 0;
eOut = allocate(outline, n);
if (!eOut) {
outline->edges = eIn;
outline->count = outline->size = n;
ImagingError_MemoryError();
return -1;
}
@ -1897,7 +1891,11 @@ ImagingOutlineTransform(ImagingOutline outline, double a[6]) {
eOut++;
}
free(eIn);
free(outline->edges);
/* FIXME: ugly! */
outline->edges = NULL;
outline->count = outline->size = 0;
return 0;
}

9
src/libImaging/ImPlatform.h
View File

@ -25,11 +25,18 @@
#endif
#endif
#ifdef _WIN32
#if defined(_WIN32) || defined(__CYGWIN__)
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#ifdef __CYGWIN__
#undef _WIN64
#undef _WIN32
#undef __WIN32__
#undef WIN32
#endif
#else
/* For System that are not Windows, we'll need to define these. */

6
src/libImaging/Jpeg2KDecode.c
View File

@ -180,9 +180,11 @@ j2ku_gray_i(
case 2:
for (y = 0; y < h; ++y) {
const UINT16 *data = (const UINT16 *)&tiledata[2 * y * w];
UINT16 *row = (UINT16 *)im->image[y0 + y] + x0;
UINT8 *row = (UINT8 *)im->image[y0 + y] + x0;
for (x = 0; x < w; ++x) {
*row++ = j2ku_shift(offset + *data++, shift);
UINT16 pixel = j2ku_shift(offset + *data++, shift);
*row++ = pixel;
*row++ = pixel >> 8;
}
}
break;

19
src/libImaging/Jpeg2KEncode.c
View File

@ -110,8 +110,15 @@ j2k_pack_i16(Imaging im, UINT8 *buf, unsigned x0, unsigned y0, unsigned w, unsig
for (y = 0; y < h; ++y) {
UINT8 *data = (UINT8 *)(im->image[y + y0] + x0);
for (x = 0; x < w; ++x) {
*ptr++ = *data++;
*ptr++ = *data++;
#ifdef WORDS_BIGENDIAN
ptr[0] = data[1];
ptr[1] = data[0];
#else
ptr[0] = data[0];
ptr[1] = data[1];
#endif
ptr += 2;
data += 2;
}
}
}
@ -301,13 +308,7 @@ j2k_encode_entry(Imaging im, ImagingCodecState state) {
components = 1;
color_space = OPJ_CLRSPC_GRAY;
pack = j2k_pack_l;
} else if (strcmp(im->mode, "I;16") == 0) {
components = 1;
color_space = OPJ_CLRSPC_GRAY;
pack = j2k_pack_i16;
prec = 16;
bpp = 12;
} else if (strcmp(im->mode, "I;16B") == 0) {
} else if (strcmp(im->mode, "I;16") == 0 || strcmp(im->mode, "I;16B") == 0) {
components = 1;
color_space = OPJ_CLRSPC_GRAY;
pack = j2k_pack_i16;

4
src/libImaging/Pack.c
View File

@ -656,7 +656,11 @@ static struct {
/* storage modes */
{"I;16", "I;16", 16, copy2},
#ifdef WORDS_BIGENDIAN
{"I;16", "I;16B", 16, packI16N_I16},
#else
{"I;16", "I;16B", 16, packI16N_I16B},
#endif
{"I;16B", "I;16B", 16, copy2},
{"I;16L", "I;16L", 16, copy2},
{"I;16", "I;16N", 16, packI16N_I16}, // LibTiff native->image endian.

2
src/libImaging/QuantOctree.c
View File

@ -317,7 +317,7 @@ void
add_lookup_buckets(ColorCube cube, ColorBucket palette, long nColors, long offset) {
long i;
Pixel p;
for (i = offset; i < offset + nColors; i++) {
for (i = offset + nColors - 1; i >= offset; i--) {
avg_color_from_color_bucket(&palette[i], &p);
set_lookup_value(cube, &p, i);
}

2
src/libImaging/TiffDecode.c
View File

@ -543,7 +543,7 @@ ImagingLibTiffDecode(
Imaging im, ImagingCodecState state, UINT8 *buffer, Py_ssize_t bytes) {
TIFFSTATE *clientstate = (TIFFSTATE *)state->context;
char *filename = "tempfile.tif";
char *mode = "r";
char *mode = "rC";
TIFF *tiff;
uint16_t photometric = 0; // init to not PHOTOMETRIC_YCBCR
uint16_t compression;

97
src/path.c
View File

@ -57,7 +57,7 @@ alloc_array(Py_ssize_t count) {
if ((unsigned long long)count > (SIZE_MAX / (2 * sizeof(double))) - 1) {
return ImagingError_MemoryError();
}
xy = malloc(2 * count * sizeof(double) + 1);
xy = calloc(2 * count * sizeof(double) + 1, sizeof(double));
if (!xy) {
ImagingError_MemoryError();
}
@ -162,42 +162,37 @@ PyPath_Flatten(PyObject *data, double **pxy) {
return -1;
}
#define assign_item_to_array(op, decref) \
if (PyFloat_Check(op)) { \
xy[j++] = PyFloat_AS_DOUBLE(op); \
} else if (PyLong_Check(op)) { \
xy[j++] = (float)PyLong_AS_LONG(op); \
} else if (PyNumber_Check(op)) { \
xy[j++] = PyFloat_AsDouble(op); \
} else if (PyArg_ParseTuple(op, "dd", &x, &y)) { \
xy[j++] = x; \
xy[j++] = y; \
} else { \
PyErr_SetString(PyExc_ValueError, "incorrect coordinate type"); \
if (decref) { \
Py_DECREF(op); \
} \
free(xy); \
return -1; \
}
/* Copy table to path array */
if (PyList_Check(data)) {
for (i = 0; i < n; i++) {
double x, y;
PyObject *op = PyList_GET_ITEM(data, i);
if (PyFloat_Check(op)) {
xy[j++] = PyFloat_AS_DOUBLE(op);
} else if (PyLong_Check(op)) {
xy[j++] = (float)PyLong_AS_LONG(op);
} else if (PyNumber_Check(op)) {
xy[j++] = PyFloat_AsDouble(op);
} else if (PyArg_ParseTuple(op, "dd", &x, &y)) {
xy[j++] = x;
xy[j++] = y;
} else {
free(xy);
return -1;
}
assign_item_to_array(op, 0);
}
} else if (PyTuple_Check(data)) {
for (i = 0; i < n; i++) {
double x, y;
PyObject *op = PyTuple_GET_ITEM(data, i);
if (PyFloat_Check(op)) {
xy[j++] = PyFloat_AS_DOUBLE(op);
} else if (PyLong_Check(op)) {
xy[j++] = (float)PyLong_AS_LONG(op);
} else if (PyNumber_Check(op)) {
xy[j++] = PyFloat_AsDouble(op);
} else if (PyArg_ParseTuple(op, "dd", &x, &y)) {
xy[j++] = x;
xy[j++] = y;
} else {
free(xy);
return -1;
}
assign_item_to_array(op, 0);
}
} else {
for (i = 0; i < n; i++) {
@ -213,20 +208,7 @@ PyPath_Flatten(PyObject *data, double **pxy) {
return -1;
}
}
if (PyFloat_Check(op)) {
xy[j++] = PyFloat_AS_DOUBLE(op);
} else if (PyLong_Check(op)) {
xy[j++] = (float)PyLong_AS_LONG(op);
} else if (PyNumber_Check(op)) {
xy[j++] = PyFloat_AsDouble(op);
} else if (PyArg_ParseTuple(op, "dd", &x, &y)) {
xy[j++] = x;
xy[j++] = y;
} else {
Py_DECREF(op);
free(xy);
return -1;
}
assign_item_to_array(op, 1);
Py_DECREF(op);
}
}
@ -327,21 +309,26 @@ path_getbbox(PyPathObject *self, PyObject *args) {
xy = self->xy;
x0 = x1 = xy[0];
y0 = y1 = xy[1];
if (self->count == 0) {
x0 = x1 = 0;
y0 = y1 = 0;
} else {
x0 = x1 = xy[0];
y0 = y1 = xy[1];
for (i = 1; i < self->count; i++) {
if (xy[i + i] < x0) {
x0 = xy[i + i];
}
if (xy[i + i] > x1) {
x1 = xy[i + i];
}
if (xy[i + i + 1] < y0) {
y0 = xy[i + i + 1];
}
if (xy[i + i + 1] > y1) {
y1 = xy[i + i + 1];
for (i = 1; i < self->count; i++) {
if (xy[i + i] < x0) {
x0 = xy[i + i];
}
if (xy[i + i] > x1) {
x1 = xy[i + i];
}
if (xy[i + i + 1] < y0) {
y0 = xy[i + i + 1];
}
if (xy[i + i + 1] > y1) {
y1 = xy[i + i + 1];
}
}
}

2
src/thirdparty/raqm/COPYING vendored
View File

@ -1,7 +1,7 @@
The MIT License (MIT)
Copyright © 2015 Information Technology Authority (ITA) <foss@ita.gov.om>
Copyright © 2016 Khaled Hosny <khaledhosny@eglug.org>
Copyright © 2016-2021 Khaled Hosny <khaled@aliftype.com>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

16
src/thirdparty/raqm/NEWS vendored
View File

@ -1,4 +1,18 @@
Overview of changes leading to 0.7.1
Overview of changes leading to 0.8.0
Monday, December 13, 2021
====================================
Remove autotools build.
Support using SheenBiDi instead of FriBiDi for Unicode BiDi support.
Fix running tests with Python <= 3.6.
New API:
* raqm_get_par_resolved_direction
* raqm_get_direction_at_index
Overview of changes leading to 0.7.2
Monday, September 27, 2021
====================================

27
src/thirdparty/raqm/README.md vendored
View File

@ -6,26 +6,26 @@ Raqm
Raqm is a small library that encapsulates the logic for complex text layout and
provides a convenient API.
It currently provides bidirectional text support (using [FriBiDi][1]), shaping
(using [HarfBuzz][2]), and proper script itemization. As a result,
Raqm can support most writing systems covered by Unicode.
It currently provides bidirectional text support (using [FriBiDi][1] or
[SheenBidi][2]), shaping (using [HarfBuzz][3]), and proper script itemization.
As a result, Raqm can support most writing systems covered by Unicode.
The documentation can be accessed on the web at:
> http://host-oman.github.io/libraqm/
Raqm (Arabic: رَقْم) is writing, also number or digit and the Arabic word for
digital (رَقَمِيّ) shares the same root, so it is a play on “digital writing”.
digital (رَقَمِيّ) shares the same root, so it is a play on “digital writing”.
Building
--------
Raqm depends on the following libraries:
* [FreeType][3]
* [HarfBuzz][2]
* [FriBiDi][1]
* [FreeType][4]
* [HarfBuzz][3]
* [FriBiDi][1] or [SheenBidi][2]
To build the documentation you will also need:
* [GTK-Doc][4]
* [GTK-Doc][5]
To install dependencies on Fedora:
@ -48,11 +48,11 @@ directory:
$ ninja -C build
$ ninja -C build install
To build the documentation, pass `-Ddocs=enable` to the `meson`.
To build the documentation, pass `-Ddocs=true` to the `meson`.
To run the tests:
$ ninja -C test
$ ninja -C build test
Contributing
------------
@ -78,6 +78,7 @@ The following projects have patches to support complex text layout using Raqm:
[1]: http://fribidi.org
[2]: http://harfbuzz.org
[3]: https://www.freetype.org
[4]: https://www.gtk.org/gtk-doc
[2]: https://github.com/Tehreer/SheenBidi
[3]: http://harfbuzz.org
[4]: https://www.freetype.org
[5]: https://www.gtk.org/gtk-doc

6
src/thirdparty/raqm/raqm-version.h vendored
View File

@ -32,10 +32,10 @@
#define _RAQM_VERSION_H_
#define RAQM_VERSION_MAJOR 0
#define RAQM_VERSION_MINOR 7
#define RAQM_VERSION_MICRO 2
#define RAQM_VERSION_MINOR 8
#define RAQM_VERSION_MICRO 0
#define RAQM_VERSION_STRING "0.7.2"
#define RAQM_VERSION_STRING "0.8.0"
#define RAQM_VERSION_ATLEAST(major,minor,micro) \
((major)*10000+(minor)*100+(micro) <= \

401
src/thirdparty/raqm/raqm.c vendored
View File

@ -1,6 +1,6 @@
/*
* Copyright © 2015 Information Technology Authority (ITA) <foss@ita.gov.om>
* Copyright © 2016 Khaled Hosny <khaledhosny@eglug.org>
* Copyright © 2016-2021 Khaled Hosny <khaled@aliftype.com>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
@ -30,11 +30,18 @@
#include <assert.h>
#include <string.h>
#ifdef RAQM_SHEENBIDI
#include <SheenBidi.h>
#else
#ifdef HAVE_FRIBIDI_SYSTEM
#include <fribidi.h>
#else
#include "../fribidi-shim/fribidi.h"
#endif
#if FRIBIDI_MAJOR_VERSION >= 1
#define USE_FRIBIDI_EX_API
#endif
#endif
#include <hb.h>
#include <hb-ft.h>
@ -56,10 +63,6 @@
#include "raqm.h"
#if FRIBIDI_MAJOR_VERSION >= 1
#define USE_FRIBIDI_EX_API
#endif
/**
* SECTION:raqm
* @title: Raqm
@ -178,6 +181,15 @@
# define RAQM_TEST(...)
#endif
#define RAQM_BIDI_LEVEL_IS_RTL(level) \
((level) & 1)
#ifdef RAQM_SHEENBIDI
typedef SBLevel _raqm_bidi_level_t;
#else
typedef FriBidiLevel _raqm_bidi_level_t;
#endif
typedef enum {
RAQM_FLAG_NONE = 0,
RAQM_FLAG_UTF8 = 1 << 0
@ -438,6 +450,53 @@ raqm_set_text (raqm_t *rq,
return true;
}
static void *
_raqm_get_utf8_codepoint (const void *str,
uint32_t *out_codepoint)
{
const char *s = (const char *)str;
if (0xf0 == (0xf8 & s[0]))
{
*out_codepoint = ((0x07 & s[0]) << 18) | ((0x3f & s[1]) << 12) | ((0x3f & s[2]) << 6) | (0x3f & s[3]);
s += 4;
}
else if (0xe0 == (0xf0 & s[0]))
{
*out_codepoint = ((0x0f & s[0]) << 12) | ((0x3f & s[1]) << 6) | (0x3f & s[2]);
s += 3;
}
else if (0xc0 == (0xe0 & s[0]))
{
*out_codepoint = ((0x1f & s[0]) << 6) | (0x3f & s[1]);
s += 2;
}
else
{
*out_codepoint = s[0];
s += 1;
}
return (void *)s;
}
static size_t
_raqm_u8_to_u32 (const char *text, size_t len, uint32_t *unicode)
{
size_t in_len = 0;
uint32_t *out_utf32 = unicode;
const char *in_utf8 = text;
while ((*in_utf8 != '\0') && (in_len < len))
{
in_utf8 = _raqm_get_utf8_codepoint (in_utf8, out_utf32);
++out_utf32;
++in_len;
}
return (out_utf32 - unicode);
}
/**
* raqm_set_text_utf8:
* @rq: a #raqm_t.
@ -482,9 +541,7 @@ raqm_set_text_utf8 (raqm_t *rq,
memcpy (rq->text_utf8, text, sizeof (char) * len);
ulen = fribidi_charset_to_unicode (FRIBIDI_CHAR_SET_UTF8,
text, len, unicode);
ulen = _raqm_u8_to_u32 (text, len, unicode);
ok = raqm_set_text (rq, unicode, ulen);
free (unicode);
@ -504,7 +561,7 @@ raqm_set_text_utf8 (raqm_t *rq,
*
* The default is #RAQM_DIRECTION_DEFAULT, which determines the paragraph
* direction based on the first character with strong bidi type (see [rule
* P2](https://unicode.org/reports/tr9/#P2) in Unicode Bidirectional Algorithm),
* P2](http://unicode.org/reports/tr9/#P2) in Unicode Bidirectional Algorithm),
* which can be good enough for many cases but has problems when a mainly
* right-to-left paragraph starts with a left-to-right character and vice versa
* as the detected paragraph direction will be the wrong one, or when text does
@ -971,17 +1028,78 @@ raqm_get_glyphs (raqm_t *rq,
return rq->glyphs;
}
/**
* raqm_get_par_resolved_direction:
* @rq: a #raqm_t.
*
* Gets the resolved direction of the paragraph;
*
* Return value:
* The #raqm_direction_t specifying the resolved direction of text,
* or #RAQM_DIRECTION_DEFAULT if raqm_layout() has not been called on @rq.
*
* Since: 0.8
*/
RAQM_API raqm_direction_t
raqm_get_par_resolved_direction (raqm_t *rq)
{
if (!rq)
return RAQM_DIRECTION_DEFAULT;
return rq->resolved_dir;
}
/**
* raqm_get_direction_at_index:
* @rq: a #raqm_t.
* @index: (in): character index.
*
* Gets the resolved direction of the character at specified index;
*
* Return value:
* The #raqm_direction_t specifying the resolved direction of text at the
* specified index, or #RAQM_DIRECTION_DEFAULT if raqm_layout() has not been
* called on @rq.
*
* Since: 0.8
*/
RAQM_API raqm_direction_t
raqm_get_direction_at_index (raqm_t *rq,
size_t index)
{
if (!rq)
return RAQM_DIRECTION_DEFAULT;
for (raqm_run_t *run = rq->runs; run != NULL; run = run->next)
{
if (run->pos <= index && index < run->pos + run->len) {
switch (run->direction) {
case HB_DIRECTION_LTR:
return RAQM_DIRECTION_LTR;
case HB_DIRECTION_RTL:
return RAQM_DIRECTION_RTL;
case HB_DIRECTION_TTB:
return RAQM_DIRECTION_TTB;
default:
return RAQM_DIRECTION_DEFAULT;
}
}
}
return RAQM_DIRECTION_DEFAULT;
}
static bool
_raqm_resolve_scripts (raqm_t *rq);
static hb_direction_t
_raqm_hb_dir (raqm_t *rq, FriBidiLevel level)
_raqm_hb_dir (raqm_t *rq, _raqm_bidi_level_t level)
{
hb_direction_t dir = HB_DIRECTION_LTR;
if (rq->base_dir == RAQM_DIRECTION_TTB)
dir = HB_DIRECTION_TTB;
else if (FRIBIDI_LEVEL_IS_RTL (level))
else if (RAQM_BIDI_LEVEL_IS_RTL(level))
dir = HB_DIRECTION_RTL;
return dir;
@ -990,9 +1108,65 @@ _raqm_hb_dir (raqm_t *rq, FriBidiLevel level)
typedef struct {
size_t pos;
size_t len;
FriBidiLevel level;
_raqm_bidi_level_t level;
} _raqm_bidi_run;
#ifdef RAQM_SHEENBIDI
static _raqm_bidi_run *
_raqm_bidi_itemize (raqm_t *rq, size_t *run_count)
{
_raqm_bidi_run *runs;
SBAlgorithmRef bidi;
SBParagraphRef par;
SBUInteger par_len;
SBLineRef line;
SBLevel base_level = SBLevelDefaultLTR;
SBCodepointSequence input = {
SBStringEncodingUTF32,
(void *) rq->text,
rq->text_len
};
if (rq->base_dir == RAQM_DIRECTION_RTL)
base_level = 1;
else if (rq->base_dir == RAQM_DIRECTION_LTR)
base_level = 0;
/* paragraph */
bidi = SBAlgorithmCreate (&input);
par = SBAlgorithmCreateParagraph (bidi, 0, INT32_MAX, base_level);
par_len = SBParagraphGetLength (par);
/* lines */
line = SBParagraphCreateLine (par, 0, par_len);
*run_count = SBLineGetRunCount (line);
if (SBParagraphGetBaseLevel (par) == 0)
rq->resolved_dir = RAQM_DIRECTION_LTR;
else
rq->resolved_dir = RAQM_DIRECTION_RTL;
runs = malloc (sizeof (_raqm_bidi_run) * (*run_count));
if (runs)
{
const SBRun *sheenbidi_runs = SBLineGetRunsPtr(line);
for (size_t i = 0; i < (*run_count); ++i)
{
runs[i].pos = sheenbidi_runs[i].offset;
runs[i].len = sheenbidi_runs[i].length;
runs[i].level = sheenbidi_runs[i].level;
}
}
SBLineRelease (line);
SBParagraphRelease (par);
SBAlgorithmRelease (bidi);
return runs;
}
#else
static void
_raqm_reverse_run (_raqm_bidi_run *run, const size_t len)
{
@ -1093,19 +1267,78 @@ _raqm_reorder_runs (const FriBidiCharType *types,
return runs;
}
static bool
_raqm_itemize (raqm_t *rq)
static _raqm_bidi_run *
_raqm_bidi_itemize (raqm_t *rq, size_t *run_count)
{
FriBidiParType par_type = FRIBIDI_PAR_ON;
_raqm_bidi_run *runs = NULL;
FriBidiCharType *types;
_raqm_bidi_level_t *levels;
int max_level = 0;
#ifdef USE_FRIBIDI_EX_API
FriBidiBracketType *btypes;
#endif
FriBidiLevel *levels;
types = calloc (rq->text_len, sizeof (FriBidiCharType));
#ifdef USE_FRIBIDI_EX_API
btypes = calloc (rq->text_len, sizeof (FriBidiBracketType));
#endif
levels = calloc (rq->text_len, sizeof (_raqm_bidi_level_t));
if (!types || !levels
#ifdef USE_FRIBIDI_EX_API
|| !btypes
#endif
)
{
goto done;
}
if (rq->base_dir == RAQM_DIRECTION_RTL)
par_type = FRIBIDI_PAR_RTL;
else if (rq->base_dir == RAQM_DIRECTION_LTR)
par_type = FRIBIDI_PAR_LTR;
fribidi_get_bidi_types (rq->text, rq->text_len, types);
#ifdef USE_FRIBIDI_EX_API
fribidi_get_bracket_types (rq->text, rq->text_len, types, btypes);
max_level = fribidi_get_par_embedding_levels_ex (types, btypes,
rq->text_len, &par_type,
levels);
#else
max_level = fribidi_get_par_embedding_levels (types, rq->text_len,
&par_type, levels);
#endif
if (par_type == FRIBIDI_PAR_LTR)
rq->resolved_dir = RAQM_DIRECTION_LTR;
else
rq->resolved_dir = RAQM_DIRECTION_RTL;
if (max_level == 0)
goto done;
/* Get the number of bidi runs */
runs = _raqm_reorder_runs (types, rq->text_len, par_type, levels, run_count);
done:
free (types);
free (levels);
#ifdef USE_FRIBIDI_EX_API
free (btypes);
#endif
return runs;
}
#endif
static bool
_raqm_itemize (raqm_t *rq)
{
_raqm_bidi_run *runs = NULL;
raqm_run_t *last;
int max_level;
size_t run_count;
size_t run_count = 0;
bool ok = true;
#ifdef RAQM_TESTING
@ -1127,67 +1360,28 @@ _raqm_itemize (raqm_t *rq)
}
#endif
types = calloc (rq->text_len, sizeof (FriBidiCharType));
#ifdef USE_FRIBIDI_EX_API
btypes = calloc (rq->text_len, sizeof (FriBidiBracketType));
#endif
levels = calloc (rq->text_len, sizeof (FriBidiLevel));
if (!types || !levels
#ifdef USE_FRIBIDI_EX_API
|| !btypes
#endif
)
{
ok = false;
goto done;
}
if (rq->base_dir == RAQM_DIRECTION_RTL)
par_type = FRIBIDI_PAR_RTL;
else if (rq->base_dir == RAQM_DIRECTION_LTR)
par_type = FRIBIDI_PAR_LTR;
if (rq->base_dir == RAQM_DIRECTION_TTB)
{
/* Treat every thing as LTR in vertical text */
max_level = 1;
memset (types, FRIBIDI_TYPE_LTR, rq->text_len);
memset (levels, 0, rq->text_len);
rq->resolved_dir = RAQM_DIRECTION_LTR;
}
else
{
fribidi_get_bidi_types (rq->text, rq->text_len, types);
#ifdef USE_FRIBIDI_EX_API
fribidi_get_bracket_types (rq->text, rq->text_len, types, btypes);
max_level = fribidi_get_par_embedding_levels_ex (types, btypes,
rq->text_len, &par_type,
levels);
#else
max_level = fribidi_get_par_embedding_levels (types, rq->text_len,
&par_type, levels);
#endif
if (par_type == FRIBIDI_PAR_LTR)
rq->resolved_dir = RAQM_DIRECTION_LTR;
else
rq->resolved_dir = RAQM_DIRECTION_RTL;
}
if (max_level == 0)
{
ok = false;
goto done;
}
if (!_raqm_resolve_scripts (rq))
{
ok = false;
goto done;
}
/* Get the number of bidi runs */
runs = _raqm_reorder_runs (types, rq->text_len, par_type, levels, &run_count);
if (rq->base_dir == RAQM_DIRECTION_TTB)
{
/* Treat every thing as LTR in vertical text */
run_count = 1;
rq->resolved_dir = RAQM_DIRECTION_TTB;
runs = malloc (sizeof (_raqm_bidi_run));
if (runs)
{
runs->pos = 0;
runs->len = rq->text_len;
runs->level = 0;
}
} else {
runs = _raqm_bidi_itemize (rq, &run_count);
}
if (!runs)
{
ok = false;
@ -1197,7 +1391,7 @@ _raqm_itemize (raqm_t *rq)
#ifdef RAQM_TESTING
RAQM_TEST ("Number of runs before script itemization: %zu\n\n", run_count);
RAQM_TEST ("Fribidi Runs:\n");
RAQM_TEST ("BiDi Runs:\n");
for (size_t i = 0; i < run_count; i++)
{
RAQM_TEST ("run[%zu]:\t start: %zu\tlength: %zu\tlevel: %d\n",
@ -1309,11 +1503,6 @@ _raqm_itemize (raqm_t *rq)
done:
free (runs);
free (types);
#ifdef USE_FRIBIDI_EX_API
free (btypes);
#endif
free (levels);
return ok;
}
@ -1328,7 +1517,7 @@ typedef struct {
/* Special paired characters for script detection */
static size_t paired_len = 34;
static const FriBidiChar paired_chars[] =
static const uint32_t paired_chars[] =
{
0x0028, 0x0029, /* ascii paired punctuation */
0x003c, 0x003e,
@ -1431,7 +1620,7 @@ _raqm_stack_push (_raqm_stack_t *stack,
}
static int
_get_pair_index (const FriBidiChar ch)
_get_pair_index (const uint32_t ch)
{
int lower = 0;
int upper = paired_len - 1;
@ -1569,6 +1758,7 @@ _raqm_resolve_scripts (raqm_t *rq)
return true;
}
#ifdef HAVE_FT_GET_TRANSFORM
static void
_raqm_ft_transform (int *x,
int *y,
@ -1583,6 +1773,7 @@ _raqm_ft_transform (int *x,
*x = vector.x;
*y = vector.y;
}
#endif
static bool
_raqm_shape (raqm_t *rq)
@ -1634,20 +1825,30 @@ _raqm_shape (raqm_t *rq)
return true;
}
/* Count equivalent UTF-8 bytes in codepoint */
static size_t
_raqm_count_codepoint_utf8_bytes (uint32_t chr)
{
if (0 == ((uint32_t) 0xffffff80 & chr))
return 1;
else if (0 == ((uint32_t) 0xfffff800 & chr))
return 2;
else if (0 == ((uint32_t) 0xffff0000 & chr))
return 3;
else
return 4;
}
/* Convert index from UTF-32 to UTF-8 */
static uint32_t
_raqm_u32_to_u8_index (raqm_t *rq,
uint32_t index)
{
FriBidiStrIndex length;
char *output = malloc ((sizeof (char) * 4 * index) + 1);
size_t length = 0;
length = fribidi_unicode_to_charset (FRIBIDI_CHAR_SET_UTF8,
rq->text,
index,
output);
for (uint32_t i = 0; i < index; ++i)
length += _raqm_count_codepoint_utf8_bytes (rq->text[i]);
free (output);
return length;
}
@ -1656,15 +1857,27 @@ static uint32_t
_raqm_u8_to_u32_index (raqm_t *rq,
uint32_t index)
{
FriBidiStrIndex length;
uint32_t *output = malloc (sizeof (uint32_t) * (index + 1));
const unsigned char *s = (const unsigned char *) rq->text_utf8;
const unsigned char *t = s;
size_t length = 0;
length = fribidi_charset_to_unicode (FRIBIDI_CHAR_SET_UTF8,
rq->text_utf8,
index,
output);
while (((size_t) (s - t) < index) && ('\0' != *s))
{
if (0xf0 == (0xf8 & *s))
s += 4;
else if (0xe0 == (0xf0 & *s))
s += 3;
else if (0xc0 == (0xe0 & *s))
s += 2;
else
s += 1;
length++;
}
if ((size_t) (s-t) > index)
length--;
free (output);
return length;
}

9
src/thirdparty/raqm/raqm.h vendored
View File

@ -1,6 +1,6 @@
/*
* Copyright © 2015 Information Technology Authority (ITA) <foss@ita.gov.om>
* Copyright © 2016 Khaled Hosny <khaledhosny@eglug.org>
* Copyright © 2016-2021 Khaled Hosny <khaled@aliftype.com>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
@ -156,6 +156,13 @@ RAQM_API raqm_glyph_t *
raqm_get_glyphs (raqm_t *rq,
size_t *length);
RAQM_API raqm_direction_t
raqm_get_par_resolved_direction (raqm_t *rq);
RAQM_API raqm_direction_t
raqm_get_direction_at_index (raqm_t *rq,
size_t index);
RAQM_API bool
raqm_index_to_position (raqm_t *rq,
size_t *index,

14
winbuild/build_prepare.py
View File

@ -154,9 +154,9 @@ deps = {
# "bins": [r"libtiff\*.dll"],
},
"libwebp": {
"url": "http://downloads.webmproject.org/releases/webp/libwebp-1.2.1.tar.gz",
"filename": "libwebp-1.2.1.tar.gz",
"dir": "libwebp-1.2.1",
"url": "http://downloads.webmproject.org/releases/webp/libwebp-1.2.2.tar.gz",
"filename": "libwebp-1.2.2.tar.gz",
"dir": "libwebp-1.2.2",
"build": [
cmd_rmdir(r"output\release-static"), # clean
cmd_nmake(
@ -257,10 +257,10 @@ deps = {
"libs": [r"bin\*.lib"],
},
"libimagequant": {
# commit: Merge branch 'master' into msvc (matches 2.16.0 tag)
"url": "https://github.com/ImageOptim/libimagequant/archive/f41ee301ff3a407b16991af3dbe03910919bbdc3.zip", # noqa: E501
"filename": "libimagequant-f41ee301ff3a407b16991af3dbe03910919bbdc3.zip",
"dir": "libimagequant-f41ee301ff3a407b16991af3dbe03910919bbdc3",
# commit: Merge branch 'master' into msvc (matches 2.17.0 tag)
"url": "https://github.com/ImageOptim/libimagequant/archive/e4c1334be0eff290af5e2b4155057c2953a313ab.zip", # noqa: E501
"filename": "libimagequant-e4c1334be0eff290af5e2b4155057c2953a313ab.zip",
"dir": "libimagequant-e4c1334be0eff290af5e2b4155057c2953a313ab",
"patch": {
"CMakeLists.txt": {
"if(OPENMP_FOUND)": "if(false)",