Commit Graph

35326 Commits

Author SHA1 Message Date
Viktor Szakats
3c6bf10863
GHA/macos: follow Homebrew and switch to pkgconf
Homebrew switched to `pkgconf`, and now pkg-config installs an extra
package. Update package list to avoid that.

The side-effect of `pkgconf` is that this former log message:
```
--   Package 'libcrypto', required by 'libssh2', not found
```
https://github.com/curl/curl/actions/runs/11779568834/job/32808325442#step:7:84

is replaced by this, and repeated 10 times:
```
Package libcrypto was not found in the pkg-config search path.
Perhaps you should add the directory containing `libcrypto.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libcrypto', required by 'libssh2', not found
```
https://github.com/curl/curl/actions/runs/11792711391/job/32846858320#step:7:85

Closes #15575
2024-11-13 18:19:38 +01:00
Viktor Szakats
2ccd1c5889
cmakelint: fix to check root CMakeLists.txt
Closes #15565
2024-11-13 12:45:06 +01:00
Viktor Szakats
93e6e4b823
cmake: work around ios.toolchain.cmake breaking feature-detections
Fix builds with CMake configured to falsely return successful detection
when using `check_function_exists()` (and `check_library_exists()`, and
anything based on `try_compile()` that's relying on the linker). After
such mis-detection the build fails when trying to use the feature that
doesn't in fact exist.

The mis-detection is caused by this CMake setting:
```
set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)
```
It is set by default (or on conditions) when using 3rd-party toolchain:
https://github.com/leetal/ios-cmake/blob/master/ios.toolchain.cmake

After this patch the curl build overrides this setting for the duration
of feature tests, and logs a message about it.

Also preset and skip feature tests for known mis-detections:
- `connect()` in `libsocket`
- `getpass_r()`
- `eventfd()` (did not cause an issue due to a separate bug)
- `sendmmsg()` (did not cause an issue because it's Linux-only)

If mis-detections are still seen, the workaround is to force-set the
specific feature by passing `-DHAVE_*=OFF` to cmake.
Also consider passing `-DENABLE_STRICT_TRY_COMPILE=ON` for
`ios.toolchain.cmake` to fix the root cause.

Interestingly curl itself uses this setting to speed up compile-only
detections: be17f298ff #3744

Also:
- OtherTests.cmake: restore original value of
  `CMAKE_TRY_COMPILE_TARGET_TYPE`. Before this patch it reset it
  to empty.
- OtherTests.cmake: unset a local variable after use, quote a string.

Follow-up to 8e34505776 #15164
Follow-up to 8b76a8aeb2 #15525
Ref: https://github.com/leetal/ios-cmake/issues/47
Ref: https://gitlab.kitware.com/cmake/cmake/-/issues/18121
Ref: https://cmake.org/cmake/help/latest/variable/CMAKE_TRY_COMPILE_TARGET_TYPE.html
Reported-by: Dan Rosser
Fixes #15557
Closes #15559
2024-11-13 10:42:38 +01:00
Dan Fandrich
e3aa2a07f9 tests: use the standard format of an IGNORED line
This is more consistent and allows Test Clutch to parse it properly.
Also, add --test-duphandle and --test-event to the runtests.pl help
text.
2024-11-12 22:23:57 -08:00
Daniel Engberg
acc73edce8
GHA/non-native: streamline installed packages on FreeBSD
Instead of installing the cmake package which is a meta-port (includes
documentation and manpages etc) install cmake-core which is cmake itself
to save a few cpu cycles.

Also drop nghttp2 in favour of the slimmer libnghttp2.

Closes #15540
2024-11-12 13:02:44 +01:00
Daniel Stenberg
448df98d92
mk-ca-bundle: remove CKA_NSS_SERVER_DISTRUST_AFTER conditions
The script's previous treatment of this meta-data was a
misunderstanding. (Added in 1ebc53df25) The mistrust is not for
the root cert at this date (it would simply be removed from the bundle
then instead) but for created server certificates:

    If a builtin certificate has a CKA_NSS_SERVER_DISTRUST_AFTER
    timestamp before the SCT or NotBefore date of a certificate that
    builtin issued, then clients can elect not to trust it.

That is however information that cannot be provided in the generated PEM
output.

Fixes #15547
Reported-by: Andrew Ayer
Closes #15552
2024-11-12 08:33:57 +01:00
Viktor Szakats
774844ab5d
curl-rustls.m4: keep existing CPPFLAGS/LDFLAGS when detected
Different variable names were used in flag save and restore operations,
which could cause existing `CPPFLAGS` and `LDFLAGS` be accidentally lost
when detecting Rustls.

Follow-up to 647e86a3ef #13179
Closes #15546
2024-11-11 14:32:01 +01:00
Viktor Szakats
8b76a8aeb2
build: use _fseeki64() on Windows, drop detections
A recent update caused CMake builds to mis-detect this symbol on iOS.
Auto-detection also seems redundant given that it's a Windows-only
function and most Windows builds were already opted-in.

Drop detections and use it in all Windows builds with large file support
enabled.

Feature history:
- pririotizing for Windows: aaacd02466 #14678
- Windows opt-in cmake: 8e74c0729d #11950
- Windows opt-in: aa6c94c5bf #11944
- use in libcurl: 9c7165e96a #11918
- use in example: 817d1c0106

Regression from 8e34505776 #15164

Reported-by: Maarten Billemont
Fixes #15525
Closes #15526
2024-11-11 14:25:04 +01:00
renovate[bot]
b723f6a445
GHA: update four dependencies
- gnutls/gnutls to v3.8.8
- rojopolis/spellcheck-github-actions digest to 403efe0
- awslabs/aws-lc to v1.38.0
- github/codeql-action digest to 4f3212b

Closes #15487
Closes #15490
Closes #15516
Closes #15528
2024-11-09 14:30:53 +01:00
Daniel Stenberg
93c65c00e5
libssh: when using IPv6 numerical address, add brackets
Reported-by: henrikjehgmti on github
Fixes #15522
Closes #15524
2024-11-09 10:34:45 +01:00
Viktor Szakats
9416e49ceb
GHA/non-native: enable nghttp2 in OmniOS job
Closes #15530
2024-11-09 01:15:03 +01:00
renovate[bot]
f6dd3a66b2
ci: Update vmactions/omnios-vm digest to 16b5996
Closes #15529
2024-11-09 00:52:35 +01:00
Daniel Stenberg
b0e53cbc4f
RELEASE-NOTES: synced 2024-11-08 23:24:50 +01:00
Ethan Everett
9089ef1f4f
nghttp2: use custom memory functions
Provide libcurl's memory functions as callbacks to replace nghttp2's own memory
functions. This allows custom memory callbacks provided by users of libcurl to
be used by nghttp2 as well.

Closes #15527
2024-11-08 23:20:32 +01:00
Viktor Szakats
1cd745a581
ECH: enable support for the AWS-LC backend
Extend existing ECH support for BoringSSL to its AWS-LC fork.

Also enable ECH in AWS-LC CI jobs.

```
curl 8.11.0-DEV (x86_64-pc-linux-gnu) libcurl/8.11.0-DEV AWS-LC/1.37.0 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libpsl/0.21.2
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli ECH HSTS HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe UnixSockets zstd
```

Closes #15499
2024-11-08 13:59:39 +01:00
Daniel Stenberg
cbafcec50b
curl: --test-duphandle in debug builds runs "duphandled"
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.

To help us catch curl_easy_duphandle() mistakes better.

Add a CI job using this.

Bonus: the previous runtests option -e is now also supported as
--test-event

Closes #15504
2024-11-08 13:22:47 +01:00
Viktor Szakats
354f3f96a1
macos: disable gcc availability workaround as needed
Homebrew gcc 14.2.0_1 fixed the issue, and the workaround is no longer
needed. Not only not needed, but the workaround is breaking builds with
the fixed gcc.

Auto-detect the upstream fix and stop applying the local workaround if
detected.

Assisted-by: Bo Anderson
Ref: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2462764619
Follow-up to e91fcbac7d #14155
Closes #15508
2024-11-08 11:15:31 +01:00
Daniel Stenberg
995db17484
RELEASE-PROCEDURE.md: adjust release dates
Due to the calendar tweak with the pending patch release on December 11.
2024-11-08 11:02:39 +01:00
Viktor Szakats
96edb5f611
cmake: drop cmake args list from buildinfo.txt
Collecting the args list has the undesired side-effect of silencing
CMake warnings about unused variables passed via the command-line.

Drop it till a better method is found to retrieve them.

Reported-by: Kai Pastor
Ref: https://github.com/curl/curl/pull/14936#issuecomment-2460350977
Follow-up to 1fdea16846 #14802
Closes #15501
2024-11-08 10:51:46 +01:00
Viktor Szakats
4d8ab1f1b8
GHA/macos: let gcc dictate the configured Apple SDK
As discovered earlier, Homebrew gcc is built against a specific Apple
SDK version and doesn't work when matched up with a different version,
e.g. the one advertised as default by the macos runner image.

Before this patch this was resolved with brute force by zapping the
hack-layer gcc component to avoid the bad interference. This worked
for us, but it's fragile, accidental and doesn't translate to
real-world build environments. Thus, impractical.

Avoid this by explicitly selecting the SDK version gcc was built for and
meant to be used with, as shown by `gcc --print-sysroot`.

It assumes that the gcc binaries preinstalled on the runner images
always ship with the SDK version they reference. It also assumes
this works with and without `brew update`.

Also:
- add 4 quick build-only jobs to test all gcc/macos combos.
- list SDKs offered via CommandLineTools.

Suggested-by: Bo Anderson
Ref: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2462764619

Follow-up to c349bd668c #14097
Closes #15518
2024-11-08 10:48:12 +01:00
Viktor Szakats
cb092c0a7e
GHA: add apt update where missing
To mitigate this kind of (repeat) errors:
```
Err:14 http://azure.archive.ubuntu.com/ubuntu noble-updates/main amd64 python3-werkzeug all 3.0.1-3ubuntu0.1
  404  Not Found [IP: 40.81.13.82 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/p/python-werkzeug/python3-werkzeug_3.0.1-3ubuntu0.1_all.deb
Fetched 10.4 MB in 4s (2593 kB/s)
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
```
https://github.com/curl/curl/actions/runs/11732257460/job/32684111508#step:3:74

Follow-up to 842f88434f #15082
Closes #15519
2024-11-08 10:48:11 +01:00
Daniel Stenberg
a3b4ffd429
TODO: consider OCSP stapling by default
Suggested-by: Nicolas F.
Closes #15483
Closes #15521
2024-11-08 08:57:45 +01:00
Ben Greear
1f6767e7a0
vtls: fix compile warning when ALPN is not available
Ref: https://curl.se/mail/lib-2024-11/0013.html

Closes #15515
2024-11-07 23:48:32 +01:00
Daniel Stenberg
f4ee7bafda
cmdline/ech.md: formatting cleanups
Reported-by: Samuel Henrique
Fixes #15506
Closes #15517
2024-11-07 23:47:17 +01:00
Daniel Stenberg
0cdde0fdfb
netrc: support large file, longer lines, longer tokens
Regression from 3b43a05e00 (shipped in 8.11.0)

Reported-by: Moritz
Fixes #15513
Closes #15514
2024-11-07 18:20:28 +01:00
Jesus Malo Poyatos
878bc429f2
setopt: fix CURLOPT_HTTP_CONTENT_DECODING
Regression from 30da1f5974 (shipped in 8.11.0)

Fixes #15511
Closes #15510
2024-11-07 17:29:36 +01:00
Daniel Stenberg
cadfe0de68
RELEASE-NOTES: synced
Bump to 8.11.1
2024-11-07 13:16:47 +01:00
Stefan Eissing
bcf8a84881
mbedtls: call psa_crypt_init() in global init
Run mbedtls' psa_crypt_init() in the general global init, optionally
protected by mbedtls locks when available.

CI: when building mbedtls, enabled thread safety

Reported-by: wxiaoguang on github
Fixes #15500
Closes #15505
2024-11-07 13:14:41 +01:00
Daniel Stenberg
f5c616930b
duphandle: also init netrc
The netrc init was only done in the Curl_open, meaning that a duplicated
handle would not get inited properly.

Added test 2309 to verify. It does netrc auth with a duplicated handle.

Regression from 3b43a05e00

Reported-by: tranzystorekk on github
Fixes #15496
Closes #15503
2024-11-07 09:47:53 +01:00
Daniel Stenberg
9919149aef
cookie: treat cookie name case sensitively
Extend test 31 to verify

Reported-by: delogicsreal on github
Fixes #15492
Closes #15493
2024-11-06 13:09:20 +01:00
Daniel Stenberg
b1ef0e1a01
RELEASE-NOTES: synced
Release time
2024-11-06 08:05:56 +01:00
Daniel Stenberg
62020546cc
THANKS: contributors from the 8.11.0 release 2024-11-06 08:05:56 +01:00
Viktor Szakats
380790b244
GHA/non-native: fix installing OpenLDAP on OpenBSD
Also:
- drop failing manual install of perl. It's there by default now.
- add link to OpenBSD package management FAQ page.

Closes #15491
2024-11-06 02:42:09 +01:00
Viktor Szakats
087f77d855
GHA/macos: drop WebSockets from job names
It's enabled in all jobs by default now.

Follow-up to cd63629733 #15076
2024-11-06 01:43:55 +01:00
Viktor Szakats
1b4897f3c8
RELEASE-NOTES: update cmake LDAP-related entry [ci skip]
Reported-by: Kai Pastor
Bug: https://github.com/curl/curl/pull/15255#issuecomment-2457558219
Follow-up to 2c90f7f69e #15255
2024-11-05 21:17:30 +01:00
Daniel Stenberg
e1ed6b8e29
mbedtls: remove failf() use from mbedtls_random
Since data can be NULL in here, calling failf() can be bad. This should
also be a terribly rare error so the lack of error message for this
should be manageable.

Reported-by: wxiaoguang on github
Fixes #15485
Closes #15486
2024-11-05 11:54:55 +01:00
Stefan Eissing
3a35901a11
wolfssl: coexist with openssl, further work
Build wolfSSL master with

./configure --prefix=/path --enable-ip-alt-name --enable-quic
--enable-earlydata --enable-psk --enable-opensslcoexist

and configure curl with openssl + wolfssl. Normal tests run.

pytest session resumption fails, as wolfssl does not handle the
new_session callback without opensslextra right now.

Closes #15481
2024-11-04 14:48:30 +01:00
Daniel Stenberg
f81abcbfeb
RELEASE-NOTES: synced 2024-11-04 10:40:42 +01:00
Daniel Stenberg
4133007798
wolfssl: no more use of the OpenSSL API
Allows curl to build with a wolfSSL built without its OpenSSL API.

It should allow curl to (soon?) build with *both* wolfSSL and OpenSSL

This change makes curl use its own sha256 implementaion when built with
wolfSSL: room for improvement.

Closes #15480
2024-11-04 10:23:05 +01:00
renovate[bot]
46bd595b77
ci: update dependency wolfSSL/wolfssh to v1.4.19
Closes #15476
2024-11-04 08:44:33 +01:00
Daniel Stenberg
6b2bc8130c
openssl: extend the OpenSSL error messages
Previously there were three error situations that only added the (unhelpful)
OpenSSL error strings, now we prefix those with a short explanation for the
error situation.

Reported-by: Jeroen Ooms
Fixes #15473
Closes #15474
2024-11-04 08:42:05 +01:00
Jonas 'Sortie' Termansen
78c3172921
curl_addrinfo: support operating systems with only getaddrinfo(3)
The gethostbyname(3) family was removed in POSIX-1.2008 in favor of
getaddrinfo(3) introduced in POSIX-1.2001. Modern POSIX systems such as
Sortix does not have gethostbyname nor the related definitions and
structures.

curl already only uses getaddrinfo(3) if available and thread safe,
although there is mild breakage if the related gethostbyname definitions
are missing.

This change attempts to fix that breakage:

Remove an unnecessary configure error if gethostbyname is missing since
getaddrinfo is enough as a fallback.

Rewrite Curl_ip2addr to not use struct hostent as it no longer is
standardized and create the struct Curl_addrinfo directly.

Only define the Curl_he2ai function on non-getaddrinfo systems where it
is going to be used with struct hoestent.

Revoke the fallback logic for when it's unknown whether getaddrinfo is
thread safe. It doesn't appear to make any sense since h_errno is
unrelated to getaddrinfo. The logic prevents new POSIX.1-2024 systems
from passing the thread safety test since h_errno does not exist anymore
and POSIX already requires getaddrinfo to be thread safe. There's
already a denylist in place for operating systems with known buggy
implementations.

Closes #15475
2024-11-02 22:28:59 +01:00
Viktor Szakats
25025419c9
pytest: include curl version string and python platform in log
For the Test Clutch matrix.

https://testclutch.curl.se/static/reports/feature-matrix.html

Assisted-by: Dan Fandrich
Closes #15470
2024-11-02 01:56:55 +01:00
Viktor Szakats
9d32724c12
certs: add missing -CAcreateserial option for LibreSSL
Also:
- display openssl path and version.
- quote a string.

Follow-up to 9b0c0d6ade #15129

Closes #15471
2024-11-01 15:36:48 +01:00
Viktor Szakats
97d69ec0da
winbuild: drop gen_resp_file.bat
`gen_resp_file.bat` could be reduced to 3 lines.
Those lines are simple to use from `Makefile.vc` as-is.

Also drop the unnecessary tabs.

Closes #15463
2024-11-01 11:36:54 +01:00
Dan Fandrich
d88db0b6b2 tests: use a set for several of the curl_props
These are used for member-of testing, so a set is more appropriate and
efficient. Also, fix a couple cases of bad exceptions.

Closes #15462
2024-10-31 20:14:44 -07:00
Stefan Eissing
9b863ac670
vquic: recv_mmsg, use fewer, but larger buffers
Reported-by: koujaz on github
Fixes #15267
Closes #15454
2024-10-31 23:40:51 +01:00
Stefan Eissing
922235e56b
ngtcp2: do not loop on recv
The vquic_recv_packets() function already loops when not all requested
packets can be received (until EAGAIN) and there is not need to do that
again in ngtcp2.

Closes #15466
2024-10-31 14:08:58 +01:00
Viktor Szakats
2f22fc10e5
GHA/linux-old: adjust configure job name 2024-10-31 13:26:27 +01:00
Viktor Szakats
329a8e9daa
unit1307: tidy up Apple OS detection
Use `__APPLE__` macro to detect Apple OS instead of relying on
the string in `CURL_OS`.

This also fixes detection with default CMake builds where `CURL_OS` is
`Darwin`. The code before this patch was expecting this substring in
lowercase.

Closes #15461
2024-10-31 09:09:01 +01:00