Commit Graph

35949 Commits

Author SHA1 Message Date
Stefan Eissing
56e40ae6a5
asyn resolver code improvements
"asyn" is the internal name under which both c-ares and threaded
resolver operate. Make the naming more consistent. Implement the c-ares
resolver in `asyn-ares.*` and the threaded resolver in `asyn-thrdd.*`.
The common functions are in `asyn-base.c`.

When `CURLRES_ASYNCH` is defined, either of the two is used and
`data->state.async` exists. Members of that struct vary for the selected
implementation, but have the fields `hostname`, `port` and `ip_version`
always present. This are populated when the async resolving starts and
eliminate the need to pass them again when checking on the status and
processing the results of the resolving.

Add a `Curl_resolv_blocking()` to `hostip.h` that relieves FTP and SOCKS
from having to repeat the same code.

`Curl_resolv_check()` remains the function to check for status of
ongoing resolving. Now it also performs internally the check if the
needed DNS entry exists in the dnscache and if so, aborts the asnyc
operation. (libcurl right now does not check for duplicate resolve
attempts. an area for future improvements).

The number of functions in `asyn.h` has been reduced. There were subtle
difference in "cancel()" and "kill()" calls, both replaced by
`Curl_async_shutdown()` now. This changes behaviour for threaded
resolver insofar as the resolving thread is now always joined unless
`data->set.quick_exit` is set. Before this was only done on some code
paths. A future improvement would be a thread pool that keeps a limit
and also could handle joins more gracefully.

DoH, not previously tagged under "asny", has its struct `doh_probes` now
also in `data->state.async`, moved there from `data->req` because it
makes more sense. Further integration of DoH underneath the "asyn"
umbrella seems like a good idea.

Closes #16963
2025-04-16 09:34:20 +02:00
Viktor Szakats
be718daf99
GHA/windows: switch a job to ARM64 on native runner
Switch a build-only MSYS2/mingw-w64 job to ARM64 using the ARM64 Windows
runner. This avoids most downsides of native ARM64 builds. Side-effect
is switching this job from GCC to clang. This adds speed, which offsets
the slower MSYS2 install step.

The new ARM64 runner hits a bunch of corner cases and inefficiencies:
- MSYS2/Cygwin misses native ARM64 support and this isn't expected to
  change. It means GH action installs x64 binaries on ARM64. vcpkg does
  the same. It runs, but not native, so slower. It affects runtests and
  possibly vcpkg build performance.
  https://github.com/msys2/MSYS2-packages/discussions/2889
  https://cygwin.com/pipermail/cygwin/2021-December/250156.html
- MSYS2 diffutils package missing for CLANGARM64.
- vcpkg boringssl build broken for ARM64.

Also:
- bump the vcpkg build time limit, as it seems the native x86_64 jobs
  also can't fit into 35 minutes.
- prepare MSVC jobs for the ARM64 runner.

Ref: https://github.com/github/roadmap/issues/1098#issuecomment-2806476117
Ref: https://github.blog/changelog/2025-04-14-windows-arm64-hosted-runners-now-available-in-public-preview/

Closes #17067
2025-04-16 04:05:23 +02:00
Joel Depooter
fe9898d26e
schannel: handle pkcs12 client certificates which contain CA certificates
The SChannel code uses the CertFindCertificateInStore function to
retrieve the client certificate from a pkcs12 certificate store.
However, when called with the CERT_FIND_ANY flag, this function does not
provide any guarantees on the order in which certificates are retrieved.
If a pkcs12 file contains an entire certificate chain instead of a
single client certificate, the CertFindCertificateInStore function may
return the CA or an intermediate certificate instead of the desired
client certificate. Since there is no associated private key for such a
certificate, the TLS handshake fails.

With this change, we now pass the CERT_FIND_HAS_PRIVATE_KEY flag. This
ensures that the CertFindCertificateInStore function will return a
certificate which has a corresponding private key. This will stop the CA
and intermediate certificates from being selected. I don't think there
would be much use in a client certificate which has no associated
private key, so this should ensure the client certificate is selected. I
suppose it may be possible for a pkcs12 file to contain multiple
certificates with private keys and the new behaviour may not guarantee
which is selected. However, this is no worse that the previous behaviour
in which any certificate may been selected.

The CERT_FIND_HAS_PRIVATE_KEY is only available in Windows 8 / Server
2012 (aka Windows NT6.2). For older versions, we will fall back to using
the CERT_FIND_ANY flag.

Closes #16825
2025-04-15 23:27:40 +02:00
Sören Tempel
fbdb1e1dbe
http: in alt-svc negotiation only allow supported HTTP versions
Without this patch, the handling of the alt-svc header added via
279a4772ae in curl-8.13.0 attempts to
connect to alternative services via different HTTP versions, even if the
target HTTP version is not supported by curl (i.e., not enabled at
compile-time). If I understand the code and RFC 7838 correctly, then we
should only attempt to migrate to supported protocols. Therefore,
`allowed_apns` should only contain such protocols, and we need to guard
its modification with `ifdefs` for supported HTTP versions.

This was discovered in a downstream bug report in Alpine Linux [1] where
it was reported that a Matrix client (using libcurl) was defunct after
the upgrade to curl-8.13.0. Further debugging revealed that this was due
to the Matrix server sending a `alt-svc: h3=":443";` HTTP header,
causing curl to attempt migration to HTTP3 even though Alpine's curl
version is compiled without HTTP3 support.

I am not sure if this is the best place in the code to address this
or if the `allowed` bitmask shouldn't contain unsupported versions
in the first place. However, since there are existing `ifdefs` in
this function for source (not destination) ALP selection, it may
be a good fit to address this here.

[1]: https://gitlab.alpinelinux.org/alpine/aports/-/issues/17062

Closes #17037
2025-04-15 23:09:10 +02:00
Viktor Szakats
a0ebac0130
GHA/windows: bump a job to windows-2025
- bump an MSYS2/mingw job to windows-2025 runner.
  (MSVC is possible, but vcpkg needs to build for windows-2025, and
  can't share these with windows-2022 builds, so not optimal for
  a single canary job.)
- skip installing OpenSSH-Windows-builtin on windows-2025.
  It's preinstalled:
  ```
  ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
  ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 9.5.0
  ```
  Still older than the manual preview install (9.8.1), so keep using that.

Closes #17066
2025-04-15 18:30:44 +02:00
Cole Helbling
2404a11d85
curl_get_line: handle lines ending on the buffer boundary
Very similar to 9f8bdd0eae, but affects
e.g. netrc file parsing.

Suggested-by: Graham Christensen <graham@grahamc.com>

Add test 744 to verify

Closes #17036
2025-04-15 17:48:32 +02:00
Stefan Eissing
39326f8ae6
easy_reset: fix dohfor_mid member
On an easy reset, the dohfor_mid must be reset to -1.

Reported-by: epicmkirzinger on github
Fixes #17052
Closes #17058
2025-04-15 17:31:17 +02:00
Stefan Eissing
6ab5afbc36
dict: move internal defines to dict.c
Move defines only used in dict.c from urldata.h to implementation.

Closes #17060
2025-04-15 17:30:02 +02:00
Stefan Eissing
a5be8e2c3f
tool_cb_write.c: handle EINTR on flush
Report-and-patch-by: Nils Goroll
Fixes #17061
Closes #17063
2025-04-15 17:28:33 +02:00
Viktor Szakats
01c429c4a8
cmake: merge CURL_WERROR logic into PickyWarnings.cmake
Safe to do this now, as the code no longer relies on setting these
options after feature detection.

Also: Tidy up the way we handle options not to be passed to feature
checks, and make sure to show them in the configure log.

Follow-up to e86542038d #17047
Closes #17062
2025-04-15 14:48:48 +02:00
Viktor Szakats
00e8ebf567
wolfssl: fix to enable ALPN when available
wolfSSL headers publish the `HAVE_ALPN` macro to tell if it has ALPN
support compiled in. Use that instead of `HAS_ALPN`, which was never
set.

Follow-up to edd573d980 #16167
Closes #17056
2025-04-15 12:34:18 +02:00
Viktor Szakats
3fbabec53c
tests/server: fix typo in comment [ci skip] 2025-04-15 02:24:58 +02:00
Viktor Szakats
111b58fbb6
cmake: append picky warnings to CMAKE_REQUIRED_FLAGS as string
Also:
- drop unnecessary type conversion. `CMAKE_REQUIRED_FLAGS` is already
  space-separated.
  https://cmake.org/cmake/help/latest/module/CheckCSourceCompiles.html

Follow-up to e86542038d #17047
Closes #17055
2025-04-15 01:49:56 +02:00
Viktor Szakats
4e203f65a1
processhelp.pm: always call taskkill with -f (force)
In the hope this avoid a possible hang in `taskkill`.

To kill processes, `runtests` first tries to kill them gently (with
"TERM", or on Windows `taskkill`), then waits some time for them
to disappear and then kills them with `KILL`, or on Windows with
`taskkill -f`. This happens within `killpid()`.

This patch bumps the gentle phase to `taskkill -f`. On the obervation
that a non-forced `taskkill` may hang in cases:

msvc, CM x64-windows wolfssl +examples:
```
  [...]
  test 3006...[SMTP with multiple invalid (all) --mail-rcpt and --mail-rcpt-allowfails]
  --p----e--- OK (1682 out of 1718, remaining: 00:04, took 0.524s, duration: 03:13)
  test 3005...[SMTP with multiple and invalid (all but one) --mail-rcpt and --mail-rcpt-allowfails]
  --p-u--e-Executing: 'taskkill -t -pid 1196 >nul 2>&1'
```
Ref: https://github.com/curl/curl/actions/runs/14445993473/job/40508986059?pr=17051#step:15:4176

Cancelling the job worked, resulting in a greyed out status, with the above
step and log entries lost.

If this change causes issues or does nothing at all, we may revert it
or limit it to CI runs.

Ref: #14854
Closes #17054
2025-04-15 01:20:24 +02:00
Viktor Szakats
160656410d
make: clean tests better (tunits)
Sync clean target with other test bundles.

Follow-up to d3761bb840 #16986
Closes #17053
2025-04-15 01:20:23 +02:00
Viktor Szakats
1d0e19a64d
tests: fixup tunit tests for cmake
Follow-up to 461ebbd336 #16983

Closes #17051
2025-04-14 21:31:39 +02:00
Viktor Szakats
e86542038d
cmake: prefer COMPILE_OPTIONS over CMAKE_C_FLAGS for custom C options
Also:
- pass `-D_GNU_SOURCE` via `COMPILE_DEFINITIONS`.
- make it explicit to pass these C flags to feature checks.
- update `_GNU_SOURCE` comment with `pipe2()`.
- enable `-pedantic-errors` picky option for GCC with CMake <3.23.
- drop redundant condition when stripping existing MSVC `/Wn` options.

CMake passes `CMAKE_C_FLAGS` to targets, feature checks and raw
`try_compile()` calls. With `COMPILE_OPTIONS`, this is limited to
targets, and we must explicitly pass them to feature checks. This
makes the build logic clearer, and offers more control. It also
reduces log noise by omitting these options from linker commands,
and from `CMAKE_C_FLAGS` dumps in feature checks.

Closes #17047
2025-04-14 21:31:39 +02:00
Stefan Eissing
d9ca7ad5cb
tests: add git ignores for tests/tunit generated files
Closes #17049
2025-04-14 14:15:05 +02:00
Viktor Szakats
ae1a861bd6
cmake: revert CURL_LTO behavior for multi-config generators
To avoid having LTO enabled for Debug configurations with multi-config
generators (e.g. MSVC.)

Reported-by: PleaseJustDont
Fixes #17042
Ref: ##17034
Follow-up to a1eaa12a83 #15829
Closes #17043
2025-04-14 02:08:24 +02:00
Viktor Szakats
c2a45bf682
runtests: split SSH_PWD into SCP_PWD and SFTP_PWD, and more
To allow configuring paths styles for SCP and SFTP servers separately.

- make `scp://` URLs use `%SCP_PWD` (was: `%SSH_PWD`).
- make `%SCP_PWD` equal to `%POSIX_PWD`.
  To fix test 3022 with OpenSSH-Windows 9.8.0 server.
  The fix works on a local machine. Remains broken in CI.
  Before this patch, it was equal to `%FILE_PWD` when using
  OpenSSH-Windows, otherwise it was `%POSIX_PWD`.
  Notice that no matter what path-style we pass, test 3022
  was and still is broken with earlier OpenSSH-Windows versions.
  (as tested with 9.5.0, 9.5.0-beta20240403, 8.0.0.1)
- rename rest of `%SSH_PWD` uses to `%SFTP_PWD`.
- drop unused `%POSIX_PWD`.
- GHA/windows: test with OpenSSH-Windows server again.
  In the LibreSSL MSVC job. This job is short enough to fit the slow
  install of the built-in OpenSSH-Windows tools, if needed.

Follow-up to 1abb087a9c #5298
Ref: #16803
Closes #17041
2025-04-14 02:08:23 +02:00
Viktor Szakats
f81647db0b
GHA/windows: add support for built-in OpenSSH-Windows
On the windows-2022 runner it installs these client/server versions:
```
ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 8.1.0
```

Not currently enabled. Slight downside (when enabled) that Windows needs
over 1 minute to install these two tiny programs.

Closes #17046
2025-04-14 00:05:17 +02:00
Viktor Szakats
d163c7cbd1
GHA/windows: bump Cygwin action, move package store to D:
- to benefit from the new download retry mechanism.
  https://github.com/cygwin/cygwin-install-action/pull/26

- to use a new setting that not only moves the Cygwin install target
  directory to the faster `D:` drive, but also the package download
  directory. Expecting a little performance improvement from this for
  the Cygwin install step.
  d3a7464b92
  https://github.com/cygwin/cygwin-install-action/pull/27

Closes #17040
2025-04-12 20:59:44 +02:00
Viktor Szakats
14d4712db7
cmake: use the LINK_OPTIONS property with CMake 3.13+
Replacing the superseded `LINK_FLAGS` in these versions.

Follow-up to 7b14449790 #14378
Closes #17039
2025-04-12 20:59:44 +02:00
bruce.yoon(윤병조)/kakao
4622099690
cmake: fix nghttp3 static linking with USE_OPENSSL_QUIC=ON
Though cmake finds nghttp3 by pkg-config, nghttp3 isn't linked properly.
Because library directory is not given by -L.

Closes #17010
2025-04-12 20:58:51 +02:00
Stefan Eissing
ff37657e4d
cpool/cshutdown: force close connections under pressure
when CURLMOPT_MAX_HOST_CONNECTIONS or CURLMOPT_MAX_TOTAL_CONNECTIONS
limits are reached, force close connections in shutdown to go below
limit when possible.

Fixes #17020
Reported-by: Fujii Hironori
Closes #17022
2025-04-11 22:46:56 +02:00
Daniel Stenberg
9f8bdd0eae
tool_parsecfg: make get_line handle lines ending on the buffer boundary
Add test 743 to verify.

Fixes #17030
Reported-by: Marius Kleidl
Closes #17031
2025-04-11 22:38:49 +02:00
Daniel Stenberg
d364f1347f
configure: catch asking for double resolver without https-rr
It is probably an unintentionally bad setup.

Found-by: Stefan Eissing
Closes #17025
2025-04-11 15:27:11 +02:00
Daniel Stenberg
a1413b4a27
docs/INSTALL.md: drop reference to removed configure option
--disable-pthreads has been removed

Fixes #17023
Reported-by: mschroeder-fzj on github
Closes #17026
2025-04-11 15:25:44 +02:00
Daniel Stenberg
960984263f
docs/libcurl: make examples build with picky compiler options
Found by improving verify-examples.pl:

- Operate directly on markdown files to remove the need to render nroff files
  first.

- Add -Wall as a compiler option to find more issues

Closes #17028
2025-04-11 15:23:51 +02:00
Pavel Kropachev
d9a86b6729
docs: add missing return statement in examples
Closes #17024
2025-04-11 14:41:28 +02:00
Viktor Szakats
4646976f3b
GHA/macos: re-generate gcc-13 hacklayer on macos-15
Add workaround for an issue related to the gcc "hacklayer" after the
GitHub macos-15-arm64 runner bumped to 20250408.1231.

Fixes:
```
configure:5175: gcc-13 -o conftest  --sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk  -w conftest.c  >&5
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:71,
                 from /opt/homebrew/Cellar/gcc@13/13.3.0/lib/gcc/13/gcc/aarch64-apple-darwin24/13/include-fixed/stdio.h:75,
                 from conftest.c:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h: In function 'fmemopen':
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:457:107: error: expected declaration specifiers before '__API_AVAILABLE_GET_MACRO_93585900'
  457 | FILE *fmemopen(void * __restrict __buf _LIBC_SIZE(__size), size_t __size, const char * __restrict __mode) __API_AVAILABLE(macos(10.13), ios(11.0), tvos(11.0), watchos(4.0));
      |                                                                                                           ^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/14378524390/job/40316589059?pr=17012#step:7:169

Assisted-by: Bo Anderson
Bug: https://github.com/curl/curl/pull/17012#issuecomment-2792572344
Bug: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2792601570

Closes #17017
2025-04-10 22:00:17 +02:00
Viktor Szakats
0e8bf75b5c
tests: require IPv6 for 1265, 1324, 2086
And also require HTTP. Also add `IPv6` to the keywords.

Fixing:

Linux AM openssl !ipv6 !--libcurl:
```
FAIL 1265: 'NO_PROXY with IPv6 numerical address' HTTP, HTTP proxy, http_proxy, NO_PROXY, noproxy
FAIL 1324: 'HTTP with --resolve and [ipv6address]' HTTP, HTTP GET, --resolve
FAIL 2086: 'Pre-request callback for HTTP IPv6' HTTP, IPv6
```
Ref: https://github.com/curl/curl/actions/runs/14378524385/job/40318328714?pr=17012#step:41:3789

Follow-up to a09e49168a #17005

Closes #17014
2025-04-10 16:24:52 +02:00
Viktor Szakats
e853ea9511
dist: drop duplicate entry from CMAKE_DIST
Closes #17012
2025-04-10 15:01:46 +02:00
Daniel Stenberg
a09e49168a
runtests: remove server verification after start
Since we start the server on our own port we know the server running is
us. By removing unnecessary verification we speed up tests a little.

Closes #17005
2025-04-10 09:12:49 +02:00
Daniel Stenberg
e20b2f3e10
RELEASE-NOTES: synced 2025-04-10 08:45:02 +02:00
Stefan Eissing
219302b4e6
openssl-quic: fix shutdown when stream not open
Check that h3 stream had been opened before telling nghttp3 to
shut it down.

Fixes #16998
Reported-by: Demi Marie Obenour
Closes #17003
2025-04-10 08:38:58 +02:00
Daniel Stenberg
4a9657a890
KNOWN_BUGS: fix link in sivg4 issue 16.3
Fixes #17007
Reported-by: Demi Marie Obenour
Closes #17009
2025-04-10 08:29:53 +02:00
Jake Yuesong Li
320eed00a4
HTTP3.md: fix incorrect variable placeholders
Closes #17008
2025-04-10 08:18:56 +02:00
Viktor Szakats
6af7ab3b39
cmake: quotes, whitespace, use VERSION_GREATER_EQUAL
- `NOT` + `VERSION_LESS` -> `VERSION_GREATER_EQUAL`
  Available since 3.7, which is the minimum required for curl:
  https://cmake.org/cmake/help/latest/command/if.html#version-greater-equal
- make `CMAKE_REQUIRED_*` argument quotes consistent.
- make `CMAKE_REQUIRED_*` space alignment consistent.
- drop quote from version value for consistency with other cases.
- formatting

Closes #17002
2025-04-10 01:42:59 +02:00
Viktor Szakats
2485a2d100
tests/ech_tests.sh: sync shebang with rest of bash scripts
Closes #17001
2025-04-10 01:42:59 +02:00
Viktor Szakats
3c868fbf7f
certs: drop unused default_bits from .prm files
Cert generation do not use these default values, some were also low,
and they were RSA-specific, and the generator recently switched to ECC.

Closes #16999
2025-04-10 01:42:59 +02:00
Daniel McCarney
2ade14b666
build: check required rustls-ffi version
Try to enforce that the Rustls vTLS backend is only used with
rustls-ffi 0.15 - the documentation already describes this as
the required version.

Follow-up from https://github.com/curl/curl/issues/16890

Closes #16922
2025-04-09 09:29:20 +02:00
Viktor Szakats
304b01b8cf
cmake: use INCLUDE_DIRECTORIES prop to specify local header dirs
To use more modern cmake, and make it somewhat more obvious where these
header directories should apply.

Also move setting the directory property _before_ defining targets,
to make them inherit this directory property.

Ref: https://cmake.org/cmake/help/latest/command/include_directories.html
Ref: https://cmake.org/cmake/help/latest/prop_dir/INCLUDE_DIRECTORIES.html

Follow-up to 45f7cb7695 #16238

Closes #16993
2025-04-08 19:09:58 +02:00
Daniel Stenberg
625f2c1644
lib: include files using known path
by including headers using "../[header]" when done from C files in
subdirectories, we do not need to specify the lib source dir as an
include path and we reduce the risk of header name collisions with
headers in the SDK using the same file names.

Idea-by: Kai Pastor

Ref: #16949
Closes #16991
2025-04-08 17:00:00 +02:00
dependabot[bot]
7e4d516bcb
GHA: bump rojopolis/spellcheck-github-actions to 0.48.0
Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.47.0 to 0.48.0.
- [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases)
- [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md)
- [Commits](ed0756273a...23dc186319)

---
updated-dependencies:
- dependency-name: rojopolis/spellcheck-github-actions
  dependency-version: 0.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Closes #16988
2025-04-08 16:58:10 +02:00
renovate[bot]
c8014fd978
GHA: update openssl/openssl to v3.5.0
Closes #16997
2025-04-08 16:57:08 +02:00
Johan Eliasson
23150149f6
docs: fix incorrect shell substitution in docker run example command
Corrected the volume mount path in the Docker run example by replacing
`(pwd)` with the shell substitution syntax `$(pwd)`. This ensures the
current working directory is properly mounted into the container.

Closes #16990
2025-04-08 16:13:30 +02:00
renovate[bot]
cb9b4a2c97
Dockerfile: update debian:bookworm-slim Docker digest to 4b44499
Closes #16992
2025-04-08 16:11:11 +02:00
Daniel Stenberg
eeed87f056
mk-ca-bundle.pl: follow redirects
The Mozilla hosted files have started to redirect. Follow them to restore
script functionality.

Reported-by: Harry Sintonen
Closes #16995
2025-04-08 11:47:05 +02:00
Daniel Stenberg
461ebbd336
tests/tunit: make a separate directory for tool-based unit tests
Separated from library based unit tests to not confuse memory management
etc. Move 1394 and 1604 there.

Closes #16983
2025-04-08 08:08:05 +02:00