Commit Graph

35949 Commits

Author SHA1 Message Date
Viktor Szakats
e5a965b676
cmake: avoid 'target is imported but not globally visible' when consuming libcurl with old cmake
Fixes:
```
CMake Error at bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:62 (add_library):
  add_library cannot create ALIAS target "CURL::libcurl" because target
  "CURL::libcurl_shared" is imported but not globally visible.
Call Stack (most recent call first):
  CMakeLists.txt:39 (find_package)
```

tests/cmake reproducer (requires #16973):
```shell
export CMAKE_CONSUMER=/path/to/CMake-3.12.0/bin/cmake
./test.sh find_package
```

I don't understand what this error says, why it happens in certain CMake
versions, and why a workaround is necessary for what seems like
a standard export/consume configuration. This patch is based on internet
suggestions and other projects ending up with this workaround.

Cherry-picked from #16973
Closes #17140
2025-04-22 19:07:03 +02:00
Viktor Szakats
d60c9aec42
openssl-quic: avoid potential -Wnull-dereference, add assert
Seen with curl-for-win, OpenSSL QUIC, gcc 14.2.0, cmake unity mode.

Silences:
```
In file included from _x86-win-ucrt-bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_5_c.c:55:
In function 'cf_osslq_check_and_unblock',
    inlined from 'cf_progress_egress' at lib/vquic/curl_osslq.c:1730:12:
lib/vquic/curl_osslq.c:1581:11: error: potential null pointer dereference [-Werror=null-dereference]
 1581 |           nghttp3_conn_unblock_stream(ctx->h3.conn, stream->s.id);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/vquic/curl_osslq.c:1582:34: error: potential null pointer dereference [-Werror=null-dereference]
 1582 |           stream->s.send_blocked = FALSE;
      |                                  ^
```

Co-authored-by: Daniel Stenberg
Co-authored-by: Stefan Eissing

Closes #17107
2025-04-22 16:50:12 +02:00
Viktor Szakats
a366552243
GHA: always use Ninja on macos runners
It's installed by default on the macos runners now.

Closes #17115
2025-04-22 16:50:12 +02:00
dependabot[bot]
4c20eb7108
GHA/windows: bump cygwin/cygwin-install-action
to f200932376

Closes #17118
2025-04-22 16:50:12 +02:00
Stefan Eissing
657aae79c0
lib: add meta_hash to connection, eliminate hash_offt
With a meta_hash at each connection (similar to easy handle, let
multi_ev.c store its pollsets as meta data, no longer needing its own
hashes.

This eliminates the last use of Curl_hash_offt. Remove it.

Closes #17095
2025-04-22 15:57:18 +02:00
Stefan Eissing
1d66a769d7
http: fix HTTP/2 handling of TE request header using "trailers"
A "TE" request header is allowed in HTTP/2 when it only carries the
"trailers" value. RFC 9113 ch. 8.2.2. Check client supplied TE values
for the "trailers" token and only pass that one in a HTTP/2 request.

Add test_01_17 to verify.

Fixes #17122
Reported-by: epicmkirzinger on github
Closes #17128
2025-04-22 15:55:36 +02:00
Stefan Eissing
ebe6612505
vquic: unblame netbsd
it was innocent.

Follow-up to 4872dafd80

Closes #17133
2025-04-22 15:54:07 +02:00
Stefan Eissing
2a25abeaf1
connect: shutdown timer fix
Fix a bug in timeout handling for connection shutdowns that led to
default timeout of 2 seconds not being in effect.

Only set the shutdown timeout expiry when operating on a non-admin
transfers. Admin handles are only temproarily tied to a connection.

Fixes #17130
Reported-by: Rasmus Melchior Jacobsen
Closes #17135
2025-04-22 15:52:17 +02:00
Stefan Eissing
24dd825ed5
cares: fix missing lazy init for CURLOPT_DNS_SERVERS
When setting option CURLOPT_DNS_SERVERS, the ares channel was not
properly initialized and the setting failed.

Fixes #17119
Reported-by: calvin2021y on github
Closes #17127
2025-04-22 14:33:41 +02:00
Stefan Eissing
4872dafd80
vquic: init for every call to recvmsg
When calling recvmsg(), always set up the msg structures for
each call as there are OS implemenations that change members
of msg.

Fixes #17120
Reported-by: Harry Sintonen
Closes #17131
2025-04-22 13:46:36 +02:00
Daniel Stenberg
2de9a97141
vquic: consistent name for the stream struct across backends
Now known as "struct h3_stream_ctx" in all four backends.

Also as a bonus: a single definition of the H3_STREAM_CTX macro

Closes #17113
2025-04-22 10:26:04 +02:00
Daniel Stenberg
c4da2bc906
curl/curlver.h: next version is 8.14.0 2025-04-22 09:44:34 +02:00
Daniel Stenberg
17bf1b9d9d
RELEASE-NOTES: synced 2025-04-22 09:43:55 +02:00
Daniel Stenberg
f142056e01
hostip: fix build when no async resolvers are present
Fixes #17124
Closes #17125
2025-04-22 09:18:40 +02:00
Daniel Stenberg
1db267eada
asyn-base: fix build with disabled threaded resolver
... but *with* DoH.

Fixes #17121
Closes #17123
2025-04-22 08:42:56 +02:00
Max Eliaser
7b375b8b91
CURLOPT_ERRORBUFFER.md: the buffer is read only after curl takes ownership
Clarifies #17100
Closes #17105
2025-04-22 08:40:42 +02:00
Daniel Stenberg
ba07dcd27b
lib: use BIT() instead of bool in structs more
Since it makes use of bitfields on supported platforms, it saves a few
bytes memory. Might as well use it consistently.

Closes #17114
2025-04-22 08:01:08 +02:00
Daniel Stenberg
23bed347b3
wcurl: import v2025.04.20 script + docs
Build/install the man page. Install the script.

This an "import" rather than just adding the script into the release tarball
with maketgz to make sure the tarball build is completely reproducible.

wcurl development, issues etc is still to be done at
https://github.com/curl/wcurl

Assisted-by: Viktor Szakats
Assisted-by: Samuel Henrique

Closes #17035
2025-04-21 11:06:44 +02:00
Daniel Stenberg
da9494eef0
tool_getparam: clear argument only when needed
Test 699 verifies

Reported-by: bsr13 on hackerone
Closes #17112
2025-04-20 23:08:11 +02:00
Viktor Szakats
929c9a7ef7
openssl-quic: fix printf mask
Fixes:
```
lib/vquic/curl_osslq.c:835:25: error: format '%zd' expects argument of type 'signed size_t', but argument 6 has type 'curl_off_t' {aka 'long long int'} [-Werror=format=]
  835 |   CURL_TRC_CF(data, cf, "[%" FMT_PRId64 "] DATA len=%zu, total=%zd",
      |                         ^~~~
  836 |               stream->s.id, buflen, stream->download_recvd);
      |                                     ~~~~~~~~~~~~~~~~~~~~~~
      |                                           |
      |                                           curl_off_t {aka long long int}
```

Closes #17106
2025-04-20 13:50:57 +02:00
Viktor Szakats
b12b812196
GHA/linux: tidy up around wolfSSH
- fix filter expression for cache step.
  It did not cause an issue because `wolfssh` is always paired with
  a `wolfssl*`.

- build `wolfssh` against `wolfssl-opensslextra` (was: `wolfssl-all`).
  It makes `wolfssh` builds `OPENSSL_COEXIST`-compatible, and clarifies
  its use in the clang-tidy job. The earlier mixup didn't cause issues
  because the clang-tidy job is compile-only (using their headers only.)
  Useful side-effect is making the wolfssh build valgrind-tested.

Reported-by: bo0tzz on github
Ref: https://github.com/curl/curl/discussions/16970#discussioncomment-12752019

Closes #17109
2025-04-20 13:50:57 +02:00
Viktor Szakats
c4e776cafa
GHA/windows: enable MSYS2 diffutils on ARM64
https://packages.msys2.org/packages/mingw-w64-clang-aarch64-diffutils

Thanks-to: Christoph Reiter
Ref: https://github.com/curl/curl/pull/17067#issuecomment-2816599349

Closes #17103
2025-04-20 13:50:56 +02:00
Viktor Szakats
e325001e52
cmake: link crypt32 for OpenSSL feature detection
Also fix to link the winsock library for WinCE there.

Reported-by: Jixinqi
Fixes #17101
Closes #17102
2025-04-20 09:36:32 +02:00
Viktor Szakats
3f297f1ecc
runtests: add retry option to reduce flakiness
Add `--retry=<num>` option to tell runtests to retry the first `<num>`
tests that failed. Retries aren't run right away, but added to the end
of the test queue. Once all retry slots are used, test fail as normal.

In CI, typically a single test fails for flakiness, and rarely over 5.

Make the `ci-test` targets default to `--retry=5`.

Closes #17091
2025-04-19 10:41:58 +02:00
Daniel Stenberg
3192eeabae
RELEASE-NOTES: synced 2025-04-19 00:05:37 +02:00
Brian Chrzanowski
c0df01fd94
websocket: add option to disable auto-pong reply
This adds another bitflag on CURLOPT_WS_OPTIONS (CURLWS_NOAUTOPONG) that
disables the default and automatic PONG reply in the WebSocket layer.

Assisted-by: Calvin Ruocco

Closes #16744
2025-04-19 00:01:28 +02:00
Stefan Eissing
21fd64645b
quic: no local idle connection timeout, ngtcp2 keep-alive
Do not set a transport parameter idle timeout, meaning we have no such
thing from our side. The remote setting then applies.

In ngtcp2, set its "keep-alive" timer to prevent a possible remote idle
timeout to tear down the connection while we have active transfers on
that connection.

Closes #17057
2025-04-18 23:52:51 +02:00
Daniel Stenberg
1102ea9f4b
cf-https-connect: use the passed in dns struct pointer
... instead of digging it out from the connection struct.

Plus: simplify struct referencing in the condition.

Closes #17092
2025-04-18 23:24:18 +02:00
Daniel Stenberg
834836dc14
doh: reduce the DNS request buffer size
No point in having it larger than the largest allowed request size.

Also removed a no longer used struct.

Closes #17087
2025-04-18 23:10:29 +02:00
sftcd
d6c8907864
doh: httpsrr fix
Closes #17099
2025-04-18 23:03:31 +02:00
Viktor Szakats
f616704980
cmake: fix shell completion install when just one flavor is enabled
Also:
- tidy up the `if` tree.
- drop `include(GNUInstallDirs)` in favor of the upper-level one.

Reported-by: Daniel Engberg
Bug: https://github.com/curl/curl/issues/16946#issuecomment-2814663246
Follow-up to c8b0f0c9ad #16833

Closes #17094
2025-04-18 10:34:11 +02:00
Viktor Szakats
cd0ec4784c
GHA/windows: add 2 basic MSVC jobs to restore some CI coverage
To keep testing these with MSVC:
- UWP !ssl
- arm64 build on the `windows-11-arm` runner
- examples
- OpenSSH-Windows (fix install on `windows-11-arm`)
- `windows-2025` runner
- cmake pre-fill checker

Surprise: UWP doesn't support SSPI, which is required by curl's Schannel
backend. Thus, no TLS support for this UWP build. It also suggests
the Schannel UWP mingw-w64 binaries may be broken and just a happy build
accident thanks to mingw-w64 headers being inaccurate.

Building zlib + libssh2 might actually take up to 2+ minutes with vcpkg,
instead of the previously estimated <1.5 minutes.

Follow-up to e3912f0f9f #17086
Follow-up to 15fb1dc7f8 #17069

Closes #17089
2025-04-17 22:15:54 +02:00
Viktor Szakats
cac50319f9
runtests: recognize lowercase windows in curl -V
As in vcpkg triplets. Test it in CI.

Closes #17088
2025-04-17 19:16:00 +02:00
Viktor Szakats
e3912f0f9f
GHA/windows: restore a simple vcpkg job (zlib + libssh2)
To keep the basics tested. Building zlib and libssh2 takes <1.5 minutes.

So far it seems the "lost" vcpkg binary cache is not a vcpkg-tool
regression, but a GitHub server-side deprecation combined with lack of
replacement service or lack of vcpkg support for such service.

Ref: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Ref: https://github.com/microsoft/vcpkg/issues/45073

Follow-up to 15fb1dc7f8 #17069

Closes #17086
2025-04-17 18:31:54 +02:00
Stefan Eissing
2b6f503570
pytest: make test_07_22 more lenient to exit codes
Depending on timing when the server aborting the connection is detected,
the reported curl exit code may vary. Check for the possible set of
expected codes instead of a single one.

Closes #17083
2025-04-17 17:30:03 +02:00
Stefan Eissing
909af1a43b
multi: do transfer book keeping using mid
Change multi's book keeping of transfers to no longer use lists, but a
special table and bitsets for unsigned int values.

`multi-xfers` is the `uint_tbl` where `multi_add_handle()` inserts a new
transfer which assigns it a unique identifier `mid`. Use bitsets to keep
track of transfers that are in state "process" or "pending" or
"msgsent".

Use sparse bitsets to replace `conn->easyq` and event handlings tracking
of transfers per socket. Instead of pointers, keep the mids involved.

Provide base data structures and document them in docs/internal:
* `uint_tbl`: a table of transfers with `mid` as lookup key,
   handing out a mid for adds between 0 - capacity.
* `uint_bset`: a bitset keeping unsigned ints from 0 - capacity.
* `uint_spbset`: a sparse bitset for keeping a small number of
  unsigned int values
* `uint_hash`: for associating `mid`s with a pointer.

This makes the `mid` the recommended way to refer to transfers inside
the same multi without risk of running into a UAF.

Modifying table and bitsets is safe while iterating over them. Overall
memory requirements are lower as with the double linked list apprach.

Closes #16761
2025-04-17 17:28:38 +02:00
Daniel Stenberg
02e9690c3e
tests/server/dnsd: basic DNS server for test suite
Currently the DNS server only responds to A and AAAA queries. It always
responds with a fixed response: the localhost address. Three times.

It should work fine over either IPv4 or IPv6, but I don't think it
matters much for curl testing.

The idea is to allow curl tests to use "normal" DNS hostnames (using the
normal name resolving code paths) and still use the local test servers.

This setup currently only works if curl is built with c-ares because
redirecting DNS requests to our test server when using getaddrinfo() is
not easy.

This should be extended to respond to HTTPS queries as well to allow
more testing there, as c-ares is always used for that.

Test 2102 is the first test using this.

Closes #17015
2025-04-17 09:13:24 +02:00
Daniel Stenberg
1609570c68
hostip: remove duplicate assignment
Follow-up from 56e40ae6a5

Pointed out by CodeSonar

Closes #17080
2025-04-17 09:12:29 +02:00
renovate[bot]
9cb73d5031
GHA: update ngtcp2/nghttp3 to v1.9.0
Closes #17073
2025-04-16 23:29:08 +02:00
Viktor Szakats
e6ce46ea03
scripts: drop unused import, formatting
Ref: #16840
Closes #17077
2025-04-16 23:07:32 +02:00
Stefan Eissing
5eefdd71a3
vquic: ngtcp2 + openssl support
With the new addition of OpenSSL QUIC API support and the support in
ngtcp2 main branch, make the necessary adjustments in curl to support
this combination.

- add support in configure.ac to detect the feature OPENSSL_QUIC_API2 in
  openssl
- initialise ngtcp2 properly in this combination
- add a Curl_vquic_init() for global initialisation that ngtcp2 likes
  for performance reasons
- add documentation on how to build in docs/HTTP3.md
- add CI testing in http3-linux.yml

Assisted-by: Viktor Szakats
Closes #17027
2025-04-16 22:32:07 +02:00
Viktor Szakats
07cc50f8eb
cmake: add openssl 3.5.0 + ngtcp2 support
```
curl 8.13.1-DEV (Darwin) libcurl/8.13.1-DEV OpenSSL/3.5.0 [...] ngtcp2/1.12.90 nghttp3/1.9.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd
```

Ref: #17027
Closes #17018
2025-04-16 17:03:56 +02:00
Daniel Stenberg
c6db2d45b2
asyn-ares: fix build with old c-ares
See https://app.circleci.com/pipelines/github/curl/curl/13081/workflows/7d80b69e-9886-4f4c-b467-0fa321a0f8c2/jobs/130479

Closes #17075
2025-04-16 16:40:40 +02:00
Stefan Eissing
1ebd92d0fd
async: DoH improvements
Adds a "meta_hash" to each easy handle for keeping special data during
operations. All meta data set needs to add its destructor callback, so
that meta data gets destroyed properly when the easy handle is cleaned
up or reset.

Add data->master_mid for "sub" transfers that belong to a "master" easy
handle. When a "sub" transfer is done, the corresponding "master" can
add a callback to be invoked. Used in DoH name resolution.

DoH: use easy meta hash to add internal structs for DoH name resolution.
One in each in each probe easy handle. When probes are done, response
data is copied from the probe to the initiating easy.

This allows DoH using transfers and their probes to be cleaned up in any
sequence correctly.

Fold DoH cleanup into the Curl_async_shutdown() and Curl_async_destroy()
functions.

Closes #16384
2025-04-16 16:06:03 +02:00
Viktor Szakats
8478365e29
cmake: enable -Wall for MSVC when PICKY_COMPILER=ON
Enable it for `_MSC_VER <= 1943`.

Omit it for not yet CI-tested MSVC versions, to avoid hitting unfixed
warnings emitted by future toolchain releases. It means we need
to explicitly opt-in to newer MSVC versions while fixing any new issues.

The newly enabled warnings did not reveal new issues. It hints that we
catch those with clang/gcc. Yet, these warnings may be useful for local
development done with MSVC.

Also:
- disable and document warnings that don't seem useful, unactionable,
  or unfixable.
- disable and document warnings found in Windows SDK headers.
- tidy up a few comments, also to avoid
  `-Wdocumentation-unknown-command`, part of llvm/clang `-Wall`.
  ```
  lib\dynhds.h(159,29): error : unknown command tag name [-Werror,-Wdocumentation-unknown-command]
  lib\ftp.c(337,15): error : unknown command tag name [-Werror,-Wdocumentation-unknown-command]
  ```
  (This patch did not end up enabling `-Wall` for clang-cl.)

Closes #17050
2025-04-16 14:34:12 +02:00
Viktor Szakats
fbe408ccd7
cmake: do not install mk-ca-bundle script and manpage
To sync with autotools builds.

Reported-by: Daniel Stenberg
Bug: https://github.com/curl/curl/pull/17035#pullrequestreview-2769964979
Follow-up to 5023ffad2c #13197

Closes #17068
2025-04-16 13:37:54 +02:00
Viktor Szakats
15fb1dc7f8
GHA: automatic vcpkg binary cache is broken, temporarily disable vcpkg jobs
In the last 1-2 days, curl's vcpkg binary cache entries stored by GHA
apparently disappeared, and each vcpkg job is now rebuilding everything
on every run. This takes up to 40 minutes per job. New cache entries
fail on upload with messages like:
```
Completed submission of nghttp2:arm64-android@1.65.0 to 0 binary cache(s) in 172 ms
```
(note the zero.)

This coincided with my adding support for windows-11-arm and touching
some jobs affected. Yet, unrelated. It also affects Android jobs. The root
cause is a runner image update bumping the vcpkg tool.

The latest vcpkg tool also has this issue.

As a temporary workaround, stop using vcpkg in CI.

Last good run: https://github.com/curl/curl/actions/runs/14473789417/job/40594304523
First bad run: https://github.com/curl/curl/actions/runs/14474616982/job/40597095025

Assisted-by: Tal Regev
Ref: #17070

Closes #17069
2025-04-16 11:56:15 +02:00
Daniel Stenberg
a92ef2fa15
RELEASE-NOTES: synced 2025-04-16 09:49:54 +02:00
Daniel Stenberg
0a0aec01b5
VERSIONS: 8.14.0 is pending 2025-04-16 09:39:06 +02:00
Christian Schmitz
8ad0243e1f
mqtt: send ping at upkeep interval
Closes #16975
2025-04-16 09:36:19 +02:00