Commit Graph

34784 Commits

Author SHA1 Message Date
Daniel McCarney
b1ba919676
rustls: add ECH support w/ string ECH config
e.g. `curl --tlsv1.3 --ech ecl:<BASE64 encoded ECH config list> ...`

Closes #16828
2025-03-27 08:48:06 +01:00
Daniel McCarney
233b668903
rustls: support ECH GREASE
e.g. `curl --tlsv1.3 --ech grease ...`
2025-03-27 08:47:51 +01:00
Yedaya Katsman
1c8c93ae15
rustls: use client cert and key if available 2025-03-27 08:47:51 +01:00
Yedaya Katsman
1d31f2592f
rustls: Add support for SSLKEYLOGFILE
With rustls-ffi 0.15+ we can set up a callback for writing TLS secrets
hooked up to call Curl_tls_keylog_write. To make sure the associated
file is cleaned up we update the Curl_ssl struct for the rustls-ffi vtls
backend to have a cleanup callback.

Closes #16828
2025-03-27 08:47:43 +01:00
Yedaya Katsman
9ad2d8c557
keylog: move some constants to header file
This allows TLS implementations to use the constants for defining sizes
and verification.
2025-03-27 08:47:43 +01:00
Daniel McCarney
6fa31a3043
ci: use rustls-ffi 0.15 deb
Take the latest rustls-ffi version and install it via the upstream
project's `.deb` for x86_64 linux.
2025-03-27 08:47:43 +01:00
Daniel McCarney
14761d8b61
docs: rework RUSTLS install instructions
Describes using a package manager or pre-built binaries and adopts the
0.15 installation from source instructions.

Previously the rustls docs described installing rustls-ffi from source
using the GNU Makefile. The upstream project has switched to using
cargo-c as a cross-platform solution that works well with the rust
toolchain, pkg-config, and Windows and so this needs an update.

Similarly, for folks that want to avoid the extra cargo-c tool
requirement, rustls-ffi provides binary releases for common platforms,
and some Linux distributions/package managers offer pre-built packages.

The install instructions are expanded to cover these options since
they're generally better for end users than building the dep. from
source (no `rustc` required).
2025-03-27 08:47:43 +01:00
Viktor Szakats
0c20e9bf1a
tests: replace remaining non-ASCII bytes with hex markup
Also drop from `binarycheck`.

Closes #16837
2025-03-26 23:38:41 +01:00
Viktor Szakats
795425f765
configure: tidy up shell completion rules
- allow generating completions in cross-builds.
  Follow-up to d055a01ce9 #16789

- fix warning messages.

- language, quotes, whitespace.

Closes #16836
2025-03-26 15:31:54 +01:00
Viktor Szakats
627e9816ff
cmake: add custom command scripts as dependencies where missing
Also some formatting.

Closes #16835
2025-03-26 15:31:53 +01:00
Daniel Stenberg
39931f7fbe
RELEASE-NOTES: synced 2025-03-26 07:54:09 +01:00
Viktor Szakats
4af9d75783
GHA/non-native: revert to bare builds for Android 21
vcpkg requires Android 28 by default after a recent update that's being
deployed onto CI runs (with `libiconv:arm64-android@1.18#1`).

Revert to bare, no-ssl, no-psl configuration for Android 21 jobs to make
them work again.

Bug: https://github.com/curl/curl/pull/16824#issuecomment-2750912507
Ref: https://github.com/microsoft/vcpkg/pull/44424#issuecomment-2753027630

Closes #16832
2025-03-26 03:31:10 +01:00
Viktor Szakats
1447726663
cmake: document SH_EXECUTABLE option
Closes #16830
2025-03-26 00:54:04 +01:00
Daniel Stenberg
7d1a5c6bdf
tests: remove base64 encoded sections
The base64 sections were added in a time when there was no other way to
do these tests. Now we remove them again to make less "hidden" data in
the test suite.

For several test cases we instead then provide a sequence of binary
octets which is not really better, but I have tried to minimize them.

Closes #16816
2025-03-25 16:47:48 +01:00
Daniel Stenberg
9a7baa34cc
mqtt: cleanups
- make a variable const
- use calloc instead of malloc + memset
- fix comment language
- improve error messages

Closes #16826
2025-03-25 09:18:08 +01:00
Daniel Stenberg
b2078c0e1c
mqtt: convert sendleftovers to dynbuf
Avoid frequent strdups/free calls, including the double-free risk.

Reported-by: Ronald Crane
Closes #16823
2025-03-25 08:22:05 +01:00
Ben Bodenmiller
38b94c80f3
SSLCERTS: list support for SSL_CERT_FILE and SSL_CERT_DIR
Closes #16762
2025-03-25 08:20:39 +01:00
tiymat
d055a01ce9
scripts: update completion.pl to parse options from docs
Reported-by: kpcyrd on github
Fixes #16072
Closes #16789
2025-03-24 23:48:26 +01:00
Gabriel Marin
c329321bf9
easy: add 'populate_fds' func to reduce size of 'wait_or_timeout'
Closes #16820
2025-03-24 23:46:52 +01:00
Daniel McCarney
088f0e6a5b
rustls: tidy up
Closes #16796
2025-03-24 23:45:02 +01:00
Jay Satiro
e601668154 openssl: fix crash on missing cert password
- Return 0 for password length if OpenSSL is expecting a certificate
  password but the user did not provide one.

Prior to this change libcurl would crash if OpenSSL called the certificate
password callback in libcurl but no password was provided (NULL).

Reported-by: Roman Zharkov

Fixes https://github.com/curl/curl/issues/16806
Closes https://github.com/curl/curl/pull/16807
2025-03-24 18:22:53 -04:00
Daniel Stenberg
18f04faef9
asyn-thread: minor cleanups
- Use 'td->' in more places instead of the full thing.

- Remove an assert that is always true since the extra dynamic malloc
  was removed

- Ignore Curl_ares_perform() errors to prioritize the "real" resolver
  info and leave RR as "less important"

Closes #16808
2025-03-24 22:54:39 +01:00
Dave Nicolson
ed0966643d
tool_operate: fix pluralization of seconds
Closes #16751
2025-03-24 22:44:09 +01:00
Stefan Eissing
a95b291ec0
conncache: eliminate cpool's diconnect callback
The callback, provided from url.c did the work that the cshutdn
functionality also implemented. Remove it.

Change some DEBUGF(infof()) to CURL_TRC_M().

Closes #16810
2025-03-24 22:41:53 +01:00
renovate[bot]
ac3c353747
GHA: update mbedtls to v3.6.3
Closes #16822
2025-03-24 22:36:05 +01:00
Viktor Szakats
579625efa5
test615: fix for Cygwin, unignore in CI
Setting a server-side file read-only by `chmod 0444` has does not
prevent overwriting it via SFTP upload (as tested in CI).

Fix it by setting its MS-DOS read-only attribute in addition. It
requires the Cygwin tool `chattr`.

Also unignore in CI.

Fixes:
```
test 0615...[SFTP put remote failure]
curl returned 0, when expecting 9
 615: exit FAILED
=== Start of file stderr615
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 100    30    0     0  100    30      0     93 --:--:-- --:--:-- --:--:--    95

 100    30    0     0  100    30      0     92 --:--:-- --:--:-- --:--:--    92
=== End of file stderr615
```
Ref: https://github.com/curl/curl/actions/runs/14037991918/job/39300723214#step:12:1269

Closes #16818
2025-03-24 16:49:57 +01:00
Viktor Szakats
89101fabf8
GHA/windows: dl-mingw PATH follow-up
Follow-up to 468bfc2618 #16813

Closes #16817
2025-03-24 15:34:00 +01:00
Daniel Stenberg
76d7483fe5
INFRASTRUCTURE.md: add IRC and Matrix details
Closes #16809
2025-03-24 15:21:59 +01:00
Viktor Szakats
ac6f7145a9
GHA/windows: use a pure Cygwin environment
Use the `PATH` `/usr/bin` to avoid any Windows system or 3rd-party tool
installed on the runner machine that may interfere with or add undesired
dependencies to the builds and tests.

Follow-up to d838d43430 #16465
Ref: #16437

Closes #16814
2025-03-24 12:41:52 +01:00
Viktor Szakats
468bfc2618
GHA/windows: boost dl-mingw and cygwin install performance
Install on drive `D:` which has much better write performance than `C:`,
on GitHub Windows runner machines.

- It's bringing down `dl-mingw` installation steps to 5-15s per job,
  from 15s-130s before this patch.

- Saving 30-90s per job in the Cygwin install step.

The before values were fluctuating, but it seems reasonable to expect
saving at least a couple of minutes for each workflow run.

Closes #16813
2025-03-24 12:37:44 +01:00
Viktor Szakats
5cce329873
libssh2: minor tidy-ups
Fix indentation and drop an unnecessary assigment.

Closes #16812
2025-03-24 12:37:44 +01:00
Viktor Szakats
be21c95740
GHA/windows: install OpenSSH-Windows manually for transparency
To have the current latest version, and to avoid the stale, misleading
versions installed by Chocolatey. It also installs transparently, faster,
and making the source of the binaries clear. Install on drive `D:` for
best performance.

After much detective work it turns out that the OpenSSH Windows versions
installed by Chocolatey aren't what they seem:

- The latest pre-release named 9.5.0-beta20240403:
  https://community.chocolatey.org/packages/openssh/9.5.0-beta20240403
  is in reality 8.6.0.0p1-Beta from 2021-05-27:
  https://github.com/PowerShell/Win32-OpenSSH/releases/download/V8.6.0.0p1-Beta/OpenSSH-Win64.zip

- The latest "stable" version 8.0.0.1 is in reality:
  https://community.chocolatey.org/packages/openssh/8.0.0.1
  is in reality 8.0.0.0p1-Beta:
  https://github.com/PowerShell/Win32-OpenSSH/releases/download/v8.0.0.0p1-Beta/OpenSSH-Win64.zip

Ref: https://github.com/curl/curl/pull/16803#issuecomment-2746365654
Follow-up to 67a7775d12 #16704
Follow-up to 0ec72c1ef8 #16672

Closes #16811
2025-03-24 12:37:44 +01:00
Viktor Szakats
e7944fb3da
GHA: extend clang-tidy jobs with more build options, add Windows job
- linux: wolfssl, wolfssh (replacing libssh2), ech, kerberos/GSSAPI,
  ssls-export (libssh2 remains tested on macos.)

- macos: brotli, zstd, c-ares, gnutls, mbedtls, gsasl, rtmp, ssls-export

- windows: new job with schannel, sspi, winidn, winldap, ssls-export

- unit3205: fix/silence remaining NULL dereferences.

Commits fixing the issues found:
cbbccb8b3a #16766
554e4c14be #16777

Closes #16764
2025-03-24 12:37:43 +01:00
Stefan Eissing
c712effda3
multi_ev: fixes regarding connection shutdowns
When handling connections (not transfers) related events, always use the
passed `conn` and not `data->conn` as the transfer does not need to have
the same connection.

Fix handling of conn pollset diffs to disregard the transfer.

Closes #16782
2025-03-24 10:55:40 +01:00
Viktor Szakats
7991b5a85e
memdebug: drop dynamic allocation from curl_dbg_log()
Closes #16745
2025-03-24 10:22:58 +01:00
Viktor Szakats
554e4c14be
windows: fix issues detected by clang-tidy, and some more
- digest_sspi: memory leak.
- digest_sspi: free buffers on `calloc()` fail.
  (not detected by clang-tidy)
- schannel_verify: avoid a `NULL` `alt_name_info`.
- schannel: fix potential `NULL` deref for `backend->cred`.
- schannel: fix uninitialized result value.
  Follow-up to 7f4c358541 #3197
- schannel: drop unused assigment.
- tool_doswin: drop unused assigment.
- testutil: fix memory leak on error.
- testutil: fix memory leak on non-error.
  (not detected by clang-tidy)

Cherry-picked from #16764
Closes #16777
2025-03-24 10:15:08 +01:00
Viktor Szakats
c48c4914a9
tests: build non-debug unit tests with autotools, run them
Before this patch, autotools disabled building unit tests for
non-debug-enabled (`DEBUGBUILD`) builds. runtests skipped running this
combination, though they were built in cmake builds. There seems to be
no technical reason anymore for these restrictions. This patch removes
them, allowing to build and run unit tests for non-debug-enabled builds.

To improve unit test build and run coverage.

- autotools: do not disable building unit tests for non-debug-enabled
  build. Bringing behavior closer to cmake builds. (There are still
  exceptions in autotools, something for another PR)

- runtests: run unit tests for non-debug-enabled builds.
  It extends coverage by 50 tests.

- `lib/altsvc.c`: fix to include `CURL_TIME` support in libcurlu, for
  unit tests. It fixes test 1654, and syncs `CURL_TIME` behavior with
  test 1660 and `lib/hsts.c`.

Ref: 10a7d05be3
Ref: fc8e0dee30 #13694
Ref: 99f78cbf6e #16770

Closes #16771
2025-03-24 10:15:05 +01:00
Viktor Szakats
cbbccb8b3a
mbedtls: re-enable an error check
While adding support for key blobs, a check and error code update moved
after some logic, resulting in the updated code not checked anymore.

Detected by clang-tidy:
```
lib/vtls/mbedtls.c:768:7: error: Value stored to 'ret' is never read [clang-analyzer-deadcode.DeadStores,-warnings-as-errors]
  768 |       ret = MBEDTLS_ERR_PK_TYPE_MISMATCH;
      |       ^
```
Ref: https://github.com/curl/curl/actions/runs/13953249156/job/39057979349?pr=16764#step:12:178

Regression from 05e0453050 #7157
Cherry-picked from #16764
Closes #16766
2025-03-24 10:15:02 +01:00
Viktor Szakats
9465327084
cmake: fix clang-tidy builds to verify tests, fix fallouts
- cmake: disable test bundles for clang-tidy builds.
  clang-tidy ignores #included .c sources, and incompatible with unity
  and bundles. It caused clang-tidy ignoring all test sources. It also
  means this is the first time tests sources are checked with
  clang-tidy. (autotools doesn't run it on tests.)

- cmake: update description for `CURL_TEST_BUNDLES` option.

- fix tests using special `CURLE_*` enums that were missing from
  `curl/curl.h`. Add them as reserved codes.

- fix about ~50 other issues detected by clang-tidy: unchecked results,
  NULL derefs, memory leaks, casts to enums, unused assigments,
  uninitialized `errno` uses, unchecked `open`, indent, and more.

- drop unnecessary casts (lib1533, lib3207).

- suppress a few impossible cases with detailed `NOLINT`s.

- lib/escape.c: drop `NOLINT` no longer necessary.
  Follow-up to 72abf7c13a #13862 (possibly)

- extend two existing `NOLINT` comments with details.

Follow-up to fabfa8e402 #15825

Closes #16756
2025-03-24 10:14:58 +01:00
Viktor Szakats
efa65b24ae
test613: make it pass on Windows, fix postprocess, unignore in CI
- on native Windows (also when using MSYS2 openssh), the group and other
  permissions do not end up as requested by Perl's chmod:
  ```diff
  --- log/8/check-expected
  +++ log/8/check-generated
  @@ -1,3 +1,3 @@
   d?????????    N U         U               N ???  N NN:NN asubdir[LF]
  --rw?rw?rw?    1 U         U              37 Jan  1  2000 plainfile.txt[LF]
  +-rw?r-?r-?    1 U         U              37 Jan  1  2000 plainfile.txt[LF]
   -r-?r-?r-?    1 U         U              47 Dec 31  2000 rofile.txt[LF]
  ```
  Ref: https://github.com/curl/curl/actions/runs/14004029192/job/39215359241?pr=16781#step:15:1596
  Fix it by ignoring group and other attributes.

- fix failing postprocess cleanup by making the read-only test file
  writeable again before deleting it. Fixing:
  ```
  Directory not empty at ../../tests/libtest/test613.pl line 83.
  ```
  (seen on Windows with Git for Windows `perl.exe`)

- unignore in GHA/windows.

Closes #16791
2025-03-24 10:05:17 +01:00
Viktor Szakats
d838d43430
runtests: use setfacl on Cygwin/MSYS, if present
To use a native Cygwin tool instead of the Windows `icacls`. It allows
running under Cygwin/MSYS without Windows system folders in the `PATH`.

Also: fix indentation and tidy up syntax of the `icacls` branch.

Note: As of this commit, these `setfacl` and `icacls` calls are not
necessary for a successful CI run. This includes OpenSSH for Windows
tests, that aren't run by default. Keep them anyway, because locally
they may be necessary depending on environment.

Reported-by: Brian Inglis
Fixes #16437
Ref: #16803
Closes #16465
2025-03-24 10:05:17 +01:00
Weng Xuetian
46c6ca789e
asyn-thread: check thread_data->init in Curl_resolver_getsock
resolver may call destroy_async_data after the name is resolved and
corresponding socketpair is already closed at this point. Any following
call to Curl_resolver_getsock should not set the fd.

Fixes #16799
Closes #16802
2025-03-24 10:03:12 +01:00
Stefan Eissing
f68eae250b
conn: eliminate conn->now
it was only used in pingpong.c to check if the overall transfer has
timed out and we do that with `Curl_timeleft()` in all other places.

Closes #16793
2025-03-24 09:57:38 +01:00
Stefan Eissing
ec4e2cd15d
conncache: eliminate conn->destination_len as premature optimization
Closes #16792
2025-03-24 09:56:13 +01:00
Stefan Eissing
646b2d6ca2
dynbuf: assert init on free
Add a DEBUGASSERT() in Curl_dyn_free() that checks that Curl_dyn_init()
has been performed before.

Fix code places that did it wrong.

Fixes #16725
Closes #16775
2025-03-24 09:53:40 +01:00
Viktor Szakats
cd7eb9e0f2
libssh2: fix to ignore known_hosts if SHA256 host public key is set
Syncing behavior with MD5 host public keys.

libcurl implemented to force a host key type for hosts is present in
`known_hosts`, and disabled this logic when an MD5 host public key is
explicitly set. libcurl later received support for SHA256 host public
keys. This update missed to extend the `known_hosts` logic with the new
key type.

This caused test 3022 to fail if a pre-existing `known_hosts` listed
the test server IP (127.0.0.1) with a non-RSA host key algo.

Follow-up to d1e7d9197b #7646
Follow-up to 272282a054 #4747

Closes #16805
2025-03-24 09:47:59 +01:00
Viktor Szakats
581a7b07a6
test612: SCP rm the uploaded remote file (not the local source), unignore in CI
It accidentally worked on all CI-tested operating systems, except on
native Windows.

Fixing:
```
=== Start of file stderr612
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
[...]
 curl: (21) rm command failed: Operation failed
```
Ref: https://github.com/curl/curl/actions/runs/14004029192/job/39215359241?pr=16781#step:15:1424

Also remove this test from the ignore list in GHA/windows.

Closes #16801
2025-03-24 09:47:59 +01:00
Daniel Stenberg
8ab3deedf4
RELEASE-NOTES: synced 2025-03-24 09:33:30 +01:00
Viktor Szakats
7d64601a8c
sshserver.pl: adjust AuthorizedKeysFile2 cutoff version
To avoid in `ssh_server.log`:
```
reprocess config line 6: Deprecated option AuthorizedKeysFile2
```
with openssh 7.1

Closes #16787
2025-03-23 20:26:41 +01:00
Viktor Szakats
72b163c301
runtests: support running tests under wine or qemu
To run curl, tests and servers via `wine`:
```shell
export CURL_TEST_EXE_RUNNER=wine
```
runtests prefixes commands with the specified runner. For systems where
this isn't automatic or supported, e.g. macOS.

Closes #16785
2025-03-23 20:26:38 +01:00