Commit Graph

33723 Commits

Author SHA1 Message Date
Jan Venekamp
1e03d4bc0b
rustls: add support for setting TLS version and ciphers
Add support for CURLOPT_SSLVERSION, CURLOPT_TLS13_CIPHERS and
CURLOPT_SSL_CIPHER_LIST.

Closes #14535
2024-08-16 09:55:02 +02:00
Viktor Szakats
0d8fdd1c74
cmake: add wolfSSH support
Enable with CMake option `-DCURL_USE_WOLFSSH=ON`. Customize with
`-DWOLFSSH_INCLUDE_DIR=<path-to-wolfssh>/include`,
`-DWOLFSSH_LIBRARY=<path-to-wolfssh>/lib/libwolfssh.a`.

It requires the wolfSSL TLS backend.

Closes #14568
2024-08-16 09:36:57 +02:00
Jan Venekamp
3ff147f8ba
cmake: TLS 1.3 warning only for bearssl and sectranp
Closes #14566
2024-08-16 09:36:37 +02:00
Daniel Stenberg
dcb51bafab
splay: use access functions, add asserts, use Curl_timediff
- add set/get functions for the custom data in a tree node

- use Curl_timediff for time comparisons instead of the custom macro, as they
  do the exact same things.

- add asserts to catch mistakes better

- updated test 1309 accordingly

Closes #14562
2024-08-16 09:12:13 +02:00
Stefan Eissing
6c5a7af754
scorecard: tweak request measurements
Increase max-parallel up to 300, the curl max

Tweak output to just give the http response code

Closes #14564
2024-08-16 09:10:09 +02:00
Daniel Stenberg
20aa8d8f31
docs/internals: new subdirectory
For libcurl internal APIs and concepts.

Closes #14552
2024-08-16 08:59:25 +02:00
Daniel Stenberg
3932964793
test1707: output diff more for debugging differences in CI outputs 2024-08-16 08:57:19 +02:00
Daniel Stenberg
0066d169e8
managen: wordwrap long example lines in ASCII output
The entire ASCII version of the manpage word wraps at a fixed column,
while example command lines can easily go wider than so.

This change now makes manage work on wrapping long example command lines
to make them look nicer. And also to avoid triggering the build error
caused by too long lines in the output.

Quoted lines cannot be wrapped, so managen now errors out if they are
"too long". With this addition, the 'maxline' script is removed as it is
no longer needed.

Closes #14543
2024-08-16 08:57:19 +02:00
Jan Venekamp
178e8ba2d1
cmake: fix find rustls
Follow-up to 2784801977 #14545
Closes #14567
2024-08-15 23:09:09 +02:00
Daniel Stenberg
160f023359
multi: on socket callback error, remove socket hash entry nonetheless
Previously an error from the callback accidentally made libcurl keep the
hash entry which would lead to the entry remaining and then potentially
getting removed *again* which would lead to internal confusions.

This is an old issue (introduced in 2b3dd01b77), caught by the new
asserts from c0233a35da.

Closes #14557
2024-08-15 18:49:16 +02:00
Viktor Szakats
ef1d606d1b
libcurl.pc: add reference to libgsasl
Closes #14556
2024-08-15 15:30:09 +02:00
Viktor Szakats
b042d5297d
tidy-up: misc spelling (bit, ASCII)
Closes #14559
2024-08-15 15:30:09 +02:00
Daniel Stenberg
551baf7d64
tests: move the disabling of 500 for hyper from CI to DISABLED
Follow-up to 136504195a

Closes #14551
2024-08-15 14:02:40 +02:00
Daniel Stenberg
5603204448
curl: fix the -w urle.* variables
urle.scheme, urle.user, urle.password and urle.options mistakenly
operated on the original URL instead of the *effective* (last) URL.

Add test 474 to verify.

Reported-by: Gruber Glass
Fixes #14550
Closes #14560
2024-08-15 14:01:24 +02:00
Viktor Szakats
2401ee68a4
cmake: show warning if libpsl is not found
Also:
- explicitly disable libpsl in CI to avoid configure warning, where
  necessary.
- add TODO to make this warning an error (to match autotools.)

Follow-up to 2998874bb6 #12661

Closes #14533
2024-08-15 10:38:46 +02:00
Stefan Eissing
c90a3f16b8
mime: avoid inifite loop in client reader
Curl_mime_read() may go into an infinite loop when called with buffer
lengths <= 4. Some encoders, like base64, are not prepared for that.

In the client reader for mime data, skip such small reads. The upload
buffers will get flushed eventually and larger reads will happen again.

Improves robustness of test652 which triggered the loop on blocked
sends.

Closes #14532
2024-08-15 10:16:55 +02:00
Viktor Szakats
9f23c8f201
cmake: fix and tidy up c-ares builds, enable in more CI jobs
- set `CARES_NO_DEPRECATED` to avoid c-ares deprecation warnings.
  Like autotools does.

- drop unused c-ares header directory when building libtests and test
  servers.

- disable TrackMemory (aka `CURLDEBUG`) feature for MSVC builds.
  It fails on test 558 and 1330, saying that TrackMemory isn't working.
  Left a FIXME about it.

- GHA/macos: enable c-ares in a cmake job.

- GHA/windows: enable c-ares in MSVC job.
  Fixes #14202

- GHA/windows: add c-ares mingw autotools job with tests.
  (move `--with-windows-unicode` option from 'default' job to
  this one to keep the former "default".)
  Put these tests on ignore for now:
  ```
  FAIL 472: 'aws-sigv4 with query' HTTP, aws-sigv4
  FAIL 1299: 'Send "OPTIONS *" with --request-target' HTTP, --request-target
  FAIL 1613: 'Send "OPTIONS *" with --request-target to a proxy' HTTP, HTTP proxy, --request-target
  ```
  Ref: https://github.com/curl/curl/actions/runs/10388126947/job/28765761441#step:15:5902

  472: fails with mingw despite Unicode enabled.
  1299, 1613: seem like a case of expanding '*' to a filename.

- GHA/windows: enable c-ares in mingw cmake job.

Closes #14541
2024-08-15 09:44:52 +02:00
Viktor Szakats
304a349e8c
GHA/configure-vs-cmake: add macOS build, fix issues
- sync cmake macOS `OS` value manually with the autotools one.
- stop exporting/subst-ing `HAVE_OPENSSL_QUIC` from autotools.
  The variable was only used internally.
- exclude a dependency detection symbol.
- allow to run when the workflow itself was updated.
- simplify cmake command.
- fix indentation.

Closes #14546
2024-08-15 09:29:07 +02:00
Viktor Szakats
2784801977
cmake: add missing pkg-config hints to Find modules
- brotli, c-ares, libpsl, libssh2, mbedtls, rustls:
  Use `pkg-config` for path hints and version info. Syncing them up with
  the rest of Find modules.

- GHA/macos: force-disable libssh2 with cmake to sync with autotools.
  After this patch, cmake auto-detects libssh2 in this job.

Closes #14545
2024-08-15 09:29:06 +02:00
Daniel Stenberg
dd3b3eca5b
RELEASE-NOTES: synced 2024-08-15 09:09:41 +02:00
Alex Snast
136504195a
getinfo: add CURLINFO_POSTTRANSFER_TIME_T
Returns the time, in microseconds, from the start until the last byte is
sent by libcurl (i.e. the request is sent off).

Closes #14189
2024-08-15 09:02:58 +02:00
Daniel Stenberg
c0233a35da
hash: provide asserts to verify API use
- converted the Curl_hash_count() macro to a function

- Discourage accessing struct fields directly

- Document the internal API in HASH.md

Closes #14503
2024-08-15 08:54:19 +02:00
Tal Regev
41a01033b3
GHA/windows: enable HTTP/3 in wolfSSL MSVC job
Closes #14383
2024-08-14 19:00:42 +02:00
Tal Regev
fd662fb3f4
GHA/windows: add GnuTLS to MSVC jobs
- enable GnuTLS in MultiSSL job.

- add new GnuTLS job with HTTP/3 enabled.
  Replacing the `!ssl` job.
  Use `Release` mode to work around a vcpkg package issue, ending up
  depending on both the debug and release versions of the gnutls DLL,
  one of them miss to copy next to the binary.

- cmake: add `pkg-config` detection for GnuTLS.

Co-authored-by: Tal Regev
Fixes #14494
Closes #14495
2024-08-14 18:53:27 +02:00
Jan Venekamp
ed76a23fcc
cmake: add rustls
Closes #14534
2024-08-14 15:05:53 +02:00
Viktor Szakats
db39c668a8
cmake: sync up result variable names in Find modules
- bearssl, c-ares, gss, libpsl, libssh2, mbedtls:
  Before this patch these Find modules returned results via
  `<NAME>_INCLUDE_DIR` and `<NAME>_LIBRARY`.

  This patch makes them return `<NAME>_INCLUDE_DIRS` (note the `S`)
  and `<NAME>_LIBRARIES` like other modules already did.

- bearssl, mbedtls:
  Before this patch these Find modules allowed custom configuration
  via `<NAME>_INCLUDE_DIRS` (note the `S`).

  This patch makes them accept `<NAME>_INCLUDE_DIR`, like the rest of
  the modules did.

  Deprecate the old variables, but keep accepting them for
  compatibility.

- bearssl: add missing `mark_as_advanced()` call.

Closes #14542
2024-08-14 14:58:58 +02:00
Viktor Szakats
65f5caee05
cmake: tidy up Find modules
Smoothen out minor differences between Find modules.

- brotli, nghttp2: drop redundant `FOUND_VAR` specifiers from
  `find_package_handle_standard_args()` calls.
  This function sets both `<NAME_UPPER>_FOUND` and `<NAME>_FOUND`
  by default.

- brotli: set result vars only when found.

- brotli: add missing `mark_as_advanced()` call.

- brotli: delete custom fail message.

- mbedtls, bearssl: use `REQUIRED_VARS` instead of `DEFAULT_MSG`.

- msh3, quiche: set `<NAME>_VERSION` (via pkg-config).

- wolfssl: also use `PC_WOLFSSL_INCLUDEDIR`, `PC_WOLFSSL_LIBDIR`
  as hints.

- libpsl, libssh2, zstd: clear temporary variables used for version
  detection.

- gss, msh3, nghttp2, nghttp3, ngtcp2, quiche, zstd: fix to apply
  `mark_as_advanced()` to internal variables only.

Closes #14538
2024-08-14 12:09:33 +02:00
Viktor Szakats
5abfe451be
cmake: update list of "advanced" variables
To hide them from the CMake GUI by default.

Closes #14540
2024-08-14 12:09:23 +02:00
Stefan Eissing
1c42ea4066
smtp: add tracing feature
Add `smtp` as tracing feature, use CURL_TRC_SMTP() in code to
trace operations.

Closes #14531
2024-08-14 11:51:55 +02:00
Stefan Eissing
8058bbae52
TODO: mqtt and gopher test fails on network blocks
Add descriptions of the test failures and how to reproduce
them to the TODO for both protocols.

Closes #14528
2024-08-14 11:50:56 +02:00
Stefan Eissing
9222f31201
test649: improve robustness
Remove check of upload data before the error is triggered as this
may vary with network conditions.

Closes #14526
2024-08-14 11:50:02 +02:00
Stefan Eissing
e434cdb836
test587: improve robustness
Remove check of server output as upload may abort before request
could fully be sent, so server output may be completely missing.

Test already used a 1 second delay to mitigate timing. This change
makes timing no longer an issue.

Closes #14525
2024-08-14 11:49:05 +02:00
Stefan Eissing
68dad8c4ea
test httpd, tweak cipher list
Configure the AES 256 instead of the AES 128 cipher in the test httpd to
make scorecard testing between httpd and caddy more comparable.

Adapt test_17 expectations, now that AES 128 can no longer, but 256 can
now be negotiated.

Closes #14502
2024-08-14 11:48:35 +02:00
Stefan Eissing
623b877504
gnutls/wolfssl: improve error message when certificate fails
Give more detailed reasons for certificate failures where available in
gnutls and wolfssi to allow user to understand the cause of the failure.

Closes #14501
2024-08-14 11:47:01 +02:00
Stefan Eissing
6905b1f867
hyper: call Curl_req_set_upload_done()
Hyper implementation was missing the call to Curl_req_set_upload_done()
as it works differently than out normal request upload handling.

Closes #14539
2024-08-14 11:34:05 +02:00
Stefan Eissing
22d292b3ec
urldata: introduce data->mid, a unique identifier inside a multi
`data->id` is unique in *most* situations, but not in all. If a libcurl
application uses more than one connection cache, they will overlap. This
is a rare situations, but libcurl apps do crazy things. However, for
informative things, like tracing, `data->id` is superior, since it
assigns new ids in curl's serial curl_easy_perform() use.

Introduce `data->mid` which is a unique identifer inside one multi
instance, assigned on multi_add_handle() and cleared on
multi_remove_handle().

Use the `mid` in DoH operations and also in h2/h3 stream hashes.

Reported-by: 罗朝辉
Fixes #14414
Closes #14499
2024-08-14 11:21:34 +02:00
Daniel Stenberg
ad6320b8a5
tool_paramhlp: bump maximum post data size in memory to 16GB
- stick to 2GB for 32bit systems.

Reported-by: Tim Yuer
Fixes #14521
Closes #14523
2024-08-14 07:57:24 +02:00
Viktor Szakats
8ae7049f4b
cmake: sync up formatting in Find modules
- lowercase internal variable names (FindGSS)
- comments
- whitespace

Closes #14527
2024-08-14 01:44:16 +02:00
Viktor Szakats
1a444e3153
runtests: log ignored but passed tests
List tests explicitly ignored, but passing. It can be useful to catch
ignore rules that are no longer necessary because the tests are not
failing anymore.

To not clutter the default view, hide this information under a GitHub
Actions foldable group.

Closes #14457
2024-08-14 01:44:07 +02:00
Viktor Szakats
d76389d82f
GHA/macos: disable AppleIDN for autotools in combinations jobs
Without libidn2, AppleIDN gets enabled by default in autotools.

To remain minimal and match cmake.

Follow-up to 8de8fe8c98 #14401
2024-08-13 13:35:34 +02:00
Dan Fandrich
0c4f05c6ee tests: don't mangle output if hostname or type unknown
If either of these couldn't be determined, the output would be missing a
newline. Make sure the line ends even in the case of an error.
2024-08-13 01:13:00 -07:00
Viktor Szakats
af73743f86
curl_sha512_256: fix symbol collisions with nettle library
`SHA512_256_BLOCK_SIZE`, `SHA512_256_DIGEST_SIZE` macros were both
defined within curl and also in the nettle library required by GnuTLS.

Fix it by namespacing the curl macros.

Cherry-picked from #14495
Closes #14514
2024-08-13 10:04:06 +02:00
Viktor Szakats
624b20c637
lib: prefer CURL_SHA256_DIGEST_LENGTH over the unprefixed name
Already used in `vtls.h`. Prefer this curl-namespaced name over the
unprefixed `SHA256_DIGEST_LENGTH`. The latter is also defined by TLS
backends with a potential to cause issues.

Also stop relying on externel headers setting this constant. It's
already defined in `vtls.h` on curl's behalf, do this also for `lib`.

Cherry-picked from #14495
Closes #14513
2024-08-13 10:04:06 +02:00
Viktor Szakats
d7e1a2dd7d
lib: avoid macro collisions between wolfSSL and GnuTLS headers
Both of these projects define the same `SHA*` macros via headers
included by curl (in MultiSSL builds, possibly only in Unity builds),
causing redefinition compiler warnings.

Fix it by disabling compatibility macros in wolfSSL.

```
Building C object lib/CMakeFiles/libcurl_static.dir/Unity/unity_0_c.c.o
In file included from _bld/lib/CMakeFiles/libcurl_static.dir/Unity/unity_0_c.c:202:
In file included from lib/http_aws_sigv4.c:33:
In file included from lib/curl_sha256.h:40:
In file included from /usr/local/Cellar/wolfssl/5.7.2/include/wolfssl/openssl/sha.h:30:
/usr/local/Cellar/wolfssl/5.7.2/include/wolfssl/wolfcrypt/sha256.h:117:13: warning: 'SHA256_BLOCK_SIZE' macro redefined [-Wmacro-redefined]
    #define SHA256_BLOCK_SIZE  WC_SHA256_BLOCK_SIZE
            ^
/usr/local/Cellar/nettle/3.10/include/nettle/sha2.h:70:9: note: previous definition is here
#define SHA256_BLOCK_SIZE 64
        ^
In file included from _bld/lib/CMakeFiles/libcurl_static.dir/Unity/unity_0_c.c:202:
In file included from lib/http_aws_sigv4.c:33:
In file included from lib/curl_sha256.h:40:
In file included from /usr/local/Cellar/wolfssl/5.7.2/include/wolfssl/openssl/sha.h:30:
[...]
    #define SHA256_DIGEST_SIZE WC_SHA256_DIGEST_SIZE
    #define SHA224_BLOCK_SIZE  WC_SHA224_BLOCK_SIZE
    #define SHA224_DIGEST_SIZE WC_SHA224_DIGEST_SIZE
    #define SHA512_BLOCK_SIZE  WC_SHA512_BLOCK_SIZE
    #define SHA512_DIGEST_SIZE WC_SHA512_DIGEST_SIZE
    #define SHA384_BLOCK_SIZE  WC_SHA384_BLOCK_SIZE
    #define SHA384_DIGEST_SIZE WC_SHA384_DIGEST_SIZE
```

Cherry-picked from #14495
Closes #14511
2024-08-13 10:04:06 +02:00
Viktor Szakats
5a45e0c562
cmake: update curl-config.cmake.in template var list [ci skip]
Omit those defined by CMake.

Follow-up to a298df7f47 #14462
2024-08-13 10:02:10 +02:00
Viktor Szakats
4111d10803
lib: fix building with wolfSSL without DES support
E.g. with Homebrew wolfssl 5.7.2 on macOS:
```
In file included from _bld/lib/CMakeFiles/libcurl_static.dir/Unity/unity_0_c.c:85:
lib/curl_ntlm_core.c:157:27: error: unknown type name 'DES_key_schedule'
                          DES_key_schedule DESKEYARG(ks))
                          ^
lib/curl_ntlm_core.c:159:3: error: use of undeclared identifier 'DES_cblock'
  DES_cblock key;
  ^
[...]
```

Earlier patch addressing this for OpenSSL:
802d864450 #7808

Cherry-picked from #14495
Closes #14512
2024-08-13 09:28:27 +02:00
Viktor Szakats
28c12bc9be
sha256: fix symbol collision between nettle (GnuTLS) and OpenSSL codepath
Fixes:
```
C:\vcpkg\installed\x64-windows\include\nettle\sha2.h(75,8): error C2011: 'sha256_ctx': 'struct' type redefinition
```
Ref: https://github.com/curl/curl/actions/runs/10341162641/job/28622681573?pr=14484#step:10:30

Cherry-picked from #14495
Closes #14515
2024-08-13 09:28:27 +02:00
Viktor Szakats
71d3ab5813
vtls: fix static function name collisions between TLS backends
When using CMake Unity build.

- use unique name for `set_ssl_version_min_max()`
  Fixes collision between GnuTLS, mbedTLS and SecureTransport.
  ```
  lib\vtls\mbedtls.c(317,1): error C2084: function 'CURLcode set_ssl_version_min_max(Curl_easy *,ssl_peer *,ssl_primary_config *,const char **,const char *)' already has a body
  lib\vtls\mbedtls.c(837,49): warning C4133: 'function': incompatible types - from 'Curl_cfilter *' to 'Curl_easy *'
  lib\vtls\mbedtls.c(837,53): warning C4133: 'function': incompatible types - from 'Curl_easy *' to 'ssl_peer *'
  lib\vtls\mbedtls.c(837,25): error C2198: 'set_ssl_version_min_max': too few arguments for call
  ```

- use unique name for `do_file_type()`
  Fixes collision between GnuTLS, OpenSSL and wolfSSL.
  ```
  lib\vtls\openssl.c(1053,12): error C2084: function 'gnutls_x509_crt_fmt_t do_file_type(const char *)' already has a body
  ```

Ref: https://github.com/curl/curl/actions/runs/10341162641/job/28622681573?pr=14484#step:10:31
Cherry-picked from #14495
Closes #14516
2024-08-13 09:28:27 +02:00
Viktor Szakats
457427e03f
build: silence C4232 MSVC warnings in vcpkg ngtcp2 builds
Silence bogus MSVC warning C4232. Use the method already used
for similar cases earlier.

Also fixup existing suppressions to use pragma push/pop.

```
lib\vquic\curl_ngtcp2.c(709,40): error C2220: the following warning is treated as an error
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'client_initial': address of dllimport 'ngtcp2_crypto_client_initial_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_crypto_data': address of dllimport 'ngtcp2_crypto_recv_crypto_data_cb' is not static, identity not guaran
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'encrypt': address of dllimport 'ngtcp2_crypto_encrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'decrypt': address of dllimport 'ngtcp2_crypto_decrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'hp_mask': address of dllimport 'ngtcp2_crypto_hp_mask_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_retry': address of dllimport 'ngtcp2_crypto_recv_retry_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'update_key': address of dllimport 'ngtcp2_crypto_update_key_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_aead_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_aead_ctx_cb' is not static, identit
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_cipher_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_cipher_ctx_cb' is not static, ide
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'get_path_challenge_data': address of dllimport 'ngtcp2_crypto_get_path_challenge_data_cb' is not static, ident
```
Ref: https://github.com/curl/curl/actions/runs/10343459009/job/28627621355#step:10:30

Cherry-picked from #14495
Co-authored-by: Tal Regev
Ref: #14383
Closes #14510
2024-08-13 09:28:27 +02:00
Viktor Szakats
b910122fe3
cmake: add CURL_USE_PKGCONFIG option
Add option to control whether to use `pkg-config` to detect
dependencies. Curl's CMake uses `pkg-config` by default for all targets
except for MSVC without vcpkg.

With the CMake option `-DCURL_USE_PKGCONFIG=ON` you can override it to
use `pkg-config` always.

If `pkg-config` is causing issues, e.g. in cross-builds or other cases,
`-DCURL_USE_PKGCONFIG=OFF` disables all use of `pkg-config`.

Also add it to `curl-config.cmake`. Not yet used, but will be once curl
starts referencing any curl-specific `Find*` module from this public
script.

Follow-up to 9dfdc6ff42 #14483
Closes #14504
2024-08-13 09:28:27 +02:00