Commit Graph

33723 Commits

Author SHA1 Message Date
Stefan Eissing
fe2a72029e
http2: auto reset stream on server eos
When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RST the stream to signal that the upload is incomplete and that the
whole transfer can be stopped.

Fixes the case where the server responds with 413 on an upload but does
not RST the stream from its side, as httpd and others do.

Reported-by: jkamp-aws on github
Fixes #15316
Closes #15325
2024-10-17 23:03:41 +02:00
Daniel Stenberg
2ae8d9b579
libtests: generate the lib1521 atomically
By renaming from a temporary file name to the .c once completed. This
avoids the risk that the checksrc job tries to verify the file before it
is complete, in parallel build setups.

Reported-by: Dan Frandrich
Fixes #15258
Closes #15327
2024-10-17 23:00:24 +02:00
Daniel Stenberg
b9877b74c5
GHA: drop the hyper job
Hyper support is being removed in 2025. No one works on it. Getting
flaky test runs with this job adds nothing to the project.

Closes #15326
2024-10-17 22:56:45 +02:00
Stefan Eissing
b42eb27c1f
openssl: improve retries on shutdown
Once SSL_shutdown() has been called, OpenSSL does not really seem to
like it when it is called again and the other side has some finally data
to deliver.

Instead SSL_read() needs to be used solely, once the close notify has
been sent from curl's side.

Closes #15321
2024-10-17 16:11:06 +02:00
Daniel Stenberg
8cdbaba4bf
tool_operate: break out of loop on error
Follow-up to 69bf530dfd

The loop could get stuck there in torture tests/OOM.

Closes #15322
2024-10-17 16:04:09 +02:00
Daniel Stenberg
38bfe1c2aa
GHA: switch off proselint
Because we cannot disable the individual warnings we do not care about,
making this tool almost unusable for our purposes. See
https://github.com/amperser/proselint/issues/1367

Instead, make 'very' a banned word (as recently that has been what
proselint most commonly points out for us).

Closes #15314
2024-10-17 13:36:42 +02:00
Daniel Stenberg
9cc246401e
source: avoid use of 'very' in comments 2024-10-17 13:36:33 +02:00
Daniel Stenberg
d1323839be
DISTROS: avoid use of "very" 2024-10-17 13:36:33 +02:00
Daniel Stenberg
193f1b4841
DISABLED: disable test 1060 with hyper
... as it has started to fail and nobody wants to debug this.

Closes #15319
2024-10-17 13:26:00 +02:00
Stefan Eissing
c97cd82821
tests/http: fix ubuntu GnuTLS CI failures
Override the system default config in test_17_09, since we want to check
all TLS versions. Provide own, empty config file to gnutls, so that any
system wide file has no effect.

The latest ubunu image in GH CI disables TLS 1.0 and 1.1
system wide for GnuTLS. Good intentions.

Closes #15310
2024-10-17 13:25:15 +02:00
Daniel Stenberg
beeeb85a78
tests: update some HTTP/2 over HTTPS tests
- improve descriptions
- require http/2, not h2c, since they are done over HTTPS

Closes #15317
2024-10-17 12:55:05 +02:00
Jay Satiro
fde5326293 winbuild/README: document how to clean a build
- Add a new section explaining that a build can be cleaned by adding the
  keyword "clean" to the build command.

- Add an example of using the "x64 Native Tools" prompt to the VS
  command prompt section.

- Update the Legacy Windows section's lack-of-cipher support warning to
  say "Windows 8 and earlier" instead of "Windows XP and earlier".

Ref: https://github.com/curl/curl/discussions/15277

Closes https://github.com/curl/curl/pull/15291
2024-10-16 18:03:06 -04:00
Viktor Szakats
1e01e2b549
GHA/macos: merge autotools and cmake jobs
To match other workflows and to avoid repetition in rules.

Also:
- fix build example step for cmake. update a job to use it.
- use `cmake` to invoke the builds (instead of ninja directly).
- extend test 2100 exclusion to more jobs.
  It fails with all `!debug gcc-12` jobs with autotools.
  With cmake this only happened for gcc-12 Secure Transport jobs
  for some reason.

Closes #15312
2024-10-16 23:25:18 +02:00
Dan Fandrich
a2f913ef6a CI: explicitly specify the OS version when necessary
Commit 8ea120f6 added --break-system-packages which works in Ubuntu
24.04 but not 22.04, so explicitly specify that version in the runner
instead of relying on ubuntu-latest to provide it.  Some runners have
regressed back to 22.04 for ubuntu-latest, resulting in build failures.
2024-10-16 09:56:14 -07:00
Dan Fandrich
41c980bb0d tests: capture stdin to get the vsftpd version number
vsftpd 3.0 at least writes its version number to stdin (!) instead of
stderr. This works due for backwards compatibility reasons in UNIX, so
we must check stdin for anything written there to reliably parse the
version string.

Closes #15278
2024-10-16 09:21:40 -07:00
Viktor Szakats
6478a36b66
src: guard for double declaration of curl_ca_embed in unity builds
Seen with curl-for-win linux-musl-from-mac build with gcc 9.2.0.

```
n file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:136:
/Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/tool_ca_embed.c:4:28: warning: redundant redeclaration of 'curl_ca_embed' [-Wredundant-decls]
    4 | extern const unsigned char curl_ca_embed[];
      |                            ^~~~~~~~~~~~~
In file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
/Users/runner/work/curl-for-win/curl-for-win/curl/src/tool_operate.c:107:28: note: previous declaration of 'curl_ca_embed' was here
  107 | extern const unsigned char curl_ca_embed[];
      |                            ^~~~~~~~~~~~~
```
https://github.com/curl/curl-for-win/actions/runs/11192203640/job/31116070669#step:3:4894

Follow-up to 8a3740bc8e #14059
Closes #15307
2024-10-16 17:27:56 +02:00
Daniel Stenberg
adf2b4fa51
libssh: use CURL_PATH_MAX instead of PATH_MAX
Follow-up to facf59c30e

Reported-by: Viktor Szakats
Bug: https://github.com/curl/curl/pull/15285#issuecomment-2416947731
Closes #15309
2024-10-16 16:27:59 +02:00
Viktor Szakats
7fbcf4b9b7
vquic: fix compiler warning with gcc + MUSL
```
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c: In function 'msghdr_get_udp_gro':
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:344: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  344 | #pragma clang diagnostic push
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:345: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  345 | #pragma clang diagnostic ignored "-Wsign-compare"
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:346: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  346 | #pragma clang diagnostic ignored "-Wcast-align"
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:350: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  350 | #pragma clang diagnostic pop
      |
```
https://github.com/curl/curl-for-win/actions/runs/11356281008/job/31587180874#step:3:9534

Follow-up to a571afc02e #14012
Closes #15303
2024-10-16 12:09:02 +02:00
Daniel Stenberg
facf59c30e
libssh2: use the filename buffer when getting the homedir
Avoids having to use a big stack buffer for this.

Closes #15285
2024-10-16 11:55:52 +02:00
Daniel Stenberg
083b4ab6e4
libssh2: put the readdir buffers into struct
... instead of separate malloc() calls:

- removes two mallocs (and associated error handling paths)
- makes cleanup easier

Also reduce maximum SFTP file path lengths to 1024 bytes universally
everywhere. Using the system's own MAX_PATH did not make sense since
this is mostly about getting a remote file name.

Closes #15285
2024-10-16 11:55:47 +02:00
renovate[bot]
1cf187a4f6
CI: update GHA dependencies
- update Mbed-TLS/mbedtls to v3.6.2
- update libressl-portable/portable to v4
- update github/codeql-action digest to f779452

Closes #15301
Closes #15299
Closes #15188
2024-10-16 08:34:19 +02:00
Viktor Szakats
3040971d12
GHA: silence proselint warnings and an error
Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: https://github.com/curl/curl/pull/15291#issuecomment-2410505100
Closes #15293
2024-10-15 16:44:17 +02:00
Daniel Stenberg
8403e5a701
tests: fix callback signatures to please UndefinedBehaviorSanitizer
Make test applications use the correct prototypes for callbacks.

Closes #15289
2024-10-15 14:33:47 +02:00
Daniel Stenberg
eed3c8f4b7
curl.h: remove the struct pointer for CURL/CURLSH/CURLM typedefs
It makes the callbacks get different signnatures when used from within
libcurl vs outside of it by libcurl-using applications (such as the
libtests) and this triggers UndefinedBehaviorSanitizer errors.

Closes #15289
2024-10-15 14:33:40 +02:00
Daniel Stenberg
ad1c49bc0e
lib: remove function pointer typecasts for hmac/sha256/md5
Make sure we use functions with the correct prototype.

Closes #15289
2024-10-15 14:32:39 +02:00
Michael Kaufmann
335d325708 conncache: More efficient implementation of cpool_remove_bundle
Closes #15292
2024-10-15 10:45:17 +02:00
Viktor Szakats
e20b139a1d
GHA/linux: add cmake job for system mbedTLS with pkg-config
Add a build-only cmake job with system mbedTLS package and `pkg-config`
enabled. Ubuntu 24.04 comes with mbedTLS 2.28.8 which supports
`pkg-config`.

Follow-up to 7bab201abe #15193
Closes #15286
2024-10-15 01:40:20 +02:00
Daniel Stenberg
e33cf006e9
server/mqttd: fix two memory leaks
Closes #15290
2024-10-14 12:47:21 +02:00
Viktor Szakats
8ea120f615
GHA/linux: fixup pip for Ubuntu 24.04
`ubuntu-latest` became `ubuntu-24.04` today.

Closes #15287
2024-10-14 03:17:36 +02:00
Daniel Stenberg
69bf530dfd
tool_operate: make --skip-existing work for --parallel
Reported-by: Tobias Wendorff
Fixes #15261
Closes #15283
2024-10-13 23:19:46 +02:00
Daniel Stenberg
9bee39bfed
url: use same credentials on redirect
Previously it could lose the username and only use the password.

Added test 998 and 999 to verify.

Reported-by: Tobias Bora
Fixes #15262
Closes #15282
2024-10-13 23:18:42 +02:00
Daniel Stenberg
eb77297ccc
lib: move curl_path.[ch] into vssh/
As this contains code only used by SSH backends.

Closes #15284
2024-10-13 23:16:18 +02:00
Stefan Eissing
a7ccd02614
ftp: move listen handling to socket filter
Move the listen/accept handling of the FTP active data connection
into the socket filter and monitor 'connected' status of that as
with passive connections - more or less.

The advantage is that the socket filter now reports being connected
only when the server has actually called and accept() has been done.
This enables to bootstrap the filter chain on the data connection
just like any other. A require SSL filter can then be added right
at the start and does not need to be patched in later.

Still, the active connection keeps on needing special handling in
ftp.c as the control connection needs to be monitored while waiting
as the server might send error responses this way. So, things did
not turn out quite as squeaky clean as hoped for, but still seems
better to do that way.

Closes #14798
2024-10-13 23:15:28 +02:00
Stefan Eissing
3455d360ce
mbedTLS: fix handling of TLSv1.3 sessions
For TLSv1.3, if supported, observer special return code to retrieve
newly arrived session from mbedTLS.

Adjust test expectations now that TLSv1.3 session resumption works in
mbedTLS >= 3.6.0.

Based on #14135 by @ad-chaos
Closes #15245
2024-10-13 23:13:04 +02:00
Stefan Eissing
513904c264
wolfSSL: fix handling of TLSv1.3 sessions
Register a callback to get notified of new SSL sessions by wolfSSL.
Remove the explicit session retrieval after handshake, since this does
not work for TLSv1.3.

Adjust test expectations now that TLSv1.3 session resumption works
in wolfSSL.

Closes #15243
2024-10-13 23:11:39 +02:00
Stefan Eissing
aa43b42461
curl-rustls.m4: set linker flags to allow rustls build on macos
Assisted-by: Viktor Szakats
Closes #15175
2024-10-13 23:10:45 +02:00
Viktor Szakats
960521d210
smb: do not redefine getpid on Windows
Replace with namespaced local macro `Curl_getpid()`.

Redefining symbols can backfire if that symbol is used in system
headers, especially with unity build. We haven't seen a fallout in CI
or supported envs, but do it anyway for good measure.

Bug report: https://datagirl.xyz/posts/wolfssl_curl_w2k.html
Closes #15263
2024-10-13 18:01:51 +02:00
Viktor Szakats
e8a007de0d
GHA: optimize test prereq steps
- Linux: move test and pytest prereqs right before test run.
  - returns build phase results faster.
  - allows skipping steps for jobs that don't need them.
  - makes dependencies more transparent.
- sync prereq install step names.
- use `tests/requirements.txt` more.

Closes #15275
2024-10-13 11:34:59 +02:00
Viktor Szakats
75dfb7b649
pytest: include buildinfo.txt in the output
If present.

It aims to provide TextClutch the same build information that
`runtests.pl` already is providing.

Ref: https://testclutch.curl.se/static/reports/feature-matrix.html
Ref: #15256
Follow-up to 1fdea16846 #14802
Closes #15279
2024-10-13 11:34:26 +02:00
Tal Regev
66cc015756
GHA/windows: drop vcpkg workaround
Reverts 841f42150a #15133
Closes #15217
2024-10-12 22:47:31 +02:00
Viktor Szakats
01a8157997
cmake: tidy-ups and rebase fixups
- limit `SIZEOF_SA_FAMILY_T` detection to non-Windows.
- make sure `sys/socket.h` exists before detecting `SIZEOF_SA_FAMILY_T`.
- limit `mach_absolute_time()` detection to `APPLE`. Drop from Windows
  pre-cache.
- skip `HAVE_LIBSOCKET` detection for Windows, drop pre-cached value.
- drop redundant pre-cached `HAVE_LIBZ` for Windows.
- `curl_required_libpaths()`: stop accepting multiple arguments.
  To prepare for `CMAKE_REQUIRED_LINK_DIRECTORIES` support.
  Follow-up to 7bab201abe #15193
- GSS: fix recent rebase mistakes:
  - fix variable name.
  - do not add a header twice.
  Follow-up to 91d451b488 #15157
- GSS: quote a variable.

Closes #15271
2024-10-12 22:33:29 +02:00
Dan Fandrich
a3601cf571 tests: allow pytests to run in out-of-tree builds
Some of the files it needs are in the build directory but the code did
not make a distinction.

Closes #15257
2024-10-12 10:47:57 -07:00
Viktor Szakats
79809ffe1e
GHA/linux: mbedTLS 3.6.1
Closes #15274
2024-10-12 08:58:19 +02:00
renovate[bot]
ba68eb02fa
CI: update rojopolis/spellcheck, actions/checkout, actions/upload-artifact
- update rojopolis/spellcheck-github-actions digest to 7ff888c
- update actions/checkout digest to eef6144
- update actions/upload-artifact digest to b4b15b8

Closes #15207
Closes #15184
Closes #15187
2024-10-11 23:53:42 +02:00
dependabot[bot]
7d53a59292
CI: bump github/codeql-action, vmactions/omnios-vm and actions/cache
- bump github/codeql-action from 3.26.10 to 3.26.11
- bump vmactions/omnios-vm from 1.0.7 to 1.0.8
- bump actions/cache from 4.0.2 to 4.1.1

Closes #15178
Closes #15179
Closes #15244
2024-10-11 23:44:19 +02:00
Stefan Eissing
fe8399f066
gnutls: use session cache for QUIC
Add session reuse for QUIC transfers using GnuTLS. This does not include
support for TLS early data, yet.

Fix check of early data support in common GnuTLS init code to not access
the filter context, as the struct varies between TCP and QUIC
connections.

Closes #15265
2024-10-11 23:37:47 +02:00
Dan Fandrich
954177b9da tool_xattr: create the user.creator xattr attribute
This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.

Closes #15237
2024-10-11 10:13:37 -07:00
Viktor Szakats
dfd36d3ee0
cmake: apply WIN32_LEAN_AND_MEAN to all feature checks
Enable `WIN32_LEAN_AND_MEAN` for all feature detections on Windows.

(Also drop it from individual detections.)

Cherry-picked from #15164
2024-10-11 17:57:03 +02:00
Viktor Szakats
8e34505776
cmake: untangle feature detection interdependencies
- reduce `check_include_file_concat()` use to those headers that either
  depend on a previously detected header, or another header or symbol
  detection depend on it.

- replace `check_symbol_exists()` with `check_function_exists()` for
  functions that are detected with `AC_CHECK_FUNCS()` in `./configure`.
  This makes `setmode()` no longer be detected with MSYS, syncing
  this with `./configure`. Instead `_setmode()` is used now also in
  CMake MSYS builds. This is consistent with Cygwin builds also.

- add comment about which header/symbol detection depends on what
  header. Based on `./configure` mainly.

- form `CURL_TEST_DEFINES` manually, and include only those macros which
  are actually used in `CMake/CurlTests.c`.

- change `curl_internal_test()` to use `CMAKE_REQUIRED_DEFINITIONS`,
  instead of `CMAKE_REQUIRED_FLAGS` to simplify the logic, and to allow
  dropping the latter macro completely.

- drop `windows.h` from header and symbol checks.

- `./configure`: add comment about whether `netinet/in6.h`, `sys/un.h`
  are indeed meant to be included for all detections. There is a chance
  they were added there by accident.

Detection resuls were cross-checked between
436bbbe7ab (master) and
48ff4694e608ccfdedf7ce5bab2b96d6b2c23cda (this PR), for CI GHA Linux,
Linux HTTP/3, non-native, macOS and Windows jobs.

Closes #15164
2024-10-11 17:44:45 +02:00
Viktor Szakats
7bff686476
ci: dump curl_config.h to log in all jobs
Also:
- GHA/windows: merge full and brief dump into a single job step.
- fix shellcheck warning 'useless cat'.

Closes #15266
2024-10-11 15:01:29 +02:00