Commit Graph

33723 Commits

Author SHA1 Message Date
Daniel Stenberg
fbf5d507ce
lib/src: white space edits to comply better with code style
... as checksrc now finds and complains about these.

Closes #14921
2024-09-19 14:59:12 +02:00
Daniel Stenberg
a57b45c386
TODO: IMAP upload unread
Proposed-by: Nicolas George
Ref: https://curl.se/mail/archive-2024-09/0003.html
Closes #14964
2024-09-19 14:55:51 +02:00
Viktor Szakats
c4f781e69d
cmake: drop redundant assigments
No need to set them `ON` again.

Closes #14924
2024-09-19 12:56:22 +02:00
Viktor Szakats
abf737b3ce
cmake: drop redundant zlib var, rename function (internals)
- drop redundant internal variable `USE_ZLIB`, rely on `HAVE_LIBZ`.

- rename `optional_dependency()` -> `curl_dependency_option()`
  Make `grep 'option('` hit this option. Namespaced.
  It has a single use with `ZLIB`.

Closes #14918
2024-09-19 12:56:22 +02:00
Viktor Szakats
56b0442e4c
urlapi: drop unused header
Closes #14867
2024-09-19 12:56:21 +02:00
Viktor Szakats
c997f3e009
processhelp.pm: improve taskkill calls (Windows)
- drop `tasklist` call before `taskkill`.
  `taskkill` offers two ways to kill a `pid`:
  1. `-pid <pid>`
     If `<pid>` is missing it returns 128 and outputs:
     ```
     ERROR: The process "<pid>" not found.
     ```
  2. `-fi "PID eq <pid>"`
     If `<pid>` is missing, it returns 0 and outputs:
     ```
     INFO: No tasks running with the specified criteria.
     ```
  The curl runner script doesn't check the result of the call and both
  stdout and stderr are redirected to NUL.
  Meaning the `tasklist` calls pre-verifying if the PID exists are not
  necessary and we can drop them to put less strain on the runner
  environment.

- log a `taskkill` call missed earlier.
  Follow-up to e53523fef0 #14859

- streamline `taskkill` calls by using the `-pid` option
  (was `-fi <filter-expression>`).

- make `taskkill` in `pidterm()` use `-t` to kill the process tree.

Ref: #11009
Closes #14959
2024-09-19 12:43:54 +02:00
Viktor Szakats
888662b74a
tests: delete duplicate macro check
Follow-up to e9a7d4a1c8 #12376

Closes #14963
2024-09-19 12:43:54 +02:00
Daniel Stenberg
8445285730
CURLMOPT_PIPELINING.md: clarify that CURLPIPE_NOTHING is not default
Fixes #14961
Reported-by: Pavel Kropachev
Closes #14962
2024-09-18 23:56:41 +02:00
Stefan Eissing
8ad3597d2d
tests: testrunner reliability improvements
- perform torture tests with '-j2' for shorter runtime
- when waiting on test results overly long, log the tests
  waited for and eventually log the test log directories
  for easier analysis what is wrong in CI jobs.
- sockfilt.c: treat the windows errno 109 (ERROR_BROKEN_PIPE)
  as a socket closed by the client and do not exit.
- when verifying https server, do not in addition check
  the http server behind it also
- when tearing down the stunnel of a non-responsive https
  server, tear down the http server with it

Closes #14960
2024-09-18 23:09:16 +02:00
Gabriel Marin
5a263710f6
lib, src, tests: added space around ternary expressions
Closes #14912
2024-09-18 15:27:26 +02:00
Daniel Stenberg
0236276c3d
RELEASE-NOTES: synced
And bump to 8.10.2 for now
2024-09-18 15:21:04 +02:00
Jon Rumsey
1ec5336b66
negotiate: conditional check around GSS & SSL specific code
Fixes #14938
Reported-by: lomberd2 on github
Fixes #14952
Closes #14954
2024-09-18 15:04:06 +02:00
Daniel Stenberg
c0a9db8424
curl_url_set.md: document HOST handling when URL is parsed
When a full URL is set (parsed), the hostname component is stored URL
decoded (with default zero flags).

While perhaps surprising and inconsistent, the API has done this for
quite some time already and changigtn this now would break existing
behaviour.

Fixes #14942
Reported-by: Venkat Krishna R
Closes #14946
2024-09-18 15:02:45 +02:00
Daniel Stenberg
6d0a48e586
sendf: add condition to max-filesize check
Since the max filesize check should not be performed while the body is
ignored.

Follow-up to aef384a7df
Closes #14958
2024-09-18 15:00:29 +02:00
Daniel Stenberg
7eb8c04847
RELEASE: synced
8.10.1 release
2024-09-18 07:55:38 +02:00
Daniel Stenberg
dabeb542fc
THANKS: contributors from the 8.10.1 release 2024-09-18 07:53:49 +02:00
Viktor Szakats
210cf7cd9d
GHA/windows: revert enabling SSPI option
Because it reduces test coverage significantly:
```
TESTINFO: "curl has SSPI support" 109 times (64, 65, 67, 68, 69, 70, 72, 81, 88 and 100 more)
```

Follow-up to e53523fef0 #14859
Closes #14947
2024-09-17 18:37:45 +02:00
Daniel Stenberg
0cfc7fcca1
tool_cb_wrt: use "curl_response" if no file name in URL
Use the same fallback for content-disposition cases as for regular -O

Add test692: verify -JO with URL without a file name

Reported-by: Brian Inglis
Fixes #14939
Closes #14940
2024-09-17 13:39:51 +02:00
dependabot[bot]
89de543204
GHA/non-native: bump vmactions/omnios-vm from 1.0.2 to 1.0.6
Bumps [vmactions/omnios-vm](https://github.com/vmactions/omnios-vm) from 1.0.2 to 1.0.6.
- [Release notes](https://github.com/vmactions/omnios-vm/releases)
- [Commits](a61ca1ebaf...2f97978b47)
2024-09-17 02:58:28 +02:00
Viktor Szakats
41290d4371
GHA/windows: fix bad typo in MSVC GnuTLS stunnel condition
Regression from e53523fef0 #14859
2024-09-16 23:49:42 +02:00
Viktor Szakats
8a7efdb876
GHA: misc updates: impacket, timeouts, mingw-w64 32-bit
- NetBSD: enable impacket. Debatable if this hefty package is worth it
  for the single SMB test (1451) run curl uses it for.
  (on the upside, it seems to install instantly on NetBSD)
  It's also a versioned package name that requires active maintenance.
  The unversioned name doesn't work, though that's what the package
  search returns: https://pkgsrc.se/net/py-impacket

- FreeBSD: stop installing impacket deps explicitly.
  They are automatically installed via the impacket package.

- Linux: put time limit on test runs.
  Started hanging yesterday:
  linux: https://github.com/curl/curl/actions/runs/10875011573/job/30172946986#step:41:3731
  awslc: https://github.com/curl/curl/actions/runs/10879782864/job/30185188509
  wolfssl: https://github.com/curl/curl/actions/runs/10879782864/job/30185188509

- distcheck: reduce time limit to 15m (was 30m)
  Linux tests started to hang, so reduce the timeout from 30 minutes
  to 15. These jobs normally take 3 to 6 minutes to finish.
  https://github.com/curl/curl/actions/runs/10879479914

- Windows: add 32-bit MSYS2/mingw-w64 job.
  Follow-up to e53523fef0 #14859

Closes #14929
2024-09-16 15:43:21 +02:00
Viktor Szakats
e53523fef0
CI: move Azure jobs to GHA, fix fallouts, sshserver, runtests tweaks
CI:

- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
  Skip `impacket` on MSYS2 due to install failure.
  Skip OpenSSH server for old/standalone mingw-w64 (building curl
  without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
  Skip `impacket`: it's compiling for 7 minutes then breaks.
  Skip `stunnel` due to sluggish test run performance.
  (This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
  - universally ignore SCP tests, they fail everywhere.
  - ignore test 612.
  - ignore test 613 616 618 with MSYS2 mingw-w64.
  - ignore test 614 with libssh.
  - ignore all SFTP with MSYS2 native.
  - ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
  - ignore a couple of SFTP tests with MSYS2 mingw-w64.
  (This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
  old mingw-w64 7.3.0 (but not for 9.5.0!):
  ```
  2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
  [...]
  curl: (7) Could not connect to server
  [...]
  FAIL 1451: 'Basic SMB request' SMB
  ```
  Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.

sshserver:

- sshserver: fix permissions for SSH host key on Windows, allowing sshd
  to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
  to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
  domainless username again.
  Follow-up to 3ee7c676ec #5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
  ```
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
  ```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
  Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
  full paths.
- sshserver: add workaround to make `logmsg` messages visible.
  Before this patch they only went to a file and never shown.

runtests:

- runtests: log details when these Windows commands are called:
  `handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
  `handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
  `servers.pm`, syncing it with the other `taskkill` call.
  Follow-up to bc72a78a11 #14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
  The call was made on all Windows versions. It's possibly overkill to
  do this, because XP Home is probably rarely used for running curl
  tests these days. In case it's needed, it'd be better to put it under
  an explicit option.
- runtests: show Perl version and path.

Fix/silence fallouts:

- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.

Closes #14859
2024-09-16 10:00:17 +02:00
Viktor Szakats
0af017d66d
GHA/non-native: install Perl for FreeBSD cmake jobs
It was implicitly installed for autotools jobs. Install it explicitly
for cmake ones.

Closes #14928
2024-09-16 09:44:16 +02:00
Viktor Szakats
2b6bdb41ce
cmake: fix MSH3 to appear on the feature list
Closes #14927
2024-09-16 09:37:19 +02:00
Viktor Szakats
71a83b8789
GHA/non-native: enable SFTP/SCP tests on FreeBSD
It's working now. Possibly fixed in the runner env?

Closes #14920
2024-09-16 01:02:55 +02:00
Viktor Szakats
0ba70dd13c
singleuse: drop Curl_memrchr() for no-HTTP builds
Make single-use function check CI test pass by dropping the global
`Curl_memrchr()` function from the build when it has no caller.

Fixes:
```
Curl_memrchr in curl_memrchr
```
https://github.com/curl/curl/actions/runs/10816280747/job/30007145114

Seen while migrating affected job from Azure to GHA.
Ref: #14859

Closes #14919
2024-09-15 16:33:24 +02:00
Viktor Szakats
5f93cad086
GHA: replace make with ninja in Cygwin, MSYS2 and mingw-w64 cmake jobs
Speed-ups:
- cygwin build: 47s -> 29s
- cygwin build tests: 3m12 -> 2m32
- MSYS2 native build: 29s -> 16s
- MSYS2 native build tests: 2m24 -> 1m52
- MSYS2/mingw-w64: already had ninja installed and used implicitly.
- linux-mingw-w64-cross: 31s -> 25s

before:
cygwin: https://github.com/curl/curl/actions/runs/10866939355/job/30155134299
windows: https://github.com/curl/curl/actions/runs/10866939361

after:
cygwin: https://github.com/curl/curl/actions/runs/10867861215/job/30157121784
windows: https://github.com/curl/curl/actions/runs/10867861208?pr=14914

Closes #14914
2024-09-15 13:57:21 +02:00
Viktor Szakats
2e76e1f7fc
GHA/non-native: replace make with ninja in cmake jobs
Comparing job times for non-native runners is unreliable, but
a speed-up is expected nonetheless.

Full job times:
- NetBSD amd64: 14m1 -> 13m15
- OpenBSD amd64: 6m14 -> 5m33
- FreeBSD arm64: 6m10 -> 5m32

Example runs:
before: https://github.com/curl/curl/actions/runs/10866939368
after: https://github.com/curl/curl/actions/runs/10870425491?pr=14917

Closes #14917
2024-09-15 13:57:21 +02:00
Viktor Szakats
8f5d73af12
GHA: add valgrind to the job titles using it, and tidy-ups
There is a 4-5x difference in test run times. Make the reason more
obvious by adding valgrind to the job names.

Also:
- tidy up job names.

Closes #14853
2024-09-15 13:57:21 +02:00
Viktor Szakats
c3a7145a4f
GHA/macos: tidy-up
- drop redundant `brew autoconf`.
  It's automatically installed via `automake`.
- whitespace
2024-09-15 13:57:20 +02:00
Tal Regev
65168b8eb9
GHA/windows: use libuv for event-based tests on openssl job
Use libuv for event-based tests on openssl job.

Closes #14868
2024-09-15 13:57:20 +02:00
Tal Regev
09b21e4755
GHA/windows: re-add GnuTLS after upstream fix
Re-add GnuTLS. The fix was in this PR:
https://github.com/microsoft/vcpkg/pull/40739

Follow-up to a5682d9cb9 #14710
Closes #14864
2024-09-15 13:57:20 +02:00
Viktor Szakats
8709404fe9
GHA/macos: make impacket found by tests
Also move impacket installation right before the test run to avoid
spending 10s installing if the build fails.

Closes #14913
2024-09-15 03:12:48 +02:00
Viktor Szakats
c89cc09ab0
GHA/macos: replace make with ninja for cmake builds
Ninja performs better than GNU Make. Also the generation step is a bit
faster. The cost is installing ninja where missing from the image, and
there is no pre-existing package install step. Otherwise the
installation is fast because the tool is tiny.

(It would probably make sense to include ninja on the runner image; it's
a near zero-cost optimization opportunity for all CI jobs using cmake.)

Speed-ups:

Build step: 3s -> 1s.
Build tests step: 37s -> 22s.

before: https://github.com/curl/curl/actions/runs/10865247916/job/30151507582?pr=14911
after: https://github.com/curl/curl/actions/runs/10865301046/job/30151610055?pr=14910

Closes #14910
2024-09-15 03:09:04 +02:00
Viktor Szakats
51fc89965b
GHA/macos: tidy-ups, install impacket for cmake jobs
- install impacket for cmake jobs.

- stop installing autotools for cmake jobs.
  (also saving the extra seconds necessary for impacket install.)

- delete no longer necessary openssl pkg-config patch.

- `brew unlink openssl` unconditionally. This is safe, we no longer need
  it at the default path, and the condition were never met anyway for
  a while now.

- stop installing openssl, it's there on the base image.
  (it was also missed for cmake, but that's okay)

before: https://github.com/curl/curl/actions/runs/10864438947/job/30149718802
after: https://github.com/curl/curl/actions/runs/10865247916/job/30151507582

Closes #14911
2024-09-15 02:34:17 +02:00
Daniel Stenberg
aef384a7df
http: make max-filesize check not count ignored bodies
Add test 477 to verify

Reported-by: MasterInQuestion on github
Fixes #14899
Closes #14900
2024-09-14 22:51:57 +02:00
Daniel Stenberg
7eda757d99
FTP: partly revert eeb7c12807
Since ASCII transfers on FTP means sending CRLF line endings, we should
still keep converting them to LF-only on platforms where text files
typically do not use CRLF.

This also DOES NOT convert existing CRLF line endings on ASCII uploads
but only does stand-alone LF => CRLF.

Regression from eeb7c12807 shipped in 8.10.0

Reported-by: finkjsc on github
Fixes #14873
Closes #14875
2024-09-14 17:23:04 +02:00
Daniel Stenberg
2b652b8634
transfer: remove redundant variable use: select_bits
Follow-up to a33bcc9b53

Pointed out by CodeSonar

Closes #14907
2024-09-14 17:20:27 +02:00
Daniel Stenberg
3e36334af6
RELEASE-NOTES: synced 2024-09-13 22:50:40 +02:00
Stefan Eissing
50166c0de8
connect: store connection info when really done
Output the 'Connected to...' info message when the connection has been
fully established and all information is available.

Due to our happy eyeballing, we should not emit info messages in
filters, because they may be part of an eyeballing attempt and may be
discarded later for another chain.

Closes #14897
2024-09-13 22:27:11 +02:00
Stefan Eissing
a33bcc9b53
transfer: fix sendrecv() without interim poll
There was a "clever" optimization that skipped sendrecv() handling when
the transfer's pollset was empty. This happens for paused transfers, for
example.

Unforunately, if the libcurl application never calls curl_multi_poll(),
the pollset is and will aways remain empty, prevent the transfer from
progressing.

Remove this "optimization" and always try send/receive where applicable.

Fixes #14898
Reported-by: Victor Kislov
Closes #14901
2024-09-13 22:21:33 +02:00
Daniel McCarney
8972845123
vtls/rustls: support strong CSRNG data
Now that the curl rustls vtls backend is using rustls 0.14 we can
address the weak random situation by using
`rustls_default_crypto_provider_random()` to provide a `Curl_ssl`
`random` callback that fills the provided buffer with cryptographically
secure random data.

The mentions in `docs/` about weak RNG when using rustls are removed as
they are no longer applicable.

Closes #14889
2024-09-13 14:11:56 +02:00
Daniel McCarney
6d9b40d6a4
vtls/rustls: simplify ciphersuite skipping
Now that the rustls vtls backend is using rustls 0.14 we can take
advantage of `rustls_supported_ciphersuite_protocol_version()` to skip
TLS 1.3 and TLS 1.2 ciphersuites as required without needing to
interrogate the ciphersuite names as `rustls_str`s.

Closes #14889
2024-09-13 14:11:52 +02:00
Daniel McCarney
f09adc3ad1
vtls/rustls: rustls-ffi 0.14.0 update
* Documentation is updated to describe new required version, and to link
  to the upstream README about cryptography providers.
* GitHub workflow is updated to fetch 0.14.0.
* Breaking changes in`lib/vtls/rustls.c` are addressed:
  * The `rustls_client_config_builder_build()` function now uses an out
    parameter for the built config instead of returning it directly.
    This allows the building process to fail if the default crypto
    provider state isn't appropriate, or another error condition occurs.
  * Default ciphersuites are collected using renamed functions named to
    make it clear the ciphersuites are associated with the default
    crypto provider.
  * Customization of ciphersuites is now done via
    a `rustls_crypto_provider_builder` used to instantiate a
    `rustls_crypto_provider`. The customized provider can then can be
    used with `rustls_client_config_builder_new_custom` in place of
    providing ciphersuites directly.
  * `rustls_connection_get_negotiated_ciphersuite()` now returns the
    ciphersuite ID directly.

Closes #14889
2024-09-13 14:11:49 +02:00
Daniel McCarney
65b8d89463
vtls/rustls: differentiate error messages
It's easier to diagnose a problem when there is one place where the
error message can be emitted. For that reason this commit updates two
errors that were shared between other fallible operations to use unique
messages.

Closes #14889
2024-09-13 14:11:42 +02:00
Daniel McCarney
d38458d823
vtls/rustls: simplify builder cleanup
Don't build `config_builder` just to free the resulting config, free the
builder directly.

When `cr_init_backend` encounters an error condition setting up the
Rustls client configuration it must do something with the
`config_builder` that was constructed earlier to avoid a memory leak.

The previous implementation preferred to use a pattern of building the
builder (thus consuming it) and then freeing the built config (to avoid
a memory leak). However, the purpose/intent is clearer when we just free
the builder directly instead of building it and freeing the result.

Closes #14889
2024-09-13 14:11:26 +02:00
Stefan Eissing
bef0acaf21
request: correctly reset the eos_sent flag
Add test cases

Bug: https://marc.info/?l=git&m=172620452502747&w=2
Reported-by: Patrick Steinhardt
Closes #14895
2024-09-13 13:28:10 +02:00
Stefan Eissing
e70c22b622
tests: tweak lock file handling and timers
- when server are killed by a test case, do not wait for the server lock
  file to go away. These tests are mostly about client timeouts and the
  server will hang until killed.
- when killing a server successfully, check for a remaining lock file,
  log its existence and remove it.
- lower the delay timings on SLOWDOWN by half
- add SLOWDOWNDATA server command to only slow down the FTP data bytes,
  not the control ones.
- lower some timeout values

Closes #14835
2024-09-13 09:46:25 +02:00
Daniel Stenberg
8ca6030839
RELEASE-NOTES: synced 2024-09-12 20:31:41 +02:00
Viktor Szakats
28fa417bf0
autotools: fix --with-ca-embed build rule
Add missing rule dependency on the user-specified CA bundle. This fixes
including it when using the curl distro tarball, and other cases.

Also:
- fix the internal name of the CA bundle to avoid nested quotes.
  It broke broke the rule dependency for the make tool.
- exclude the generated (empty) `tool_ca_embed.c` file from the distro
  tarball.
  Patch-by: Daniel Stenberg

Follow-up to 8a3740bc8e #14059
Reported-by: rampageX on github
Fixes #14879
Closes #14882
2024-09-12 19:58:14 +02:00