Commit Graph

35949 Commits

Author SHA1 Message Date
Nigel Brittain
c19465ca55
http_aws_sigv4: improve sigv4 url encoding and canonicalization
Closes #17129
2025-05-21 08:07:18 +02:00
Daniel Stenberg
5763449112
RELEASE-NOTES: synced 2025-05-21 07:56:42 +02:00
Daniel Stenberg
6422005a3b
DEPRECATE: restore the Windows CE text
Accidentally wiped out with 63e513b106
2025-05-20 23:11:23 +02:00
Daniel Stenberg
155af7ac77
ftplistparser: add two overflow preventions
These two conditions probably cannot actually happen, but these two
checks make that certain and should please the static code analyzers.

Pointed out by Coverity

Closes #17397
2025-05-20 16:08:29 +02:00
Stefan Eissing
be45e014c6
dns_entry: move from conn to data->state
The `struct Curl_dns_entry *` used to established a connection
do not have the connection's lifetime, but the transfer's lifetime
(of the transfer that initiates the connect).

`Curl_dns_entry *` is reference counted with the "dns cache". That
cache might be owned by the multi or the transfer's share. In the
share, the reference count needs updating under lock.

Therefore, the dns entry can only be kept *and* released using the
same transfer it was initially looked up from. But a connection is
often discarded using another transfer.

So far, the problem of this has been avoided in clearing the connection's
dns entries in the "multi_don()" handling. So, connections had NULL
dns entries after the initial transfers and its connect had been handled.

Keeping the dns entries in data->state seems therefore a better choice.

Also: remove the `struct Curl_dns_entry *` from the connect filters
contexts. Use `data->state.dns` every time instead and fail correctly
when not present and needed.

Closes #17383
2025-05-20 14:49:17 +02:00
Daniel Stenberg
3ec6aa5c07
formdata: fix memory leak in OOM situation
Fixes #17390
Follow-up to c26da713e7
Reported-by: Viktor Szakats
Closes #17393
2025-05-20 10:44:53 +02:00
Viktor Szakats
1e4a59f8fd
DEPRECATE.md: fix typo
Bug: https://github.com/curl/curl/actions/runs/15112183952/job/42473983311

Closes #17391
2025-05-20 00:40:31 +02:00
renovate[bot]
1952467ea1
GHA/non-native: update cross-platform-actions/action digest to 97419d1
Closes #17389
2025-05-19 22:29:29 +02:00
Viktor Szakats
e785e898a6
checksrc: avoid extra runs in CI, enable more check locally, fix fallouts
To avoid redundant work in CI and to avoid a single checksrc issue make
all autotools jobs fail. After this patch checksrc issues make fail
the checksrc job, the `dist / verify-out-of-tree-autotools-debug`,
`dist / maketgz-and-verify-in-tree`  jobs and the fuzzer job (if run).
Of these, the `dist` jobs replicate local builds, also testing the build
logic.

Also add a script to check the complete local repository, optionally
with the build tree to verify generated C files.

Also:
- automatically run checksrc in subdirectories having a `checksrc`
  target. (examples, OS400, tests http/client, unit and tunit)
- tests/libtest: make sure to run `checksrc` on generated `lib1521.c`.
  (requires in-tree autotools build.)
- tests: run `checksrc` on targets also for non-`DEBUGBUILD`
  builds. It ensures to check `lib1521.c` in CI via job
  `dist / maketgz-and-verify-in-tree`.
- src: drop redundant `$(builddir)` in autotools builds.
- scripts: add `checksrc-all.sh` script to check all C sources and
  the build directory as an option.
- use the above from CI, also make it verify all generated sources.
- silence `checksrc` issues in generated C sources.
- checksrc: add `-v` option to enable verbose mode.
- checksrc: make verbose mode show checked filename and fix to only
  return error on failure.
- make sure that generated C files pass `checksrc`.

Assisted-by: Daniel Stenberg

Closes #17376
2025-05-19 18:59:50 +02:00
Viktor Szakats
414ec13840
GHA/non-native: bump to FreeBSD 14.2, OpenBSD 7.7
Follow-up to f56309fdfb #17387

Closes #17388
2025-05-19 16:49:48 +02:00
renovate[bot]
f56309fdfb
GHA/non-native: update cross-platform-actions/action action to v0.28.0
Closes #17387
2025-05-19 16:24:23 +02:00
Viktor Szakats
42331cb48a
pytest: enable in 5 GHA jobs, fix fallouts
They take about 4.5 minutes of CI time in GHA/macos.

Also:
- autotools: improve `caddy`, `vsftpd` detection.
  Bringing it closer to cmake.
- autotools: fix `--with-test-caddy=no`, `--with-test-vsftps=no`,
  `--with-test-nghttpx=no` options.
- cmake: sync `nghttpx` default with autotools.
- pytest: disable failing mbedTLS tests on macOS CI.
- pytest: disable failing earlydata tests on macOS CI.
- GHA/macos: keep vsftpd pytests disabled due to lengthy run times.
- pytest: fix test_05_04 for LibreSSL. Ref: #17367
  Authored-by: Stefan Eissing

Remaining issues:
- some unidentified tests taking a long time with mbedTLS:
  `================= 462 passed, 278 skipped in 347.93s (0:05:47) =================`
  Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374999041#step:17:1536
  Workaround: not enabling pytest for mbedTLS jobs
- 17 FTP tests taking a long time (affecting all TLS backends):
  without vsftpd:
  `====================== 496 passed, 244 skipped in 56.15s =======================`
  Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374998230#step:17:1536
  with vsftpd:
  `================= 513 passed, 227 skipped in 409.37s (0:06:49) =================`
  Ref: https://github.com/curl/curl/actions/runs/15073678568/job/42376039672?pr=17362#step:17:1537
  Workaround: force-disable vsftpd.
- 100 tests failing with SecureTransport. Let's ignore that due to imminent deprecation.
  Ref: https://github.com/curl/curl/actions/runs/15055652333/job/42320873732#step:17:15362

Follow-up to 30ef79ed93 #17295
Follow-up to 9147903366 #16518

Closes #17362
2025-05-19 15:38:05 +02:00
Viktor Szakats
60f8f6d5cb
GHA/linux: restore native arm job
Previously deleted due to flakiness in stunnel package install step.

Follow-up to 0005f91259 #16303
Closes #17375
2025-05-19 15:38:05 +02:00
Daniel Stenberg
c0f724e887
urldata: move a struct field in ssl_primary_config
... which makes the struct eight bytes smaller with no gaps.

Closes #17386
2025-05-19 15:14:23 +02:00
Daniel Stenberg
63e513b106
DEPRECATE.md: drop support for VS2008
Closes #17380
2025-05-19 13:48:41 +02:00
Daniel Stenberg
e83e8c30a8
ftplistparser: split up into more functions
And some general cleanups

Closes #17384
2025-05-19 13:46:12 +02:00
Daniel Stenberg
8491e6574c
DEPRECATE.md: drop Windows CE support
Closes #17379
2025-05-19 13:32:26 +02:00
Stefan Eissing
a85f1df480
pytest tls: extend coverage
Add possibility to reload QUIC test server with another certificate. Add
tests for more coverage of handshakes.

Closes #17382
2025-05-19 13:30:02 +02:00
Daniel Stenberg
dd22442e3b
tool_operate: make retrycheck() a separate function
Simplifies post_per_transfer()

Closes #17381
2025-05-19 13:27:54 +02:00
Daniel Stenberg
c26da713e7
formdata: cleanups
- use memchr() instead of for() loop
- add and use free_formlist() instead of duplicate code
- shorten some variable names
- reduce flag struct field from 'long' to 'unsigned char'
- pass in struct pointer, not individual fields, to addhttppost()

Closes #17370
2025-05-19 13:26:02 +02:00
Daniel Stenberg
30e40c1a11
RELEASE-NOTES: synced 2025-05-19 10:10:40 +02:00
Daniel Stenberg
99f5c5c794
tool_operate: when retrying, only truncate regular files
If /dev/null or another character device etc is used for output, trying
to truncate that only causes errors.

Add test 1497 to verify

Fixes #17371
Reported-by: Brendan Dolan-Gavitt
Closes #17374
2025-05-19 09:42:54 +02:00
Christian Schmitz
6414cc6ae1
setopt: split setopt_long
Split off the biggest options RTSP_REQUEST, SSLVERSION and HTTP_VERSION
into sub functions to reduce complexity.

Closes #17372
2025-05-19 09:09:23 +02:00
Daniel Stenberg
0013bbdfb8
libtests: define CURL_DISABLE_DEPRECATION first
Instead of using the kludgey ignore macros inline. They make setting
break-points in a debugger and single-stepping code impossible.

Closes #17373
2025-05-16 23:10:43 +02:00
Christian Schmitz
f5a4d3c916
formdata: split up FormAdd
To reduce complexity of FormAdd, split the checking part into a separate
function.

Closes #17366
2025-05-16 16:14:29 +02:00
Daniel Stenberg
74cfba6a67
libssh2: remove the 'actualcode' struct field
Return and use CURLcode to a larger extent to avoid the complicated
double return code setup previously used.

Closes #17369
2025-05-16 14:56:59 +02:00
Jacob Mealey
e0f05bcf69
urlapi: remove unneeded guards around PUNY2IDN
Add more IDN/punycode tests to 1560

Closes #17364
2025-05-16 14:44:30 +02:00
Stefan Eissing
73b8912580
asnyc-thrdd: explain how this is okay with a comment
Coverity assess correctly that a variable write under mutex lock could
overwrite values from another thread - if the function were ever called
from multiple thread for the same transfer - which it is not.

Closes #17365
2025-05-16 12:55:17 +02:00
Daniel Stenberg
96726af95a
libssh: add NULL check for Curl_meta_get()
It really cannot return NULL in a working condition, but ...

Pointed out by Coverity.

Closes #17359
2025-05-15 23:04:37 +02:00
Daniel Stenberg
f7ff1d1e5e
libssh: remove a condition that always equals false
Found by Coverity

Closes #17358
2025-05-15 23:03:35 +02:00
Daniel Stenberg
f51c6d2dc2
test1173.pl: whitelist some option-looking names that aren't options
Referring to the names in the CURLOPT_DEBUGFUNCTION by name is still
fine. They start with CURLINFO but aren't really options for *getinfo().

Closes #17361
2025-05-15 22:43:54 +02:00
Stefan Eissing
30ef79ed93
pytest-xdist: pytest in parallel
Require now pytest-xdist from tests/http/requirements.txt and
run pytest in 'auto' parallel mode (counts cpu cores).

For CI runs, set the worker count to 4, overriding the
core count of 2 exposed in the images.

- use Filelock to generate allocated ports at start for all
  workers and have subsequent workers just read the file and
  take the ports for their slot
- make httpd config clearing a function fixture so every test
  starts with a clean httpd config
- have fixture `configures_httpd` as parameter of test cases
  that configure httpd anyway, saving one reload
- add pytest-xdist and filelock to required pyhton modules
- add installs to ruff CI
- give live checks waiting for a server to start up longer time
- add fixtures to tests that rely on a server
- do not stop servers unnecessarily. failures may not start them
  properly again, leading to unexpected fails in whatever follows
- add a https: port to httpd that is *not* back by QUIC to allow
  failover tests without stopping the QUIC server

Closes #17295
2025-05-15 21:44:31 +02:00
Daniel Stenberg
f0bf43e209
libssh2: split up ssh_statemachine
Into many sub functions.

Complexity taken down from 150 to 37.

Closes #17356
2025-05-15 19:50:15 +02:00
Daniel Stenberg
b4310c0fb9
docs/libcurl: mention sensitive data/headers
In the CURLOPT_DEBUGFUNCTION and CURLOPT_VERBOSE documentation.

Mentioned-by: Gordon Parke
Fixes #17353
Closes #17355
2025-05-15 19:42:32 +02:00
Viktor Szakats
8076824870
GHA/curl-for-win: build one job with classic zlib
curl-for-win switched to zlib-ng by default. Switch curl's explicit
zlib-ng job to classic zlib to keep testing that build path.

Ref: https://github.com/curl/curl-for-win/pull/79
Ref: 5aed6363cd

Closes #17357
2025-05-15 18:58:54 +02:00
Daniel Stenberg
f97d372703
tool_operate: move config2setopts to separate file, split into subs
To decrease size and complexity. Complexity taken down from 190 to 80.

Bonus:

- remove leftover HTTP/0.9 warning never triggered since hyper was dropped
- remove the ftp-skip-ip option unless FTP is used
- only set HTTP options if HTTP(S) is used
- remove use of the pointless SETOPT_CHECK macro

Side-effect:

- The order of the options in --libcurl is modified

Closes #17352
2025-05-15 14:32:45 +02:00
Daniel Stenberg
9c8cbbda5b
RELEASE-NOTES: synced 2025-05-15 14:16:15 +02:00
Daniel Stenberg
a9aafbea95
CONTRIBUTE: add project guidelines for AI use
Feedback-by: Daniel Fosco
Feedback-by: Jimmy Sjölund
Feedback-by: Christoph Jabs
Feedback-by: Manuel Strehl
Feedback-by: Dan Fandrich
Feedback-by: Sarah Gooding

Closes #17325
2025-05-15 14:07:39 +02:00
Stefan Eissing
d59d8530c6
ngtcp2: clarify ignoring of result
In shutdown, the result of a bufq_write() is intentionally ignored, but
it was not obvious why. Add a (void) cast to declare intent and a
comment explaining why.

Closes #17354
2025-05-15 11:10:02 +02:00
Stefan Eissing
0c749d8177
pytest: give parameterised tests better ids for read- and parsability
Closes #17340
2025-05-15 11:08:24 +02:00
Daniel Stenberg
f5b7eb27f0
netrc: deal with null token better
If there is no length of the token, assign a blank string to avoid
risking it being NULL.

Pointed out by Coverity

Closes #17351
2025-05-15 00:48:10 +02:00
Daniel Stenberg
473b3e51d6
tests/FILEFORMAT.md: document the aws feature
Closes #17350
2025-05-14 23:59:25 +02:00
Maksim Ściepanienka
235b575d3a
tests: add aws feature to the related tests
Closes #17347
2025-05-14 23:41:38 +02:00
Daniel Stenberg
b97d5f931a
GHA: add libssh to the openssl O3 job
To make sure we run libssh tests with valgrind enabled

Closes #17345
2025-05-14 23:36:54 +02:00
Daniel Stenberg
f0f6c610a9
_SEEALSO.md: remove spaces around command and man page section
Follows traditions better

Closes #17348
2025-05-14 23:34:28 +02:00
Daniel Stenberg
6f6ee601b9
libssh: split up the state machine function
This reduces the "complexity score" for myssh_statemach_act from 160 to
100, taking it down from the most complex function in libcurl to the
5th.

Also fixes a memory leak of the sftp session.

Closes #17346
2025-05-14 23:15:58 +02:00
Dan Fandrich
028c12948c scripts: move the ruff linter command into a script
This makes it easier for users to run it with the correct options
locally. Also, update the ruff, pytype and cmakelint versions to the
latest current ones.

Closes #17343
2025-05-13 16:41:32 -07:00
Viktor Szakats
828f998493
spacecheck.pl: drop more exceptions
- replace ß (scharfes S) with links.
- replace § (section sign) with links.
- replace 🙏 emoji with `🙏`.
 Supported by GitHub, Forgejo/Gitea and most likely GitLab.
- docs/libcurl/curl_mprintf.md: replace Unicode ± with `{+|-}`.
- docs/CIPHERS.md: URL encode Unicode in URLs.
- lib1560: use hex encoding in `räksmörgås.se`.
- unit1307: use hex encoding in `Lindmätarv`.
- drop LATIN SMALL LETTER A WITH ACUTE exception.
  No longer appears in tests.

This leaves the single character exception: `ö`
And file exceptions holding contributor names.

Follow-up to 9243ed59b3 #17329
Follow-up to 838dc53bb7 #17247

Closes #17335
2025-05-13 16:01:07 +02:00
Daniel Stenberg
4efeeaa22d
tests/server/Makefile.inc: cleanup
- drop curl_ctype.h as it is not used by test servers
- list all curlx files in the CURLX_* lists
- sort the lists alphabetically

Closes #17339
2025-05-13 14:19:01 +02:00
Daniel Stenberg
0cce66b3b3
setopt: provide info for CURLE_BAD_FUNCTION_ARGUMENT
If CURLE_BAD_FUNCTION_ARGUMENT is returned and failf() has not provided
any details, this adds a generic error string that includes the option
number.

This helps debugging for example the curl tool which does a lot of
setopt calls and in reading post fact logs it is not always easy to tell
exactly which call that failed.

Closes #17337
2025-05-13 14:18:19 +02:00