Commit Graph

34209 Commits

Author SHA1 Message Date
Viktor Szakats
1dce2a1746
openssl: drop unused HAVE_SSL_GET_SHUTDOWN macro
Follow-up to c31041b17e #13904
Closes #16103
2025-01-28 11:15:09 +01:00
Daniel Stenberg
58c22dda66
tool_getparam: fail --hostpubsha256 if libssh2 is not used
Reported-by: Harry Sintonen
Closes #16109
2025-01-28 10:30:40 +01:00
Andrew Kaster
1b740aedcd
ws: Reject frames with unknown reserved bits set
RFC 6455 Section 5.2 notes that for bits RSV1, RSV2, and RSV3 of the
framing header, a non-zero value that is not defined by a negotiated
extension MUST Fail the WebSocket connection.

Test 2310 verifies

Closes #16069
2025-01-28 10:19:39 +01:00
Jay Satiro
e2ec7bf1b4 vtls: fix default SSL backend as a fallback
- Use build-time CURL_DEFAULT_SSL_BACKEND as a fallback when environment
  variable CURL_SSL_BACKEND contains a backend that is unavailable.

Prior to this change if CURL_SSL_BACKEND was set then
CURL_DEFAULT_SSL_BACKEND was ignored even if the backend of the former
was unavailable. In that case libcurl would instead select the first
available backend in the list of backends.

Bug: https://github.com/curl/curl/issues/16076#issuecomment-2617354254
Reported-by: Jeroen Ooms

Closes https://github.com/curl/curl/pull/16108
2025-01-28 03:29:35 -05:00
Jay Satiro
4f99efb192 easy: allow connect-only handle reuse with easy_perform
- Detach and disconnect an attached connection before performing.

Prior to this change it was not possible to safely reuse an easy handle
with an attached connection in a second call to curl_easy_perform. The
only known case of this is a connect-only type handle where the
connection was detached when curl_easy_perform returned, only to be
reattached by either curl_easy_send/recv.

This commit effectively reverts 2f8ecd5d and be82a360, the latter of
which treated the reuse as an error. Prior to that change undefined
behavior may occur in such a case.

Bug: https://curl.se/mail/lib-2025-01/0044.html
Reported-by: Aleksander Mazur

Closes https://github.com/curl/curl/pull/16008
2025-01-28 03:27:04 -05:00
Dexter Gerig
f25a807a7d
asyn-ares: fix memory leak
Closes #16107
2025-01-28 09:23:48 +01:00
Dexter Gerig
afb3fd7112
asyn-thread: fix HTTPS RR resolution
Store the received data correctly.

Closes #16107
2025-01-28 09:23:33 +01:00
Viktor Szakats
fd12b56832
gitignore: add generated unity sources for lib and src
Follow-up to 71cf0d1fca #14772
2025-01-27 22:55:39 +01:00
Viktor Szakats
81e271f45c
checksrc: exclude generated bundle files to avoid race condition
Necessary to catch rare cases when `checksrc` hits these files when they
are not populated yet:
```
./curltool_unity.c:1:1: error: Missing copyright statement (COPYRIGHT)
 ^
```
https://github.com/curl/curl/actions/runs/12995546740/job/36242556713?pr=16094#step:37:123

Follow-up to 71cf0d1fca #14772
Closes #16102
2025-01-27 20:59:47 +01:00
Viktor Szakats
23b41224e3
msvc: add missing push/pop for warning pragmas
Also fix indentation/formatting around similar pragmas.

Closes #16101
2025-01-27 20:59:47 +01:00
Viktor Szakats
108b2153ac
Makefile.dist: delete
It had shorthand aliases to launch `./configure` and
`./configure --with-openssl`. The former hasn't worked for a long while
because of missing TLS.

Its `ca-bundle` and `ca-firefox` targets have been broken for 2.5 years
till recently. These targets also exist in `./configure` and have been
working all along.

Also:
- cmake: add support `curl-ca-bundle` and `curl-ca-firefox` targets.
- tests/testcurl.pl: drop obsolete build logic.

Closes #16094
2025-01-27 20:59:47 +01:00
Viktor Szakats
5c31c2e670
tidy-up: .gitignore lines mostly
- `.gitignore`: delete, dedupe and move rules upwards.
  Ref: 6389ba87b8 #13311
- `.gitignore`: fix generated test sources.
  Follow-up to 71cf0d1fca #14772
- `.gitignore`: replace exe listings with a wildcard.
- lib: move `setup-*.h` from `EXTRA_DIST` to `CURL_HFILES`.
- `makedebug.bat`: uppercase an argument to match docs.
- GHA/non-native: delete stray env.
  Follow-up to 12a6de2f66 #16043
- sort source lists.

Closes #16093
2025-01-27 20:59:46 +01:00
Daniel Stenberg
cb4cd36fe7
VULN-DISCLOSURE-POLICY: on legacy dependencies
Problems that only trigger using *legacy* dependencies are not
considered security problems.

Closes #16086
2025-01-27 15:48:13 +01:00
Daniel Stenberg
35b1c1585b
urldata: tweak the UserDefined struct
By better sticking to listing the struct members sorted by size, this
struct is now 48 bytes smaller on my fairly maximized build, without
removing anything.

Turned 'connect_only' into two bits instead of an unsigned char with two
magic values.

Also put the 'gssapi_delegation' field within ifdef HAVE_GSSAPI.

Closes #16097
2025-01-27 15:47:18 +01:00
Stefan Eissing
5b73a7b637
http2: fix data_pending check
The h2 filter mistakenly also checked `sendbuf` when asked
about pending data. The call is only meant to account for
buffered data that still needs to be received.

Also, remove obsolete recvbuf in stream as we write received
headers and data directly.

Fixes #16084
Closes #16098
Reported-by: Deniz Sökmen
2025-01-27 15:46:33 +01:00
Viktor Szakats
b01f44ec09
INSTALL.md: add CMake examples for macOS and iOS
Closes #16095
2025-01-27 12:49:12 +01:00
Daniel Stenberg
e5d7a154d6
RELEASE-NOTES: synced 2025-01-27 08:18:45 +01:00
Daniel Stenberg
2a49cad39e
version: fix the IDN feature for winidn and appleidn
Closes #16091
2025-01-27 08:13:38 +01:00
Daniel Stenberg
e27abfe2f1
version: rename c-ares-rr to asyn-rr
Works better if we later introduce another way to do the asynch RR
resolves (together with the threaded resolver) that does not use c-ares.

Closes #16090
2025-01-27 08:11:44 +01:00
Viktor Szakats
5ce5c62662
GHA: fixup CM/AM order within Apple steps, move name: first, quote them
Closes #16092
2025-01-26 15:18:59 +01:00
Viktor Szakats
7c198b59e4
projects: fix INSTALL-CMAKE.md references
Follow-up to 0f4c19b66a #12772
2025-01-26 15:09:16 +01:00
Viktor Szakats
af892b5400
windows: drop two missed buildconf.bat references
Follow-up to 0035ff45c5 #16081
2025-01-26 15:04:54 +01:00
Viktor Szakats
0035ff45c5
build: drop tool_hugehelp.c.cvs, tidy up macros, drop buildconf.bat
Rework the way `tool_hugehelp.c` is included in builds.

After this patch, with `./configure` and CMake `tool_hugehelp.c` is only
compiled when building with manuals enabled. With manuals disabled this
source file is not used anymore. The method is similar to how
8a3740bc8e implemented `tool_ca_embed.c`.
`./configure` always generates it as before, otherwise the build fails.

- winbuild: rework to not need `buildconf.bat`, but automatically use
  `tool_hugehelp.c` if present (e.g. when building from an official
  source tarball) and enable `USE_MANUAL` accordingly.

- `buildconf.bat`: after dropping `tool_hugehelp.c` generation, the only
  logic left was `cp Makefile.dist Makefile`. This allowed to launch
  winbuild builds via GNU Make in a Git repo. Drop this option together
  with the batch file.

- build `libcurltool` without `USE_MANUAL` macro to exclude the manual
  and the dependence on the generator commands. Drop relying on
  `UNITTESTS` for this purpose.
  Follow-up to 96843f4ef7 #16068

- `src/mkhelp.pl`: include `tool_hugehelp.h` before using `USE_MANUAL`
  to have it set in `config-*.h` builds with source tarballs created
  with manual but without zlib.

Closes #16081
2025-01-26 14:22:49 +01:00
Jay Satiro
1a36923d73 tests: change the behavior of swsbounce
- Change the swsbounce keyword to override the part number on a
  subsequent request to the previous part number + 1.

Note the previous part number in this case is the part number that
was returned as a response to the previous request and contained
the swsbounce keyword.

Prior to this change swsbounce incremented the part number of the
subsequent request instead of overriding it, and did so in a more
limited fashion that prevented chaining swsbounce in multiple responses.

For example, if the test makes a request that causes the sws server to
return `<data>` as a response and that response contains `swsbounce`
then for the next response the sws server returns `<data1>`. If
`<data1>` also contains `swsbounce` then for the next response the sws
server now returns `<data2>` instead of the requested part.

Fixes https://github.com/curl/curl/discussions/16074
Closes https://github.com/curl/curl/pull/16085
2025-01-26 02:29:21 -05:00
Viktor Szakats
516cb98733
docs/HTTPSRR.md: fix typo in option name
Follow-up to 7f4f192f04 #16052
2025-01-26 02:23:23 +01:00
Viktor Szakats
3e552ef833
cmake/Find: add iphlpapi for c-ares, omit syslibs if dep not found
- add `iphlpapi` library for c-ares.
  Ref: 082d98ba6b

- fix to not add system libs if the dependency was not found.
  librtmp, Rustls, wolfSSL
  Follow-up to 421e592db2 #15832
  Follow-up to 7bab201abe #15193

Closes #16089
2025-01-26 01:52:47 +01:00
Daniel Stenberg
7f4f192f04
docs/HTTPSRR.md: initial HTTPS RR documentation
Closes #16052
2025-01-25 23:51:01 +01:00
Daniel Stenberg
0d4fdbf15d
asyn-thread: use c-ares to resolve HTTPS RR
Allow building with c-ares and yet use threaded resolver for the main
host A/AAAA resolving:

  `--with-ares` provides the c-ares install path and defaults to use
  c-ares for name resolving

  `--with-threaded-resolver` still uses c-ares in the build (for HTTPS)
  but uses the threaded resolver for "normal" resolves.

It works similarly for cmake: ENABLE_ARES enables ares, and if
ENABLE_THREADED_RESOLVER also is set, c-ares is used for HTTPS RR and
the threaded resolver for "normal" resolves.

HTTPSRR and c-ares-rr are new features return by curl_version_info() and
thus shown by curl -V.

The c-ares-rr feature bit is there to make it possible to distinguish
between builds using c-ares for all name resolves and builds that use
the threaded resolves for the regular name resolves and c-ares for
HTTPSRR only. "c-ares-rr" means it does not use c-ares for "plain" name
resolves.

HTTPSRR support is EXPERIMENTAL only.

Closes #16054
2025-01-25 23:46:14 +01:00
Viktor Szakats
520e67dd39
GHA/non-native: bump to NetBSD 10.1
Closes #16088
2025-01-25 23:03:24 +01:00
renovate[bot]
604cb65fbd
GHA/non-native: update cross-platform-actions/action action to v0.27.0
Closes #16071
2025-01-25 22:50:52 +01:00
Viktor Szakats
1bc83abfb7
cmake: drop CURL_USE_PKGCONFIG from curl-config.cmake.in
This variable was meant to be used by curl Find modules, but it turns
out it makes no sense to use those from `curl-config.cmake.in`. It means
this variable was not used before and will not be used in the future,
and therefore safe to delete.

Also add missing macros passed to `curl-config.cmake` to comment.

Ref: https://github.com/curl/curl/pull/14930#discussion_r1929537797

Closes #16087
2025-01-25 15:21:17 +01:00
renovate[bot]
49ece646c0
ci: update dependency cloudflare/quiche to v0.23.2
Closes #16082
2025-01-25 12:06:01 +01:00
Viktor Szakats
d8f1aa4b11
GHA/curl-for-win: fix DOCKER_CONTENT_TRUST for a job [ci skip]
Follow-up to 28230bec1f #15141
2025-01-25 00:42:54 +01:00
Daniel Stenberg
5426cd5948
content_encoding: put the decomp buffers into the writer structs
- no more malloc/free per chunk
- removes the extra malloc entirely
- make the buffer (much) smaller (10MB => 16KB!)
- rename 'decomp' to 'buffer' to clarify purpose

Closes #16079
2025-01-24 14:04:23 +01:00
Daniel Stenberg
76f83f0db2
content_encoding: drop support for zlib before 1.2.0.4
zlib 1.2.0.4 was released on 10 August 2003

Closes #16079
2025-01-24 14:04:23 +01:00
Stefan Eissing
1213c31272
lib: redirect handling by protocol handler
Adds a `follow()` callback to protocol handlers, so they may decide how
to act on a `newurl` after a request has been done. This is optional.

This moves the HTTP code for handling redirects from multi.c to http.c
where it should be. If we ever add a protocol with its own logic, it
would install its own follow function.

Closes #16075
2025-01-24 11:00:34 +01:00
Stefan Eissing
e83818cae1
lib: clarify 'conn->httpversion'
The variable `conn->httpversion` was used for several purposes and it
was unclear at which time the value represents what.

- rename `conn->httpversion` to `conn->httpversion_seen`
  This makes clear that the variable only records the last
  HTTP version seen on the connection - if any. And that it
  no longer is an indication of what version to use.
- Change Alt-Svc handling to no longer modify `conn->httpversion`
  but set `data->state.httpwant` for influencing the HTTP version
  to use on a transfer.
- Add `data->req.httpversion_sent` to have a record of what
  HTTP version was sent in a request
- Add connection filter type CF_TYPE_HTTP
- Add filter query `CF_QUERY_HTTP_VERSION` to ask what HTTP
  filter version is in place
- Lookup filters HTTP version instead of using `conn->httpversion`

Test test_12_05 now switches to HTTP/1.1 correctly and the
expectations have been fixed.

Removed the connection fitler "is_httpN()" checks and using
the version query instead.

Closes #16073
2025-01-24 10:59:02 +01:00
Viktor Szakats
7e814c8717
build: fix compiling with GCC 4.x versions
- silence false positive picky warnings.
- avoid "possible noreturn" warnings for standalone tests and examples.
- fix to compile without `#pragma GCC diagnostic push` support.
- fix "#pragma GCC diagnostic not allowed inside functions".

Prerequisite for #15975 that needs GCC 4.4 for the latest pre-built
CeGCC/mingw32ce toolchain for Windows CE.

Cherry-picked from #15975
Closes #16062
2025-01-22 11:26:15 +01:00
Viktor Szakats
96843f4ef7
src: omit hugehelp and ca-embed from libcurltool
CMake builds using the Xcode generator broke with an error saying it
doesn't support multiple targets depending on the same custom commands.
These custom commands are generating `tool_hugehelp.c` and
`tool_c_embed.c` for the curl tool and libcurltool.

`unit1394` and `unit1604` tests use libcurltool to test tool-specific
functions. They don't need hugehelp and ca-embed. It's thus safe to
disable and exclude them when compiling the sources for libcurltool.
Use the `UNITTESTS` macro to detect a libcurltool build within C.

After this patch these sources are solely used for building the curl
tool. Making the build compatible with the CMake Xcode generator.

Apply the change to autotools too to keep build systems synchronized.

Follow-up to 12a6de2f66 #16043
Closes #16068
2025-01-22 11:21:43 +01:00
Viktor Szakats
8dfd271c35
cmake: prefer dash-style MSVC options
They play better with Unixy shells. The compiler has been supporting
dash options since its early versions.

Also fix to detect warnings options passed in dash-style.

Closes #16063
2025-01-22 11:21:42 +01:00
Stefan Eissing
cd9107e012
cfilters: kill connection filter events attach+detach
Make transfer attach/detach to/from connections chepaer.

- the "attach" event was no longer implemented by any filter
- the "detach" did the same as the "done" event for the filters
  who still implemented it. It should be superfluous as the "done"
  must always happen.

Closes #16067
2025-01-22 08:28:38 +01:00
Daniel Stenberg
f5f8f2c892
RELEASE-NOTES: synced 2025-01-22 08:09:51 +01:00
Daniel Stenberg
7c2b325004
curl_multi_fdset: include the shutdown connections in the set
They were previously missing.

Follow-up from c9b95c0bb3

Fixes #15156
Reported-by: Christopher Dannemiller
Closes #16049
2025-01-22 07:55:31 +01:00
Neil Johari
c80715169c
content_encoding: support use of custom libzstd memory functions
If ZSTD_STATIC_LINKING_ONLY is defined.

This functionality was introduced in zstd v0.8.1 in 2016 here:
facebook/zstd@be6180c

Closes #16028
2025-01-21 23:37:03 +01:00
Viktor Szakats
5fd7bd4379
appveyor: bump to OpenSSL 3.4
Closes #16064
2025-01-21 12:42:33 +01:00
Yedaya Katsman
38ad3902bf
docs/TLS-SESSIONS: fix typo, the->they
Closes #16057
2025-01-21 09:23:26 +01:00
Jay Satiro
789c7f1b6c easy_lock: use Sleep(1) for thread yield on old Windows
- Prefer Sleep(1) over sched_yield() for pre-Vista thread yield.

On Windows sched_yield is often implemented as Sleep(0) which only
yields to threads of highest priority to current priority. However,
during libcurl initialization if there is thread contention then it's
possible that there is a wait for a different library or OS thread of
a lesser priority and then the yield is not effective during that time.
On the other hand Sleep(1) will wait the minimum time slice which is
usually like 15ms or more.

Prior to this change 2c4bfef removed sched_yield detection on Windows,
which effectively removed the yield in the spin lock, and therefore this
change restores the yield but in a different way.

For Windows Vista and later we use SRW locks and do not have this issue.

Ref: https://github.com/curl/curl/pull/16037#issuecomment-2600161764
Ref: https://devblogs.microsoft.com/oldnewthing/20051004-09/?p=33923

Closes https://github.com/curl/curl/pull/16048
2025-01-21 01:45:21 -05:00
Viktor Szakats
7c039292ad
GHA/non-native: fix TLS backend in Android job names
Also drop whitespaces/alignment.

Reported-by: Tal Regev

Follow-up to 12a6de2f66 #16043
2025-01-21 01:10:16 +01:00
Viktor Szakats
12a6de2f66
GHA: add iOS jobs with LibreSSL, enable dependencies for Android via vcpkg
iOS:

- add jobs with autotools, CMake, CMake Xcode generator.
  The Xcode generator is >10x slower than Unix Makefiles. Keep it
  because it's the one recommended by CMake and for having its own
  quirks we may want to know about.
- build, cache and use LibreSSL for these jobs.
  With workaround for an iOS build issue fixed in master.
- make Xcode generator work by explicitly disabling code signing.
- make tests and examples build with the Xcode generator by setting
  `-DMACOSX_BUNDLE_GUI_IDENTIFIER=se.curl`, to avoid
  "Bundle identifier is missing" errors.
- cmake: disable `CURL_USE_PKGCONFIG` by default for Apple device.
- cmake: add `stdc++` library for BoringSSL and AWS-LC, with
  `OPENSSL_USE_STATIC_LIBS=ON` set.
- cmake: add workaround for Xcode generator issue, where it cannot
  handle two targets depending on one custom command. A better fix may
  be dropping `tool_hugehelp.c` and `tool_ca_embed.c` from curltool
  library. For a future PR.

Android:

- add vcpkg to Android jobs, enable dependencies.
  Assisted-by: Tal Regev via #16045
- make vcpkg work with autotools.
- pass `--with-brotli` to autotools to detect the vcpkg-supplied brotli.
- enable BoringSSL for Android and add a job with it.
- silence 457 CMake configure warnings about the Android NDK CMake
  scripts targeting freshly deprecated CMake versions.

These were much more involved than imagined. Basically nothing works out
of the box, and when combined, everything becomes a unique edge case.
autotools builds were a much easier to make work than CMake ones.

Also:

- GHA/non-native: re-sync names to be shorter and more aligned with
  other workflows.
- GHA: add `persist-credentials: false` where missing.

Unresolved issues:

- `OPENSSL_ROOT_DIR` ignored/mis-used when pointing it to LibreSSL.
  CMake seems to prepend the sysroot to the passed absolute directory.
  Found no workaround.
- CMake when combined with Android, both the Google-recommended method
  and the built-in CMake method fail to provide a way to avoid
  `pkg-config` packages at system directories. Failed to find a knob
  that can remove `/usr/include` from the search path. The workaround is
  to disable zstd. (I enabled it by default in this release, maybe
  premature?: f2adb3b6d7 #15431)
  Disabling `pkg-config` doesn't work because vcpkg dependencies do not
  link without it.
- CMake's Xcode generator is slow because each `try_compile()` feature
  check springs a new CMake + Xcode project taking a long time to run,
  just to compile single-liner C files. A known issue, with no solution.
  `-DCMAKE_MACOSX_BUNDLE=OFF` did not help, limiting build types to
  a single one (e.g. `Debug`) also had no effect.
   make | Xcode | GHA run
  :---- | :---- | :--------------------------------------------------------------------
    16s | 2m57s | https://github.com/curl/curl/actions/runs/12866334102/job/35868712426
    23s | 4m13s | https://github.com/curl/curl/actions/runs/12868128013/job/35874212461
    16s | 3m39s | https://github.com/curl/curl/actions/runs/12859073531/job/35849041880
    14s | 2m23s | https://github.com/curl/curl/actions/runs/12858298423/job/35847201313
    15s | 2m36s | https://github.com/curl/curl/actions/runs/12858058492/job/35846669761
    19s | 3m19s | https://github.com/curl/curl/actions/runs/12868919430/job/35876601168

Closes #16043
2025-01-20 22:32:06 +01:00
Viktor Szakats
292d81ca54
windows: merge config-win32ce.h into config-win32.h
They were more or less the same, but each missed some things the other
had. Windows CE is a subset of Win32, make the headers reflect that and
avoid duplications.

Ref: #15975
Closes #16038
2025-01-20 22:32:06 +01:00