Commit Graph

34209 Commits

Author SHA1 Message Date
Viktor Szakats
f2adb3b6d7
cmake: extend zlib's AUTO option to brotli, zstd and enable if found
- make `curl_dependency_option()` more generic.

- extend `CURL_BROTLI` and `CURL_ZSTD` options to accept
  `AUTO` in addition to existing `ON` and `OFF`.

- change `CURL_BROTLI` and `CURL_ZSTD` option default
  to `AUTO`. Was: `OFF`.
  It brings cmake behavior closer to `./configure`.
  Still different:
  - `./configure` defaults to `off` which means to check default
    locations. cmake checks more locations by default.
    (Also tried `NO_CMAKE_PATH`, but then it checked less locations.)
  - cmake returns both `brotlicommon` and `brotlidec` libs,
    while `./configure` only returns the latter.

- ci: drop explicit cmake options, that are now unnecessary.

- GHA/configure-vs-cmake: make adjustments to make tests pass.

Closes #15431
2024-12-17 04:06:45 +01:00
Viktor Szakats
290607f9d4
Revert: cmake: enable CURL_USE_PKGCONFIG for cross-MINGW
This reverts commit 39c06f7883 #15005.

Combined with most Find modules now supporting `pkg-config`
(39c741b7b0 #15408) this change made
mingw-cross builds fragile by picking up OS-native components. Also
adding `/usr/include` to the header path, confusing feature detection.
2024-12-17 03:21:22 +01:00
Viktor Szakats
39c741b7b0
cmake: add native pkg-config detection for remaining Find modules
brotli, c-ares, libpsl, libssh2, nghttp2, nghttp3, ntgcp2, zstd.

Also:

Add workaround for CMake reporting successful libssh2 detection, but
leaving the header directory empty, and causing `libssh2.h` not found
while compiling. It happens when `pkgconf` is not detecting libssh2
dependency libcrypto in Homebrew after `brew unlink openssl` (as in
GHA/macos). The workaround is to require a non-empty header directory
to consider the detection successful. This workaround may need to be
tweaked and/or applied to other Find modules.

Follow-up to 7bab201abe #15193

Closes #15408
2024-12-17 02:30:55 +01:00
Viktor Szakats
a3585c9576
Makefile.mk: drop in favour of autotools and cmake (MS-DOS, AmigaOS3)
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.

This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.

`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.

Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.

Details:

- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
  - default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
  - add support for `WATT_ROOT`.
  - use static libcurl with MS-DOS.
  - fixup default CMake suffixes/prefixes for DJGPP.
  - disable hidden symbols for MS-DOS. Not supported on MS-DOS.
  - opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
  - default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
  `Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
  Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
  - configure: detect `CloseSocket()` when it's a macro.
  - configure: fix `IoctlSocket` detection on AmigaOS.
  - curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
  - cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
    `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
  - set necessary system libs.
  - add AmiSSL support.
  - inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
    and they did not compile with AmigaOS.
  - cmake: better sync `gethostname` detection with autotools.
    Fixes detection for AmigaOS, where `gethostname` is a macro.
  - cmake: fix `sys/utime.h` detection on AmigaOS.
  - cmake: force-disable `getaddrinfo` for AmigaOS.
  - cmake: tweak threading and static/shared default for AmigaOS.
  - cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
  Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
  `zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
  `ioctl` is also detected, but fails when used. Make the above override
  it for a successful build.
  Authored-by: Darren Banfi
  Fixes #15537
  Closes #15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
  Overriding the default 4096.
  Suggested-by: Darren Banfi
  Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
  Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.

Non-MS-DOS/AmigeOS-specific tidy-ups:

- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
  configurations/requirements.
  This allows to honor global requirements added earlier.
  Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
  No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.

Follow-up to a8861b6ccd #9764

Closes #15543
2024-12-16 23:20:55 +01:00
Daniel Stenberg
607bec04d9
INSTALL.md: sync with my brag slide "104 operating systems"
Minoca, Nintendo 3DS and Maemo were missing.

Rename OS/400 to IBM I

Closes #15755
2024-12-16 23:09:11 +01:00
Viktor Szakats
39c06f7883
cmake: enable CURL_USE_PKGCONFIG for cross-MINGW
Enable `CURL_USE_PKGCONFIG` by default for MinGW cross-builds.

Note: This may cause fallouts in certain envs where `pkg-config` picks
up native packages.

Follow-up to e1ab01d1bd #14658
Follow-up to c555ab469d #14575

Closes #15005
2024-12-16 22:16:06 +01:00
Viktor Szakats
b9895b9d3f
cmake: namespace functions and macros
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.

Closes #15498
2024-12-16 21:55:00 +01:00
Viktor Szakats
fa676a6985
cmake: clang-cl improvements
- drop `/clang:` prefix for `-W` options for clang-cl.
  Except for `-Wall` which gets interpreted as MSVC `/Wall`
  and translated to `-Weverything`, which is undesired.
  Related: https://github.com/llvm/llvm-project/issues/102982

- include `MSVC_VERSION` in target flags.
  Useful for clang-cl builds where this information doesn't appear
  elsewhere in the cmake configure log.

- suppress `-Wlanguage-extension-token` more for clang-cl.
  This fixes clang-cl builds with default `CURL_WERROR=OFF` and
  `PICKY_COMPILER=ON`.
  This warning is enabled by `-pedantic` as a warning and by
  `-pedantic-errors` as an error. Verifiable using llvm's
  `diagtool show-enabled -pedantic test.c`.
  Follow-up to fb711b5098 #15449

Closes #15478
2024-12-16 21:45:07 +01:00
Viktor Szakats
b4aa93114c
cmake: replace unset(VAR) with set(VAR "") for init
Also add cleanup `unset()`s where missing.

Reported-by: Kai Pastor
Bug: https://github.com/curl/curl/pull/15255#issuecomment-2458659626
Follow-up to 8b09138083 #14610

Closes #15497
2024-12-16 21:12:51 +01:00
Viktor Szakats
ca348a64f4
GHA/codeql-analysis: install libpsl
Stay with the default CMake build. It requires libpsl now, but
the latest Ubuntu runner no longer seems to provide it. Install it
manually.

Bug: https://github.com/curl/curl/pull/15464#issuecomment-2546602052
Follow-up to 7afbc39173 #15464

Closes #15760
2024-12-16 21:09:10 +01:00
Viktor Szakats
7afbc39173
cmake: make libpsl required by default
As done earlier in `./configure`:

To force users to explictily disable it if they really don't want it
used and make it harder to accidentally miss it.

`-DCURL_USE_LIBPSL=OFF` is the option to use if PSL is not wanted.

Follow-up to 2998874bb6 #12661

Closes #15464
2024-12-16 19:39:46 +01:00
Viktor Szakats
49f2a23d50
cmake: add LDAP Find module
Move LDAP detection to its own Find module.

It supports `pkg-config` and the standard detection method used for
other dependencies, with version detection.

In curl CI it fixes LDAP detection in the OpenBSD job.

Closes #15273
2024-12-16 19:36:43 +01:00
Viktor Szakats
ae3ca135d1
lib: replace inline redefine with CURL_INLINE macro
Instead of redefining the `inline` keyword, introduce curl's own
`CURL_INLINE` macro and set it depending on the compiler's capabilities,
or use its value set via custom C flags.

Also keep honoring a custom `inline` macro, if set.

Closes #15523
2024-12-16 19:26:22 +01:00
Viktor Szakats
3dbd4362fd
cmake: use CMAKE_REQUIRED_LINK_DIRECTORIES
Use `CMAKE_REQUIRED_LINK_DIRECTORIES` with CMake 3.31.0 and upper,
in local macro `curl_required_libpaths()`.

9e95bd49f2
9e95bd49f2
https://gitlab.kitware.com/cmake/cmake/-/merge_requests/9795
https://cmake.org/cmake/help/v3.31/module/CheckSymbolExists.html

Tested OK with cmake 3.31.0-rc1.

Follow-up to 01a8157997 #15271
Follow-up to 7bab201abe #15193

Closes #15280
2024-12-16 18:35:34 +01:00
Viktor Szakats
05ba353435
cmake: drop redundant FOUND checks (libgsasl, libssh, libuv)
With `find_package(... REQUIRED)` the configuration fails and exits
if the package is not found. The `..._FOUND` check afterwards always
evaluates true and safe to delete.

Also true for brotli and zstd, but those are addressed differently
via #15431.

Closes #15465
2024-12-16 18:11:15 +01:00
Viktor Szakats
f58342ae21
cmake: improve curl_dumpvars() and move to Utilities.cmake
Also:
- dump variable types.
- show which variables are marked as advanced.
- use `IN ITEMS`.

Closes #15562
2024-12-16 18:08:13 +01:00
Viktor Szakats
dc874d4369
build: drop unused feature macros, update exception list
- cmp-config.pl: add remaining exceptions. Sort list.

- drop unused `HAVE_SYS_WAIT_H`.
  Follow-up to 50def7c881 #13249

- drop unused `HAVE_FCHMOD`.
  Follow-up to 03cb1ff4d6 #12395

- autotools: stop promoting variables to macros:
  `USE_NGTCP2_CRYPTO_*`, `USE_NGTCP2_H3`, `USE_OPENSSL_H3`,
  `HAVE_LIBRESSL`.
  They are not used in the source.

- cmake: drop unused `HAVE_O_NONBLOCK`,
  `HAVE_DISABLED_NONBLOCKING`.

- lib: drop `NEED_MALLOC_H`.
  It was used in manual-build cases for Amiga/MS-DOS/Windows/WinCE, but
  never by autotools/cmake, thus apparently unnecessary.

- lib: drop unused `NEED_MEMORY_H`.

- lib: simplify classic mac feature guards,
  drop `HAVE_EXTRA_STRICMP_H` and `HAVE_EXTRA_STRDUP_H`.

- autotools: drop unused `HAVE_GETHOSTBYNAME` detection.

- autotools: drop unused OpenSSL feature tests:
  `HAVE_ERR_H`, `HAVE_PEM_H`, `HAVE_RSA_H`

- autotools: drop unused OpenSSL feature tests:
  `HAVE_X509_H`, `HAVE_CRYPTO_H`, `HAVE_SSL_H`.

  They performed a fallback check when the primary check missed
  `openssl/x509.h`. Though if any other prefixed headers were found,
  OpenSSL is already assumed detected.

  The fallback check was looking for 3 unprefixed OpenSSL headers, and
  if all found, marked OpenSSL found internally, but did not promote
  it to `curl_config.h` via `USE_OPENSSL`. Meaning it either didn't do
  anything or may have continued with an inconsistent state.
  Added in d99c20f628 (2008)

  At the time, there was an extra `AC_DEFINE(USE_SSLEAY, 1 ...` logic
  after this code, which kicked in in the fallback case, but that code
  was deleted in 709cf76f6b (2015)

  Follow-up to 709cf76f6b

- autotools: drop `AC_SUBST()` where the value is explicitly set anyway
  and the macro is unused.

- autotools: replace `AC_SUBST(VAR, 1)` with local variable assigments,
  where the `@VAR@` macro is unused. Also dedupe the local variable if
  there was a parallel one used for the same purpose.

- autotools: drop local feature variables that were never used.

- autotools: drop unused `CURL_CHECK_OPTION_NTLM_WB`,
  `CURL_CHECK_NTLM_WB`.
  Also stop setting unused `NTLM_WB_ENABLED` macro for VMS.
  Follow-up to 50def7c881 #13249

- autotools: drop unused `PKGADD_*`.
  Follow-up to bae0d473f5 #3331

- autotools: drop unused `CURL_NETWORK_LIBS`.
  Follow-up to 3af75e18d6 #14697

Closes #15577
2024-12-16 18:05:24 +01:00
Viktor Szakats
ba9fe58d43
GHA: set persist-credentials: false
Suggested by zizmor GHA analysis tool.

Also:
- Move GH variables within single-quotes.
- Prefer single-quotes in shell code. (tidy-up)

Ref: https://github.com/actions/checkout/issues/485
Ref: https://github.com/actions/checkout/pull/1687
Ref: https://woodruffw.github.io/zizmor/

Closes #15746
2024-12-16 18:00:22 +01:00
Daniel Stenberg
9991f255dd
RELEASE-NOTES: synced 2024-12-16 10:09:22 +01:00
Daniel Stenberg
0af5ce162f
variable.md: mention --expand-variable for variables to variables
To assign a variable using contents from another variable.

Closes #15752
2024-12-16 08:17:22 +01:00
Hermes Zhang
fc5c9ed253 tool_getparam: fix memory leak on error in parse_ech
- Free tmpcfg memory before returning an error code if aprintf failed.

Closes https://github.com/curl/curl/pull/15753
2024-12-16 00:56:11 -05:00
Daniel Stenberg
6eb86e428d
ws-docs: remove the outdated texts saying ws support is experimental
It stopped being experimental in 8.10.0

Fixes #15749
Reported-by: Mohammed Sadiq
Closes #15751
2024-12-15 18:14:52 +01:00
Daniel Stenberg
c3857eca70
altsvc: avoid integer overflow in expire calculation
A bad value here just makes for a bad alt-svc experience, not a security
problem.

Detected by OSS-Fuzz

Bug: https://issues.oss-fuzz.com/issues/383911309

Closes #15745
2024-12-15 00:11:24 +01:00
Viktor Szakats
f7cd713d38
cmake/FindMbedTLS: drop lib duplicates early
When de-duplicating the list of raw libs, make sure to drop duplicates
from the beginning of the list.

Reported-by: Kai Pastor
Ref: https://github.com/curl/curl/pull/15273#pullrequestreview-2417191841
Closes #15495
2024-12-14 23:43:02 +01:00
Daniel Stenberg
9294ca8916
variable.md: show function use with examples
Closes #15743
2024-12-14 17:00:23 +01:00
Daniel Stenberg
cd80716cb6
docs/BUGS.md: remove leading space from a link 2024-12-14 14:22:38 +01:00
Daniel Stenberg
62515e8e9d
scripts/mdlinkcheck: markdown link checker
This script parses all markdown files in the repository, extracts all
links and verifies that they work.

It makes sure to only check the URLs once, even if used in multiple
links. There is a whitelist for URLs we deem unnecessary to check.

It uses curl to do the checks.

As a bonus, this makes it easy to run this check locally.

Closes #15742
2024-12-14 14:22:30 +01:00
Daniel Stenberg
6bb76d92e1
tool_getparam: fix the ECH parser
The pn: and ecl: prefixes were not handled correctly.

CodeSonar helped me realize as it poined out an "Unreachable Data Flow"

Closes #15741
2024-12-14 11:51:30 +01:00
Daniel Stenberg
cffbbc693e
tool_getparam: fix "Ignored Return Value"
Follow-up from a300879b63

Pointed out by CodeSonar

Closes #15740
2024-12-14 11:50:23 +01:00
Daniel Stenberg
7347ddc9e2
tool_operate: make --etag-compare always accept a non-existing file
Consider it a blank etag. It allows for more use cases when the file
just might not have been created yet.

Closes #15737
2024-12-13 15:48:27 +01:00
Daniel Stenberg
c64c3527e5
RELEASE-NOTES: synced 2024-12-13 11:50:43 +01:00
renovate[bot]
97866f1620
GHA: pin dependencies
For recently added actions in checkdocs.yml and windows.yml

Closes #15722
2024-12-13 10:52:40 +01:00
Daniel Stenberg
a300879b63
curl: return error if etag options are used with multiple URLs
And document it.

Add tests 484 and 485

Fixes #15729
Reported-by: Tamir Duberstein
Closes #15731
2024-12-13 10:11:26 +01:00
Kevin Sun
0439499170
netrc: restore _netrc fallback logic
Regression from 05977f4f75fd08837a877, shipped in 8.11.0

Fixes #15734
Closes #15735
2024-12-13 10:08:00 +01:00
Daniel Stenberg
c2ac9ea1ee
checksrc: ban use of sscanf()
Using sscanf() is not a (security) problem in itself, but we strongly
discorage using it for parsing input since it is hard to use right, easy
to mess up and often makes for sloppy error checking.

Allow it in examples and tests

Closes #15687
2024-12-13 09:43:05 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00
Andy Pan
ff5091aa9f
async-thread: avoid closing eventfd twice
When employing eventfd for socketpair, there is only one file
descriptor. Closing that fd twice might result in fd corruption.
Thus, we should avoid closing the eventfd twice, following the
pattern in lib/multi.c.

Fixes #15725
Closes #15727
Reported-by: Christian Heusel
2024-12-12 15:58:47 +01:00
Randall S. Becker
aed732acb1
tests/http/clients: use proper sleep() call on NonStop
This fix impacts ws-data.c and ws-pingpong.c. sleep() replaced with
system PROCESS_DELAY_() having the same resolution as usleep().

Fixes #15711
Closes #15712

Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
2024-12-12 14:30:28 +01:00
Randall S. Becker
ae4ec1d2ec
system.h: add 64-bit curl_off_t definitions for NonStop
Fixes #15723
Closes #15724

Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
2024-12-12 14:27:40 +01:00
Christian Schmitz
df1d08ec0d
sectransp: free certificate on error
Otherwise the certificate memory was leaked.

Closes #15721
2024-12-11 13:09:44 +01:00
Christian Schmitz
017e6440ce
conncache: result_cb comment removed from function docs
result_cb doesn't exist anymore

Closes #15720
2024-12-11 13:07:18 +01:00
Daniel Stenberg
a8397643f3
GHA/checkdocs: change markdown link checker to linkspector
Fixes #15717
Closes #15719
2024-12-11 13:05:11 +01:00
Daniel Stenberg
6907638631
RELEASE-NOTES: synced
Start working on the next release
2024-12-11 11:16:48 +01:00
Daniel Stenberg
98b30eda79
RELEASE-PROCEDURE.md: mention how to publish security advisories
As they are releated to the release procedure

Closes #15714
2024-12-11 11:11:32 +01:00
Daniel Stenberg
a8c852b9a5
cookie: parse only the exact expire date
The date parser function is very forgiving and skips most "irrelevant"
characters in its hunt for a date to figure out. Therefore it is
important to make sure the date string is properly null terminated so
that it does not accidentally parse a piece of whatever text follows
after the date.

Add test483: test (overly) long expire dates in cookies

Closes #15709
2024-12-11 09:31:53 +01:00
Daniel Stenberg
75f0835513
lib517: extend the getdate test with quotes and leading "junk"
Closes #15708
2024-12-11 09:19:50 +01:00
Daniel Stenberg
75a2079d5c
RELEASE: synced
curl 8.11.1 release
2024-12-11 08:05:13 +01:00
Daniel Stenberg
cff5a7b641
THANKS: contributors from 8.11.1 2024-12-11 08:05:13 +01:00
Viktor Szakats
2d4852af83
build: fix tests when documentation/manual is disabled
- build: fix to exclude 'documentation' tests when building
  the documentation is explicitly disabled. Both for cmake
  and `./configure`.

  Reported-by: Daniel Engberg
  Fixes #15703

- test481, test482: fix for builds with no manual.

  Reported-by: Daniel Engberg
  Fixes #15703

- configure: fix to always detect Perl. Running tests require
  it when run. Before this patch Perl wasn't detected when
  documentation/manual/embedded-CA were all disabled,
  making tests fail to start.

- test1177: add keyword `documentation`. It depends no
  `curl_version_info.3`.

- GHA/linux: test cmake and `./configure` with docs and
  manual disabled and tests run.

Closes #15704
2024-12-09 12:45:29 +01:00
renovate[bot]
72266a3e9f
GHA: update four depencencies
- github/codeql-action digest to aa57810
- cross-platform-actions/action action to v0.26.0
- actions/cache digest to 1bd1e32
- msys2/setup-msys2 digest to d44ca8e

Closes #15674
Closes #15686
Closes #15696
Closes #15702
2024-12-09 10:31:24 +01:00