Update IP related information at the connection and the transfer in two
places only: once the filter chain connects and when a transfer is added
to a connection. The latter only updates on reuse when the filters
already are connected.
The only user of that information before a full connect is the HAProxy
filter. Add cfilter CF_QUERY_IP_INFO query to let it find the
information from the filters "below".
This solves two issues with the previous version:
- updates where often done twice with the same info
- happy eyeballing filter "forks" could overwrite each others
updates before the full winner was determined.
Closes#14699
When we downloaded all we wanted, and we did not want a response body,
and no Trailer: has been announced, and the receive gives EAGAIN, do not
hang around unnecessarily.
Some servers are buggy in HEAD processing and fail to send the HTTP/2
EOS. Since we do not need any more data, end the request right there.
This will cause us to send a RST_STREAM to the server.
Fixes#14670
Reported-by: Gruber Glass
Closes#14685
This is a better match for what they do and the general "cpool"
var/function prefix works well.
The pool now handles very long hostnames correctly.
The following changes have been made:
* 'struct connectdata', e.g. connections, keep new members
named `destination` and ' destination_len' that fully specifies
interface+port+hostname of where the connection is going to.
This is used in the pool for "bundling" of connections with
the same destination. There is no limit on the length any more.
* Locking: all locks are done inside conncache.c when calling
into the pool and released on return. This eliminates hazards
of the callers keeping track.
* 'struct connectbundle' is now internal to the pool. It is no
longer referenced by a connection.
* 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters
no longer need to set it. Instead, the multi checks on leaving
MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now
multiplexed and new, e.g. not conn->bits.reuse. In that case
the processing of pending handles is triggered.
* The pool's init is provided with a callback to invoke on all
connections being discarded. This allows the cleanups in
`Curl_disconnect` to run, wherever it is decided to retire
a connection.
* Several pool operations can now be fully done with one call.
Pruning dead connections, upkeep and checks on pool limits
can now directly discard connections and need no longer return
those to the caller for doing that (as we have now the callback
described above).
* Finding a connection for reuse is now done via `Curl_cpool_find()`
and the caller provides callbacks to evaluate the connection
candidates.
* The 'Curl_cpool_check_limits()' now directly uses the max values
that may be set in the transfer's multi. No need to pass them
around. Curl_multi_max_host_connections() and
Curl_multi_max_total_connections() are gone.
* Add method 'Curl_node_llist()' to get the llist a node is in.
Used in cpool to verify connection are indeed in the list (or
not in any list) as they need to.
I left the conncache.[ch] as is for now and also did not touch the
documentation. If we update that outside the feature window, we can
do this in a separate PR.
Multi-thread safety is not achieved by this PR, but since more details
on how pools operate are now "internal" it is a better starting
point to go for this in the future.
Closes#14662
- GHA/macos: set timeout for test runs.
Double the value for autotools to fit torture tests.
- GHA/cygwin: reduce test run timeout for autotools.
- GHA/cygwin: enable building tests with cmake.
- GHA/windows: enable building tests with MSYS cmake.
- GHA/windows: enable building tests with MSVC UWP.
- appveyor: enable building tests with VS2008 x86 and VS2010 x64.
- tests: add workaround compiler warnings when building with VS2010:
```
tests\server\util.c(482): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(486): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(490): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(720): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(726): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(732): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(781): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(785): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
tests\server\util.c(789): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/50485633/job/4ujlwxod3cexmn2q#L1535
With this, tests are built in all GHA cygwin/windows jobs.
Timeouts avoid long runs with runaway issues, example:
https://github.com/curl/curl/actions/runs/10575522173Closes#14700
1. GHA/windows: enable WinIDN in Linux cross-builds.
(to reveal the issue in CI.)
2. fix compiler warning when building with mingw-w64 supporting
WinIDN, while targeting pre-Vista Windows, with a `WINVER` set to
target Vista or newer. (Such was Ubuntu's mingw-w64 with the
classic-mingw-specific trick in point 3 of this PR.)
```
../../lib/idn.c:154:23: error: redundant redeclaration of ‘IdnToAscii’ [-Werror=redundant-decls]
154 | WINBASEAPI int WINAPI IdnToAscii(DWORD dwFlags,
| ^~~~~~~~~~
In file included from /usr/share/mingw-w64/include/windows.h:73,
from /usr/share/mingw-w64/include/winsock2.h:23,
from ../../lib/setup-win32.h:91,
from ../../lib/curl_setup.h:308,
from ../../lib/idn.c:29:
/usr/share/mingw-w64/include/winnls.h:1075:30: note: previous declaration of ‘IdnToAscii’ was here
1075 | WINNORMALIZEAPI int WINAPI IdnToAscii (DWORD dwFlags, LPCWSTR lpUnicodeCharStr, int cchUnicodeChar, LPWSTR lpASCIICharStr, int cchASCIIChar);
| ^~~~~~~~~~
[...same for IdnToUnicode...]
```
Ref: https://github.com/curl/curl/actions/runs/10542832783/job/29210098553#step:7:89
3. drop `WINVER` override for classic-mingw. curl no longer supports
building with classic-mingw.
Reverts 37f1c21cb9#7581
4. sync `if IdnToUnicode can be linked` detection snippet with the live
code in `lib/idn.c`. It fixes detection for the scenario in point 2.
5. delete unused `WINIDN_DIR` variable.
Bug: https://github.com/curl/curl/pull/12606#issuecomment-1885381038
Previous abandoned attempt: #12684
Reviewed-by: Jay Satiro
Closes#14680
Before this patch, `libhostname.so` and `chkhostname` were a test
facility for overriding `gethostname()` in non-debug builds on
Linux and other Unix platforms supporting `LD_PRELOAD`.
`gethostname()` has a single use with SMTP.
The alternative way to override `gethostname()` is building in debug
mode, which allows to do this via the `CURL_GETHOSTNAME` env, on all
platforms.
Drop the `LD_PRELOAD` solution in favour of the above.
Also:
- delete inactive NTLM code with a `gethostname()` call made from it.
- streamline NTLM code by dropping a `printf()` and a macro.
- tests: stop setting `CURL_GETHOSTNAME` where unnecessary.
Closes#14695
A number of checks don't match our style or are buggy and so are disabled.
Co-authored-by: Viktor Szakats <vszakats@users.noreply.github.com>
Fixes#14580Closes#14665
Before this change, calling curl_share_setopt w/ CURL_LOCK_DATA_CONNECT
a second time would re-initialize the connection cache, rather than use
the existing one.
After this change, calling curl_share_setopt w/ CURL_LOCK_DATA_CONNECT
multiple times will have no effect after the first call.
Closes#14696
- skip adding pkg-config libdirs if they are system locations.
- replace custom regexes with `get_filename_component()`.
- collect `-L` and `-framework` separately.
It means these will appear before libs in the `Libs.private` entry,
syncing it with `./configure`.
- collect in a list variable (was: string).
- use `list(REMOVE_DUPLICATES)` to deduplicate libdirs.
- rename internal variable that is now solely used for system libdirs.
Follow-up to 7c0b6eb3bd#14652Closes#14668
Curl_xfer_send and Curl_xfer_recv had commented FIXMEs about protocol
setting up the transfers badly, but in reality these functions are too
low-level to be able to depend on the protocol transfer setups having
been done yet. Removed.
The functions had checks for data and data->conn that I convered to
asserts since they SHOULD always be valid in this function. The same
goes for the runtime check for buffer_size > 0 that I also converted to
an assert since that should never be set to an invalid value.
Closes#14688
Remove the "hardcoded" logic for the pop3 transfer handler and instead
use the generic protocol handler write_resp function.
Remove the check for 'data->req.ignorebody' because I cannot find a code
flow where this is set for POP3.
Closes#14684
- configure: disable pthreads by default on Windows.
- configure: disable detecting `fseeko()` on Windows.
(It exists in mingw-w64 2.0.0 and newer, but it's permanently ignored
in CMake, as this function is never necessary on Windows.)
- extend existing exceptions with their Windows variants.
- `lib/formdata.c`: prioritize `_fseeki64()` over `fseeko()`.
To reduce the difference between Windows builds, which now all use
`_fseeki64()`.
- cmake: perm-enable `HAVE_DIRENT_H` and `HAVE_OPENDIR` for mingw-w64,
to match configure.
Follow-up to bfe54b0e88#13137
This in theory could make the dir listing feature work in mingw-w64
build, but in my tests (on WINE) it failed at the preceding `open()`
call.
- cmake: perm-enable `HAVE_STRINGS_H` and `HAVE_UTIME_H` for mingw-w64,
to match configure. (They are wrappers and make no difference in the build.)
Also:
- configure: sync `USE_MANUAL` macro with cmake, by only setting it for
`src`. Drop checker exception.
- CI: use `--disable-dependency-tracking` in existing jobs.
- CI: install packages before git checkout, in existing jobs.
Closes#14678
If AppleIDN or WinIDN is selected, don't look for libidn2. Do this by
moving libidn2 detection after AppleIDN/WinIDN and skipping it if any
of them was selected.
Also:
- disable AppleIDN by default with autotools to sync behaviour with
CMake.
- limit WinIDN checks to native Windows with autotools, as with CMake.
Before this patch libidn2 was detected by default even if AppleIDN or
WinIDN was explicitly selected. libidn2 wasn't used in the build, but
it was left enabled as a dependency and appeared in `libcurl.pc` and
`curl-config`.
Closes#14674
- show `OpenSSL v3+` when detected (as in `./configure`).
(this string also makes its way to `curl-config`.)
- prefer `unset(VAR)` over `set(VAR)`.
Same effect, but `unset()` tells the intent unambiguously.
https://cmake.org/cmake/help/latest/command/set.html
- drop "implementation" from an `option()` description.
- FindGSS: replace legacy keyword alias with modern alternative.
https://cmake.org/cmake/help/latest/command/get_filename_component.html
- move `CURL_STATIC_CRT` logic next to its `option()`.
- improve order of `libcurl.pc`/`curl-config` variable init lines.
- tests: drop/shorten custom target names.
They inflated generated make files by 550KB.
Keep target name logic for sync between code snippets.
Follow-up to a2ef5d36b3#14660
- clear a variable after use.
- restore `STATUS` for `Features:`/`Protocols:` `message()`s:
Without it the output goes to stderr, and appears in red in CMake GUI.
It doesn't seem possible to show a line on stdout without leading
underscores to match `curl -V` and `./configure` output.
Partial revert of acbc6b703f#14197
- WindowsCache: move `HAVE_LINUX_TCP_H` into the header group.
- move strings to the same line as their `STRING` keyword.
- formatting in generated code.
- delete bogus comment.
- unfold lines for readability.
- fix a too long line. (for cmakelint)
- missing quotes, whitespace, comments.
Closes#14610
- call them "manpages", not "man pages" and drop the .1 and .3
- replace link to "manual" with a link to everything curl
- remove the link to the FAQ
- add a new "open source "subtitle for the license link, and rephrase
- mention and link GitHub services in the contact section
- rename the "git" subtitle to "source code"
- shorten the source code section somewhat
Closes#14663
The missing message only uses a single line now and has a consistent
prefix making it easier for log parsers to skip. Remove a precheck test
looking for impacket in test1451 because it's incomplete, unnecessary
and in the wrong layer; the test harness will automatically discover
that the smbserver won't start and the test was only checking a single
dependency. Also update the tests README to no longer mention Python 2.
The servers were switched to Python 3 several years ago and no attempt
is being made to maintain Python 2 compatibility
Closes#14669
Also:
- add a line to CMake log output showing the host and target OS and CPU
when doing cross-builds.
- reduce excessive timeout for `curl -V` steps.
Closes#14661
- reuse local variable names.
- sync `PROJECT_LABEL`, add where missing.
- namespace all target names.
- bind header directories to each target.
- tests/server: limit `CURL_STATICLIB` to Windows (as in autotools.)
- drop functions with a single caller.
Closes#14660
A bunch of tiny jobs that run various source or documentation checks are
consolidated into two workflow files: checksrc.yml and checkdocs.yml.
This reduces the proliferation of new files containing one-line checks
and brings those that operate similarly together for better reasoning
about them. The man-examples check is also now running again for the
first time in 7 months.
Various calls to find, xargs and git ls-files are changed where possible
to use NUL line terminators in pipes to avoid issues with oddly-named
files that might find their way into the repo.
Closes#14654
It was previously wrongly verifying the input in its URL encoded format
when setting the hostname component with curl_url_set(), so it wrongly
rejected '%'.
Now it URL decodes the name appropriately before the check.
Added tests to lib1560 to verify that a fine %-code is okay and that a
bad %-code (that decodes to '%') is rejected.
Regression from 0a0c9b6dfa, shipped in 8.0.0
Fixes#14656
Reported-by: Venkat Krishna R
Closes#14657
- Install stunnel.
- Regenerate certificates (as SecureTransport requires a validity period
less than 398 days).
- Restart server if it is unresponsive.
- Do not hardcode the SHA-256 base64 public pinned key.
- Ignore test 313 as SecureTransport does not support crl file.
- Ignore tests 1631 and 1632 as SecureTransport is not yet able to shut
down FTP over HTTPS gracefully.
- Add a CMake target for generating certificates.
Closes#14486
In Find modules with native pkg-config detection (libgsasl, libidn2,
libssh, libuv, nettle) use the C compiler flags returned by pkg-config.
Also use the library paths, and return the pathless library names.
Also:
- add these library paths to `libcurl.pc`/`curl-config`.
- fix libgsasl detection to use the detected header directory.
FindGSS already did this before this patch.
Fixes#14641Closes#14652
- add pkg-config-specific 'Found' message to the Find module.
- update non-pkg-config 'Found' message to show the flavour instead of
the library name. (= the first value listed after `REQUIRED_VARS`)
- delete extra 'Found' message from `CMakeLists.txt`.
- rename internal result variables to match with default pkg-config
names, in preparation of introducing them in other Find modules:
- `GSS_LINK_DIRECTORIES` -> `GSS_LIBRARY_DIRS`
- `GSS_LINKER_FLAGS` -> `GSS_LDFLAGS`
- `GSS_COMPILER_FLAGS` -> `GSS_CFLAGS`
Ref: #14652Closes#14651
Previously this test allowed several error values when setting options.
This made this test miss #14629.
Now, errors are generally not accepted for setopts:
- numerical setopts accept CURLE_BAD_FUNCTION_ARGUMENT for funny input
- the first setopt to an option accepts CURLE_NOT_BUILT_IN or
CURLE_UNKNOWN_OPTION for when they are disabled/not built-in
- there is an allowlist concept for some return code for some variables,
managed at the top of the mk-lib1521.pl script
In curl.h: remove the OBSOLETE named values from the setopt list.
Closes#14634
