Further testing with timeouts in event based processing revealed that
our current shutdown handling in the connection pool was not clear
enough. Graceful shutdowns can only happen inside a multi handle and it
was confusing to track in the code which situation actually applies. It
seems better to split the shutdown handling off and have that code
always be part of a multi handle.
Add `cshutdn.[ch]` with its own struct to maintain connections being
shut down. A `cshutdn` always belongs to a multi handle and uses that
for socket/timeout monitoring.
The `cpool`, which can be part of a multi or share, either passes
connections to a `cshutdn` or terminates them with a one-time, best
effort.
Add an `admin` easy handle to each multi and share. This is used to
perform all maintenance operations where no "real" easy handle is
available. This solves the problem that the multi admin handle requires
some additional initialisation (e.g. timeout list).
The share needs its admin handle as it is often cleaned up when no other
transfer or multi handle exists any more. But we need a `data` in almost
every call.
Fix file:// handling of errors when adding a new connection to the pool.
Changes in `curl` itself:
- for parallel transfers, do not set a connection pool in the share,
rely on the multi's connection pool instead. While not a requirement
for the new `cshutdn` to work, this is
a) helpful in testing to trigger graceful shutdowns
b) a broader code coverage of libcurl via the curl tool
- on test_event with uv, cleanup the multi handle before returning from
parallel_event(). The uv struct is on the stack, cleanup of the multi
later will crash when it tries to register sockets. This is a "eat
your own dogfood" related fix.
Closes#16508
To make sure we store and use the correct port used for this particular
lookup.
Partial revert of 8ded8e5f3fFixes#16531
Reported-by: Jay Satiro
Closes#16532
Add a standalong hash table for curl_offt_t as key. This allows a
smaller memory footprint and faster lookups as we do not need to deal
with variable key lengths.
Use in all places we had the standard hash for this purpose.
Closes#16442
Fixes#14973
Reported-by: stevenpackardblp on github
When curl negotiated with a http: proxy for a https: request, it
wrongly believed there must be an SSL filter present, which during
CONNECT, there is not.
25b445e fixed this. This PR adds a pytest case for the setup.
Closes#16136
Keep the `@for %%i in [...]` lines within limits by stripping whitespace
from the input `.c` source lists read from `Makefile.inc`. To avoid this
error after adding a new `.c` source:
```
configuration name: libcurl-vc14-x64-release-dll-ssl-dll-ipv6-sspi
NMAKE : fatal error U1095: expanded command line 'for %i in (altsvc.obj amigaos.obj
asyn-ares.obj asyn-thread.obj base64.obj bufq.obj
bufref.obj cf-h1-proxy.obj cf-h2-proxy.obj cf-haproxy.obj [...]
vssh/wolfssh.obj) do @echo ..\builds\libcurl-vc14-x64-release-dll-ssl-dll-ipv6-sspi-obj-lib/%i \
' too long
Stop.
Command exited with code 2
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51605338/job/dqg6qtebtscb279g#L44
Reported-by: Stefan Eissing
Bug: https://github.com/curl/curl/pull/16508#issuecomment-2690443409Fixes#16521Closes#16528
- enable quictls in autotools and cmake jobs. autotools requires
a workaround due to wrong libpath in the quictls pkg-config.
nghttp3 is offered by Homebrew, but not ngtcp2, to enable H3.
- install `libnghttp2` rather than `nghttp2`.
`libnghttp2` is preinstalled and smaller. It also avoids detecting
`nghttpx`, which confuses `pytest`.
- limit `brew unlink openssl` to libressl/quictls jobs.
Closes#16517
Fixes#16249
Forwarded-to-us-by: Carlos Henrique Lima Melara
Always use `gnutls_certificate_set_x509_key_file2()` for loading keys
and certificates, even without a password, since this function support
pkcs11 urls.
Thanks to @tatsuhiro-t for finding this out.
Help-by: Tatsuhiro Tsujikawa
Closes#16472
To avoid breaking the control flow and align to majority of code
already using `return`.
`exit()` has the side-effect of suppressing leak detection in cases.
Fix fallouts detected after switching to `return`.
- configure:
- fix `getaddrinfo` run test to call `freeaddrinfo()` to pacify ASAN,
and call `WSACleanup()` to deinit winsock2.
- fix `getifaddrs` run test to call `freeifaddrs()` to pacify ASAN.
- tests/server:
- setup `atexit(win32_cleanup)` via `win32_init()`.
- return 2 instead of 1 on winsock2 init failures.
- sws: goto cleanup instead of `exit()` in `http_connect()`.
Follow-up to 02dfe71937#7235
- tests/client/http:
- cleanup memory to pacify ASAN in `h2-upgrade-extreme`,
`tls-session-reuse`.
- examples:
- block_ip: fix memory leak reported by CI.
- http2-upload: avoid handle leaks.
Untouched `exit()` calls, made from callbacks:
- docs/examples: ephiperfifo.c, ghiper.c, hiperfifo.c
- tests/libtest: lib582.c, lib655.c, lib670.c
- tests/server: tftpd.c
Closes#16507
The initial curl CMake commit introduced it in 2009-04-02 via
4c5307b456. Suppressing a stray
`-Wlong-double` warning in `mprintf.c`. This was before Apple switched
to clang, and likely affected the Apple distributed GCC, version 4.2.1
at the time. It applied the workaround to CMake builds only, though
the issue seems build-tool agnostic. Yet, it was not suppressed or
reported for autotools builds.
For these reasons this logic seems obsolete and this patch drops it with
no replacement. It saves a feature detection for GCC builds for macOS.
In PR sub-commits I added (and reverted) in-source suppression. In case
it becomes necessary, that should fix it for all build tools.
Closes#16513
This now makes sure to trim off exact matches for curl symbols and long
curl commanad line options instead of using pattern matching as before.
This should catch typoed names (that still follow the pattern) better.
The cleanspell.pl script is no longer used. cleancmd.pl is used for all
markdown files.
Closes#16504
Switch VS2008 job the oldest runner machine. It adds the oldest CMake to
the Windows mix, from 2018-11-30. Not a beauty, missing support for `-B`
and Unity, but it's a version curl supports. It's newer than Old Linux.
The previous oldest was 3.16.2. It remains used with VS2010-VS2017.
Also:
- fix VS2008 job to actually build examples.
- switch VS2019 job to OpenSSL 1.1.0 that wasn't tested before.
Migrate OpenSSL 1.0.2 to the VS2008 job.
- measure run time of individual build steps.
Follow-up to 01c25e3b00#16458Closes#16505
- replace `--parallel <n>` and `-j<n>` for individual commands with
`MAKEFLAGS`, for jobs not yet doing it.
This enables parallel builds in distcheck / maketgz-and-verify-in-tree,
where `-j` option was missing.
- add `--parallel` for iOS Xcode job for improved performance.
- drop redundant `-j5` for Android jobs.
- drop stray `cmake --config` options from single-target jobs (cygwin,
msys/mingw, dl-mingw, non-native). Drop redundant
`CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` settings too.
- GHA/windows: add timeout for package install steps where missing.
- GHA/non-native: specify target type explicitly for iOS cmake jobs.
Xcode default was already Debug, single-target default was generic,
now it's Release, with unity batch to keep it fast.
`MAKEFLAGS` is necessary for autotools jobs and CMake jobs using
the default (GNU Make) generator. It's ignored by Ninja and other tools.
`cmake --parallel` is still necessary for jobs with Visual Studio or
Xcode generators. Parallelism is 5 for GHA Linux and Windows runners,
4 for macOS, 3 for VMs, 2 for AppVeyor.
Closes#16502
- fix redirect from file:// URL with query part
- find_host_sep() simplify
- urlencode_str() simplify
- redirect_url() simplify
- made more const char *
- add more redirect URL test cases to test 1560
Closes#16498
- GHA/windows/WinCE:
- set `-O3 -DNDEBUG` C flags manually for the CMake mingw32ce build.
CMake doesn't recognize the platform and fails to add them. To match
autotools (using `-O2`), and hit similar compiler warnings.
- enable parallel builds for cmake.
- tune parallelism for cmake using unity batches.
- tune parallelism for autotools.
Follow-up to 2a292c3984#15975
- tests: fix potentially uninitialized value in `readline()` in
`getpart.c`. Detected by gcc 4.4.0 `-O2` (Windows CE) jobs:
```
tests/server/getpart.c: In function 'getpart':
tests/server/getpart.c:298: error: 'datalen' may be used uninitialized in this function
```
Ref: https://github.com/curl/curl/actions/runs/13522595237/job/37785147505?pr=16476#step:11:25
Follow-up to 592880a3ca
- vtls_scache: rework returning pointer to avoid compiler warning seen
with `-O3` gcc 4.4.0 builds (Windows CE/schannel):
```
lib/vtls/schannel.c: In function 'schannel_connect_step1':
lib/vtls/vtls_scache.c:975: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
lib/vtls/vtls_scache.c:985: error: dereferencing pointer 'old_cred.4474' does break strict-aliasing rules
lib/vtls/schannel.c:959: note: initialized from here
```
Ref: https://github.com/curl/curl/actions/runs/13523868335/job/37789610845#step:9:25
Follow-up to fa0ccd9f1f#15774Closes#16476
Before this patch, it missed this in `curl/curl.h`:
```c
# define __has_declspec_attribute(x) 0
```
After this patch:
```
test 1167...[Verify curl prefix of public symbols in header files]
/usr/bin/perl -I. -I. returned 1, when expecting 0
1167: exit FAILED
== Contents of files in the log/14/ dir after test 1167
=== Start of file server.cmd
Testnum 1167
=== End of file server.cmd
=== Start of file stdout1167
Bad symbols in public header files:
__has_declspec_attribute(x)
=== End of file stdout1167
FAIL 1167: 'Verify curl prefix of public symbols in header files' source analysis
TESTFAIL: These test cases failed: 1167
```
Ref: #16491Closes#16496
Public curl headers are best not to define 3rd-party or system macros.
Introduce `CURL_HAS_DECLSPEC_ATTRIBUTE` to cover this system macro and
use it.
Detected by test1167 after dropping the indentation:
```
test 1167...[Verify curl prefix of public symbols in header files]
/usr/bin/perl -I. -I. returned 1, when expecting 0
1167: exit FAILED
== Contents of files in the log/10/ dir after test 1167
=== Start of file server.cmd
Testnum 1167
=== End of file server.cmd
=== Start of file stdout1167
Bad symbols in public header files:
__has_declspec_attribute(x)
=== End of file stdout1167
```
Ref: https://github.com/curl/curl/actions/runs/13533200900/job/37819784405?pr=16490#step:42:2087
Ref: https://clang.llvm.org/docs/LanguageExtensions.html#has-declspec-attribute
Follow-up to 50482b8c0a#3616
Ref: #16496 (fixing test1167)
Closes#16491
`./configure` mingw32ce builds enable C99 mode automatically, that
triggers compiler warnings in gcc 4.4.0. We initially worked it around
in CI by suppressing the detection of C99 with `ac_cv_prog_cc_c99=no`.
Replace it with automatically silencing the bogus warnings in C99 mode,
for all build systems:
```
lib/ftp.c: In function 'Curl_GetFTPResponse':
lib/ftp.c:726: error: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
lib/ws.c: In function 'ws_dec_pass_payload':
lib/ws.c:304: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/ws.c: In function 'ws_enc_write_head':
lib/ws.c:581: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'long int'
lib/vtls/schannel.c: In function 'schannel_connect_step1':
lib/vtls/schannel.c:1122: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_connect_step2':
lib/vtls/schannel.c:1311: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_send':
lib/vtls/schannel.c:1793: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c:1810: error: format '%zd' expects type 'signed size_t', but argument 3 has type 'ssize_t'
lib/vtls/schannel.c: In function 'schannel_shutdown':
lib/vtls/schannel.c:2286: error: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t'
lib/vtls/vtls.c: In function 'ssl_cf_recv':
lib/vtls/vtls.c:1422: error: format '%zd' expects type 'signed size_t', but argument 5 has type 'ssize_t'
```
Ref: https://github.com/curl/curl/actions/runs/13533841306/job/37821720902?pr=16492#step:9:20
Also: simplify Windows CE job configuration in GHA/windows.
Follow-up to 2a292c3984#15975Closes#16492
We decided not to show warnings for this issue.
Also this:
```
CMake Warning at CMakeLists.txt:783 (message):
OpenSSL does not support TLS 1.3.
```
as seen in #16483 with CMake 3.12.4, `OPENSSL_VERSION` is empty.
Perhaps detection isn't reliable? I haven't seen this so far with
mainline OpenSSL. An `if(OPENSSL_VERSION AND ...)` can fix it, if we
opt for this warning in the future.
Follow-up to #16122
This reverts commit 34c1c653fc#16120Closes#16485
- Show verbose message if the CA native import option is set but
the wolfSSL build does not support it.
wolfSSL has to be built with WOLFSSL_SYS_CA_CERTS to import native
CA certificates and that may not be common.
Closes https://github.com/curl/curl/pull/16417
To help applications do the right thing easier, change some enum values
into defines with L suffixes so that they get the corect type (long)
easier when used with curl_easy_setopt(). This also fixes a few of our
own libtests.
To reduce the risk that this change breaks the compile for any existing
users, the previously provided enums are still provided, but the values
to use are not defined by the enums.
This change "magically" fixes a few RTSP test failures we have had on
64-bit platforms because those options were not see using longs
properly.
Closes#16482
