Now using 'sort' for sorting the names. This has the small side-effect
that it sorts slightly different than the previously used sort function
(emacs).
I think this is a better sort and over all it makes it more convenient
to use the script as it removes a manual step.
Closes#16448
Enable TLS Early Data for wolfSSL:
- merge WOLFSSL_CTX and WOLFSSL setup from ngtcp2 with the general
implemenation in wolfssl.c
- enable for QUIC via ngtcp2
- give Curl_vquic_tls_init() a `struct alpn_spec` like used for the TCP
case. Adapt gnutls and other users.
- enable pytest test cases for early data with wolfSSL
and while this messes up wolfssl.c anyway, do
- rename all struct/functions with prefix 'wolfssl_' to 'wssl_' to not
pollute that name prefix
- rename `ctx/handle` to `ssl_ctx/ssl`, as used in openssl case
Closes#16167
This now insists on using a server auth option unless --insecure is
provided. As an added bonus, it now also only checks for the knownhosts
file once (if found).
Ref: #16197Closes#16205
- make it only accept version 1.0, as that is the version curl supports
- convert the parser to use strparse
- the status code max is now 999, but it does allow != 3 digits
Closes#16435
Rework the event based handling of transfers and connections to
be "localized" into a single source file with clearer dependencies.
- add multi_ev.c and multi_ev.h
- add docs/internal/MULTI-EV.md to explain the overall workings
- only do event handling book keeping when the socket callback
is set
- add handling for "connection only" event tracking, when internal
easy handles are used that are not really tied to a connection.
Used in connection pool.
- remove transfer member "last_poll" and connections "shutdown_poll"
and keep all that internal to multi_ev.c
- add CURL_TRC_M() for tracing of "multi" related things, including
event handling and connection pool operations. Add new trace
feature "multi" for trace config.
multi traces will show exactly what is going on in regard to
event handling.
- multi: trace transfers "mstate" in every CURL_TRC_M() call
- make internal trace buffer 2048 bytes and end the silliness
with +n here -m there. Adjust test 1652 expectations of resulting
length and input edge cases.
- add trace feature "lib-ids" to perfix libcurl traces with transfer
and connection ids. Useful for debugging libcurl applications.
Closes#16308
Verified in test 455 and 487.
If the provided string cannot be base64-decoded, it will instead use
"[64dec-fail]" (without the quotes).
Documented
Ref: #16288Closes#16330
When running curl event based, connect attempts stalled as the 'done'
check was using the wrong state in gnutls.
Add event based pytest runs to all http3 jobs and the openssl and
mbedtls ones on linux.
Closes#16423
It has the same name as the one used in asyn-thread, but for a slightly
different purpose. This not only caused unity build problems, but would
also be confusing and error-prone.
Stop compilation with a clear error when build with versions older than
decribed in `docs/INTERNALS.md` and `docs/HTTPSRR.md` docs.
Also drop an unused macro and another one that's no longer used with
the required minimum c-ares version.
Closes#16407
Silence compiler warnings (200 of them across the main CI workflows):
```
warning #2193: null argument provided for parameter marked with attribute "nonnull"
warning: Null pointer passed to 1st parameter expecting 'nonnull' [core.NonNullParamChecker]
warning: Null pointer passed to 2nd parameter expecting 'nonnull' [core.NonNullParamChecker]
warning: argument 1 null where non-null expected [-Wnonnull]
warning: argument 2 null where non-null expected [-Wnonnull]
warning: comparison of unsigned expression in '< 0' is always false [-Wtype-limits]
warning: null argument where non-null required (argument 1) [-Wnonnull]
```
Also drop `if ... can be linked` feature checks that were identical to
`if ... is compilable` checks, for:
`closesocket`, `ioctlsocket`, `socket`, `freeaddrinfo`, `getaddrinfo`,
`gethostname`, `getpeername`, `getsockname`,
`CloseSocket` (AmigaOS), `IoctlSocket` (AmigaOS).
Another option is to really do the link checks. But, if they weren't
missed so far, it seems safer to drop than risk a detection failure,
as was the case with AmigaOS functions while working on this PR.
There remain 22 `-Wnonnull` warnings in `gethostbyname_r()`,
`getpeername()` `getsockname()`. Most of the rest is necessary for
detection, or originate from autotools and CMake detection code
templates. Some still fixable, like duplicate libs.
Follow-up to ca2f49ded0#16287Closes#16377
Make it possible to build curl for Windows CE using the CeGCC toolchain.
With both CMake and autotools, including tests and examples, also in CI.
The build configuration is the default one with Schannel enabled. No
3rd-party dependencies have been tested.
Also revive old code to make Schannel build with Windows CE, including
certificate verification.
Builds have been throughougly tested. But, I've made no functional tests
for this PR. Some parts (esp. file operations, like truncate and seek)
are stubbed out and likely broken as a result. Test servers build, but
they do not work on Windows CE. This patch substitutes `fstat()` calls
with `stat()`, which operate on filenames, not file handles. This may or
may not work and/or may not be secure.
About CeGCC: I used the latest available macOS binary build v0.59.1
r1397 from 2009, in native `mingw32ce` build mode. CeGCC is in effect
MinGW + GCC 4.4.0 + old/classic-mingw Windows headers. It targets
Windows CE v3.0 according to its `_WIN32_WCE` value. It means this PR
restores portions of old/classic-mingw support. It makes the Windows CE
codepath compatible with GCC 4.4.0. It also adds workaround for CMake,
which cannot identify and configure this toolchain out of the box.
Notes:
- CMake doesn't recognize CeGCC/mingw32ce, necessitating tricks as seen
with Amiga and MS-DOS.
- CMake doesn't set `MINGW` for mingw32ce. Set it and `MINGW32CE`
manually as a helper variable, in addition to `WINCE` which CMake sets
based on `CMAKE_SYSTEM_NAME`.
- CMake fails to create an implib for `libcurl.dll`, due to not
recognizing the platform as a Windowsy one. This patch adds the
necessary workaround to make it work.
- headers shipping with CeGCC miss some things curl needs for Schannel
support. Fixed by restoring and renovating code previously deleted
old-mingw code.
- it's sometime non-trivial to figure out if a fallout is WinCE,
mingw32ce, old-mingw, or GCC version-specific.
- WinCE is always Unicode. With exceptions: no `wmain`,
`GetProcAddress()`.
- `_fileno()` is said to convert from `FILE *` to `void *` which is
a Win32 file `HANDLE`. (This patch doesn't use this, but with further
effort it probably could be.)
https://stackoverflow.com/questions/3989545/how-do-i-get-the-file-handle-from-the-fopen-file-structure
- WinCE has no signals, current directory, stdio/CRT file handles, no
`_get_osfhandle()`, no `errno`, no `errno.h`. Some of this stuff is
standard C89, yet missing from this platform. Microsoft expects
Windows CE apps to use Win32 file API and `FILE *` exclusively.
- revived CeGCC here (not tested for this PR):
https://building.enlyze.com/posts/a-new-windows-ce-x86-compiler-in-2024/
On `UNDER_CE` vs. `_WIN32_WCE`: (This patch settled on `UNDER_CE`)
- A custom VS2008 WinCE toolchain does not set any of these.
The compiler binaries don't contain these strings, and has no compiler
option for targeting WinCE, hinting that a vanilla toolchain isn't
setting any of them either.
- `UNDER_CE` is automatically defined by the CeGCC compiler.
https://cegcc.sourceforge.net/docs/details.html
- `UNDER_CE` is similar to `_WIN32`, except it's not set automatically
by all compilers. It's not supposed to have any value, like a version.
(Though e.g. OpenSSL sets it to a version)
- `_WIN32_WCE` is the CE counterpart of the non-CE `_WIN32_WINNT` macro.
That does return the targeted Windows CE version.
- `_WIN32_WCE` is not defined by compilers, and relies on a header
setting it to a default, or the build to set it to the desired target
version. This is also how `_WIN32_WINNT` works.
- `_WIN32_WCE` default is set by `windef.h` in CeGCC.
- `_WIN32_WCE` isn't set to a default by MSVC Windows CE headers (the
ones I checked at least).
- CMake sets `_WIN32_WCE=<ver>`, `UNDER_CE`, `WINCE` for MSVC WinCE.
- `_WIN32_WCE` seems more popular in other projects, including CeGCC
itself. `zlib` is a notable exception amongst curl dependencies,
which uses `UNDER_CE`.
- Since `_WIN32_WCE` needs "certain" headers to have it defined, it's
undefined depending on headers included beforehand.
- `curl/curl.h` re-uses `_WIN32_WCE`'s as a self-guard, relying on
its not-(necessarily)-defined-by-default property:
25b445e479/include/curl/curl.h (L77)
Toolchain downloads:
- Windows:
https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_cygwin1.7_r1399.tar.bz2
- macOS Intel:
https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_snowleopard_r1397.tar.bz2Closes#15975
After fixing support for x32, unlock eventfd support for all CPUs.
Before this patch, it was explicitly limited to 64-bit ones.
You can disable eventfs manually on systems where it's auto-detected:
- cmake: `-DHAVE_EVENTFD=0`
- configure: `export ac_cv_func_eventfd=0`
Ref: c2aa504ab9#16239Closes#16277
curl builds tests with CMake when explicitly building the `testdeps`
target. It's not built by default. It seems overkill to have
a curl-specific variant of this (over CMake's `BUILD_TESTING`)
to disable generating this target.
Its history also doesn't make it obvious why this was necessary,
and there was a long debate how to do it, by the time the original
submitter abandoned CMake. The option also remained uninitialized
and thus undocumented.
Let me know if I missed something.
Ref: #6036
Ref: 3a1e798009#6072Closes#16134
curl detects Vista as part of its global initialization. Use that result
instead of detecting it again in Schannel, to save some cycles and API
calls.
Follow-up to 46e97b10ba#16400Closes#16408
- Do not set x509_store_setup until after all CA certificate imports.
- If CA blob import fails then return error unconditionally.
This is to sync the behavior with OpenSSL and other backends.
Prior to this change, setting CA native and/or CA blob were treated as
mutually exclusive from setting CA file and/or CA path.
CA blob import still overrides CA file import as documented.
Closes https://github.com/curl/curl/pull/16391
Avoids having to use the correct index into the line. Avoids repeated
use of is_valid_auth_separator.
Require that the following letter is not an alnum instead of checking
explicitly for ch == '\0' || ch == ',' || ISSPACE(ch). After all, the
point is to not erroneously match another auth string using the same
prefix.
Follow-up to b75620b9a0Closes#16406
In curl 8.12 I tried to improve the logic on how we handle connections
that "upgrade" to TLS later, e.g. with a STARTTLS. I found the existing
code hard to read in this regard. But of course, the "improvements" blew
up in my face.
We fixed issues with imap, opo3, smtp in 8.12.1, but ftp was no longer
reusing existing, upgraded control connections as before. This PR adds
checks in our pytest FTP tests that verify reuse is happening as
intended.
I rewrote the logic in url.c again, so that the new test checks now pass.
Reported-by: Zenju on github
Fixes#16384Closes#16392
Adds a "cw-pause" client writer in the PROTOCOL phase that buffers
output when the client paused the transfer. This prevents content
decoding from blowing the buffer in the "cw-out" writer.
Added test_02_35 that downloads 2 100MB gzip bombs in parallel and
pauses after 1MB of decoded 0's.
This is a solution to issue #16280, with some limitations:
- cw-out still needs buffering of its own, since it can be paused
"in the middle" of a write that started with some KB of gzipped
zeros and exploded into several MB of calls to cw-out.
- cw-pause will then start buffering on its own *after* the write
that caused the pause. cw-pause has no buffer limits, but the
data it buffers is still content-encoded.
Protocols like http/1.1 stop receiving, h2/h3 have window sizes,
so the cw-pause buffer should not grow out of control, at least
for these protocols.
- the current limit on cw-out's buffer is ~75MB (for whatever
historical reason). A potential content-encoding that blows 16KB
(the common h2 chunk size) into > 75MB would still blow the buffer,
making the transfer fail. A gzip of 0's makes 16KB into ~16MB, so
that still works.
A better solution would be to allow CURLE_AGAIN handling in the client
writer chain and make all content encoders handle that. This would stop
explosion of encoding on a pause right away. But this is a large change
of the deocoder operations.
Reported-by: lf- on github
Fixes#16280Closes#16296
Add a 'wanted' major HTTP version bitmask next to the 'allowed' bitmask
in HTTP version negotiation. This will try connections as specified in
'wanted', but enabled Alt-Svc and HTTPS-RR to redirect to other major
HTTP versions, if those are 'allowed'.
Changes libcurl internal default to `CURL_HTTP_VERSION_NONE` and removes
the code in curl that sets `CURL_HTTP_VERSION_2TLS` if the command line
does not say anything else.
Closes#16117
Remove `blocking` argument from cfilter's connect method.
Implement blocking behaviour in Curl_conn_connect() instead for all
filter chains.
Update filters implementations. Several of which did never use the
paramter (QUIC for example). Simplifies connect handling in TLS filters
that no longer need to loop
Fixed a blocking connect call in FTP when waiting on a socket accept()
which only worked because the filter did not implement it.
Closes#16397
curl attempts to clear some flags to hide them from snooping neighbors
(on platforms where it works). For example the credentials provided with
-u. Previously it would only do that if there was a space between the
option and the credentials as in "-u joe:s3cr3t" but not when done
without a separating space as in "-ujoe:s3cr3t".
This addresses that previous shortcoming.
Reported-by: kayrus on github
Fixes#16396Closes#16401
