Commit Graph

33088 Commits

Author SHA1 Message Date
Daniel Stenberg
e7112a726b
CI: verify libcurl function SYNPOSIS sections
With the .github/scripits/verify-synopsis.pl script

Closes #12402
2023-11-24 20:35:50 +01:00
Daniel Stenberg
ad1dfc594f
docs/libcurl: SYNSOPSIS cleanup
- use the correct include file
- make sure they are declared as in the header file
- fix minor nroff syntax mistakes (missing .fi)

These are verified by verify-synopsis.pl, which extracts the SYNPOSIS
code and runs it through gcc.

Closes #12402
2023-11-24 20:35:38 +01:00
Daniel Stenberg
3c30c165a8
sendf: fix comment typo 2023-11-24 14:36:25 +01:00
Daniel Stenberg
f27b8dba73
fopen: allocate the dir after fopen
Move the allocation of the directory name down to after the fopen() call
to allow that shortcut code path to avoid a superfluous malloc+free
cycle.

Follow-up to 73b65e94f3

Closes #12398
2023-11-24 13:24:08 +01:00
Stefan Eissing
5b65e7d1ae
transfer: cleanup done+excess handling
- add `SingleRequest->download_done` as indicator that
  all download bytes have been received
- remove `stop_reading` bool from readwrite functions
- move excess body handling into client download writer

Closes #12371
2023-11-24 13:22:07 +01:00
Daniel Stenberg
03cb1ff4d6
fopen: create new file using old file's mode
Because the function renames the temp file to the target name as a last
step, if the file was previously owned by a different user, not ORing
the old mode could otherwise end up creating a file that was no longer
readable by the original owner after save.

Reported-by: Loïc Yhuel
Fixes #12299
Closes #12395
2023-11-23 22:30:42 +01:00
Daniel Stenberg
242e6d019f
test1476: require proxy
Follow-up from 323df4261c

Closes #12394
2023-11-23 19:56:12 +01:00
Daniel Stenberg
73b65e94f3
fopen: create short(er) temporary file name
Only using random letters in the name plus a ".tmp" extension. Not by
appending characters to the final file name.

Reported-by: Maksymilian Arciemowicz

Closes #12388
2023-11-23 15:46:34 +01:00
Stefan Eissing
2692d41c23
tests: git ignore generated second-hsts.txt file
File is generated in test lib1900

Follow-up to 7cb03229d9

Closes #12393
2023-11-23 15:35:58 +01:00
Viktor Szakats
294194d689
openssl: enable infof_certstack for 1.1 and LibreSSL 3.6
Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to
OpenSSL 1.1.x, and LibreSSL 3.6 or upper.

With the caveat, that "group name" and "type name" are missing from
the log output with these TLS backends.

Follow-up to b6e6d4ff8f #12030

Reviewed-by: Daniel Stenberg
Closes #12385
2023-11-23 12:41:48 +00:00
Daniel Stenberg
fafefdf9dc
urldata: fix typo in comment 2023-11-23 13:07:53 +01:00
Daniel Stenberg
481755955b
CI: codespell
The list of words to ignore is in the file
.github/scripts/codespell-ignore.txt

Closes #12390
2023-11-23 13:07:26 +01:00
Daniel Stenberg
0510e8b58c
lib: fix comment typos
Five separate ones, found by codespell

Closes #12390
2023-11-23 12:35:59 +01:00
Daniel Stenberg
323df4261c
test1476: verify cookie PSL mixed case 2023-11-23 10:09:01 +01:00
Daniel Stenberg
2b0994c29a
cookie: lowercase the domain names before PSL checks
Reported-by: Harry Sintonen

Closes #12387
2023-11-23 10:08:56 +01:00
Viktor Szakats
006977859d
openssl: fix building with v3 no-deprecated + add CI test
- build quictls with `no-deprecated` in CI to have test coverage for
  this OpenSSL 3 configuration.

- don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`.
  The caller code is meant for OpenSSL 3, while these two functions were
  only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3
  if built with option `no-deprecated`, causing build errors:
  ```
  vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_algorithms'; ISO C99 and later do not   support implicit function declarations [-Wimplicit-function-declaration]
  vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_digests'; ISO C99 and later do not   support implicit function declarations [-Wimplicit-function-declaration]
  ```
  Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?fullLog=true#L7667

  Regression from b6e6d4ff8f #12030
  Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669
  Reviewed-by: Alex Bozarth

- vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with
  `no-deprecated` quictls 3 builds.
  Do it by moving an existing solution for this from `vtls/openssl.c`
  to `vtls/openssl.h` and adjusting caller code.
  ```
  vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_get_peer_certificate'; did you mean   'SSL_get1_peer_certificate'? [-Wimplicit-function-declaration]
  ```
  Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#step:24:1178

- curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and
  `-Wunused-function` when trying to build curl with NTLM enabled but
  without the necessary TLS backend (with DES) support.

Closes #12384
2023-11-23 02:03:02 +00:00
Viktor Szakats
2c4c780472
curl.h: delete Symbian OS references
curl deprecated Symbian OS in 3d64031fa7
via #5989. Delete references to it from public headers, because there
is no fresh release to use those headers with.

Reviewed-by: Dan Fandrich
Reviewed-by: Jay Satiro
Closes #12378
2023-11-22 15:43:10 +00:00
Viktor Szakats
e9a7d4a1c8
windows: use built-in _WIN32 macro to detect Windows
Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Closes #12376
2023-11-22 15:42:25 +00:00
Stefan Eissing
1c37d472a0
url: ConnectionExists revisited
- have common pattern of `if not match, continue`
- revert pages long if()s to return early
- move dead connection check to later since it may
  be relatively expensive
- check multiuse also when NOT building with NGHTTP2
- for MULTIUSE bundles, verify that the inspected
  connection indeed supports multiplexing when in use
  (bundles may contain a mix of connection, afaict)

Closes #12373
2023-11-22 15:10:58 +01:00
Daniel Stenberg
9a8bb20d48
CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range
... or use the default value.

Also clarify the documentation language somewhat.

Closes #12382
2023-11-22 15:01:04 +01:00
Daniel Stenberg
2d06eebf28
urldata: make maxconnects a 32 bit value
"2^32 idle connections ought to be enough for anybody"

Closes #12375
2023-11-22 13:27:38 +01:00
Daniel Stenberg
14612e5fbe
FEATURES: update the URL phrasing
The URL is length limited since a while back so "no limit" simply is not
true anymore. Mention the URL RFC standard used instead.

Closes #12383
2023-11-22 13:26:51 +01:00
Daniel Stenberg
cfdcdc8d2f
wolfssh: remove redundant static prototypes
vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-Werror=redundant-decls]

Closes #12381
2023-11-22 09:39:08 +01:00
Daniel Stenberg
92efb39c00
setopt: remove superfluous use of ternary expressions
Closes #12374
2023-11-22 07:44:22 +01:00
Daniel Stenberg
7f766637aa
mime: store "form escape" as a single bit
Closes #12374
2023-11-22 07:44:18 +01:00
Daniel Stenberg
49c413cce2
setopt: check CURLOPT_TFTP_BLKSIZE range on set
... instead of later when the transfer is about to happen.

Closes #12374
2023-11-22 07:44:05 +01:00
Viktor Szakats
84338c4de2
build: add more picky warnings and fix them
Enable more picky compiler warnings. I've found these options in the
nghttp3 project when implementing the CMake quick picky warning
functionality for it [1].

`-Wunused-macros` was too noisy to keep around, but fixed a few issues
it revealed while testing.

- autotools: reflect the more precisely-versioned clang warnings.
  Follow-up to 033f8e2a08 #12324
- autotools: sync between clang and gcc the way we set `no-multichar`.
- autotools: avoid setting `-Wstrict-aliasing=3` twice.
- autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2].
  It triggers in libtool-generated stub code.

- lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch.

- lib/curl_setup.h: delete duplicate declaration for `fileno`.
  Added in initial commit ae1912cb0d
  (1999-12-29). This suggests this may not be needed anymore, but if
  it does, we may restore this for those specific (non-Windows) systems.
- lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since
  c1d6fe2aaa.
- lib: delete unused macro `isxdigit_ascii` since
  f65f750742.
- lib/mqtt: delete unused macro `MQTT_HEADER_LEN`.
- lib/multi: delete unused macro `SH_READ`/`SH_WRITE`.
- lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN`
  macro.
- lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`.
- lib/rand: fix `-Wunreachable-code` and related fallouts [3].
- lib/setopt: fix `-Wunreachable-code-break`.
- lib/system_win32 and lib/timeval: fix double declarations for
  `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4].
- lib/warnless: fix double declarations in CMake UNITY mode [5].
  This was due to force-disabling the header guard of `warnless.h` to
  to reapply it to source code coming after `warnless.c` in UNITY
  builds. This reapplied declarations too, causing the warnings.
  Solved by adding a header guard for the lines that actually need
  to be reapplied.
- lib/vauth/digest: fix `-Wunreachable-code-break` [6].
- lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant
  block.
- lib/vtls/sectransp: fix `-Wunreachable-code-break` [7].
- lib/vtls/sectransp: suppress `-Wunreachable-code`.
  Detected in `else` branches of dynamic feature checks, with results
  known at compile-time, e.g.
  ```c
  if(SecCertificateCopySubjectSummary)  /* -> true */
  ```
  Likely fixable as a separate micro-project, but given SecureTransport
  is deprecated anyway, let's just silence these locally.
- src/tool_help: delete duplicate declaration for `helptext`.
- src/tool_xattr: fix `-Wunreachable-code`.
- tests: delete duplicate declaration for `unitfail` [8].
- tests: delete duplicate declaration for `strncasecompare`.
- tests/libtest: delete duplicate declaration for `gethostname`.
  Originally added in 687df5c8c3
  (2010-08-02).
  Got complicated later: c49e9683b8
  If there are still systems around with warnings, we may restore the
  prototype, but limited for those systems.
- tests/lib2305: delete duplicate declaration for
  `libtest_debug_config`.
- tests/h2-download: fix `-Wunreachable-code-break`.

[1] a70edb08e9/cmake/PickyWarningsC.cmake
[2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjauiqla5fj45?fullLog=true#L1675
[3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=12331#step:7:72
    https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=12331#step:7:109
[4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L204
[5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L218
[6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=12331#step:7:290
[7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=12331#step:9:1193
[8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=12331#step:33:1870

Closes #12331
2023-11-21 16:35:42 +00:00
Daniel Stenberg
891f1041d6
transfer: avoid unreachable expression
If curl_off_t and size_t have the same size (which is common on modern
64 bit systems), a condition cannot occur which Coverity pointed
out. Avoid the warning by having the code conditionally only used if
curl_off_t actually is larger.

Follow-up to 1cd2f0072f

Closes #12370
2023-11-21 13:51:22 +01:00
Stefan Eissing
1cd2f0072f
transfer: readwrite improvements
- changed header/chunk/handler->readwrite prototypes to accept `buf`,
  `blen` and a `pconsumed` pointer. They now get the buffer to work on
  and report back how many bytes they consumed
- eliminated `k->str` in SingleRequest
- improved excess data handling to properly calculate with any body data
  left in the headerb buffer
- eliminated `k->badheader` enum to only be a bool

Closes #12283
2023-11-21 08:03:45 +01:00
Daniel Stenberg
d9e7643ced
RELEASE-NOTES: synced 2023-11-21 08:01:42 +01:00
Jiří Hruška
6a095da1f3
transfer: avoid calling the read callback again after EOF
Regression since 7f43f3dc59 (7.84.0)

Bug: https://curl.se/mail/lib-2023-11/0017.html

Closes #12363
2023-11-21 07:58:53 +01:00
Daniel Stenberg
db5bf23103
doh: provide better return code for responses w/o addresses
Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the
response did not contain any addresses. Now it more accurately returns
CURLE_COULDNT_RESOLVE_HOST.

Reported-by: lRoccoon on github

Fixes #12365
Closes #12366
2023-11-21 07:57:23 +01:00
Stefan Eissing
b06b6216a3
HTTP/2, HTTP/3: handle detach of onoing transfers
- refs #12356 where a UAF is reported when closing a connection
  with a stream whose easy handle was cleaned up already
- handle DETACH events same as DONE events in h2/h3 filters

Fixes #12356
Reported-by: Paweł Wegner
Closes #12364
2023-11-21 07:55:16 +01:00
Viktor Szakats
413a0fedd0
autotools: stop setting -std=gnu89 with --enable-warnings
Do not alter the C standard when building with `--enable-warnings` when
building with gcc.

On one hand this alters warning results compared to a default build.
On the other, it may produce different binaries, which is unexpected.

Also fix new warnings that appeared after removing `-std=gnu89`:

- include: fix public curl headers to use the correct printf mask for
  `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64
  and Visual Studio 2013 and newer. This fixes the printf mask warnings
  in examples and tests. E.g. [1]

- conncache: fix printf format string [2].

- http2: fix potential null pointer dereference [3].
  (seen on Slackware with gcc 11.)

- libssh: fix printf format string in SFTP code [4].
  Also make MSVC builds compatible with old CRT versions.

- libssh2: fix printf format string in SFTP code for MSVC.
  Applying the same fix as for libssh above.

- unit1395: fix `argument is null` and related issues [5]:
  - stop calling `strcmp()` with NULL to avoid undefined behaviour.
  - fix checking results if some of them were NULL.
  - do not pass NULL to printf `%s`.

- ci: keep a build job with `-std=gnu89` to continue testing for
  C89-compliance. We can apply this to other gcc jobs as needed.
  Ref: b23ce2cee7 (2022-09-23) #9542

[1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=logs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b
[2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=12346#step:6:67
[3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=12346#step:30:214
[4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=12346#step:29:895
[5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=12346#step:33:1689

Closes #12346
2023-11-20 22:28:17 +00:00
Viktor Szakats
95231921b2
autotools: fix/improve gcc and Apple clang version detection
- Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu
  may return `n-win32` (also with `-dumpfullversion`). Causing these
  errors and failing to enable picky warnings:
  ```
  ../configure: line 23845: test: : integer expression expected
  ```
  Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/17007893718#step:5:143

  Fix that by stripping any dash-suffix and handling a dotless (major-only)
  version number by assuming `.0` in that case.

  `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0`
  Ref: https://github.com/mamedev/mame/pull/9767

- fix Apple clang version detection for releases between
  'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the
  version was under-detected as 3.7 llvm/clang equivalent.

- fix Apple clang version detection for 'Apple clang version 11.0.0'
  and newer where the Apple clang version was detected, instead of its
  llvm/clang equivalent.

- display detected clang/gcc/icc compiler version.

Via libssh2:
- 00a3b88c51
  https://github.com/libssh2/libssh2/pull/1187
- 89ccc83c7d
  https://github.com/libssh2/libssh2/pull/1232

Closes #12362
2023-11-20 22:28:17 +00:00
Viktor Szakats
3215a7916e
autotools: delete LCC compiler support bits
Follow-up to fd7ef00f43 #12222

Closes #12357
2023-11-20 22:28:17 +00:00
Viktor Szakats
33493db2af
cmake: add test for DISABLE options, add CURL_DISABLE_HEADERS_API
- tests: verify CMake `DISABLE` options.

  Make an exception for 2 CMake-only ones, and one more that's
  using a different naming scheme, also in autotools and source.

- cmake: add support for `CURL_DISABLE_HEADERS_API`.

Suggested-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641

Closes #12353
2023-11-20 22:28:17 +00:00
Jacob Hoffman-Andrews
b9b50f3193
hyper: temporarily remove HTTP/2 support
The current design of the Hyper integration requires rebuilding the
Hyper clientconn for each request. However, building the clientconn
requires resending the HTTP/2 connection preface, which is incorrect
from a protocol perspective. That in turn causes servers to send GOAWAY
frames, effectively degrading performance to "no connection reuse" in
the best case. It may also be triggering some bugs where requests get
dropped entirely and reconnects take too long.

This doesn't rule out HTTP/2 support with Hyper, but it may take a
redesign of the Hyper integration in order to make things work.

Closes #12191
2023-11-20 11:01:48 +01:00
Jay Satiro
9ac6023d36 schannel: fix unused variable warning
Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846
Reported-by: Viktor Szakats

Closes https://github.com/curl/curl/pull/12361
2023-11-20 02:55:50 -05:00
Daniel Stenberg
b2d8f3f7f9
url: find scheme with a "perfect hash"
Instead of a loop to scan over the potentially 30+ scheme names, this
uses a "perfect hash" table. This works fine because the set of schemes
is known and cannot change in a build. The hash algorithm and table size
is made to only make a single scheme index per table entry.

The perfect hash is generated by a separate tool (scripts/schemetable.c)

Closes #12347
2023-11-19 13:59:24 +01:00
Daniel Stenberg
d6447b1bd2
scripts: add schemetable.c
This tool generates a scheme-matching table.

It iterates over a number of different initial and shift values in order
to find the hash algorithm that needs the smallest possible table.

The generated hash function, table and table size then needs to be used
by the url.c:Curl_getn_scheme_handler() function.
2023-11-19 13:57:31 +01:00
Stefan Eissing
fa714830e9
vtls/vquic, keep peer name information together
- add `struct ssl_peer` to keep hostname, dispname and sni
  for a filter
- allocate `sni` for use in VTLS backend
- eliminate `Curl_ssl_snihost()` and its use of the download buffer
- use ssl_peer in SSL and QUIC filters

Closes #12349
2023-11-19 13:55:22 +01:00
Viktor Szakats
a9fd0d0083
build: always revert #pragma GCC diagnostic after use
Before this patch some source files were overriding gcc warning options,
but without restoring them at the end of the file. In CMake UNITY builds
these options spilled over to the remainder of the source code,
effecitvely disabling them for a larger portion of the codebase than
intended.

`#pragma clang diagnostic` didn't have such issue in the codebase.

Reviewed-by: Marcel Raad
Closes #12352
2023-11-18 11:16:32 +00:00
Viktor Szakats
ab50027ddf
tidy-up: casing typos, delete unused Windows version aliases
- cmake: fix casing of `UnixSockets` to match the rest of the codebase.

- curl-compilers.m4: fix casing in a comment.

- setup-win32: delete unused Windows version constant aliases.

Reviewed-by: Marcel Raad
Closes #12351
2023-11-18 11:16:32 +00:00
Viktor Szakats
7f33848c98
keylog: disable if unused
Fully disable keylog code if there is no TLS or QUIC subsystem using it.

Closes #12350
2023-11-18 11:16:31 +00:00
Viktor Szakats
20bb363f25
cmake: add CURL_DISABLE_BINDLOCAL option
To match similar autotools option.

Default is `ON`.

Reviewed-by: Daniel Stenberg
Closes #12345
2023-11-17 11:26:40 +00:00
Viktor Szakats
a5daac6009
url: fix -Wzero-length-array with no protocols
Fixes:
```
./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [-Wc2x-extensions]
  178 | static const struct Curl_handler * const protocols[] = {
      |                                                        ^
./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length-array]
```

Closes #12344
2023-11-17 11:26:40 +00:00
Viktor Szakats
47c00fce4e
url: fix builds with CURL_DISABLE_HTTP
Fixes:
```
./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState'
  456 |   Curl_mime_cleanpart(data->state.formp);
      |                       ~~~~~~~~~~~ ^
```

Regression from 74b87a8af1 #11682

Closes #12343
2023-11-17 11:26:40 +00:00
Viktor Szakats
4c1ef6d7bf
http: fix -Wunused-parameter with no auth and no proxy
```
lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter]
                    bool proxy)
                         ^
```

Reviewed-by: Marcel Raad
Closes #12338
2023-11-17 11:26:40 +00:00
Daniel Stenberg
f996d4aad4
TODO: Some TLS options are not offered for HTTPS proxies
Closes #12286
Closes #12342
2023-11-16 23:41:21 +01:00