Commit graph

2315 commits

Author SHA1 Message Date
Andrew Clayton
375556f9aa Don't conflate the error variable in nxt_kqueue_poll().
In nxt_kqueue_poll() error is declared as a nxt_bool_t aka unsigned int
(on x86-64 anyway).

It is used both as a boolean and as the return storage for a bitwise AND
operation.

This has potential to go awry.

If nxt_bool_t was changed to be a u8 then we would have the following
issue

gcc12 -c -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-prototypes -Werror -g -O2 -I src -I build     -I/usr/local/include  -o build/src/nxt_kqueue_engine.o  -MMD -MF build/src/nxt_kqueue_engine.dep -MT build/src/nxt_kqueue_engine.o  src/nxt_kqueue_engine.c
src/nxt_kqueue_engine.c: In function 'nxt_kqueue_poll':
src/nxt_kqueue_engine.c:728:17: error: overflow in conversion from 'int' to 'nxt_bool_t' {aka 'unsigned char'} changes value from '(int)kev->flags & 16384' to '0' [-Werror=overflow]
  728 |         error = (kev->flags & EV_ERROR);
      |                 ^
cc1: all warnings being treated as errors

EV_ERROR has the value 16384, after the AND operation error holds 16384,
however this overflows and wraps around (64 times) exactly to 0.

With nxt_bool_t defined as a u32, we would have a similar issue if
EV_ERROR ever became UINT_MAX + 1 (or a multiple thereof)...

Rather than conflating the use of error, keep error as a boolean (it is
used further down the function) but do the AND operation inside the
if ().

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-24 19:40:16 +01:00
Andrew Clayton
b9177d36e7 Remove a bunch of dead code.
This removes a bunch of unused files that would have been touched by
subsequent commits that switch to using nxt_bool_t (AKA unit6_t) in
structures.

In auto/sources we have

  NXT_LIB_SRC0=" \
      src/nxt_buf_filter.c \
      src/nxt_job_file.c \
      src/nxt_stream_module.c \
      src/nxt_stream_source.c \
      src/nxt_upstream_source.c \
      src/nxt_http_source.c \
      src/nxt_fastcgi_source.c \
      src/nxt_fastcgi_record_parse.c \
  \
      src/nxt_mem_pool_cleanup.h \
      src/nxt_mem_pool_cleanup.c \
  "

None of these seem to actually be used anywhere (other than within
themselves). That variable is _not_ referenced anywhere else.

Also remove the unused related header files: src/nxt_buf_filter.h,
src/nxt_fastcgi_source.h, src/nxt_http_source.h, src/nxt_job_file.h,
src/nxt_stream_source.h and src/nxt_upstream_source.h

Also, these files do not seem to be used, no mention under auto/ or build/

  src/nxt_file_cache.c
  src/nxt_cache.c
  src/nxt_job_file_cache.c

src/nxt_cache.h is #included in src/nxt_main.h, but AFAICT is not
actually used.

With all the above removed

  $ ./configure --openssl --debug --tests && make -j && make -j tests &&
  make libnxt

all builds.

Buildbot passes.

NOTE: You may need to do a 'make clean' before the next build attempt.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-24 19:39:09 +01:00
Alejandro Colomar
6f36a67fc3 Tools: setup-unit: unified repeated code.
Instead of doing the same operation in each subcommand, do it once in
the parent.

Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-04-15 00:29:52 +02:00
Konstantin Pavlov
4f8a9e305d contrib: updated njs to 0.7.12. 2023-04-11 15:48:01 -07:00
Alejandro Colomar
fcff55acb6 HTTP: optimizing $request_line.
Don't reconstruct a new string for the $request_line from the parsed
method, target, and HTTP version, but rather keep a pointer to the
original memory where the request line was received.

This will be necessary for implementing URI rewrites, since we want to
log the original request line, and not one constructed from the
rewritten target.

This implementation changes behavior (only for invalid requests) in the
following way:

Previous behavior was to log as many tokens from the request line as
were parsed validly, thus:

Request              -> access log              ; error log

"GET / HTTP/1.1"     -> "GET / HTTP/1.1"     OK ; =
"GET   / HTTP/1.1"   -> "GET / HTTP/1.1"    [1] ; =
"GET / HTTP/2.1"     -> "GET / HTTP/2.1"     OK ; =
"GET / HTTP/1."      -> "GET / HTTP/1."     [2] ; "GET / HTTP/1. [null]"
"GET / food"         -> "GET / food"        [2] ; "GET / food [null]"
"GET / / HTTP/1.1"   -> "GET / / HTTP/1.1"  [2] ; =
"GET /  / HTTP/1.1"  -> "GET /  / HTTP/1.1" [2] ; =
"GET food HTTP/1.1"  -> "GET"                   ; "GET [null] [null]"
"OPTIONS * HTTP/1.1" -> "OPTIONS"           [3] ; "OPTIONS [null] [null]"
"FOOBAR baz HTTP/1.1"-> "FOOBAR"                ; "FOOBAR [null] [null]"
"FOOBAR / HTTP/1.1"  -> "FOOBAR / HTTP/1.1"     ; =
"get / HTTP/1.1"     -> "-"                     ; " [null] [null]"
""                   -> "-"                     ; " [null] [null]"

This behavior was rather inconsistent.  We have several options to go
forward with this patch:

-  NGINX behavior.

   Log the entire request line, up to '\r' | '\n', even if it was
   invalid.

   This is the most informative alternative.  However, RFC-complying
   requests will probably not send invalid requests.

   This information would be interesting to users where debugging
   requests constructed manually via netcat(1) or a similar tool, or
   maybe for debugging a client, are important.  It might be interesting
   to support this in the future if our users are interested; for now,
   since this approach requires looping over invalid requests twice,
   that's an overhead that we better avoid.

-  Previous Unit behavior

   This is relatively fast (almost as fast as the next alternative, the
   one we chose), but the implementation is ugly, in that we need to
   perform the same operation in many places around the code.

   If we want performance, probably the next alternative is better; if
   we want to be informative, then the first one is better (maybe in
   combination with the third one too).

-  Chosen behavior

   Only logging request lines when the request is valid.  For any
   invalid request, or even unsupported ones, the request line will be
   logged as "-".  Thus:

   Request              -> access log [4]

   "GET / HTTP/1.1"     -> "GET / HTTP/1.1"     OK
   "GET   / HTTP/1.1"   -> "GET   / HTTP/1.1"  [1]
   "GET / HTTP/2.1"     -> "-"                 [3]
   "GET / HTTP/1."      -> "-"
   "GET / food"         -> "-"
   "GET / / HTTP/1.1"   -> "GET / / HTTP/1.1"  [2]
   "GET /  / HTTP/1.1"  -> "GET /  / HTTP/1.1" [2]
   "GET food HTTP/1.1"  -> "-"
   "OPTIONS * HTTP/1.1" -> "-"
   "FOOBAR baz HTTP/1.1"-> "-"
   "FOOBAR / HTTP/1.1"  -> "FOOBAR / HTTP/1.1"
   "get / HTTP/1.1"     -> "-"
   ""                   -> "-"

   This is less informative than previous behavior, but considering how
   inconsistent it was, and that RFC-complying agents will probably not
   send us such requests, we're ready to lose that information in the
   log.  This is of course the fastest and simplest implementation we
   can get.

   We've chosen to implement this alternative in this patch.  Since we
   modified the behavior, this patch also changes the affected tests.

[1]:  Multiple successive spaces as a token delimiter is allowed by the
      RFC, but it is discouraged, and considered a security risk.  It is
      currently supported by Unit, but we will probably drop support for
      it in the future.

[2]:  Unit currently supports spaces in the request-target.  This is
      a violation of the relevant RFC (linked below), and will be fixed
      in the future, and consider those targets as invalid, returning
      a 400 (Bad Request), and thus the log lines with the previous
      inconsistent behavior would be changed.

[3]:  Not yet supported.

[4]:  In the error log, regarding the "log_routes" conditional logging
      of the request line, we only need to log the request line if it
      was valid.  It doesn't make sense to log "" or "-" in case that
      the request was invalid, since this is only useful for
      understanding decisions of the router.  In this case, the access
      log is more appropriate, which shows that the request was invalid,
      and a 400 was returned.  When the request line is valid, it is
      printed in the error log exactly as in the access log.

Link: <https://datatracker.ietf.org/doc/html/rfc9112#section-3>
Suggested-by: Liam Crilly <liam@nginx.com>
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Cc: Timo Stark <t.stark@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Cc: Artem Konev <a.konev@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-04-12 11:50:56 +02:00
Konstantin Pavlov
bfd9a0a30a Docker: fixed a typo. 2023-04-11 15:39:42 -07:00
Andrew Clayton
45c45eaeb4 Add per-application logging.
Currently when running in the foreground, unit application processes
will send stdout to the current TTY and stderr to the unit log file.

That behaviour won't change.

When running as a daemon, unit application processes will send stdout to
/dev/null and stderr to the unit log file.

This commit allows to alter the latter case of unit running as a daemon,
by allowing applications to redirect stdout and/or stderr to specific
log files. This is done via two new application options, 'stdout' &
'stderr', e.g

  "applications": {
      "myapp": {
          ...
          "stdout": "/path/to/log/unit/app/stdout.log",
          "stderr": "/path/to/log/unit/app/stderr.log"
      }
  }

These log files are created by the application processes themselves and
thus the log directories need to be writable by the user (and or group)
of the application processes.

E.g

  $ sudo mkdir -p /path/to/log/unit/app
  $ sudo chown APP_USER /path/to/log/unit/app

These need to be setup before starting unit with the above config.

Currently these log files do not participate in log-file rotation
(SIGUSR1), that may change in a future commit. In the meantime these
logs can be rotated using the traditional copy/truncate method.

NOTE:

You may or may not see stuff printed to stdout as stdout was
traditionally used by CGI applications to communicate with the
webserver.

Closes: <https://github.com/nginx/unit/issues/197>
Closes: <https://github.com/nginx/unit/issues/846>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-11 19:08:32 +01:00
Andrew Clayton
8b8952930c Add nxt_file_stdout().
This is analogous to the nxt_file_stderr() function and will be used in
a subsequent commit.

This function redirects stdout to a given file descriptor.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-11 19:08:12 +01:00
Andrew Clayton
edbc43558d PHP: Make the filter_input() function work.
On GitHub, @jamesRUS52 reported that the PHP filter_input()[0] function
would just return NULL.

To enable this function we need to run the variables through the
sapi_module.input_filter() function when we call
php_register_variable_safe().

In PHP versions prior to 7.0.0, input_filter() takes 'len' as an
unsigned int, while later versions take it as a size_t.

Now, with this commit and the following PHP

  <?php

  var_dump(filter_input(INPUT_SERVER, 'REMOTE_ADDR'));
  var_dump(filter_input(INPUT_SERVER, 'REQUEST_URI'));
  var_dump(filter_input(INPUT_GET, 'get', FILTER_SANITIZE_SPECIAL_CHARS));

  ?>

you get

  $ curl 'http://localhost:8080/854.php?get=foo<>'
  string(3) "::1"
  string(18) "/854.php?get=foo<>"
  string(13) "foo&#60;&#62;"

[0]: <https://www.php.net/manual/en/function.filter-input.php>

Tested-by: <https://github.com/jamesRUS52>
Closes: <https://github.com/nginx/unit/issues/854>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-11 19:08:12 +01:00
Konstantin Pavlov
028e537bef Docker: fixed git references. 2023-03-28 14:45:10 -07:00
Konstantin Pavlov
5d83ee6660 Docker: drop apt-get clean usage.
It's automatic in the Debian and Ubuntu containers: 5cf7949ecf/scripts/debuerreotype-minimizing-config (L85-L109)
2023-04-06 20:43:22 -07:00
Konstantin Pavlov
472c6d0fb0 Docker: explicitely set uid/gid to 999 for unit user.
This allows us to be consistent through possible updates of default
settings used in distributions.  Previous behaviour was uid/gid were
chosen automatically based on what uids/gids are already taken on the
system.
2023-04-06 20:43:22 -07:00
Konstantin Pavlov
6d51672d8d Packages: use groupadd/useradd on Debian-based operating systems.
addgroup/adduser will no longer be installed by default in the
"minbase".  Also, moving to lower-level utilities saves us one runtime
dependency.
2023-04-06 20:43:22 -07:00
Konstantin Pavlov
886aa17e79 Docker: added OCI image-spec labels. 2023-04-06 20:43:22 -07:00
Konstantin Pavlov
09cd3793aa Docker: specified explicit variants of images to use.
This allows us to decide when to move to a newer underlying distribution
version with our pace instead of relying on Docker Hub cadence.
2023-04-06 20:43:22 -07:00
Konstantin Pavlov
6ed5f1654e Docker: dropped a leftover from a multi-stage build. 2023-04-06 20:43:22 -07:00
Konstantin Pavlov
8392f8c902 Docker: check out packaging tags.
This will ensure we're checking out source code that is close to what we
have in binary packages.

While at it, remove the checkout directory when it's no longer needed.
2023-04-10 15:36:48 -07:00
Konstantin Pavlov
31424f409e Docker: added njs support. 2023-03-30 16:17:59 -07:00
Konstantin Pavlov
73c6c8a7f7 Packages: added unitc and setup-unit. 2023-03-30 16:03:41 -07:00
Liam Crilly
c54331fa3d Tools: use control socket and log file from running instance.
If unitd was started with an explicit path then unitc will use that
binary instead of the default PATH to obtain the default control socket
and log file locations.
2023-04-04 11:29:53 +01:00
Andrew Clayton
4852057124 Remove a useless assignment in nxt_mem_zone_alloc_pages().
This was reported by the 'Clang Static Analyzer' as a 'dead nested
assignment'.

We assign prev_size then check if it's != 0 and if true we then set
prev_pages to page_size right shifted by two at the same time setting
prev_size to be right shifted by two (>>=), however page_size is never
used again so no need to set it here.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-03 14:53:04 +01:00
Andrew Clayton
8a9e078e54 Prevent a possible NULL de-reference in nxt_job_create().
We allocate 'job' we then have a check if it's not NULL and do stuff
with it, but then we accessed it outside this check.

Simply return if job is NULL.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-03 14:53:04 +01:00
Andrew Clayton
22993fe41a Remove a useless assignment in nxt_fs_mkdir_all().
This was reported by the 'Clang Static Analyzer' as a 'dead nested
assignment'.

We set end outside the loop but the first time we use it is to assign it
in the loop (not used anywhere else).

Further cleanup could be to reduce the scope of end by moving its
declaration inside the loop.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-04-03 14:53:04 +01:00
Konstantin Pavlov
497b84c18f Packages: Used a stricter check for Amazon Linux 2023.
Previously, findstring matched on amazonlinux2 too, breaking the build on that OS.
2023-03-30 10:51:32 -07:00
Konstantin Pavlov
219b8363a4 Packages: fixed rpm builds after 817968931c58. 2023-03-29 12:42:54 -07:00
Konstantin Pavlov
24243ecab3 Packages: Added Amazon Linux 2023. 2023-03-22 16:55:36 -07:00
Konstantin Pavlov
700ee28bbf Packages: check rpm database for actual provides.
Previously, we required an exact non-virtual package, however it's fine
if some package has a fully-virtual provides for what we need.
2023-03-22 16:55:25 -07:00
Alejandro Colomar
6e16d7ac5b Auto: mirroring installation structure in build tree.
This makes the build tree more organized, which is good for adding new
stuff.  Now, it's useful for example for adding manual pages in man3/,
but it may be useful in the future for example for extending the build
system to run linters (e.g., clang-tidy(1), Clang analyzer, ...) on the
C source code.

Previously, the build tree was quite flat, and looked like this (after
`./configure && make`):

    $ tree -I src build
    build
    ├── Makefile
    ├── autoconf.data
    ├── autoconf.err
    ├── echo
    ├── libnxt.a
    ├── nxt_auto_config.h
    ├── nxt_version.h
    ├── unitd
    └── unitd.8

    1 directory, 9 files

And after this patch, it looks like this:

    $ tree -I src build
    build
    ├── Makefile
    ├── autoconf.data
    ├── autoconf.err
    ├── bin
    │   └── echo
    ├── include
    │   ├── nxt_auto_config.h
    │   └── nxt_version.h
    ├── lib
    │   ├── libnxt.a
    │   └── unit
    │       └── modules
    ├── sbin
    │   └── unitd
    ├── share
    │   └── man
    │       └── man8
    │           └── unitd.8
    └── var
        ├── lib
        │   └── unit
        ├── log
        │   └── unit
        └── run
            └── unit

    17 directories, 9 files

It also solves one issue introduced in
5a37171f73 ("Added default values for pathnames.").  Before that
commit, it was possible to run unitd from the build system
(`./build/unitd`).  Now, since it expects files in a very specific
location, that has been broken.  By having a directory structure that
mirrors the installation, it's possible to trick it to believe it's
installed, and run it from there:

    $ ./configure --prefix=./build
    $ make
    $ ./build/sbin/unitd

Fixes: 5a37171f73 ("Added default values for pathnames.")
Reported-by: Liam Crilly <liam@nginx.com>
Reviewed-by: Konstantin Pavlov <thresh@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Cc: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-03-29 00:41:08 +02:00
Alejandro Colomar
5ba79b9b52 Renamed --libstatedir to --statedir.
In BSD systems, it's usually </var/db> or some other dir under </var>
that is not </var/lib>, so $statedir is a more generic name.  See
hier(7).

Reported-by: Andrei Zeliankou <zelenkov@nginx.com>
Reported-by: Zhidao Hong <z.hong@f5.com>
Reviewed-by: Konstantin Pavlov <thresh@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Cc: Liam Crilly <liam@nginx.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-03-29 00:40:40 +02:00
Konstantin Pavlov
e242b1454d Tests: relaxed jar glob.
We install jars with names like websocket-api-${NXT_JAVA_MODULE}-$NXT_VERSION.jar,
which translates to versioned NXT_JAVA_MODULE in the packaging system, e.g.
websocket-api-java11-1.30.0.jar.
2023-03-27 13:16:47 -07:00
Konstantin Pavlov
565dcbb139 Docker: bumped language versions. 2023-02-13 17:04:24 -08:00
Konstantin Pavlov
2a597c5c7a Docker: limited the waiting time for control socket creation.
While at it, fixed a typo.
2023-02-13 17:04:24 -08:00
Konstantin Pavlov
054518bb36 Docker: made dockerfiles use a single stage build process. 2023-02-13 17:04:24 -08:00
Konstantin Pavlov
eb47174875 Docker: added a target to generate Docker library definition. 2023-02-13 17:04:24 -08:00
Konstantin Pavlov
a8f21079e0 Docker: cleanup unused targets. 2023-02-13 17:04:24 -08:00
Andrei Zeliankou
b24257c8f6 Tests: added tests for the "log_route" option. 2023-03-21 19:20:23 +00:00
Alejandro Colomar
0ebce31c92 HTTP: added route logging.
-  Configuration: added "/config/settings/http/log_route".

   Type: bool
   Default: false

   This adds configurability to the error log.  It allows enabling and
   disabling logs related to how the router performs selection of the
   routes.

-  HTTP: logging request line.

   Log level: [notice]

   The request line is essential to understand which logs correspond to
   which request when reading the logs.

-  HTTP: logging route that's been discarded.

   Log level: [info]

-  HTTP: logging route whose action is selected.

   Log level: [notice]

-  HTTP: logging when "fallback" action is taken.

   Log level: [notice]

Closes: <https://github.com/nginx/unit/issues/758>
Link: <https://github.com/nginx/unit/pull/824>
Link: <https://github.com/nginx/unit/pull/839>
Suggested-by: Timo Stark <t.stark@nginx.com>
Suggested-by: Mark L Wood-Patrick <mwoodpatrick@gmail.com>
Suggested-by: Liam Crilly <liam@nginx.com>
Tested-by: Liam Crilly <liam@nginx.com>
Acked-by: Artem Konev <a.konev@f5.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-03-21 13:02:38 +01:00
Alejandro Colomar
773c341d70 HTTP: rewrote while loop as for loop.
This considerably simplifies the function, and will also help log the
iteration in which we are, which corresponds to the route array element.

Link: <https://github.com/nginx/unit/pull/824>
Link: <https://github.com/nginx/unit/pull/839>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Tested-by: Liam Crilly <liam@nginx.com>
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
2023-03-21 13:02:38 +01:00
Andrew Clayton
c18dd1f65b Default PR_SET_NO_NEW_PRIVS to off.
This prctl(2) option was enabled in commit 0277d8f1 ("Isolation: Fix the
enablement of PR_SET_NO_NEW_PRIVS.") and this was being set by default.

This prctl(2) when enabled renders (amongst other things) the set-UID
and set-GID bits on executables ineffective after an execve(2).

This causes an issue for applications that want to execute the
sendmail(8) binary, this includes the PHP mail() function, which is
usually set-GID.

After some internal discussion it was decided to disable this option by
default.

Closes: <https://github.com/nginx/unit/issues/852>
Fixes: 0277d8f1 ("Isolation: Fix the enablement of PR_SET_NO_NEW_PRIVS.")
Fixes: e2b53e16 ("Added "rootfs" feature.")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-17 04:28:46 +00:00
Andrew Clayton
7d0ceb82c7 Improve an error message regarding Unix domain sockets.
When starting unit, if its Unix domain control socket was already active
you would get an error message like

  2023/03/15 18:07:55 [alert] 53875#8669650 connect(5, unix:/tmp/control.sock) succeed, address already in use

which is confusing in a couple of regards, firstly we have the classic
success/failure message and secondly 'address already in use' is an
actual errno value, EADDRINUSE and we didn't get an error from this
connect(2).

Re-word this error message for greater clarity.

Reported-by: Liam Crilly <liam.crilly@nginx.com>
Cc: Liam Crilly <liam.crilly@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-17 04:28:46 +00:00
Andrew Clayton
2e3e1c7e7b Socket: Remove Unix domain listen sockets upon reconfigure.
Currently when using Unix domain sockets for requests, if unit is
reconfigured then it will fail if it tries to bind(2) again to a Unix
domain socket with something like

  2023/02/25 19:15:50 [alert] 35274#35274 bind(\"unix:/tmp/unit.sock\") failed (98: Address already in use)

When closing such a socket we really need to unlink(2) it. However that
presents a problem in that when running as root, while the main process
runs as root and creates the socket, it's the router process, that runs
as an unprivileged user, e.g nobody, that closes the socket and would
thus remove it, but couldn't due to not having permission, even if the
socket is mode 0666, you need write permissions on the containing
directory to remove a file.

There are several options to solve this, all with varying degrees of
complexity and utility.

  1) Give the user who the router process runs as write permission to
     the directory containing the listen sockets. These can then be
     unlink(2)'d from the router process.

     Simple and would work, but perhaps not the most elegant.

  2) Using capabilities(7). The router process could temporarily attain
     the CAP_DAC_OVERRIDE capability, unlink(7) the socket, then
     relinquish the capability until required again.

     These are Linux specific (other systems may have similar mechanisms
     which would be extra work to support). There is also a, albeit
     small, window where the router process is running with elevated
     privileges.

  3) Have the main process do the unlink(2), it is after all the process
     that created the socket.

     This is what this commit implements.

We create a new port IPC message type of NXT_PORT_MSG_SOCKET_UNLINK,
that is used by the router process to notify the main process about a
Unix domain socket to unlink(2).

Upon doing a reconfigure the router process will call
nxt_router_listen_socket_release() which will close the socket, we
extend this function in the case of non-abstract Unix domain sockets, so
that it will send a message to the main process containing a copy of the
nxt_sockaddr_t structure that will contain the filename of the socket.

In the main process the handler that we have defined,
nxt_main_port_socket_unlink_handler(), for this message type will run
and allow us to look for the socket in question in the listen_sockets
array and remove it and unlink(2) the socket.

This then allows the reconfigure to work if it tries to bind(2) again to
a socket that previously existed.

Link: <https://github.com/nginx/unit/issues/669>
Link: <https://github.com/nginx/unit/pull/735>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-17 04:28:23 +00:00
Andrew Clayton
8f0192bb4c Remove some dormant code from nxt_process_quit().
In nxt_process_quit() there is a loop that iterates over the
task->thread->runtime->listen_sockets array and closes the connections.

This code has been there from the beginning

  $ git log --pretty=oneline -S'if (rt->listen_sockets != NULL)'
  e9e5ddd5a5 Refactor of process management.
  6f2c9acd18 Processes refactoring. The cycle has been renamed to the runtime.
  $ git log --pretty=oneline -S'if (cycle->listen_sockets != NULL) {'
  6f2c9acd18 Processes refactoring. The cycle has been renamed to the runtime.
  16cbf3c076 Initial version.

but never seems to have been used (AFAICT and certainly not recently,
confirmed by code inspection and running pytests with a bunch of
language modules enabled and the code in question was never executed) as
the listen_sockets array has never been populated... until now.

The previous commit now adds Unix domain sockets to this array so that
they can be unlink(2)'d upon exit and reconfiguration.

This has now caused this dormant code to become active as it now tries
to close these sockets (from at least the prototype processes), this
array is inherited via fork by other processes.

The file descriptor for these sockets is set to -1 when they are put
into this array. This then results in close(-1) calls which caused
multiple failures in the pytests such as

  >       assert not alerts, 'alert(s)'
  E       AssertionError: alert(s)
  E       assert not ['2023/03/09 23:26:14 [alert] 137673#137673 socket close(-1) failed (9: Bad file descriptor)']

I think the simplest thing is to just remove this code.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-17 04:03:56 +00:00
Andrew Clayton
ccaad38bc5 Socket: Remove Unix domain listen sockets at shutdown.
If we don't remove the Unix domain listen socket file then when Unit
restarts it get an error like

  2023/02/25 23:10:11 [alert] 36388#36388 bind(\"unix:/tmp/unit.sock\") failed (98: Address already in use)

This patch makes use of the listen_sockets array, that is already
allocated in the main process but never populated, to place the Unix
domain listen sockets into.

At shutdown we can then loop through this array and unlink(2) any Unix
domain sockets found therein.

Closes: <https://github.com/nginx/unit/issues/792>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-17 04:03:56 +00:00
Andrew Clayton
172ceba5b6 Router: More accurately allocate request buffer memory.
In nxt_router_prepare_msg() we create a buffer (nxt_unit_request_t *req)
that gets sent to an application process that contains details about a
client request.

This buffer was always a little larger than needed due to allocating space
for the remote address _and_ port and the local address _and_ port. We
also allocate space for the local port separately.

->{local,remote}->length includes the port number and ':' and also the
'[]' for IPv6. E.g [2001:db8::1]:8080

->{local,remote}->address_length represents the length of the unadorned
IP address. E.g 2001:db8::1

Update the buffer size so that we only allocate what is actually needed.

Suggested-by: Zhidao HONG <z.hong@f5.com>
Cc: Zhidao HONG <z.hong@f5.com>
Reviewed-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-14 17:07:08 +00:00
Andrei Zeliankou
28bdeec614 Tests: added Perl test with many responses using streaming body. 2023-03-14 11:06:09 +00:00
Andrew Clayton
fa81d7a11a Perl: Fix a crash in the language module.
User @bes-internal reported a Perl module crasher on GitHub.

This was due to a Perl application sending back two responses, for each
response we would call down into XS_NGINX__Unit__Sandbox_cb(), the first
time pctx->req would point to a valid nxt_unit_request_info_t, the
second time pctx->req would be NULL.

Add an invalid responses check which covers this case.

Closes: <https://github.com/nginx/unit/issues/841>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-10 21:40:28 +00:00
Andrew Clayton
78e1122a3c Router: Fix allocation of request buffer sent to application.
This fixes an issue reported by @Peter2121 on GitHub.

In nxt_router_prepare_msg() we create a buffer (nxt_unit_request_t *req)
that gets sent to an application process that contains details about a
client request.

The req structure comprises various members with the final member being
an array (specified as a flexible array member, with its actual length
denoted by the req->fields_count member) of nxt_unit_field_t's. These
structures specify the length and offset for the various request headers
name/value pairs which are stored after some request metadata that is
stored immediately after this array of structs as individual nul
terminated strings.

After this we have the body content data (if any). So it looks a little
like

  (gdb) x /64bs 0x7f38c976e060
  0x7f38c976e060: "\353\346\244\t\006"		<-- First nxt_unit_field_t
  0x7f38c976e066: ""
  0x7f38c976e067: ""
  0x7f38c976e068: "T\001"
  0x7f38c976e06b: ""
  0x7f38c976e06c: "Z\001"
  0x7f38c976e06f: ""
  ...
  0x7f38c976e170: "\362#\244\v$"		<-- Last nxt_unit_field_t
  0x7f38c976e176: ""
  0x7f38c976e177: ""
  0x7f38c976e178: "\342\002"
  0x7f38c976e17b: ""
  0x7f38c976e17c: "\352\002"
  0x7f38c976e17f: ""
  0x7f38c976e180: "POST"			<-- Start of request metadata
  0x7f38c976e185: "HTTP/1.1"
  0x7f38c976e18e: "unix:"
  0x7f38c976e194: "unix:/dev/shm/842.sock"
  0x7f38c976e1ab: ""
  0x7f38c976e1ac: "fedora"
  0x7f38c976e1b3: "/842.php"
  0x7f38c976e1bc: "HTTP_HOST"			<-- Start of header fields
  0x7f38c976e1c6: "fedora"
  0x7f38c976e1cd: "HTTP_X_FORWARDED_PROTO"
  0x7f38c976e1e4: "https"
  ...
  0x7f38c976e45a: "HTTP_COOKIE"
  0x7f38c976e466: "PHPSESSID=8apkg25r9s9vju3pi085i21eh4"
  0x7f38c976e48b: "public_form=sended"		<-- Body content

Well that's how things are supposed to look! When using Unix domain
sockets what we actually got looked like

  ...
  0x7f6141f3445a: "HTTP_COOKIE"
  0x7f6141f34466: "PHPSESSID=uo5b2nu9buijkc89jotbgmd60vpublic_form=sended"

Here, the body content (from a POST for example) has been appended
straight onto the end of the last header field value. In this case
corrupting the PHP session cookie.  The body content would still be
found by the application as its offset into this buffer is correct.

This problem was actually caused by a0327445 ("PHP: allowed to specify
URLs without a trailing '/'.") which added an extra item into this
request buffer specifying the port number that unit is listening on that
handled this request.

Unfortunately when I wrote that patch I didn't increase the size of this
request buffer to accommodate it.

When using normal TCP sockets we actually end up allocating more space
than required for this buffer, we track the end of this buffer up to
where the body content would go and so we have a few spare bytes between
the nul byte of the last field header value and the start of the body
content.

When using Unix domain sockets, they have no associated port number and
thus the port number has a length of 0 bytes, but we still write a '\0'
in there using up a byte that we didn't account for, this causes us to
loose the nul byte of the last header fields value causing the body data
to be appended to the last header field value.

The fix is simple, account for the local port length, we also add 1 to
it, this covers the nul byte, even if there is no port as with Unix
domain sockets.

Closes: <https://github.com/nginx/unit/issues/842>
Fixes: a0327445 ("PHP: allowed to specify URLs without a trailing '/'.")
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-10 21:39:57 +00:00
Andrei Zeliankou
4ed4283cff Tests: _clear_temp_dir() function introduced.
Also added temporary directory clearing after checking available
modules to prevent garbage environment when tests start.
2023-03-07 14:47:39 +00:00
Dave McAllister
d13c4fb503 Adding the NGINX Code of Conduct to the repo.
This adds the NGINX Code of Conduct file to the repo, one of the
community guides requested and tracked for community health by the NGINX
community and by GitHub insights.

[ Re-flowed the commit message - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
2023-03-07 13:05:41 +00:00
Konstantin Pavlov
1d515879f1 contrib: fixed njs make rule. 2023-02-28 10:42:18 -08:00