Commit graph

211 commits

Author SHA1 Message Date
Axel Duch
e3af18834d Router: matching regular expressions support. 2020-11-17 15:03:30 +00:00
Max Romanov
8340ca0b9c Libunit: improving logging consistency.
Debug logging depends on macros defined in nxt_auto_config.h.
2020-11-18 22:33:53 +03:00
Max Romanov
4ca9ba3408 Java: fixing isolation mounts for Alpine musl.
Thanks to @wujjpp.

This closes #490 PR on GitHub.
2020-11-10 22:27:08 +03:00
Max Romanov
4ba9e1d005 Java: upgrading 3rd-party components. 2020-11-02 21:05:04 +03:00
Tiago Natel de Moura
0390cb3a61 Isolation: mounting of procfs by default when using "rootfs". 2020-10-29 20:30:53 +00:00
Valentin Bartenev
84136eb49d Configure: using comma instead of space for passing -rpath value.
This variant will be more interoperable across various systems
and it's already used in Ruby module.

Otherwise, configure tests fail on NetBSD with:

  gcc: Missing argument for -Wl,-rpath
2020-10-26 22:24:32 +03:00
Max Romanov
c4c2f90c5b Python: ASGI server introduced.
This closes #461 issue on GitHub.
2020-10-01 23:55:23 +03:00
Tiago Natel de Moura
c2eb245b32 PHP: fixed "rootfs" isolation dependency on system mounts. 2020-09-09 19:28:44 +01:00
Max Romanov
d94dac091f Python: split module initialization from WSGI implementation.
This is required for futher ASGI implementation.
2020-09-14 13:27:02 +03:00
Max Romanov
d483aa74e6 Python: source file moved to 'python' sub-directory.
No functional changes.  Get ready for an increase in file number.
2020-09-14 12:07:30 +03:00
Tiago Natel de Moura
b65a8636bb Isolation: added "automount" option.
Now it's possible to disable default bind mounts of
languages by setting:

  {
      "isolation": {
           "automount": {
               "language_deps": false
           }
     }
  }

In this case, the user is responsible to provide a "rootfs"
containing the language libraries and required files for
the application.
2020-08-25 15:25:51 +01:00
Tiago Natel de Moura
30a242aa3c PHP: added bind mounts for extensions directory. 2020-08-25 13:28:14 +01:00
Tiago Natel de Moura
a8a7eeb1fc Moved isolation related code to "nxt_isolation.c". 2020-08-20 15:22:58 +01:00
Max Romanov
7ffc617ae8 Supporting platforms without sendfile() implementation.
This is a quick and dirty sendfile() replacement.

This closes #452 PR on GitHub.
2020-08-17 12:28:48 +03:00
Valentin Bartenev
93146616cf Basic variables support. 2020-08-13 02:46:54 +03:00
Max Romanov
a82cf4ffb6 Circular queues implementations and a test.
- naive circular queue, described in the article "A Scalable, Portable, and
 Memory-Efficient Lock-Free FIFO Queue" by Ruslan Nikolaev:
https://drops.dagstuhl.de/opus/volltexte/2019/11335/pdf/LIPIcs-DISC-2019-28.pdf
- circular queue, proposed by Valentin Bartenev in the "Unit router application
IPC" design draft
2020-08-11 19:20:32 +03:00
Valentin Bartenev
91280b4c0b Ruby: fixed gem mount paths.
The gem paths must depend on the specified interpreter.
Also, gemdir looks redundant as it's already included in Gem.default_path().
2020-08-09 01:00:44 +03:00
Valentin Bartenev
317fabc83e Ruby: simplified commands in ./configure script.
There is no reason to use printf instead of just print.

No functional changes.
2020-08-09 01:00:44 +03:00
Valentin Bartenev
7e1d800f08 Ruby: removed unused variable from ./configure script. 2020-08-09 01:00:44 +03:00
Valentin Bartenev
0d7a6885a3 Configure: verifying the Ruby library path.
An attempt to build a Ruby module for a custom Ruby installation that has the
same major version as the system Ruby may unexpectedly cause the use of the
system Ruby library.

This closes #449 issue on GitHub.
2020-08-08 23:52:31 +03:00
Tiago Natel de Moura
b28b4459b0 Isolation: fixed the generation of mounts table.
Since the introduction of rootfs feature, some language modules
can't be configured multiple times.

Now the configure generates a separate nxt_<module>_mounts.h for
each module compiled.
2020-07-31 12:21:21 +01:00
Tiago Natel de Moura
f3471c29c0 PHP: fixed version comparison in configure script.
Some PPAs for Ubuntu package PHP with versions like:
 7.2.28-3+ubuntu18.04.1+deb.sury.org+1

But the script expected only "X.Y.Z".

The issue was introduced in:
 http://hg.nginx.org/unit/rev/2ecb15904ba5
2020-07-28 16:17:18 +01:00
Igor Sysoev
65799c7252 Upstream chunked transfer encoding support. 2020-06-23 14:16:45 +03:00
Tiago Natel de Moura
f8ba5d6c00 Isolation: fixed build when features aren't detected. 2020-06-23 12:11:27 +01:00
Tiago Natel de Moura
0326cefef5 Python: fixed interpreter path in ./configure. 2020-06-23 11:01:20 +01:00
Tiago Natel de Moura
d3ca5fb47a Packages: fixed java configure script.
Now the configure script appends /server to --lib-path argument.
2020-05-28 17:04:00 +01:00
Tiago Natel de Moura
e2b53e16c6 Added "rootfs" feature. 2020-05-28 14:57:41 +01:00
Tiago Natel de Moura
e9e5ddd5a5 Refactor of process management.
The process abstraction has changed to:

  setup(task, process)
  start(task, process_data)
  prefork(task, process, mp)

The prefork() occurs in the main process right before fork.

The file src/nxt_main_process.c is completely free of process
specific logic.

The creation of a process now supports a PROCESS_CREATED state.  The
The setup() function of each process can set its state to either
created or ready.  If created, a MSG_PROCESS_CREATED is sent to main
process, where external setup can be done (required for rootfs under
container).

The core processes (discovery, controller and router) doesn't need
external setup, then they all proceeds to their start() function
straight away.

In the case of applications, the load of the module happens at the
process setup() time and The module's init() function has changed
to be the start() of the process.

The module API has changed to:

  setup(task, process, conf)
  start(task, data)

As a direct benefit of the PROCESS_CREATED message, the clone(2) of
processes using pid namespaces now doesn't need to create a pipe
to make the child block until parent setup uid/gid mappings nor it
needs to receive the child pid.
2020-03-09 16:28:25 +00:00
Remi Collet
140b81208e PHP: building with PHP 8 (development version). 2020-05-20 11:18:03 +03:00
Valentin Bartenev
8d727774e3 Implemented "return" action.
The "return" action can be used to immediately generate a simple HTTP response
with an arbitrary status:

  {
      "action": {
          "return": 404
      }
  }

This is especially useful for denying access to specific resources.
2020-03-27 17:22:52 +03:00
Max Romanov
efbcd517fc Checking sendfile() availability in configure.
Removing SF_NODISKIO flag for FreeBSD sendfile() check because it
is not used yet and to support DragonFlyBSD.

This closes #414 issue on GitHub.
2020-03-17 14:44:06 +03:00
Max Romanov
5296be0b82 Using disk file to store large request body.
This closes #386 on GitHub.
2020-03-12 17:54:29 +03:00
Igor Sysoev
7935ea4543 Round robin upstream added. 2020-03-06 18:28:54 +03:00
Valentin Bartenev
afa2f86ecf PHP: added ZTS indication to ./configure output. 2020-03-04 15:24:27 +03:00
Valentin Bartenev
75cb2a947d PHP: rearranged feature checks in ./configure.
Now it prints version even if PHP was built without embed SAPI.
2020-03-04 15:24:27 +03:00
Tiago Natel de Moura
f99d20ad39 PHP: optimization to avoid surplus chdir(2) calls.
For each request, the worker calls the php_execute_script function
from libphp that changes to the script directory before doing its
work and then restores the process directory before returning.  The
chdir(2) calls it performs are unnecessary in Unit design.  In simple
benchmarks, profiling shows that the chdir syscall code path (syscall,
FS walk, etc.) is where the CPU spends most of its time.

PHP SAPI semantics requires the script to be run from the script
directory.  In Unit's PHP implementation, we have two use cases:

- script
- arbitrary path

The "script" configuration doesn't have much need for a working
directory change: it can be changed once at module initialization.
The module needs to chdir again only if the user's PHP script also
calls chdir to switch to another directory during execution.

If "script" is not used in Unit configuration, we must ensure the
script is run from its directory (thus calling chdir before exec),
but there's no need to restore the working directory later.

Our implementation disables mandatory chdir calls with the SAPI
option SAPI_OPTION_NO_CHDIR, instead calling chdir only when needed.

To detect the user's calls to chdir, a simple "unit" extension is
added that hooks the built-in chdir() PHP call.
2020-03-03 14:38:08 +00:00
Max Romanov
4024b18bcc Go: adding main configure CFLAGS and LDFLAGS to module build flags.
This makes ASAN buildbot workers to work out-of-the-box.
2020-01-28 19:20:46 +03:00
Max Romanov
4e0a2002fe Java: fixing configure errors reporting. 2020-01-28 17:03:19 +03:00
Max Romanov
1949be644c Java: introducing SHA512 sum validation for external JARs. 2020-01-28 17:02:51 +03:00
Max Romanov
04bf6457c6 Java: fixing maven repository URL.
It is required to use https scheme and different host to download
packages from maven repository.
2020-01-28 17:01:21 +03:00
Max Romanov
693a458b4e Go: changing import name for "unit.nginx.org/go".
This patch includes packaging changes - update unit-go installation
directory.
2019-12-25 20:00:32 +03:00
Axel Duch
8677bf8d41 Router: introducing routing on client address. 2019-12-24 13:58:10 +00:00
Max Romanov
f5a2984acf Go: moving source files to the root of the project.
This patch includes packaging changes related to files move.
2019-12-24 17:59:52 +03:00
Max Romanov
a427ecd4c5 Go: installing go module for tests into build directory. 2019-12-24 17:59:48 +03:00
Max Romanov
823f658c77 Go: linking against libunit. 2019-12-24 17:59:37 +03:00
Max Romanov
55a368e505 Java: fixing racing condition in external JAR download.
Temporary file name with configure process PID used to download
JAR from external repository.  Then file renamed using command 'mv'.

The issue reproduced in clean environment when 2 or more concurrent builds
started.
2019-12-09 18:17:16 +03:00
Tiago Natel
411daeaa53 Isolation: allowed the use of credentials with unpriv userns.
The setuid/setgid syscalls requires root capabilities but if the kernel
supports unprivileged user namespace then the child process has the full
set of capabilities in the new namespace, then we can allow setting "user"
and "group" in such cases (this is a common security use case).

Tests were added to ensure user gets meaningful error messages for
uid/gid mapping misconfigurations.
2019-12-06 16:52:50 +00:00
Tiago Natel
ed2492a66a Moved credential-related code to nxt_credential.c.
This is required to avoid include cycles, as some nxt_clone_* functions
depend on the credential structures, but nxt_process depends on clone
structures.
2019-12-06 13:28:05 +00:00
Tiago Natel
2f23923e44 Changed the group listing to run unprivileged when possible.
Now the nxt_user_groups_get() function uses getgrouplist(3) when available
(except MacOS, see below).  For some platforms, getgrouplist() supports
a method of probing how much groups the user has but the behavior is not
consistent.  The method used here consists of optimistically trying to get up
to min(256, NGROUPS_MAX) groups; only if ngroups returned exceeds the original
value, we do a second call.  This method can block main's process if LDAP/NDIS+
is in use.

MacOS has getgrouplist(3) but it's buggy.  It doesn't update ngroups if the
value passed is smaller than the number of groups the user has.  Some
projects (like Go stdlib) call getgrouplist() in a loop, increasing ngroups
until it exceeds the number of groups user belongs to or fail when a limit
is reached.  For performance reasons, this is to be avoided and MacOS is
handled in the fallback implementation.

The fallback implementation is the old Unit approach.  It saves main's
user groups (getgroups(2)) and then calls initgroups(3) to load application's
groups in main, then does a second getgroups(2) to store the gids and restore
main's groups in the end.  Because of initgroups(3)' call to setgroups(2),
this method requires root capabilities.  In the case of OSX, which has
small NGROUPS_MAX by default (16), it's not possible to restore main's groups
if it's large; if so, this method fallbacks again: user_cred gids aren't
stored, and the worker process calls initgroups() itself and may block for
some time if LDAP/NDIS+ is in use.
2019-11-26 16:15:23 +00:00
Sergey Kandaurov
94a9162baa Configure: fixed posix_spawn() detection with glic 2.30.
In particular, it was previously broken on Ubuntu 19.10 and Fedora 31.
See for details: https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=2ab5741
2019-11-22 14:06:02 +03:00