Commit graph

1252 commits

Author SHA1 Message Date
Tiago Natel
411daeaa53 Isolation: allowed the use of credentials with unpriv userns.
The setuid/setgid syscalls requires root capabilities but if the kernel
supports unprivileged user namespace then the child process has the full
set of capabilities in the new namespace, then we can allow setting "user"
and "group" in such cases (this is a common security use case).

Tests were added to ensure user gets meaningful error messages for
uid/gid mapping misconfigurations.
2019-12-06 16:52:50 +00:00
Tiago Natel
ed2492a66a Moved credential-related code to nxt_credential.c.
This is required to avoid include cycles, as some nxt_clone_* functions
depend on the credential structures, but nxt_process depends on clone
structures.
2019-12-06 13:28:05 +00:00
Tiago Natel
c6a11260e8 Tests: added tests for setting user and group. 2019-11-11 14:36:22 +00:00
Tiago Natel
abbad122bb Tests: added support for testing "user" and "group". 2019-11-11 14:35:29 +00:00
Tiago Natel
417cc7be7c Refactor of process init.
Introduces the functions nxt_process_init_create() and
nxt_process_init_creds_set().
2019-11-26 16:26:24 +00:00
Andrei Belov
a3e5d09501 Packages: added Python 3 support on RHEL/CentOS 7. 2019-12-02 15:09:53 +03:00
Konstantin Pavlov
9a7881d131 Regenerated Dockerfiles. 2019-11-20 18:24:50 +03:00
Konstantin Pavlov
c87111cbb2 Moved docker images to Debian 10 "buster".
This changes the language versions we provide:
 - python 3.5 -> python 3.7
 - go 1.7/1.8 -> go 1.11
 - perl 5.24 -> perl 5.28
 - ruby 2.3 -> ruby 2.5
 - php 7.0 -> php 7.3
2019-11-20 18:24:46 +03:00
Tiago Natel
2f23923e44 Changed the group listing to run unprivileged when possible.
Now the nxt_user_groups_get() function uses getgrouplist(3) when available
(except MacOS, see below).  For some platforms, getgrouplist() supports
a method of probing how much groups the user has but the behavior is not
consistent.  The method used here consists of optimistically trying to get up
to min(256, NGROUPS_MAX) groups; only if ngroups returned exceeds the original
value, we do a second call.  This method can block main's process if LDAP/NDIS+
is in use.

MacOS has getgrouplist(3) but it's buggy.  It doesn't update ngroups if the
value passed is smaller than the number of groups the user has.  Some
projects (like Go stdlib) call getgrouplist() in a loop, increasing ngroups
until it exceeds the number of groups user belongs to or fail when a limit
is reached.  For performance reasons, this is to be avoided and MacOS is
handled in the fallback implementation.

The fallback implementation is the old Unit approach.  It saves main's
user groups (getgroups(2)) and then calls initgroups(3) to load application's
groups in main, then does a second getgroups(2) to store the gids and restore
main's groups in the end.  Because of initgroups(3)' call to setgroups(2),
this method requires root capabilities.  In the case of OSX, which has
small NGROUPS_MAX by default (16), it's not possible to restore main's groups
if it's large; if so, this method fallbacks again: user_cred gids aren't
stored, and the worker process calls initgroups() itself and may block for
some time if LDAP/NDIS+ is in use.
2019-11-26 16:15:23 +00:00
Tiago Natel
224787bbac Tests: added getjson() helper. 2019-11-26 16:00:11 +00:00
Tiago Natel
01103c5005 Tests: parsing of "Transfer-Encoding: chunked" responses. 2019-11-26 15:58:25 +00:00
Max Romanov
4eecf1cb6a Refactoring reference counting of req_app_link.
The reason for the change is that the req_app_link reference count
was incorrect if the application crashed at start; in this case,
the nxt_request_app_link_update_peer() function was never called.

This closes #332 issue on GitHub.
2019-11-26 17:14:53 +03:00
Tiago Natel
19b974674c Tests: fixed tests to run as root.
- The mode of testdir was changed to allow reading from other users/groups.
- The java multipart test now uploads the file into an app writable dir.
- The build directory was made readable for other users.
- The python environment test now uses the HOME env var instead of PWD
  because the latter is not set by the root shell (/bin/sh) by default.
- The node `node_modules` directory now is copied into the `testdir` instead
  of using symlinks.
2019-11-09 20:14:52 +00:00
Sergey Kandaurov
94a9162baa Configure: fixed posix_spawn() detection with glic 2.30.
In particular, it was previously broken on Ubuntu 19.10 and Fedora 31.
See for details: https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=2ab5741
2019-11-22 14:06:02 +03:00
Sergey Kandaurov
217914df7c Version bump. 2019-11-22 16:39:15 +03:00
Andrei Belov
5758a06374 Packages: added Ubuntu 19.10 "eoan" support. 2019-11-21 14:13:24 +03:00
Andrei Belov
bc14605fd6 Packages: added Python 3.8 on Ubuntu 18.04, 19.04. 2019-11-19 12:43:51 +03:00
Valentin Bartenev
0a27f137de Added tag 1.13.0 for changeset 3313bf222e6e 2019-11-14 19:25:20 +03:00
Valentin Bartenev
2241847efb Generated Dockerfiles for Unit 1.13.0. 2019-11-14 19:24:14 +03:00
Valentin Bartenev
182107c10e Added version 1.13.0 CHANGES. 2019-11-14 19:23:38 +03:00
Andrei Zeliankou
1072c94829 Tests: Python exception tests. 2019-11-14 18:46:28 +03:00
Valentin Bartenev
c2976fb337 Python: avoiding buffering of exception backtraces.
A quote from the Python 3 documentation:

 | When interactive, stdout and stderr streams are line-buffered.
 | Otherwise, they are block-buffered like regular text files.

As a result, if an exception occurred and PyErr_Print() was called, its output
could be buffered but not printed to the log for a while (ultimately, until
the interpreter finalization).  If the application process crashed shortly,
the backtrace was completely lost.

Buffering can be disabled by redefining the sys.stderr stream object.
However, interference with standard environment objects was deemed undesirable.
Instead, sys.stderr.flush() is called every time after printing exceptions.
A potential advantage here is that lines from backtraces won't be mixed
with other lines in the log.
2019-11-14 17:48:48 +03:00
Valentin Bartenev
4e41d90006 Python: removed wrong PyErr_Print() call.
PyCallable_Check() doesn't produce errors.

The needless call was introduced in fdd6ed28e3b9.
2019-11-14 17:48:48 +03:00
Valentin Bartenev
b3db7a30c0 Python: optimized response object close() calling.
PyObject_HasAttrString() is just a wrapper over PyObject_GetAttrString(),
while PyObject_CallMethod() calls it as the first step.  As a result,
PyObject_GetAttrString() was called twice if close() was present.

To get rid of PyObject_HasAttrString() while keeping the same behaviour,
the PyObject_CallMethod() call has been decomposed into separate calls of
PyObject_GetAttrString() and PyObject_CallFunction().
2019-11-14 17:48:48 +03:00
Valentin Bartenev
75c2c23bb6 Python: fixed an object leak when response close() is called.
On success, PyObject_CallMethod() returns a new reference to
the result of the call, which previously got lost.

Also, error logging on failure was added.

The issue was introduced by b0148ec28c4d.
2019-11-14 17:48:48 +03:00
Valentin Bartenev
c667bb446d Python: refactored nxt_python_request_handler(). 2019-11-14 17:48:48 +03:00
Valentin Bartenev
cd6cb02be0 Python: fixed potential object leak in case of allocation error. 2019-11-14 17:48:48 +03:00
Valentin Bartenev
e9429324c2 Python: improved error handling if response object isn't iterable.
According to the documentation, PyObject_GetIter():

 | Raises TypeError and returns NULL if the object cannot be iterated.

Previously, this exception wasn't printed or cleared and remained unhandled.
2019-11-14 17:48:48 +03:00
Valentin Bartenev
e189d0a96c Python: fixed handling of errors on response object iteration.
According to the documentation, PyIter_Next():

 | If there are no remaining values, returns NULL with no exception set.
 | If an error occurs while retrieving the item, returns NULL and passes
 | along the exception.

Previously, this exception wasn't properly handled and the response was
finalized as successful.

This issue was introduced in b0148ec28c4d.

A check for PyErr_Occurred() located in the code below might print this
traceback or occasionally catch an exception from one of the two response
close() calls.

Albeit that exceptions from the close() calls also need to be catched,
it's clear that this particular check wasn't supposed to do so.  This is
another issue and it will be fixed later.
2019-11-14 17:48:48 +03:00
Andrei Zeliankou
c7726c0eb1 Tests: proxy tests. 2019-11-14 17:15:29 +03:00
Andrei Zeliankou
efb461c0e1 Tests: run_process() and stop_processes() introduced. 2019-11-14 17:15:20 +03:00
Andrei Zeliankou
b5e3e22a46 Tests: waitforsocket() introduced. 2019-11-14 17:14:55 +03:00
Igor Sysoev
02e197e978 Processing inconsistent proxied response length.
Keepalive connection is disabled if upstream response length
differs from specified in the "Content-Length" field value.
2019-11-14 16:40:02 +03:00
Igor Sysoev
ddde9c23cf Initial proxy support. 2019-11-14 16:39:54 +03:00
Igor Sysoev
57e326b411 Introduced chained buffer completion handlers. 2019-11-14 16:39:48 +03:00
Igor Sysoev
643c433f8e Using event engine memory buffers in HTTP/1 layer. 2019-11-14 16:39:48 +03:00
Igor Sysoev
1ec6353733 Introduced event engine memory buffers. 2019-11-14 16:39:48 +03:00
Igor Sysoev
07c007af57 Event engine memory cache refactored. 2019-11-14 16:39:48 +03:00
Igor Sysoev
d4e3951c4d Using request task. 2019-11-14 16:39:48 +03:00
Igor Sysoev
14e56fe8c8 Replacing pass with action. 2019-11-14 16:39:48 +03:00
Igor Sysoev
96cd6558ce Fixed connect(2) errors processing on old Linuxes.
While connect(2) states that non-blocking connect should use EPOLLOUT:

  EINPROGRESS
    The socket is non-blocking and the connection cannot be completed
    immediately.  It is possible to select(2) or poll(2) for completion by
    selecting the socket for writing.  After select(2) indicates writability,
    use getsockopt(2) to read the SO_ERROR option at level SOL_SOCKET to
    determine whether connect() completed successfully (SO_ERROR is zero)
    or unsuccessfully (SO_ERROR is one of the usual error codes listed here,
    explaining the reason for the failure).

On connect error, Linux 2.6.32 (CentOS 6) may return EPOLLRDHUP, EPOLLERR,
EPOLLHUP, EPOLLIN, but not EPOLLOUT.
2019-11-14 16:39:48 +03:00
Andrei Zeliankou
5452ee458d Tests: fixed websocket tests. 2019-11-13 18:11:24 +03:00
Andrei Zeliankou
45d75ee2cb Tests: added Python test with threading. 2019-11-13 16:51:12 +03:00
Valentin Bartenev
69ff7ce731 Python: releasing GIL while waiting for a request.
It unblocks other threads that can be forked by the application
to work in background.

This closes #336 issue on GitHub.
2019-11-13 16:45:37 +03:00
Andrey Zelenkov
defb14f165 Tests: added Ruby test with constants. 2019-11-13 15:53:56 +03:00
Max Romanov
bdd96bc9ed Ruby: fixing initialization sequence.
There was a change (ruby/ruby@6c70fed) in Ruby 2.6 that moved
RUBY_DESCRIPTION global constant definition out of Init_version().
Unit initialized Ruby incorrectly, so the constant was not defined.

This closes #330 issue on GitHub.
2019-11-13 15:53:49 +03:00
Max Romanov
f2610d2160 Fixing libunit 'off by 2' issue in library.
Name and value in each header are 0-terminated, so additional 2 bytes
should be allocated for them.  There were several attempts to add these
2 bytes to headers in language modules, but some modules weren't updated.
Also, adding these 2 bytes is specific to the implementation which may be
changed later, so extending this mechanics to modules may cause errors.
2019-11-11 18:04:17 +03:00
Andrey Zelenkov
ed3298a3c6 Tests: fixed operator in http.py. 2019-11-08 14:04:32 +03:00
Valentin Bartenev
cc82547782 Respecting AR environment variable to configure ar binary. 2019-11-07 19:53:25 +03:00
Hong Zhi Dao
5d42599e33 Process port refactoring.
- Introduced nxt_runtime_process_port_create().
- Moved nxt_process_use() into nxt_process.c from nxt_runtime.c.
- Renamed nxt_runtime_process_remove_pid() as nxt_runtime_process_remove().
- Some public functions transformed to static.

This closes #327 issue on GitHub.
2019-10-29 16:07:21 +03:00