This extends the approach used for debian-based packages in 3f805bc64e
to rpm as well. Notable change for both deb and rpm packaging is to use
CFLAGS as defined in the build/Makefile, and not pass them from the
environment which might not be there (as is the case for rpm).
While at it, stop passing CFLAGS in the install phase, as it should no
longer invoke builds (see d54af163c4).
The rpm part was overlooked in 7a6405566c, since testing was not done
on the platforms where problem manifested itself, notably Amazon Linux
2023 and Fedora 38+.
cargo build creates the language module under
src/wasm-wasi-component/target/release/libwasm_wasi_component.so and not
build/lib/unit/modules/wasm_wasi_component.unit.so which is what we were
using as a target dependency in the Makefile which doesn't exist so this
resulted in the following
$ make wasm-wasi-component-install
cargo build --release --manifest-path src/wasm-wasi-component/Cargo.toml
Finished release [optimized] target(s) in 0.17s
install -d /opt/unit/modules
install -p src/wasm-wasi-component/target/release/libwasm_wasi_component.so \
/opt/unit/modules/wasm_wasi_component.unit.so
I.e it wanted to rebuild the module, after this patch we get the more
correct
$ make wasm-wasi-component-install
install -d /opt/unit/modules
install -p src/wasm-wasi-component/target/release/libwasm_wasi_component.so \
/opt/unit/modules/wasm_wasi_component.unit.so
This is all a little ugly because we're fighting against cargo wanting
to do its own thing and this wasm-wasi-component language module build
process is likely going to get some re-working anyway, so this will do
for now.
Reported-by: Konstantin Pavlov <thresh@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
Rather than calling make itself to build nxt_unit.o make nxt_unit.o a
dependency of the main module build target.
Reported-by: Konstantin Pavlov <thresh@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
We need to redefine CFLAGS to drop missing-prototypes as warning since
third-party code such as wasmtime fails to build from source when
building a debian package. This happens only for packages because we
use DPKG_EXPORT_BUILDFLAGS=1 propagating build flags to the environment,
so cargo build picks it up as well. Since we have no control over
third-party code, the easiest solution is to disable this warning.
Python applications are used only to generate responses here and can be
replaced by applications written in any other language. While the
"_python" prefix is used to indicate that the file contains tests
specific to the Python module.
These tests cause router crash when run with AddressSanitizer:
=================================================================
==77196==ERROR: AddressSanitizer: heap-use-after-free on address 0x60c000079340 at pc 0x55d56b132d4b bp 0x7f8cc7f346b0 sp 0x7f8cc7f346a0
READ of size 1 at 0x60c000079340 thread T1
#0 0x55d56b132d4a in nxt_openssl_conn_io_shutdown src/nxt_openssl.c:1466
#1 0x55d56b0f6a25 in nxt_h1p_closing src/nxt_h1proto.c:2069
#2 0x55d56b1009a6 in nxt_h1p_shutdown src/nxt_h1proto.c:2038
#3 0x55d56b1014c3 in nxt_h1p_request_close src/nxt_h1proto.c:1718
#4 0x55d56b1045c0 in nxt_http_request_close_handler src/nxt_http_request.c:864
#5 0x55d56b104988 in nxt_http_request_done src/nxt_http_request.c:795
#6 0x55d56b0ba0c3 in nxt_event_engine_start src/nxt_event_engine.c:542
#7 0x55d56b0dcac2 in nxt_router_thread_start src/nxt_router.c:3645
#8 0x55d56b0b421b in nxt_thread_trampoline src/nxt_thread.c:126
#9 0x7f8ccab95ac2 (/lib/x86_64-linux-gnu/libc.so.6+0x94ac2)
#10 0x7f8ccac2784f (/lib/x86_64-linux-gnu/libc.so.6+0x12684f)
This flag is necessary to either run or skip certain tests that have
specific behavior depending on whether AddressSanitizer is enabled.
For instance, some tests may fail only when the binary is compiled
with AddressSanitizer.
It can fail with reporting following alert:
[alert] 137462#137462 mount("none", "/tmp/unit-test-636e0uh8/proc", "proc", 2097162, "") (16: Device or resource busy)
This now includes support for the 'wasm-wasi-component' module.
This targets the upcoming 1.32.0 release which is required by
wasm-wasi-component. However of course the 1.32.0 tag doesn't exist yet,
so there will be a small window where this image won't build.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
The minimum version required to build wasmtime 17 which is required by
wasm-wasi-component is 1.73.0
But no point not using the latest version.
This also now needs the libclang-dev package installed, we install this
via MODULE_PREBUILD_wasm.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This exposes the various WebAssembly Component Model language module
specific options.
The application type is "wasm-wasi-component".
There is a "component" option that is required, this specifies the full
path to the WebAssembly component to be run. This component should be in
binary format, i.e a .wasm file.
There is also currently one optional option
"access"
Due to the sandboxed nature of WebAssembly, by default Wasm
modules/components don't have any access to the underlying filesystem.
There is however a capabilities based mechanism[0] for allowing such
access.
This adds a config option to the 'wasm-wasi-component' application type
(same as for 'wasm');
'access.filesystem' which takes an array of
directory paths that are then made available to the wasm
module/component. This access works recursively, i.e everything under a
specific path is allowed access to.
Example config might look like
"applications": {
"my-wasm-component": {
"type": "wasm-wasi-component",
"component": "/path/to/component.wasm",
"access" {
"filesystem": [
"/tmp",
"/var/tmp"
]
}
}
}
The actual mechanism used allows directories to be mapped differently in
the guest. But at the moment we don't support that and just map say /tmp
to /tmp. This can be revisited if it's something users clamour for.
[0]: <https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-capabilities.md>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
It seems we do want to track this thing. This is just the latest version
that cargo had generated for me.
Cc: Dan Callahan <d.callahan@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
With the initial port to wasmtime 17 we could no longer use the
'reactor' adaptor but had to switch to the more restrictive 'proxy'
adaptor.
This meant amongst other things (probably) we could no longer access the
filesystem.
Thanks to Joel Dice for pointing out the fix.
With this we can go back to using the 'reactor' adaptor again and things
are back to working as before.
It's worth noting that you can use either the 'proxy' or 'reactor'
adaptor depending on your requirements.
Cc: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
When Unit receives a request, if the body of that request is greater
than a certain amount (16KiB by default) then it is written to a
temporary file.
When a language module goes to read the request body in such situations
it will end up using read(2).
The wasm-wasi-component language module was failing to properly read
request bodies of around 2GiB or more.
This is because (on Linux at least) read(2) (and other related system
calls) will only read (or write) at most 0x7ffff000 (2,147,479,552)
bytes, this is the case for both 32 and 64-bit systems.
Regardless, it's probably not a good idea doing IO in such large chunks
anyway.
This patch changes the wasm-wasi-component language module to read the
request buffer in 32MiB chunks (this matches the original 'wasm'
language module).
We are still limited to a 4GiB address space and can only upload files a
little under 4GiB.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
Run from the repository root like
$ rustfmt --edition 2021 src/wasm-wasi-component/src/lib.rs
Also manually fix up some overly long comments.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This is used by the rustfmt program to format Rust code according to the
rules contained in this file.
Currently we just set the line width limit to 80 characters to match our
C code.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This is the work of Alex Crichton.
This is written in Rust. The problem is that there is currently no
support on the C side of things for the component model, which is the
point of this module.
It talks to Unit via automatically generated bindings.
I've (Andrew) just made some minor tweaks to src/lib.rs, build.rs &
Cargo.toml to adjust some paths, adjust where we get the language module
config from and the module name and where it's located in the source
tree,
I also removed and disabled the tracking of the Cargo.lock file, this is
constantly changing and not tracking it seems right for 'libraries' and
dropped the README's...
Other than that I have tried to leave his work intact, subsequent
commits will make some larger changes, but I didn't want to intermix
them with Alex's work.
One such commit will update the module to use wasmtime 17 which brings
WASI 0.2.0 support.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This is required to actually _build_ the 'wasm-wasi-componet' language
module.
The nxt_wasm_wc_app_conf_t structure consists of the component name, e.g
my_component.wasm, this is required. It also consists of an object to
store the directories that are allowed access to by the component, this
is optional.
The bulk of the configuration infrastructure will be added in a
subsequent commit.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This is the first commit in adding WebAssembly Component Model language
module support.
This just adds a new NXT_APP_WASM_WC type, required by subsequent
commits.
The WC stands for WASI_COMPONENT
This new module will have a type of 'wasm-wasi-component'.
Link: <https://github.com/nginx/unit/issues/1098>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This commit adds GitHub Actions configuration, running tests on
pull-requests and master push changes.
This change is meant to be a first-pass at our evolving CI processes.
- Tests run in parallel per language for speed and isolation
- Test matrix is composed by a string list of languages and versions
- `setup-${language}` actions are preferred over base (and changing)
versions from `ubuntu-latest` operating system
A few caveats with the current setup:
- Only tests on Ubuntu (no FreeBSD or Alpine)
- Unpriviledged tests only
- No core dumps available on failure
@filiphanes requested support for bytearray
and memoryview in the request body here:
<https://github.com/nginx/unit/issues/648>
This patch implements bytearray body support only.
Memoryview body still need to be implemented.
This allows the editor(1) to enable syntax highlighting. See
suffixes(7).
Cc: Liam Crilly <liam@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
When editing the configuration in-place, it's easy to make a mistake.
If the configuration is wrong, it will be passed to the control socket,
which will reject it, keeping the old configuration. Those manual edits
would be lost, which can make it very uncomfortable to edit in-place.
By printing the name of the temporary file, we allow the user to recover
the changes.
Cc: Liam Crilly <liam@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
Centralize handling of the ssh(1) tunnel in the ctl command.
This is possible now that we do the cleanup with trap(1).
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
This allows listening to command exit statuses. Before this change, we
had to ignore the exit status of curl(1) (and a few other commands),
since otherwise the script would go kaboom and not cleanup the ssh(1)
tunnels.
Fixes: 543d478e12 ("Tools: setup-unit: ctl: added "edit" subcommand.")
Fixes: 3778877eb3 ("Tools: Added subcommands to setup-unit.")
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
- Go: Drop 1.20, Add 1.22
- Node: Drop 18, Add 21
- PHP: Add 8.3
- Ruby: Add 3.3
Perl and Python are still up-to-date with upstream releases
Regenerating the Dockerfiles also picks up the logging change from
183a1e9d63
Add more entries for Andrei and Konstantin and an entry for Dan.
Reviewed-by: Dan Callahan <d.callahan@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
Saves on the order of 10 MBs of transfer for each build.
We call `rm -rf /usr/src/unit` later in this step, so the full repo has
never appeared in our published images anyway.
In that particular issue the compiled nuxt files end up importing the
http module as node:http rather than http only. This bypasses unit's
custom loader implementation which only check for the http or unit-http
modules, and their websocket counterparts.
This changeset adds replace sources for both the node:http and
node:websocket import signatures.
Closes: https://github.com/nginx/unit/issues/1013
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
This commit introduces the 'vars' JavaScript object to NJS,
enabling direct access to native variables such as $uri and $arg_foo.
The syntax is `${vars.var_name}` or `${'vars[var_name]'}`.
For example:
{
"action": {
"share": "`/www/html${vars.uri}`"
}
}
This commit is for subsequent commits that will support njs variable
accessing. In this commit, nxt_var_get() is introduced to extend
the variable handling capabilities. Concurrently, nxt_var_ref_get()
has been refactored to use in both configuration and request phases.
This commit enhances nxt_var_cache_value() to enable variable access
using string names, complementing the existing reference index method.
The modification ensures future compatibility with njs variable access.
