From c175e47cfee0215ad7386e7c1d9a4865280ba76f Mon Sep 17 00:00:00 2001
From: Andrew Clayton <a.clayton@nginx.com>
Date: Fri, 2 Dec 2022 17:20:37 +0000
Subject: [PATCH] Autodetect endianness.

In configure we set NXT_HAVE_LITTLE_ENDIAN for i386, amd64 and x86_64.
However that misses at least AArch64 (arm64) where it's usually run in
little endian mode.

However none of that really matters as NXT_HAVE_LITTLE_ENDIAN isn't used
anywhere.  So why this patch?

The only place we need to explicitly know about endianness is the
nxt_websocket_header_t structure where we lay it out differently
depending on endianness.

This is currently done using BYTE_ORDER, LITTLE_ENDIAN and BIG_ENDIAN
macros.

However on at least illumos (OpenSolaris / OpenIndiana) those macros are
not defined and we get compiler errors due to duplicate structure
members.

So let's use our own NXT_HAVE_{BIG,LITTLE}_ENDIAN macros.  However it
would be better to detect endianness programmatically as some
architectures can run in either mode, e.g Linux used to run in big
endian on PowerPC but has since switched to little endian (to match
x86).

This commit adds an auto/endian script (using a slightly modified
version of the test program from nginx's auto script), that checks for
the endianness of the platform being built on.  E.g

  checking for endianness ... little endian

The next commit will switch the nxt_websocket_header_t structure over to
these new macros.

Link: <https://github.com/nginx/unit/pull/298>
Link: <https://developer.ibm.com/articles/l-power-little-endian-faq-trs/>
Tested-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
---
 auto/endian | 31 +++++++++++++++++++++++++++++++
 configure   |  2 +-
 2 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100644 auto/endian

diff --git a/auto/endian b/auto/endian
new file mode 100644
index 00000000..cb23639b
--- /dev/null
+++ b/auto/endian
@@ -0,0 +1,31 @@
+# Copyright (C) Igor Sysoev
+# Copyright (C) Andrew Clayton
+# Copyright (C) Nginx, Inc.
+
+
+nxt_feature="endianness"
+nxt_feature_name=
+nxt_feature_run=value
+nxt_feature_incs=
+nxt_feature_libs=
+nxt_feature_test="#include <stdint.h>
+                  #include <stdio.h>
+
+                  int main(void) {
+                      int i = 0x11223344;
+                      uint8_t *p;
+
+                      p = (uint8_t *)&i;
+                      if (*p == 0x44)
+                          printf(\"little endian\");
+                      else
+                          printf(\"big endian\");
+                      return 0;
+                  }"
+. auto/feature
+
+if [ "$nxt_feature_value" = "little endian" ]; then
+    nxt_have=NXT_HAVE_LITTLE_ENDIAN . auto/have
+else
+    nxt_have=NXT_HAVE_BIG_ENDIAN . auto/have
+fi
diff --git a/configure b/configure
index 1d897f1d..8482b514 100755
--- a/configure
+++ b/configure
@@ -109,6 +109,7 @@ fi
 
 NXT_LIBRT=
 
+. auto/endian
 . auto/types
 . auto/clang
 . auto/atomic
@@ -136,7 +137,6 @@ fi
 
 case "$NXT_SYSTEM_PLATFORM" in
     i386 | amd64 | x86_64)
-        nxt_have=NXT_HAVE_LITTLE_ENDIAN . auto/have
         nxt_have=NXT_HAVE_NONALIGNED . auto/have
     ;;
 esac