Squashed 'external/toxcore/c-toxcore/' changes from e2c01e457b..b03b571272

b03b571272 fix: flaky tcp test This only fixes the symptoms, not the real problem. Sometimes or consistently on some platforms a socket might need a moment before it can be written to.
32e67ab4c2 cleanup: use typedef for private message ID's in callback
7b1db6adc1 feat: add message IDs to private group messages
99e0bcc27d refactor: Observers/ignored peers can now send and receive custom packets
b3c3c49d26 fix: Disable IPv6 in Windows cross-compilation tests
e742deddff feat: Check hashes of Windows dependencies when cross-compiling
dfb9a0b02b fix: Test the current Windows Dockerfile, not an old Dockerhub image
14de93ccec chore: Use WineHQ's Wine as Debian Bookworm's crashes
ed37616249 docs: Update the Windows cross-compilation section
9bb79c174f cleanup: Remove a couple of unnecessary misc_tools dependencies
19475adb70 chore: Statically link OpenMP into the cracker fun util on Windows
1be311e51f feat: Build the fun utils when cross-compiling to Windows
88133f8446 chore: Strip Windows binaries
3cc0ae7535 refactor: Copy over all of the required static dependencies
c4fa8f7fb1 feat: Generate .def, .exp and .lib files when building for Windows
74bbac5363 feat: Let CMake create the dll instead of doing so ourselves
246642e9ae feat: Harden Windows cross-compilation
8d431c0d11 chore: Bump Windows build dependency versions
e519f7998b fix: Remove unnecessary wsock32 dependency on Windows
ed2b60c217 chore: Use a specific non-broken slimcc version.
d7f21010a1 chore: Update github actions.
e71a68b7f2 docs: Update the list of CMake options
77e08876ff chore: Remove mod and founder from group API naming scheme
12bc042767 docs: add the experimental api build option to INSTALL.md
e1fa5cae96 refactor: Rename Queries to Query to align with other enums.
be82a3ea30 fix: Correct type for conference offline peer numbers.
0627c36716 test: Add pkgsrc build.
92578afe4b test: Add FreeBSD VM action on GitHub.
52ece0f57b test: Build toxcore on NetBSD (VM).
3fe8ee2c11 chore: Only install tox_private.h on request.
9a8dfa06ab fix: save_compatibility_test failing on big-endian systems
86f5e55578 fix: Don't serve files from websockify.
710eb674a5 fix: Correctly pass extended public keys to group moderation code.
021db7031c refactor: Use `struct`s for extended public/secret keys.
a1e999fd80 chore: Compile libsodium reference implementation with compcert.
fbe3c19cf5 cleanup: correct a few nullable annotations
623e3ee5c3 cleanup: Don't use `memcpy` to cast arbitrary `struct`s to `uint8_t[]`.
c71567dc18 fix: Pass array, not array pointer, to `memcmp`.
9b46a08144 cleanup: Never pass `void*` directly to `memcpy`.
5d7b7a7bbc refactor: Use tox rng to seed the keypair generation.
961891d568 cleanup: Small improvements found by PVS Studio.
8201019f0d chore: Disable NGC saving by default, enable through Tox_Options.
5dd9ee3f65 cleanup: Replace pointer arithmetic with explicit `&arr[i]`.
ca4606d49d refactor: Use strong typedef for NGC peer id.
442213b722 cleanup: Simplify custom packet length check in NGC.
08d3393def fix: Correct a few potential null derefs in bootstrap daemon.
b9877b32b0 fix: Add missing memunlock of local variable when it goes out of scope.
dab5fe44b9 fix: Zero out stack-allocated secret key before return.
f058103299 refactor: Make prune_gc_sanctions_list more obviously correct.
3ba7a0dec9 docs: Add static analysis tool list to README.
8d0811a0f3 docs: Run prettier-markdown on markdown files.
969e3a2bfc refactor: Fix network test not using the strong typedef
93c83fbc7c refactor: Use strong typedef instead of struct for `Socket`.
9fe18b176f fix: Fix some false positive from PVS Studio.
7c44379ccb cleanup: Check that WINXP macro exists before comparing it.
5c93231bef refactor: Make tox mutex non-recursive.
aacff73939 docs: Fix up doxyfile.
d55fc85ff5 docs: Add more documentation to crypto_core.
5bdaaaedb6 refactor: Remove `Tox *` from `tox_dispatch`.
e202341e76 refactor: Don't rely on tox_dispatch passing tox in tests.
34df938f52 chore: Use C++ mode for clang-tidy.
8b05296a78 chore: Check that both gtest and gmock exist for tests.
42010660e1 test: Add slimcc compiler compatibility test.
b473630321 chore: Add some comments to the astyle config.
b7404f24f6 cleanup: Remove implicit bool conversions.
4e2dba4d9f chore: Reformat sources with astyle.
4359e3a6bc chore: Rename C++ headers to .hh suffixes.
0c05566e58 cleanup: Further `#include` cleanups.
8d29935b7a chore: Only check the bootstrap daemon checksum on release.
f70e588bc6 cleanup: Add more `const` where possible.
511bfe39c8 cleanup: Use Bazel modules to enforce proper `#include` hygiene.
1710a0d091 refactor: Move pack/unpack `IP_Port` from DHT into network module.
a975943564 chore: Really fix coverage docker image build.
c08409390f chore: Fix post-submit coverage image.
39aadf8922 fix: Don't use `memcmp` to compare `IP_Port`s.
d94246a906 fix: partially fix a bug that prevented group part messages from sending.
eeaa039222 chore: Fix rpm build; add a CI check for it.
8328449c1a chore: Speed up docker builds a bit by reducing layer count.
d6d67d56f3 cleanup: Add `const` where possible in auto tests.
6aa9e6850d cleanup: Minor cleanup of event unpack code.
bdf460a3a9 refactor: Rename `system_{memory,...}` to `os_{memory,...}`.
203e1af81e fix: a few off by one errors in group autotests
5c093c4888 cleanup: Remove all uses of `SIZEOF_VLA`.
662c2140f3 test: Add goblint static analyser.
8f07755834 cleanup: Use `memzero(x, s)` instead of `memset(x, 0, s)`.
a7258e40cf cleanup: Use explicit 0 instead of `PACKET_ID_PADDING`.
6370d0f15d cleanup: Expand the `Tox_Options` accessor macros.
14a1a0b9bd cleanup: Remove plan9 support.
a05dccad13 test: Add a simple new/delete test for Tox.
1cdcf938b9 cleanup: Add comment after every `#endif`.
ba99d4dc4b test: Fix comment I broke in the events test PR.
e07248debb refactor: Migrate auto_tests to new events API.
bdd42b5452 refactor: Add common msgpack array packer with callback.
3c659f5288 cleanup: Rename group to conference in groupav documentation.
89957be230 cleanup: Ensure handler params are named after callback params.
c650d9d345 refactor: Pass `this` pointer as first param to s11n callbacks.
e7fb91ddb8 refactor: Allow NULL pointers for byte arrays in events.
5e2c8cabc1 cleanup: make some improvements to group moderation test
259de4867e cleanup: Remove `bin_pack_{new,free}`.
21a8ff5895 cleanup: skip a do_gc iteration before removing peers marked for deletion
16809dc36e feat: Add dht_get_nodes_response event to the events system.

git-subtree-dir: external/toxcore/c-toxcore
git-subtree-split: b03b5712720de9a9901ea12fd741f177327a7021

.github/scripts/cmake-windows.sh

@@ -18,7 +18,7 @@ docker run \
   -e ENABLE_ARCH_i686="$i686" \
   -e ENABLE_ARCH_x86_64="$x86_64" \
   -e ENABLE_TEST=true \
-  -e EXTRA_CMAKE_FLAGS="-DBOOTSTRAP_DAEMON=OFF -DMIN_LOGGER_LEVEL=DEBUG -DTEST_TIMEOUT_SECONDS=90 -DAUTOTEST=ON" \
+  -e EXTRA_CMAKE_FLAGS="-DBOOTSTRAP_DAEMON=OFF -DMIN_LOGGER_LEVEL=DEBUG -DTEST_TIMEOUT_SECONDS=90 -DAUTOTEST=ON -DUSE_IPV6=OFF" \
   -e CMAKE_C_FLAGS="$C_FLAGS" \
   -e CMAKE_CXX_FLAGS="$CXX_FLAGS" \
   -e CMAKE_EXE_LINKER_FLAGS="$LD_FLAGS" \
@@ -26,4 +26,6 @@ docker run \
   -v "$PWD:/toxcore" \
   -v "$PWD/result:/prefix" \
   --rm \
+  -t \
+  --pull never \
   "toxchat/windows:$WINDOWS_ARCH"

.github/scripts/tox-bootstrapd-docker

@@ -3,10 +3,11 @@
 set -exu -o pipefail
 
 LOCAL="${1:-}"
+CHECK="${2:-}"
 
 readarray -t FILES <<<"$(git ls-files)"
 
-if ! tar c "${FILES[@]}" | docker build -f other/bootstrap_daemon/docker/Dockerfile -t toxchat/bootstrap-node - 2>&1 | tee docker-build.log; then
+if ! tar c "${FILES[@]}" | docker build --build-arg="CHECK=$CHECK" -f other/bootstrap_daemon/docker/Dockerfile -t toxchat/bootstrap-node - 2>&1 | tee docker-build.log; then
   grep -o "::error.*::[a-f0-9]*  /usr/local/bin/tox-bootstrapd" docker-build.log
   false
 fi

.github/workflows/ci.yml

@@ -15,16 +15,24 @@ jobs:
 
   analysis:
     strategy:
+      fail-fast: false
       matrix:
-        tool: [autotools, clang-tidy, compcert, cppcheck, doxygen, infer, misra, tcc, tokstyle]
+        tool: [autotools, clang-tidy, compcert, cppcheck, doxygen, goblint, infer, freebsd, misra, modules, pkgsrc, rpm, slimcc, sparse, tcc, tokstyle]
     runs-on: ubuntu-latest
     steps:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
-      - name: Docker Build
-        uses: docker/build-push-action@v4
         with:
-          file: other/docker/${{ matrix.tool }}/Dockerfile
+          driver: docker
+      - name: Build toxchat/c-toxcore:sources
+        uses: docker/build-push-action@v5
+        with:
+          file: other/docker/sources/sources.Dockerfile
+          tags: toxchat/c-toxcore:sources
+      - name: Docker Build
+        uses: docker/build-push-action@v5
+        with:
+          file: other/docker/${{ matrix.tool }}/${{ matrix.tool }}.Dockerfile
 
   coverage-linux:
     runs-on: ubuntu-latest
@@ -75,7 +83,7 @@ jobs:
       - name: Build and test
         run: .github/scripts/cmake-osx
 
-  build-msvc:
+  build-windows-msvc:
     strategy:
       matrix:
         version: [2019, 2022]
@@ -102,17 +110,77 @@ jobs:
           cd _build
           ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:6 --build-config Debug
 
-  build-windows:
-    strategy:
-      matrix:
-        bits: [32, 64]
+  build-netbsd:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
         with:
           submodules: recursive
-      - name: Cross compilation
-        run: .github/scripts/cmake-win${{ matrix.bits }} script
+      - name: Test in NetBSD
+        id: test
+        uses: vmactions/netbsd-vm@v1
+        with:
+          usesh: true
+          copyback: false
+          prepare:
+            /usr/sbin/pkg_add
+              cmake
+              googletest
+              libconfig
+              libopus
+              libsodium
+              libvpx
+              pkg-config
+
+          run: |
+            # TODO(iphydf): Investigate NetBSD failures on these tests.
+            sed -Ei -e '/\((TCP|dht_getnodes_api)\)/s/^/#/' auto_tests/CMakeLists.txt
+            cmake . \
+              -DMIN_LOGGER_LEVEL=TRACE \
+              -DMUST_BUILD_TOXAV=ON \
+              -DNON_HERMETIC_TESTS=ON \
+              -DTEST_TIMEOUT_SECONDS=90 \
+              -DUSE_IPV6=OFF \
+              -DAUTOTEST=ON
+            cmake --build . --target install
+            ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:6
+
+  build-freebsd:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+      - name: Test in FreeBSD
+        id: test
+        uses: vmactions/freebsd-vm@v1
+        with:
+          usesh: true
+          copyback: false
+          prepare:
+            PAGER=cat ASSUME_ALWAYS_YES=YES pkg install
+              cmake
+              git
+              gmake
+              googletest
+              libconfig
+              libsodium
+              libvpx
+              opus
+              pkgconf
+
+          run: |
+            # TODO(iphydf): Investigate FreeBSD failures on these tests.
+            sed -Ei -e '/\(dht_getnodes_api\)/s/^/#/' auto_tests/CMakeLists.txt
+            cmake . \
+              -DMIN_LOGGER_LEVEL=TRACE \
+              -DMUST_BUILD_TOXAV=ON \
+              -DNON_HERMETIC_TESTS=ON \
+              -DTEST_TIMEOUT_SECONDS=50 \
+              -DUSE_IPV6=OFF \
+              -DAUTOTEST=ON
+            cmake --build . --target install
+            ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:6
 
   mypy:
     runs-on: ubuntu-latest

.github/workflows/coverity-scan.yml

@@ -8,7 +8,7 @@ jobs:
   latest:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           submodules: recursive
       - name: Install libraries

.github/workflows/docker.yml

@@ -25,7 +25,9 @@ jobs:
         with:
           submodules: recursive
       - name: Docker Build
-        run: .github/scripts/tox-bootstrapd-docker local
+        run: .github/scripts/tox-bootstrapd-docker local "$CHECK"
+        env:
+          CHECK: "${{ contains(github.event.pull_request.title, 'chore: Release ') && 'sha256sum' || 'echo' }}"
       - name: Push latest image to DockerHub
         if: ${{ github.event_name == 'push' }}
         run: docker push toxchat/bootstrap-node:latest
@@ -46,7 +48,7 @@ jobs:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build and push
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
           context: "{{defaultContext}}:other/bootstrap_daemon/websocket"
           push: ${{ github.event_name == 'push' }}
@@ -67,7 +69,7 @@ jobs:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build and push
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
           context: "."
           file: .clusterfuzzlite/Dockerfile
@@ -88,7 +90,7 @@ jobs:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build and push
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
           file: testing/Dockerfile
           push: ${{ github.event_name == 'push' }}
@@ -108,7 +110,7 @@ jobs:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build and push
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
           file: other/emscripten/Dockerfile
           push: ${{ github.event_name == 'push' }}
@@ -121,21 +123,14 @@ jobs:
     steps:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
-        with:
-          driver: docker
       - name: Login to DockerHub
         if: ${{ github.event_name == 'push' }}
         uses: docker/login-action@v3
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
-      - name: Build toxchat/c-toxcore:sources
-        uses: docker/build-push-action@v4
-        with:
-          file: other/docker/sources/Dockerfile
-          tags: toxchat/c-toxcore:sources
       - name: Build and push
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
           file: other/docker/esp32/Dockerfile
           push: ${{ github.event_name == 'push' }}
@@ -143,9 +138,15 @@ jobs:
           cache-from: type=registry,ref=toxchat/c-toxcore:esp32
           cache-to: type=inline
 
-  docker-win32:
+  docker-windows-mingw:
+    strategy:
+      matrix:
+        bits: [32, 64]
     runs-on: ubuntu-latest
     steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
       - name: Login to DockerHub
@@ -154,39 +155,27 @@ jobs:
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
-      - name: Build and push
-        uses: docker/build-push-action@v4
+      - name: Build and store to local Docker daemon
+        uses: docker/build-push-action@v5
         with:
-          context: "{{defaultContext}}:other/docker/windows"
-          push: ${{ github.event_name == 'push' }}
-          tags: toxchat/windows:win32
-          cache-from: type=registry,ref=toxchat/windows:win32
-          cache-to: type=inline
+          context: other/docker/windows
+          load: true
+          tags: toxchat/windows:win${{ matrix.bits }}
+          cache-from: type=registry,ref=toxchat/windows:win${{ matrix.bits }}
           build-args: |
-            SUPPORT_ARCH_i686=true
-            SUPPORT_ARCH_x86_64=false
+            SUPPORT_ARCH_i686=${{ matrix.bits == '32' }}
+            SUPPORT_ARCH_x86_64=${{ matrix.bits == '64' }}
             SUPPORT_TEST=true
-
-  docker-win64:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-      - name: Login to DockerHub
+      - name: Push the stored image to Dockerhub
         if: ${{ github.event_name == 'push' }}
-        uses: docker/login-action@v3
+        uses: docker/build-push-action@v5
         with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-      - name: Build and push
-        uses: docker/build-push-action@v4
-        with:
-          context: "{{defaultContext}}:other/docker/windows"
+          context: other/docker/windows
           push: ${{ github.event_name == 'push' }}
-          tags: toxchat/windows:win64
-          cache-from: type=registry,ref=toxchat/windows:win64
-          cache-to: type=inline
+          tags: toxchat/windows:win${{ matrix.bits }}
           build-args: |
-            SUPPORT_ARCH_i686=false
-            SUPPORT_ARCH_x86_64=true
+            SUPPORT_ARCH_i686=${{ matrix.bits == '32' }}
+            SUPPORT_ARCH_x86_64=${{ matrix.bits == '64' }}
             SUPPORT_TEST=true
+      - name: Cross-compile
+        run: .github/scripts/cmake-win${{ matrix.bits }} script

.github/workflows/post-submit.yml

@@ -11,7 +11,7 @@ jobs:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
       - name: Docker Build
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
           file: other/docker/alpine-s390x/Dockerfile
 
@@ -29,14 +29,14 @@ jobs:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build toxchat/c-toxcore:sources
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
-          file: other/docker/sources/Dockerfile
+          file: other/docker/sources/sources.Dockerfile
           tags: toxchat/c-toxcore:sources
       - name: Build and push
-        uses: docker/build-push-action@v4
+        uses: docker/build-push-action@v5
         with:
-          file: other/docker/coverage/Dockerfile
+          file: other/docker/coverage/coverage.Dockerfile
           push: ${{ github.event_name == 'push' }}
           tags: toxchat/c-toxcore:coverage
           cache-from: type=registry,ref=toxchat/c-toxcore:coverage