Squashed 'external/toxcore/c-toxcore/' changes from d9b8fa6098d..81b1e4f6348

81b1e4f6348 chore: Release v0.2.21-rc.1 9303e2e49a1 chore: Update the pkgsrc versions in the update-versions tool 71ec4b3b1e9 chore: Update the version-sync script to work in a post-tox.api.h world 66da842f753 chore: Add version update script compatible with ci-tools. 199878f7660 chore: Use new bazel script for circle ci. 8278e9cda46 chore: Add release issue template and workflow. a9bb3a1c4d1 chore: Fix alpine-s390x build. 6e0a641272e chore: Add a source tarball deploy workflow. 4adebe4d8b1 chore: Don't upload ios/macos variants in deploy workflows. 18f1d858ccb chore: Move one of the 3 freebsd builds to post-submit. 432ab60c002 feat: Add a Makefile for the single file deploy build. a86c0011fd5 chore: Add deploy job for single C file library. 2e7495e8f2a docs: Update changelog format to use the new clog-compatible way. a682da99e84 chore: Export wasmExports from the wasm binary. 12f34cdff27 chore: Add wasm to the nightly binary deploys. 1451029613f chore: Add strict-abi support for macOS/iOS. c53c30e09d9 chore: Add time option to manual fuzz trigger. 2ccecdc2a1a chore: Add remaining fuzz tests to cflite. 4626c2e230e test: Add a Net_Crypto fuzz test. b4a0e617c48 refactor: Use IP string length from ip_ntoa instead of strlen. b85b91f22f6 cleanup: rename getnodes/sendnodes to nodes request/response This change alignes the naming to be closer to the spec and make it less ambiguous. This change also changes the naming of some private/experimental marked APIs. - tox_callback_dht_nodes_response() - tox_dht_nodes_request() - Tox_Event_Dht_Get_Nodes_Response f1991aaa029 perf: Use stack allocation for strerror rendering. 3984211ccbf cleanup: remove kicked peers from saved peers list 26a991ed2be fix: ip to string function not accepting tcp families 712861f2e6d cleanup: Make websockify output qtox-compatible logging. 01932ea2f73 chore: Add opus and vpx to the toxcore wasm build. d29c42ef631 refactor: don't fully discard received DHT nodes. This is mostly forward thinking, where we might introduce other ip families, in addition to ipv4, ipv6, tcp_ipv4 etc. 21e2325934f chore: Fix xcframework tarball creation. b10c8b766ba chore: Fix xcframework checksum creation. 93787a9322e chore: Add ios/macos framework build. 9f723f891d3 fix: run do_gca also in bootstrap nodes 496cc703556 chore: Support arm64 iphone simulator. aa0e2a8e928 chore: Add support for more iOS architectures. 13ad8e81cbf chore: Add binary deploy workflows. c8344726378 refactor: Move tox_log_level out into its own file. 8799bea76c3 cleanup: Mark events/dispatch headers as experimental. d4164edb548 refactor: Remove tox_types.h; use `struct` tags instead. d408c982090 refactor: Move `Tox_Options` to `tox_options.h`. 5ab42d41209 chore: Move most cirrus jobs to circleci. 463eeae1144 cleanup: Avoid clashing with global define `DEBUG`. 92cc1e91747 refactor: Make Tox_Options own the passed proxy host and savedata. f276b397226 test: Add some more asserts for I/O and alloc to succeed. edb4dfc4869 fix: Don't crash on malloc failures in bin_unpack. be457d5d0b2 cleanup: Use tox memory for bin_unpack and net_strerror. git-subtree-dir: external/toxcore/c-toxcore git-subtree-split: 81b1e4f6348124784088591c4fe9ab41e273031d
2025-03-12 19:16:50 +01:00
parent 3b6bb15e86
commit 3105cc20ef
130 changed files with 3604 additions and 1776 deletions
--- a/other/docker/alpine-s390x/alpine-s390x.Dockerfile
+++ b/other/docker/alpine-s390x/alpine-s390x.Dockerfile
--- a/other/docker/alpine-s390x/alpine-s390x.Dockerfile.dockerignore
+++ b/other/docker/alpine-s390x/alpine-s390x.Dockerfile.dockerignore
@ -0,0 +1,24 @@
+# ===== common =====
+# Ignore everything ...
+**/*
+# ... except sources
+!**/*.[ch]
+!**/*.cc
+!**/*.hh
+!CHANGELOG.md
+!LICENSE
+!README.md
+!auto_tests/data/*
+!other/bootstrap_daemon/bash-completion/**
+!other/bootstrap_daemon/tox-bootstrapd.*
+!other/proxy/*.mod
+!other/proxy/*.sum
+!other/proxy/*.go
+# ... and CMake build files (used by most builds).
+!**/CMakeLists.txt
+!.github/scripts/flags*.sh
+!cmake/*.cmake
+!other/pkgconfig/*
+!other/rpm/*
+!so.version
+!.github/scripts/cmake-alpine-s390x
--- a/other/docker/alpine-s390x/dockerignore
+++ b/other/docker/alpine-s390x/dockerignore
@ -0,0 +1 @@
+!.github/scripts/cmake-alpine-s390x
--- a/other/docker/alpine-s390x/run
+++ b/other/docker/alpine-s390x/run
@ -1,5 +1,3 @@
-#!/bin/sh
+#!/usr/bin/env bash

-set -eux
-BUILD=alpine-s390x
-docker build -t "toxchat/c-toxcore:$BUILD" -f "other/docker/$BUILD/Dockerfile" .
+. "$(cd "$(dirname "${BASH_SOURCE[0]}")/../sources" && pwd)/run.sh"
--- a/other/docker/cppcheck/toxcore.cfg
+++ b/other/docker/cppcheck/toxcore.cfg
@ -92,10 +92,6 @@
    <alloc init="true">new_networking_no_udp</alloc>
    <dealloc arg="1">kill_networking</dealloc>
  </resource>
-  <resource>
-    <alloc init="true">net_new_strerror</alloc>
-    <dealloc arg="1">net_kill_strerror</dealloc>
-  </resource>
  <resource>
    <alloc init="true">new_onion</alloc>
    <dealloc arg="1">kill_onion</dealloc>
--- a/other/docker/goblint/goblint.Dockerfile
+++ b/other/docker/goblint/goblint.Dockerfile
@ -11,9 +11,9 @@ RUN apt-get update && \
 WORKDIR /work
 COPY --from=sources /src/ /work/

-COPY other/make_single_file /work/other/
+COPY other/deploy/single-file/make_single_file /work/

-RUN other/make_single_file -core auto_tests/tox_new_test.c other/docker/goblint/sodium.c > analysis.c
+RUN ./make_single_file -core auto_tests/tox_new_test.c other/docker/goblint/sodium.c > analysis.c
 # Try compiling+linking just to make sure we have all the fake functions.
 RUN tcc analysis.c

--- a/other/docker/goblint/sodium.c
+++ b/other/docker/goblint/sodium.c
@ -98,6 +98,17 @@ int crypto_hash_sha512(unsigned char *out, const unsigned char *in,
 {
    return 0;
 }
+int crypto_pwhash_scryptsalsa208sha256(unsigned char *const out,
+                                       unsigned long long outlen,
+                                       const char *const passwd,
+                                       unsigned long long passwdlen,
+                                       const unsigned char *const salt,
+                                       unsigned long long opslimit,
+                                       size_t memlimit)
+{
+    memset(out, 0, outlen);
+    return 0;
+}
 void randombytes(unsigned char *const buf, const unsigned long long buf_len)
 {
    memset(buf, 0, buf_len);
--- a/other/docker/modules/check
+++ b/other/docker/modules/check
@ -12,24 +12,24 @@ MODS = {}
 STD_MODULE = """module std [system] {
  textual header "/usr/include/alloca.h"
  textual header "/usr/include/assert.h"
-  textual header "/usr/include/c++/13.2.1/algorithm"
-  textual header "/usr/include/c++/13.2.1/array"
-  textual header "/usr/include/c++/13.2.1/chrono"
-  textual header "/usr/include/c++/13.2.1/cstddef"
-  textual header "/usr/include/c++/13.2.1/cstdint"
-  textual header "/usr/include/c++/13.2.1/cstdio"
-  textual header "/usr/include/c++/13.2.1/cstdlib"
-  textual header "/usr/include/c++/13.2.1/cstring"
-  textual header "/usr/include/c++/13.2.1/iomanip"
-  textual header "/usr/include/c++/13.2.1/iosfwd"
-  textual header "/usr/include/c++/13.2.1/limits"
-  textual header "/usr/include/c++/13.2.1/memory"
-  textual header "/usr/include/c++/13.2.1/ostream"
-  textual header "/usr/include/c++/13.2.1/random"
-  textual header "/usr/include/c++/13.2.1/stdlib.h"
-  textual header "/usr/include/c++/13.2.1/thread"
-  textual header "/usr/include/c++/13.2.1/type_traits"
-  textual header "/usr/include/c++/13.2.1/vector"
+  textual header "/usr/include/c++/14.2.0/algorithm"
+  textual header "/usr/include/c++/14.2.0/array"
+  textual header "/usr/include/c++/14.2.0/chrono"
+  textual header "/usr/include/c++/14.2.0/cstddef"
+  textual header "/usr/include/c++/14.2.0/cstdint"
+  textual header "/usr/include/c++/14.2.0/cstdio"
+  textual header "/usr/include/c++/14.2.0/cstdlib"
+  textual header "/usr/include/c++/14.2.0/cstring"
+  textual header "/usr/include/c++/14.2.0/iomanip"
+  textual header "/usr/include/c++/14.2.0/iosfwd"
+  textual header "/usr/include/c++/14.2.0/limits"
+  textual header "/usr/include/c++/14.2.0/memory"
+  textual header "/usr/include/c++/14.2.0/ostream"
+  textual header "/usr/include/c++/14.2.0/random"
+  textual header "/usr/include/c++/14.2.0/stdlib.h"
+  textual header "/usr/include/c++/14.2.0/thread"
+  textual header "/usr/include/c++/14.2.0/type_traits"
+  textual header "/usr/include/c++/14.2.0/vector"
  textual header "/usr/include/errno.h"
  textual header "/usr/include/fortify/stdio.h"
  textual header "/usr/include/fortify/string.h"
--- a/other/docker/modules/dockerignore
+++ b/other/docker/modules/dockerignore
@ -0,0 +1,2 @@
+!other/docker/modules/check
+!**/BUILD.bazel
--- a/other/docker/modules/modules.Dockerfile
+++ b/other/docker/modules/modules.Dockerfile
@ -1,5 +1,4 @@
-FROM toxchat/c-toxcore:sources AS sources
-FROM alpine:3.19.0
+FROM alpine:3.21.0

 RUN ["apk", "add", "--no-cache", \
 "bash", \
@ -15,7 +14,7 @@ RUN ["apk", "add", "--no-cache", \
 "python3"]

 WORKDIR /work
-COPY --from=sources /src/ /work/
+COPY . /work/

 COPY toxcore/BUILD.bazel /work/toxcore/
 COPY other/docker/modules/check /work/other/docker/modules/
--- a/other/docker/modules/modules.Dockerfile.dockerignore
+++ b/other/docker/modules/modules.Dockerfile.dockerignore
@ -0,0 +1,25 @@
+# ===== common =====
+# Ignore everything ...
+**/*
+# ... except sources
+!**/*.[ch]
+!**/*.cc
+!**/*.hh
+!CHANGELOG.md
+!LICENSE
+!README.md
+!auto_tests/data/*
+!other/bootstrap_daemon/bash-completion/**
+!other/bootstrap_daemon/tox-bootstrapd.*
+!other/proxy/*.mod
+!other/proxy/*.sum
+!other/proxy/*.go
+# ... and CMake build files (used by most builds).
+!**/CMakeLists.txt
+!.github/scripts/flags*.sh
+!cmake/*.cmake
+!other/pkgconfig/*
+!other/rpm/*
+!so.version
+!other/docker/modules/check
+!**/BUILD.bazel
--- a/other/docker/modules/run
+++ b/other/docker/modules/run
@ -1,6 +1,3 @@
-#!/bin/sh
+#!/usr/bin/env bash

-set -eux
-BUILD=modules
-other/docker/sources/build
-docker build -t "toxchat/c-toxcore:$BUILD" -f "other/docker/$BUILD/$BUILD.Dockerfile" .
+. "$(cd "$(dirname "${BASH_SOURCE[0]}")/../sources" && pwd)/run.sh"
--- a/other/docker/pkgsrc/dockerignore
+++ b/other/docker/pkgsrc/dockerignore
@ -0,0 +1 @@
+!other/docker/pkgsrc/pkgsrc.patch
--- a/other/docker/pkgsrc/pkgsrc.Dockerfile
+++ b/other/docker/pkgsrc/pkgsrc.Dockerfile
@ -1,11 +1,14 @@
 FROM toxchat/pkgsrc:latest

 WORKDIR /work
-COPY . /work/c-toxcore-0.2.18
-RUN ["tar", "zcf", "c-toxcore.tar.gz", "c-toxcore-0.2.18"]
+COPY . /work/c-toxcore-0.2.21
+RUN ["tar", "zcf", "c-toxcore.tar.gz", "c-toxcore-0.2.21"]
+
+WORKDIR /work/pkgsrc
+COPY other/docker/pkgsrc/pkgsrc.patch /tmp/pkgsrc.patch
+RUN ["patch", "-p1", "-i", "/tmp/pkgsrc.patch"]

 WORKDIR /work/pkgsrc/chat/toxcore
-RUN ["sed", "-i", "-e", "s/libtoxcore.so.2.18.0/libtoxcore.so.2.20.0/g", "PLIST"]
 RUN ["bmake", "clean"]
 RUN ["bmake", "DISTFILES=c-toxcore.tar.gz", "DISTDIR=/work", "NO_CHECKSUM=yes"]
-RUN ["bmake", "install"]
+RUN ["bmake", "DISTFILES=c-toxcore.tar.gz", "DISTDIR=/work", "NO_CHECKSUM=yes", "install"]
--- a/other/docker/pkgsrc/pkgsrc.Dockerfile.dockerignore
+++ b/other/docker/pkgsrc/pkgsrc.Dockerfile.dockerignore
@ -21,3 +21,4 @@
 !other/pkgconfig/*
 !other/rpm/*
 !so.version
+!other/docker/pkgsrc/pkgsrc.patch
--- a/other/docker/pkgsrc/pkgsrc.patch
+++ b/other/docker/pkgsrc/pkgsrc.patch
@ -0,0 +1,31 @@
+diff --git a/chat/toxcore/Makefile b/chat/toxcore/Makefile
+index 70466704d..53a08ad08 100644
+--- a/chat/toxcore/Makefile
+++ b/chat/toxcore/Makefile
+@@ -1,6 +1,6 @@
+ # $NetBSD: Makefile,v 1.6 2024/01/22 13:16:56 ryoon Exp $
+
+-DISTNAME=	toxcore-0.2.18
+DISTNAME=	toxcore-0.2.21
+ PKGREVISION=	2
+ CATEGORIES=	chat
+ MASTER_SITES=	${MASTER_SITE_GITHUB:=TokTok/}
+diff --git a/chat/toxcore/PLIST b/chat/toxcore/PLIST
+index f0a5e4f04..4122b0867 100644
+--- a/chat/toxcore/PLIST
+++ b/chat/toxcore/PLIST
+@@ -4,11 +4,11 @@ bin/tox-bootstrapd
+ include/tox/tox.h
+-include/tox/tox_dispatch.h
+-include/tox/tox_events.h
+include/tox/tox_log_level.h
+include/tox/tox_options.h
+ include/tox/toxav.h
+ include/tox/toxencryptsave.h
+ lib/libtoxcore.a
+ lib/libtoxcore.so
+ lib/libtoxcore.so.2
+-lib/libtoxcore.so.2.18.0
+lib/libtoxcore.so.2.21.0
+ lib/pkgconfig/toxcore.pc
+ share/bash-completion/completions/tox-bootstrapd
--- a/other/docker/sparse/sparse.Dockerfile
+++ b/other/docker/sparse/sparse.Dockerfile
@ -27,8 +27,8 @@ RUN make install -j4 PREFIX=/usr/local

 WORKDIR /work/c-toxcore
 COPY --from=sources /src/ /work/c-toxcore
-#COPY other/make_single_file /work/c-toxcore/other/
-#RUN other/make_single_file auto_tests/tox_new_test.c > crash.c
+#COPY other/deploy/single-file/make_single_file /work/c-toxcore/
+#RUN ./make_single_file auto_tests/tox_new_test.c > crash.c
 #RUN sparsec $(pkg-config --cflags --libs libsodium opus vpx) crash.c

 COPY other/docker/sparse/Makefile /work/c-toxcore/
--- a/other/docker/tcc/tcc.Dockerfile
+++ b/other/docker/tcc/tcc.Dockerfile
@ -33,9 +33,9 @@ RUN tcc \
 $(pkg-config --cflags --libs libsodium opus vpx) \
 && ./send_message_test | grep 'tox clients connected'

-COPY other/make_single_file /work/other/
+COPY other/deploy/single-file/make_single_file /work/
 RUN \
- other/make_single_file -core \
+ ./make_single_file -core \
   auto_tests/auto_test_support.c \
   auto_tests/send_message_test.c \
   testing/misc_tools.c | \
@ -47,7 +47,7 @@ RUN \
 && ./send_message_test | grep 'tox clients connected'

 RUN \
- other/make_single_file \
+ ./make_single_file \
   auto_tests/auto_test_support.c \
   auto_tests/toxav_basic_test.c \
   testing/misc_tools.c | \
--- a/other/docker/wasm/run
+++ b/other/docker/wasm/run
@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+. "$(cd "$(dirname "${BASH_SOURCE[0]}")/../sources" && pwd)/run.sh"
--- a/other/docker/wasm/wasm.Dockerfile
+++ b/other/docker/wasm/wasm.Dockerfile
@ -0,0 +1,136 @@
+FROM ubuntu:20.04
+
+ENV DEBIAN_FRONTEND="noninteractive"
+
+# Install dependencies.
+RUN apt-get update && apt-get install --no-install-recommends -y \
+ autoconf \
+ automake \
+ ca-certificates \
+ cmake \
+ curl \
+ git \
+ libtool \
+ make \
+ ninja-build \
+ pkg-config \
+ python3 \
+ unzip \
+ wget \
+ xz-utils \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /work/emsdk
+RUN git clone --depth=1 https://github.com/emscripten-core/emsdk /work/emsdk \
+ && ./emsdk install 4.0.3 \
+ && ./emsdk activate 4.0.3
+
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+WORKDIR /work
+
+# Build libsodium.
+RUN . "/work/emsdk/emsdk_env.sh" \
+ && tar zxf <(curl -L https://github.com/jedisct1/libsodium/releases/download/1.0.20-RELEASE/libsodium-1.0.20.tar.gz) \
+ && cd /work/libsodium-* \
+ && emconfigure ./configure \
+ --prefix=/wasm \
+ --enable-static \
+ --disable-shared \
+ --without-pthreads \
+ --disable-ssp \
+ --disable-asm \
+ --disable-pie \
+ && emmake make -j"$(nproc)" \
+ && emmake make install \
+ && rm -rf /work/libsodium-*
+
+# Build libvpx.
+RUN . "/work/emsdk/emsdk_env.sh" \
+ && tar zxf <(curl -L https://github.com/webmproject/libvpx/archive/v1.15.0.tar.gz) \
+ && cd /work/libvpx-* \
+ && emconfigure ./configure \
+ --prefix=/wasm \
+ --enable-static \
+ --disable-shared \
+ --target=generic-gnu \
+ --disable-examples \
+ --disable-tools \
+ --disable-docs \
+ --disable-unit-tests \
+ --enable-pic \
+ && emmake make -j"$(nproc)" \
+ && emmake make install \
+ && rm -rf /work/libvpx-*
+
+# Build opus.
+RUN . "/work/emsdk/emsdk_env.sh" \
+ && tar zxf <(curl -L https://github.com/xiph/opus/releases/download/v1.5.2/opus-1.5.2.tar.gz) \
+ && cd /work/opus-* \
+ && emconfigure ./configure \
+ --prefix=/wasm \
+ --enable-static \
+ --disable-shared \
+ --host wasm32-unknown-emscripten \
+ --disable-extra-programs \
+ --disable-doc \
+ CFLAGS="-O3 -flto -fPIC" \
+ && emmake make -j"$(nproc)" \
+ && emmake make install \
+ && rm -rf /work/opus-*
+
+# Build an unused binding without toxcore first so emcc caches all the system
+# libraries. This makes rebuilds of toxcore below much faster.
+RUN . "/work/emsdk/emsdk_env.sh" \
+ && mkdir -p /wasm/bin \
+ && emcc -O3 -flto \
+ -s EXPORT_NAME=libtoxcore \
+ -s MAIN_MODULE=1 \
+ -s MALLOC=emmalloc \
+ -s MODULARIZE=1 \
+ -s STRICT=1 \
+ -s WEBSOCKET_URL=ws:// \
+ --no-entry \
+ /wasm/lib/libopus.a \
+ /wasm/lib/libsodium.a \
+ /wasm/lib/libvpx.a \
+ -o /wasm/bin/libtoxcore.js
+
+ENV PKG_CONFIG_PATH="/wasm/lib/pkgconfig"
+
+# Build c-toxcore.
+COPY . /work/c-toxcore
+RUN . "/work/emsdk/emsdk_env.sh" \
+ && cd /work/c-toxcore \
+ && emcmake cmake \
+ -B_build \
+ -GNinja \
+ -DCMAKE_INSTALL_PREFIX="/wasm" \
+ -DCMAKE_C_FLAGS="-O3 -flto -fPIC" \
+ -DCMAKE_UNITY_BUILD=ON \
+ -DMUST_BUILD_TOXAV=ON \
+ -DENABLE_SHARED=OFF \
+ -DBOOTSTRAP_DAEMON=OFF \
+ -DMIN_LOGGER_LEVEL=TRACE \
+ . \
+ && emmake cmake --build _build \
+ && emmake cmake --install _build
+
+# Link together all the libraries.
+RUN . "/work/emsdk/emsdk_env.sh" \
+ && mkdir -p /wasm/bin \
+ && emcc -O3 -flto \
+ -s EXPORT_NAME=libtoxcore \
+ -s EXPORTED_RUNTIME_METHODS='["HEAPU8", "wasmExports"]' \
+ -s MAIN_MODULE=1 \
+ -s MALLOC=emmalloc \
+ -s MODULARIZE=1 \
+ -s STRICT=1 \
+ -s WEBSOCKET_URL=ws:// \
+ --no-entry \
+ /wasm/lib/libopus.a \
+ /wasm/lib/libsodium.a \
+ /wasm/lib/libvpx.a \
+ /wasm/lib/libtoxcore.a \
+ -o /wasm/bin/libtoxcore.js
+RUN ls -lh /wasm/bin/libtoxcore.js /wasm/bin/libtoxcore.wasm
--- a/other/docker/wasm/wasm.Dockerfile.dockerignore
+++ b/other/docker/wasm/wasm.Dockerfile.dockerignore
@ -0,0 +1,23 @@
+# ===== common =====
+# Ignore everything ...
+**/*
+# ... except sources
+!**/*.[ch]
+!**/*.cc
+!**/*.hh
+!CHANGELOG.md
+!LICENSE
+!README.md
+!auto_tests/data/*
+!other/bootstrap_daemon/bash-completion/**
+!other/bootstrap_daemon/tox-bootstrapd.*
+!other/proxy/*.mod
+!other/proxy/*.sum
+!other/proxy/*.go
+# ... and CMake build files (used by most builds).
+!**/CMakeLists.txt
+!.github/scripts/flags*.sh
+!cmake/*.cmake
+!other/pkgconfig/*
+!other/rpm/*
+!so.version