Compare commits

...

10 Commits

Author SHA1 Message Date
Green Sky
22823c5ca2 fix sdl video sources who dont report a proper frame ts
Some checks are pending
ContinuousDelivery / linux-ubuntu (push) Waiting to run
ContinuousDelivery / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Waiting to run
ContinuousDelivery / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Waiting to run
ContinuousDelivery / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Waiting to run
ContinuousDelivery / windows (windows-2022, ) (push) Waiting to run
ContinuousDelivery / windows (windows-2022, asan) (push) Waiting to run
ContinuousDelivery / dumpsyms (push) Blocked by required conditions
ContinuousDelivery / release (push) Blocked by required conditions
ContinuousIntegration / on ubuntu-24.04-arm (push) Waiting to run
ContinuousIntegration / asan on ubuntu-24.04-arm (push) Waiting to run
ContinuousIntegration / on ubuntu-latest (push) Waiting to run
ContinuousIntegration / asan on ubuntu-latest (push) Waiting to run
ContinuousIntegration / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Waiting to run
ContinuousIntegration / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Waiting to run
ContinuousIntegration / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Waiting to run
ContinuousIntegration / macos (push) Waiting to run
ContinuousIntegration / windows (push) Waiting to run
see gamescope (0fps + constant ts)
2026-01-02 20:47:43 +01:00
Green Sky
940d9e5c2c sub update and minor cmake stuff 2026-01-02 16:09:17 +01:00
Green Sky
f696ea5fea Merge commit 'e95f2cbb1c94c7f2c50f8f208260ad639373564d' 2026-01-01 19:15:15 +01:00
Green Sky
e95f2cbb1c Squashed 'external/toxcore/c-toxcore/' changes from e58eb27a8..1828c5356
1828c5356 fix(toxav): remove extra copy of video frame on encode
b66b8ded6 refactor: improve group stability, moderation determinism, and DHT dual-stack handling
4fbd7c10a fix(toxav): fix heap buffer overflow in RTP video packet handling
809fe8c78 refactor(tox): make the `#define` consts int literals.
50d242a37 refactor(toxav): improve MSI safety and testability
da1c13a2f fix(toxav): harden video processing and fix large frame handling
472825288 fix(toxav): fix multiple logic bugs in audio module
dc963d9a9 fix(toxav): fix multiple bugs in bandwidth controller and add tests
3bf5778ef refactor(toxav): split out RTP module and add exhaustive unit tests
b79b7d436 fix(autotools): add tox_log_level.h to public headers list
ea2e34ff2 chore: Disable cirrus. We're out of quota again.
b449ea2ed chore(ci): update azure runner image to windows-2022 windows-2019 is EOL
e115b136d refactor: Make add_to_list non-recursive.
REVERT: e58eb27a8 fix(toxav): remove extra copy of video frame on encode Tested and works, but there might be alignment issues and other stuff.

git-subtree-dir: external/toxcore/c-toxcore
git-subtree-split: 1828c5356b2daf1d5f680854e776d74b181d268c
2026-01-01 19:15:15 +01:00
Green Sky
0d3696c0c5 fix local message duplication if joining a group via invite
Some checks failed
ContinuousDelivery / linux-ubuntu (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, ) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, asan) (push) Has been cancelled
ContinuousIntegration / on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousIntegration / macos (push) Has been cancelled
ContinuousIntegration / windows (push) Has been cancelled
ContinuousDelivery / dumpsyms (push) Has been cancelled
ContinuousDelivery / release (push) Has been cancelled
2025-12-19 16:32:18 +01:00
Green Sky
811a673b0d correctly move buffer
Some checks failed
ContinuousDelivery / linux-ubuntu (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, ) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, asan) (push) Has been cancelled
ContinuousDelivery / dumpsyms (push) Has been cancelled
ContinuousDelivery / release (push) Has been cancelled
ContinuousIntegration / on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousIntegration / macos (push) Has been cancelled
ContinuousIntegration / windows (push) Has been cancelled
other fixes
2025-12-19 16:06:50 +01:00
Green Sky
aef9593095 remove unused var 2025-12-19 14:15:46 +01:00
Green Sky
4dc22c012e update example config
Some checks failed
ContinuousDelivery / linux-ubuntu (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, ) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, asan) (push) Has been cancelled
ContinuousDelivery / dumpsyms (push) Has been cancelled
ContinuousDelivery / release (push) Has been cancelled
ContinuousIntegration / on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousIntegration / macos (push) Has been cancelled
ContinuousIntegration / windows (push) Has been cancelled
2025-12-19 14:01:18 +01:00
Green Sky
aa8bacc18f fix more warnings
Some checks failed
ContinuousDelivery / linux-ubuntu (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousDelivery / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, ) (push) Has been cancelled
ContinuousDelivery / windows (windows-2022, asan) (push) Has been cancelled
ContinuousDelivery / dumpsyms (push) Has been cancelled
ContinuousDelivery / release (push) Has been cancelled
ContinuousIntegration / on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-24.04-arm (push) Has been cancelled
ContinuousIntegration / on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / asan on ubuntu-latest (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:arm64-v8a vcpkg_toolkit:arm64-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:armeabi-v7a vcpkg_toolkit:arm-neon-android-23]) (push) Has been cancelled
ContinuousIntegration / android (map[ndk_abi:x86_64 vcpkg_toolkit:x64-android-23]) (push) Has been cancelled
ContinuousIntegration / macos (push) Has been cancelled
ContinuousIntegration / windows (push) Has been cancelled
2025-12-19 13:28:26 +01:00
Green Sky
47278807f6 fix image loader warning by better handling errors 2025-12-19 13:23:38 +01:00
58 changed files with 4659 additions and 1171 deletions

View File

@@ -28,16 +28,6 @@ else()
endif() endif()
# HACK: "install" api headers into binary dir # HACK: "install" api headers into binary dir
configure_file(
./c-toxcore/toxcore/tox_log_level.h
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_log_level.h
@ONLY
)
configure_file(
./c-toxcore/toxcore/tox_options.h
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_options.h
@ONLY
)
configure_file( configure_file(
./c-toxcore/toxcore/tox.h ./c-toxcore/toxcore/tox.h
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox.h ${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox.h
@@ -48,6 +38,16 @@ configure_file(
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_events.h ${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_events.h
@ONLY @ONLY
) )
configure_file(
./c-toxcore/toxcore/tox_log_level.h
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_log_level.h
@ONLY
)
configure_file(
./c-toxcore/toxcore/tox_options.h
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_options.h
@ONLY
)
configure_file( configure_file(
./c-toxcore/toxcore/tox_private.h ./c-toxcore/toxcore/tox_private.h
${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_private.h ${CMAKE_CURRENT_BINARY_DIR}/include/tox/tox_private.h

View File

@@ -1,33 +1,33 @@
--- # ---
freebsd_task: # freebsd_task:
timeout_in: 5m # timeout_in: 5m
freebsd_instance: # freebsd_instance:
image_family: freebsd-14-2 # image_family: freebsd-14-2
configure_script: # configure_script:
- PAGER=cat ASSUME_ALWAYS_YES=YES pkg install # - PAGER=cat ASSUME_ALWAYS_YES=YES pkg install
cmake # cmake
git # git
gmake # gmake
googletest # googletest
libconfig # libconfig
libsodium # libsodium
libvpx # libvpx
ninja # ninja
opus # opus
pkgconf # pkgconf
- git submodule update --init --recursive # - git submodule update --init --recursive
test_all_script: # test_all_script:
- | # - |
# TODO(iphydf): Investigate FreeBSD failures on these tests. # # TODO(iphydf): Investigate FreeBSD failures on these tests.
sed -Ei -e '/\(dht_nodes_response_api\)/s/^/#/' auto_tests/CMakeLists.txt # sed -Ei -e '/\(dht_nodes_response_api\)/s/^/#/' auto_tests/CMakeLists.txt
cmake . \ # cmake . \
-DMIN_LOGGER_LEVEL=TRACE \ # -DMIN_LOGGER_LEVEL=TRACE \
-DMUST_BUILD_TOXAV=ON \ # -DMUST_BUILD_TOXAV=ON \
-DNON_HERMETIC_TESTS=OFF \ # -DNON_HERMETIC_TESTS=OFF \
-DTEST_TIMEOUT_SECONDS=50 \ # -DTEST_TIMEOUT_SECONDS=50 \
-DUSE_IPV6=OFF \ # -DUSE_IPV6=OFF \
-DAUTOTEST=ON \ # -DAUTOTEST=ON \
-GNinja # -GNinja
cmake --build . --target install # cmake --build . --target install
ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:3 || # ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:3 ||
ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:6 # ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:6

View File

@@ -94,7 +94,7 @@ jobs:
strategy: strategy:
matrix: matrix:
arch: [arm64, x86_64] arch: [arm64, x86_64]
runs-on: ${{ matrix.arch == 'arm64' && 'macos-14' || 'macos-13' }} runs-on: ${{ matrix.arch == 'arm64' && 'macos-14' || 'macos-15-intel' }}
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:

View File

@@ -418,7 +418,6 @@ if(BUILD_TOXAV)
toxav/rtp.h toxav/rtp.h
toxav/toxav.c toxav/toxav.c
toxav/toxav.h toxav/toxav.h
toxav/toxav_hacks.h
toxav/toxav_old.c toxav/toxav_old.c
toxav/video.c toxav/video.c
toxav/video.h) toxav/video.h)
@@ -578,8 +577,15 @@ endfunction()
# The actual unit tests follow. # The actual unit tests follow.
# #
if(UNITTEST AND TARGET GTest::gtest AND TARGET GTest::gmock) if(UNITTEST AND TARGET GTest::gtest AND TARGET GTest::gmock)
if(BUILD_TOXAV)
unit_test(toxav audio)
unit_test(toxav bwcontroller)
unit_test(toxav msi)
unit_test(toxav ring_buffer) unit_test(toxav ring_buffer)
unit_test(toxav rtp) unit_test(toxav rtp)
unit_test(toxav video)
endif()
unit_test(toxcore DHT) unit_test(toxcore DHT)
unit_test(toxcore bin_pack) unit_test(toxcore bin_pack)
unit_test(toxcore crypto_core) unit_test(toxcore crypto_core)

View File

@@ -39,7 +39,7 @@ extra_data = {
[cc_test( [cc_test(
name = src[:-2], name = src[:-2],
size = "small", timeout = "moderate",
srcs = [src], srcs = [src],
args = ["$(location %s)" % src] + extra_args.get( args = ["$(location %s)" % src] + extra_args.get(
src[:-2], src[:-2],

View File

@@ -51,7 +51,7 @@ static void handle_friend_connection_status(
} }
} }
static void audio_callback(void *tox, uint32_t groupnumber, uint32_t peernumber, static void audio_callback(void *tox, uint32_t conference_number, uint32_t peer_number,
const int16_t *pcm, unsigned int samples, uint8_t channels, uint32_t const int16_t *pcm, unsigned int samples, uint8_t channels, uint32_t
sample_rate, void *user_data) sample_rate, void *user_data)
{ {
@@ -63,14 +63,14 @@ static void audio_callback(void *tox, uint32_t groupnumber, uint32_t peernumber,
State *state = (State *)autotox->state; State *state = (State *)autotox->state;
for (uint32_t i = 0; i < state->received_audio_num; ++i) { for (uint32_t i = 0; i < state->received_audio_num; ++i) {
if (state->received_audio_peers[i] == peernumber) { if (state->received_audio_peers[i] == peer_number) {
return; return;
} }
} }
ck_assert(state->received_audio_num < NUM_AV_GROUP_TOX); ck_assert(state->received_audio_num < NUM_AV_GROUP_TOX);
state->received_audio_peers[state->received_audio_num] = peernumber; state->received_audio_peers[state->received_audio_num] = peer_number;
++state->received_audio_num; ++state->received_audio_num;
} }

View File

@@ -89,7 +89,7 @@ static void handle_conference_connected(
static uint32_t num_recv; static uint32_t num_recv;
static void handle_conference_message( static void handle_conference_message(
Tox *tox, uint32_t groupnumber, uint32_t peernumber, Tox_Message_Type type, Tox *tox, uint32_t conference_number, uint32_t peer_number, Tox_Message_Type type,
const uint8_t *message, size_t length, void *user_data) const uint8_t *message, size_t length, void *user_data)
{ {
if (length == (sizeof(GROUP_MESSAGE) - 1) && memcmp(message, GROUP_MESSAGE, sizeof(GROUP_MESSAGE) - 1) == 0) { if (length == (sizeof(GROUP_MESSAGE) - 1) && memcmp(message, GROUP_MESSAGE, sizeof(GROUP_MESSAGE) - 1) == 0) {

View File

@@ -3,6 +3,10 @@
#include <string.h> #include <string.h>
#include <time.h> #include <time.h>
#if !defined(_WIN32) && !defined(__WIN32__) && !defined(WIN32)
#include <pthread.h>
#endif
#include <vpx/vpx_image.h> #include <vpx/vpx_image.h>
#include "../testing/misc_tools.h" #include "../testing/misc_tools.h"
@@ -10,6 +14,7 @@
#include "../toxcore/crypto_core.h" #include "../toxcore/crypto_core.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/tox.h" #include "../toxcore/tox.h"
#include "../toxcore/tox_struct.h"
#include "../toxcore/util.h" #include "../toxcore/util.h"
#include "auto_test_support.h" #include "auto_test_support.h"
#include "check_compat.h" #include "check_compat.h"
@@ -38,6 +43,33 @@ typedef struct {
uint32_t state; uint32_t state;
} CallControl; } CallControl;
typedef struct Time_Data {
pthread_mutex_t lock;
uint64_t clock;
} Time_Data;
static uint64_t get_state_clock_callback_basic(void *user_data)
{
Time_Data *time_data = (Time_Data *)user_data;
pthread_mutex_lock(&time_data->lock);
uint64_t clock = time_data->clock;
pthread_mutex_unlock(&time_data->lock);
return clock;
}
static void increment_clock(Time_Data *time_data, uint64_t count)
{
pthread_mutex_lock(&time_data->lock);
time_data->clock += count;
pthread_mutex_unlock(&time_data->lock);
}
static void set_current_time_callback(Tox *tox, Time_Data *time_data)
{
Mono_Time *mono_time = tox->mono_time;
mono_time_set_current_time_callback(mono_time, get_state_clock_callback_basic, time_data);
}
static void clear_call_control(CallControl *cc) static void clear_call_control(CallControl *cc)
{ {
const CallControl empty = {0}; const CallControl empty = {0};
@@ -89,12 +121,17 @@ static void t_accept_friend_request_cb(Tox *m, const uint8_t *public_key, const
/** /**
* Iterate helper * Iterate helper
*/ */
static void iterate_tox(Tox *bootstrap, Tox *alice, Tox *bob) static void iterate_tox(Tox *bootstrap, Tox *alice, Tox *bob, Time_Data *time_data)
{ {
c_sleep(100);
tox_iterate(bootstrap, nullptr); tox_iterate(bootstrap, nullptr);
tox_iterate(alice, nullptr); tox_iterate(alice, nullptr);
tox_iterate(bob, nullptr); tox_iterate(bob, nullptr);
if (time_data) {
increment_clock(time_data, 50);
}
c_sleep(5);
} }
static bool toxav_audio_send_frame_helper(ToxAV *av, uint32_t friend_number, Toxav_Err_Send_Frame *error) static bool toxav_audio_send_frame_helper(ToxAV *av, uint32_t friend_number, Toxav_Err_Send_Frame *error)
@@ -107,7 +144,7 @@ static void regular_call_flow(
Tox *alice, Tox *bob, Tox *bootstrap, Tox *alice, Tox *bob, Tox *bootstrap,
ToxAV *alice_av, ToxAV *bob_av, ToxAV *alice_av, ToxAV *bob_av,
CallControl *alice_cc, CallControl *bob_cc, CallControl *alice_cc, CallControl *bob_cc,
int a_br, int v_br) int a_br, int v_br, Time_Data *time_data)
{ {
clear_call_control(alice_cc); clear_call_control(alice_cc);
clear_call_control(bob_cc); clear_call_control(bob_cc);
@@ -117,7 +154,7 @@ static void regular_call_flow(
ck_assert_msg(call_err == TOXAV_ERR_CALL_OK, "toxav_call failed: %u\n", call_err); ck_assert_msg(call_err == TOXAV_ERR_CALL_OK, "toxav_call failed: %u\n", call_err);
const time_t start_time = time(nullptr); const uint64_t start_time = get_state_clock_callback_basic(time_data);
do { do {
if (bob_cc->incoming) { if (bob_cc->incoming) {
@@ -128,7 +165,7 @@ static void regular_call_flow(
bob_cc->incoming = false; bob_cc->incoming = false;
} else { /* TODO(mannol): rtp */ } else { /* TODO(mannol): rtp */
if (time(nullptr) - start_time >= 1) { if (get_state_clock_callback_basic(time_data) - start_time >= 1000) {
Toxav_Err_Call_Control cc_err; Toxav_Err_Call_Control cc_err;
toxav_call_control(alice_av, 0, TOXAV_CALL_CONTROL_CANCEL, &cc_err); toxav_call_control(alice_av, 0, TOXAV_CALL_CONTROL_CANCEL, &cc_err);
@@ -137,7 +174,7 @@ static void regular_call_flow(
} }
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, time_data);
} while (bob_cc->state != TOXAV_FRIEND_CALL_STATE_FINISHED); } while (bob_cc->state != TOXAV_FRIEND_CALL_STATE_FINISHED);
printf("Success!\n"); printf("Success!\n");
@@ -151,22 +188,34 @@ static void test_av_flows(void)
CallControl alice_cc, bob_cc; CallControl alice_cc, bob_cc;
Time_Data time_data;
pthread_mutex_init(&time_data.lock, nullptr);
{ {
Tox_Options *opts = tox_options_new(nullptr);
ck_assert(opts != nullptr);
tox_options_set_experimental_thread_safety(opts, true);
Tox_Err_New error; Tox_Err_New error;
bootstrap = tox_new_log(nullptr, &error, &index[0]); bootstrap = tox_new_log(opts, &error, &index[0]);
ck_assert(error == TOX_ERR_NEW_OK); ck_assert(error == TOX_ERR_NEW_OK);
time_data.clock = current_time_monotonic(bootstrap->mono_time);
set_current_time_callback(bootstrap, &time_data);
alice = tox_new_log(nullptr, &error, &index[1]); alice = tox_new_log(opts, &error, &index[1]);
ck_assert(error == TOX_ERR_NEW_OK); ck_assert(error == TOX_ERR_NEW_OK);
set_current_time_callback(alice, &time_data);
bob = tox_new_log(nullptr, &error, &index[2]); bob = tox_new_log(opts, &error, &index[2]);
ck_assert(error == TOX_ERR_NEW_OK); ck_assert(error == TOX_ERR_NEW_OK);
set_current_time_callback(bob, &time_data);
tox_options_free(opts);
} }
printf("Created 3 instances of Tox\n"); printf("Created 3 instances of Tox\n");
printf("Preparing network...\n"); printf("Preparing network...\n");
long long unsigned int cur_time = time(nullptr); uint64_t cur_time = get_state_clock_callback_basic(&time_data);
uint8_t address[TOX_ADDRESS_SIZE]; uint8_t address[TOX_ADDRESS_SIZE];
@@ -186,12 +235,12 @@ static void test_av_flows(void)
uint8_t off = 1; uint8_t off = 1;
while (true) { while (true) {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
if (tox_self_get_connection_status(bootstrap) && if (tox_self_get_connection_status(bootstrap) &&
tox_self_get_connection_status(alice) && tox_self_get_connection_status(alice) &&
tox_self_get_connection_status(bob) && off) { tox_self_get_connection_status(bob) && off) {
printf("Toxes are online, took %llu seconds\n", time(nullptr) - cur_time); printf("Toxes are online, took %llu seconds\n", (unsigned long long)(get_state_clock_callback_basic(&time_data) - cur_time) / 1000);
off = 0; off = 0;
} }
@@ -200,7 +249,7 @@ static void test_av_flows(void)
break; break;
} }
c_sleep(20); increment_clock(&time_data, 100);
} }
{ {
@@ -223,24 +272,24 @@ static void test_av_flows(void)
toxav_callback_audio_receive_frame(bob_av, t_toxav_receive_audio_frame_cb, &bob_cc); toxav_callback_audio_receive_frame(bob_av, t_toxav_receive_audio_frame_cb, &bob_cc);
printf("Created 2 instances of ToxAV\n"); printf("Created 2 instances of ToxAV\n");
printf("All set after %llu seconds!\n", time(nullptr) - cur_time); printf("All set after %llu seconds!\n", (unsigned long long)(get_state_clock_callback_basic(&time_data) - cur_time) / 1000);
if (TEST_REGULAR_AV) { if (TEST_REGULAR_AV) {
printf("\nTrying regular call (Audio and Video)...\n"); printf("\nTrying regular call (Audio and Video)...\n");
regular_call_flow(alice, bob, bootstrap, alice_av, bob_av, &alice_cc, &bob_cc, regular_call_flow(alice, bob, bootstrap, alice_av, bob_av, &alice_cc, &bob_cc,
48, 4000); 48, 4000, &time_data);
} }
if (TEST_REGULAR_A) { if (TEST_REGULAR_A) {
printf("\nTrying regular call (Audio only)...\n"); printf("\nTrying regular call (Audio only)...\n");
regular_call_flow(alice, bob, bootstrap, alice_av, bob_av, &alice_cc, &bob_cc, regular_call_flow(alice, bob, bootstrap, alice_av, bob_av, &alice_cc, &bob_cc,
48, 0); 48, 0, &time_data);
} }
if (TEST_REGULAR_V) { if (TEST_REGULAR_V) {
printf("\nTrying regular call (Video only)...\n"); printf("\nTrying regular call (Video only)...\n");
regular_call_flow(alice, bob, bootstrap, alice_av, bob_av, &alice_cc, &bob_cc, regular_call_flow(alice, bob, bootstrap, alice_av, bob_av, &alice_cc, &bob_cc,
0, 4000); 0, 4000, &time_data);
} }
if (TEST_REJECT) { /* Alice calls; Bob rejects */ if (TEST_REJECT) { /* Alice calls; Bob rejects */
@@ -257,7 +306,7 @@ static void test_av_flows(void)
} }
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (!bob_cc.incoming); } while (!bob_cc.incoming);
/* Reject */ /* Reject */
@@ -269,7 +318,7 @@ static void test_av_flows(void)
} }
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (alice_cc.state != TOXAV_FRIEND_CALL_STATE_FINISHED); } while (alice_cc.state != TOXAV_FRIEND_CALL_STATE_FINISHED);
printf("Success!\n"); printf("Success!\n");
@@ -289,7 +338,7 @@ static void test_av_flows(void)
} }
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (!bob_cc.incoming); } while (!bob_cc.incoming);
/* Cancel */ /* Cancel */
@@ -302,7 +351,7 @@ static void test_av_flows(void)
/* Alice will not receive end state */ /* Alice will not receive end state */
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (bob_cc.state != TOXAV_FRIEND_CALL_STATE_FINISHED); } while (bob_cc.state != TOXAV_FRIEND_CALL_STATE_FINISHED);
printf("Success!\n"); printf("Success!\n");
@@ -323,7 +372,7 @@ static void test_av_flows(void)
} }
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (!bob_cc.incoming); } while (!bob_cc.incoming);
/* At first try all stuff while in invalid state */ /* At first try all stuff while in invalid state */
@@ -341,39 +390,39 @@ static void test_av_flows(void)
ck_assert_msg(rc == TOXAV_ERR_ANSWER_OK, "toxav_answer failed: %u\n", rc); ck_assert_msg(rc == TOXAV_ERR_ANSWER_OK, "toxav_answer failed: %u\n", rc);
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
/* Pause and Resume */ /* Pause and Resume */
printf("Pause and Resume\n"); printf("Pause and Resume\n");
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_PAUSE); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_PAUSE);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state == 0); ck_assert(bob_cc.state == 0);
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_RESUME); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_RESUME);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state & (TOXAV_FRIEND_CALL_STATE_SENDING_A | TOXAV_FRIEND_CALL_STATE_SENDING_V)); ck_assert(bob_cc.state & (TOXAV_FRIEND_CALL_STATE_SENDING_A | TOXAV_FRIEND_CALL_STATE_SENDING_V));
/* Mute/Unmute single */ /* Mute/Unmute single */
printf("Mute/Unmute single\n"); printf("Mute/Unmute single\n");
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_MUTE_AUDIO); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_MUTE_AUDIO);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state ^ TOXAV_FRIEND_CALL_STATE_ACCEPTING_A); ck_assert(bob_cc.state ^ TOXAV_FRIEND_CALL_STATE_ACCEPTING_A);
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_UNMUTE_AUDIO); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_UNMUTE_AUDIO);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_ACCEPTING_A); ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_ACCEPTING_A);
/* Mute/Unmute both */ /* Mute/Unmute both */
printf("Mute/Unmute both\n"); printf("Mute/Unmute both\n");
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_MUTE_AUDIO); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_MUTE_AUDIO);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state ^ TOXAV_FRIEND_CALL_STATE_ACCEPTING_A); ck_assert(bob_cc.state ^ TOXAV_FRIEND_CALL_STATE_ACCEPTING_A);
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_HIDE_VIDEO); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_HIDE_VIDEO);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state ^ TOXAV_FRIEND_CALL_STATE_ACCEPTING_V); ck_assert(bob_cc.state ^ TOXAV_FRIEND_CALL_STATE_ACCEPTING_V);
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_UNMUTE_AUDIO); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_UNMUTE_AUDIO);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_ACCEPTING_A); ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_ACCEPTING_A);
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_SHOW_VIDEO); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_SHOW_VIDEO);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_ACCEPTING_V); ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_ACCEPTING_V);
{ {
@@ -383,7 +432,7 @@ static void test_av_flows(void)
ck_assert_msg(rc == TOXAV_ERR_CALL_CONTROL_OK, "toxav_call_control failed: %u\n", rc); ck_assert_msg(rc == TOXAV_ERR_CALL_CONTROL_OK, "toxav_call_control failed: %u\n", rc);
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state == TOXAV_FRIEND_CALL_STATE_FINISHED); ck_assert(bob_cc.state == TOXAV_FRIEND_CALL_STATE_FINISHED);
printf("Success!\n"); printf("Success!\n");
@@ -404,7 +453,7 @@ static void test_av_flows(void)
} }
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (!bob_cc.incoming); } while (!bob_cc.incoming);
{ {
@@ -414,25 +463,25 @@ static void test_av_flows(void)
ck_assert_msg(rc == TOXAV_ERR_ANSWER_OK, "toxav_answer failed: %u\n", rc); ck_assert_msg(rc == TOXAV_ERR_ANSWER_OK, "toxav_answer failed: %u\n", rc);
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
printf("Call started as audio only\n"); printf("Call started as audio only\n");
printf("Turning on video for Alice...\n"); printf("Turning on video for Alice...\n");
ck_assert(toxav_video_set_bit_rate(alice_av, 0, 1000, nullptr)); ck_assert(toxav_video_set_bit_rate(alice_av, 0, 1000, nullptr));
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_SENDING_V); ck_assert(bob_cc.state & TOXAV_FRIEND_CALL_STATE_SENDING_V);
printf("Turning off video for Alice...\n"); printf("Turning off video for Alice...\n");
ck_assert(toxav_video_set_bit_rate(alice_av, 0, 0, nullptr)); ck_assert(toxav_video_set_bit_rate(alice_av, 0, 0, nullptr));
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(!(bob_cc.state & TOXAV_FRIEND_CALL_STATE_SENDING_V)); ck_assert(!(bob_cc.state & TOXAV_FRIEND_CALL_STATE_SENDING_V));
printf("Turning off audio for Alice...\n"); printf("Turning off audio for Alice...\n");
ck_assert(toxav_audio_set_bit_rate(alice_av, 0, 0, nullptr)); ck_assert(toxav_audio_set_bit_rate(alice_av, 0, 0, nullptr));
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(!(bob_cc.state & TOXAV_FRIEND_CALL_STATE_SENDING_A)); ck_assert(!(bob_cc.state & TOXAV_FRIEND_CALL_STATE_SENDING_A));
{ {
@@ -442,7 +491,7 @@ static void test_av_flows(void)
ck_assert_msg(rc == TOXAV_ERR_CALL_CONTROL_OK, "toxav_call_control failed: %u\n", rc); ck_assert_msg(rc == TOXAV_ERR_CALL_CONTROL_OK, "toxav_call_control failed: %u\n", rc);
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state == TOXAV_FRIEND_CALL_STATE_FINISHED); ck_assert(bob_cc.state == TOXAV_FRIEND_CALL_STATE_FINISHED);
printf("Success!\n"); printf("Success!\n");
@@ -463,7 +512,7 @@ static void test_av_flows(void)
} }
do { do {
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
} while (!bob_cc.incoming); } while (!bob_cc.incoming);
{ {
@@ -473,16 +522,16 @@ static void test_av_flows(void)
ck_assert_msg(rc == TOXAV_ERR_ANSWER_OK, "toxav_answer failed: %u\n", rc); ck_assert_msg(rc == TOXAV_ERR_ANSWER_OK, "toxav_answer failed: %u\n", rc);
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_PAUSE); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_PAUSE);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(!toxav_audio_send_frame_helper(alice_av, 0, nullptr)); ck_assert(!toxav_audio_send_frame_helper(alice_av, 0, nullptr));
ck_assert(!toxav_audio_send_frame_helper(bob_av, 0, nullptr)); ck_assert(!toxav_audio_send_frame_helper(bob_av, 0, nullptr));
ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_RESUME); ck_assert_call_control(alice_av, 0, TOXAV_CALL_CONTROL_RESUME);
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(toxav_audio_send_frame_helper(alice_av, 0, nullptr)); ck_assert(toxav_audio_send_frame_helper(alice_av, 0, nullptr));
ck_assert(toxav_audio_send_frame_helper(bob_av, 0, nullptr)); ck_assert(toxav_audio_send_frame_helper(bob_av, 0, nullptr));
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
{ {
Toxav_Err_Call_Control rc; Toxav_Err_Call_Control rc;
@@ -491,7 +540,7 @@ static void test_av_flows(void)
ck_assert_msg(rc == TOXAV_ERR_CALL_CONTROL_OK, "toxav_call_control failed: %u\n", rc); ck_assert_msg(rc == TOXAV_ERR_CALL_CONTROL_OK, "toxav_call_control failed: %u\n", rc);
} }
iterate_tox(bootstrap, alice, bob); iterate_tox(bootstrap, alice, bob, &time_data);
ck_assert(bob_cc.state == TOXAV_FRIEND_CALL_STATE_FINISHED); ck_assert(bob_cc.state == TOXAV_FRIEND_CALL_STATE_FINISHED);
printf("Success!\n"); printf("Success!\n");
@@ -503,6 +552,8 @@ static void test_av_flows(void)
tox_kill(alice); tox_kill(alice);
tox_kill(bootstrap); tox_kill(bootstrap);
pthread_mutex_destroy(&time_data.lock);
printf("\nTest successful!\n"); printf("\nTest successful!\n");
} }

View File

@@ -1,5 +1,5 @@
pool: pool:
vmImage: "windows-2019" vmImage: "windows-2022"
jobs: jobs:
- job: "vcpkg" - job: "vcpkg"
strategy: strategy:

View File

@@ -1,4 +1,5 @@
load("@rules_cc//cc:defs.bzl", "cc_test") load("@rules_cc//cc:defs.bzl", "cc_test")
load("@rules_fuzzing//fuzzing:cc_defs.bzl", "cc_fuzz_test")
load("//tools:no_undefined.bzl", "cc_library") load("//tools:no_undefined.bzl", "cc_library")
exports_files( exports_files(
@@ -42,17 +43,179 @@ cc_library(
) )
cc_library( cc_library(
name = "toxav", name = "rtp",
srcs = glob( srcs = ["rtp.c"],
[ hdrs = ["rtp.h"],
"*.c", visibility = ["//c-toxcore:__subpackages__"],
"*.h", deps = [
"//c-toxcore/toxcore:ccompat",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:net_crypto",
"//c-toxcore/toxcore:network",
"//c-toxcore/toxcore:util",
"@libsodium",
],
)
cc_test(
name = "rtp_test",
size = "small",
srcs = ["rtp_test.cc"],
deps = [
":rtp",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:net_crypto",
"//c-toxcore/toxcore:os_memory",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
cc_fuzz_test(
name = "rtp_fuzz_test",
size = "small",
srcs = ["rtp_fuzz_test.cc"],
copts = ["-UNDEBUG"],
deps = [
":rtp",
"//c-toxcore/testing/fuzzing:fuzz_support",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:os_memory",
],
)
cc_library(
name = "bwcontroller",
srcs = ["bwcontroller.c"],
hdrs = ["bwcontroller.h"],
deps = [
":ring_buffer",
"//c-toxcore/toxcore:ccompat",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:network",
"//c-toxcore/toxcore:util",
],
)
cc_test(
name = "bwcontroller_test",
size = "small",
srcs = ["bwcontroller_test.cc"],
deps = [
":bwcontroller",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:os_memory",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
cc_library(
name = "audio",
srcs = ["audio.c"],
hdrs = ["audio.h"],
deps = [
":ring_buffer",
":rtp",
"//c-toxcore/toxcore:ccompat",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:network",
"//c-toxcore/toxcore:util",
"@opus",
],
)
cc_test(
name = "audio_test",
timeout = "moderate",
srcs = ["audio_test.cc"],
deps = [
":audio",
":rtp",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:os_memory",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
cc_library(
name = "video",
srcs = ["video.c"],
hdrs = ["video.h"],
deps = [
":ring_buffer",
":rtp",
"//c-toxcore/toxcore:ccompat",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:util",
"@libvpx",
],
)
cc_test(
name = "video_test",
timeout = "moderate",
srcs = ["video_test.cc"],
deps = [
":rtp",
":video",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:mono_time",
"//c-toxcore/toxcore:os_memory",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
cc_library(
name = "msi",
srcs = ["msi.c"],
hdrs = ["msi.h"],
visibility = ["//c-toxcore:__subpackages__"],
deps = [
"//c-toxcore/toxcore:ccompat",
"//c-toxcore/toxcore:logger",
"//c-toxcore/toxcore:util",
],
)
cc_test(
name = "msi_test",
size = "small",
srcs = ["msi_test.cc"],
deps = [
":msi",
"//c-toxcore/toxcore:os_memory",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
cc_library(
name = "toxav",
srcs = [
"groupav.c",
"groupav.h",
"toxav.c",
"toxav_old.c",
], ],
exclude = ["toxav.h"],
),
hdrs = ["toxav.h"], hdrs = ["toxav.h"],
visibility = ["//c-toxcore:__subpackages__"], visibility = ["//c-toxcore:__subpackages__"],
deps = [ deps = [
":audio",
":bwcontroller",
":msi",
":rtp",
":video",
"//c-toxcore/toxcore:Messenger", "//c-toxcore/toxcore:Messenger",
"//c-toxcore/toxcore:ccompat", "//c-toxcore/toxcore:ccompat",
"//c-toxcore/toxcore:group", "//c-toxcore/toxcore:group",
@@ -63,24 +226,10 @@ cc_library(
"//c-toxcore/toxcore:tox", "//c-toxcore/toxcore:tox",
"//c-toxcore/toxcore:util", "//c-toxcore/toxcore:util",
"@libsodium", "@libsodium",
"@libvpx",
"@opus", "@opus",
], ],
) )
cc_test(
name = "rtp_test",
size = "small",
srcs = ["rtp_test.cc"],
deps = [
":toxav",
"//c-toxcore/toxcore:crypto_core",
"//c-toxcore/toxcore:os_random",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
sh_library( sh_library(
name = "cimple_files", name = "cimple_files",
srcs = glob([ srcs = glob([

View File

@@ -19,7 +19,6 @@ libtoxav_la_SOURCES = ../toxav/rtp.h \
../toxav/ring_buffer.h \ ../toxav/ring_buffer.h \
../toxav/ring_buffer.c \ ../toxav/ring_buffer.c \
../toxav/toxav.h \ ../toxav/toxav.h \
../toxav/toxav_hacks.h \
../toxav/toxav.c \ ../toxav/toxav.c \
../toxav/toxav_old.c ../toxav/toxav_old.c

View File

@@ -5,6 +5,8 @@
#include "audio.h" #include "audio.h"
#include <assert.h> #include <assert.h>
#include <opus.h>
#include <pthread.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
@@ -14,6 +16,37 @@
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/mono_time.h" #include "../toxcore/mono_time.h"
#include "../toxcore/network.h" #include "../toxcore/network.h"
#include "../toxcore/util.h"
struct ACSession {
Mono_Time *mono_time;
const Logger *log;
/* encoding */
OpusEncoder *encoder;
uint32_t le_sample_rate; /* Last encoder sample rate */
uint8_t le_channel_count; /* Last encoder channel count */
uint32_t le_bit_rate; /* Last encoder bit rate */
/* decoding */
OpusDecoder *decoder;
uint8_t lp_channel_count; /* Last packet channel count */
uint32_t lp_sampling_rate; /* Last packet sample rate */
uint32_t lp_frame_duration; /* Last packet frame duration */
uint32_t ld_sample_rate; /* Last decoder sample rate */
uint8_t ld_channel_count; /* Last decoder channel count */
uint64_t ldrts; /* Last decoder reconfiguration time stamp */
void *j_buf;
pthread_mutex_t queue_mutex[1];
uint32_t friend_number;
/* Audio frame receive callback */
ac_audio_receive_frame_cb *acb;
void *user_data;
};
static struct JitterBuffer *jbuf_new(uint32_t capacity); static struct JitterBuffer *jbuf_new(uint32_t capacity);
static void jbuf_clear(struct JitterBuffer *q); static void jbuf_clear(struct JitterBuffer *q);
@@ -28,8 +61,8 @@ static bool reconfigure_audio_decoder(ACSession *ac, uint32_t sampling_rate, uin
ACSession *ac_new(Mono_Time *mono_time, const Logger *log, ToxAV *av, uint32_t friend_number, ACSession *ac_new(Mono_Time *mono_time, const Logger *log, uint32_t friend_number,
toxav_audio_receive_frame_cb *cb, void *cb_data) ac_audio_receive_frame_cb *cb, void *user_data)
{ {
ACSession *ac = (ACSession *)calloc(1, sizeof(ACSession)); ACSession *ac = (ACSession *)calloc(1, sizeof(ACSession));
@@ -84,10 +117,9 @@ ACSession *ac_new(Mono_Time *mono_time, const Logger *log, ToxAV *av, uint32_t f
ac->lp_sampling_rate = AUDIO_DECODER_START_SAMPLE_RATE; ac->lp_sampling_rate = AUDIO_DECODER_START_SAMPLE_RATE;
ac->lp_channel_count = AUDIO_DECODER_START_CHANNEL_COUNT; ac->lp_channel_count = AUDIO_DECODER_START_CHANNEL_COUNT;
ac->av = av;
ac->friend_number = friend_number; ac->friend_number = friend_number;
ac->acb = cb; ac->acb = cb;
ac->acb_user_data = cb_data; ac->user_data = user_data;
return ac; return ac;
@@ -132,37 +164,73 @@ void ac_iterate(ACSession *ac)
return; return;
} }
int rc = 0;
pthread_mutex_lock(ac->queue_mutex); pthread_mutex_lock(ac->queue_mutex);
struct JitterBuffer *const j_buf = (struct JitterBuffer *)ac->j_buf; struct JitterBuffer *const j_buf = (struct JitterBuffer *)ac->j_buf;
int rc = 0; while (true) {
struct RTPMessage *msg = jbuf_read(j_buf, &rc);
if (msg == nullptr && rc != 2) {
break;
}
for (struct RTPMessage *msg = jbuf_read(j_buf, &rc); msg != nullptr || rc == 2; msg = jbuf_read(j_buf, &rc)) {
pthread_mutex_unlock(ac->queue_mutex); pthread_mutex_unlock(ac->queue_mutex);
if (rc == 2) { if (rc == 2) {
/* Packet Loss Concealment (PLC) */
LOGGER_DEBUG(ac->log, "OPUS correction"); LOGGER_DEBUG(ac->log, "OPUS correction");
const int fs = (ac->lp_sampling_rate * ac->lp_frame_duration) / 1000;
rc = opus_decode(ac->decoder, nullptr, 0, temp_audio_buffer, fs, 1); /* Use safe defaults or last known good values */
const uint32_t sampling_rate = ac->lp_sampling_rate;
const uint32_t frame_duration = ac->lp_frame_duration;
if (sampling_rate == 0 || sampling_rate > AUDIO_MAX_SAMPLE_RATE || frame_duration > AUDIO_MAX_FRAME_DURATION_MS) {
LOGGER_WARNING(ac->log, "Invalid PLC parameters: sr %u, dur %u", sampling_rate, frame_duration);
} else { } else {
assert(msg->len > 4); const int fs = (sampling_rate * frame_duration) / 1000;
rc = opus_decode(ac->decoder, nullptr, 0, temp_audio_buffer, fs, 1);
}
} else {
const uint8_t *msg_data = rtp_message_data(msg);
const uint32_t msg_length = rtp_message_len(msg);
if (msg_length <= 4) {
LOGGER_WARNING(ac->log, "Packet too short: %u", msg_length);
free(msg);
pthread_mutex_lock(ac->queue_mutex);
continue;
}
/* Pick up sampling rate from packet */ /* Pick up sampling rate from packet */
memcpy(&ac->lp_sampling_rate, msg->data, 4); uint32_t sampling_rate;
ac->lp_sampling_rate = net_ntohl(ac->lp_sampling_rate); memcpy(&sampling_rate, msg_data, 4);
sampling_rate = net_ntohl(sampling_rate);
ac->lp_channel_count = opus_packet_get_nb_channels(msg->data + 4); const int channels = opus_packet_get_nb_channels(msg_data + 4);
if (channels < 1 || channels > AUDIO_MAX_CHANNEL_COUNT ||
sampling_rate == 0 || sampling_rate > AUDIO_MAX_SAMPLE_RATE) {
LOGGER_WARNING(ac->log, "Invalid packet parameters: sr %u, cc %d", sampling_rate, channels);
free(msg);
pthread_mutex_lock(ac->queue_mutex);
continue;
}
/** NOTE: even though OPUS supports decoding mono frames with stereo decoder and vice versa, /** NOTE: even though OPUS supports decoding mono frames with stereo decoder and vice versa,
* it didn't work quite well. * it didn't work quite well.
*/ */
if (!reconfigure_audio_decoder(ac, ac->lp_sampling_rate, ac->lp_channel_count)) { if (!reconfigure_audio_decoder(ac, sampling_rate, (uint8_t)channels)) {
LOGGER_WARNING(ac->log, "Failed to reconfigure decoder!"); LOGGER_WARNING(ac->log, "Failed to reconfigure decoder!");
free(msg); free(msg);
pthread_mutex_lock(ac->queue_mutex); pthread_mutex_lock(ac->queue_mutex);
continue; continue;
} }
ac->lp_sampling_rate = sampling_rate;
ac->lp_channel_count = (uint8_t)channels;
/* /*
* frame_size = opus_decode(dec, packet, len, decoded, max_size, 0); * frame_size = opus_decode(dec, packet, len, decoded, max_size, 0);
* where * where
@@ -172,7 +240,7 @@ void ac_iterate(ACSession *ac)
* max_size is the max duration of the frame in samples (per channel) that can fit * max_size is the max duration of the frame in samples (per channel) that can fit
* into the decoded_frame array * into the decoded_frame array
*/ */
rc = opus_decode(ac->decoder, msg->data + 4, msg->len - 4, temp_audio_buffer, 5760, 0); rc = opus_decode(ac->decoder, msg_data + 4, msg_length - 4, temp_audio_buffer, AUDIO_MAX_BUFFER_SIZE_PCM16, 0);
free(msg); free(msg);
} }
@@ -181,13 +249,11 @@ void ac_iterate(ACSession *ac)
} else if (ac->acb != nullptr) { } else if (ac->acb != nullptr) {
ac->lp_frame_duration = (rc * 1000) / ac->lp_sampling_rate; ac->lp_frame_duration = (rc * 1000) / ac->lp_sampling_rate;
ac->acb(ac->av, ac->friend_number, temp_audio_buffer, rc, ac->lp_channel_count, ac->acb(ac->friend_number, temp_audio_buffer, (size_t)rc, ac->lp_channel_count,
ac->lp_sampling_rate, ac->acb_user_data); ac->lp_sampling_rate, ac->user_data);
} }
free(temp_audio_buffer); pthread_mutex_lock(ac->queue_mutex);
return;
} }
pthread_mutex_unlock(ac->queue_mutex); pthread_mutex_unlock(ac->queue_mutex);
@@ -204,13 +270,13 @@ int ac_queue_message(const Mono_Time *mono_time, void *cs, struct RTPMessage *ms
return -1; return -1;
} }
if ((msg->header.pt & 0x7f) == (RTP_TYPE_AUDIO + 2) % 128) { if ((rtp_message_pt(msg) & 0x7f) == (RTP_TYPE_AUDIO + 2) % 128) {
LOGGER_WARNING(ac->log, "Got dummy!"); LOGGER_WARNING(ac->log, "Got dummy!");
free(msg); free(msg);
return 0; return 0;
} }
if ((msg->header.pt & 0x7f) != RTP_TYPE_AUDIO % 128) { if ((rtp_message_pt(msg) & 0x7f) != RTP_TYPE_AUDIO % 128) {
LOGGER_WARNING(ac->log, "Invalid payload type!"); LOGGER_WARNING(ac->log, "Invalid payload type!");
free(msg); free(msg);
return -1; return -1;
@@ -243,6 +309,22 @@ int ac_reconfigure_encoder(ACSession *ac, uint32_t bit_rate, uint32_t sampling_r
return 0; return 0;
} }
uint32_t ac_get_lp_frame_duration(const ACSession *ac)
{
return ac->lp_frame_duration;
}
int ac_encode(ACSession *ac, const int16_t *pcm, size_t sample_count, uint8_t *dest, size_t dest_max)
{
const int vrc = opus_encode(ac->encoder, pcm, (int)sample_count, dest, (int)dest_max);
if (vrc < 0) {
LOGGER_WARNING(ac->log, "Failed to encode frame %s", opus_strerror(vrc));
}
return vrc;
}
struct JitterBuffer { struct JitterBuffer {
struct RTPMessage **queue; struct RTPMessage **queue;
uint32_t size; uint32_t size;
@@ -303,11 +385,17 @@ static void jbuf_free(struct JitterBuffer *q)
*/ */
static int jbuf_write(const Logger *log, struct JitterBuffer *q, struct RTPMessage *m) static int jbuf_write(const Logger *log, struct JitterBuffer *q, struct RTPMessage *m)
{ {
const uint16_t sequnum = m->header.sequnum; const uint16_t sequnum = rtp_message_sequnum(m);
const unsigned int num = sequnum % q->size; const unsigned int num = sequnum % q->size;
if ((uint32_t)(sequnum - q->bottom) > q->size) { const int16_t diff = (int16_t)(sequnum - q->bottom);
if (diff < 0) {
return -1;
}
if (diff > (int32_t)q->size) {
LOGGER_DEBUG(log, "Clearing filled jitter buffer: %p", (void *)q); LOGGER_DEBUG(log, "Clearing filled jitter buffer: %p", (void *)q);
jbuf_clear(q); jbuf_clear(q);
@@ -347,7 +435,7 @@ static struct RTPMessage *jbuf_read(struct JitterBuffer *q, int32_t *success)
return ret; return ret;
} }
if ((uint32_t)(q->top - q->bottom) > q->capacity) { if ((uint16_t)(q->top - q->bottom) > q->capacity) {
++q->bottom; ++q->bottom;
*success = 2; *success = 2;
return nullptr; return nullptr;

View File

@@ -5,14 +5,15 @@
#ifndef C_TOXCORE_TOXAV_AUDIO_H #ifndef C_TOXCORE_TOXAV_AUDIO_H
#define C_TOXCORE_TOXAV_AUDIO_H #define C_TOXCORE_TOXAV_AUDIO_H
#include <opus.h> #include <stdint.h>
#include <pthread.h> #include <stddef.h>
#include "toxav.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/util.h" #include "../toxcore/mono_time.h"
#include "rtp.h"
#ifdef __cplusplus
extern "C" {
#endif
#define AUDIO_JITTERBUFFER_COUNT 3 #define AUDIO_JITTERBUFFER_COUNT 3
#define AUDIO_MAX_SAMPLE_RATE 48000 #define AUDIO_MAX_SAMPLE_RATE 48000
@@ -34,40 +35,26 @@
#define AUDIO_MAX_BUFFER_SIZE_PCM16 ((AUDIO_MAX_SAMPLE_RATE * AUDIO_MAX_FRAME_DURATION_MS) / 1000) #define AUDIO_MAX_BUFFER_SIZE_PCM16 ((AUDIO_MAX_SAMPLE_RATE * AUDIO_MAX_FRAME_DURATION_MS) / 1000)
#define AUDIO_MAX_BUFFER_SIZE_BYTES (AUDIO_MAX_BUFFER_SIZE_PCM16 * 2) #define AUDIO_MAX_BUFFER_SIZE_BYTES (AUDIO_MAX_BUFFER_SIZE_PCM16 * 2)
typedef struct ACSession { typedef void ac_audio_receive_frame_cb(uint32_t friend_number, const int16_t *pcm, size_t sample_count,
Mono_Time *mono_time; uint8_t channels, uint32_t sampling_rate, void *user_data);
const Logger *log;
/* encoding */ typedef struct ACSession ACSession;
OpusEncoder *encoder;
uint32_t le_sample_rate; /* Last encoder sample rate */
uint8_t le_channel_count; /* Last encoder channel count */
uint32_t le_bit_rate; /* Last encoder bit rate */
/* decoding */ struct RTPMessage;
OpusDecoder *decoder;
uint8_t lp_channel_count; /* Last packet channel count */
uint32_t lp_sampling_rate; /* Last packet sample rate */
uint32_t lp_frame_duration; /* Last packet frame duration */
uint32_t ld_sample_rate; /* Last decoder sample rate */
uint8_t ld_channel_count; /* Last decoder channel count */
uint64_t ldrts; /* Last decoder reconfiguration time stamp */
void *j_buf;
pthread_mutex_t queue_mutex[1]; ACSession *ac_new(Mono_Time *mono_time, const Logger *log, uint32_t friend_number,
ac_audio_receive_frame_cb *cb, void *user_data);
ToxAV *av;
uint32_t friend_number;
/* Audio frame receive callback */
toxav_audio_receive_frame_cb *acb;
void *acb_user_data;
} ACSession;
ACSession *ac_new(Mono_Time *mono_time, const Logger *log, ToxAV *av, uint32_t friend_number,
toxav_audio_receive_frame_cb *cb, void *cb_data);
void ac_kill(ACSession *ac); void ac_kill(ACSession *ac);
void ac_iterate(ACSession *ac); void ac_iterate(ACSession *ac);
int ac_queue_message(const Mono_Time *mono_time, void *cs, struct RTPMessage *msg); int ac_queue_message(const Mono_Time *mono_time, void *cs, struct RTPMessage *msg);
int ac_reconfigure_encoder(ACSession *ac, uint32_t bit_rate, uint32_t sampling_rate, uint8_t channels); int ac_reconfigure_encoder(ACSession *ac, uint32_t bit_rate, uint32_t sampling_rate, uint8_t channels);
uint32_t ac_get_lp_frame_duration(const ACSession *ac);
int ac_encode(ACSession *ac, const int16_t *pcm, size_t sample_count, uint8_t *dest, size_t dest_max);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* C_TOXCORE_TOXAV_AUDIO_H */ #endif /* C_TOXCORE_TOXAV_AUDIO_H */

View File

@@ -0,0 +1,668 @@
#include "audio.h"
#include <gtest/gtest.h>
#include <algorithm>
#include <vector>
#include "../toxcore/logger.h"
#include "../toxcore/mono_time.h"
#include "../toxcore/network.h"
#include "../toxcore/os_memory.h"
#include "rtp.h"
namespace {
struct AudioTimeMock {
uint64_t t;
};
uint64_t audio_mock_time_cb(void *ud) { return static_cast<AudioTimeMock *>(ud)->t; }
struct AudioTestData {
uint32_t friend_number = 0;
std::vector<int16_t> last_pcm;
size_t sample_count = 0;
uint8_t channels = 0;
uint32_t sampling_rate = 0;
static void receive_frame(uint32_t friend_number, const int16_t *pcm, size_t sample_count,
uint8_t channels, uint32_t sampling_rate, void *user_data)
{
auto *self = static_cast<AudioTestData *>(user_data);
self->friend_number = friend_number;
self->last_pcm.assign(pcm, pcm + sample_count * channels);
self->sample_count = sample_count;
self->channels = channels;
self->sampling_rate = sampling_rate;
}
};
struct AudioRtpMock {
RTPSession *recv_session = nullptr;
std::vector<std::vector<uint8_t>> captured_packets;
bool auto_forward = true;
static int send_packet(void *user_data, const uint8_t *data, uint16_t length)
{
auto *self = static_cast<AudioRtpMock *>(user_data);
self->captured_packets.push_back(std::vector<uint8_t>(data, data + length));
if (self->auto_forward && self->recv_session) {
rtp_receive_packet(self->recv_session, data, length);
}
return 0;
}
static int audio_cb(const Mono_Time *mono_time, void *cs, RTPMessage *msg)
{
return ac_queue_message(mono_time, cs, msg);
}
};
class AudioTest : public ::testing::Test {
protected:
void SetUp() override
{
const Memory *mem = os_memory();
log = logger_new(mem);
tm.t = 1000;
mono_time = mono_time_new(mem, audio_mock_time_cb, &tm);
mono_time_update(mono_time);
}
void TearDown() override
{
const Memory *mem = os_memory();
mono_time_free(mem, mono_time);
logger_kill(log);
}
Logger *log;
Mono_Time *mono_time;
AudioTimeMock tm;
};
TEST_F(AudioTest, BasicNewKill)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
ac_kill(ac);
}
TEST_F(AudioTest, EncodeDecodeLoop)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint32_t sampling_rate = 48000;
uint8_t channels = 1;
size_t sample_count = 960; // 20ms at 48kHz
// Reconfigure to mono
ASSERT_EQ(ac_reconfigure_encoder(ac, 48000, sampling_rate, channels), 0);
std::vector<int16_t> pcm(sample_count * channels);
for (size_t i = 0; i < pcm.size(); ++i) {
pcm[i] = static_cast<int16_t>(i * 10);
}
std::vector<uint8_t> encoded(2000);
int encoded_size = ac_encode(ac, pcm.data(), sample_count, encoded.data(), encoded.size());
ASSERT_GT(encoded_size, 0);
// Prepare payload: 4 bytes sampling rate + Opus data
std::vector<uint8_t> payload(4 + static_cast<size_t>(encoded_size));
uint32_t net_sr = net_htonl(sampling_rate);
memcpy(payload.data(), &net_sr, 4);
memcpy(payload.data() + 4, encoded.data(), static_cast<size_t>(encoded_size));
// Send via RTP
int rc = rtp_send_data(
log, send_rtp, payload.data(), static_cast<uint32_t>(payload.size()), false);
ASSERT_EQ(rc, 0);
// Decode
ac_iterate(ac);
ASSERT_EQ(data.friend_number, 123u);
ASSERT_EQ(data.sample_count, sample_count);
ASSERT_EQ(data.channels, channels);
ASSERT_EQ(data.sampling_rate, sampling_rate);
ASSERT_EQ(data.last_pcm.size(), pcm.size());
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, ReconfigureEncoder)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
// Initial state: 48kHz, mono, 48kbps
// Change to 24kHz, stereo, 32kbps
int rc = ac_reconfigure_encoder(ac, 32000, 24000, 2);
ASSERT_EQ(rc, 0);
size_t sample_count = 480; // 20ms at 24kHz
uint8_t channels = 2;
std::vector<int16_t> pcm(sample_count * channels, 0);
std::vector<uint8_t> encoded(1000);
int encoded_size = ac_encode(ac, pcm.data(), sample_count, encoded.data(), encoded.size());
ASSERT_GT(encoded_size, 0);
ac_kill(ac);
}
TEST_F(AudioTest, GetFrameDuration)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
// Default duration in audio.c is 120ms (AUDIO_MAX_FRAME_DURATION_MS)
EXPECT_EQ(ac_get_lp_frame_duration(ac), 120u);
ac_kill(ac);
}
TEST_F(AudioTest, QueueInvalidMessage)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
// Create a video RTP session but try to queue to audio session
RTPSession *video_rtp = rtp_new(log, RTP_TYPE_VIDEO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *audio_recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = audio_recv_rtp;
std::vector<uint8_t> dummy_video(100, 0);
int rc = rtp_send_data(
log, video_rtp, dummy_video.data(), static_cast<uint32_t>(dummy_video.size()), true);
ASSERT_EQ(rc, 0);
// Iterate should NOT trigger callback because payload type was wrong
ac_iterate(ac);
EXPECT_EQ(data.sample_count, 0u);
rtp_kill(log, video_rtp);
rtp_kill(log, audio_recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, JitterBufferDuplicate)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr = net_htonl(48000);
memcpy(dummy_data, &net_sr, 4);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
ASSERT_EQ(rtp_mock.captured_packets.size(), 1u);
// Feed the same packet twice
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
// First iterate should process the packet
ac_iterate(ac);
EXPECT_GT(data.sample_count, 0u);
data.sample_count = 0;
// Second iterate should NOT process anything (duplicate was dropped in queue)
ac_iterate(ac);
EXPECT_EQ(data.sample_count, 0u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, JitterBufferOutOfOrder)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr = net_htonl(48000);
memcpy(dummy_data, &net_sr, 4);
// Capture 3 packets
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
ASSERT_EQ(rtp_mock.captured_packets.size(), 3u);
// Receive in order 0, 2, 1
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[2].data(), rtp_mock.captured_packets[2].size());
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[1].data(), rtp_mock.captured_packets[1].size());
// Iterate once, should process all 3 packets (because ac_iterate now loops)
data.sample_count = 0;
ac_iterate(ac);
EXPECT_GT(data.sample_count, 0u);
// Subsequent iterate should find nothing
data.sample_count = 0;
ac_iterate(ac);
EXPECT_EQ(data.sample_count, 0u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, PacketLossConcealment)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr = net_htonl(48000);
memcpy(dummy_data, &net_sr, 4);
// Send packet 0 and deliver it immediately.
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
ac_iterate(ac);
EXPECT_GT(data.sample_count, 0u);
data.sample_count = 0;
// Send packets 1 through 5 but do not deliver them, creating a gap in the sequence.
for (int i = 0; i < 5; ++i) {
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
}
// Send and deliver packet 6. The gap (1-5) exceeds the jitter buffer capacity (3).
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[6].data(), rtp_mock.captured_packets[6].size());
// The next iteration should trigger Packet Loss Concealment (PLC) for the missing packets.
// In audio.c, a return code of 2 from jbuf_read indicates that PLC should be performed.
ac_iterate(ac);
EXPECT_GT(data.sample_count, 0u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, JitterBufferReset)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr = net_htonl(48000);
memcpy(dummy_data, &net_sr, 4);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
ac_iterate(ac);
// The jitter buffer size is (capacity * 4) rounded up to the next power of 2.
// With AUDIO_JITTERBUFFER_COUNT = 3, the size is 16.
// A jump in sequence number greater than the buffer size triggers a full reset of the jitter
// buffer.
for (int i = 0; i < 20; ++i) {
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
}
// Deliver the latest packet, which is well beyond the current window.
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// The session should recover after the reset and process the new packet normally.
data.sample_count = 0;
ac_iterate(ac);
EXPECT_GT(data.sample_count, 0u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, DecoderReconfigureCooldown)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr_48 = net_htonl(48000);
uint32_t net_sr_24 = net_htonl(24000);
// 1. Reconfigure to 24kHz. The initial sampling rate is 48kHz.
memcpy(dummy_data, &net_sr_24, 4);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
ac_iterate(ac);
EXPECT_EQ(data.sampling_rate, 24000u);
data.sampling_rate = 0;
// 2. Advance time by only 100ms. This is less than the 500ms cooldown required for decoder
// reconfiguration.
tm.t += 100;
mono_time_update(mono_time);
// 3. Attempt to reconfigure back to 48kHz.
memcpy(dummy_data, &net_sr_48, 4);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// Reconfiguration should be rejected due to the cooldown, so the callback should not be
// invoked.
ac_iterate(ac);
EXPECT_EQ(data.sampling_rate, 0u);
// 4. Advance time beyond the 500ms cooldown period (measured from the first reconfiguration).
tm.t += 500;
mono_time_update(mono_time);
// 5. Attempt reconfiguration to 48kHz again.
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// Reconfiguration should now succeed.
ac_iterate(ac);
EXPECT_EQ(data.sampling_rate, 48000u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, QueueDummyMessage)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
// RTP_TYPE_AUDIO + 2 is the dummy type
RTPSession *dummy_rtp = rtp_new(log, RTP_TYPE_AUDIO + 2, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *audio_recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = audio_recv_rtp;
std::vector<uint8_t> dummy_payload(100, 0);
int rc = rtp_send_data(
log, dummy_rtp, dummy_payload.data(), static_cast<uint32_t>(dummy_payload.size()), false);
ASSERT_EQ(rc, 0);
// Iterate should NOT trigger callback because it was a dummy packet
ac_iterate(ac);
EXPECT_EQ(data.sample_count, 0u);
rtp_kill(log, dummy_rtp);
rtp_kill(log, audio_recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, LatePacketReset)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr = net_htonl(48000);
memcpy(dummy_data, &net_sr, 4);
// 1. Send and process the first packet.
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false); // seq 0
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
ac_iterate(ac);
ASSERT_GT(data.sample_count, 0u);
data.sample_count = 0;
// 2. Buffer another packet with a different sampling rate (24kHz) but don't process it yet.
uint32_t net_sr_24 = net_htonl(24000);
memcpy(dummy_data, &net_sr_24, 4);
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false); // seq 1
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[1].data(), rtp_mock.captured_packets[1].size());
// 3. Receive the late packet (seq 0) again.
// This triggers the bug: (uint32_t)(0 - 1) > 16, causing a full jitter buffer reset.
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets[0].data(), rtp_mock.captured_packets[0].size());
// 4. Try to process the next packet.
// Due to the bug, packet 1 was cleared. We will likely get PLC (48kHz) instead of packet 1
// (24kHz).
ac_iterate(ac);
// If the bug is present, sampling_rate will be 48000 (from PLC) instead of 24000.
EXPECT_EQ(data.sampling_rate, 24000u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, InvalidSamplingRate)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
// 1. Send a packet with an absurdly large sampling rate.
uint8_t malicious_data[100] = {0};
uint32_t net_sr = net_htonl(1000000000); // 1 GHz
memcpy(malicious_data, &net_sr, 4);
// Add some dummy Opus data so it's not too short
malicious_data[4] = 0x08;
rtp_send_data(log, send_rtp, malicious_data, sizeof(malicious_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// This packet should fail reconfiguration and be discarded.
ac_iterate(ac);
EXPECT_EQ(data.sample_count, 0u);
// 2. Trigger PLC. It should NOT use the malicious sampling rate.
// Send 5 packets to create a gap.
for (int i = 0; i < 5; ++i) {
rtp_send_data(log, send_rtp, malicious_data, sizeof(malicious_data), false);
}
// Deliver the next one.
rtp_send_data(log, send_rtp, malicious_data, sizeof(malicious_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// Next iterate triggers PLC. If it uses 1GHz, it might overflow/crash.
ac_iterate(ac);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, ShortPacket)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
// 1. Send a packet that is too short (only sampling rate, no Opus data).
// The protocol requires 4 bytes SR + at least 1 byte Opus data.
uint8_t short_data[4] = {0, 0, 0xBB, 0x80}; // 48000
// rtp_send_data might not like 4 bytes if it expects more, but let's see.
rtp_send_data(log, send_rtp, short_data, sizeof(short_data), false);
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// This should not crash. In debug it might hit an assert.
// In production it might do an OOB read.
ac_iterate(ac);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
TEST_F(AudioTest, JitterBufferWrapAround)
{
AudioTestData data;
ACSession *ac = ac_new(mono_time, log, 123, AudioTestData::receive_frame, &data);
ASSERT_NE(ac, nullptr);
AudioRtpMock rtp_mock;
rtp_mock.auto_forward = false;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, AudioRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, ac, AudioRtpMock::audio_cb);
rtp_mock.recv_session = recv_rtp;
uint8_t dummy_data[100] = {0};
uint32_t net_sr = net_htonl(48000);
memcpy(dummy_data, &net_sr, 4);
// Send enough packets to reach the sequence number wrap-around point (0xFFFF -> 0x0000).
// We detect the current sequence number to minimize the number of iterations.
uint16_t seq = 0;
{
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
const uint8_t *pkt = rtp_mock.captured_packets.back().data();
seq = (pkt[3] << 8) | pkt[4];
rtp_receive_packet(recv_rtp, pkt, rtp_mock.captured_packets.back().size());
rtp_mock.captured_packets.clear();
ac_iterate(ac);
}
// Aim for sequence number 65532 to be the last processed packet before the gap.
int to_send = (65532 - seq + 65536) % 65536;
for (int i = 0; i < to_send; ++i) {
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false);
const uint8_t *pkt = rtp_mock.captured_packets.back().data();
rtp_receive_packet(recv_rtp, pkt, rtp_mock.captured_packets.back().size());
rtp_mock.captured_packets.clear();
ac_iterate(ac);
}
// Now 'bottom' should be at 65533 (next expected).
data.sample_count = 0;
// Create a gap of 2 missing packets: 65533, 65534.
// Packet 65535 is delivered. Gap is 2. Capacity is 3. Should NOT trigger PLC.
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false); // 65533 (dropped)
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false); // 65534 (dropped)
rtp_send_data(log, send_rtp, dummy_data, sizeof(dummy_data), false); // 65535 (delivered)
rtp_receive_packet(
recv_rtp, rtp_mock.captured_packets.back().data(), rtp_mock.captured_packets.back().size());
// Iteration should result in no frames processed because the gap is within capacity.
// If there is a bug in wrap-around distance calculation, it will trigger PLC here.
ac_iterate(ac);
EXPECT_EQ(data.sample_count, 0u);
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
ac_kill(ac);
}
} // namespace

View File

@@ -10,17 +10,14 @@
#include <string.h> #include <string.h>
#include "ring_buffer.h" #include "ring_buffer.h"
#include "toxav_hacks.h"
#include "../toxcore/ccompat.h" #include "../toxcore/ccompat.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/mono_time.h" #include "../toxcore/mono_time.h"
#include "../toxcore/network.h" #include "../toxcore/network.h"
#include "../toxcore/tox_private.h"
#include "../toxcore/util.h" #include "../toxcore/util.h"
#define BWC_PACKET_ID 196
#define BWC_SEND_INTERVAL_MS 950 // 0.95s #define BWC_SEND_INTERVAL_MS 950 // 0.95s
#define BWC_AVG_PKT_COUNT 20 #define BWC_AVG_PKT_COUNT 20
#define BWC_AVG_LOSS_OVER_CYCLES_COUNT 30 #define BWC_AVG_LOSS_OVER_CYCLES_COUNT 30
@@ -40,9 +37,10 @@ typedef struct BWCRcvPkt {
} BWCRcvPkt; } BWCRcvPkt;
struct BWController { struct BWController {
m_cb *mcb; bwc_loss_report_cb *mcb;
void *mcb_user_data; void *mcb_user_data;
Tox *tox; bwc_send_packet_cb *send_packet;
void *send_packet_user_data;
const Logger *log; const Logger *log;
uint32_t friend_number; uint32_t friend_number;
@@ -60,11 +58,12 @@ struct BWCMessage {
uint32_t recv; uint32_t recv;
}; };
static void bwc_handle_data(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length, void *user_data);
static void send_update(BWController *bwc); static void send_update(BWController *bwc);
BWController *bwc_new(const Logger *log, Tox *tox, uint32_t friendnumber, m_cb *mcb, void *mcb_user_data, BWController *bwc_new(const Logger *log, uint32_t friendnumber,
bwc_loss_report_cb *mcb, void *mcb_user_data,
bwc_send_packet_cb *send_packet, void *send_packet_user_data,
Mono_Time *bwc_mono_time) Mono_Time *bwc_mono_time)
{ {
BWController *retu = (BWController *)calloc(1, sizeof(BWController)); BWController *retu = (BWController *)calloc(1, sizeof(BWController));
@@ -77,12 +76,13 @@ BWController *bwc_new(const Logger *log, Tox *tox, uint32_t friendnumber, m_cb *
retu->mcb = mcb; retu->mcb = mcb;
retu->mcb_user_data = mcb_user_data; retu->mcb_user_data = mcb_user_data;
retu->send_packet = send_packet;
retu->send_packet_user_data = send_packet_user_data;
retu->friend_number = friendnumber; retu->friend_number = friendnumber;
retu->bwc_mono_time = bwc_mono_time; retu->bwc_mono_time = bwc_mono_time;
const uint64_t now = current_time_monotonic(bwc_mono_time); const uint64_t now = current_time_monotonic(bwc_mono_time);
retu->cycle.last_sent_timestamp = now; retu->cycle.last_sent_timestamp = now;
retu->cycle.last_refresh_timestamp = now; retu->cycle.last_refresh_timestamp = now;
retu->tox = tox;
retu->log = log; retu->log = log;
retu->bwc_receive_active = true; retu->bwc_receive_active = true;
retu->rcvpkt.rb = rb_new(BWC_AVG_PKT_COUNT); retu->rcvpkt.rb = rb_new(BWC_AVG_PKT_COUNT);
@@ -141,7 +141,7 @@ static void send_update(BWController *bwc)
if (bwc->cycle.lost != 0) { if (bwc->cycle.lost != 0) {
LOGGER_DEBUG(bwc->log, "%p Sent update rcv: %u lost: %u percent: %f %%", LOGGER_DEBUG(bwc->log, "%p Sent update rcv: %u lost: %u percent: %f %%",
(void *)bwc, bwc->cycle.recv, bwc->cycle.lost, (void *)bwc, bwc->cycle.recv, bwc->cycle.lost,
((double)bwc->cycle.lost / (bwc->cycle.recv + bwc->cycle.lost)) * 100.0); ((double)bwc->cycle.lost / ((double)bwc->cycle.recv + (double)bwc->cycle.lost)) * 100.0);
uint8_t bwc_packet[sizeof(struct BWCMessage) + 1]; uint8_t bwc_packet[sizeof(struct BWCMessage) + 1];
size_t offset = 0; size_t offset = 0;
@@ -152,11 +152,8 @@ static void send_update(BWController *bwc)
offset += net_pack_u32(bwc_packet + offset, bwc->cycle.recv); offset += net_pack_u32(bwc_packet + offset, bwc->cycle.recv);
assert(offset == sizeof(bwc_packet)); assert(offset == sizeof(bwc_packet));
Tox_Err_Friend_Custom_Packet error; if (bwc->send_packet != nullptr && bwc->send_packet(bwc->send_packet_user_data, bwc_packet, sizeof(bwc_packet)) != 0) {
tox_friend_send_lossy_packet(bwc->tox, bwc->friend_number, bwc_packet, sizeof(bwc_packet), &error); LOGGER_WARNING(bwc->log, "BWC send failed");
if (error != TOX_ERR_FRIEND_CUSTOM_PACKET_OK) {
LOGGER_WARNING(bwc->log, "BWC send failed: %u", error);
} }
} }
@@ -171,7 +168,7 @@ static int on_update(BWController *bwc, const struct BWCMessage *msg)
LOGGER_DEBUG(bwc->log, "%p Got update from peer", (void *)bwc); LOGGER_DEBUG(bwc->log, "%p Got update from peer", (void *)bwc);
/* Peers sent update too soon */ /* Peers sent update too soon */
if (bwc->cycle.last_recv_timestamp + BWC_SEND_INTERVAL_MS > current_time_monotonic(bwc->bwc_mono_time)) { if (current_time_monotonic(bwc->bwc_mono_time) - bwc->cycle.last_recv_timestamp < BWC_SEND_INTERVAL_MS) {
LOGGER_INFO(bwc->log, "%p Rejecting extra update", (void *)bwc); LOGGER_INFO(bwc->log, "%p Rejecting extra update", (void *)bwc);
return -1; return -1;
} }
@@ -183,49 +180,28 @@ static int on_update(BWController *bwc, const struct BWCMessage *msg)
if (lost != 0 && bwc->mcb != nullptr) { if (lost != 0 && bwc->mcb != nullptr) {
const uint32_t recv = msg->recv; const uint32_t recv = msg->recv;
LOGGER_DEBUG(bwc->log, "recved: %u lost: %u percentage: %f %%", recv, lost, LOGGER_DEBUG(bwc->log, "recved: %u lost: %u percentage: %f %%", recv, lost,
((double) lost / (recv + lost)) * 100.0); ((double) lost / ((double)recv + (double)lost)) * 100.0);
bwc->mcb(bwc, bwc->friend_number, bwc->mcb(bwc, bwc->friend_number,
(float)lost / (recv + lost), (float)((double)lost / ((double)recv + (double)lost)),
bwc->mcb_user_data); bwc->mcb_user_data);
} }
return 0; return 0;
} }
static void bwc_handle_data(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length, void *user_data) void bwc_handle_packet(BWController *bwc, const uint8_t *data, size_t length)
{ {
/* get BWController object from Tox and friend number */ if (bwc == nullptr) {
ToxAV *toxav = (ToxAV *)tox_get_av_object(tox);
if (toxav == nullptr) {
// LOGGER_ERROR(log, "Could not get ToxAV object from Tox");
return; return;
} }
const Logger *log = toxav_get_logger(toxav);
if (length - 1 != sizeof(struct BWCMessage)) { if (length - 1 != sizeof(struct BWCMessage)) {
LOGGER_ERROR(log, "Got BWCMessage of insufficient size."); LOGGER_ERROR(bwc->log, "Got BWCMessage of insufficient size.");
return;
}
const ToxAVCall *call = call_get(toxav, friend_number);
if (call == nullptr) {
LOGGER_ERROR(log, "Could not get ToxAVCall object from ToxAV.");
return;
}
/* get Call object from Tox and friend number */
BWController *bwc = bwc_controller_get(call);
if (bwc == nullptr) {
LOGGER_WARNING(log, "No session!");
return; return;
} }
if (!bwc->bwc_receive_active) { if (!bwc->bwc_receive_active) {
LOGGER_WARNING(log, "receiving not allowed!"); LOGGER_WARNING(bwc->log, "receiving not allowed!");
return; return;
} }
@@ -237,13 +213,3 @@ static void bwc_handle_data(Tox *tox, uint32_t friend_number, const uint8_t *dat
on_update(bwc, &msg); on_update(bwc, &msg);
} }
void bwc_allow_receiving(Tox *tox)
{
tox_callback_friend_lossy_packet_per_pktid(tox, bwc_handle_data, BWC_PACKET_ID);
}
void bwc_stop_receiving(Tox *tox)
{
tox_callback_friend_lossy_packet_per_pktid(tox, nullptr, BWC_PACKET_ID);
}

View File

@@ -6,23 +6,37 @@
#define C_TOXCORE_TOXAV_BWCONTROLLER_H #define C_TOXCORE_TOXAV_BWCONTROLLER_H
#include <stdint.h> #include <stdint.h>
#include <stddef.h>
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/mono_time.h" #include "../toxcore/mono_time.h"
#include "../toxcore/tox.h"
#ifdef __cplusplus
extern "C" {
#endif
#define BWC_PACKET_ID 196
typedef struct BWController BWController; typedef struct BWController BWController;
typedef void m_cb(BWController *bwc, uint32_t friend_number, float loss, void *user_data); typedef void bwc_loss_report_cb(BWController *bwc, uint32_t friend_number, float loss, void *user_data);
BWController *bwc_new(const Logger *log, Tox *tox, uint32_t friendnumber, typedef int bwc_send_packet_cb(void *user_data, const uint8_t *data, uint16_t length);
m_cb *mcb, void *mcb_user_data, Mono_Time *bwc_mono_time);
BWController *bwc_new(const Logger *log, uint32_t friendnumber,
bwc_loss_report_cb *mcb, void *mcb_user_data,
bwc_send_packet_cb *send_packet, void *send_packet_user_data,
Mono_Time *bwc_mono_time);
void bwc_kill(BWController *bwc); void bwc_kill(BWController *bwc);
void bwc_add_lost(BWController *bwc, uint32_t bytes_lost); void bwc_add_lost(BWController *bwc, uint32_t bytes_lost);
void bwc_add_recv(BWController *bwc, uint32_t recv_bytes); void bwc_add_recv(BWController *bwc, uint32_t recv_bytes);
void bwc_allow_receiving(Tox *tox);
void bwc_stop_receiving(Tox *tox); void bwc_handle_packet(BWController *bwc, const uint8_t *data, size_t length);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* C_TOXCORE_TOXAV_BWCONTROLLER_H */ #endif /* C_TOXCORE_TOXAV_BWCONTROLLER_H */

View File

@@ -0,0 +1,390 @@
#include "bwcontroller.h"
#include <gtest/gtest.h>
#include <cmath>
#include <vector>
#include "../toxcore/logger.h"
#include "../toxcore/mono_time.h"
#include "../toxcore/network.h"
#include "../toxcore/os_memory.h"
namespace {
struct BwcTimeMock {
uint64_t t;
};
uint64_t bwc_mock_time_cb(void *ud) { return static_cast<BwcTimeMock *>(ud)->t; }
struct MockBwcData {
std::vector<std::vector<uint8_t>> sent_packets;
std::vector<float> reported_losses;
uint32_t friend_number = 0;
static int send_packet(void *user_data, const uint8_t *data, uint16_t length)
{
auto *sd = static_cast<MockBwcData *>(user_data);
if (sd->fail_send) {
return -1;
}
sd->sent_packets.emplace_back(data, data + length);
return 0;
}
static void loss_report(
BWController * /*bwc*/, uint32_t friend_number, float loss, void *user_data)
{
auto *sd = static_cast<MockBwcData *>(user_data);
sd->friend_number = friend_number;
sd->reported_losses.push_back(loss);
}
bool fail_send = false;
};
class BwcTest : public ::testing::Test {
protected:
void SetUp() override
{
const Memory *mem = os_memory();
log = logger_new(mem);
tm.t = 1000;
mono_time = mono_time_new(mem, bwc_mock_time_cb, &tm);
mono_time_update(mono_time);
}
void TearDown() override
{
const Memory *mem = os_memory();
mono_time_free(mem, mono_time);
logger_kill(log);
}
Logger *log;
Mono_Time *mono_time;
BwcTimeMock tm;
};
TEST_F(BwcTest, BasicNewKill)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
ASSERT_NE(bwc, nullptr);
bwc_kill(bwc);
}
TEST_F(BwcTest, SendUpdate)
{
MockBwcData sd;
uint32_t friend_number = 123;
BWController *bwc = bwc_new(log, friend_number, MockBwcData::loss_report, &sd,
MockBwcData::send_packet, &sd, mono_time);
ASSERT_NE(bwc, nullptr);
// BWC_AVG_LOSS_OVER_CYCLES_COUNT is 30
// BWC_SEND_INTERVAL_MS is 950
// Add some received and lost bytes
for (int i = 0; i < 30; ++i) {
bwc_add_recv(bwc, 1000);
}
bwc_add_lost(bwc, 500);
// Should not have sent anything yet because interval not reached
EXPECT_EQ(sd.sent_packets.size(), 0);
// Advance time
tm.t += 1000;
mono_time_update(mono_time);
// Trigger another update
bwc_add_recv(bwc, 1000);
ASSERT_EQ(sd.sent_packets.size(), 1);
EXPECT_EQ(sd.sent_packets[0][0], BWC_PACKET_ID);
// Packet contains lost (4 bytes) and recv (4 bytes)
uint32_t lost, recv;
net_unpack_u32(sd.sent_packets[0].data() + 1, &lost);
net_unpack_u32(sd.sent_packets[0].data() + 5, &recv);
EXPECT_EQ(lost, 500);
EXPECT_EQ(recv, 30 * 1000 + 1000);
bwc_kill(bwc);
}
TEST_F(BwcTest, HandlePacket)
{
MockBwcData sd;
uint32_t friend_number = 123;
BWController *bwc = bwc_new(log, friend_number, MockBwcData::loss_report, &sd,
MockBwcData::send_packet, &sd, mono_time);
ASSERT_NE(bwc, nullptr);
uint8_t packet[9];
packet[0] = BWC_PACKET_ID;
net_pack_u32(packet + 1, 100); // lost
net_pack_u32(packet + 5, 900); // recv
bwc_handle_packet(bwc, packet, sizeof(packet));
ASSERT_EQ(sd.reported_losses.size(), 1);
EXPECT_EQ(sd.friend_number, friend_number);
EXPECT_FLOAT_EQ(sd.reported_losses[0], 100.0f / (100.0f + 900.0f));
// Try sending another update too soon
bwc_handle_packet(bwc, packet, sizeof(packet));
EXPECT_EQ(sd.reported_losses.size(), 1);
// Advance time
tm.t += 1000;
mono_time_update(mono_time);
bwc_handle_packet(bwc, packet, sizeof(packet));
EXPECT_EQ(sd.reported_losses.size(), 2);
bwc_kill(bwc);
}
TEST_F(BwcTest, NullArgs)
{
// These should just return without crashing
bwc_kill(nullptr);
bwc_add_lost(nullptr, 100);
bwc_add_recv(nullptr, 100);
bwc_handle_packet(nullptr, nullptr, 0);
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
bwc_add_recv(bwc, 0); // Should return early
bwc_add_lost(bwc, 0); // Should return early
bwc_kill(bwc);
}
TEST_F(BwcTest, InvalidPacketSize)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
uint8_t packet[10] = {0};
// Correct size is 9
bwc_handle_packet(bwc, packet, 8);
bwc_handle_packet(bwc, packet, 10);
EXPECT_EQ(sd.reported_losses.size(), 0);
bwc_kill(bwc);
}
TEST_F(BwcTest, SendFailure)
{
MockBwcData sd;
sd.fail_send = true;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
for (int i = 0; i < 31; ++i) {
bwc_add_recv(bwc, 1000);
}
bwc_add_lost(bwc, 500);
tm.t += 1000;
mono_time_update(mono_time);
bwc_add_recv(bwc, 1000);
// Send should have failed (logged, but doesn't crash)
EXPECT_EQ(sd.sent_packets.size(), 0);
bwc_kill(bwc);
}
TEST_F(BwcTest, NullCallback)
{
MockBwcData sd;
// Pass NULL for the loss report callback
BWController *bwc
= bwc_new(log, 123, nullptr, nullptr, MockBwcData::send_packet, &sd, mono_time);
uint8_t packet[9];
packet[0] = BWC_PACKET_ID;
net_pack_u32(packet + 1, 100); // lost
net_pack_u32(packet + 5, 900); // recv
bwc_handle_packet(bwc, packet, sizeof(packet));
// Should not crash, and no loss should be reported
EXPECT_EQ(sd.reported_losses.size(), 0);
bwc_kill(bwc);
}
TEST_F(BwcTest, ZeroLoss)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
// 1. Peer sends update with zero loss
uint8_t packet[9];
packet[0] = BWC_PACKET_ID;
net_pack_u32(packet + 1, 0); // lost
net_pack_u32(packet + 5, 1000); // recv
bwc_handle_packet(bwc, packet, sizeof(packet));
EXPECT_EQ(sd.reported_losses.size(), 0);
// 2. We have zero loss to report
for (int i = 0; i < 31; ++i) {
bwc_add_recv(bwc, 1000);
}
// No bwc_add_lost called
tm.t += 1000;
mono_time_update(mono_time);
bwc_add_recv(bwc, 1000);
// Should NOT send update if loss is 0
EXPECT_EQ(sd.sent_packets.size(), 0);
bwc_kill(bwc);
}
TEST_F(BwcTest, Overflow)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
// Set lost/recv to near max to check for overflow (though they are just added)
bwc_add_lost(bwc, 0xFFFFFFFF);
bwc_add_recv(bwc, 0xFFFFFFFF);
// Trigger update
for (int i = 0; i < 32; ++i) {
bwc_add_recv(bwc, 1);
}
tm.t += 1000;
mono_time_update(mono_time);
bwc_add_recv(bwc, 1);
ASSERT_EQ(sd.sent_packets.size(), 1);
uint32_t lost, recv;
net_unpack_u32(sd.sent_packets[0].data() + 1, &lost);
net_unpack_u32(sd.sent_packets[0].data() + 5, &recv);
// 0xFFFFFFFF + 32 + 1 should wrap to 32
EXPECT_EQ(lost, 0xFFFFFFFF);
EXPECT_EQ(recv, 32);
bwc_kill(bwc);
}
TEST_F(BwcTest, CycleCountThreshold)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
// BWC_AVG_LOSS_OVER_CYCLES_COUNT is 30.
// We need more than 30 cycles AND the time interval to pass.
tm.t += 2000; // Time is sufficient
mono_time_update(mono_time);
for (int i = 0; i < 30; ++i) {
bwc_add_recv(bwc, 100);
bwc_add_lost(bwc, 1);
ASSERT_EQ(sd.sent_packets.size(), 0) << "Should not send at cycle " << i;
}
// 31st call to bwc_add_recv (or bwc_add_lost)
bwc_add_recv(bwc, 100);
EXPECT_EQ(sd.sent_packets.size(), 1);
bwc_kill(bwc);
}
TEST_F(BwcTest, TimeIntervalStrict)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
// Enough cycles
for (int i = 0; i < 40; ++i) {
bwc_add_recv(bwc, 100);
}
bwc_add_lost(bwc, 10);
EXPECT_EQ(sd.sent_packets.size(), 0); // Time not advanced
// Advance just below 950ms
tm.t += 949;
mono_time_update(mono_time);
bwc_add_recv(bwc, 100);
EXPECT_EQ(sd.sent_packets.size(), 0);
// Advance to 951ms (Total > 950)
tm.t += 2;
mono_time_update(mono_time);
bwc_add_recv(bwc, 100);
EXPECT_EQ(sd.sent_packets.size(), 1);
bwc_kill(bwc);
}
TEST_F(BwcTest, RecvPlusLostOverflowBug)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
uint8_t packet[9];
packet[0] = BWC_PACKET_ID;
net_pack_u32(packet + 1, 1);
net_pack_u32(packet + 5, 0xFFFFFFFF);
bwc_handle_packet(bwc, packet, sizeof(packet));
ASSERT_EQ(sd.reported_losses.size(), 1);
// Loss should be very small, but if it's inf or > 1.0, it's a bug
EXPECT_LE(sd.reported_losses[0], 1.0f);
bwc_kill(bwc);
}
TEST_F(BwcTest, RateLimitBypassBug)
{
MockBwcData sd;
BWController *bwc = bwc_new(
log, 123, MockBwcData::loss_report, &sd, MockBwcData::send_packet, &sd, mono_time);
tm.t = 0xFFFFFFF0;
mono_time_update(mono_time);
uint8_t packet[9];
packet[0] = BWC_PACKET_ID;
net_pack_u32(packet + 1, 1);
net_pack_u32(packet + 5, 100);
bwc_handle_packet(bwc, packet, sizeof(packet));
EXPECT_EQ(sd.reported_losses.size(), 1);
// Only 5ms passed, should be rejected
tm.t = 0xFFFFFFF5;
mono_time_update(mono_time);
bwc_handle_packet(bwc, packet, sizeof(packet));
EXPECT_EQ(sd.reported_losses.size(), 1);
bwc_kill(bwc);
}
TEST_F(BwcTest, NoCrashOnNullSendPacket)
{
BWController *bwc = bwc_new(log, 123, nullptr, nullptr, nullptr, nullptr, mono_time);
for (int i = 0; i < 31; ++i) {
bwc_add_recv(bwc, 100);
}
bwc_add_lost(bwc, 10);
tm.t += 1000;
mono_time_update(mono_time);
// This should no longer crash
bwc_add_recv(bwc, 100);
bwc_kill(bwc);
}
} // namespace

View File

@@ -255,7 +255,7 @@ static Group_AV *new_group_av(const Logger *log, Tox *tox, Group_Chats *g_c, aud
return group_av; return group_av;
} }
static void group_av_peer_new(void *object, uint32_t conference_number, uint32_t peer_number) static void group_av_peer_new(void *object, Tox_Conference_Number conference_number, Tox_Conference_Peer_Number peer_number)
{ {
const Group_AV *group_av = (const Group_AV *)object; const Group_AV *group_av = (const Group_AV *)object;
Group_Peer_AV *peer_av = (Group_Peer_AV *)calloc(1, sizeof(Group_Peer_AV)); Group_Peer_AV *peer_av = (Group_Peer_AV *)calloc(1, sizeof(Group_Peer_AV));
@@ -272,7 +272,7 @@ static void group_av_peer_new(void *object, uint32_t conference_number, uint32_t
} }
} }
static void group_av_peer_delete(void *object, uint32_t conference_number, void *peer_object) static void group_av_peer_delete(void *object, Tox_Conference_Number conference_number, void *peer_object)
{ {
Group_Peer_AV *peer_av = (Group_Peer_AV *)peer_object; Group_Peer_AV *peer_av = (Group_Peer_AV *)peer_object;
@@ -288,7 +288,7 @@ static void group_av_peer_delete(void *object, uint32_t conference_number, void
free(peer_object); free(peer_object);
} }
static void group_av_groupchat_delete(void *object, uint32_t conference_number) static void group_av_groupchat_delete(void *object, Tox_Conference_Number conference_number)
{ {
Group_AV *group_av = (Group_AV *)object; Group_AV *group_av = (Group_AV *)object;
if (group_av != nullptr) { if (group_av != nullptr) {
@@ -296,8 +296,8 @@ static void group_av_groupchat_delete(void *object, uint32_t conference_number)
} }
} }
static int decode_audio_packet(Group_AV *group_av, Group_Peer_AV *peer_av, uint32_t conference_number, static int decode_audio_packet(Group_AV *group_av, Group_Peer_AV *peer_av, Tox_Conference_Number conference_number,
uint32_t peer_number) Tox_Conference_Peer_Number peer_number)
{ {
if (group_av == nullptr || peer_av == nullptr) { if (group_av == nullptr || peer_av == nullptr) {
return -1; return -1;
@@ -313,7 +313,7 @@ static int decode_audio_packet(Group_AV *group_av, Group_Peer_AV *peer_av, uint3
int16_t *out_audio = nullptr; int16_t *out_audio = nullptr;
int out_audio_samples = 0; int out_audio_samples = 0;
const unsigned int sample_rate = 48000; const uint32_t sample_rate = 48000;
if (success == 1) { if (success == 1) {
const int channels = opus_packet_get_nb_channels(pk->data); const int channels = opus_packet_get_nb_channels(pk->data);
@@ -363,7 +363,7 @@ static int decode_audio_packet(Group_AV *group_av, Group_Peer_AV *peer_av, uint3
return -1; return -1;
} }
peer_av->last_packet_samples = out_audio_samples; peer_av->last_packet_samples = (unsigned int)out_audio_samples;
} else { } else {
if (peer_av->audio_decoder == nullptr) { if (peer_av->audio_decoder == nullptr) {
return -1; return -1;
@@ -391,8 +391,8 @@ static int decode_audio_packet(Group_AV *group_av, Group_Peer_AV *peer_av, uint3
if (out_audio != nullptr) { if (out_audio != nullptr) {
if (group_av->audio_data != nullptr) { if (group_av->audio_data != nullptr) {
group_av->audio_data(group_av->tox, conference_number, peer_number, out_audio, out_audio_samples, group_av->audio_data(group_av->tox, conference_number, peer_number, out_audio, (uint32_t)out_audio_samples,
peer_av->decoder_channels, sample_rate, group_av->userdata); (uint8_t)peer_av->decoder_channels, sample_rate, group_av->userdata);
} }
free(out_audio); free(out_audio);
@@ -402,7 +402,7 @@ static int decode_audio_packet(Group_AV *group_av, Group_Peer_AV *peer_av, uint3
return -1; return -1;
} }
static int handle_group_audio_packet(void *object, uint32_t conference_number, uint32_t peer_number, void *peer_object, static int handle_group_audio_packet(void *object, Tox_Conference_Number conference_number, Tox_Conference_Peer_Number peer_number, void *peer_object,
const uint8_t *packet, uint16_t length) const uint8_t *packet, uint16_t length)
{ {
Group_AV *group_av = (Group_AV *)object; Group_AV *group_av = (Group_AV *)object;
@@ -447,7 +447,7 @@ static int handle_group_audio_packet(void *object, uint32_t conference_number, u
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int groupchat_enable_av(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t conference_number, int groupchat_enable_av(const Logger *log, Tox *tox, Group_Chats *g_c, Tox_Conference_Number conference_number,
audio_data_cb *audio_callback, void *userdata) audio_data_cb *audio_callback, void *userdata)
{ {
if (group_get_type(g_c, conference_number) != GROUPCHAT_TYPE_AV if (group_get_type(g_c, conference_number) != GROUPCHAT_TYPE_AV
@@ -489,7 +489,7 @@ int groupchat_enable_av(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int groupchat_disable_av(const Group_Chats *g_c, uint32_t conference_number) int groupchat_disable_av(const Group_Chats *g_c, Tox_Conference_Number conference_number)
{ {
if (group_get_type(g_c, conference_number) != GROUPCHAT_TYPE_AV) { if (group_get_type(g_c, conference_number) != GROUPCHAT_TYPE_AV) {
return -1; return -1;
@@ -526,7 +526,7 @@ int groupchat_disable_av(const Group_Chats *g_c, uint32_t conference_number)
} }
/** Return whether A/V is enabled in the conference. */ /** Return whether A/V is enabled in the conference. */
bool groupchat_av_enabled(const Group_Chats *g_c, uint32_t conference_number) bool groupchat_av_enabled(const Group_Chats *g_c, Tox_Conference_Number conference_number)
{ {
return group_get_object(g_c, conference_number) != nullptr; return group_get_object(g_c, conference_number) != nullptr;
} }
@@ -544,8 +544,8 @@ int add_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, audio_data_c
return -1; return -1;
} }
if (groupchat_enable_av(log, tox, g_c, conference_number, audio_callback, userdata) == -1) { if (groupchat_enable_av(log, tox, g_c, (Tox_Conference_Number)conference_number, audio_callback, userdata) == -1) {
del_groupchat(g_c, conference_number, true); del_groupchat(g_c, (Tox_Conference_Number)conference_number, true);
return -1; return -1;
} }
@@ -557,17 +557,17 @@ int add_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, audio_data_c
* @return conference number on success * @return conference number on success
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int join_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t peer_number, const uint8_t *data, int join_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, Tox_Friend_Number friend_number, const uint8_t *data,
uint16_t length, audio_data_cb *audio_callback, void *userdata) uint16_t length, audio_data_cb *audio_callback, void *userdata)
{ {
const int conference_number = join_groupchat(g_c, peer_number, GROUPCHAT_TYPE_AV, data, length); const int conference_number = join_groupchat(g_c, friend_number, GROUPCHAT_TYPE_AV, data, length);
if (conference_number == -1) { if (conference_number == -1) {
return -1; return -1;
} }
if (groupchat_enable_av(log, tox, g_c, conference_number, audio_callback, userdata) == -1) { if (groupchat_enable_av(log, tox, g_c, (Tox_Conference_Number)conference_number, audio_callback, userdata) == -1) {
del_groupchat(g_c, conference_number, true); del_groupchat(g_c, (Tox_Conference_Number)conference_number, true);
return -1; return -1;
} }
@@ -579,7 +579,7 @@ int join_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t pe
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
static int send_audio_packet(const Group_Chats *g_c, uint32_t conference_number, const uint8_t *packet, uint16_t length) static int send_audio_packet(const Group_Chats *g_c, Tox_Conference_Number conference_number, const uint8_t *packet, uint16_t length)
{ {
if (length == 0 || length > MAX_CRYPTO_DATA_SIZE - 1 - sizeof(uint16_t)) { if (length == 0 || length > MAX_CRYPTO_DATA_SIZE - 1 - sizeof(uint16_t)) {
return -1; return -1;
@@ -614,7 +614,7 @@ static int send_audio_packet(const Group_Chats *g_c, uint32_t conference_number,
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int group_send_audio(const Group_Chats *g_c, uint32_t conference_number, const int16_t *pcm, unsigned int samples, uint8_t channels, int group_send_audio(const Group_Chats *g_c, Tox_Conference_Number conference_number, const int16_t pcm[], uint32_t samples, uint8_t channels,
uint32_t sample_rate) uint32_t sample_rate)
{ {
Group_AV *group_av = (Group_AV *)group_get_object(g_c, conference_number); Group_AV *group_av = (Group_AV *)group_get_object(g_c, conference_number);

View File

@@ -14,9 +14,9 @@
#define GROUP_AUDIO_PACKET_ID 192 #define GROUP_AUDIO_PACKET_ID 192
// TODO(iphydf): Use this better typed one instead of the void-pointer one below. // TODO(iphydf): Use this better typed one instead of the void-pointer one below.
// typedef void audio_data_cb(Tox *tox, uint32_t conference_number, uint32_t peernumber, const int16_t *pcm, // typedef void audio_data_cb(Tox *tox, uint32_t conference_number, uint32_t peer_number, const int16_t *pcm,
// uint32_t samples, uint8_t channels, uint32_t sample_rate, void *userdata); // uint32_t samples, uint8_t channels, uint32_t sample_rate, void *userdata);
typedef void audio_data_cb(void *tox, uint32_t conference_number, uint32_t peernumber, const int16_t *pcm, typedef void audio_data_cb(void *tox, Tox_Conference_Number conference_number, Tox_Conference_Peer_Number peer_number, const int16_t pcm[],
uint32_t samples, uint8_t channels, uint32_t sample_rate, void *userdata); uint32_t samples, uint8_t channels, uint32_t sample_rate, void *userdata);
/** @brief Create and connect to a new toxav group. /** @brief Create and connect to a new toxav group.
@@ -31,7 +31,7 @@ int add_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, audio_data_c
* @return conference number on success * @return conference number on success
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int join_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t peer_number, const uint8_t *data, int join_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, Tox_Friend_Number friend_number, const uint8_t *data,
uint16_t length, audio_data_cb *audio_callback, void *userdata); uint16_t length, audio_data_cb *audio_callback, void *userdata);
/** @brief Send audio to the conference. /** @brief Send audio to the conference.
@@ -39,7 +39,8 @@ int join_av_groupchat(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t pe
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int group_send_audio(const Group_Chats *g_c, uint32_t conference_number, const int16_t *pcm, unsigned int samples, uint8_t channels, int group_send_audio(const Group_Chats *g_c, Tox_Conference_Number conference_number, const int16_t pcm[], uint32_t samples,
uint8_t channels,
uint32_t sample_rate); uint32_t sample_rate);
/** @brief Enable A/V in a conference. /** @brief Enable A/V in a conference.
@@ -47,7 +48,7 @@ int group_send_audio(const Group_Chats *g_c, uint32_t conference_number, const i
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int groupchat_enable_av(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t conference_number, int groupchat_enable_av(const Logger *log, Tox *tox, Group_Chats *g_c, Tox_Conference_Number conference_number,
audio_data_cb *audio_callback, void *userdata); audio_data_cb *audio_callback, void *userdata);
/** @brief Disable A/V in a conference. /** @brief Disable A/V in a conference.
@@ -55,9 +56,9 @@ int groupchat_enable_av(const Logger *log, Tox *tox, Group_Chats *g_c, uint32_t
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int groupchat_disable_av(const Group_Chats *g_c, uint32_t conference_number); int groupchat_disable_av(const Group_Chats *g_c, Tox_Conference_Number conference_number);
/** Return whether A/V is enabled in the conference. */ /** Return whether A/V is enabled in the conference. */
bool groupchat_av_enabled(const Group_Chats *g_c, uint32_t conference_number); bool groupchat_av_enabled(const Group_Chats *g_c, Tox_Conference_Number conference_number);
#endif /* C_TOXCORE_TOXAV_GROUPAV_H */ #endif /* C_TOXCORE_TOXAV_GROUPAV_H */

View File

@@ -9,13 +9,8 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include "toxav_hacks.h"
#include "../toxcore/ccompat.h" #include "../toxcore/ccompat.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/net_crypto.h"
#include "../toxcore/tox.h"
#include "../toxcore/tox_private.h"
#include "../toxcore/util.h" #include "../toxcore/util.h"
#define MSI_MAXMSG_SIZE 256 #define MSI_MAXMSG_SIZE 256
@@ -64,52 +59,22 @@ static void kill_call(const Logger *log, MSICall *call);
static int msg_parse_in(const Logger *log, MSIMessage *dest, const uint8_t *data, uint16_t length); static int msg_parse_in(const Logger *log, MSIMessage *dest, const uint8_t *data, uint16_t length);
static uint8_t *msg_parse_header_out(MSIHeaderID id, uint8_t *dest, const uint8_t *value, uint8_t value_len, static uint8_t *msg_parse_header_out(MSIHeaderID id, uint8_t *dest, const uint8_t *value, uint8_t value_len,
uint16_t *length); uint16_t *length);
static int send_message(const Logger *log, Tox *tox, uint32_t friend_number, const MSIMessage *msg); static int send_message(const Logger *log, MSISession *session, uint32_t friend_number, const MSIMessage *msg);
static int send_error(const Logger *log, Tox *tox, uint32_t friend_number, MSIError error); static int send_error(const Logger *log, MSISession *session, uint32_t friend_number, MSIError error);
static MSICall *get_call(MSISession *session, uint32_t friend_number); static MSICall *get_call(MSISession *session, uint32_t friend_number);
static MSICall *new_call(MSISession *session, uint32_t friend_number); static MSICall *new_call(MSISession *session, uint32_t friend_number);
static bool invoke_callback(const Logger *log, MSICall *call, MSICallbackID cb); static bool invoke_callback(const Logger *log, MSICall *call, MSICallbackID cb);
static void handle_init(const Logger *log, MSICall *call, const MSIMessage *msg); static void handle_init(const Logger *log, MSICall *call, const MSIMessage *msg);
static void handle_push(const Logger *log, MSICall *call, const MSIMessage *msg); static void handle_push(const Logger *log, MSICall *call, const MSIMessage *msg);
static void handle_pop(const Logger *log, MSICall *call, const MSIMessage *msg); static void handle_pop(const Logger *log, MSICall *call, const MSIMessage *msg);
static void handle_msi_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length,
void *user_data);
/* /*
* Public functions * Public functions
*/ */
void msi_callback_invite(MSISession *session, msi_action_cb *callback) MSISession *msi_new(const Logger *log, msi_send_packet_cb *send_packet, void *send_packet_user_data,
const MSICallbacks *callbacks, void *user_data)
{ {
session->invite_callback = callback;
}
void msi_callback_start(MSISession *session, msi_action_cb *callback)
{
session->start_callback = callback;
}
void msi_callback_end(MSISession *session, msi_action_cb *callback)
{
session->end_callback = callback;
}
void msi_callback_error(MSISession *session, msi_action_cb *callback)
{
session->error_callback = callback;
}
void msi_callback_peertimeout(MSISession *session, msi_action_cb *callback)
{
session->peertimeout_callback = callback;
}
void msi_callback_capabilities(MSISession *session, msi_action_cb *callback)
{
session->capabilities_callback = callback;
}
MSISession *msi_new(const Logger *log, Tox *tox)
{
if (tox == nullptr) {
return nullptr;
}
MSISession *retu = (MSISession *)calloc(1, sizeof(MSISession)); MSISession *retu = (MSISession *)calloc(1, sizeof(MSISession));
if (retu == nullptr) { if (retu == nullptr) {
@@ -123,25 +88,28 @@ MSISession *msi_new(const Logger *log, Tox *tox)
return nullptr; return nullptr;
} }
retu->tox = tox; retu->send_packet = send_packet;
retu->send_packet_user_data = send_packet_user_data;
retu->user_data = user_data;
// register callback retu->invite_callback = callbacks->invite;
tox_callback_friend_lossless_packet_per_pktid(tox, handle_msi_packet, PACKET_ID_MSI); retu->start_callback = callbacks->start;
retu->end_callback = callbacks->end;
retu->error_callback = callbacks->error;
retu->peertimeout_callback = callbacks->peertimeout;
retu->capabilities_callback = callbacks->capabilities;
LOGGER_DEBUG(log, "New msi session: %p ", (void *)retu); LOGGER_DEBUG(log, "New msi session: %p ", (void *)retu);
return retu; return retu;
} }
int msi_kill(const Logger *log, Tox *tox, MSISession *session) int msi_kill(const Logger *log, MSISession *session)
{ {
if (session == nullptr) { if (session == nullptr) {
LOGGER_ERROR(log, "Tried to terminate non-existing session"); LOGGER_ERROR(log, "Tried to terminate non-existing session");
return -1; return -1;
} }
// UN-register callback
tox_callback_friend_lossless_packet_per_pktid(tox, nullptr, PACKET_ID_MSI);
if (pthread_mutex_trylock(session->mutex) != 0) { if (pthread_mutex_trylock(session->mutex) != 0) {
LOGGER_ERROR(log, "Failed to acquire lock on msi mutex"); LOGGER_ERROR(log, "Failed to acquire lock on msi mutex");
return -1; return -1;
@@ -154,7 +122,7 @@ int msi_kill(const Logger *log, Tox *tox, MSISession *session)
MSICall *it = get_call(session, session->calls_head); MSICall *it = get_call(session, session->calls_head);
while (it != nullptr) { while (it != nullptr) {
send_message(log, session->tox, it->friend_number, &msg); send_message(log, session, it->friend_number, &msg);
MSICall *temp_it = it; MSICall *temp_it = it;
it = it->next; it = it->next;
kill_call(log, temp_it); /* This will eventually free session->calls */ kill_call(log, temp_it); /* This will eventually free session->calls */
@@ -169,37 +137,23 @@ int msi_kill(const Logger *log, Tox *tox, MSISession *session)
return 0; return 0;
} }
/* void msi_call_timeout(MSISession *session, const Logger *log, uint32_t friend_number)
* return true if friend is offline and the call was canceled.
*/
bool check_peer_offline_status(const Logger *log, const Tox *tox, MSISession *session, uint32_t friend_number)
{ {
if (tox == nullptr || session == nullptr) { if (session == nullptr) {
return false; return;
} }
Tox_Err_Friend_Query f_con_query_error;
const Tox_Connection f_con_status = tox_friend_get_connection_status(tox, friend_number, &f_con_query_error);
if (f_con_status == TOX_CONNECTION_NONE) {
/* Friend is now offline */
LOGGER_DEBUG(log, "Friend %u is now offline", friend_number);
pthread_mutex_lock(session->mutex); pthread_mutex_lock(session->mutex);
MSICall *call = get_call(session, friend_number); MSICall *call = get_call(session, friend_number);
if (call == nullptr) { if (call == nullptr) {
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
return true; return;
} }
invoke_callback(log, call, MSI_ON_PEERTIMEOUT); /* Failure is ignored */ invoke_callback(log, call, MSI_ON_PEERTIMEOUT); /* Failure is ignored */
kill_call(log, call); kill_call(log, call);
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
return true;
}
return false;
} }
int msi_invite(const Logger *log, MSISession *session, MSICall **call, uint32_t friend_number, uint8_t capabilities) int msi_invite(const Logger *log, MSISession *session, MSICall **call, uint32_t friend_number, uint8_t capabilities)
@@ -238,7 +192,7 @@ int msi_invite(const Logger *log, MSISession *session, MSICall **call, uint32_t
msg.capabilities.exists = true; msg.capabilities.exists = true;
msg.capabilities.value = capabilities; msg.capabilities.value = capabilities;
send_message(log, temp->session->tox, temp->friend_number, &msg); send_message(log, session, temp->friend_number, &msg);
temp->state = MSI_CALL_REQUESTING; temp->state = MSI_CALL_REQUESTING;
@@ -274,7 +228,7 @@ int msi_hangup(const Logger *log, MSICall *call)
MSIMessage msg; MSIMessage msg;
msg_init(&msg, REQU_POP); msg_init(&msg, REQU_POP);
send_message(log, session->tox, call->friend_number, &msg); send_message(log, session, call->friend_number, &msg);
kill_call(log, call); kill_call(log, call);
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
@@ -313,7 +267,7 @@ int msi_answer(const Logger *log, MSICall *call, uint8_t capabilities)
msg.capabilities.exists = true; msg.capabilities.exists = true;
msg.capabilities.value = capabilities; msg.capabilities.value = capabilities;
send_message(log, session->tox, call->friend_number, &msg); send_message(log, session, call->friend_number, &msg);
call->state = MSI_CALL_ACTIVE; call->state = MSI_CALL_ACTIVE;
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
@@ -351,7 +305,7 @@ int msi_change_capabilities(const Logger *log, MSICall *call, uint8_t capabiliti
msg.capabilities.exists = true; msg.capabilities.exists = true;
msg.capabilities.value = capabilities; msg.capabilities.value = capabilities;
send_message(log, call->session->tox, call->friend_number, &msg); send_message(log, session, call->friend_number, &msg);
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
return 0; return 0;
@@ -488,46 +442,8 @@ static uint8_t *msg_parse_header_out(MSIHeaderID id, uint8_t *dest, const uint8_
return dest + value_len; /* Set to next position ready to be written */ return dest + value_len; /* Set to next position ready to be written */
} }
/* Send an msi packet. static int send_message(const Logger *log, MSISession *session, uint32_t friend_number, const MSIMessage *msg)
*
* return 1 on success
* return 0 on failure
*/
static int m_msi_packet(Tox *tox, int32_t friendnumber, const uint8_t *data, uint16_t length)
{ {
// TODO(Zoff): make this better later! -------------------
/* we need to prepend 1 byte (packet id) to data
* do this without malloc, memcpy and free in the future
*/
const size_t length_new = (size_t)length + 1;
uint8_t *data_new = (uint8_t *)malloc(length_new);
if (data_new == nullptr) {
return 0;
}
data_new[0] = PACKET_ID_MSI;
if (length != 0) {
memcpy(data_new + 1, data, length);
}
Tox_Err_Friend_Custom_Packet error;
tox_friend_send_lossless_packet(tox, friendnumber, data_new, length_new, &error);
free(data_new);
if (error == TOX_ERR_FRIEND_CUSTOM_PACKET_OK) {
return 1;
}
return 0;
}
static int send_message(const Logger *log, Tox *tox, uint32_t friend_number, const MSIMessage *msg)
{
assert(tox != nullptr);
/* Parse and send message */ /* Parse and send message */
uint8_t parsed[MSI_MAXMSG_SIZE]; uint8_t parsed[MSI_MAXMSG_SIZE];
@@ -562,7 +478,7 @@ static int send_message(const Logger *log, Tox *tox, uint32_t friend_number, con
*it = 0; *it = 0;
++size; ++size;
if (m_msi_packet(tox, friend_number, parsed, size) == 1) { if (session->send_packet != nullptr && session->send_packet(session->send_packet_user_data, friend_number, parsed, size) == 0) {
LOGGER_DEBUG(log, "Sent message"); LOGGER_DEBUG(log, "Sent message");
return 0; return 0;
} }
@@ -570,10 +486,8 @@ static int send_message(const Logger *log, Tox *tox, uint32_t friend_number, con
return -1; return -1;
} }
static int send_error(const Logger *log, Tox *tox, uint32_t friend_number, MSIError error) static int send_error(const Logger *log, MSISession *session, uint32_t friend_number, MSIError error)
{ {
assert(tox != nullptr);
/* Send error message */ /* Send error message */
LOGGER_DEBUG(log, "Sending error: %u to friend: %u", error, friend_number); LOGGER_DEBUG(log, "Sending error: %u to friend: %u", error, friend_number);
@@ -583,7 +497,7 @@ static int send_error(const Logger *log, Tox *tox, uint32_t friend_number, MSIEr
msg.error.exists = true; msg.error.exists = true;
msg.error.value = error; msg.error.value = error;
send_message(log, tox, friend_number, &msg); send_message(log, session, friend_number, &msg);
return 0; return 0;
} }
@@ -594,22 +508,22 @@ static int invoke_callback_inner(const Logger *log, MSICall *call, MSICallbackID
switch (id) { switch (id) {
case MSI_ON_INVITE: case MSI_ON_INVITE:
return session->invite_callback(session->av, call); return session->invite_callback(session->user_data, call);
case MSI_ON_START: case MSI_ON_START:
return session->start_callback(session->av, call); return session->start_callback(session->user_data, call);
case MSI_ON_END: case MSI_ON_END:
return session->end_callback(session->av, call); return session->end_callback(session->user_data, call);
case MSI_ON_ERROR: case MSI_ON_ERROR:
return session->error_callback(session->av, call); return session->error_callback(session->user_data, call);
case MSI_ON_PEERTIMEOUT: case MSI_ON_PEERTIMEOUT:
return session->peertimeout_callback(session->av, call); return session->peertimeout_callback(session->user_data, call);
case MSI_ON_CAPABILITIES: case MSI_ON_CAPABILITIES:
return session->capabilities_callback(session->av, call); return session->capabilities_callback(session->user_data, call);
} }
LOGGER_FATAL(log, "invalid callback id: %u", id); LOGGER_FATAL(log, "invalid callback id: %u", id);
@@ -694,6 +608,20 @@ static MSICall *new_call(MSISession *session, uint32_t friend_number)
rc->next = session->calls[session->calls_head]; rc->next = session->calls[session->calls_head];
session->calls[session->calls_head]->prev = rc; session->calls[session->calls_head]->prev = rc;
session->calls_head = friend_number; session->calls_head = friend_number;
} else { /* Inserting in a hole */
uint32_t i = friend_number - 1;
while (session->calls[i] == nullptr) {
--i;
}
rc->prev = session->calls[i];
rc->next = session->calls[i]->next;
rc->prev->next = rc;
if (rc->next != nullptr) {
rc->next->prev = rc;
}
} }
session->calls[friend_number] = rc; session->calls[friend_number] = rc;
@@ -774,13 +702,23 @@ static bool try_handle_init(const Logger *log, MSICall *call, const MSIMessage *
LOGGER_INFO(log, "Friend is recalling us"); LOGGER_INFO(log, "Friend is recalling us");
if (call->peer_capabilities != msg->capabilities.value) {
LOGGER_INFO(log, "Friend is changing capabilities to: %u", msg->capabilities.value);
call->peer_capabilities = msg->capabilities.value;
if (!invoke_callback(log, call, MSI_ON_CAPABILITIES)) {
return false;
}
}
MSIMessage out_msg; MSIMessage out_msg;
msg_init(&out_msg, REQU_PUSH); msg_init(&out_msg, REQU_PUSH);
out_msg.capabilities.exists = true; out_msg.capabilities.exists = true;
out_msg.capabilities.value = call->self_capabilities; out_msg.capabilities.value = call->self_capabilities;
send_message(log, call->session->tox, call->friend_number, &out_msg); send_message(log, call->session, call->friend_number, &out_msg);
/* If peer changed capabilities during re-call they will /* If peer changed capabilities during re-call they will
* be handled accordingly during the next step * be handled accordingly during the next step
@@ -806,7 +744,7 @@ static void handle_init(const Logger *log, MSICall *call, const MSIMessage *msg)
"Session: %p Handling 'init' friend: %u", (void *)call->session, call->friend_number); "Session: %p Handling 'init' friend: %u", (void *)call->session, call->friend_number);
if (!try_handle_init(log, call, msg)) { if (!try_handle_init(log, call, msg)) {
send_error(log, call->session->tox, call->friend_number, call->error); send_error(log, call->session, call->friend_number, call->error);
kill_call(log, call); kill_call(log, call);
} }
} }
@@ -863,7 +801,7 @@ static void handle_push(const Logger *log, MSICall *call, const MSIMessage *msg)
return; return;
FAILURE: FAILURE:
send_error(log, call->session->tox, call->friend_number, call->error); send_error(log, call->session, call->friend_number, call->error);
kill_call(log, call); kill_call(log, call);
} }
@@ -913,41 +851,25 @@ static void handle_pop(const Logger *log, MSICall *call, const MSIMessage *msg)
kill_call(log, call); kill_call(log, call);
} }
static void handle_msi_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length, void msi_handle_packet(MSISession *session, const Logger *log, uint32_t friend_number, const uint8_t *data,
void *user_data) size_t length)
{ {
const ToxAV *toxav = (ToxAV *)tox_get_av_object(tox);
if (toxav == nullptr) {
return;
}
const Logger *log = toxav_get_logger(toxav);
if (length < 2) {
LOGGER_ERROR(log, "MSI packet is less than 2 bytes in size");
// we need more than the ID byte for MSI messages
return;
}
const uint16_t payload_length = (uint16_t)(length - 1);
// Zoff: do not show the first byte, its always "PACKET_ID_MSI"
const uint8_t *data_strip_id_byte = data + 1;
LOGGER_DEBUG(log, "Got msi message");
MSISession *session = tox_av_msi_get(toxav);
if (session == nullptr) { if (session == nullptr) {
return; return;
} }
if (length < 1) {
LOGGER_ERROR(log, "MSI packet is empty");
return;
}
LOGGER_DEBUG(log, "Got msi message");
MSIMessage msg; MSIMessage msg;
if (msg_parse_in(log, &msg, data_strip_id_byte, payload_length) == -1) { if (msg_parse_in(log, &msg, data, length) == -1) {
LOGGER_WARNING(log, "Error parsing message"); LOGGER_WARNING(log, "Error parsing message");
send_error(log, tox, friend_number, MSI_E_INVALID_MESSAGE); send_error(log, session, friend_number, MSI_E_INVALID_MESSAGE);
return; return;
} }
@@ -958,7 +880,7 @@ static void handle_msi_packet(Tox *tox, uint32_t friend_number, const uint8_t *d
if (call == nullptr) { if (call == nullptr) {
if (msg.request.value != REQU_INIT) { if (msg.request.value != REQU_INIT) {
send_error(log, tox, friend_number, MSI_E_STRAY_MESSAGE); send_error(log, session, friend_number, MSI_E_STRAY_MESSAGE);
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
return; return;
} }
@@ -966,7 +888,7 @@ static void handle_msi_packet(Tox *tox, uint32_t friend_number, const uint8_t *d
call = new_call(session, friend_number); call = new_call(session, friend_number);
if (call == nullptr) { if (call == nullptr) {
send_error(log, tox, friend_number, MSI_E_SYSTEM); send_error(log, session, friend_number, MSI_E_SYSTEM);
pthread_mutex_unlock(session->mutex); pthread_mutex_unlock(session->mutex);
return; return;
} }

View File

@@ -8,11 +8,12 @@
#include <pthread.h> #include <pthread.h>
#include <stdint.h> #include <stdint.h>
#include "audio.h"
#include "video.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#ifdef __cplusplus
extern "C" {
#endif
/** /**
* Error codes. * Error codes.
*/ */
@@ -72,7 +73,7 @@ typedef struct MSICall {
uint32_t friend_number; /* Index of this call in MSISession */ uint32_t friend_number; /* Index of this call in MSISession */
MSIError error; /* Last error */ MSIError error; /* Last error */
struct ToxAVCall *av_call; /* Pointer to av call handler */ void *user_data; /* Pointer to av call handler */
struct MSICall *next; struct MSICall *next;
struct MSICall *prev; struct MSICall *prev;
@@ -85,6 +86,25 @@ typedef struct MSICall {
*/ */
typedef int msi_action_cb(void *object, MSICall *call); typedef int msi_action_cb(void *object, MSICall *call);
/**
* Send packet callback.
*
* @return 0 on success, -1 on failure.
*/
typedef int msi_send_packet_cb(void *user_data, uint32_t friend_number, const uint8_t *data, size_t length);
/**
* MSI callbacks.
*/
typedef struct MSICallbacks {
msi_action_cb *_Nonnull invite;
msi_action_cb *_Nonnull start;
msi_action_cb *_Nonnull end;
msi_action_cb *_Nonnull error;
msi_action_cb *_Nonnull peertimeout;
msi_action_cb *_Nonnull capabilities;
} MSICallbacks;
/** /**
* Control session struct. Please do not modify outside msi.c * Control session struct. Please do not modify outside msi.c
*/ */
@@ -94,53 +114,63 @@ typedef struct MSISession {
uint32_t calls_tail; uint32_t calls_tail;
uint32_t calls_head; uint32_t calls_head;
void *av; void *user_data;
Tox *tox;
msi_send_packet_cb *send_packet;
void *send_packet_user_data;
pthread_mutex_t mutex[1]; pthread_mutex_t mutex[1];
msi_action_cb *invite_callback; msi_action_cb *_Nonnull invite_callback;
msi_action_cb *start_callback; msi_action_cb *_Nonnull start_callback;
msi_action_cb *end_callback; msi_action_cb *_Nonnull end_callback;
msi_action_cb *error_callback; msi_action_cb *_Nonnull error_callback;
msi_action_cb *peertimeout_callback; msi_action_cb *_Nonnull peertimeout_callback;
msi_action_cb *capabilities_callback; msi_action_cb *_Nonnull capabilities_callback;
} MSISession; } MSISession;
/** /**
* Start the control session. * Start the control session.
*/ */
MSISession *msi_new(const Logger *log, Tox *tox); MSISession *_Nullable msi_new(const Logger *_Nonnull log,
msi_send_packet_cb *_Nonnull send_packet, void *_Nullable send_packet_user_data,
const MSICallbacks *_Nonnull callbacks,
void *_Nullable user_data);
/** /**
* Terminate control session. NOTE: all calls will be freed * Terminate control session. NOTE: all calls will be freed
*/ */
int msi_kill(const Logger *log, Tox *tox, MSISession *session); int msi_kill(const Logger *_Nonnull log, MSISession *_Nullable session);
/**
* Callback setters.
*/
void msi_callback_invite(MSISession *session, msi_action_cb *callback);
void msi_callback_start(MSISession *session, msi_action_cb *callback);
void msi_callback_end(MSISession *session, msi_action_cb *callback);
void msi_callback_error(MSISession *session, msi_action_cb *callback);
void msi_callback_peertimeout(MSISession *session, msi_action_cb *callback);
void msi_callback_capabilities(MSISession *session, msi_action_cb *callback);
/** /**
* Send invite request to friend_number. * Send invite request to friend_number.
*/ */
int msi_invite(const Logger *log, MSISession *session, MSICall **call, uint32_t friend_number, uint8_t capabilities); int msi_invite(const Logger *_Nonnull log, MSISession *_Nonnull session, MSICall *_Nonnull *_Nonnull call,
uint32_t friend_number, uint8_t capabilities);
/** /**
* Hangup call. NOTE: `call` will be freed * Hangup call. NOTE: `call` will be freed
*/ */
int msi_hangup(const Logger *log, MSICall *call); int msi_hangup(const Logger *_Nonnull log, MSICall *_Nullable call);
/** /**
* Answer call request. * Answer call request.
*/ */
int msi_answer(const Logger *log, MSICall *call, uint8_t capabilities); int msi_answer(const Logger *_Nonnull log, MSICall *_Nullable call, uint8_t capabilities);
/** /**
* Change capabilities of the call. * Change capabilities of the call.
*/ */
int msi_change_capabilities(const Logger *log, MSICall *call, uint8_t capabilities); int msi_change_capabilities(const Logger *_Nonnull log, MSICall *_Nullable call, uint8_t capabilities);
bool check_peer_offline_status(const Logger *log, const Tox *tox, MSISession *session, uint32_t friend_number); /**
* Handle incoming MSI packet.
*/
void msi_handle_packet(MSISession *_Nullable session, const Logger *_Nonnull log, uint32_t friend_number,
const uint8_t *_Nonnull data, size_t length);
/**
* Mark a call as timed out.
*/
void msi_call_timeout(MSISession *_Nullable session, const Logger *_Nonnull log, uint32_t friend_number);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* C_TOXCORE_TOXAV_MSI_H */ #endif /* C_TOXCORE_TOXAV_MSI_H */

View File

@@ -0,0 +1,454 @@
/* SPDX-License-Identifier: GPL-3.0-or-later
* Copyright © 2016-2025 The TokTok team.
* Copyright © 2013-2015 Tox project.
*/
#include "msi.h"
#include <gtest/gtest.h>
#include <vector>
#include "../toxcore/logger.h"
#include "../toxcore/os_memory.h"
namespace {
struct MockMsi {
std::vector<std::vector<uint8_t>> sent_packets;
std::vector<uint32_t> sent_to_friends;
struct CallbackStats {
int invite = 0;
int start = 0;
int end = 0;
int error = 0;
int peertimeout = 0;
int capabilities = 0;
} stats;
MSICall *last_call = nullptr;
MSIError last_error = MSI_E_NONE;
static int send_packet(
void *user_data, uint32_t friend_number, const uint8_t *data, size_t length)
{
auto *self = static_cast<MockMsi *>(user_data);
self->sent_packets.emplace_back(data, data + length);
self->sent_to_friends.push_back(friend_number);
return 0;
}
static int on_invite(void *object, MSICall *call)
{
auto *self = static_cast<MockMsi *>(object);
self->stats.invite++;
self->last_call = call;
return 0;
}
static int on_start(void *object, MSICall *call)
{
auto *self = static_cast<MockMsi *>(object);
self->stats.start++;
self->last_call = call;
return 0;
}
static int on_end(void *object, MSICall *call)
{
auto *self = static_cast<MockMsi *>(object);
self->stats.end++;
self->last_call = call;
return 0;
}
static int on_error(void *object, MSICall *call)
{
auto *self = static_cast<MockMsi *>(object);
self->stats.error++;
self->last_call = call;
self->last_error = call->error;
return 0;
}
static int on_peertimeout(void *object, MSICall *call)
{
auto *self = static_cast<MockMsi *>(object);
self->stats.peertimeout++;
self->last_call = call;
return 0;
}
static int on_capabilities(void *object, MSICall *call)
{
auto *self = static_cast<MockMsi *>(object);
self->stats.capabilities++;
self->last_call = call;
return 0;
}
};
class MsiTest : public ::testing::Test {
protected:
void SetUp() override
{
const Memory *mem = os_memory();
log = logger_new(mem);
MSICallbacks callbacks = {MockMsi::on_invite, MockMsi::on_start, MockMsi::on_end,
MockMsi::on_error, MockMsi::on_peertimeout, MockMsi::on_capabilities};
session = msi_new(log, MockMsi::send_packet, &mock, &callbacks, &mock);
}
void TearDown() override
{
if (session) {
msi_kill(log, session);
}
logger_kill(log);
}
Logger *log;
MSISession *session = nullptr;
MockMsi mock;
};
TEST_F(MsiTest, BasicNewKill)
{
// setup/teardown handles it
}
TEST_F(MsiTest, Invite)
{
MSICall *call = nullptr;
uint32_t friend_number = 123;
uint8_t capabilities = MSI_CAP_S_AUDIO | MSI_CAP_R_AUDIO;
int rc = msi_invite(log, session, &call, friend_number, capabilities);
ASSERT_EQ(rc, 0);
ASSERT_NE(call, nullptr);
EXPECT_EQ(call->friend_number, friend_number);
EXPECT_EQ(call->self_capabilities, capabilities);
EXPECT_EQ(call->state, MSI_CALL_REQUESTING);
ASSERT_EQ(mock.sent_packets.size(), 1u);
EXPECT_EQ(mock.sent_to_friends[0], friend_number);
// Verify packet: |ID_REQUEST(1)| |len(1)| |REQU_INIT(0)| |ID_CAPABILITIES(3)| |len(1)| |caps|
// |0|
const auto &pkt = mock.sent_packets[0];
ASSERT_GE(pkt.size(), 7u);
EXPECT_EQ(pkt[0], 1); // ID_REQUEST
EXPECT_EQ(pkt[2], 0); // REQU_INIT
EXPECT_EQ(pkt.back(), 0);
}
TEST_F(MsiTest, HandleIncomingInvite)
{
uint32_t friend_number = 456;
uint8_t peer_caps = MSI_CAP_S_VIDEO | MSI_CAP_R_VIDEO;
// Craft invite packet
uint8_t invite_pkt[] = {
1, 1, 0, // ID_REQUEST, len 1, REQU_INIT
3, 1, peer_caps, // ID_CAPABILITIES, len 1, caps
0 // end
};
msi_handle_packet(session, log, friend_number, invite_pkt, sizeof(invite_pkt));
EXPECT_EQ(mock.stats.invite, 1);
ASSERT_NE(mock.last_call, nullptr);
EXPECT_EQ(mock.last_call->friend_number, friend_number);
EXPECT_EQ(mock.last_call->peer_capabilities, peer_caps);
EXPECT_EQ(mock.last_call->state, MSI_CALL_REQUESTED);
}
TEST_F(MsiTest, Answer)
{
// 1. Receive invite first
uint32_t friend_number = 456;
uint8_t peer_caps = MSI_CAP_S_VIDEO | MSI_CAP_R_VIDEO;
uint8_t invite_pkt[] = {1, 1, 0, 3, 1, peer_caps, 0};
msi_handle_packet(session, log, friend_number, invite_pkt, sizeof(invite_pkt));
MSICall *call = mock.last_call;
// 2. Answer it
uint8_t my_caps = MSI_CAP_S_AUDIO | MSI_CAP_R_AUDIO;
int rc = msi_answer(log, call, my_caps);
ASSERT_EQ(rc, 0);
EXPECT_EQ(call->state, MSI_CALL_ACTIVE);
EXPECT_EQ(call->self_capabilities, my_caps);
ASSERT_GE(mock.sent_packets.size(), 1u);
const auto &pkt = mock.sent_packets.back();
// REQU_PUSH (1)
EXPECT_EQ(pkt[0], 1);
EXPECT_EQ(pkt[2], 1); // REQU_PUSH
}
TEST_F(MsiTest, Hangup)
{
MSICall *call = nullptr;
msi_invite(log, session, &call, 123, 0);
mock.sent_packets.clear();
int rc = msi_hangup(log, call);
ASSERT_EQ(rc, 0);
ASSERT_EQ(mock.sent_packets.size(), 1u);
const auto &pkt = mock.sent_packets[0];
// REQU_POP (2)
EXPECT_EQ(pkt[2], 2);
}
TEST_F(MsiTest, ChangeCapabilities)
{
// Setup active call
uint32_t friend_number = 123;
uint8_t invite_pkt[] = {1, 1, 0, 3, 1, 0, 0};
msi_handle_packet(session, log, friend_number, invite_pkt, sizeof(invite_pkt));
MSICall *call = mock.last_call;
msi_answer(log, call, 0);
mock.sent_packets.clear();
uint8_t new_caps = MSI_CAP_S_VIDEO;
int rc = msi_change_capabilities(log, call, new_caps);
ASSERT_EQ(rc, 0);
EXPECT_EQ(call->self_capabilities, new_caps);
ASSERT_EQ(mock.sent_packets.size(), 1u);
EXPECT_EQ(mock.sent_packets[0][2], 1); // REQU_PUSH
EXPECT_EQ(mock.sent_packets[0][5], new_caps);
}
TEST_F(MsiTest, PeerTimeout)
{
MSICall *call = nullptr;
uint32_t friend_number = 123;
msi_invite(log, session, &call, friend_number, 0);
msi_call_timeout(session, log, friend_number);
EXPECT_EQ(mock.stats.peertimeout, 1);
}
TEST_F(MsiTest, RemoteHangup)
{
uint32_t friend_number = 123;
MSICall *call = nullptr;
msi_invite(log, session, &call, friend_number, 0);
// Craft pop packet
uint8_t pop_pkt[] = {1, 1, 2, 0}; // REQU_POP
msi_handle_packet(session, log, friend_number, pop_pkt, sizeof(pop_pkt));
EXPECT_EQ(mock.stats.end, 1);
}
TEST_F(MsiTest, RemoteError)
{
uint32_t friend_number = 123;
MSICall *call = nullptr;
msi_invite(log, session, &call, friend_number, 0);
// Craft error packet (ID_ERROR = 2)
uint8_t error_pkt[] = {1, 1, 2, 2, 1, 1, 0}; // REQU_POP + MSI_E_INVALID_MESSAGE
msi_handle_packet(session, log, friend_number, error_pkt, sizeof(error_pkt));
EXPECT_EQ(mock.stats.error, 1);
ASSERT_NE(mock.last_call, nullptr);
EXPECT_EQ(mock.last_error, MSI_E_INVALID_MESSAGE);
}
TEST_F(MsiTest, MultipleConcurrentCalls)
{
MSICall *call1 = nullptr;
MSICall *call2 = nullptr;
msi_invite(log, session, &call1, 1, 0);
msi_invite(log, session, &call2, 2, 0);
EXPECT_NE(call1, call2);
EXPECT_EQ(call1->friend_number, 1u);
EXPECT_EQ(call2->friend_number, 2u);
// End call 1
msi_hangup(log, call1);
// Call 2 should still be there
uint8_t pop_pkt[] = {1, 1, 2, 0};
msi_handle_packet(session, log, 2, pop_pkt, sizeof(pop_pkt));
EXPECT_EQ(mock.stats.end, 1);
}
TEST_F(MsiTest, RemoteAnswer)
{
MSICall *call = nullptr;
msi_invite(log, session, &call, 123, 0);
uint8_t peer_caps = MSI_CAP_S_AUDIO;
uint8_t push_pkt[] = {1, 1, 1, 3, 1, peer_caps, 0}; // REQU_PUSH + capabilities
msi_handle_packet(session, log, 123, push_pkt, sizeof(push_pkt));
EXPECT_EQ(mock.stats.start, 1);
EXPECT_EQ(call->state, MSI_CALL_ACTIVE);
EXPECT_EQ(call->peer_capabilities, peer_caps);
}
TEST_F(MsiTest, RemoteCapabilitiesChange)
{
uint32_t friend_number = 123;
uint8_t invite_pkt[] = {1, 1, 0, 3, 1, 0, 0};
msi_handle_packet(session, log, friend_number, invite_pkt, sizeof(invite_pkt));
MSICall *call = mock.last_call;
msi_answer(log, call, 0);
uint8_t new_caps = MSI_CAP_S_VIDEO;
uint8_t push_pkt[] = {1, 1, 1, 3, 1, new_caps, 0}; // REQU_PUSH + new capabilities
msi_handle_packet(session, log, friend_number, push_pkt, sizeof(push_pkt));
EXPECT_EQ(mock.stats.capabilities, 1);
EXPECT_EQ(call->peer_capabilities, new_caps);
}
TEST_F(MsiTest, FriendRecall)
{
uint32_t friend_number = 123;
uint8_t invite_pkt[] = {1, 1, 0, 3, 1, 0, 0};
msi_handle_packet(session, log, friend_number, invite_pkt, sizeof(invite_pkt));
MSICall *call = mock.last_call;
msi_answer(log, call, 0);
mock.sent_packets.clear();
// Friend sends invite again while we are active
msi_handle_packet(session, log, friend_number, invite_pkt, sizeof(invite_pkt));
// We should have sent a REQU_PUSH back
ASSERT_GE(mock.sent_packets.size(), 1u);
EXPECT_EQ(mock.sent_packets.back()[2], 1); // REQU_PUSH
}
TEST_F(MsiTest, GapInFriendNumbers)
{
MSICall *call1 = nullptr;
MSICall *call3 = nullptr;
MSICall *call2 = nullptr;
msi_invite(log, session, &call1, 1, 0);
msi_invite(log, session, &call3, 3, 0);
msi_invite(log, session, &call2, 2, 0); // This fills a hole
EXPECT_EQ(call2->prev, call1);
EXPECT_EQ(call2->next, call3);
EXPECT_EQ(call1->next, call2);
EXPECT_EQ(call3->prev, call2);
}
TEST_F(MsiTest, InvalidPackets)
{
uint32_t friend_number = 123;
// Empty packet
uint8_t empty = 0;
msi_handle_packet(session, log, friend_number, &empty, 0);
// Missing end byte
uint8_t no_end[] = {1, 1, 0};
msi_handle_packet(session, log, friend_number, no_end, sizeof(no_end));
// Invalid ID
uint8_t invalid_id[] = {99, 1, 0, 0};
msi_handle_packet(session, log, friend_number, invalid_id, sizeof(invalid_id));
// Invalid size (too large)
uint8_t invalid_size[] = {1, 10, 0, 0};
msi_handle_packet(session, log, friend_number, invalid_size, sizeof(invalid_size));
// Invalid size (mismatch)
uint8_t size_mismatch[] = {1, 2, 0, 0};
msi_handle_packet(session, log, friend_number, size_mismatch, sizeof(size_mismatch));
// Missing request field
uint8_t no_request[] = {3, 1, 0, 0};
msi_handle_packet(session, log, friend_number, no_request, sizeof(no_request));
}
TEST_F(MsiTest, CallbackFailure)
{
struct FailMock {
static int fail_cb(void *, MSICall *) { return -1; }
};
// Create a new session with a failing callback
MSICallbacks callbacks = {FailMock::fail_cb, MockMsi::on_start, MockMsi::on_end,
MockMsi::on_error, MockMsi::on_peertimeout, MockMsi::on_capabilities};
MSISession *fail_session = msi_new(log, MockMsi::send_packet, &mock, &callbacks, &mock);
uint8_t invite_pkt[] = {1, 1, 0, 3, 1, 0, 0};
msi_handle_packet(fail_session, log, 123, invite_pkt, sizeof(invite_pkt));
// Should have sent an error back
ASSERT_GE(mock.sent_packets.size(), 1u);
// REQU_POP (2) + ID_ERROR (2)
EXPECT_EQ(mock.sent_packets.back()[2], 2);
EXPECT_EQ(mock.sent_packets.back()[3], 2);
msi_kill(log, fail_session);
}
TEST_F(MsiTest, InvalidStates)
{
MSICall *call = nullptr;
msi_invite(log, session, &call, 123, 0);
// Cannot answer a REQUESTING call
EXPECT_EQ(msi_answer(log, call, 0), -1);
// Cannot change capabilities of a REQUESTING call
EXPECT_EQ(msi_change_capabilities(log, call, 0), -1);
// Cannot invite a friend we are already in call with
MSICall *call2 = nullptr;
EXPECT_EQ(msi_invite(log, session, &call2, 123, 0), -1);
}
TEST_F(MsiTest, StrayPackets)
{
uint32_t friend_number = 123;
// PUSH for non-existent call
uint8_t push_pkt[] = {1, 1, 1, 3, 1, 0, 0};
msi_handle_packet(session, log, friend_number, push_pkt, sizeof(push_pkt));
// POP for non-existent call
uint8_t pop_pkt[] = {1, 1, 2, 0};
msi_handle_packet(session, log, friend_number, pop_pkt, sizeof(pop_pkt));
// Error sent back for stray PUSH
ASSERT_GE(mock.sent_packets.size(), 1u);
}
TEST_F(MsiTest, KillMultipleCalls)
{
MSICall *call1 = nullptr;
MSICall *call2 = nullptr;
msi_invite(log, session, &call1, 1, 0);
msi_invite(log, session, &call2, 2, 0);
mock.sent_packets.clear();
// msi_kill is called in TearDown, but we can call it here to verify
msi_kill(log, session);
// Should have sent POP for both calls
EXPECT_EQ(mock.sent_packets.size(), 2u);
// Set session to NULL so TearDown doesn't double-kill it (though msi_kill handles it)
session = nullptr;
}
} // namespace

View File

@@ -10,17 +10,180 @@
#include <sodium.h> #include <sodium.h>
#include "bwcontroller.h"
#include "toxav_hacks.h"
#include "../toxcore/ccompat.h" #include "../toxcore/ccompat.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/mono_time.h" #include "../toxcore/mono_time.h"
#include "../toxcore/net_crypto.h" #include "../toxcore/net_crypto.h"
#include "../toxcore/network.h" #include "../toxcore/network.h"
#include "../toxcore/tox_private.h"
#include "../toxcore/util.h" #include "../toxcore/util.h"
/**
* Maximum size of a single RTP frame in bytes.
* This limit prevents memory exhaustion attacks where a malicious peer sends
* a header indicating a very large frame size, causing the receiver to allocate
* excessive memory.
*/
#define MAX_RTP_FRAME_SIZE (32 * 1024 * 1024)
struct RTPHeader {
/* Standard RTP header */
unsigned ve: 2; /* Version has only 2 bits! */
unsigned pe: 1; /* Padding */
unsigned xe: 1; /* Extra header */
unsigned cc: 4; /* Contributing sources count */
unsigned ma: 1; /* Marker */
unsigned pt: 7; /* Payload type */
uint16_t sequnum;
uint32_t timestamp;
uint32_t ssrc;
/* Non-standard Tox-specific fields */
/**
* Bit mask of `RTPFlags` setting features of the current frame.
*/
uint64_t flags;
/**
* The full 32 bit data offset of the current data chunk. The
* @ref offset_lower data member contains the lower 16 bits of this value.
* For frames smaller than 64KiB, @ref offset_full and @ref offset_lower are
* equal.
*/
uint32_t offset_full;
/**
* The full 32 bit payload length without header and packet id.
*/
uint32_t data_length_full;
/**
* Only the receiver uses this field (why do we have this?).
*/
uint32_t received_length_full;
/**
* Data offset of the current part (lower bits).
*/
uint16_t offset_lower;
/**
* Total message length (lower bits).
*/
uint16_t data_length_lower;
};
struct RTPMessage {
/**
* This is used in the old code that doesn't deal with large frames, i.e.
* the audio code or receiving code for old 16 bit messages. We use it to
* record the number of bytes received so far in a multi-part message. The
* multi-part message in the old code is stored in `RTPSession::mp`.
*/
uint32_t len;
struct RTPHeader header;
uint8_t data[];
};
/**
* One slot in the work buffer list. Represents one frame that is currently
* being assembled.
*/
struct RTPWorkBuffer {
/**
* Whether this slot contains a key frame. This is true iff
* `buf->header.flags & RTP_KEY_FRAME`.
*/
bool is_keyframe;
/**
* The number of bytes received so far, regardless of which pieces. I.e. we
* could have received the first 1000 bytes and the last 1000 bytes with
* 4000 bytes in the middle still to come, and this number would be 2000.
*/
uint32_t received_len;
/**
* The message currently being assembled.
*/
struct RTPMessage *buf;
};
struct RTPWorkBufferList {
int8_t next_free_entry;
struct RTPWorkBuffer work_buffer[USED_RTP_WORKBUFFER_COUNT];
};
/**
* RTP control session.
*/
struct RTPSession {
uint8_t payload_type;
uint16_t sequnum; /* Sending sequence number */
uint16_t rsequnum; /* Receiving sequence number */
uint32_t rtimestamp;
uint32_t ssrc; // this seems to be unused!?
struct RTPMessage *mp; /* Expected parted message */
struct RTPWorkBufferList *work_buffer_list;
uint8_t first_packets_counter; /* dismiss first few lost video packets */
const Logger *log;
Mono_Time *mono_time;
bool rtp_receive_active; /* if this is set to false then incoming rtp packets will not be processed by rtp_receive_packet() */
rtp_send_packet_cb *send_packet;
void *send_packet_user_data;
rtp_add_recv_cb *add_recv;
rtp_add_lost_cb *add_lost;
void *bwc_user_data;
void *cs;
rtp_m_cb *mcb;
};
const uint8_t *rtp_message_data(const RTPMessage *msg)
{
return msg->data;
}
uint32_t rtp_message_len(const RTPMessage *msg)
{
return msg->len;
}
uint8_t rtp_message_pt(const RTPMessage *msg)
{
return msg->header.pt;
}
uint16_t rtp_message_sequnum(const RTPMessage *msg)
{
return msg->header.sequnum;
}
uint64_t rtp_message_flags(const RTPMessage *msg)
{
return msg->header.flags;
}
uint32_t rtp_message_data_length_full(const RTPMessage *msg)
{
return msg->header.data_length_full;
}
bool rtp_session_is_receiving_active(const RTPSession *session)
{
return session->rtp_receive_active;
}
uint32_t rtp_session_get_ssrc(const RTPSession *session)
{
return session->ssrc;
}
void rtp_session_set_ssrc(RTPSession *session, uint32_t ssrc)
{
session->ssrc = ssrc;
}
/** /**
* The number of milliseconds we want to keep a keyframe in the buffer for, * The number of milliseconds we want to keep a keyframe in the buffer for,
* even though there are no free slots for incoming frames. * even though there are no free slots for incoming frames.
@@ -31,11 +194,15 @@
static struct RTPMessage *new_message(const Logger *log, const struct RTPHeader *header, size_t allocate_len, static struct RTPMessage *new_message(const Logger *log, const struct RTPHeader *header, size_t allocate_len,
const uint8_t *data, uint16_t data_length) const uint8_t *data, uint16_t data_length)
{ {
assert(allocate_len >= data_length); if (allocate_len < data_length) {
LOGGER_WARNING(log, "new_message: allocate_len (%zu) < data_length (%u)", allocate_len, data_length);
return nullptr;
}
struct RTPMessage *msg = (struct RTPMessage *)calloc(1, sizeof(struct RTPMessage) + allocate_len); struct RTPMessage *msg = (struct RTPMessage *)calloc(1, sizeof(struct RTPMessage) + allocate_len);
if (msg == nullptr) { if (msg == nullptr) {
LOGGER_DEBUG(log, "Could not allocate RTPMessage buffer"); LOGGER_WARNING(log, "Could not allocate RTPMessage buffer");
return nullptr; return nullptr;
} }
@@ -204,7 +371,8 @@ static struct RTPMessage *process_frame(const Logger *log, struct RTPWorkBufferL
struct RTPWorkBuffer *const slot = &wkbl->work_buffer[slot_id]; struct RTPWorkBuffer *const slot = &wkbl->work_buffer[slot_id];
// Move ownership of the frame out of the slot into m_new. // Move ownership of the frame out of the slot into m_new.
struct RTPMessage *const m_new = slot->buf; struct RTPMessage *msg = slot->buf;
msg->len = msg->header.data_length_full;
slot->buf = nullptr; slot->buf = nullptr;
assert(wkbl->next_free_entry >= 1 && wkbl->next_free_entry <= USED_RTP_WORKBUFFER_COUNT); assert(wkbl->next_free_entry >= 1 && wkbl->next_free_entry <= USED_RTP_WORKBUFFER_COUNT);
@@ -226,7 +394,7 @@ static struct RTPMessage *process_frame(const Logger *log, struct RTPWorkBufferL
wkbl->work_buffer[wkbl->next_free_entry] = empty; wkbl->work_buffer[wkbl->next_free_entry] = empty;
// Move ownership of the frame to the caller. // Move ownership of the frame to the caller.
return m_new; return msg;
} }
/** /**
@@ -253,6 +421,11 @@ static bool fill_data_into_slot(const Logger *log, struct RTPWorkBufferList *wkb
if (slot->received_len == 0) { if (slot->received_len == 0) {
assert(slot->buf == nullptr); assert(slot->buf == nullptr);
if (header->data_length_full > MAX_RTP_FRAME_SIZE) {
LOGGER_WARNING(log, "RTP frame too large: %u > %u", (unsigned)header->data_length_full, (unsigned)MAX_RTP_FRAME_SIZE);
return false;
}
// No data for this slot has been received, yet, so we create a new // No data for this slot has been received, yet, so we create a new
// message for it with enough memory for the entire frame. // message for it with enough memory for the entire frame.
struct RTPMessage *msg = (struct RTPMessage *)calloc(1, sizeof(struct RTPMessage) + header->data_length_full); struct RTPMessage *msg = (struct RTPMessage *)calloc(1, sizeof(struct RTPMessage) + header->data_length_full);
@@ -275,12 +448,24 @@ static bool fill_data_into_slot(const Logger *log, struct RTPWorkBufferList *wkb
assert(wkbl->next_free_entry < USED_RTP_WORKBUFFER_COUNT); assert(wkbl->next_free_entry < USED_RTP_WORKBUFFER_COUNT);
++wkbl->next_free_entry; ++wkbl->next_free_entry;
} else {
if (slot->buf->header.data_length_full != header->data_length_full) {
LOGGER_WARNING(log, "Received packet with different length than previous packets in same frame: %u != %u",
header->data_length_full, slot->buf->header.data_length_full);
return false;
}
} }
// We already checked this when we received the packet, but we rely on it // We already checked this when we received the packet, but we rely on it
// here, so assert again. // here, so assert again.
assert(header->offset_full < header->data_length_full); assert(header->offset_full < header->data_length_full);
if (header->data_length_full - header->offset_full < incoming_data_length) {
LOGGER_ERROR(log, "Packet too long for buffer: offset %u + len %u > total %u",
(unsigned)header->offset_full, (unsigned)incoming_data_length, (unsigned)header->data_length_full);
return false;
}
// Copy the incoming chunk of data into the correct position in the full // Copy the incoming chunk of data into the correct position in the full
// frame data array. // frame data array.
memcpy( memcpy(
@@ -305,11 +490,15 @@ static void update_bwc_values(RTPSession *session, const struct RTPMessage *msg)
} else { } else {
const uint32_t data_length_full = msg->header.data_length_full; // without header const uint32_t data_length_full = msg->header.data_length_full; // without header
const uint32_t received_length_full = msg->header.received_length_full; // without header const uint32_t received_length_full = msg->header.received_length_full; // without header
bwc_add_recv(session->bwc, data_length_full); if (session->add_recv) {
session->add_recv(session->bwc_user_data, data_length_full);
}
if (received_length_full < data_length_full) { if (received_length_full < data_length_full) {
LOGGER_DEBUG(session->log, "BWC: full length=%u received length=%u", data_length_full, received_length_full); LOGGER_DEBUG(session->log, "BWC: full length=%u received length=%u", data_length_full, received_length_full);
bwc_add_lost(session->bwc, data_length_full - received_length_full); if (session->add_lost) {
session->add_lost(session->bwc_user_data, data_length_full - received_length_full);
}
} }
} }
} }
@@ -320,12 +509,7 @@ static void update_bwc_values(RTPSession *session, const struct RTPMessage *msg)
* The packet may or may not be part of a multipart frame. This function will * The packet may or may not be part of a multipart frame. This function will
* find out and handle it appropriately. * find out and handle it appropriately.
* *
* @param session The current RTP session with: * @param session The current RTP session
* <code>
* session->mcb == vc_queue_message() // this function is called from here
* session->mp == struct RTPMessage *
* session->cs == call->video.second // == VCSession created by vc_new() call
* </code>
* @param header The RTP header deserialised from the packet. * @param header The RTP header deserialised from the packet.
* @param incoming_data The packet data *not* header, i.e. this is the actual * @param incoming_data The packet data *not* header, i.e. this is the actual
* payload. * payload.
@@ -370,13 +554,17 @@ static int handle_video_packet(const Logger *log, RTPSession *session, const str
// get_slot just told us it's full, so process_frame must return non-null. // get_slot just told us it's full, so process_frame must return non-null.
assert(m_new != nullptr); assert(m_new != nullptr);
if (m_new->len >= 2) {
LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-001a b0=%d b1=%d", (int)m_new->data[0], LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-001a b0=%d b1=%d", (int)m_new->data[0],
(int)m_new->data[1]); (int)m_new->data[1]);
} else if (m_new->len == 1) {
LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-001a b0=%d", (int)m_new->data[0]);
} else {
LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-001a (empty)");
}
update_bwc_values(session, m_new); update_bwc_values(session, m_new);
// Pass ownership of m_new to the callback. // Pass ownership of m_new to the callback.
Mono_Time *mt = toxav_get_av_mono_time(session->toxav); session->mcb(session->mono_time, session->cs, m_new);
assert(mt != nullptr);
session->mcb(mt, session->cs, m_new);
// Now we no longer own m_new. // Now we no longer own m_new.
m_new = nullptr; m_new = nullptr;
@@ -411,12 +599,16 @@ static int handle_video_packet(const Logger *log, RTPSession *session, const str
struct RTPMessage *m_new = process_frame(log, session->work_buffer_list, slot_id); struct RTPMessage *m_new = process_frame(log, session->work_buffer_list, slot_id);
if (m_new != nullptr) { if (m_new != nullptr) {
if (m_new->len >= 2) {
LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-003a b0=%d b1=%d", (int)m_new->data[0], LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-003a b0=%d b1=%d", (int)m_new->data[0],
(int)m_new->data[1]); (int)m_new->data[1]);
} else if (m_new->len == 1) {
LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-003a b0=%d", (int)m_new->data[0]);
} else {
LOGGER_DEBUG(log, "-- handle_video_packet -- CALLBACK-003a (empty)");
}
update_bwc_values(session, m_new); update_bwc_values(session, m_new);
Mono_Time *mt = toxav_get_av_mono_time(session->toxav); session->mcb(session->mono_time, session->cs, m_new);
assert(mt != nullptr);
session->mcb(mt, session->cs, m_new);
m_new = nullptr; m_new = nullptr;
} }
@@ -427,41 +619,15 @@ static int handle_video_packet(const Logger *log, RTPSession *session, const str
/** /**
* receive custom lossypackets and process them. they can be incoming audio or video packets * receive custom lossypackets and process them. they can be incoming audio or video packets
*/ */
void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length, void *user_data) void rtp_receive_packet(RTPSession *session, const uint8_t *data, size_t length)
{ {
ToxAV *toxav = (ToxAV *)tox_get_av_object(tox); const Logger *log = session->log;
if (toxav == nullptr) {
// LOGGER_WARNING(log, "ToxAV is NULL!");
return;
}
const Logger *log = toxav_get_logger(toxav);
if (length < RTP_HEADER_SIZE + 1) { if (length < RTP_HEADER_SIZE + 1) {
LOGGER_WARNING(log, "Invalid length of received buffer!"); LOGGER_WARNING(log, "Invalid length of received buffer!");
return; return;
} }
ToxAVCall *call = call_get(toxav, friend_number);
if (call == nullptr) {
LOGGER_WARNING(log, "ToxAVCall is NULL!");
return;
}
RTPSession *session = rtp_session_get(call, data[0]);
if (session == nullptr) {
LOGGER_WARNING(log, "No session!");
return;
}
if (!session->rtp_receive_active) {
LOGGER_WARNING(log, "receiving not allowed!");
return;
}
// Get the packet type. // Get the packet type.
const uint8_t packet_type = data[0]; const uint8_t packet_type = data[0];
const uint8_t *payload = &data[1]; const uint8_t *payload = &data[1];
@@ -513,13 +679,13 @@ void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, si
/* Message is not late; pick up the latest parameters */ /* Message is not late; pick up the latest parameters */
session->rsequnum = header.sequnum; session->rsequnum = header.sequnum;
session->rtimestamp = header.timestamp; session->rtimestamp = header.timestamp;
bwc_add_recv(session->bwc, payload_size); if (session->add_recv) {
session->add_recv(session->bwc_user_data, payload_size);
}
/* Invoke processing of active multiparted message */ /* Invoke processing of active multiparted message */
if (session->mp != nullptr) { if (session->mp != nullptr) {
Mono_Time *mt = toxav_get_av_mono_time(session->toxav); session->mcb(session->mono_time, session->cs, session->mp);
assert(mt != nullptr);
session->mcb(mt, session->cs, session->mp);
session->mp = nullptr; session->mp = nullptr;
} }
@@ -527,9 +693,7 @@ void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, si
*/ */
session->mp = new_message(log, &header, payload_size - RTP_HEADER_SIZE, &payload[RTP_HEADER_SIZE], payload_size - RTP_HEADER_SIZE); session->mp = new_message(log, &header, payload_size - RTP_HEADER_SIZE, &payload[RTP_HEADER_SIZE], payload_size - RTP_HEADER_SIZE);
Mono_Time *mt = toxav_get_av_mono_time(session->toxav); session->mcb(session->mono_time, session->cs, session->mp);
assert(mt != nullptr);
session->mcb(mt, session->cs, session->mp);
session->mp = nullptr; session->mp = nullptr;
return; return;
} }
@@ -550,25 +714,24 @@ void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, si
/* Make sure we have enough allocated memory */ /* Make sure we have enough allocated memory */
if (session->mp->header.data_length_lower - session->mp->len < payload_size - RTP_HEADER_SIZE || if (session->mp->header.data_length_lower - session->mp->len < payload_size - RTP_HEADER_SIZE ||
session->mp->header.data_length_lower <= header.offset_lower) { session->mp->header.data_length_lower <= header.offset_lower ||
/* There happened to be some corruption on the stream; session->mp->header.data_length_lower - header.offset_lower < payload_size - RTP_HEADER_SIZE) {
* continue wihtout this part LOGGER_WARNING(log, "Corruption on the stream: multipart audio packet does not fit");
*/
return; return;
} }
memcpy(session->mp->data + header.offset_lower, &payload[RTP_HEADER_SIZE], memcpy(session->mp->data + header.offset_lower, &payload[RTP_HEADER_SIZE],
payload_size - RTP_HEADER_SIZE); payload_size - RTP_HEADER_SIZE);
session->mp->len += payload_size - RTP_HEADER_SIZE; session->mp->len += payload_size - RTP_HEADER_SIZE;
bwc_add_recv(session->bwc, payload_size); if (session->add_recv) {
session->add_recv(session->bwc_user_data, payload_size);
}
if (session->mp->len == session->mp->header.data_length_lower) { if (session->mp->len == session->mp->header.data_length_lower) {
/* Received a full message; now push it for the further /* Received a full message; now push it for the further
* processing. * processing.
*/ */
Mono_Time *mt = toxav_get_av_mono_time(session->toxav); session->mcb(session->mono_time, session->cs, session->mp);
assert(mt != nullptr);
session->mcb(mt, session->cs, session->mp);
session->mp = nullptr; session->mp = nullptr;
} }
} else { } else {
@@ -581,9 +744,7 @@ void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, si
} }
/* Push the previous message for processing */ /* Push the previous message for processing */
Mono_Time *mt = toxav_get_av_mono_time(session->toxav); session->mcb(session->mono_time, session->cs, session->mp);
assert(mt != nullptr);
session->mcb(mt, session->cs, session->mp);
session->mp = nullptr; session->mp = nullptr;
goto NEW_MULTIPARTED; goto NEW_MULTIPARTED;
@@ -594,10 +755,19 @@ void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, si
/* This is also a point for new multiparted messages */ /* This is also a point for new multiparted messages */
NEW_MULTIPARTED: NEW_MULTIPARTED:
if (header.data_length_lower - header.offset_lower < payload_size - RTP_HEADER_SIZE) {
LOGGER_WARNING(log, "Packet too long for buffer: offset %u + len %u > total %u",
(unsigned)header.offset_lower, (unsigned)(payload_size - RTP_HEADER_SIZE),
(unsigned)header.data_length_lower);
return;
}
/* Message is not late; pick up the latest parameters */ /* Message is not late; pick up the latest parameters */
session->rsequnum = header.sequnum; session->rsequnum = header.sequnum;
session->rtimestamp = header.timestamp; session->rtimestamp = header.timestamp;
bwc_add_recv(session->bwc, payload_size); if (session->add_recv) {
session->add_recv(session->bwc_user_data, payload_size);
}
/* Store message. /* Store message.
*/ */
@@ -679,8 +849,10 @@ static uint32_t rtp_random_u32(void)
return randombytes_random(); return randombytes_random();
} }
RTPSession *rtp_new(const Logger *log, const Memory *mem, int payload_type, Tox *tox, ToxAV *toxav, uint32_t friendnumber, RTPSession *rtp_new(const Logger *log, int payload_type, Mono_Time *mono_time,
BWController *bwc, void *cs, rtp_m_cb *mcb) rtp_send_packet_cb *send_packet, void *send_packet_user_data,
rtp_add_recv_cb *add_recv, rtp_add_lost_cb *add_lost, void *bwc_user_data,
void *cs, rtp_m_cb *mcb)
{ {
assert(mcb != nullptr); assert(mcb != nullptr);
assert(cs != nullptr); assert(cs != nullptr);
@@ -706,18 +878,20 @@ RTPSession *rtp_new(const Logger *log, const Memory *mem, int payload_type, Tox
session->ssrc = payload_type == RTP_TYPE_VIDEO ? 0 : rtp_random_u32(); // Zoff: what is this?? session->ssrc = payload_type == RTP_TYPE_VIDEO ? 0 : rtp_random_u32(); // Zoff: what is this??
session->payload_type = payload_type; session->payload_type = payload_type;
session->log = log; session->log = log;
session->mem = mem; session->mono_time = mono_time;
session->tox = tox;
session->toxav = toxav;
session->friend_number = friendnumber;
session->rtp_receive_active = true; session->rtp_receive_active = true;
session->send_packet = send_packet;
session->send_packet_user_data = send_packet_user_data;
session->add_recv = add_recv;
session->add_lost = add_lost;
session->bwc_user_data = bwc_user_data;
// set NULL just in case // set NULL just in case
session->mp = nullptr; session->mp = nullptr;
session->first_packets_counter = 1; session->first_packets_counter = 1;
/* Also set payload type as prefix */ /* Also set payload type as prefix */
session->bwc = bwc;
session->cs = cs; session->cs = cs;
session->mcb = mcb; session->mcb = mcb;
@@ -735,10 +909,13 @@ void rtp_kill(const Logger *log, RTPSession *session)
LOGGER_DEBUG(log, "Terminated RTP session V3 work_buffer_list->next_free_entry: %d", LOGGER_DEBUG(log, "Terminated RTP session V3 work_buffer_list->next_free_entry: %d",
(int)session->work_buffer_list->next_free_entry); (int)session->work_buffer_list->next_free_entry);
if (session->work_buffer_list) {
for (int8_t i = 0; i < session->work_buffer_list->next_free_entry; ++i) { for (int8_t i = 0; i < session->work_buffer_list->next_free_entry; ++i) {
free(session->work_buffer_list->work_buffer[i].buf); free(session->work_buffer_list->work_buffer[i].buf);
} }
free(session->work_buffer_list); free(session->work_buffer_list);
}
free(session->mp);
free(session); free(session);
} }
@@ -756,37 +933,7 @@ void rtp_stop_receiving_mark(RTPSession *session)
} }
} }
void rtp_allow_receiving(Tox *tox) static void rtp_send_piece(RTPSession *session, const struct RTPHeader *header,
{
// register callback
tox_callback_friend_lossy_packet_per_pktid(tox, handle_rtp_packet, RTP_TYPE_AUDIO);
tox_callback_friend_lossy_packet_per_pktid(tox, handle_rtp_packet, RTP_TYPE_VIDEO);
}
void rtp_stop_receiving(Tox *tox)
{
// UN-register callback
tox_callback_friend_lossy_packet_per_pktid(tox, nullptr, RTP_TYPE_AUDIO);
tox_callback_friend_lossy_packet_per_pktid(tox, nullptr, RTP_TYPE_VIDEO);
}
/**
* Log the neterror error if any.
*
* @param error the error from rtp_send_custom_lossy_packet.
* @param rdata_size The package length to be shown in the log.
*/
static void rtp_report_error_maybe(const Logger *log, const Memory *mem, Tox_Err_Friend_Custom_Packet error, uint16_t rdata_size)
{
if (error != TOX_ERR_FRIEND_CUSTOM_PACKET_OK) {
Net_Strerror error_str;
const char *toxerror = tox_err_friend_custom_packet_to_string(error);
LOGGER_WARNING(log, "RTP send failed (len: %u)! tox error: %s net error: %s",
rdata_size, toxerror, net_strerror(net_error(), &error_str));
}
}
static void rtp_send_piece(const Logger *log, const Memory *mem, Tox *tox, uint32_t friend_number, const struct RTPHeader *header,
const uint8_t *data, uint8_t *rdata, uint16_t length) const uint8_t *data, uint8_t *rdata, uint16_t length)
{ {
rtp_header_pack(rdata + 1, header); rtp_header_pack(rdata + 1, header);
@@ -794,10 +941,9 @@ static void rtp_send_piece(const Logger *log, const Memory *mem, Tox *tox, uint3
const uint16_t rdata_size = length + RTP_HEADER_SIZE + 1; const uint16_t rdata_size = length + RTP_HEADER_SIZE + 1;
Tox_Err_Friend_Custom_Packet error; if (session->send_packet) {
tox_friend_send_lossy_packet(tox, friend_number, rdata, rdata_size, &error); session->send_packet(session->send_packet_user_data, rdata, rdata_size);
}
rtp_report_error_maybe(log, mem, error, rdata_size);
} }
static struct RTPHeader rtp_default_header(const RTPSession *session, uint32_t length, bool is_keyframe) static struct RTPHeader rtp_default_header(const RTPSession *session, uint32_t length, bool is_keyframe)
@@ -825,9 +971,8 @@ static struct RTPHeader rtp_default_header(const RTPSession *session, uint32_t l
header.ma = 0; header.ma = 0;
header.pt = session->payload_type % 128; header.pt = session->payload_type % 128;
header.sequnum = session->sequnum; header.sequnum = session->sequnum;
const Mono_Time *mt = toxav_get_av_mono_time(session->toxav); if (session->mono_time != nullptr) {
if (mt != nullptr) { header.timestamp = current_time_monotonic(session->mono_time);
header.timestamp = current_time_monotonic(mt);
} else { } else {
header.timestamp = 0; header.timestamp = 0;
} }
@@ -835,7 +980,6 @@ static struct RTPHeader rtp_default_header(const RTPSession *session, uint32_t l
header.offset_lower = 0; header.offset_lower = 0;
header.data_length_lower = length_safe; header.data_length_lower = length_safe;
header.data_length_full = length; // without header header.data_length_full = length; // without header
header.offset_lower = 0;
header.offset_full = 0; header.offset_full = 0;
return header; return header;
@@ -870,7 +1014,7 @@ int rtp_send_data(const Logger *log, RTPSession *session, const uint8_t *data, u
* Send the packet in single piece. * Send the packet in single piece.
*/ */
assert(length < UINT16_MAX); assert(length < UINT16_MAX);
rtp_send_piece(log, session->mem, session->tox, session->friend_number, &header, data, rdata, length); rtp_send_piece(session, &header, data, rdata, (uint16_t)length);
} else { } else {
/* /*
* The length is greater than the maximum allowed length (including header) * The length is greater than the maximum allowed length (including header)
@@ -880,18 +1024,18 @@ int rtp_send_data(const Logger *log, RTPSession *session, const uint8_t *data, u
uint16_t piece = MAX_CRYPTO_DATA_SIZE - (RTP_HEADER_SIZE + 1); uint16_t piece = MAX_CRYPTO_DATA_SIZE - (RTP_HEADER_SIZE + 1);
while ((length - sent) + RTP_HEADER_SIZE + 1 > MAX_CRYPTO_DATA_SIZE) { while ((length - sent) + RTP_HEADER_SIZE + 1 > MAX_CRYPTO_DATA_SIZE) {
rtp_send_piece(log, session->mem, session->tox, session->friend_number, &header, data + sent, rdata, piece); rtp_send_piece(session, &header, data + sent, rdata, piece);
sent += piece; sent += piece;
header.offset_lower = sent; header.offset_lower = (uint16_t)sent;
header.offset_full = sent; // raw data offset, without any header header.offset_full = sent; // raw data offset, without any header
} }
/* Send remaining */ /* Send remaining */
piece = length - sent; piece = (uint16_t)(length - sent);
if (piece != 0) { if (piece != 0) {
rtp_send_piece(log, session->mem, session->tox, session->friend_number, &header, data + sent, rdata, piece); rtp_send_piece(session, &header, data + sent, rdata, piece);
} }
} }

View File

@@ -6,11 +6,11 @@
#define C_TOXCORE_TOXAV_RTP_H #define C_TOXCORE_TOXAV_RTP_H
#include <stdbool.h> #include <stdbool.h>
#include <stddef.h>
#include "bwcontroller.h" #include <stdint.h>
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/tox.h" #include "../toxcore/mono_time.h"
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
@@ -35,11 +35,6 @@ typedef enum RTP_Type {
RTP_TYPE_VIDEO = 193, RTP_TYPE_VIDEO = 193,
} RTP_Type; } RTP_Type;
#ifndef TOXAV_DEFINED
#define TOXAV_DEFINED
typedef struct ToxAV ToxAV;
#endif /* TOXAV_DEFINED */
/** /**
* A bit mask (up to 64 bits) specifying features of the current frame affecting * A bit mask (up to 64 bits) specifying features of the current frame affecting
* the behaviour of the decoder. * the behaviour of the decoder.
@@ -56,124 +51,33 @@ typedef enum RTPFlags {
RTP_KEY_FRAME = 1 << 1, RTP_KEY_FRAME = 1 << 1,
} RTPFlags; } RTPFlags;
struct RTPHeader { typedef struct RTPHeader RTPHeader;
/* Standard RTP header */ typedef struct RTPMessage RTPMessage;
unsigned ve: 2; /* Version has only 2 bits! */ typedef struct RTPSession RTPSession;
unsigned pe: 1; /* Padding */
unsigned xe: 1; /* Extra header */
unsigned cc: 4; /* Contributing sources count */
unsigned ma: 1; /* Marker */ /* RTPMessage accessors */
unsigned pt: 7; /* Payload type */ const uint8_t *rtp_message_data(const RTPMessage *msg);
uint32_t rtp_message_len(const RTPMessage *msg);
uint8_t rtp_message_pt(const RTPMessage *msg);
uint16_t rtp_message_sequnum(const RTPMessage *msg);
uint64_t rtp_message_flags(const RTPMessage *msg);
uint32_t rtp_message_data_length_full(const RTPMessage *msg);
uint16_t sequnum; /* RTPSession accessors */
uint32_t timestamp; bool rtp_session_is_receiving_active(const RTPSession *session);
uint32_t ssrc; uint32_t rtp_session_get_ssrc(const RTPSession *session);
void rtp_session_set_ssrc(RTPSession *session, uint32_t ssrc);
/* Non-standard Tox-specific fields */
/**
* Bit mask of `RTPFlags` setting features of the current frame.
*/
uint64_t flags;
/**
* The full 32 bit data offset of the current data chunk. The
* @ref offset_lower data member contains the lower 16 bits of this value.
* For frames smaller than 64KiB, @ref offset_full and @ref offset_lower are
* equal.
*/
uint32_t offset_full;
/**
* The full 32 bit payload length without header and packet id.
*/
uint32_t data_length_full;
/**
* Only the receiver uses this field (why do we have this?).
*/
uint32_t received_length_full;
/**
* Data offset of the current part (lower bits).
*/
uint16_t offset_lower;
/**
* Total message length (lower bits).
*/
uint16_t data_length_lower;
};
struct RTPMessage {
/**
* This is used in the old code that doesn't deal with large frames, i.e.
* the audio code or receiving code for old 16 bit messages. We use it to
* record the number of bytes received so far in a multi-part message. The
* multi-part message in the old code is stored in `RTPSession::mp`.
*/
uint16_t len;
struct RTPHeader header;
uint8_t data[];
};
#define USED_RTP_WORKBUFFER_COUNT 3 #define USED_RTP_WORKBUFFER_COUNT 3
/**
* One slot in the work buffer list. Represents one frame that is currently
* being assembled.
*/
struct RTPWorkBuffer {
/**
* Whether this slot contains a key frame. This is true iff
* `buf->header.flags & RTP_KEY_FRAME`.
*/
bool is_keyframe;
/**
* The number of bytes received so far, regardless of which pieces. I.e. we
* could have received the first 1000 bytes and the last 1000 bytes with
* 4000 bytes in the middle still to come, and this number would be 2000.
*/
uint32_t received_len;
/**
* The message currently being assembled.
*/
struct RTPMessage *buf;
};
struct RTPWorkBufferList {
int8_t next_free_entry;
struct RTPWorkBuffer work_buffer[USED_RTP_WORKBUFFER_COUNT];
};
#define DISMISS_FIRST_LOST_VIDEO_PACKET_COUNT 10 #define DISMISS_FIRST_LOST_VIDEO_PACKET_COUNT 10
typedef int rtp_m_cb(const Mono_Time *mono_time, void *cs, struct RTPMessage *msg); typedef int rtp_m_cb(const Mono_Time *mono_time, void *cs, RTPMessage *msg);
/** typedef int rtp_send_packet_cb(void *user_data, const uint8_t *data, uint16_t length);
* RTP control session. typedef void rtp_add_recv_cb(void *user_data, uint32_t bytes);
*/ typedef void rtp_add_lost_cb(void *user_data, uint32_t bytes);
typedef struct RTPSession {
uint8_t payload_type;
uint16_t sequnum; /* Sending sequence number */
uint16_t rsequnum; /* Receiving sequence number */
uint32_t rtimestamp;
uint32_t ssrc; // this seems to be unused!?
struct RTPMessage *mp; /* Expected parted message */
struct RTPWorkBufferList *work_buffer_list;
uint8_t first_packets_counter; /* dismiss first few lost video packets */
const Logger *log;
const Memory *mem;
Tox *tox;
ToxAV *toxav;
uint32_t friend_number;
bool rtp_receive_active; /* if this is set to false then incoming rtp packets will not be processed by handle_rtp_packet() */
BWController *bwc;
void *cs;
rtp_m_cb *mcb;
} RTPSession;
void rtp_receive_packet(RTPSession *session, const uint8_t *data, size_t length);
void handle_rtp_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length, void *user_data);
/** /**
* Serialise an RTPHeader to bytes to be sent over the network. * Serialise an RTPHeader to bytes to be sent over the network.
@@ -193,13 +97,13 @@ size_t rtp_header_pack(uint8_t *rdata, const struct RTPHeader *header);
*/ */
size_t rtp_header_unpack(const uint8_t *data, struct RTPHeader *header); size_t rtp_header_unpack(const uint8_t *data, struct RTPHeader *header);
RTPSession *rtp_new(const Logger *log, const Memory *mem, int payload_type, Tox *tox, ToxAV *toxav, uint32_t friendnumber, RTPSession *rtp_new(const Logger *log, int payload_type, Mono_Time *mono_time,
BWController *bwc, void *cs, rtp_m_cb *mcb); rtp_send_packet_cb *send_packet, void *send_packet_user_data,
rtp_add_recv_cb *add_recv, rtp_add_lost_cb *add_lost, void *bwc_user_data,
void *cs, rtp_m_cb *mcb);
void rtp_kill(const Logger *log, RTPSession *session); void rtp_kill(const Logger *log, RTPSession *session);
void rtp_allow_receiving_mark(RTPSession *session); void rtp_allow_receiving_mark(RTPSession *session);
void rtp_stop_receiving_mark(RTPSession *session); void rtp_stop_receiving_mark(RTPSession *session);
void rtp_allow_receiving(Tox *tox);
void rtp_stop_receiving(Tox *tox);
/** /**
* @brief Send a frame of audio or video data, chunked in @ref RTPMessage instances. * @brief Send a frame of audio or video data, chunked in @ref RTPMessage instances.

View File

@@ -0,0 +1,81 @@
#include "rtp.h"
#include <cstdlib>
#include <vector>
#include "../testing/fuzzing/fuzz_support.hh"
#include "../toxcore/logger.h"
#include "../toxcore/mono_time.h"
#include "../toxcore/os_memory.h"
namespace {
struct MockSessionData { };
static int mock_send_packet(void * /*user_data*/, const uint8_t * /*data*/, uint16_t /*length*/)
{
return 0;
}
static int mock_m_cb(const Mono_Time * /*mono_time*/, void * /*cs*/, RTPMessage *msg)
{
std::free(msg);
return 0;
}
void fuzz_rtp_receive(Fuzz_Data &input)
{
const Memory *mem = os_memory();
struct LoggerDeleter {
void operator()(Logger *l) { logger_kill(l); }
};
std::unique_ptr<Logger, LoggerDeleter> log(logger_new(mem));
auto time_cb = [](void *) -> uint64_t { return 0; };
struct MonoTimeDeleter {
const Memory *m;
void operator()(Mono_Time *t) { mono_time_free(m, t); }
};
std::unique_ptr<Mono_Time, MonoTimeDeleter> mono_time(
mono_time_new(mem, time_cb, nullptr), MonoTimeDeleter{mem});
MockSessionData sd;
CONSUME1_OR_RETURN(uint8_t, payload_type_byte, input);
int payload_type = (payload_type_byte % 2 == 0) ? RTP_TYPE_AUDIO : RTP_TYPE_VIDEO;
struct RtpSessionDeleter {
Logger *l;
void operator()(RTPSession *s) { rtp_kill(l, s); }
};
std::unique_ptr<RTPSession, RtpSessionDeleter> session(
rtp_new(log.get(), payload_type, mono_time.get(), mock_send_packet, &sd, nullptr, nullptr,
nullptr, &sd, mock_m_cb),
RtpSessionDeleter{log.get()});
while (!input.empty()) {
CONSUME1_OR_RETURN(uint16_t, len, input);
if (input.size() < len) {
len = input.size();
}
if (len == 0) {
break;
}
const uint8_t *pkt_data = input.consume(__func__, len);
rtp_receive_packet(session.get(), pkt_data, len);
}
}
} // namespace
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
Fuzz_Data input(data, size);
fuzz_rtp_receive(input);
return 0;
}

File diff suppressed because it is too large Load Diff

View File

@@ -11,7 +11,9 @@
#include "msi.h" #include "msi.h"
#include "rtp.h" #include "rtp.h"
#include "toxav_hacks.h" #include "audio.h"
#include "video.h"
#include "bwcontroller.h"
#include "../toxcore/Messenger.h" #include "../toxcore/Messenger.h"
#include "../toxcore/ccompat.h" #include "../toxcore/ccompat.h"
@@ -29,10 +31,11 @@
// iteration interval that is used when no call is active // iteration interval that is used when no call is active
#define IDLE_ITERATION_INTERVAL_MS 1000 #define IDLE_ITERATION_INTERVAL_MS 1000
#ifndef TOXAV_CALL_DEFINED
#define TOXAV_CALL_DEFINED
typedef struct ToxAVCall ToxAVCall; typedef struct ToxAVCall ToxAVCall;
#endif /* TOXAV_CALL_DEFINED */
static ToxAVCall *call_get(ToxAV *av, uint32_t friend_number);
static RTPSession *rtp_session_get(ToxAVCall *call, int payload_type);
static BWController *bwc_controller_get(const ToxAVCall *call);
struct ToxAVCall { struct ToxAVCall {
ToxAV *av; ToxAV *av;
@@ -49,7 +52,7 @@ struct ToxAVCall {
bool active; bool active;
MSICall *msi_call; MSICall *msi_call;
uint32_t friend_number; Tox_Friend_Number friend_number;
uint32_t audio_bit_rate; /* Sending audio bit rate */ uint32_t audio_bit_rate; /* Sending audio bit rate */
uint32_t video_bit_rate; /* Sending video bit rate */ uint32_t video_bit_rate; /* Sending video bit rate */
@@ -57,6 +60,9 @@ struct ToxAVCall {
/** Required for monitoring changes in states */ /** Required for monitoring changes in states */
uint8_t previous_self_capabilities; uint8_t previous_self_capabilities;
toxav_audio_receive_frame_cb *acb;
void *acb_user_data;
pthread_mutex_t toxav_call_mutex[1]; pthread_mutex_t toxav_call_mutex[1];
struct ToxAVCall *prev; struct ToxAVCall *prev;
@@ -114,32 +120,149 @@ struct ToxAV {
Mono_Time *toxav_mono_time; // ToxAV's own mono_time instance Mono_Time *toxav_mono_time; // ToxAV's own mono_time instance
}; };
static void callback_bwc(BWController *bwc, uint32_t friend_number, float loss, void *user_data); static void callback_bwc(BWController *bwc, Tox_Friend_Number friend_number, float loss, void *user_data);
static int msi_send_packet(void *user_data, uint32_t friend_number, const uint8_t *data, size_t length)
{
Tox *tox = (Tox *)user_data;
const size_t length_new = length + 1;
VLA(uint8_t, data_new, length_new);
data_new[0] = PACKET_ID_MSI;
memcpy(data_new + 1, data, length);
Tox_Err_Friend_Custom_Packet error;
tox_friend_send_lossless_packet(tox, friend_number, data_new, length_new, &error);
return error == TOX_ERR_FRIEND_CUSTOM_PACKET_OK ? 0 : -1;
}
static void handle_msi_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length,
void *user_data)
{
ToxAV *toxav = (ToxAV *)tox_get_av_object(tox);
if (toxav == nullptr) {
return;
}
if (length < 2) {
LOGGER_ERROR(toxav->log, "MSI packet is less than 2 bytes in size");
return;
}
msi_handle_packet(toxav->msi, toxav->log, friend_number, data + 1, length - 1);
}
static int rtp_send_packet(void *user_data, const uint8_t *data, uint16_t length)
{
ToxAVCall *call = (ToxAVCall *)user_data;
Tox_Err_Friend_Custom_Packet error;
tox_friend_send_lossy_packet(call->av->tox, call->friend_number, data, length, &error);
return error == TOX_ERR_FRIEND_CUSTOM_PACKET_OK ? 0 : -1;
}
static void rtp_add_recv(void *user_data, uint32_t bytes)
{
BWController *bwc = (BWController *)user_data;
bwc_add_recv(bwc, bytes);
}
static void rtp_add_lost(void *user_data, uint32_t bytes)
{
BWController *bwc = (BWController *)user_data;
bwc_add_lost(bwc, bytes);
}
static void handle_rtp_packet(Tox *tox, Tox_Friend_Number friend_number, const uint8_t *data, size_t length, void *user_data)
{
ToxAV *toxav = (ToxAV *)tox_get_av_object(tox);
if (toxav == nullptr) {
return;
}
ToxAVCall *call = call_get(toxav, friend_number);
if (call == nullptr) {
return;
}
RTPSession *session = rtp_session_get(call, data[0]);
if (session == nullptr) {
return;
}
if (!rtp_session_is_receiving_active(session)) {
return;
}
rtp_receive_packet(session, data, length);
}
static void handle_bwc_packet(Tox *tox, uint32_t friend_number, const uint8_t *data, size_t length, void *user_data)
{
ToxAV *toxav = (ToxAV *)tox_get_av_object(tox);
if (toxav == nullptr) {
return;
}
const ToxAVCall *call = call_get(toxav, friend_number);
if (call == nullptr) {
return;
}
BWController *bwc = bwc_controller_get(call);
if (bwc == nullptr) {
return;
}
bwc_handle_packet(bwc, data, length);
}
static void handle_audio_frame(uint32_t friend_number, const int16_t *pcm, size_t sample_count, uint8_t channels,
uint32_t sampling_rate, void *user_data)
{
ToxAVCall *call = (ToxAVCall *)user_data;
toxav_audio_receive_frame_cb *acb = call->acb;
void *acb_user_data = call->acb_user_data;
if (acb != nullptr) {
acb(call->av, friend_number, pcm, sample_count, channels, sampling_rate, acb_user_data);
}
}
static void handle_video_frame(uint32_t friend_number, uint16_t width, uint16_t height,
const uint8_t *y, const uint8_t *u, const uint8_t *v,
int32_t ystride, int32_t ustride, int32_t vstride,
void *user_data)
{
ToxAVCall *call = (ToxAVCall *)user_data;
toxav_video_receive_frame_cb *vcb = call->av->vcb;
void *vcb_user_data = call->av->vcb_user_data;
if (vcb != nullptr) {
vcb(call->av, friend_number, width, height, y, u, v, ystride, ustride, vstride, vcb_user_data);
}
}
static int callback_invite(void *object, MSICall *call); static int callback_invite(void *object, MSICall *call);
static int callback_start(void *object, MSICall *call); static int callback_start(void *object, MSICall *call);
static int callback_end(void *object, MSICall *call); static int callback_end(void *object, MSICall *call);
static int callback_error(void *object, MSICall *call); static int callback_error(void *object, MSICall *call);
static int callback_capabilites(void *object, MSICall *call); static int callback_capabilities(void *object, MSICall *call);
static bool audio_bit_rate_invalid(uint32_t bit_rate); static bool audio_bit_rate_invalid(uint32_t bit_rate);
static bool video_bit_rate_invalid(uint32_t bit_rate); static bool video_bit_rate_invalid(uint32_t bit_rate);
static bool invoke_call_state_callback(ToxAV *av, uint32_t friend_number, uint32_t state); static bool invoke_call_state_callback(ToxAV *av, Tox_Friend_Number friend_number, uint32_t state);
static ToxAVCall *call_new(ToxAV *av, uint32_t friend_number, Toxav_Err_Call *error); static ToxAVCall *call_new(ToxAV *av, Tox_Friend_Number friend_number, Toxav_Err_Call *error);
static ToxAVCall *call_remove(ToxAVCall *call); static ToxAVCall *call_remove(ToxAVCall *call);
static bool call_prepare_transmission(ToxAVCall *call); static bool call_prepare_transmission(ToxAVCall *call);
static void call_kill_transmission(ToxAVCall *call); static void call_kill_transmission(ToxAVCall *call);
MSISession *tox_av_msi_get(const ToxAV *av) static ToxAVCall *call_get(ToxAV *av, uint32_t friend_number)
{
if (av == nullptr) {
return nullptr;
}
return av->msi;
}
ToxAVCall *call_get(ToxAV *av, uint32_t friend_number)
{ {
if (av == nullptr) { if (av == nullptr) {
return nullptr; return nullptr;
@@ -153,7 +276,7 @@ ToxAVCall *call_get(ToxAV *av, uint32_t friend_number)
return av->calls[friend_number]; return av->calls[friend_number];
} }
RTPSession *rtp_session_get(ToxAVCall *call, int payload_type) static RTPSession *rtp_session_get(ToxAVCall *call, int payload_type)
{ {
if (call == nullptr) { if (call == nullptr) {
return nullptr; return nullptr;
@@ -166,7 +289,7 @@ RTPSession *rtp_session_get(ToxAVCall *call, int payload_type)
} }
} }
BWController *bwc_controller_get(const ToxAVCall *call) static BWController *bwc_controller_get(const ToxAVCall *call)
{ {
if (call == nullptr) { if (call == nullptr) {
return nullptr; return nullptr;
@@ -190,6 +313,15 @@ static void init_decode_time_stats(DecodeTimeStats *d)
ToxAV *toxav_new(Tox *tox, Toxav_Err_New *error) ToxAV *toxav_new(Tox *tox, Toxav_Err_New *error)
{ {
const MSICallbacks callbacks = {
callback_invite,
callback_start,
callback_end,
callback_error,
callback_error, // peertimeout
callback_capabilities,
};
Toxav_Err_New rc = TOXAV_ERR_NEW_OK; Toxav_Err_New rc = TOXAV_ERR_NEW_OK;
ToxAV *av = nullptr; ToxAV *av = nullptr;
@@ -213,10 +345,13 @@ ToxAV *toxav_new(Tox *tox, Toxav_Err_New *error)
av->mem = tox->sys.mem; av->mem = tox->sys.mem;
av->log = tox->m->log; av->log = tox->m->log;
av->tox = tox; av->tox = tox;
av->msi = msi_new(av->log, av->tox);
rtp_allow_receiving(av->tox); av->msi = msi_new(av->log, msi_send_packet, av->tox, &callbacks, av);
bwc_allow_receiving(av->tox);
tox_callback_friend_lossy_packet_per_pktid(av->tox, handle_rtp_packet, RTP_TYPE_AUDIO);
tox_callback_friend_lossy_packet_per_pktid(av->tox, handle_rtp_packet, RTP_TYPE_VIDEO);
tox_callback_friend_lossy_packet_per_pktid(av->tox, handle_bwc_packet, BWC_PACKET_ID);
tox_callback_friend_lossless_packet_per_pktid(av->tox, handle_msi_packet, PACKET_ID_MSI);
av->toxav_mono_time = mono_time_new(tox->sys.mem, nullptr, nullptr); av->toxav_mono_time = mono_time_new(tox->sys.mem, nullptr, nullptr);
@@ -228,18 +363,10 @@ ToxAV *toxav_new(Tox *tox, Toxav_Err_New *error)
init_decode_time_stats(&av->audio_stats); init_decode_time_stats(&av->audio_stats);
init_decode_time_stats(&av->video_stats); init_decode_time_stats(&av->video_stats);
av->msi->av = av;
// save ToxAV object into toxcore // save ToxAV object into toxcore
tox_set_av_object(av->tox, av); tox_set_av_object(av->tox, av);
msi_callback_invite(av->msi, callback_invite);
msi_callback_start(av->msi, callback_start);
msi_callback_end(av->msi, callback_end);
msi_callback_error(av->msi, callback_error);
msi_callback_peertimeout(av->msi, callback_error);
msi_callback_capabilities(av->msi, callback_capabilites);
RETURN: RETURN:
if (error != nullptr) { if (error != nullptr) {
@@ -269,11 +396,13 @@ void toxav_kill(ToxAV *av)
tox_callback_friend_lossy_packet_per_pktid(av->tox, nullptr, i); tox_callback_friend_lossy_packet_per_pktid(av->tox, nullptr, i);
} }
rtp_stop_receiving(av->tox); tox_callback_friend_lossy_packet_per_pktid(av->tox, nullptr, RTP_TYPE_AUDIO);
bwc_stop_receiving(av->tox); tox_callback_friend_lossy_packet_per_pktid(av->tox, nullptr, RTP_TYPE_VIDEO);
tox_callback_friend_lossy_packet_per_pktid(av->tox, nullptr, BWC_PACKET_ID);
tox_callback_friend_lossless_packet_per_pktid(av->tox, nullptr, PACKET_ID_MSI);
/* To avoid possible deadlocks */ /* To avoid possible deadlocks */
while (av->msi != nullptr && msi_kill(av->log, av->tox, av->msi) != 0) { while (av->msi != nullptr && msi_kill(av->log, av->msi) != 0) {
pthread_mutex_unlock(av->mutex); pthread_mutex_unlock(av->mutex);
pthread_mutex_lock(av->mutex); pthread_mutex_lock(av->mutex);
} }
@@ -305,11 +434,6 @@ Tox *toxav_get_tox(const ToxAV *av)
return av->tox; return av->tox;
} }
const Logger *toxav_get_logger(const ToxAV *av)
{
return av->log;
}
uint32_t toxav_audio_iteration_interval(const ToxAV *av) uint32_t toxav_audio_iteration_interval(const ToxAV *av)
{ {
return av->calls != nullptr ? av->audio_stats.interval : IDLE_ITERATION_INTERVAL_MS; return av->calls != nullptr ? av->audio_stats.interval : IDLE_ITERATION_INTERVAL_MS;
@@ -372,9 +496,11 @@ static void iterate_common(ToxAV *av, bool audio)
pthread_mutex_unlock(av->mutex); pthread_mutex_unlock(av->mutex);
const uint32_t fid = i->friend_number; const uint32_t fid = i->friend_number;
const bool is_offline = check_peer_offline_status(av->log, av->tox, i->msi_call->session, fid); Tox_Err_Friend_Query f_con_query_error;
const bool is_offline = tox_friend_get_connection_status(av->tox, fid, &f_con_query_error) == TOX_CONNECTION_NONE;
if (is_offline) { if (is_offline) {
msi_call_timeout(i->msi_call->session, av->log, fid);
pthread_mutex_unlock(i->toxav_call_mutex); pthread_mutex_unlock(i->toxav_call_mutex);
pthread_mutex_lock(av->mutex); pthread_mutex_lock(av->mutex);
break; break;
@@ -385,16 +511,16 @@ static void iterate_common(ToxAV *av, bool audio)
if ((i->msi_call->self_capabilities & MSI_CAP_R_AUDIO) != 0 && if ((i->msi_call->self_capabilities & MSI_CAP_R_AUDIO) != 0 &&
(i->msi_call->peer_capabilities & MSI_CAP_S_AUDIO) != 0) { (i->msi_call->peer_capabilities & MSI_CAP_S_AUDIO) != 0) {
frame_time = min_s32(i->audio->lp_frame_duration, frame_time); frame_time = min_s32(ac_get_lp_frame_duration(i->audio), frame_time);
} }
} else { } else {
vc_iterate(i->video); vc_iterate(i->video);
if ((i->msi_call->self_capabilities & MSI_CAP_R_VIDEO) != 0 && if ((i->msi_call->self_capabilities & MSI_CAP_R_VIDEO) != 0 &&
(i->msi_call->peer_capabilities & MSI_CAP_S_VIDEO) != 0) { (i->msi_call->peer_capabilities & MSI_CAP_S_VIDEO) != 0) {
pthread_mutex_lock(i->video->queue_mutex); pthread_mutex_lock(vc_get_queue_mutex(i->video));
frame_time = min_s32(i->video->lcfd, frame_time); frame_time = min_s32(vc_get_lcfd(i->video), frame_time);
pthread_mutex_unlock(i->video->queue_mutex); pthread_mutex_unlock(vc_get_queue_mutex(i->video));
} }
} }
@@ -429,7 +555,7 @@ void toxav_iterate(ToxAV *av)
toxav_video_iterate(av); toxav_video_iterate(av);
} }
bool toxav_call(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate, bool toxav_call(ToxAV *av, Tox_Friend_Number friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate,
Toxav_Err_Call *error) Toxav_Err_Call *error)
{ {
Toxav_Err_Call rc = TOXAV_ERR_CALL_OK; Toxav_Err_Call rc = TOXAV_ERR_CALL_OK;
@@ -463,7 +589,7 @@ bool toxav_call(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, uint
goto RETURN; goto RETURN;
} }
call->msi_call->av_call = call; call->msi_call->user_data = call;
RETURN: RETURN:
pthread_mutex_unlock(av->mutex); pthread_mutex_unlock(av->mutex);
@@ -483,7 +609,7 @@ void toxav_callback_call(ToxAV *av, toxav_call_cb *callback, void *user_data)
pthread_mutex_unlock(av->mutex); pthread_mutex_unlock(av->mutex);
} }
bool toxav_answer(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate, bool toxav_answer(ToxAV *av, Tox_Friend_Number friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate,
Toxav_Err_Answer *error) Toxav_Err_Answer *error)
{ {
pthread_mutex_lock(av->mutex); pthread_mutex_lock(av->mutex);
@@ -683,7 +809,7 @@ static Toxav_Err_Call_Control call_control_handle(ToxAVCall *call, Toxav_Call_Co
return TOXAV_ERR_CALL_CONTROL_INVALID_TRANSITION; return TOXAV_ERR_CALL_CONTROL_INVALID_TRANSITION;
} }
static Toxav_Err_Call_Control call_control(ToxAV *av, uint32_t friend_number, Toxav_Call_Control control) static Toxav_Err_Call_Control call_control(ToxAV *av, Tox_Friend_Number friend_number, Toxav_Call_Control control)
{ {
if (!tox_friend_exists(av->tox, friend_number)) { if (!tox_friend_exists(av->tox, friend_number)) {
return TOXAV_ERR_CALL_CONTROL_FRIEND_NOT_FOUND; return TOXAV_ERR_CALL_CONTROL_FRIEND_NOT_FOUND;
@@ -697,7 +823,7 @@ static Toxav_Err_Call_Control call_control(ToxAV *av, uint32_t friend_number, To
return call_control_handle(call, control); return call_control_handle(call, control);
} }
bool toxav_call_control(ToxAV *av, uint32_t friend_number, Toxav_Call_Control control, Toxav_Err_Call_Control *error) bool toxav_call_control(ToxAV *av, Tox_Friend_Number friend_number, Toxav_Call_Control control, Toxav_Err_Call_Control *error)
{ {
pthread_mutex_lock(av->mutex); pthread_mutex_lock(av->mutex);
@@ -712,7 +838,7 @@ bool toxav_call_control(ToxAV *av, uint32_t friend_number, Toxav_Call_Control co
return rc == TOXAV_ERR_CALL_CONTROL_OK; return rc == TOXAV_ERR_CALL_CONTROL_OK;
} }
bool toxav_audio_set_bit_rate(ToxAV *av, uint32_t friend_number, uint32_t bit_rate, bool toxav_audio_set_bit_rate(ToxAV *av, Tox_Friend_Number friend_number, uint32_t bit_rate,
Toxav_Err_Bit_Rate_Set *error) Toxav_Err_Bit_Rate_Set *error)
{ {
Toxav_Err_Bit_Rate_Set rc = TOXAV_ERR_BIT_RATE_SET_OK; Toxav_Err_Bit_Rate_Set rc = TOXAV_ERR_BIT_RATE_SET_OK;
@@ -785,7 +911,7 @@ RETURN:
return rc == TOXAV_ERR_BIT_RATE_SET_OK; return rc == TOXAV_ERR_BIT_RATE_SET_OK;
} }
bool toxav_video_set_bit_rate(ToxAV *av, uint32_t friend_number, uint32_t bit_rate, bool toxav_video_set_bit_rate(ToxAV *av, Tox_Friend_Number friend_number, uint32_t bit_rate,
Toxav_Err_Bit_Rate_Set *error) Toxav_Err_Bit_Rate_Set *error)
{ {
Toxav_Err_Bit_Rate_Set rc = TOXAV_ERR_BIT_RATE_SET_OK; Toxav_Err_Bit_Rate_Set rc = TOXAV_ERR_BIT_RATE_SET_OK;
@@ -874,7 +1000,7 @@ void toxav_callback_video_bit_rate(ToxAV *av, toxav_video_bit_rate_cb *callback,
pthread_mutex_unlock(av->mutex); pthread_mutex_unlock(av->mutex);
} }
bool toxav_audio_send_frame(ToxAV *av, uint32_t friend_number, const int16_t *pcm, size_t sample_count, bool toxav_audio_send_frame(ToxAV *av, Tox_Friend_Number friend_number, const int16_t pcm[], size_t sample_count,
uint8_t channels, uint32_t sampling_rate, Toxav_Err_Send_Frame *error) uint8_t channels, uint32_t sampling_rate, Toxav_Err_Send_Frame *error)
{ {
Toxav_Err_Send_Frame rc = TOXAV_ERR_SEND_FRAME_OK; Toxav_Err_Send_Frame rc = TOXAV_ERR_SEND_FRAME_OK;
@@ -934,11 +1060,10 @@ bool toxav_audio_send_frame(ToxAV *av, uint32_t friend_number, const int16_t *pc
sampling_rate = net_htonl(sampling_rate); sampling_rate = net_htonl(sampling_rate);
memcpy(dest, &sampling_rate, sizeof(sampling_rate)); memcpy(dest, &sampling_rate, sizeof(sampling_rate));
const int vrc = opus_encode(call->audio->encoder, pcm, sample_count, const int vrc = ac_encode(call->audio, pcm, sample_count,
dest + sizeof(sampling_rate), dest_size - sizeof(sampling_rate)); dest + sizeof(sampling_rate), dest_size - sizeof(sampling_rate));
if (vrc < 0) { if (vrc < 0) {
LOGGER_WARNING(av->log, "Failed to encode frame %s", opus_strerror(vrc));
pthread_mutex_unlock(call->mutex_audio); pthread_mutex_unlock(call->mutex_audio);
rc = TOXAV_ERR_SEND_FRAME_INVALID; rc = TOXAV_ERR_SEND_FRAME_INVALID;
goto RETURN; goto RETURN;
@@ -963,26 +1088,16 @@ RETURN:
static Toxav_Err_Send_Frame send_frames(const ToxAV *av, ToxAVCall *call) static Toxav_Err_Send_Frame send_frames(const ToxAV *av, ToxAVCall *call)
{ {
vpx_codec_iter_t iter = nullptr; uint8_t *data;
uint32_t size;
for (const vpx_codec_cx_pkt_t *pkt = vpx_codec_get_cx_data(call->video->encoder, &iter); bool is_keyframe;
pkt != nullptr;
pkt = vpx_codec_get_cx_data(call->video->encoder, &iter)) {
if (pkt->kind != VPX_CODEC_CX_FRAME_PKT) {
continue;
}
const bool is_keyframe = (pkt->data.frame.flags & VPX_FRAME_IS_KEY) != 0;
// https://www.webmproject.org/docs/webm-sdk/structvpx__codec__cx__pkt.html
// pkt->data.frame.sz -> size_t
const uint32_t frame_length_in_bytes = pkt->data.frame.sz;
while (vc_get_cx_data(call->video, &data, &size, &is_keyframe)) {
const int res = rtp_send_data( const int res = rtp_send_data(
av->log, av->log,
call->video_rtp, call->video_rtp,
(const uint8_t *)pkt->data.frame.buf, data,
frame_length_in_bytes, size,
is_keyframe); is_keyframe);
if (res < 0) { if (res < 0) {
@@ -995,13 +1110,13 @@ static Toxav_Err_Send_Frame send_frames(const ToxAV *av, ToxAVCall *call)
return TOXAV_ERR_SEND_FRAME_OK; return TOXAV_ERR_SEND_FRAME_OK;
} }
bool toxav_video_send_frame(ToxAV *av, uint32_t friend_number, uint16_t width, uint16_t height, const uint8_t *y, bool toxav_video_send_frame(ToxAV *av, Tox_Friend_Number friend_number, uint16_t width, uint16_t height,
const uint8_t *u, const uint8_t *v, Toxav_Err_Send_Frame *error) const uint8_t y[], const uint8_t u[], const uint8_t v[], Toxav_Err_Send_Frame *error)
{ {
Toxav_Err_Send_Frame rc = TOXAV_ERR_SEND_FRAME_OK; Toxav_Err_Send_Frame rc = TOXAV_ERR_SEND_FRAME_OK;
ToxAVCall *call; ToxAVCall *call;
int vpx_encode_flags = 0; int video_encode_flags = 0;
if (!tox_friend_exists(av->tox, friend_number)) { if (!tox_friend_exists(av->tox, friend_number)) {
rc = TOXAV_ERR_SEND_FRAME_FRIEND_NOT_FOUND; rc = TOXAV_ERR_SEND_FRAME_FRIEND_NOT_FOUND;
@@ -1045,58 +1160,27 @@ bool toxav_video_send_frame(ToxAV *av, uint32_t friend_number, uint16_t width, u
} }
// we start with I-frames (full frames) and then switch to normal mode later // we start with I-frames (full frames) and then switch to normal mode later
if (call->video_rtp->ssrc < VIDEO_SEND_X_KEYFRAMES_FIRST) { if (rtp_session_get_ssrc(call->video_rtp) < VIDEO_SEND_X_KEYFRAMES_FIRST) {
// Key frame flag for first frames // Key frame flag for first frames
vpx_encode_flags = VPX_EFLAG_FORCE_KF; video_encode_flags = VC_EFLAG_FORCE_KF;
LOGGER_DEBUG(av->log, "I_FRAME_FLAG:%u only-i-frame mode", call->video_rtp->ssrc); LOGGER_DEBUG(av->log, "I_FRAME_FLAG:%u only-i-frame mode", rtp_session_get_ssrc(call->video_rtp));
++call->video_rtp->ssrc; rtp_session_set_ssrc(call->video_rtp, rtp_session_get_ssrc(call->video_rtp) + 1);
} else if (call->video_rtp->ssrc == VIDEO_SEND_X_KEYFRAMES_FIRST) { } else if (rtp_session_get_ssrc(call->video_rtp) == VIDEO_SEND_X_KEYFRAMES_FIRST) {
// normal keyframe placement // normal keyframe placement
vpx_encode_flags = 0; video_encode_flags = VC_EFLAG_NONE;
LOGGER_DEBUG(av->log, "I_FRAME_FLAG:%u normal mode", call->video_rtp->ssrc); LOGGER_DEBUG(av->log, "I_FRAME_FLAG:%u normal mode", rtp_session_get_ssrc(call->video_rtp));
++call->video_rtp->ssrc; rtp_session_set_ssrc(call->video_rtp, rtp_session_get_ssrc(call->video_rtp) + 1);
} }
{ /* Encode */ if (vc_encode(call->video, width, height, y, u, v, video_encode_flags) != 0) {
vpx_image_t img;
// TODO(Green-Sky): figure out stride_align
// TODO(Green-Sky): check memory alignment?
if (vpx_img_wrap(&img, VPX_IMG_FMT_I420, width, height, 0, (uint8_t *)y) != nullptr) {
// vpx_img_wrap assumes contigues memory, so we fix that
img.planes[VPX_PLANE_U] = (uint8_t *)u;
img.planes[VPX_PLANE_V] = (uint8_t *)v;
} else {
// call to wrap failed, falling back to copy
img.w = 0;
img.h = 0;
img.d_w = 0;
img.d_h = 0;
vpx_img_alloc(&img, VPX_IMG_FMT_I420, width, height, 0);
/* I420 "It comprises an NxM Y plane followed by (N/2)x(M/2) V and U planes."
* http://fourcc.org/yuv.php#IYUV
*/
memcpy(img.planes[VPX_PLANE_Y], y, width * height);
memcpy(img.planes[VPX_PLANE_U], u, (width / 2) * (height / 2));
memcpy(img.planes[VPX_PLANE_V], v, (width / 2) * (height / 2));
}
const vpx_codec_err_t vrc = vpx_codec_encode(call->video->encoder, &img,
call->video->frame_counter, 1, vpx_encode_flags, VPX_DL_REALTIME);
vpx_img_free(&img);
if (vrc != VPX_CODEC_OK) {
pthread_mutex_unlock(call->mutex_video); pthread_mutex_unlock(call->mutex_video);
LOGGER_ERROR(av->log, "Could not encode video frame: %s", vpx_codec_err_to_string(vrc));
rc = TOXAV_ERR_SEND_FRAME_INVALID; rc = TOXAV_ERR_SEND_FRAME_INVALID;
goto RETURN; goto RETURN;
} }
}
++call->video->frame_counter; vc_increment_frame_counter(call->video);
rc = send_frames(av, call); rc = send_frames(av, call);
@@ -1116,6 +1200,16 @@ void toxav_callback_audio_receive_frame(ToxAV *av, toxav_audio_receive_frame_cb
pthread_mutex_lock(av->mutex); pthread_mutex_lock(av->mutex);
av->acb = callback; av->acb = callback;
av->acb_user_data = user_data; av->acb_user_data = user_data;
if (av->calls != nullptr) {
for (ToxAVCall *i = av->calls[av->calls_head]; i != nullptr; i = i->next) {
pthread_mutex_lock(i->toxav_call_mutex);
i->acb = callback;
i->acb_user_data = user_data;
pthread_mutex_unlock(i->toxav_call_mutex);
}
}
pthread_mutex_unlock(av->mutex); pthread_mutex_unlock(av->mutex);
} }
@@ -1132,7 +1226,7 @@ void toxav_callback_video_receive_frame(ToxAV *av, toxav_video_receive_frame_cb
* :: Internal * :: Internal
* *
******************************************************************************/ ******************************************************************************/
static void callback_bwc(BWController *bwc, uint32_t friend_number, float loss, void *user_data) static void callback_bwc(BWController *bwc, Tox_Friend_Number friend_number, float loss, void *user_data)
{ {
/* Callback which is called when the internal measure mechanism reported packet loss. /* Callback which is called when the internal measure mechanism reported packet loss.
* We report suggested lowered bitrate to an app. If app is sending both audio and video, * We report suggested lowered bitrate to an app. If app is sending both audio and video,
@@ -1191,7 +1285,7 @@ static int callback_invite(void *object, MSICall *call)
return -1; return -1;
} }
call->av_call = av_call; call->user_data = av_call;
av_call->msi_call = call; av_call->msi_call = call;
if (toxav->ccb != nullptr) { if (toxav->ccb != nullptr) {
@@ -1243,9 +1337,9 @@ static int callback_end(void *object, MSICall *call)
invoke_call_state_callback(toxav, call->friend_number, TOXAV_FRIEND_CALL_STATE_FINISHED); invoke_call_state_callback(toxav, call->friend_number, TOXAV_FRIEND_CALL_STATE_FINISHED);
if (call->av_call != nullptr) { if (call->user_data != nullptr) {
call_kill_transmission(call->av_call); call_kill_transmission((ToxAVCall *)call->user_data);
call_remove(call->av_call); call_remove((ToxAVCall *)call->user_data);
} }
pthread_mutex_unlock(toxav->mutex); pthread_mutex_unlock(toxav->mutex);
@@ -1259,30 +1353,32 @@ static int callback_error(void *object, MSICall *call)
invoke_call_state_callback(toxav, call->friend_number, TOXAV_FRIEND_CALL_STATE_ERROR); invoke_call_state_callback(toxav, call->friend_number, TOXAV_FRIEND_CALL_STATE_ERROR);
if (call->av_call != nullptr) { if (call->user_data != nullptr) {
call_kill_transmission(call->av_call); call_kill_transmission((ToxAVCall *)call->user_data);
call_remove(call->av_call); call_remove((ToxAVCall *)call->user_data);
} }
pthread_mutex_unlock(toxav->mutex); pthread_mutex_unlock(toxav->mutex);
return 0; return 0;
} }
static int callback_capabilites(void *object, MSICall *call) static int callback_capabilities(void *object, MSICall *call)
{ {
ToxAV *toxav = (ToxAV *)object; ToxAV *toxav = (ToxAV *)object;
pthread_mutex_lock(toxav->mutex); pthread_mutex_lock(toxav->mutex);
ToxAVCall *av_call = (ToxAVCall *)call->user_data;
if ((call->peer_capabilities & MSI_CAP_S_AUDIO) != 0) { if ((call->peer_capabilities & MSI_CAP_S_AUDIO) != 0) {
rtp_allow_receiving_mark(call->av_call->audio_rtp); rtp_allow_receiving_mark(av_call->audio_rtp);
} else { } else {
rtp_stop_receiving_mark(call->av_call->audio_rtp); rtp_stop_receiving_mark(av_call->audio_rtp);
} }
if ((call->peer_capabilities & MSI_CAP_S_VIDEO) != 0) { if ((call->peer_capabilities & MSI_CAP_S_VIDEO) != 0) {
rtp_allow_receiving_mark(call->av_call->video_rtp); rtp_allow_receiving_mark(av_call->video_rtp);
} else { } else {
rtp_stop_receiving_mark(call->av_call->video_rtp); rtp_stop_receiving_mark(av_call->video_rtp);
} }
invoke_call_state_callback(toxav, call->friend_number, call->peer_capabilities); invoke_call_state_callback(toxav, call->friend_number, call->peer_capabilities);
@@ -1308,7 +1404,7 @@ static bool video_bit_rate_invalid(uint32_t bit_rate)
return bit_rate > 1000000; return bit_rate > 1000000;
} }
static bool invoke_call_state_callback(ToxAV *av, uint32_t friend_number, uint32_t state) static bool invoke_call_state_callback(ToxAV *av, Tox_Friend_Number friend_number, uint32_t state)
{ {
if (av->scb != nullptr) { if (av->scb != nullptr) {
av->scb(av, friend_number, state, av->scb_user_data); av->scb(av, friend_number, state, av->scb_user_data);
@@ -1319,7 +1415,7 @@ static bool invoke_call_state_callback(ToxAV *av, uint32_t friend_number, uint32
return true; return true;
} }
static ToxAVCall *call_new(ToxAV *av, uint32_t friend_number, Toxav_Err_Call *error) static ToxAVCall *call_new(ToxAV *av, Tox_Friend_Number friend_number, Toxav_Err_Call *error)
{ {
/* Assumes mutex locked */ /* Assumes mutex locked */
Toxav_Err_Call rc = TOXAV_ERR_CALL_OK; Toxav_Err_Call rc = TOXAV_ERR_CALL_OK;
@@ -1430,7 +1526,7 @@ static ToxAVCall *call_remove(ToxAVCall *call)
* removed from the msi call. * removed from the msi call.
*/ */
if (call->msi_call != nullptr) { if (call->msi_call != nullptr) {
call->msi_call->av_call = nullptr; call->msi_call->user_data = nullptr;
} }
pthread_mutex_destroy(call->toxav_call_mutex); pthread_mutex_destroy(call->toxav_call_mutex);
@@ -1493,17 +1589,21 @@ static bool call_prepare_transmission(ToxAVCall *call)
} }
/* Prepare bwc */ /* Prepare bwc */
call->bwc = bwc_new(av->log, av->tox, call->friend_number, callback_bwc, call, av->toxav_mono_time); call->bwc = bwc_new(av->log, call->friend_number, callback_bwc, call, rtp_send_packet, call, av->toxav_mono_time);
{ /* Prepare audio */ { /* Prepare audio */
call->audio = ac_new(av->toxav_mono_time, av->log, av, call->friend_number, av->acb, av->acb_user_data); call->acb = av->acb;
call->acb_user_data = av->acb_user_data;
call->audio = ac_new(av->toxav_mono_time, av->log, call->friend_number, handle_audio_frame, call);
if (call->audio == nullptr) { if (call->audio == nullptr) {
LOGGER_ERROR(av->log, "Failed to create audio codec session"); LOGGER_ERROR(av->log, "Failed to create audio codec session");
goto FAILURE; goto FAILURE;
} }
call->audio_rtp = rtp_new(av->log, av->mem, RTP_TYPE_AUDIO, av->tox, av, call->friend_number, call->bwc, call->audio_rtp = rtp_new(av->log, RTP_TYPE_AUDIO, av->toxav_mono_time,
rtp_send_packet, call,
rtp_add_recv, rtp_add_lost, call->bwc,
call->audio, ac_queue_message); call->audio, ac_queue_message);
if (call->audio_rtp == nullptr) { if (call->audio_rtp == nullptr) {
@@ -1512,14 +1612,16 @@ static bool call_prepare_transmission(ToxAVCall *call)
} }
} }
{ /* Prepare video */ { /* Prepare video */
call->video = vc_new(av->log, av->toxav_mono_time, av, call->friend_number, av->vcb, av->vcb_user_data); call->video = vc_new(av->log, av->toxav_mono_time, call->friend_number, handle_video_frame, call);
if (call->video == nullptr) { if (call->video == nullptr) {
LOGGER_ERROR(av->log, "Failed to create video codec session"); LOGGER_ERROR(av->log, "Failed to create video codec session");
goto FAILURE; goto FAILURE;
} }
call->video_rtp = rtp_new(av->log, av->mem, RTP_TYPE_VIDEO, av->tox, av, call->friend_number, call->bwc, call->video_rtp = rtp_new(av->log, RTP_TYPE_VIDEO, av->toxav_mono_time,
rtp_send_packet, call,
rtp_add_recv, rtp_add_lost, call->bwc,
call->video, vc_queue_message); call->video, vc_queue_message);
if (call->video_rtp == nullptr) { if (call->video_rtp == nullptr) {
@@ -1579,12 +1681,3 @@ static void call_kill_transmission(ToxAVCall *call)
pthread_mutex_destroy(call->mutex_audio); pthread_mutex_destroy(call->mutex_audio);
pthread_mutex_destroy(call->mutex_video); pthread_mutex_destroy(call->mutex_video);
} }
Mono_Time *toxav_get_av_mono_time(const ToxAV *av)
{
if (av == nullptr) {
return nullptr;
}
return av->toxav_mono_time;
}

View File

@@ -66,12 +66,27 @@ extern "C" {
/** /**
* External Tox type. * External Tox type.
*/ */
#ifndef APIGEN_IGNORE
#ifndef TOX_DEFINED #ifndef TOX_DEFINED
#define TOX_DEFINED #define TOX_DEFINED
typedef struct Tox Tox; typedef struct Tox Tox;
#endif /* !TOX_DEFINED */ #endif /* !TOX_DEFINED */
#endif /* !APIGEN_IGNORE */
#ifndef TOX_CONFERENCE_NUMBER_DEFINED
#define TOX_CONFERENCE_NUMBER_DEFINED
typedef uint32_t Tox_Conference_Number;
#endif /* !TOX_CONFERENCE_NUMBER_DEFINED */
#ifndef TOX_FRIEND_NUMBER_DEFINED
#define TOX_FRIEND_NUMBER_DEFINED
typedef uint32_t Tox_Friend_Number;
#endif /* !TOX_FRIEND_NUMBER_DEFINED */
#ifndef TOX_CONFERENCE_PEER_NUMBER_DEFINED
#define TOX_CONFERENCE_PEER_NUMBER_DEFINED
typedef uint32_t Tox_Conference_Peer_Number;
#endif /* !TOX_CONFERENCE_PEER_NUMBER_DEFINED */
#ifndef TOXAV_DEFINED #ifndef TOXAV_DEFINED
#define TOXAV_DEFINED #define TOXAV_DEFINED
@@ -254,7 +269,7 @@ typedef enum Toxav_Err_Call {
* @param video_bit_rate Video bit rate in kbit/sec. Set this to 0 to disable * @param video_bit_rate Video bit rate in kbit/sec. Set this to 0 to disable
* video sending. * video sending.
*/ */
bool toxav_call(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate, bool toxav_call(ToxAV *av, Tox_Friend_Number friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate,
Toxav_Err_Call *error); Toxav_Err_Call *error);
/** /**
@@ -264,7 +279,7 @@ bool toxav_call(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, uint
* @param audio_enabled True if friend is sending audio. * @param audio_enabled True if friend is sending audio.
* @param video_enabled True if friend is sending video. * @param video_enabled True if friend is sending video.
*/ */
typedef void toxav_call_cb(ToxAV *av, uint32_t friend_number, bool audio_enabled, bool video_enabled, void *user_data); typedef void toxav_call_cb(ToxAV *av, Tox_Friend_Number friend_number, bool audio_enabled, bool video_enabled, void *user_data);
/** /**
* Set the callback for the `call` event. Pass NULL to unset. * Set the callback for the `call` event. Pass NULL to unset.
@@ -323,7 +338,7 @@ typedef enum Toxav_Err_Answer {
* @param video_bit_rate Video bit rate in kbit/sec. Set this to 0 to disable * @param video_bit_rate Video bit rate in kbit/sec. Set this to 0 to disable
* video sending. * video sending.
*/ */
bool toxav_answer(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate, bool toxav_answer(ToxAV *av, Tox_Friend_Number friend_number, uint32_t audio_bit_rate, uint32_t video_bit_rate,
Toxav_Err_Answer *error); Toxav_Err_Answer *error);
/** @} */ /** @} */
@@ -385,7 +400,7 @@ enum Toxav_Friend_Call_State {
* paused. The bitmask represents all the activities currently performed by * paused. The bitmask represents all the activities currently performed by
* the friend. * the friend.
*/ */
typedef void toxav_call_state_cb(ToxAV *av, uint32_t friend_number, uint32_t state, void *user_data); typedef void toxav_call_state_cb(ToxAV *av, Tox_Friend_Number friend_number, uint32_t state, void *user_data);
/** /**
* Set the callback for the `call_state` event. Pass NULL to unset. * Set the callback for the `call_state` event. Pass NULL to unset.
@@ -485,7 +500,7 @@ typedef enum Toxav_Err_Call_Control {
* *
* @return true on success. * @return true on success.
*/ */
bool toxav_call_control(ToxAV *av, uint32_t friend_number, Toxav_Call_Control control, Toxav_Err_Call_Control *error); bool toxav_call_control(ToxAV *av, Tox_Friend_Number friend_number, Toxav_Call_Control control, Toxav_Err_Call_Control *error);
/** @} */ /** @} */
@@ -596,8 +611,8 @@ typedef enum Toxav_Err_Send_Frame {
* @param sampling_rate Audio sampling rate used in this frame. Valid sampling * @param sampling_rate Audio sampling rate used in this frame. Valid sampling
* rates are 8000, 12000, 16000, 24000, or 48000. * rates are 8000, 12000, 16000, 24000, or 48000.
*/ */
bool toxav_audio_send_frame(ToxAV *av, uint32_t friend_number, const int16_t pcm[], size_t sample_count, bool toxav_audio_send_frame(ToxAV *av, Tox_Friend_Number friend_number, const int16_t pcm[/*! sample_count * channels */],
uint8_t channels, uint32_t sampling_rate, Toxav_Err_Send_Frame *error); size_t sample_count, uint8_t channels, uint32_t sampling_rate, Toxav_Err_Send_Frame *error);
/** /**
* Set the bit rate to be used in subsequent audio frames. * Set the bit rate to be used in subsequent audio frames.
@@ -608,7 +623,7 @@ bool toxav_audio_send_frame(ToxAV *av, uint32_t friend_number, const int16_t pcm
* *
* @return true on success. * @return true on success.
*/ */
bool toxav_audio_set_bit_rate(ToxAV *av, uint32_t friend_number, uint32_t bit_rate, Toxav_Err_Bit_Rate_Set *error); bool toxav_audio_set_bit_rate(ToxAV *av, Tox_Friend_Number friend_number, uint32_t bit_rate, Toxav_Err_Bit_Rate_Set *error);
/** /**
* The function type for the audio_bit_rate callback. The event is triggered * The function type for the audio_bit_rate callback. The event is triggered
@@ -619,7 +634,7 @@ bool toxav_audio_set_bit_rate(ToxAV *av, uint32_t friend_number, uint32_t bit_ra
* bit rate. * bit rate.
* @param audio_bit_rate Suggested maximum audio bit rate in kbit/sec. * @param audio_bit_rate Suggested maximum audio bit rate in kbit/sec.
*/ */
typedef void toxav_audio_bit_rate_cb(ToxAV *av, uint32_t friend_number, uint32_t audio_bit_rate, void *user_data); typedef void toxav_audio_bit_rate_cb(ToxAV *av, Tox_Friend_Number friend_number, uint32_t audio_bit_rate, void *user_data);
/** /**
* Set the callback for the `audio_bit_rate` event. Pass NULL to unset. * Set the callback for the `audio_bit_rate` event. Pass NULL to unset.
@@ -644,7 +659,7 @@ void toxav_callback_audio_bit_rate(ToxAV *av, toxav_audio_bit_rate_cb *callback,
* @param v V (Chroma) plane data. * @param v V (Chroma) plane data.
*/ */
bool toxav_video_send_frame( bool toxav_video_send_frame(
ToxAV *av, uint32_t friend_number, uint16_t width, uint16_t height, ToxAV *av, Tox_Friend_Number friend_number, uint16_t width, uint16_t height,
const uint8_t y[/*! width * height */], const uint8_t y[/*! width * height */],
const uint8_t u[/*! width/2 * height/2 */], const uint8_t u[/*! width/2 * height/2 */],
const uint8_t v[/*! width/2 * height/2 */], const uint8_t v[/*! width/2 * height/2 */],
@@ -659,7 +674,7 @@ bool toxav_video_send_frame(
* *
* @return true on success. * @return true on success.
*/ */
bool toxav_video_set_bit_rate(ToxAV *av, uint32_t friend_number, uint32_t bit_rate, Toxav_Err_Bit_Rate_Set *error); bool toxav_video_set_bit_rate(ToxAV *av, Tox_Friend_Number friend_number, uint32_t bit_rate, Toxav_Err_Bit_Rate_Set *error);
/** /**
* The function type for the video_bit_rate callback. The event is triggered * The function type for the video_bit_rate callback. The event is triggered
@@ -670,7 +685,7 @@ bool toxav_video_set_bit_rate(ToxAV *av, uint32_t friend_number, uint32_t bit_ra
* bit rate. * bit rate.
* @param video_bit_rate Suggested maximum video bit rate in kbit/sec. * @param video_bit_rate Suggested maximum video bit rate in kbit/sec.
*/ */
typedef void toxav_video_bit_rate_cb(ToxAV *av, uint32_t friend_number, uint32_t video_bit_rate, void *user_data); typedef void toxav_video_bit_rate_cb(ToxAV *av, Tox_Friend_Number friend_number, uint32_t video_bit_rate, void *user_data);
/** /**
* Set the callback for the `video_bit_rate` event. Pass NULL to unset. * Set the callback for the `video_bit_rate` event. Pass NULL to unset.
@@ -696,7 +711,7 @@ void toxav_callback_video_bit_rate(ToxAV *av, toxav_video_bit_rate_cb *callback,
* @param sampling_rate Sampling rate used in this frame. * @param sampling_rate Sampling rate used in this frame.
* *
*/ */
typedef void toxav_audio_receive_frame_cb(ToxAV *av, uint32_t friend_number, const int16_t pcm[], size_t sample_count, typedef void toxav_audio_receive_frame_cb(ToxAV *av, Tox_Friend_Number friend_number, const int16_t pcm[/*! sample_count * channels */], size_t sample_count,
uint8_t channels, uint32_t sampling_rate, void *user_data); uint8_t channels, uint32_t sampling_rate, void *user_data);
/** /**
@@ -727,7 +742,7 @@ void toxav_callback_audio_receive_frame(ToxAV *av, toxav_audio_receive_frame_cb
* @param vstride V chroma plane stride. * @param vstride V chroma plane stride.
*/ */
typedef void toxav_video_receive_frame_cb( typedef void toxav_video_receive_frame_cb(
ToxAV *av, uint32_t friend_number, ToxAV *av, Tox_Friend_Number friend_number,
uint16_t width, uint16_t height, uint16_t width, uint16_t height,
const uint8_t y[/*! max(width, abs(ystride)) * height */], const uint8_t y[/*! max(width, abs(ystride)) * height */],
const uint8_t u[/*! max(width/2, abs(ustride)) * (height/2) */], const uint8_t u[/*! max(width/2, abs(ustride)) * (height/2) */],
@@ -741,7 +756,7 @@ typedef void toxav_video_receive_frame_cb(
*/ */
void toxav_callback_video_receive_frame(ToxAV *av, toxav_video_receive_frame_cb *callback, void *user_data); void toxav_callback_video_receive_frame(ToxAV *av, toxav_video_receive_frame_cb *callback, void *user_data);
#ifndef APIGEN_IGNORE
/*** /***
* NOTE Compatibility with old ToxAV group calls. TODO(iphydf): remove * NOTE Compatibility with old ToxAV group calls. TODO(iphydf): remove
@@ -751,17 +766,19 @@ void toxav_callback_video_receive_frame(ToxAV *av, toxav_video_receive_frame_cb
* userdata per group. * userdata per group.
*/ */
#ifndef APIGEN_IGNORE
// TODO(iphydf): Use this better typed one instead of the void-pointer one // TODO(iphydf): Use this better typed one instead of the void-pointer one
// below. // below.
typedef void toxav_group_audio_cb(Tox *tox, uint32_t groupnumber, uint32_t peernumber, const int16_t pcm[], typedef void toxav_group_audio_cb(Tox *tox, Tox_Conference_Number conference_number, Tox_Conference_Peer_Number peer_number, const int16_t pcm[],
uint32_t samples, uint8_t channels, uint32_t sample_rate, void *user_data); uint32_t samples, uint8_t channels, uint32_t sample_rate, void *user_data);
#endif /* APIGEN_IGNORE */
typedef void toxav_audio_data_cb(void *tox, uint32_t groupnumber, uint32_t peernumber, const int16_t pcm[], typedef void toxav_audio_data_cb(void *tox, Tox_Conference_Number conference_number, Tox_Conference_Peer_Number peer_number, const int16_t pcm[],
uint32_t samples, uint8_t channels, uint32_t sample_rate, void *userdata); uint32_t samples, uint8_t channels, uint32_t sample_rate, void *userdata);
/** @brief Create a new ToxAV group. /** @brief Create a new ToxAV group.
* *
* @return group number on success. * @return conference number on success.
* @retval -1 on failure. * @retval -1 on failure.
* *
* Note that total size of pcm in bytes is equal to * Note that total size of pcm in bytes is equal to
@@ -771,14 +788,14 @@ int32_t toxav_add_av_groupchat(Tox *tox, toxav_audio_data_cb *audio_callback, vo
/** @brief Join a AV group (you need to have been invited first). /** @brief Join a AV group (you need to have been invited first).
* *
* @return group number on success. * @return conference number on success.
* @retval -1 on failure. * @retval -1 on failure.
* *
* Note that total size of pcm in bytes is equal to * Note that total size of pcm in bytes is equal to
* `samples * channels * sizeof(int16_t)`. * `samples * channels * sizeof(int16_t)`.
*/ */
int32_t toxav_join_av_groupchat( int32_t toxav_join_av_groupchat(
Tox *tox, uint32_t friendnumber, const uint8_t data[], uint16_t length, Tox *tox, Tox_Friend_Number friend_number, const uint8_t data[], uint16_t length,
toxav_audio_data_cb *audio_callback, void *userdata); toxav_audio_data_cb *audio_callback, void *userdata);
/** @brief Send audio to the group chat. /** @brief Send audio to the group chat.
@@ -797,7 +814,7 @@ int32_t toxav_join_av_groupchat(
* Recommended values are: samples = 960, channels = 1, sample_rate = 48000 * Recommended values are: samples = 960, channels = 1, sample_rate = 48000
*/ */
int32_t toxav_group_send_audio( int32_t toxav_group_send_audio(
Tox *tox, uint32_t groupnumber, const int16_t pcm[], uint32_t samples, uint8_t channels, Tox *tox, Tox_Conference_Number conference_number, const int16_t pcm[], uint32_t samples, uint8_t channels,
uint32_t sample_rate); uint32_t sample_rate);
/** @brief Enable A/V in a groupchat. /** @brief Enable A/V in a groupchat.
@@ -817,7 +834,7 @@ int32_t toxav_group_send_audio(
* `samples * channels * sizeof(int16_t)`. * `samples * channels * sizeof(int16_t)`.
*/ */
int32_t toxav_groupchat_enable_av( int32_t toxav_groupchat_enable_av(
Tox *tox, uint32_t groupnumber, Tox *tox, Tox_Conference_Number conference_number,
toxav_audio_data_cb *audio_callback, void *userdata); toxav_audio_data_cb *audio_callback, void *userdata);
/** @brief Disable A/V in a groupchat. /** @brief Disable A/V in a groupchat.
@@ -825,12 +842,12 @@ int32_t toxav_groupchat_enable_av(
* @retval 0 on success. * @retval 0 on success.
* @retval -1 on failure. * @retval -1 on failure.
*/ */
int32_t toxav_groupchat_disable_av(Tox *tox, uint32_t groupnumber); int32_t toxav_groupchat_disable_av(Tox *tox, Tox_Conference_Number conference_number);
/** @brief Return whether A/V is enabled in the groupchat. */ /** @brief Return whether A/V is enabled in the groupchat. */
bool toxav_groupchat_av_enabled(Tox *tox, uint32_t groupnumber); bool toxav_groupchat_av_enabled(Tox *tox, Tox_Conference_Number conference_number);
#endif /* !APIGEN_IGNORE */
/** @} */ /** @} */

View File

@@ -1,29 +0,0 @@
/* SPDX-License-Identifier: GPL-3.0-or-later
* Copyright © 2016-2025 The TokTok team.
* Copyright © 2013-2015 Tox project.
*/
#ifndef C_TOXCORE_TOXAV_HACKS_H
#define C_TOXCORE_TOXAV_HACKS_H
#include "bwcontroller.h"
#include "msi.h"
#include "rtp.h"
#ifndef TOXAV_CALL_DEFINED
#define TOXAV_CALL_DEFINED
typedef struct ToxAVCall ToxAVCall;
#endif /* TOXAV_CALL_DEFINED */
ToxAVCall *_Nullable call_get(ToxAV *_Nonnull av, uint32_t friend_number);
RTPSession *_Nullable rtp_session_get(ToxAVCall *_Nonnull call, int payload_type);
MSISession *_Nullable tox_av_msi_get(const ToxAV *_Nonnull av);
BWController *_Nullable bwc_controller_get(const ToxAVCall *_Nonnull call);
Mono_Time *_Nullable toxav_get_av_mono_time(const ToxAV *_Nonnull av);
const Logger *_Nonnull toxav_get_logger(const ToxAV *_Nonnull av);
#endif /* C_TOXCORE_TOXAV_HACKS_H */

View File

@@ -11,35 +11,39 @@
#include "../toxcore/tox_struct.h" #include "../toxcore/tox_struct.h"
#include "groupav.h" #include "groupav.h"
int toxav_add_av_groupchat(Tox *tox, audio_data_cb *audio_callback, void *userdata) int32_t toxav_add_av_groupchat(Tox *tox, toxav_audio_data_cb *audio_callback, void *userdata)
{ {
return add_av_groupchat(tox->m->log, tox, tox->m->conferences_object, audio_callback, userdata); return add_av_groupchat(tox->m->log, tox, tox->m->conferences_object, (audio_data_cb *)audio_callback, userdata);
} }
int toxav_join_av_groupchat(Tox *tox, uint32_t friendnumber, const uint8_t *data, uint16_t length, int32_t toxav_join_av_groupchat(Tox *tox, Tox_Friend_Number friend_number, const uint8_t *data, uint16_t length,
audio_data_cb *audio_callback, void *userdata) toxav_audio_data_cb *audio_callback, void *userdata)
{ {
return join_av_groupchat(tox->m->log, tox, tox->m->conferences_object, friendnumber, data, length, audio_callback, userdata); return join_av_groupchat(tox->m->log, tox, tox->m->conferences_object, friend_number, data, length,
(audio_data_cb *)audio_callback, userdata);
} }
int toxav_group_send_audio(Tox *tox, uint32_t groupnumber, const int16_t *pcm, unsigned int samples, uint8_t channels, int32_t toxav_group_send_audio(Tox *tox, Tox_Conference_Number conference_number, const int16_t *pcm, uint32_t samples,
uint8_t channels,
uint32_t sample_rate) uint32_t sample_rate)
{ {
return group_send_audio(tox->m->conferences_object, groupnumber, pcm, samples, channels, sample_rate); return group_send_audio(tox->m->conferences_object, conference_number, pcm, samples, channels, sample_rate);
} }
int toxav_groupchat_enable_av(Tox *tox, uint32_t groupnumber, audio_data_cb *audio_callback, void *userdata) int32_t toxav_groupchat_enable_av(Tox *tox, Tox_Conference_Number conference_number, toxav_audio_data_cb *audio_callback,
void *userdata)
{ {
return groupchat_enable_av(tox->m->log, tox, tox->m->conferences_object, groupnumber, audio_callback, userdata); return groupchat_enable_av(tox->m->log, tox, tox->m->conferences_object, conference_number, (audio_data_cb *)audio_callback,
userdata);
} }
int toxav_groupchat_disable_av(Tox *tox, uint32_t groupnumber) int32_t toxav_groupchat_disable_av(Tox *tox, Tox_Conference_Number conference_number)
{ {
return groupchat_disable_av(tox->m->conferences_object, groupnumber); return groupchat_disable_av(tox->m->conferences_object, conference_number);
} }
/** @brief Return whether A/V is enabled in the groupchat. */ /** @brief Return whether A/V is enabled in the groupchat. */
bool toxav_groupchat_av_enabled(Tox *tox, uint32_t groupnumber) bool toxav_groupchat_av_enabled(Tox *tox, Tox_Conference_Number conference_number)
{ {
return groupchat_av_enabled(tox->m->conferences_object, groupnumber); return groupchat_av_enabled(tox->m->conferences_object, conference_number);
} }

View File

@@ -4,6 +4,13 @@
*/ */
#include "video.h" #include "video.h"
#include <vpx/vpx_decoder.h>
#include <vpx/vpx_encoder.h>
#include <vpx/vpx_image.h>
#include <vpx/vp8cx.h>
#include <vpx/vp8dx.h>
#include <assert.h> #include <assert.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
@@ -14,6 +21,31 @@
#include "../toxcore/ccompat.h" #include "../toxcore/ccompat.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/mono_time.h" #include "../toxcore/mono_time.h"
#include "../toxcore/util.h"
struct VCSession {
/* encoding */
vpx_codec_ctx_t encoder[1];
uint32_t frame_counter;
/* decoding */
vpx_codec_ctx_t decoder[1];
struct RingBuffer *vbuf_raw; /* Un-decoded data */
uint64_t linfts; /* Last received frame time stamp */
uint32_t lcfd; /* Last calculated frame duration for incoming video payload */
uint32_t friend_number;
/* Video frame receive callback */
vc_video_receive_frame_cb *vcb;
void *user_data;
pthread_mutex_t queue_mutex[1];
const Logger *log;
vpx_codec_iter_t iter;
};
/** /**
* Codec control function to set encoder internal speed settings. Changes in * Codec control function to set encoder internal speed settings. Changes in
@@ -33,6 +65,12 @@
#define VIDEO_BITRATE_INITIAL_VALUE 5000 #define VIDEO_BITRATE_INITIAL_VALUE 5000
#define VIDEO_DECODE_BUFFER_SIZE 5 // this buffer has normally max. 1 entry #define VIDEO_DECODE_BUFFER_SIZE 5 // this buffer has normally max. 1 entry
/**
* Security limits to prevent resource exhaustion.
*/
#define VIDEO_MAX_FRAME_SIZE (10 * 1024 * 1024)
#define VIDEO_MAX_RESOLUTION_LIMIT 4096
static vpx_codec_iface_t *video_codec_decoder_interface(void) static vpx_codec_iface_t *video_codec_decoder_interface(void)
{ {
return vpx_codec_vp8_dx(); return vpx_codec_vp8_dx();
@@ -120,9 +158,13 @@ static void vc_init_encoder_cfg(const Logger *log, vpx_codec_enc_cfg_t *cfg, int
#endif /* 0 */ #endif /* 0 */
} }
VCSession *vc_new(const Logger *log, const Mono_Time *mono_time, ToxAV *av, uint32_t friend_number, VCSession *vc_new(const Logger *log, const Mono_Time *mono_time, uint32_t friend_number,
toxav_video_receive_frame_cb *cb, void *cb_data) vc_video_receive_frame_cb *cb, void *user_data)
{ {
if (mono_time == nullptr) {
return nullptr;
}
VCSession *vc = (VCSession *)calloc(1, sizeof(VCSession)); VCSession *vc = (VCSession *)calloc(1, sizeof(VCSession));
vpx_codec_err_t rc; vpx_codec_err_t rc;
@@ -231,9 +273,8 @@ VCSession *vc_new(const Logger *log, const Mono_Time *mono_time, ToxAV *av, uint
vc->linfts = current_time_monotonic(mono_time); vc->linfts = current_time_monotonic(mono_time);
vc->lcfd = 60; vc->lcfd = 60;
vc->vcb = cb; vc->vcb = cb;
vc->vcb_user_data = cb_data; vc->user_data = user_data;
vc->friend_number = friend_number; vc->friend_number = friend_number;
vc->av = av;
vc->log = log; vc->log = log;
return vc; return vc;
@@ -285,21 +326,30 @@ void vc_iterate(VCSession *vc)
const uint16_t log_rb_size = rb_size(vc->vbuf_raw); const uint16_t log_rb_size = rb_size(vc->vbuf_raw);
pthread_mutex_unlock(vc->queue_mutex); pthread_mutex_unlock(vc->queue_mutex);
const struct RTPHeader *const header = &p->header;
uint32_t full_data_len; uint32_t full_data_len;
if ((header->flags & RTP_LARGE_FRAME) != 0) { if ((rtp_message_flags(p) & RTP_LARGE_FRAME) != 0) {
full_data_len = header->data_length_full; full_data_len = rtp_message_data_length_full(p);
LOGGER_DEBUG(vc->log, "vc_iterate:001:full_data_len=%d", (int)full_data_len); LOGGER_DEBUG(vc->log, "vc_iterate:001:full_data_len=%u", full_data_len);
} else { } else {
full_data_len = p->len; full_data_len = rtp_message_len(p);
LOGGER_DEBUG(vc->log, "vc_iterate:002"); LOGGER_DEBUG(vc->log, "vc_iterate:002");
} }
LOGGER_DEBUG(vc->log, "vc_iterate: rb_read p->len=%d p->header.xe=%d", (int)full_data_len, p->header.xe); /* Security check: Ensure the reported full data length does not exceed the actual buffer size.
* rtp_message_len(p) returns the actual allocated payload size.
*/
if (full_data_len > rtp_message_len(p)) {
LOGGER_ERROR(vc->log, "vc_iterate: Malicious packet detected! Lying length: %u actual: %u",
full_data_len, (uint32_t)rtp_message_len(p));
free(p);
return;
}
LOGGER_DEBUG(vc->log, "vc_iterate: rb_read p->len=%u", full_data_len);
LOGGER_DEBUG(vc->log, "vc_iterate: rb_read rb size=%d", (int)log_rb_size); LOGGER_DEBUG(vc->log, "vc_iterate: rb_read rb size=%d", (int)log_rb_size);
const vpx_codec_err_t rc = vpx_codec_decode(vc->decoder, p->data, full_data_len, nullptr, 0); const vpx_codec_err_t rc = vpx_codec_decode(vc->decoder, rtp_message_data(p), full_data_len, nullptr, 0);
free(p); free(p);
if (rc != VPX_CODEC_OK) { if (rc != VPX_CODEC_OK) {
@@ -314,12 +364,10 @@ void vc_iterate(VCSession *vc)
dest != nullptr; dest != nullptr;
dest = vpx_codec_get_frame(vc->decoder, &iter)) { dest = vpx_codec_get_frame(vc->decoder, &iter)) {
if (vc->vcb != nullptr) { if (vc->vcb != nullptr) {
vc->vcb(vc->av, vc->friend_number, dest->d_w, dest->d_h, vc->vcb(vc->friend_number, dest->d_w, dest->d_h,
dest->planes[0], dest->planes[1], dest->planes[2], dest->planes[0], dest->planes[1], dest->planes[2],
dest->stride[0], dest->stride[1], dest->stride[2], vc->vcb_user_data); dest->stride[0], dest->stride[1], dest->stride[2], vc->user_data);
} }
vpx_img_free(dest); // is this needed? none of the VPx examples show that
} }
} }
@@ -337,24 +385,29 @@ int vc_queue_message(const Mono_Time *mono_time, void *cs, struct RTPMessage *ms
return -1; return -1;
} }
const struct RTPHeader *const header = &msg->header; if (rtp_message_pt(msg) == (RTP_TYPE_VIDEO + 2) % 128) {
if (msg->header.pt == (RTP_TYPE_VIDEO + 2) % 128) {
LOGGER_WARNING(vc->log, "Got dummy!"); LOGGER_WARNING(vc->log, "Got dummy!");
free(msg); free(msg);
return 0; return 0;
} }
if (msg->header.pt != RTP_TYPE_VIDEO % 128) { if (rtp_message_pt(msg) != RTP_TYPE_VIDEO % 128) {
LOGGER_WARNING(vc->log, "Invalid payload type! pt=%d", (int)msg->header.pt); LOGGER_WARNING(vc->log, "Invalid payload type! pt=%d", (int)rtp_message_pt(msg));
free(msg);
return -1;
}
/* Security check: Sanitize message size to prevent memory exhaustion */
if (rtp_message_data_length_full(msg) > VIDEO_MAX_FRAME_SIZE) {
LOGGER_ERROR(vc->log, "Message too large! size=%u", (uint32_t)rtp_message_data_length_full(msg));
free(msg); free(msg);
return -1; return -1;
} }
pthread_mutex_lock(vc->queue_mutex); pthread_mutex_lock(vc->queue_mutex);
if ((header->flags & RTP_LARGE_FRAME) != 0 && header->pt == RTP_TYPE_VIDEO % 128) { if ((rtp_message_flags(msg) & RTP_LARGE_FRAME) != 0 && rtp_message_pt(msg) == RTP_TYPE_VIDEO % 128) {
LOGGER_DEBUG(vc->log, "rb_write msg->len=%d b0=%d b1=%d", (int)msg->len, (int)msg->data[0], (int)msg->data[1]); LOGGER_DEBUG(vc->log, "rb_write msg->len=%d b0=%d b1=%d", (int)rtp_message_len(msg), (int)rtp_message_data(msg)[0], (int)rtp_message_data(msg)[1]);
} }
free(rb_write(vc->vbuf_raw, msg)); free(rb_write(vc->vbuf_raw, msg));
@@ -373,6 +426,12 @@ int vc_reconfigure_encoder(VCSession *vc, uint32_t bit_rate, uint16_t width, uin
return -1; return -1;
} }
/* Security check: Sanitize resolution to prevent resource exhaustion */
if (width == 0 || height == 0 || width > VIDEO_MAX_RESOLUTION_LIMIT || height > VIDEO_MAX_RESOLUTION_LIMIT) {
LOGGER_ERROR(vc->log, "Invalid resolution requested: %ux%u", (uint32_t)width, (uint32_t)height);
return -1;
}
vpx_codec_enc_cfg_t cfg2 = *vc->encoder->config.enc; vpx_codec_enc_cfg_t cfg2 = *vc->encoder->config.enc;
if (cfg2.rc_target_bitrate == bit_rate && cfg2.g_w == width && cfg2.g_h == height && kf_max_dist == -1) { if (cfg2.rc_target_bitrate == bit_rate && cfg2.g_w == width && cfg2.g_h == height && kf_max_dist == -1) {
@@ -394,15 +453,16 @@ int vc_reconfigure_encoder(VCSession *vc, uint32_t bit_rate, uint16_t width, uin
* reconfiguring encoder to use resolutions greater than initially set. * reconfiguring encoder to use resolutions greater than initially set.
*/ */
LOGGER_DEBUG(vc->log, "Have to reinitialize vpx encoder on session %p", (void *)vc); LOGGER_DEBUG(vc->log, "Have to reinitialize vpx encoder on session %p", (void *)vc);
vpx_codec_ctx_t new_c;
vpx_codec_enc_cfg_t cfg; vpx_codec_enc_cfg_t cfg;
vc_init_encoder_cfg(vc->log, &cfg, kf_max_dist); vc_init_encoder_cfg(vc->log, &cfg, kf_max_dist);
cfg.rc_target_bitrate = bit_rate; cfg.rc_target_bitrate = bit_rate;
cfg.g_w = width; cfg.g_w = width;
cfg.g_h = height; cfg.g_h = height;
/* Atomic reconfiguration: Initialize new encoder first */
vpx_codec_ctx_t new_encoder;
LOGGER_DEBUG(vc->log, "Using VP8 codec for encoder"); LOGGER_DEBUG(vc->log, "Using VP8 codec for encoder");
vpx_codec_err_t rc = vpx_codec_enc_init(&new_c, video_codec_encoder_interface(), &cfg, VPX_CODEC_USE_FRAME_THREADING); vpx_codec_err_t rc = vpx_codec_enc_init(&new_encoder, video_codec_encoder_interface(), &cfg, VPX_CODEC_USE_FRAME_THREADING);
if (rc != VPX_CODEC_OK) { if (rc != VPX_CODEC_OK) {
LOGGER_ERROR(vc->log, "Failed to initialize encoder: %s", vpx_codec_err_to_string(rc)); LOGGER_ERROR(vc->log, "Failed to initialize encoder: %s", vpx_codec_err_to_string(rc));
@@ -411,17 +471,99 @@ int vc_reconfigure_encoder(VCSession *vc, uint32_t bit_rate, uint16_t width, uin
const int cpu_used_value = VP8E_SET_CPUUSED_VALUE; const int cpu_used_value = VP8E_SET_CPUUSED_VALUE;
rc = vpx_codec_control(&new_c, VP8E_SET_CPUUSED, cpu_used_value); rc = vpx_codec_control(&new_encoder, VP8E_SET_CPUUSED, cpu_used_value);
if (rc != VPX_CODEC_OK) { if (rc != VPX_CODEC_OK) {
LOGGER_ERROR(vc->log, "Failed to set encoder control setting: %s", vpx_codec_err_to_string(rc)); LOGGER_ERROR(vc->log, "Failed to set encoder control setting: %s", vpx_codec_err_to_string(rc));
vpx_codec_destroy(&new_c); vpx_codec_destroy(&new_encoder);
return -1; return -1;
} }
/* Swap only on success */
vpx_codec_destroy(vc->encoder); vpx_codec_destroy(vc->encoder);
memcpy(vc->encoder, &new_c, sizeof(new_c)); *vc->encoder = new_encoder;
return 0;
} }
return 0; return 0;
} }
int vc_encode(VCSession *vc, uint16_t width, uint16_t height, const uint8_t *y,
const uint8_t *u, const uint8_t *v, int encode_flags)
{
vpx_image_t img;
// TODO(Green-Sky): figure out stride_align
// TODO(Green-Sky): check memory alignment?
if (vpx_img_wrap(&img, VPX_IMG_FMT_I420, width, height, 0, (uint8_t *)y) != nullptr) {
// vpx_img_wrap assumes contigues memory, so we fix that
img.planes[VPX_PLANE_U] = (uint8_t *)u;
img.planes[VPX_PLANE_V] = (uint8_t *)v;
} else {
// call to wrap failed, falling back to copy
if (vpx_img_alloc(&img, VPX_IMG_FMT_I420, width, height, 0) == nullptr) {
LOGGER_ERROR(vc->log, "Could not allocate image for frame");
return -1;
}
/* I420 "It comprises an NxM Y plane followed by (N/2)x(M/2) V and U planes."
* http://fourcc.org/yuv.php#IYUV
*/
memcpy(img.planes[VPX_PLANE_Y], y, width * height);
memcpy(img.planes[VPX_PLANE_U], u, (width / 2) * (height / 2));
memcpy(img.planes[VPX_PLANE_V], v, (width / 2) * (height / 2));
}
int vpx_flags = 0;
if ((encode_flags & VC_EFLAG_FORCE_KF) != 0) {
vpx_flags |= VPX_EFLAG_FORCE_KF;
}
const vpx_codec_err_t vrc = vpx_codec_encode(vc->encoder, &img,
vc->frame_counter, 1, vpx_flags, VPX_DL_REALTIME);
vpx_img_free(&img);
if (vrc != VPX_CODEC_OK) {
LOGGER_ERROR(vc->log, "Could not encode video frame: %s", vpx_codec_err_to_string(vrc));
return -1;
}
vc->iter = nullptr;
return 0;
}
int vc_get_cx_data(VCSession *vc, uint8_t **data, uint32_t *size, bool *is_keyframe)
{
const vpx_codec_cx_pkt_t *pkt = vpx_codec_get_cx_data(vc->encoder, &vc->iter);
while (pkt != nullptr && pkt->kind != VPX_CODEC_CX_FRAME_PKT) {
pkt = vpx_codec_get_cx_data(vc->encoder, &vc->iter);
}
if (pkt == nullptr) {
return 0;
}
*data = (uint8_t *)pkt->data.frame.buf;
*size = (uint32_t)pkt->data.frame.sz;
*is_keyframe = (pkt->data.frame.flags & VPX_FRAME_IS_KEY) != 0;
return 1;
}
uint32_t vc_get_lcfd(const VCSession *vc)
{
return vc->lcfd;
}
pthread_mutex_t *vc_get_queue_mutex(VCSession *vc)
{
return &vc->queue_mutex[0];
}
void vc_increment_frame_counter(VCSession *vc)
{
++vc->frame_counter;
}

View File

@@ -5,50 +5,46 @@
#ifndef C_TOXCORE_TOXAV_VIDEO_H #ifndef C_TOXCORE_TOXAV_VIDEO_H
#define C_TOXCORE_TOXAV_VIDEO_H #define C_TOXCORE_TOXAV_VIDEO_H
#include <vpx/vpx_decoder.h>
#include <vpx/vpx_encoder.h>
#include <vpx/vpx_image.h>
#include <vpx/vp8cx.h>
#include <vpx/vp8dx.h>
#include <pthread.h> #include <pthread.h>
#include <stdint.h>
#include "toxav.h"
#include "../toxcore/logger.h" #include "../toxcore/logger.h"
#include "../toxcore/util.h" #include "../toxcore/mono_time.h"
#include "ring_buffer.h"
#include "rtp.h"
typedef struct VCSession { #ifdef __cplusplus
/* encoding */ extern "C" {
vpx_codec_ctx_t encoder[1]; #endif
uint32_t frame_counter;
/* decoding */ typedef void vc_video_receive_frame_cb(uint32_t friend_number, uint16_t width, uint16_t height,
vpx_codec_ctx_t decoder[1]; const uint8_t *y, const uint8_t *u, const uint8_t *v,
struct RingBuffer *vbuf_raw; /* Un-decoded data */ int32_t ystride, int32_t ustride, int32_t vstride,
void *user_data);
uint64_t linfts; /* Last received frame time stamp */ typedef struct VCSession VCSession;
uint32_t lcfd; /* Last calculated frame duration for incoming video payload */
ToxAV *av; #define VC_EFLAG_NONE 0
uint32_t friend_number; #define VC_EFLAG_FORCE_KF (1 << 0)
/* Video frame receive callback */ struct RTPMessage;
toxav_video_receive_frame_cb *vcb;
void *vcb_user_data;
pthread_mutex_t queue_mutex[1]; VCSession *vc_new(const Logger *log, const Mono_Time *mono_time, uint32_t friend_number,
const Logger *log; vc_video_receive_frame_cb *cb, void *user_data);
} VCSession;
VCSession *vc_new(const Logger *log, const Mono_Time *mono_time, ToxAV *av, uint32_t friend_number,
toxav_video_receive_frame_cb *cb, void *cb_data);
void vc_kill(VCSession *vc); void vc_kill(VCSession *vc);
void vc_iterate(VCSession *vc); void vc_iterate(VCSession *vc);
int vc_queue_message(const Mono_Time *mono_time, void *cs, struct RTPMessage *msg); int vc_queue_message(const Mono_Time *mono_time, void *cs, struct RTPMessage *msg);
int vc_reconfigure_encoder(VCSession *vc, uint32_t bit_rate, uint16_t width, uint16_t height, int16_t kf_max_dist); int vc_reconfigure_encoder(VCSession *vc, uint32_t bit_rate, uint16_t width, uint16_t height, int16_t kf_max_dist);
int vc_encode(VCSession *vc, uint16_t width, uint16_t height, const uint8_t *y,
const uint8_t *u, const uint8_t *v, int encode_flags);
int vc_get_cx_data(VCSession *vc, uint8_t **data, uint32_t *size, bool *is_keyframe);
uint32_t vc_get_lcfd(const VCSession *vc);
pthread_mutex_t *vc_get_queue_mutex(VCSession *vc);
void vc_increment_frame_counter(VCSession *vc);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* C_TOXCORE_TOXAV_VIDEO_H */ #endif /* C_TOXCORE_TOXAV_VIDEO_H */

View File

@@ -0,0 +1,412 @@
#include "video.h"
#include <gtest/gtest.h>
#include <algorithm>
#include <vector>
#include "../toxcore/logger.h"
#include "../toxcore/mono_time.h"
#include "../toxcore/network.h"
#include "../toxcore/os_memory.h"
#include "rtp.h"
namespace {
struct VideoTimeMock {
uint64_t t;
};
uint64_t video_mock_time_cb(void *ud) { return static_cast<VideoTimeMock *>(ud)->t; }
void test_logger_cb(void *context, Logger_Level level, const char *file, uint32_t line,
const char *func, const char *message, void *userdata)
{
(void)context;
(void)userdata;
const char *level_str = "UNKNOWN";
switch (level) {
case LOGGER_LEVEL_TRACE:
level_str = "TRACE";
break;
case LOGGER_LEVEL_DEBUG:
level_str = "DEBUG";
break;
case LOGGER_LEVEL_INFO:
level_str = "INFO";
break;
case LOGGER_LEVEL_WARNING:
level_str = "WARN";
break;
case LOGGER_LEVEL_ERROR:
level_str = "ERROR";
break;
}
printf("[%s] %s:%u %s: %s\n", level_str, file, line, func, message);
}
struct VideoTestData {
uint32_t friend_number = 0;
uint16_t width = 0;
uint16_t height = 0;
std::vector<uint8_t> y, u, v;
int32_t ystride = 0, ustride = 0, vstride = 0;
VideoTestData();
~VideoTestData();
static void receive_frame(uint32_t friend_number, uint16_t width, uint16_t height,
const uint8_t *y, const uint8_t *u, const uint8_t *v, int32_t ystride, int32_t ustride,
int32_t vstride, void *user_data)
{
auto *self = static_cast<VideoTestData *>(user_data);
self->friend_number = friend_number;
self->width = width;
self->height = height;
self->ystride = ystride;
self->ustride = ustride;
self->vstride = vstride;
self->y.assign(y, y + static_cast<size_t>(std::abs(ystride)) * height);
self->u.assign(u, u + static_cast<size_t>(std::abs(ustride)) * (height / 2));
self->v.assign(v, v + static_cast<size_t>(std::abs(vstride)) * (height / 2));
}
};
VideoTestData::VideoTestData() = default;
VideoTestData::~VideoTestData() = default;
struct VideoRtpMock {
RTPSession *recv_session = nullptr;
std::vector<std::vector<uint8_t>> captured_packets;
bool auto_forward = true;
static int send_packet(void *user_data, const uint8_t *data, uint16_t length)
{
auto *self = static_cast<VideoRtpMock *>(user_data);
self->captured_packets.push_back(std::vector<uint8_t>(data, data + length));
if (self->auto_forward && self->recv_session) {
rtp_receive_packet(self->recv_session, data, length);
}
return 0;
}
static int video_cb(const Mono_Time *mono_time, void *cs, RTPMessage *msg)
{
return vc_queue_message(mono_time, cs, msg);
}
};
class VideoTest : public ::testing::Test {
protected:
void SetUp() override
{
const Memory *mem = os_memory();
log = logger_new(mem);
logger_callback_log(log, test_logger_cb, nullptr, nullptr);
tm.t = 1000;
mono_time = mono_time_new(mem, video_mock_time_cb, &tm);
mono_time_update(mono_time);
}
void TearDown() override
{
const Memory *mem = os_memory();
mono_time_free(mem, mono_time);
logger_kill(log);
}
Logger *log;
Mono_Time *mono_time;
VideoTimeMock tm;
};
TEST_F(VideoTest, BasicNewKill)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
vc_kill(vc);
}
TEST_F(VideoTest, EncodeDecodeLoop)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
VideoRtpMock rtp_mock;
RTPSession *send_rtp = rtp_new(log, RTP_TYPE_VIDEO, mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_VIDEO, mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
rtp_mock.recv_session = recv_rtp;
uint16_t width = 320;
uint16_t height = 240;
uint32_t bitrate = 500;
ASSERT_EQ(vc_reconfigure_encoder(vc, bitrate, width, height, -1), 0);
std::vector<uint8_t> y(width * height, 128);
std::vector<uint8_t> u((width / 2) * (height / 2), 64);
std::vector<uint8_t> v((width / 2) * (height / 2), 192);
ASSERT_EQ(vc_encode(vc, width, height, y.data(), u.data(), v.data(), VC_EFLAG_FORCE_KF), 0);
vc_increment_frame_counter(vc);
uint8_t *pkt_data;
uint32_t pkt_size;
bool is_keyframe;
while (vc_get_cx_data(vc, &pkt_data, &pkt_size, &is_keyframe)) {
int rc = rtp_send_data(log, send_rtp, pkt_data, pkt_size, is_keyframe);
ASSERT_EQ(rc, 0);
}
vc_iterate(vc);
ASSERT_EQ(data.friend_number, 123u);
ASSERT_EQ(data.width, width);
ASSERT_EQ(data.height, height);
ASSERT_FALSE(data.y.empty());
rtp_kill(log, send_rtp);
rtp_kill(log, recv_rtp);
vc_kill(vc);
}
TEST_F(VideoTest, ReconfigureEncoder)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
// Initial reconfigure
ASSERT_EQ(vc_reconfigure_encoder(vc, 500, 320, 240, -1), 0);
// Change bitrate and resolution
ASSERT_EQ(vc_reconfigure_encoder(vc, 1000, 640, 480, -1), 0);
std::vector<uint8_t> y(640 * 480, 128);
std::vector<uint8_t> u(320 * 240, 64);
std::vector<uint8_t> v(320 * 240, 192);
ASSERT_EQ(vc_encode(vc, 640, 480, y.data(), u.data(), v.data(), VC_EFLAG_NONE), 0);
vc_kill(vc);
}
TEST_F(VideoTest, GetLcfd)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
// Default lcfd is 60 in video.c
EXPECT_EQ(vc_get_lcfd(vc), 60u);
vc_kill(vc);
}
TEST_F(VideoTest, QueueInvalidMessage)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
VideoRtpMock rtp_mock;
// Create an audio RTP session but try to queue to video session
RTPSession *audio_rtp = rtp_new(log, RTP_TYPE_AUDIO, mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
RTPSession *video_recv_rtp = rtp_new(log, RTP_TYPE_VIDEO, mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
rtp_mock.recv_session = video_recv_rtp;
std::vector<uint8_t> dummy_audio(100, 0);
int rc = rtp_send_data(
log, audio_rtp, dummy_audio.data(), static_cast<uint32_t>(dummy_audio.size()), false);
ASSERT_EQ(rc, 0);
// Iterate should NOT trigger callback because payload type was wrong
vc_iterate(vc);
EXPECT_EQ(data.width, 0u);
rtp_kill(log, audio_rtp);
rtp_kill(log, video_recv_rtp);
vc_kill(vc);
}
TEST_F(VideoTest, ReconfigureOptimizations)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
// 1. Reconfigure with same values (should do nothing)
// vc_new initializes encoder with 800x600 and 5000 bitrate.
EXPECT_EQ(vc_reconfigure_encoder(vc, 5000, 800, 600, -1), 0);
// 2. Reconfigure with only bitrate change
EXPECT_EQ(vc_reconfigure_encoder(vc, 2000, 800, 600, -1), 0);
// 3. Reconfigure with kf_max_dist > 1 (triggers re-init and kf_max_dist branch)
EXPECT_EQ(vc_reconfigure_encoder(vc, 2000, 800, 600, 60), 0);
vc_kill(vc);
}
TEST_F(VideoTest, LcfdAndSpecialPackets)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
VideoRtpMock rtp_mock;
RTPSession *video_recv_rtp = rtp_new(log, RTP_TYPE_VIDEO, mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
rtp_mock.recv_session = video_recv_rtp;
// 1. Test lcfd update
tm.t += 50; // Advance time by 50ms
mono_time_update(mono_time);
std::vector<uint8_t> dummy_frame(10, 0);
rtp_send_data(
log, video_recv_rtp, dummy_frame.data(), static_cast<uint32_t>(dummy_frame.size()), true);
// lcfd should be updated. Initial linfts was set at vc_new (tm.t=1000).
// Now tm.t is 1050. t_lcfd = 1050 - 1000 = 50.
EXPECT_EQ(vc_get_lcfd(vc), 50u);
// 2. Test lcfd threshold (t_lcfd > 100 should be ignored)
tm.t += 200;
mono_time_update(mono_time);
rtp_send_data(
log, video_recv_rtp, dummy_frame.data(), static_cast<uint32_t>(dummy_frame.size()), true);
EXPECT_EQ(vc_get_lcfd(vc), 50u); // Should still be 50
// 3. Test dummy packet PT = (RTP_TYPE_VIDEO + 2) % 128
RTPSession *dummy_rtp = rtp_new(log, (RTP_TYPE_VIDEO + 2), mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
rtp_mock.recv_session = dummy_rtp;
rtp_send_data(
log, dummy_rtp, dummy_frame.data(), static_cast<uint32_t>(dummy_frame.size()), false);
// Should return 0 but do nothing (logged as "Got dummy!")
// 4. Test GetQueueMutex
EXPECT_NE(vc_get_queue_mutex(vc), nullptr);
rtp_kill(log, video_recv_rtp);
rtp_kill(log, dummy_rtp);
vc_kill(vc);
}
TEST_F(VideoTest, MultiReconfigureEncode)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
for (int i = 0; i < 5; ++i) {
uint16_t w = static_cast<uint16_t>(160 + (i * 16));
uint16_t h = static_cast<uint16_t>(120 + (i * 16));
std::vector<uint8_t> y(static_cast<size_t>(w) * h, 128);
std::vector<uint8_t> u((static_cast<size_t>(w) / 2) * (h / 2), 64);
std::vector<uint8_t> v((static_cast<size_t>(w) / 2) * (h / 2), 192);
ASSERT_EQ(vc_reconfigure_encoder(vc, 1000, w, h, -1), 0);
ASSERT_EQ(vc_encode(vc, w, h, y.data(), u.data(), v.data(), VC_EFLAG_NONE), 0);
}
vc_kill(vc);
}
TEST_F(VideoTest, NewWithNullMonoTime)
{
VideoTestData data;
VCSession *vc = vc_new(log, nullptr, 123, VideoTestData::receive_frame, &data);
EXPECT_EQ(vc, nullptr);
}
TEST_F(VideoTest, ReconfigureFailDoS)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
// Trigger failure by passing invalid resolution (0)
// This currently destroys the encoder.
ASSERT_EQ(vc_reconfigure_encoder(vc, 1000, 0, 0, -1), -1);
// Attempt to encode. This is expected to crash because vc->encoder is destroyed.
std::vector<uint8_t> y(320 * 240, 128);
std::vector<uint8_t> u(160 * 120, 64);
std::vector<uint8_t> v(160 * 120, 192);
// This call will crash in the current unfixed code.
vc_encode(vc, 320, 240, y.data(), u.data(), v.data(), VC_EFLAG_NONE);
vc_kill(vc);
}
TEST_F(VideoTest, LyingLengthOOB)
{
VideoTestData data;
VCSession *vc = vc_new(log, mono_time, 123, VideoTestData::receive_frame, &data);
ASSERT_NE(vc, nullptr);
VideoRtpMock rtp_mock;
RTPSession *recv_rtp = rtp_new(log, RTP_TYPE_VIDEO, mono_time, VideoRtpMock::send_packet,
&rtp_mock, nullptr, nullptr, nullptr, vc, VideoRtpMock::video_cb);
rtp_mock.recv_session = recv_rtp;
// Craft a malicious RTP packet
uint16_t payload_len = 10;
uint8_t packet[RTP_HEADER_SIZE + 11]; // +1 for Tox ID
memset(packet, 0, sizeof(packet));
// Tox ID
packet[0] = static_cast<uint8_t>(RTP_TYPE_VIDEO);
auto pack_u16 = [](uint8_t *p, uint16_t v) {
p[0] = static_cast<uint8_t>(v >> 8);
p[1] = static_cast<uint8_t>(v & 0xff);
};
auto pack_u32 = [](uint8_t *p, uint32_t v) {
p[0] = static_cast<uint8_t>(v >> 24);
p[1] = static_cast<uint8_t>((v >> 16) & 0xff);
p[2] = static_cast<uint8_t>((v >> 8) & 0xff);
p[3] = static_cast<uint8_t>(v & 0xff);
};
auto pack_u64 = [&](uint8_t *p, uint64_t v) {
pack_u32(p, static_cast<uint32_t>(v >> 32));
pack_u32(p + 4, static_cast<uint32_t>(v & 0xffffffff));
};
// RTP Header starts at packet[1]
packet[1] = 2 << 6; // ve = 2
packet[2] = static_cast<uint8_t>(RTP_TYPE_VIDEO % 128);
pack_u16(packet + 3, 1); // sequnum
pack_u32(packet + 5, 1000); // timestamp
pack_u32(packet + 9, 0x12345678); // ssrc
pack_u64(packet + 13, RTP_LARGE_FRAME); // flags
pack_u32(packet + 21, 0); // offset_full
pack_u32(packet + 25, 1000); // data_length_full (LYING! Actual is 10)
pack_u32(packet + 29, 0); // received_length_full
// Skip padding fields (11 * 4 = 44 bytes)
pack_u16(packet + 77, 0); // offset_lower
pack_u16(packet + 79, payload_len); // data_length_lower
// Send the malicious packet
rtp_receive_packet(recv_rtp, packet, sizeof(packet));
// Trigger vc_iterate. This will call vpx_codec_decode with length 1000.
// This is expected to cause OOB read.
vc_iterate(vc);
rtp_kill(log, recv_rtp);
vc_kill(vc);
}
} // namespace

View File

@@ -591,9 +591,18 @@ static bool client_or_ip_port_in_list(const Logger *_Nonnull log, const Mono_Tim
LOGGER_DEBUG(log, "coipil[%u]: switching public_key (ipv%d)", index, ip_version); LOGGER_DEBUG(log, "coipil[%u]: switching public_key (ipv%d)", index, ip_version);
/* kill the other address, if it was set */ /* kill the other address, if it was set.
* We just updated `assoc` (which is either assoc4 or assoc6) with the new public_key.
* If there was an association for the other IP version, it's now invalid for this new identity.
*/
if (ip_version == 4) {
const IPPTsPng empty_ipptspng = {{{{0}}}}; const IPPTsPng empty_ipptspng = {{{{0}}}};
*assoc = empty_ipptspng; list[index].assoc6 = empty_ipptspng;
} else {
const IPPTsPng empty_ipptspng = {{{{0}}}};
list[index].assoc4 = empty_ipptspng;
}
return true; return true;
} }
@@ -601,27 +610,30 @@ bool add_to_list(
Node_format *nodes_list, uint32_t length, const uint8_t pk[CRYPTO_PUBLIC_KEY_SIZE], Node_format *nodes_list, uint32_t length, const uint8_t pk[CRYPTO_PUBLIC_KEY_SIZE],
const IP_Port *ip_port, const uint8_t cmp_pk[CRYPTO_PUBLIC_KEY_SIZE]) const IP_Port *ip_port, const uint8_t cmp_pk[CRYPTO_PUBLIC_KEY_SIZE])
{ {
uint8_t pk_cur[CRYPTO_PUBLIC_KEY_SIZE];
memcpy(pk_cur, pk, CRYPTO_PUBLIC_KEY_SIZE);
IP_Port ip_port_cur = *ip_port;
bool inserted = false;
for (uint32_t i = 0; i < length; ++i) { for (uint32_t i = 0; i < length; ++i) {
Node_format *node = &nodes_list[i]; Node_format *node = &nodes_list[i];
if (id_closest(cmp_pk, node->public_key, pk) == 2) { if (id_closest(cmp_pk, node->public_key, pk_cur) == 2) {
uint8_t pk_bak[CRYPTO_PUBLIC_KEY_SIZE]; uint8_t pk_bak[CRYPTO_PUBLIC_KEY_SIZE];
memcpy(pk_bak, node->public_key, CRYPTO_PUBLIC_KEY_SIZE); memcpy(pk_bak, node->public_key, CRYPTO_PUBLIC_KEY_SIZE);
const IP_Port ip_port_bak = node->ip_port; const IP_Port ip_port_bak = node->ip_port;
memcpy(node->public_key, pk, CRYPTO_PUBLIC_KEY_SIZE);
node->ip_port = *ip_port; memcpy(node->public_key, pk_cur, CRYPTO_PUBLIC_KEY_SIZE);
node->ip_port = ip_port_cur;
if (i != length - 1) { memcpy(pk_cur, pk_bak, CRYPTO_PUBLIC_KEY_SIZE);
add_to_list(nodes_list, length, pk_bak, &ip_port_bak, cmp_pk); ip_port_cur = ip_port_bak;
} inserted = true;
return true;
} }
} }
return false; return inserted;
} }
/** /**
@@ -638,12 +650,6 @@ static void get_close_nodes_inner(uint64_t cur_time, const uint8_t *_Nonnull pub
for (uint32_t i = 0; i < client_list_length; ++i) { for (uint32_t i = 0; i < client_list_length; ++i) {
const Client_data *const client = &client_list[i]; const Client_data *const client = &client_list[i];
/* node already in list? */
if (index_of_node_pk(nodes_list, MAX_SENT_NODES, client->public_key) != UINT32_MAX) {
continue;
}
const IPPTsPng *ipptp; const IPPTsPng *ipptp;
if (net_family_is_ipv4(sa_family)) { if (net_family_is_ipv4(sa_family)) {
@@ -674,6 +680,11 @@ static void get_close_nodes_inner(uint64_t cur_time, const uint8_t *_Nonnull pub
#endif /* CHECK_ANNOUNCE_NODE */ #endif /* CHECK_ANNOUNCE_NODE */
/* node already in list? */
if (index_of_node_pk(nodes_list, num_nodes, client->public_key) != UINT32_MAX) {
continue;
}
if (num_nodes < MAX_SENT_NODES) { if (num_nodes < MAX_SENT_NODES) {
memcpy(nodes_list[num_nodes].public_key, client->public_key, CRYPTO_PUBLIC_KEY_SIZE); memcpy(nodes_list[num_nodes].public_key, client->public_key, CRYPTO_PUBLIC_KEY_SIZE);
nodes_list[num_nodes].ip_port = ipptp->ip_port; nodes_list[num_nodes].ip_port = ipptp->ip_port;
@@ -2009,7 +2020,7 @@ static uint32_t foreach_ip_port(const DHT *_Nonnull dht, const DHT_Friend *_Nonn
static bool send_packet_to_friend(const DHT *_Nonnull dht, const IP_Port *_Nonnull ip_port, uint32_t *_Nonnull n, void *_Nonnull userdata) static bool send_packet_to_friend(const DHT *_Nonnull dht, const IP_Port *_Nonnull ip_port, uint32_t *_Nonnull n, void *_Nonnull userdata)
{ {
const Packet *packet = (const Packet *)userdata; const Packet *packet = (const Packet *)userdata;
const int retval = send_packet(dht->net, ip_port, *packet); const int retval = net_send_packet(dht->net, ip_port, *packet);
if ((uint32_t)retval == packet->length) { if ((uint32_t)retval == packet->length) {
++*n; ++*n;
@@ -2078,7 +2089,7 @@ static uint32_t routeone_to_friend(const DHT *_Nonnull dht, const uint8_t *_Nonn
} }
const uint32_t rand_idx = random_range_u32(dht->rng, n); const uint32_t rand_idx = random_range_u32(dht->rng, n);
const int retval = send_packet(dht->net, &ip_list[rand_idx], *packet); const int retval = net_send_packet(dht->net, &ip_list[rand_idx], *packet);
if ((unsigned int)retval == packet->length) { if ((unsigned int)retval == packet->length) {
return 1; return 1;

View File

@@ -2,6 +2,7 @@ lib_LTLIBRARIES += libtoxcore.la
libtoxcore_la_include_HEADERS = \ libtoxcore_la_include_HEADERS = \
../toxcore/tox.h \ ../toxcore/tox.h \
../toxcore/tox_log_level.h \
../toxcore/tox_options.h ../toxcore/tox_options.h
libtoxcore_la_includedir = $(includedir)/tox libtoxcore_la_includedir = $(includedir)/tox

View File

@@ -142,12 +142,7 @@ bool crypto_memunlock(void *data, size_t length)
bool pk_equal(const uint8_t pk1[CRYPTO_PUBLIC_KEY_SIZE], const uint8_t pk2[CRYPTO_PUBLIC_KEY_SIZE]) bool pk_equal(const uint8_t pk1[CRYPTO_PUBLIC_KEY_SIZE], const uint8_t pk2[CRYPTO_PUBLIC_KEY_SIZE])
{ {
#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
// Hope that this is better for the fuzzer
return memcmp(pk1, pk2, CRYPTO_PUBLIC_KEY_SIZE) == 0; return memcmp(pk1, pk2, CRYPTO_PUBLIC_KEY_SIZE) == 0;
#else
return crypto_verify_32(pk1, pk2) == 0;
#endif /* FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION */
} }
void pk_copy(uint8_t dest[CRYPTO_PUBLIC_KEY_SIZE], const uint8_t src[CRYPTO_PUBLIC_KEY_SIZE]) void pk_copy(uint8_t dest[CRYPTO_PUBLIC_KEY_SIZE], const uint8_t src[CRYPTO_PUBLIC_KEY_SIZE])
@@ -167,12 +162,7 @@ bool crypto_sha512_eq(const uint8_t cksum1[CRYPTO_SHA512_SIZE], const uint8_t ck
bool crypto_sha256_eq(const uint8_t cksum1[CRYPTO_SHA256_SIZE], const uint8_t cksum2[CRYPTO_SHA256_SIZE]) bool crypto_sha256_eq(const uint8_t cksum1[CRYPTO_SHA256_SIZE], const uint8_t cksum2[CRYPTO_SHA256_SIZE])
{ {
#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
// Hope that this is better for the fuzzer
return memcmp(cksum1, cksum2, CRYPTO_SHA256_SIZE) == 0; return memcmp(cksum1, cksum2, CRYPTO_SHA256_SIZE) == 0;
#else
return crypto_verify_32(cksum1, cksum2) == 0;
#endif /* FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION */
} }
uint8_t random_u08(const Random *rng) uint8_t random_u08(const Random *rng)

View File

@@ -3717,6 +3717,10 @@ Group_Chats *new_groupchats(const Mono_Time *mono_time, const Memory *mem, Messe
/** main groupchats loop. */ /** main groupchats loop. */
void do_groupchats(Group_Chats *g_c, void *userdata) void do_groupchats(Group_Chats *g_c, void *userdata)
{ {
if (g_c == nullptr) {
return;
}
for (uint16_t i = 0; i < g_c->num_chats; ++i) { for (uint16_t i = 0; i < g_c->num_chats; ++i) {
Group_c *g = get_group_c(g_c, i); Group_c *g = get_group_c(g_c, i);

View File

@@ -1759,11 +1759,6 @@ static bool unpack_gc_sync_announce(GC_Chat *_Nonnull chat, const uint8_t *_Nonn
static int handle_gc_sync_response(const GC_Session *_Nonnull c, GC_Chat *_Nonnull chat, uint32_t peer_number, const uint8_t *_Nullable data, static int handle_gc_sync_response(const GC_Session *_Nonnull c, GC_Chat *_Nonnull chat, uint32_t peer_number, const uint8_t *_Nullable data,
uint16_t length, void *_Nullable userdata) uint16_t length, void *_Nullable userdata)
{ {
if (chat->connection_state == CS_CONNECTED && get_gc_confirmed_numpeers(chat) >= chat->shared_state.maxpeers
&& !peer_is_founder(chat, peer_number)) {
return -1;
}
if (length > 0) { if (length > 0) {
if (!unpack_gc_sync_announce(chat, data, length)) { if (!unpack_gc_sync_announce(chat, data, length)) {
return -1; return -1;
@@ -1778,6 +1773,14 @@ static int handle_gc_sync_response(const GC_Session *_Nonnull c, GC_Chat *_Nonnu
return -2; return -2;
} }
/* If the group is full, we only allow already confirmed peers to sync.
* This prevents disconnecting existing peers during re-syncing.
*/
if (!gconn->confirmed && get_gc_confirmed_numpeers(chat) >= chat->shared_state.maxpeers
&& !peer_is_founder(chat, peer_number)) {
return -1;
}
if (!send_gc_peer_exchange(chat, gconn)) { if (!send_gc_peer_exchange(chat, gconn)) {
LOGGER_WARNING(chat->log, "Failed to send peer exchange on sync response"); LOGGER_WARNING(chat->log, "Failed to send peer exchange on sync response");
} }
@@ -2200,7 +2203,7 @@ static int handle_gc_invite_request(GC_Chat *_Nonnull chat, uint32_t peer_number
uint8_t invite_error; uint8_t invite_error;
if (get_gc_confirmed_numpeers(chat) >= chat->shared_state.maxpeers && !peer_is_founder(chat, peer_number)) { if (!gconn->confirmed && get_gc_confirmed_numpeers(chat) >= chat->shared_state.maxpeers && !peer_is_founder(chat, peer_number)) {
invite_error = GJ_GROUP_FULL; invite_error = GJ_GROUP_FULL;
goto FAILED_INVITE; goto FAILED_INVITE;
} }
@@ -3828,12 +3831,6 @@ static bool handle_gc_topic_validate(const GC_Chat *_Nonnull chat, const GC_Peer
return true; return true;
} }
if (chat->topic_prev_checksum == topic_info->checksum &&
!mono_time_is_timeout(chat->mono_time, chat->topic_time_set, GC_CONFIRMED_PEER_TIMEOUT)) {
LOGGER_DEBUG(chat->log, "Topic reversion (probable sync error)");
return false;
}
return true; return true;
} }
@@ -3893,12 +3890,18 @@ static int handle_gc_topic(const GC_Session *_Nonnull c, GC_Chat *_Nonnull chat,
chat->topic_info = topic_info; chat->topic_info = topic_info;
memcpy(chat->topic_sig, signature, SIGNATURE_SIZE); memcpy(chat->topic_sig, signature, SIGNATURE_SIZE);
if (!skip_callback && chat->connection_state == CS_CONNECTED && c->topic_change != nullptr) { if (!skip_callback && chat->connection_state == CS_CONNECTED) {
if (gc_get_self_role(chat) == GR_FOUNDER) {
broadcast_gc_topic(chat);
}
if (c->topic_change != nullptr) {
const int setter_peer_number = get_peer_number_of_sig_pk(chat, topic_info.public_sig_key); const int setter_peer_number = get_peer_number_of_sig_pk(chat, topic_info.public_sig_key);
const GC_Peer_Id peer_id = setter_peer_number >= 0 ? chat->group[setter_peer_number].peer_id : gc_unknown_peer_id(); const GC_Peer_Id peer_id = setter_peer_number >= 0 ? chat->group[setter_peer_number].peer_id : gc_unknown_peer_id();
c->topic_change(c->messenger, chat->group_number, peer_id, topic_info.topic, topic_info.length, userdata); c->topic_change(c->messenger, chat->group_number, peer_id, topic_info.topic, topic_info.length, userdata);
} }
}
return 0; return 0;
} }
@@ -5530,7 +5533,9 @@ static bool send_gc_handshake_packet(const GC_Chat *_Nonnull chat, GC_Connection
} }
if (ret != length && gconn->tcp_relays_count == 0) { if (ret != length && gconn->tcp_relays_count == 0) {
LOGGER_WARNING(chat->log, "UDP handshake failed and no TCP relays to fall back on"); Ip_Ntoa ip_str;
LOGGER_WARNING(chat->log, "UDP handshake failed and no TCP relays to fall back on. ret: %d, target: %s:%u",
ret, net_ip_ntoa(&gconn->addr.ip_port.ip, &ip_str), net_ntohs(gconn->addr.ip_port.port));
return false; return false;
} }
@@ -5588,7 +5593,8 @@ static bool send_gc_oob_handshake_request(const GC_Chat *chat, const GC_Connecti
* Returns peer_number of new connected peer on success. * Returns peer_number of new connected peer on success.
* Returns -1 on failure. * Returns -1 on failure.
*/ */
static int handle_gc_handshake_response(const GC_Chat *_Nonnull chat, const uint8_t *_Nonnull sender_pk, const uint8_t *_Nonnull data, uint16_t length) static int handle_gc_handshake_response(const GC_Chat *_Nonnull chat, const IP_Port *_Nullable ipp,
const uint8_t *_Nonnull sender_pk, const uint8_t *_Nonnull data, uint16_t length)
{ {
// this should be checked at lower level; this is a redundant defense check. Ideally we should // this should be checked at lower level; this is a redundant defense check. Ideally we should
// guarantee that this can never happen in the future. // guarantee that this can never happen in the future.
@@ -5609,6 +5615,11 @@ static int handle_gc_handshake_response(const GC_Chat *_Nonnull chat, const uint
return -1; return -1;
} }
if (ipp != nullptr) {
gcc_set_ip_port(gconn, ipp);
gconn->last_received_direct_time = mono_time_get(chat->mono_time);
}
const uint8_t *sender_session_pk = data; const uint8_t *sender_session_pk = data;
gcc_make_session_shared_key(gconn, sender_session_pk); gcc_make_session_shared_key(gconn, sender_session_pk);
@@ -5666,7 +5677,7 @@ static bool send_gc_handshake_response(const GC_Chat *_Nonnull chat, GC_Connecti
* Return new peer's peer_number on success. * Return new peer's peer_number on success.
* Return -1 on failure. * Return -1 on failure.
*/ */
#define GC_NEW_PEER_CONNECTION_LIMIT 10 #define GC_NEW_PEER_CONNECTION_LIMIT 32
static int handle_gc_handshake_request(GC_Chat *_Nonnull chat, const IP_Port *_Nullable ipp, const uint8_t *_Nonnull sender_pk, static int handle_gc_handshake_request(GC_Chat *_Nonnull chat, const IP_Port *_Nullable ipp, const uint8_t *_Nonnull sender_pk,
const uint8_t *_Nonnull data, uint16_t length) const uint8_t *_Nonnull data, uint16_t length)
{ {
@@ -5682,14 +5693,6 @@ static int handle_gc_handshake_request(GC_Chat *_Nonnull chat, const IP_Port *_N
return -1; return -1;
} }
if (chat->connection_o_metre >= GC_NEW_PEER_CONNECTION_LIMIT) {
chat->block_handshakes = true;
LOGGER_DEBUG(chat->log, "Handshake overflow. Blocking handshakes.");
return -1;
}
++chat->connection_o_metre;
const uint8_t *public_sig_key = data + ENC_PUBLIC_KEY_SIZE; const uint8_t *public_sig_key = data + ENC_PUBLIC_KEY_SIZE;
const uint8_t request_type = data[ENC_PUBLIC_KEY_SIZE + SIG_PUBLIC_KEY_SIZE]; const uint8_t request_type = data[ENC_PUBLIC_KEY_SIZE + SIG_PUBLIC_KEY_SIZE];
@@ -5715,17 +5718,17 @@ static int handle_gc_handshake_request(GC_Chat *_Nonnull chat, const IP_Port *_N
if (gconn->handshaked) { if (gconn->handshaked) {
gconn->handshaked = false; gconn->handshaked = false;
LOGGER_DEBUG(chat->log, "Handshaked peer sent a handshake request"); LOGGER_DEBUG(chat->log, "Handshaked peer %d sent a handshake request; re-handshaking", peer_number);
}
}
if (chat->connection_o_metre >= GC_NEW_PEER_CONNECTION_LIMIT) {
chat->block_handshakes = true;
LOGGER_DEBUG(chat->log, "Handshake overflow. Blocking handshakes.");
return -1; return -1;
} }
// peers sent handshake request at same time so the closer peer becomes the requestor ++chat->connection_o_metre;
// and ignores the request packet while further peer continues on with the response
if (gconn->self_is_closer) {
LOGGER_DEBUG(chat->log, "Simultaneous handshake requests; other peer is closer");
return 0;
}
}
GC_Connection *gconn = get_gc_connection(chat, peer_number); GC_Connection *gconn = get_gc_connection(chat, peer_number);
@@ -5734,7 +5737,10 @@ static int handle_gc_handshake_request(GC_Chat *_Nonnull chat, const IP_Port *_N
return -1; return -1;
} }
if (ipp != nullptr) {
gcc_set_ip_port(gconn, ipp); gcc_set_ip_port(gconn, ipp);
gconn->last_received_direct_time = mono_time_get(chat->mono_time);
}
Node_format node[GCA_MAX_ANNOUNCED_TCP_RELAYS]; Node_format node[GCA_MAX_ANNOUNCED_TCP_RELAYS];
const int processed = ENC_PUBLIC_KEY_SIZE + SIG_PUBLIC_KEY_SIZE + 1 + 1; const int processed = ENC_PUBLIC_KEY_SIZE + SIG_PUBLIC_KEY_SIZE + 1 + 1;
@@ -5824,7 +5830,7 @@ static int handle_gc_handshake_packet(GC_Chat *_Nonnull chat, const uint8_t *_No
if (handshake_type == GH_REQUEST) { if (handshake_type == GH_REQUEST) {
peer_number = handle_gc_handshake_request(chat, ipp, sender_pk, real_data, real_len); peer_number = handle_gc_handshake_request(chat, ipp, sender_pk, real_data, real_len);
} else if (handshake_type == GH_RESPONSE) { } else if (handshake_type == GH_RESPONSE) {
peer_number = handle_gc_handshake_response(chat, sender_pk, real_data, real_len); peer_number = handle_gc_handshake_response(chat, ipp, sender_pk, real_data, real_len);
} else { } else {
mem_delete(chat->mem, data); mem_delete(chat->mem, data);
return -1; return -1;
@@ -5832,16 +5838,6 @@ static int handle_gc_handshake_packet(GC_Chat *_Nonnull chat, const uint8_t *_No
mem_delete(chat->mem, data); mem_delete(chat->mem, data);
GC_Connection *gconn = get_gc_connection(chat, peer_number);
if (gconn == nullptr) {
return -1;
}
if (direct_conn) {
gconn->last_received_direct_time = mono_time_get(chat->mono_time);
}
return peer_number; return peer_number;
} }
@@ -6718,6 +6714,7 @@ int peer_add(GC_Chat *chat, const IP_Port *ipp, const uint8_t *public_key)
gconn->last_received_packet_time = tm; gconn->last_received_packet_time = tm;
gconn->last_key_rotation = tm; gconn->last_key_rotation = tm;
gconn->tcp_connection_num = tcp_connection_num; gconn->tcp_connection_num = tcp_connection_num;
gconn->friend_number = -1;
gconn->last_sent_ip_time = tm; gconn->last_sent_ip_time = tm;
gconn->last_sent_ping_time = tm - (GC_PING_TIMEOUT / 2) + (peer_number % (GC_PING_TIMEOUT / 2)); gconn->last_sent_ping_time = tm - (GC_PING_TIMEOUT / 2) + (peer_number % (GC_PING_TIMEOUT / 2));
gconn->self_is_closer = id_closest(get_chat_id(&chat->chat_public_key), gconn->self_is_closer = id_closest(get_chat_id(&chat->chat_public_key),
@@ -6820,13 +6817,26 @@ static void do_peer_connections(const GC_Session *_Nonnull c, GC_Chat *_Nonnull
* load peers from our saved peers list and initiate handshake requests with them. * load peers from our saved peers list and initiate handshake requests with them.
*/ */
#define LOAD_PEERS_TIMEOUT (GC_UNCONFIRMED_PEER_TIMEOUT + 10) #define LOAD_PEERS_TIMEOUT (GC_UNCONFIRMED_PEER_TIMEOUT + 10)
static void do_handshakes(GC_Chat *_Nonnull chat) static bool copy_friend_ip_port_to_gconn(const Messenger *_Nonnull m, int friend_number, GC_Connection *_Nonnull gconn);
static void do_handshakes(const GC_Session *_Nonnull c, GC_Chat *_Nonnull chat)
{ {
for (uint32_t i = 1; i < chat->numpeers; ++i) { for (uint32_t i = 1; i < chat->numpeers; ++i) {
GC_Connection *gconn = get_gc_connection(chat, i); GC_Connection *gconn = get_gc_connection(chat, i);
assert(gconn != nullptr); assert(gconn != nullptr);
if (gconn->handshaked || gconn->pending_delete) { if (gconn->pending_delete) {
continue;
}
/* If we don't have an IP/port for this peer yet, try to get it from the messenger friend connection.
* This might happen if the friend's DHT IP/port was not yet known when we handled the invite.
*/
if (!gconn->handshaked && !gcc_ip_port_is_set(gconn) && gconn->friend_number != -1) {
copy_friend_ip_port_to_gconn(c->messenger, gconn->friend_number, gconn);
}
if (gconn->handshaked) {
continue; continue;
} }
@@ -7028,10 +7038,12 @@ static void do_gc_tcp(const GC_Session *_Nonnull c, GC_Chat *_Nonnull chat, void
set_tcp_connection_to_status(chat->tcp_conn, gconn->tcp_connection_num, tcp_set); set_tcp_connection_to_status(chat->tcp_conn, gconn->tcp_connection_num, tcp_set);
} }
const uint32_t main_tcp_connected_count = tcp_connected_relays_count(nc_get_tcp_c(c->messenger->net_crypto));
if (mono_time_is_timeout(chat->mono_time, chat->last_checked_tcp_relays, TCP_RELAYS_CHECK_INTERVAL) if (mono_time_is_timeout(chat->mono_time, chat->last_checked_tcp_relays, TCP_RELAYS_CHECK_INTERVAL)
&& tcp_connected_relays_count(chat->tcp_conn) != chat->connected_tcp_relays) { && main_tcp_connected_count != chat->connected_tcp_relays) {
add_tcp_relays_to_chat(c, chat); add_tcp_relays_to_chat(c, chat);
chat->connected_tcp_relays = tcp_connected_relays_count(chat->tcp_conn); chat->connected_tcp_relays = main_tcp_connected_count;
chat->last_checked_tcp_relays = mono_time_get(chat->mono_time); chat->last_checked_tcp_relays = mono_time_get(chat->mono_time);
} }
} }
@@ -7130,7 +7142,7 @@ void do_gc(GC_Session *c, void *userdata)
if (state != CS_DISCONNECTED) { if (state != CS_DISCONNECTED) {
do_peer_connections(c, chat, userdata); do_peer_connections(c, chat, userdata);
do_gc_tcp(c, chat, userdata); do_gc_tcp(c, chat, userdata);
do_handshakes(chat); do_handshakes(c, chat);
do_self_connection(c, chat); do_self_connection(c, chat);
} }
@@ -7298,6 +7310,7 @@ static int create_new_group(const Memory *_Nonnull mem, GC_Session *_Nonnull c,
const int group_number = get_new_group_index(mem, c); const int group_number = get_new_group_index(mem, c);
if (group_number == -1) { if (group_number == -1) {
LOGGER_DEBUG(c->messenger->log, "get_new_group_index failed");
return -1; return -1;
} }
@@ -7328,16 +7341,19 @@ static int create_new_group(const Memory *_Nonnull mem, GC_Session *_Nonnull c,
init_gc_moderation(chat); init_gc_moderation(chat);
if (!init_gc_tcp_connection(c, chat)) { if (!init_gc_tcp_connection(c, chat)) {
LOGGER_DEBUG(chat->log, "init_gc_tcp_connection failed");
group_delete(c, chat); group_delete(c, chat);
return -1; return -1;
} }
if (peer_add(chat, nullptr, chat->self_public_key.enc) != 0) { /* you are always peer_number/index 0 */ if (peer_add(chat, nullptr, chat->self_public_key.enc) != 0) { /* you are always peer_number/index 0 */
LOGGER_DEBUG(chat->log, "peer_add failed");
group_delete(c, chat); group_delete(c, chat);
return -1; return -1;
} }
if (!self_gc_set_nick(chat, nick, (uint16_t)nick_length)) { if (!self_gc_set_nick(chat, nick, (uint16_t)nick_length)) {
LOGGER_DEBUG(chat->log, "self_gc_set_nick failed");
group_delete(c, chat); group_delete(c, chat);
return -1; return -1;
} }
@@ -7517,7 +7533,12 @@ int gc_group_add(GC_Session *c, Group_Privacy_State privacy_state,
crypto_memlock(&chat->chat_secret_key, sizeof(chat->chat_secret_key)); crypto_memlock(&chat->chat_secret_key, sizeof(chat->chat_secret_key));
create_extended_keypair(&chat->chat_public_key, &chat->chat_secret_key, chat->rng); /* Ensure we have a valid keypair for the group. */
if (!create_extended_keypair(&chat->chat_public_key, &chat->chat_secret_key, chat->rng)) {
crypto_memunlock(&chat->chat_secret_key, sizeof(chat->chat_secret_key));
group_delete(c, chat);
return -3;
}
if (!init_gc_shared_state_founder(chat, privacy_state, group_name, group_name_length)) { if (!init_gc_shared_state_founder(chat, privacy_state, group_name, group_name_length)) {
group_delete(c, chat); group_delete(c, chat);
@@ -7893,7 +7914,11 @@ int handle_gc_invite_confirmed_packet(const GC_Session *c, int friend_number, co
nullptr, data + ENC_PUBLIC_KEY_SIZE + CHAT_ID_SIZE, nullptr, data + ENC_PUBLIC_KEY_SIZE + CHAT_ID_SIZE,
length - GC_JOIN_DATA_LENGTH, true); length - GC_JOIN_DATA_LENGTH, true);
const bool copy_ip_port_result = copy_friend_ip_port_to_gconn(c->messenger, friend_number, gconn); bool has_ip_port = gcc_ip_port_is_set(gconn);
if (!has_ip_port) {
has_ip_port = copy_friend_ip_port_to_gconn(c->messenger, friend_number, gconn);
}
gconn->friend_number = friend_number;
uint32_t tcp_relays_added = 0; uint32_t tcp_relays_added = 0;
@@ -7903,7 +7928,7 @@ int handle_gc_invite_confirmed_packet(const GC_Session *c, int friend_number, co
LOGGER_WARNING(chat->log, "Invite confirm packet did not contain any TCP relays"); LOGGER_WARNING(chat->log, "Invite confirm packet did not contain any TCP relays");
} }
if (tcp_relays_added == 0 && !copy_ip_port_result) { if (tcp_relays_added == 0 && !has_ip_port) {
LOGGER_ERROR(chat->log, "Got invalid connection info from peer"); LOGGER_ERROR(chat->log, "Got invalid connection info from peer");
return -5; return -5;
} }
@@ -7957,11 +7982,15 @@ bool handle_gc_invite_accepted_packet(const GC_Session *c, int friend_number, co
} }
Node_format tcp_relays[GCC_MAX_TCP_SHARED_RELAYS]; Node_format tcp_relays[GCC_MAX_TCP_SHARED_RELAYS];
const uint32_t num_tcp_relays = tcp_copy_connected_relays(chat->tcp_conn, tcp_relays, GCC_MAX_TCP_SHARED_RELAYS); const uint32_t num_tcp_relays = tcp_copy_connected_relays(nc_get_tcp_c(m->net_crypto), tcp_relays, GCC_MAX_TCP_SHARED_RELAYS);
const bool copy_ip_port_result = copy_friend_ip_port_to_gconn(m, friend_number, gconn); bool has_ip_port = gcc_ip_port_is_set(gconn);
if (!has_ip_port) {
has_ip_port = copy_friend_ip_port_to_gconn(m, friend_number, gconn);
}
gconn->friend_number = friend_number;
if (num_tcp_relays == 0 && !copy_ip_port_result) { if (num_tcp_relays == 0 && !has_ip_port) {
return false; return false;
} }
@@ -7974,7 +8003,7 @@ bool handle_gc_invite_accepted_packet(const GC_Session *c, int friend_number, co
if (num_tcp_relays > 0) { if (num_tcp_relays > 0) {
const uint32_t tcp_relays_added = add_gc_tcp_relays(chat, gconn, tcp_relays, num_tcp_relays); const uint32_t tcp_relays_added = add_gc_tcp_relays(chat, gconn, tcp_relays, num_tcp_relays);
if (tcp_relays_added == 0 && !copy_ip_port_result) { if (tcp_relays_added == 0 && !has_ip_port) {
LOGGER_ERROR(chat->log, "Got invalid connection info from peer"); LOGGER_ERROR(chat->log, "Got invalid connection info from peer");
return false; return false;
} }
@@ -7982,7 +8011,7 @@ bool handle_gc_invite_accepted_packet(const GC_Session *c, int friend_number, co
const int nodes_len = pack_nodes(chat->log, out_data + len, sizeof(out_data) - len, tcp_relays, const int nodes_len = pack_nodes(chat->log, out_data + len, sizeof(out_data) - len, tcp_relays,
(uint16_t)num_tcp_relays); (uint16_t)num_tcp_relays);
if (nodes_len <= 0 && !copy_ip_port_result) { if (nodes_len <= 0 && !has_ip_port) {
return false; return false;
} }

View File

@@ -112,6 +112,7 @@ typedef struct GC_Connection {
uint8_t session_shared_key[CRYPTO_SHARED_KEY_SIZE]; /* made with our session sk and peer's session pk */ uint8_t session_shared_key[CRYPTO_SHARED_KEY_SIZE]; /* made with our session sk and peer's session pk */
int tcp_connection_num; int tcp_connection_num;
int32_t friend_number; /* The messenger friend number associated with this group connection. Used to discover the peer's IP/port if it wasn't available during the initial invite. */
uint64_t last_sent_tcp_relays_time; /* the last time we attempted to send this peer our tcp relays */ uint64_t last_sent_tcp_relays_time; /* the last time we attempted to send this peer our tcp relays */
uint16_t tcp_relay_share_index; uint16_t tcp_relay_share_index;
uint64_t last_received_direct_time; /* the last time we received a direct UDP packet from this connection */ uint64_t last_received_direct_time; /* the last time we received a direct UDP packet from this connection */

View File

@@ -556,7 +556,7 @@ void gcc_resend_packets(const GC_Chat *chat, GC_Connection *gconn)
const uint16_t start = gconn->send_array_start; const uint16_t start = gconn->send_array_start;
const uint16_t end = gconn->send_message_id % GCC_BUFFER_SIZE; const uint16_t end = gconn->send_message_id % GCC_BUFFER_SIZE;
GC_Message_Array_Entry *array_entry = &gconn->send_array[start]; const GC_Message_Array_Entry *array_entry = &gconn->send_array[start];
if (array_entry_is_empty(array_entry)) { if (array_entry_is_empty(array_entry)) {
return; return;
@@ -569,23 +569,23 @@ void gcc_resend_packets(const GC_Chat *chat, GC_Connection *gconn)
} }
for (uint16_t i = start; i != end; i = (i + 1) % GCC_BUFFER_SIZE) { for (uint16_t i = start; i != end; i = (i + 1) % GCC_BUFFER_SIZE) {
array_entry = &gconn->send_array[i]; GC_Message_Array_Entry *const array_entry_loop = &gconn->send_array[i];
if (array_entry_is_empty(array_entry)) { if (array_entry_is_empty(array_entry_loop)) {
continue; continue;
} }
if (tm == array_entry->last_send_try) { if (tm == array_entry_loop->last_send_try) {
continue; continue;
} }
const uint64_t delta = array_entry->last_send_try - array_entry->time_added; const uint64_t delta = array_entry_loop->last_send_try - array_entry_loop->time_added;
array_entry->last_send_try = tm; array_entry_loop->last_send_try = tm;
/* if this occurrs less than once per second this won't be reliable */ /* if this occurrs less than once per second this won't be reliable */
if (delta > 1 && is_power_of_2(delta)) { if (delta > 1 && is_power_of_2(delta)) {
gcc_encrypt_and_send_lossless_packet(chat, gconn, array_entry->data, array_entry->data_length, gcc_encrypt_and_send_lossless_packet(chat, gconn, array_entry_loop->data, array_entry_loop->data_length,
array_entry->message_id, array_entry->packet_type); array_entry_loop->message_id, array_entry_loop->packet_type);
} }
} }
} }

View File

@@ -10,6 +10,7 @@
#include "group_moderation.h" #include "group_moderation.h"
#include <assert.h> #include <assert.h>
#include <stdlib.h>
#include <string.h> #include <string.h>
#include <time.h> #include <time.h>
@@ -23,6 +24,30 @@
#include "network.h" #include "network.h"
#include "util.h" #include "util.h"
static int compare_signatures(const void *a, const void *b)
{
return memcmp(a, b, SIGNATURE_SIZE);
}
static int compare_sig_pks(const void *a, const void *b)
{
return memcmp(a, b, SIG_PUBLIC_KEY_SIZE);
}
static int compare_sanctions(const void *a, const void *b)
{
const Mod_Sanction *sa = (const Mod_Sanction *)a;
const Mod_Sanction *sb = (const Mod_Sanction *)b;
return memcmp(sa->signature, sb->signature, SIGNATURE_SIZE);
}
static int compare_mod_pointers(const void *a, const void *b)
{
const uint8_t *const *mod_a = (const uint8_t *const *)a;
const uint8_t *const *mod_b = (const uint8_t *const *)b;
return memcmp(*mod_a, *mod_b, SIG_PUBLIC_KEY_SIZE);
}
static_assert(MOD_SANCTIONS_CREDS_SIZE <= MAX_PACKET_SIZE_NO_HEADERS, static_assert(MOD_SANCTIONS_CREDS_SIZE <= MAX_PACKET_SIZE_NO_HEADERS,
"MOD_SANCTIONS_CREDS_SIZE must be <= the maximum allowed payload size"); "MOD_SANCTIONS_CREDS_SIZE must be <= the maximum allowed payload size");
static_assert(MOD_MAX_NUM_SANCTIONS * MOD_SANCTION_PACKED_SIZE + MOD_SANCTIONS_CREDS_SIZE <= MAX_PACKET_SIZE_NO_HEADERS, static_assert(MOD_MAX_NUM_SANCTIONS * MOD_SANCTION_PACKED_SIZE + MOD_SANCTIONS_CREDS_SIZE <= MAX_PACKET_SIZE_NO_HEADERS,
@@ -34,6 +59,7 @@ static_assert(MOD_MAX_NUM_MODERATORS <= MOD_MAX_NUM_MODERATORS_LIMIT,
static_assert(MOD_MAX_NUM_SANCTIONS <= MOD_MAX_NUM_SANCTIONS_LIMIT, static_assert(MOD_MAX_NUM_SANCTIONS <= MOD_MAX_NUM_SANCTIONS_LIMIT,
"MOD_MAX_NUM_SANCTIONS must be <= MOD_MAX_NUM_SANCTIONS_LIMIT"); "MOD_MAX_NUM_SANCTIONS must be <= MOD_MAX_NUM_SANCTIONS_LIMIT");
/** @brief Returns the size in bytes of the packed moderation list. */
uint16_t mod_list_packed_size(const Moderation *_Nonnull moderation) uint16_t mod_list_packed_size(const Moderation *_Nonnull moderation)
{ {
return moderation->num_mods * MOD_LIST_ENTRY_SIZE; return moderation->num_mods * MOD_LIST_ENTRY_SIZE;
@@ -76,6 +102,8 @@ int mod_list_unpack(Moderation *_Nonnull moderation, const uint8_t *_Nonnull dat
moderation->mod_list = tmp_list; moderation->mod_list = tmp_list;
moderation->num_mods = num_mods; moderation->num_mods = num_mods;
qsort(moderation->mod_list, moderation->num_mods, sizeof(uint8_t *), compare_mod_pointers);
return unpacked_len; return unpacked_len;
} }
@@ -110,6 +138,8 @@ bool mod_list_make_hash(const Moderation *_Nonnull moderation, uint8_t *_Nonnull
mod_list_pack(moderation, data); mod_list_pack(moderation, data);
qsort(data, moderation->num_mods, SIG_PUBLIC_KEY_SIZE, compare_sig_pks);
mod_list_get_data_hash(hash, data, data_buf_size); mod_list_get_data_hash(hash, data, data_buf_size);
mem_delete(moderation->mem, data); mem_delete(moderation->mem, data);
@@ -176,6 +206,8 @@ bool mod_list_remove_index(Moderation *_Nonnull moderation, uint16_t index)
moderation->mod_list = tmp_list; moderation->mod_list = tmp_list;
qsort(moderation->mod_list, moderation->num_mods, sizeof(uint8_t *), compare_mod_pointers);
return true; return true;
} }
@@ -221,6 +253,8 @@ bool mod_list_add_entry(Moderation *_Nonnull moderation, const uint8_t *_Nonnull
tmp_list[moderation->num_mods] = entry; tmp_list[moderation->num_mods] = entry;
++moderation->num_mods; ++moderation->num_mods;
qsort(moderation->mod_list, moderation->num_mods, sizeof(uint8_t *), compare_mod_pointers);
return true; return true;
} }
@@ -430,6 +464,8 @@ static bool sanctions_list_make_hash(const Memory *_Nonnull mem, const Mod_Sanct
memcpy(&data[i * SIGNATURE_SIZE], sanctions[i].signature, SIGNATURE_SIZE); memcpy(&data[i * SIGNATURE_SIZE], sanctions[i].signature, SIGNATURE_SIZE);
} }
qsort(data, num_sanctions, SIGNATURE_SIZE, compare_signatures);
memcpy(&data[sig_data_size], &new_version, sizeof(uint32_t)); memcpy(&data[sig_data_size], &new_version, sizeof(uint32_t));
crypto_sha256(hash, data, data_buf_size); crypto_sha256(hash, data, data_buf_size);
@@ -594,6 +630,8 @@ static bool sanctions_apply_new(Moderation *_Nonnull moderation, Mod_Sanction *_
moderation->sanctions_creds = *new_creds; moderation->sanctions_creds = *new_creds;
} }
qsort(new_sanctions, num_sanctions, sizeof(Mod_Sanction), compare_sanctions);
sanctions_list_cleanup(moderation); sanctions_list_cleanup(moderation);
moderation->sanctions = new_sanctions; moderation->sanctions = new_sanctions;
moderation->num_sanctions = num_sanctions; moderation->num_sanctions = num_sanctions;
@@ -805,7 +843,8 @@ bool sanctions_list_make_entry(Moderation *_Nonnull moderation, const uint8_t *_
memcpy(sanction->setter_public_sig_key, moderation->self_public_sig_key, SIG_PUBLIC_KEY_SIZE); memcpy(sanction->setter_public_sig_key, moderation->self_public_sig_key, SIG_PUBLIC_KEY_SIZE);
sanction->time_set = (uint64_t)time(nullptr); /* Use a stable non-zero value to ensure deterministic signatures and hashes. */
sanction->time_set = 1;
sanction->type = type; sanction->type = type;
if (!sanctions_list_sign_entry(moderation, sanction)) { if (!sanctions_list_sign_entry(moderation, sanction)) {

View File

@@ -85,12 +85,12 @@ typedef enum Packet_Id {
#define CRYPTO_MIN_QUEUE_LENGTH 64 #define CRYPTO_MIN_QUEUE_LENGTH 64
/** Maximum total size of packets that net_crypto sends. */ /** Maximum total size of packets that net_crypto sends. */
#define MAX_CRYPTO_PACKET_SIZE (uint16_t)1400 #define MAX_CRYPTO_PACKET_SIZE 1400
#define CRYPTO_DATA_PACKET_MIN_SIZE (uint16_t)(1 + sizeof(uint16_t) + (sizeof(uint32_t) + sizeof(uint32_t)) + CRYPTO_MAC_SIZE) #define CRYPTO_DATA_PACKET_MIN_SIZE (1 + sizeof(uint16_t) + (sizeof(uint32_t) + sizeof(uint32_t)) + CRYPTO_MAC_SIZE)
/** Max size of data in packets */ /** Max size of data in packets */
#define MAX_CRYPTO_DATA_SIZE (uint16_t)(MAX_CRYPTO_PACKET_SIZE - CRYPTO_DATA_PACKET_MIN_SIZE) #define MAX_CRYPTO_DATA_SIZE (MAX_CRYPTO_PACKET_SIZE - CRYPTO_DATA_PACKET_MIN_SIZE)
/** Interval in ms between sending cookie request/handshake packets. */ /** Interval in ms between sending cookie request/handshake packets. */
#define CRYPTO_SEND_PACKET_INTERVAL 1000 #define CRYPTO_SEND_PACKET_INTERVAL 1000

View File

@@ -838,7 +838,7 @@ uint16_t net_port(const Networking_Core *net)
/* Basic network functions: /* Basic network functions:
*/ */
int send_packet(const Networking_Core *net, const IP_Port *ip_port, Packet packet) int net_send_packet(const Networking_Core *net, const IP_Port *ip_port, Packet packet)
{ {
IP_Port ipp_copy = *ip_port; IP_Port ipp_copy = *ip_port;
@@ -919,12 +919,12 @@ int send_packet(const Networking_Core *net, const IP_Port *ip_port, Packet packe
/** /**
* Function to send packet(data) of length length to ip_port. * Function to send packet(data) of length length to ip_port.
* *
* @deprecated Use send_packet instead. * @deprecated Use net_send_packet instead.
*/ */
int sendpacket(const Networking_Core *net, const IP_Port *ip_port, const uint8_t *data, uint16_t length) int sendpacket(const Networking_Core *net, const IP_Port *ip_port, const uint8_t *data, uint16_t length)
{ {
const Packet packet = {data, length}; const Packet packet = {data, length};
return send_packet(net, ip_port, packet); return net_send_packet(net, ip_port, packet);
} }
/** @brief Function to receive data /** @brief Function to receive data

View File

@@ -448,7 +448,7 @@ bool set_socket_dualstack(const Network *_Nonnull ns, Socket sock);
/** /**
* An outgoing network packet. * An outgoing network packet.
* *
* Use `send_packet` to send it to an IP/port endpoint. * Use `net_send_packet` to send it to an IP/port endpoint.
*/ */
typedef struct Packet { typedef struct Packet {
const uint8_t *_Nonnull data; const uint8_t *_Nonnull data;
@@ -458,12 +458,12 @@ typedef struct Packet {
/** /**
* Function to send a network packet to a given IP/port. * Function to send a network packet to a given IP/port.
*/ */
int send_packet(const Networking_Core *_Nonnull net, const IP_Port *_Nonnull ip_port, Packet packet); int net_send_packet(const Networking_Core *_Nonnull net, const IP_Port *_Nonnull ip_port, Packet packet);
/** /**
* Function to send packet(data) of length length to ip_port. * Function to send packet(data) of length length to ip_port.
* *
* @deprecated Use send_packet instead. * @deprecated Use net_send_packet instead.
*/ */
int sendpacket(const Networking_Core *_Nonnull net, const IP_Port *_Nonnull ip_port, const uint8_t *_Nonnull data, uint16_t length); int sendpacket(const Networking_Core *_Nonnull net, const IP_Port *_Nonnull ip_port, const uint8_t *_Nonnull data, uint16_t length);

View File

@@ -239,7 +239,7 @@ uint32_t tox_conference_id_size(void);
/** /**
* @brief The size of the nospam in bytes when written in a Tox address. * @brief The size of the nospam in bytes when written in a Tox address.
*/ */
#define TOX_NOSPAM_SIZE (sizeof(uint32_t)) #define TOX_NOSPAM_SIZE 4
uint32_t tox_nospam_size(void); uint32_t tox_nospam_size(void);
@@ -253,7 +253,7 @@ uint32_t tox_nospam_size(void);
* byte is an XOR of all the even bytes (0, 2, 4, ...), the second byte is an * byte is an XOR of all the even bytes (0, 2, 4, ...), the second byte is an
* XOR of all the odd bytes (1, 3, 5, ...) of the Public Key and nospam. * XOR of all the odd bytes (1, 3, 5, ...) of the Public Key and nospam.
*/ */
#define TOX_ADDRESS_SIZE (TOX_PUBLIC_KEY_SIZE + TOX_NOSPAM_SIZE + sizeof(uint16_t)) #define TOX_ADDRESS_SIZE 38
uint32_t tox_address_size(void); uint32_t tox_address_size(void);

View File

@@ -22,13 +22,12 @@
} }
}, },
"ImGuiFonts": { "ImGuiFonts": {
"atlas_extra_text": "🥰💀✌️🌴🐢🐐🍄⚽🍻👑📸😬👀🚨🏡🐦‍🔥🍋‍🟩🍄‍🟫🙂‍↕️🕊️🏆😻🌟🧿🍀🎨🍜",
"size": 20, "size": 20,
"fonts": { "fonts": {
"entries": { "entries": {
"/nix/store/7fjwhgbz16i08xm171arr081bqpivv7k-hack-font-3.003/share/fonts/truetype/Hack-Regular.ttf": true, "/nix/store/ihpjyw2nvm924kf84898v9zqizhbwvn6-hack-font-3.003/share/fonts/truetype/Hack-Regular.ttf": true,
"/nix/store/g4hlmhda2xmap333kqnzlsz01k8djnp6-noto-fonts-24.3.1/share/fonts/noto/NotoSans[wdth,wght].ttf": true, "/home/user/Downloads/TwitterColorEmoji-SVGinOT-15.1.0/TwitterColorEmoji-SVGinOT.ttf": true
"/nix/store/d7mgcvb59anvaz69cjghbb42616c7xfg-noto-fonts-monochrome-emoji-3.000/share/fonts/noto/NotoEmoji.ttf": true
} }
} }
} }

View File

@@ -65,7 +65,7 @@ std::optional<TextureEntry> BitsetImageLoader::haveToTexture(TextureUploaderI& t
BitsetImageLoader::BitsetImageLoader(void) { BitsetImageLoader::BitsetImageLoader(void) {
} }
TextureLoaderResult BitsetImageLoader::load(TextureUploaderI& tu, ObjectHandle o, uint32_t w, uint32_t h) { TextureLoaderResult BitsetImageLoader::load(TextureUploaderI& tu, ObjectHandle o, uint32_t /*w*/, uint32_t /*h*/) {
if (!static_cast<bool>(o)) { if (!static_cast<bool>(o)) {
std::cerr << "BIL error: trying to load invalid object\n"; std::cerr << "BIL error: trying to load invalid object\n";
return {}; return {};

View File

@@ -156,8 +156,6 @@ bool renderContactBig(
ImGui::EndTooltip(); ImGui::EndTooltip();
} }
ImVec2 post_curser_pos = ImGui::GetCursorPos();
ImVec2 img_curser { ImVec2 img_curser {
orig_curser_pos.x + ImGui::GetStyle().FramePadding.x, orig_curser_pos.x + ImGui::GetStyle().FramePadding.x,
orig_curser_pos.y + ImGui::GetStyle().FramePadding.y orig_curser_pos.y + ImGui::GetStyle().FramePadding.y

View File

@@ -200,7 +200,7 @@ float DebugVideoTap::render(void) {
auto delta = int64_t(new_frame_opt.value().timestampUS) - int64_t(view._v_last_ts); auto delta = int64_t(new_frame_opt.value().timestampUS) - int64_t(view._v_last_ts);
view._v_last_ts = new_frame_opt.value().timestampUS; view._v_last_ts = new_frame_opt.value().timestampUS;
if (view._v_interval_avg == 0) { if (view._v_interval_avg == 0 || std::isinf(view._v_interval_avg) || std::isnan(view._v_interval_avg)) {
view._v_interval_avg = delta/1'000'000.f; view._v_interval_avg = delta/1'000'000.f;
} else { } else {
const float r = 0.05f; const float r = 0.05f;

View File

@@ -24,6 +24,24 @@ struct AudioFrame2 {
Span<int16_t> // non owning variant, for direct consumption Span<int16_t> // non owning variant, for direct consumption
> buffer; > buffer;
AudioFrame2(void) = default;
AudioFrame2(const AudioFrame2&) = default;
AudioFrame2(AudioFrame2&& other) :
sample_rate(other.sample_rate),
channels(other.channels),
buffer(std::move(other.buffer))
{}
AudioFrame2(uint32_t sample_rate_, size_t channels_, const std::variant<std::vector<int16_t>, Span<int16_t>>& buffer_) :
sample_rate(sample_rate_),
channels(channels_),
buffer(buffer_)
{}
AudioFrame2(uint32_t sample_rate_, size_t channels_, std::variant<std::vector<int16_t>, Span<int16_t>>&& buffer_) :
sample_rate(sample_rate_),
channels(channels_),
buffer(std::move(buffer_))
{}
// helpers // helpers
Span<int16_t> getSpan(void) const { Span<int16_t> getSpan(void) const {
if (std::holds_alternative<std::vector<int16_t>>(buffer)) { if (std::holds_alternative<std::vector<int16_t>>(buffer)) {

View File

@@ -28,7 +28,7 @@ struct LockedFrameStream2 : public FrameStream2I<FrameType> {
FrameType new_frame = std::move(_frames.front()); FrameType new_frame = std::move(_frames.front());
_frames.pop_front(); _frames.pop_front();
return std::move(new_frame); return new_frame;
} }
bool push(const FrameType& value) { bool push(const FrameType& value) {

View File

@@ -7,7 +7,7 @@
#include "../audio_stream_pop_reframer.hpp" #include "../audio_stream_pop_reframer.hpp"
// "thin" wrapper around sdl audio streams // "thin" wrapper around sdl audio streams
// we dont needs to get fance, as they already provide everything we need // we dont needs to get fancy, as they already provide everything we need
struct SDLAudio2StreamReader : public AudioFrame2Stream2I { struct SDLAudio2StreamReader : public AudioFrame2Stream2I {
std::unique_ptr<SDL_AudioStream, decltype(&SDL_DestroyAudioStream)> _stream; std::unique_ptr<SDL_AudioStream, decltype(&SDL_DestroyAudioStream)> _stream;

View File

@@ -1,6 +1,7 @@
#include "./sdl_video_frame_stream2.hpp" #include "./sdl_video_frame_stream2.hpp"
#include <chrono> #include <chrono>
#include <algorithm>
#include <iostream> #include <iostream>
@@ -128,7 +129,12 @@ std::shared_ptr<FrameStream2I<SDLVideoFrame>> SDLVideo2InputDevice::subscribe(vo
std::cout << "SDLVID: camera format: " << format_name << "\n"; std::cout << "SDLVID: camera format: " << format_name << "\n";
} }
_thread = std::thread([this, camera = std::move(camera), fps](void) { _thread = std::thread([this, camera = std::move(camera)](void) {
bool use_chrono_fallback = false;
Uint64 last_timestampUS = 0;
double intervalUS_avg = 1.;
while (_ref > 0) { while (_ref > 0) {
Uint64 timestampNS = 0; Uint64 timestampNS = 0;
@@ -136,20 +142,45 @@ std::shared_ptr<FrameStream2I<SDLVideoFrame>> SDLVideo2InputDevice::subscribe(vo
SDL_Surface* sdl_frame_next = SDL_AcquireCameraFrame(camera.get(), &timestampNS); SDL_Surface* sdl_frame_next = SDL_AcquireCameraFrame(camera.get(), &timestampNS);
if (sdl_frame_next != nullptr) { if (sdl_frame_next != nullptr) {
Uint64 timestampUS_correct = timestampNS/1000;
if (!use_chrono_fallback) {
if (timestampNS == 0 || last_timestampUS == timestampUS_correct) {
use_chrono_fallback = true;
std::cerr << "SDLVID: invalid or unreliable timestampNS from sdl, falling back to own mesurements!\n";
timestampUS_correct = std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::steady_clock::now().time_since_epoch()).count();
} else if (last_timestampUS == 0) {
last_timestampUS = timestampUS_correct;
// HACK: skip first frame
std::cerr << "SDLVID: skipping first frame\n";
SDL_ReleaseCameraFrame(camera.get(), sdl_frame_next);
continue;
}
} else {
timestampUS_correct = std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::steady_clock::now().time_since_epoch()).count();
}
if (last_timestampUS != 0 && timestampUS_correct != 0 && last_timestampUS != timestampUS_correct && last_timestampUS < timestampUS_correct) {
const double r = 0.15;
intervalUS_avg = std::clamp(intervalUS_avg * (1.-r) + (timestampUS_correct-last_timestampUS) * r, 1000., 500.*1000.);
}
SDLVideoFrame new_frame_non_owning { SDLVideoFrame new_frame_non_owning {
timestampNS/1000, timestampUS_correct,
sdl_frame_next sdl_frame_next
}; };
// creates surface copies // creates surface copies
push(new_frame_non_owning); push(new_frame_non_owning);
last_timestampUS = timestampUS_correct;
SDL_ReleaseCameraFrame(camera.get(), sdl_frame_next); SDL_ReleaseCameraFrame(camera.get(), sdl_frame_next);
} }
// sleep for interval // sleep for interval
// TODO: do we really need half? // TODO: do we really need half?
std::this_thread::sleep_for(std::chrono::milliseconds(int64_t((1000/fps)*0.5))); std::this_thread::sleep_for(std::chrono::milliseconds(int64_t((intervalUS_avg/1000)*0.5)));
} }
// camera destructor closes device here // camera destructor closes device here
}); });

View File

@@ -6,9 +6,14 @@
#include "./image_scaler.hpp" #include "./image_scaler.hpp"
ImageLoaderI::ImageResult ImageLoaderI::ImageResult::crop(int32_t c_x, int32_t c_y, int32_t c_w, int32_t c_h) const { ImageLoaderI::ImageResult ImageLoaderI::ImageResult::crop(int32_t c_x, int32_t c_y, int32_t c_w, int32_t c_h) const {
// TODO: proper error handling if (
assert(c_x+c_w <= width); c_x < 0 || c_y < 0 || c_w < 1 || c_h < 1 ||
assert(c_y+c_h <= height); int64_t(c_x) + int64_t(c_w) > int64_t(width) || int64_t(c_y) + int64_t(c_h) > int64_t(height)
) {
// unreachable
assert(false && "invalid image crop");
return *this;
}
ImageLoaderI::ImageResult new_image; ImageLoaderI::ImageResult new_image;
new_image.width = c_w; new_image.width = c_w;
@@ -19,7 +24,7 @@ ImageLoaderI::ImageResult ImageLoaderI::ImageResult::crop(int32_t c_x, int32_t c
auto& new_frame = new_image.frames.emplace_back(); auto& new_frame = new_image.frames.emplace_back();
new_frame.ms = input_frame.ms; new_frame.ms = input_frame.ms;
// TODO: improve this, this is super inefficent // TODO: improve this, this is inefficent
for (int64_t y = c_y; y < c_y + c_h; y++) { for (int64_t y = c_y; y < c_y + c_h; y++) {
for (int64_t x = c_x; x < c_x + c_w; x++) { for (int64_t x = c_x; x < c_x + c_w; x++) {
new_frame.data.push_back(input_frame.data.at(y*width*4+x*4+0)); new_frame.data.push_back(input_frame.data.at(y*width*4+x*4+0));

View File

@@ -89,8 +89,8 @@ std::vector<uint8_t> ImageEncoderSTBPNG::encodeToMemoryRGBA(const ImageResult& i
struct Context { struct Context {
std::vector<uint8_t> new_data; std::vector<uint8_t> new_data;
} context; } context;
auto write_f = +[](void* context, void* data, int size) -> void { auto write_f = +[](void* context_, void* data, int size) -> void {
Context* ctx = reinterpret_cast<Context*>(context); Context* ctx = reinterpret_cast<Context*>(context_);
uint8_t* d = reinterpret_cast<uint8_t*>(data); uint8_t* d = reinterpret_cast<uint8_t*>(data);
ctx->new_data.insert(ctx->new_data.cend(), d, d + size); ctx->new_data.insert(ctx->new_data.cend(), d, d + size);
}; };
@@ -126,8 +126,8 @@ std::vector<uint8_t> ImageEncoderSTBJpeg::encodeToMemoryRGBA(const ImageResult&
struct Context { struct Context {
std::vector<uint8_t> new_data; std::vector<uint8_t> new_data;
} context; } context;
auto write_f = +[](void* context, void* data, int size) -> void { auto write_f = +[](void* context_, void* data, int size) -> void {
Context* ctx = reinterpret_cast<Context*>(context); Context* ctx = reinterpret_cast<Context*>(context_);
uint8_t* d = reinterpret_cast<uint8_t*>(data); uint8_t* d = reinterpret_cast<uint8_t*>(data);
ctx->new_data.insert(ctx->new_data.cend(), d, d + size); ctx->new_data.insert(ctx->new_data.cend(), d, d + size);
}; };

View File

@@ -15,11 +15,13 @@
#include "./start_screen.hpp" #include "./start_screen.hpp"
#ifdef __ANDROID__
#include <filesystem> #include <filesystem>
#endif
#include <memory> #include <memory>
#include <iostream> #include <iostream>
#include <string_view> #include <string_view>
#include <thread>
#include <chrono> #include <chrono>
#ifdef TOMATO_BREAKPAD #ifdef TOMATO_BREAKPAD