Green-Sky/tomato
1
0
Fork 1
Code Issues Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

Squashed 'external/toxcore/c-toxcore/' changes from e2c01e457b..b03b571272

Browse Source 
b03b571272 fix: flaky tcp test This only fixes the symptoms, not the real problem. Sometimes or consistently on some platforms a socket might need a moment before it can be written to.
32e67ab4c2 cleanup: use typedef for private message ID's in callback
7b1db6adc1 feat: add message IDs to private group messages
99e0bcc27d refactor: Observers/ignored peers can now send and receive custom packets
b3c3c49d26 fix: Disable IPv6 in Windows cross-compilation tests
e742deddff feat: Check hashes of Windows dependencies when cross-compiling
dfb9a0b02b fix: Test the current Windows Dockerfile, not an old Dockerhub image
14de93ccec chore: Use WineHQ's Wine as Debian Bookworm's crashes
ed37616249 docs: Update the Windows cross-compilation section
9bb79c174f cleanup: Remove a couple of unnecessary misc_tools dependencies
19475adb70 chore: Statically link OpenMP into the cracker fun util on Windows
1be311e51f feat: Build the fun utils when cross-compiling to Windows
88133f8446 chore: Strip Windows binaries
3cc0ae7535 refactor: Copy over all of the required static dependencies
c4fa8f7fb1 feat: Generate .def, .exp and .lib files when building for Windows
74bbac5363 feat: Let CMake create the dll instead of doing so ourselves
246642e9ae feat: Harden Windows cross-compilation
8d431c0d11 chore: Bump Windows build dependency versions
e519f7998b fix: Remove unnecessary wsock32 dependency on Windows
ed2b60c217 chore: Use a specific non-broken slimcc version.
d7f21010a1 chore: Update github actions.
e71a68b7f2 docs: Update the list of CMake options
77e08876ff chore: Remove mod and founder from group API naming scheme
12bc042767 docs: add the experimental api build option to INSTALL.md
e1fa5cae96 refactor: Rename Queries to Query to align with other enums.
be82a3ea30 fix: Correct type for conference offline peer numbers.
0627c36716 test: Add pkgsrc build.
92578afe4b test: Add FreeBSD VM action on GitHub.
52ece0f57b test: Build toxcore on NetBSD (VM).
3fe8ee2c11 chore: Only install tox_private.h on request.
9a8dfa06ab fix: save_compatibility_test failing on big-endian systems
86f5e55578 fix: Don't serve files from websockify.
710eb674a5 fix: Correctly pass extended public keys to group moderation code.
021db7031c refactor: Use `struct`s for extended public/secret keys.
a1e999fd80 chore: Compile libsodium reference implementation with compcert.
fbe3c19cf5 cleanup: correct a few nullable annotations
623e3ee5c3 cleanup: Don't use `memcpy` to cast arbitrary `struct`s to `uint8_t[]`.
c71567dc18 fix: Pass array, not array pointer, to `memcmp`.
9b46a08144 cleanup: Never pass `void*` directly to `memcpy`.
5d7b7a7bbc refactor: Use tox rng to seed the keypair generation.
961891d568 cleanup: Small improvements found by PVS Studio.
8201019f0d chore: Disable NGC saving by default, enable through Tox_Options.
5dd9ee3f65 cleanup: Replace pointer arithmetic with explicit `&arr[i]`.
ca4606d49d refactor: Use strong typedef for NGC peer id.
442213b722 cleanup: Simplify custom packet length check in NGC.
08d3393def fix: Correct a few potential null derefs in bootstrap daemon.
b9877b32b0 fix: Add missing memunlock of local variable when it goes out of scope.
dab5fe44b9 fix: Zero out stack-allocated secret key before return.
f058103299 refactor: Make prune_gc_sanctions_list more obviously correct.
3ba7a0dec9 docs: Add static analysis tool list to README.
8d0811a0f3 docs: Run prettier-markdown on markdown files.
969e3a2bfc refactor: Fix network test not using the strong typedef
93c83fbc7c refactor: Use strong typedef instead of struct for `Socket`.
9fe18b176f fix: Fix some false positive from PVS Studio.
7c44379ccb cleanup: Check that WINXP macro exists before comparing it.
5c93231bef refactor: Make tox mutex non-recursive.
aacff73939 docs: Fix up doxyfile.
d55fc85ff5 docs: Add more documentation to crypto_core.
5bdaaaedb6 refactor: Remove `Tox *` from `tox_dispatch`.
e202341e76 refactor: Don't rely on tox_dispatch passing tox in tests.
34df938f52 chore: Use C++ mode for clang-tidy.
8b05296a78 chore: Check that both gtest and gmock exist for tests.
42010660e1 test: Add slimcc compiler compatibility test.
b473630321 chore: Add some comments to the astyle config.
b7404f24f6 cleanup: Remove implicit bool conversions.
4e2dba4d9f chore: Reformat sources with astyle.
4359e3a6bc chore: Rename C++ headers to .hh suffixes.
0c05566e58 cleanup: Further `#include` cleanups.
8d29935b7a chore: Only check the bootstrap daemon checksum on release.
f70e588bc6 cleanup: Add more `const` where possible.
511bfe39c8 cleanup: Use Bazel modules to enforce proper `#include` hygiene.
1710a0d091 refactor: Move pack/unpack `IP_Port` from DHT into network module.
a975943564 chore: Really fix coverage docker image build.
c08409390f chore: Fix post-submit coverage image.
39aadf8922 fix: Don't use `memcmp` to compare `IP_Port`s.
d94246a906 fix: partially fix a bug that prevented group part messages from sending.
eeaa039222 chore: Fix rpm build; add a CI check for it.
8328449c1a chore: Speed up docker builds a bit by reducing layer count.
d6d67d56f3 cleanup: Add `const` where possible in auto tests.
6aa9e6850d cleanup: Minor cleanup of event unpack code.
bdf460a3a9 refactor: Rename `system_{memory,...}` to `os_{memory,...}`.
203e1af81e fix: a few off by one errors in group autotests
5c093c4888 cleanup: Remove all uses of `SIZEOF_VLA`.
662c2140f3 test: Add goblint static analyser.
8f07755834 cleanup: Use `memzero(x, s)` instead of `memset(x, 0, s)`.
a7258e40cf cleanup: Use explicit 0 instead of `PACKET_ID_PADDING`.
6370d0f15d cleanup: Expand the `Tox_Options` accessor macros.
14a1a0b9bd cleanup: Remove plan9 support.
a05dccad13 test: Add a simple new/delete test for Tox.
1cdcf938b9 cleanup: Add comment after every `#endif`.
ba99d4dc4b test: Fix comment I broke in the events test PR.
e07248debb refactor: Migrate auto_tests to new events API.
bdd42b5452 refactor: Add common msgpack array packer with callback.
3c659f5288 cleanup: Rename group to conference in groupav documentation.
89957be230 cleanup: Ensure handler params are named after callback params.
c650d9d345 refactor: Pass `this` pointer as first param to s11n callbacks.
e7fb91ddb8 refactor: Allow NULL pointers for byte arrays in events.
5e2c8cabc1 cleanup: make some improvements to group moderation test
259de4867e cleanup: Remove `bin_pack_{new,free}`.
21a8ff5895 cleanup: skip a do_gc iteration before removing peers marked for deletion
16809dc36e feat: Add dht_get_nodes_response event to the events system.

git-subtree-dir: external/toxcore/c-toxcore
git-subtree-split: b03b5712720de9a9901ea12fd741f177327a7021
This commit is contained in:
Green Sky
2024-03-07 23:12:55 +01:00
parent 61accfe184
commit aae086cc65
358 changed files with 8093 additions and 5229 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
docs/Doxyfile
View File

@ -1295,15 +1295,6 @@ HTML_COLORSTYLE_SAT = 100
HTML_COLORSTYLE_GAMMA = 80
# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML
# page will contain the date and time when the page was generated. Setting this
# to YES can help to show when doxygen was last run and thus if the
# documentation is up to date.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_TIMESTAMP = NO
# If the HTML_DYNAMIC_MENUS tag is set to YES then the generated HTML
# documentation will contain a main index with vertical navigation menus that
# are dynamically created via JavaScript. If disabled, the navigation index will
@ -1950,14 +1941,6 @@ LATEX_HIDE_INDICES = NO
LATEX_BIB_STYLE = plain
# If the LATEX_TIMESTAMP tag is set to YES then the footer of each generated
# page will contain the date and time when the page was generated. Setting this
# to NO can help when comparing the output of multiple runs.
# The default value is: NO.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_TIMESTAMP = NO
# The LATEX_EMOJI_DIRECTORY tag is used to specify the (relative or absolute)
# path from which the emoji images will be read. If a relative path is entered,
# it will be relative to the LATEX_OUTPUT directory. If left blank the
@ -2587,3 +2570,19 @@ GENERATE_LEGEND = YES
# The default value is: YES.
DOT_CLEANUP = YES
# If the CLANG_ASSISTED_PARSING tag is set to YES then doxygen will use the
# clang parser for more accurate parsing at the cost of reduced performance.
# This can be particularly helpful with template rich C++ code for which
# doxygen's built-in parser lacks the necessary type information.
CLANG_ASSISTED_PARSING = NO
# If clang assisted parsing is enabled you can provide the clang parser with
# the path to the directory containing a file called compile_commands.json.
# This file is the compilation database containing the options used when the
# source files were built. This is equivalent to specifying the -p option to a
# clang tool, such as clang-check. These options will then be passed to the
# parser. Any options specified with CLANG_OPTIONS will be added as well.
CLANG_DATABASE_PATH = _build

109
docs/Group-Chats.md
View File

@ -2,77 +2,122 @@ Group chats.
Note: we assume everyone in the chat trusts each other.
These group chats work by temporarily adding the 4 "closest" people defined by a distance function
in group.c in order to form a circle of connected peers. These peers then relay messages to each other.
A friend invites another friend to a group chat by sending them an invite packet. The friend either ignores
the invite or responds with a response packet if he wants to join the chat. The friend invite contains the type
of groupchat (text only, A/V) the friend is being invited to.
These group chats work by temporarily adding the 4 "closest" people defined by a
distance function in group.c in order to form a circle of connected peers. These
peers then relay messages to each other.
A friend invites another friend to a group chat by sending them an invite
packet. The friend either ignores the invite or responds with a response packet
if he wants to join the chat. The friend invite contains the type of groupchat
(text only, A/V) the friend is being invited to.
TODO(irungentoo): write more of this.
## Protocol
Invite packets:
Invite packet:
### Invite packet:
```
[uint8_t id 96][uint8_t id 0][uint16_t group chat number][33 bytes group chat identifier[1 byte type][32 bytes id]]
```
Response packet
### Response packet
```
[uint8_t id 96][uint8_t id 1][uint16_t group chat number(local)][uint16_t group chat number to join][33 bytes group chat identifier[1 byte type][32 bytes id]]
```
### Peer online packet:
Peer online packet:
```
[uint8_t id 97][uint16_t group chat number (local)][33 bytes group chat identifier[1 byte type][32 bytes id]]
```
Peer leave packet:
### Peer leave packet:
```
[uint8_t id 98][uint16_t group chat number][uint8_t id 1]
```
Peer query packet:
### Peer query packet:
```
[uint8_t id 98][uint16_t group chat number][uint8_t id 8]
```
Peer response packet:
[uint8_t id 98][uint16_t group chat number][uint8_t id 9][Repeated times number of peers: [uint16_t peer num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key][uint8_t name length][name]]
### Peer response packet:
Title response packet:
```
[uint8_t id 98][uint16_t group chat number][uint8_t id 9][Repeated times number of peers: [uint16_t peer num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key][uint8_t name length][name]]
```
### Title response packet:
```
[uint8_t id 98][uint16_t group chat number][uint8_t id 10][title]
```
Message packets:
### Message packets:
```
[uint8_t id 99][uint16_t group chat number][uint16_t peer number][uint32_t message number][uint8_t with a value representing id of message][data]
```
Lossy Message packets:
### Lossy Message packets:
```
[uint8_t id 199][uint16_t group chat number][uint16_t peer number][uint16_t message number][uint8_t with a value representing id of message][data]
```
Group chat types:
0: text
1: AV
## Group chat types:
- 0: text
- 1: AV
Note: the message number is increased by 1 for each sent message.
message ids:
0 - ping
sent every ~60 seconds by every peer.
No data.
## message ids:
### 0 - ping
sent every ~60 seconds by every peer. No data.
### 16 - new_peer
16 - new_peer
Tell everyone about a new peer in the chat.
```
[uint16_t peer_num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key]
```
17 - kill_peer
### 17 - kill_peer
```
[uint16_t peer_num]
```
48 - name change
### 48 - name change
```
[uint8_t name[namelen]]
```
49 - groupchat title change
### 49 - groupchat title change
```
[uint8_t title[titlelen]]
```
64 - chat message
### 64 - chat message
```
[uint8_t message[messagelen]]
```
65 - action (/me)
### 65 - action (/me)
```
[uint8_t message[messagelen]]
```

51
docs/apidsl.md
View File

@ -1,51 +0,0 @@
This folder contains the input file (``tox.in.h``) that has to be used to generate the ``tox.h`` api with: https://github.com/TokTok/apidsl
# Minimal requirements
There are some minimal requirements to contribute to ``tox.h``:
* unix environment
* ``astyle`` ``>=2.03``
* [``apidsl``](https://github.com/TokTok/apidsl) (you can use provided service with curl instead)
## Quick way
If you want to do it quickly and you don't have time for anything other than copypasting commands, you should have ``curl`` installed.
1. Make sure that you have ``curl`` and ``>=astyle-2.03`` installed
2. Modify [``tox.api.h``](/toxcore/tox.api.h)
3. Run command below ↓
Command to run from ``toxcore`` directory (quick way, involves using curl):
```bash
# For tox.h:
curl -X POST --data-binary @- https://apidsl.herokuapp.com/apidsl \
< toxcore/tox.api.h \
| astyle --options=other/astyle/astylerc \
> toxcore/tox.h
# For toxav.h:
curl -X POST --data-binary @- https://apidsl.herokuapp.com/apidsl \
< toxav/toxav.api.h \
| astyle --options=other/astyle/astylerc \
> toxav/toxav.h
```
You may want to make sure with ``git diff`` that changes made in ``tox.h`` reflect changes in ``tox.in.h``.
And you're done.
## Manually
If you prefer to have more control over what is happening, there are steps below:
1. Install [``apidsl``](https://github.com/TokTok/apidsl)
2. Install ``astyle``, version 2.03 or later.
3. Modify [``tox.api.h``](/toxcore/tox.api.h)
4. Use ``apidsl`` ``??``
5. Parse generated ``tox.h`` with astyle, minimal command for it would be:
```bash
astyle --options=other/astyle/astylerc toxcore/tox.h
```
**Always pass output from ``apidsl`` through astyle.**

129
docs/av_api.md
View File

@ -8,38 +8,36 @@
phone_t* initPhone(uint16_t _listen_port, uint16_t _send_port);
```
function initializes sample phone. _listen_port and _send_port are variables only meant
for local testing. You will not have to do anything regarding to that since
everything will be started within a messenger.
function initializes sample phone. `_listen_port` and `_send_port` are variables
only meant for local testing. You will not have to do anything regarding to that
since everything will be started within a messenger.
Phone requires one msi session and two rtp sessions ( one for audio and one for
video ).
Phone requires one msi session and two rtp sessions (one for audio and one for
video).
```
msi_session_t* msi_init_session( void* _core_handler, const uint8_t* _user_agent );
```
initializes msi session.
Params:
initializes msi session. Params:
```
void* _core_handler - pointer to an object handling networking,
const uint8_t* _user_agent - string describing phone client version.
```
Return value:
msi_session_t* - pointer to a newly created msi session handler.
Return value: `msi_session_t*` - pointer to a newly created msi session handler.
### msi_session_t reference:
### `msi_session_t` reference:
How to handle msi session:
Controlling is done via callbacks and action handlers.
First register callbacks for every state/action received and make sure
NOT TO PLACE SOMETHING LIKE LOOPS THAT TAKES A LOT OF TIME TO EXECUTE; every callback is being called
directly from event loop. You can find examples in phone.c.
How to handle msi session: Controlling is done via callbacks and action
handlers. First register callbacks for every state/action received and make sure
NOT TO PLACE SOMETHING LIKE LOOPS THAT TAKES A LOT OF TIME TO EXECUTE; every
callback is being called directly from event loop. You can find examples in
phone.c.
Register callbacks:
Register callbacks:
```
void msi_register_callback_call_started ( MCALLBACK );
void msi_register_callback_call_canceled ( MCALLBACK );
@ -55,10 +53,9 @@ void msi_register_callback_recv_error ( MCALLBACK );
void msi_register_callback_requ_timeout ( MCALLBACK );
```
MCALLBACK is defined as: void (*callback) (void* _arg)
msi_session_t* handler is being thrown as \_arg so you can use that and \_agent_handler to get to your own phone handler
directly from callback.
MCALLBACK is defined as: `void (*callback) (void* _arg)` `msi_session_t*`
handler is being thrown as `_arg` so you can use that and `_agent_handler` to
get to your own phone handler directly from callback.
Actions:
@ -66,80 +63,93 @@ Actions:
int msi_invite ( msi_session_t* _session, call_type _call_type, uint32_t _timeoutms );
```
Sends call invite. Before calling/sending invite msi_session_t::_friend_id is needed to be set or else
it will not work. _call_type is type of the call ( Audio/Video ) and _timeoutms is how long
will poll wait until request is terminated.
Sends call invite. Before calling/sending invite `msi_session_t::_friend_id` is
needed to be set or else it will not work. `_call_type` is type of the call (
Audio/Video ) and `_timeoutms` is how long will poll wait until request is
terminated.
```
int msi_hangup ( msi_session_t* _session );
```
Hangs up active call
```
int msi_answer ( msi_session_t* _session, call_type _call_type );
```
Answer incoming call. _call_type set's callee call type.
Answer incoming call. `_call_type` set's callee call type.
```
int msi_cancel ( msi_session_t* _session );
```
Cancel current request.
```
int msi_reject ( msi_session_t* _session );
```
Reject incoming call.
Reject incoming call.
### Now for rtp:
You will need 2 sessions; one for audio one for video.
You start them with:
You will need 2 sessions; one for audio one for video. You start them with:
```
rtp_session_t* rtp_init_session ( int _max_users, int _multi_session );
```
Params:
```
int _max_users - max users. -1 if undefined
int _multi_session - any positive number means uses multi session; -1 if not.
```
Return value:
```
rtp_session_t* - pointer to a newly created rtp session handler.
```
### How to handle rtp session:
Take a look at
```
void* phone_handle_media_transport_poll ( void* _hmtc_args_p ) in phone.c
```
on example. Basically what you do is just receive a message via:
```
struct rtp_msg_s* rtp_recv_msg ( rtp_session_t* _session );
```
and then you use payload within the rtp_msg_s struct. Don't forget to deallocate it with:
void rtp_free_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg );
and then you use payload within the `rtp_msg_s` struct. Don't forget to
deallocate it with:
`void rtp_free_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg );`
Receiving should be thread safe so don't worry about that.
When you capture and encode a payload you want to send it ( obviously ).
first create a new message with:
```
struct rtp_msg_s* rtp_msg_new ( rtp_session_t* _session, const uint8_t* _data, uint32_t _length );
```
and then send it with:
```
int rtp_send_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg, void* _core_handler );
```
_core_handler is the same network handler as in msi_session_s struct.
`_core_handler` is the same network handler as in `msi_session_s` struct.
## A/V initialization:
```
int init_receive_audio(codec_state *cs);
int init_receive_video(codec_state *cs);
@ -156,39 +166,62 @@ In the future, VP8 should be used directly and ffmpeg should be dropped from the
The variable bps is the required bitrate in bits per second.
```
### A/V encoding/decoding:
```
void *encode_video_thread(void *arg);
```
Spawns the video encoding thread. The argument should hold a pointer to a codec_state.
This function should only be called if video encoding is supported (when init_send_video returns 1).
Each video frame gets encoded into a packet, which is sent via RTP. Every 60 frames a new bidirectional interframe is encoded.
Spawns the video encoding thread. The argument should hold a pointer to a
`codec_state`. This function should only be called if video encoding is
supported (when `init_send_video` returns 1). Each video frame gets encoded into
a packet, which is sent via RTP. Every 60 frames a new bidirectional interframe
is encoded.
```
void *encode_audio_thread(void *arg);
```
Spawns the audio encoding thread. The argument should hold a pointer to a codec_state.
This function should only be called if audio encoding is supported (when init_send_audio returns 1).
Audio frames are read from the selected audio capture device during initialisation. This audio capturing can be rerouted to a different device on the fly.
Each audio frame is encoded into a packet, and sent via RTP. All audio frames have the same amount of samples, which is defined in AV_codec.h.
Spawns the audio encoding thread. The argument should hold a pointer to a
`codec_state`. This function should only be called if audio encoding is
supported (when `init_send_audio` returns 1). Audio frames are read from the
selected audio capture device during initialisation. This audio capturing can be
rerouted to a different device on the fly. Each audio frame is encoded into a
packet, and sent via RTP. All audio frames have the same amount of samples,
which is defined in `AV_codec.h`.
```
int video_decoder_refresh(codec_state *cs, int width, int height);
```
Sets the SDL window dimensions and creates a pixel buffer with the requested size. It also creates a scaling context, which will be used to convert the input image format to YUV420P.
Sets the SDL window dimensions and creates a pixel buffer with the requested
size. It also creates a scaling context, which will be used to convert the input
image format to YUV420P.
```
void *decode_video_thread(void *arg);
```
Spawns a video decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised video decoder.
This function reads video packets and feeds them to the video decoder. If the video frame's resolution has changed, video_decoder_refresh() is called. Afterwards, the frame is displayed on the SDL window.
Spawns a video decoding thread. The argument should hold a pointer to a
`codec_state`. The `codec_state` is assumed to contain a successfully
initialised video decoder. This function reads video packets and feeds them to
the video decoder. If the video frame's resolution has changed,
`video_decoder_refresh()` is called. Afterwards, the frame is displayed on the
SDL window.
```
void *decode_audio_thread(void *arg);
```
Spawns an audio decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised audio decoder.
All received audio packets are pushed into a jitter buffer and are reordered. If there is a missing packet, or a packet has arrived too late, it is treated as a lost packet and the audio decoder is informed of the packet loss. The audio decoder will then try to reconstruct the lost packet, based on information from previous packets.
Audio is played on the default OpenAL output device.
Spawns an audio decoding thread. The argument should hold a pointer to a
`codec_state`. The `codec_state` is assumed to contain a successfully
initialised audio decoder. All received audio packets are pushed into a jitter
buffer and are reordered. If there is a missing packet, or a packet has arrived
too late, it is treated as a lost packet and the audio decoder is informed of
the packet loss. The audio decoder will then try to reconstruct the lost packet,
based on information from previous packets. Audio is played on the default
OpenAL output device.
If you have any more qustions/bug reports/feature request contact the following users on the irc channel #tox-dev on irc.freenode.net:
For RTP and MSI: mannol
If you have any more qustions/bug reports/feature request contact the following
users on the irc channel #tox-dev on irc.freenode.net: For RTP and MSI: mannol
For audio and video: Martijnvdc

111
docs/minpgc.md
View File

@ -3,29 +3,29 @@
This document describes the "minpgc" simple persistent conferences
implementation of PR #1069.
Many of the ideas derive from isotoxin's persistent conferences
implementation, PR #826.
Many of the ideas derive from isotoxin's persistent conferences implementation,
PR #826.
## Specification of changes from pre-existing conference specification
We add one new packet type:
Rejoin Conference packet
| Length | Contents |
|:-------|:--------------------------------|
| `1` | `uint8_t` (0x64) |
| `33` | Group chat identifier |
| Length | Contents |
| :----- | :-------------------- |
| `1` | `uint8_t` (0x64) |
| `33` | Group chat identifier |
A peer times out from a group if it has been inactive for 60s. When a peer times
out, we flag it as _frozen_. Frozen peers are disregarded for all purposes
except those discussed below - in particular no packets are sent to them except
as described below, they are omitted from the peer lists sent to the client or
in a Peer Response packet, and they are not considered when determining closest
peers for establishing direct connections.
A peer times out from a group if it has been inactive for 60s. When a peer
times out, we flag it as _frozen_. Frozen peers are disregarded for all
purposes except those discussed below - in particular no packets are sent to
them except as described below, they are omitted from the peer lists sent to
the client or in a Peer Response packet, and they are not considered when
determining closest peers for establishing direct connections.
A peer is considered to be active if we receive a group message or Rejoin
packet from it, or a New Peer message for it.
A peer is considered to be active if we receive a group message or Rejoin packet
from it, or a New Peer message for it.
If a frozen peer is seen to be active, we remove its 'frozen' flag and send a
Name group message. (We can hold off on sending this message until the next
@ -40,77 +40,83 @@ message. (This is current behaviour; it's mentioned here because it's important
and not currently mentioned in the spec.)
If we receive a Rejoin packet from a peer we update its DHT pubkey, add a
temporary groupchat connection for the peer, and, once the connection is
online, send out a New Peer message announcing the peer, and a Name message.
temporary groupchat connection for the peer, and, once the connection is online,
send out a New Peer message announcing the peer, and a Name message.
Whenever we make a new friend connection, we check if the public key is that
of any frozen peer. If so, we send it a Rejoin packet, add a temporary
groupchat connection for it, and, once the connection is online, send the
peer a Peer Query packet.
Whenever we make a new friend connection, we check if the public key is that of
any frozen peer. If so, we send it a Rejoin packet, add a temporary groupchat
connection for it, and, once the connection is online, send the peer a Peer
Query packet.
We do the same with a peer when we are setting it as frozen if we have a
friend connection to it.
We do the same with a peer when we are setting it as frozen if we have a friend
connection to it.
The temporary groupchat connections established in sending and handling Rejoin
packets are not immediately operational (because group numbers are not known);
rather, an Online packet is sent when we handle a Rejoin packet.
When a connection is set as online as a result of an Online packet, we ping
the group.
When a connection is set as online as a result of an Online packet, we ping the
group.
When processing the reply to a Peer Query, we update the DHT pubkey of an
existing peer if and only if it is frozen or has not had its DHT pubkey
updated since it last stopped being frozen.
existing peer if and only if it is frozen or has not had its DHT pubkey updated
since it last stopped being frozen.
When we receive a Title Response packet, we set the title if it has never been
set or if at some point since it was last set, there were no unfrozen peers
(except us).
## Discussion
### Overview
The intention is to recover seamlessly from splits in the group, the most
common form of which is a single peer temporarily losing all connectivity.
To see how this works, first note that groups (even before the changes
discussed here) have the property that for a group to be connected in the
sense that any peer will receive the messages of any other peer and have them
in their peerlist, it is necessary and sufficient that there is a path of
direct group connections between any two peers.
### Overview
The intention is to recover seamlessly from splits in the group, the most common
form of which is a single peer temporarily losing all connectivity.
To see how this works, first note that groups (even before the changes discussed
here) have the property that for a group to be connected in the sense that any
peer will receive the messages of any other peer and have them in their
peerlist, it is necessary and sufficient that there is a path of direct group
connections between any two peers.
Now suppose the group is split into two connected components, with each member
of one component frozen according to the members of the other. Suppose there
are two peers, one in each component, which are using the above protocol, and
of one component frozen according to the members of the other. Suppose there are
two peers, one in each component, which are using the above protocol, and
suppose they establish a friend connection. Then each will rejoin the other,
forming a direct group connection. Hence the whole group will become connected
(even if all other peers are using the unmodified protocol).
The Peer Query packet sent on rejoining hastens this process.
Peers who leave the group during a split will not be deleted by all peers
after the merge - but they will be set as frozen due to ping timeouts, which
is sufficient.
Peers who leave the group during a split will not be deleted by all peers after
the merge - but they will be set as frozen due to ping timeouts, which is
sufficient.
### Titles
If we have a split into components each containing multiple peers, and the
title is changed in one component, then peers will continue to disagree on the
title after the split. Short of a complicated voting system, this seems the
only reasonable behaviour.
If we have a split into components each containing multiple peers, and the title
is changed in one component, then peers will continue to disagree on the title
after the split. Short of a complicated voting system, this seems the only
reasonable behaviour.
### Implementation notes
Although I've described the logic in terms of an 'frozen' flag, it might
actually make more sense in the implementation to have a separate list for
Although I've described the logic in terms of an 'frozen' flag, it might
actually make more sense in the implementation to have a separate list for
frozen peers.
## Saving
Saving is implemented by simply saving all live groups with their group numbers
and full peer info for all peers. On reload, all peers are set as frozen.
Clients needs to support this by understanding that groups may exist on
start-up. Clients should call `tox_conference_get_chatlist` to obtain them. A
group which is deleted (with `tox_conference_delete`) is removed permanently
and will not be saved.
group which is deleted (with `tox_conference_delete`) is removed permanently and
will not be saved.
## Limitations
If a peer disconnects from the group for a period short enough that group
timeouts do not occur, and a name change occurs during this period, then the
name change will never be propagated.
@ -120,9 +126,8 @@ requesting missed group messages. But this is considered out of scope of this
PR.
If a peer changes its DHT pubkey, the change might not be properly propagated
under various circumstances - in particular, if connections do not go down
long enough for the peer to become frozen.
under various circumstances - in particular, if connections do not go down long
enough for the peer to become frozen.
One way to deal with this would be to add a group message announcing the
sending peer's current DHT pubkey, and treat it analogously to the Name
message.
One way to deal with this would be to add a group message announcing the sending
peer's current DHT pubkey, and treat it analogously to the Name message.