Green-Sky/tomato
1
0
Fork 1
Code Issues Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

Merge commit '227425b90e9a671118026689dd30967e127a1090' as 'external/toxcore/c-toxcore'

Browse Source
This commit is contained in:
Green Sky
2023-07-25 11:53:09 +02:00
parent 310a099f14 227425b90e
commit 5568b70d31
467 changed files with 116591 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2638
external/toxcore/c-toxcore/docs/Doxyfile vendored Normal file
View File

File diff suppressed because it is too large Load Diff

78
external/toxcore/c-toxcore/docs/Group-Chats.md vendored Normal file
View File

@ -0,0 +1,78 @@
Group chats.
Note: we assume everyone in the chat trusts each other.
These group chats work by temporarily adding the 4 "closest" people defined by a distance function
in group.c in order to form a circle of connected peers. These peers then relay messages to each other.
A friend invites another friend to a group chat by sending them an invite packet. The friend either ignores
the invite or responds with a response packet if he wants to join the chat. The friend invite contains the type
of groupchat (text only, A/V) the friend is being invited to.
TODO(irungentoo): write more of this.
## Protocol
Invite packets:
Invite packet:
[uint8_t id 96][uint8_t id 0][uint16_t group chat number][33 bytes group chat identifier[1 byte type][32 bytes id]]
Response packet
[uint8_t id 96][uint8_t id 1][uint16_t group chat number(local)][uint16_t group chat number to join][33 bytes group chat identifier[1 byte type][32 bytes id]]
Peer online packet:
[uint8_t id 97][uint16_t group chat number (local)][33 bytes group chat identifier[1 byte type][32 bytes id]]
Peer leave packet:
[uint8_t id 98][uint16_t group chat number][uint8_t id 1]
Peer query packet:
[uint8_t id 98][uint16_t group chat number][uint8_t id 8]
Peer response packet:
[uint8_t id 98][uint16_t group chat number][uint8_t id 9][Repeated times number of peers: [uint16_t peer num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key][uint8_t name length][name]]
Title response packet:
[uint8_t id 98][uint16_t group chat number][uint8_t id 10][title]
Message packets:
[uint8_t id 99][uint16_t group chat number][uint16_t peer number][uint32_t message number][uint8_t with a value representing id of message][data]
Lossy Message packets:
[uint8_t id 199][uint16_t group chat number][uint16_t peer number][uint16_t message number][uint8_t with a value representing id of message][data]
Group chat types:
0: text
1: AV
Note: the message number is increased by 1 for each sent message.
message ids:
0 - ping
sent every ~60 seconds by every peer.
No data.
16 - new_peer
Tell everyone about a new peer in the chat.
[uint16_t peer_num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key]
17 - kill_peer
[uint16_t peer_num]
48 - name change
[uint8_t name[namelen]]
49 - groupchat title change
[uint8_t title[titlelen]]
64 - chat message
[uint8_t message[messagelen]]
65 - action (/me)
[uint8_t message[messagelen]]

60
external/toxcore/c-toxcore/docs/Hardening.txt vendored Normal file
View File

@ -0,0 +1,60 @@
Currently an attacker with sufficient resources could launch a large scale
denial of service type attack by flooding the Tox network with a bunch of nodes
that do not act like real nodes to prevent people from finding each other.
Due to the design of Tox, this is the worst thing an attacker can do to disrupt
the network.
This solution's goal is to make these denial of service attack very very hard
to accomplish.
For the network to work every Tox node must:
1. Respond to ping requests.
2. Respond to get node requests with the ids of nodes closest to a queried id
(It is assumed each nodes know at least the 32 nodes closest to them.)
3. Properly send crypto request packets to their intended destination.
Currently the only thing a node needs to do to be part of the network is
respond correctly to ping requests.
The only people we really trust on the network are the nodes in our friends
list.
The behavior of each Tox node is easily predictable. This means that it possible
for Tox nodes to test the nodes that they are connected to to see if they
behave like normal Tox nodes and only send nodes that are confirmed to behave
like real Tox nodes as part of send node replies when other nodes query them.
If correctly done, this means that to poison the network an attacker can only
infiltrate the network if his "fake" nodes behave exactly like real nodes
completely defeating the purpose of the attack. Of course nodes must be
rechecked regularly to defeat an attack where someone floods the network with
many good nodes then suddenly turns them all bad.
This also prevents someone from accidentally killing the tox network with a bad
implementation of the protocol.
Implementation ideas (In Progress):
1. Use our friends to check if the nodes in our close list are good.
EX: If our friend queries a node close to us and it correctly returns our
ip/port and then sends a crypto request packet to it and it routes it correctly
to us then it is good.
Problems with this: People don't always have at least one online friend.
2. Pick random nodes (add ourselves some random (fake) friends to increase the
pool of available nodes) and make then send requests to other nodes, the
response is then relayed back to us and compared to how the node should have
behaved. If the node is found to be behaving correctly, it is set as trusted.
Only trusted nodes are sent in send node packets, that is unless the exact node
being queried for in the getnode packet is present, it will be sent in the
sendnode packet even if it is not trusted.
The hypothesis is that if to be part of the network nodes have to behave
correctly it should prevent disruption from nodes that behave incorrectly.
(This idea is currently being implemented in the harden branch.)
...

30
external/toxcore/c-toxcore/docs/Hardening_docs.txt vendored Normal file
View File

@ -0,0 +1,30 @@
Hardening request packets are sent as crypto request packets (see crypto docs.)
NOTE: currently only get nodes requests are tested in the code which is why
there is only one test (more will be added soon.)
All hardening requests must contain exactly 768 bytes of data. (The data sent
must be padded with zeros if it is smaller than that.)
1. Get the information (IP_port, client_id) of the node we want to test.
2. Find a couple random nodes that is not that node (one for each test.)
3. Send crypto request packets to each of these random nodes with the data being:
[byte with value: 02 (get nodes test request)][struct Node_format (the node to
test.)][client_id(32 bytes) the id to query the node with.][padding]
4. The random node receives a packet.
-The packet is a get nodes test request:
send a get_node request to that node with the id to query in the request.
when a send_node response is received, send the following response to the
person who sent us the get nodes test request packet:
[byte with value: 03 (get nodes test response)][client_id(32 bytes):
the id of the tested node][The list of nodes it responded with in IPv6
Node format (struct Node_Format)]
PROTIP: (get node requests and response contain an encrypted part that you
can use to store information so that you don't
have to store in your memory where/if to send back the response from the
send node)
5. Receive the test responses.
-If the test(s) pass (the nodes behave in a satisfactory manner), make these
nodes have priority over those who don't pass the test(s).

160
external/toxcore/c-toxcore/docs/Prevent_Tracking.txt vendored Normal file
View File

@ -0,0 +1,160 @@
Current privacy issues with the Tox DHT:
1. It makes tracking people across different IPs very easy.
Solution: Have each new DHT use a temporary public/private key pair not related
to the long term public/private key pair.
2. Metadata on which key is friends to which can be collected (The hardening
makes this somewhat harder by introducing a bunch of random traffic but not
impossible.).
Solution: If no long term keys were used in the DHT it would solve this
problem. (possibly knowing which ip is connected to which is much less
precious.)
So, it seems all our privacy problems are solved if we can manage to make every
node in the DHT have a keypair that is not related to the long term keys and is
generated every session.
So, every node in the DHT now has a temporary keypair not related to their real
long term one.
But, how do people find themselves then? We have to add a way for people to
tell their friends what their DHT public key is. We also have to somehow make
it so people can send/receive friend requests. This has to be done without
non-friends being able to find out where a node is.
The solution: Onion routing + enable the storage of some small amount of data
on DHT nodes.
Alice and bob are friends. Before joining the DHT they generate temporary
session keypairs to be used for the DHT instead of their long term keys.
Bob finds a bunch of random nodes then picks 3 random working ones (A, B, C).
Bob gets the known working node with an id closest to his real one from his list (D)
Bob then creates an onion (the packet will go through A, B, C and will end up at D)
announce request packet with his real public key, ping_id as zeros and
searching for his real public key.
Bob will announce response packets and will recursively send onion announce request
packets to closer and closer nodes until he finds the ones closest to his real public key.
Once he has done this, he will send some onion announce request packets with the right
ping_id previously received from the node when he queried it to announce himself to the node.
The nodes he announces himself to keep the information to send onion packets to that node in
memory.
Alice meanwhile searches for the nodes closest to Bobs real id using a temporary keypair and
announce request packets. She does this until she finds nodes that respond with a ping_id of zero.
She sends data to route request packet with information telling Bob her temporary id in the DHT
(or a friend request if she is not friends with him).
Bob finds her by using her temporary id and they connect to each other.
NOTE: crypto_box is used for all the asymmetric encryption and crypto_secretbox is used for all
the symmetric. Also every DHT node have a random symmetric key which they use to encrypt the stuff
in normal get node request that is used to encrypt stuff in the following.
Onion packet (request):
initial (sent from us to node A):
[uint8_t packet id (128)][nonce]
[our temp DHT public key]encrypted with our temp DHT private key and the pub key of Node A and the nonce:[
[IP_Port of node B][a random public key]encrypted with the random private key and the pub key of Node B and the nonce:[
[IP_Port of node C][a random public key]encrypted with the random private key and the pub key of Node C and the nonce:[
[IP_Port of node D][data to send to Node D]]]]
(sent from node A to node B):
[uint8_t packet id (129)][nonce]
[a random public key]encrypted with the random private key and the pub key of Node B and the nonce:[
[IP_Port of node C][a random public key]encrypted with the random private key and the pub key of Node C and the nonce:[
[IP_Port of node D][data to send to Node D]]][nonce (for the following symmetric encryption)]encrypted with temp symmetric key of Node A: [IP_Port (of us)]
(sent from node B to node C):
[uint8_t packet id (130)][nonce]
[a random public key]encrypted with the random private key and the pub key of Node C and the nonce:[
[IP_Port of node D][data to send to Node D]][nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node A: [IP_Port (of us)]]
(sent from node C to node D):
[data to send to Node D][nonce (for the following symmetric encryption)]encrypted with temp symmetric key of Node C:
[IP_Port (of Node B)[nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node A: [IP_Port (of us)]]]
Data sent to Node D:
announce request packet:
[uint8_t packet id (131)][nonce][our real long term public key or a temporary one (see next)]
encrypted (with our real long term private key if we want to announce ourselves, a temporary one if we are searching for friends) and the pub key of Node D and the nonce:
[[(32 bytes) ping_id][client id we are searching for][public key that we want those sending back data packets to use.][data to send back in response(fixed size)]]
(if the ping id is zero, respond with a announce response packet)
(If the ping id matches the one the node sent in the announce response and the public key matches the one being searched for,
add the part used to send data to our list (if the list is full make it replace the furthest entry))
data to route request packet:
[uint8_t packet id (133)][public key of destination node][nonce][temporary just generated public key]
encrypted with that temporary private key and the nonce and the public key from the announce response packet of the destination node:[data]
(if Node D contains the ret data for the node, it sends the stuff in this packet as a data to route response packet to the right node)
The data in the previous packet is in format: [real public key of sender]
encrypted with real private key of the sender, the nonce in the data packet and
the real public key of the receiver:[[uint8_t id][data (optional)]]
Data sent to us:
announce response packet:
[uint8_t packet id (132)][data to send back in response(fixed size)][nonce]
encrypted with the DHT private key of Node D, the public key in the request and the nonce:[[uint8_t is_stored]
[(32 bytes) ping_id if is_stored is 0 or 2, public key that must be used to send data packets if is_stored is 1][Node_Format * (maximum of 8)]]
(if the is_stored is not 0, it means the information to reach the client id we are searching for is stored on this node)
is_stored is 2 as a response to a peer trying to announce himself to tell the
peer that he is currently announced successfully.
data to route response packet:
[uint8_t packet id (134)][nonce][temporary just generated public key]
encrypted with that temporary private key, the nonce and the public key from the announce response packet of the destination node:[data]
Onion packet (response):
initial (sent from node D to node C):
[uint8_t packet id (140)][nonce (for the following symmetric encryption)]encrypted with temp symmetric key of Node C:
[IP_Port (of Node B)[nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node A: [IP_Port (of us)]]][data to send back]
(sent from node C to node B):
[uint8_t packet id (141)][nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node B:[IP_Port (of Node A)[nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node A: [IP_Port (of us)]][data to send back]
(sent from node B to node A):
[uint8_t packet id (142)][nonce (for the following symmetric encryption)]
encrypted with temp symmetric key of Node A: [IP_Port (of us)][data to send back]
(sent from node A to us):
[data to send back]
Data packets:
To tell our friend what our DHT public key is so that he can connect to us we send a data packet
with id 156 and the data being:[uint64_t (in network byte order) no_replay, the packet will only be
accepted if this number is bigger than the last one received] [our dht public key][Node_Format * (
maximum of 8) nodes closest to us so that the friend can find us faster]

154
external/toxcore/c-toxcore/docs/TCP_Network.txt vendored Normal file
View File

@ -0,0 +1,154 @@
It has come to our attention that to achieve decent market penetration Tox
must work behind ALL internet connections, may they be behind enterprise NATs
or any other bad network conditions.
The people who have issues with the UDP direct connection approach seem to be a
small minority though it is hard to estimate how many.
This means that routing their packets using good nodes on the network will
probably not take a huge toll on the network and will assure that people
can use Tox regardless of the quality of their internet connection.
How it's going to work:
1. Alice, a Tox client on a TCP only network generates a temporary public key
and connects to a bootstrap node.
2. Using the bootstrap node she finds and connects to a couple (exact number
to be determined later) number of random nodes that have TCP relay support.
3. She uses the onion through the TCP relay connections to send friend requests
or tell online friends which TCP nodes she is connected to and her temporary
public key.
4. Bob receives an onion packet from Alice telling him which nodes she is
connected to. Bob connects to these nodes and establishes a routed connection
with Alice using that temporary public key.
5. That connection is used by both to transmit encrypted Messenger and A/V
packets.
6. If one of the nodes shuts down while it is currently routing traffic, Alice
and bob just switch to one of the other nodes they are both connected to.
Detailed implementation details:
There are two distinct parts for TCP relays, the client part and the server
part.
The server acts as the actual relay. Servers must have fully forwarded TCP
ports (NAT-PMP and uPNP can help here). The first port the server will try
binding to is 443 followed by port 3389 and possibly some others. Onion packets
can be sent/received through the TCP servers.
Server:
The public/private key pair the TCP server uses is the same one he uses for the
DHT.
all crypto for communication with the server uses the crypto_box() function of
NaCl.
TCP doesn't have packets so what we will refer to as packets are sent this way:
[[uint16_t (length of data)][data]]
So if you would inspect the TCP stream you would see:
[[uint16_t (length of data)][data]][[uint16_t (length of
data)][data]][[uint16_t (length of data)][data]]
Note that both handshake packets don't have this format (the length for them is
always the same so we don't need to specify it.)
When the client connects to the server, he sends this packet:
[public key of client (32 bytes)][nonce for the encrypted data [24
bytes]][encrypted with the private key of the client and public key of the
server and the nonce:[public key (32 bytes) and][base nonce we want the server
to use to encrypt the packets sent to us (24 bytes)]]
The server responds with:
[nonce for the encrypted data [24 bytes]][encrypted with the public key of the
client and private key of the server and the nonce:[public key (32 bytes)
and][base nonce we want the client to use to encrypt the packets sent to us (24
bytes)]]
All packets to the server are end to end encrypted with the information
received
(and sent) in the handshake.
(first packet is encrypted with the base nonce the private key for which the
client sent the server the public key and the public key we sent to the client,
the next with base nonce + 1...)
The connection is set to an unconfirmed state until a packet is received and
decrypted correctly using the information in the handshake.
each packet sent to/from the server has an id (the first byte of the plain text
data of the packet.)
ids 0 to 15 are reserved for special packets, ids 16 to 255 are used to denote
who we want the data to be routed to/who the packet is from.
special ids and packets:
0 - Routing request.
[uint8_t id (0)][public key (32 bytes)]
1 - Routing request response.
[uint8_t id (1)][uint8_t (rpid) 0 if refused, packet id if accepted][public key
(32 bytes)]
2 - Connect notification:
[uint8_t id (2)][uint8_t (packet id of connection that got connected)]
3 - Disconnect notification:
[uint8_t id (3)][uint8_t (packet id of connection that got disconnected)]
4 - ping packet
[uint8_t id (4)][uint64_t ping_id (0 is invalid)]
5 - ping response (pong)
[uint8_t id (5)][uint64_t ping_id (0 is invalid)]
6 - OOB send
[uint8_t id (6)][destination public key (32 bytes)][data]
7 - OOB recv
[uint8_t id (7)][senders public key (32 bytes)][data]
8 - onion packet (same format as initial onion packet (See: Prevent
tracking.txt) but packet id is 8 instead of 128)
9 - onion packet response (same format as onion packet with id 142 but id is 9
instead.)
The rest of the special ids are reserved for possible future usage.
If the server receives a routing request he stores server side that the client
wants to connect to the person with that public key and sends back a Routing
request response with the rpid along with the public key sent in the request.
If for some reason the server must refuse the routing request (too many) he
sends the response with a rpid of 0.
If the person who the client wants to connect to is also online and wants to
connect to the client a connect notification is sent to both with the
appropriate packet id.
If either one disconnects, a disconnect notification is sent to the other with
appropriate packet id.
If a client sends a disconnect notification, the entry on the server for that
routed connection is cleared and a disconnect notification is sent to the peer
(if he was online)
If the server receives an onion packet he handles it the same as he would if it
was one received normally via UDP, he must also assure himself that any
responses must be sent to the proper client.
Ping responses must have the same ping_id as the request.
If the server receives a ping packet he must respond with a ping response.
The server will send a ping packet to clients every 30 seconds, they have 30
seconds to respond, if they don't the connection is deleted.
OOB send packets will be sent to the peer connected to the TCP server with the
destination public key as a OOB recv packet. The client sending this packet has
no way of knowing if the packet reached its destination.
Client:
Implementation details coming soon.

120
external/toxcore/c-toxcore/docs/Tox_middle_level_network_protocol.txt vendored Normal file
View File

@ -0,0 +1,120 @@
The TCP client and TCP server part are in a state that can be considered
feature complete. Why doesn't Tox support TCP yet even if those parts are
complete?
The answer is that a way to ensure a smooth switchover between the TCP and UDP
needs to be added. If Tox first connects to the other user using TCP but then,
due to pure chance, manages to connect using the faster direct UDP connection,
Tox must switch seamlessly from the TCP to the UDP connection without there
being any data loss or the other user going offline and then back online. The
transition must be seamless whatever both connected users are doing - be it
transferring files or simply chatting together.
Possible evil/bad or simply TCP relays going offline must not impact the
connection between both clients.
Typically, Tox will use more than one TCP relay to connect to other peers for
maximum connection stability, which means there must be a way for Tox to take
advantage of multiple relays in a way that the user will never be aware of, if one
of them goes offline/tries to slow down the connection/decides to corrupt
packets/etc.
To accomplish this, Tox needs something between the low level protocol (TCP) and
high level Tox messaging protocol; hence the name middle level.
The plan is to move some functionality from lossless_UDP to a higher level:
more specifically, the functionality for detecting which packets a peer is
missing, and the ability to request and send them again. Lossless UDP uses plain
text packets to request missing packets from the other peer, while Tox is
currently designed to kill the connection if any packet tampering is detected.
This works very well when connecting directly with someone because if the
attacker can modify packets, it means he can kill your connection anyway. With
TCP relays, however, that is not the case. As such the packets used to request
missing packets must be encrypted. If it is detected that a packet has been
tampered, the connection must stay intact while the evil relay must be
disconnected from and replaced with a good relay; the behavior must be the same
as if the relay had just suddenly gone offline. Of course, something to protect
from evil "friends" framing relays must also be implemented.
Detailed implementation details:
cookie request packet:
[uint8_t 24][Sender's DHT Public key (32 bytes)][Random nonce (24
bytes)][Encrypted message containing: [Sender's real public key (32
bytes)][padding (32 bytes)][uint64_t number (must be sent
back untouched in cookie response)]]
Encrypted message is encrypted with sender's DHT private key, receiver's DHT
public key and the nonce.
cookie response packet:
[uint8_t 25][Random nonce (24 bytes)][Encrypted message containing:
[Cookie][uint64_t number (that was sent in the request)]]
Encrypted message is encrypted with sender's DHT private key, receiver's DHT
public key and the nonce.
The Cookie should be basically:
[nonce][encrypted data:[uint64_t time][Sender's real public key (32
bytes)][Sender's DHT public key (32 bytes)]]
Handshake packet:
[uint8_t 26][Cookie][nonce][Encrypted message containing: [random 24 bytes base
nonce][session public key of the peer (32 bytes)][sha512 hash of the entire
Cookie sitting outside the encrypted part][Other Cookie (used by the other to
respond to the handshake packet)]]
The handshake packet is encrypted using the real private key of the sender, the
real public key of the receiver and the nonce.
Alice wants to connect to Bob:
Alice sends a cookie request packet to Bob and gets a cookie response back.
Alice then generates a nonce and a temporary public/private keypair.
Alice then takes that nonce and just generated private key, the obtained
cookie, creates a new cookie and puts them in a handshake packet, which she
sends to Bob.
Bob gets the handshake packet, accepts the connection request, then generates a
nonce and a temporary public/private keypair and sends a handshake packet back
with this just generated information and with the cookie field being the Other
Cookie contained in the received handshake.
Both then use these temporary keys to generate the session key, with which every
data packet sent and received will be encrypted and decrypted. The nonce sent
in the handshake will be used to encrypt the first data packet sent, the nonce
+ 1 for the second, the nonce + 2 for the third, and so on.
Data packets:
[uint8_t 27][uint16_t (in network byte order) the last 2 bytes of the nonce
used to encrypt this][encrypted with the session key and a nonce:[plain data]]
Plain data in the data packets:
[uint32_t our recvbuffers buffer_start, (highest packet number handled +
1)][uint32_t packet number if lossless, our sendbuffer buffer_end if
lossy][data]
data ids:
0: padding (skipped until we hit a non zero (data id) byte)
1: packet request packet (lossy packet)
2: connection kill packet (lossy packet) (tells the other that the connection is over)
...
16+: reserved for Messenger usage (lossless packets).
192+: reserved for Messenger usage (lossy packets).
255: reserved for Messenger usage (lossless packet)
packet request packet: [uint8_t (1)][uint8_t num][uint8_t num][uint8_t
num]...[uint8_t num]
The list of nums are a list of packet numbers the other is requesting.
In order to get the real packet numbers from this list, take the recvbuffers buffer_start
from the packet, subtract 1 from it and put it in packet_num, then start from the
beginning of the num list: if num is zero, add 255 to packet_num, then do the
next num. If num isn't zero, add its value to packet_num, note that the other
has requested we send this packet again to them, then continue to the next num in
the list.

51
external/toxcore/c-toxcore/docs/apidsl.md vendored Normal file
View File

@ -0,0 +1,51 @@
This folder contains the input file (``tox.in.h``) that has to be used to generate the ``tox.h`` api with: https://github.com/TokTok/apidsl
# Minimal requirements
There are some minimal requirements to contribute to ``tox.h``:
* unix environment
* ``astyle`` ``>=2.03``
* [``apidsl``](https://github.com/TokTok/apidsl) (you can use provided service with curl instead)
## Quick way
If you want to do it quickly and you don't have time for anything other than copypasting commands, you should have ``curl`` installed.
1. Make sure that you have ``curl`` and ``>=astyle-2.03`` installed
2. Modify [``tox.api.h``](/toxcore/tox.api.h)
3. Run command below ↓
Command to run from ``toxcore`` directory (quick way, involves using curl):
```bash
# For tox.h:
curl -X POST --data-binary @- https://apidsl.herokuapp.com/apidsl \
< toxcore/tox.api.h \
| astyle --options=other/astyle/astylerc \
> toxcore/tox.h
# For toxav.h:
curl -X POST --data-binary @- https://apidsl.herokuapp.com/apidsl \
< toxav/toxav.api.h \
| astyle --options=other/astyle/astylerc \
> toxav/toxav.h
```
You may want to make sure with ``git diff`` that changes made in ``tox.h`` reflect changes in ``tox.in.h``.
And you're done.
## Manually
If you prefer to have more control over what is happening, there are steps below:
1. Install [``apidsl``](https://github.com/TokTok/apidsl)
2. Install ``astyle``, version 2.03 or later.
3. Modify [``tox.api.h``](/toxcore/tox.api.h)
4. Use ``apidsl`` ``??``
5. Parse generated ``tox.h`` with astyle, minimal command for it would be:
```bash
astyle --options=other/astyle/astylerc toxcore/tox.h
```
**Always pass output from ``apidsl`` through astyle.**

194
external/toxcore/c-toxcore/docs/av_api.md vendored Normal file
View File

@ -0,0 +1,194 @@
# A/V API reference
## Take toxmsi/phone.c as a reference
### Initialization:
```
phone_t* initPhone(uint16_t _listen_port, uint16_t _send_port);
```
function initializes sample phone. _listen_port and _send_port are variables only meant
for local testing. You will not have to do anything regarding to that since
everything will be started within a messenger.
Phone requires one msi session and two rtp sessions ( one for audio and one for
video ).
```
msi_session_t* msi_init_session( void* _core_handler, const uint8_t* _user_agent );
```
initializes msi session.
Params:
```
void* _core_handler - pointer to an object handling networking,
const uint8_t* _user_agent - string describing phone client version.
```
Return value:
msi_session_t* - pointer to a newly created msi session handler.
### msi_session_t reference:
How to handle msi session:
Controlling is done via callbacks and action handlers.
First register callbacks for every state/action received and make sure
NOT TO PLACE SOMETHING LIKE LOOPS THAT TAKES A LOT OF TIME TO EXECUTE; every callback is being called
directly from event loop. You can find examples in phone.c.
Register callbacks:
```
void msi_register_callback_call_started ( MCALLBACK );
void msi_register_callback_call_canceled ( MCALLBACK );
void msi_register_callback_call_rejected ( MCALLBACK );
void msi_register_callback_call_ended ( MCALLBACK );
void msi_register_callback_recv_invite ( MCALLBACK );
void msi_register_callback_recv_ringing ( MCALLBACK );
void msi_register_callback_recv_starting ( MCALLBACK );
void msi_register_callback_recv_ending ( MCALLBACK );
void msi_register_callback_recv_error ( MCALLBACK );
void msi_register_callback_requ_timeout ( MCALLBACK );
```
MCALLBACK is defined as: void (*callback) (void* _arg)
msi_session_t* handler is being thrown as \_arg so you can use that and \_agent_handler to get to your own phone handler
directly from callback.
Actions:
```
int msi_invite ( msi_session_t* _session, call_type _call_type, uint32_t _timeoutms );
```
Sends call invite. Before calling/sending invite msi_session_t::_friend_id is needed to be set or else
it will not work. _call_type is type of the call ( Audio/Video ) and _timeoutms is how long
will poll wait until request is terminated.
```
int msi_hangup ( msi_session_t* _session );
```
Hangs up active call
```
int msi_answer ( msi_session_t* _session, call_type _call_type );
```
Answer incoming call. _call_type set's callee call type.
```
int msi_cancel ( msi_session_t* _session );
```
Cancel current request.
```
int msi_reject ( msi_session_t* _session );
```
Reject incoming call.
### Now for rtp:
You will need 2 sessions; one for audio one for video.
You start them with:
```
rtp_session_t* rtp_init_session ( int _max_users, int _multi_session );
```
Params:
```
int _max_users - max users. -1 if undefined
int _multi_session - any positive number means uses multi session; -1 if not.
```
Return value:
```
rtp_session_t* - pointer to a newly created rtp session handler.
```
### How to handle rtp session:
Take a look at
```
void* phone_handle_media_transport_poll ( void* _hmtc_args_p ) in phone.c
```
on example. Basically what you do is just receive a message via:
```
struct rtp_msg_s* rtp_recv_msg ( rtp_session_t* _session );
```
and then you use payload within the rtp_msg_s struct. Don't forget to deallocate it with:
void rtp_free_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg );
Receiving should be thread safe so don't worry about that.
When you capture and encode a payload you want to send it ( obviously ).
first create a new message with:
```
struct rtp_msg_s* rtp_msg_new ( rtp_session_t* _session, const uint8_t* _data, uint32_t _length );
```
and then send it with:
```
int rtp_send_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg, void* _core_handler );
```
_core_handler is the same network handler as in msi_session_s struct.
## A/V initialization:
```
int init_receive_audio(codec_state *cs);
int init_receive_video(codec_state *cs);
Initialises the A/V decoders. On failure it will print the reason and return 0. On success it will return 1.
int init_send_audio(codec_state *cs);
int init_send_video(codec_state *cs);
Initialises the A/V encoders. On failure it will print the reason and return 0. On success it will return 1.
init_send_audio will also let the user select an input device. init_send_video will determine the webcam's output codec and initialise the appropriate decoder.
int video_encoder_refresh(codec_state *cs, int bps);
Reinitialises the video encoder with a new bitrate. ffmpeg does not expose the needed VP8 feature to change the bitrate on the fly, so this serves as a workaround.
In the future, VP8 should be used directly and ffmpeg should be dropped from the dependencies.
The variable bps is the required bitrate in bits per second.
```
### A/V encoding/decoding:
```
void *encode_video_thread(void *arg);
```
Spawns the video encoding thread. The argument should hold a pointer to a codec_state.
This function should only be called if video encoding is supported (when init_send_video returns 1).
Each video frame gets encoded into a packet, which is sent via RTP. Every 60 frames a new bidirectional interframe is encoded.
```
void *encode_audio_thread(void *arg);
```
Spawns the audio encoding thread. The argument should hold a pointer to a codec_state.
This function should only be called if audio encoding is supported (when init_send_audio returns 1).
Audio frames are read from the selected audio capture device during initialisation. This audio capturing can be rerouted to a different device on the fly.
Each audio frame is encoded into a packet, and sent via RTP. All audio frames have the same amount of samples, which is defined in AV_codec.h.
```
int video_decoder_refresh(codec_state *cs, int width, int height);
```
Sets the SDL window dimensions and creates a pixel buffer with the requested size. It also creates a scaling context, which will be used to convert the input image format to YUV420P.
```
void *decode_video_thread(void *arg);
```
Spawns a video decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised video decoder.
This function reads video packets and feeds them to the video decoder. If the video frame's resolution has changed, video_decoder_refresh() is called. Afterwards, the frame is displayed on the SDL window.
```
void *decode_audio_thread(void *arg);
```
Spawns an audio decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised audio decoder.
All received audio packets are pushed into a jitter buffer and are reordered. If there is a missing packet, or a packet has arrived too late, it is treated as a lost packet and the audio decoder is informed of the packet loss. The audio decoder will then try to reconstruct the lost packet, based on information from previous packets.
Audio is played on the default OpenAL output device.
If you have any more qustions/bug reports/feature request contact the following users on the irc channel #tox-dev on irc.freenode.net:
For RTP and MSI: mannol
For audio and video: Martijnvdc

128
external/toxcore/c-toxcore/docs/minpgc.md vendored Normal file
View File

@ -0,0 +1,128 @@
# Persistent conferences
This document describes the "minpgc" simple persistent conferences
implementation of PR #1069.
Many of the ideas derive from isotoxin's persistent conferences
implementation, PR #826.
## Specification of changes from pre-existing conference specification
We add one new packet type:
Rejoin Conference packet
| Length | Contents |
|:-------|:--------------------------------|
| `1` | `uint8_t` (0x64) |
| `33` | Group chat identifier |
A peer times out from a group if it has been inactive for 60s. When a peer
times out, we flag it as _frozen_. Frozen peers are disregarded for all
purposes except those discussed below - in particular no packets are sent to
them except as described below, they are omitted from the peer lists sent to
the client or in a Peer Response packet, and they are not considered when
determining closest peers for establishing direct connections.
A peer is considered to be active if we receive a group message or Rejoin
packet from it, or a New Peer message for it.
If a frozen peer is seen to be active, we remove its 'frozen' flag and send a
Name group message. (We can hold off on sending this message until the next
`tox_iterate`, and only send one message if many frozen peers become active at
once).
If we receive a New Peer message for a peer, we update its DHT pubkey.
If we receive a group message originating from an unknown peer, we drop the
message but send a Peer Query packet back to the peer who directly sent us the
message. (This is current behaviour; it's mentioned here because it's important
and not currently mentioned in the spec.)
If we receive a Rejoin packet from a peer we update its DHT pubkey, add a
temporary groupchat connection for the peer, and, once the connection is
online, send out a New Peer message announcing the peer, and a Name message.
Whenever we make a new friend connection, we check if the public key is that
of any frozen peer. If so, we send it a Rejoin packet, add a temporary
groupchat connection for it, and, once the connection is online, send the
peer a Peer Query packet.
We do the same with a peer when we are setting it as frozen if we have a
friend connection to it.
The temporary groupchat connections established in sending and handling Rejoin
packets are not immediately operational (because group numbers are not known);
rather, an Online packet is sent when we handle a Rejoin packet.
When a connection is set as online as a result of an Online packet, we ping
the group.
When processing the reply to a Peer Query, we update the DHT pubkey of an
existing peer if and only if it is frozen or has not had its DHT pubkey
updated since it last stopped being frozen.
When we receive a Title Response packet, we set the title if it has never been
set or if at some point since it was last set, there were no unfrozen peers
(except us).
## Discussion
### Overview
The intention is to recover seamlessly from splits in the group, the most
common form of which is a single peer temporarily losing all connectivity.
To see how this works, first note that groups (even before the changes
discussed here) have the property that for a group to be connected in the
sense that any peer will receive the messages of any other peer and have them
in their peerlist, it is necessary and sufficient that there is a path of
direct group connections between any two peers.
Now suppose the group is split into two connected components, with each member
of one component frozen according to the members of the other. Suppose there
are two peers, one in each component, which are using the above protocol, and
suppose they establish a friend connection. Then each will rejoin the other,
forming a direct group connection. Hence the whole group will become connected
(even if all other peers are using the unmodified protocol).
The Peer Query packet sent on rejoining hastens this process.
Peers who leave the group during a split will not be deleted by all peers
after the merge - but they will be set as frozen due to ping timeouts, which
is sufficient.
### Titles
If we have a split into components each containing multiple peers, and the
title is changed in one component, then peers will continue to disagree on the
title after the split. Short of a complicated voting system, this seems the
only reasonable behaviour.
### Implementation notes
Although I've described the logic in terms of an 'frozen' flag, it might
actually make more sense in the implementation to have a separate list for
frozen peers.
## Saving
Saving is implemented by simply saving all live groups with their group numbers
and full peer info for all peers. On reload, all peers are set as frozen.
Clients needs to support this by understanding that groups may exist on
start-up. Clients should call `tox_conference_get_chatlist` to obtain them. A
group which is deleted (with `tox_conference_delete`) is removed permanently
and will not be saved.
## Limitations
If a peer disconnects from the group for a period short enough that group
timeouts do not occur, and a name change occurs during this period, then the
name change will never be propagated.
One way to deal with this would be a general mechanism for storing and
requesting missed group messages. But this is considered out of scope of this
PR.
If a peer changes its DHT pubkey, the change might not be properly propagated
under various circumstances - in particular, if connections do not go down
long enough for the peer to become frozen.
One way to deal with this would be to add a group message announcing the
sending peer's current DHT pubkey, and treat it analogously to the Name
message.

54
external/toxcore/c-toxcore/docs/updates/Crypto.md vendored Normal file
View File

@ -0,0 +1,54 @@
Encryption library used: http://nacl.cr.yp.to/
When running the program for the first time the crypto_box_keypair() function is used to
generate the users public-private key pair. (32 bytes each)
The generated public key is set as the client_id of the peer.
Adding a friend
---------------
Alice adds Bob to her friend list by adding his 32 byte public key (client_id) to her friend list.
2 cases:
case 1: Alice adds the public key of Bob, then Bob waits for Alice to attempt to connect to him.
case 2: Bob and Alice add their respective public keys to their friend lists at the same time.
case 1:
Alice sends an onion data (see: Prevent_tracking.txt) packet to Bob with the encrypted part containing the friend request like so:
```
[char with a value of 32][nospam number (4 bytes)][Message]
```
Ex message: hello Bob it's me Alice -_- add me pl0x.
For more info on the nospam see: Spam_Prevention.txt
Bob receives the request and decrypts the message using the function crypto_box_open()
If the message decrypts successfully:
If Alice is already in Bob's friend list: case 2
If Alice is not in Bob's friend list and the nospam is good: Bob is prompt to add Alice and is shown the message from her.
If Bob accepts Alice friend request he adds her public key to his friend list.
case 2:
Bob and Alice both have the others public key in their friend list, they are ready for the next step: Connecting to an already added friend
In the next step only crypto_box() is used for encryption and only crypto_box_open() for decryption (just like in the last step.)
Connecting to an already added friend
-------------------------------------
see: Tox_middle_level_network_protocol.txt
Crypto request packets
--------------------------------------
```
[char with a value of 32][Bob (The receiver's) Public key (client_id) (32 bytes))][Alice's (The sender's) Public key (client_id) (32 bytes)][Random nonce (24 bytes)][Encrypted message]
```
The encrypted message is encrypted with crypto_box() (using Bob's public key, Alice's private key and the nonce (randomly generated 24 bytes)) and is a message from Alice in which she tells Bob who she is.
Each node can route the request to the receiver if they are connected to him. This is to bypass bad NATs.

107
external/toxcore/c-toxcore/docs/updates/DHT.md vendored Normal file
View File

@ -0,0 +1,107 @@
DHT protocol
============
NOTE: only the protocol section is up to date, the rest needs to be rewritten.
Follows pretty much the principle of the torrent DHT: http://www.bittorrent.org/beps/bep_0005.html (READ IT)
But:
Vastly simplified packet format and encryption.
Boostrapping:
The first time you install the client we bootstrap it with a node. (bandwidth should not be a problem as the client only needs to be sent one reply.)
Basics
------
(All the numbers here are just guesses and are probably not optimal values)
client list: A list of node ids closest (mathematically see bittorrent doc) to ours matched with ip addresses + port number corresponding to that id and a timestamp containing the time or time since the client was successfully pinged.
"friends" list: A list containing the node_ids of all our "friends" or clients we want to connect to.
Also contains the ip addresses + port + node_ids + timestamp(of last ping like in the client list) of the 8 clients closest (mathematically see bittorrent doc) to each "friend"
One pinged lists:
-One for storing a list of ips along with their ping_ids and a timestamp for the ping requests
Entries in the pinged lists expire after 5 seconds.
If one of the lists becomes full, the expire rate reduces itself one second or the new ping takes the place of the oldest one.
Entries in client list and "friends" list expire after 300 seconds without ping response.
Each client stores a maximum of 32 entries in its client list.
Each client in the client list and "friends" list is pinged every 60 seconds.
Each client in the client list and "friends" list has a timestamp which denote the last time it was successfully pinged.
If the corresponding clients timestamp is more than 130 seconds old it is considered bad.
Send a get nodes request every 20 seconds to a random good node for each "friend" in our "friends" list.
Send a get nodes request every 20 seconds to a random good node in the client list.
When a client receives any request from another
-----------------------------------------------
- Respond to the request
- Ping request is replied to with with a ping response containing the same encrypted data
- Get nodes request is replied with a send nodes reply containing the same encrypted data and the good nodes from the client list and/or the "friends" list that are closest to the requested_node_id
- If the requesting client is not in the client list:
- If there are no bad clients in the list and the list is full:
- If the id of the other client is closer (mathematically see bittorrent doc) than at least one of the clients in the list or our "friends" list:
- Send a ping request to the client.
- if not forget about the client.
- If there are bad clients and/or the list isn't full:
- Send a ping request to the client
When a client receives a response
---------------------------------
- Ping response
- If the node was previously pinged with a matching ping_id (check in the corresponding pinged list.)
- If the node is in the client list the matching client's timestamp is set to current time.
- If the node is in the "friends" list the matching client's timestamp is set to current time for every occurrence.
- If the node is not in the client list:
- If the list isn't full, add it to the list.
- If the list is full, the furthest away (mathematically see bittorrent doc) bad client is replaced by the new one.
- If the list is filled with good nodes replace the furthest client with it only if it is closer than the replaced node.
- for each friend in the "friends" list:
- If that friend's client list isn't full, add that client to it
- If that friend's client list contains bad clients, replace the furthest one with that client.
- If that friend's client list contains only good clients
- If the client is closer to the friend than one of the other clients, it replaces the farthest one
- If not, nothing happens.
- Send nodes
- If the ping_id matches what we sent previously (check in the corresponding pinged list.):
- Each node in the response is pinged.
Protocol
--------
Node format:
```
[uint8_t family (2 == IPv4, 10 == IPv6, 130 == TCP IPv4, 138 == TCP IPv6)][ip (in network byte order), length=4 bytes if ipv4, 16 bytes if ipv6][port (in network byte order), length=2 bytes][char array (node_id), length=32 bytes]
```
see also: DHT.h (pack_nodes() and unpack_nodes())
Valid queries and Responses:
Ping(Request and response):
```
[byte with value: 00 for request, 01 for response][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender: [1 byte type (0 for request, 1 for response)][random 8 byte (ping_id)]]
```
ping_id = a random integer, the response must contain the exact same number as the request
Get nodes (Request):
Packet contents:
```
[byte with value: 02][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender:[char array: requested_node_id (node_id of which we want the ip), length=32 bytes][Sendback data (must be sent back unmodified by in the response), length=8 bytes]]
```
Valid replies: a send_nodes packet
Send_nodes (response (for all addresses)):
```
[byte with value: 04][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender:[uint8_t number of nodes in this packet][Nodes in node format, length=?? * (number of nodes (maximum of 4 nodes)) bytes][Sendback data, length=8 bytes]]
```

12
external/toxcore/c-toxcore/docs/updates/Spam-Prevention.md vendored Normal file
View File

@ -0,0 +1,12 @@
Situation 1:
Someone randomly goes around the DHT sending friend requests to everyone.
Prevented by:
Every friend address:
[client_id (32 bytes)][nospam number (4 bytes)][checksum (2 bytes)]
contains a number (nospam).
The nospam in every friend request to that friend must be that number.
If not it is rejected.

43
external/toxcore/c-toxcore/docs/updates/Symmetric-NAT-Transversal.md vendored Normal file
View File

@ -0,0 +1,43 @@
Notes:
Friend requests need to be routed.
The current DHT should be capable of punching all NATs except symmetric ones.
######
Symmetric NAT hole punching:
If we are not connected to the friend and if the DHT is queried and ips
returned for the friend are the same but the port is different, the friend is
assumed to be behind a symmetric NAT.
Before attempting the procedure we first send a routed ping request to the
friend. This request is to be routed through the nodes who returned the ip of
the peer.
As soon as we receive one routed ping request from the other peer, we respond
with a ping response.
Ping request/response packet:
See: Crypto request packets in [[Crypto]]
Message:
For the ping request:
[char with a value of 254][char with 0][8 byte random number]
For the ping response:
[char with a value of 254][char with 1][8 byte random number (The same that was sent in the request)]
As soon as we get a proper ping response from the other we run the different
ports returned by the DHT through our port guessing algorithm.
######
Port guessing algorithm:
Right now it just tries all the ports directly beside the known ports.(A better one is needed)
######
We send DHT ping requests to all the guessed ports, only a couple at a time.