Commit Graph

644 Commits

Author SHA1 Message Date
f22f523774 minor frag store refactor 2024-04-12 13:42:03 +02:00
8b17ed195f more testing and file2 zstd now passes tests with varying frame sizes
and 1.5gig files
2024-04-12 13:42:03 +02:00
def7fc1959 add file2 impl for zstd (lightly tested and not integrated yet) 2024-04-12 13:42:02 +02:00
318be9cd62 throw update 2024-04-12 13:42:02 +02:00
2772c8ee69 reduce excessive message frag saving (queue dedup + waiting 10sec)
prepare for frag updates
2024-04-12 13:42:02 +02:00
eac2927379 try to tame log spam 2024-04-12 13:42:02 +02:00
77a0ae6acd fix accel structure being wrong and mark empty frags and dont count them 2024-04-12 13:42:02 +02:00
7879a0927b combat memory leaks with smart pointers 2024-04-12 13:42:02 +02:00
88ea3e177d refactor saving and save on exit 2024-04-12 13:42:01 +02:00
bc22451524 dirty frag on message updates (if still open) 2024-04-12 13:42:01 +02:00
7b8e93eec3 refactor message fuid -> fid
save alot of memory by using fid instead of fuid
2024-04-12 13:42:01 +02:00
71be5c3c6e reduce log spam 2024-04-12 13:42:01 +02:00
2b8cee6a29 remove old code 2024-04-12 13:42:01 +02:00
5bf4640d61 forgot to throw update on read 2024-04-12 13:42:01 +02:00
0e0e81720b dont sync messages we dont know enough about 2024-04-12 13:42:00 +02:00
592a4cb9cf make adjacency loading work, extend range and use loops 2024-04-12 13:42:00 +02:00
93f60bd073 replace old bad prev/next code with way better code 2024-04-12 13:42:00 +02:00
6a6de77ae9 smaller contact frag fixes 2024-04-12 13:42:00 +02:00
89f065a610 impl new acceleration structure for components, not exploited yet
disable funky load at first msg
2024-04-12 13:42:00 +02:00
52e95ca654 forgot to check contact 2024-04-12 13:41:59 +02:00
eaa316a2aa rework cursers for cg, keep views between switching. will be refactored later 2024-04-12 13:41:59 +02:00
bdf4e60f2f fix one inverted comparator 2024-04-12 13:41:59 +02:00
2e3c779bec stop ignoring mfs interval and sort after 2024-04-12 13:41:59 +02:00
461a4f1aa7 make inital curser a range 2024-04-12 13:41:59 +02:00
78488daa9b loading logic implemented but broken (very funky and sometimes even out of contact) 2024-04-12 13:41:59 +02:00
22f2c8f514 load based on view cursers (untested and not used yet) 2024-04-12 13:41:58 +02:00
e442191aad msg frag before and after helper 2024-04-12 13:41:58 +02:00
795ab2d4e1 fix potential tsrange errors and deduplicate state 2024-04-12 13:41:58 +02:00
0896038dd6 make writing safe (by using a tmp file and moving to actual location) 2024-04-12 13:41:58 +02:00
67c6f9adb0 make empty contacts from ids on message load 2024-04-12 13:41:58 +02:00
6aac44cda9 change binary meta format and add zstd to metadata 2024-04-12 13:41:58 +02:00
4fb2b51b7d switch to streaming compressor for data to drastically improve ratio.
would still benefit from a abstract file refactor
2024-04-12 13:41:57 +02:00
182d844e32 update fs readme a little 2024-04-12 13:41:57 +02:00
6f511016bc save msg json zstd compressed (3x compression) 2024-04-12 13:41:57 +02:00
fb885b5c21 simplify array cast a little 2024-04-12 13:41:57 +02:00
527a7c63f6 add zstd dep 2024-04-12 13:41:57 +02:00
d21dbb43e2 comp refactor and make groups work 2024-04-12 13:41:57 +02:00
7ac62274f4 move json around and disable files for now 2024-04-12 13:41:56 +02:00
4ec87337c8 reverse message write order 2024-04-12 13:41:56 +02:00
20f7c6d011 add dup check, would work for ngc if we saved tox group msg id yet 2024-04-12 13:41:56 +02:00
24dc5a03f3 fix dup on write 2024-04-12 13:41:56 +02:00
3d0863ff9a basically working, but some dup glitch is still there 2024-04-12 13:41:56 +02:00
97aedca844 scan laters 2024-04-12 13:41:56 +02:00
2e7d5538d1 fragment events + 256bit uuids 2024-04-12 13:41:55 +02:00
1bfd04680e refactor message serializer to allow access to eg contacts 2024-04-12 13:41:55 +02:00
73d1d65142 further serializer refactoring 2024-04-12 13:41:55 +02:00
f6e55851cc improve deserialization and provide message comp deserl 2024-04-12 13:41:55 +02:00
0b0245d844 loading fragments mostly working (not notifying anyone yet) 2024-04-12 13:41:55 +02:00
d278391528 add contact id to meta 2024-04-12 13:41:55 +02:00
aa7a5d6013 more comps 2024-04-12 13:41:54 +02:00