Commit Graph

417 Commits

Author SHA1 Message Date
a5e67d0ee8 move uuid gen to util 2024-04-12 19:44:24 +02:00
dfbb1dea68 move os and backend to sub 2024-04-12 13:42:08 +02:00
2597edd579 new messages objcomp names 2024-04-12 13:42:08 +02:00
85a29372f4 adding message frag object version and conversion work, but comp name changes incoming 2024-04-12 13:42:07 +02:00
a9f6a5d763 move mfs to os, works, convert tool still incomplete 2024-04-12 13:42:07 +02:00
73180195fe some more backend interface changes i realized i had to do 2024-04-12 13:42:07 +02:00
10b689ca95 refactor the serializer again 2024-04-12 13:42:07 +02:00
3796841961 conversion improvements 2024-04-12 13:42:07 +02:00
854ed851b4 fixes missing comps and conversion tool is almost working (most of meta is transfered) 2024-04-12 13:42:07 +02:00
5c3b797a99 new object (os + fsb mostly done) 2024-04-12 13:42:06 +02:00
8a580e2fbb backend fs read 2024-04-12 13:42:06 +02:00
3cede91aa0 more work on backend and moving frags to objs 2024-04-12 13:42:06 +02:00
0610a6a64a continue os refactor, start with fs backend 2024-04-12 13:42:06 +02:00
26d07b06db start refactoring in the name of object store 2024-04-12 13:42:06 +02:00
fd0b210bbb fragment store fully file2 zstd. no more zstd.h in fragment store 2024-04-12 13:42:06 +02:00
268cbe137e move file stack creation to separate file, removing some scope 2024-04-12 13:42:05 +02:00
31bb0d3e61 refactor meta data decompression using file2 and mem 2024-04-12 13:42:05 +02:00
248f68f6a2 minimal stack creation refactor (wip) 2024-04-12 13:42:05 +02:00
3b010bd16f catch write errors (eg unsanitzed strings) 2024-04-12 13:42:05 +02:00
7e285290fe test in memory comp 2024-04-12 13:42:05 +02:00
5767834f71 since its working, remove old commented code 2024-04-12 13:42:05 +02:00
53ce292e82 fix stack overflow (i am stupid) and some file stack error handling 2024-04-12 13:42:04 +02:00
84bd24807d small change to flush work space. writing new frags is broken rn 2024-04-12 13:42:04 +02:00
8d0518c2e3 harden against some parsing exceptions
(by disabling them, since the error case is already handled)
2024-04-12 13:42:04 +02:00
6d150ba441 roll back meta comp (did it wrong) and enable data comp 2024-04-12 13:42:04 +02:00
c737715c66 more refactor and transition meta write 2024-04-12 13:42:04 +02:00
b640b5a06b variant to span helper 2024-04-12 13:42:04 +02:00
1b9363e7b5 tested and works, cleaning up commented code 2024-04-12 13:42:03 +02:00
16d2238f35 fixes for ci 2024-04-12 13:42:03 +02:00
19844a9423 use file2 zstd wrapper to read frag data (untested) 2024-04-12 13:42:03 +02:00
19fd99f713 refactor out uuidgenerator 2024-04-12 13:42:03 +02:00
f22f523774 minor frag store refactor 2024-04-12 13:42:03 +02:00
8b17ed195f more testing and file2 zstd now passes tests with varying frame sizes
and 1.5gig files
2024-04-12 13:42:03 +02:00
def7fc1959 add file2 impl for zstd (lightly tested and not integrated yet) 2024-04-12 13:42:02 +02:00
318be9cd62 throw update 2024-04-12 13:42:02 +02:00
2772c8ee69 reduce excessive message frag saving (queue dedup + waiting 10sec)
prepare for frag updates
2024-04-12 13:42:02 +02:00
eac2927379 try to tame log spam 2024-04-12 13:42:02 +02:00
77a0ae6acd fix accel structure being wrong and mark empty frags and dont count them 2024-04-12 13:42:02 +02:00
7879a0927b combat memory leaks with smart pointers 2024-04-12 13:42:02 +02:00
88ea3e177d refactor saving and save on exit 2024-04-12 13:42:01 +02:00
bc22451524 dirty frag on message updates (if still open) 2024-04-12 13:42:01 +02:00
7b8e93eec3 refactor message fuid -> fid
save alot of memory by using fid instead of fuid
2024-04-12 13:42:01 +02:00
71be5c3c6e reduce log spam 2024-04-12 13:42:01 +02:00
2b8cee6a29 remove old code 2024-04-12 13:42:01 +02:00
5bf4640d61 forgot to throw update on read 2024-04-12 13:42:01 +02:00
0e0e81720b dont sync messages we dont know enough about 2024-04-12 13:42:00 +02:00
592a4cb9cf make adjacency loading work, extend range and use loops 2024-04-12 13:42:00 +02:00
93f60bd073 replace old bad prev/next code with way better code 2024-04-12 13:42:00 +02:00
6a6de77ae9 smaller contact frag fixes 2024-04-12 13:42:00 +02:00
89f065a610 impl new acceleration structure for components, not exploited yet
disable funky load at first msg
2024-04-12 13:42:00 +02:00