Draft: Work in progres for backup and restore data from running conduit instance #971
No reviewers
Labels
No labels
Android
CS::needs customer feedback
CS::needs follow up
CS::needs on prem installation
CS::waiting
Chrome
Design:: Ready
Design:: in progress
Design::UX
E2EE
Edge
Firefox
GDPR
Iteration 13 IM
Linux
MacOS
Need::Discussion
Need::Steps to reproduce
Need::Upstream fix
Needs:: Planning
Needs::Dev-Team
Needs::More information
Needs::Priority
Needs::Product
Needs::Refinement
Needs::Severity
Priority::1-Critical
Priority::2-Max
Priority::3-Impending
Priority::4-High
Priority::5-Medium
Priority::6-Low
Priority::7-None
Progress::Backlog
Progress::Review
Progress::Started
Progress::Testing
Progress::Triage
Progress::Waiting
Reporter::Sentry
Safari
Target::Community
Target::Customer
Target::Internal
Target::PoC
Target::Security
Team:Customer-Success
Team:Design
Team:Infrastructure
Team:Instant-Messaging
Team:Product
Team:Workflows
Type::Bug
Type::Design
Type::Documentation
Type::Feature
Type::Improvement
Type::Support
Type::Tests
Windows
blocked
blocked-by-spec
cla-signed
conduit
contribution::advanced
contribution::easy
contribution::help needed
from::review
iOS
p::ti-tenant
performance
product::triage
proposal
refactor
release-blocker
s: dart_openapi_codegen
s::Famedly-Patient
s::Org-Directory
s::Passport-Generator
s::Requeuest
s:CRM
s:Famedly-App
s:Famedly-Web
s:Fhiroxide
s:Fhiroxide-cli
s:Fhiroxide-client
s:Fhirs
s:Hedwig
s:LISA
s:Matrix-Dart-SDK
s:Role-Manager
s:Synapse
s:User-Directory
s:WFS-Matrix
s:Workflow Engine
s:dtls
s:famedly-error
s:fcm-shared-isolate
s:matrix-api-lite
s:multiple-tab-detector
s:native-imaging
severity::1
severity::2
severity::3
severity::4
technical-debt
voip
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Matthias/conduit#971
Loading…
Reference in a new issue
No description provided.
Delete branch "WIP_backup"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Hi,
In the context of #16
This add some API in the KVEngine for export all the data, I used in all implementation I could the snapshot mechanism, to have a point in time read that do not include data that refer not existing data, the implementations that do not support snapshots are sqlite,sled and heed, sqlite just because rusqlite do not support it by now, but is available in sqlite itself so in long term should be possible to support also that.
An additional limitation is that as today an export of data may include partial "executions" because operations are not executed as a single transaction in the underlying storage as today,
to fix this is needed that the high level operations are executed in transaction nothing can be done at export level.
As far this include only the low level implementation, more work is needed to define a simple export format to a file and a way to schedule or trigger the export, together with some import/restore functionalities.
added 1 commit
Compare with previous version
added 25 commits
famedly:next
5d764947
- refactor: changed tree specific impls to be one liner5498863d
- test: add tests for rocksdb and persy impleaec8286
- test:add entry for seamless remove of not existing key from database1c198f79
- fix:update implementation to recent changes on next418aeee0
- chore: fix tests after refactordb60a7c0
- fix: make heed implementation compile089c7284
- feat: first implementation of KeyValue data export5ecc1e35
- feat: additional implementation of export logic08cde527
- test: add test for exporting and importing data from tree storeba52bcef
- feat: add implementation to export and import low level data in jsonCompare with previous version
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.