Matthias/conduit
1
0
Fork 0
Code Issues 179 Pull requests 44 Projects Releases 5 Packages Wiki Activity

Use MongoDB as an alternative storage solution #345

New issue
Open
opened 2023-03-27 16:35:50 +00:00 by lmauromb · 3 comments
lmauromb commented 2023-03-27 16:35:50 +00:00 (Migrated from gitlab.com)
Copy link

Is your feature request related to a problem? Please describe.

Hello Conduit Team, here at the federation team at Rocket.Chat are evaluating the use of Conduit as our matrix homeserver for our federation capabilities, so we wanted to reach out to ask if it would be possible to contribute back with some changes that we need to do to achieve the integration.

Describe the solution you'd like

Since our application relies on MongoDB for our storage we wanted to know if it would be architecturally possible to add Mongo as an alternative solution for the Conduit storage.

I don't know if for Conduit you still need to have an embedded database, so we wanted to check first if it is possible to add the Mongo abstraction similar to what you have for RocksDB/SQLite.

### Is your feature request related to a problem? Please describe. Hello Conduit Team, here at the federation team at Rocket.Chat are evaluating the use of Conduit as our matrix homeserver for our federation capabilities, so we wanted to reach out to ask if it would be possible to contribute back with some changes that we need to do to achieve the integration. ### Describe the solution you'd like Since our application relies on MongoDB for our storage we wanted to know if it would be architecturally possible to add Mongo as an alternative solution for the Conduit storage. I don't know if for Conduit you still need to have an embedded database, so we wanted to check first if it is possible to add the Mongo abstraction similar to what you have for RocksDB/SQLite.
timokoesters commented 2023-05-21 07:48:50 +00:00 (Migrated from gitlab.com)
Copy link

Hello, thanks for offering your help! We currently only support key-value databases, but there is a lot of interest in Postgres and other relational databases. Last year I rewrote some of the core code to make this more simple:

Each "service" defines the set of database interaction it needs, see for example src/service/users/data.rs. These interactions can have multiple implementations. Currently we just have an implementation for key value databases, see for example src/database/key_value/users.rs

In the future we can extend this to src/database/relational/users.rs to support proper SQL databases.

If you want to combine efforts on this, send me a message on Matrix or an email.

Hello, thanks for offering your help! We currently only support key-value databases, but there is a lot of interest in Postgres and other relational databases. Last year I rewrote some of the core code to make this more simple: Each "service" defines the set of database interaction it needs, see for example `src/service/users/data.rs`. These interactions can have multiple implementations. Currently we just have an implementation for key value databases, see for example `src/database/key_value/users.rs` In the future we can extend this to `src/database/relational/users.rs` to support proper SQL databases. If you want to combine efforts on this, send me a message on Matrix or an email.
erlend-sh commented 2024-02-12 08:24:33 +00:00 (Migrated from gitlab.com)
Copy link

Possible avenue: https://www.ferretdb.com/

FerretDB is an open-source proxy, converting the MongoDB 5.0+ wire protocol queries to SQL - using PostgreSQL or SQLite as a database engine.

Possible avenue: https://www.ferretdb.com/ > FerretDB is an open-source proxy, converting the MongoDB 5.0+ wire protocol queries to SQL - using PostgreSQL or SQLite as a database engine.
avdb13 commented 2024-02-12 08:44:47 +00:00 (Migrated from gitlab.com)
Copy link

This could work but I doubt OP is gonna have a great time with Conduit unless they find a way to convert a data (which is gonna be painful in case the objects even differ slightly). MongoDB created more problems than it solved for my uses. Handy tool though!

This could work but I doubt OP is gonna have a great time with Conduit unless they find a way to convert a data (which is gonna be painful in case the objects even differ slightly). MongoDB created more problems than it solved for my uses. Handy tool though!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
next
performance
jonas-test-ci-docker-building-quemu-on-fsinfo-runner
presence
master
askstate
recoverbroken
Nyaaori/temp-combined-feature-branch
Nyaaori/destructive-rocksdb-recovery
Nyaaori/full-profile-directory
Nyaaori/read-receipt-config-options
Nyaaori/private-read-receipts
Nyaaori/federation-backfill
Nyaaori/declare-msc3827
Nyaaori/remove-misisng-e2ee-readme
refactor
Nyaaori/refactor-next
refactor-ci-fix
bump-docker-alpine-to-3.16.2
jplatte/up-ruma
mdbook-docs-preparation
async
244-support-well-known
complement-integration
brokenjoinfix
state_full_ids_cache
ticho34782694/conduit-rocksdb-memory-usage
docs-the-second
serve-element-web
perf
nyaaori/older-room-versions
serve-well-known
unbeatable-101-next-patch-64009
docs-overhaul
state-res-perf
stablecrosssigning
member-reason
admin-command-for-cache-memory-usage
measure-cache-memory-usage
lesscache
locks
state-res-closure-rebased
ruma-client
ruma
v0.6.0
v0.5.0
v0.4.0
v0.3.0
v0.2.0
Labels
Clear labels   
Android

This label indicates that the issue affects Android devices.

  
CS::needs customer feedback

Issue has been finished and needs feedback from the customer if everything is okay OR the customer needs to answer questions before issue can be worked on.

  
CS::needs follow up

We have received questions or new information from the customer that needs discussing in the meeting with Product and Sales

  
CS::needs on prem installation

This Issue needs to be installed on prem at the customer after closing

  
CS::waiting

This Issue is in progress in other teams (prod/tech)

  
Chrome

This label indicates that the issue affects the chrome browser. You should always combine it with other platforms like Android, if it only affects Android.

  
Design:: Ready

   
Design:: in progress

   
Design::UX

@Famedly: Do not use this label

  
E2EE

   
Edge

This label indicates that the issue affects Edge devices.

  
Firefox

This label indicates that the issue affects Firefox devices.

  
GDPR

   
Iteration 13 IM

   
Linux

This label indicates that the issue affects Linux devices.

  
MacOS

This label indicates that the issue affects MacOS devices.

  
Need::Discussion

Topic needs more discussion

  
Need::Steps to reproduce

   
Need::Upstream fix

   
Needs:: Planning

   
Needs::Dev-Team

This label indicates that the issue needs to be assigned to a dev team.

  
Needs::More information

   
Needs::Priority

This label should be set if there is no priority on feature or improvement requests.

  
Needs::Product

   
Needs::Refinement

the issue is triaged and needs to go to the refinement meeting to be able to set story points to the issue

  
Needs::Severity

This label should be set if bug reports need a severity.

  
Priority::1-Critical

The Priority 1 label indicates a critically dire issue such as a regression, or failure in existing features. We look at issues with this label frequently. If you find yourself assigning a Prio 1 label to an issue, please be sure that there's a positive handoff between filing and a prospective owner for the issue. Prio 1 issues should be rare. Normally there should be zero open P1 issues. "The world is on fire."

  
Priority::2-Max

The Priority 2 label indicates that the issue deserves immediate attention, such as affecting a top-tier customer. There are generally only one or two Prio 2 bugs at a time, and we aim for there to be none. "A customer is on fire."

  
Priority::3-Impending

A bug with Priority 3 flag indicates that a top-tier customer is about to encounter this bug. Generally, there are less than ten Priority 3 bugs. Issues may not be handled immediately (sometimes some Prio 4 issues are handled first), but are top-of-mind and we explicitly consider them weekly. "A customer is about to be on fire."

  
Priority::4-High

The Priority 4 label indicates high-priority issues that are at the top of the work list. This is the highest priority level an issue can have if it isn't affecting a top-tier customer or breaking the app. Bugs marked Priority 4 are generally actively being worked on unless the assignee is dealing with a P1-P3 bug(or another P4 issue).

  
Priority::5-Medium

The Priority 5 label indicates issues that we agree are important to work on, but not at the top of the work list. This is the default level for a bug. An issue at this priority level may not be worked on for a long time. Often an issue at this level will first migrate to Prio 4 before we work on them.

  
Priority::6-Low

The Priority 6 label indicates issues we think are valid but not important. This is the default level for new feature requests. We are unlikely to work on such issues unless they form part of a long-term strategic plan (see the Roadmap).

  
Priority::7-None

The Priority 7 label indicates valid issues that are unlikely to ever be worked on. We use "thumbs-up" on these issues to promote them to Priority 6 or higher based on demand.

  
Progress::Backlog

The Refinement of the issue is completed and DoR for the Squad is fulfilled

  
Progress::Review

Merge Request is ready for review. By definition MR review needs to be done by a second developer.

  
Progress::Started

The issue has been picked up for work by a developer in the team and the developer started to work on actively on the issue.

  
Progress::Testing

In testing

  
Progress::Triage

This is the first step of the process. Issue has been recently opened. Product team will identify/evaluate the following: - Clarify questions if required / verify DoR for Product is met. - It’s been identified if Design work is required or not. - Technical Feasibility - Components involved. - Team to work on that issue. - Severity (in case the issue refers to a bug)

  
Progress::Waiting

A blocker has been identified and work cannot continue.

  
Reporter::Sentry

   
Safari

This label indicates that the issue affects Safari devices.

  
Target::Community

   
Target::Customer

   
Target::Internal

   
Target::PoC

Proof of Concept - issues necessary for potential customer acquisition. Can be scoped by deadline for planned presentations

  
Target::Security

   
Team:Customer-Success

   
Team:Design

Issues which are in the competence area of the frontend chapter. It is not a scoped label anymore as issues can be part of different chapters. Before we can start implementing/changing this, we need a UI/UX design first.

  
Team:Infrastructure

Issues which are in the competence area of the infrastructure chapter. It is not a scoped label anymore as issues can be part of different chapters.

  
Team:Instant-Messaging

   
Team:Product

Issues which are in the responsibility of the product team.

  
Team:Workflows

   
Type::Bug

A software bug is an error, flaw or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways.

  
Type::Design

A task about creating or improving the UI/UX design

  
Type::Documentation

   
Type::Feature

Nothing broken; Request for a new feature.

  
Type::Improvement

An improvement or enhancement to an existing feature, technically solution or code. It makes a working functionality faster, more secure, add some additional value, refactors code etc. or helps to design a better UX and UI.

  
Type::Support

   
Type::Tests

A task about writing or improving automated or manual tests.

  
Windows

This label indicates that the issue affects Windows devices.

  
blocked

   
blocked-by-spec

   
cla-signed

   
conduit

   
contribution::advanced

   
contribution::easy

Easy task to start with a contribution

  
contribution::help needed

   
from::review

Reported by famedly review

  
iOS

This label indicates that the issue affects iOS devices.

  
p::ti-tenant

This label indicates that the issues is a requirement of the ti-tenant

  
performance

   
product::triage

   
proposal

A feature propsal.

  
refactor

   
release-blocker

This issue needs to be fixed before the next release.

  
s: dart_openapi_codegen

   
s::Famedly-Patient

Service: Famedly-Patient App

  
s::Org-Directory

For issues that relate to the organisation directory https://gitlab.com/famedly/company/backend/services/organisation-directory/

  
s::Passport-Generator

Service: PASSporT-Generator

  
s::Requeuest

For anything relevant to the requeuest library

  
s:CRM

Service: CRM

  
s:Famedly-App

Service: Famedly-App

  
s:Famedly-Web

Service: Famedly-Web

  
s:Fhiroxide

Service: Fhiroxide

  
s:Fhiroxide-cli

Fhir(oxide) maintenance and debug CLI

  
s:Fhiroxide-client

Fhiroxide client library

  
s:Fhirs

Service: FHIRS

  
s:Hedwig

Service: Hedwig

  
s:LISA

Service: LISA

  
s:Matrix-Dart-SDK

Service: Matrix-Dart-SDK

  
s:Role-Manager

Service: Role-Manager

  
s:Synapse

Service: Synapse

  
s:User-Directory

Service: User-Directory

  
s:WFS-Matrix

Service: Workflow-Service-Matrix

  
s:Workflow Engine

Service: Workflow-Engine

  
s:dtls

   
s:famedly-error

   
s:fcm-shared-isolate

   
s:matrix-api-lite

   
s:multiple-tab-detector

   
s:native-imaging

   
severity::1

Blocker: Broken feature with no workaround or any data-loss. It makes the feature unusable!

  
severity::2

Critical: Broken feature with an unacceptably complex workaround.

  
severity::3

Major: Broken feature with a workaround.

  
severity::4

Low: Functionality is inconvenient.

  
technical-debt

This Issue is a technical debt. Mark issues with this label to have an overview of the technical debts in a repository or group.

  
voip
No labels
Android
CS::needs customer feedback
CS::needs follow up
CS::needs on prem installation
CS::waiting
Chrome
Design:: Ready
Design:: in progress
Design::UX
E2EE
Edge
Firefox
GDPR
Iteration 13 IM
Linux
MacOS
Need::Discussion
Need::Steps to reproduce
Need::Upstream fix
Needs:: Planning
Needs::Dev-Team
Needs::More information
Needs::Priority
Needs::Product
Needs::Refinement
Needs::Severity
Priority::1-Critical
Priority::2-Max
Priority::3-Impending
Priority::4-High
Priority::5-Medium
Priority::6-Low
Priority::7-None
Progress::Backlog
Progress::Review
Progress::Started
Progress::Testing
Progress::Triage
Progress::Waiting
Reporter::Sentry
Safari
Target::Community
Target::Customer
Target::Internal
Target::PoC
Target::Security
Team:Customer-Success
Team:Design
Team:Infrastructure
Team:Instant-Messaging
Team:Product
Team:Workflows
Type::Bug
Type::Design
Type::Documentation
Type::Feature
Type::Improvement
Type::Support
Type::Tests
Windows
blocked
blocked-by-spec
cla-signed
conduit
contribution::advanced
contribution::easy
contribution::help needed
from::review
iOS
p::ti-tenant
performance
product::triage
proposal
refactor
release-blocker
s: dart_openapi_codegen
s::Famedly-Patient
s::Org-Directory
s::Passport-Generator
s::Requeuest
s:CRM
s:Famedly-App
s:Famedly-Web
s:Fhiroxide
s:Fhiroxide-cli
s:Fhiroxide-client
s:Fhirs
s:Hedwig
s:LISA
s:Matrix-Dart-SDK
s:Role-Manager
s:Synapse
s:User-Directory
s:WFS-Matrix
s:Workflow Engine
s:dtls
s:famedly-error
s:fcm-shared-isolate
s:matrix-api-lite
s:multiple-tab-detector
s:native-imaging
severity::1
severity::2
severity::3
severity::4
technical-debt
voip
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Matthias/conduit#345
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?