Fetch server keys concurrently #985
No reviewers
Labels
No labels
Android
CS::needs customer feedback
CS::needs follow up
CS::needs on prem installation
CS::waiting
Chrome
Design:: Ready
Design:: in progress
Design::UX
E2EE
Edge
Firefox
GDPR
Iteration 13 IM
Linux
MacOS
Need::Discussion
Need::Steps to reproduce
Need::Upstream fix
Needs:: Planning
Needs::Dev-Team
Needs::More information
Needs::Priority
Needs::Product
Needs::Refinement
Needs::Severity
Priority::1-Critical
Priority::2-Max
Priority::3-Impending
Priority::4-High
Priority::5-Medium
Priority::6-Low
Priority::7-None
Progress::Backlog
Progress::Review
Progress::Started
Progress::Testing
Progress::Triage
Progress::Waiting
Reporter::Sentry
Safari
Target::Community
Target::Customer
Target::Internal
Target::PoC
Target::Security
Team:Customer-Success
Team:Design
Team:Infrastructure
Team:Instant-Messaging
Team:Product
Team:Workflows
Type::Bug
Type::Design
Type::Documentation
Type::Feature
Type::Improvement
Type::Support
Type::Tests
Windows
blocked
blocked-by-spec
cla-signed
conduit
contribution::advanced
contribution::easy
contribution::help needed
from::review
iOS
p::ti-tenant
performance
product::triage
proposal
refactor
release-blocker
s: dart_openapi_codegen
s::Famedly-Patient
s::Org-Directory
s::Passport-Generator
s::Requeuest
s:CRM
s:Famedly-App
s:Famedly-Web
s:Fhiroxide
s:Fhiroxide-cli
s:Fhiroxide-client
s:Fhirs
s:Hedwig
s:LISA
s:Matrix-Dart-SDK
s:Role-Manager
s:Synapse
s:User-Directory
s:WFS-Matrix
s:Workflow Engine
s:dtls
s:famedly-error
s:fcm-shared-isolate
s:matrix-api-lite
s:multiple-tab-detector
s:native-imaging
severity::1
severity::2
severity::3
severity::4
technical-debt
voip
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Matthias/conduit#985
Loading…
Reference in a new issue
No description provided.
Delete branch "concurrent-fetch-keys"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This speeds up handling of /_matrix/federation/v1/send/:transaction_id
when more than one event contains unknown keys.
In particular, when receiving multiple PDUs with dead servers in their
auth chain, timeouts of each server accumulate and can make handling of
incoming requests take several minutes, to the point the client closes
the connection (eg. matrix.org has a 2 minute timeout), causing new
events to be dropped eventually.
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
28db5ada
- Fetch server keys concurrentlyCompare with previous version
mentioned in commit Aranjedeath/conduit@e6938db25b
added 1 commit
e7f4dd47
- Add missing pub key in send_join handlingCompare with previous version
I have been using this patch since written. it works well.
I only skimmed this code so far, but it looks like you reimplemented some of the functionality found in event_handler::fetch_join_signing_keys, which is a bit more powerful because it tries to get all keys in one go from the trusted server. I think instead of introducing more functions, we can make fetch_join_signing_keys more generic and use it in more places, what do you think?
But I also feel like this MR is not as useful as it sounds, because it's only a single transaction that takes long. After that, either the keys are loaded, or the servers are on the backoff blacklist.
this isn't introducing a new function, I only generalized
fetch_required_signing_keys
.But indeed, they do look similar. Maybe
It is a requirement to run a server with
trusted_servers = []
if you want to join a handful of non-trivial rooms. Without it, most PDUs frommatrix.org
can't be processedor the client closed the connection before we finished processing, causing the request to be killed before anything is persisted.
I don't think this MR changes anything about room joins, also see my chat messages here: https://matrix.to/#/!SMloEYlhCiqKwRLAgY:fachschaften.org/$GwAOU5o9gh_ULd2rjZO4PoNF4wDphZCkvz_IqgV2ENM?via=flipdot.org&via=matrix.org&via=conduit.rs
All requests are handled in a new tokio task that will not be cancelled when the client disconnects, see
spawn_task
in main.rsI meant that after joining the rooms, PDUs from matrix.org time out
well I don't know then, but I can tell you that for two weeks my server was unable to process PDUs from matrix.org in times of high activity and logging that it was fetching the same keys over and over.
Interesting, I will try to look through the code again with this bug in mind
mentioned in commit Aranjedeath/conduit@3122a7417b
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.