aboutsummaryrefslogtreecommitdiff
path: root/federationapi
AgeCommit message (Collapse)Author
2023-03-23Move every `db.Prepare` to `sqlutil.Statementlist`, remove trace driver (#3026)Till
Doesn't buy us much, but makes everything a bit more consistent. Also removes the SQL trace driver, as it is unused and the output is hard to read anyway.
2023-03-22Remove `BaseDendrite` (#3023)Till
Removes `BaseDendrite` to, hopefully, make testing and composing of components easier in the future.
2023-03-17Preparations for removing `BaseDendrite` (#3016)Till
Preparations to actually remove/replace `BaseDendrite`. Quite a few changes: - SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed from `BaseDendrite`) - Caches are removed from `BaseDendrite` - Introduces a `Router` struct (likely to change) - also fixes #2903 - Introduces a `sqlutil.ConnectionManager`, which should remove `base.DatabaseConnection` later on - probably more
2023-03-01Refactor `StoreEvent`, add `MaybeRedactEvent`, create an `EventDatabase` (#2989)Till
This PR changes the following: - `StoreEvent` now only stores an event (and possibly prev event), instead of also doing redactions - Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should be called after storing events - a few other things
2023-02-20Merge keyserver & userapi (#2972)Till
As discussed yesterday, a first draft of merging the keyserver and the userapi.
2023-02-14Remove polylith/API mode (#2967)Till
This removes most of the code used for polylith/API mode. This removes the `/api` internal endpoints entirely. Binary size change roughly 5%: ``` 51437560 Feb 13 10:15 dendrite-monolith-server # old 48759008 Feb 13 10:15 dendrite-monolith-server # new ```
2023-01-28Relay integration to pinecone demos (#2955)devonh
This extends the dendrite monolith for pinecone to integrate the s&f features into the mobile apps. Also makes a few tweaks to federation queueing/statistics to make some edge cases more robust.
2023-01-23Initial Store & Forward Implementation (#2917)devonh
This adds store & forward relays into dendrite for p2p. A few things have changed: - new relay api serves new http endpoints for s&f federation - updated outbound federation queueing which will attempt to forward using s&f if appropriate - database entries to track s&f relays for other nodes
2023-01-20Make tests more reliable (#2948)Till
When using `testrig.CreateBase` and then using that base for other `NewInternalAPI` calls, we never actually shutdown the components. `testrig.CreateBase` returns a `close` function, which only removes the database, so still running components have issues connecting to the database, since we ripped it out underneath it - which can result in "Disk I/O" or "pq deadlock detected" issues.
2023-01-19Add `/_dendrite/admin/purgeRoom/{roomID}` (#2662)Neil
This adds a new admin endpoint `/_dendrite/admin/purgeRoom/{roomID}`. It completely erases all database entries for a given room ID. The roomserver will start by clearing all data for that room and then will generate an output event to notify downstream components (i.e. the sync API and federation API) to do the same. It does not currently clear media and it is currently not implemented for SQLite since it relies on SQL array operations right now. Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-01-17`/sync` performance optimizations (#2927)Till
Since #2849 there is no limit for the current state we fetch to calculate history visibility. In large rooms this can cause us to fetch thousands of membership events we don't really care about. This now only gets the state event types and senders in our timeline, which should significantly reduce the amount of events we fetch from the database. Also removes `MaxTopologicalPosition`, as it is an unnecessary DB call, given we use the result in `topological_position < $1` calls.
2023-01-12Always initialize statistics server mapDevon Hudson
2022-12-22More flakey testsTill Faelligen
2022-12-22Fix flakey testTill Faelligen
2022-12-22Add federation peeking table tests (#2920)Till
As the title says, adds tests for inbound/outbound peeking federation table tests. Also removes some unused code
2022-12-06Fix issue with sending presence events to invalid serversTill Faelligen
2022-12-05Enable/Disable internal metrics (#2899)Till
Basically enables us to use `test.WithAllDatabases` when testing internal HTTP APIs, as this would otherwise result in Prometheus complaining about already registered metric names.
2022-11-18Fix registration for virtual hostingNeil Alexander
2022-11-18Add p2p wakeup broadcast handling to pinecone demos (#2841)devonh
Adds wakeup broadcast handling to the pinecone demos. This will reset their blacklist status and interrupt any ongoing federation queue backoffs currently in progress for this peer. The end result is that any queued events will quickly be sent to the peer if they had disconnected while attempting to send events to them.
2022-11-17Try that againNeil Alexander
2022-11-17Fix `/key/v2/server`, add HTTP `Host` matchingNeil Alexander
2022-11-16Fix bug in a2f72dd9Neil Alexander
2022-11-16Fix slice out of bounds in federation APINeil Alexander
2022-11-16Virtual host server name workaroundNeil Alexander
2022-11-16Tweak `/key/v2/server`Neil Alexander
2022-11-15Don't get blacklisted hosts when querying joined servers (#2880)Neil Alexander
Otherwise we just waste time/CPU.
2022-11-15Federation fixes for virtual hostingNeil Alexander
2022-11-11Virtual hosting schema and logic changes (#2876)Neil Alexander
Note that virtual users cannot federate correctly yet.
2022-11-11Send presence to newly added servers (#2869)Till
This should make `New federated private chats get full presence information (SYN-115)` happy.
2022-11-04Send presence to joined hosts only (#2858)Till
Send presence events only to rooms the user is participating, not all servers we know about. Should fix #2752
2022-10-27Refactor `claimRemoteKeys`Neil Alexander
2022-10-27Add AS specific public room list endpoints (#2836)Till
Adds `PUT /_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and `DELTE /_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` support, as well as the ability to filter `/publicRooms` on networkID and including all networks.
2022-10-26Associate events in db before queueing them to send (#2833)devonh
Fixes a race condition between sending federation events and having them fully associated in the database.
2022-10-26Update federation API consumersNeil Alexander
2022-10-26Initial support for multiple server names (#2829)Neil Alexander
This PR is the first step towards virtual hosting by laying the groundwork for multiple server names being configured.
2022-10-21Optimize inserting pending PDUs/EDUs (#2821)Till
This optimizes the association of PDUs/EDUs to their destination by inserting all destinations in one transaction.
2022-10-20Fix flakey queue test (#2818)devonh
Ensure both events are added to the database, even if the destination is already blacklisted.
2022-10-20Fix race conditionTill Faelligen
2022-10-19Refactor Federation Destination Queues (#2807)devonh
This is a refactor of the federation destination queues. It fixes a few things, namely: - actually retry outgoing events with backoff behaviour - obtain enough events from the database to fill messages as much as possible - minimize the amount of running goroutines - use pure timers for backoff - don't restart queue unless necessary - close the background task when backing off - increase max edus in a transaction to match the spec - cleanup timers more aggresively to reduce memory usage - add jitter to backoff timers to reduce resource spikes - add a bunch of tests (with real and fake databases) to ensure everything is working
2022-10-13Revert "Federation backoff fixes and tests (#2792)"Neil Alexander
This reverts commit dcedd1b6bf1e890ff425bdf1fcd8a2e0850778b5.
2022-10-13Federation backoff fixes and tests (#2792)devonh
This fixes some edge cases where federation queue backoffs and blacklisting weren't behaving as expected. It also adds new tests for the federation queues to ensure their behaviour continues to work correctly.
2022-10-11Private read receipts (#2789)Neil Alexander
Implement behaviours for `m.read.private` receipts.
2022-10-11Tweak federation `M_NOT_FOUND` errorsNeil Alexander
2022-10-10Fix joined hosts with `RewritesState` (#2785)Neil Alexander
This ensures that the joined hosts in the federation API are correct after the state is rewritten. This might fix some races around the time of joining federated rooms.
2022-10-10Send E2EE related errors to sentry (#2784)Till
Only sends errors if we're not retrying them in NATS. Not sure if those should be scoped/tagged with something like "E2EE".
2022-10-07Fix issue with DMs shown as normal rooms (#2776)Till
Fixes #2121, test added in https://github.com/matrix-org/complement/pull/494
2022-10-06Allow specifying old signing keys with the public key and key ID only (#2770)Neil Alexander
If the private key file is lost, it's often possible to retrieve the public key from another server elsewhere, so we should make it possible to configure it in that way.
2022-10-06Tweak logging for federated room joinsNeil Alexander
2022-10-05Add `RoomEventType` nats.Header to avoid unneeded unmarshalling (#2765)Till
2022-09-29Remove `Suppressing send-to-device` log lineNeil Alexander
The behaviour is correct and I have no idea why we are logging it.