aboutsummaryrefslogtreecommitdiff
path: root/roomserver/storage/tables/interface.go
AgeCommit message (Collapse)Author
2024-03-22Add getting/deleting single event report (#3344)Till
Based on https://github.com/matrix-org/dendrite/pull/3342 Adds `GET /_synapse/admin/v1/event_reports/{reportID}` and `DELETE /_synapse/admin/v1/event_reports/{reportID}`
2024-03-22Add `/_synapse/admin/v1/event_reports` endpoint (#3342)Till
Based on #3340 This adds a `/_synapse/admin/v1/event_reports` endpoint, the same Synapse has. This way existing tools also work with Dendrite. Given this is already getting huge (even though many test lines), splitting this into two PRs. (The next adds "getting one report" and "deleting reports") [skip ci]
2024-03-21Add event reporting (#3340)Till
Part of #3216 and #3226 There will be a follow up PR which is going to add the same admin endpoints Synapse has, so existing tools also work for Dendrite.
2024-03-05Query rooms with ACLs instead of all rooms (#3338)Till
This now should actually speed up startup times. This is because _many_ rooms (like DMs) don't have room ACLs, this means that we had around 95% pointless DB queries. (as queried on d.m.org)
2024-02-21Speed up start up time by batch querying ACL events (#3334)Till
This should significantly speed up start up times on servers with many rooms.
2023-08-15[pseudoID] More pseudo ID fixes (#3167)Sam Wedgwood
Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-06-14Merge SenderID & Per Room User Key work (#3109)devonh
2023-06-12Add initial support for storing user room keys (#3098)Till
2023-05-24Drop `reference_sha` column (#3083)Till
Companion PR to https://github.com/matrix-org/gomatrixserverlib/pull/383
2023-04-27Remove gmsl.HeaderedEvent (#3068)kegsay
Replaced with types.HeaderedEvent _for now_. In reality we want to move them all to gmsl.Event and only use HeaderedEvent when we _need_ to bundle the version/event ID with the event (seriailsation boundaries, and even then only when we don't have the room version). Requires https://github.com/matrix-org/gomatrixserverlib/pull/373
2023-04-19refactor: update GMSL (#3058)kegsay
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/364 Read this commit by commit to avoid going insane.
2023-02-24Refactor `StoreEvent` and create a new `RoomDatabase` interface (#2985)Till
This PR changes a few things: - It pulls out the creation of several NIDs from the `StoreEvent` function to make the functions more reusable - Uses more caching when using those NIDs to avoid DB round trips
2023-02-07Optimize `/sync` and history visibility (#2961)Till
Should fix the following issues or make a lot less worse when using Postgres: The main issue behind #2911: The client gives up after a certain time, causing a cascade of context errors, because the response couldn't be built up fast enough. This mostly happens on accounts with many rooms, due to the inefficient way we're getting recent events and current state For #2777: The queries for getting the membership events for history visibility were being executed for each room (I think 185?), resulting in a whooping 2k queries for membership events. (Getting the statesnapshot -> block nids -> actual wanted membership event) Both should now be better by: - Using a LATERAL join to get all recent events for all joined rooms in one go (TODO: maybe do the same for room summary and current state etc) - If we're lazy loading on initial syncs, we're now not getting the whole current state, just to drop the majority of it because we're lazy loading members - we add a filter to exclude membership events on the first call to `CurrentState`. - Using an optimized query to get the membership events needed to calculate history visibility --------- Co-authored-by: kegsay <kegan@matrix.org>
2023-01-19Add `/_dendrite/admin/purgeRoom/{roomID}` (#2662)Neil
This adds a new admin endpoint `/_dendrite/admin/purgeRoom/{roomID}`. It completely erases all database entries for a given room ID. The roomserver will start by clearing all data for that room and then will generate an output event to notify downstream components (i.e. the sync API and federation API) to do the same. It does not currently clear media and it is currently not implemented for SQLite since it relies on SQL array operations right now. Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2022-12-12Cleanup stale device lists for users we don't share a room with anymore (#2857)Till
The stale device lists table might contain entries for users we don't share a room with anymore. This now asks the roomserver about left users and removes those entries from the table. Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-01Optimize history visibility checks (#2848)Till
This optimizes history visibility checks by (mostly) avoiding database hits. Possibly solves https://github.com/matrix-org/dendrite/issues/2777 Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-27Add AS specific public room list endpoints (#2836)Till
Adds `PUT /_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and `DELTE /_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` support, as well as the ability to filter `/publicRooms` on networkID and including all networks.
2022-10-07Fix issue with DMs shown as normal rooms (#2776)Till
Fixes #2121, test added in https://github.com/matrix-org/complement/pull/494
2022-10-05Optimise `QuerySharedUsers` so that we can only work on local users (#2766)Neil Alexander
Otherwise the sync API key change consumer wastes a lot of time trying to wake up the notifiers for non-local users.
2022-08-18State handling tweaks (#2652)Neil Alexander
This tweaks how rejected events are handled in room state and also to not apply checks we can't complete to outliers.
2022-08-18Reprocess rejected input events (#2647)Neil Alexander
* Reprocess outliers that were previously rejected * Might as well do all events this way * More useful errors * Fix queries * Tweak condition * Don't wrap errors * Report more useful error * Flatten error on `r.Queryer.QueryStateAfterEvents` * Some more debug logging * Flatten error in `QueryRestrictedJoinAllowed` * Revert "Flatten error in `QueryRestrictedJoinAllowed`" This reverts commit 1238b4184c30e0c31ffb0f364806fa1275aba483. * Tweak `QueryStateAfterEvents` * Handle MissingStateError too * Scope to room * Clean up * Fix the error * Only apply rejection check to outliers
2022-08-05Fix linter issues (#2624)Till
* Try that again * All hail the mighty linter? * And once again * goimport all the things
2022-08-01Optimise checking other servers allowed to see events (#2596)Neil Alexander
* Try optimising checking if server is allowed to see event * Fix error * Handle case where snapshot NID is 0 * Fix query * Update SQL * Clean up `CheckServerAllowedToSeeEvent` * Not supported on SQLite * Maybe placate the unit tests * Review comments
2022-07-22Membership updater refactoring (#2541)Neil Alexander
* Membership updater refactoring * Pass in membership state * Use membership check rather than referring to state directly * Delete irrelevant membership states * We don't need the leave event after all * Tweaks * Put a log entry in that I might stand a chance of finding * Be less panicky * Tweak invite handling * Don't freak if we can't find the event NID * Use event NID from `types.Event` * Clean up * Better invite handling * Placate the almighty linter * Blacklist a Sytest which is otherwise fine under Complement for reasons I don't understand * Fix the sytest after all (thanks @S7evinK for the spot)
2022-05-16Add roomserver tests (3/4) (#2447)Till
* Add Room Aliases tests * Add Rooms table test * Move StateKeyTuplerSorter to the types package * Add StateBlock tests Some optimizations * Add State Snapshot tests Some optimization * Return []int64 and convert to pq.Int64Array for postgres * Move []types.EventNID back to rows.Next() * Update tests, rename SelectRoomIDs
2022-05-09Add roomserver tests (1/?) (#2434)Till
* Add EventJSONTable tests * Add eventJSON tests * Add EventStateKeysTable tests * Add EventTypesTable tests * Add Events Table tests Move variable declaration outside loops Switch to testify/assert for tests * Move variable declaration outside loop * Remove random data * Fix issue where the EventReferenceSHA256 is not set * Add more tests * Revert "Fix issue where the EventReferenceSHA256 is not set" This reverts commit 8ae34c4e5f78584f0edb479f5a893556d2b95d19. * Update GMSL * Add tests for duplicate entries * Test what happens if we select non-existing NIDs * Add test for non-existing eventType * Really update GMSL
2022-03-17Various bug fixes and tweaks around invites and membershipNeil Alexander
2022-03-01Limit `JoinedUsersSetInRooms` to interested users (#2234)Neil Alexander
* Limit database work in `JoinedUsersSetInRooms` to changed user IDs only * Comments * Fix variadic params for SQLite, update comments
2022-02-17Don't re-send sent events in `add_state_events` (#2195)Neil Alexander
* Only add events to `add_state_events` that haven't already been sent to the roomserver output before * Filter on event NIDs instead, hopefully bring joy to SQLite * UnsentFilter, review comments
2022-02-04Full roomserver input transactional isolation (#2141)Neil Alexander
* Add transaction to all database tables in roomserver, rename latest events updater to room updater, use room updater for all RS input * Better transaction management * Tweak order * Handle cases where the room does not exist * Other fixes * More tweaks * Fill some gaps * Fill in the gaps * good lord it gets worse * Don't roll back transactions when events rejected * Pass through errors properly * Fix bugs * Fix incorrect error check * Don't panic on nil txns * Tweaks * Hopefully fix panics for good in SQLite this time * Fix rollback * Minor bug fixes with latest event updater * Some review comments * Revert "Some review comments" This reverts commit 0caf8cf53e62c33f7b83c52e9df1d963871f751e. * Fix a couple of bugs * Clearer commit and rollback results * Remove unnecessary prepares
2021-11-22Don't persist transaction IDs in the roomserver (#2048)Neil Alexander
2021-07-22Track knocking in membership updater (#1935)Neil Alexander
* Topologically sort outliers in SendEventWithState * Knock in membership updater * Update gomatrixserverlib * Update gomatrixserverlib * Get the NID of the knock event properly for the membership updater
2021-07-21Optimise QueryServerJoinedToRoom (#1933)Neil Alexander
* Optimise checking if a server is in a room * Fix queries * Fix queries
2021-07-09Add more optimised code path for checking if we're in a room (#1909)Neil Alexander
* Add more optimised code path for checking if we're in a room * Fix database queries * Fix federation API test * Fix logging * Review comments * Make separate API call for room membership
2021-04-26State storage refactor (#1839)Neil Alexander
* Hash-deduplicated state storage (and migrations) for PostgreSQL and SQLite * Refactor droomserver database setup for migrations * Fix conflict statements * Update migration names * Set a boundary for old to new block/snapshot IDs so we don't rewrite them more than once accidentally * Create sequence if not exists * Fix boundary queries * Fix boundary queries * Use Query * Break out queries a bit * More sequence tweaks * Query parameters are not playing the game * Injection escaping may not work for CREATE SEQUENCE after all * Fix snapshot sequence name * Use boundaried IDs in SQLite too * Use IFNULL for SQLite * Use COALESCE in PostgreSQL * Review comments @Kegsay
2020-12-16Hit the database far less in Events to find room NIDs and room versions (#1643)Neil Alexander
* Hit the database far less to find room NIDs for event NIDs * Close the rows * Fix SQLite selectRoomNIDsForEventNIDsSQL * Give same treatment to room version lookups
2020-11-05Implement forgetting about rooms (#1572)S7evinK
* Add basic storage methods * Add internal api handler * Add check for forgotten room * Add /rooms/{roomID}/forget endpoint * Add missing rsAPI method * Remove unused parameters * Add passing tests Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing file * Add postgres migration * Add sqlite migration * Use Forgetter to forget room * Remove empty line * Update HTTP status codes It looks like the spec calls for these to be 400, rather than 403: https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-forget Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-09-16Implement rejected events (#1426)Kegsay
* WIP Event rejection * Still send back errors for rejected events Instead, discard them at the federationapi /send layer rather than re-implementing checks at the clientapi/PerformJoin layer. * Implement rejected events Critically, rejected events CAN cause state resolution to happen as it can merge forks in the DAG. This is fine, _provided_ we do not add the rejected event when performing state resolution, which is what this PR does. It also fixes the error handling when NotAllowed happens, as we were checking too early and needlessly handling NotAllowed in more than one place. * Update test to match reality * Modify InputRoomEvents to no longer return an error Errors do not serialise across HTTP boundaries in polylith mode, so instead set fields on the InputRoomEventsResponse. Add `Err()` function to make the API shape basically the same. * Remove redundant returns; linting * Update blacklist
2020-09-07Remove QueryBulkStateContent from current state server (#1404)Kegsay
* Remove QueryBulkStateContent from current state server Expected fail due to db impl not existing * Implement query bulk state content * Fix up rejecting invites over federation * Fix bulk content marshalling
2020-09-03Implement more CSS storage functions in roomserver (#1388)Kegsay
2020-09-03Move currentstateserver API to roomserver (#1387)Kegsay
* Move currentstateserver API to roomserver Stub out DB functions for now, nothing uses the roomserver version yet. * Allow it to startup * Implement some current-state-server storage interface functions * Add missing package
2020-09-02Expand RoomInfo to cover more DB storage functions (#1377)Kegsay
* Factor more things to RoomInfo * Factor out remaining bits for RoomInfo * Linting for now
2020-09-01Add RoomInfo metadata struct (#1367)Kegsay
* Add RoomInfo struct * Remove RoomNID and replace with RoomInfo * Bugfix and remove another needless query * nil guard
2020-08-20Roomserver per-room input parallelisation (Postgres) (#1289)Neil Alexander
* Per-room input mutex * GetMembership should use transaction when assigning state key NID * Actually use writer transactions rather than ignoring them * Limit per-room mutexes to Postgres * Flip the check in InputRoomEvents
2020-07-07Emit redacted_event from the roomserver when redactions are validated (#1186)Kegsay
* Emit redacted_event from the roomserver when redactions are validated - Consume them in the currentstateserver and act accordingly. - Add integration test for the roomserver to check that injecting `m.room.redaction` events result in `redacted_event` being emitted. * Linting * Ignore events that redact themselves
2020-07-06Implement core redaction logic (#1185)Kegsay
* Implement core redaction logic - Add a new `redactions_table.go` which tracks the mapping of the redaction event ID and the redacted event ID - Mark redactions as 'validated' when we have both events. - When redactions are validated, add `unsigned.redacted_because` and modify the `eventJSON` accordingly. Note: We currently do NOT redact the event content - it's gated behind a feature flag - until we have tested redactions a bit more. * Linting
2020-07-02Replace publicroomsapi with a combination of ↵Kegsay
clientapi/roomserver/currentstateserver (#1174) * Use content_value instead of membership * Fix build * Replace publicroomsapi with a combination of clientapi/roomserver/currentstateserver - All public rooms paths are now handled by clientapi - Requests to (un)publish rooms are sent to the roomserver via `PerformPublish` which are stored in a new `published_table.go` - Requests for public rooms are handled in clientapi by: * Fetch all room IDs which are published using `QueryPublishedRooms` on the roomserver. * Apply pagination parameters to the slice. * Do a `QueryBulkStateContent` request to the currentstateserver to pull out required state event *content* (not entire events). * Aggregate and return the chunk. Mostly but not fully implemented (DB queries on currentstateserver are missing) * Fix pq query * Make postgres work * Make sqlite work * Fix tests * Unbreak pagination tests * Linting
2020-06-26Finish implementing retiring invites (#1166)Kegsay
* Pass retired invites to the syncapi with the event ID of the invite * Implement retire invite streaming * Update whitelist
2020-05-27Convert remaining roomserver tables (#1065)Kegsay
* Convert invites table * Convert membership table * Factor out remaining functions except for *Updater structs * Remove 'implements' comments from long-forgotten interfaces. Move those comments to storage.Database
2020-05-27Convert room_aliases previous_events state_block and state_snapshot tables ↵Kegsay
(#1064) * Convert state_snapshot and state_block tables * Convert room_aliases and previous_events tables * Add missing table
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-26 18:54:38 +0000