aboutsummaryrefslogtreecommitdiff
path: root/roomserver
AgeCommit message (Collapse)Author
2022-05-06Simplify `calculateLatest` (#2430)Neil Alexander
* Simplify `calculateLatest` * Comments
2022-05-06Clean up interface definitions (#2427)kegsay
* tidy up interfaces * remove unused GetCreatorIDForAlias * Add RoomserverUserAPI interface * Define more interfaces * Use AppServiceInternalAPI for consistent naming * clean up federationapi constructor a bit * Fix monolith in -http mode
2022-05-05Define component interfaces based on consumers (2/2) (#2425)kegsay
* convert remaining interfaces * Tidy up the userapi interfaces
2022-05-05Don't log consumer errors on shutdownNeil Alexander
2022-05-05Define component interfaces based on consumers (1/2) (#2423)kegsay
* Specify interfaces used by appservice, do half of clientapi * convert more deps of clientapi to finer-grained interfaces * Convert mediaapi and rest of clientapi * Somehow this got missed
2022-05-05syncapi: define specific interfaces for internal HTTP communications (#2416)kegsay
* syncapi: use finer-grained interfaces when making the syncapi * Use specific interfaces for syncapi-roomserver interactions * Define query access token api for shared http auth code
2022-05-03Global database connection pool (for monolith mode) (#2411)Neil Alexander
* Allow monolith components to share a single database pool * Don't yell about missing connection strings * Rename field * Setup tweaks * Fix panic * Improve configuration checks * Update config * Fix lint errors * Update comments
2022-04-28Add `/_dendrite/admin/evacuateRoom/{roomID}` (#2401)Neil Alexander
* Add new endpoint to allow admins to evacuate the local server from the room * Guard endpoint * Use right prefix * Auth API * More useful return error rather than a panic * More useful return value again * Update the path * Try using inputer instead * oh provide the config * Try that again * Return affected user IDs * Don't create so many forward extremities * Add missing `Path` to name Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2022-04-25Return M_NOT_FOUND for rejected events (#2371)Till
* Return M_NOT_FOUND for rejected events * Add passing tests
2022-04-08Add metrics for internal API requests (#2310)Till
* Add response size and requests total to internal handler * Move MustRegister calls to New* funcs * Move MustRegister back to init * Init at some place, minimize changes
2022-04-08Add test infrastructure code for dendrite unit/integ tests (#2331)kegsay
* Add test infrastructure code for dendrite unit/integ tests Start re-enabling some syncapi storage tests in the process. * Linting * Add postgres service to unit tests * dendrite not syncv3 * Skip test which doesn't work * Linting * Add `jetstream.PrepareForTests` Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-06Reduce the number of allocations made by `localRoomMembers` when consuming ↵Neil Alexander
stream events for push notifications (#2324)
2022-04-05Update to matrix-org/gomatrixserverlib#300Neil Alexander
2022-04-05Added /upgrade endpoint (#2307)David Spenler
* Added /upgrade endpoint * fix * Fix lints * More lint lifex * Move room upgrading to the roomserver * Remove extraneous arg * Fix HTTP API for `PerformUpgrade` * Reduce number of API calls in `generateInitialEvents`, preserve membership fields * Refactor `generateInitialEvents` to preserve old state events for all but the essential room setup events * Handle ban events in the state transfer * Refactor and comment `createTemporaryPowerLevels` * Only send two power levels if we needed to override the levels, preserve miscellaneous fields in the create event * Fix copyrights * Review comments @S7evinK * Update sytest whitelist * Specify empty state keys, use `EventLevel`, remove unnecessary check on state copy * Add comment to `restrictOldRoomPowerLevels` * Ensure canonical aliases exist before clearing * Copy invites as well as bans * Fix return error on `m.room.tombstone` handling in client API * Relax checks for well-formedness of join rules, membership event etc Co-authored-by: Alex Kursell <alex@awk.run> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: kegsay <kegan@matrix.org>
2022-03-24Update database when rejecting federated invite (#2300)S7evinK
* Actually set the DB entry to "leave" * Try to rollback in case of error * Fix linter issue
2022-03-24Fix lint errorsNeil Alexander
2022-03-23Per-room consumers (#2293)Neil Alexander
* Roomserver input refactoring — again! * Ensure the actor runs again * Preserve consumer after unsubscribe * Another sprinkling of magic * Rename `TopicFor` to `Prefixed` * Recreate the stream if the config is bad * Check streams too * Prefix subjects, preserve inboxes * Recreate if subjects wrong * Remove stream subject * Reconstruct properly * Fix mutex unlock * Comments * Fix tests * Don't drop events * Review comments * Separate `queueInputRoomEvents` function * Re-jig control flow a bit
2022-03-21Wait for safe shutdown of NATS Server (#2289)Neil Alexander
2022-03-17Assign room NIDs, event type NIDs and state key NIDs outside of database ↵Neil Alexander
transactions (#2265) * Assign room NIDs and state key NIDs outside of database transactions * In roomserver storage package too * Don't take a `txn` parameter, clean up SQLite
2022-03-17Various bug fixes and tweaks around invites and membershipNeil Alexander
2022-03-16Stream tweaks, use same codepath for sync vs async input room events, wait ↵Neil Alexander
for error response via NATS messages (#2283)
2022-03-09Fix `GET /directory/list/room/{roomID}` (#2262)Neil Alexander
* Let's try to work out why this endpoint lies * Try that again * Fix `QueryPublishedRooms` * Remove logging * Remove unnecessary change * Remove unnecessary change
2022-03-07Don't send `adds_state_events` in roomserver output events anymore (#2258)Neil Alexander
* Don't send `adds_state_events` in roomserver output events anymore * Set `omitempty` on some output fields that aren't always set * Add `AddsState` helper function * No-op if no added state event IDs * Revert "No-op if no added state event IDs" This reverts commit 71a0ef3df10e0d94234d916246c30b0a4e82b26e. * Revert "Add `AddsState` helper function" This reverts commit c9fbe45475eb12ae44d2a8da7c0fc3a002ad9819.
2022-03-07Add canonical alias support (#2236)S7evinK
* Add canonical support * Add test * Check that the send event is actually an m.room.canonical_alias Check that we got an event from the database * Update to get correct required events * Add flakey test to blacklist
2022-03-04Mark soft-failed events as rejected in `roomserver_events` (#2252)Neil Alexander
2022-03-04Fix memory leaks with SQLite prepared statements (#2253)Neil Alexander
2022-03-01Remove unnecessary error line (#2237)Neil Alexander
Previously this error line would print because we were pulling out all user memberships, but now this is no longer necessary — an event state key that we don't know will no longer get passed to `SelectJoinedUsersSetForRooms` at all.
2022-03-01MSC2946: Spaces Summary (round 2) (#2232)kegsay
* Initial cut at fixing up MSC2946 to work with latest spec * bugfix: send response back correctly * Initial working version of MSC2946 * msc2946: handle suggested_only; remove custom database As the MSC doesn't require reverse lookups, we can just pull the room state and inspect via the roomserver database. To handle this, expand QueryCurrentState to support wildcards. Use all this and handle `?suggested_only`. * Sort child rooms * msc2946: Make TestClientSpacesSummary pass * msc2946: allow invited rooms to be spidered * msc2946: support basic federation requests * fix up go mod
2022-03-01Limit `JoinedUsersSetInRooms` to interested users (#2234)Neil Alexander
* Limit database work in `JoinedUsersSetInRooms` to changed user IDs only * Comments * Fix variadic params for SQLite, update comments
2022-02-28Update `Events` call-sites which now don't return an error, update ↵Neil Alexander
`parsedRespState` to sort (#2227) * Topologically sort with `SendEventWithState`, so that earlier events should satisfy auth for later ones * Revert "Topologically sort with `SendEventWithState`, so that earlier events should satisfy auth for later ones" This reverts commit b0cd706012b4c9b6724b11e16f19c4cb732ab286. * Update to matrix-org/gomatrixserverlib#293 * `Events` no longer returns an error, other tweaks * Make sure `Events` is sorted for `parsedRespState` too
2022-02-24State resolution v2 micro-optimisations (#2226)Neil Alexander
* Don't populate duplicates into auth events * Only append the single event * Potentially reduce number of iterations in `isInAllAuthLists
2022-02-23Relax roomserver input transactional isolation (#2224)Neil Alexander
* Don't force full transactional isolation on roomserver input * Set succeeded * Tweak `MissingAuthPrevEvents`
2022-02-22Remove error when state keys are missing for user NIDs (#2213)Neil Alexander
* Remove error when state keys are missing for user NIDs There is still an actual bug here somewhere in the membership updater, but this check does more harm than good, since it means that the key consumers don't actually distribute updates to *anyone*. It's better just to deal with this silently for now. To find these broken rows: ``` SELECT * FROM roomserver_membership AS m WHERE NOT EXISTS ( SELECT event_state_key_nid FROM roomserver_event_state_keys AS s WHERE m.sender_nid = s.event_state_key_nid ); ``` * Logging
2022-02-21Return `ErrRoomNoExists` if insufficient state is available for a ↵Neil Alexander
`buildEvent` to succeed when joining a room (#2210) This may help cases like #2206, since it should prompt us to try a federated join again instead.
2022-02-21Reset invalid state snapshots for events during state storage refactor ↵Neil Alexander
migration (#2209) This should help with #2204. We can't do this for rooms, only events.
2022-02-18Implement server notices (#2180)S7evinK
* Add server_notices config * Disallow rejecting "server notice" invites * Update config * Slightly refactor sendEvent and CreateRoom so it can be reused * Implement unspecced server notices * Validate the request * Set the user api when starting * Rename function/variables * Update comments * Update config * Set the avatar on account creation * Update test * Only create the account when starting Only add routes if sever notices are enabled * Use reserver username Check that we actually got roomData * Add check for admin account Enable server notices for CI Return same values as Synapse * Add custom error for rejecting server notice invite * Move building an invite to it's own function, for reusability * Don't create new rooms, use the existing one (follow Synapse behavior) Co-authored-by: kegsay <kegan@matrix.org>
2022-02-18Remove event type and state key caches (#2200)Neil Alexander
* Don't proactively cache event types and state keys when we don't know if the transaction has persisted yet * Remove event type and state key caches altogether
2022-02-17Use process context for roomserver input (#2198)Neil Alexander
2022-02-17Don't re-send sent events in `add_state_events` (#2195)Neil Alexander
* Only add events to `add_state_events` that haven't already been sent to the roomserver output before * Filter on event NIDs instead, hopefully bring joy to SQLite * UnsentFilter, review comments
2022-02-11Ensure only one transaction is used for RS input per room (#2178)Neil Alexander
* Ensure the input API only uses a single transaction * Remove more of the dead query API call * Tidy up * Fix tests hopefully * Don't do unnecessary work for rooms that don't exist * Improve error, fix another case where transaction wasn't used properly * Add a unit test for checking single transaction on RS input API * Fix logic oops when deciding whether to use a transaction in storeEvent
2022-02-10Fix fetching missing state (#2163)Neil Alexander
* Check that we have a populated state snapshot when determining if we closed the gap * Do the same in the query API * Use HasState more opportunistically * Try to avoid falling down the hole of using a trustworthy but empty state snapshot for non-create events * Refactor missing state and make sure that we really solve the problem for the new event * Comments * Review comments * Tweak that check again * Tidy up that create check further * Fix build hopefully * Update sendOutliers to use OrderAuthAndStateEvents * Don't go out of bounds on missingEvents
2022-02-10Fix transaction issues in events table in PSQL (#2165)Neil Alexander
* Revert "Revert "Fix storage bug in PSQL events table"" This reverts commit cf447dd52a0015c2c5b10813ed11e59a3712607e. * Membership updater to use updater * Fix membership updater to use transactions properly
2022-02-09Unmarshal events at the Dendrite level not GMSL level (#2164)kegsay
* Use new event json types in gmsl * Fix EventJSON to actually unmarshal events * Update GMSL * Bump GMSL and improve error messages * Send back the correct RespState * Update GMSL
2022-02-09Revert "Fix storage bug in PSQL events table"Neil Alexander
This reverts commit b4687f2ed24ae4f397e039776118c6efee306fa9.
2022-02-09Fix storage bug in PSQL events tableNeil Alexander
2022-02-08More relaxed auth event fetching (#2161)Neil Alexander
* Tweaks around auth event fetching * More tweaking
2022-02-08Allow events to be un-rejected (#2159)Neil Alexander
* Allow un-rejecting an event later * SQL * Only un-reject, don't re-reject * Clarify ambiguous column reference
2022-02-07Fix bugs that could wedge rooms (#2154)Neil Alexander
* Don't flake so badly for rejected events * Moar * Fix panic * Don't count rejected events as missing * Don't treat rejected events without state as missing * Revert "Don't count rejected events as missing" This reverts commit 4b6139b62eb91ba059b47415b0275964b37d9b43. * Missing events should be KindOld * If we have state, use it, regardless of memberships which could be stale now * Fetch missing state for KindOld too * Tweak the condition again * Clean up a bit * Use room updater to get latest events in a race-free way * Return the correct error * Improve errors
2022-02-04Remove sarama/saramajetstream dependencies (#2138)S7evinK
* Remove dependency on saramajetstream & sarama Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Remove internal.ContinualConsumer from federationapi * Remove internal.ContinualConsumer from syncapi * Remove internal.ContinualConsumer from keyserver * Move to new Prepare function * Remove saramajetstream & sarama dependency * Delete unneeded file * Remove duplicate import * Log error instead of silently irgnoring it * Move `OffsetNewest` and `OffsetOldest` into keyserver types, change them to be more sane values * Fix comments Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-04Remove roomserver input deadlines (#2144)Neil Alexander
It isn't really clear that the deadlines actually help in any way. Currently we can use up our 2 minutes doing something, run out of context time and then return an error which causes the transaction to rollback and forgetting everything we've done. If the message came to us from NATS then we probably will end up retrying just to be in the same situation. We'd be really a lot better if we just spent the time reconciling the problem in the first place, and then we're much less likely to need to fetch those missing auth or prev events in the future. Also includes matrix-org/gomatrixserverlib#287 so we don't wait so long for servers that are obviously dead.
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-22 22:08:24 +0000