aboutsummaryrefslogtreecommitdiff
path: root/roomserver/state
AgeCommit message (Collapse)Author
2022-02-24State resolution v2 micro-optimisations (#2226)Neil Alexander
* Don't populate duplicates into auth events * Only append the single event * Potentially reduce number of iterations in `isInAllAuthLists
2022-02-04Full roomserver input transactional isolation (#2141)Neil Alexander
* Add transaction to all database tables in roomserver, rename latest events updater to room updater, use room updater for all RS input * Better transaction management * Tweak order * Handle cases where the room does not exist * Other fixes * More tweaks * Fill some gaps * Fill in the gaps * good lord it gets worse * Don't roll back transactions when events rejected * Pass through errors properly * Fix bugs * Fix incorrect error check * Don't panic on nil txns * Tweaks * Hopefully fix panics for good in SQLite this time * Fix rollback * Minor bug fixes with latest event updater * Some review comments * Revert "Some review comments" This reverts commit 0caf8cf53e62c33f7b83c52e9df1d963871f751e. * Fix a couple of bugs * Clearer commit and rollback results * Remove unnecessary prepares
2022-01-27Roomserver/federation input refactor (#2104)Neil Alexander
* Put federation client functions into their own file * Look for missing auth events in RS input * Remove retrieveMissingAuthEvents from federation API * Logging * Sorta transplanted the code over * Use event origin failing all else * Don't get stuck on mutexes: * Add verifier * Don't mark state events with zero snapshot NID as not existing * Check missing state if not an outlier before storing the event * Reject instead of soft-fail, don't copy roominfo so much * Use synchronous contexts, limit time to fetch missing events * Clean up some commented out bits * Simplify `/send` endpoint significantly * Submit async * Report errors on sending to RS input * Set max payload in NATS to 16MB * Tweak metrics * Add `workerForRoom` for tidiness * Try skipping unmarshalling errors for RespMissingEvents * Track missing prev events separately to avoid calculating state when not possible * Tweak logic around checking missing state * Care about state when checking missing prev events * Don't check missing state for create events * Try that again * Handle create events better * Send create room events as new * Use given event kind when sending auth/state events * Revert "Use given event kind when sending auth/state events" This reverts commit 089d64d271b5fca8c104e1554711187420dbebca. * Only search for missing prev events or state for new events * Tweaks * We only have missing prev if we don't supply state * Room version tweaks * Allow async inputs again * Apply backpressure to consumers/synchronous requests to hopefully stop things being overwhelmed * Set timeouts on roomserver input tasks (need to decide what timeout makes sense) * Use work queue policy, deliver all on restart * Reduce chance of duplicates being sent by NATS * Limit the number of servers we attempt to reduce backpressure * Some review comment fixes * Tidy up a couple things * Don't limit servers, randomise order using map * Some context refactoring * Update gmsl * Don't resend create events * Set stateIDs length correctly or else the roomserver thinks there are missing events when there aren't * Exclude our own servername * Try backing off servers * Make excluding self behaviour optional * Exclude self from g_m_e * Update sytest-whitelist * Update consumers for the roomserver output stream * Remember to send outliers for state returned from /gme * Make full HTTP tests less upsetti * Remove 'If a device list update goes missing, the server resyncs on the next one' from the sytest blacklist * Remove debugging test * Fix blacklist again, remove unnecessary duplicate context * Clearer contexts, don't use background in case there's something happening there * Don't queue up events more than once in memory * Correctly identify create events when checking for state * Fill in gaps again in /gme code * Remove `AuthEventIDs` from `InputRoomEvent` * Remove stray field Co-authored-by: Kegan Dougal <kegan@matrix.org>
2021-11-04Try to reduce re-allocations a bit in resolveConflictsV2Neil Alexander
2021-09-08- Removed double imports (#1989)Ryan W
- Lower cased error messages Signed-off-by: Ryan Whittington <twentybitdev@gmail.com> Co-authored-by: kegsay <kegan@matrix.org>
2021-07-19Metric fixesNeil Alexander
Squashed commit of the following: commit c6eb4d8bbf80320ec2b6d416c77659b0343e5e47 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Mon Jul 19 16:52:57 2021 +0100 Fix bug commit d420966d9ac44936728960a8d38602662b58f1c3 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Mon Jul 19 16:46:12 2021 +0100 Update metric commit 0ad6e37846e2ebbbd0e33a38274094bd15b8f11b Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Mon Jul 19 16:30:14 2021 +0100 Fix observe for calculateStateDurations
2021-07-19Set buckets for dendrite_roomserver_calculate_state_duration_microsecondsNeil Alexander
2021-07-19dendrite_roomserver_calculate_state_duration_microseconds as histogram ↵Neil Alexander
rather than summary
2021-03-03Increase gocyclo complexity to 25 (and remove all but 2 golint directives ↵Neil Alexander
related to it) (#1783)
2021-02-16Don't query for servers so often in /send (#1766)Neil Alexander
* Look up servers less often, don't hit API for missing auth events unless there are actually missing auth events * Remove ResolveConflictsAdhoc (since it is already in GMSL), other tweaks * Update gomatrixserverlib to matrix-org/gomatrixserverlib#254 * Fix resolve-state * Initialise t.servers on first use
2021-01-26Don't retrieve same state events over and over again (#1737)Neil Alexander
2021-01-18Don't create so many state snapshots when updating forward extremities (#1718)Neil Alexander
* Light-weight checking of state changes when updating forward extremities * Only do this for non-state events, since state events will always result in state change at extremities
2020-11-16Pass pointers to events — reloaded (#1583)Neil Alexander
* Pass events as pointers * Fix lint errors * Update gomatrixserverlib * Update gomatrixserverlib * Update to matrix-org/gomatrixserverlib#240
2020-10-21Refactor forward extremities (#1556)Neil Alexander
* Add resolve-state helper * Tweaks * Refactor forward extremities, again * Tweaks * Minor optimisation * Make path a bit clearer * Only process state/membership if forward extremities have changed * Usage comments in resolve-state
2020-10-05Resolve state after event against current room state when determining latest ↵Neil Alexander
state changes (#1479) * Resolve state after event against current room state when determining latest state changes * Update sytest-whitelist * Update sytest-whitelist, blacklist
2020-10-01Fix golangci-lint issues (#1464)S7evinK
* Fix S1039: unnecessary use of fmt.Sprintf * Fix S1036: unnecessary guard around map access Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-09-16Implement rejected events (#1426)Kegsay
* WIP Event rejection * Still send back errors for rejected events Instead, discard them at the federationapi /send layer rather than re-implementing checks at the clientapi/PerformJoin layer. * Implement rejected events Critically, rejected events CAN cause state resolution to happen as it can merge forks in the DAG. This is fine, _provided_ we do not add the rejected event when performing state resolution, which is what this PR does. It also fixes the error handling when NotAllowed happens, as we were checking too early and needlessly handling NotAllowed in more than one place. * Update test to match reality * Modify InputRoomEvents to no longer return an error Errors do not serialise across HTTP boundaries in polylith mode, so instead set fields on the InputRoomEventsResponse. Add `Err()` function to make the API shape basically the same. * Remove redundant returns; linting * Update blacklist
2020-09-02Expand RoomInfo to cover more DB storage functions (#1377)Kegsay
* Factor more things to RoomInfo * Factor out remaining bits for RoomInfo * Linting for now
2020-08-19Transaction writer changes, move roomserver writers (#1285)Neil Alexander
* Updated TransactionWriters, moved locks in roomserver, various other tweaks * Fix redaction deadlocks * Fix lint issue * Rename SQLiteTransactionWriter to ExclusiveTransactionWriter * Fix us not sending transactions through in latest events updater
2020-05-21Fix #897 and shuffle directory around (#1054)Kegsay
* Fix #897 and shuffle directory around * Update find-lint * goimports Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-04-29Honour history_visibility when backfilling (#990)Kegsay
* Make backfill work for shared history visibility * fetch missing state on backfill to remember snapshots correctly * Fix gmsl to not mux in auth events into room state * Whoops * Linting
2020-04-24Use a single storage.Database interface (#978)Kegsay
2020-04-09Federation for v3/v4 rooms (#954)Neil Alexander
* Update gomatrixserverlib * Default to room version 4 * Update gomatrixserverlib * Limit prev_events and auth_events * Fix auth_events, prev_events * Fix linter issues * Update gomatrixserverlib * Fix getState * Update sytest-whitelist * Squashed commit of the following: commit 067b87506357c996fd6ddb11271db9469ad4ce80 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Fri Apr 3 14:29:06 2020 +0100 Invites v2 endpoint (#952) * Start converting v1 invite endpoint to v2 * Update gomatrixserverlib * Early federationsender code for sending invites * Sending invites sorta happens now * Populate invite request with stripped state * Remodel a bit, don't reflect received invites * Handle invite_room_state * Handle room versions a bit better * Update gomatrixserverlib * Tweak order in destinationQueue.next * Revert check in processMessage * Tweak federation sender destination queue code a bit * Add comments commit 955244c09298d0e6c870377dad3af2ffa1f5e578 Author: Ben B <benne@klimlive.de> Date: Fri Apr 3 12:40:50 2020 +0200 use custom http client instead of the http DefaultClient (#823) This commit replaces the default client from the http lib with a custom one. The previously used default client doesn't come with a timeout. This could cause unwanted locks. That solution chosen here creates a http client in the base component dendrite with a constant timeout of 30 seconds. If it should be necessary to overwrite this, we could include the timeout in the dendrite configuration. Here it would be a good idea to extend the type "Address" by a timeout and create an http client for each service. Closes #820 Signed-off-by: Benedikt Bongartz <benne@klimlive.de> Co-authored-by: Kegsay <kegan@matrix.org> * Update sytest-whitelist, sytest-blacklist * Update go.mod/go.sum * Add some error wrapping for debug * Add a NOTSPEC to common/events.go * Perform state resolution at send_join * Set default room version to v2 again * Tweak GetCapabilities * Add comments to ResolveConflictsAdhoc * Update sytest-blacklist * go mod tidy * Update sytest-whitelist, sytest-blacklist * Update versions * Updates from review comments * Update sytest-blacklist, sytest-whitelist * Check room versions compatible at make_join, add some comments, update gomatrixserverlib, other tweaks * Set default room version back to v2 * Update gomatrixserverlib, sytest-whitelist
2020-03-30Ensure state res results are uniqueNeil Alexander
2020-03-19Room server changes for room versions (#930)Neil Alexander
* Rearrange state package a bit, add some code to look up the right state resolution algorithm * Remove shared * Add GetRoomVersionForRoomNID * Try to use room version to get correct state resolution algorithm * Fix room joins over federation * nolint resolveConflictsV2 because all attempts to break it up so far just result in it being awfully less obvious how it works * Rename Prepare to NewStateResolution * Update comments * Re-add missing tests
2020-03-16bugfix: Fix #908 by setting the correct state after the event (#913)Kegsay
* bugfix: Fix #908 by setting the correct state after the event Previously, this would only happen if the state already existed previously! * Structured logging
2020-02-05Implement room version capabilities in CS API (#866)Neil Alexander
* Add wiring for querying the roomserver for the default room version * Try to implement /capabilities for room versions * Update copyright notices * Update sytests, add /capabilities endpoint into CS API * Update sytest-whitelist * Add GetDefaultRoomVersion * Fix cases where state package was shadowed * Fix version formatting * Update Dockerfile to Go 1.13.6 * oh yes types I remember * And fix the default too
2020-02-05Room version abstractions (#865)Neil Alexander
* Rough first pass at adding room version abstractions * Define newer room versions * Update room version metadata * Fix roomserver/versions * Try to fix whitespace in roomsSchema
2019-05-21use go module for dependencies (#594)ruben