aboutsummaryrefslogtreecommitdiff
path: root/roomserver
AgeCommit message (Collapse)Author
2022-08-12Show/hide users in user directory (#2637)Till
* CS API changes * Query remote profiles * Add passing tests * Don't create a new FullyQualifiedProfile * Handle sql.ErrNoRows Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-08-11Restore the room version cache in the roomserver internal API HTTP clientNeil Alexander
2022-08-11Implement history visibility on `/messages`, `/context`, `/sync` (#2511)Till
* Add possibility to set history_visibility and user AccountType * Add new DB queries * Add actual history_visibility changes for /messages * Add passing tests * Extract check function * Cleanup * Cleanup * Fix build on 386 * Move ApplyHistoryVisibilityFilter to internal * Move queries to topology table * Add filtering to /sync and /context Some cleanup * Add passing tests; Remove failing tests :( * Re-add passing tests * Move filtering to own function to avoid duplication * Re-add passing test * Use newly added GMSL HistoryVisibility * Update gomatrixserverlib * Set the visibility when creating events * Default to shared history visibility * Remove unused query * Update history visibility checks to use gmsl Update tests * Remove unused statement * Update migrations to set "correct" history visibility * Add method to fetch the membership at a given event * Tweaks and logging * Use actual internal rsAPI, default to shared visibility in tests * Revert "Move queries to topology table" This reverts commit 4f0d41be9c194a46379796435ce73e79203edbd6. * Remove noise/unneeded code * More cleanup * Try to optimize database requests * Fix imports * PR peview fixes/changes * Move setting history visibility to own migration, be more restrictive * Fix unit tests * Lint * Fix missing entries * Tweaks for incremental syncs * Adapt generic changes Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: kegsay <kegan@matrix.org>
2022-08-11Set default room version to 9Neil Alexander
2022-08-11Generic-based internal HTTP API (#2626)Neil Alexander
* Generic-based internal HTTP API (tested out on a few endpoints in the federation API) * Add `PerformInvite` * More tweaks * Fix metric name * Fix LookupStateIDs * Lots of changes to clients * Some serverside stuff * Some error handling * Use paths as metric names * Revert "Use paths as metric names" This reverts commit a9323a6a343f5ce6461a2e5bd570fe06465f1b15. * Namespace metric names * Remove duplicate entry * Remove another duplicate entry * Tweak error handling * Some more tweaks * Update error behaviour * Some more error tweaking * Fix API path for `PerformDeleteKeys` * Fix another path * Tweak federation client proxying * Fix another path * Don't return typed nils * Some more tweaks, not that it makes any difference * Tweak federation client proxying * Maybe fix the key backup test
2022-08-08Fix issues with migrations not getting executed (#2628)Till
* Fix issues with migrations not getting executed * Check actual postgres error * Return error if it's not "column does not exist"
2022-08-05Fix linter issues (#2624)Till
* Try that again * All hail the mighty linter? * And once again * goimport all the things
2022-08-03Try more servers when calling `/state_ids` (#2610)Neil Alexander
* Try more servers when calling `/state_ids` * More logging * Maybe fix concurrent map write * Revert "Maybe fix concurrent map write" This reverts commit da0dbb836207a911afe77e6f6d63c4809669693c. * Enforce a limit of 20s per server, 5 mins total
2022-08-03Remove roominfo cache (#2615)Neil Alexander
* Remove roominfo cache It's the source of a number of race conditions which are seemingly causing bugs and CI failures. * Make the linter less sad
2022-08-02Fix data race in `lookupMissingStateViaStateIDs`Neil Alexander
2022-08-02Various roominfo tweaks (#2607)Neil Alexander
2022-08-01De-race `types.RoomInfo` (#2600)Neil Alexander
2022-08-01Optimise checking other servers allowed to see events (#2596)Neil Alexander
* Try optimising checking if server is allowed to see event * Fix error * Handle case where snapshot NID is 0 * Fix query * Update SQL * Clean up `CheckServerAllowedToSeeEvent` * Not supported on SQLite * Maybe placate the unit tests * Review comments
2022-07-25Update database migrations, remove goose (#2264)Till
* Add new db migration * Update migrations Remove goose * Add possibility to test direct upgrades * Try to fix WASM test * Add checks for specific migrations * Remove AddMigration Use WithTransaction Add Dendrite version to table * Fix linter issues * Update tests * Update comments, outdent if * Namespace migrations * Add direct upgrade tests, skipping over one version * Split migrations * Update go version in CI * Fix copy&paste mistake * Use contexts in migrations Co-authored-by: kegsay <kegan@matrix.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-22Try to fix HTTP 500s on `/members` (#2581)Neil Alexander
2022-07-22Membership updater refactoring (#2541)Neil Alexander
* Membership updater refactoring * Pass in membership state * Use membership check rather than referring to state directly * Delete irrelevant membership states * We don't need the leave event after all * Tweaks * Put a log entry in that I might stand a chance of finding * Be less panicky * Tweak invite handling * Don't freak if we can't find the event NID * Use event NID from `types.Event` * Clean up * Better invite handling * Placate the almighty linter * Blacklist a Sytest which is otherwise fine under Complement for reasons I don't understand * Fix the sytest after all (thanks @S7evinK for the spot)
2022-07-20Fix issue with membership event_nid being 0 (#2580)Till
2022-07-19Add event state key cache (#2576)Neil Alexander
2022-07-13Pointerise `types.RoomInfo` in the cache so we can update it in-place in the ↵Neil Alexander
latest events updater
2022-07-11Ristretto cache (#2563)Neil Alexander
* Try Ristretto cache * Tweak * It's beautiful * Update GMSL * More strict keyable interface * Fix that some more * Make less panicky * Don't enforce mutability checks for now * Determine mutability using deep equality * Tweaks * Namespace keys * Make federation caches mutable * Update cost estimation, add metric * Update GMSL * Estimate cost for metrics better * Reduce counters a bit * Try caching events * Some guards * Try again * Try this * Use separate caches for hopefully better hash distribution * Fix bug with admitting events into cache * Try to fix bugs * Check nil * Try that again * Preserve order jeezo this is messy * thanks VS Code for doing exactly the wrong thing * Try this again * Be more specific * aaaaargh * One more time * That might be better * Stronger sorting * Cache expiries, async publishing of EDUs * Put it back * Use a shared cache again * Cost estimation fixes * Update ristretto * Reduce counters a bit * Clean up a bit * Update GMSL * 1GB * Configurable cache sizees * Tweaks * Add `config.DataUnit` for specifying friendly cache sizes * Various tweaks * Update GMSL * Add back some lazy loading caching * Include key in cost * Include key in cost * Tweak max age handling, config key name * Only register prometheus metrics if requested * Review comments @S7evinK * Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere) * Review comments * Update sample configs * Update GHA Workflow * Update Complement images to Go 1.18 * Remove the cache test from the federation API as we no longer guarantee immediate cache admission * Don't check the caches in the renewal test * Possibly fix the upgrade tests * Update to matrix-org/gomatrixserverlib#322 * Update documentation to refer to Go 1.18
2022-07-07Fix nil pointer access when redacting events (#2560)Till
2022-07-05Wrap error from `SnapshotNIDFromEventID`Neil Alexander
2022-07-05Return clearer error when no state NID exists for an event (#2555)Neil Alexander
2022-07-05Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)Till
* Make more use of base.BaseDendrite * Fix QuerySharedUsers if no UserIDs are supplied
2022-07-01Roomserver producers package (#2546)Neil Alexander
* Give the roomserver a producers package * Change init point * Populate ACLs API * Fix build issues * `RoomEventProducer` naming
2022-07-01Try to fix backfilling (#2548)Till
* Try to fix backfilling * Return start/end to not confuse clients * Update GMSL * Update GMSL
2022-06-29Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)Neil Alexander
* Add `evacuateUser` endpoint, use it when deactivating accounts * Populate the API * Clean up user devices when deactivating * Include invites, delete pushers
2022-06-29Return an error if trying to invite a malformed user ID (#2543)Neil Alexander
2022-06-13Handle state before, send history visibility in output (#2532)Neil Alexander
* Check state before event * Tweaks * Refactor a bit, include in output events * Don't waste time if soft failed either * Tweak control flow, comments, use GMSL history visibility type
2022-06-09Correctly redact events over federation (#2526)Till
* Ensure we check powerlevel/origin before redacting an event * Add passing test * Use pl.UserLevel * Make check more readable, also check for the sender
2022-06-07Optimise `loadAuthEvents`, add roomserver tracingNeil Alexander
2022-06-06Send tombstone to other servers when upgrading roomsNeil Alexander
2022-06-06Room upgrade tweaksNeil Alexander
Squashed commit of the following: commit 7a1568c716866594af6d0b1d561c58c96de29b20 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Mon Jun 6 15:17:49 2022 +0100 Make errors more useful commit 64befe7c9a901b00650442171660c2dc4ea575fa Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Mon Jun 6 15:02:40 2022 +0100 Tweak ordering a bit
2022-06-06Fix panic in `QueryRestrictedJoinAllowed`Neil Alexander
2022-06-01Revert #2457Neil Alexander
Squashed commit of the following: commit 2bd0daf4d61376d2dd56628eaff267b0bc63e116 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Wed Jun 1 09:55:54 2022 +0100 Revert resolving old extremities as well as new This may no longer be needed with the new state fixes and probably just burns more CPU time than is strictly necessary.
2022-06-01Fix bugs related to state resolution (#2507)Neil Alexander
* Fix bugs related to state resolution * Clean up `resolve-state` * Don't panic when entries can't be found * Ensure we have state entries for the auth events * Revert "Ensure we have state entries for the auth events" This reverts commit 9b13b7ed37f40ce6d1301d9cb423a27b0db9c897. * Revert "Revert "Ensure we have state entries for the auth events"" This reverts commit d86db197e3e317f7d64ec6722cc60533872f4617. * Fix bug * Try that again * Update gomatrixserverlib * Remove recursion from `loadAuthEvents`
2022-05-26Add support for MSC3787 and `org.matrix.msc3787` room version (update to ↵Neil Alexander
matrix-org/gomatrixserverlib#310)
2022-05-26Make logging output for state deletions a bit betterNeil Alexander
2022-05-25Try to spot state deletions when they happen (#2489)Neil Alexander
2022-05-25Restricted join support on `/make_join`, `/send_join` (#2478)Neil Alexander
* Add `QueryRestrictedJoinAllowed` * Add `Resident` flag to `QueryRestrictedJoinAllowedResponse` * Check restricted joins on federation API * Return `Restricted` to determine if the room was restricted or not * Populate `AuthorisedVia` properly * Sign the event on `/send_join`, return it in the `/send_join` response in the `"event"` key * Kick back joins with invalid authorising user IDs, use event from `"event"` key if returned in `RespSendJoin` * Use invite helper in `QueryRestrictedJoinAllowed` * Only use users with the power to invite, change error bubbling a bit * Placate the almighty linter One day I will nuke `gocyclo` from orbit and everything in the world will be much better for it. * Review comments
2022-05-17bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)kegsay
* Fix flakey sytest 'Local device key changes get to remote servers' * Debug logs * Remove internal/test and use /test only Remove a lot of ancient code too. * Use FederationRoomserverAPI in more places * Use more interfaces in federationapi; begin adding regression test * Linting * Add regression test * Unbreak tests * ALL THE LOGS * Fix a race condition which could cause events to not be sent to servers If a new room event which rewrites state arrives, we remove all joined hosts then re-calculate them. This wasn't done in a transaction so for a brief period we would have no joined hosts. During this interim, key change events which arrive would not be sent to destination servers. This would sporadically fail on sytest. * Unbreak new tests * Linting
2022-05-16Add roomserver tests (3/4) (#2447)Till
* Add Room Aliases tests * Add Rooms table test * Move StateKeyTuplerSorter to the types package * Add StateBlock tests Some optimizations * Add State Snapshot tests Some optimization * Return []int64 and convert to pq.Int64Array for postgres * Move []types.EventNID back to rows.Next() * Update tests, rename SelectRoomIDs
2022-05-13Resolve over old and new extremities (#2457)Neil Alexander
* Feed existing state into state res when calculating state from new extremities * Remove duplicates * Fix bug * Sort and unique * Update to matrix-org/gomatrixserverlib#308 * Trim the slice properly * Update gomatrixserverlib again * Update to matrix-org/gomatrixserverlib#308
2022-05-11Add RoomExists flag to QueryMembershipForUser (#2450)kegsay
Fixes https://github.com/matrix-org/complement/pull/369
2022-05-10Add roomserver tests (2/?) (#2445)Till
* Add invite table tests; move variable declarations * Add Membership table tests * Move variable declarations * Add PrevEvents table tests * Add Published table test * Add Redactions tests Fix bug in SQLite markRedactionValidatedSQL * PR comments, better readability for invite tests
2022-05-09Federation consumer `adds_state_event_ids` tweak (#2441)Neil Alexander
* Don't ask roomserver for events we already have in federation API * Check number of events returned is as expected * Preallocate array * Improve shape a bit
2022-05-09Add roomserver tests (1/?) (#2434)Till
* Add EventJSONTable tests * Add eventJSON tests * Add EventStateKeysTable tests * Add EventTypesTable tests * Add Events Table tests Move variable declaration outside loops Switch to testify/assert for tests * Move variable declaration outside loop * Remove random data * Fix issue where the EventReferenceSHA256 is not set * Add more tests * Revert "Fix issue where the EventReferenceSHA256 is not set" This reverts commit 8ae34c4e5f78584f0edb479f5a893556d2b95d19. * Update GMSL * Add tests for duplicate entries * Test what happens if we select non-existing NIDs * Add test for non-existing eventType * Really update GMSL
2022-05-09One NATS instance per `BaseDendrite` (#2438)Neil Alexander
* One NATS instance per `BaseDendrite` * Fix roomserver
2022-05-06Simplify `calculateLatest` (#2430)Neil Alexander
* Simplify `calculateLatest` * Comments
2022-05-06Clean up interface definitions (#2427)kegsay
* tidy up interfaces * remove unused GetCreatorIDForAlias * Add RoomserverUserAPI interface * Define more interfaces * Use AppServiceInternalAPI for consistent naming * clean up federationapi constructor a bit * Fix monolith in -http mode
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-18 09:20:12 +0000