aboutsummaryrefslogtreecommitdiff
path: root/clientapi/routing
AgeCommit message (Collapse)Author
2022-05-24Fix `join_authorised_via_users_server` key name in `SendEvent`Neil Alexander
2022-05-24Give PL100 to invited users in `trusted_private_chat` preset (#2485)Neil Alexander
2022-05-24Strip `join_authorised_by_users_server` when updating membership events ↵Neil Alexander
using `/rooms/{roomID}/state` (#2482)
2022-05-11Add RoomExists flag to QueryMembershipForUser (#2450)kegsay
Fixes https://github.com/matrix-org/complement/pull/369
2022-05-06Clean up interface definitions (#2427)kegsay
* tidy up interfaces * remove unused GetCreatorIDForAlias * Add RoomserverUserAPI interface * Define more interfaces * Use AppServiceInternalAPI for consistent naming * clean up federationapi constructor a bit * Fix monolith in -http mode
2022-05-05Define component interfaces based on consumers (2/2) (#2425)kegsay
* convert remaining interfaces * Tidy up the userapi interfaces
2022-05-05Define component interfaces based on consumers (1/2) (#2423)kegsay
* Specify interfaces used by appservice, do half of clientapi * convert more deps of clientapi to finer-grained interfaces * Convert mediaapi and rest of clientapi * Somehow this got missed
2022-05-05Fix user already joined when using server notices (#2364)Till
2022-04-29Move admin functions into their own file in the client APINeil Alexander
2022-04-28Add `/_dendrite/admin/evacuateRoom/{roomID}` (#2401)Neil Alexander
* Add new endpoint to allow admins to evacuate the local server from the room * Guard endpoint * Use right prefix * Auth API * More useful return error rather than a panic * More useful return value again * Update the path * Try using inputer instead * oh provide the config * Try that again * Return affected user IDs * Don't create so many forward extremities * Add missing `Path` to name Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2022-04-26Add transactionsCache to redact endpoint (#2375)Till
2022-04-19Make sure resp.Username is defined before hashing. Fixes #2356 (#2357)fcwoknhenuxdfiyv-nextcloud
Co-authored-by: Jason Quigley <jason@onecha.net>
2022-04-08Add metrics for internal API requests (#2310)Till
* Add response size and requests total to internal handler * Move MustRegister calls to New* funcs * Move MustRegister back to init * Init at some place, minimize changes
2022-04-08Prevent setting push rules via account data endpoint (#2333)Neil Alexander
2022-04-07Send avatar updates asynchronously, same as display name updatesNeil Alexander
2022-04-07Add possibility to ignore users (#2329)Till
* Add ignore users * Ignore users in pushrules Add passing tests * Update sytest lists * Store ignore knowledge in the sync API * Fix copyrights Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-07Simplify presence stringification (should help with ↵Neil Alexander
vector-im/element-android#5712)
2022-04-06Add presence module V2 (#2312)Till
* Syncapi presence * Clientapi http presence handler * Why is this here? * Missing files * FederationAPI presence implementation * Add new presence stream * Pinecone update * Pinecone update * Add passing tests * Make linter happy * Add presence producer * Add presence config option * Set user to unavailable after x minutes * Only set currently_active if online Avoid unneeded presence updates when syncing * Tweaks * Query devices for last_active_ts Fixes & tweaks * Export SharedUsers/SharedUsers * Presence stream in MemoryStorage * Remove status_msg_nil * Fix sytest crashes * Make presence types const and use stringer for it * Change options to allow inbound/outbound presence * Fix option & typo * Update configs Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-05Added /upgrade endpoint (#2307)David Spenler
* Added /upgrade endpoint * fix * Fix lints * More lint lifex * Move room upgrading to the roomserver * Remove extraneous arg * Fix HTTP API for `PerformUpgrade` * Reduce number of API calls in `generateInitialEvents`, preserve membership fields * Refactor `generateInitialEvents` to preserve old state events for all but the essential room setup events * Handle ban events in the state transfer * Refactor and comment `createTemporaryPowerLevels` * Only send two power levels if we needed to override the levels, preserve miscellaneous fields in the create event * Fix copyrights * Review comments @S7evinK * Update sytest whitelist * Specify empty state keys, use `EventLevel`, remove unnecessary check on state copy * Add comment to `restrictOldRoomPowerLevels` * Ensure canonical aliases exist before clearing * Copy invites as well as bans * Fix return error on `m.room.tombstone` handling in client API * Relax checks for well-formedness of join rules, membership event etc Co-authored-by: Alex Kursell <alex@awk.run> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: kegsay <kegan@matrix.org>
2022-03-29Remove eduserver (#2306)S7evinK
* Move receipt sending to own JetStream producer * Move SendToDevice to producer * Remove most parts of the EDU server * Fix SendToDevice & copyrights * Move structs, cleanup EDU Server traces * Use HeadersOnly subscription * Missing file * Fix linter issues * Move consumers to own files * Rename durable consumer; Consumer cleanup * Docs/config cleanup
2022-03-28User directory for nearby Pinecone peers (P2P demo) (#2311)Neil Alexander
* User directory for nearby Pinecone peers * Fix mux routing * Use config to determine which server notices user to exclude
2022-03-28Allow stored session parameters to be overwritten in the registration ↵Neil Alexander
request (#2309) * Allow stored session parameters to be overwritten in the registration request * Remove logging * Close request body * Use `httputil.UnmarshalJSON` as that should enforce UTF-8 correctness * Return `M_NOT_JSON` on read error * Whoops, return the value of `httputil.UnmarshalJSON` * Remove redundant comment
2022-03-24Refactor appservice & client API to use userapi internal (#2290)S7evinK
* Refactor user api internal * Refactor clientapi to use internal userapi * Use internal userapi instead of user DB directly * Remove AccountDB dependency * Fix linter issues Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-07Fix appservice username check (#2223)S7evinK
* Fix appservice username check * Flakey test moved to blocklist * Move tests to blacklist
2022-03-07Add canonical alias support (#2236)S7evinK
* Add canonical support * Add test * Check that the send event is actually an m.room.canonical_alias Check that we got an event from the database * Update to get correct required events * Add flakey test to blacklist
2022-03-03Send profile updates asynchronously (#2246)Neil Alexander
2022-03-03Give more time to `TestSessionCleanUp` testsNeil Alexander
2022-03-03Only store notifications for users with pushers, de-parallelise ↵Neil Alexander
`TestSessionCleanUp` for now
2022-03-03Implement Push Notifications (#1842)Dan
* Add Pushserver component with Pushers API Co-authored-by: Tommie Gannert <tommie@gannert.se> Co-authored-by: Dan Peleg <dan@globekeeper.com> * Wire Pushserver component Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Add PushGatewayClient. The full event format is required for Sytest. * Add a pushrules module. * Change user API account creation to use the new pushrules module's defaults. Introduces "scope" as required by client API, and some small field tweaks to make some 61push Sytests pass. * Add push rules query/put API in Pushserver. This manipulates account data over User API, and fires sync messages for changes. Those sync messages should, according to an existing TODO in clientapi, be moved to userapi. Forks clientapi/producers/syncapi.go to pushserver/ for later extension. * Add clientapi routes for push rules to Pushserver. A cleanup would be to move more of the name-splitting logic into pushrules.go, to depollute routing.go. * Output rooms.join.unread_notifications in /sync. This is the read-side. Pushserver will be the write-side. * Implement pushserver/storage for notifications. * Use PushGatewayClient and the pushrules module in Pushserver's room consumer. * Use one goroutine per user to avoid locking up the entire server for one bad push gateway. * Split pushing by format. * Send one device per push. Sytest does not support coalescing multiple devices into one push. Matches Synapse. Either we change Sytest, or remove the group-by-url-and-format logic. * Write OutputNotificationData from push server. Sync API is already the consumer. * Implement read receipt consumers in Pushserver. Supports m.read and m.fully_read receipts. * Add clientapi route for /unstable/notifications. * Rename to UpsertPusher for clarity and handle pusher update * Fix linter errors * Ignore body.Close() error check * Fix push server internal http wiring * Add 40 newly passing 61push tests to whitelist * Add next 12 newly passing 61push tests to whitelist * Send notification data before notifying users in EDU server consumer * NATS JetStream * Goodbye sarama * Fix `NewStreamTokenFromString` * Consume on the correct topic for the roomserver * Don't panic, NAK instead * Move push notifications into the User API * Don't set null values since that apparently causes Element upsetti * Also set omitempty on conditions * Fix bug so that we don't override the push rules unnecessarily * Tweak defaults * Update defaults * More tweaks * Move `/notifications` onto `r0`/`v3` mux * User API will consume events and read/fully read markers from the sync API with stream positions, instead of consuming directly Co-authored-by: Piotr Kozimor <p1996k@gmail.com> Co-authored-by: Tommie Gannert <tommie@gannert.se> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-01Only allow device deletion from session UIA was initiated from (#2235)S7evinK
* Only allow device deletion if the session matches * Make the challenge response available to other packages * Remove userID, as it's not in the spec * Remove tests * Add passing test & remove obsolete config * Rename field, add comment Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-25Remember parameters on registration (#2225)S7evinK
* Remember parameters for sessions Cleanup sessions on successfully registering or after a while * Add flakey test * Update to use time.AfterFunc, add more tests * Try to drain the channel, if possible
2022-02-22Remote banned user is kicked and may not rejoin until unbanned (#2216)S7evinK
* Remote banned user is kicked and may not rejoin until unbanned * Use gmsl constant
2022-02-18Implement server notices (#2180)S7evinK
* Add server_notices config * Disallow rejecting "server notice" invites * Update config * Slightly refactor sendEvent and CreateRoom so it can be reused * Implement unspecced server notices * Validate the request * Set the user api when starting * Rename function/variables * Update comments * Update config * Set the avatar on account creation * Update test * Only create the account when starting Only add routes if sever notices are enabled * Use reserver username Check that we actually got roomData * Add check for admin account Enable server notices for CI Return same values as Synapse * Add custom error for rejecting server notice invite * Move building an invite to it's own function, for reusability * Don't create new rooms, use the existing one (follow Synapse behavior) Co-authored-by: kegsay <kegan@matrix.org>
2022-02-18Merge both user API databases into one (#2186)Neil Alexander
* Merge user API databases into one * Remove DeviceDatabase from config * Fix tests * Try that again * Clean up keyserver device keys when the devices no longer exist in the user API * Tweak ordering * Fix UserExists flag, device check * Allow including empty entries so we can clean them up * Remove logging
2022-02-18Update /whoami response to match Spec v1.2 (#2201)kegsay
Basically include `is_guest` and `device_id`. The latter is needed for https://github.com/matrix-org/complement/pull/305
2022-02-18Allow preventing guest registration (#2199)Neil Alexander
* Allow disabling guest registration separately * Update sample config * Set `guests_disabled` to `true` in the sample config
2022-02-17Listen for /v3 on CSAPI (#2197)kegsay
* Listen for /v3 on CSAPI * Docs * More docs * Rename path variable to fix key backup tests * Update routing.go Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-16Add account type (#2171)S7evinK
* Add account_type for sqlite3 * Add account_type for postgres * Remove CreateGuestAccount from interface * Add new AccountTypes & update test * Use newly added AccountType for account creation * Add migrations * Reuse type * Add AccounnType to Device, so it can be verified on requests * Rename migration, add missing update for appservices * Rename sqlite3 migration * Add missing AccountType to return value * Update sqlite migration Change allowance check on /admin/whois * Fix migration, add IS NULL * Move accountType to completeRegistration * Fix migrations * Add passing test
2022-02-10Support for `m.login.token` (#2014)tommie
* Add GOPATH to PATH in find-lint.sh. The user doesn't necessarily have it in PATH. * Refactor LoginTypePassword and Type to support m.login.token and m.login.sso. For login token: * m.login.token will require deleting the token after completeAuth has generated an access token, so a cleanup function is returned by Type.Login. * Allowing different login types will require parsing the /login body twice: first to extract the "type" and then the type-specific parsing. Thus, we will have to buffer the request JSON in /login, like UserInteractive already does. For SSO: * NewUserInteractive will have to also use GetAccountByLocalpart. It makes more sense to just pass a (narrowed-down) accountDB interface to it than adding more function pointers. Code quality: * Passing around (and down-casting) interface{} for login request types has drawbacks in terms of type-safety, and no inherent benefits. We always decode JSON anyway. Hence renaming to Type.LoginFromJSON. Code that directly uses LoginTypePassword with parsed data can still use Login. * Removed a TODO for SSO. This is already tracked in #1297. * httputil.UnmarshalJSON is useful because it returns a JSONResponse. This change is intended to have no functional changes. * Support login tokens in User API. This adds full lifecycle functions for login tokens: create, query, delete. * Support m.login.token in /login. * Fixes for PR review. * Set @matrix-org/dendrite-core as repository code owner * Return event NID from `StoreEvent`, match PSQL vs SQLite behaviour, tweak backfill persistence (#2071) Co-authored-by: kegsay <kegan@matrix.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-10Allow user to forget a room, even if they never were a member (#2166)S7evinK
* Allow user to forget a room, even if they never were a member * Return "M_UNKNOWN" as per the spec Co-authored-by: kegsay <kegan@matrix.org>
2022-01-28"Enable" remote room search (#2099)S7evinK
* "Enable" remote room search Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Update go.mod * Fix formatting
2022-01-27Roomserver/federation input refactor (#2104)Neil Alexander
* Put federation client functions into their own file * Look for missing auth events in RS input * Remove retrieveMissingAuthEvents from federation API * Logging * Sorta transplanted the code over * Use event origin failing all else * Don't get stuck on mutexes: * Add verifier * Don't mark state events with zero snapshot NID as not existing * Check missing state if not an outlier before storing the event * Reject instead of soft-fail, don't copy roominfo so much * Use synchronous contexts, limit time to fetch missing events * Clean up some commented out bits * Simplify `/send` endpoint significantly * Submit async * Report errors on sending to RS input * Set max payload in NATS to 16MB * Tweak metrics * Add `workerForRoom` for tidiness * Try skipping unmarshalling errors for RespMissingEvents * Track missing prev events separately to avoid calculating state when not possible * Tweak logic around checking missing state * Care about state when checking missing prev events * Don't check missing state for create events * Try that again * Handle create events better * Send create room events as new * Use given event kind when sending auth/state events * Revert "Use given event kind when sending auth/state events" This reverts commit 089d64d271b5fca8c104e1554711187420dbebca. * Only search for missing prev events or state for new events * Tweaks * We only have missing prev if we don't supply state * Room version tweaks * Allow async inputs again * Apply backpressure to consumers/synchronous requests to hopefully stop things being overwhelmed * Set timeouts on roomserver input tasks (need to decide what timeout makes sense) * Use work queue policy, deliver all on restart * Reduce chance of duplicates being sent by NATS * Limit the number of servers we attempt to reduce backpressure * Some review comment fixes * Tidy up a couple things * Don't limit servers, randomise order using map * Some context refactoring * Update gmsl * Don't resend create events * Set stateIDs length correctly or else the roomserver thinks there are missing events when there aren't * Exclude our own servername * Try backing off servers * Make excluding self behaviour optional * Exclude self from g_m_e * Update sytest-whitelist * Update consumers for the roomserver output stream * Remember to send outliers for state returned from /gme * Make full HTTP tests less upsetti * Remove 'If a device list update goes missing, the server resyncs on the next one' from the sytest blacklist * Remove debugging test * Fix blacklist again, remove unnecessary duplicate context * Clearer contexts, don't use background in case there's something happening there * Don't queue up events more than once in memory * Correctly identify create events when checking for state * Fill in gaps again in /gme code * Remove `AuthEventIDs` from `InputRoomEvent` * Remove stray field Co-authored-by: Kegan Dougal <kegan@matrix.org>
2022-01-05Add NATS JetStream support (#1866)S7evinK
* Add NATS JetStream support Update shopify/sarama * Fix addresses * Don't change Addresses in Defaults * Update saramajetstream * Add missing error check Keep typing events for at least one minute * Use all configured NATS addresses * Update saramajetstream * Try setting up with NATS * Make sure NATS uses own persistent directory (TODO: make this configurable) * Update go.mod/go.sum * Jetstream package * Various other refactoring * Build fixes * Config tweaks, make random jetstream storage path for CI * Disable interest policies * Try to sane default on jetstream base path * Try to use in-memory for CI * Restore storage/retention * Update nats.go dependency * Adapt changes to config * Remove unneeded TopicFor * Dep update * Revert "Remove unneeded TopicFor" This reverts commit f5a4e4a339b6f94ec215778dca22204adaa893d1. * Revert changes made to streams * Fix build problems * Update nats-server * Update go.mod/go.sum * Roomserver input API queuing using NATS * Fix topic naming * Prometheus metrics * More refactoring to remove saramajetstream * Add missing topic * Don't try to populate map that doesn't exist * Roomserver output topic * Update go.mod/go.sum * Message acknowledgements * Ack tweaks * Try to resume transaction re-sends * Try to resume transaction re-sends * Update to matrix-org/gomatrixserverlib@91dadfb * Remove internal.PartitionStorer from components that don't consume keychanges * Try to reduce re-allocations a bit in resolveConflictsV2 * Tweak delivery options on RS input * Publish send-to-device messages into correct JetStream subject * Async and sync roomserver input * Update dendrite-config.yaml * Remove roomserver tests for now (they need rewriting) * Remove roomserver test again (was merged back in) * Update documentation * Docker updates * More Docker updates * Update Docker readme again * Fix lint issues * Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that * Go 1.16 instead of Go 1.13 for upgrade tests and Complement * Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. * Don't report any errors on `/send` to see what fun that creates * Fix panics on closed channel sends * Enforce state key matches sender * Do the same for leave * Various tweaks to make tests happier Squashed commit of the following: commit 13f9028e7a63662759ce7c55504a9d2423058668 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:47:14 2022 +0000 Do the same for leave commit e6be7f05c349fafbdddfe818337a17a60c867be1 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:33:42 2022 +0000 Enforce state key matches sender commit 85ede6d64bf10ce9b91cdd6d80f87350ee55242f Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 14:07:04 2022 +0000 Fix panics on closed channel sends commit 9755494a98bed62450f8001d8128e40481d27e15 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:38:22 2022 +0000 Don't report any errors on `/send` to see what fun that creates commit 3bb4f87b5dd56882febb4db5621db484c8789b7c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:00:26 2022 +0000 Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. commit fe2673ed7be9559eaca134424e403a4faca100b0 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 12:09:34 2022 +0000 Go 1.16 instead of Go 1.13 for upgrade tests and Complement commit 368675283fc44501f227639811bdb16dd5deef8c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 11:51:45 2022 +0000 Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that commit b028dfc08577bcf52e6cb498026e15fa5d46d07c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 10:29:08 2022 +0000 Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Merge in NATS Server v2.6.6 and nats.go v1.13 into the in-process connection fork * Add `jetstream.WithJetStreamMessage` to make ack/nak-ing less messy, use process context in consumers * Fix consumer component name in federation API * Add comment explaining where streams are defined * Tweaks to roomserver input with comments * Finish that sentence that I apparently forgot to finish in INSTALL.md * Bump version number of config to 2 * Add comments around asynchronous sends to roomserver in processEventWithMissingState * More useful error message when the config version does not match * Set version in generate-config * Fix version in config.Defaults Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-03Add missing HTTP mode for userapi (#1982)S7evinK
* Add missing internal api endpoint Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing performKeyBackup endpoint * Add missing http mode for userapi * Fix failing tests * Add error checks * Fix sytest * Update startup logic for HTTP mode * Use userImpl for AS (annoying) * Don't send device list updates for appservice devices * Fix build Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-11-24Ratelimit requests to /media/r0/download|upload (#2020)S7evinK
* Add /media/r0/config handler Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add rate limiting to media api * Rename variable * Add passing tests * Don't send multiple headers Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-11-24Don't populate config defaults where it doesn't make sense (#2058)Neil Alexander
* Don't populate config defaults where it doesn't make sense * Fix dendritejs builds
2021-11-24Merge `federationapi`, `federationsender`, `signingkeyserver` components (#2055)Neil Alexander
* Initial federation sender -> federation API refactoring * Move base into own package, avoids import cycle * Fix build errors * Fix tests * Add signing key server tables * Try to fold signing key server into federation API * Fix dendritejs builds * Update embedded interfaces * Fix panic, fix lint error * Update configs, docker * Rename some things * Reuse same keyring on the implementing side * Fix federation tests, `NewBaseDendrite` can accept freeform options * Fix build * Update create_db, configs * Name tables back * Don't rename federationsender consumer for now
2021-09-08- Removed double imports (#1989)Ryan W
- Lower cased error messages Signed-off-by: Ryan Whittington <twentybitdev@gmail.com> Co-authored-by: kegsay <kegan@matrix.org>
2021-08-24Hopefully fix UIA for cross-signing setup (closes #1983) (#1986)Neil Alexander
2021-08-18Add and use M_ROOM_IN_USE (#1972)Kevin Liu
Signed-off-by: nivekuil <mail@nivekuil.com> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-26 14:21:16 +0000