aboutsummaryrefslogtreecommitdiff
path: root/setup
AgeCommit message (Collapse)Author
2022-03-01Also don't send null back when the target room isn't a space roomKegan Dougal
2022-03-01Always send [] from federated rooms, not nullKegan Dougal
2022-03-01MSC2946: Spaces Summary (round 2) (#2232)kegsay
* Initial cut at fixing up MSC2946 to work with latest spec * bugfix: send response back correctly * Initial working version of MSC2946 * msc2946: handle suggested_only; remove custom database As the MSC doesn't require reverse lookups, we can just pull the room state and inspect via the roomserver database. To handle this, expand QueryCurrentState to support wildcards. Use all this and handle `?suggested_only`. * Sort child rooms * msc2946: Make TestClientSpacesSummary pass * msc2946: allow invited rooms to be spidered * msc2946: support basic federation requests * fix up go mod
2022-02-28Update `Events` call-sites which now don't return an error, update ↵Neil Alexander
`parsedRespState` to sort (#2227) * Topologically sort with `SendEventWithState`, so that earlier events should satisfy auth for later ones * Revert "Topologically sort with `SendEventWithState`, so that earlier events should satisfy auth for later ones" This reverts commit b0cd706012b4c9b6724b11e16f19c4cb732ab286. * Update to matrix-org/gomatrixserverlib#293 * `Events` no longer returns an error, other tweaks * Make sure `Events` is sorted for `parsedRespState` too
2022-02-18Implement server notices (#2180)S7evinK
* Add server_notices config * Disallow rejecting "server notice" invites * Update config * Slightly refactor sendEvent and CreateRoom so it can be reused * Implement unspecced server notices * Validate the request * Set the user api when starting * Rename function/variables * Update comments * Update config * Set the avatar on account creation * Update test * Only create the account when starting Only add routes if sever notices are enabled * Use reserver username Check that we actually got roomData * Add check for admin account Enable server notices for CI Return same values as Synapse * Add custom error for rejecting server notice invite * Move building an invite to it's own function, for reusability * Don't create new rooms, use the existing one (follow Synapse behavior) Co-authored-by: kegsay <kegan@matrix.org>
2022-02-18Merge both user API databases into one (#2186)Neil Alexander
* Merge user API databases into one * Remove DeviceDatabase from config * Fix tests * Try that again * Clean up keyserver device keys when the devices no longer exist in the user API * Tweak ordering * Fix UserExists flag, device check * Allow including empty entries so we can clean them up * Remove logging
2022-02-18Allow preventing guest registration (#2199)Neil Alexander
* Allow disabling guest registration separately * Update sample config * Set `guests_disabled` to `true` in the sample config
2022-02-17Update NATS Server to v2.7.2 (#2193)Neil Alexander
* Update NATS JetStream to v2.7.2 * Remove deprecated option
2022-02-17Remove outbound proxy, http.ProxyFromEnvironment is now used (#2191)S7evinK
2022-02-09Unmarshal events at the Dendrite level not GMSL level (#2164)kegsay
* Use new event json types in gmsl * Fix EventJSON to actually unmarshal events * Update GMSL * Bump GMSL and improve error messages * Send back the correct RespState * Update GMSL
2022-02-04Remove sarama/saramajetstream dependencies (#2138)S7evinK
* Remove dependency on saramajetstream & sarama Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Remove internal.ContinualConsumer from federationapi * Remove internal.ContinualConsumer from syncapi * Remove internal.ContinualConsumer from keyserver * Move to new Prepare function * Remove saramajetstream & sarama dependency * Delete unneeded file * Remove duplicate import * Log error instead of silently irgnoring it * Move `OffsetNewest` and `OffsetOldest` into keyserver types, change them to be more sane values * Fix comments Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-02Use pull consumers (#2140)Neil Alexander
* Pull consumers * Pull consumers * Only nuke consumers if they are push consumers * Clean up old consumers * Better error handling * Update comments
2022-01-27Roomserver/federation input refactor (#2104)Neil Alexander
* Put federation client functions into their own file * Look for missing auth events in RS input * Remove retrieveMissingAuthEvents from federation API * Logging * Sorta transplanted the code over * Use event origin failing all else * Don't get stuck on mutexes: * Add verifier * Don't mark state events with zero snapshot NID as not existing * Check missing state if not an outlier before storing the event * Reject instead of soft-fail, don't copy roominfo so much * Use synchronous contexts, limit time to fetch missing events * Clean up some commented out bits * Simplify `/send` endpoint significantly * Submit async * Report errors on sending to RS input * Set max payload in NATS to 16MB * Tweak metrics * Add `workerForRoom` for tidiness * Try skipping unmarshalling errors for RespMissingEvents * Track missing prev events separately to avoid calculating state when not possible * Tweak logic around checking missing state * Care about state when checking missing prev events * Don't check missing state for create events * Try that again * Handle create events better * Send create room events as new * Use given event kind when sending auth/state events * Revert "Use given event kind when sending auth/state events" This reverts commit 089d64d271b5fca8c104e1554711187420dbebca. * Only search for missing prev events or state for new events * Tweaks * We only have missing prev if we don't supply state * Room version tweaks * Allow async inputs again * Apply backpressure to consumers/synchronous requests to hopefully stop things being overwhelmed * Set timeouts on roomserver input tasks (need to decide what timeout makes sense) * Use work queue policy, deliver all on restart * Reduce chance of duplicates being sent by NATS * Limit the number of servers we attempt to reduce backpressure * Some review comment fixes * Tidy up a couple things * Don't limit servers, randomise order using map * Some context refactoring * Update gmsl * Don't resend create events * Set stateIDs length correctly or else the roomserver thinks there are missing events when there aren't * Exclude our own servername * Try backing off servers * Make excluding self behaviour optional * Exclude self from g_m_e * Update sytest-whitelist * Update consumers for the roomserver output stream * Remember to send outliers for state returned from /gme * Make full HTTP tests less upsetti * Remove 'If a device list update goes missing, the server resyncs on the next one' from the sytest blacklist * Remove debugging test * Fix blacklist again, remove unnecessary duplicate context * Clearer contexts, don't use background in case there's something happening there * Don't queue up events more than once in memory * Correctly identify create events when checking for state * Fill in gaps again in /gme code * Remove `AuthEventIDs` from `InputRoomEvent` * Remove stray field Co-authored-by: Kegan Dougal <kegan@matrix.org>
2022-01-25Increase maximum message size to 16MB (#2109)Neil Alexander
2022-01-07NATS JetStream tweaks (#2086)Neil Alexander
* Use named NATS durable consumers * Build fixes * Remove dupe call to SetFederationAPI * Use namespaced consumer name * Fix namespacing * Fix unit tests hopefully
2022-01-05Add NATS JetStream support (#1866)S7evinK
* Add NATS JetStream support Update shopify/sarama * Fix addresses * Don't change Addresses in Defaults * Update saramajetstream * Add missing error check Keep typing events for at least one minute * Use all configured NATS addresses * Update saramajetstream * Try setting up with NATS * Make sure NATS uses own persistent directory (TODO: make this configurable) * Update go.mod/go.sum * Jetstream package * Various other refactoring * Build fixes * Config tweaks, make random jetstream storage path for CI * Disable interest policies * Try to sane default on jetstream base path * Try to use in-memory for CI * Restore storage/retention * Update nats.go dependency * Adapt changes to config * Remove unneeded TopicFor * Dep update * Revert "Remove unneeded TopicFor" This reverts commit f5a4e4a339b6f94ec215778dca22204adaa893d1. * Revert changes made to streams * Fix build problems * Update nats-server * Update go.mod/go.sum * Roomserver input API queuing using NATS * Fix topic naming * Prometheus metrics * More refactoring to remove saramajetstream * Add missing topic * Don't try to populate map that doesn't exist * Roomserver output topic * Update go.mod/go.sum * Message acknowledgements * Ack tweaks * Try to resume transaction re-sends * Try to resume transaction re-sends * Update to matrix-org/gomatrixserverlib@91dadfb * Remove internal.PartitionStorer from components that don't consume keychanges * Try to reduce re-allocations a bit in resolveConflictsV2 * Tweak delivery options on RS input * Publish send-to-device messages into correct JetStream subject * Async and sync roomserver input * Update dendrite-config.yaml * Remove roomserver tests for now (they need rewriting) * Remove roomserver test again (was merged back in) * Update documentation * Docker updates * More Docker updates * Update Docker readme again * Fix lint issues * Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that * Go 1.16 instead of Go 1.13 for upgrade tests and Complement * Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. * Don't report any errors on `/send` to see what fun that creates * Fix panics on closed channel sends * Enforce state key matches sender * Do the same for leave * Various tweaks to make tests happier Squashed commit of the following: commit 13f9028e7a63662759ce7c55504a9d2423058668 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:47:14 2022 +0000 Do the same for leave commit e6be7f05c349fafbdddfe818337a17a60c867be1 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:33:42 2022 +0000 Enforce state key matches sender commit 85ede6d64bf10ce9b91cdd6d80f87350ee55242f Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 14:07:04 2022 +0000 Fix panics on closed channel sends commit 9755494a98bed62450f8001d8128e40481d27e15 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:38:22 2022 +0000 Don't report any errors on `/send` to see what fun that creates commit 3bb4f87b5dd56882febb4db5621db484c8789b7c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:00:26 2022 +0000 Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. commit fe2673ed7be9559eaca134424e403a4faca100b0 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 12:09:34 2022 +0000 Go 1.16 instead of Go 1.13 for upgrade tests and Complement commit 368675283fc44501f227639811bdb16dd5deef8c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 11:51:45 2022 +0000 Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that commit b028dfc08577bcf52e6cb498026e15fa5d46d07c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 10:29:08 2022 +0000 Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Merge in NATS Server v2.6.6 and nats.go v1.13 into the in-process connection fork * Add `jetstream.WithJetStreamMessage` to make ack/nak-ing less messy, use process context in consumers * Fix consumer component name in federation API * Add comment explaining where streams are defined * Tweaks to roomserver input with comments * Finish that sentence that I apparently forgot to finish in INSTALL.md * Bump version number of config to 2 * Add comments around asynchronous sends to roomserver in processEventWithMissingState * More useful error message when the config version does not match * Set version in generate-config * Fix version in config.Defaults Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-03Cherry-pick removal of unused HTTP client from #2061Neil Alexander
Co-authored-by: Tommie Gannert <tommie@gannert.se>
2021-12-03Add missing HTTP mode for userapi (#1982)S7evinK
* Add missing internal api endpoint Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing performKeyBackup endpoint * Add missing http mode for userapi * Fix failing tests * Add error checks * Fix sytest * Update startup logic for HTTP mode * Use userImpl for AS (annoying) * Don't send device list updates for appservice devices * Fix build Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-11-24Ratelimit requests to /media/r0/download|upload (#2020)S7evinK
* Add /media/r0/config handler Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add rate limiting to media api * Rename variable * Add passing tests * Don't send multiple headers Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-11-24Don't populate config defaults where it doesn't make sense (#2058)Neil Alexander
* Don't populate config defaults where it doesn't make sense * Fix dendritejs builds
2021-11-24Merge `federationapi`, `federationsender`, `signingkeyserver` components (#2055)Neil Alexander
* Initial federation sender -> federation API refactoring * Move base into own package, avoids import cycle * Fix build errors * Fix tests * Add signing key server tables * Try to fold signing key server into federation API * Fix dendritejs builds * Update embedded interfaces * Fix panic, fix lint error * Update configs, docker * Rename some things * Reuse same keyring on the implementing side * Fix federation tests, `NewBaseDendrite` can accept freeform options * Fix build * Update create_db, configs * Name tables back * Don't rename federationsender consumer for now
2021-09-10Added .well-known/matrix/server endpoint (#1988)Ryan W
* Added .well-known/matrix/server endpoint Signed-off-by: Ryan Whittington <twentybitdev@gmail.com> * Replaced tabs with spaces Signed-off-by: Ryan Whittington <twentybitdev@gmail.com>
2021-09-08- Removed double imports (#1989)Ryan W
- Lower cased error messages Signed-off-by: Ryan Whittington <twentybitdev@gmail.com> Co-authored-by: kegsay <kegan@matrix.org>
2021-08-17Cross-signing fixes, notifications via sync, federation (#1974)Neil Alexander
* Initial work on signing key update EDUs * Fix build * Produce/consume EDUs * Producer logging * Only produce key change notifications for local users * Better naming * Try to notify sync * Enable feature * Use key change topic * Don't bother verifying signatures, validate key lengths if we can, notifier fixes * Copyright notices * Remove tests from whitelist until matrix-org/sytest#1117 * Some review comment fixes * Update to matrix-org/gomatrixserverlib@f9416ac * Remove unneeded parameter
2021-08-04Cross-signing groundwork (#1953)Neil Alexander
* Cross-signing groundwork * Update to matrix-org/gomatrixserverlib#274 * Fix gobind builds, which stops unit tests in CI from yelling * Some changes from review comments * Fix build by passing in UIA * Update to matrix-org/gomatrixserverlib@bec8d22 * Process master/self-signing keys from devices call * nolint * Enum-ify the key type in the database * Process self-signing key too * Fix sanity check in device list updater * Fix check * Fix sytest, hopefully * Fix build
2021-08-03Add tracing to user API (#1948)kegsay
Use the trace version in tests so we can just implement the required API functions.
2021-07-27Key backups (1/2) : Add E2E session backup metadata tables (#1943)kegsay
* Initial key backup paths and userapi API * Fix unit tests * Add key backup table * Glue REST API to database * Linting * use writer on sqlite
2021-07-19Set MaxFileSizeBytes <= 0 to unlimited (#1876)S7evinK
* Revert "Set MaxFileSizeBytes <= 0 to "unlimited" (#1875)" This reverts commit 9ed044042156a9bdfafb4ae9830b3ed41225444e. * Actually allow unlimited upload Signed-off-by: Till Faelligen <tfaelligen@gmail.com> Co-authored-by: kegsay <kegan@matrix.org>
2021-07-16Fix DNS CacheLifetime (#1926)S7evinK
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2021-07-09Implement /_synapse/admin/v1/register (#1911)kegsay
* Implement /_synapse/admin/v1/register This is implemented identically to Synapse, so scripts which work with Synapse should work with Dendrite. ``` Test 27 POST /_synapse/admin/v1/register with shared secret... OK Test 28 POST /_synapse/admin/v1/register admin with shared secret... OK Test 29 POST /_synapse/admin/v1/register with shared secret downcases capitals... OK Test 30 POST /_synapse/admin/v1/register with shared secret disallows symbols... OK ``` Sytest however has `implementation_specific => "synapse"` which stops these tests from running. * Add missing muxes to gobind * Linting
2021-06-30Change how servers are selected for missing auth/prev events (#1892)Neil Alexander
* Change how servers are selected for missing auth/prev events * Shuffle order * Move ServersInRoomProvider into api package
2021-06-14Set MaxFileSizeBytes <= 0 to "unlimited" (#1875)S7evinK
* Set MaxFileSizeBytes < 0 to "unlimited" Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * int64 overflows later in mediaapi/routing/upload.go[doUpload] * Prevent int overflow when uploading
2021-06-07Fix SIGSEGV in IsInterestedInRoomID (#1846)bodqhrohro
* Avoid crash on non-compiled room regex Roughly fixes #1845 (actual compiling still needed) Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com> * Compile regexes for all namespaces Deadheres the regex compiling from building larger regexes for possibly exclusive namespaces only. A complete fix for #1845, so regexes for rooms namespaces and other non-whitelisted namespaces can be used more safely. Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com> * Appservice config: handle regexp parsing errors Signed-off-by: diamondburned <datutbrus@gmail.com> Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com> Co-authored-by: Kegsay <kegan@matrix.org>
2021-05-18Update MSC2946 implementation for stable spaces (#1859)Michael Telatynski
Now that MSC1772 passed FCP its identifiers have stabilised This outright drops support for experimental spaces but that's what you get for being on the bleeding edge
2021-04-07Implement OpenID module (#599) (#1812)Bruce MacDonald
* Implement OpenID module (#599) - Unrelated: change Riot references to Element in client API routing Signed-off-by: Bruce MacDonald <contact@bruce-macdonald.com> * OpenID module tweaks (#599) - specify expiry is ms rather than vague ts - add OpenID token lifetime to configuration - use Go naming conventions for the path params - store plaintext token rather than hash - remove openid table sqllite mutex * Add default OpenID token lifetime (#599) * Update dendrite-config.yaml Co-authored-by: Kegsay <kegsay@gmail.com> Co-authored-by: Kegsay <kegan@matrix.org>
2021-03-24Add Sentry support (#1803)Kegsay
* Add Sentry support * Use HTTP Sentry properly maybe * Capture panics * Log fed Sentry stuff correctly * British english linter
2021-03-08Make sure the component max open conns doesn't exceed 100Kegan Dougal
2021-03-08Make bcrypt cost configurable (#1793)Kegsay
2021-03-08Fix database default connection limits for CI (#1794)Neil Alexander
2021-03-05Use default transport for AS traffic (#1789)Neil Alexander
* Use default transport for AS traffic * Update gmsl and use default client * Remove replace * Fix go.sum * Update gomatrixserverlib * Go back to appservices managing their own HTTP clients because argh * Add missing context
2021-03-05Treat the sender_localpart as an exclusive namespace of one user (#1790)Will Hunt
2021-03-05Tweak AS registration check and AS component HTTP clients (#1785)Neil Alexander
* Tweak AS registration check * Check appservice usernames using correct function * Update sytest-whitelist * Use gomatrixserverlib.Client since that allows us to disable TLS validation using the config * Add appservice-specific client and ability to control TLS validation for appservices only * Set timeout on appservice client * Review comments * Remove dead code * Enforce LoginTypeApplicationService after all * Check correct auth type field
2021-03-03Increase gocyclo complexity to 25 (and remove all but 2 golint directives ↵Neil Alexander
related to it) (#1783)
2021-03-02Use new path for MSC2946 (#1778)Kegsay
* Use new path for MSC2946 * Update GMSL
2021-02-06Remove 100 defaultKegan Dougal
2021-02-06Increase limitKegan Dougal
2021-01-26Graceful shutdowns (#1734)Neil Alexander
* Initial graceful stop * Fix dendritejs * Use process context for outbound federation requests in destination queues * Reduce logging * Fix log level
2021-01-22NewClient and NewFederationClient updates (#1730)Neil Alexander
* Use matrix-org/gomatrixserverlib#252 * Add missing WithSkipVerify to test * Functions instead * Update gomatrixserverlib to matrix-org/gomatrixserverlib#252 * Fix disabling TLS validation
2021-01-22Gate peeking behind msc flags (#1731)Kegsay
2021-01-22Fix build errorNeil Alexander
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-19 08:42:44 +0000