aboutsummaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)Author
2022-05-26Version 0.8.6 (#2497)v0.8.6Neil Alexander
2022-05-26Upgrade various dependencies (#2496)Neil Alexander
* Upgrade various dependencies * Yaml dependency too
2022-05-17bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)kegsay
* Fix flakey sytest 'Local device key changes get to remote servers' * Debug logs * Remove internal/test and use /test only Remove a lot of ancient code too. * Use FederationRoomserverAPI in more places * Use more interfaces in federationapi; begin adding regression test * Linting * Add regression test * Unbreak tests * ALL THE LOGS * Fix a race condition which could cause events to not be sent to servers If a new room event which rewrites state arrives, we remove all joined hosts then re-calculate them. This wasn't done in a transaction so for a brief period we would have no joined hosts. During this interim, key change events which arrive would not be sent to destination servers. This would sporadically fail on sytest. * Unbreak new tests * Linting
2022-05-13Version 0.8.5 (#2461)v0.8.5Neil Alexander
* Version 0.8.5 * Update changelog * Update changelog
2022-05-10Version 0.8.4v0.8.4Neil Alexander
2022-05-10Back out matrix-org/dendrite#2421 by restoring `http.Client`sNeil Alexander
This creates problems with non-HTTPS endpoints and should fix #2444.
2022-05-09Version 0.8.3 (#2431)v0.8.3Neil Alexander
* Version 0.8.3 * Update changelog
2022-05-06Version 0.8.3rc1v0.8.3rc1Neil Alexander
2022-05-06Move LL cache (#2429)Till
2022-05-06Clean up interface definitions (#2427)kegsay
* tidy up interfaces * remove unused GetCreatorIDForAlias * Add RoomserverUserAPI interface * Define more interfaces * Use AppServiceInternalAPI for consistent naming * clean up federationapi constructor a bit * Fix monolith in -http mode
2022-05-05Define component interfaces based on consumers (1/2) (#2423)kegsay
* Specify interfaces used by appservice, do half of clientapi * convert more deps of clientapi to finer-grained interfaces * Convert mediaapi and rest of clientapi * Somehow this got missed
2022-05-05Use `gomatrixserverlib.Client` instead of `http.Client` (#2421)Neil Alexander
* Update to matrix-org/gomatrixserverlib#303 * Use `gomatrixserverlib.Client` for phone-home stats * Use `gomatrixserverlib.Client` for push notifications * Use `gomatrixserverlib.Client` for appservices * Use `gomatrixserverlib.Client` for three-PID invites
2022-05-05syncapi: define specific interfaces for internal HTTP communications (#2416)kegsay
* syncapi: use finer-grained interfaces when making the syncapi * Use specific interfaces for syncapi-roomserver interactions * Define query access token api for shared http auth code
2022-05-03Global database connection pool (for monolith mode) (#2411)Neil Alexander
* Allow monolith components to share a single database pool * Don't yell about missing connection strings * Rename field * Setup tweaks * Fix panic * Improve configuration checks * Update config * Fix lint errors * Update comments
2022-04-27Version 0.8.2 (#2386)v0.8.2Neil Alexander
* Version 0.8.2 * Correct account data position mapping * Try that again * Don't duplicate wait-for-shutdowns
2022-04-19Change `pushkey_ts` to be seconds (fix #2354) (#2358)Neil Alexander
2022-04-19Implement lazy loading on `/sync` (#2346)Till
* Initial work on lazyloading * Partially implement lazy loading on /sync * Rename methods * Make missing tests pass * Preallocate slice, even if it will end up with fewer values * Let the cache handle the user mapping * Linter * Cap cache growth
2022-04-08Add metrics for internal API requests (#2310)Till
* Add response size and requests total to internal handler * Move MustRegister calls to New* funcs * Move MustRegister back to init * Init at some place, minimize changes
2022-04-07Dendrite 0.8.1v0.8.1Neil Alexander
2022-04-07Version 0.8.0 (#2330)v0.8.0Neil Alexander
* Version 0.8.0 * Update changelog again
2022-04-07Add possibility to ignore users (#2329)Till
* Add ignore users * Ignore users in pushrules Add passing tests * Update sytest lists * Store ignore knowledge in the sync API * Fix copyrights Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-04remove obsolete config entry "federation_certificates" (#2318)Sebastian Spaeth
* Remove all federation_certificates occurencs This configuration value has not been used since 2019 apparently, and indeed it is never really used in the code base. So remove all traces of it from the various configuration files. Also remove the unused variable FederationCertificatePaths Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de> * setup/config/config_test.go: remove federation_sender config snippet The federation_sender: section was folded into the federation_api some time ago, and this seems to be the only leftover in the code base. So remove it.
2022-03-29Remove eduserver (#2306)S7evinK
* Move receipt sending to own JetStream producer * Move SendToDevice to producer * Remove most parts of the EDU server * Fix SendToDevice & copyrights * Move structs, cleanup EDU Server traces * Use HeadersOnly subscription * Missing file * Fix linter issues * Move consumers to own files * Rename durable consumer; Consumer cleanup * Docs/config cleanup
2022-03-25Version 0.7.0 (#2304)v0.7.0Neil Alexander
2022-03-25Healthcheck endpoints (#2303)Neil Alexander
* Health monitoring endpoints * Rename endpoints
2022-03-21Remove unused partition_offset_table (#2288)S7evinK
2022-03-04Version 0.6.5 (#2254)v0.6.5Neil Alexander
* Version and changelog * Update changelog * Update changelog * Update readme * Update readme some more * Fix date in changelog
2022-03-03Implement Push Notifications (#1842)Dan
* Add Pushserver component with Pushers API Co-authored-by: Tommie Gannert <tommie@gannert.se> Co-authored-by: Dan Peleg <dan@globekeeper.com> * Wire Pushserver component Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> * Add PushGatewayClient. The full event format is required for Sytest. * Add a pushrules module. * Change user API account creation to use the new pushrules module's defaults. Introduces "scope" as required by client API, and some small field tweaks to make some 61push Sytests pass. * Add push rules query/put API in Pushserver. This manipulates account data over User API, and fires sync messages for changes. Those sync messages should, according to an existing TODO in clientapi, be moved to userapi. Forks clientapi/producers/syncapi.go to pushserver/ for later extension. * Add clientapi routes for push rules to Pushserver. A cleanup would be to move more of the name-splitting logic into pushrules.go, to depollute routing.go. * Output rooms.join.unread_notifications in /sync. This is the read-side. Pushserver will be the write-side. * Implement pushserver/storage for notifications. * Use PushGatewayClient and the pushrules module in Pushserver's room consumer. * Use one goroutine per user to avoid locking up the entire server for one bad push gateway. * Split pushing by format. * Send one device per push. Sytest does not support coalescing multiple devices into one push. Matches Synapse. Either we change Sytest, or remove the group-by-url-and-format logic. * Write OutputNotificationData from push server. Sync API is already the consumer. * Implement read receipt consumers in Pushserver. Supports m.read and m.fully_read receipts. * Add clientapi route for /unstable/notifications. * Rename to UpsertPusher for clarity and handle pusher update * Fix linter errors * Ignore body.Close() error check * Fix push server internal http wiring * Add 40 newly passing 61push tests to whitelist * Add next 12 newly passing 61push tests to whitelist * Send notification data before notifying users in EDU server consumer * NATS JetStream * Goodbye sarama * Fix `NewStreamTokenFromString` * Consume on the correct topic for the roomserver * Don't panic, NAK instead * Move push notifications into the User API * Don't set null values since that apparently causes Element upsetti * Also set omitempty on conditions * Fix bug so that we don't override the push rules unnecessarily * Tweak defaults * Update defaults * More tweaks * Move `/notifications` onto `r0`/`v3` mux * User API will consume events and read/fully read markers from the sync API with stream positions, instead of consuming directly Co-authored-by: Piotr Kozimor <p1996k@gmail.com> Co-authored-by: Tommie Gannert <tommie@gannert.se> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-01Set max age of 5 minutes for spaces summary cacheNeil Alexander
2022-03-01Allow specifying max age for caches (#2239)Neil Alexander
* Allow specifying max age for caches * Evict cache entry if it's found to be stale when we call Get * Fix bugs
2022-03-01msc2946: add federation cache (#2238)kegsay
2022-02-22Version 0.6.4 (#2212)v0.6.4Neil Alexander
* Version 0.6.4 * Tweaks * Update changelog * Update changelog one last time
2022-02-18Merge both user API databases into one (#2186)Neil Alexander
* Merge user API databases into one * Remove DeviceDatabase from config * Fix tests * Try that again * Clean up keyserver device keys when the devices no longer exist in the user API * Tweak ordering * Fix UserExists flag, device check * Allow including empty entries so we can clean them up * Remove logging
2022-02-18Remove event type and state key caches (#2200)Neil Alexander
* Don't proactively cache event types and state keys when we don't know if the transaction has persisted yet * Remove event type and state key caches altogether
2022-02-10Version 0.6.3 (#2170)v0.6.3Neil Alexander
2022-02-04Version 0.6.2v0.6.2Neil Alexander
2022-02-04Version 0.6.1 (#2145)v0.6.1Neil Alexander
2022-02-04Remove sarama/saramajetstream dependencies (#2138)S7evinK
* Remove dependency on saramajetstream & sarama Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Remove internal.ContinualConsumer from federationapi * Remove internal.ContinualConsumer from syncapi * Remove internal.ContinualConsumer from keyserver * Move to new Prepare function * Remove saramajetstream & sarama dependency * Delete unneeded file * Remove duplicate import * Log error instead of silently irgnoring it * Move `OffsetNewest` and `OffsetOldest` into keyserver types, change them to be more sane values * Fix comments Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-01Support CA certificates in CI (#2136)kegsay
* Support CA setting in generate-keys * Set DNS names correctly * Use generate-config -server not sed
2022-01-28Version 0.6 (#2117)v0.6Neil Alexander
* Bump version, release notes * Update changelog * Update changelog
2022-01-28Add debug logging for incoming CSAPI calls on authentication failure (#2116)kegsay
* Add debug logging for incoming CSAPI calls on authentication failure Will help to debug Complement failures, and just generally useful. * Update httpapi.go Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-01-05Add NATS JetStream support (#1866)S7evinK
* Add NATS JetStream support Update shopify/sarama * Fix addresses * Don't change Addresses in Defaults * Update saramajetstream * Add missing error check Keep typing events for at least one minute * Use all configured NATS addresses * Update saramajetstream * Try setting up with NATS * Make sure NATS uses own persistent directory (TODO: make this configurable) * Update go.mod/go.sum * Jetstream package * Various other refactoring * Build fixes * Config tweaks, make random jetstream storage path for CI * Disable interest policies * Try to sane default on jetstream base path * Try to use in-memory for CI * Restore storage/retention * Update nats.go dependency * Adapt changes to config * Remove unneeded TopicFor * Dep update * Revert "Remove unneeded TopicFor" This reverts commit f5a4e4a339b6f94ec215778dca22204adaa893d1. * Revert changes made to streams * Fix build problems * Update nats-server * Update go.mod/go.sum * Roomserver input API queuing using NATS * Fix topic naming * Prometheus metrics * More refactoring to remove saramajetstream * Add missing topic * Don't try to populate map that doesn't exist * Roomserver output topic * Update go.mod/go.sum * Message acknowledgements * Ack tweaks * Try to resume transaction re-sends * Try to resume transaction re-sends * Update to matrix-org/gomatrixserverlib@91dadfb * Remove internal.PartitionStorer from components that don't consume keychanges * Try to reduce re-allocations a bit in resolveConflictsV2 * Tweak delivery options on RS input * Publish send-to-device messages into correct JetStream subject * Async and sync roomserver input * Update dendrite-config.yaml * Remove roomserver tests for now (they need rewriting) * Remove roomserver test again (was merged back in) * Update documentation * Docker updates * More Docker updates * Update Docker readme again * Fix lint issues * Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that * Go 1.16 instead of Go 1.13 for upgrade tests and Complement * Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. * Don't report any errors on `/send` to see what fun that creates * Fix panics on closed channel sends * Enforce state key matches sender * Do the same for leave * Various tweaks to make tests happier Squashed commit of the following: commit 13f9028e7a63662759ce7c55504a9d2423058668 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:47:14 2022 +0000 Do the same for leave commit e6be7f05c349fafbdddfe818337a17a60c867be1 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:33:42 2022 +0000 Enforce state key matches sender commit 85ede6d64bf10ce9b91cdd6d80f87350ee55242f Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 14:07:04 2022 +0000 Fix panics on closed channel sends commit 9755494a98bed62450f8001d8128e40481d27e15 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:38:22 2022 +0000 Don't report any errors on `/send` to see what fun that creates commit 3bb4f87b5dd56882febb4db5621db484c8789b7c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:00:26 2022 +0000 Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. commit fe2673ed7be9559eaca134424e403a4faca100b0 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 12:09:34 2022 +0000 Go 1.16 instead of Go 1.13 for upgrade tests and Complement commit 368675283fc44501f227639811bdb16dd5deef8c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 11:51:45 2022 +0000 Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that commit b028dfc08577bcf52e6cb498026e15fa5d46d07c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 10:29:08 2022 +0000 Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Merge in NATS Server v2.6.6 and nats.go v1.13 into the in-process connection fork * Add `jetstream.WithJetStreamMessage` to make ack/nak-ing less messy, use process context in consumers * Fix consumer component name in federation API * Add comment explaining where streams are defined * Tweaks to roomserver input with comments * Finish that sentence that I apparently forgot to finish in INSTALL.md * Bump version number of config to 2 * Add comments around asynchronous sends to roomserver in processEventWithMissingState * More useful error message when the config version does not match * Set version in generate-config * Fix version in config.Defaults Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-16Added std type to logging config. (#2060)Martin Ashby
* Added std type to logging level. Added demux-ing of logging to stdout/err working around https://github.com/sirupsen/logrus/issues/403 Turned off the default logging in favour of using hooks always. Added fallback, if no stdout/err logger was configured, add one automatically. This prevents unexpected lack of logging after an upgrade in case the user's configuration file is not updated. Fixes: https://github.com/matrix-org/dendrite/issues/2054 Signed-off-by: Martin Ashby <martin@ashbysoft.com> * Fix build on Windows - revert function rename SetupHookLogging Fixes https://github.com/matrix-org/dendrite/pull/2060/files#r761692681 * Revert logging formatter changes to log.go, base.go. The same formatting should be applied on both windows and unix, so it makes sense to keep it in files which are shared between both platforms. Fixes https://github.com/matrix-org/dendrite/pull/2060#discussion_r762557679 * Remove unnecessary startup logging about info level logger. Fixes https://github.com/matrix-org/dendrite/pull/2060#discussion_r763959474 Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-03Add missing HTTP mode for userapi (#1982)S7evinK
* Add missing internal api endpoint Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing performKeyBackup endpoint * Add missing http mode for userapi * Fix failing tests * Add error checks * Fix sytest * Update startup logic for HTTP mode * Use userImpl for AS (annoying) * Don't send device list updates for appservice devices * Fix build Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-11-24Ratelimit requests to /media/r0/download|upload (#2020)S7evinK
* Add /media/r0/config handler Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add rate limiting to media api * Rename variable * Add passing tests * Don't send multiple headers Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-11-24Don't populate config defaults where it doesn't make sense (#2058)Neil Alexander
* Don't populate config defaults where it doesn't make sense * Fix dendritejs builds
2021-11-24Merge `federationapi`, `federationsender`, `signingkeyserver` components (#2055)Neil Alexander
* Initial federation sender -> federation API refactoring * Move base into own package, avoids import cycle * Fix build errors * Fix tests * Add signing key server tables * Try to fold signing key server into federation API * Fix dendritejs builds * Update embedded interfaces * Fix panic, fix lint error * Update configs, docker * Rename some things * Reuse same keyring on the implementing side * Fix federation tests, `NewBaseDendrite` can accept freeform options * Fix build * Update create_db, configs * Name tables back * Don't rename federationsender consumer for now
2021-11-16Version 0.5.1 (#2047)v0.5.1Neil Alexander
* Version bump, changelog * Update to matrix-org/pinecone@8c7a1ad * Update to matrix-org/pinecone@febf350
2021-11-02Fix building on Windows (#2008)S7evinK
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2021-11-02Run gofmt on dendrite - apply go 1.17 preferred build tags (#2021)PiotrKozimor
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-02 22:03:03 +0000