aboutsummaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)Author
2022-09-27Fulltext implementation using Bleve (#2675)Till
Based on #2480 This actually indexes events based on their event type. They are removed from the index if we receive a `m.room.redaction` event on the `OutputRoomEvent` stream. An admin endpoint is added to reindex all existing events. Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-22Version 0.9.9 (#2732)v0.9.9Neil Alexander
Changelog and version bump.
2022-09-22Tweak `InsertMigration` to avoid logging (#2720)Till
We'd still produce logs in Postgres when trying to insert a migration we already ran. This should stop us from creating those log entries.
2022-09-20Update dependencies (#2729)Neil Alexander
This updates Dendrite dependencies.
2022-09-13Check unique constraint errors when manually inserting migrations (#2712)Till
This should avoid unnecessary logging on startup if the migration (were we need `InsertMigration`) was already executed. This now checks for "unique constraint errors" for SQLite and Postgres and fails the startup process if the migration couldn't be manually inserted for some other reason.
2022-09-12Version 0.9.8v0.9.8Neil Alexander
2022-09-09Version 0.9.7 (#2707)v0.9.7Neil Alexander
Changelog and version bump.
2022-09-09Change detection of already executed migrations (#2665)Till
This changes the detection of already executed migrations for the roomserver state block and keychange refactor. It now uses schema tables provided by the database engine to check if the column was already removed. We now also store the migration in the migrations table. This should stop e.g. Postgres from logging errors like `ERROR: column "event_nid" does not exist at character 8`.
2022-09-09Update getting pushrules, add tests, tweak pushrules (#2705)Till
This PR - adds tests for `evaluatePushrules` - removes the need for the UserAPI on the `OutputStreamEventConsumer` (for easier testing) - adds a method to get the pushrules from the database - adds a new default pushrule for `m.reaction` events (and some other tweaks)
2022-09-07Fulltext implementation incl. config (#2480)Till
This adds the main component of the fulltext search. This PR doesn't do anything yet, besides creating an empty fulltextindex folder if enabled. Indexing events is done in a separate PR.
2022-09-07Handle errors differently in the `DeviceListUpdater` (#2695)Till
`If a device list update goes missing, the server resyncs on the next one` was failing because a previous test would receive a `waitTime` of 1h, resulting in the test timing out. This now tries to handle the returned errors differently, e.g. by using the default `waitTime` of 2s. Also doesn't try further users in the list, if one of the errors would cause a longer `waitTime`.
2022-09-06Stronger default power levelsNeil Alexander
2022-09-05Tweak Sentry againNeil Alexander
2022-09-05Set Sentry user where knownNeil Alexander
2022-09-05Improve Sentry reportingNeil Alexander
2022-09-01Version 0.9.6 (#2689)v0.9.6Neil Alexander
Changelog and version bump.
2022-08-25Version 0.9.5 (#2673)v0.9.5Neil Alexander
Changelog and version bump.
2022-08-19Version 0.9.4 (#2658)v0.9.4Neil Alexander
### Fixes * A bug in the roomserver around handling rejected outliers has been fixed * Backfilled events will now use the correct history visibility where possible * The device list updater backoff has been fixed, which should reduce the number of outbound HTTP requests and `Failed to query device keys for some users` log entries for dead servers * The `/sync` endpoint will no longer incorrectly return room entries for retired invites which could cause some rooms to show up in the client "Historical" section * The `/createRoom` endpoint will now correctly populate `is_direct` in invite membership events, which may help clients to classify direct messages correctly * The `create-account` tool will now log an error if the shared secret is not set in the Dendrite config * A couple of minor bugs have been fixed in the membership lazy-loading * Queued EDUs in the federation API are now cached properly
2022-08-15Version 0.9.3v0.9.3Neil Alexander
2022-08-12Version 0.9.2 (#2638)v0.9.2Neil Alexander
2022-08-12Use `/admin/v1/register` in `create-account` (#2484)Till
* Get all account data on CompleteSync * Revert "Get all account data on CompleteSync" This reverts commit 44a3e566d8fb940b0b757aea9b8408fa19ea9f54. * Use /_synapse/admin/v1/register to create account * Linting * Linter again :) * Update docs * Use HTTP API to reset password, add option to User API `PerformPasswordUpdate` to invalidate sessions * Fix routing name * Tell me more about what went wrong * Deprecate the `-reset-password` flag, document the new API Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-08-12Protect user_interactive reads and writes with locks (#2635)Tak Wai Wong
* Protect user_interactive reads and writes with locks * Ignore golangci-lint false positive * fix lint Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
2022-08-11Generic-based internal HTTP API (#2626)Neil Alexander
* Generic-based internal HTTP API (tested out on a few endpoints in the federation API) * Add `PerformInvite` * More tweaks * Fix metric name * Fix LookupStateIDs * Lots of changes to clients * Some serverside stuff * Some error handling * Use paths as metric names * Revert "Use paths as metric names" This reverts commit a9323a6a343f5ce6461a2e5bd570fe06465f1b15. * Namespace metric names * Remove duplicate entry * Remove another duplicate entry * Tweak error handling * Some more tweaks * Update error behaviour * Some more error tweaking * Fix API path for `PerformDeleteKeys` * Fix another path * Tweak federation client proxying * Fix another path * Don't return typed nils * Some more tweaks, not that it makes any difference * Tweak federation client proxying * Maybe fix the key backup test
2022-08-05Invalidate lazyLoadCache if we're doing an initial sync (#2623)Till
* Bypass lazyLoadCache if we're doing an initial sync * Make the linter happy again? * Revert "Make the linter happy again?" This reverts commit 52a5691ba3c17c05698bcc6a13092090f27ace63. * Try that again * Invalidate LazyLoadCache on initial syncs * Remove unneeded check * Add TODO * Rename Invalite -> InvalidateLazyLoadedUser * Thanks IDE
2022-08-05Do not use `ioutil` as it is deprecated (#2625)Neil Alexander
2022-08-05Fix linter issues (#2624)Till
* Try that again * All hail the mighty linter? * And once again * goimport all the things
2022-08-03Version 0.9.1 (#2616)v0.9.1Neil Alexander
* Version 0.9.1 * Update CHANGES.md
2022-08-03Remove roominfo cache (#2615)Neil Alexander
* Remove roominfo cache It's the source of a number of race conditions which are seemingly causing bugs and CI failures. * Make the linter less sad
2022-08-01Version 0.9.0 (#2602)v0.9.0Neil Alexander
2022-07-25Update database migrations, remove goose (#2264)Till
* Add new db migration * Update migrations Remove goose * Add possibility to test direct upgrades * Try to fix WASM test * Add checks for specific migrations * Remove AddMigration Use WithTransaction Add Dendrite version to table * Fix linter issues * Update tests * Update comments, outdent if * Namespace migrations * Add direct upgrade tests, skipping over one version * Split migrations * Update go version in CI * Fix copy&paste mistake * Use contexts in migrations Co-authored-by: kegsay <kegan@matrix.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-19Add event state key cache (#2576)Neil Alexander
2022-07-18Tweak cache counters (#2575)Neil Alexander
* Tweak cache counters This makes the number of counters relative to the maximum cache size. Since the counters effectively manage the size of the bloom filter, larger caches need more counters and smaller caches need less. 10 counters per 1KB data means that the default cache size of 1GB should result in a bloom filter and TinyLRU admission set of about 16MB estimated. * Remove line left by accident
2022-07-13Pointerise `types.RoomInfo` in the cache so we can update it in-place in the ↵Neil Alexander
latest events updater
2022-07-11Ristretto cache (#2563)Neil Alexander
* Try Ristretto cache * Tweak * It's beautiful * Update GMSL * More strict keyable interface * Fix that some more * Make less panicky * Don't enforce mutability checks for now * Determine mutability using deep equality * Tweaks * Namespace keys * Make federation caches mutable * Update cost estimation, add metric * Update GMSL * Estimate cost for metrics better * Reduce counters a bit * Try caching events * Some guards * Try again * Try this * Use separate caches for hopefully better hash distribution * Fix bug with admitting events into cache * Try to fix bugs * Check nil * Try that again * Preserve order jeezo this is messy * thanks VS Code for doing exactly the wrong thing * Try this again * Be more specific * aaaaargh * One more time * That might be better * Stronger sorting * Cache expiries, async publishing of EDUs * Put it back * Use a shared cache again * Cost estimation fixes * Update ristretto * Reduce counters a bit * Clean up a bit * Update GMSL * 1GB * Configurable cache sizees * Tweaks * Add `config.DataUnit` for specifying friendly cache sizes * Various tweaks * Update GMSL * Add back some lazy loading caching * Include key in cost * Include key in cost * Tweak max age handling, config key name * Only register prometheus metrics if requested * Review comments @S7evinK * Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere) * Review comments * Update sample configs * Update GHA Workflow * Update Complement images to Go 1.18 * Remove the cache test from the federation API as we no longer guarantee immediate cache admission * Don't check the caches in the renewal test * Possibly fix the upgrade tests * Update to matrix-org/gomatrixserverlib#322 * Update documentation to refer to Go 1.18
2022-07-01Version 0.8.9 (#2549)v0.8.9Neil Alexander
* Version 0.8.9 * Update changelog
2022-06-09Version 0.8.8 (#2525)v0.8.8Neil Alexander
2022-06-07Rate limiting changes (#2519)Neil Alexander
* Rate limiting changes This makes the following changes: * For logged in users, the rate limiting now applies to the device session rather than the remote IP address; * For non-logged in users, the rate limiting continues to apply to remote address as it does today; * It is now possible to add user IDs to the `exempt_user_ids` option under `rate_limiting` to exclude bots from rate limiting; * Admin and appservice users are now exempt from rate limiting by default. * Fix build with media API
2022-06-01Version 0.8.7 (#2509)v0.8.7Neil Alexander
2022-05-26Version 0.8.6 (#2497)v0.8.6Neil Alexander
2022-05-26Upgrade various dependencies (#2496)Neil Alexander
* Upgrade various dependencies * Yaml dependency too
2022-05-17bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)kegsay
* Fix flakey sytest 'Local device key changes get to remote servers' * Debug logs * Remove internal/test and use /test only Remove a lot of ancient code too. * Use FederationRoomserverAPI in more places * Use more interfaces in federationapi; begin adding regression test * Linting * Add regression test * Unbreak tests * ALL THE LOGS * Fix a race condition which could cause events to not be sent to servers If a new room event which rewrites state arrives, we remove all joined hosts then re-calculate them. This wasn't done in a transaction so for a brief period we would have no joined hosts. During this interim, key change events which arrive would not be sent to destination servers. This would sporadically fail on sytest. * Unbreak new tests * Linting
2022-05-13Version 0.8.5 (#2461)v0.8.5Neil Alexander
* Version 0.8.5 * Update changelog * Update changelog
2022-05-10Version 0.8.4v0.8.4Neil Alexander
2022-05-10Back out matrix-org/dendrite#2421 by restoring `http.Client`sNeil Alexander
This creates problems with non-HTTPS endpoints and should fix #2444.
2022-05-09Version 0.8.3 (#2431)v0.8.3Neil Alexander
* Version 0.8.3 * Update changelog
2022-05-06Version 0.8.3rc1v0.8.3rc1Neil Alexander
2022-05-06Move LL cache (#2429)Till
2022-05-06Clean up interface definitions (#2427)kegsay
* tidy up interfaces * remove unused GetCreatorIDForAlias * Add RoomserverUserAPI interface * Define more interfaces * Use AppServiceInternalAPI for consistent naming * clean up federationapi constructor a bit * Fix monolith in -http mode
2022-05-05Define component interfaces based on consumers (1/2) (#2423)kegsay
* Specify interfaces used by appservice, do half of clientapi * convert more deps of clientapi to finer-grained interfaces * Convert mediaapi and rest of clientapi * Somehow this got missed
2022-05-05Use `gomatrixserverlib.Client` instead of `http.Client` (#2421)Neil Alexander
* Update to matrix-org/gomatrixserverlib#303 * Use `gomatrixserverlib.Client` for phone-home stats * Use `gomatrixserverlib.Client` for push notifications * Use `gomatrixserverlib.Client` for appservices * Use `gomatrixserverlib.Client` for three-PID invites