aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-01-31Tweak some logging (#2130)Neil Alexander
* Modify some log levels * Update gomatrixserverlib to matrix-org/gomatrixserverlib@336334f * Update gomatrixserverlib to matrix-org/gomatrixserverlib@cde7ac8 * Demote warning about key change producer * Add more useful roomserver logging * Further tweaking
2022-01-31Revert consumer changeNeil Alexander
2022-01-31Only limit context for fetching missing auth/prev events (#2131)Neil Alexander
2022-01-28Update Sarama to fix 32-bit builds (#2120)v0.6.0Neil Alexander
2022-01-28Require Go 1.16 (#2122)Neil Alexander
2022-01-28Version 0.6 (#2117)v0.6Neil Alexander
* Bump version, release notes * Update changelog * Update changelog
2022-01-28Call hooks for outliers (#2119)Neil Alexander
* Move hook call when processing room events * Fix build * Call hooks for outliers too
2022-01-28Move hook call when processing room events (#2118)Neil Alexander
* Move hook call when processing room events * Fix build
2022-01-28Add debug logging for incoming CSAPI calls on authentication failure (#2116)kegsay
* Add debug logging for incoming CSAPI calls on authentication failure Will help to debug Complement failures, and just generally useful. * Update httpapi.go Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-01-28"Enable" remote room search (#2099)S7evinK
* "Enable" remote room search Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Update go.mod * Fix formatting
2022-01-28Don't flood Sentry with context cancelled/deadline exceeded errors (#2115)Neil Alexander
2022-01-28Upgrade dependencies (#2074)Neil Alexander
* Upgrade dependencies * Revert gjson/sjson due to panics * Revert sarama as it requires Go 1.16 * Revert quic-go as it requires Go 1.16 * Revert sarama again
2022-01-28Update gomatrixserverlibNeil Alexander
2022-01-27Try federation when backfill fails to find events in the database (#2113)Neil Alexander
* Try to backfill via federation in error cases * Cleaner retry for backfill * Simpler condition
2022-01-27Roomserver/federation input refactor (#2104)Neil Alexander
* Put federation client functions into their own file * Look for missing auth events in RS input * Remove retrieveMissingAuthEvents from federation API * Logging * Sorta transplanted the code over * Use event origin failing all else * Don't get stuck on mutexes: * Add verifier * Don't mark state events with zero snapshot NID as not existing * Check missing state if not an outlier before storing the event * Reject instead of soft-fail, don't copy roominfo so much * Use synchronous contexts, limit time to fetch missing events * Clean up some commented out bits * Simplify `/send` endpoint significantly * Submit async * Report errors on sending to RS input * Set max payload in NATS to 16MB * Tweak metrics * Add `workerForRoom` for tidiness * Try skipping unmarshalling errors for RespMissingEvents * Track missing prev events separately to avoid calculating state when not possible * Tweak logic around checking missing state * Care about state when checking missing prev events * Don't check missing state for create events * Try that again * Handle create events better * Send create room events as new * Use given event kind when sending auth/state events * Revert "Use given event kind when sending auth/state events" This reverts commit 089d64d271b5fca8c104e1554711187420dbebca. * Only search for missing prev events or state for new events * Tweaks * We only have missing prev if we don't supply state * Room version tweaks * Allow async inputs again * Apply backpressure to consumers/synchronous requests to hopefully stop things being overwhelmed * Set timeouts on roomserver input tasks (need to decide what timeout makes sense) * Use work queue policy, deliver all on restart * Reduce chance of duplicates being sent by NATS * Limit the number of servers we attempt to reduce backpressure * Some review comment fixes * Tidy up a couple things * Don't limit servers, randomise order using map * Some context refactoring * Update gmsl * Don't resend create events * Set stateIDs length correctly or else the roomserver thinks there are missing events when there aren't * Exclude our own servername * Try backing off servers * Make excluding self behaviour optional * Exclude self from g_m_e * Update sytest-whitelist * Update consumers for the roomserver output stream * Remember to send outliers for state returned from /gme * Make full HTTP tests less upsetti * Remove 'If a device list update goes missing, the server resyncs on the next one' from the sytest blacklist * Remove debugging test * Fix blacklist again, remove unnecessary duplicate context * Clearer contexts, don't use background in case there's something happening there * Don't queue up events more than once in memory * Correctly identify create events when checking for state * Fill in gaps again in /gme code * Remove `AuthEventIDs` from `InputRoomEvent` * Remove stray field Co-authored-by: Kegan Dougal <kegan@matrix.org>
2022-01-26Use std logging when running under CIKegan Dougal
2022-01-25Exclude our own server name in `GetJoinedHostsForRooms` (#2110)Neil Alexander
* Exclude our own servername * Make excluding self behaviour optional
2022-01-25Increase maximum message size to 16MB (#2109)Neil Alexander
2022-01-24Add Complement to GHA (#2108)kegsay
* Add Complement to GHA * Only run on push on master
2022-01-24Update bridge FAQ & README (#2106)S7evinK
* Update bridge FAQ Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Update README
2022-01-24Update to matrix-org/gomatrixserverlib@f3e2ef8 (matrix-org/matrix-doc#3667)Neil Alexander
2022-01-21Expand issue template (#2103)kegsay
2022-01-21Add `Forward extremities remain so even after the next events are populated ↵Neil Alexander
as outliers` to `sytest-whitelist`
2022-01-21Document log levels (#2101)kegsay
2022-01-21Update monolith-sample.conf (#2087)FORCHA
* Update monolith-sample.conf -Replaced undefined monolith value with server_name (my.hostname.com) value in reference tho ths issue https://github.com/matrix-org/dendrite/issues/2078 * Update monolith-sample.conf Changed IP to location of monolith server Co-authored-by: kegsay <kegan@matrix.org>
2022-01-21Fix #2027 by gracefully handling stub rooms (#2100)kegsay
The server ACL code on startup will grab all known rooms from the rooms_table and then call `GetStateEvent` with each found room ID to find the server ACL event. This can fail for stub rooms, which will be present in the rooms table. Previously this would result in an error being returned and the server failing to start (!). Now we just return no event for stub rooms.
2022-01-21Remodel how device list change IDs are created (#2098)kegsay
* Remodel how device list change IDs are created Previously we made them using the offset Kafka supplied. We don't run Kafka anymore, so now we make the SQL table assign the change ID via an AUTOINCREMENTing ID. Redesign the `keyserver_key_changes` table to have `UNIQUE(user_id)` so we don't accumulate key changes forevermore, we now have at most 1 row per user which contains the highest change ID. This needs a SQL migration. * Ensure we bump the change ID on sqlite * Actually read the DeviceChangeID not the Offset in synapi * Add SQL migrations * Prepare after migration; fixup dendrite-upgrade-test logging * Use higher version numbers; fix sqlite query to increment better * Default 0 on postgres * fixup postgres migration on fresh dendrite instances
2022-01-20BREAKING: Remove Partitioned Stream Positions (#2096)kegsay
* go mod tidy * Break complement to check it fails CI * Remove partitioned stream positions This was used by the device list stream position. The device list position now corresponds to the `Offset`, and the partition is always 0, in prep for removing reliance on Kafka topics for device list changes. * Linting * Migrate old style tokens to new style because element-web doesn't soft-logoout on 4xx errors on /sync
2022-01-07NATS JetStream tweaks (#2086)Neil Alexander
* Use named NATS durable consumers * Build fixes * Remove dupe call to SetFederationAPI * Use namespaced consumer name * Fix namespacing * Fix unit tests hopefully
2022-01-07Fix panic at startup if roomserver was not given federation API reference by ↵Neil Alexander
the time NATS consumes an event, tweak backpressure metrics
2022-01-06Fix #2084 - incorrect /event_auth response (#2085)kegsay
* Fix #2084 * Return early * Linting
2022-01-05Add NATS JetStream support (#1866)S7evinK
* Add NATS JetStream support Update shopify/sarama * Fix addresses * Don't change Addresses in Defaults * Update saramajetstream * Add missing error check Keep typing events for at least one minute * Use all configured NATS addresses * Update saramajetstream * Try setting up with NATS * Make sure NATS uses own persistent directory (TODO: make this configurable) * Update go.mod/go.sum * Jetstream package * Various other refactoring * Build fixes * Config tweaks, make random jetstream storage path for CI * Disable interest policies * Try to sane default on jetstream base path * Try to use in-memory for CI * Restore storage/retention * Update nats.go dependency * Adapt changes to config * Remove unneeded TopicFor * Dep update * Revert "Remove unneeded TopicFor" This reverts commit f5a4e4a339b6f94ec215778dca22204adaa893d1. * Revert changes made to streams * Fix build problems * Update nats-server * Update go.mod/go.sum * Roomserver input API queuing using NATS * Fix topic naming * Prometheus metrics * More refactoring to remove saramajetstream * Add missing topic * Don't try to populate map that doesn't exist * Roomserver output topic * Update go.mod/go.sum * Message acknowledgements * Ack tweaks * Try to resume transaction re-sends * Try to resume transaction re-sends * Update to matrix-org/gomatrixserverlib@91dadfb * Remove internal.PartitionStorer from components that don't consume keychanges * Try to reduce re-allocations a bit in resolveConflictsV2 * Tweak delivery options on RS input * Publish send-to-device messages into correct JetStream subject * Async and sync roomserver input * Update dendrite-config.yaml * Remove roomserver tests for now (they need rewriting) * Remove roomserver test again (was merged back in) * Update documentation * Docker updates * More Docker updates * Update Docker readme again * Fix lint issues * Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that * Go 1.16 instead of Go 1.13 for upgrade tests and Complement * Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. * Don't report any errors on `/send` to see what fun that creates * Fix panics on closed channel sends * Enforce state key matches sender * Do the same for leave * Various tweaks to make tests happier Squashed commit of the following: commit 13f9028e7a63662759ce7c55504a9d2423058668 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:47:14 2022 +0000 Do the same for leave commit e6be7f05c349fafbdddfe818337a17a60c867be1 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 15:33:42 2022 +0000 Enforce state key matches sender commit 85ede6d64bf10ce9b91cdd6d80f87350ee55242f Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 14:07:04 2022 +0000 Fix panics on closed channel sends commit 9755494a98bed62450f8001d8128e40481d27e15 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:38:22 2022 +0000 Don't report any errors on `/send` to see what fun that creates commit 3bb4f87b5dd56882febb4db5621db484c8789b7c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 13:00:26 2022 +0000 Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that" This reverts commit 368675283fc44501f227639811bdb16dd5deef8c. commit fe2673ed7be9559eaca134424e403a4faca100b0 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 12:09:34 2022 +0000 Go 1.16 instead of Go 1.13 for upgrade tests and Complement commit 368675283fc44501f227639811bdb16dd5deef8c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 11:51:45 2022 +0000 Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that commit b028dfc08577bcf52e6cb498026e15fa5d46d07c Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Jan 4 10:29:08 2022 +0000 Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset) * Merge in NATS Server v2.6.6 and nats.go v1.13 into the in-process connection fork * Add `jetstream.WithJetStreamMessage` to make ack/nak-ing less messy, use process context in consumers * Fix consumer component name in federation API * Add comment explaining where streams are defined * Tweaks to roomserver input with comments * Finish that sentence that I apparently forgot to finish in INSTALL.md * Bump version number of config to 2 * Add comments around asynchronous sends to roomserver in processEventWithMissingState * More useful error message when the config version does not match * Set version in generate-config * Fix version in config.Defaults Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-20Use non http API as fsAPI to avoid network requests (#2075)S7evinK
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2021-12-20Allow partial test runs (#2076)S7evinK
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2021-12-16Added std type to logging config. (#2060)Martin Ashby
* Added std type to logging level. Added demux-ing of logging to stdout/err working around https://github.com/sirupsen/logrus/issues/403 Turned off the default logging in favour of using hooks always. Added fallback, if no stdout/err logger was configured, add one automatically. This prevents unexpected lack of logging after an upgrade in case the user's configuration file is not updated. Fixes: https://github.com/matrix-org/dendrite/issues/2054 Signed-off-by: Martin Ashby <martin@ashbysoft.com> * Fix build on Windows - revert function rename SetupHookLogging Fixes https://github.com/matrix-org/dendrite/pull/2060/files#r761692681 * Revert logging formatter changes to log.go, base.go. The same formatting should be applied on both windows and unix, so it makes sense to keep it in files which are shared between both platforms. Fixes https://github.com/matrix-org/dendrite/pull/2060#discussion_r762557679 * Remove unnecessary startup logging about info level logger. Fixes https://github.com/matrix-org/dendrite/pull/2060#discussion_r763959474 Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-13Fix build errorNeil Alexander
2021-12-13Update to matrix-org/pinecone@aa2808dNeil Alexander
2021-12-13Fix keyring regressions in previous P2P demoNeil Alexander
2021-12-09Return event NID from `StoreEvent`, match PSQL vs SQLite behaviour, tweak ↵Neil Alexander
backfill persistence (#2071)
2021-12-09Set @matrix-org/dendrite-core as repository code ownerNeil Alexander
2021-12-03Cherry-pick removal of unused HTTP client from #2061Neil Alexander
Co-authored-by: Tommie Gannert <tommie@gannert.se>
2021-12-03Cherry-pick typing fix from #2061Neil Alexander
Co-authored-by: Tommie Gannert <tommie@gannert.se>
2021-12-03Add missing HTTP mode for userapi (#1982)S7evinK
* Add missing internal api endpoint Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing performKeyBackup endpoint * Add missing http mode for userapi * Fix failing tests * Add error checks * Fix sytest * Update startup logic for HTTP mode * Use userImpl for AS (annoying) * Don't send device list updates for appservice devices * Fix build Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-12-03Add missing internal routes (#2064)S7evinK
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2021-12-03Squash username to lowercase at login (#2065)nymori
Signed-off-by: Bernard Zhao <bernard.zhao.us@gmail.com>
2021-11-29Updating example to Postgres v14 (#2062)Ben Yanke
See issue #2052
2021-11-25Fixes for multiple Pinecone peersNeil Alexander
2021-11-25Fix P2P demo buildsNeil Alexander
2021-11-25Update to matrix-org/pinecone@0cc483bNeil Alexander
2021-11-25Support connecting to multiple Pinecone static peers in the P2P demos ↵Neil Alexander
(supply a comma-separated list)
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-26 01:44:28 +0000