aboutsummaryrefslogtreecommitdiff
path: root/setup
AgeCommit message (Collapse)Author
2022-08-05Do not use `ioutil` as it is deprecated (#2625)Neil Alexander
2022-08-05Fix linter issues (#2624)Till
* Try that again * All hail the mighty linter? * And once again * goimport all the things
2022-08-05Add race testing to tests, and fix a few small race conditions in the tests ↵Brian Meek
(#2587) * Add race testing to tests, and fix a few small race conditions in the tests * Enable run-sytest on MacOS * Remove deadlock detecting mutex, per code review feedback * Remove autoformatting related changes and a closure that is not needed * Adjust to importing nats client as 'natsclient' Signed-off-by: Brian Meek <brian@hntlabs.com> * Clarify the use of gooseMutex to proect goose internal state Signed-off-by: Brian Meek <brian@hntlabs.com> * Remove no longer needed mutex for guarding goose Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-08-02Disable NATS Server logging, allow self-signed certificates (#2605)Till
* Disable NATS Server logs in CI * Add option to disable TLS validation for NATS
2022-08-01Set CORS headers for HTTP 404 and 405 errors (#2599)Neil Alexander
* Set CORS headers for the 404s * Use custom handlers, plus one for HTTP 405 too * Tweak setup * Add to muxes too * Tidy up some more * Use built-in HTTP 404 handler * Don't bother setting it for federation-facing
2022-07-25Remove `room_id` field from MSC2946 stripped events (closes #2588)Neil Alexander
2022-07-25Add .well-known/matrix/client to clientapi (#2551)Jonathan Bartlett
Signed-off-by: Jonathan Bartlett <jonathan@jonnobrow.co.uk> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-19Update FAQNeil Alexander
2022-07-11Ristretto cache (#2563)Neil Alexander
* Try Ristretto cache * Tweak * It's beautiful * Update GMSL * More strict keyable interface * Fix that some more * Make less panicky * Don't enforce mutability checks for now * Determine mutability using deep equality * Tweaks * Namespace keys * Make federation caches mutable * Update cost estimation, add metric * Update GMSL * Estimate cost for metrics better * Reduce counters a bit * Try caching events * Some guards * Try again * Try this * Use separate caches for hopefully better hash distribution * Fix bug with admitting events into cache * Try to fix bugs * Check nil * Try that again * Preserve order jeezo this is messy * thanks VS Code for doing exactly the wrong thing * Try this again * Be more specific * aaaaargh * One more time * That might be better * Stronger sorting * Cache expiries, async publishing of EDUs * Put it back * Use a shared cache again * Cost estimation fixes * Update ristretto * Reduce counters a bit * Clean up a bit * Update GMSL * 1GB * Configurable cache sizees * Tweaks * Add `config.DataUnit` for specifying friendly cache sizes * Various tweaks * Update GMSL * Add back some lazy loading caching * Include key in cost * Include key in cost * Tweak max age handling, config key name * Only register prometheus metrics if requested * Review comments @S7evinK * Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere) * Review comments * Update sample configs * Update GHA Workflow * Update Complement images to Go 1.18 * Remove the cache test from the federation API as we no longer guarantee immediate cache admission * Don't check the caches in the renewal test * Possibly fix the upgrade tests * Update to matrix-org/gomatrixserverlib#322 * Update documentation to refer to Go 1.18
2022-07-05feat+fix: Ignore unknown keys and verify required fields are present in ↵Kabir Kwatra
appservice registration files (#2550) * fix: ignore unknown keys in appservice configs fixes matrix-org/dendrite#1567 * feat: verify required fields in appservice configs
2022-06-15Add `InputDeviceListUpdate` to the keyserver, remove old input API (#2536)Neil Alexander
* Add `InputDeviceListUpdate` to the keyserver, remove old input API * Fix copyright * Log more information when a device list update fails
2022-06-10Attempt to raise the file descriptor limit at startup (#2527)Neil Alexander
2022-06-07Rate limiting changes (#2519)Neil Alexander
* Rate limiting changes This makes the following changes: * For logged in users, the rate limiting now applies to the device session rather than the remote IP address; * For non-logged in users, the rate limiting continues to apply to remote address as it does today; * It is now possible to add user IDs to the `exempt_user_ids` option under `rate_limiting` to exclude bots from rate limiting; * Admin and appservice users are now exempt from rate limiting by default. * Fix build with media API
2022-05-13Shuffle config Verify/Defaults a bit around (#2459)Till
2022-05-09Begin adding syncapi component tests (#2442)kegsay
* Add very basic syncapi tests * Add a way to inject jetstream messages * implement add_state_ids * bugfixes * Unbreak tests * Remove now un-needed API call * Linting
2022-05-09One NATS instance per `BaseDendrite` (#2438)Neil Alexander
* One NATS instance per `BaseDendrite` * Fix roomserver
2022-05-06Add `PolylithMode` base config option (#2428)Neil Alexander
* Add `PolylithMode` base config option * Polylith mode always uses HTTP APIs
2022-05-06Clean up interface definitions (#2427)kegsay
* tidy up interfaces * remove unused GetCreatorIDForAlias * Add RoomserverUserAPI interface * Define more interfaces * Use AppServiceInternalAPI for consistent naming * clean up federationapi constructor a bit * Fix monolith in -http mode
2022-05-05Define component interfaces based on consumers (2/2) (#2425)kegsay
* convert remaining interfaces * Tidy up the userapi interfaces
2022-05-05Update to matrix-org/gomatrixserverlib#303Neil Alexander
2022-05-05syncapi: define specific interfaces for internal HTTP communications (#2416)kegsay
* syncapi: use finer-grained interfaces when making the syncapi * Use specific interfaces for syncapi-roomserver interactions * Define query access token api for shared http auth code
2022-05-04Add opt-in anonymous stats reporting (#2249)Till
* Initial phone home stats queries * Add userAgent to UpdateDeviceLastSeen Add new Table for tracking daily user vists * Add user_daily_visits table * Fix queries * userapi stats tables & queries * userapi interface and internal api * sycnapi stats queries * testing phone home stats * Add complete config to syncapi * add missing files * Fix queries * Send empty request * Add version & monolith stats * Add configuration for phone home stats * Move WASM to its own file, add config and comments * Add tracing methods * Add total rooms * Add more fields, actually send data somewhere * Move stats to the userapi * Move phone home stats to util package * Cleanup * Linter & parts of GH comments * More GH comments changes - Move comments to SQL statements - Shrink interface, add struct for stats - No fatal errors, use defaults * Be more explicit when querying * Fix wrong calculation & wrong query params Add tests * Add Windows stats * ADd build constraint * Use new testing structure Fix issues with getting values when using SQLite Fix wrong AddDate value Export UpdateUserDailyVisits * Fix query params * Fix test * Add comment about countR30UsersSQL and countR30UsersV2SQL; fix test * Update config * Also update example config file * Use OS level proxy, update logging Co-authored-by: kegsay <kegan@matrix.org>
2022-05-03Fix bug in database global setupNeil Alexander
2022-05-03Tidy up `AddPublicRoutes` (#2412)Neil Alexander
* Simplify federation API `AddPublicRoutes` * Simplify client API `AddPublicRoutes` * Simplify media API `AddPublicRoutes` * Simplify sync API `AddPublicRoutes` * Simplify `AddAllPublicRoutes`
2022-05-03Global database connection pool (for monolith mode) (#2411)Neil Alexander
* Allow monolith components to share a single database pool * Don't yell about missing connection strings * Rename field * Setup tweaks * Fix panic * Improve configuration checks * Update config * Fix lint errors * Update comments
2022-05-02Return `null` if MaxFileSizeBytes is 0 (#2409)Till
* Return "null" if MaxFileSizeBytes is 0 * Add comment and nil check (better save than sorry) * Simplify config
2022-04-29Add restrictions for open registration (#2402)Till
* Add restrications for open registration * Make enable open registration a parameter * Enable registration for CI * Update error message * Shuffle things around a bit * Add a warning at every startup just to be extra annoying * Ignore shared secret when warning about open registration, since it's not strictly required when it is set if registration is otherwise enabled * Make CI happy? * Add missing parameter; try new parameter in upgrade-test Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-28Add `/_dendrite/admin/evacuateRoom/{roomID}` (#2401)Neil Alexander
* Add new endpoint to allow admins to evacuate the local server from the room * Guard endpoint * Use right prefix * Auth API * More useful return error rather than a panic * More useful return value again * Update the path * Try using inputer instead * oh provide the config * Try that again * Return affected user IDs * Don't create so many forward extremities * Add missing `Path` to name Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2022-04-27Version 0.8.2 (#2386)v0.8.2Neil Alexander
* Version 0.8.2 * Correct account data position mapping * Try that again * Don't duplicate wait-for-shutdowns
2022-04-27Fix graceful shutdownNeil Alexander
2022-04-27Defer cancel on shutdown contextNeil Alexander
2022-04-27Use process context as base context for all HTTPNeil Alexander
2022-04-27Add UserAPI storage tests (#2384)Till
* Add tests for parts of the userapi storage * Add tests for keybackup * Add LoginToken tests * Add OpenID tests * Add profile tests * Add pusher tests * Add ThreePID tests * Add notification tests * Add more device tests, fix numeric localpart query * Fix failing CI * Fix numeric local part query
2022-04-27Prevent JetStream from handling OS signals, allow running as a Windows ↵Neil Alexander
service (#2385) * Prevent JetStream from handling OS signals, allow running as a Windows service (fixes #2374) * Remove double import
2022-04-08Add test infrastructure code for dendrite unit/integ tests (#2331)kegsay
* Add test infrastructure code for dendrite unit/integ tests Start re-enabling some syncapi storage tests in the process. * Linting * Add postgres service to unit tests * dendrite not syncv3 * Skip test which doesn't work * Linting * Add `jetstream.PrepareForTests` Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-06Add presence module V2 (#2312)Till
* Syncapi presence * Clientapi http presence handler * Why is this here? * Missing files * FederationAPI presence implementation * Add new presence stream * Pinecone update * Pinecone update * Add passing tests * Make linter happy * Add presence producer * Add presence config option * Set user to unavailable after x minutes * Only set currently_active if online Avoid unneeded presence updates when syncing * Tweaks * Query devices for last_active_ts Fixes & tweaks * Export SharedUsers/SharedUsers * Presence stream in MemoryStorage * Remove status_msg_nil * Fix sytest crashes * Make presence types const and use stringer for it * Change options to allow inbound/outbound presence * Fix option & typo * Update configs Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-04remove obsolete config entry "federation_certificates" (#2318)Sebastian Spaeth
* Remove all federation_certificates occurencs This configuration value has not been used since 2019 apparently, and indeed it is never really used in the code base. So remove all traces of it from the various configuration files. Also remove the unused variable FederationCertificatePaths Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de> * setup/config/config_test.go: remove federation_sender config snippet The federation_sender: section was folded into the federation_api some time ago, and this seems to be the only leftover in the code base. So remove it.
2022-03-29Remove eduserver (#2306)S7evinK
* Move receipt sending to own JetStream producer * Move SendToDevice to producer * Remove most parts of the EDU server * Fix SendToDevice & copyrights * Move structs, cleanup EDU Server traces * Use HeadersOnly subscription * Missing file * Fix linter issues * Move consumers to own files * Rename durable consumer; Consumer cleanup * Docs/config cleanup
2022-03-28User directory for nearby Pinecone peers (P2P demo) (#2311)Neil Alexander
* User directory for nearby Pinecone peers * Fix mux routing * Use config to determine which server notices user to exclude
2022-03-25Healthcheck endpoints (#2303)Neil Alexander
* Health monitoring endpoints * Rename endpoints
2022-03-25Try to recover from corrupted NATS streams in memory temporarily (#2301)Neil Alexander
2022-03-24Refactor appservice & client API to use userapi internal (#2290)S7evinK
* Refactor user api internal * Refactor clientapi to use internal userapi * Use internal userapi instead of user DB directly * Remove AccountDB dependency * Fix linter issues Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-24fixup treat the sender_localpart as an exclusive namespace of one user (#2255)Nick Cao
Signed-off-by: Nick Cao <nickcao@nichi.co> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-24Fix lint errorsNeil Alexander
2022-03-23Per-room consumers (#2293)Neil Alexander
* Roomserver input refactoring — again! * Ensure the actor runs again * Preserve consumer after unsubscribe * Another sprinkling of magic * Rename `TopicFor` to `Prefixed` * Recreate the stream if the config is bad * Check streams too * Prefix subjects, preserve inboxes * Recreate if subjects wrong * Remove stream subject * Reconstruct properly * Fix mutex unlock * Comments * Fix tests * Don't drop events * Review comments * Separate `queueInputRoomEvents` function * Re-jig control flow a bit
2022-03-21Wait for safe shutdown of NATS Server (#2289)Neil Alexander
2022-03-16Stream tweaks, use same codepath for sync vs async input room events, wait ↵Neil Alexander
for error response via NATS messages (#2283)
2022-03-11Platform sanity checks (#2271)Neil Alexander
* Platform sanity checks * Don't yell about `root` because Docker
2022-03-09Remove references to `userapi_devices` (fixes #2259)Neil Alexander
2022-03-08Get MSC2946 working for restricted rooms locally/over federation (#2260)kegsay
* Get MSC2946 working for restricted rooms locally * Get MSC2946 working for restricted rooms over federation * Allow invited in addition to joined to enable child walking