aboutsummaryrefslogtreecommitdiff
path: root/keyserver
AgeCommit message (Collapse)Author
2021-06-07Try to optimize SelectOneTimeKeys (#1851)S7evinK
* Try to optimize SelectOneTimeKeys Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Use pg.Array when using ANY... Co-authored-by: Kegsay <kegan@matrix.org>
2021-04-08Optimise memory usage when calling /g_m_e (#1819)Kegsay
* Optimise memory usage when calling /g_m_e * cache more events * refactor handling of device list update pokes * Sigh
2021-03-23Add more metrics (#1802)Kegsay
* Add more metrics * Linting
2021-03-23log less for failed key querys, add counters for incoming pdus/edus (#1801)Kegsay
* log less for failed key querys, add counters for incoming pdus/edus * use labels * Blacklist flakey test * Fix metrics
2021-03-08device lists: backoff for longer if the wrong error type is returned (#1796)Kegsay
2021-03-02Return the current OTK count on an empty upload request (#1774)Neil Alexander
* Always return OTK counts * Fix parameter ordering * Send IDs over to keyserver internal API * Review comments * Fix syntax error * Fix panic, hopefully * Require user ID to be set * Fix user API call
2021-01-22NewClient and NewFederationClient updates (#1730)Neil Alexander
* Use matrix-org/gomatrixserverlib#252 * Add missing WithSkipVerify to test * Functions instead * Update gomatrixserverlib to matrix-org/gomatrixserverlib#252 * Fix disabling TLS validation
2021-01-18Fix integer overflow in device_list_update.go (#1717)Loïck Bonniot
Fix #1511 On 32-bits systems, int(hash.Sum32()) can be negative. This makes the computation of array indices using modulo invalid, crashing dendrite. Signed-off-by: Loïck Bonniot <git@lesterpig.com>
2021-01-12Reduce device list GetUserDevices timeout (#1704)Neil Alexander
2020-12-18More sane next batch handling, typing notification tweaks, give invites ↵Neil Alexander
their own stream position, device list fix (#1641) * Update sync responses * Fix positions, add ApplyUpdates * Fix MarshalText as non-pointer, PrevBatch is optional * Increment by number of read receipts * Merge branch 'master' into neilalexander/devicelist * Tweak typing * Include keyserver position tweak * Fix typing next position in all cases * Tweaks * Fix typo * Tweaks, restore StreamingToken.MarshalText which somehow went missing? * Rely on positions from notifier rather than manually advancing them * Revert "Rely on positions from notifier rather than manually advancing them" This reverts commit 53112a62cc3bfd9989acab518e69eeb27938117a. * Give invites their own position, fix other things * Fix test * Fix invites maybe * Un-whitelist tests that look to be genuinely wrong * Use real receipt positions * Ensure send-to-device uses real positions too
2020-12-02Top-level setup package (#1605)Neil Alexander
* Move config, setup, mscs into "setup" top-level folder * oops, forgot the EDU server * Add setup * goimports
2020-10-15Start Kafka connections for each component that needs them (#1527)Neil Alexander
* Start Kafka connection for each component that needs one * Fix roomserver unit tests * Rename to naffkaInstance (@Kegsay review comment) * Fix import cycle
2020-10-09Update federation timeouts (#1504)Neil Alexander
* Update to matrix-org/gomatrixserverlib#234 * Update gomatrixserverlib * Update federation timeouts * Fix dendritejs * Increase /send context time in destination queue
2020-09-24Update all usages of tx.Stmt to sqlutil.TxStmt (#1423)Sam
* Replace all usages of txn.Stmt with sqlutil.TxStmt Signed-off-by: Sam Day <me@samcday.com> * Fix sign off link in PR template. Signed-off-by: Sam Day <me@samcday.com> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-09-10Peeking via MSC2753 (#1370)Matthew Hodgson
Initial implementation of MSC2753, as tested by https://github.com/matrix-org/sytest/pull/944. Doesn't yet handle unpeeks, peeked EDUs, or history viz changing during a peek - these will follow. https://github.com/matrix-org/dendrite/pull/1370 has full details.
2020-09-08Exclude deleted keys from selectBatchDeviceKeysSQL (#1412)Neil Alexander
2020-09-03Defer keyserver and federationsender wakeups to give HTTP listeners time to ↵Neil Alexander
start (#1389)
2020-08-26Wait for 8h between device list updates for blacklisted servers (#1344)Kegsay
2020-08-26Modify DeviceListUpdater to retry requests according to RetryAfter (#1342)Kegsay
* Modify DeviceListUpdater to retry requests according to RetryAfter * Reduce wait time for sytest test pollution
2020-08-25Use Writer in shared package (#1296)Neil Alexander
2020-08-21Component-wide TransactionWriters (#1290)Neil Alexander
* Offset updates take place using TransactionWriter * Refactor TransactionWriter in current state server * Refactor TransactionWriter in federation sender * Refactor TransactionWriter in key server * Refactor TransactionWriter in media API * Refactor TransactionWriter in server key API * Refactor TransactionWriter in sync API * Refactor TransactionWriter in user API * Fix deadlocking Sync API tests * Un-deadlock device database * Fix appservice API * Rename TransactionWriters to Writers * Move writers up a layer in sync API * Document sqlutil.Writer interface * Add note to Writer documentation
2020-08-20Add FederationClient interface to federationsender (#1284)Kegsay
* Add FederationClient interface to federationsender - Use a shim struct in HTTP mode to keep the same API as `FederationClient`. - Use `federationsender` instead of `FederationClient` in `keyserver`. * Pointers not values * Review comments * Fix unit tests * Rejig backoff * Unbreak test * Remove debug logs * Review comments and linting
2020-08-19Transaction writer changes, move roomserver writers (#1285)Neil Alexander
* Updated TransactionWriters, moved locks in roomserver, various other tweaks * Fix redaction deadlocks * Fix lint issue * Rename SQLiteTransactionWriter to ExclusiveTransactionWriter * Fix us not sending transactions through in latest events updater
2020-08-18Summarise key change logs (#1278)Kegsay
2020-08-18Only emit key changes which are different from what we had before (#1279)Kegsay
We did this already for local `/keys/upload` but didn't for remote `/users/devices`. This meant any resyncs would spam produce events, hammering disk i/o and spamming the logs.
2020-08-13Make 'Device list doesn't change if remote server is down' pass (#1268)Kegsay
- As a last resort, query the DB when exhausting all possible remote query endpoints, but keep the field in `failures` so clients can detect that this is stale data. - Unblock `DeviceListUpdater.Update` on failures rather than timing out. - Use a mutex when writing directly to `res`, not just for failures.
2020-08-12Fix more E2E sytests (#1265)Kegsay
* WIP: Eagerly sync device lists on /user/keys/query requests Also notify servers when a user's device display name changes. Few caveats: - sytest `Device deletion propagates over federation` fails - `populateResponseWithDeviceKeysFromDatabase` is called from multiple goroutines and hence is unsafe. * Handle deleted devices correctly over federation
2020-08-12Add sync mechanism to block when updating device lists (#1264)Kegsay
* Add sync mechanism to block when updating device lists With a timeout, mainly for sytest to fix the test "Server correctly handles incoming m.device_list_update" which is flakey because it assumes that when `/send` 200 OKs that the server has updated the device lists in prep for `/keys/query` which is not always true when using workers. * Fix UT * Add new working test
2020-08-12Bugfixes for 'If remote user leaves room we no longer receive device ↵Kegsay
updates' (#1262) * Bugfixes for 'If remote user leaves room we no longer receive device updates' * Update whitelist and README
2020-08-11Reduce cooldown to make sure sytest doesn't give up (#1257)Kegsay
* Reduce cooldown to make sure sytest doesn't give up * More sytests pass weeeeeee
2020-08-10Prefix-defined Kafka topics (#1254)Neil Alexander
* Prefix-defined Kafka topics * Fix current state server test
2020-08-10Configuration format v1 (#1230)Neil Alexander
* Initial pass at refactoring config (not finished) * Don't forget current state and EDU servers * More shifting around * Update server key API tests * Fix roomserver test * Fix more tests * Further tweaks * Fix current state server test (sort of) * Maybe fix appservices * Fix client API test * Include database connection string in database options * Fix sync API build * Update config test * Fix unit tests * Fix federation sender build * Fix gobind build * Set Listen address for all services in HTTP monolith mode * Validate config, reinstate appservice derived in directory, tweaks * Tweak federation API test * Set MaxOpenConnections/MaxIdleConnections to previous values * Update generate-config
2020-08-07Fix unit testsKegan Dougal
2020-08-07Finish inbound E2E device lists (#1243)Kegsay
* Add tests for device list updates * Add stale_device_lists table and use db before asking remote for device keys * Fetch remote keys if all devices are requested * Add display_name col to store remote device names Few other tweaks to make `Server correctly handles incoming m.device_list_update` pass. * Fix sqlite otk bug * Unbuffered channel to block /send causing sytest to not race anymore * Linting and fix bug whereby we didn't send updated dl tokens to the client causing a tightloop on /sync sometimes * No longer assert staleness as Update blocks on workers now * Back out tweaks * Bugfixes
2020-08-06Add device list updater which manages updating remote device lists (#1242)Kegsay
* Add device list updater which manages updating remote device lists - Doesn't persist stale lists to the database yet - Doesn't have tests yet * Mark device lists as fresh when we persist
2020-08-05Process inbound device list updates from federation (#1240)Kegsay
* Add InputDeviceListUpdate * Unbreak unit tests * Process inbound device list updates from federation - Persist the keys in the keyserver and produce key changes - Does not currently fetch keys from the remote server if the prev IDs are missing * Linting
2020-08-05Use TransactionWriter in SQLite keyserver (#1239)Neil Alexander
* Use TransactionWriter in SQLite keyserver * Fix keyserver storage tests
2020-08-04Send device list updates to servers (outbound only) (#1237)Kegsay
* Add QueryDeviceMessages to serve up device keys and stream IDs * Consume key change events in fedsender Don't yet send them to destinations as we haven't worked them out yet * Send device list updates to all required servers * Glue it all together
2020-08-03Generate stream IDs for locally uploaded device keys (#1236)Kegsay
* Breaking: add stream_id to keyserver_device_keys table * Add tests for stream ID generation * Fix whitelist
2020-08-03Produce OTK counts in /sync response (#1235)Kegsay
* Add QueryOneTimeKeys for /sync extensions * Unbreak tests * Produce OTK counts in /sync response * Linting
2020-07-31Fix edge cases around device lists (#1234)Kegsay
* Fix New users appear in /keys/changes * Create blank device keys when logging in on a new device * Add PerformDeviceUpdate and fix a few bugs - Correct device deletion query on sqlite - Return no keys on /keys/query rather than an empty key * Unbreak sqlite properly * Use a real DB for currentstateserver integration tests * Race fix
2020-07-30Notify clients when devices are deleted (#1233)Kegsay
* Recheck device lists when join/leave events come in * Add PerformDeviceDeletion * Notify clients when devices are deleted * Unbreak things * Remove debug logging
2020-07-30Implement /keys/changes (#1232)Kegsay
* Implement /keys/changes And refactor QueryKeyChanges to accept a `to` offset. * Unbreak tests * Sort keys when serialising log tokens
2020-07-30Hook up device list updates to the sync notifier (#1231)Kegsay
* WIP hooking up key changes * Fix import cycle, get tests passing and binary compiling * Linting and update whitelist
2020-07-28Add QueryKeyChanges (#1228)Kegsay
Hook some things up to call it as well.
2020-07-28Persist partition|offset|user_id in the keyserver (#1226)Kegsay
* Persist partition|offset|user_id in the keyserver Required for a query API which will be used by the syncapi which will be called when a `/sync` request comes in which will return a list of user IDs of people who have changed their device keys between two tokens. * Add tests and fix maxOffset bug * s/offset/log_offset/g because 'offset' is a reserved word in postgres
2020-07-23keyserver: produce key change events (#1218)Kegsay
* Produce kafka events when keys are added * Consume key changes in syncapi with TODO markers for handling them and catching up * unbreak tests * Linting
2020-07-22Handle inbound federation E2E key queries/claims (#1215)Kegsay
* Handle inbound /keys/claim and /keys/query requests * Add display names to device key responses * Linting
2020-07-21Perform outbound federation hits for querying/claiming E2E keys (#1212)Kegsay
* Perform outbound federation hits for querying/claiming E2E keys Untested currently because we need the receiving end to work before sytest will be happy. * Linting
2020-07-21Implement claiming one-time keys locally (#1210)Kegsay
* Add API shape for claiming keys * Implement claiming one-time keys locally Fairly boring, nothing too special going on.
generated by cgit v1.2.3 (git 2.26.2) at 2025-01-21 08:56:00 +0000