aboutsummaryrefslogtreecommitdiff
path: root/syncapi
AgeCommit message (Collapse)Author
2023-09-12[pseudoIDs] changing event ID fix (#3195)Sam Wedgwood
power levels events in pseudo IDs sometimes changed event IDs (this was already fixed earlier, but one of the edgecases was not covered, and is now covered) Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-31Handle event_format federation in /sync responses (#3192)devonh
2023-08-24[pseudoIDs] More pseudo ID fixes - Part 2 (#3181)Sam Wedgwood
Fixes include: - Translating state keys that contain user IDs to their respective room keys for both querying and sending state events - **NOTE**: there may be design discussion needed on what should happen when sender keys cannot be found for users - A simple fix for kicking guests from rooms properly - Logic for boundary history visibilities was slightly off (I'm surprised this only manifested in pseudo ID room versions) Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-15[pseudoID] More pseudo ID fixes (#3167)Sam Wedgwood
Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-02Use `*spec.SenderID` for `QuerySenderIDForUser` (#3164)Sam Wedgwood
There are cases where a dendrite instance is unaware of a pseudo ID for a user, the user is not a member of that room. To represent this case, we currently use the 'zero' value, which is often not checked and so causes errors later down the line. To make this case more explict, and to be consistent with `QueryUserIDForSender`, this PR changes this to use a pointer (and `nil` to mean no sender ID). Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-07-31[pseudoIDs] Fixes for room alias tests (#3159)Sam Wedgwood
Some (deceptively) simple fixes for some bugs that caused room alias tests to fail (sytext `tests/30rooms/05aliases.pl`). Each commit has details about what it fixes. Sytest results: - Sytest before (79d4a0e): https://gist.github.com/swedgwood/972ac4ef93edd130d3db0930703d6c82 - Sytest after (4b09bed): https://gist.github.com/swedgwood/504b00ac4ee892acb757b7fac55fa28a Room aliases go from `8/15` to `15/15`, but looks like these fixes also managed to fix about `4` other tests, which is a nice bonus :) Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-07-19Use pointer when passing the connection manager around (#3152)Till
As otherwise existing connections aren't reused.
2023-07-14Discard "illegal base64 data at input byte 0" errors in the SyncAPITill Faelligen
2023-07-13Tweaks around `/messages` (#3149)Till
Try to mitigate some issues with `/messages`
2023-07-08Avoid loops by setting end to an empty string if start == end (#3146)Till
2023-07-07Fix issues reported by Sentry (#3143)Till
This should fix a few issues reported by Sentry
2023-07-06Populate syncapi state event prev_sender with userID (#3142)devonh
2023-07-06Fix prev event lookup in syncapi (#3141)devonh
The syncapi operates using userID's so when querying for the previous state event we need to lookup the userID from the given senderID before the state query.
2023-07-06Add pseudoID compatibility to Invites (#3126)devonh
2023-06-28Add `MXIDMapping` for pseudoID rooms (#3112)Till
Add `MXIDMapping` on membership events when creating/joining rooms.
2023-06-28Fix syncAPI redactions (#3118)Till
Previously we were setting `redacted_because` to the PDU event, but as per the spec it should really be a client event. This fixes it.
2023-06-14Merge SenderID & Per Room User Key work (#3109)devonh
2023-06-13Add `sender_key` to ClientEvent (#3110)Till
2023-06-12Cleanup remaining statekey usage for senderIDs (#3106)devonh
2023-06-07Use SenderID Type (#3105)devonh
2023-06-06PDU Sender split (#3100)devonh
Initial cut of splitting PDU Sender into SenderID & looking up UserID where required.
2023-06-02Fix newly found linter issues (#3099)Till
Fixes the issues found in https://github.com/matrix-org/dendrite/actions/runs/5155539352/jobs/9285342056#step:5:22. Only naked returns in longer functions.
2023-05-30Docs restructure (#2953)Till
Needs to be merged into `gh-pages` later on.
2023-05-17Move MakeJoin logic to GMSL (#3081)devonh
2023-05-09Move json errors over to gmsl (#3080)devonh
2023-05-03Use PDU in even more places (#3074)kegsay
- No longer rely on *Event returning from NewEventFrom... functions Requires https://github.com/matrix-org/gomatrixserverlib/pull/377
2023-05-02Use PDU not *Event in HeaderedEvent (#3073)kegsay
Requires https://github.com/matrix-org/gomatrixserverlib/pull/376 This has numerous upsides: - Less type casting to `*Event` is required. - Making Dendrite work with `PDU` interfaces means we can swap out Event impls more easily. - Tests which represent weird event shapes are easier to write. Part of a series of refactors on GMSL.
2023-04-28Use PDU in more places (#3072)kegsay
2023-04-27Use PDU interface (#3070)kegsay
We only use it in a few places currently, enough to get things to compile and run. We should be using it in much more places. Similarly, in some places we cast []PDU back to []*Event, we need to not do that. Likewise, in some places we cast PDU to *Event, we need to not do that. For now though, hopefully this is a start.
2023-04-27Remove gmsl.HeaderedEvent (#3068)kegsay
Replaced with types.HeaderedEvent _for now_. In reality we want to move them all to gmsl.Event and only use HeaderedEvent when we _need_ to bundle the version/event ID with the event (seriailsation boundaries, and even then only when we don't have the room version). Requires https://github.com/matrix-org/gomatrixserverlib/pull/373
2023-04-21Implement new RoomVersionImpl API (#3062)kegsay
As outlined in https://github.com/matrix-org/gomatrixserverlib/pull/368 The main change Dendrite side is that `RoomVersion` no longer has any methods on it. Instead, you need to bounce via `gmsl.GetRoomVersion`. It's very interesting to see where exactly Dendrite cares about this. For some places it's creating events (fine) but others are way more specific. Those areas will need to migrate to GMSL at some point.
2023-04-20refactor: funnel event creation through room versions (#3060)kegsay
In preparation of interfacing up the room version value.
2023-04-19refactor: update GMSL (#3058)kegsay
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/364 Read this commit by commit to avoid going insane.
2023-04-17Add sync API db tests (#3043)Till
Co-authored-by: kegsay <kegan@matrix.org>
2023-04-04Move GMSL client types to Dendrite (#3045)kegsay
GMSL is intended for Federation only. Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/357
2023-03-27Add `/search` tests (#3025)Till
2023-03-23Move every `db.Prepare` to `sqlutil.Statementlist`, remove trace driver (#3026)Till
Doesn't buy us much, but makes everything a bit more consistent. Also removes the SQL trace driver, as it is unused and the output is hard to read anyway.
2023-03-22Update Go, use go tool covdata for coverage files?Till Faelligen
2023-03-22Remove `BaseDendrite` (#3023)Till
Removes `BaseDendrite` to, hopefully, make testing and composing of components easier in the future.
2023-03-17Preparations for removing `BaseDendrite` (#3016)Till
Preparations to actually remove/replace `BaseDendrite`. Quite a few changes: - SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed from `BaseDendrite`) - Caches are removed from `BaseDendrite` - Introduces a `Router` struct (likely to change) - also fixes #2903 - Introduces a `sqlutil.ConnectionManager`, which should remove `base.DatabaseConnection` later on - probably more
2023-03-06Add tests for `UpdateRelations` (#2999)Till
This also fixes an issue regarding updates to relations for invalid events, which could result in us retrying said event over and over again, if we fail to unmarshal the event to `gomatrixserverlib.RelationContent`, this was discovered by `@sleroq:virto.community`
2023-03-01Refactor `StoreEvent`, add `MaybeRedactEvent`, create an `EventDatabase` (#2989)Till
This PR changes the following: - `StoreEvent` now only stores an event (and possibly prev event), instead of also doing redactions - Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should be called after storing events - a few other things
2023-02-20Merge keyserver & userapi (#2972)Till
As discussed yesterday, a first draft of merging the keyserver and the userapi.
2023-02-14Remove polylith/API mode (#2967)Till
This removes most of the code used for polylith/API mode. This removes the `/api` internal endpoints entirely. Binary size change roughly 5%: ``` 51437560 Feb 13 10:15 dendrite-monolith-server # old 48759008 Feb 13 10:15 dendrite-monolith-server # new ```
2023-02-07Optimize `/sync` and history visibility (#2961)Till
Should fix the following issues or make a lot less worse when using Postgres: The main issue behind #2911: The client gives up after a certain time, causing a cascade of context errors, because the response couldn't be built up fast enough. This mostly happens on accounts with many rooms, due to the inefficient way we're getting recent events and current state For #2777: The queries for getting the membership events for history visibility were being executed for each room (I think 185?), resulting in a whooping 2k queries for membership events. (Getting the statesnapshot -> block nids -> actual wanted membership event) Both should now be better by: - Using a LATERAL join to get all recent events for all joined rooms in one go (TODO: maybe do the same for room summary and current state etc) - If we're lazy loading on initial syncs, we're now not getting the whole current state, just to drop the majority of it because we're lazy loading members - we add a filter to exclude membership events on the first call to `CurrentState`. - Using an optimized query to get the membership events needed to calculate history visibility --------- Co-authored-by: kegsay <kegan@matrix.org>
2023-02-03Add Sytest/Complement coverage to scheduled runs (#2962)Till
This adds Sytest and Complement coverage reporting to the nightly scheduled CI runs. Fixes a few API mode related issues as well, since we seemingly never really ran them with Complement. Also fixes a bug related to device list changes: When we pass in an empty `newlyLeftRooms` slice, we got a list of all currently joined rooms with the corresponding members. When we then got the `newlyJoinedRooms`, we wouldn't update the `changed` slice, because we already got the user from the `newlyLeftRooms` query. This is fixed by simply ignoring empty `newlyLeftRooms`.
2023-01-19Add `/_dendrite/admin/purgeRoom/{roomID}` (#2662)Neil
This adds a new admin endpoint `/_dendrite/admin/purgeRoom/{roomID}`. It completely erases all database entries for a given room ID. The roomserver will start by clearing all data for that room and then will generate an output event to notify downstream components (i.e. the sync API and federation API) to do the same. It does not currently clear media and it is currently not implemented for SQLite since it relies on SQL array operations right now. Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-01-17`/sync` performance optimizations (#2927)Till
Since #2849 there is no limit for the current state we fetch to calculate history visibility. In large rooms this can cause us to fetch thousands of membership events we don't really care about. This now only gets the state event types and senders in our timeline, which should significantly reduce the amount of events we fetch from the database. Also removes `MaxTopologicalPosition`, as it is an unnecessary DB call, given we use the result in `topological_position < $1` calls.
2023-01-17Omit state field from `/messages` response if empty (#2940)Umar Getagazov
The field type is `[ClientEvent]` in the [spec](https://spec.matrix.org/v1.5/client-server-api/#get_matrixclientv3roomsroomidmessages), but right now `null` can also be returned. Omit the field completely if it's empty. Some clients (rightfully) assume it's either not present at all or it's of the right type (see https://github.com/matrix-org/matrix-react-sdk/pull/9913). ### Pull Request Checklist <!-- Please read https://matrix-org.github.io/dendrite/development/contributing before submitting your pull request --> * [x] I have added Go unit tests or [Complement integration tests](https://github.com/matrix-org/complement) for this PR _or_ I have justified why this PR doesn't need tests * The PR is a simple struct tag fix * [x] Pull request includes a [sign off below using a legally identifiable name](https://matrix-org.github.io/dendrite/development/contributing#sign-off) _or_ I have already signed off privately Signed-off-by: `Umar Getagazov <umar@handlerug.me>` Signed-off-by: Umar Getagazov <umar@handlerug.me>
2023-01-12Fix room summary returning wrong heroes (#2930)Till
This should fix #2910. Probably makes Sytest/Complement a bit upset, since this not using `sort.Strings` anymore.