aboutsummaryrefslogtreecommitdiff
path: root/syncapi/routing
AgeCommit message (Collapse)Author
2022-07-18History visibility database changes (#2533)Till
* Add new history_visibility column * Update SQL queries to include history_visibility * Store the history visibilty calculated by the roomserver * Update GMSL * Update migrations * Fix migration * Update GMSL * Fix `go.sum` * Update GMSL to use sql.Scanner & sql.Valuer * Re-order migration/table creation * Update gomatrixserverlib * Add history_visibility column to current_room_state * Fix migrations * Return error instead of Fatal log Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-01Try to fix backfilling (#2548)Till
* Try to fix backfilling * Return start/end to not confuse clients * Update GMSL * Update GMSL
2022-06-29Don't return `end` if there are not more messages (#2542)Till
* Be more spec compliant * Move lazyLoadMembers to own method
2022-06-13Handle state before, send history visibility in output (#2532)Neil Alexander
* Check state before event * Tweaks * Refactor a bit, include in output events * Don't waste time if soft failed either * Tweak control flow, comments, use GMSL history visibility type
2022-05-11Add RoomExists flag to QueryMembershipForUser (#2450)kegsay
Fixes https://github.com/matrix-org/complement/pull/369
2022-05-06Move LL cache (#2429)Till
2022-05-05syncapi: define specific interfaces for internal HTTP communications (#2416)kegsay
* syncapi: use finer-grained interfaces when making the syncapi * Use specific interfaces for syncapi-roomserver interactions * Define query access token api for shared http auth code
2022-04-26Send all account data on complete sync by defaultNeil Alexander
Squashed commit of the following: commit 0ec8de57261d573a5f88577aa9d7a1174d3999b9 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Apr 26 16:56:30 2022 +0100 Select filter onto provided target filter commit da40b6fffbf5737864b223f49900048f557941f9 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Apr 26 16:48:00 2022 +0100 Specify other field too commit ffc0b0801f63bb4d3061b6813e3ce5f3b4c8fbcb Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Tue Apr 26 16:45:44 2022 +0100 Send as much account data as possible during complete sync
2022-04-22Reuse the existing lazyload cache on /context and /messages (#2367)Till
2022-04-13Make `/messages` filterable (#2347)Till
* Make /messages filterable Fix bug when determining if an event contains an URL * Add newly passing test * Fix test
2022-04-11Correctly use provided filters (#2339)Till
* Apply filters correctly * Fix issues; Use prepareWithFilters * Update gmsl & tests * go.mod.. * PR comments
2022-03-18Better mapping of stream positions to topological positions in `/messages` ↵Neil Alexander
(#2263) * Convert stream positions into topological positions for both `from` and `to` in `/messages` * Hopefully it works now * Remove unnecessary logging * Return sane values if `StreamToTopologicalPosition` can't work out the right thing to do * Revert logging change * tweaks * Fix `selectEventIDsInRangeASCSQL` * Test `Getting messages going forward is limited for a departed room (SPEC-216)` was passing incorrectly so un-whitelist it
2022-03-14Fix /context with lazy_load_members (#2277)S7evinK
* Add membership events to the end of the list, to ensure Sytest sees them * Move tests to allowlist * Append to correct list, fix logging message * Add flakey tests to blacklist * Remove flakey tests from whitelist
2022-03-04Fix a panic in `OnIncomingMessagesRequest` (#2250)Neil Alexander
It's possible for `GetStateEvent` to return `nil` if there was no error but the state event wasn't found. Therefore we need to be prepared for that case. This should fix #2247.
2022-03-03Return 404 if event given to `/context` was not found (#2245)Neil Alexander
2022-03-01Return state on calls to /message and lazy load members (#2218)S7evinK
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-21Fix logic error on context history visibility (#2211)Neil Alexander
2022-02-21Implement `/context` (#2207)S7evinK
* Add QueryEventsAfter * Add /context * Make all tests pass on sqlite * Add queries to get the events for /context requests * Move /context to the syncapi * Revert "Add QueryEventsAfter" This reverts commit 440a771d10632622e8c65d35fe90f0804bc98862. * Simplify getting the required events * Apply RoomEventFilter when getting events * Add passing tests * Remove logging * Remove unused SQL statements Update comments & add TODO
2022-02-18Expose sync endpoints via `/v3` (#2203)Neil Alexander
2021-07-20Rename Riot to Element (#1874)S7evinK
* s/riot/element/g Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * fix formatting Co-authored-by: kegsay <kegan@matrix.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-03-03Increase gocyclo complexity to 25 (and remove all but 2 golint directives ↵Neil Alexander
related to it) (#1783)
2021-01-19Basic sync filtering (#1721)Neil Alexander
* Add some filtering (postgres only for now) * Fix build error * Try to use request filter * Use default filter as a template when retrieving from the database * Remove unused strut * Update sytest-whitelist * Add filtering to SelectEarlyEvents * Fix Postgres selectEarlyEvents query * Attempt filtering on SQLite * Test limit, set field for limit/order in prepareWithFilters * Remove debug logging, add comments * Tweaks, debug logging * Separate SQLite stream IDs * Fix filtering in current state table * Fix lock issues * More tweaks * Current state requires room ID * Review comments
2021-01-13Update /messages pagination token behaviour (#1708)Neil Alexander
* Tweak pagination tokens * start should be the specified from * Don't reverse start and end * Tweak getStartEnd again * Update sytest-whitelist * NOTSPEC: Re-add iOS end of topology
2020-12-16Add start_stream to /messages (#1648)Kegsay
2020-12-16NOTSPEC: Make ?from= optional in /messages (#1647)Kegsay
2020-12-10Refactor sync tokens (#1628)Neil Alexander
* Refactor sync tokens * Comment out broken notifier test * Update types, sytest-whitelist * More robust token checking * Remove New functions for streaming tokens * Export Logs in StreamingToken * Fix tests
2020-12-02Top-level setup package (#1605)Neil Alexander
* Move config, setup, mscs into "setup" top-level folder * oops, forgot the EDU server * Add setup * goimports
2020-11-16Pass pointers to events — reloaded (#1583)Neil Alexander
* Pass events as pointers * Fix lint errors * Update gomatrixserverlib * Update gomatrixserverlib * Update to matrix-org/gomatrixserverlib#240
2020-11-05Implement forgetting about rooms (#1572)S7evinK
* Add basic storage methods * Add internal api handler * Add check for forgotten room * Add /rooms/{roomID}/forget endpoint * Add missing rsAPI method * Remove unused parameters * Add passing tests Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing file * Add postgres migration * Add sqlite migration * Use Forgetter to forget room * Remove empty line * Update HTTP status codes It looks like the spec calls for these to be 400, rather than 403: https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-forget Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-05Don't store backfilled events using request context (#1478)Neil Alexander
2020-10-02Add history visibility guards (#1470)Kegsay
* Add history visibility guards Default to 'joined' visibility to avoid leaking events, until we get around to implementing history visibility completely. Related #617 * Don't apply his vis checks on shared rooms * Fix order of checks * Linting and remove another misleading check * Update whitelist
2020-09-27unbreak CORS on /capabilities.Matthew Hodgson
fixes https://github.com/vector-im/element-web/issues/15297
2020-08-13API setup refactoring (#1266)Neil Alexander
* Start HTTP endpoint refactoring * Update SetupAndServeHTTP * Fix builds * Don't set up external listener if no address configured * TLS HTTP setup * Break apart client/federation/key/media muxes * Tweaks * Fix P2P demos * Fix media API routing * Review comments @Kegsay * Update sample config * Fix gobind build * Fix External -> Public in federation API test
2020-08-10Configuration format v1 (#1230)Neil Alexander
* Initial pass at refactoring config (not finished) * Don't forget current state and EDU servers * More shifting around * Update server key API tests * Fix roomserver test * Fix more tests * Further tweaks * Fix current state server test (sort of) * Maybe fix appservices * Fix client API test * Include database connection string in database options * Fix sync API build * Update config test * Fix unit tests * Fix federation sender build * Fix gobind build * Set Listen address for all services in HTTP monolith mode * Validate config, reinstate appservice derived in directory, tweaks * Tweak federation API test * Set MaxOpenConnections/MaxIdleConnections to previous values * Update generate-config
2020-07-30Implement /keys/changes (#1232)Kegsay
* Implement /keys/changes And refactor QueryKeyChanges to accept a `to` offset. * Unbreak tests * Sort keys when serialising log tokens
2020-06-26Implement /sync `limited` and read timeline limit from stored filters (#1168)Kegsay
* Move filter table to syncapi where it is used * Implement /sync `limited` and read timeline limit from stored filters We now fully handle `room.timeline.limit` filters (in-line + stored) and return the right value for `limited` syncs. * Update whitelist * Default to the default timeline limit if it's unset, also strip the extra event correctly * Update whitelist
2020-06-17Hacks for supporting Riot iOS (#1148)Neil Alexander
* Join room body is optional * Support deprecated login by user/password * Implement dummy key upload endpoint * Make a very determinate end to /messages if we hit the create event in back-pagination * Linting
2020-06-16Make userapi responsible for checking access tokens (#1133)Kegsay
* Make userapi responsible for checking access tokens There's still plenty of dependencies on account/device DBs, but this is a start. This is a breaking change as it adds a required config value `listen.user_api`. * Cleanup * Review comments and test fix
2020-06-12Rehuffle where things are in the internal package (#1122)Kegsay
renamed: internal/eventcontent.go -> internal/eventutil/eventcontent.go renamed: internal/events.go -> internal/eventutil/events.go renamed: internal/types.go -> internal/eventutil/types.go renamed: internal/http/http.go -> internal/httputil/http.go renamed: internal/httpapi.go -> internal/httputil/httpapi.go renamed: internal/httpapi_test.go -> internal/httputil/httpapi_test.go renamed: internal/httpapis/paths.go -> internal/httputil/paths.go renamed: internal/routing.go -> internal/httputil/routing.go renamed: internal/basecomponent/base.go -> internal/setup/base.go renamed: internal/basecomponent/flags.go -> internal/setup/flags.go renamed: internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go renamed: internal/postgres.go -> internal/sqlutil/postgres.go renamed: internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go renamed: internal/sql.go -> internal/sqlutil/sql.go
2020-06-11Roomserver API changes (#1118)Kegsay
* s/QueryBackfill/PerformBackfill/g * OutputEvent now includes AddStateEvents which contain the full event of extra state events * Only include adds not the current event * Get adding state right
2020-05-22Separate muxes for public and internal APIs (#1056)Neil Alexander
* Separate muxes for public and internal APIs * Update client-api-proxy and federation-api-proxy so they don't add /api to the path * Tidy up * Consistent HTTP setup * Set up prefixes properly
2020-05-21Fix #897 and shuffle directory around (#1054)Kegsay
* Fix #897 and shuffle directory around * Update find-lint * goimports Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-05-20sytest: Make 'Inbound federation can backfill events' pass (#1051)Kegsay
* sytest: Make 'Inbound federation can backfill events' pass This breaks 'Outbound federation can backfill events' because now we are returning the right number of events, which the previous test was relying on. Previously, /messages was backfilling the membership event, causing the test to pass. Now we are no longer backfilling the membership event due to the change in this commit, causing the test to fail. The test should instead be returning the membership event locally from synacpis database, but it doesn't do it fast enough, resulting in a no-op /sync response with a next_batch=s0_0 which will never pick up the local membership event when it rolls in. The test does attempt to retry, but doesn't take the new next_batch=s1_0 resulting in it missing from the /messages response. * Linting
2020-05-19Make "Outbound federation can backfill events" pass sytest (#1049)Kegsay
- Use a backfill limit of 100 regardless of what was asked. - Special case the create event for `StateIDsBeforeEvent` - Trim to the limit in `syncapi`
2020-05-14Cleanup syncapi topology logic (#1035)Kegsay
* Cleanup syncapi topology logic * Variable renaming * comments
2020-05-13syncapi: Rename and split out tokens (#1025)Kegsay
* syncapi: Rename and split out tokens Previously we used the badly named `PaginationToken` which was used for both `/sync` and `/messages` requests. This quickly became confusing because named fields like `PDUPosition` meant different things depending on the token type. Instead, we now have two token types: `TopologyToken` and `StreamingToken`, both of which have fields which make more sense for their specific situations. Updated the codebase to use one or the other. `PaginationToken` still lives on as `syncToken`, an unexported type which both tokens rely on. This allows us to guarantee that the specific mappings of positions to a string remain solely under the control of the `types` package. This enables us to move high-level conceptual things like "decrement this topological token" to function calls e.g `TopologicalToken.Decrement()`. Currently broken because `/messages` seemingly used both stream and topological tokens, though I need to confirm this. * final tweaks/hacks * spurious logging * Review comments and linting
2020-05-01Fix ordering when backfilling (#1000)Kegsay
* Fix ordering when backfilling The problem was that we weren't sorting the returned events by depth when sending them back to the caller, instead we were sorting by prev_events which is not the same thing. * Fixup tests
2020-05-01Correctly generate backpagination tokens for events which have the same ↵Kegsay
depth (#996) * Correctly generate backpagination tokens for events which have the same depth With tests. Unfortunately the code around here is hard to understand. There will be a subsequent PR which fixes this up now that we have a test harness in place. * Add postgres impl * More linting * Fix psql statement so it actually works
2020-05-01Consolidation of roomserver APIs (#994)Neil Alexander
* Consolidation of roomserver APIs * Comment out alias tests for now, they are broken * Wire AS API into roomserver again * Roomserver didn't take asAPI param before so return to that * Prevent roomserver asking AS API for alias info * Rename some files * Remove alias_test, incoherent tests and unwanted appservice integration * Remove FS API inject on syncapi component
2020-04-28Implement backfill in the roomserver (#983)Kegsay
* Initial cut for backfilling The syncserver now asks the roomserver via QueryBackfill (which already existed to *handle* backfill requests) which then makes federation requests via gomatrixserverlib.RequestBackfill. Currently, tests fail on subsequent /messages requests because we don't know which servers are in the room, because we are unable to get state snapshots from a backfilled event because that code doesn't exist yet. * WIP backfill, doesn't work * Make initial backfill pass checks * Persist backfilled events with state snapshots * Remove debug lines * Linting * Review comments
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-18 16:20:17 +0000