aboutsummaryrefslogtreecommitdiff
path: root/federationapi
AgeCommit message (Collapse)Author
2020-07-02Replace publicroomsapi with a combination of ↵Kegsay
clientapi/roomserver/currentstateserver (#1174) * Use content_value instead of membership * Fix build * Replace publicroomsapi with a combination of clientapi/roomserver/currentstateserver - All public rooms paths are now handled by clientapi - Requests to (un)publish rooms are sent to the roomserver via `PerformPublish` which are stored in a new `published_table.go` - Requests for public rooms are handled in clientapi by: * Fetch all room IDs which are published using `QueryPublishedRooms` on the roomserver. * Apply pagination parameters to the slice. * Do a `QueryBulkStateContent` request to the currentstateserver to pull out required state event *content* (not entire events). * Aggregate and return the chunk. Mostly but not fully implemented (DB queries on currentstateserver are missing) * Fix pq query * Make postgres work * Make sqlite work * Fix tests * Unbreak pagination tests * Linting
2020-06-29Don't include current state in processEventWithMissingState (#1126)Neil Alexander
* Don't include current state in processEventWithMissingState * Remove lookupCurrentState as not needed Co-authored-by: Kegsay <kegan@matrix.org>
2020-06-25Handle invite v1 (#1165)Kegsay
* Implement invite v1 for sytest mainly * Bump gmsl version which falls back to invite v1 if v2 404s * Update whitelist
2020-06-24Add PerformInvite and refactor how errors get handled (#1158)Kegsay
* Add PerformInvite and refactor how errors get handled - Rename `JoinError` to `PerformError` - Remove `error` from the API function signature entirely. This forces errors to be bundled into `PerformError` which makes it easier for callers to detect and handle errors. On network errors, HTTP clients will make a `PerformError`. * Unbreak everything; thanks Go! * Send back JSONResponse according to the PerformError * Update federation invite code too
2020-06-23Add /send restrictions and return correct error codes (#1156)Kegsay
* Add /send restrictions and return correct error codes - Max 50 PDUs / 100 EDUs - Fail the transaction when PDUs contain bad JSON * Update whitelist * Unbreak test * Linting
2020-06-23Fix room checks for /state and /state_ids (#1155)Kegsay
We would return a 403 first (as the server would not be allowed to see this event) and only then return a 404 if the event is not in the given room. We now invert those checks for /state and /state_ids to make the tests pass.
2020-06-23/send auth errors are silent (#1149)Kegsay
* /send auth errors are silent * Fix test
2020-06-17Do not wrap send_join errors on /v1/send_join (#1143)Kegsay
* Do not wrap v1 send_join errors in [code, body] * Don't wrap errors
2020-06-16Make federationapi use userapi (#1135)Kegsay
Removes dependencies on account DB, device DB and ASAPI.
2020-06-15Fix rooms v3 url paths for good - with tests (#1130)Kegsay
* Fix rooms v3 url paths for good - with tests - Add a test rig around `federationapi` to test routing. - Use `JSONVerifier` over `KeyRing` so we can stub things out more easily. - Add `test.NopJSONVerifier` which verifies nothing. - Add `base.BaseMux` which is the original `mux.Router` used to spawn public/internal routers. - Listen on `base.BaseMux` and not the default serve mux as it cleans paths which we don't want. - Factor out `ListenAndServe` to `test.ListenAndServe` and add flag for listening on TLS. * Fix comments * Linting
2020-06-12Minor perf/debugging improvements (#1121)Kegsay
* Minor perf/debugging improvements - publicroomsapi: Don't call QueryEventsByID with no event IDs - appservice: Consume only if there are 1 or more ASes - roomserver: don't keep a copy of the request "for debugging" - we trace now * fedsender: return early if we have no destinations * Unbreak tests
2020-06-12Rehuffle where things are in the internal package (#1122)Kegsay
renamed: internal/eventcontent.go -> internal/eventutil/eventcontent.go renamed: internal/events.go -> internal/eventutil/events.go renamed: internal/types.go -> internal/eventutil/types.go renamed: internal/http/http.go -> internal/httputil/http.go renamed: internal/httpapi.go -> internal/httputil/httpapi.go renamed: internal/httpapi_test.go -> internal/httputil/httpapi_test.go renamed: internal/httpapis/paths.go -> internal/httputil/paths.go renamed: internal/routing.go -> internal/httputil/routing.go renamed: internal/basecomponent/base.go -> internal/setup/base.go renamed: internal/basecomponent/flags.go -> internal/setup/flags.go renamed: internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go renamed: internal/postgres.go -> internal/sqlutil/postgres.go renamed: internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go renamed: internal/sql.go -> internal/sqlutil/sql.go
2020-06-11Roomserver API changes (#1118)Kegsay
* s/QueryBackfill/PerformBackfill/g * OutputEvent now includes AddStateEvents which contain the full event of extra state events * Only include adds not the current event * Get adding state right
2020-06-11Remove unused internal APIs (#1117)Kegsay
2020-06-10Remove clientapi producers which aren't actually producers (#1111)Kegsay
* Remove clientapi producers which aren't actually producers They are actually just convenience wrappers around the internal APIs for roomserver/eduserver. Move their logic to their respective `api` packages and call them directly. * Remove TODO * unbreak ygg
2020-06-08Split out SetupFooComponent (#1106)Kegsay
* Split out adding HTTP routes from making internal APIs for clarity * Split out more components * Split out more things * Finish converting * internal mux for internal routes
2020-06-05Use AuthChainProvider to try and speed up federated joins (#1100)Neil Alexander
* Use MissingAuthEventHandler on performjoin to try and speed up cases where we have missing events * Update gomatrixserverlib * Use supplied room version * Use AuthChainProvider * Tweaks * Update gomatrixserverlib * Signature checks
2020-06-04s/Base64String/Base64Bytes/g (#1093)Kegsay
* s/Base64String/Base64Bytes/g Requires https://github.com/matrix-org/gomatrixserverlib/pull/203 to land first * update gmsl
2020-06-04Encode v3 event IDs correctly (#1090)Kegsay
2020-06-04Enable room version 6 (#1087)Neil Alexander
* Return bad request on CS API /send if bad JSON * Return some more M_BAD_JSON in the right places * nolint because damnit gocyclo all I added was a type check for an error * Update gomatrixserverlib * Update gomatrixserverlib * Update sytest-whitelist * Update gomatrixserverlib * Update sytest-whitelist * NotJSON -> BadJSON
2020-06-02Reset backoff on incoming federation (#1080)Neil Alexander
* Reset backoffs in response to incoming federation requests * Federation wakeups no more than once per minute per origin
2020-06-02Use gomatrixserverlib device structs (#1079)Neil Alexander
2020-06-01Send-to-device support (#1072)Neil Alexander
* Groundwork for send-to-device messaging * Update sample config * Add unstable routing for now * Send to device consumer in sync API * Start the send-to-device consumer * fix indentation in dendrite-config.yaml * Create send-to-device database tables, other tweaks * Add some logic for send-to-device messages, add them into sync stream * Handle incoming send-to-device messages, count them with EDU stream pos * Undo changes to test * pq.Array * Fix sync * Logging * Fix a couple of transaction things, fix client API * Add send-to-device test, hopefully fix bugs * Comments * Refactor a bit * Fix schema * Fix queries * Debug logging * Fix storing and retrieving of send-to-device messages * Try to avoid database locks * Update sync position * Use latest sync position * Jiggle about sync a bit * Fix tests * Break out the retrieval from the update/delete behaviour * Comments * nolint on getResponseWithPDUsForCompleteSync * Try to line up sync tokens again * Implement wildcard * Add all send-to-device tests to whitelist, what could possibly go wrong? * Only care about wildcard when targeted locally * Deduplicate transactions * Handle tokens properly, return immediately if waiting send-to-device messages * Fix sync * Update sytest-whitelist * Fix copyright notice (need to do more of this) * Comments, copyrights * Return errors from Do, fix dendritejs * Review comments * Comments * Constructor for TransactionWriter * defletions * Update gomatrixserverlib, sytest-blacklist
2020-05-28Hopefully fix get_missing_events (#1070)Neil Alexander
2020-05-27Return 500 when processing a transaction fails fatally (#1066)Neil Alexander
2020-05-22Separate muxes for public and internal APIs (#1056)Neil Alexander
* Separate muxes for public and internal APIs * Update client-api-proxy and federation-api-proxy so they don't add /api to the path * Tidy up * Consistent HTTP setup * Set up prefixes properly
2020-05-21Fix #897 and shuffle directory around (#1054)Kegsay
* Fix #897 and shuffle directory around * Update find-lint * goimports Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-05-20sytest: Add remaining backfill tests (#1052)Kegsay
One failed because of `null` instead of `[]` in HTTP responses. One failed because we hadn't implemented in-line filter limits!
2020-05-20sytest: Make 'Inbound federation can backfill events' pass (#1051)Kegsay
* sytest: Make 'Inbound federation can backfill events' pass This breaks 'Outbound federation can backfill events' because now we are returning the right number of events, which the previous test was relying on. Previously, /messages was backfilling the membership event, causing the test to pass. Now we are no longer backfilling the membership event due to the change in this commit, causing the test to fail. The test should instead be returning the membership event locally from synacpis database, but it doesn't do it fast enough, resulting in a no-op /sync response with a next_batch=s0_0 which will never pick up the local membership event when it rolls in. The test does attempt to retry, but doesn't take the new next_batch=s1_0 resulting in it missing from the /messages response. * Linting
2020-05-15Return user_id and stream_id in federated devices query (#1040)Neil Alexander
2020-05-13Reduce 500s (#1017)Neil Alexander
* Try to avoid returning 500s on /send * Don't return 500s from media API download requests * Don't 500 on context errors * Update sytest-whitelist * Fix lint, add comments
2020-05-12Implement /get_missing_events (#1022)Kegsay
* WIP get_missing_events work * More WIP get_missing_events work * First working /get_missing_events implementation Flakey currently due to racing between /sync and /send * Final tweaks * Remove log lines * Linting * go mod tidy * Clamp min depth to 0 * sort events by depth because sytest makes me sad Specifically I think it's https://github.com/matrix-org/sytest/blob/4172585c2521ec6d640b4b580080276da1ab5353/lib/SyTest/Federation/Client.pm#L265 to blame here.
2020-05-06only send new events to RS; add tests for /state_ids and /event (#1011)Kegsay
* only send new events to RS; add tests for /state_ids and /event * Review comments: send in auth event order * Ignore order of state events for this test as RespState.Events is non-deterministic
2020-05-06Add tests around federationapi's txnReq (#1010)Kegsay
* Add necessary stubs for testing txnReq * Add basic tests
2020-05-05Fetch events by ID rather than use current state as this includes auth ↵Kegsay
events (#1009)
2020-05-05Prefer /state_ids when missing state across federation (#1008)Kegsay
* Prefer /state_ids when missing state across federation * Linting * Better logging
2020-05-05Fix RespState/RespSendJoin (#1005)Neil Alexander
* Update gmsl * Update gomatrixserverlib * Add link to spec
2020-05-04Roomserver perform leave (#1004)Neil Alexander
* First pass at PerformLeave * Fix SQLite bulkSelectEventStateKey * Update gomatrixserverlib * Fix bugs * Tidy a bit * Satisfy King Linter * Review comments * Review comments * Fix constants in SQLite event state keys table
2020-05-04Roomserver perform join (#1001)Neil Alexander
* Add PerformJoin template * Try roomserver perform join * Send correct server name to FS API * Pass through content, try to handle multiple server names * Fix local server checks * Don't refer to non-existent error * Add directory lookups of aliases * Remove unneeded parameters * Don't repeat join events into the roomserver * Unmarshal the content, that would help * Check if the user is already in the room in the fedeationapi too * Return incompatible room version error * Use Membership, don't try more servers than needed * Review comments, make FS API take list of servernames, dedupe them, break out of loop properly on success * Tweaks
2020-05-01Consolidation of roomserver APIs (#994)Neil Alexander
* Consolidation of roomserver APIs * Comment out alias tests for now, they are broken * Wire AS API into roomserver again * Roomserver didn't take asAPI param before so return to that * Prevent roomserver asking AS API for alias info * Rename some files * Remove alias_test, incoherent tests and unwanted appservice integration * Remove FS API inject on syncapi component
2020-04-29Honour history_visibility when backfilling (#990)Kegsay
* Make backfill work for shared history visibility * fetch missing state on backfill to remember snapshots correctly * Fix gmsl to not mux in auth events into room state * Whoops * Linting
2020-04-29Federation sender API remodel (#988)Neil Alexander
* Define an input API for the federationsender * Wiring for rooomserver input API and federation sender input API * Whoops, commit common too * Merge input API into query API * Rename FederationSenderQueryAPI to FederationSenderInternalAPI * Fix dendritejs * Rename Input to Perform * Fix a couple of inputs -> performs * Remove needless storage interface, add comments
2020-04-28Implement backfill in the roomserver (#983)Kegsay
* Initial cut for backfilling The syncserver now asks the roomserver via QueryBackfill (which already existed to *handle* backfill requests) which then makes federation requests via gomatrixserverlib.RequestBackfill. Currently, tests fail on subsequent /messages requests because we don't know which servers are in the room, because we are unable to get state snapshots from a backfilled event because that code doesn't exist yet. * WIP backfill, doesn't work * Make initial backfill pass checks * Persist backfilled events with state snapshots * Remove debug lines * Linting * Review comments
2020-04-27Update gomatrixserverlibNeil Alexander
2020-04-24More invite support (#979)Neil Alexander
* Update gomatixserverlib * Try to build invite stripped state if not given to us * SendInvite improvements * Transpose invite_room_state into invite_state.events for sync API * Remove syncapi debugging output * Use RespInviteV2 * Update gomatrixserverlib * Send the invite event as a normal roomserver event too, for incorporating into room (should this be done by the roomserver automatically for invite inputs?) * Federation sender use invite_room_state, room server try to insert membership state * Check supported room versions on the invite endpoint * Prevent roomserver query API from trying to handle requests for stub rooms * Adding a nolint * Replace IsRoomStub with RoomNIDExcludingStubs, fix query API to use that instead * Review comments
2020-04-22Response from /send_join now use gomatrixserverlib.RespSendJoin (#796)Behouba Manassé
* response from /send_join now use gomatrixserverlib.RespSendJoin * Update Dendrite gomatrixserverlib version * Fix spelling Co-authored-by: Andrew Morgan <andrew@amorgan.xyz> Co-authored-by: Cnly <minecnly@gmail.com> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-04-20Perspective key fetching, some federation room join fixes (#975)Neil Alexander
* Update gomatrixserverlib * Test matrix.org as perspective key server * Base64 decode better * Optional strict validity checking in gmsl * Update gomatrixserverlib * Attempt to find missing auth events over federation (this shouldn't happen but I am guessing there is a synapse bug involved where we don't get all of the auth events) * Update gomatrixserverlib, debug logging * Remove debugging output * More verbose debugging * Print outliers * Increase timeouts for testing, observe contexts before trying to join over more servers * Don't block on roomserver (experimental) * Don't block on roomserver * Update gomatrixserverlib * Update gomatrixserverlib * Configurable perspective key fetchers * Output number of configured keys for perspective * Example perspective config included * Undo debug stack trace * Undo debug stack trace * Restore original HTTP listener in monolith * Fix lint * Review comments * Set default HTTP server timeout to 5 minutes now, block again when joining * Don't use HTTP address for HTTPS whoops * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Actually add perspectives * Actually add perspectives * Update gomatrixserverlib
2020-04-16Federation fixes and error handling (#970)Neil Alexander
* Improve error handling in federation /send endpoint a bit * Remove unknownRoomError, use unmarshalError when unable to get room ID * Swap out a couple more internal server errors * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Return bad limit in error * Same with domain/userid
2020-04-15Tweaks to backfill (#964)Neil Alexander
2020-04-14Move /room/{roomID}/state endpoints into client API (#606) (#962)Neil Alexander
* Move /room/{roomID}/state endpoints into client API (#606) * Update sytest-whitelist * Blacklist tests which rely on endpoints we don't implement
generated by cgit v1.2.3 (git 2.26.2) at 2025-02-21 12:49:12 +0000