aboutsummaryrefslogtreecommitdiff
path: root/roomserver/api
AgeCommit message (Collapse)Author
2021-07-21Various alias fixes (#1934)Neil Alexander
* Generate m.room.canonical_alias instead of legacy m.room.aliases * Add omitempty tags * Add aliases endpoint to client API * Check power levels when setting aliases * Don't return null on /aliases * Don't return error if the state event fails * Update sytest-whitelist * Don't send updated m.room.canonical_alias events * Don't check PLs after all because for local aliases they are apparently irrelevant * Fix some bugs * Allow deleting a local alias with enough PL * Fix some more bugs * Update sytest-whitelist * Fix copyright notices * Review comments
2021-07-21Optimise QueryServerJoinedToRoom (#1933)Neil Alexander
* Optimise checking if a server is in a room * Fix queries * Fix queries
2021-07-09Add more optimised code path for checking if we're in a room (#1909)Neil Alexander
* Add more optimised code path for checking if we're in a room * Fix database queries * Fix federation API test * Fix logging * Review comments * Make separate API call for room membership
2021-03-03Fix appsevice alias queries part 2 (#1684)Will Hunt
* Check membership of room * Use QueryStateAfterEventsResponse * Fix complexity * Add field ShouldHitAppservice to GetRoomIDForAlias * Hit appservice when trying to join a non-existent alias * remove unused * Changes that I made a long time ago * Rename to appserviceJoinedAtEvent * Check membership in GetMemberships * Update QueryMembershipsForRoom * Tweaks in client API * Update appserviceJoinedAtEvent * Comments * Try QueryMembershipForUser instead * Undo some changes to client API that shouldn't be needed * More /event tweaks * Refactor /event bit * Go back to QueryMembershipsForRoom because appservices are hard * Fix bugs in onMessage * Add comments * More logical naming, clean up a bit Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-03-03Send events to appservice based on room membership (#1680)Will Hunt
* Check membership of room * Use QueryStateAfterEventsResponse * Fix complexity * Changes that I made a long time ago * Rename to appserviceJoinedAtEvent * Check membership in GetMemberships * Update QueryMembershipsForRoom * Tweaks in client API * Update appserviceJoinedAtEvent * Comments * Try QueryMembershipForUser instead * Undo some changes to client API that shouldn't be needed * More /event tweaks * Refactor /event bit * Go back to QueryMembershipsForRoom because appservices are hard * Fix bugs in onMessage * Add comments Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-03-03Increase gocyclo complexity to 25 (and remove all but 2 golint directives ↵Neil Alexander
related to it) (#1783)
2021-01-22Peeking over federation via MSC2444 (#1391)Matthew Hodgson
* a very very WIP first cut of peeking via MSC2753. doesn't yet compile or work. needs to actually add the peeking block into the sync response. checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it. * make PeekingDeviceSet private * add server_name param * blind stab at adding a `peek` section to /sync * make it build * make it launch * add peeking to getResponseWithPDUsForCompleteSync * cancel any peeks when we join a room * spell out how to runoutside of docker if you want speed * fix SQL * remove unnecessary txn for SelectPeeks * fix s/join/peek/ cargocult fail * HACK: Track goroutine IDs to determine when we write by the wrong thread To use: set `DENDRITE_TRACE_SQL=1` then grep for `unsafe` * Track partition offsets and only log unsafe for non-selects * Put redactions in the writer goroutine * Update filters on writer goroutine * wrap peek storage in goid hack * use exclusive writer, and MarkPeeksAsOld more efficiently * don't log ascii in binary at sql trace... * strip out empty roomd deltas * re-add txn to SelectPeeks * re-add accidentally deleted field * reject peeks for non-worldreadable rooms * move perform_peek * fix package * correctly refactor perform_peek * WIP of implementing MSC2444 * typo * Revert "Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking" This reverts commit 3cebd8dbfbccdf82b7930b7b6eda92095ca6ef41, reversing changes made to ed4b3a58a7855acc43530693cc855b439edf9c7c. * (almost) make it build * clean up bad merge * support SendEventWithState with optional event * fix build & lint * fix build & lint * reinstate federated peeks in the roomserver (doh) * fix sql thinko * todo for authenticating state returned by /peek * support returning current state from QueryStateAndAuthChain * handle SS /peek * reimplement SS /peek to prod the RS to tell the FS about the peek * rename RemotePeeks as OutboundPeeks * rename remote_peeks_table as outbound_peeks_table * add perform_handle_remote_peek.go * flesh out federation doc * add inbound peeks table and hook it up * rename ambiguous RemotePeek as InboundPeek * rename FSAPI's PerformPeek as PerformOutboundPeek * setup inbound peeks db correctly * fix api.SendEventWithState with no event * track latestevent on /peek * go fmt * document the peek send stream race better * fix SendEventWithRewrite not to bail if handed a non-state event * add fixme * switch SS /peek to use SendEventWithRewrite * fix comment * use reverse topo ordering to find latest extrem * support postgres for federated peeking * go fmt * back out bogus go.mod change * Fix performOutboundPeekUsingServer * Fix getAuthChain -> GetAuthChain * Fix build issues * Fix build again * Fix getAuthChain -> GetAuthChain * Don't repeat outbound peeks for the same room ID to the same servers * Fix lint * Don't omitempty to appease sytest Co-authored-by: Kegan Dougal <kegan@matrix.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-12-18Fix #1655 by re-adding the appservice alias query (#1660)Neil Alexander
2020-12-04MSC2836 threading: part 2 (#1596)Kegsay
* Update GMSL * Add MSC2836EventRelationships to fedsender * Call MSC2836EventRelationships in reqCtx * auth remote servers * Extract room ID and servers from previous events; refactor a bit * initial cut of federated threading * Use the right client/fed struct in the response * Add QueryAuthChain for use with MSC2836 * Add auth chain to federated response * Fix pointers * under CI: more logging and enable mscs, nil fix * Handle direction: up * Actually send message events to the roomserver.. * Add children and children_hash to unsigned, with tests * Add logic for exploring threads and tracking children; missing storage functions * Implement storage functions for children * Add fetchUnknownEvent * Do federated hits for include_children if we have unexplored children * Use /ev_rel rather than /event as the former includes child metadata * Remove cross-room threading impl * Enable MSC2836 in the p2p demo * Namespace mscs db * Enable msc2836 for ygg Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-12-03Peeking updates (#1607)Neil Alexander
* Add unpeek * Don't allow peeks into encrypted rooms * Fix send tests * Update consumers
2020-11-19MSC2836: Threading - part one (#1589)Kegsay
* Add mscs/hooks package, begin work for msc2836 * Flesh out hooks and add SQL schema * Begin implementing core msc2836 logic * Add test harness * Linting * Implement visibility checks; stub out APIs for tests * Flesh out testing * Flesh out walkThread a bit * Persist the origin_server_ts as well * Edges table instead of relationships * Add nodes table for event metadata * LEFT JOIN to extract origin_server_ts for children * Add graph walking structs * Implement walking algorithm * Add more graph walking tests * Add auto_join for local rooms * Fix create table syntax on postgres * Add relationship_room_id|servers to the unsigned section of events * Persist the parent room_id/servers in edge metadata Other events cannot assert the true room_id/servers for the parent event, only make claims to them, hence why this is edge metadata. * guts to pass through room_id/servers * Refactor msc2836 to allow handling from federation * Add JoinedVia to PerformJoin responses * Fix tests; review comments
2020-11-16Pass pointers to events — reloaded (#1583)Neil Alexander
* Pass events as pointers * Fix lint errors * Update gomatrixserverlib * Update gomatrixserverlib * Update to matrix-org/gomatrixserverlib#240
2020-11-05Implement forgetting about rooms (#1572)S7evinK
* Add basic storage methods * Add internal api handler * Add check for forgotten room * Add /rooms/{roomID}/forget endpoint * Add missing rsAPI method * Remove unused parameters * Add passing tests Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add missing file * Add postgres migration * Add sqlite migration * Use Forgetter to forget room * Remove empty line * Update HTTP status codes It looks like the spec calls for these to be 400, rather than 403: https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-forget Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-19KindOld (#1531)Neil Alexander
* Add KindOld * Don't process latest events/memberships for old events * Allow federationsender to ignore duplicate key entries when LatestEventIDs is duplicated by RS output events * Signal to downstream components if an event has become a forward extremity * Don't exclude from sync * Soft-fail checks on KindNew * Don't run the latest events updater at all for KindOld * Don't make federation sender change after all * Kind in federation sender join * Don't send isForwardExtremity * Fix syncapi * Update comments * Fix SendEventWithState * Update sytest-whitelist * Generate old output events * Sync API consumes old room events * Update comments
2020-10-14Improved state handling in /send (#1521)Neil Alexander
* Capture errors * Don't request only state key tuples needed for auth (we end up discarding room state this way) * QueryStateAfterEvent returns all state when no tuples supplied * Resolve state * Comments
2020-10-07Deep checking of forward extremities (#1491)Neil Alexander
* Deep forward extremity calculation * Use updater txn * Update error * Update error * Create previous event references in StoreEvent * Use latest events updater to row-lock prev events * Fix unexpected fallthrough * Fix deadlock * Don't roll back * Update comments in calculateLatest * Don't include events that we can't find references for in the forward extremities * Add another passing test
2020-10-06Remove KindRewrite (#1481)Neil Alexander
* Don't send rewrite events * Remove final traces of rewrite events * Remove test that is no longer needed * Revert "Remove test that is no longer needed" This reverts commit 9a45babff690480acd656a52f2c2950a5f7e9ada. * Update test to use KindOutlier
2020-09-29Fetch missing auth events, implement QueryMissingAuthPrevEvents, try other ↵Neil Alexander
servers in room for /event and /get_missing_events (#1450) * Try to ask other servers in the room for missing events if the origin won't provide them * Logging * More logging * Implement QueryMissingAuthPrevEvents * Try to get missing auth events badly * Use processEvent * Logging * Update QueryMissingAuthPrevEvents * Try to find missing auth events * Patchy fix for test * Logging tweaks * Send auth events as outliers * Update check in QueryMissingAuthPrevEvents * Error responses * More return codes * Don't return error on reject/soft-fail since it was ultimately handled * More tweaks * More error tweaks
2020-09-24Reject make_join for empty rooms (#1439)Neil Alexander
* Sanity-check room version on RS event input * Update gomatrixserverlib * Reject make_join when no room members are left * Revert some changes from wrong branch * Distinguish between room not existing and room being abandoned on this server * nolint
2020-09-21Soft-fail (#1364)Neil Alexander
* Initial work on soft-fail * Fix state block retrieval * Copy-pasta QueryLatestEventsAndState code * Fix state lookup * Clean up * Fix up failing sytest * Linting * Update previous events SQLite insert query * Update SQLite InsertPreviousEvent properly * Hopefully fix the event references updates Co-authored-by: Kegan Dougal <kegan@matrix.org>
2020-09-16Implement rejected events (#1426)Kegsay
* WIP Event rejection * Still send back errors for rejected events Instead, discard them at the federationapi /send layer rather than re-implementing checks at the clientapi/PerformJoin layer. * Implement rejected events Critically, rejected events CAN cause state resolution to happen as it can merge forks in the DAG. This is fine, _provided_ we do not add the rejected event when performing state resolution, which is what this PR does. It also fixes the error handling when NotAllowed happens, as we were checking too early and needlessly handling NotAllowed in more than one place. * Update test to match reality * Modify InputRoomEvents to no longer return an error Errors do not serialise across HTTP boundaries in polylith mode, so instead set fields on the InputRoomEventsResponse. Add `Err()` function to make the API shape basically the same. * Remove redundant returns; linting * Update blacklist
2020-09-15Handle state with input event as new events (#1415)Neil Alexander
* SendEventWithState events as new * Use cumulative state IDs for final event * Error wrapping in calculateAndSetState * Handle overwriting same event type and state key * Hacky way to spot historical events * Don't exclude from sync * Don't generate output events when rewriting forward extremities * Update output event check * Historical output events * Define output room event type * Notify key changes on state * Don't send our membership event twice * Deduplicate state entries * Tweaks * Remove unnecessary nolint * Fix current state upsert in sync API * Send auth events as outliers, state events as rewrite * Sync API don't consume state events * Process events actually * Improve outlier check * Fix local room check * Remove extra room check, it seems to break the whole damn world * Fix federated join check * Fix nil pointer exception * Better comments on DeduplicateStateEntries * Reflow forced federated joins * Don't force federated join for possibly even local invites * Comment SendEventWithState better * Rewrite room state in sync API storage * Add TODO * Clean up all room data when receiving create event * Don't generate output events for rewrites, but instead notify that state is rewritten on the final new event * Rename to PurgeRoom * Exclude backfilled messages from /sync * Split out rewriting state from updating state from state res Co-authored-by: Kegan Dougal <kegan@matrix.org>
2020-09-10Peeking via MSC2753 (#1370)Matthew Hodgson
Initial implementation of MSC2753, as tested by https://github.com/matrix-org/sytest/pull/944. Doesn't yet handle unpeeks, peeked EDUs, or history viz changing during a peek - these will follow. https://github.com/matrix-org/dendrite/pull/1370 has full details.
2020-09-07Remove QueryBulkStateContent from current state server (#1404)Kegsay
* Remove QueryBulkStateContent from current state server Expected fail due to db impl not existing * Implement query bulk state content * Fix up rejecting invites over federation * Fix bulk content marshalling
2020-09-03Move currentstateserver API to roomserver (#1387)Kegsay
* Move currentstateserver API to roomserver Stub out DB functions for now, nothing uses the roomserver version yet. * Allow it to startup * Implement some current-state-server storage interface functions * Add missing package
2020-09-03FIFO ordering of input events (#1386)Neil Alexander
* Initial FIFOing of roomserver inputs * Remove EventID response from api.InputRoomEventsResponse * Don't send back event ID unnecessarily * Fix ordering hopefully * Reduce copies, use buffered task channel to reduce contention on other rooms * Fix error handling
2020-08-17Make PerformJoin responsible for sending invite to RS input (#1277)Neil Alexander
* Make PerformJoin send input membership event * Invite input room events in separate goroutine * Don't limit roomserver input events using request context * Synchronous input room events * Nope, that didn't work * oops send state key to GetMembership * Don't generate stripped state in client API more times than necessary, generate output events on receiving end of federated invite * Commit membership updater changes * Tweaks
2020-08-17Synchronous invites (#1273)Neil Alexander
* Refactor invites to be synchronous * Fix synchronous invites * Fix client API return type for send invite error * Linter * Restore PerformError on rsAPI.PerformInvite * Update sytest-whitelist * Don't override PerformError with normal errors * Fix error passing * Un-whitelist a couple of tests * Update sytest-whitelist * Try to handle multiple invite rejections better * nolint * Update gomatrixserverlib * Fix /v1/invite test * Remove replace from go.mod
2020-07-08Finish implementing redactions (#1189)Kegsay
* Add a bit more logging to the fedsender * bugfix: continue sending PDUs if ones are added whilst sending another PDU Without this, the queue goes back to sleep on `<-oq.notifyPDUs` which won't fire because `pendingPDUs` is already > 0. This should fix a flakey sytest. * Break if no txn is sent * WIP syncapi work * More debugging * Bump GMSL version to pull in working Event.Redact * Remove logging * Make redactions work on v3+ * Fix more tests
2020-07-07Emit redacted_event from the roomserver when redactions are validated (#1186)Kegsay
* Emit redacted_event from the roomserver when redactions are validated - Consume them in the currentstateserver and act accordingly. - Add integration test for the roomserver to check that injecting `m.room.redaction` events result in `redacted_event` being emitted. * Linting * Ignore events that redact themselves
2020-07-03Implement local redaction (#1182)Kegsay
* Create redaction events and apply checks (but do not send them) * Send redactions to the roomserver * Linting * Slightly better wording
2020-07-02Replace publicroomsapi with a combination of ↵Kegsay
clientapi/roomserver/currentstateserver (#1174) * Use content_value instead of membership * Fix build * Replace publicroomsapi with a combination of clientapi/roomserver/currentstateserver - All public rooms paths are now handled by clientapi - Requests to (un)publish rooms are sent to the roomserver via `PerformPublish` which are stored in a new `published_table.go` - Requests for public rooms are handled in clientapi by: * Fetch all room IDs which are published using `QueryPublishedRooms` on the roomserver. * Apply pagination parameters to the slice. * Do a `QueryBulkStateContent` request to the currentstateserver to pull out required state event *content* (not entire events). * Aggregate and return the chunk. Mostly but not fully implemented (DB queries on currentstateserver are missing) * Fix pq query * Make postgres work * Make sqlite work * Fix tests * Unbreak pagination tests * Linting
2020-06-25Return remote errors from FS.PerformJoin (#1164)Kegsay
* Return remote errors from FS.PerformJoin Follows the same pattern as PerformJoin on roomserver (no error return). Also return the right format for incompatible room version errors. Makes a bunch of tests pass! * Handle network errors better when returning remote HTTP errors * Linting * Fix tests * Update whitelist, pass network errors through in API=1 mode
2020-06-24Refactor SendMembership - make ban test pass (#1160)Kegsay
* Refactor SendMembership - make ban test pass * Only check invite auth events for local invites
2020-06-24Make same membership transitions 403, add sytests (#1161)Kegsay
* Make same membership transitions 403, add sytests * Update blacklist
2020-06-24Add PerformInvite and refactor how errors get handled (#1158)Kegsay
* Add PerformInvite and refactor how errors get handled - Rename `JoinError` to `PerformError` - Remove `error` from the API function signature entirely. This forces errors to be bundled into `PerformError` which makes it easier for callers to detect and handle errors. On network errors, HTTP clients will make a `PerformError`. * Unbreak everything; thanks Go! * Send back JSONResponse according to the PerformError * Update federation invite code too
2020-06-24Pass join errors through internal API boundaries (#1157)Kegsay
* Pass join errors through internal API boundaries Required for certain invite sytests. We will need to think of a better way of handling this going forwards. * Include m.room.avatar in stripped state; handle trailing slashes when GETing state events * Update whitelist * Update whitelist
2020-06-12Minor perf/debugging improvements (#1121)Kegsay
* Minor perf/debugging improvements - publicroomsapi: Don't call QueryEventsByID with no event IDs - appservice: Consume only if there are 1 or more ASes - roomserver: don't keep a copy of the request "for debugging" - we trace now * fedsender: return early if we have no destinations * Unbreak tests
2020-06-12Add trace logging to RoomserverInternalAPI (#1120)Kegsay
This is a wrapper around whatever impl we have which then logs the function name/request/response/error. Also tweak when we log on kafka streams: only log on the producer side not the consumer side: we've never had issues with comms and having 1 message rather than N would be nice.
2020-06-11Roomserver API changes (#1118)Kegsay
* s/QueryBackfill/PerformBackfill/g * OutputEvent now includes AddStateEvents which contain the full event of extra state events * Only include adds not the current event * Get adding state right
2020-06-11Remove unused internal APIs (#1117)Kegsay
2020-06-10Remove clientapi producers which aren't actually producers (#1111)Kegsay
* Remove clientapi producers which aren't actually producers They are actually just convenience wrappers around the internal APIs for roomserver/eduserver. Move their logic to their respective `api` packages and call them directly. * Remove TODO * unbreak ygg
2020-06-04Convert everything but serverkeyapi to inthttp (#1096)Kegsay
* Convert roomserver to new inthttp format * Convert eduserver to new inthttp format * Convert appservice to new inthttp format
2020-05-26Miscellaneous fixes (#1060)Neil Alexander
* Add missing routing for PerformDirectoryLookupRequest * Tweak output * Fix some bugs in devices * Don't default to federated room joins in response to invite * Update sytest-whitelist * Update comments * Return correct room ID from PerformJoin * Fix appservice and EDU server API setup, update sytest-whitelist * Update sytest-whitelist
2020-05-22Separate muxes for public and internal APIs (#1056)Neil Alexander
* Separate muxes for public and internal APIs * Update client-api-proxy and federation-api-proxy so they don't add /api to the path * Tidy up * Consistent HTTP setup * Set up prefixes properly
2020-05-21Fix #897 and shuffle directory around (#1054)Kegsay
* Fix #897 and shuffle directory around * Update find-lint * goimports Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-05-20sytest: Make 'Inbound federation can backfill events' pass (#1051)Kegsay
* sytest: Make 'Inbound federation can backfill events' pass This breaks 'Outbound federation can backfill events' because now we are returning the right number of events, which the previous test was relying on. Previously, /messages was backfilling the membership event, causing the test to pass. Now we are no longer backfilling the membership event due to the change in this commit, causing the test to fail. The test should instead be returning the membership event locally from synacpis database, but it doesn't do it fast enough, resulting in a no-op /sync response with a next_batch=s0_0 which will never pick up the local membership event when it rolls in. The test does attempt to retry, but doesn't take the new next_batch=s1_0 resulting in it missing from the /messages response. * Linting
2020-05-05Prefer /state_ids when missing state across federation (#1008)Kegsay
* Prefer /state_ids when missing state across federation * Linting * Better logging
2020-05-04Roomserver perform leave (#1004)Neil Alexander
* First pass at PerformLeave * Fix SQLite bulkSelectEventStateKey * Update gomatrixserverlib * Fix bugs * Tidy a bit * Satisfy King Linter * Review comments * Review comments * Fix constants in SQLite event state keys table
2020-05-04Roomserver perform join (#1001)Neil Alexander
* Add PerformJoin template * Try roomserver perform join * Send correct server name to FS API * Pass through content, try to handle multiple server names * Fix local server checks * Don't refer to non-existent error * Add directory lookups of aliases * Remove unneeded parameters * Don't repeat join events into the roomserver * Unmarshal the content, that would help * Check if the user is already in the room in the fedeationapi too * Return incompatible room version error * Use Membership, don't try more servers than needed * Review comments, make FS API take list of servernames, dedupe them, break out of loop properly on success * Tweaks
generated by cgit v1.2.3 (git 2.26.2) at 2025-02-07 02:56:32 +0000