aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-06-17Return the correct /joined_members response and allow ?format=event (#1146)Kegsay
2020-06-17Enable more sytests (#1145)Kegsay
2020-06-17Are we synapse yet: Accept tests without a group rather than dying (#1142)Kegsay
Produces output like: ``` Non-Spec APIs: 0% (0/52 tests) -------------- Non-Spec API : 0% (0/50 tests) Unknown API (no group specified): 0% (0/2 tests) ``` Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-06-17Fallback parsing for Content-Disposition minefield (#1144)Neil Alexander
2020-06-17Do not wrap send_join errors on /v1/send_join (#1143)Kegsay
* Do not wrap v1 send_join errors in [code, body] * Don't wrap errors
2020-06-17add testKegan Dougal
2020-06-17Move account/device DBs to userapi (#1141)Kegsay
2020-06-17Media filename handling improvements (#1140)Neil Alexander
* Derive content ID from hash+filename but preserve dedupe, improve Content-Disposition handling and ASCII handling * Linter fix * Some more comments * Update sytest-whitelist
2020-06-17Make userapi control account creation entirely (#1139)Kegsay
This makes a chokepoint with which we can finally fix 'database is locked' errors on sqlite during account creation
2020-06-16Update whitelist for sytest media fix (#1137)Neil Alexander
* Update sytest-whitelist, are-we-synapse-yet.list * Update gomatrixserverlib * Update gomatrixserverlib * Loop avoidance * Return UTF-8 filenames * Replace quotes only, instead of using strconv.Quote * Update sytest-whitelist * Update sytest-whitelist
2020-06-16BREAKING: Make eduserver/appservice use userapi (#1138)Kegsay
* BREAKING: Make eduserver/appservice use userapi This is a breaking change because this PR restructures how the AS API tracks its position in Kafka streams. Previously, it used the account DB to store partition offsets. However, this is also being used by `clientapi` for the same purpose, which is bad (each component needs to store offsets independently or else you might lose messages across restarts). This PR changes this behaviour to now store partition offsets in the `appservice` database. This means that: - Upon restart, the `appservice` component will attempt to replay all room events from the beginning of time. - An additional table will be created in the appservice database, which in and of itself is backwards compatible. * Return ErrorConflict
2020-06-16Make syncapi use userapi (#1136)Kegsay
* Make syncapi use userapi * Unbreak things * Fix tests * Lint
2020-06-16Make federationapi use userapi (#1135)Kegsay
Removes dependencies on account DB, device DB and ASAPI.
2020-06-16Update sytest-whitelistNeil Alexander
2020-06-16Fix media API for demos and possibly Synapse (#1134)Neil Alexander
* Fix media API for demos and possibly Synapse * User API * goimports
2020-06-16Make userapi responsible for checking access tokens (#1133)Kegsay
* Make userapi responsible for checking access tokens There's still plenty of dependencies on account/device DBs, but this is a start. This is a breaking change as it adds a required config value `listen.user_api`. * Cleanup * Review comments and test fix
2020-06-16More server key updates, tests (#1129)Neil Alexander
* More key tweaks * Start testing stuff * Move responsibility for generating local keys into server key API, don't register prom in caches unless needed, start tests * Don't store our own keys in the database * Don't store our own keys in the database * Don't run tests for now * Tweak caching behaviour, update tests * Update comments, add fixes from forward-merge * Debug logging * Debug logging * Perform final comparison against original set of requests * oops * Fetcher timeouts * Fetcher timeouts * missing func * Tweaks * Update gomatrixserverlib * Fix Federation API test * Break up FetchKeys * Add comments to caching * Add URL check in test * Partially revert "Move responsibility for generating local keys into server key API, don't register prom in caches unless needed, start tests" This reverts commit d7eb54c5b30b2f6a9d6514b643e32e6ad2b602f3. * Fix federation API test * Fix internal cache stuff again * Fix server key API test * Update comments * Update comments from review * Fix lint
2020-06-15Unbreak HTTP mode (#1131)Kegsay
2020-06-15Fix rooms v3 url paths for good - with tests (#1130)Kegsay
* Fix rooms v3 url paths for good - with tests - Add a test rig around `federationapi` to test routing. - Use `JSONVerifier` over `KeyRing` so we can stub things out more easily. - Add `test.NopJSONVerifier` which verifies nothing. - Add `base.BaseMux` which is the original `mux.Router` used to spawn public/internal routers. - Listen on `base.BaseMux` and not the default serve mux as it cleans paths which we don't want. - Factor out `ListenAndServe` to `test.ListenAndServe` and add flag for listening on TLS. * Fix comments * Linting
2020-06-15Unbreak buildKegan Dougal
2020-06-15Current wiring (#1125)Kegsay
* Current wiring * Add ServerKeyAPI lines
2020-06-15Add bare bones user API (#1127)Kegsay
* Add bare bones user API with tests! * linting
2020-06-12Minor perf/debugging improvements (#1121)Kegsay
* Minor perf/debugging improvements - publicroomsapi: Don't call QueryEventsByID with no event IDs - appservice: Consume only if there are 1 or more ASes - roomserver: don't keep a copy of the request "for debugging" - we trace now * fedsender: return early if we have no destinations * Unbreak tests
2020-06-12Rehuffle where things are in the internal package (#1122)Kegsay
renamed: internal/eventcontent.go -> internal/eventutil/eventcontent.go renamed: internal/events.go -> internal/eventutil/events.go renamed: internal/types.go -> internal/eventutil/types.go renamed: internal/http/http.go -> internal/httputil/http.go renamed: internal/httpapi.go -> internal/httputil/httpapi.go renamed: internal/httpapi_test.go -> internal/httputil/httpapi_test.go renamed: internal/httpapis/paths.go -> internal/httputil/paths.go renamed: internal/routing.go -> internal/httputil/routing.go renamed: internal/basecomponent/base.go -> internal/setup/base.go renamed: internal/basecomponent/flags.go -> internal/setup/flags.go renamed: internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go renamed: internal/postgres.go -> internal/sqlutil/postgres.go renamed: internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go renamed: internal/sql.go -> internal/sqlutil/sql.go
2020-06-12Add trace logging to RoomserverInternalAPI (#1120)Kegsay
This is a wrapper around whatever impl we have which then logs the function name/request/response/error. Also tweak when we log on kafka streams: only log on the producer side not the consumer side: we've never had issues with comms and having 1 message rather than N would be nice.
2020-06-12More key tweaks (#1116)Neil Alexander
2020-06-11Roomserver API changes (#1118)Kegsay
* s/QueryBackfill/PerformBackfill/g * OutputEvent now includes AddStateEvents which contain the full event of extra state events * Only include adds not the current event * Get adding state right
2020-06-11Remove unused internal APIs (#1117)Kegsay
2020-06-11Fix tiebreak in Yggdrasil demo (uses wrong key)Neil Alexander
2020-06-11Support embedding Riot Web (#1113)Neil Alexander
* Embed * Support for embedding is better now * HTTP on localhost for Ygg demo * Move embed into Yggdrasil demo
2020-06-11Update Yggdrasil demo timeouts againNeil Alexander
2020-06-10Yggdrasil demo tweaksNeil Alexander
2020-06-10Remove federationsender producer, which in fact was not a producer (#1115)Kegsay
* Remove federationsender producer, which in fact was not a producer * Set the signing struct
2020-06-10Improvements to Yggdrasil demo (#1114)Neil Alexander
* Improvements to Yggdrasil demo * Fix missing copyright * Fix tie-break
2020-06-10Remove public appservice routes (#1112)Kegsay
2020-06-10Remove clientapi producers which aren't actually producers (#1111)Kegsay
* Remove clientapi producers which aren't actually producers They are actually just convenience wrappers around the internal APIs for roomserver/eduserver. Move their logic to their respective `api` packages and call them directly. * Remove TODO * unbreak ygg
2020-06-10Use ServerKeyAPI as keyring (#1110)Neil Alexander
* Use ServerKeyAPI as keyring * Update fetcher name for debug logs
2020-06-10Yggdrasil-based P2P demo (#1108)Neil Alexander
* Initial work on Yggdrasil demo * Muxing? * Yamux * Updates to yamux * Updates * Comments * Update to use monolith stuff * Update go.mod/go.sum * Set defaults * Tweaks * Update yggdrasil * Update config * MarshalIndent * Change default instance name/port * add -peer switch * gocyclo, for a change * Determinate yamux roles * Fix copyright notices * Remove HTTP API checks as always false, remove unused topic
2020-06-10Remove unused UserUpdates producer (#1109)Kegsay
2020-06-09Factor out how monolith routes get added (#1107)Kegsay
Previously we had 3 monoliths: - dendrite-monolith-server - dendrite-demo-libp2p - dendritejs which all had their own of setting up public routes. Factor this out into a new `setup.Monolith` struct which gets all dependencies set as fields. This is different to `basecomponent.Base` which doesn't provide any way to set configured deps (e.g public rooms db) Part of a larger process to clean up how we initialise Dendrite.
2020-06-08Split out SetupFooComponent (#1106)Kegsay
* Split out adding HTTP routes from making internal APIs for clarity * Split out more components * Split out more things * Finish converting * internal mux for internal routes
2020-06-08Update gomatrixserverlibNeil Alexander
2020-06-05Nuke old Docker stuff (#1104)Neil Alexander
* Remove old Docker stuff * Move hub stuff to upper level docker directory * Build monolith images * Update README.md * Update paths * Update README.md
2020-06-05More flexible caching (#1101)Neil Alexander
2020-06-05Use AuthChainProvider to try and speed up federated joins (#1100)Neil Alexander
* Use MissingAuthEventHandler on performjoin to try and speed up cases where we have missing events * Update gomatrixserverlib * Use supplied room version * Use AuthChainProvider * Tweaks * Update gomatrixserverlib * Signature checks
2020-06-05General cleanup when making components (#1098)Kegsay
* Remove ParseMonolith/LoadMonolith * cleanup which components need to be made
2020-06-04Convert serverkeys to inthttp (#1097)Kegsay
2020-06-04Convert everything but serverkeyapi to inthttp (#1096)Kegsay
* Convert roomserver to new inthttp format * Convert eduserver to new inthttp format * Convert appservice to new inthttp format
2020-06-04Add key validity fetching to server key API (#1094)Neil Alexander
* Add key validity checks * Store fetched keys * Don't double-cache key results * Perform server key API operations using new context * Revert "Perform server key API operations using new context" This reverts commit 02172223f5cb7850b0852c6ed6836ad82508ea76. * Perform server key API operations using new context
2020-06-04Refactor how federationsender gets created (#1095)Kegsay
* Refactor how federationsender gets created * s/httpint/inthttp/ for alphabetical niceness with internal package