aboutsummaryrefslogtreecommitdiff
path: root/clientapi
AgeCommit message (Collapse)Author
2020-05-01Limit database connections (#980, #564) (#998)Neil Alexander
* Limit database connections (#564) - Add new options to the config file database: max_open_conns: 100 max_idle_conns: 2 conn_max_lifetime: -1 - Implement connection parameter setup on the *DB (database/sql) in internal/sqlutil/trace.go:Open() - Propagate the values in the form of DbProperties interface via all the Open() and NewDatabase() functions Signed-off-by: Tomas Jirka <tomas.jirka@email.cz> * Fix wasm builds * Remove file accidentally added from working tree Co-authored-by: Tomas Jirka <tomas.jirka@email.cz>
2020-05-01Consolidation of roomserver APIs (#994)Neil Alexander
* Consolidation of roomserver APIs * Comment out alias tests for now, they are broken * Wire AS API into roomserver again * Roomserver didn't take asAPI param before so return to that * Prevent roomserver asking AS API for alias info * Rename some files * Remove alias_test, incoherent tests and unwanted appservice integration * Remove FS API inject on syncapi component
2020-04-29Join room support in federation sender (#989)Neil Alexander
* Implement PerformJoinRequest * Rename perform functions * Check send join response * Temporary wiring to test federation sender room joins * Actually pass through the config * Make sure membership content shows join
2020-04-29Federation sender API remodel (#988)Neil Alexander
* Define an input API for the federationsender * Wiring for rooomserver input API and federation sender input API * Whoops, commit common too * Merge input API into query API * Rename FederationSenderQueryAPI to FederationSenderInternalAPI * Fix dendritejs * Rename Input to Perform * Fix a couple of inputs -> performs * Remove needless storage interface, add comments
2020-04-28Loopback event from invite response (#982)Neil Alexander
* Working invite v2 support * Fix copyright notice * Update gomatrixserverlib * Add fallthrough * Add missing continue * Update sytest-whitelist, gomatrixserverlib * Update gomatrixserverlib to test matrix-org/gomatrixserverlib#181 * Update gomatrixserverlib
2020-04-24More invite support (#979)Neil Alexander
* Update gomatixserverlib * Try to build invite stripped state if not given to us * SendInvite improvements * Transpose invite_room_state into invite_state.events for sync API * Remove syncapi debugging output * Use RespInviteV2 * Update gomatrixserverlib * Send the invite event as a normal roomserver event too, for incorporating into room (should this be done by the roomserver automatically for invite inputs?) * Federation sender use invite_room_state, room server try to insert membership state * Check supported room versions on the invite endpoint * Prevent roomserver query API from trying to handle requests for stub rooms * Adding a nolint * Replace IsRoomStub with RoomNIDExcludingStubs, fix query API to use that instead * Review comments
2020-04-22Fix sarama import URLs (#856)Neil Alexander
* Fix sarama import URLs * Update gomatrixserverlib * Update naffka * Update naffka * Update in kafka-producer
2020-04-22Response from /send_join now use gomatrixserverlib.RespSendJoin (#796)Behouba Manassé
* response from /send_join now use gomatrixserverlib.RespSendJoin * Update Dendrite gomatrixserverlib version * Fix spelling Co-authored-by: Andrew Morgan <andrew@amorgan.xyz> Co-authored-by: Cnly <minecnly@gmail.com> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-04-20Perspective key fetching, some federation room join fixes (#975)Neil Alexander
* Update gomatrixserverlib * Test matrix.org as perspective key server * Base64 decode better * Optional strict validity checking in gmsl * Update gomatrixserverlib * Attempt to find missing auth events over federation (this shouldn't happen but I am guessing there is a synapse bug involved where we don't get all of the auth events) * Update gomatrixserverlib, debug logging * Remove debugging output * More verbose debugging * Print outliers * Increase timeouts for testing, observe contexts before trying to join over more servers * Don't block on roomserver (experimental) * Don't block on roomserver * Update gomatrixserverlib * Update gomatrixserverlib * Configurable perspective key fetchers * Output number of configured keys for perspective * Example perspective config included * Undo debug stack trace * Undo debug stack trace * Restore original HTTP listener in monolith * Fix lint * Review comments * Set default HTTP server timeout to 5 minutes now, block again when joining * Don't use HTTP address for HTTPS whoops * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Actually add perspectives * Actually add perspectives * Update gomatrixserverlib
2020-04-16Add SQL tracing via DENDRITE_TRACE_SQL (#968)Kegsay
* Add SQL tracing via DENDRITE_TRACE_SQL Add this to `internal/sqlutil` in preparation for #897 * Not entirely
2020-04-14Move /room/{roomID}/state endpoints into client API (#606) (#962)Neil Alexander
* Move /room/{roomID}/state endpoints into client API (#606) * Update sytest-whitelist * Blacklist tests which rely on endpoints we don't implement
2020-04-14support non-exclusive namespaces for AS (#828)mohit kumar singh
Signed-off-by: MohitKS5 <mohitkumarsingh907@gmail.com> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com> Co-authored-by: Kegsay <kegan@matrix.org>
2020-04-12Added checks for JSON body in accounts_data endpoint (#863)Prateek Sachan
Signed-off-by: Prateek Sachan <psachan@cs.iitr.ac.in>
2020-04-09Federation for v3/v4 rooms (#954)Neil Alexander
* Update gomatrixserverlib * Default to room version 4 * Update gomatrixserverlib * Limit prev_events and auth_events * Fix auth_events, prev_events * Fix linter issues * Update gomatrixserverlib * Fix getState * Update sytest-whitelist * Squashed commit of the following: commit 067b87506357c996fd6ddb11271db9469ad4ce80 Author: Neil Alexander <neilalexander@users.noreply.github.com> Date: Fri Apr 3 14:29:06 2020 +0100 Invites v2 endpoint (#952) * Start converting v1 invite endpoint to v2 * Update gomatrixserverlib * Early federationsender code for sending invites * Sending invites sorta happens now * Populate invite request with stripped state * Remodel a bit, don't reflect received invites * Handle invite_room_state * Handle room versions a bit better * Update gomatrixserverlib * Tweak order in destinationQueue.next * Revert check in processMessage * Tweak federation sender destination queue code a bit * Add comments commit 955244c09298d0e6c870377dad3af2ffa1f5e578 Author: Ben B <benne@klimlive.de> Date: Fri Apr 3 12:40:50 2020 +0200 use custom http client instead of the http DefaultClient (#823) This commit replaces the default client from the http lib with a custom one. The previously used default client doesn't come with a timeout. This could cause unwanted locks. That solution chosen here creates a http client in the base component dendrite with a constant timeout of 30 seconds. If it should be necessary to overwrite this, we could include the timeout in the dendrite configuration. Here it would be a good idea to extend the type "Address" by a timeout and create an http client for each service. Closes #820 Signed-off-by: Benedikt Bongartz <benne@klimlive.de> Co-authored-by: Kegsay <kegan@matrix.org> * Update sytest-whitelist, sytest-blacklist * Update go.mod/go.sum * Add some error wrapping for debug * Add a NOTSPEC to common/events.go * Perform state resolution at send_join * Set default room version to v2 again * Tweak GetCapabilities * Add comments to ResolveConflictsAdhoc * Update sytest-blacklist * go mod tidy * Update sytest-whitelist, sytest-blacklist * Update versions * Updates from review comments * Update sytest-blacklist, sytest-whitelist * Check room versions compatible at make_join, add some comments, update gomatrixserverlib, other tweaks * Set default room version back to v2 * Update gomatrixserverlib, sytest-whitelist
2020-04-03Invites v2 endpoint (#952)Neil Alexander
* Start converting v1 invite endpoint to v2 * Update gomatrixserverlib * Early federationsender code for sending invites * Sending invites sorta happens now * Populate invite request with stripped state * Remodel a bit, don't reflect received invites * Handle invite_room_state * Handle room versions a bit better * Update gomatrixserverlib * Tweak order in destinationQueue.next * Revert check in processMessage * Tweak federation sender destination queue code a bit * Add comments
2020-03-30Implement typing over federation (#949)Kegsay
Also fix a pet peeve of mine: not putting units on things!!! Manually tested on p2p and works well, with some fudge factor delay.
2020-03-30Rename the typing server to EDU server (#948)Kegsay
* Blunt move and sed rename * Rename common/ refs to typing * Rename internal stuff in eduserver * Rename docs and scripts * Rename constants/filenames, goimports everything to re-order imports
2020-03-27Further room version wiring (#936)Neil Alexander
* Room version 2 by default, other wiring updates, update gomatrixserverlib * Fix nil pointer exception * Fix some more nil pointer exceptions hopefully * Update gomatrixserverlib * Send all room versions when joining, not just stable ones * Remove room version cquery * Get room version when getting events from the roomserver database * Reset default back to room version 2 * Don't generate event IDs unless needed * Revert "Remove room version cquery" This reverts commit a170d5873360dd059614460acc8b21ab2cda9767. * Query room version in federation API, client API as needed * Improvements to make_join send_join dance * Make room server producers use headered events * Lint tweaks * Update gomatrixserverlib * Versioned SendJoin * Query room version in syncapi backfill * Handle transaction marshalling/unmarshalling within Dendrite * Sorta fix federation (kinda) * whoops commit federation API too * Use NewEventFromTrustedJSON when getting events from the database * Update gomatrixserverlib * Strip headers on federationapi endpoints * Fix bug in clientapi profile room version query * Update gomatrixserverlib * Return more useful error if room version query doesn't find the room * Update gomatrixserverlib * Update gomatrixserverlib * Maybe fix federation * Fix formatting directive * Update sytest whitelist and blacklist * Temporarily disable room versions 3 and 4 until gmsl is fixed * Fix count of EDUs in logging * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Rely on EventBuilder in gmsl to generate the event IDs for us * Some review comments fixed * Move function out of common and into gmsl * Comment in federationsender destinationqueue * Update gomatrixserverlib
2020-03-19p2p: Implement published rooms (#923)Kegsay
* Create and glue ExternalPublicRoomsProvider into the public rooms component This is how we will link p2p stuff to dendrite proper. * Use gmsl structs rather than our own * Implement federated public rooms - Make thirdparty endpoint r0 so riot-web loads the public room list * Typo * Missing callsites
2020-03-19Implement /joined_rooms (#911)Prateek Sachan
* Implemented /joined_rooms * Removed account endpoint added by mistake * trigger ci
2020-03-18maintenance: Fix matrix-org#896 use %w format verb to wrap errors (#916)Abhinav Krishna C K
* maintenance: Fix matrix-org#896 use %w format verb to wrap errors * In Go version 1.13 a new formatting verb introduced for fmt.Errorf %w https://blog.golang.org/go1.13-errors * update %s to %w to wrap errors. * Update all instances of error type to use %w Signed-off-by: Abhinav Krishna C K <me@abhy.me> Co-authored-by: Kegsay <kegan@matrix.org>
2020-03-18Log errors from rows.Close (#920)Prateek Sachan
* Log errors from rows.Close * fixed imports * Added contextual messages * fixed review changes
2020-03-17Add room version to room create request, persist in storage (#915)Neil Alexander
* Add room version into createRoomReq * Extract room version from m.room.create event when persisting * Reduce cyclomatic complexity * Update whitelist, gomatrixserverlib, tweaks to roomserver * Update sytest-whitelist again
2020-03-17Implement gomatrixserverlib.HeaderedEvent in roomserver Kafka output (#914)Neil Alexander
* Use Event.Headered * Use HeaderedEvent in roomserver kafka output * Fix syncserver-integration-tests * Update producers to roomserver inputs * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib * Update gomatrixserverlib
2020-03-16Implement gomatrixserverlib.HeaderedEvent in roomserver query API (#912)Neil Alexander
* Implement gomatrixserverlib.HeaderedEvent, which should allow us to store room version headers along with the event across API boundaries and consumers/producers, and intercept unmarshalling to get the event structure right * Add federationsender to previous
2020-03-09Improve logging when sending events (#883)Kegsay
We have some failing sytests on sqlite but it's very difficult to debug due to lack of useful logging. This adds a log line for when a new event is sent (incl. logging the event ID) as well as adding a user_id field for all contextual logs so we know who initiated certain actions.
2020-03-06bugfix: Fix a race condition when creating guest accounts (#882)Kegsay
* bugfix: Fix a race condition when creating guest accounts It was possible to both select the same next numeric ID and then both attempt to INSERT this into the table. This would cause a UNIQUE violation which then presented itself as an error in sqlite because it does not implement `common.IsUniqueConstraintViolationErr`. The fix here is NOT to implement `common.IsUniqueConstraintViolationErr` otherwise the 2 users would get the SAME guest account. Instead, all of these operations should be done inside a transaction. This is what this PR does. * Update postgres * Typo * Actually use the txn when creating accounts * bugfix for database is locked on guest reg
2020-03-06Try to recursively find auth events (to a point) if they are missing (#881)Neil Alexander
* Try to recursively find auth events (to a point) if they are missing * Remove recursion limit for now and other review fixes * Simplify error handling for recursion * Pass room version 1 only to MakeJoin until room version support comes later
2020-03-06Add peer-to-peer support into Dendrite via libp2p and fetch (#880)Kegsay
* Use a fork of pq which supports userCurrent on wasm * Use sqlite3_js driver when running in JS * Add cmd/dendritejs to pull in sqlite3_js driver for wasm only * Update to latest go-sqlite-js version * Replace prometheus with a stub. sigh * Hard-code a config and don't use opentracing * Latest go-sqlite3-js version * Generate a key for now * Listen for fetch traffic rather than HTTP * Latest hacks for js * libp2p support * More libp2p * Fork gjson to allow us to enforce auth checks as before Previously, all events would come down redacted because the hash checks would fail. They would fail because sjson.DeleteBytes didn't remove keys not used for hashing. This didn't work because of a build tag which included a file which no-oped the index returned. See https://github.com/tidwall/gjson/issues/157 When it's resolved, let's go back to mainline. * Use gjson@1.6.0 as it fixes https://github.com/tidwall/gjson/issues/157 * Use latest gomatrixserverlib for sig checks * Fix a bug which could cause exclude_from_sync to not be set Caused when sending events over federation. * Use query variadic to make lookups actually work! * Latest gomatrixserverlib * Add notes on getting p2p up and running Partly so I don't forget myself! * refactor: Move p2p specific stuff to cmd/dendritejs This is important or else the normal build of dendrite will fail because the p2p libraries depend on syscall/js which doesn't work on normal builds. Also, clean up main.go to read a bit better. * Update ho-http-js-libp2p to return errors from RoundTrip * Add an LRU cache around the key DB We actually need this for P2P because otherwise we can *segfault* with things like: "runtime: unexpected return pc for runtime.handleEvent" where the event is a `syscall/js` event, caused by spamming sql.js caused by "Checking event signatures for 14 events of room state" which hammers the key DB repeatedly in quick succession. Using a cache fixes this, though the underlying cause is probably a bug in the version of Go I'm on (1.13.7) * breaking: Add Tracing.Enabled to toggle whether we do opentracing Defaults to false, which is why this is a breaking change. We need this flag because WASM builds cannot do opentracing. * Start adding conditional builds for wasm to handle lib/pq The general idea here is to have the wasm build have a `NewXXXDatabase` that doesn't import any postgres package and hence we never import `lib/pq`, which doesn't work under WASM (undefined `userCurrent`). * Remove lib/pq for wasm for syncapi * Add conditional building to remaining storage APIs * Update build script to set env vars correctly for dendritejs * sqlite bug fixes * Docs * Add a no-op main for dendritejs when not building under wasm * Use the real prometheus, even for WASM Instead, the dendrite-sw.js must mock out `process.pid` and `fs.stat` - which must invoke the callback with an error (e.g `EINVAL`) in order for it to work: ``` global.process = { pid: 1, }; global.fs.stat = function(path, cb) { cb({ code: "EINVAL", }); } ``` * Linting
2020-03-02Remove httputil.LogThenError so that the line numbers are reported properly ↵Neil Alexander
- make error reporting slightly more useful (#879)
2020-02-28Fix bug in devices endpoint (#877)Neil Alexander
2020-02-20sqlite: fixes from sytest (#872)Kegsay
* bugfix: fix panic on new invite events from sytest I'm unsure why the previous code didn't work, but it's clearer, quicker and easier to read the `LastInsertID()` way. Previously, the code would panic as the SELECT would fail to find the last inserted row ID. * sqlite: Fix UNIQUE violations and close more cursors - Add missing `defer rows.Close()` - Do not have the state block NID as a PRIMARY KEY else it breaks for blocks with >1 state event in them. Instead, rejig the queries so we can still have monotonically increasing integers without using AUTOINCREMENT (which mandates PRIMARY KEY). * sqlite: Add missing variadic function * Use LastInsertId because empirically it works over the SELECT form (though I don't know why that is) * sqlite: Fix invite table by using the global stream pos rather than one specific to invites If we don't use the global, clients don't get notified about any invites because the position is too low. * linting: shadowing * sqlite: do not use last rowid, we already know the stream pos! * sqlite: Fix account data table in syncapi by commiting insert txns! * sqlite: Fix failing federation invite Was failing with 'database is locked' due to multiple write txns being taken out. * sqlite: Ensure we return exactly the number of events found in the database Previously we would return exactly the number of *requested* events, which meant that several zero-initialised events would bubble through the system, failing at JSON serialisation time. * sqlite: let's just ignore the problem for now.... * linting
2020-02-13Support sqlite in addition to postgres (#869)Kegsay
* Move current work into single branch * Initial massaging of clientapi etc (not working yet) * Interfaces for accounts/devices databases * Duplicate postgres package for sqlite3 (no changes made to it yet) * Some keydb, accountdb, devicedb, common partition fixes, some more syncapi tweaking * Fix accounts DB, device DB * Update naffka dependency for SQLite * Naffka SQLite * Update naffka to latest master * SQLite support for federationsender * Mostly not-bad support for SQLite in syncapi (although there are problems where lots of events get classed incorrectly as backward extremities, probably because of IN/ANY clauses that are badly supported) * Update Dockerfile -> Go 1.13.7, add build-base (as gcc and friends are needed for SQLite) * Implement GET endpoints for account_data in clientapi * Nuke filtering for now... * Revert "Implement GET endpoints for account_data in clientapi" This reverts commit 4d80dff4583d278620d9b3ed437e9fcd8d4674ee. * Implement GET endpoints for account_data in clientapi (#861) * Implement GET endpoints for account_data in clientapi * Fix accountDB parameter * Remove fmt.Println * Fix insertAccountData SQLite query * Fix accountDB storage interfaces * Add empty push rules into account data on account creation (#862) * Put SaveAccountData into the right function this time * Not sure if roomserver is better or worse now * sqlite work * Allow empty last sent ID for the first event * sqlite: room creation works * Support sending messages * Nuke fmt.println * Move QueryVariadic etc into common, other device fixes * Fix some linter issues * Fix bugs * Fix some linting errors * Fix errcheck lint errors * Make naffka use postgres as fallback, fix couple of compile errors * What on earth happened to the /rooms/{roomID}/send/{eventType} routing Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11Always defer *sql.Rows.Close and consult with Err (#844)Kiril Vladimiroff
* Always defer *sql.Rows.Close and consult with Err database/sql.Rows.Next() makes sure to call Close only after exhausting result rows which would NOT happen when returning early from a bad Scan. Close being idempotent makes it a great candidate to get always deferred regardless of what happens later on the result set. This change also makes sure call Err() after exhausting Next() and propagate non-nil results from it as the documentation advises. Closes #764 Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org> * Override named result parameters in last returns Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org> * Do the same over new changes that got merged Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org> Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11Implement missing device management features (#835)S7evinK
* Implement missing device management features Signed-off-by: Till Faelligen <tfaelligen@gmail.com> * Add a little more documentation * Undo changes * Use non-anonymous struct to decode devices list * Update sytest-whitelist * Update sytest-whitelist * Update sytest-blacklist Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11Pass cfg by reference around the codebase (#819)Behouba Manassé
* Pass cfg by reference around the codebase * Merge branch 'master' into pass-cfg-by-ref Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-05Implement room version capabilities in CS API (#866)Neil Alexander
* Add wiring for querying the roomserver for the default room version * Try to implement /capabilities for room versions * Update copyright notices * Update sytests, add /capabilities endpoint into CS API * Update sytest-whitelist * Add GetDefaultRoomVersion * Fix cases where state package was shadowed * Fix version formatting * Update Dockerfile to Go 1.13.6 * oh yes types I remember * And fix the default too
2020-01-30Handle kind=guest query parameter on /register (#860)Prateek Sachan
* Handle kind=guest query parameter on /register * Reorganized imports * Pass device_id as nil * Added tests to systest-whitelist
2020-01-30Add empty push rules into account data on account creation (#862)Neil Alexander
2020-01-29Implement GET endpoints for account_data in clientapi (#861)Neil Alexander
* Implement GET endpoints for account_data in clientapi * Fix accountDB parameter * Remove fmt.Println
2020-01-20Identifier key loginNeil Alexander
Merge commit '56318dcca400b5ada4dfa4f8fac922522fc88128'
2020-01-10Federation fixes (#845)Neil Alexander
* Update gomatrixserverlib to p2p commit 92c0338, other tweaks * Update gomatrixserverlib to p2p commit e5dcc65 * Rewrite getAuthChain * Update gomatrixserverlib in go.mod/go.sum * Correct a couple of package refs for updated gmsl/gomatrix * Update gomatrixserverlib ref in go.mod/go.sum * Update getAuthChain comments following @babolivier review
2019-11-27Fix login using identifier keySerra Allgood
Not a full fix, it only really supports logging in with the localpart of an mxid. Signed-off-by: Serra Allgood <serra@allgood.dev>
2019-10-07selectAccountDataByType shouldn't error when no rows (#804)Alex Chen
Signed-off-by: Alex Chen <minecnly@gmail.com>
2019-10-02Add missing servers field in /directory/room/:alias response (#732)Alex Chen
2019-09-30selectAccountDataByType return ClientEvent pointer instead of slice of ↵Behouba Manassé
ClientEvent (#798) This pull request is an attempt to fix #773. Signed-off-by: Kouame Behouba Manassé behouba@gmail.com
2019-08-24Associate transactions with session IDs instead of device IDs (#789)Alex Chen
2019-08-16Fix data race in clientapi/routing/register.go (#787)Alex Chen
2019-08-16Replace event content types with ones in gomatrixserverlib (#785)Alex Chen
The types that are not in gomatrixserverlib are not replaced. This also updates the gomatrixserverlib dependency.
2019-08-15Add auth fallback endpoint (#405)Parminder Singh
Also adds support for the recaptcha auth type.
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-04 19:04:58 +0000