aboutsummaryrefslogtreecommitdiff
path: root/docs/development
diff options
context:
space:
mode:
Diffstat (limited to 'docs/development')
-rw-r--r--docs/development/CONTRIBUTING.md1
-rw-r--r--docs/development/PROFILING.md1
-rw-r--r--docs/development/coverage.md164
-rw-r--r--docs/development/sytest.md70
-rw-r--r--docs/development/tracing/opentracing.md114
-rw-r--r--docs/development/tracing/setup.md57
6 files changed, 114 insertions, 293 deletions
diff --git a/docs/development/CONTRIBUTING.md b/docs/development/CONTRIBUTING.md
index 2aec4c36..71e7516a 100644
--- a/docs/development/CONTRIBUTING.md
+++ b/docs/development/CONTRIBUTING.md
@@ -1,6 +1,7 @@
---
title: Contributing
parent: Development
+nav_order: 1
permalink: /development/contributing
---
diff --git a/docs/development/PROFILING.md b/docs/development/PROFILING.md
index 57c37a90..dc4eca7b 100644
--- a/docs/development/PROFILING.md
+++ b/docs/development/PROFILING.md
@@ -1,6 +1,7 @@
---
title: Profiling
parent: Development
+nav_order: 4
permalink: /development/profiling
---
diff --git a/docs/development/coverage.md b/docs/development/coverage.md
index c4a8a117..1b15f71a 100644
--- a/docs/development/coverage.md
+++ b/docs/development/coverage.md
@@ -1,78 +1,130 @@
---
title: Coverage
parent: Development
+nav_order: 3
permalink: /development/coverage
---
-To generate a test coverage report for Sytest, a small patch needs to be applied to the Sytest repository to compile and use the instrumented binary:
-```patch
-diff --git a/lib/SyTest/Homeserver/Dendrite.pm b/lib/SyTest/Homeserver/Dendrite.pm
-index 8f0e209c..ad057e52 100644
---- a/lib/SyTest/Homeserver/Dendrite.pm
-+++ b/lib/SyTest/Homeserver/Dendrite.pm
-@@ -337,7 +337,7 @@ sub _start_monolith
-
- $output->diag( "Starting monolith server" );
- my @command = (
-- $self->{bindir} . '/dendrite',
-+ $self->{bindir} . '/dendrite', '--test.coverprofile=' . $self->{hs_dir} . '/integrationcover.log', "DEVEL",
- '--config', $self->{paths}{config},
- '--http-bind-address', $self->{bind_host} . ':' . $self->unsecure_port,
- '--https-bind-address', $self->{bind_host} . ':' . $self->secure_port,
-diff --git a/scripts/dendrite_sytest.sh b/scripts/dendrite_sytest.sh
-index f009332b..7ea79869 100755
---- a/scripts/dendrite_sytest.sh
-+++ b/scripts/dendrite_sytest.sh
-@@ -34,7 +34,8 @@ export GOBIN=/tmp/bin
- echo >&2 "--- Building dendrite from source"
- cd /src
- mkdir -p $GOBIN
--go install -v ./cmd/dendrite
-+# go install -v ./cmd/dendrite
-+go test -c -cover -covermode=atomic -o $GOBIN/dendrite -coverpkg "github.com/matrix-org/..." ./cmd/dendrite
- go install -v ./cmd/generate-keys
- cd -
- ```
+## Running unit tests with coverage enabled
+
+Running unit tests with coverage enabled can be done with the following commands, this will generate a `integrationcover.log`
+```bash
+go test -covermode=atomic -coverpkg=./... -coverprofile=integrationcover.log $(go list ./... | grep -v '/cmd/')
+go tool cover -func=integrationcover.log
+```
+
+## Running Sytest with coverage enabled
+
+To run Sytest with coverage enabled:
+
+```bash
+docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest" \
+ -v "/Users/kegan/github/dendrite:/src" -v "$(pwd)/sytest_logs:/logs" \
+ -v "/Users/kegan/go/:/gopath" -e "POSTGRES=1" \
+ -e "COVER=1" \
+ matrixdotorg/sytest-dendrite:latest
+
+# to get a more accurate coverage you may also need to run Sytest using SQLite as the database:
+docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest" \
+ -v "/Users/kegan/github/dendrite:/src" -v "$(pwd)/sytest_logs:/logs" \
+ -v "/Users/kegan/go/:/gopath" \
+ -e "COVER=1" \
+ matrixdotorg/sytest-dendrite:latest
+```
+
+This will generate a folder `covdatafiles` in each server's directory, e.g `server-0/covdatafiles`. To parse them,
+ensure your working directory is under the Dendrite repository then run:
- Then run Sytest. This will generate a new file `integrationcover.log` in each server's directory e.g `server-0/integrationcover.log`. To parse it,
- ensure your working directory is under the Dendrite repository then run:
```bash
- go tool cover -func=/path/to/server-0/integrationcover.log
+ go tool covdata func -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)"
```
which will produce an output like:
```
...
- github.com/matrix-org/util/json.go:83: NewJSONRequestHandler 100.0%
-github.com/matrix-org/util/json.go:90: Protect 57.1%
-github.com/matrix-org/util/json.go:110: RequestWithLogging 100.0%
-github.com/matrix-org/util/json.go:132: MakeJSONAPI 70.0%
-github.com/matrix-org/util/json.go:151: respond 61.5%
-github.com/matrix-org/util/json.go:180: WithCORSOptions 0.0%
-github.com/matrix-org/util/json.go:191: SetCORSHeaders 100.0%
-github.com/matrix-org/util/json.go:202: RandomString 100.0%
-github.com/matrix-org/util/json.go:210: init 100.0%
-github.com/matrix-org/util/unique.go:13: Unique 91.7%
-github.com/matrix-org/util/unique.go:48: SortAndUnique 100.0%
-github.com/matrix-org/util/unique.go:55: UniqueStrings 100.0%
-total: (statements) 53.7%
+github.com/matrix-org/util/json.go:132: MakeJSONAPI 70.0%
+github.com/matrix-org/util/json.go:151: respond 84.6%
+github.com/matrix-org/util/json.go:180: WithCORSOptions 0.0%
+github.com/matrix-org/util/json.go:191: SetCORSHeaders 100.0%
+github.com/matrix-org/util/json.go:202: RandomString 100.0%
+github.com/matrix-org/util/json.go:210: init 100.0%
+github.com/matrix-org/util/unique.go:13: Unique 91.7%
+github.com/matrix-org/util/unique.go:48: SortAndUnique 100.0%
+github.com/matrix-org/util/unique.go:55: UniqueStrings 100.0%
+total (statements) 64.0%
+```
+(after running Sytest for Postgres _and_ SQLite)
+
+The total coverage for this run is the last line at the bottom. However, this value is misleading because Dendrite can run in different configurations,
+which will never be tested in a single test run (e.g sqlite or postgres). To get a more accurate value, you'll need run Sytest for Postgres and SQLite (see commands above).
+Additional processing is required also to remove packages which will never be tested and extension MSCs:
+
+```bash
+# If you executed both commands from above, you can get the total coverage using the following commands
+go tool covdata textfmt -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)" -o sytest.cov
+grep -Ev 'relayapi|setup/mscs' sytest.cov > final.cov
+go tool cover -func=final.cov
+
+# If you only executed the one for Postgres:
+go tool covdata textfmt -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)" -o sytest.cov
+grep -Ev 'relayapi|sqlite|setup/mscs' sytest.cov > final.cov
+go tool cover -func=final.cov
+
+# If you only executed the one for SQLite:
+go tool covdata textfmt -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)" -o sytest.cov
+grep -Ev 'relayapi|postgres|setup/mscs' sytest.cov > final.cov
+go tool cover -func=final.cov
```
-The total coverage for this run is the last line at the bottom. However, this value is misleading because Dendrite can run in many different configurations,
-which will never be tested in a single test run (e.g sqlite or postgres). To get a more accurate value, additional processing is required
-to remove packages which will never be tested and extension MSCs:
+
+## Getting coverage from Complement
+
+Getting the coverage for Complement runs is a bit more involved.
+
+First you'll need a docker image compatible with Complement, one can be built using
+```bash
+docker build -t complement-dendrite -f build/scripts/Complement.Dockerfile .
+```
+from within the Dendrite repository.
+
+Clone complement to a directory of your liking:
```bash
-# These commands are all similar but change which package paths are _removed_ from the output.
+git clone https://github.com/matrix-org/complement.git
+cd complement
+```
-# For Postgres
-go tool cover -func=/path/to/server-0/integrationcover.log | grep 'github.com/matrix-org/dendrite' | grep -Ev 'inthttp|sqlite|setup/mscs|api_trace' > coverage.txt
+Next we'll need a script to execute after a test finishes, create a new file `posttest.sh`, make the file executable (`chmod +x posttest.sh`)
+and add the following content:
+```bash
+#!/bin/bash
-# For SQLite
-go tool cover -func=/path/to/server-0/integrationcover.log | grep 'github.com/matrix-org/dendrite' | grep -Ev 'inthttp|postgres|setup/mscs|api_trace' > coverage.txt
+mkdir -p /tmp/Complement/logs/$2/$1/
+docker cp $1:/tmp/covdatafiles/. /tmp/Complement/logs/$2/$1/
```
+This will copy the `covdatafiles` files from each container to something like
+`/tmp/Complement/logs/TestLogin/94f9c428de95779d2b62a3ccd8eab9d5ddcf65cc259a40ece06bdc61687ffed3/`. (`$1` is the containerID, `$2` the test name)
-A total value can then be calculated using:
+Now that we have set up everything we need, we can finally execute Complement:
```bash
-cat coverage.txt | awk -F '\t+' '{x = x + $3} END {print x/NR}'
+COMPLEMENT_BASE_IMAGE=complement-dendrite \
+COMPLEMENT_SHARE_ENV_PREFIX=COMPLEMENT_DENDRITE_ \
+COMPLEMENT_DENDRITE_COVER=1 \
+COMPLEMENT_POST_TEST_SCRIPT=$(pwd)/posttest.sh \
+ go test -tags dendrite_blacklist ./tests/... -count=1 -v -timeout=30m -failfast=false
```
+Once this is done, you can copy the resulting `covdatafiles` files to your Dendrite repository for the next step.
+```bash
+cp -pr /tmp/Complement/logs PathToYourDendriteRepository
+```
-We currently do not have a way to combine Sytest/Complement/Unit Tests into a single coverage report. \ No newline at end of file
+You can also run the following to get the coverage for Complement runs alone:
+```bash
+go tool covdata func -i="$(find /tmp/Complement -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)"
+```
+
+## Combining the results of (almost) all runs
+
+Now that we have all our `covdatafiles` files within the Dendrite repository, you can now execute the following command, to get the coverage
+overall (excluding unit tests):
+```bash
+go tool covdata func -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)"
+``` \ No newline at end of file
diff --git a/docs/development/sytest.md b/docs/development/sytest.md
index 4fae2ea3..2f681f3e 100644
--- a/docs/development/sytest.md
+++ b/docs/development/sytest.md
@@ -1,6 +1,7 @@
---
title: SyTest
parent: Development
+nav_order: 2
permalink: /development/sytest
---
@@ -23,7 +24,7 @@ After running the tests, a script will print the tests you need to add to
You should proceed after you see no build problems for dendrite after running:
```sh
-./build.sh
+go build -o bin/ ./cmd/...
```
If you are fixing an issue marked with
@@ -61,6 +62,8 @@ When debugging, the following Docker `run` options may also be useful:
* `-e "DENDRITE_TRACE_HTTP=1"`: Adds HTTP tracing to server logs.
* `-e "DENDRITE_TRACE_INTERNAL=1"`: Adds roomserver internal API tracing to
server logs.
+* `-e "COVER=1"`: Run Sytest with an instrumented binary, producing a Go coverage file per server.
+* `-e "RACE_DETECTION=1"`: Build the binaries with the `-race` flag (Note: This will significantly slow down test runs)
The docker command also supports a single positional argument for the test file to
run, so you can run a single `.pl` file rather than the whole test suite. For example:
@@ -71,68 +74,3 @@ docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest"
-v "/Users/kegan/go/:/gopath" -e "POSTGRES=1" -e "DENDRITE_TRACE_HTTP=1"
matrixdotorg/sytest-dendrite:latest tests/50federation/40devicelists.pl
```
-
-### Manually Setting up SyTest
-
-**We advise AGAINST using manual SyTest setups.**
-
-If you don't want to use the Docker image, you can also run SyTest by hand. Make
-sure you have Perl 5 or above, and get SyTest with:
-
-(Note that this guide assumes your SyTest checkout is next to your
-`dendrite` checkout.)
-
-```sh
-git clone -b develop https://github.com/matrix-org/sytest
-cd sytest
-./install-deps.pl
-```
-
-Set up the database:
-
-```sh
-sudo -u postgres psql -c "CREATE USER dendrite PASSWORD 'itsasecret'"
-sudo -u postgres psql -c "ALTER USER dendrite CREATEDB"
-for i in dendrite0 dendrite1 sytest_template; do sudo -u postgres psql -c "CREATE DATABASE $i OWNER dendrite;"; done
-mkdir -p "server-0"
-cat > "server-0/database.yaml" << EOF
-args:
- user: dendrite
- password: itsasecret
- database: dendrite0
- host: 127.0.0.1
- sslmode: disable
-type: pg
-EOF
-mkdir -p "server-1"
-cat > "server-1/database.yaml" << EOF
-args:
- user: dendrite
- password: itsasecret
- database: dendrite1
- host: 127.0.0.1
- sslmode: disable
-type: pg
-EOF
-```
-
-Run the tests:
-
-```sh
-POSTGRES=1 ./run-tests.pl -I Dendrite::Monolith -d ../dendrite/bin -W ../dendrite/sytest-whitelist -O tap --all | tee results.tap
-```
-
-where `tee` lets you see the results while they're being piped to the file, and
-`POSTGRES=1` enables testing with PostgeSQL. If the `POSTGRES` environment
-variable is not set or is set to 0, SyTest will fall back to SQLite 3. For more
-flags and options, see <https://github.com/matrix-org/sytest#running>.
-
-Once the tests are complete, run the helper script to see if you need to add
-any newly passing test names to `sytest-whitelist` in the project's root
-directory:
-
-```sh
-../dendrite/show-expected-fail-tests.sh results.tap ../dendrite/sytest-whitelist ../dendrite/sytest-blacklist
-```
-
-If the script prints nothing/exits with 0, then you're good to go.
diff --git a/docs/development/tracing/opentracing.md b/docs/development/tracing/opentracing.md
deleted file mode 100644
index 8528c2ba..00000000
--- a/docs/development/tracing/opentracing.md
+++ /dev/null
@@ -1,114 +0,0 @@
----
-title: OpenTracing
-has_children: true
-parent: Development
-permalink: /development/opentracing
----
-
-# OpenTracing
-
-Dendrite extensively uses the [opentracing.io](http://opentracing.io) framework
-to trace work across the different logical components.
-
-At its most basic opentracing tracks "spans" of work; recording start and end
-times as well as any parent span that caused the piece of work.
-
-A typical example would be a new span being created on an incoming request that
-finishes when the response is sent. When the code needs to hit out to a
-different component a new span is created with the initial span as its parent.
-This would end up looking roughly like:
-
-```
-Received request Sent response
- |<───────────────────────────────────────>|
- |<────────────────────>|
- RPC call RPC call returns
-```
-
-This is useful to see where the time is being spent processing a request on a
-component. However, opentracing allows tracking of spans across components. This
-makes it possible to see exactly what work goes into processing a request:
-
-```
-Component 1 |<─────────────────── HTTP ────────────────────>|
- |<──────────────── RPC ─────────────────>|
-Component 2 |<─ SQL ─>| |<── RPC ───>|
-Component 3 |<─ SQL ─>|
-```
-
-This is achieved by serializing span information during all communication
-between components. For HTTP requests, this is achieved by the sender
-serializing the span into a HTTP header, and the receiver deserializing the span
-on receipt. (Generally a new span is then immediately created with the
-deserialized span as the parent).
-
-A collection of spans that are related is called a trace.
-
-Spans are passed through the code via contexts, rather than manually. It is
-therefore important that all spans that are created are immediately added to the
-current context. Thankfully the opentracing library gives helper functions for
-doing this:
-
-```golang
-span, ctx := opentracing.StartSpanFromContext(ctx, spanName)
-defer span.Finish()
-```
-
-This will create a new span, adding any span already in `ctx` as a parent to the
-new span.
-
-Adding Information
-------------------
-
-Opentracing allows adding information to a trace via three mechanisms:
-
-- "tags" ─ A span can be tagged with a key/value pair. This is typically
- information that relates to the span, e.g. for spans created for incoming HTTP
- requests could include the request path and response codes as tags, spans for
- SQL could include the query being executed.
-- "logs" ─ Key/value pairs can be looged at a particular instance in a trace.
- This can be useful to log e.g. any errors that happen.
-- "baggage" ─ Arbitrary key/value pairs can be added to a span to which all
- child spans have access. Baggage isn't saved and so isn't available when
- inspecting the traces, but can be used to add context to logs or tags in child
- spans.
-
-See
-[specification.md](https://github.com/opentracing/specification/blob/master/specification.md)
-for some of the common tags and log fields used.
-
-Span Relationships
-------------------
-
-Spans can be related to each other. The most common relation is `childOf`, which
-indicates the child span somehow depends on the parent span ─ typically the
-parent span cannot complete until all child spans are completed.
-
-A second relation type is `followsFrom`, where the parent has no dependence on
-the child span. This usually indicates some sort of fire and forget behaviour,
-e.g. adding a message to a pipeline or inserting into a kafka topic.
-
-Jaeger
-------
-
-Opentracing is just a framework. We use
-[jaeger](https://github.com/jaegertracing/jaeger) as the actual implementation.
-
-Jaeger is responsible for recording, sending and saving traces, as well as
-giving a UI for viewing and interacting with traces.
-
-To enable jaeger a `Tracer` object must be instansiated from the config (as well
-as having a jaeger server running somewhere, usually locally). A `Tracer` does
-several things:
-
-- Decides which traces to save and send to the server. There are multiple
- schemes for doing this, with a simple example being to save a certain fraction
- of traces.
-- Communicating with the jaeger backend. If not explicitly specified uses the
- default port on localhost.
-- Associates a service name to all spans created by the tracer. This service
- name equates to a logical component, e.g. spans created by clientapi will have
- a different service name than ones created by the syncapi. Database access
- will also typically use a different service name.
-
- This means that there is a tracer per service name/component.
diff --git a/docs/development/tracing/setup.md b/docs/development/tracing/setup.md
deleted file mode 100644
index cef1089e..00000000
--- a/docs/development/tracing/setup.md
+++ /dev/null
@@ -1,57 +0,0 @@
----
-title: Setup
-parent: OpenTracing
-grand_parent: Development
-permalink: /development/opentracing/setup
----
-
-# OpenTracing Setup
-
-Dendrite uses [Jaeger](https://www.jaegertracing.io/) for tracing between microservices.
-Tracing shows the nesting of logical spans which provides visibility on how the microservices interact.
-This document explains how to set up Jaeger locally on a single machine.
-
-## Set up the Jaeger backend
-
-The [easiest way](https://www.jaegertracing.io/docs/1.18/getting-started/) is to use the all-in-one Docker image:
-
-```
-$ docker run -d --name jaeger \
- -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
- -p 5775:5775/udp \
- -p 6831:6831/udp \
- -p 6832:6832/udp \
- -p 5778:5778 \
- -p 16686:16686 \
- -p 14268:14268 \
- -p 14250:14250 \
- -p 9411:9411 \
- jaegertracing/all-in-one:1.18
-```
-
-## Configuring Dendrite to talk to Jaeger
-
-Modify your config to look like: (this will send every single span to Jaeger which will be slow on large instances, but for local testing it's fine)
-
-```
-tracing:
- enabled: true
- jaeger:
- serviceName: "dendrite"
- disabled: false
- rpc_metrics: true
- tags: []
- sampler:
- type: const
- param: 1
-```
-
-then run the monolith server:
-
-```
-./dendrite --tls-cert server.crt --tls-key server.key --config dendrite.yaml
-```
-
-## Checking traces
-
-Visit <http://localhost:16686> to see traces under `DendriteMonolith`.
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-25 13:39:36 +0000