1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
|
---
title: Coverage
parent: Development
nav_order: 3
permalink: /development/coverage
---
{% include deprecation.html %}
## Running unit tests with coverage enabled
Running unit tests with coverage enabled can be done with the following commands, this will generate a `integrationcover.log`
```bash
go test -covermode=atomic -coverpkg=./... -coverprofile=integrationcover.log $(go list ./... | grep -v '/cmd/')
go tool cover -func=integrationcover.log
```
## Running Sytest with coverage enabled
To run Sytest with coverage enabled:
```bash
docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest" \
-v "/Users/kegan/github/dendrite:/src" -v "$(pwd)/sytest_logs:/logs" \
-v "/Users/kegan/go/:/gopath" -e "POSTGRES=1" \
-e "COVER=1" \
matrixdotorg/sytest-dendrite:latest
# to get a more accurate coverage you may also need to run Sytest using SQLite as the database:
docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest" \
-v "/Users/kegan/github/dendrite:/src" -v "$(pwd)/sytest_logs:/logs" \
-v "/Users/kegan/go/:/gopath" \
-e "COVER=1" \
matrixdotorg/sytest-dendrite:latest
```
This will generate a folder `covdatafiles` in each server's directory, e.g `server-0/covdatafiles`. To parse them,
ensure your working directory is under the Dendrite repository then run:
```bash
go tool covdata func -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)"
```
which will produce an output like:
```
...
github.com/matrix-org/util/json.go:132: MakeJSONAPI 70.0%
github.com/matrix-org/util/json.go:151: respond 84.6%
github.com/matrix-org/util/json.go:180: WithCORSOptions 0.0%
github.com/matrix-org/util/json.go:191: SetCORSHeaders 100.0%
github.com/matrix-org/util/json.go:202: RandomString 100.0%
github.com/matrix-org/util/json.go:210: init 100.0%
github.com/matrix-org/util/unique.go:13: Unique 91.7%
github.com/matrix-org/util/unique.go:48: SortAndUnique 100.0%
github.com/matrix-org/util/unique.go:55: UniqueStrings 100.0%
total (statements) 64.0%
```
(after running Sytest for Postgres _and_ SQLite)
The total coverage for this run is the last line at the bottom. However, this value is misleading because Dendrite can run in different configurations,
which will never be tested in a single test run (e.g sqlite or postgres). To get a more accurate value, you'll need run Sytest for Postgres and SQLite (see commands above).
Additional processing is required also to remove packages which will never be tested and extension MSCs:
```bash
# If you executed both commands from above, you can get the total coverage using the following commands
go tool covdata textfmt -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)" -o sytest.cov
grep -Ev 'relayapi|setup/mscs' sytest.cov > final.cov
go tool cover -func=final.cov
# If you only executed the one for Postgres:
go tool covdata textfmt -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)" -o sytest.cov
grep -Ev 'relayapi|sqlite|setup/mscs' sytest.cov > final.cov
go tool cover -func=final.cov
# If you only executed the one for SQLite:
go tool covdata textfmt -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)" -o sytest.cov
grep -Ev 'relayapi|postgres|setup/mscs' sytest.cov > final.cov
go tool cover -func=final.cov
```
## Getting coverage from Complement
Getting the coverage for Complement runs is a bit more involved.
First you'll need a docker image compatible with Complement, one can be built using
```bash
docker build -t complement-dendrite -f build/scripts/Complement.Dockerfile .
```
from within the Dendrite repository.
Clone complement to a directory of your liking:
```bash
git clone https://github.com/matrix-org/complement.git
cd complement
```
Next we'll need a script to execute after a test finishes, create a new file `posttest.sh`, make the file executable (`chmod +x posttest.sh`)
and add the following content:
```bash
#!/bin/bash
mkdir -p /tmp/Complement/logs/$2/$1/
docker cp $1:/tmp/covdatafiles/. /tmp/Complement/logs/$2/$1/
```
This will copy the `covdatafiles` files from each container to something like
`/tmp/Complement/logs/TestLogin/94f9c428de95779d2b62a3ccd8eab9d5ddcf65cc259a40ece06bdc61687ffed3/`. (`$1` is the containerID, `$2` the test name)
Now that we have set up everything we need, we can finally execute Complement:
```bash
COMPLEMENT_BASE_IMAGE=complement-dendrite \
COMPLEMENT_SHARE_ENV_PREFIX=COMPLEMENT_DENDRITE_ \
COMPLEMENT_DENDRITE_COVER=1 \
COMPLEMENT_POST_TEST_SCRIPT=$(pwd)/posttest.sh \
go test -tags dendrite_blacklist ./tests/... -count=1 -v -timeout=30m -failfast=false
```
Once this is done, you can copy the resulting `covdatafiles` files to your Dendrite repository for the next step.
```bash
cp -pr /tmp/Complement/logs PathToYourDendriteRepository
```
You can also run the following to get the coverage for Complement runs alone:
```bash
go tool covdata func -i="$(find /tmp/Complement -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)"
```
## Combining the results of (almost) all runs
Now that we have all our `covdatafiles` files within the Dendrite repository, you can now execute the following command, to get the coverage
overall (excluding unit tests):
```bash
go tool covdata func -i="$(find -name 'covmeta*' -type f -exec dirname {} \; | uniq | paste -s -d ',' -)"
```
|
