Update guides for running sytest (#958)

* Update guides for running sytest

Bring the details for running sytest against postgres up to date; prefer docker over manual setup

Signed-off-by: Alex Chen <minecnly@gmail.com>

* Better flags

Signed-off-by: Alex Chen <minecnly@gmail.com>

1 files changed, 47 insertions, 22 deletions

diff --git a/docs/sytest.md b/docs/sytest.md
index 9385ebff..26a46c0b 100644
--- a/docs/sytest.md
+++ b/docs/sytest.md
@@ -10,9 +10,9 @@ passes.
 
 ## Finding out which tests to add
 
-We recommend you run the tests locally by manually setting up SyTest or using a
-SyTest docker image. After running the tests, a script will print the tests you
-need to add to `sytest-whitelist`.
+We recommend you run the tests locally by using the SyTest docker image or
+manually setting up SyTest. After running the tests, a script will print the
+tests you need to add to `sytest-whitelist`.
 
 You should proceed after you see no build problems for dendrite after running:
 
@@ -20,9 +20,32 @@ You should proceed after you see no build problems for dendrite after running:
 ./build.sh
 ```
 
+### Using the SyTest Docker image
+
+Use the following commands to pull the latest SyTest image and run the tests:
+
+```sh
+docker pull matrixdotorg/sytest-dendrite
+docker run --rm -v /path/to/dendrite/:/src/ -v /path/to/log/output/:/logs/ matrixdotorg/sytest-dendrite
+```
+
+`/path/to/dendrite/` should be replaced with the actual path to your dendrite
+source code. The test results TAP file and homeserver logging output will go to
+`/path/to/log/output`. The output of the command should tell you if you need to
+add any tests to `sytest-whitelist`.
+
+When debugging, the following Docker `run` options may also be useful:
+* `-v /path/to/sytest/:/sytest/`: Use your local SyTest repository at
+  `/path/to/sytest` instead of pulling from GitHub. This is useful when you want
+  to speed things up or make modifications to SyTest.
+* `--entrypoint bash`: Prevent the container from automatically starting the
+  tests.  When used, you need to manually run `/bootstrap.sh dendrite` inside
+  the container to start them.
+
 ### Manually Setting up SyTest
 
-Make sure you have Perl v5+ installed, and get SyTest with:
+If you don't want to use the Docker image, you can also run SyTest by hand. Make
+sure you have Perl 5 or above, and get SyTest with:
 
 (Note that this guide assumes your SyTest checkout is next to your
 `dendrite` checkout.)
@@ -37,12 +60,23 @@ Set up the database:
 
 ```sh
 sudo -u postgres psql -c "CREATE USER dendrite PASSWORD 'itsasecret'"
-sudo -u postgres psql -c "CREATE DATABASE sytest_template OWNER dendrite"
+for i in dendrite0 dendrite1 sytest_template; do sudo -u postgres psql -c "CREATE DATABASE $i OWNER dendrite;"; done
 mkdir -p "server-0"
 cat > "server-0/database.yaml" << EOF
 args:
     user: dendrite
-    database: dendrite
+    password: itsasecret
+    database: dendrite0
+    host: 127.0.0.1
+    sslmode: disable
+type: pg
+EOF
+mkdir -p "server-1"
+cat > "server-1/database.yaml" << EOF
+args:
+    user: dendrite
+    password: itsasecret
+    database: dendrite1
     host: 127.0.0.1
     sslmode: disable
 type: pg
@@ -52,29 +86,20 @@ EOF
 Run the tests:
 
 ```sh
-./run-tests.pl -I Dendrite::Monolith -d ../dendrite/bin -W ../dendrite/sytest-whitelist -O tap --all | tee results.tap
+POSTGRES=1 ./run-tests.pl -I Dendrite::Monolith -d ../dendrite/bin -W ../dendrite/sytest-whitelist -O tap --all | tee results.tap
 ```
 
-where `tee` lets you see the results while they're being piped to the file.
+where `tee` lets you see the results while they're being piped to the file, and
+`POSTGRES=1` enables testing with PostgeSQL. If the `POSTGRES` environment
+variable is not set or is set to 0, SyTest will fall back to SQLite 3. For more
+flags and options, see https://github.com/matrix-org/sytest#running.
 
 Once the tests are complete, run the helper script to see if you need to add
-any newly passing test names to `sytest-whitelist` in the project's root directory:
+any newly passing test names to `sytest-whitelist` in the project's root
+directory:
 
 ```sh
 ../dendrite/show-expected-fail-tests.sh results.tap ../dendrite/sytest-whitelist ../dendrite/sytest-blacklist
 ```
 
 If the script prints nothing/exits with 0, then you're good to go.
-
-### Using a SyTest Docker image
-
-Ensure you have the latest image for SyTest, then run the tests:
-
-```sh
-docker pull matrixdotorg/sytest-dendrite
-docker run --rm -v /path/to/dendrite/:/src/ matrixdotorg/sytest-dendrite
-```
-
-where `/path/to/dendrite/` should be replaced with the actual path to your
-dendrite source code. The output should tell you if you need to add any tests to
-`sytest-whitelist`.