aboutsummaryrefslogtreecommitdiff
path: root/tests/docker
AgeCommit message (Collapse)Author
2016-10-17tests/docker/Makefile.include: add a generic docker-run targetAlex Bennée
This re-factors the docker makefile to include a docker-run target which can be controlled entirely from environment variables specified on the make command line. This allows us to run against any given docker image we may have in our repository, for example: make docker-run TEST="test-quick" IMAGE="debian:arm64" \ EXECUTABLE=./aarch64-linux-user/qemu-aarch64 The existing docker-foo@bar targets still work but the inline verification has been dropped because we already don't hit that due to other pattern rules in rules.mak. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20161011161625.9070-5-alex.bennee@linaro.org> Message-Id: <20161011161625.9070-6-alex.bennee@linaro.org> [Squash in the verification removal patch. - Fam] Signed-off-by: Fam Zheng <famz@redhat.com>
2016-10-17tests/docker: make test-mingw honour TARGET_LISTAlex Bennée
The other builders honour this variable, so should the mingw build. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20161011161625.9070-4-alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-10-17tests/docker: test-build scriptAlex Bennée
Much like test-quick but only builds. This is useful for some of the build targets like ThreadSanitizer that don't yet pass "make check". Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20161011161625.9070-3-alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-10-17tests/docker: add travis dockerfileAlex Bennée
This target grabs the latest Travis containers from their repository at quay.io and then installs QEMU's build dependencies. With this it is possible to run on broadly the same setup as they have on travis-ci.org. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20161011161625.9070-2-alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-10-06rules.mak: quiet-command: Split command name and args to printPeter Maydell
The quiet-command make rule currently takes two arguments: the command and arguments to run, and a string to print if the V flag is not set (ie we are not being verbose). By convention, the string printed is of the form " NAME some args". Unfortunately to get nicely lined up output all the strings have to agree about what column the arguments should start in, which means that if we add a new quiet-command usage which wants a slightly longer CMD name then we either put up with misalignment or change every quiet-command string. Split the quiet-mode string into two, the "NAME" and the "same args" part, and use printf(1) to format the string automatically. This means we only need to change one place if we want to support a longer maximum name. In particular, we can now print 7-character names lined up properly (they are needed for the OSX "SETTOOL" invocation). Change all the uses of quiet-command to the new syntax. (Any which are missed or inadvertently reintroduced via later merges will result in slightly misformatted quiet output rather than disaster.) A few places in the pc-bios/ makefiles are updated to use "BUILD", "SIGN" and "STRIP" rather than "Building", "Signing" and "Stripping" for consistency and to keep them below 7 characters. Module .mo links now print "LD" rather than the nonstandard "LD -r". Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Eric Blake <eblake@redhat.com> Message-id: 1475598441-27908-1-git-send-email-peter.maydell@linaro.org
2016-10-02docker: Build in a clean directoryFam Zheng
Currently we configure and build under "$QEMU_SRC/tests/docker" which is dubious. Create a fixed directory (to be friendly to ccache) and change to there before calling build_qemu. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1475047892-11955-1-git-send-email-famz@redhat.com>
2016-09-23docker: exec $CMDFam Zheng
This is the last command to run (unless DEBUG), make it 'exec' to simplify the process tree. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-10-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Terminate instances at SIGTERM and SIGHUPFam Zheng
Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-9-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Support showing environment informationFam Zheng
Add a make variable SHOW_ENV. When it's set to non empty, print the package information and environment variables. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-8-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Print used options before doing configureFam Zheng
This makes the configure command more obvious which usually has useful information. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-7-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Flatten default target list in test-quickFam Zheng
Previously it is expanded to a whitespace separated list which is not the most appropriate format. Since it's only two items, flatten it. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-6-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Update fedora image to latestFam Zheng
Now that 23 is becoming an "old" release with 24 available. Fedora has a quick release cycle, so use latest to follow more closely. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-5-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Generate /packages.txt in ubuntu imageFam Zheng
Put the list of package names in an environment, and output their package names to the target file in the end. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-4-git-send-email-famz@redhat.com> Reviewed-by: Daniel. P. Berrange <berrange@redhat.com>
2016-09-23docker: Generate /packages.txt in fedora imageFam Zheng
Put the list of package names in an environment, and output their package names to the target file in the end. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-3-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Generate /packages.txt in centos6 imageFam Zheng
Put the list of package names in an environment, and output their package names to the target file in the end. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1474429768-25027-2-git-send-email-famz@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2016-09-23docker: Handle exceptions when looking for docker commandEduardo Habkost
When trying to run docker tests on a host without the docker command, we get the following Python backtrace: $ make docker-test-quick@centos6 V=1 .../qemu/tests/docker/docker.py build qemu:centos6 .../qemu/tests/docker/dockerfiles/centos6.docker Traceback (most recent call last): File ".../qemu/tests/docker/docker.py", line 339, in <module> sys.exit(main()) File ".../qemu/tests/docker/docker.py", line 336, in main return args.cmdobj.run(args, argv) File ".../qemu/tests/docker/docker.py", line 231, in run dkr = Docker() File ".../qemu/tests/docker/docker.py", line 98, in __init__ self._command = _guess_docker_command() File ".../qemu/tests/docker/docker.py", line 41, in _guess_docker_command stdout=DEVNULL, stderr=DEVNULL) == 0: File "/usr/lib64/python2.7/subprocess.py", line 523, in call return Popen(*popenargs, **kwargs).wait() File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1343, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory .../qemu/tests/docker/Makefile.include:47: recipe for target 'docker-image-centos6' failed make: *** [docker-image-centos6] Error 1 Change _guess_docker_command() to handle OSError exceptions raised by subprocess.call(), so we will keep looking for other commands and print a better error message. New output will be: $ make docker-test-quick@centos6 V=1 .../qemu/tests/docker/docker.py build qemu:centos6 .../qemu/tests/docker/dockerfiles/centos6.docker Traceback (most recent call last): File ".../qemu/tests/docker/docker.py", line 343, in <module> sys.exit(main()) File ".../qemu/tests/docker/docker.py", line 340, in main return args.cmdobj.run(args, argv) File ".../qemu/tests/docker/docker.py", line 235, in run dkr = Docker() File ".../qemu/tests/docker/docker.py", line 102, in __init__ self._command = _guess_docker_command() File ".../qemu/tests/docker/docker.py", line 49, in _guess_docker_command commands_txt) Exception: Cannot find working docker command. Tried: docker sudo -n docker .../qemu/tests/docker/Makefile.include:47: recipe for target 'docker-image-centos6' failed make: *** [docker-image-centos6] Error 1 Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <1474369559-16903-1-git-send-email-ehabkost@redhat.com> [exceptions.OSError -> OSError and drop the import. - Fam] Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: silence debootstrap when --quiet is givenSascha Silbe
If we silence docker when --quiet is given, we should also silence the .pre script (i.e. debootstrap). Only discards stdout, so some diagnostics (e.g. from git clone) are still printed. Most of the verbose output is gone however and this way we still have a chance to see error messages. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-9-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: build debootstrap after cloningSascha Silbe
When using the git version of debootstrap (because no usable version of debootstrap was installed on the host), we need to run 'make' so that devices.tar.gz gets built. Otherwise the first debootstrap stage will fail without printing any error message. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-8-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: make sure debootstrap is at least 1.0.67Sascha Silbe
debootstrap prior to 1.0.67 generated an empty sources.list during foreign bootstraps (Debian#732255 [1]). Fall back to the git checkout if the installed debootstrap version is too old. [1] https://bugs.debian.org/732255 Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-7-git-send-email-silbe@linux.vnet.ibm.com> [Update 'sort -C' to 'sorc -c &>/dev/null' - Fam] Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: print warning if EXECUTABLE is not set when building debootstrap imageSascha Silbe
Building the debian-debootstrap image will usually fail if EXECUTABLE isn't set (when using the Makefile). Warn the user in this case so they know why it's failing. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-6-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: debian-bootstrap.pre: print helpful message if DEB_ARCH/DEB_TYPE unsetSascha Silbe
The debian-bootstrap image doesn't choose a default architecture and distribution version, instead the user has to set both DEB_ARCH and DEB_TYPE in the environment. Print a reasonably helpful message if either of them isn't set instead of complaining about "qemu-" being missing or erroring out because we cannot cd to the mirror URL. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-5-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: debian-bootstrap.pre: print error messages to stderrSascha Silbe
Send error messages where they belong so they're seen even if stdout is redirected to /dev/null. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-4-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: avoid dependency on 'realpath' packageSascha Silbe
The 'realpath' executable is shipped in a separate package that isn't installed by default on some distros. We already use 'readlink -e' (provided by GNU coreutils) in some other part of the code, so let's settle for that instead. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Message-Id: <1473192351-601-3-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker.py: don't hang on large docker outputSascha Silbe
Unlike Popen.communicate(), subprocess.call() doesn't read from the stdout file descriptor. If the child process produces more output than fits into the pipe buffer, it will block indefinitely. If we don't intend to consume the output, just send it straight to /dev/null to avoid this issue. Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com> Reviewed-by: Janosch Frank <frankja@linux.vnet.ibm.com> Message-Id: <1473192351-601-2-git-send-email-silbe@linux.vnet.ibm.com> Signed-off-by: Fam Zheng <famz@redhat.com>
2016-09-08docker: Add a glib2-2.22 imageFam Zheng
It's a variation of our existing centos6, plus two more lines to downgrade glib2 to version 2.22 which we download from vault.centos.org. Suggested-by: Paolo Bonzini <pbonzoni@redhat.com> Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1470708908-12885-1-git-send-email-famz@redhat.com>
2016-08-05docker: Add "--enable-werror" to configure command lineFam Zheng
We don't have .git in the docker checkout, add this to enable -Werror explicitly. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1469453510-658-1-git-send-email-famz@redhat.com
2016-08-05docker: Be compatible with older dockerFam Zheng
By not using "--format" with docker images command. The option is not available on RHEL 7 docker command. Use an awk matching command instead. Reported-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1470202928-3392-1-git-send-email-famz@redhat.com>
2016-07-20docker: pass EXECUTABLE to build scriptAlex Bennée
To build a docker image with which needs qemu linux-user emulation we need to pass --include-executable to the build script. Using the same mechanism as for other container controls we enable the option is EXECUTABLE is set on the make command line e.g: make docker-image-debian-bootstrap V=1 J=9 DEB_ARCH=armhf \ DEB_TYPE=stable EXECUTABLE=./arm-linux-user/qemu-arm Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-11-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20docker: Don't start a container that doesn't existFam Zheng
Image building targets are dependencies of test running targets, so when a docker image doesn't exist, it means it's skipped (due to dependency checks in pre script). Therefore, skip the test too. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-10-git-send-email-famz@redhat.com
2016-07-20docker: Add "images" subcommand to docker.pyFam Zheng
This is a wrapper for the 'docker images' command. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-9-git-send-email-famz@redhat.com
2016-07-20docker: Fix exit code if $CMD failedFam Zheng
Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-8-git-send-email-famz@redhat.com
2016-07-20docker: More sensible run scriptFam Zheng
It is very easy to figure out current directory and bash option from the execution, so do less in the Makefile invocation command line, and figure both options in the script. This makes the next patch easier. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-7-git-send-email-famz@redhat.com
2016-07-20tests/docker/docker.py: add update operationAlex Bennée
This adds a new operation to the docker script to allow updating of binaries in an existing container. This is because it would be inefficient to re-build the whole container just for an update to the QEMU binary. To update the executable run: ./tests/docker/docker.py update \ debian:armhf ./arm-linux-user/qemu-arm Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-6-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/dockerfiles: new debian-bootstrap.dockerAlex Bennée
Together with the debian-bootstrap.pre script can now build an arbitrary architecture of Debian using debootstrap. This allows debootstrap to set up its first stage before the container is built. To build a container you need a command line like: DEB_ARCH=armhf DEB_TYPE=testing \ ./tests/docker/docker.py build \ --include-executable=arm-linux-user/qemu-arm debian:armhf \ ./tests/docker/dockerfiles/debian-bootstrap.docker Although a number of non-debian systems package the debootstrap script it is fairly portable in itself. Assuming we have some sort of fakeroot implementation we can just clone the upstream repository and use the script from there. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-5-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: check and run .pre scriptAlex Bennée
The docker script will now search for an associated $dockerfile.pre script which gets run in the same build context as the dockerfile will be. This is to support pre-seeding the build context before running the docker build. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-4-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: support --include-executableAlex Bennée
When passed the path to a binary we copy it and any linked libraries (if it is dynamically linked) into the docker build context. These can then be included by a dockerfile with the line: # Copy all of context into container ADD . / This is mainly intended for setting up foreign architecture docker images which use qemu-$arch to do cross-architecture linux-user execution. It also relies on the host and guest file-system following reasonable multi-arch layouts so the copied libraries don't clash with the guest ones. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-3-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: docker_dir outside buildAlex Bennée
Instead of letting the build_image create the temporary working dir we move the creation to the build command. This is preparation for the later patches where additional files can be added to the build context before the build step is run. We also ensure we remove the build context after we are done (mkdtemp doesn't do this automatically for you). Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-2-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-08tests/docker: build all targets in test-clangPaolo Bonzini
Warnings specific to clang may affect devices that are not build by x86_64-softmmu and aarch64-softmmu. Build all targets since that is also what Peter does. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1465224417-141321-7-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-08tests/docker: support travis test with fedora imagePaolo Bonzini
Install sparse and PyYAML. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1465224417-141321-6-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-08tests/docker: remove unused feature "ccache"Paolo Bonzini
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1465224417-141321-5-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-08tests/docker: fix test-mingwPaolo Bonzini
Add flex and bison for use in test-mingw, because test-mingw uses the in-tree libdtc. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1465224417-141321-4-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-08tests/docker: make test-full build all targets, not nonePaolo Bonzini
Fix common.rc to avoid passing an empty --target-list= option to configure. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1465224417-141321-3-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-08tests/docker: fix make-archive-maybePaolo Bonzini
make-archive-maybe expects an archive path relative to $1, but receives a path relative to the current directory. Redirect the output outside the subshell to bypass the "cd $1". Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1465224417-141321-2-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-06-07docker: Don't use eval trick on MakefileEduardo Habkost
The eval trick for defining DOCKER_SRC_COPY doesn't do anything useful, as DOCKER_SRC_COPY is immediately expanded just after it is defined, and CUR_TIME is already defined using ":=". Simply define it using ":=" so it is evaluated only once. The eval trick was also triggering an weird error on Travis builds: qemu/tests/docker/Makefile.include:34: *** unterminated variable reference. Stop. The issue is not easily reproducible (maybe it's a bug in some versions of Make), but it is avoided if removing the eval trick. Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-06-01docker: Add EXTRA_CONFIGURE_OPTSFam Zheng
Whatever passed in this variable will be appended to all configure commands. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1464755128-32490-15-git-send-email-famz@redhat.com
2016-06-01docker: Add travis toolFam Zheng
The script is not prefixed with test- so it won't run with "make docker-test", because it can take too long. Run it with "make docker-travis@ubuntu". Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1464755128-32490-13-git-send-email-famz@redhat.com
2016-06-01docker: Add mingw testFam Zheng
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1464755128-32490-12-git-send-email-famz@redhat.com
2016-06-01docker: Add clang testFam Zheng
The (currently partially commented out) configure options are suggested by John Snow <jsnow@redhat.com>. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1464755128-32490-11-git-send-email-famz@redhat.com
2016-06-01docker: Add full testFam Zheng
This builds all available targets. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1464755128-32490-10-git-send-email-famz@redhat.com
2016-06-01docker: Add quick testFam Zheng
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1464755128-32490-9-git-send-email-famz@redhat.com
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-28 06:25:55 +0000