Including test of a build ISO-Image

Answered

Permanently deleted user

Created October 14, 2020 09:07

We are using teamcity to build a custom ubuntu ISO which works fine so far. I already distribute the build .iso-file via ssh to some storage where our VMWare vSphere Hypervisor are able to access it. I would like to go the next step now and automatically install the ISO via the vSphere API, execute some tests on the installed VM and report that back to the build configuration from which the ISO originated from.

How do I do that conceptually with Teamcity? Do I just create some more buildsteps for this and handle it via own scripts and define that my test succeded if the build step succeded? From https://www.youtube.com/watch?v=kraM_K4QyGs&list=PLQ176FUIyIUZYwvmi81qloBtZGi40C05m&index=47 It looked for me that there is a destinct tab for tests, however that video is really old and I cant find any scope especially for "tests" in Teamcity.

However, I also would like to not only run one VM from that ISO but multiple, all of them running different tests in the installed VMs. From https://teamcity-support.jetbrains.com/hc/en-us/community/posts/360000877980-Build-parallel-steps it looks like I have to create multiple build configurations for this?

Also, if there is any more documentation, guides, videos, best practices etc. on how to include tests in teamcity I would be glad for some links.

Greetings,

Niklas

1 comment

Fedor Rumyantsev

Created October 14, 2020 13:16

Hello Niklas,

There are several things which I would like to mention in relation to this request:

1) What test framework do you plan to use in order to run the tests on newly-started VM? While there could be no built-in integration, you may still report tests to the TeamCity during build execution so that they will appear on the build results. This is possible with the Service Messages feature (https://www.jetbrains.com/help/teamcity/service-messages.html#Reporting+Tests); essentially, if you print following lines to stdout during the tests execution:

##teamcity[testStarted name='testName' captureStandardOutput='<true/false>']
<here go all the test service messages with the same name>
##teamcity[testFinished name='testName' duration='<test_duration_in_milliseconds>']

then a build would register a new instance of test testName and mark it as a successful one. More examples of usage are available on the link above; of course, this also allows to fail a build if specific test fails, keep track of the tests from build to build and otherwise utilize the test-related functionality of TeamCity. The general details on the tests in TeamCity is available here: https://www.jetbrains.com/help/teamcity/viewing-tests-and-configuration-problems.html

2) Yes indeed, if you would like to run several test sequences in parallel, running multiple configurations could be a way to go. As setup and teardown operations are likely to be the same for each of the "test VM" configurations, you could either:

a) consider first running a custom script which will upload ISO to vSphere if it is not already there, start a VM and start a connection, then saving it as a meta-runner (https://www.jetbrains.com/help/teamcity/working-with-meta-runner.html) so you may reuse it on the different configurations (or save it as a build configuration template (https://www.jetbrains.com/help/teamcity/build-configuration-template.html) for the same goal).

b) there is a ready integration for vSphere with TeamCity allowing to start and stop cloud agents on demand, but as far as I can see it is incapable of uploading images to the cloud or starting machines from image directly, not from the template or existing VM. Still, if you would upload the image and create a template with a script, this integration could maybe assist you with starting and stopping agents automatically. The details of setup are described here: https://www.jetbrains.com/help/teamcity/setting-up-teamcity-for-vmware-vsphere-and-vcenter.html#top

I hope this helps; please do not hesitate to reach out if there is anything else I could assist with.

Please sign in to leave a comment.