Reporting Results of Integration Tests with Nuget Packages
My org is trying to improve our integration testing across the board. We have a private Nuget repository on a separate server with multiple Nuget packages that reference each other. Basically, on any commit to a branch in a package repo, we want everything that consumes the package to update the package version and run its own test suite with the new package.
So what we just set up is like this. For simplicity, let's say we have 2 projects - named "Package" and "App." App references Package.
- Someone commits to Package repo, triggering a Package Unit Tests build
- The Package Unit Tests build sets a unique build number based on what branch it's in. If it's in branch "my-feature," the build number will be like "1.0.0-myfeature.{build.counter}"
- If the Package Unit Tests build passes all tests, it publishes the package to a CI nuget feed that we have on our Nuget repository.
- App "CI Tests" build has a Finish Build Trigger that watches Package Unit Tests.
- CI Tests build parses the %teamcity.build.triggeredBy% parameter to get the build number/package version of the newly finished Package build. It also sets its own build number to %build.counter%-{Package build number} so we can easily see where the changes came from.
- CI Tests build uses the Nuget Restore build step to update to the new package, then runs its own test suite.
This seems to be working ok for now. However, we'd like more automation for getting the results of the CI Test's builds. For example, we'd like to see the changes in the Package repo along with each App build. We would also potentially like to report the results to Jira and Bitbucket.
I see two options to do this right now.
- Add a Package VCS root to the CI Tests build. One issue here is that the changes for both the App and Package would be shown here. For the Bitbucket integration, we can select which VCS root to report status to, but I could easily see the Jira integration statuses getting tangled up.
- We implement it ourselves using the way too much direct API interaction with TeamCity, Jira, and Bitbucket. That seems like a big job.
I guess, actually typing this out, I have a pretty good idea of how to accomplish what we want. So my question is more, are we using the right tools for this? It feels a little like we're trying to fit a square peg in a round hole.
Please sign in to leave a comment.
As a side note, you don't necessarily need to attach the Package VCS root to the CI build if you want to see the Package changes in CI/App builds. If the CI/App build configuration has a snapshot dependency on the Package configuration, it should be sufficient to just tick the 'Show changes from snapshot dependencies' checkbox in the CI/App settings > Version Control Settings.
Cheers,
Anatoly
Hi Anatoly,
Thanks for the advice!
I think you're right about the Jira thing. If it ends up getting confusing, that's probably another part of our process that we can improve.
The snapshot dependency suggestion is very helpful. I had looked into them before, but I thought it was intended for multiple builds in the same repo. It was very easy to set up and get the changes showing up. I think it will also simplify the "parse triggeredBy to get Package's build number" step.
Is there an easy way to get the snapshot dependency's build ID? My current plan is, the first time a build runs, get it from REST via /app/rest/buildTypes/AppCiBuild/snapshot-dependencies, then save it as a parameter. Then we can easily access the dep.PackageBuild.x parameters.
Thanks!
Jordan