Report custom runner output

For the time being there is no dedicated ServiceMessage-targeted extension point. (I've created an issue on this: TW-13347)

You can listen for all the messages logged by the builds via BuildServerListener.messageReceived and use "ServiceMessage.parse" to try to parse the message. Please ensure that you do not perform any long tasks inside BuildServerListener as it is performed synchroniously.

You will probably need to persist the data associated with the build. The options here seems to be a custom stoage on the file system under .BuildServer/system or in the build log, dependning on the logic associated with the data processing.

yaegor wrote:

You will probably need to persist the data associated with the build. The options here seems to be a custom stoage on the file system under .BuildServer/system or in the build log, dependning on the logic associated with the data processing.

Any advice on how to do this? Isn't the data in the build log already?

Simone,

This depends too much on what information do you need to store and what reports to produce.
When you need to generate a report, you can probably go through the build log and parse all the necessary data. However, this will only work if the build log is not large and the report can be built quickly.

yaegor ha scritto:

Simone,

This depends too much on what information do you need to store and what reports to produce.
When you need to generate a report, you can probably go through the build log and parse all the necessary data. However, this will only work if the build log is not large and the report can be built quickly.

Yegor, something similar to how tests are reported - I would liketo parse the build log to find my custom messages - and show them formatted somehow on a custom tab as soon as they are generated.

Yegor, btw the reason is this. I have created a custom runner plugin which can run C# code supplied inline. This is quite useful for me to execute small scripts quickly, for example to perform some analysis on the code or make web calls.
I would like to capture the output which comes from the external process which compiles and runs the .NET code in order to show it on its own tab on the fly. At the moment it is only displayed on the build log and on a custom report tab I have created.

In case you're going to try it at the moment it only runs simple one liners like Console.WriteLine() (I'm planning to support statements and entire programs along the lines of LINQPad) and the external executable runs on the .NET fx 4. I've tested it on the latest EAP but I have not relied (I suppose) on any new classes introduced there since I will have to run it on 5.1.x at work.

Simone,

A greate initiative!

As to displaying the output in a separate tab on the fly:
Currently there is no way to add something into build log that will not be displayed on the build log tab itself (though, we might need to add something like that in the future).

So you can either add messages into the build log and later parse the log to display them also on a separate tab, or you can publish a build artifact file with the output (probably continiously while the build is running) and displaying the content of that file in the web.

function(){return A.apply(null,[this].concat($A(arguments)))}

yaegor ha scritto:

Simone,

A greate initiative!

As to displaying the output in a separate tab on the fly:
Currently there is no way to add something into build log that will not be displayed on the build log tab itself (though, we might need to add something like that in the future).

So you can either add messages into the build log and later parse the log to display them also on a separate tab, or you can publish a build artifact file with the output (probably continiously while the build is running) and displaying the content of that file in the web.

Yegor, it's fine for me if they are displayed in the build log, as tests do, what I would like to know is where I can access those messages for displaying them in my custom tab. Is there a class which provides the build log built so far as the build is running?

I've thought about publishing the output file as an artifact continuously, I was a bit worried about the network traffic it would generate once I start dumping large collections of data on the output.

Simone,

> Is there a class which provides the build log built so far as the build is running?
You can probably use SBuild.getBuildLog().getMessagesIterator() to iterate through the build log.

Hi Yegor,

I fiddled a little more with this one, and I have a couple of questions.

I was trying to figure out if I could create my own service message, but I cannot get TC to pick it up. I have implemented my custome message class by extending ServiceMessage then created and registered my message translator by implementing ServiceMessageTranslator<MyMessage>, but it seems they are not used at all.

Some documentation about how this stuff work would be very useful.

The other question is about how to publish artifacts as the build is running from the API. I know you can do that with service messages, but in the API I only found the ArtifactsWatcher.addNewArtifactsPath method.

Thanks

Simone,

ArtifactsWatcher is available on agent only and it is correct service to use if you need to publish additional artifacts at runtime. The argument can be in the same format as on the General Settings page in the build configuration administration area.

Simone,

As to service messages, as I've noted in my first reply there is no extension point to add processing of own servcice messages, you will need to read the build log and parse the lines for your service messages.

Thanks Yegor,

I thought I had misunderstood you since in this page I read that one extension point is ServiceMessageTranslator, so I thought there was a way to plug-in your custom messages. So it's not really an extension point, is it?

Simone,

This extension is to process known service messages, but there is no way to add new service types kinds into the system.
Actually, the extension point you mentioned is not very usable at the moment and will probably be rewoeked in the scope of TW-13347.

Hi Yegor, is it possible to have the build of the plugin hosted on your demo server? It's reached a point where it's usable right now.

As for output reporting, I've gone the route of publishing fragments of output on the agent side and loading them via ajax from the server, it's proven to be quite nice from the UI perspective.

Simone,

I've added the build. If you register on the server we can make you administrator of the project.

I did, simoneb. I see that it's failing right now, maybe I'm using something that's not installed on your server, specifically a class used to encode in base64.

Another question, I'd need to build also the .NET runner part, which being in the same folder structure just copies the resulting executable into the plugin folder, which then picks it up when building the output. Any suggestions on how to better do that?

Simone,

The error is because of JDK 1.6 API usage.
You can use getBytes(String) and jetbrains.buildServer.util.StringUtil.isEmpty() instead of String.isEmpty()  if you want to get JDK 1.5 cmpatibility.
Othervise, you can switch to using JDK 1.6 in the build.

I've made you administrator for the project.

> Another question, I'd need to build also the .NET runner part, which being in the same folder structure just copies the resulting executable into the plugin folder,
> which then picks it up when building the output. Any suggestions on how to better do that?

The main issue here is that we do not have .Net framework 4.0 installed on the agents on teamcity.jetbrains.com at this time.
We will add it in some time. For now you can just check-in the binaries of the C# code.

Overall, here are several options:

Option 1: build all in a single build. This will require calling either MSBuild from Ant or Ant from MSBuild.
Option 2: make building C# code a separate build and use the artifacts in the Java build. If you anticipate making changes to both parts in the same commits it also makes sense to add snapshot dependency between them.
Current TeamCity 6.0 EAP also has option 3: use both Ant and VS Solution in the same build.

I've just setup Option 2 (without snapshot dependencies yet). It should become compatible with agents when we install framework 4 on the AMI used by the server.

Thanks Yegor,

I've switched to using stuff which is in JDK 1.5 and now the plugin part compiles fine.

WRT the external runner, I'm using VS2010 but I'm not using anything specifically which requires the .NET fx 4.0, but I've thought a bit about future developments.

Inside the external runner I compile the code using the CSharpCodeProvider class. Now this class is in the System.dll assembly which of course is different for .NET 2.0/3.5 and 4.0. The implementation of this class in System.dll-2.0 cannot pick up the 4.0 compiler even if the .NET fx 4 is installed on the machine, while the one in System.dll-4.0 can pick up both. So compiling in VS2008 won't let me target the fx 4.0.

Therefore I'm planning to keep compiling in VS2010 since it lets me target both 3.5 and 4.0, and create two projects generating two executables which reference the same sources but just target a different fx so that the plugin will be able to run both on the .NET fx 3.5 and 4.0 just by switching the executable. Of course the fx 4.0 runner would be able to compile fx 3.5 coee just fine, but it would impose the fx 4.0 as a requirement.

I will be providing a choice on the runner UI. Makes sense? Then, WRT this, I noticed that some classes which are not in the open API would be very useful as they expose agent parameters that can be checked to provide dynamic requirements for agents depending on whether the user wants to compile for 3.5 or 4.0.

Unfortunately I could not find a way to make this code run fine at runtime:

@Override
    public List<Requirement> getRunnerSpecificRequirements(@NotNull Map<String, String> runParameters) {
        MSBuildVersion version = MSBuildVersion.V3_5;
        RunPlatform platform = RunPlatform.x86;

        return Collections.singletonList(new Requirement(version.getDotNetProperty(platform), null, RequirementType.EXISTS));
    }

which fails with a:

Error message: jetbrains/buildServer/runner/MSBuild/MSBuildVersion

Show stacktrace

Trace: java.lang.NoClassDefFoundError: jetbrains/buildServer/runner/MSBuild/MSBuildVersion
     at csharpRunner.server.CSharpRunnerRunType.getRunnerSpecificRequirements(CSharpRunnerRunType.java:30)

I can of course hard-code the name of the parameters which signify that a specific version of the fx is installed, but relying on what's already there would be safer.

Thanks for setting up the build btw.

Your code crashed in the server because of classes isolation that is done while plugins are loaded. .NET plugin have configuration that prescribes loading it's classes to separated classloader, thus another plugins will not be able to use classes of .NET plugin.

Recently, I've checked in a refactoring that moved all system.DotNetFramework*, system.VS20**, system.WindowsSDK* system properties to configuration parameter. For your code it only means that you may ommit 'system.' prefix in the names.

For now, I would recommend you to hardcode property names in your build runner. To make your build runner compatible with both 5.1 and 6.0, create the following requirements:

new Requirement(RequirementQualifier.EXISTS_QUIALIFIER + "(system\\.)?DotNetFramework4.0_x86", null, EXISTS))

Using EXISTS_QUIALIFIER allows you to put regex for the name of a property the is checked.

If you targeting your plugin for 6.0 only, consider using BuildServiceAdapter class as a base class for your CommandLineBuildService implementation.

From Yegor:
> I will be providing a choice on the runner UI. Makes sense?
If you can make a choice without user's setting (e.g. try to use latest from the avaialble ones), this can be better.
However, if user may care for the version of the runtime to use (e.g. to  use 4.0 features), then the setting seems to be the way to go.

Eugene, Yegor,

thanks for your replies. I've actually gone a totally different route compared to what I described before, which I think complies with your suggestions.
I chose to build with VS2008 only (and thus change the configuration on the build server, which now compiles fine) targeting 3.5 and providing support (preference, actually) for v4.0 using the configuration file. This should lead to the following situations:

1. if only the fx 3.5 is installed then the plugin will run on fx 3.5 and will be able to compile C# 3 code which will run on CLR 2
2. if both fx 3.5 and 4.0 are installed then the plugin will run on fx 4.0 and will be able to compile both C# 3 and 4 code, which will run on CLR 4
3. not sure what happens if only fx 4.0 is installed as the executable is compiled for targeting fx 3.5, I don't have a machine with fx 4.0-only installed but my guess is that it should be the same as case 2.

Given the above assumptions, therefore, the only strict requirement is that at least fx 3.5 is installed - and I will follow you suggestions for imposing that, Eugene.
It will be a bit harder to detect if you're supplying C# 4 code and impose the fx 4.0 requirement dynamically, but I think I can live with it for now.

WRT the dependency between the external runner and the plugin itself I have created both a snaphot and artifact dependencies between the build types. Now the zip built on the server is usable.

Simone,

Cool!

Keep us informed on the progress!

Well, for the most part it's already usable, it works pretty much the same as LINQPad does, and it was my intent, you can either write simple expressions, statements or programs and they will be compiled and run on the agent.

Then there is a Dump extension method which prints a nice html output which is published as an artifact on the server, and an overall report is shown in a custom tab and updated on the fly via ajax.

You can for example type this code:

new { Prop1 = 1, Prop2 = "ciao" }.Dump();
Thread.Sleep(2000);
new KeyValuePair<int, string>(1, "ciao").Dump();
Thread.Sleep(2000);
new[]{new{Prop1 = 1, Prop2 = 2}, new{Prop1 = 1, Prop2 = 2}}.Dump();
Thread.Sleep(2000);
1.Dump();
Thread.Sleep(2000);
new[]{1,2,3}.Dump();
Thread.Sleep(2000);
new { Prop1 = new[] { new {Prop11 = "ciao"} },   Prop2 = 1  }.Dump();
Thread.Sleep(2000);
new object[]{ 1, "ciao", new { Prop1 = "a", Prop2 = 234}, "pollo", new{ Prop2 = new[]{1,2,3} }, new[]{4,5,6}, new { Prop3 = DateTime.Now }}.Dump();

and you'd get the report below:

Hi Yegor,

the plugin is pretty much ready for usage, if you like you can link to it somewhere from the TC docs or where you feel like the best place for such stuff.

Simone,

Thank you for the update!

It's been some time since I've added your plugin to our plugins page. Just updated the description, though.
If you want to add it to other places - please let me know.

BTW, I'd appreciate if you can share your experience with TeamCity plugin writing: overal impresison, pain points, etc.

Hi Yegor, sure, here they are:

• considering that I'm a java noob it's been quite easy to startup, mostly thanks to the project template that you provide on the svn repository, I started from there, and looking at the other OS plugins
• the automatically generated ant script is great, I'm not used to something like this in the .NET world
• the wrapper around running external processes from the agent is very straightforward to use
• the open API docs are a bit lacking and I had to resort to decompiling the binaries of the plugins on the server to make sense of how some stuff worked - service messages, build log - that is
• having the server restarted each time you make changes is painful, even after following the guidelines on the docs. To automate this a bit I set up a build on my local TC server which pulled the sources of the plugin, built, deployed and restarted the server so that I could see the changes in the least amount of time
• you guys are quite responsive on the forums so asking for and getting help is straightforward
• the overall object model is quite good, nice to develop against
• reference implementation for some kinds of plugin is missing, for instance issue tracker integration
• documentation about web UI framework is totally missing, it's hard to figure out that many tags for common tasks are already there unless you dig into the TC web app sources yourself

Simone,

Thank you for the feedback!

We will try to address the inconveniences that you've mentioned.

To automate this a bit I set up a build on my local TC server which pulled the sources of the plugin, built, deployed and restarted the server so that I could see the changes in the least amount of time

That's smart!

reference implementation for some kinds of plugin is missing, for instance issue tracker integration

Please vote for TW-9103

documentation about web UI framework is totally missing, it's hard to figure out that many tags for common tasks are already there unless you dig into the TC web app sources yourself

That is true, looking into the existing pages is the best approach for now. Not sure we will go the route of fully documenting all that staff as the documentation will always be behind the code... May be something like JavaDoc can work there...

That is true, looking into the existing pages is the best approach for 
now. Not sure we will go the route of fully documenting all that staff 
as the documentation will always be behind the code... May be something 
like JavaDoc can work there...

Yes that would be enough, the issue is not that it's hard to use, but that you don't know that tags and the rest even exist.

simone_b wrote:

• documentation about web UI framework is totally missing, it's hard to figure out that many tags for common tasks are already there unless you dig into the TC web app sources yourself

I'd agree with all of Simone's points there - could do with a better idea of which classes/interfaces are used at which stage. Maybe some kind of flowchart/lifecycle of a build would be useful, showing which extension points apply?

Also with the web UI stuff specifically -- it would be very useful to have some more examples as most of us do just copy/paste to learn from existing examples.

A couple of examples that have me stumped at the moment:
* a simple text property editor that doesn't show all of the different parameters available
* how to do a simple drop-down list (combo box) with a specified set of options

Looking at other plugins, especially things like the fxcop and rake runner, were very useful to see the right way of doing JSP pages.

Without people answering questions on the forum I'd be quite stuck by now! So I'm happy that I'm not ;)


It would also be good if there was a sample plugin that showed both server- and agent-side things - e.g. a VERY simple BuildRunner.  I've just put one together myself and it took a couple of days to figure it out as I've been away from java/ant for some time!

I wonder if there should be a recommended layout for plugins e.g. where they get their libs from, a standard ant build script --  teamcity-common.xml is very useful but not everything uses it?