Internal error message when running a build - but has no details

Avatar
Steve Tapley
Created

Hi,

We are having an issue where TeamCity reports an error but gives no details as to whats gone wrong.

In our build log in the UI, I see

Internal error message details: BuildMessage1 2916 Text DefaultMessage ERROR 1894394472147300

or

Internal error message details: BuildMessage1 2901 Text DefaultMessage ERROR 1840463546471600

The numbers change every build.

Looking on the agent, in the teamcity-agent.log and everything is marked as INFO - no error message. Also on the agent, looking at the teamcity-build.log has the messages around the error message with the same timestamps, but the message I posted is not present in those logs.

Looking on the server, in teamcity-server log, and once again everything is marked as INFO, no error message

I'm not really sure where to look now.

Some background is that the error occurs when running an angular "ng build" command. It does take a while (a few minutes) to complete but it is building successfully (the TC build does continue and everything works as expected as far as deployment goes)

I currently use a command like this

ng build project-name --progress=false

which is generating the above error.

 

Prior to this it was simply

ng build project-name

However, the progress messes with TeamCity and it reports an error and fails the build because of the progress messages the Angular CLI is reporting.  See a snippet of the build log

We have gotten around this error by muting the message, but it hides real error messages so I'm keen to fix this properly.  Note that this is NOT an error in the ng build process - everything is ok and building properly.  I suspect TC sees the angular CLI interim error message and triggers a build failure.

 

Any ideas please?

 

 

 

Votes

0

Share

12 comments
Avatar
Fedor Rumyantsev
Created

Hello Steve,

It appears that you will see this error when a service message is sent by agent but service message type is not recognized. May I ask you to temporarily add the following build parameter to affected build?

teamcity.internal.build.serviceMessages.logOriginal=true

It will make all service messages sent by agent to appear in the build log. After that, can you please rerun build and upload build log via uploads.jetbrains.com site?

0
Avatar
Steve Tapley
Created

Hi,

I have set the build parameter like so:

I've uploaded a file here: Upload id: 2023_04_28_M6o8diqqSYhpuP2vG2x8pU (file: Methodology_Publish_to_QA_10557.log.zip)

 

From what I can tell it didn't log anything different....but maybe I've uploaded the wrong file :(

Thanks,

Steve

0
Avatar
Fedor Rumyantsev
Created

Hello Steve,

Apologies, this is my bad - looks like this parameter captures only those service messages where service message type was resolved. May I ask you to instead enable XMLRPC debug on any agent where this build may run, run the build and share all teamcity-xmlrpc.log files? They will capture raw communication between agent and server. 

0
Avatar
Steve Tapley
Created

Hi Fedor,

Sorry about the delay

The uploaded the log file from the build agent in question here: 2023_05_04_7GphDXjHacUVSnQ4q5XaSw

Thanks,

Steve

0
Avatar
Fedor Rumyantsev
Created
Hello Steve,

Thank you! Can you please also share the build log for this build so I will cross-reference the ID mentioned in the error message and log?
0
Avatar
Steve Tapley
Created

Ok, I've uploaded and the id is: 2023_05_05_ir2cbTUmH1uySUUvmnaxDw

Thank you!

0
Avatar
Fedor Rumyantsev
Created
Hello Steve,

Thanks again! At a glance, this looks like an average message (just an empty line sent by agent, there are a lot of similar messages in the log prior to the error) , so I have reached out to our devs with the data and hope to provide you with an update shortly.
0
Avatar
Fedor Rumyantsev
Created
Hello Steve,

Unfortunately, we are still not sure on the issue here. The problem is that the text of the problematic message may not be processed and is not logged in XMLRPC log. I have tried to reproduce the issue by running Angular builds locally, but had no luck with that.

Can you please try to run the same command manually and pipe the output to a text file, then share it? Alternatively, if you build an open-source project, can you let me know the repository name so I will try to reproduce it on my instance?

I am most sorry about this investigation taking a while; for any questions or concerns, please do not hesitate to reach out.
0
Avatar
Steve Tapley
Created

Hi Fedor,

No worries about the time, just hope we can get to the root cause here....

 

I've uploaded those files and the id is:2023_05_11_7BsqLAmhx4dVwT9HVBXgKp

The command line for when we don't show angular CLI build progress is:

npx --node-options="--max-old-space-size=8192" ng build alto --configuration=staging --progress=false >> stdout.txt 2>> stderr.txt

 

The command line showing progress is:

npx --node-options="--max-old-space-size=8192" ng build alto --configuration=staging --progress=true  >> stdout_withprogress.txt 2>> stderr_withprogress.txt

 

From my look at those files, there's nothing in there, that might be of help.  The timing to do this on my dev machine is about 2 minutes for a build.  Our QA server is prob about 6 minutes.

Thanks,

Steve

0
Avatar
Fedor Rumyantsev
Created
Hello Steve,

Unfortunately, I still could not reproduce the issue locally - I can only assume that it somehow has to do with the project you are building. The output files you have shared, if fed line by line to the agent, do not trigger the error either.

When you redirect output to the files, do you see the same issue with the build?
0
Avatar
Steve Tapley
Created

Hi Fedor,

We don't see the same issue with the build, unfortunately.  What I'll do is try to get this simplified and isolate the issue (Unfortunately, it'll be a spare-time task for a couple of weeks though).

Cheers,

Steve

0
Avatar
Fedor Rumyantsev
Created
Hello Steve,

OK, looking forward to hear from you! 
On a side note, you may also try to capture the build output with tee command - hopefully this will let us see the raw output that produces the error.

Thanks so much for your patience with this issue.
0

Please sign in to leave a comment.