Configurations with Large Build Logs

Answered

Hello,

I used the Server Health tool and it discovered that there were 2 monstrously large build logs (4.1GB and 13GB). I wasn't sure what the right way to fix this was but here's what I did:
I went directly to the log files in system/messages and did something like this: echo "Deleted build log because it was 13GB." > system/messages/CH94/88894.msg5

This reduced the excessive amount of disk space used. However, when I run the server health report, it still thinks the files are the same size and the log files now read, for example:
Attempt to read outside of build log file. It seems the build log "/data/teamCity/system/messages/CH94/88894.msg5" is broken. This can be caused by incorrect server shutdown. Please contact your system administrator.

I have 2 questions:
1. Is there now a way to make the log file display the plain text I added to it without displaying an error?
2. What is the right way to delete just one or two large build logs?

Thanks,
Dave

6 comments
Comment actions Permalink

Build log has a binary format, so you can't modify it easily. To get rid of these build logs you can either remove them from disk (in this case TeamCity will show message in UI that it can't find build logs), or remove the build itself (Actions -> Remove).

0
Comment actions Permalink

TeamCity is written in Java ... so perhaps a TeamCity lib/jar could be used to generate a buildLog.msg5 file with generic message (no contextualisation like exact size or date), used to replace the original file ?

If is there a source code or doc entry point, I'll take it ! :-)

0
Comment actions Permalink

Hello Alix,

Why do you need to replace the original .msg5 file?

0
Comment actions Permalink

Hi Alina,

In a global / mutualized unique CI platform for all projects in a company, when there are some bad practices, they could impacts the exploitation of the platform.

One case is the "too many log generation". If project generates more ~2-3Go by build (DEBUG/TRACE log level during unit tests ... so Spring/Hinernate/etc kilometer of logs), with builds history & build type (integration, release, ...) the result could be ~50Go for one project.

In this case:
1) The build log tab is unusable by the users
2) The TeamCity hard-disk could be full quickly (=> exploitation task)

So replacing this logs (ex: more one 1Go, old than 2 days) could be useful.

Easy to do with a Linux shell script searching this file (perhaps with TeamCity REST API, but a little bit more complicated).

I have done the job by generating a "generic" buildLog.msg5 & buildLog.msg5.i1 based on a Command Line step & echo the wanted message, used to replace these big original files in the previous shell script.
But this workaround is a "real build log" => The message is here but with a wrong date, build name, ...

If it could contain only one line, or generated dynamically (in Java) with contextualized message (original size, date, ...), it would be better.

Best regards.

0
Comment actions Permalink

Alix,

At the moment, the log file format is not a part of TeamCity plugins API, so the code you are looking for may change without any notification (e.g., your workaround may stop working after next TeamCity upgrade).

In upcoming release, we are going to deliver a related feature: https://youtrack.jetbrains.com/issue/TW-20272

Also, you can configure Clean Up  with level "Artifacts" (https://confluence.jetbrains.com/display/TCD10/Clean-Up) and secify pattern ".teamcity/logs/buidLog.msg*" so that build logs are cleaned up properly. (since TeamCity 9.0)

1
Comment actions Permalink

Hi Nikita,

Many thanks for these informations and links.

Clean-Up is a good option / complement to my workaround ... and TC v10.1 is the ultime fix for this problem !.

What a great tool :-)
Best regards.

0

Please sign in to leave a comment.