Configurations with Large Build Logs
Hello,
I used the Server Health tool and it discovered that there were 2 monstrously large build logs (4.1GB and 13GB). I wasn't sure what the right way to fix this was but here's what I did:
I went directly to the log files in system/messages and did something like this: echo "Deleted build log because it was 13GB." > system/messages/CH94/88894.msg5
This reduced the excessive amount of disk space used. However, when I run the server health report, it still thinks the files are the same size and the log files now read, for example:
Attempt to read outside of build log file. It seems the build log "/data/teamCity/system/messages/CH94/88894.msg5" is broken. This can be caused by incorrect server shutdown. Please contact your system administrator.
I have 2 questions:
1. Is there now a way to make the log file display the plain text I added to it without displaying an error?
2. What is the right way to delete just one or two large build logs?
Thanks,
Dave
Please sign in to leave a comment.
Build log has a binary format, so you can't modify it easily. To get rid of these build logs you can either remove them from disk (in this case TeamCity will show message in UI that it can't find build logs), or remove the build itself (Actions -> Remove).
TeamCity is written in Java ... so perhaps a TeamCity lib/jar could be used to generate a buildLog.msg5 file with generic message (no contextualisation like exact size or date), used to replace the original file ?
If is there a source code or doc entry point, I'll take it ! :-)
Hello Alix,
Why do you need to replace the original .msg5 file?
Hi Alina,
In a global / mutualized unique CI platform for all projects in a company, when there are some bad practices, they could impacts the exploitation of the platform.
One case is the "too many log generation". If project generates more ~2-3Go by build (DEBUG/TRACE log level during unit tests ... so Spring/Hinernate/etc kilometer of logs), with builds history & build type (integration, release, ...) the result could be ~50Go for one project.
In this case:
1) The build log tab is unusable by the users
2) The TeamCity hard-disk could be full quickly (=> exploitation task)
So replacing this logs (ex: more one 1Go, old than 2 days) could be useful.
Easy to do with a Linux shell script searching this file (perhaps with TeamCity REST API, but a little bit more complicated).
I have done the job by generating a "generic" buildLog.msg5 & buildLog.msg5.i1 based on a Command Line step & echo the wanted message, used to replace these big original files in the previous shell script.
But this workaround is a "real build log" => The message is here but with a wrong date, build name, ...
If it could contain only one line, or generated dynamically (in Java) with contextualized message (original size, date, ...), it would be better.
Best regards.
Alix,
At the moment, the log file format is not a part of TeamCity plugins API, so the code you are looking for may change without any notification (e.g., your workaround may stop working after next TeamCity upgrade).
In upcoming release, we are going to deliver a related feature: https://youtrack.jetbrains.com/issue/TW-20272
Also, you can configure Clean Up with level "Artifacts" (https://confluence.jetbrains.com/display/TCD10/Clean-Up) and secify pattern ".teamcity/logs/buidLog.msg*" so that build logs are cleaned up properly. (since TeamCity 9.0)
Hi Nikita,
Many thanks for these informations and links.
Clean-Up is a good option / complement to my workaround ... and TC v10.1 is the ultime fix for this problem !.
What a great tool :-)
Best regards.