Opening build details is sometimes slow Follow
Sometimes it takes couple of seconds to open a build details (the URI is e.g. /viewLog.html?buildId=6709&tab=buildResultsDiv&buildTypeId=bt8). It's not consistent, the median load time is about 3 seconds, but sometimes it takes much longer.
I am using TeamCity 9.0.1 (build 32116). The database is running on PostgreSQL 9.3.
In those moments where it takes really long time to load the page, this is the only query TeamCity is issuing to the database:
select I.build_id, I.test_name_id, I.status
from test_info I
join build$ B
on I.build_id = B.build_id
Is this kind of performance expected?
Best regards,
Stanislav Kurik
Please sign in to leave a comment.
Hi Stanislav,
Could you please attach serveral server thread dumps collected while the issue is reproduced?
Dear Alina,
a while ago it happened again and after a few seconds I took a few thread dumps. The server generated some on its own, so I included them as well.
Best regards,
Stanislav
Attachment(s):
TeamCityThreadDumps.zip
So far looks like a slow down in database. Please try applying our suggested settings for PostgreSQL:
https://confluence.jetbrains.com/display/TCD9/How+To...#HowTo...-ConfigureNewlyInstalledPostgreSQLServer
Thanks, I modified the settings to match those you linked. In fact, the only difference was that my shared_buffers were 8192 MB. But even after changing it to 512 MB opening a build detail is very slow. I installed the pg_stat_statements extension and after opening a build detail couple of times, this is how it looks like:
query | per_query | calls | rows
-------------------------------------------------------------------------------------------------------+------------------+-------+-------
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4299.6744 | 10 | 5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4158.84751724138 | 29 | 15549
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4144.5404 | 10 | 5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4143.7066 | 10 | 5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4075.15206666667 | 90 | 44152
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4064.418 | 10 | 5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4048.97025 | 20 | 10888
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 3991.276 | 10 | 5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 3900.44356028369 | 141 | 72601
select * from pg_stat_statements; | 63.4373333333333 | 3 | 29805
(10 rows)
per_query specifies how long, on average, the query took to complete. calls is the number of times it has been called and rows is the total number of rows it returned (over all calls) (see http://www.postgresql.org/docs/current/static/pgstatstatements.html for details).
I use the following query to generate this dump:
select query, (total_time / calls) as per_query, calls, rows from public.pg_stat_statements order by (total_time / calls) desc limit 10
If you had much bigger shared_buffers then it does not make sense to make it smaller. It looks like you experience this problem: https://youtrack.jetbrains.com/issue/TW-39886
Please try installing 9.0.3 eap build from this ftp:
ftp://ftp.intellij.net/pub/.teamcity/nightly/9.0.x
Build does not change data format, so in case of any problems you can revert to 9.0.2 without necessity to restore from backup. Let us know whether it helps.