Opening build details is sometimes slow

Sometimes it takes couple of seconds to open a build details (the URI is e.g. /viewLog.html?buildId=6709&tab=buildResultsDiv&buildTypeId=bt8). It's not consistent, the median load time is about 3 seconds, but sometimes it takes much longer.

I am using TeamCity 9.0.1 (build 32116). The database is running on PostgreSQL 9.3.

In those moments where it takes really long time to load the page, this is the only query TeamCity is issuing to the database:

select I.build_id, I.test_name_id, I.status
from test_info I
join build$ B
  on I.build_id = B.build_id

Is this kind of performance expected?

Best regards,
Stanislav Kurik

5 comments
Comment actions Permalink

Hi Stanislav,

Could you please attach serveral server thread dumps collected while the issue is reproduced?

0
Comment actions Permalink

Dear Alina,

a while ago it happened again and after a few seconds I took a few thread dumps. The server generated some on its own, so I included them as well.

Best regards,
Stanislav



Attachment(s):
TeamCityThreadDumps.zip
0
Comment actions Permalink

So far looks like a slow down in database. Please try applying our suggested settings for PostgreSQL:
https://confluence.jetbrains.com/display/TCD9/How+To...#HowTo...-ConfigureNewlyInstalledPostgreSQLServer

0
Comment actions Permalink

Thanks, I modified the settings to match those you linked. In fact, the only difference was that my shared_buffers were 8192 MB. But even after changing it to 512 MB opening a build detail is very slow. I installed the pg_stat_statements extension and after opening a build detail couple of times, this is how it looks like:

                                                 query                                                 |    per_query     | calls | rows  
-------------------------------------------------------------------------------------------------------+------------------+-------+-------
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id |        4299.6744 |    10 |  5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4158.84751724138 |    29 | 15549
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id |        4144.5404 |    10 |  5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id |        4143.7066 |    10 |  5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 4075.15206666667 |    90 | 44152
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id |         4064.418 |    10 |  5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id |       4048.97025 |    20 | 10888
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id |         3991.276 |    10 |  5444
select I.build_id, I.test_name_id, I.status from test_info I join build$ B on I.build_id = B.build_id | 3900.44356028369 |   141 | 72601
select * from pg_stat_statements;                                                                     | 63.4373333333333 |     3 | 29805
(10 rows)

per_query specifies how long, on average, the query took to complete. calls is the number of times it has been called and rows is the total number of rows it returned (over all calls) (see http://www.postgresql.org/docs/current/static/pgstatstatements.html for details).

I use the following query to generate this dump:

select query, (total_time / calls) as per_query, calls, rows from public.pg_stat_statements order by (total_time / calls) desc limit 10

0
Comment actions Permalink

If you had much bigger shared_buffers then it does not make sense to make it smaller. It looks like you experience this problem: https://youtrack.jetbrains.com/issue/TW-39886

Please try installing 9.0.3 eap build from this ftp:
ftp://ftp.intellij.net/pub/.teamcity/nightly/9.0.x

Build does not change data format, so in case of any problems you can revert to 9.0.2 without necessity to restore from backup. Let us know whether it helps.

0

Please sign in to leave a comment.