High-Availability setup: intra-node communication requirements

Possibly related to https://teamcity-support.jetbrains.com/hc/en-us/community/posts/360010004360-Secondary-server-URLs-and-other-challenges but that's quite an old post.

I am in the process of creating a multinode HA TeamCity setup:

  • 1 MAIN_NODE and 2 secondaries. 
  • The three nodes share 1DB and 1 DATA_DIRECTORY
  • They all listen on http://localhost
  • I have a reverse proxy sitting in front of the MAIN_NODE that listens on https://teamcity.example.com but no failover configured just yet, all requests go to the MAIN_NODE.

According to https://www.jetbrains.com/help/teamcity/2023.11/multinode-setup.html and https://resources.jetbrains.com/help/img/teamcity/2023.11/multinode-setup-2022.png intra-node communication also happens via http(s) .

  1. What kind of traffic is that? Ie. if not set correctly, what goes wrong? 
  2. Can I reuse https://teamcity.example.com for teamcity.server.rootURL ? In screenshot [1] I see no indication of failing communication?
  3. Is teamcity.server.rootURL the same as `<server rootURL` as mentioned in https://www.jetbrains.com/help/teamcity/configuring-server-url.html
  4. Will setting teamcity.server.rootURL influence the URL's that TeamCity itself generates (eg. commit status publisher backlink, etc…)

https://www.jetbrains.com/help/teamcity/2023.11/multinode-setup.html#Automated+Addition+of+a+Secondary+Node

<node_URL> is the secondary node root URL, which should be accessible from the main node and agents.

This conflicts with

This URL is required for inter-node communication and a firewall should be configured to allow connections to this URL from one node to another.

 

5. Should it be accessible from the agents? 

 

[1]:

0
2 comments
Hi Jan,

> 1. What kind of traffic is that? Ie. if not set correctly, what goes wrong?
It can be used to route the requests from the main node in case secondary nodes have additional responsibilities assigned: https://www.jetbrains.com/help/teamcity/2023.11/multinode-setup.html#Responsibilities.
If not configured correctly, the requests redirect between nodes (for example, requests from agents), etc. won't work. The nodes are also checking the heartbeat of each other to know if one of the nodes is down, so if not configured correctly, it also may not work.

> 2. Can I reuse https://teamcity.example.com for teamcity.server.rootURL ? In screenshot [1] I see no indication of failing communication?
Each node should have its own URL. This URL is required for inter-node communication and a firewall should be configured to allow connections to this URL from one node to another.

> 3. Is teamcity.server.rootURL the same as `The URL set in the "server rootURL" property (or in the Administration UI) is a URL to which the external requests come to. The teamcity.server.rootURL is a node URL for inter-node communication. 
The UI (server rootURL) setting should have a proxy URL, as all the external requests come to the proxy first. 

> 4. Will setting teamcity.server.rootURL influence the URL's that TeamCity itself generates (eg. commit status publisher backlink, etc…)
No, the URL that is set up in the UI (server rootURL) is used.

> 5. Should it be accessible from the agents?
No, this requirement was lifted in https://youtrack.jetbrains.com/issue/TW-66859. The documentation seems to be out of date in this one sentence. Thank you for pointing it out, we will fix it.

Best regards,
Anton
1

Thank you for the clarification Anton

0

Please sign in to leave a comment.