Agents randomly disconnects

Running TeamCity Enterprise 2017.1.3 (build 46961) on a machine, with 3 agents distributed over 2 separate machines. We did a upgrade from 10.0.1 to 2017.1.3 yesterday (missed the 2017.1.4 version with an hour)

The main machine (machine1) runs the TeamCity server and 2 agents (agent1 and agent2).

One agent (agent3) runs on another machine (machine2) and points to the machine1 via serverUrl (conf. file below).

All agents (on both machines) and TeamCity server run under the Local System account.

After the upgrade when the agents connected and upgraded, they seem to Disconnect randomly and frequently.

During a 10 minute period, 2 out of 3 agents disconnected 4-5 times.

The inactivity reason is Cannot access agent

When the agent is disconnected, the Communication Protocol goes from unidirectional to Communication protocol: unknown. (it is pretty logical that the protocol is unknown when the agent cannot be accessed, but just wanted to point that out).

Main configuration:
Agent1 on machine1 configuration:
name=Machine1_Agent1
ownPort=9090
serverUrl=http\://localhost\:6060
workDir=E\:\\TeamCity\\buildAgent\\work
tempDir=E\:\\TeamCity\\buildAgent\\temp
env.TEAMCITY_JRE=C\:\\Program Files (x86)\\TeamCity\\buildAgent\\BuildAgent\\jre

Agent2 on machine1 configuration:
name=Machine1_agent2
ownPort=9091
serverUrl=http\://localhost\:6060
workDir=E\:\\TeamCity\\buildAgent2\\work
tempDir=E\:\\TeamCity\\buildAgent2\\temp
env.TEAMCITY_JRE=C\:\\Program Files (x86)\\TeamCity\\buildAgent2\\BuildAgent\\jre

Agent3 on machine2 configuration:
name=Machine2_agent3
ownPort=9090
## http\://192.168.200.4:6060 is machine1 TeamCity instance
serverUrl=http\://192.168.200.4:6060
workDir=E\:\\Teamcity\\BuildAgent\\work
tempDir=E\:\\Teamcity\\BuildAgent\\temp
systemDir=E\:\\Teamcity\\BuildAgent\\system
env.TEAMCITY_JRE=E\:\\Teamcity\\BuildAgent\\jre

2 agents using 9090, but they are on different machines, so that shouldn't be a problem.

It seems to me that the agents are fully functional, as in when they get assigned and start to build, even when they disconnect, they still successfully runs the build when it connects again. I have stopped one build that hanged, but the reason of the hang was unknown.

I do not really know which logs are interesting. I have checked all the files under buildAgent\logs, but cannot really find anything that could explain the disconnects.