Azure cloud agents - No agents connected after instance start

Answered

Hello,

I have installed and configured a VM on Azure with TeamCity Server and a VM image with a TeamCity Agent.

There appears to be an issue with (de)privisioning the agents.

 

It seems to work correctly on first run after saving the Agent Cloud Config:

  1. A new VM is provisioned as shown in the Agents / Cloud tab
  2. Queued builds are started and run
  3. The VM is deprovisioned and removed after 30 minutes

This is the point where it stops working. When going to the Agent / Cloud tab it shows a red exclamation mark with the following message:

No agents connected after instance start. Please check the image has TeamCity agent configured and it can connect to the server using http://10.x.x.x address. Start the instance manually to check for agent again.

If at this point I change 'Agent Cloud > Agent Images > Name Prefix' to some different value and save this everything start over from #1.

What could be going wrong?

Thanks!

6 comments

Hello Tijs,

What TeamCity version do you use? Please attach full teamcity-clouds.log file.

1

Thank you for the reply.

I am using TeamCity v9.1.7

Below is the clouds.log file contents from today. You can see the VM getting created and stopped after 30min so I'm at the point described above.

[2016-06-07 06:15:31,300] INFO [re'{id=arm-1} 2] - louds.base.AbstractCloudClient - Cloud profile 'profile 'Azure'{id=arm-1}' initialized
[2016-06-07 07:04:21,333] INFO [re'{id=arm-1} 1] - louds.base.AbstractCloudClient - Cloud profile 'profile 'Azure'{id=arm-1}' initialized
[2016-06-07 12:25:01,112] INFO [p-nio-80-exec-8] - .web.admin.UpdateProfileAction - Updating cloud profile: id=arm-1
[2016-06-07 12:25:01,221] INFO [p-nio-80-exec-8] - .server.impl.CloudEventsLogger - Profile reloaded: arm-1 and is enabled
[2016-06-07 12:25:04,579] INFO [re'{id=arm-1} 2] - louds.base.AbstractCloudClient - Cloud profile 'profile 'Azure'{id=arm-1}' initialized
[2016-06-07 12:25:05,015] INFO [4 Flush Queue 1] - .instances.StartInstanceAction - Starting cloud instance: profile 'Azure'{id=arm-1}, AzureCloudImage{myName='FooAgent'}, hash=z1QKwGyx1tKf30D3w9gyyunSnKvkWb4L, reason=Automatic start: Start instance to get instance agent details
[2016-06-07 12:25:05,026] INFO [4 Flush Queue 1] - uds.base.AbstractCloudInstance - Changing FooAgent1(1178e17) status from Unknown to Scheduled to start
[2016-06-07 12:25:05,560] INFO [4 Flush Queue 1] - .server.impl.CloudEventsLogger - Cloud instance entered 'scheduled to start' state, profile 'Azure'{id=arm-1}, AzureCloudInstance{myName='FooAgent1'}
[2016-06-07 12:25:05,560] INFO [4 Flush Queue 1] - .server.impl.CloudEventsLogger - Cloud instance start succeeded: profile 'Azure'{id=arm-1}, AzureCloudInstance{myName='FooAgent1'}
[2016-06-07 12:30:12,619] INFO [re'{id=arm-1} 1] - base.tasks.UpdateInstancesTask - Updated instance 'FooAgent1' status to Running based on API information
[2016-06-07 12:30:12,619] INFO [re'{id=arm-1} 1] - uds.base.AbstractCloudInstance - Changing FooAgent1(1178e17) status from Scheduled to start to Running
[2016-06-07 12:30:15,445] INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'running' state, profile 'Azure'{id=arm-1}, AzureCloudInstance{myName='FooAgent1'}
[2016-06-07 13:05:20,314] INFO [uled executor 2] - l.instances.StopInstanceAction - Terminating instance: AzureCloudInstance{myName='FooAgent1'}, profile 'Azure'{id=arm-1}, reason: idle timeout of 30 minutes reached
[2016-06-07 13:05:20,347] INFO [uled executor 2] - r.impl.DBCloudStateManagerImpl - Image: AzureCloudImage{myName='FooAgent'}, profile: profile 'Azure'{id=arm-1} was marked to NOT_CONTAIN agent
[2016-06-07 13:05:20,399] INFO [uled executor 2] - uds.base.AbstractCloudInstance - Changing FooAgent1(1178e17) status from Running to Scheduled to stop
[2016-06-07 13:05:24,077] INFO [uled executor 2] - .server.impl.CloudEventsLogger - Cloud instance entered 'scheduled to stop' state, profile 'Azure'{id=arm-1}, AzureCloudInstance{myName='FooAgent1'}
[2016-06-07 13:07:44,269] INFO [uled executor 2] - .server.impl.CloudEventsLogger - Cloud instance has gone (is not reported by cloud profile): profileId=arm-1, imageId=FooAgent, instanceId=FooAgent1

0

Can you please start one agent machine (I assume you do this from TC UI) and check the TCagent/logs/teamcity-agent.log?

If you look to the teamcity-agent.log, there's usually some log information that you can locate by word vmware. Please ensure that OS user TeamCity agent is running under has an access to C:\WindowsAzure\Config for Windows or /var/lib/waagent/ for Linux.

1

There is no mention of vmware in the logs nor any interesting warning or error messages. The agent does have access to WindowsAzure\Config. As stated before, builds are run just fine, it's the management of the agents after this that is failing. So the agent gets deleted after 30 minutes, but then I get the red exclamation mark and warning message and no new agents are created after..

0

So just to be clear:

- The first cloud agent is created as expected
- Builds are run as expected on that cloud agent
- The cloud agent is deleted after 30 minutes

After this point TC stops working as expected:
- The cloud tabs shows 0 agents, but an exclamation mark with an error message. See screenshot: http://imgur.com/A9vm98r
- I start manually start new agents using the Start button, but the same thing keeps happening

0

Please sign in to leave a comment.