Running TeamCity in AWS EC2 without an public facing EIP - Working model

We have deployed the TeamCity CloudFormation template, however it builds the server with a public facing ElasticIP. We want to make our server accessible available only internal networks.

With this in mind I updated the CF template to work within our VPC, use our subnets, security groups, etc. and removed the EIP and replace it's reference in the startup Userdata with the private IP.

 

The above method does work, the template is outlined below

2 comments
Comment actions Permalink
Official comment

Hi, Chris

It is hard to determine what goes wrong without looking at actual template version.

Please share the content of your template version.

Since your initial request might be actual for other TeamCity users I've created issue to support this type of deployment out of the box

https://youtrack.jetbrains.com/issue/TW-53860

Feel free to watch / vote / comment it

Comment actions Permalink

Unformatted template below with private info removed.

 

AWSTemplateFormatVersion: 2010-09-09
Description: >
This template deploys a CoreOS EC2 instance with the official TeamCity server
of the specified version from DockerHub, one RDS database and one TeamCity
Build Agent.

Last Modified: 19 Nov 2017
Parameters:
KeyName:
Description: >-
The existing EC2 KeyPair for SSH access to the TeamCity Server EC2
instance
Type: 'AWS::EC2::KeyPair::KeyName'
AllowedPattern: .+
TeamCityVersion:
Description: TeamCity Version
Type: String
AllowedPattern: .+
Default: latest
ServerInstanceType:
Description: The instance type for the TeamCity Server EC2 instance
Type: String
Default: t2.medium
AllowedValues:
- t2.small
- t2.medium
- t2.large
- t2.xlarge
- t2.2xlarge
- m4.large
- m4.xlarge
- m4.2xlarge
- m4.4xlarge
- m4.10xlarge
- m4.16xlarge
- m3.medium
- m3.large
- m3.xlarge
- m3.2xlarge
- c4.large
- c4.xlarge
- c4.2xlarge
- c4.4xlarge
- c4.8xlarge
- c3.large
- c3.xlarge
- c3.2xlarge
- c3.4xlarge
- c3.8xlarge
- r4.large
- r4.xlarge
- r4.2xlarge
- r4.4xlarge
- r4.8xlarge
- r4.16xlarge
- r3.large
- r3.xlarge
- r3.2xlarge
- r3.4xlarge
- r3.8xlarge
DBInstanceType:
Description: The instance type for the RDS instance for TeamCity Database
Type: String
Default: db.t2.small
AllowedValues:
- db.t2.small
- db.t2.medium
- db.m4.large
- db.m4.xlarge
- db.m4.2xlarge
- db.m4.4xlarge
- db.m4.10xlarge
- db.m3.medium
- db.m3.large
- db.m3.xlarge
- db.m3.2xlarge
- db.r3.large
- db.r3.xlarge
- db.r3.2xlarge
- db.r3.4xlarge
- db.r3.8xlarge
DBPassword:
Description: Provide the password for the TeamCity RDS Database (min 8 chars)
Type: String
AllowedPattern: .+
MinLength: 8
NoEcho: true
Metadata:
'AWS::CloudFormation::Interface':
ParameterGroups:
- Label:
default: TeamCity Server
Parameters:
- ServerInstanceType
- KeyName
- TeamCityVersion
- Label:
default: TeamCity Database
Parameters:
- DBInstanceType
- DBPassword
ParameterLabels:
ServerInstanceType:
default: EC2 Instance Type
KeyName:
default: EC2 KeyPair
TeamCityVersion:
default: TeamCity Version
DBInstanceType:
default: RDS Database Instance Type
DBPassword:
default: TeamCity Database Password
Mappings:
AWSRegionToAMI:
eu-central-1:
AMI: ami-9501c8fa
ap-northeast-1:
AMI: ami-885f19ef
us-gov-west-1:
AMI: ami-12c67c73
ap-northeast-2:
AMI: ami-d65889b8
ca-central-1:
AMI: ami-c8c67bac
ap-south-1:
AMI: ami-7e641511
sa-east-1:
AMI: ami-3e5d3952
ap-southeast-2:
AMI: ami-d92422ba
ap-southeast-1:
AMI: ami-14cc7877
us-east-1:
AMI: ami-fd6c94eb
us-east-2:
AMI: ami-72032617
us-west-2:
AMI: ami-4c49f22c
us-west-1:
AMI: ami-b6bae7d6
eu-west-1:
AMI: ami-ac8fd4ca
eu-west-2:
AMI: ami-054c5961
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
ImageId: !FindInMap
- AWSRegionToAMI
- !Ref 'AWS::Region'
- AMI
KeyName: !Ref KeyName
InstanceType: !Ref ServerInstanceType
SecurityGroupIds:
- sg-12345
- sg-23456
- sg-34567
SubnetId: subnet-12345
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeSize: 10
VolumeType: gp2
- DeviceName: /dev/xvdg
Ebs:
VolumeSize: 10
VolumeType: gp2
DeleteOnTermination: false
Tags:
- Key: Name
Value: !Sub 'TeamCity ${AWS::StackName} Server'
- Key: Version
Value: !Sub '${TeamCityVersion}'
UserData:
'Fn::Base64': !Sub
- |
#cloud-config

write_files:
- path: "/etc/teamcity-version"
content: |
TEAMCITY_VERSION=${TeamCityVersion}
- path: "/etc/coreos/update.conf"
content: |
REBOOT_STRATEGY="off"

coreos:
units:
- name: "format-mnt-data.service"
enable: true
content: |
[Unit]
Requires=network-online.target
Before=teamcity-server.service mnt-data.mount
ConditionPathExists=!/dev/mapper/app-data

[Service]
Type=oneshot
ExecStart=/bin/bash -c \
'/usr/sbin/pvcreate /dev/xvdg && \
/usr/sbin/vgcreate app /dev/xvdg && \
/usr/sbin/lvcreate -l 100%FREE -n data app && \
/usr/sbin/mkfs.ext4 /dev/mapper/app-data'

[Install]
WantedBy=multi-user.target
- name: "mnt-data.mount"
enable: true
content: |
[Unit]
Before=teamcity-server.service
After=format-mnt-data.service
Requires=format-mnt-data.service
ConditionVirtualization=!container
Conflicts=umount.target

[Mount]
What=/dev/mapper/app-data
Where=/mnt/data
Type=ext4
Options=

[Install]
RequiredBy=teamcity-server.service
- name: "get-mysql-connector.service"
enable: true
content: |
[Unit]
Before=teamcity-server.service
After=mnt-data.mount
Requires=mnt-data.mount
ConditionPathExists=!/mnt/data/teamcity/lib/jdbc/mysql-connector-java-bin.jar

[Service]
Type=oneshot
ExecStart=/usr/bin/mkdir -p /mnt/data/teamcity/lib/jdbc
ExecStart=/usr/bin/wget -O /mnt/data/teamcity/lib/jdbc/mysql-connector-java-bin.jar http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.43/mysql-connector-java-5.1.43.jar

[Install]
WantedBy=multi-user.target
- name: "prepare-db-properties.service"
enable: true
content: |
[Unit]
Before=teamcity-server.service
After=mnt-data.mount
Requires=mnt-data.mount network-online.target
ConditionPathExists=!/mnt/data/teamcity/config/database.properties

[Service]
Type=oneshot
ExecStart=/usr/bin/mkdir -p /mnt/data/teamcity/config
ExecStart=/bin/bash -c 'echo connectionProperties.user=teamcity > /mnt/data/teamcity/config/database.properties'
ExecStart=/bin/bash -c 'echo connectionProperties.password=${RDSPassword} >> /mnt/data/teamcity/config/database.properties'
ExecStart=/bin/bash -c 'echo connectionUrl=jdbc\\:mysql\\://${RDSAddress}/teamcitydb >> /mnt/data/teamcity/config/database.properties'

[Install]
WantedBy=multi-user.target
- name: "teamcity-server.service"
command: "start"
content: |
[Unit]
Description=TeamCity Server
After=docker.service mnt-data.mount get-mysql-connector.service prepare-db-properties.service
Requires=docker.service mnt-data.mount get-mysql-connector.service prepare-db-properties.service
[Service]
EnvironmentFile=/etc/teamcity-version
TimeoutStartSec=1200s
ExecStartPre=/bin/sh -c "docker images --filter 'before=jetbrains/teamcity-server:${!TEAMCITY_VERSION}' --format '{{.ID}} {{.Repository}}' | grep 'jetbrains/teamcity-server' | grep -Eo '^[^ ]+' | xargs -r docker rmi"
ExecStart=/usr/bin/docker run \
-v /mnt/data/teamcity:/data/teamcity_server/datadir \
-v /mnt/data/logs/teamcity:/opt/teamcity/logs \
-v /mnt/data/temp/teamcity:/opt/teamcity/temp \
-p 80:8111 \
--name teamcity-server \
jetbrains/teamcity-server:${!TEAMCITY_VERSION}
ExecStop=-/usr/bin/docker exec teamcity-server /opt/teamcity/bin/teamcity-server.sh stop 60
ExecStopPost=-/usr/bin/docker stop teamcity-server
ExecStopPost=-/usr/bin/docker rm teamcity-server
Restart=always

[Install]
WantedBy=multi-user.target
- name: "teamcity-agent.service"
command: "start"
content: |
[Unit]
Description=TeamCity Agent
After=teamcity-server.service coreos-metadata.service
Requires=teamcity-server.service coreos-metadata.service

[Service]
EnvironmentFile=/etc/teamcity-version
TimeoutStartSec=1200s
EnvironmentFile=/run/metadata/coreos
ExecStartPre=/bin/sh -c "docker images --filter 'before=jetbrains/teamcity-agent:${!TEAMCITY_VERSION}' --format '{{.ID}} {{.Repository}}' | grep 'jetbrains/teamcity-agent' | grep -Eo '^[^ ]+' | xargs -r docker rmi"
ExecStart=/usr/bin/docker run \
-v /mnt/data/temp/teamcity:/opt/teamcity/temp \
-e SERVER_URL=${!COREOS_EC2_IPV4_LOCAL} \
-e AGENT_NAME=Default \
--name teamcity-agent \
jetbrains/teamcity-agent:${!TEAMCITY_VERSION}
ExecStop=-/usr/bin/docker stop teamcity-agent
ExecStopPost=-/usr/bin/docker rm teamcity-agent
Restart=always

[Install]
WantedBy=multi-user.target
- RDSAddress: !GetAtt
- RDSDB
- Endpoint.Address
RDSPassword: !Ref DBPassword
TeamCityVersion: !Ref TeamCityVersion
RDSDB:
Type: 'AWS::RDS::DBInstance'
Properties:
AllocatedStorage: '5'
DBInstanceClass: !Ref DBInstanceType
DBInstanceIdentifier: !Sub 'TeamCity-${AWS::StackName}'
DBName: teamcitydb
DBSubnetGroupName: subnetgroupname
VPCSecurityGroups:
- sg-45678
Engine: MySQL
EngineVersion: 5.5.54
MasterUsername: teamcity
MasterUserPassword: !Ref DBPassword
DBParameterGroupName: !Ref RDSParamGroup
PubliclyAccessible: false
DeletionPolicy: Snapshot
RDSParamGroup:
Type: 'AWS::RDS::DBParameterGroup'
Properties:
Family: MySQL5.5
Description: !Sub 'TeamCity ${AWS::StackName} DB'
Parameters:
long_query_time: '5'
slow_query_log: '1'
innodb_flush_log_at_trx_commit: '2'
character_set_server: utf8
collation_server: utf8_bin
Outputs:
TeamCityServerUrl:
Description: TeamCity Server endpoint
Value: !Join
- ''
- - 'http://'
- !GetAtt EC2Instance.PrivateIp
- /

0

Please sign in to leave a comment.