tag:blogger.com,1999:blog-92384052024-03-18T02:04:50.468-07:00Agile TestingDid anybody say webscale?Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.comBlogger561125tag:blogger.com,1999:blog-9238405.post-70621100813829983772018-01-11T16:00:00.000-08:002018-01-11T16:00:01.036-08:00Modifying EC2 security groups via AWS Lambda functionsOne task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructure. This can be automated either fully or partially with the help of simple AWS Lambda functions.<br /><h4>
Example 1: Checking a Dynamic DNS IP and replacing it in an EC2 security group</h4>
This scenario arises when you have a user without a static IP. They can still get a Dynamic DNS name and have it automatically point to their local dynamic IP. You can check for that name periodically, and update the appropriate rules within EC2 security group(s).<br /><br />Here is an AWS Lambda function named UpdateSecurityGroupWithHomeIP and written in Python 2.7 that achieves this goal:<br /><br /><div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import boto3</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import hashlib</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import json</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import copy</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import urllib2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># ID of the security group we want to update</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SECURITY_GROUP_ID = "sg-XXXX"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># Description of the security rule we want to replace</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SECURITY_RULE_DESCR = "My Home IP"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def lambda_handler(event, context):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> new_ip_address = list(event.values())[0]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result = update_security_group(new_ip_address)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return result</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def update_security_group(new_ip_address):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> client = boto3.client('ec2')</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> response = client.describe_security_groups(GroupIds=[SECURITY_GROUP_ID])</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> group = response['SecurityGroups'][0]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for permission in group['IpPermissions']:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> new_permission = copy.deepcopy(permission)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ip_ranges = new_permission['IpRanges']</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for ip_range in ip_ranges:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if ip_range['Description'] == 'My Home IP':</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ip_range['CidrIp'] = "%s/32" % new_ip_address</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> client.revoke_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[permission])</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> client.authorize_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[new_permission])</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return ""</span></div>
<br /><br />A few observations:<br /><ul>
<li>it’s not trivial to do DNS lookups within Lambda, so I preferred to do the DNS lookup in the caller, and pass the resulting IP address as the sole argument to the above Lambda function — which is retrieved as <span style="font-family: Courier New, Courier, monospace;">new_ip_address</span> in the <span style="font-family: Courier New, Courier, monospace;">lambda_handler</span> function</li>
<li>in the <span style="font-family: Courier New, Courier, monospace;">update_security_group</span> function I iterate through all permission objects in the <span style="font-family: Courier New, Courier, monospace;">IpPermissions</span> list associated to the given security group and I create a deep copy of every permission</li>
<li>if any IP range in a permission object has the description “My Home IP”, I change its <span style="font-family: Courier New, Courier, monospace;">CidrIp</span> property to the CIDR block corresponding to <span style="font-family: Courier New, Courier, monospace;">new_ip_address</span></li>
<li>finally, I revoke the old permission and authorize the new (deep-copied) permission</li>
</ul>
<br />This Lambda function needs the proper permissions to modify security groups in EC2. I associated it with an IAM role which allows that. Here is the policy associated with that role:<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">{</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Version": "2012-10-17",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Statement": [</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Effect": "Allow",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Action": [</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "logs:CreateLogGroup",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "logs:CreateLogStream",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "logs:PutLogEvents"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ],</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Resource": "arn:aws:logs:*:*:*"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> },</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Effect": "Allow",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Action": [</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "ec2:DescribeSecurityGroups",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "ec2:AuthorizeSecurityGroupIngress",</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "ec2:RevokeSecurityGroupIngress"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ],</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> "Resource": "*"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">}</span></div>
<br /><br />I call this Lambda function from a Jenkins job set to run periodically which does the DNS lookup first, then calls the above AWS Lambda function:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">IPADDRESS=`dig my.homeip.example.com | grep IN | grep -v ';' | awk '{print $5}'`<br />aws lambda invoke \<br />--invocation-type RequestResponse \<br />--function-name UpdateSecurityGroupWithHomeIP \<br />--region us-west-2 \<br />--log-type Tail \<br />--payload "{\"ip\":\"$IPADDRESS\"}" \<br />outputfile.txt</span><br /><br /><h4>
Example 2: adding a new IP/CIDR block to a several security groups</h4>
<br />This is useful when you have several security groups and you need to add a new source CIDR block to all of them.<br /><br />Here is a Lambda function for this purpose:<br /><br /><div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import boto3</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import hashlib</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import json</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import urllib2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># Ports your application uses that need inbound permissions from the service for</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">INGRESS_PORTS = { </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> 'web' : [80, 443], </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> 'ssh': [22,] </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">}</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># Tags which identify the security groups you want to update</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SECURITY_GROUP_TAG_FOR_WEB = { 'LambdaUpdate': 'web'}</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SECURITY_GROUP_TAG_FOR_SSH = { 'LambdaUpdate': 'ssh'}</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def lambda_handler(event, context):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> cidr_blocks = list(event.values())</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result = update_security_groups(cidr_blocks)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return result</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def update_security_groups(cidr_blocks):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> client = boto3.client('ec2')</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> web_group = get_security_groups_for_update(client, SECURITY_GROUP_TAG_FOR_WEB)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ssh_group = get_security_groups_for_update(client, SECURITY_GROUP_TAG_FOR_SSH)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print ('Found ' + str(len(web_group)) + ' WebSecurityGroups to update')</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print ('Found ' + str(len(ssh_group)) + ' SshSecurityGroups to update')</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result = list()</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> web_updated = 0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ssh_updated = 0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for group in web_group:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for port in INGRESS_PORTS['web']:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if update_security_group(client, group, cidr_blocks, port):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> web_updated += 1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result.append('Updated ' + group['GroupId'])</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for group in ssh_group:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for port in INGRESS_PORTS['ssh']:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if update_security_group(client, group, cidr_blocks, port):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ssh_updated += 1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result.append('Updated ' + group['GroupId'])</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result.append('Updated ' + str(web_updated) + ' of ' + str(len(web_group)) + ' WebSecurityGroups')</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> result.append('Updated ' + str(ssh_updated) + ' of ' + str(len(ssh_group)) + ' SshSecurityGroups')</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return result</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def update_security_group(client, group, cidr_blocks, port):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> added = 0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if len(group['IpPermissions']) > 0:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for permission in group['IpPermissions']:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if permission['FromPort'] <= port and permission['ToPort'] >= port:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> old_prefixes = list()</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> to_add = list()</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for cidr_block in cidr_blocks:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if old_prefixes.count(cidr_block) == 0:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> to_add.append({ 'CidrIp': cidr_block })</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print(group['GroupId'] + ": Adding " + cidr_block + ":" + str(permission['ToPort']))</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> added += add_permissions(client, group, permission, to_add)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> else:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> to_add = list()</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for cidr_block in cidr_blocks:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> to_add.append({ 'CidrIp': cidr_block })</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print(group['GroupId'] + ": Adding " + cidr_block + ":" + str(port))</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> permission = { 'ToPort': port, 'FromPort': port, 'IpProtocol': 'tcp'}</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> added += add_permissions(client, group, permission, to_add)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> print (group['GroupId'] + ": Added " + str(added))</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return (added > 0)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def add_permissions(client, group, permission, to_add):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if len(to_add) > 0:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> add_params = {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> 'ToPort': permission['ToPort'],</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> 'FromPort': permission['FromPort'],</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> 'IpRanges': to_add,</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> 'IpProtocol': permission['IpProtocol']</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> client.authorize_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[add_params])</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return len(to_add)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">def get_security_groups_for_update(client, security_group_tag):</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> filters = list();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> for key, value in security_group_tag.iteritems():</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> filters.extend(</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> [</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> { 'Name': "tag-key", 'Values': [ key ] },</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> { 'Name': "tag-value", 'Values': [ value ] }</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> )</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> response = client.describe_security_groups(Filters=filters)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> return response['SecurityGroups']</span></div>
<br /><br />This function acts on security groups tagged “web” and “ssh”. For the ones tagged “web”, it adds new rules allowing the IP/CIDR block access to ports 80 and 443. For the groups tagged “ssh”, it does the same but for port 22.<br /><br />The input for this function is <span style="font-family: Courier New, Courier, monospace;">{“ip1”: “$IPAddressBlock”}</span> where <span style="font-family: Courier New, Courier, monospace;">IPAddressBlock</span> is a Jenkins parameter that the user specifies when running the appropriate Jenkins job. In this case, I used the AWS Lambda Invocation build step in Jenkins.</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-11391414935560148782017-12-29T10:11:00.000-08:002017-12-29T10:22:55.098-08:00Experiences with the Kong API Gateway<a href="https://getkong.org/">Kong</a> is an open-source API Gateway that you can install on-premise if you don't want or cannot use the AWS API Gateway. The benefits of putting an API Gateway in front of your actual API endpoints are many: rate limiting, authentication, security, better logging, etc. Kong offers all of these and more via <a href="https://getkong.org/plugins/">plugins</a>. I've experimented with Kong a bit and I am writing down my notes here for future reference.<br />
<h3>
Installing Kong on Ubuntu 16.04</h3>
<h4>
Install Kong from deb package</h4>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># wget https://bintray.com/kong/kong-community-edition-deb/download_file?file_path=dists/kong-community-edition-0.11.2.xenial.all.deb</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># dpkg -i kong-community-edition-0.11.2.xenial.all.deb</span></div>
</div>
<h4>
Install PostgreSQL </h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># apt-get install postgresql</span></span></div>
<h4>
Create kong database and user</h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># su - postgres</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">$ psql</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">psql (10.1)</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Type "help" for help.</span></span></div>
<b id="docs-internal-guid-4d77cd74-a309-ba53-dcfc-79a36c9154a5" style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">postgres=# CREATE USER kong; CREATE DATABASE kong OWNER kong;</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">postgres=# ALTER USER kong PASSWORD 'somepassword';</span></span></div>
<h4>
Modify kong configuration</h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># cp /etc/kong/kong.conf.default /etc/kong/kong.conf</span></span></div>
<div>
<br /></div>
Edit kong.conf and set:<b style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">pg_host = 127.0.0.1 </span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">pg_port = 5432 </span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">pg_user = kong </span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">pg_password = somepassword</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">pg_database = kong </span></span><span style="background-color: transparent; color: black; font-family: "courier new"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre;"> </span></div>
<h4>
Run kong migrations job</h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># kong migrations up</span></span></div>
<h4>
Increase open files limit</h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># ulimit -n 4096</span></span></div>
<h4>
Start kong</h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># kong start</span></span></div>
<h4>
Install kong dashboard</h4>
Install updated version of node first.<br />
<div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># </span><span style="background-color: transparent; color: #3a3a3a; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">curl -sL https://deb.nodesource.com/setup_</span><span style="background-color: transparent; color: #e94849; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">6.x</span><span style="background-color: transparent; color: #3a3a3a; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> -o nodesource_setup.sh</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: #3a3a3a; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># bash nodesource_setup.sh</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># apt-get install nodejs</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># npm install -g kong-dashboard</span></span></div>
<h4>
Create systemd service to start kong-dashboard with basic auth</h4>
I specified port 8001 for kong-dashboard, instead of the default port 8080.<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># cat /etc/systemd/system/multi-user.target.wants/kong-dashboard.service</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">[Unit]</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Description=kong dashboard</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">After=network.target</span></span></div>
<b style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">[Service]</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">ExecStart=/usr/local/bin/kong-dashboard start --kong-url http://localhost:8001 --basic-auth someadminuser=someadminpassword</span></span></div>
<b style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">[Install]</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">WantedBy=multi-user.target</span></span></div>
<b style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># systemctl daemon-reload</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"># systemctl start kong-dashboard</span></span></div>
<h3>
Adding an API to Kong</h3>
I used the Kong admin dashboard to add a new API Gateway object which points to the actual API endpoint I have. Note that the URL name as far as Kong is concerned can be anything you want. In this example I chose /<span style="font-family: "courier new" , "courier" , monospace;">kong1</span>. The upstream URL is your actual API endpoint. The Hosts value is a comma-separated list of the host headers you want Kong to reply to. Here is contains the domain name for my actual API endpoint (<span style="font-family: "courier new" , "courier" , monospace;">stage.mydomain.com</span>) as well as a new domain name that I will use later in conjunction with a load balancer in front of Kong (<span style="font-family: "courier new" , "courier" , monospace;">stage-api.mydomain.com</span>).</div>
<div>
<br />
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "times" , "times new roman" , serif; font-size: 11pt;">Name: </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">my-kong-api</span></span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "times" , "times new roman" , serif; font-size: 11pt;">Hosts: </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">stage.mydomain.com, stage-api.mydomain.com</span></span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "times" , "times new roman" , serif; font-size: 11pt;">URLs: </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/kong1</span></span></div>
</li>
<li dir="ltr" style="font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "times" , "times new roman" , serif; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Upstream URL:</span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> https://stage.mydomain.com/api/graphql/pub</span></div>
</li>
</ul>
<br />
At this point you can query the Kong admin endpoint for the existing APIs. We’ll use <a href="https://github.com/jakubroztocil/httpie">HTTPie</a> (on a Mac you can install HTTPie via <span style="font-family: "courier new" , "courier" , monospace;">brew install httpie</span>).<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">$ </span></b></span><span style="font-family: "courier new";"><span style="white-space: pre-wrap;"><b><span style="font-size: x-small;">http http://stage.mydomain.com:8001/apis</span></b><span style="font-size: 14.6667px;">
</span></span><span style="white-space: pre-wrap;"><span style="font-size: x-small;">HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Fri, 29 Dec 2017 16:23:59 GMT
Server: kong/0.11.2
Transfer-Encoding: chunked
{
"data": [
{
"created_at": 1513401906560,
"hosts": [
"stage-api.mydomain.com",
"stage.mydomain.com"
],
"http_if_terminated": true,
"https_only": true,
"id": "ad23a91a-b76a-417d-9889-0088a13e3419",
"name": "my-kong-api",
"preserve_host": true,
"retries": 5,
"strip_uri": true,
"upstream_connect_timeout": 60000,
"upstream_read_timeout": 60000,
"upstream_send_timeout": 60000,
"upstream_url": "https://stage.mydomain.com/api/graphql/pub",
"uris": [
"/kong1"
]
}
],
"total": 1
}</span></span></span></div>
<div>
<br /></div>
Note that any operation that you do via the Kong dashboard can also be done via API calls to the Kong admin backend. For lots of examples on how to do that, see the <a href="https://getkong.org/docs/0.11.x/getting-started/adding-your-api/">official documentation</a> as well as this <a href="https://dev9.com/blog-posts/2016/6/introduction-to-kong-api-gateway">blog post from John Nikolai</a>.<br />
<h3>
Using the Kong rate-limiting plugin</h3>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
In the Kong admin dashboard, go to Plugins -> Add:</div>
<ul>
<li>Plugin name: <b>rate-limiting</b></li>
<li>Config: minute = 10</li>
</ul>
This means that we are limiting the rate of calls to our API to 10 per minute. You can also specify limits per other units of time. See the <a href="https://getkong.org/plugins/rate-limiting/">rate-limiting plugin documentation</a> for more details.<br />
<h4>
Verifying that our API endpoint is rate-limited</h4>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre;">Now we can verify that the API endpoint is working by issuing a POST request to the Kong URL (</span><span style="background-color: transparent; color: black; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">/kong1</span></span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre;">) which points to our actual API endpoint:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-size: x-small;"><span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ http --verify=no -v POST "</span><span style="background-color: transparent; color: #1155cc; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">https://stage.mydomain.com:8443/kong1?query=version</span><span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"</span></span></b></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">POST /kong1?query=version HTTP/1.1</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Accept: */*</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Accept-Encoding: gzip, deflate</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Connection: keep-alive</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Content-Length: 0</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Host: stage.mydomain.com:8443</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">User-Agent: HTTPie/0.9.9</span></span></div>
<b style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">HTTP/1.1 200 OK</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Connection: keep-alive</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Content-Length: 43</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Content-Type: application/json</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Date: Sat, 16 Dec 2017 18:16:53 GMT</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Server: nginx/1.10.3 (Ubuntu)</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Via: kong/0.11.2</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">X-Kong-Proxy-Latency: 86</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">X-Kong-Upstream-Latency: 33</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">X-RateLimit-Limit-minute: 10</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">X-RateLimit-Remaining-minute: 9</span></span></div>
<b style="font-weight: normal;"><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">{</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> "data": {</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> "version": "21633901644387004745"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> }</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">}</span></span></div>
<b style="font-weight: normal;"><br /></b>
<b style="font-weight: normal;">A few things to notice in the call above:</b><br />
<br />
<ul>
<li>we are making an SSL call; by default Kong exposes SSL on port 8443</li>
<li>we are passing a query string to the <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/kong1</span> URL endpoint; Kong will pass this along to our actual API</li>
<li>the HTTP reply contains 2 headers related to rate limiting:</li>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">X-RateLimit-Limit-minute: 10</span> (this specifies the rate we set for the rate-limiting plugin)</li>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">X-RateLimit-Remaining-minute: 9</span> (this specifies the remaining calls we have)</li>
</ul>
</ul>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre;">After 9 requests in quick succession, the rate-limiting headers are:</span></div>
<b><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">X-RateLimit-Limit-minute: 10</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">X-RateLimit-Remaining-minute: 1</span></b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre;">After 10 requests:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">X-RateLimit-Limit-minute: 10</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">X-RateLimit-Remaining-minute: 0</span></b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre;">After the 11th request we get an HTTP 429 error:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">HTTP/1.1 429</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Connection: keep-alive</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Content-Type: application/json; charset=utf-8</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Date: Sat, 16 Dec 2017 18:21:19 GMT</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Server: kong/0.11.2</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Transfer-Encoding: chunked</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">X-RateLimit-Limit-minute: 10</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">X-RateLimit-Remaining-minute: 0</span></b></span></div>
<b><span style="font-size: x-small;"><br /></span></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">{</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;"> "message": "API rate limit exceeded"</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">}</span></b></span></div>
<h3>
Putting a load balancer in front of Kong</h3>
<b style="font-weight: normal;">Calling Kong APIs on non-standard port numbers gets cumbersome. To make things cleaner, I put an AWS ALB in front of Kong. I added a proper SSL certificate via AWS Certificate Manager for the domain <span style="font-family: "courier new" , "courier" , monospace;">stage-api.mydomain.com</span> and associated it to a listener on port 443 of the ALB. I also created two Target Groups, one for HTTP traffic to port 80 on the ALB and one for HTTPS </b><br />
<b style="font-weight: normal;">traffic to port 443 on the ALB. The first Target Group points to port 8000 on the server running Kong, because that's the port Kong uses for plain HTTP requests. The second Target Group points to port 8443 on the server running Kong, for HTTPS requests.</b><br />
<b style="font-weight: normal;"><br /></b>
I was now able to make calls such as these:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ http -v POST "https://stage-api.mydomain.com/kong1?query=version"</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">POST /kong1?query=version HTTP/1.1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Accept: */*</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Accept-Encoding: gzip, deflate</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Connection: keep-alive</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Content-Length: 0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Host: stage-api.mydomain.com</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">User-Agent: HTTPie/0.9.9</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">HTTP/1.1 200 OK</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Connection: keep-alive</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Content-Length: 42</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Content-Type: application/json</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Date: Fri, 29 Dec 2017 17:42:14 GMT</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Server: nginx/1.10.3 (Ubuntu)</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Via: kong/0.11.2</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">X-Kong-Proxy-Latency: 94</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">X-Kong-Upstream-Latency: 29</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">X-RateLimit-Limit-minute: 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">X-RateLimit-Remaining-minute: 9</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">{</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "data": {</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> "version": "</span><span style="font-family: "courier new"; font-size: x-small; white-space: pre-wrap;">21633901644387004745</span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">}</span><br />
<h3>
Using the Kong syslog plugin</h3>
I added the <a href="https://getkong.org/plugins/syslog/">syslog plugin</a> via the Kong admin dashboard. I set the following values:<br />
<br />
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">server_errors_severity: err</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">successful_severity: notice</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">client_errors_severity: err</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">log_level: notice</span></li>
</ul>
<div>
I then created a file called <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">kong.conf</span> in <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/etc/rsyslog.d</span> on the server running Kong:</div>
<div>
<br /></div>
<br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># cat kong.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">if ($programname == 'kong' and $syslogseverity-text == 'notice') then -/var/log/kong.log</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">& ~</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># service rsyslog restart</span></div>
<div>
<br /></div>
<div>
Now every call to Kong is logged in syslog format in <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/var/log/kong.log</span>. The message portion of the log entry is in JSON format.</div>
<h4>
Sending Kong logs to AWS ElasticSearch/Kibana</h4>
<div>
I used the process I described in my <a href="http://agiletesting.blogspot.com/2017/11/using-aws-cloudwatch-logs-and-aws.html">previous blog post on AWS CloudWatch Logs and AWS ElasticSearch</a>. One difference was that for the Kong logs I had to use a new index in ElasticSearch. </div>
<div>
<br /></div>
<div>
This was because the JSON object logged by Kong contains a field called <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">response</span>, which was clashing with another field called <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">response</span> already present in the <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">cwl-*</span> index in Kibana. What I ended up doing was copying the Lambda function used to send CloudWatch logs to ElasticSearch and replacing <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">cwl-</span> with <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">cwl-kong-</span> in the <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">transform</span> function. This created a new index <span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">cwl-kong-*</span> in ElasticSearch, and at that point I was able to add that index to a Kibana dashboard and visualize and query the Kong logs.</div>
<div>
<br /></div>
<div>
This is just scratching the surface of what you can do with Kong. Here are a few more resources:</div>
<div>
<ul>
<li>Webinar on <a href="https://www.youtube.com/watch?v=S6CeWL2qvl4">API & Microservices with Kong</a></li>
<li>Presentation on <a href="https://www.youtube.com/watch?v=OUUiS28hZuw">API Gateway Pattern and Kong in a Microservices World</a></li>
<li>Blog post on <a href="http://www.tothenew.com/blog/getting-started-with-application-authentication-via-kong-api-gateway/">API authentication with Kong</a></li>
</ul>
</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-60393663947890885532017-11-24T16:51:00.001-08:002017-11-24T16:51:22.773-08:00Using AWS CloudWatch Logs and AWS ElasticSearch for log aggregation and visualizationIf you run your infrastructure in AWS, then you can use CloudWatch Logs and AWS ElasticSearch + Kibana for log aggregation/searching/visualization as an alternative to either rolling your own ELK stack, or using a 3rd party SaaS solution such as Logentries, Loggly, Papertrail or the more expensive Splunk, Sumo Logic etc.<br />
<br />
Here are some pointers on how to achieve this.<br />
<br />
<b>1) Create IAM policy and role allowing read/write access to CloudWatch logs</b><br />
<br />
I created a IAM policy called <b>cloudwatch-logs-access</b> with the following content:<br />
<span style="font-size: x-small;"><b><br /></b></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>{</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Version": "2012-10-17",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Statement": [</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> {</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Effect": "Allow",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Action": [</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "logs:CreateLogGroup",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "logs:CreateLogStream",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "logs:PutLogEvents",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "logs:DescribeLogStreams"</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> ],</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Resource": [</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "arn:aws:logs:*:*:*"</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> ]</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> }</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> ]</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>}</b></span><br />
<br />
<br />
Then I create an IAM role called <b>cloudwatch-logs-role</b> and attached the <b>cloudwatch-logs-access</b> policy to it.<br />
<br />
<b>2) Attach IAM role to EC2 instances</b><br />
<br />
I attached the cloudwatch-logs-role IAM role to all EC2 instances from which I wanted to send logs to CloudWatch (I went to Actions --> Instance Settings --> Attach/Replace IAM Role and attached the role)<br />
<br />
<b>3) Install and configure CloudWatch Logs Agent on EC2 instances</b><br />
<br />
I followed the instructions <a href="http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html">here</a> for my OS, which is Ubuntu.<br />
<br />
I first downloaded a Python script:<br />
<br />
<span id="docs-internal-guid-a3bb8769-f03e-f8df-019d-66093c0a2b9c"><span style="font-family: "Courier New"; font-size: 10pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"># curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O</span></span><br />
<span><span style="font-family: "Courier New"; font-size: 10pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span>
Then I ran the script in the region where my EC2 instances are:<br />
<br />
<span id="docs-internal-guid-a3bb8769-f03f-a8f0-c006-2d012603a6a7"></span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New;"><span style="font-size: 13.3333px; white-space: pre-wrap;"><b># python awslogs-agent-setup.py --region us-west-2
Launching interactive setup of CloudWatch Logs agent ...
Step 1 of 5: Installing pip ...DONE
Step 2 of 5: Downloading the latest CloudWatch Logs agent bits ... DONE
Step 3 of 5: Configuring AWS CLI ...
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [us-west-2]:
Default output format [None]:
Step 4 of 5: Configuring the CloudWatch Logs Agent ...
Path of log file to upload [/var/log/syslog]:
Destination Log Group name [/var/log/syslog]:
Choose Log Stream name:
1. Use EC2 instance id.
2. Use hostname.
3. Custom.
Enter choice [1]: 2
Choose Log Event timestamp format:
1. %b %d %H:%M:%S (Dec 31 23:59:59)
2. %d/%b/%Y:%H:%M:%S (10/Oct/2000:13:55:36)
3. %Y-%m-%d %H:%M:%S (2008-09-08 11:52:54)
4. Custom
Enter choice [1]: 3
Choose initial position of upload:
1. From start of file.
2. From end of file.
Enter choice [1]: 1
More log files to configure? [Y]:</b></span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New;"><span style="font-size: 13.3333px; white-space: pre-wrap;"><b><br /></b></span></span></div>
I continued by adding more log files such as apache access and error logs, and other types of logs.<div>
<br /></div>
<div>
You can start/stop/restart the CloudWatch Logs agent via:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b># service awslogs start</b></span></div>
<div>
<br /></div>
<div>
The <span style="font-family: Courier New, Courier, monospace;">awslogs</span> service writes its logs in <span style="font-family: Courier New, Courier, monospace;">/var/log/awslogs.log</span> and its configuration file is in <span style="font-family: Courier New, Courier, monospace;">/var/awslogs/etc/awslogs.conf</span>.</div>
<div>
<br /></div>
<div>
<b>4) Create AWS ElasticSearch cluster</b></div>
<div>
<br /></div>
<div>
Not much to say here. Follow the prompts in the AWS console :)</div>
<div>
<br /></div>
<div>
For the initial Access Policy for the ES cluster, I chose an IP-based policy and specified the source CIDR blocks allowed to connect:</div>
<div>
<br /></div>
<div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 16.2px; font: 10.3px Menlo; color: #454545; -webkit-text-stroke: #454545; background-color: #f9f9f9}
span.s1 {font-kerning: none}
</style>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"Statement": [</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>{</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"Effect": "Allow",</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"Principal": {</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"AWS": "*"</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>},</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"Action": "es:*",</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"Resource": "arn:aws:es:us-west-2:accountID:domain/my-es-cluster/*",</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"Condition": {</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"IpAddress": {</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"aws:SourceIp": [</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"1.2.3.0/24",</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>"4.5.6.7/32"</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>]</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>}</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>}</b></span></div>
<div class="p1">
<span class="s1"><b><span class="Apple-converted-space"> </span>}</b></span></div>
<br />
<b>5) Create subscription filters for streaming CloudWatch logs to ElasticSearch</b><br />
<br />
First, make sure that the log files you configured with the AWS CloudWatch Log agent are indeed sent to CloudWatch. For each log file name, you should see a CloudWatch Log Group with that name, and inside the Log Group you should see multiple Log Streams, each Log Stream having the same name as the hostname sending those logs to CloudWatch.<br />
<br />
I chose one of the Log Streams, went to Actions --> Stream to Amazon Elasticsearch Service, chose the ElasticSearch cluster created above, then created a new Lambda function to do the streaming. I had to create a new IAM role for the Lambda function. I created a role I called <b>lambda-execution-role</b> and associated with it the pre-existing IAM policy <b>AWSLambdaBasicExecutionRole</b>.<br />
<br />
Once this Lambda function is created, subsequent log subscription filters for other Log Groups will reuse it for streaming to the same ES cluster.<br />
<br />
One important note here is that you also need to allow the role <b>lambda-execution-role</b> to access the ES cluster. To do that, I modified the ES access policy and added a statement for the ARN of this role:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> {</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Effect": "Allow",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Principal": {</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "AWS": "arn:aws:iam::accountID:role/lambda-execution-role"</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> },</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Action": "es:*",</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> "Resource": "arn:aws:es:us-west-2:accountID:domain/my-es-cluster/*"</b></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b> }</b></span><br />
<br />
<b>6) Configure index pattern in Kibana</b><br />
<br />
The last step is to configure Kibana to use the ElasticSearch index for the CloudWatch logs. If you look under Indices in the ElasticSearch dashboard, you should see indices of the form <b>cwl-2017.11.24</b>. In Kibana, add an Index Pattern of the form <b>cwl-*</b>. It should recognize the @timestamp field as the timestamp for the log entries, and create the Index Pattern correctly.<br />
<br />
Now if you go to the Discover screen in Kibana, you should be able to visualize and search your log entries streamed from CloudWatch.</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-47821407967638116892017-07-31T09:46:00.000-07:002017-07-31T09:46:10.643-07:00Apache 2.4 authentication and whitelisting scenariosI have these examples scattered among many Apache installations, so I wanted to gather my notes here for my benefit, and hopefully for others as well. The following scenarios depict various requirements for Apache 2.4 authentication and whitelisting. They are all for Apache 2.4.x running on Ubuntu 14.04/16.04.<br />
<br />
<b>Scenario 1: block all access to Apache except to a list of whitelisted IP addresses and networks</b><br />
<br />
Apache configuration snippet:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> <Directory /var/www/html/></span><br />
<span style="font-family: Courier New, Courier, monospace;"> IncludeOptional /etc/apache2/whitelist.conf</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Order allow,deny</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Allow from all</span><br />
<span style="font-family: Courier New, Courier, monospace;"> </Directory></span><br />
<br />
Contents of whitelist.conf file:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"># local server IPs</span><br />
<span style="font-family: Courier New, Courier, monospace;">Require ip 127.0.0.1</span><br />
<span style="font-family: Courier New, Courier, monospace;">Require ip 172.31.2.2</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"># Office network</span><br />
<span style="font-family: Courier New, Courier, monospace;">Require ip 1.2.3.0/24</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"># Other IP addresses</span><br />
<span style="font-family: Courier New, Courier, monospace;">Require ip 4.5.6.7/32</span><br />
<span style="font-family: "Courier New", Courier, monospace;">Require ip 5.6.7.8/32</span><br />
<span style="font-family: Courier New, Courier, monospace;">etc.</span><br />
<br />
<b>Scenario 2: enable basic HTTP authentication but allow specific IP addresses through with no authentication</b><br />
<br />
Apache configuration snippet:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> <Directory /var/www/html/></span><br />
<span style="font-family: Courier New, Courier, monospace;"> AuthType basic</span><br />
<span style="font-family: Courier New, Courier, monospace;"> AuthBasicProvider file</span><br />
<span style="font-family: Courier New, Courier, monospace;"> AuthName "Restricted Content"</span><br />
<span style="font-family: Courier New, Courier, monospace;"> AuthUserFile /etc/apache2/.htpasswd</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> Require valid-user</span><br />
<span style="font-family: Courier New, Courier, monospace;"> IncludeOptional /etc/apache2/whitelist.conf</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> Satisfy Any</span><br />
<span style="font-family: Courier New, Courier, monospace;"> </Directory></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><br /></span>
The contents of whitelist.conf are similar to the ones in Scenario 1.<div>
<br /></div>
<div>
<b>Scenario 3: enable basic HTTP authentication but allow access to specific URLs with no authentication</b></div>
<div>
<br /></div>
<div>
Apache configuration snippet:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> <Directory /var/www/html/></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Order allow,deny</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Allow from all</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> AuthType Basic</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> AuthName "Restricted Content"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> AuthUserFile /etc/apache2/.htpasswd</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> SetEnvIf Request_URI /.well-known/acme-challenge/* noauth=1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> <RequireAny></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Require env noauth</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Require valid-user</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> </RequireAny></span></div>
</div>
<div>
<span style="font-family: "Courier New", Courier, monospace;"> </Directory></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace;"><br /></span></div>
<div>
This is useful when you install SSL certificates from Let's Encrypt and you need to allow the Let's Encrypt servers access to the HTTP challenge directory.</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-19860616221312179242017-06-01T15:53:00.002-07:002017-06-01T15:53:29.390-07:00SSL termination and http caching with HAProxy, Varnish and Apache<br />
A common requirement when setting up a development or staging server is to try to mimic production as much as possible. One scenario I've implemented a few times is to use Varnish in front of a web site but also use SSL. Since Varnish can't handle encrypted traffic, SSL needs to be terminated before it hits Varnish. One fairly easy way to do it is using HAProxy to terminate both HTTP and HTTPS traffic, then forwarding the unencrypted traffic to Varnish, which then forwards non-cached traffic to Apache or nginx. Here are the steps to achieve this on an Ubuntu 16.04 box.<br />
<br />
<b>1) Install HAProxy and Varnish</b><br />
<span style="font-size: x-small;"><br /><span style="font-family: Courier New, Courier, monospace;"># apt-get install haproxy varnish</span></span><br />
<br />
<b>2) Get SSL certificates from Let’s Encrypt</b><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># wget https://dl.eff.org/certbot-auto</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># chmod +x certbot-auto</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># ./certbot-auto -a webroot --webroot-path=/var/www/mysite.com -d mysite.com certonly</span><br />
<br />
<b>3) Generate combined chain + key PEM file to be used by HAProxy</b><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># cat /etc/letsencrypt/live/mysite.com/fullchain.pem /etc/letsencrypt/live/mysite.com/privkey.pem > /etc/ssl/private/mysite.com.pem</span><br />
<br />
<b>4) Configure HAProxy</b><br />
<br />
Edit haproxy.cfg and add frontend sections for ports 80 and 443 + backend section pointing to varnish on port 8888<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># cat /etc/haproxy/haproxy.cfg</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">global</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> log /dev/log local0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> log /dev/log local1 notice</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> chroot /var/lib/haproxy</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> stats socket /run/haproxy/admin.sock mode 660 level admin</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> stats timeout 30s</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> user haproxy</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> group haproxy</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> daemon</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> # Default SSL material locations</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ca-base /etc/ssl/certs</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> crt-base /etc/ssl/private</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> # Default ciphers to use on SSL-enabled listening sockets.</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> # For more information, see ciphers(1SSL). This list is from:</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ssl-default-bind-options no-sslv3</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> tune.ssl.default-dh-param 2048</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">defaults</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> log global</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> mode http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> option httplog</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> option dontlognull</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> timeout connect 5000</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> timeout client 50000</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> timeout server 50000</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 400 /etc/haproxy/errors/400.http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 403 /etc/haproxy/errors/403.http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 408 /etc/haproxy/errors/408.http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 500 /etc/haproxy/errors/500.http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 502 /etc/haproxy/errors/502.http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 503 /etc/haproxy/errors/503.http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> errorfile 504 /etc/haproxy/errors/504.http</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">frontend www-http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> bind 172.31.8.204:80</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> http-request set-header "SSL-OFFLOADED" "1"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> reqadd X-Forwarded-Proto:\ http</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> default_backend varnish-backend</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">frontend www-https</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> bind 172.31.8.204:443 ssl crt mysite.com.pem</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> http-request set-header "SSL-OFFLOADED" "1"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> reqadd X-Forwarded-Proto:\ https</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> default_backend varnish-backend</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">backend varnish-backend</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> redirect scheme https if !{ ssl_fc }</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> server varnish 172.31.8.204:8888 check</span><br />
<br />
Enable UDP in rsyslog for haproxy logging by uncommenting 2 lines in /etc/rsyslog.conf:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># provides UDP syslog reception</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">module(load="imudp")</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">input(type="imudp" port="514")</span><br />
<br />
Restart rsyslog and haproxy<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># service rsyslog restart</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># service haproxy restart</span><br />
<br />
<b>5) Configure varnish to listen on port 8888</b><br />
<br />
Ubuntu 16.04 is using systemd for service management. You need to edit 2 files to configure the port varnish will listen on:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/lib/systemd/system/varnish.service</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/etc/default/varnish</span><br />
<br />
In both, set the port after the <span style="font-family: Courier New, Courier, monospace;">-a</span> flag to <span style="font-family: Courier New, Courier, monospace;">8888</span>, then stop the varnish service, reload the systemctl daemon and restart the varnish service:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># systemctl stop varnish.service</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># systemctl daemon-reload</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># systemctl start varnish.service</span><br />
<br />
By default, Varnish will send non-cached traffic to port 8080 on localhost.<br />
<br />
<b>6) Configure Apache or nginx to listen on 8080</b><br />
<br />
For Apache, change port 80 to 8080 in all virtual hosts, and also change 80 to 8080 in <span style="font-family: Courier New, Courier, monospace;">/etc/apache2/ports.conf</span>.<br />
<br />
<br />
<br />
<br />Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-82055727997263201522017-03-30T15:18:00.001-07:002017-03-30T15:19:03.728-07:00Working with AWS CodeDeployAs usual when I make a breakthrough after bumping my head against the wall for a few days trying to get something to work, I hasten to write down my notes here so I can remember what I've done ;) In this case, the head-against-the-wall routine was caused by trying to get <a href="https://aws.amazon.com/documentation/codedeploy/">AWS CodeDeploy</a> to work within the regular code deployment procedures that we have in place using Jenkins and Capistrano.<br />
<br />
Here is the 30,000 foot view of how the deployment process works using a combination of Jenkins, Docker, Capistrano and AWS CodeDeploy:<br />
<ol>
<li>Code gets pushed to GitHub</li>
<li>Jenkins deployment job fires off either automatically (for development environments, if so desired) or manually</li>
<ul>
<li>Jenkins spins up a Docker container running Capistrano and passes it several environment variables such as GitHub repository URL and branch, target deployment directory, etc.</li>
<li>The Capistrano Docker image is built beforehand and contains rake files that specify how the code it checks out from GitHub is supposed to be built</li>
<li>The Capistrano Docker container builds the code and exposes the target deployment directory as a Docker volume</li>
<li>Jenkins archives the files from the exposed Docker volume locally as a tar.gz file</li>
<li>Jenkins uploads the tar.gz to an S3 bucket</li>
<li>For good measure, Jenkins also builds a Docker image of a webapp container which includes the built artifacts, tags the image and pushes it to Amazon ECR so it can be later used if needed by an orchestration system such as Kubernetes </li>
</ul>
<li>AWS CodeDeploy runs a code deployment (via the AWS console currently, using the awscli soon) while specifying the S3 bucket and the tar.gz file above as the source of the deployment and an AWS AutoScaling group as the destination of the deployment</li>
<li>Everybody is happy </li>
</ol>
You may ask: why Capistrano? Why not use a shell script or some other way of building the source code into artifacts? Several reasons:<br />
<ul>
<li>Capistrano is still one of the most popular deployment tools. Many developers are familiar with it.</li>
<li>You get many good features for free just by using Capistrano. For example, it automatically creates a releases directory under your target directory, creates a timestamped subdirectory under releases where it checks out the source code, builds the source code, and if everything works well creates a 'current' symlink pointing to the releases/timestamped subdirectory </li>
<li>This strategy is portable. Instead of building the code locally and uploading it to S3 for use with AWS CodeDeploy, you can use the regular Capistrano deployment and build the code directly on a target server via ssh. The rake files are the same, only the deploy configuration differs.</li>
</ul>
I am not going to go into details for the Jenkins/Capistrano/Docker setup. I've touched on some of these topics in previous posts.<br />
<br />
I will go into details for the AWS CodeDeploy setup. Here goes.<br />
<br />
<b>Create IAM policies and roles</b><br />
<br />
There are two roles that need to be created for AWS CodeDeploy to work. One is to be attached to EC2 instances that you want to deploy to, and one is to be used by the CodeDeploy agent running on each instance.<br />
<br />
- Create following IAM policy for EC2 instances, which allows those instances to list S3 buckets and download fobject from S3 buckets (in this case the permissions cover all S3 buckets, but you can specify specific ones in the Resource variable):<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">{<br /> "Version": "2012-10-17",<br /> "Statement": [<br /> {<br /> "Action": [<br /> "s3:Get*",<br /> "s3:List*"<br /> ],<br /> "Effect": "Allow",<br /> "Resource": "*"<br /> }<br /> ]<br />}</span></span><br />
<br />
- Attach above policy to an IAM role and name the role e.g. <b>CodeDeploy-EC2-Instance-Profile</b><br />
<br />
- Create following IAM policy to be used by the CodeDeploy agent running on the EC2 instances you want to deploy to:<br />
<br />
<pre><span style="font-size: small;"><span style="font-family: "courier new" , "courier" , monospace;">{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"autoscaling:CompleteLifecycleAction",
"autoscaling:DeleteLifecycleHook",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLifecycleHooks",
"autoscaling:PutLifecycleHook",
"autoscaling:RecordLifecycleActionHeartbeat",
"autoscaling:CreateAutoScalingGroup",
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:EnableMetricsCollection",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribePolicies",
"autoscaling:DescribeScheduledActions",
"autoscaling:DescribeNotificationConfigurations",
"autoscaling:DescribeLifecycleHooks",
"autoscaling:SuspendProcesses",
"autoscaling:ResumeProcesses",
"autoscaling:AttachLoadBalancers",
"autoscaling:PutScalingPolicy",
"autoscaling:PutScheduledUpdateGroupAction",
"autoscaling:PutNotificationConfiguration",
"autoscaling:PutLifecycleHook",
"autoscaling:DescribeScalingActivities",
"autoscaling:DeleteAutoScalingGroup",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
"ec2:TerminateInstances",
"tag:GetTags",
"tag:GetResources",
"sns:Publish",
"cloudwatch:DescribeAlarms",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer"
],
"Resource": "*"
}
]
} </span></span></pre>
- Attach above policy to an IAM role and name the role e.g. <b>CodeDeployServiceRole</b><br />
<br />
<b>Create a 'golden image' AMI</b><br />
<br />
The whole purpose of AWS CodeDeploy is to act in conjunction with Auto Scaling Groups so that the app server layer of your infrastructure becomes horizontally scalable. You need to start somewhere, so I recommend the following:<br />
<ul>
<li>set up an EC2 instance for your app server the old-fashioned way, either with Ansible/Chef/Puppet or with Terraform</li>
<li>configure this EC2 instance to talk to any other layers it needs, i.e. the database layer (either running on EC2 instances or, if you are in AWS, on RDS), the caching layer (dedicated EC2 instances running Redis/memcached, or AWS ElastiCache), etc. </li>
<li> deploy some version of your code to the instance and make sure your application is fully functioning</li>
</ul>
If all this works as expected, take an AMI image from this EC2 instance. This image will serve as the 'golden image' that all other instances launched by the Auto Scaling Group / Launch Configuration will be based on.<br />
<br />
<b>Create Application Load Balancer (ALB) and Target Group</b><br />
<br />
The ALB will be the entry point into your infrastructure. For now just create an ALB and an associated Target Group. Make sure you add your availability zones into the AZ pool of the ALB.<br />
<br />
If you want the ALB to handle the SSL certificate for your domain, add the SSL cert to Amazon Certificate Manager and add a listener on the ALB mapping port 443 to the Target Group. Of course, also add a listener for port 80 on the ALB and map it to the Target Group.<br />
<br />
I recommend creating a dedicated Security Group for the ALB and allowing ports 80 and 443, either from everywhere or from a restricted subnet if you want to test it first. <br />
<br />
For the Target Group, make sure you set the correct health check for your application (something like requesting a special file healthcheck.html over port 80). No need to select any EC2 instances in the Target Group yet. <br />
<br />
<b>Create Launch Configuration and Auto Scaling Group</b><br />
<br />
Here are the main elements to keep in mind when creating a Launch Configuration to be used in conjunction with AWS CodeDeploy:<br />
<ul>
<li><b>AMI ID</b>: specify the AMI ID of the 'golden image' created above</li>
<li><b>IAM Instance Profile</b>: specify <b>CodeDeploy-EC2-Instance-Profile</b> (role created above)</li>
<li><b>Security Groups</b>: create a Security Group that allows access to ports 80 and 443 from the ALB Security Group above </li>
<li><b>User data</b>: each newly launched EC2 instance based on your golden image AMI will have to get the AWS CodeDeploy agent installed. Here's the user data for an Ubuntu-based AMI (taken from the AWS CodeDeploy documentation):</li>
</ul>
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash<br />apt-get -y update<br />apt-get -y install awscli<br />apt-get -y install ruby<br />cd /home/ubuntu<br />aws s3 cp s3://aws-codedeploy-us-west-2/latest/install . --region us-west-2<br />chmod +x ./install</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">./install auto </span></span><br />
<br />
Alternatively, you can run these commands your initial EC2 instance, then take a golden image AMI based off of that instance. That way you make sure that the CodeDeploy agent will be running on each new EC2 instance that gets provisioned via the Launch Configuration. In this case, there is no need to specify a User data section for the Launch Configuration.<br />
<br />
Once the Launch Configuration is created, you'll be able to create an Auto Scaling Group (ASG) associated with it. Here are the main configuration elements for the ASG:<br />
<ul>
<li><b>Launch Configuration</b>: the one defined above</li>
<li><b>Target Groups</b>: the Target Group defined above</li>
<li><b>Min/Max/Desired</b>: up to you to define the EC2 instance count for each of these. You can start with 1/1/1 to test</li>
<li><b>Scaling Policies</b>: you can start with a static policy (corresponding to Min/Max/Desired counts) and add policies based on alarms triggered by various Cloud Watch metrics such as CPU usage, memory usage, etc as measured on the EC2 instances comprising the ASG</li>
</ul>
Once the ASG is created, depending on the Desired instance count, that many EC2 instances will be launched.<br />
<br />
<b>Create AWS CodeDeploy Application and Deployment Group</b><br />
<br />
We finally get to the meat of this post. Go to the AWS CodeDeploy page and create a new Application. You also need to create a Deployment Group while you are at it. For Deployment Type, you can start with 'In-place deployment' and once you are happy with that, move to 'Blue/green deployment, which is more complex but better from a high-availability and rollback perspective.<br />
<br />
In the Add Instances section, choose 'Auto scaling group' as the tag type, and the name of the ASG created above as the key. Under 'Total matching instances' below the Tag and Key you should see a number of EC2 instances corresponding to the Desired count in your ASG.<br />
<br />
For Deployment Configuration, you can start with the default value, which is OneAtATime, then experiment with other types such as HalfAtATime (I don't recommend AllAtOnce unless you know what you're doing)<br />
<br />
For Service Role, you need to specify the <b>CodeDeployServiceRole</b> service role created above.<br />
<br />
<b>Create scaffoding files for AWS CodeDeploy Application lifecycle</b><br />
<br />
At a minimum, the tar.gz or zip archive of your application's built code also needs to contain what is called an AppSpec file, which is a YAML file named appspec.yml. The file needs to be in the root directory of the archive. Here's what I have in mine:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;">version: 0.0<br />os: linux<br />files:<br /> - source: /<br /> destination: /var/www/mydomain.com/<br />hooks:<br /> BeforeInstall:<br /> - location: before_install<br /> timeout: 300<br /> runas: root<br /> AfterInstall:<br /> - location: after_install<br /> timeout: 300<br /> runas: root</span></span><br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">before_install</span> and <span style="font-family: "courier new" , "courier" , monospace;">after_install</span> scripts (you can name them anything you want) are shell scripts that will be executed after the archive is downloaded on the target EC2 instance.<br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">before_install</span> script will be run <b>before</b> the files inside the archive are copied into the destination directory (as specified in the <span style="font-family: "courier new" , "courier" , monospace;">destination</span> variable <span style="font-family: "courier new" , "courier" , monospace;">/var/www/mydomain.com</span>). You can do things like create certain directories that need to exist, or change the ownership/permissions of certain files and directories.<br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">after_install</span> script script will be run <b>after</b> the files inside the archive are copied into
the destination directory. You can do things like create symlinks, run any scripts that need to complete the application installation (such as scripts that need to hit the database), etc.<br />
<br />
One note specific to archives obtained from code built by Capistrano: it's customary to have Capistrano tasks create symlinks for directories such as media or var to volumes outside of the web server document root (when media files are mounted over NFS/EFS for example). When these symlinks are unarchived by CodeDeploy, they tend to turn into regular directories, and the contents of potentially large mounted file systems get copied in them. Not what you want. I ended up creating all symlinks I need in the <span style="font-family: "courier new" , "courier" , monospace;">after_install</span> script, and not creating them in Capistrano.<br />
<br />
There are other points in the Application deploy lifecycle where you can insert your own scripts. See the <a href="http://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html">AppSpec hook documentation</a>.<br />
<br />
<b><br /></b>
<b>Deploy the application with AWS CodeDeploy</b><br />
<br />
Once you have an Application and its associated Deployment Group, you can select this group and choose 'Deploy new revision' from the Action drop-down. For the Revision Type, choose 'My application is stored in Amazon S3'. For the Revision Location, type in the name of the S3 bucket where Jenkins uploaded the tar.gz of the application build. You can play with the other options according to the needs of your deployment.<br />
<br />
Finally, hit the Deploy button, baby! If everything goes well, you'll see a nice green bar showing success. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuoc_AtL0tEy4CmY8OG-C4vHIDmNABjqxqXH-0mGiiLgKqKbBAVNjuZCaKfPoURLBgYZNNhevTCx1CEjE7eo3zI6jWxaaaEGiFWr4HJsjVQBeeTzAed6jV3Tp94QKnlXiPO-gK9Q/s1600/Screen+Shot+2017-03-30+at+3.12.02+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="97" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuoc_AtL0tEy4CmY8OG-C4vHIDmNABjqxqXH-0mGiiLgKqKbBAVNjuZCaKfPoURLBgYZNNhevTCx1CEjE7eo3zI6jWxaaaEGiFWr4HJsjVQBeeTzAed6jV3Tp94QKnlXiPO-gK9Q/s400/Screen+Shot+2017-03-30+at+3.12.02+PM.png" width="400" /></a></div>
<br />
If everything does not go well, you can usually troubleshoot things pretty well by looking at the logs of the Events associated with that particular Deployment. Here's an example of an error log:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><b>ScriptFailed</b> <br /> <b>Script Name</b> after_install <br /> <b>Message Script</b> at specified location: after_install run as user root failed with exit code 1 </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><b>Log Tail</b> [stderr]chown: changing ownership of ‘/var/www/mydomain.com/shared/media/images/85.jpg’: <br />Operation not permitted</span></span><br />
<pre class="log-label-right ng-binding" data-ng-bind="diagnostics.logTail" id="logTail"> </pre>
In this case, I the 'shared' directory was mounted over NFS, so I had to make sure the permissions and ownership of the source file system on the NFS server were correct.<br />
<br />
I am still experimenting with AWS CodeDeploy and haven't quite used it 'in anger' yet, so I'll report back with any other findings.<br />
<br />Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-39985309384325075902017-01-31T15:28:00.002-08:002017-01-31T15:28:35.832-08:00Notes on setting up Elasticsearch, Kibana and Fluentd on UbuntuI've been experimenting with an EFK stack (with Fluentd replacing Logstash) and I hasten to write down some of my notes. I could have just as well used Logstash, but my goal is to also use the EFK stack for capturing logs out of Kubernetes clusters, and I wanted to become familiar with <a href="http://www.fluentd.org/">Fluentd</a>, which is a <a href="https://www.cncf.io/">Cloud Native Computing Foundation</a> project.<br />
<br />
<b>1) Install Java 8</b><br />
<br />
On Ubuntu 16.04:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"># apt-get install openjdk-8-jre-headless</span><br />
<br />
On Ubuntu 14.04:<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"># add-apt-repository -y ppa:webupd8team/java</span><br />
<span style="font-family: Courier New, Courier, monospace;"># apt-get update</span><br />
<span style="font-family: Courier New, Courier, monospace;"># apt-get -y install oracle-java8-installer</span><br />
<br />
<b>2) Download and install Elasticsearch (latest version is 5.1.2 currently)</b><br />
<br /><span style="font-family: Courier New, Courier, monospace;"># wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.2.deb<br /># dpkg -i elasticsearch-5.1.2.deb</span><br /><br />Edit /etc/default/elasticsearch/elasticsearch.yml and set<br /><span style="font-family: Courier New, Courier, monospace;"><br /></span><div>
<span style="font-family: Courier New, Courier, monospace;">network.host: 0.0.0.0</span><br /><br /><span style="font-family: Courier New, Courier, monospace;"># service elasticsearch restart</span></div>
<div>
<br /></div>
<b>3) Download and install Kibana</b><br /><br /><span style="font-family: Courier New, Courier, monospace;"># wget https://artifacts.elastic.co/downloads/kibana/kibana-5.1.2-amd64.deb<br /># dpkg -i kibana-5.1.2-amd64.deb</span><br /><br />Edit /etc/kibana/kibana.yml and set<br /><span style="font-family: Courier New, Courier, monospace;"><br />server.host: "local_ip_address"</span><br /><br /><div>
<span style="font-family: Courier New, Courier, monospace;"># service kibana restart</span></div>
<div>
<b><br /></b></div>
<div>
<b>4) Install Fluentd agent (td-agent)</b></div>
<div>
<br /></div>
On Ubuntu 16.04:<div>
<br /><span style="font-family: Courier New, Courier, monospace;"># curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-xenial-td-agent2.sh | sh</span></div>
<div>
<br /></div>
<div>
On Ubuntu 14.04:</div>
<span style="font-family: Courier New, Courier, monospace;"><br /># curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-trusty-td-agent2.sh | sh</span><br /><div>
<br /></div>
<div>
Install Fluentd elasticsearch plugin (note that td-agent comes with its own gem installer):</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># td-agent-gem install fluent-plugin-elasticsearch</span><br /><br /><b>5) Configure Fluentd agent</b></div>
<div>
<br /></div>
<div>
To specify the Elasticsearch server to send the local logs to, use a <span style="font-family: Courier New, Courier, monospace;">match</span> stanza in /<span style="font-family: Courier New, Courier, monospace;">etc/td-agent/td-agent.conf</span>:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><match **></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> @type elasticsearch</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> logstash_format true</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> host IP_ADDRESS_OF_ELASTICSEARCH_SERVER</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> port 9200</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> index_name fluentd</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> type_name fluentd.project.stage.web01</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></match></span></div>
</div>
<div>
<br /></div>
<div>
Note that Fluentd is backwards compatible with logstash, so if you set <span style="font-family: Courier New, Courier, monospace;">logstash_format true</span>, Elasticsearch will create an index called logstash-*. Also, port 9200 needs to be open from the client to the Elasticsearch server.</div>
<div>
<br /></div>
<div>
I found it useful to set the <span style="font-family: Courier New, Courier, monospace;">type_name</span> property to a name specific to the client running the Fluentd agent. For example, if you have several projects/tenants, each with multiple environments (dev, stage, prod) and each environment with multiple servers, you could use something like <span style="font-family: "Courier New", Courier, monospace;">type_name fluentd.project.stage.web01. </span>This label will then be parsed and shown in Kibana and will allow you to easily tell the source of a given log entry.</div>
<div>
<br /></div>
<div>
If you want Fluentd to parse Apache logs and send the log entries to Elasticsearch, use stanzas of this form in <span style="font-family: Courier New, Courier, monospace;">td-agent.conf</span>:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><source></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> type tail</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> format apache2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> path /var/log/apache2/mysite.com-access.log</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> pos_file /var/log/td-agent/mysite.com-access.pos</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> tag apache.access</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></source></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><source></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> type tail</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> format apache2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> path /var/log/apache2/mysite.com-ssl-access.log</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> pos_file /var/log/td-agent/mysite.com-ssl-access.pos</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> tag apache.ssl.access</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></source></span></div>
</div>
<div>
<br /></div>
<div>
For syslog logs, use:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><source></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> @type syslog</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> port 5140</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> bind 0.0.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> tag system.local</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></source></span></div>
</div>
<div>
<br /></div>
<div>
Restart td-agent:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># service td-agent restart</span></div>
<div>
<br /></div>
<div>
Inspect the td-agent log file:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># tail -f /var/log/td-agent/td-agent.log</span></div>
<div>
<br /></div>
<div>
Some things I've had to do to fix errors emitted by td-agent:</div>
<div>
<ul>
<li>change permissions on apache log directory and log files so they are readable by user td-agent</li>
<li>make sure port 9200 is open from the client to the Elasticsearch server</li>
</ul>
<div>
<br /></div>
</div>
<div>
That's it in a nutshell. In the next installment, I'll show how to secure the communication between the Fluentd agent and the Elasticsearch server.</div>
<div>
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com1tag:blogger.com,1999:blog-9238405.post-91759916296295368842016-12-06T14:15:00.002-08:002016-12-06T14:15:57.039-08:00Using Helm to install Traefik as an Ingress Controller in KubernetesThat was a mouthful of a title...Hope this post lives up to it :)<br />
<br />
First of all, just a bit of theory. If you want to expose your application running on Kubernetes to the outside world, you have several choices.<br />
<br />
One choice you have is to expose the pods running your application via a <a href="http://kubernetes.io/docs/user-guide/services/">Service</a> of type <a href="http://kubernetes.io/docs/user-guide/services/#type-nodeport">NodePort</a> or <a href="http://kubernetes.io/docs/user-guide/services/#type-loadbalancer">LoadBalancer</a>. If you run your service as a NodePort, Kubernetes will allocate a random high port on every node in the cluster, and it will proxy traffic to that port to your service. Services of type LoadBalancer are only supported if you run your Kubernetes cluster using certain specific cloud providers such as AWS and GCE. In this case, the cloud provider will create a specific load balancer resource, for example an Elastic Load Balancer in AWS, which will then forward traffic to the pods comprising your service. Either way, the load balancing you get by exposing a service is fairly crude, at the TCP layer and using a round-robin algorithm.<br />
<br />
A better choice for exposing your Kubernetes application is to use <a href="http://kubernetes.io/docs/user-guide/ingress/#what-is-ingress">Ingress</a> resources together with <a href="http://kubernetes.io/docs/user-guide/ingress/#ingress-controllers">Ingress Controllers</a>. An ingress resource is a fancy name for a set of layer 7 load balancing rules, as you might be familiar with if you use HAProxy or Pound as a software load balancer. An Ingress Controller is a piece of software that actually implements those rules by watching the Kubernetes API for requests to Ingress resources. Here is a fragment from the <a href="https://github.com/kubernetes/contrib/blob/master/ingress/controllers/README.md">Ingress Controller documentation</a> on GitHub:<br />
<br />
<i>What is an Ingress Controller?<br /><br />An Ingress Controller is a daemon, deployed as a Kubernetes Pod, that watches the ApiServer's /ingresses endpoint for updates to the <a href="https://github.com/kubernetes/kubernetes/blob/master/docs/user-guide/ingress.md">Ingress resource</a>. Its job is to satisfy requests for ingress.<br /><a href="https://github.com/kubernetes/contrib/blob/master/ingress/controllers/README.md#writing-an-ingress-controller"></a>Writing an Ingress Controller<br /><br />Writing an Ingress controller is simple. By way of example, the <a href="https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx-alpha">nginx controller</a> does the following:<br /><ul>
<li><i>Poll until apiserver reports a new Ingress</i></li>
<li><i>Write the nginx config file based on a <a href="https://golang.org/pkg/text/template/">go text/template</a></i></li>
<li><i>Reload nginx</i></li>
</ul>
</i>As I mentioned in a previous post, I warmly recommend watching a KubeCon presentation from Gerred Dillon on "<a href="https://www.youtube.com/watch?v=Syw2PzRudIM">Kubernetes Ingress: Your Router, Your Rules</a>" if you want to further delve into the advantages of using Ingress Controllers as opposed to plain Services.<br />
While nginx is the only software currently included in the Kubernetes source code as an Ingress Controller, I wanted to experiment with a full-fledged HTTP reverse proxy such as <a href="https://docs.traefik.io/">Traefik</a>. I should add from the beginning that only nginx offers the <a href="https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/README.md#https">TLS feature of Ingress resources</a>. Traefik can terminate SSL of course, and I'll show how you can do that, but it is outside of the Ingress resource spec.<br />
<br />
I've also been looking at <a href="https://github.com/kubernetes/helm">Helm</a>, the Kubernetes package manager, and I noticed that Traefik is one of the '<a href="https://github.com/kubernetes/charts/tree/master/stable">stable</a>' packages (or Charts as they are called) currently offered by Helm, so I went the Helm route in order to install Traefik. In the following instructions I will assume that you are already running a Kubernetes cluster in AWS and that your local <span style="font-family: Courier New, Courier, monospace;">kubectl</span> environment is configured to talk to that cluster.<br />
<br />
<b>Install Helm</b><br />
<br />
This is pretty easy. Follow the <a href="https://github.com/kubernetes/helm/blob/master/docs/install.md">instructions</a> on GitHub to download or install a binary for your OS.<br />
<br />
<b>Initialize Helm</b><br />
<br />
Run <span style="font-family: Courier New, Courier, monospace;">helm init</span> in order to install the server component of Helm, called <span style="font-family: Courier New, Courier, monospace;">tiller</span>, which will be run as a Kubernetes Deployment in the <span style="font-family: Courier New, Courier, monospace;">kube-system</span> namespace of your cluster.<br />
<br />
<b>Get the Traefik Helm chart from GitHub</b><br />
<br />
I git cloned the entire <span style="font-family: Courier New, Courier, monospace;">kubernetes/charts</span> repo, then copied the <span style="font-family: Courier New, Courier, monospace;">traefik</span> directory locally under my own source code repo which contains the rest of the yaml files for my Kubernetes resource manifests.<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># git clone https://github.com/kubernetes/charts.git helmcharts</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cp -r helmcharts/stable/traefik traefik-helm-chart</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
It is instructive to look at the contents of a Helm chart. The main advantage of a chart in my view is the bundling together of all the Kubernetes resources necessary to run a specific set of services. The other advantage is that you can use Go-style templates for the resource manifests, and the variables in those template files can be passed to helm via a values.yaml file or via the command line.<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Times; font-size: small; font-weight: normal; white-space: normal;"><br /></span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Times; font-size: small; font-weight: normal; white-space: normal;">For more details on Helm charts and templates, I recommend this </span><a href="https://www.linux.com/learn/helm-kubernetes-package-manager" style="font-family: Times; font-size: medium; font-weight: normal; white-space: normal;">linux.com article</a><span style="font-family: Times; font-size: small; font-weight: normal; white-space: normal;">.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: Times; font-size: small; font-weight: normal; white-space: normal;"><br /></span></span></div>
<b>Create an Ingress resource for your application service</b><div>
<br /></div>
<div>
I copied the <span style="font-family: Courier New, Courier, monospace;">dashboard-ingress.yaml</span> template file from the Traefik chart and customized it so as to refer to my application's web service, which is running in a Kubernetes namespace called <span style="font-family: Courier New, Courier, monospace;">tenant1</span>.<br /><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cd traefik-helm-chart/templates</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cp dashboard-ingress.yaml web-ingress.yaml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cat web-ingress.yaml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">{{- if .Values.tenant1.enabled }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">apiVersion: extensions/v1beta1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">kind: Ingress</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">metadata:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> namespace: {{ .Values.tenant1.namespace }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> name: {{ template "fullname" . }}-web-ingress</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> labels:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> app: {{ template "fullname" . }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> release: "{{ .Release.Name }}"</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> heritage: "{{ .Release.Service }}"</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">spec:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> rules:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - host: {{ .Values.tenant1.domain }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> http:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> paths:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - path: /</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> backend:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> serviceName: {{ .Values.tenant1.serviceName }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> servicePort: {{ .Values.tenant1.servicePort }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">{{- end }}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br />The variables referenced in the template above are defined in the <span style="font-family: Courier New, Courier, monospace;">values.yaml</span> file in the Helm chart. I started with the variables in the <span style="font-family: Courier New, Courier, monospace;">values.yaml</span> file that came with the Traefik chart and added my own customizations:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; font-size: 14.6667px; font-weight: 700; white-space: pre-wrap;"># vi traefik-helm-chart/values.yaml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>ssl:</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> enabled: true</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>acme:</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> enabled: true</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> email: admin@mydomain.com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> staging: false</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> # Save ACME certs to a persistent volume. WARNING: If you do not do this, you will re-request</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> # certs every time a pod (re-)starts and you WILL be rate limited!</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> persistence:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> enabled: true</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> storageClass: kubernetes.io/aws-ebs</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> accessMode: ReadWriteOnce</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> size: 1Gi</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dashboard:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> enabled: true</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> domain: tenant1-lb.dev.mydomain.com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">gzip:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> enabled: false</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>tenant1:</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> enabled: true</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> namespace: tenant1</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> domain: tenant1.dev.mydomain.com</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> serviceName: web</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-size: 14.6667px; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> servicePort: http</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
Note that I added a section called <span style="font-family: Courier New, Courier, monospace;">tenant1</span>, where I defined the variables referenced in the <span style="font-family: Courier New, Courier, monospace;">web-ingress.yaml</span> template above. I also enabled the <span style="font-family: Courier New, Courier, monospace;">ssl</span> and <span style="font-family: Courier New, Courier, monospace;">acme</span> sections, so that Traefik can automatically install SSL certificates from <a href="https://letsencrypt.org/">Let's Encrypt</a> via the <a href="https://github.com/ietf-wg-acme/acme/">ACME</a> protocol.</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b>Install your customized Helm chart for Traefik</b></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
With these modifications done, I ran '<span style="font-family: Courier New, Courier, monospace;">helm install'</span> to actually deploy the various Kubernetes resources included in the Traefik chart. </div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
I specified the directory containing my Traefik chart files (<span style="font-family: Courier New, Courier, monospace;">traefik-helm-chart</span>) as the last argument passed to <span style="font-family: Courier New, Courier, monospace;">helm install:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># helm install --name tenant1-lb --namespace tenant1 traefik-helm-chart/</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME: tenant1-lb</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">LAST DEPLOYED: Tue Nov 29 09:51:12 2016</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAMESPACE: tenant1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">STATUS: DEPLOYED</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">RESOURCES:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> extensions/Ingress</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME HOSTS ADDRESS PORTS AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-web-ingress tenant1.dev.mydomain.com 80 1s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-dashboard tenant1-lb.dev.mydomain.com 80 0s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/PersistentVolumeClaim</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME STATUS VOLUME CAPACITY ACCESSMODES AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-acme Pending 0s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/Secret</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME TYPE DATA AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-default-cert Opaque 2 1s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/ConfigMap</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME DATA AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik 1 1s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/Service</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-dashboard 10.3.0.15 <none> 80/TCP 1s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik 10.3.0.215 <pending> 80/TCP,443/TCP 1s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> extensions/Deployment</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik 1 1 1 0 1s</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NOTES:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">1. Get Traefik's load balancer IP/hostname:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> NOTE: It may take a few minutes for this to become available.</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> You can watch the status by running:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> $ kubectl get svc tenant1-lb-traefik --namespace tenant1 -w</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Once 'EXTERNAL-IP' is no longer '<pending>':</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> $ kubectl describe svc tenant1-lb-traefik --namespace tenant1 | grep Ingress | awk '{print $3}'</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">2. Configure DNS records corresponding to Kubernetes ingress resources to point to the load balancer IP/hostname found in step 1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
At this point you should see two Ingress resources, one for the Traefik dashboard and on for the custom web ingress resource:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># kubectl --namespace tenant1 get ingress</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">NAME HOSTS ADDRESS PORTS AGE</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">tenant1-lb-traefik-dashboard tenant1-lb.dev.mydomain.com 80 50s</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "Courier New"; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">tenant1-lb-traefik-web-ingress tenant1.dev.mydomain.com 80 51s</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
As per the Helm notes above (shown as part of the output of <span style="font-family: Courier New, Courier, monospace;">helm install</span>), run this command to figure out the CNAME of the AWS ELB created by Kubernetes during the creation of the tenant1-lb-traefik service of type LoadBalancer:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># kubectl describe svc tenant1-lb-traefik --namespace tenant1 | grep Ingress | awk '{print $3}'</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">a5be275d8b65c11e685a402e9ec69178-91587212.us-west-2.elb.amazonaws.com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
Create <span style="font-family: Courier New, Courier, monospace;">tenant1.dev.mydomain.com</span> and <span style="font-family: Courier New, Courier, monospace;">tenant1-lb.dev.mydomain.com</span> as DNS CNAME records pointing to <span style="font-family: "Courier New"; font-size: 14.6667px; white-space: pre-wrap;">a5be275d8b65c11e685a402e9ec69178-91587212.us-west-2.elb.amazonaws.com.</span><br /><br />Now, if you hit http://tenant1-lb.dev.mydomain.com you should see the Traefik dashboard showing the frontends on the left and the backends on the right:<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img alt="Screen Shot 2016-11-29 at 10.54.07 AM.png" height="344" src="https://lh5.googleusercontent.com/UMgdhPBLRKKubgZAgSAxBF54lv-1IVIZc7QH2dHyFlM3ZNYP8Nkj5jXEhywFWlA4z8SJInxtUO7Slma1syWUdQ4-wFlgBmopJ55ycMqGCw6iFhCL15cF-xj9BryuLfDTMoOFLl0H" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
If you hit http://tenant1.dev.mydomain.com you should see your web service in action.</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
You can also inspect the logs of the tenant1-lb-traefik pod to see what's going on under the covers when Traefik is launched and to verify that the Let's Encrypt SSL certificates were properly downloaded via ACME:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; font-size: 14.6667px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"># kubectl --namespace tenant1 logs tenant1-lb-traefik-3710322105-o2887</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:51Z" level=info msg="Traefik version v1.1.0 built on 2016-11-18_09:20:46AM"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:51Z" level=info msg="Using TOML configuration file /config/traefik.toml"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:51Z" level=info msg="Preparing server http &{Network: Address::80 TLS:<nil> Redirect:<nil> Auth:<nil> Compress:false}"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:51Z" level=info msg="Preparing server https &{Network: Address::443 TLS:0xc4201b1800 Redirect:<nil> Auth:<nil> Compress:false}"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:51Z" level=info msg="Starting server on :80"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:58Z" level=info msg="Loading ACME Account..."</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:03:59Z" level=info msg="Loaded ACME config from store /acme/acme.json"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Starting provider *main.WebProvider {\"Address\":\":8080\",\"CertFile\":\"\",\"KeyFile\":\"\",\"ReadOnly\":false,\"Auth\":null}"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Starting provider *provider.Kubernetes {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Endpoint\":\"\",\"DisablePassHostHeaders\":false,\"Namespaces\":null,\"LabelSelector\":\"\"}"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Retrieving ACME certificates..."</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Retrieved ACME certificates"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Starting server on :443"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Server configuration reloaded on :80"</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">time="2016-11-29T00:04:01Z" level=info msg="Server configuration reloaded on :443"</span></span></div>
</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
To get an even better warm and fuzzy feeling about the SSL certificates installed via ACME, you can run this command against the live endpoint <span style="font-family: Courier New, Courier, monospace;">tenant1.dev.mydomain.com</span>:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; font-size: 14.6667px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"># echo | openssl s_client -showcerts -servername tenant1.dev.mydomain.com -connect tenant1.dev.mydomain.com:443 2>/dev/null</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">CONNECTED(00000003)</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">---</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">Certificate chain</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-size: x-small;"><span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"> 0 s:/CN=</span><span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;">tenant1.dev.mydomain.com</span></span></b></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;"> i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">-----BEGIN CERTIFICATE-----</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">MIIGEDCCBPigAwIBAgISAwNwBNVU7ZHlRtPxBBOPPVXkMA0GCSqGSIb3DQEBCwUA</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">-----END CERTIFICATE-----</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> i:/O=Digital Signature Trust Co./CN=DST Root CA X3</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">-----BEGIN CERTIFICATE-----</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">-----END CERTIFICATE-----</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">---</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">Server certificate</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-size: x-small;"><span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;">subject=/CN=</span><span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;">tenant1.dev.mydomain.com</span></span></b></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><b><span style="font-size: x-small;">issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3</span></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">---</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">No client certificate CA names sent</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">---</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">SSL handshake has read 3009 bytes and written 713 bytes</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">---</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">New, TLSv1/SSLv3, Cipher is AES128-SHA</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Server public key is 4096 bit</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Secure Renegotiation IS supported</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Compression: NONE</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">Expansion: NONE</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">SSL-Session:</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> Protocol : TLSv1</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> Cipher : AES128-SHA</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> Start Time: 1480456552</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> Timeout : 300 (sec)</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;"> Verify return code: 0 (ok)</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: x-small;">etc.</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
</div>
<b>Other helm commands</b></div>
<div>
<br /></div>
<div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
You can list the Helm releases that are currently running (a Helm release is a particular versioned instance of a Helm chart) with <span style="font-family: Courier New, Courier, monospace;">helm list</span>:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; font-size: 14.6667px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"># helm list</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">NAME </span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">REVISION </span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">UPDATED </span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">STATUS </span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">CHART</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb </span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">1 </span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">Tue Nov 29 10:13:47 2016</span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">DEPLOYED</span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;">traefik-1.1.0-a</span></div>
</div>
<div>
<span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "Courier New"; font-size: 12px; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div>
If you change any files or values in a Helm chart, you can apply the changes by means of the '<span style="font-family: Courier New, Courier, monospace;">helm upgrade</span>' command:<br /><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># helm upgrade tenant1-lb traefik-helm-chart</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
You can see the status of a release with <span style="font-family: Courier New, Courier, monospace;">helm status</span>:</div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># helm status tenant1-lb</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">LAST DEPLOYED: Tue Nov 29 10:13:47 2016</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAMESPACE: tenant1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">STATUS: DEPLOYED</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">RESOURCES:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/Service</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik 10.3.0.76 a92601b47b65f... 80/TCP,443/TCP 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-dashboard 10.3.0.36 <none> 80/TCP 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> extensions/Deployment</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik 1 1 1 1 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> extensions/Ingress</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME HOSTS ADDRESS PORTS AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-web-ingress tenant1.dev.mydomain.com 80 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-dashboard tenant1-lb.dev.mydomain.com 80 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/PersistentVolumeClaim</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME STATUS VOLUME CAPACITY ACCESSMODES AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-acme Bound pvc-927df794-b65f-11e6-85a4-02e9ec69178b 1Gi RWO 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/Secret</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME TYPE DATA AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik-default-cert Opaque 2 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">==> v1/ConfigMap</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">NAME DATA AGE</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tenant1-lb-traefik 1 35m</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<span id="docs-internal-guid-a06c8a87-d152-3ae3-03c3-342441ce30b3"></span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-2545786563580050312016-11-29T14:58:00.003-08:002016-11-29T14:59:44.314-08:00Kubernetes resource graphing with Heapster, InfluxDB and GrafanaI know that the <a href="https://www.cncf.io/">Cloud Native Computing Foundation</a> chose Prometheus as the monitoring platform of choice for Kubernetes, but in this post I'll show you how to quickly get started with graphing CPU, memory, disk and network in a Kubernetes cluster using Heapster, InfluxDB and Grafana.<br />
<br />
The <a href="https://github.com/kubernetes/heapster/blob/master/docs/influxdb.md">documentation</a> in the kubernetes/heapster GitHub repo is actually pretty good. Here's what I did:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ git clone https://github.com/kubernetes/heapster.git</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cd heapster/deploy/kube-config/influxdb</b></span><br />
<br />
Look at the yaml manifests to see if you need to customize anything. I left everything 'as is' and ran:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ kubectl create -f .</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">deployment "monitoring-grafana" created</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">service "monitoring-grafana" created</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">deployment "heapster" created</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">service "heapster" created</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">deployment "monitoring-influxdb" created</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">service "monitoring-influxdb" created</span><br />
<br />
Then you can run '<span style="font-family: "courier new" , "courier" , monospace;">kubectl cluster-info</span>' and look for the <span style="font-family: "courier new" , "courier" , monospace;">monitoring-grafana</span> endpoint. Since the <span style="font-family: "courier new" , "courier" , monospace;">monitoring-grafana</span> service is of type LoadBalancer, if you run your Kubernetes cluster in AWS, the service creation will also involve the creation of an ELB. By default the ELB security group allows 80 from all, so I edited that to restrict it to some known IPs.<br />
<br />
After a few minutes, you should see CPU and memory graphs shown in the Kubernetes dashboard. Here is an example showing pods running in the <span style="font-family: "courier new" , "courier" , monospace;">kube-system</span> namespace:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ1AaRqpm0eQzcafQB4fCMU9hrTmPnBuVDua866x3TsOiAzaVsx7aqxtfY1ler8izgacfhDzuNHeQ_yAOCglG9l1iUDlY-G6di6W-YCg34ZHRdOQyGGcWIAg8tlY_EBF2yHSD5qg/s1600/Screen+Shot+2016-11-29+at+2.52.31+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ1AaRqpm0eQzcafQB4fCMU9hrTmPnBuVDua866x3TsOiAzaVsx7aqxtfY1ler8izgacfhDzuNHeQ_yAOCglG9l1iUDlY-G6di6W-YCg34ZHRdOQyGGcWIAg8tlY_EBF2yHSD5qg/s400/Screen+Shot+2016-11-29+at+2.52.31+PM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
You can also hit the Grafana endpoint and choose the Cluster or Pods dashboards. Note that if you have a namespace different from <span style="font-family: "courier new" , "courier" , monospace;">default</span> and <span style="font-family: "courier new" , "courier" , monospace;">kube-system</span>, you have to enter its name manually in the namespace field of the Grafana Pods dashboard. Only then you'll be able to see data corresponding to pods running in that namespace (or at least I had to jump through that hoop.)<br />
<br />
Here is an example of graphs for the <span style="font-family: "courier new" , "courier" , monospace;">kubernetes-dashboard</span> pod running in the <span style="font-family: "courier new" , "courier" , monospace;">kube-system</span> namespace:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzmIMtlV6h0Td-kFwxxZAOnjlk9uyIIEgzi0Zsyd9DUMHrNGlVN5EUO1VbTrF8MxdEWtQHK_Bu8AgvCVpONZbBArDZqe6wc_FEwPCoGkfov1g2NKlLzxb2I64NXhsLPHJJ8NBbQw/s1600/Screen+Shot+2016-11-29+at+2.55.14+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzmIMtlV6h0Td-kFwxxZAOnjlk9uyIIEgzi0Zsyd9DUMHrNGlVN5EUO1VbTrF8MxdEWtQHK_Bu8AgvCVpONZbBArDZqe6wc_FEwPCoGkfov1g2NKlLzxb2I64NXhsLPHJJ8NBbQw/s400/Screen+Shot+2016-11-29+at+2.55.14+PM.png" width="400" /></a></div>
<br />
For info on how to customize the Grafana graphs, here's a good <a href="https://deis.com/blog/2016/monitoring-kubernetes-with-heapster/">post</a> from Deis.<br />
<br />Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-51294970760737199222016-11-22T17:13:00.002-08:002016-11-23T09:28:32.171-08:00Running an application using Kubernetes on AWSI've been knee-deep in <a href="http://kubernetes.io/">Kubernetes</a> for the past few weeks and to say that I like it is an understatement. It's exhilarating to have at your fingertips a distributed platfom created by Google's massive brain power.<br />
<br />
I'll jump right in and talk about how I installed Kubernetes in AWS and how I created various resources in Kubernetes in order to run a database-backed PHP-based web application.<br />
<br />
<b>Installing Kubernetes</b><br />
<br />
I used the <a href="https://github.com/kz8s/tack"><span style="font-family: "courier new" , "courier" , monospace;">tack</span></a> tool from my laptop running OSX to spin up a Kubernetes cluster in AWS. <span style="font-family: inherit;">Tack</span> uses <span style="font-family: "courier new" , "courier" , monospace;"><a href="https://www.terraform.io/">terraform</a></span> under the hood, which I liked a lot because it makes it very easy to delete all AWS resources and start from scratch while you are experimenting with it. I went with the <span style="font-family: inherit;">tack</span> defaults and spun up 3 m3.medium EC2 instances for running etcd and the Kubernetes API, the scheduler and the controller manager in an HA configuration. <span style="font-family: inherit;">Tack</span> also provisioned 3 m3.medium EC2 instances as Kubernetes workers/minions, in an EC2 auto-scaling group. Finally, <span style="font-family: inherit;">tack</span> spun up a t2.nano EC2 instance to server as a bastion host for getting access into the Kubernetes cluster. All 7 EC2 instances launched by <span style="font-family: inherit;">tack</span> run CoreOS.<br />
<br />
<b>Using kubectl</b><br />
<br />
Tack also installs <a href="http://kubernetes.io/docs/user-guide/kubectl-overview/"><span style="font-family: "courier new" , "courier" , monospace;">kubectl</span></a>, which is the Kubernetes command-line management tool. I used <span style="font-family: "courier new" , "courier" , monospace;">kubectl</span> to create the various Kubernetes resources needed to run my application: deployments, services, secrets, config maps, persistent volumes etc. It pays to become familiar with the syntax and arguments of <span style="font-family: "courier new" , "courier" , monospace;">kubectl</span>.<br />
<br />
<b>Creating namespaces</b><br />
<br />
One thing I needed to do right off the bat was to think about ways to achieve multi-tenancy in my Kubernetes cluster. This is done with <a href="http://kubernetes.io/docs/user-guide/namespaces/"><b>namespaces</b></a>. Here's my <span style="font-family: "courier new" , "courier" , monospace;">namespace.yaml</span> file:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat namespace.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Namespace</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1</span><br />
<br />
To create the namespace tenant1, I used <span style="font-family: "courier new" , "courier" , monospace;">kubectl create</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f namespace.yam</span>l<br />
<br />
To list all namespaces:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl get namespaces</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">NAME STATUS AGE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">default Active 12d</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kube-system Active 12d</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>tenant1</b> Active 11d </span><br />
<br />
If you don't need a dedicated namespace per tenant, you can just run <span style="font-family: "courier new" , "courier" , monospace;">kubectl</span> commands in the '<span style="font-family: "courier new" , "courier" , monospace;">default</span>' namespace.<br />
<br />
<b>Creating persistent volumes, storage classes and persistent volume claims</b><br />
<br />
I'll show how you can create two types of Kubernetes <a href="http://kubernetes.io/docs/user-guide/persistent-volumes/">persistent volumes</a> in AWS: one based on EFS, and one based on EBS. I chose the EFS one for my web application layer, for things such as shared configuration and media files. I chose the EBS one for my database layer, to be mounted as the data volume.<br />
<br />
First, I created an EFS share using the AWS console (although I recommend using terraform to do it automatically, but I am not there yet). I allowed the Kubernetes worker security group to access this share. I noted one of the DNS names available for it, e.g. us-west-2a.fs-c830ab1c.efs.us-west-2.amazonaws.com. I used this Kubernetes manifest to define a persistent volume (PV) based on this EFS share:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat web-pv-efs.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>PersistentVolume</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: pv-efs-web</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> capacity:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> storage: 50Gi</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> accessModes:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - ReadWriteMany</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> nfs:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> server: s-west-2a.fs-c830ab1c.efs.us-west-2.amazonaws.com</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> path: "/"</span><br />
<br />
To create the PV, I used <span style="font-family: "courier new" , "courier" , monospace;">kubectl create</span>, and I also specified the namespace <span style="font-family: "courier new" , "courier" , monospace;">tenant1</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f web-pv-efs.yaml --namespace tenant1</span><br />
<br />
However, creating a PV is not sufficient. Pods use <a href="http://kubernetes.io/docs/user-guide/persistent-volumes/#persistentvolumeclaims">persistent volume claims</a> (PVC) to refer to persistent volumes in their manifests. So I had to create a PVC:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat web-pvc.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>PersistentVolumeClaim</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: web-pvc</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> accessModes:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - ReadWriteMany</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> resources:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> requests:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> storage: 50Gi </span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f web-pvc.yaml</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;">--namespace tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
Note that a PVC does not refer directly to a PV. The storage specified in the PVC is provisioned from available persistent volumes.<br />
<br />
Instead of defining a persistent volume for the EBS volume I wanted to use for the database, I created a <a href="http://kubernetes.io/docs/user-guide/persistent-volumes/#storageclasses">storage class</a>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat db-storageclass-ebs.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>StorageClass</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: storage.k8s.io/v1beta1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: db-ebs</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">provisioner: kubernetes.io/aws-ebs</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">parameters:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> type: gp2</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f </span><span style="font-family: "courier new" , "courier" , monospace;">db-storageclass-ebs</span><span style="font-family: "courier new" , "courier" , monospace;">.yaml</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;">--namespace tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
I also created a PVC which does refer directly to the storage class name <span style="font-family: "courier new" , "courier" , monospace;">db-ebs</span>. When the PVC is used in a pod, the underlying resource (i.e. the EBS volume in this case) will be automatically provisioned by Kubernetes.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat db-pvc-ebs.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>PersistentVolumeClaim</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: db-pvc-ebs</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> annotations:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volume.beta.kubernetes.io/storage-class: '<b>db-ebs</b>'</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> accessModes:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - ReadWriteMany</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> resources:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> requests:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> storage: 50Gi</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f </span><span style="font-family: "courier new" , "courier" , monospace;">db-pvc-ebs</span><span style="font-family: "courier new" , "courier" , monospace;">.yaml</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;">--namespace tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
To list the newly created resource, you can use:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl get pv,pvc,storageclass --namespace tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<b>Creating secrets and ConfigMaps</b><br />
<div>
<br /></div>
<div>
I followed the "<a href="https://github.com/kubernetes/kubernetes/tree/master/examples/mysql-wordpress-pd">Persistent Installation of MySQL and Wordpress on Kubernetes</a>" guide to figure out how to create and use Kubernetes <a href="http://kubernetes.io/docs/user-guide/secrets/">secrets</a>. Here is how to create a secret for the MySQL root password, necessary when you spin up a pod based on a Percona or plain MySQL image:</div>
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ echo -n $MYSQL_ROOT_PASSWORD > mysql-root-pass.secret <br />$ kubectl create secret generic mysql-root-pass --from-file=mysql-root-pass.secret --namespace tenant1 </span><br />
<div>
<br /></div>
<div>
Kubernetes also has the handy notion of <a href="http://kubernetes.io/docs/user-guide/configmap/">ConfigMap</a>, a resource where you can store either entire configuration files, or key/value properties that you can then use in other Kubernetes resource definitions. For example, I save the GitHub branch and commit environment variables for the code I deploy in a ConfigMap:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create configmap git-config --namespace tenant1 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> --from-literal=GIT_BRANCH=$GIT_BRANCH \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> --from-literal=GIT_COMMIT=$GIT_COMMIT</span></div>
<div>
<br /></div>
<div>
I'll show how to use secrets and ConfigMaps in pod definitions a bit later on.</div>
<br />
<b>Creating an ECR image pull secret and a service account</b><br />
<br />
We use AWS ECR to store our Docker images. Kubernetes can access <a href="http://kubernetes.io/docs/user-guide/images/#using-a-private-registry">images stored in ECR</a>, but you need to jump through a couple of hoops to make that happen. First, you need to create a Kubernetes secret of type <span style="font-family: "courier new" , "courier" , monospace;">dockerconfigjson</span> which encapsulates the ECR credentials in base64 format. Here's a shell script that generates a file called <span style="font-family: "courier new" , "courier" , monospace;">ecr-pull-secret.yaml</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">TMP_JSON_CONFIG=/tmp/ecr_config.json</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">PASSWORD=$(aws --profile default --region us-west-2 ecr get-login | cut -d ' ' -f 6)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">cat > $TMP_JSON_CONFIG << EOF</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">{"https://YOUR_AWS_ECR_ID.dkr.ecr.us-west-2.amazonaws.com":{"username":"AWS","email":"none","password":"$PASSWORD"}}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">BASE64CONFIG=$(cat $TMP_JSON_CONFIG | base64)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cat > <b>ecr-pull-secret.yaml</b> << EOF</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Secret</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: <b>ecr-key</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> namespace: tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">data:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> .dockerconfigjson: $BASE64CONFIG</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">type: kubernetes.io/dockerconfigjson</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">rm -rf $TMP_JSON_CONFIG</span><br />
<br />
Once you run the script and generate the file, you can then define a Kubernetes <a href="http://kubernetes.io/docs/user-guide/service-accounts/">service account</a> that will use this secret:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat service-account.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>ServiceAccount</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> namespace: tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-dev</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">imagePullSecrets:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: <b>ecr-key</b></span><br />
<br />
Note that the service account refers to the <span style="font-family: "courier new" , "courier" , monospace;"><b>ecr-key</b></span> secret in the <span style="font-family: "courier new" , "courier" , monospace;"><b>imagePullSecrets</b></span> property.<br />
<br />
As usual, kubectl create will create these resources based on their manifests:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f ecr-pull-secret.yaml<br />$ kubectl create -f service-account.yaml</span><br />
<br />
<b>Creating deployments</b><br />
<br />
The atomic unit of scheduling in Kubernetes is a <a href="http://kubernetes.io/docs/user-guide/pods/">pod</a>. You don't usually create a pod directly (though you can, and I'll show you a case where it makes sense.) Instead, you create a <a href="http://kubernetes.io/docs/user-guide/deployments/">deployment</a>, which keeps track of how many pod replicas you need, and spins up the exact number of pods to fulfill your requirement. A deployment actually creates a <a href="http://kubernetes.io/docs/user-guide/replicasets/">replica set</a> under the covers, but in general you don't deal with replica sets directly. Note that deployments are the new recommended way to create multiple pods. The old way, which is still predominant in the documentation, was to use <a href="http://kubernetes.io/docs/user-guide/replication-controller/">replication controllers</a>.<br />
<br />
Here's my deployment manifest for a pod running a database image:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat db-deployment.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: extensions/v1beta1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Deployment</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: db-deployment</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> app: myapp</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> strategy:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> type: Recreate</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <b>template</b>:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> labels:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> app: myapp</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> tier: db</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> containers:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: db</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: MY_ECR_ID.dkr.ecr.us-west-2.amazonaws.com/myapp-db:tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> imagePullPolicy: Always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> env:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: MYSQL_ROOT_PASSWORD</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> secretKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: mysql-root-pass</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: mysql-root-pass.secret</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: MYSQL_DATABASE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: MYSQL_DATABASE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: MYSQL_USER</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: MYSQL_USER</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: MYSQL_DUMP_FILE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: MYSQL_DUMP_FILE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: S3_BUCKET</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: S3_BUCKET</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - containerPort: 3306</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: mysql</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> volumeMounts:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: ebs</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> mountPath: /var/lib/mysql</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> volumes:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: ebs</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> persistentVolumeClaim:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> claimName: db-pvc-ebs</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> serviceAccount: tenant1-dev</span><br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">template</span> section specifies the elements necessary for spinning up new pods. Of particular importance are the <span style="font-family: "courier new" , "courier" , monospace;">labels</span>, which, as we will see, are used by services to select pods that are included in a given service. The image property specifies the ECR Docker image used to spin up new containers. In my case, the image is called <span style="font-family: "courier new" , "courier" , monospace;">myapp-db</span> and it is tagged with the tenant name <span style="font-family: "courier new" , "courier" , monospace;">tenant1</span>. Here is the Dockerfile from which this image was generated:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat Dockerfile</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">FROM mysql:5.6</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># disable interactive functions</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">ARG DEBIAN_FRONTEND=noninteractive</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">RUN apt-get update && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> apt-get install -y python-pip</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN pip install awscli</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">VOLUME /var/lib/mysql</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">COPY etc/mysql/my.cnf /etc/mysql/my.cnf</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">COPY scripts/db_setup.sh /usr/local/bin/db_setup.sh</span><br />
<br />
Nothing out of the ordinary here. The image is based on the <span style="font-family: "courier new" , "courier" , monospace;">mysql</span> DockerHub image, specifically version <span style="font-family: "courier new" , "courier" , monospace;">5.6</span>. The <span style="font-family: "courier new" , "courier" , monospace;">my.cnf</span> is getting added in as a customization, and a <span style="font-family: "courier new" , "courier" , monospace;">db_setup.sh</span> script is copied over so it can be run at a later time.<br />
<br />
Some other things to note about the deployment manifest:<br />
<br />
<ul>
<li>I made pretty heavy use of secrets and ConfigMap key/values</li>
<li>I also used the <span style="font-family: "courier new" , "courier" , monospace;">db-pvc-ebs</span> Persistent Volume Claim and mounted the underlying physical resource (an EBS volume in this case) as /var/lib/mysql</li>
<li>I used the <span style="font-family: "courier new" , "courier" , monospace;">tenant1-dev</span> service account, which allows the deployment to pull down the container image from ECR</li>
<li>I didn't specify the number of replicas I wanted, which means that 1 pod will be created (the default)</li>
</ul>
<br />
To create the deployment, I ran <span style="font-family: "courier new" , "courier" , monospace;">kubectl</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f db-deployment.yaml --record --namespace tenant1</span><br />
<br />
Note that I used the <span style="font-family: Courier New, Courier, monospace;">--record</span> flag, which tells Kubernetes to keep a history of the commands used to create or update that deployment. You can show this history with the <span style="font-family: Courier New, Courier, monospace;">kubectl rollout history</span> command:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ kubectl --namespace tenant1 rollout history deployment db-deployment </span><br />
<br />
To list the running deployments, replica sets and pods, you can use:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl get get deployments,rs,pods --namespace tenant1 --show-all</span><br />
<br />
Here is another example of a deployment manifest, this time for redis:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat redis-deployment.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: extensions/v1beta1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Deployment</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: redis-deployment</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> replicas: 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> minReadySeconds: 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> template:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> labels:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> app: myapp</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> tier: redis</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> containers:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: redis</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> command: ["redis-server", "/etc/redis/redis.conf", "--requirepass", "<b>$(REDIS_PASSWORD)</b>"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: </span><span style="font-family: "courier new" , "courier" , monospace;">MY_ECR_ID</span><span style="font-family: "courier new" , "courier" , monospace;">.dkr.ecr.us-west-2.amazonaws.com/myapp-redis:tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> imagePullPolicy: Always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> env:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: <b>REDIS_PASSWORD</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> secretKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: redis-pass</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: redis-pass.secret</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - containerPort: 6379</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> protocol: TCP</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> serviceAccount: tenant1-dev</span><br />
<br />
One thing that is different from the db deployment is the way a secret (<span style="font-family: "courier new" , "courier" , monospace;">REDIS_PASSWORD</span>) is used as a command-line parameter for the container command. Make sure you use in this case the syntax <span style="font-family: "courier new" , "courier" , monospace;">$(VARIABLE_NAME)</span> because that's what Kubernetes expects.<br />
<br />
Also note the labels, which have <span style="font-family: "courier new" , "courier" , monospace;">app: myapp</span> in common with the db deployment, but a different value for <span style="font-family: "courier new" , "courier" , monospace;">tier</span>, <span style="font-family: "courier new" , "courier" , monospace;">redis</span> instead of <span style="font-family: "courier new" , "courier" , monospace;">db</span>.<br />
<br />
My last deployment example for now is the one for the web application pods:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat web-deployment.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: extensions/v1beta1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Deployment</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: web-deployment</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <b>replicas: 2</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> strategy:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> type: Recreate</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> template:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> labels:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> app: myapp</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> tier: frontend</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> containers:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: web</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: </span><span style="font-family: "courier new" , "courier" , monospace;">MY_ECR_ID</span><span style="font-family: "courier new" , "courier" , monospace;">.dkr.ecr.us-west-2.amazonaws.com/myapp-web:tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> imagePullPolicy: Always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - containerPort: 80</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: web</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> volumeMounts:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: web-persistent-storage</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> mountPath: /var/www/html/shared</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> volumes:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: web-persistent-storage</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> persistentVolumeClaim:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> claimName: web-pvc</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> serviceAccount: tenant1-dev</span><br />
<br />
Note that <span style="font-family: "courier new" , "courier" , monospace;">replicas</span> is set to 2, so that 2 pods will be launched and kept running at all times. The <span style="font-family: "courier new" , "courier" , monospace;">labels</span> have the same common part <span style="font-family: "courier new" , "courier" , monospace;">app: myapp</span>, but the <span style="font-family: "courier new" , "courier" , monospace;">tier</span> is different, set to <span style="font-family: "courier new" , "courier" , monospace;">frontend</span>. The persistent volume claim <span style="font-family: "courier new" , "courier" , monospace;">web-pvc</span> for the underlying physical EFS volume is used to mount <span style="font-family: "courier new" , "courier" , monospace;">/var/www/html/shared</span> over EFS.<br />
<br />
The image used for the container is derived from a stock <span style="font-family: "courier new" , "courier" , monospace;">ubuntu:14.04</span> DockerHub image, with apache and php 5.6 installed on top. Something along these lines:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">FROM ubuntu:14.04</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">RUN apt-get update && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> apt-get install -y ntp build-essential binutils zlib1g-dev telnet </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">git acl lzop unzip mcrypt expat xsltproc python-pip curl language-pack-en-base && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> pip install awscli</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">RUN export LC_ALL=en_US.UTF-8 && export LC_ALL=en_US.UTF-8 && export LANG=en_US.UTF-8 && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> apt-get install -y mysql-client-5.6 software-properties-common && add-apt-repository ppa:ondrej/php</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">RUN apt-get update && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> apt-get install -y --allow-unauthenticated apache2 apache2-utils libapache2-mod-php5.6 </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">php5.6 php5.6-mcrypt php5.6-curl php-pear php5.6-common php5.6-gd </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">php5.6-dev php5.6-opcache php5.6-json php5.6-mysql</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">RUN apt-get remove -y libapache2-mod-php5 php7.0-cli php7.0-common php7.0-json php7.0-opcache php7.0-readline php7.0-xml</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">RUN curl -sSL https://getcomposer.org/composer.phar -o /usr/bin/composer \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> && chmod +x /usr/bin/composer \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> && composer selfupdate</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">COPY files/apache2-foreground /usr/local/bin/</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">RUN chmod +x /usr/local/bin/apache2-foreground</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">EXPOSE 80</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">CMD bash /usr/local/bin/apache2-foreground</span><br />
<br />
<b>Creating services</b><br />
<br />
In Kubernetes, you are not supposed to refer to individual pods when you want to target the containers running inside them. Instead, you need to use <a href="http://kubernetes.io/docs/user-guide/services/">services</a>, which provide endpoints for accessing a set of pods based on a set of labels.<br />
<br />
Here is an example of a service for the <span style="font-family: "courier new" , "courier" , monospace;">db-deployment</span> I created above:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />$ cat db-service.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Service</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: db</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> app: myapp</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - port: 3306</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> selector:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> app: myapp</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> tier: db</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> clusterIP: None</span><br />
<br />
Note the <span style="font-family: "courier new" , "courier" , monospace;">selector</span> property, which is set to <span style="font-family: "courier new" , "courier" , monospace;">app: myapp</span> and <span style="font-family: "courier new" , "courier" , monospace;">tier: db</span>. By specifying these labels, we make sure that only the deployments tagged with those labels will be included in this service. There is only one deployment with those 2 labels, and that is <span style="font-family: "courier new" , "courier" , monospace;">db-deployment</span>.<br />
<br />
Here are similar service manifests for the redis and web deployments:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat redis-service.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Service</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: redis</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> app: myapp</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - port: 6379</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> selector:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> app: myapp</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> tier: redis</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> clusterIP: None</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">$ cat web-service.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Service</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: web</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> app: myapp</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - port: 80</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> selector:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> app: myapp</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> tier: frontend</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <b>type: LoadBalancer</b></span><br />
<br />
The selector properties for each service are set so that the proper deployment is included in each service.<br />
<br />
One important thing to note in the definition of the web service: its type is set to <span style="font-family: "courier new" , "courier" , monospace;">LoadBalancer</span>. Since Kubernetes is AWS-aware, the service creation will create an actual ELB in AWS, so that the application can be accessible from the outside world. It turns out that this is not the best way to expose applications externally, since this <span style="font-family: "courier new" , "courier" , monospace;">LoadBalancer</span> resource operates only at the TCP layer. What we need is a proper layer 7 load balancer, and in a future post I'll show how to use a Kubernetes <a href="http://kubernetes.io/docs/user-guide/ingress/">ingress controller</a> in conjunction with the <a href="https://traefik.io/"><b>traefik</b></a> proxy to achieve that. In the mean time, here is a KubeCon presentation from Gerred Dillon on "<a href="https://www.youtube.com/watch?v=Syw2PzRudIM">Kubernetes Ingress: Your Router, Your Rules</a>".<br />
<br />
To create the services defined above, I used <span style="font-family: "courier new" , "courier" , monospace;">kubectl</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f db-service.yaml --namespace tenant1</span><br />
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f redis-service.yaml --namespace tenant1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f web-service.yaml --namespace tenant1</span></div>
<div>
<br /></div>
At this point, the web application can refer to the database 'host' in its configuration files by simply using the name of the database service, which is <span style="font-family: "courier new" , "courier" , monospace;"><b>db</b></span> in our example. Similarly, the web application can refer to the redis 'host' by using the name of the redis service, which is <span style="font-family: "courier new" , "courier" , monospace;"><b>redis</b></span>. The Kubernetes magic will make sure calls to <span style="font-family: "courier new" , "courier" , monospace;"><b>db</b></span> and <span style="font-family: "courier new" , "courier" , monospace;"><b>redis</b></span> are properly routed to their end destinations, which are the actual containers running those services.<br />
<br />
<b>Running commands inside pods with kubectl exec</b><br />
<br />
Although you are not really supposed to do this in a container world, I found it useful to run a command such as loading a database from a MySQL dump file on a newly created pod. Kubernetes makes this relatively easy via the <span style="font-family: "courier new" , "courier" , monospace;">kubectl exec</span> functionality. Here's how I did it:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">DEPLOYMENT=db-deployment</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">NAMESPACE=tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">POD=$(kubectl --namespace $NAMESPACE get pods --show-all | grep $DEPLOYMENT | awk '{print $1}')</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">echo Running db_setup.sh command on pod $POD</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kubectl --namespace $NAMESPACE exec $POD -it /usr/local/bin/db_setup.sh</span><br />
<br />
where db_setup.sh downloads a sql.tar.gz file from S3 and loads it into MySQL.<br />
<br />
A handy troubleshooting tool is to get a shell prompt inside a pod. First you get the pod name (via <span style="font-family: "courier new" , "courier" , monospace;">kubectl get pods --show-all</span>), then you run:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl --namespace tenant1 exec -it $POD -- bash -il</span><br />
<br />
<b>Sharing volumes across containers</b><br />
<b><br /></b>
One of the patterns I found useful in docker-compose files is to mount a container volume into another container, for example to check out the source code in a container volume, then mount it as <span style="font-family: "courier new" , "courier" , monospace;">/var/www/html</span> in another container running the web application. This pattern is not extremely well supported in Kubernetes, but you can find your way around it by using <a href="http://kubernetes.io/docs/user-guide/petset/bootstrapping/">init-containers</a>.<br />
<br />
Here's an example of creating an individual pod for the sole purpose of running a Capistrano task against the web application source code. Simply running two regular containers inside the same pod would not achieve this goal, because the order of creation for those containers is random. What we need is to force one container to start before any regular containers by declaring it to be an 'init-container'.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat capistrano-pod.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: <b>Pod</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: capistrano</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> annotations:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> pod.beta.kubernetes.io/init-containers</b>: '[</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "name": "data4capistrano",</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "image": "MY_ECR_ID.dkr.ecr.us-west-2.amazonaws.com/myapp-web:tenant1",</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "command": ["cp", "-rH", "/var/www/html/current", "/tmpfsvol/"],</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "volumeMounts": [</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "name": "crtvol",</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "mountPath": "/tmpfsvol"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ]'</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> containers:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: capistrano</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: MY_ECR_ID.dkr.ecr.us-west-2.amazonaws.com/capistrano:tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> imagePullPolicy: Always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> command: [ "cap", "$(CAP_STAGE)", "$(CAP_TASK)", "--trace" ]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> env:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: CAP_STAGE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-cap-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: CAP_STAGE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: CAP_TASK</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-cap-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: CAP_TASK</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: DEPLOY_TO</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-cap-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: DEPLOY_TO</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> volumeMounts:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: crtvol</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> mountPath: /var/www/html</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: web-persistent-storage</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> mountPath: /var/www/html/shared</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> volumes:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: web-persistent-storage</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> persistentVolumeClaim:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> claimName: web-pvc</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> - name: crtvol</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> emptyDir: {}</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <b>restartPolicy: Never</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> serviceAccount: tenant1-dev</span><br />
<br />
The logic is here is a bit convoluted. Hopefully some readers of this post will know a better way to achieve the same thing. What I am doing here is launching a container based on the <span style="font-family: "courier new" , "courier" , monospace;">myapp-web:tenant1</span> Docker image, which already contains the source code checked out from GitHub. This container is declared as an <span style="font-family: "courier new" , "courier" , monospace;">init-container</span>, so it's guaranteed to run first. What it does is it mounts a special Kubernetes volume declared at the bottom of the pod manifest as an <span style="font-family: "courier new" , "courier" , monospace;"><a href="http://kubernetes.io/docs/user-guide/volumes/#emptydir">emptyDir</a></span>. This means that Kubernetes will allocate some storage on the node where this pod will run. The <span style="font-family: "courier new" , "courier" , monospace;">data4capistrano</span> container runs a command which copies the contents of the <span style="font-family: "courier new" , "courier" , monospace;">/var/www/html/current</span> directory from the <span style="font-family: "courier new" , "courier" , monospace;">myapp-web</span> image into this storage space mounted as <span style="font-family: "courier new" , "courier" , monospace;">/tmpfsvol</span> inside <span style="font-family: "courier new" , "courier" , monospace;">data4capistrano</span>. One other thing to note is that init-containers are a beta feature currently, so their declaration needs to be embedded into an <a href="http://kubernetes.io/docs/user-guide/annotations/">annotation</a>.<br />
<br />
When the regular <span style="font-family: "courier new" , "courier" , monospace;">capistrano</span> container is created inside the pod, it also mounts the same <span style="font-family: "courier new" , "courier" , monospace;">emptyDir</span> container (which is not empty at this point, because it was populated by the init-container), this time as <span style="font-family: "courier new" , "courier" , monospace;">/var/www/html</span>. It also mounts the shared EFS file system as <span style="font-family: "courier new" , "courier" , monospace;">/var/www/html/shared</span>. With these volumes in place, it has all it needs in order to run Capistrano locally via the <span style="font-family: "courier new" , "courier" , monospace;">cap</span> command. The stage, task, and target directory for Capistrano are passed via ConfigMaps values.<br />
<br />
One thing to note is that the <span style="font-family: "courier new" , "courier" , monospace;">RestartPolicy</span> is set to <span style="font-family: "courier new" , "courier" , monospace;">Never</span> for this pod, because we only want to run it once and be done with it.<br />
<br />
To run the pod, I used kubectl again:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f capistrano-pod.yaml --namespace tenant1</span><br />
<br />
<b>Creating jobs</b><br />
<br />
Kubernetes also has the concept of <a href="http://kubernetes.io/docs/user-guide/jobs/">jobs</a>, which differ from deployments in that they run one instance of a pod and make sure it completes. Jobs are useful for one-off tasks that you want to run, or for periodic tasks such as cron commands. Here is an example of a job manifest which runs a script that uses the <a href="http://twig.sensiolabs.org/">twig</a> template engine under the covers in order to generate a configuration file for the web application:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat template-job.yaml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">apiVersion: batch/v1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">kind: Job</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: myapp-template</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> template:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> metadata:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: myapp-template</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> spec:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> containers:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: myapp-template</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: </span><span style="font-family: "courier new" , "courier" , monospace;">Y_ECR_ID</span><span style="font-family: "courier new" , "courier" , monospace;">.dkr.ecr.us-west-2.amazonaws.com/myapp-template:tenant1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> imagePullPolicy: Always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> command: [ "php", "/root/scripts/templatize.php"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> env:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: DBNAME</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: MYSQL_DATABASE</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: DBUSER</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> configMapKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: tenant1-config</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: MYSQL_USER</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: DBPASSWORD</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> secretKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: mysql-db-pass</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: mysql-db-pass.secret</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: REDIS_PASSWORD</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> valueFrom:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> secretKeyRef:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name: redis-pass</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> key: redis-pass.secret</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volumeMounts:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: web-persistent-storage</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> mountPath: /var/www/html/shared</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volumes:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - name: web-persistent-storage</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> persistentVolumeClaim:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> claimName: web-pvc</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> restartPolicy: Never</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> serviceAccount: tenant1-dev</span><br />
<br />
The templatize.php script substitutes DBNAME, DBUSER, DBPASSWORD and REDIS_PASSWORD with the values passed in the job manifest, obtained from either Kubernetes secrets or ConfigMaps.<br />
<br />
To create the job, I used <span style="font-family: "courier new" , "courier" , monospace;">kubectl</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ kubectl create -f template-job.yaml --namespace tenant1</span><br />
<br />
<b>Performing rolling updates and rollbacks for Kubernetes deployments</b><br />
<br />
Once your application pods are running, you'll need to update the application to a new version. Kubernetes allows you to do a rolling update of your deployments. One advantage of using deployments as opposed to the older method of using replication controllers is that the update process for deployment happens on the Kubernetes server side, and can be paused and restarted. There are a few ways of doing a rolling update for a deployment (and a recent linux.com <a href="https://www.linux.com/learn/rolling-updates-and-rollbacks-using-kubernetes-deployments">article</a> has a good overview as well).<br />
<br />
a) You can modify the deployment's yaml file and change a label such as a version or a git commit, then run <span style="font-family: Courier New, Courier, monospace;">kubectl apply</span>:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ kubectl --namespace tenant1 apply -f deployment.yaml</span><br />
<br />
Note from the Kubernetes documentation on <a href="http://kubernetes.io/docs/user-guide/deployments/#updating-a-deployment">updating deployments</a>:<br />
<i><span style="background-color: white; font-family: Roboto, sans-serif; font-size: 14px; letter-spacing: 0.1px;"><br /></span></i>
<i><span style="background-color: white; font-family: Roboto, sans-serif; font-size: 14px; letter-spacing: 0.1px;">a Deployment’s rollout is triggered if and only if the Deployment’s pod template (i.e. </span><code class="highlighter-rouge" style="background: none rgb(247, 247, 247); border: 0px; box-sizing: border-box; color: #303030; display: inline-block; font-family: "Roboto Mono", monospace; font-size: 14px; font-weight: bold; letter-spacing: 0.1px; margin: 0px; padding: 2px 4px; vertical-align: bottom;">.spec.template</code></i><span style="background-color: white; font-family: Roboto, sans-serif; font-size: 14px; letter-spacing: 0.1px;"><i>) is changed, e.g. updating labels or container images of the template. Other updates, such as scaling the Deployment, will not trigger a rollout.</i></span><br />
<br />
b) You can use <span style="font-family: Courier New, Courier, monospace;">kubectl set</span> to specify a new image for the deployment containers. Example from the documentation:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">deployment "nginx-deployment" image update</span><br /><br />c) You can use <span style="font-family: Courier New, Courier, monospace;">kubectl patch</span> to add a unique label to the deployment spec template on the fly. This is the method I've been using, with the label being set to a timestamp:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ kubectl patch deployment web-deployment --namespace tenant1 -p \</span><span style="font-family: Courier New, Courier, monospace;"> "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"`date +'%Y%M%d%H%M%S'`\"}}}}}"</span><br />
<br />When updating a deployment, a new replica set will be created for that deployment, and the specified number of pods will be launched by that replica set, while the pods from the old replica set will be shut down. However, the old replica set itself will be preserved, allowing you to perform a rollback if needed. </div>
<div>
<br /></div>
<div>
If you want to roll back to a previous version, you can use <span style="font-family: Courier New, Courier, monospace;">kubectl rollout history</span> to show the revisions of your deployment updates:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ kubectl --namespace tenant1 rollout history deployment web-deployment</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">deployments "web-deployment"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">REVISION<span class="Apple-tab-span" style="white-space: pre;"> </span>CHANGE-CAUSE</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">1<span class="Apple-tab-span" style="white-space: pre;"> </span>kubectl create -f web-deployment.yaml --record --namespace tenant1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">2<span class="Apple-tab-span" style="white-space: pre;"> </span>kubectl patch deployment web-deployment --namespace tenant1 -p {"spec":{"template":{"metadata":{"labels":{"date":"1479161196"}}}}}</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">3<span class="Apple-tab-span" style="white-space: pre;"> </span>kubectl patch deployment web-deployment --namespace tenant1 -p {"spec":{"template":{"metadata":{"labels":{"date":"1479161573"}}}}}</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">4<span class="Apple-tab-span" style="white-space: pre;"> </span>kubectl patch deployment web-deployment --namespace tenant1 -p {"spec":{"template":{"metadata":{"labels":{"date":"1479243444"}}}}}</span></div>
<br />Now use <span style="font-family: Courier New, Courier, monospace;">kubectl rollout undo</span> to rollback to a previous revision:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ kubectl --namespace tenant1 rollout undo deployments web-deployment --to-revision=3</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">deployment "web-deployment" rolled back</span></div>
<br />
I should note that all these kubectl commands can be easily executed out of Jenkins pipeline scripts or shell steps. I use a Docker image to wrap kubectl and its keys so that they I don't have to install it on the Jenkins worker nodes.<br />
<br />
And there you have it. I hope the examples I provided will shed some light on some aspects of Kubernetes that go past the 'Kubernetes 101' stage. Before I forget, here's a good overview from the official documentation on <a href="http://kubernetes.io/docs/user-guide/production-pods/">using Kubernetes in production</a>.<br />
<br />
I have a lot more Kubernetes things on my plate, and I hope to write blog posts on all of them. Some of these:<br />
<br />
<ul>
<li>ingress controllers based on traefik</li>
<li>creation and renewal of Let's Encrypt certificates</li>
<li>monitoring</li>
<li>logging</li>
<li>using the <a href="https://github.com/kubernetes/helm">Helm</a> package manager</li>
<li>...and more</li>
</ul>
<br />
<br />
<br />
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-75620667148036011342016-10-20T14:14:00.003-07:002016-10-20T15:44:57.591-07:00Creating EC2 and Route 53 resources with TerraformInspired by the great series of <a href="https://www.terraform.io/">Terraform</a>-related posts published on the <a href="https://blog.gruntwork.io/">Gruntwork blog</a>, I've been experimenting with Terraform the last couple of days. So far, I like it a lot, and I think the point that Yevgeniy Brikman makes in the <a href="https://blog.gruntwork.io/why-we-use-terraform-and-not-chef-puppet-ansible-saltstack-or-cloudformation-7989dad2865c#.2w4m0ahrt">first post of the series</a>, on why they chose Terraform over other tools (which are the usual suspects Chef, Puppet, Ansible), is a very valid point: Terraform is a <b>declarative</b> and <b>client-only</b> <b>orchestration</b> tool that allows you to manage <b>immutable infrastructure</b>. Read that post for more details about why this is a good thing.<br />
<br />
In this short post I'll show how I am using Terraform in its Docker image incarnation to create an AWS ELB and a Route 53 CNAME record pointing to the name of the newly-created ELB.<br />
<br />
I created a directory called <span style="font-family: "courier new" , "courier" , monospace;">terraform</span> and created this <span style="font-family: "courier new" , "courier" , monospace;">Dockerfile</span> inside it:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat Dockerfile</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">FROM hashicorp/terraform:full</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">COPY data /data/</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">WORKDIR /data</span><br />
<br />
My Terraform configuration files are under <span style="font-family: "courier new" , "courier" , monospace;">terraform/data</span> locally. I have 2 files, one for variable declarations and one for the actual resource declarations.<br />
<br />
Here is the variable declaration file:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat data/vars.tf</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "access_key" {}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">variable "secret_key" {}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">variable "region" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> default = "us-west-2"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "exposed_http_port" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> description = "The HTTP port exposed by the application"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> default = 8888</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "security_group_id" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> description = "The ID of the ELB security group"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> default = "sg-SOMEID"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "host1_id" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> description = "EC2 Instance ID for ELB Host #1"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> default = "i-SOMEID1"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "host2_id" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> description = "EC2 Instance ID for ELB Host #2"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> default = "i-SOMEID2"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "elb_cname" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> description = "CNAME for the ELB"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">variable "route53_zone_id" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> description = "Zone ID for the Route 53 zone "</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> default = "MY_ZONE_ID"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
Note that I am making some assumptions, namely that the ELB will point to 2 EC2 instances. This is because I know beforehand which 2 instances I want to point it to. In my case, those instances are configured as Rancher hosts, and the ELB's purpose is to expose as port 80 to the world an internal Rancher load balancer port (say 8888).<br />
<br />
Here is the resource declaration file:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /><b>$ cat data/main.tf</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># --------------------------------------------------------</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># CONFIGURE THE AWS CONNECTION</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># --------------------------------------------------------</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">provider "aws" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> access_key = "${var.access_key}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> secret_key = "${var.secret_key}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> region = "${var.region}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># --------------------------------------------------------</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># CREATE A NEW ELB</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># --------------------------------------------------------</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">resource "aws_elb" "my-elb" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name = "MY-ELB"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> listener {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> instance_port = "${var.exposed_http_port}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> instance_protocol = "http"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> lb_port = 80</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> lb_protocol = "http"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> health_check {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> healthy_threshold = 2</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> unhealthy_threshold = 2</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> timeout = 3</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> target = "TCP:${var.exposed_http_port}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> interval = 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> instances = ["${var.host1_id}", "${var.host2_id}"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> security_groups = ["${var.security_group_id}"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> cross_zone_load_balancing = true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> idle_timeout = 400</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> connection_draining = true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> connection_draining_timeout = 400</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> tags {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> Name = "MY-ELB"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># --------------------------------------------------------</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># CREATE A ROUTE 53 CNAME FOR THE ELB</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># --------------------------------------------------------</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">resource "aws_route53_record" "my-elb-cname" {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> zone_id = "${var.route53_zone_id}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> name = "${var.elb_cname}"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> type = "CNAME"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ttl = "300"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> records = ["${aws_elb.my-elb.dns_name}"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
First I declare a provider of type <span style="font-family: "courier new" , "courier" , monospace;">aws</span>, then I declare 2 resources, one of type <span style="font-family: "courier new" , "courier" , monospace;">aws_elb</span>, and another one of type <span style="font-family: "courier new" , "courier" , monospace;">aws_route53_record</span>. These types are all detailed in the very good Terraform documentation for the <a href="https://www.terraform.io/docs/providers/aws/index.html">AWS provider</a>.<br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">aws_elb</span> resource defines an ELB named <span style="font-family: "courier new" , "courier" , monospace;"><b>my-elb</b></span>, which points to the 2 EC2 instances mentioned above. The instances are specified by their variable names from vars.tf, using the syntax <span style="font-family: "courier new" , "courier" , monospace;">${var.VARIABLE_NAME}, </span>e.g.<span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;">${var.host1_id}</span>. For the other properties of the ELB resource, consult the <a href="https://www.terraform.io/docs/providers/aws/r/elb.html">Terraform aws_elb documentation</a>.<br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">aws_route53_record</span> defined a CNAME record in the given Route 53 zone file (specified via the <span style="font-family: "courier new" , "courier" , monospace;">route53_zone_id</span> variable). An important thing to note here is that the CNAME points to the name of the ELB just created via the <span style="font-family: Courier New, Courier, monospace;">aws_elb.my-elb.dns_name</span> variable. This is one of the powerful things you can do in Terraform - reference properties of resources in other resources.<br />
<br />
Again, for more details on <span style="font-family: Courier New, Courier, monospace;">aws_route53_record</span>, consult the <a href="https://www.terraform.io/docs/providers/aws/r/route53_record.html">Terraform documentation</a>.<br />
<br />
Given these files, I built a local Docker image:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ docker build -t terraform:local</b></span><br />
<br />
I can then run the Terraform '<span style="font-family: "courier new" , "courier" , monospace;">plan</span>' command to see what Terraform intends to do:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">$ docker run -t --rm terraform:local plan \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "access_key=$TERRAFORM_AWS_ACCESS_KEY" \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "secret_key=$TERRAFORM_AWS_SECRET_KEY" \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "exposed_http_port=$LB_EXPOSED_HTTP_PORT" \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "elb_cname=$ELB_CNAME"</span><br />
<br />
The nice thing about this is that I can run Terraform in exactly the same way via Jenkins. The variables above are defined in Jenkins either as credentials of type 'secret text' (the 2 AWS keys), or as build parameters of type string. In Jenkins, the Docker image name would be specified as an ECR image, something of the type <span style="font-family: "courier new" , "courier" , monospace;">ECR_ID.dkr.ecr.us-west-2.amazonaws.com/terraform</span>.<br />
<br />
After making sure that the plan corresponds to what I expected, I ran the Terraform apply command:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ docker run -t --rm terraform:local apply \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "access_key=$TERRAFORM_AWS_ACCESS_KEY" \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "secret_key=$TERRAFORM_AWS_SECRET_KEY" \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "exposed_http_port=$LB_EXPOSED_HTTP_PORT" \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">-var "elb_cname=$ELB_CNAME"</span><br />
<br />
One thing to note is that the AWS credentials I used are for an IAM user that has only the privileges needed to create the resources I need. I tinkered with the IAM policy generator until I got it right. Terraform will emit various AWS errors when it's not able to make certain calls. Those errors help you add the required privileges to the IAM policies. In my case, here are some example of policies.<br />
<br />
Allow all ELB operations:<br />
<br />
<pre>{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1476919435000",
"Effect": "Allow",
"Action": [ "elasticloadbalancing:*"
],
"Resource": [
"*"
]
}
]
}</pre>
<pre></pre>
Allow the <span style="font-family: "courier new" , "courier" , monospace;">ec2:DescribeSecurityGroups</span> operation:<br />
<div>
<br /></div>
<div>
<pre>{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1476983196000",
"Effect": "Allow",
"Action": [
"ec2:DescribeSecurityGroups"
],
"Resource": [
"*"
]
}
]
}</pre>
</div>
<div>
Allow the <span style="font-family: "courier new" , "courier" , monospace;">route53:GetHostedZone</span> and <span style="font-family: "courier new" , "courier" , monospace;">GetChange</span> operations:</div>
<div>
<br /></div>
<div>
<pre>{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1476986919000",
"Effect": "Allow",
"Action": [
"route53:GetHostedZone",
"route53:GetChange"
],
"Resource": [
"*"
]
}
]
}</pre>
<br />
Allow the creation, changing and listing of Route 53 record sets in a given Route 53 zone file:</div>
<div>
<br /></div>
<div>
<pre>{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1476987070000",
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets"
],
"Resource": [
"arn:aws:route53:::hostedzone/MY_ZONE_ID"
]
}
]
}</pre>
</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-23565754463694148102016-09-09T12:27:00.000-07:002016-09-09T12:27:15.263-07:00Docker orchestration with RancherFor the last month or so I've been experimenting with <a href="http://rancher.com/">Rancher</a> as the orchestration layer for Docker-based deployments. I've been pretty happy with it so far. Here are some of my notes and a few tips and tricks. I also recommend reading through the very good Rancher <a href="http://docs.rancher.com/rancher/v1.2/en/">documentation</a>. In what follows I'll assume that the cluster management engine used by Rancher is its own engine called Cattle. Rancher also supports Kubernetes, Mesos and Docker Swarm.<br />
<br />
<b><span style="font-size: large;">Running the Rancher server</span></b><br />
<br />
I provisioned an EC2 instance, installed Docker on it, then ran this command to launch the Rancher server as a Docker container (it will also get launched automatically if you reboot the EC2 instance):<br />
<br />
<span id="docs-internal-guid-cce51a89-05b9-dd85-654d-22895d1caa31"></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b># docker run -d --restart=always -p 8080:8080 rancher/server</b></span><br />
<div>
<br /></div>
<div>
<b><span style="font-size: large;">Creating Rancher environments</span></b></div>
<div>
<br /></div>
<div>
It's important to think about the various environments you want to manage in Rancher. If you have multiple projects that you want to manage with Rancher, as well as multiple environments for your infrastructure, such as development, staging and production, I recommend you create a Rancher environment per project/infrastructure-environment combination, for example a Rancher environment called <span style="font-family: Courier New, Courier, monospace;">proj1dev</span>, another one called <span style="font-family: Courier New, Courier, monospace;">proj1stage</span>, another called <span style="font-family: Courier New, Courier, monospace;">proj1prod</span>, and similarly for other projects: <span style="font-family: Courier New, Courier, monospace;">proj2dev</span>, <span style="font-family: Courier New, Courier, monospace;">proj2stage</span>, <span style="font-family: Courier New, Courier, monospace;">proj2prod</span> etc.</div>
<div>
<br /></div>
<div>
<b>Tip:</b> Since all containers in the same Rancher environment can by default connect to all other containers in that Rancher environment, having a project/infrastructure-environment combination as detailed above will provide good isolation and security from one project to another, and from one infrastructure environment to another within the same project. I recommend you become familiar with Rancher environments by reading more about them in the <a href="http://docs.rancher.com/rancher/v1.2/en/environments/">documentation</a>.</div>
<div>
<br /></div>
<div>
In what follows I'll assume the current environment is <span style="font-family: "courier new" , "courier" , monospace;">proj1dev</span>.</div>
<div>
<br /></div>
<div>
<b><span style="font-size: large;">Creating Rancher API key pairs</span></b></div>
<div>
<br /></div>
<div>
Within each environment, create an API key pair. Copy and paste the two keys (one access key and one secret access key) somewhere safe.<br />
<br /></div>
<div>
<b><span style="font-size: large;">Adding Rancher hosts</span></b></div>
<div>
<br /></div>
<div>
Within each environment, you need to add Rancher hosts. They are the compute nodes that will run the various Docker containers that you will orchestrate with Rancher. In my case, I provisioned two hosts per environment as EC2 instances running Docker.</div>
<div>
<br /></div>
<div>
In the Rancher UI, when you go to Infrastructure -> Hosts then click the Add Host button, you should see a <span style="font-family: "courier new" , "courier" , monospace;">docker run</span> command that you can run on each host in order to launch the Rancher Agent on that host. Something like this:</div>
<div>
<br /></div>
<div>
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;"><b># docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.0.2 http://your-rancher-server-name.example.com:8080/v1/scripts/5536854597A70149E388:1473267600000:rfQVqxXcvIPulNw72fUOQG66iGM</b></span></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">Note that you need to allow <b>UDP ports 500 and 4500</b> from each Rancher host to/from any other host and to/from the Rancher server. This is because Rancher uses IPSec tunnels for inter-host communication. The Rancher hosts also need to talk to the Rancher server over <b>port 8080</b> (or whatever port you have exposed for the Rancher server container).</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1" style="font-size: large;"><b>Adding ECR registries</b></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">We use ECR as our Docker registry. Within each environment, I had to add our ECR registry. In the Rancher UI, I went to Infrastructure -> Registries, then clicked Add Registry and chose Custom as the registry type. In the attribute fields, I specified:</span></div>
<div class="p1">
</div>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li>Address: <span id="docs-internal-guid-cce51a89-05c8-ca72-8f90-9f46d4a1c69b"><span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com</span></span></span></li>
<li><span style="font-family: "arial"; font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">Email: </span><span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">none</span></span></li>
<li><span style="font-family: "arial"; font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">Username: </span><span style="font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">AWS</span></span></li>
<li><span style="font-family: "arial"; font-size: 14.6667px; vertical-align: baseline; white-space: pre-wrap;">Password: the result of running these commands (you need to install and configure the awscli for this to work):</span></li>
<ul>
<li><span style="font-family: "courier new"; font-size: 14.6667px; line-height: 1.38; white-space: pre-wrap;">apt-get install python-pip; pip install awscli</span></li>
<li><span style="font-family: "courier new"; font-size: 14.6667px; line-height: 1.38; white-space: pre-wrap;">aws configure (specify the keys for an IAM user allowed to access the ECR registry)</span></li>
<li><span style="font-size: 14.6667px; line-height: 1.38; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"><b>aws ecr get-login | cut -d ' ' -f 6</b></span></span></li>
</ul>
</ul>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: large; line-height: 20.24px; white-space: pre-wrap;"><b>Application architecture</b></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">For this example I will consider an application composed of a Web application based on Apache/PHP running in 2 or more containers and mounting its shared files (configuration, media) over NFS. The Web app talks to a MySQL database server mounting its data files over NFS. The Web app containers are behind one or more instances of a Rancher load balancer, and the Rancher LB instances are fronted by an Amazon Elastic Load Balancer. </span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: large; line-height: 20.24px; white-space: pre-wrap;"><b>Rancher stacks</b></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">A 'stack' in Rancher corresponds to a set of services defined in a </span><span style="font-family: "courier new" , "courier" , monospace;">docker-compose</span><span style="font-family: "arial";"> YAML file. These services can also have Rancher-specific attributes (</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">such as desired number of containers aka 'scale', health checks, etc) </span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">defined in a special rancher-compose YAML file. I'll show plenty of examples of these files in what follows. </span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">My stack naming convention will be </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">projname-environment-stacktype</span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">, for example </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">proj1-development-nfs</span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">, </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">proj1--development-database </span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">etc.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><b>Tip</b>: Try to experiment with creating stacks in the Rancher UI, then either view or export their configurations via the stack settings button in the UI:</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiO5a04vHPH0LOgoFwVHnWjla0dMd0lTcIeNQSKEXgqT6gWdlBXa7X8EDmmlamrQFqIdzmFMfvcrFyZEHeRf4xlJXBTZZ53wh6wnoRW9R4rZ4QwwP9tQEDYI5DjoNPXp1s1jp_tEA/s1600/Screen+Shot+2016-09-07+at+12.47.53+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiO5a04vHPH0LOgoFwVHnWjla0dMd0lTcIeNQSKEXgqT6gWdlBXa7X8EDmmlamrQFqIdzmFMfvcrFyZEHeRf4xlJXBTZZ53wh6wnoRW9R4rZ4QwwP9tQEDYI5DjoNPXp1s1jp_tEA/s320/Screen+Shot+2016-09-07+at+12.47.53+PM.png" width="320" /></a></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">This was a life saver for me especially when it comes to lower-level stacks such as NFS or Rancher load balancers. Exporting the configuration will download a zip file containing two files: </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">docker-compose.yml</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> and </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">rancher-compose.yml</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">. It will save you from figuring out on your own the exact syntax you need to use in these files.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: large; line-height: 20.24px; white-space: pre-wrap;"><b>Creating an NFS stack</b></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">One of the advantages of using Rancher is that it offers an extensive catalog of services ready to be used within your infrastructure. One such service is Convoy NFS. To use it, I started out by going to the Catalog menu option in the Rancher UI, then selecting Convoy NFS. In the following screen I specified </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">proj1-development-nfs</span><span style="font-family: "arial";"> as the stack name, as well as the NFS server's IP address and mount point.</span></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjn4Cx_2keUv2MSpTRpMB5uNmn9Nsvn3nqKhcOpRozDd8cXZDjvJ4eMPlOw_usLyBtps3YD_bLPE_nne4VF9L9OcxO3aejYCIJlfo9hMdlEn8iFqeUy6mcOR9NdKXJzbXSviCVemQ/s1600/Screen+Shot+2016-09-07+at+1.02.51+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="186" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjn4Cx_2keUv2MSpTRpMB5uNmn9Nsvn3nqKhcOpRozDd8cXZDjvJ4eMPlOw_usLyBtps3YD_bLPE_nne4VF9L9OcxO3aejYCIJlfo9hMdlEn8iFqeUy6mcOR9NdKXJzbXSviCVemQ/s320/Screen+Shot+2016-09-07+at+1.02.51+PM.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">Note that I had already set up an EC2 instance to act as an NFS server. I attached an EBS volume per project/environment. So in the example above, I exported a directory called </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">/nfs/development/proj1</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">.</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">After launching the NFS stack, you should see it in the Stacks screen in the Rancher UI. The stack will consist of 2 services, one called </span><span style="font-family: "courier new" , "courier" , monospace;">convoy-nfs</span><span style="font-family: "arial";"> and the other called </span><span style="font-family: "courier new" , "courier" , monospace;">convoy-nfs-storagepool</span><span style="font-family: "arial";">:</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5yPbxPHptrvfFw_Cn7i65NknIe-Ztr1Zoa7K1E0F30WPmeRdWEGZRknL95PlZ6gkeKyvK_i8mwpkZY97gDKPlC0bWfEyPDW3_8KorKRFKNf_NTaHQPP7lfhBkxKKraRBZc9t3Bw/s1600/Screen+Shot+2016-09-07+at+1.04.52+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="21" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5yPbxPHptrvfFw_Cn7i65NknIe-Ztr1Zoa7K1E0F30WPmeRdWEGZRknL95PlZ6gkeKyvK_i8mwpkZY97gDKPlC0bWfEyPDW3_8KorKRFKNf_NTaHQPP7lfhBkxKKraRBZc9t3Bw/s320/Screen+Shot+2016-09-07+at+1.04.52+PM.png" width="320" /></a></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
Once the NFS stack is up and running, you can export its configuration as explained above.<br />
<div>
<br /></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">To create or update a stack programmatically, I used the </span><a href="http://docs.rancher.com/rancher/v1.2/en/cattle/rancher-compose/" style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">rancher-compose</a><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";"> utility and wrapped it inside shell scripts. Here is an example of a shell script that calls </span><span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span><span style="font-family: "arial";"> to create an NFS stack:</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat rancher-nfssetup.sh</b></span></div>
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash<br /><br />COMMAND=$@<br /><br />rancher-compose -p <b>proj1-development-nfs</b> --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-nfssetup.yml --rancher-file rancher-compose.yml $COMMAND</span><br />
<div style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<br /></div>
<div style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
Note that there is no command line option for the target Rancher environment. It suffices to use the Rancher API keys for a given environment in order to target that environment.<br />
<br />
Here is the docker-compose file for this stack, which I obtained by exporting the stack configuration from the UI:</div>
<div style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<br /></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat docker-compose-nfssetup.yml</b>
convoy-nfs-storagepool:
labels:
io.rancher.container.create_agent: 'true'
command:
- storagepool-agent
image: rancher/convoy-agent:v0.9.0
volumes:
- /var/run:/host/var/run
- /run:/host/run
convoy-nfs:
labels:
io.rancher.scheduler.global: 'true'
io.rancher.container.create_agent: 'true'
command:
- volume-agent-nfs
image: rancher/convoy-agent:v0.9.0
pid: host
privileged: true
volumes:
- /lib/modules:/lib/modules:ro
- /proc:/host/proc
- /var/run:/host/var/run
- /run:/host/run
- /etc/docker/plugins:/etc/docker/plugins</span></span></div>
<div style="font-family: Arial;">
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">Here is the portion of my </span><span style="font-family: "courier new" , "courier" , monospace;">rancher-compose.yml</span><span style="font-family: "arial";"> file that has to do with the NFS stack, again obtained by exporting the NFS stack configuration:</span></span></div>
<div style="font-family: Arial;">
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">convoy-nfs-storagepool:
scale: 1
health_check:
port: 10241
interval: 2000
unhealthy_threshold: 3
strategy: recreate
response_timeout: 2000
request_line: GET /healthcheck HTTP/1.0
healthy_threshold: 2
metadata:
mount_dir: /nfs/development/proj1
nfs_server: 172.31.41.108
convoy-nfs:
health_check:
port: 10241
interval: 2000
unhealthy_threshold: 3
strategy: recreate
response_timeout: 2000
request_line: GET /healthcheck HTTP/1.0
healthy_threshold: 2
metadata:
mount_dir: /nfs/development/proj1
nfs_server: 172.31.41.108
mount_opts: ''</span></span></div>
<br />
<br />
To create the NFS stack, all I need to do at this point is to call:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ ./rancher-nfssetup.sh up</span><br />
<br />
To inspect the logs for the stack, I can call:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ ./rancher-nfssetup.sh logs</span><br />
<div style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<br /></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";">Note that I passed various arguments to the rancher-compose utility. Most of them are specified as environment variables. This allows me to add the bash script to version control without worrying about credentials, secrets etc. I also use the </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px;">--env-file .envvars </span><span style="font-size: 14.6667px; line-height: 20.24px;"><span style="font-family: "arial";">option, which allows me to define environment variables in the </span><span style="font-family: "courier new" , "courier" , monospace;">.envvars</span><span style="font-family: "arial";"> file and have them interpolated by </span><span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span><span style="font-family: "arial";"> in the various yml files it uses. </span></span></div>
<div style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<br /></div>
<div>
<div style="font-family: times; line-height: normal; white-space: normal;">
<span style="font-family: "arial"; line-height: 20.24px; white-space: pre-wrap;"><b>Creating volumes using the NFS stack</b></span></div>
<div style="font-family: Times; font-size: medium; line-height: normal; white-space: normal;">
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div style="font-size: medium; line-height: normal; white-space: normal;">
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">One of my goals was to attach NFS-based volumes to Docker containers in my infrastructure. To do this, I needed to create volumes in Rancher. One way to do it is to go to Infrastructure -> Storage in the Rancher UI, then go to the area corresponding to the NFS stack you want and click Add Volume, giving the volume a name and a description. Doing it manually is well and good, but I wanted to do it automatically, so I used another bash script around </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> together with another docker-compose file:</span></div>
<div style="font-family: Times; font-size: medium; line-height: normal; white-space: normal;">
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat rancher-volsetup.sh</b>
#!/bin/bash
COMMAND=$@
rancher-compose -p <b>proj1-development-volsetup</b> --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-volsetup.yml --rancher-file rancher-compose.yml $COMMAND</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"></span></span><br />
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: 14.6667px; line-height: 20.24px;"><b>$ cat docker-compose-volsetup.yml</b></span></span></span></div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">
</span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">volsetup:
image: ubuntu:14.04
labels:
io.rancher.container.start_once: true
volumes:
- volMysqlData:/var/lib/mysql
- volAppShared:/var/www/shared
volume_driver: proj1-development-nfs</span></span></div>
</div>
</div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">A few things to note in the </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">docker-compose-volsetup.yml</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> file:</span></div>
<div>
<ul>
<li><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">I used the </span><span style="font-family: "courier new" , "courier" , monospace;">ubuntu:14.04</span><span style="font-family: "arial";"> Docker image and I attached two volumes, one called </span><span style="font-family: "courier new" , "courier" , monospace;">volMysqlData</span><span style="font-family: "arial";"> and once called </span><span style="font-family: "courier new" , "courier" , monospace;">volAppSharedData</span><span style="font-family: "arial";">. The first one will be mounted on the Docker container as </span><span style="font-family: "courier new" , "courier" , monospace;">/var/lib/mysql</span><span style="font-family: "arial";"> and the second one will be mounted as </span><span style="font-family: "courier new" , "courier" , monospace;">/var/www/shared</span><span style="font-family: "arial";">. These are arbitrary paths, since my goal was just to create the volumes as Rancher resources.</span></span></li>
<li><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">I wanted the </span><span style="font-family: "courier new" , "courier" , monospace;">volsetup</span><span style="font-family: "arial";"> service to run once so that the volumes get created, then stop. For that, I used the special Rancher label </span></span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">io.rancher.container.start_once: true</span></li>
<li>I used as the <span style="font-family: "courier new" , "courier" , monospace;">volume_driver</span> the NFS stack <span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">proj1-development-nfs </span>I created above. This is important, because I want these volumes to be created within this NFS stack.</li>
</ul>
</div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">I used the following commands to create and start the </span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">proj1-development-volsetup</span></span><span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> stack, then to show its logs, and finally to shut it down and remove its containers, which are not needed anymore once the volumes get created:</span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"> </span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">./rancher-volsetup.sh up -d
sleep 30
./rancher-volsetup.sh logs
./rancher-volsetup.sh down
./rancher-volsetup.sh rm --force</span></span></div>
<div>
<span style="font-family: "arial"; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">I haven't figured out yet how to remove a Rancher stack programmatically, so for these 'helper' type stacks I had to use the Rancher UI to delete them.</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
At this point, if you look in the /nfs/development/proj1 directory on the NFS server, you should see 2 directories with the same names as the volumes we created.</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: large; line-height: 20.24px; white-space: pre-wrap;"><b>Creating a database stack</b></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">So far I haven't used any custom Docker images. For the database layer of my application, I will want to use a custom image which I will push to the Amazon ECR registry. I will use this image in a docker-compose file in order to set up and start the database in Rancher.</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">I have a directory called db containing the following </span><span style="font-family: "courier new" , "courier" , monospace;">Dockerfile</span><span style="font-family: "arial";">:</span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ cat Dockerfile<br />FROM percona<br /><br />VOLUME /var/lib/mysql<br /><br />COPY etc/mysql/my.cnf /etc/mysql/my.cnf<br />COPY scripts/db_setup.sh /usr/local/bin/db_setup.sh</span><br />
<div style="font-family: Arial; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<br /></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";">I have a customized MySQL configuration file </span><span style="font-family: "courier new" , "courier" , monospace;">my.cnf</span><span style="font-family: "arial";"> (in my local directory </span><span style="font-family: "courier new" , "courier" , monospace;">db/etc/mysql</span><span style="font-family: "arial";">) which gets copied to the Docker image as </span><span style="font-family: "courier new" , "courier" , monospace;">/etc/mysql.my.cnf</span><span style="font-family: "arial";">. I also have a </span><span style="font-family: "courier new" , "courier" , monospace;">db_setup.sh</span><span style="font-family: "arial";"> bash script in my local directory </span><span style="font-family: "courier new" , "courier" , monospace;">db/scripts</span><span style="font-family: "arial";"> which gets copied to </span><span style="font-family: "courier new" , "courier" , monospace;">/usr/local/bin</span><span style="font-family: "arial";"> in the Docker image. In this script I grant rights to a MySQL user used by the Web app, and I also load a MySQL dump file if it exists:</span></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">$ cat scripts/db_setup.sh</b>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">#</span></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">!/bin/bash
set -e
host="$1"
until mysql -h "$host" -uroot -p$MYSQL_ROOT_PASSWORD -e "SHOW DATABASES"; do
>&2 echo "MySQL is unavailable - sleeping"
sleep 1
done
>&2 echo "MySQL is up - executing GRANT statement"
mysql -h "$host" -uroot -p$MYSQL_ROOT_PASSWORD \
-e "GRANT ALL ON $MYSQL_DATABASE.* TO $MYSQL_USER@'%' IDENTIFIED BY \"$MYSQL_PASSWORD\""
>&2 echo "Starting to load SQL dump"
mysql -h "$host" -uroot -p$MYSQL_ROOT_PASSWORD $MYSQL_DATABASE < /dbdump/$MYSQL_DUMP_FILE
>&2 echo "Finished loading SQL dump"</span></span></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";"><br /></span></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";">Note that the database name, database user name and password, as well as the MySQL root password are all passed in environment variables.</span></div>
</div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";"><br /></span></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";">To build this Docker image, I ran:</span></div>
<div style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">$ docker build -t </span><span style="font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com</span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">/db:proj1-development .</span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "arial";">Note that I tagged the image with the </span><span style="font-family: "courier new" , "courier" , monospace;">proj1-development</span><span style="font-family: "arial";"> tag.</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">To push this image to Amazon ECR, I first called: </span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><span style="font-family: "courier new" , "courier" , monospace;">$(aws get-login)</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">then:</span></span></div>
<div>
<span style="font-family: "arial";"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;"><br /></span></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">$ docker push </span><span style="font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com</span><span style="font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">/db:proj1-development</span></span></div>
<br />
To run the <span style="font-family: "courier new" , "courier" , monospace;">db_setup.sh</span> script inside a Docker container in order to set up the database, I put together the following docker-compose file:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat docker-compose-dbsetup.yml</b></span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">ECRCredentials:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_REGION: $AWS_REGION</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.create_agent: 'true'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.agent.role: environment</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.start_once: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> tty: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> image: objectpartners/rancher-ecr-credentials</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> stdin_open: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">db:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> image: </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com</span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">/db:proj1-development</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.scheduler.affinity:host_label: dbsetup=proj1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> volumes:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - volMysqlData:/var/lib/mysql</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> volume_driver: proj1-development-nfs</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">dbsetup:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> image: </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com</span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">/db:proj1-development</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.start_once: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.scheduler.affinity:host_label: dbsetup=proj1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> command: /usr/local/bin/db_setup.sh db</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> links:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - db:db</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> volumes:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - volMysqlData:/var/lib/mysql</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - /dbdump/proj1:/dbdump</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> volume_driver: proj1-development-nfs</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - MYSQL_DATABASE=$MYSQL_DATABASE</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - MYSQL_USER=$MYSQL_USER</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - MYSQL_PASSWORD=$MYSQL_PASSWORD</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - MYSQL_DUMP_FILE=$MYSQL_DUMP_FILE</span></div>
<br />
A few things to note:</div>
<div>
<ul>
<li>there are 3 services in this docker-compose file</li>
<ul>
<li>a <span style="font-family: "courier new" , "courier" , monospace;">ECRCredentials </span>service which connects to Amazon ECR and allows the ECR image <span style="font-family: "courier new" , "courier" , monospace;">db:proj1-development</span> to be used by the other 2 services</li>
<li>a <span style="font-family: "courier new" , "courier" , monospace;">db</span> service which runs a Docker container based on the <span style="font-family: "courier new" , "courier" , monospace;">db:proj1-development </span>ECR image, and which launches a MySQL database with the root password set to the value of the <span style="font-family: "courier new" , "courier" , monospace;">MYSQL_ROOT_PASSWORD </span>environment variable</li>
<li>a <span style="font-family: "courier new" , "courier" , monospace;">dbsetup</span> service that also runs a Docker container based on the <span style="font-family: "courier new" , "courier" , monospace;">db:proj1-development </span>ECR image, but instead of the default command, which would run MySQL, it runs the <span style="font-family: "courier new" , "courier" , monospace;">db_setup.sh</span> script (specified in the <span style="font-family: "courier new" , "courier" , monospace;">command</span> directive); this service also uses environment variables specifying the database to be loaded from the SQL dump file, as well as the user and password that will get grants to that database</li>
</ul>
<li>the <span style="font-family: "courier new" , "courier" , monospace;">dbsetup</span> service links to the <span style="font-family: "courier new" , "courier" , monospace;">db</span> service via the <span style="font-family: "courier new" , "courier" , monospace;">links</span> directive</li>
<li>the dbsetup service is a 'run once then stop' type of service, which is why it has the label <span style="font-family: "courier new" , "courier" , monospace;">io.rancher.container.start_once: true </span>attached</li>
<li>both the <span style="font-family: "courier new" , "courier" , monospace;">db</span> and the <span style="font-family: "courier new" , "courier" , monospace;">dbsetup</span> service will run on a Rancher host with the label '<span style="font-family: "courier new" , "courier" , monospace;">dbsetup=proj1</span>'; this is because we want to load the SQL dump from a file that the <span style="font-family: "courier new" , "courier" , monospace;">dbsetup</span> service can find</li>
<ul>
<li>we will put this file on a specific Rancher host in a directory called <span style="font-family: "courier new" , "courier" , monospace;">/dbdump/proj1</span>, which will then be mounted by the <span style="font-family: "courier new" , "courier" , monospace;">dbsetup</span> container as <span style="font-family: "courier new" , "courier" , monospace;">/dbdump</span></li>
<li>the <span style="font-family: "courier new" , "courier" , monospace;">db_setup.sh</span> script will then load the SQL file called <span style="font-family: "courier new" , "courier" , monospace;">MYSQL_DUMP_FILE </span>from the<span style="font-family: "courier new" , "courier" , monospace;"> /dbdump </span>directory</li>
<li>this can also work if we'd just put the SQL file in the same NFS volume as the MySQL data files, but I wanted to experiment with host labels in this case</li>
</ul>
<li>wherever NFS volumes are used, for example for <span style="font-family: "courier new" , "courier" , monospace;">volMysqlData</span>, the <span style="font-family: "courier new" , "courier" , monospace;">volume_driver</span> needs to be set to the proper NFS stack, <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-nfs </span>in this case</li>
</ul>
<div>
It goes without saying that mounting the MySQL data files from NFS is a potential performance bottleneck, so you probably wouldn't do this in production. I wanted to experiment with NFS in Rancher, and the performance I've seen in development and staging for some of our projects doesn't seem too bad.</div>
<div>
<br /></div>
<div>
To run a Rancher stack based on this <span style="font-family: "courier new" , "courier" , monospace;">docker-compose-dbsetup.yml</span> file, I used this bash script:</div>
</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ cat rancher-dbsetup.sh</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">COMMAND=$@</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">rancher-compose -p <b>proj1-development-dbsetup</b> --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-dbsetup.yml --rancher-file rancher-compose.yml $COMMAND</span></div>
</div>
<div>
<br /></div>
<div>
Note that all environment variables referenced in the <span style="font-family: "courier new" , "courier" , monospace;">docker-compose-dbsetup.yml</span> file are set in the <span style="font-family: "courier new" , "courier" , monospace;">.envvars</span> file.</div>
<div>
<br /></div>
<div>
I wanted to run the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-dbsetup</span> stack and then shut down its services once the <span style="font-family: "courier new" , "courier" , monospace;">dbsetup</span> service completes. I used these commands as part of a bash script:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-dbsetup.sh up -d</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">while :</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">do</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ./rancher-dbsetup.sh logs --lines "10" > dbsetup.log 2>&1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> grep 'Finished loading SQL dump' dbsetup.log</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> result=$?</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> if [ $result -eq 0 ]; then</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> break</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> fi</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> echo Waiting 10 seconds for DB load to finish...</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> sleep 10</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">done</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-dbsetup.sh logs</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-dbsetup.sh down</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-dbsetup.sh rm --force</span></div>
<div>
<br /></div>
</div>
<div>
Once the database is setup, I want to launch MySQL and keep it running so it can be used by the Web application. I have a separate docker-compose file for that:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat docker-compose-dblaunch.yml</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">ECRCredentials:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_REGION: $AWS_REGION</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.create_agent: 'true'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.agent.role: environment</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.start_once: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> tty: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> image: objectpartners/rancher-ecr-credentials</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> stdin_open: true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">db:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> image: </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com</span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; line-height: 20.24px; white-space: pre-wrap;">/db:proj1-development</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> volumes:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - volMysqlData:/var/lib/mysql</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> volume_driver: proj1-development-nfs</span></div>
</div>
<div>
<br /></div>
<div>
The db service is similar to the one in the docker-compose-dbsetup.yml file. In this case the database is all set up, so we don't need anything except the NFS volume to mount the MySQL data files from.</div>
<div>
<br /></div>
<div>
As usual, I have a bash script that calls docker-compose in order to create a stack called <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-database</span>:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ cat rancher-dblaunch.sh</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">COMMAND=$@</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">rancher-compose -p <b>proj1-development-database</b> --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-dblaunch.yml --rancher-file rancher-compose.yml $COMMAND</span></div>
</div>
<div>
<br /></div>
<div>
I call this script like this:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-dblaunch.sh up -d</span></div>
<div>
<br /></div>
<div>
At this point, the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-database</span> stack is up and running and contains the <span style="font-family: "courier new" , "courier" , monospace;">db</span> service running as a container on one of the Rancher hosts in the Rancher '<span style="font-family: "courier new" , "courier" , monospace;">proj1dev</span>' environment.</div>
<div>
<br />
<b><span style="font-size: large;">Creating a Web application stack</span></b><br />
<br />
So far, I've been using either off-the-shelf or slightly customized Docker images. For the Web application stack I will be using more heavily customized images. The building block is a '<span style="font-family: "courier new" , "courier" , monospace;">base</span>' image whose Dockerfile contains directives for installing commonly used packages and for adding users.<br />
<br />
<b>Here is the Dockerfile for a 'base' image running Ubuntu 14.04:</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">FROM ubuntu:14.04</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">RUN apt-get update && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> apt-get install -y ntp build-essential build-essential binutils zlib1g-dev \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> git acl cronolog lzop unzip mcrypt expat xsltproc python-pip curl language-pack-en-base</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN pip install awscli</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">RUN adduser --ui 501 --ingroup www-data --shell /bin/bash --home /home/myuser myuser</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN mkdir /home/myuser/.ssh</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">COPY files/myuser_authorized_keys /home/myuser/.ssh/authorized_keys</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN chown -R myuser:www-data /home/myuser/.ssh && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> chmod 700 /home/myuser/.ssh && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> chmod 600 /home/myuser/.ssh/authorized_keys </span><br />
<br />
When I built this image, I tagged it as <b><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/</span><span style="font-family: "courier new" , "courier" , monospace;">base:proj1-development</span></b>.<br />
<b><br /></b>
<b>Here is the Dockerfile for an image (based on the <span style="font-family: "courier new" , "courier" , monospace;">base</span> image above) that installs Apache, PHP 5.6 (using a custom apt repository), RVM, Ruby and the <span style="font-family: "courier new" , "courier" , monospace;">compass</span> gem:</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">FROM </span> <span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/</span><span style="font-family: "courier new" , "courier" , monospace;">base:proj1-development</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">RUN export LC_ALL=en_US.UTF-8 && export LC_ALL=en_US.UTF-8 && export LANG=en_US.UTF-8 && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>apt-get install -y mysql-client-5.6 software-properties-common && add-apt-repository ppa:ondrej/php5-5.6</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">RUN apt-get update && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> apt-get install -y --allow-unauthenticated apache2 apache2-utils libapache2-mod-php5 \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> php5 php5-mcrypt php5-curl php-pear php5-gd \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> php5-dev php5-mysql php5-readline php5-xsl php5-xmlrpc php5-intl</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># Install composer</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN curl -sSL https://getcomposer.org/composer.phar -o /usr/bin/composer \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> && chmod +x /usr/bin/composer \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> && composer selfupdate</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># Install rvm and compass gem for SASS image compilation</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN curl https://raw.githubusercontent.com/rvm/rvm/master/binscripts/rvm-installer -o /tmp/rvm-installer.sh && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>chmod 755 /tmp/rvm-installer.sh && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>gpg --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3 && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/tmp/rvm-installer.sh stable --path /home/myuser/.rvm --auto-dotfiles --user-install && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/bin/rvm get stable && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/bin/rvm reload && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/bin/rvm autolibs 3</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">RUN /home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/bin/rvm install ruby-2.2.2 && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/bin/rvm alias create default ruby-2.2.2 && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/wrappers/ruby-2.2.2/gem install bundler && \</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>/home/</span><span style="font-family: "courier new" , "courier" , monospace;">myuser</span><span style="font-family: "courier new" , "courier" , monospace;">/.rvm/wrappers/ruby-2.2.2/gem install compass</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">COPY files/apache2-foreground /usr/local/bin/</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">EXPOSE 80</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">CMD ["apache2-foreground"]</span><br />
<br />
When I built this image, I tagged it as <b> <span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/apache-php</span><span style="font-family: "courier new" , "courier" , monospace;">:proj1-development</span></b><br />
<br />
With these 2 images as building blocks, I put together 2 more images, one for building artifacts for the Web application, and one for launching it.<br />
<br />
<b>Here is the Dockerfile for an image that builds the artifacts for the Web application:</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">FROM </span><span style="font-family: "courier new" , "courier" , monospace; font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/apache-php</span><span style="font-family: "courier new" , "courier" , monospace;">:proj1-development</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">ADD ./scripts/app_setup.sh /usr/local/bin/</span><br />
<br />
The heavy lifting takes place in the <span style="font-family: "courier new" , "courier" , monospace;">app_setup.sh</span> script. That's where you would do things such as pull a specified git branch from application repo on GitHub, then run composer (if it's a PHP app) or other build tools in order to generate the artifacts necessary for running the application. At the end of this script, I generate a tar.gz of the code + any artifacts and upload it to S3 so I can use it when I generate the Docker image for the Web app.<br />
<br />
When I built this image, I tagged it as <b> <span style="font-family: "courier new" , "courier" , monospace; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/appsetup</span><span style="font-family: "courier new" , "courier" , monospace;">:proj1-development</span></b><br />
<br />
To actually run a Docker container based on the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> image, I used this docker-compose file:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat docker-compose-appsetup.yml</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">ECRCredentials:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_REGION: $AWS_REGION</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.create_agent: 'true'</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.agent.role: environment</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.start_once: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> tty: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: objectpartners/rancher-ecr-credentials</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> stdin_open: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">appsetup:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: <span style="font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/appsetup</span>:proj1-development</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> command: /usr/local/bin/app_setup.sh</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> external_links:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - proj1-development-database/db:db</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volumes:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - volAppShared:/var/www/shared</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volume_driver: proj1-development-nfs</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - GIT_URL=$GIT_URL</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - GIT_BRANCH=$GIT_BRANCH</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - AWS_S3_REGION=$AWS_S3_REGION</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - AWS_S3_ACCESS_KEY_ID=$AWS_S3_ACCESS_KEY_ID</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - AWS_S3_SECRET_ACCESS_KEY=$AWS_S3_SECRET_ACCESS_KEY</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - AWS_S3_RELEASE_BUCKET=$AWS_S3_RELEASE_BUCKET</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - AWS_S3_RELEASE_FILENAME=$AWS_S3_RELEASE_FILENAME</span><br />
<br />
Some things to note:<br />
<ul>
<li>the command executed when a Docker container based on the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service is launched is <span style="font-family: "courier new" , "courier" , monospace;">/usr/local/bin/app_setup.sh</span>, as specified in the <span style="font-family: "courier new" , "courier" , monospace;">command</span> directive</li>
<ul>
<li>the <span style="font-family: "courier new" , "courier" , monospace;">app_setup.sh</span> script runs commands that connect to the database, hence the need for the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service to link to the MySQL database running in the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-database</span> stack launched above; for that, I used the <span style="font-family: "courier new" , "courier" , monospace;">external_links</span> directive</li>
</ul>
<li>the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service mounts an NFS volume (<span style="font-family: "courier new" , "courier" , monospace;">volAppShared</span>) as <span style="font-family: "courier new" , "courier" , monospace;">/var/www/shared</span></li>
<ul>
<li>the <span style="font-family: "courier new" , "courier" , monospace;">volume_driver</span> needs to be <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-nfs</span></li>
<li>before running the service, I created proper application configuration files under <span style="font-family: "courier new" , "courier" , monospace;">/nfs/development/proj1/volAppShared</span> on the NFS server, specifying things such as the database server name (which needs to be '<span style="font-family: "courier new" , "courier" , monospace;">db</span>', since this is how the database container is linked as), the database name, user name and password, etc.</li>
</ul>
<li>the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service uses various environment variables referenced in the <span style="font-family: "courier new" , "courier" , monospace;">environment</span> directive; it will pass these variables to the <span style="font-family: "courier new" , "courier" , monospace;">app_setup.sh</span> script</li>
</ul>
<div>
To run the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service, I used another bash script around the <span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span> command:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat rancher-appsetup.sh</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">COMMAND=$@</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">rancher-compose -p proj1-development-appsetup --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-appsetup.yml --rancher-file rancher-compose.yml $COMMAND</span></div>
<div>
<br /></div>
<div>
<br /></div>
<b>Tip</b>: When using its Cattle cluster management engine, Rancher does not add services linked to each other as static entries in <span style="font-family: "courier new" , "courier" , monospace;">/etc/hosts</span> on the containers. Instead, it provides an <a href="http://docs.rancher.com/rancher/v1.1/zh/cattle/internal-dns-service/">internal DNS service</a> so that containers in the same environment can reach each other by DNS names as long as they link to each other in docker-compose files. If you go to a shell prompt inside a container, you can ping other containers by name even from one Rancher stack to another. For example, from a web container in the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-app</span> stack you can ping a database container in the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-database</span> stack linked in the docker-compose file as <span style="font-family: "courier new" , "courier" , monospace;">db</span> and you would get back a name of the type <span style="font-family: "courier new" , "courier" , monospace;">db.proj1-development-app.rancher.internal</span>.</div>
<div>
<br /></div>
<div>
<b>Tip</b>: There is no need to expose ports from containers within the same Rancher environment. I spent many hours troubleshooting issues related to ports and making sure ports are unique across stacks, only to realize that the internal ports that the services listen on (3306 for MySQL, 80 and 443 for Apache) are reachable from the other containers in the same Rancher environment. The only ports you need exposed to the external world in the architecture I am describing are the load balancer ports, as I'll describe below.</div>
<div>
<br /></div>
<div>
<b>Here is the Dockerfile for an image that runs the Web application:</b></div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">FROM </span><span style="font-size: 14.6667px; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/apache-php</span><span style="font-family: "courier new" , "courier" , monospace;">:proj1-development</span></span></div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /># disable interactive functions<br />ARG DEBIAN_FRONTEND=noninteractive<br /><br />RUN a2enmod headers \<br /> && a2enmod rewrite \<br /> && a2enmod ssl<br /><br />RUN rm -rf /etc/apache2/ports.conf /etc/apache2/sites-enabled/*<br />ADD etc/apache2/sites-enabled /etc/apache2/sites-enabled<br />ADD etc/apache2/ports.conf /etc/apache2/ports.conf<br /><br />ADD release /var/www/html/release<br />RUN chown -R myuser:www-data /var/www/html/release</span></div>
<div>
<br />
This image is based on the <span style="font-family: "courier new" , "courier" , monospace;">apache-php</span> image but adds Apache customizations, as well as the release directory obtained from the tar.gz file uploaded to S3 by the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service.<br />
<br />
When I built this image, I tagged it as <b> <span style="font-family: "courier new" , "courier" , monospace; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/app</span><span style="font-family: "courier new" , "courier" , monospace;">:proj1-development</span></b><br />
<br />
<span style="font-size: large;"><b>Code deployment</b></span><br />
<br />
My code deployment process is a bash script (which can be used standalone, or as part of a Jenkins job, or can be turned into a Jenkins pipeline) that first runs the <span style="font-family: "courier new" , "courier" , monospace;">appsetup</span> service in order to generate a tar.gz of the code and artifacts, then downloads it from S3 and uses it as the local <span style="font-family: "courier new" , "courier" , monospace;">release</span> directory to be copied into the <span style="font-family: "courier new" , "courier" , monospace;">app</span> image. The script then pushes the <span style="font-family: "courier new" , "courier" , monospace;">app</span> Docker image to Amazon ECR. The environment variables are either defined in an <span style="font-family: "courier new" , "courier" , monospace;">.envvars</span> file or passed via Jenkins parameters. The script assumes that the <span style="font-family: "courier new" , "courier" , monospace;">Dockerfile</span> for the <span style="font-family: "courier new" , "courier" , monospace;">app</span> image is in the current directory, and that the <span style="font-family: "courier new" , "courier" , monospace;">etc</span> directory structure used for the Apache files in the <span style="font-family: "courier new" , "courier" , monospace;">app</span> image is also in the current directory (they are all checked into the project repository, so Jenkins will find them).<br />
<br />
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-appsetup.sh up -d</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">sleep 20</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">cp /dev/null appsetup.log</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">while :</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">do</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> ./rancher-appsetup.sh logs >> appsetup.log 2>&1</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> grep 'Restarting web server apache2' appsetup.log</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> result=$?</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> if [ $result -eq 0 ]; then</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> break</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> fi</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> echo Waiting 10 seconds for app code deployment to finish...</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> sleep 10</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">done</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-appsetup.sh logs</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-appsetup.sh down</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">./rancher-appsetup.sh rm --force</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># download release.tar.gz from S3 and unpack it</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">set -a</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">. .envvars</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">set +a</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">export AWS_ACCESS_KEY_ID=$AWS_S3_ACCESS_KEY_ID</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">export AWS_SECRET_ACCESS_KEY=$AWS_S3_SECRET_ACCESS_KEY</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">rm -rf $AWS_S3_RELEASE_FILENAME.tar.gz</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">aws s3 --region $AWS_S3_REGION ls s3://$AWS_S3_RELEASE_BUCKET/</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">aws s3 --region $AWS_S3_REGION cp s3://$AWS_S3_RELEASE_BUCKET/$AWS_S3_RELEASE_FILENAME.tar.gz .</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">tar xfz $AWS_S3_RELEASE_FILENAME.tar.gz</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># build app docker image and push it to ECR</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">cat << "EOF" > awscreds</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">[default]</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">aws_access_key_id=$AWS_ACCESS_KEY_ID</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">aws_secret_access_key=$AWS_SECRET_ACCESS_KEY</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">export AWS_SHARED_CREDENTIALS_FILE=./awscreds</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"> $(aws ecr --region=$AWS_REGION get-login)</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">/usr/bin/docker build -t </span><span style="font-family: "courier new" , "courier" , monospace; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/</span><span style="font-family: "courier new" , "courier" , monospace;">app:proj1-development .</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">/usr/bin/docker push </span><span style="font-family: "courier new" , "courier" , monospace; white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/</span><span style="font-family: "courier new" , "courier" , monospace;">app:proj1-development</span></div>
<br />
<div class="p2">
<br /></div>
<b><span style="font-size: large;">Launching the app service</span></b><br />
<br />
At this point, the Docker image for the app service has been pushed to Amazon ECR, but the service itself hasn't been started. To do that, I use this docker-compose file:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat docker-compose-app.yml</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">ECRCredentials:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> environment:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_REGION: $AWS_REGION</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.create_agent: 'true'</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.agent.role: environment</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.start_once: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> tty: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: objectpartners/rancher-ecr-credentials</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> stdin_open: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">app:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: <span style="white-space: pre-wrap;">my_ecr_registry_id.dkr.ecr.my_region.amazonaws.com/</span>app:proj1-development</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.container.pull_image: always</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> external_links:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - proj1-development-database/db:db</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volumes:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - volAppShared:/var/www/shared</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> volume_driver: proj1-development-nfs</span><br />
<br />
Nothing very different about this file compare to the files I've shown so far. The <span style="font-family: "courier new" , "courier" , monospace;">app</span> service mounts the <span style="font-family: "courier new" , "courier" , monospace;">volAppShared</span> NFS volume as <span style="font-family: "courier new" , "courier" , monospace;">/var/www/shared</span>, and links to the MySQL database service <span style="font-family: "courier new" , "courier" , monospace;">db</span> already running in the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-database</span> Rancher stack, giving it the name '<span style="font-family: "courier new" , "courier" , monospace;">db</span>'.<br />
<br />
To run the app service, I use this bash script wrapping <span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cat rancher-app.sh</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">#!/bin/bash</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">COMMAND=$@</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">rancher-compose -p <b>proj1-development-app</b> --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-app.yml --rancher-file rancher-compose.yml $COMMAND</span><br />
<br />
Since the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-app</span> stack may already be running with an old version of the <span style="font-family: "courier new" , "courier" , monospace;">app</span> Docker image, I will invoke <span style="font-family: "courier new" , "courier" , monospace;">rancher-app.sh</span> with the <span style="font-family: "courier new" , "courier" , monospace;">force-upgrade</span> option of the <span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span> command:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span>
<span style="font-family: "courier new" , "courier" , monospace;"><b>./rancher-app.sh up -d --force-upgrade --confirm-upgrade --pull --batch-size "1"</b></span><br />
<br />
This will perform a <b>rolling upgrade </b>of the <span style="font-family: "courier new" , "courier" , monospace;">app</span> service, by stopping the containers for the <span style="font-family: "courier new" , "courier" , monospace;">app</span> service one at a time (as indicated by the<span style="font-family: "courier new" , "courier" , monospace;"> batch-size</span> parameter), then pulling the latest Docker image for the <span style="font-family: "courier new" , "courier" , monospace;">app</span> service, and finally starting each container again. Speaking of 'containers' plural, you can indicate how many containers should run at all times for the app service by adding these lines to <span style="font-family: "courier new" , "courier" , monospace;">rancher-compose.yml</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>app:</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b> scale: 2</b></span><br />
<br />
In my case, I want 2 containers to run at all times. If you stop one container from the Rancher UI, you will see another one restarted automatically by Rancher in order to preserve the value specified for the '<span style="font-family: "courier new" , "courier" , monospace;">scale</span>' parameter.<br />
<br />
<b><span style="font-size: large;">Creating a load balancer stack</span></b><br />
<br />
When I started to run load balancers in Rancher, I created them via the Rancher UI. I created a new stack, then added a load balancer service to it. It took me a while to figure out that I can then export the stack configuration and generate a <span style="font-family: "courier new" , "courier" , monospace;">docker-compose</span> file and a <span style="font-family: "courier new" , "courier" , monospace;">rancher-compose</span> snippet I can add to my main <span style="font-family: "courier new" , "courier" , monospace;">rancher-compose.yml</span> file.<br />
<br />
Here is the docker-compose file I use:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ cat docker-compose-lbsetup.yml</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">lb:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ports:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - 8000:80</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - 8001:443</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> external_links:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - proj1-development-app/app:app</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> labels:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> io.rancher.loadbalancer.ssl.ports: '8001'</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="s1">io.rancher.loadbalancer.target.proj1-development-app/app</span><span class="s2">:</span><span class="s3"> proj1.dev.mydomain.com</span><span class="s2">:</span><span class="s3">8000=80</span><span class="s2">,</span><span class="s3">8001=443</span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> tty: true</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> image: rancher/load-balancer-service</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> stdin_open: true</span><br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">ports</span> directive tell the load balancer which ports to expose externally and what ports to map them to. This example shows that port 8000 will be exposed externally and mapped to port 80 on the target service, and port 8001 will be exposed externally and mapped to port 443 on the target service.<br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">external_links</span> directive tells the load balancer which service to load balance. In this example, it is the <span style="font-family: "courier new" , "courier" , monospace;">app</span> service in the <span style="font-family: "courier new" , "courier" , monospace;">proj1-development-app </span>stack.<br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">labels</span> directive does layer 7 load balancing by allowing you to specify a domain name that you want to send to a specific port. In this example, I want to send HTTP requests coming on port 8000 for <span style="font-family: "courier new" , "courier" , monospace;">proj1.dev.mydomain.com</span> to port 80 on the target containers for the <span style="font-family: "courier new" , "courier" , monospace;">app</span> service, and HTTPS requests coming on port 8001 for the same <span style="font-family: "courier new" , "courier" , monospace;">proj1.dev.mydomain.com</span> name to port 443 on the target containers.<br />
<br />
I could have also added a new line under labels, specifying that I want requests for <span style="font-family: "courier new" , "courier" , monospace;">proj1-admin.dev.mydomain.com</span> coming on port 8000 to be sent to a<span style="font-family: "courier new" , "courier" , monospace;"> </span>different port on the target containers, assuming that I had Apache configured to listen on that port. You can read more about the load balancing features available in Rancher in the <a href="http://docs.rancher.com/rancher/v1.1/zh/cattle/adding-load-balancers/">documentation</a>.<br />
<div>
<br />
Here is the load balancer section in <span style="font-family: Courier New, Courier, monospace;">rancher-compose.yml</span>:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">lb:</span><br />
<span style="font-family: Courier New, Courier, monospace;"> scale: 2</span><br />
<span style="font-family: Courier New, Courier, monospace;"> load_balancer_config:</span><br />
<span style="font-family: Courier New, Courier, monospace;"> haproxy_config: {}</span><br />
<span style="font-family: Courier New, Courier, monospace;"> default_cert: proj1.dev.mydomain.com</span><br />
<span style="font-family: Courier New, Courier, monospace;"> health_check:</span><br />
<span style="font-family: Courier New, Courier, monospace;"> port: 42</span><br />
<span style="font-family: Courier New, Courier, monospace;"> interval: 2000</span><br />
<span style="font-family: Courier New, Courier, monospace;"> unhealthy_threshold: 3</span><br />
<span style="font-family: Courier New, Courier, monospace;"> healthy_threshold: 2</span><br />
<span style="font-family: Courier New, Courier, monospace;"> response_timeout: 2000</span><br />
<br />
Note that there is a mention of a <span style="font-family: Courier New, Courier, monospace;">default_cert</span>. This is an SSL key + cert that I uploaded to Rancher via the UI by going to Infrastructure -> Certificates and that I named <span style="font-family: 'Courier New', Courier, monospace;">proj1.dev.mydomain.com</span>. The Rancher Catalog does contain an integration for Let's Encrypt but I haven't had a chance to test it yet (from the Rancher Catalog: "<i>The Let's Encrypt Certificate Manager obtains a free (SAN) SSL Certificate from the <a href="https://letsencrypt.org/">Let's Encrypt CA</a> and adds it to Rancher's certificate store. Once the certificate is created it is scheduled for auto-renewal 14-days before expiration. The renewed certificate is propagated to all applicable load balancer services.</i>")<br />
<br />
Note also that the <span style="font-family: Courier New, Courier, monospace;">scale</span> value is 2, which means that there will be 2 containers for the <span style="font-family: Courier New, Courier, monospace;">lb</span> service.<br />
<br /></div>
<div>
<b>Tip</b>: In the Rancher UI, you can open a shell into any container, or view the logs for any container by going to the Settings icon of that container, and choosing Execute Shell or View Logs:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM8-ukSxhztyfGa2oK-dpjAl6KYwTR-iZmHM1fJ1H8yeMcZ2g87n2Dc5QvCko6p5HiKHhDXW8Cq-jZVFTHkI1mhkai8Bdnok6dz-shfwd9fcFsm2iogx7HXcT2SM0lbXK3F8x4Vw/s1600/Screen+Shot+2016-09-08+at+5.10.02+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM8-ukSxhztyfGa2oK-dpjAl6KYwTR-iZmHM1fJ1H8yeMcZ2g87n2Dc5QvCko6p5HiKHhDXW8Cq-jZVFTHkI1mhkai8Bdnok6dz-shfwd9fcFsm2iogx7HXcT2SM0lbXK3F8x4Vw/s1600/Screen+Shot+2016-09-08+at+5.10.02+PM.png" /></a></div>
<div>
<br /></div>
<div>
<b>Tip</b>: Rancher load balancers are based on haproxy. You can open a shell into a container running for the lb service, then look at the haproxy configuration file in <span style="font-family: "courier new" , "courier" , monospace;">/etc/haproxy/haproxy.cfg</span>. To troubleshoot haproxy issues, you can enable UDP logging in <span style="font-family: "courier new" , "courier" , monospace;">/etc/rsyslog.conf</span> by removing the comments before the following 2 lines:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">#$ModLoad imudp</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">#$UDPServerRun 514</span><br />
<div>
<br /></div>
<div>
then restarting the rsyslog service. Then you can restart the haproxy service and inspect its log file in <span style="font-family: "courier new" , "courier" , monospace;">/var/log/haproxy.log</span>.</div>
</div>
<div>
<br /></div>
<div>
To run the <span style="font-family: Courier New, Courier, monospace;">lb</span> service, I use this bash script:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ cat rancher-lbsetup.sh</span><br />
<span style="font-family: Courier New, Courier, monospace;">#!/bin/bash</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">COMMAND=$@</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">rancher-compose -p proj1-development-lb --url $RANCHER_URL --access-key $RANCHER_API_ACCESS_KEY --secret-key $RANCHER_API_SECRET_KEY --env-file .envvars --file docker-compose-lbsetup.yml --rancher-file rancher-compose.yml $COMMAND</span><br />
<br />
I want to do a rolling upgrade of the <span style="font-family: Courier New, Courier, monospace;">lb</span> service in case anything has changed, so I invoke the <span style="font-family: Courier New, Courier, monospace;">rancher-compose</span> wrapper script in a similar way to the one for the <span style="font-family: Courier New, Courier, monospace;">app</span> service:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><b>./rancher-lbsetup.sh up -d --force-upgrade --confirm-upgrade --batch-size "1"</b></span><br />
<br /></div>
<span style="font-size: large;"><b>Putting it all together in Jenkins</b></span><br />
<br />
First I created a GitHub repository with the following structure:<br />
<br />
<ul>
<li>All <span style="font-family: Courier New, Courier, monospace;">docker-compose-*.yml</span> files</li>
<li>The <span style="font-family: Courier New, Courier, monospace;">rancher-compose.yml</span> file</li>
<li>All <span style="font-family: Courier New, Courier, monospace;">rancher-*.sh</span> bash scripts wrapping the rancher-compose command</li>
<li>A directory for the <span style="font-family: Courier New, Courier, monospace;">base</span> Docker image (containing its Dockerfile and any other files that need to go into that image)</li>
<li>A directory for the <span style="font-family: Courier New, Courier, monospace;">apache-php</span> Docker image</li>
<li>A directory for the <span style="font-family: Courier New, Courier, monospace;">db</span> Docker image</li>
<li>A directory for the <span style="font-family: Courier New, Courier, monospace;">appsetup</span> Docker image</li>
<li>A <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span> in the current directory for the <span style="font-family: Courier New, Courier, monospace;">app</span> Docker image</li>
<li>An <span style="font-family: Courier New, Courier, monospace;">etc</span> directory in the current directory used by the Dockerfile for the <span style="font-family: Courier New, Courier, monospace;">app</span> image</li>
</ul>
<br />
Each project/environment combination has a branch created in this GitHub repository. For example, for the proj1 development environment I would create a <span style="font-family: Courier New, Courier, monospace;">proj1dev</span> branch which would then contain any customizations I need for this project -- usually stack names, Docker tags, Apache configuration files under the etc directory.<br />
<br />
My end goal was to use Jenkins to drive the launching of the Rancher services and the deployment of the code. Eventually I will use a <a href="https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin">Jenkins Pipeline</a> to string together the various steps of the workflow, but for now I have 5 individual Jenkins jobs which all check out the <span style="font-family: Courier New, Courier, monospace;">proj1dev</span> branch of the GitHub repo above. The jobs contain shell-type build steps where I actually call the various rancher bash scripts around <span style="font-family: Courier New, Courier, monospace;">rancher-compose</span>. The Jenkins jobs also take parameters corresponding to the environment variables used in the docker-compose files and in the rancher bash scripts. I also use the Credentials section in Jenkins to store any secrets such as the Rancher API keys, AWS keys, S3 keys, ECR keys etc. On the Jenkins master and executor nodes I installed the <span style="font-family: Courier New, Courier, monospace;">rancher</span> and <span style="font-family: Courier New, Courier, monospace;">rancher-compose</span> CLI utilities (I downloaded the rancher CLI from the footer of the Rancher UI).<br />
<br />
Job #1 builds the Docker images discussed above: <span style="font-family: Courier New, Courier, monospace;">base</span>, <span style="font-family: Courier New, Courier, monospace;">apache-php</span>, <span style="font-family: Courier New, Courier, monospace;">db</span>, and <span style="font-family: Courier New, Courier, monospace;">appsetup</span> (but not the <span style="font-family: Courier New, Courier, monospace;">app</span> image yet).<br />
<br />
Job #2 runs <span style="font-family: Courier New, Courier, monospace;">rancher-nfssetup.sh</span> and <span style="font-family: Courier New, Courier, monospace;">rancher-volsetup.sh</span> in order to set up the NFS stack and the volumes used by the <span style="font-family: Courier New, Courier, monospace;">dbsetup</span>, <span style="font-family: Courier New, Courier, monospace;">appsetup</span>, <span style="font-family: Courier New, Courier, monospace;">db</span> and <span style="font-family: Courier New, Courier, monospace;">app</span> services.<br /><br />
Job #3 runs <span style="font-family: Courier New, Courier, monospace;">rancher-dbsetup.sh</span> and <span style="font-family: Courier New, Courier, monospace;">rancher-dblaunch.sh</span> in order to set up the database via the <span style="font-family: Courier New, Courier, monospace;">dbsetup</span> service, then launch the <span style="font-family: Courier New, Courier, monospace;">db</span> service.<br />
<br />
At this point, everything is ready for deployment of the application.<br />
<br />
Job #4 is the code deployment job. It runs the sequence of steps detailed in the Code Deployment section above.<br />
<br />
Job #5 is the rolling upgrade job for the <span style="font-family: Courier New, Courier, monospace;">app</span> service and the <span style="font-family: Courier New, Courier, monospace;">lb</span> service. If those services have never been started before, they will get started. If they are already running, they will be upgraded in a rolling fashion, <span style="font-family: Courier New, Courier, monospace;">batch-size</span> containers at a time as I detailed above.<br />
<br />
When a new code release needs to be pushed to the proj1dev Rancher environment, I would just run job #4 followed by job #5. Obviously you can string these jobs together in a Jenkins Pipeline, which I intend to do next.<br />
<br />
<span style="font-size: large;"><b>Some more Rancher tips and tricks</b></span><br />
<br />
<span id="docs-internal-guid-78f36dab-1065-fc01-b244-c00b39b47c7f"></span><br />
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li>To troubleshoot Rancher infrastructure-related issues, it helps to inspect the cattle-debug and cattle-error log files on the Rancher server:</li>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: circle; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">cd /var/lib/docker</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: circle; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">find . -name \*cattle\*log -ls</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: circle; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tail -f ./volumes/e426db7ae3def92c737d1b2eb7ed9423c93148e17df5c253a3b3a4dddfb42810/_data/logs/cattle-error.log</span></div>
</li>
</ul>
<li>I recommend this great series of posts on "<a href="http://rancher.com/lessons-learned-building-a-deployment-pipeline-with-docker-docker-compose-and-rancher-part-1/">Lessons learned building a deployment pipeline with Docker, docker-compose and Rancher</a>" from <a href="http://rancher.com/author/jpatterson/">John Patterson</a> and <a href="http://rancher.com/author/clunsford/">Chris Lunsford</a> at This End Out</li>
<li>For production environments, I recommend looking into running your Rancher server in a <a href="http://docs.rancher.com/rancher/v1.2/en/installing-rancher/installing-server/multi-nodes/">multi-node HA configuration</a></li>
</ul>
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-36452789518608427042016-07-28T09:43:00.001-07:002016-07-28T09:43:19.185-07:00Exposing a private Amazon RDS instance with iptables NAT rulesI needed to expose a private Amazon MySQL RDS instance to a 3rd party SaaS tool. I tried several approaches and finally found one that seemed to work pretty well.<br />
<br />
I ended up creating a small EC2 instance in the same VPC as the RDS instance, and applied these iptables NAT/masquerading rules to it, mapping local port 3307 to port 3306 on the RDS instance, whose internal IP address is in this case 172.16.11.2.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># cat iptables_tunnel_port_3307.sh</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#!/bin/bash</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -F</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -F -t nat</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -X</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -t nat -A PREROUTING -p tcp --dport 3307 -j DNAT --to 172.16.11.2:3306</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -A FORWARD -p tcp -d 172.16.11.2 --dport 3306 -j ACCEPT</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -t nat -A OUTPUT -p tcp -o lo --dport 3307 -j DNAT --to 172.16.11.2:3306</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">iptables -t nat -A POSTROUTING -j MASQUERADE</span><br />
<br />
I also had to enable IP forwarding on the EC2 instance:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># sysctl net.ipv4.ip_forward</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># sysctl -p</span><br />
<br />
At this point, I was able to hit the external IP of the EC2 instance on port 3307, and get to the private RDS instance on port 3306. I was also able to attach the EC2 instance to an EC2 Security Group allowing the 3rd party SaaS tool IP addresses to access port 3307 on the EC2 instance.<br />
<br />
My thanks to the people discussing a similar issue on this <a href="http://www.linuxquestions.org/questions/linux-networking-3/iptables-how-to-redirect-locally-generated-packets-to-a-remote-server-797173/page2.html">thread</a> of LinuxQuestions. Without their discussion, I don't think I'd have been able to figure out a solution.Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-11498762637449362662016-07-13T14:05:00.003-07:002016-07-14T09:52:56.924-07:00Using JMESPath queries with the AWS CLIThe <a href="https://github.com/aws/aws-cli">AWS CLI</a>, based on the <a href="https://github.com/boto/boto3">boto3</a> Python library, is the recommended way of automating interactions with AWS. In this post I'll show some examples of more advanced AWS CLI usage using the query mechanism based on the <a href="http://jmespath.org/">JMESPath</a> JSON query language.<br />
<br />
Installing the AWS CLI tools is straightforward. On Ubuntu via <span style="font-family: "courier new" , "courier" , monospace;">apt-get</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># apt-get install awscli</span><br />
<br />
Or via <span style="font-family: "courier new" , "courier" , monospace;">pip</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># apt-get install python-pip</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"># pip install awscli</span><br />
<br />
The next step is to configure <span style="font-family: "courier new" , "courier" , monospace;">awscli</span> by specifying the AWS Access Key ID and AWS Secret Access Key, as well as the default region and output format:<br />
<br />
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;"># aws configure</span></span></div>
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;">AWS Access Key ID: your-aws-access-key-id</span></span></div>
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;">AWS Secret Access Key: your-aws-secret-access-key</span></span></div>
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;">Default region name [us-west-2]: us-west-2</span></span></div>
<br />
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;">Default output format [None]: json</span></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">The configure command creates a <span style="font-family: "courier new" , "courier" , monospace;">~/.aws</span> directory containing two files: <span style="font-family: "courier new" , "courier" , monospace;">config</span> and <span style="font-family: "courier new" , "courier" , monospace;">credentials</span>.</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
You can specify more than one pair of AWS keys by creating profiles in these files. For example, in <span style="font-family: "courier new" , "courier" , monospace;">~/.aws/credentials</span> you can have:</div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">[profile profile1]</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">AWS_ACCESS_KEY_ID=key1</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">AWS_SECRET_ACCESS_KEY=secretkey1</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br class="Apple-interchange-newline" />[profile profile2]</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">AWS_ACCESS_KEY_ID=key2</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">AWS_SECRET_ACCESS_KEY=secretkey2</span></div>
<div class="p1">
<br /></div>
<div class="p1">
In <span style="font-family: "courier new" , "courier" , monospace;">~/.aws/config</span> you can have:</div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">[profile profile1]</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">region = us-west-2</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">[profile profile2]</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">region = us-east-1</span></div>
<div class="p1">
<br /></div>
<div class="p1">
You can specify a given profile when you run awscli:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># awscli --profile profile1</span></div>
<div class="p1">
<br /></div>
<div class="p1">
Let's assume you want to write a script using awscli that deletes EBS snapshots older than N days. Let's go through this one step at a time.</div>
<div class="p1">
<br /></div>
<div class="p1">
Here's how you can list all snapshots owned by you:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER --query "Snapshots[]"</span></div>
<div class="p1">
<br /></div>
<div class="p1">
Note the use of the <span style="font-family: "courier new" , "courier" , monospace;">--query</span> option. It takes a parameter representing a JMESPath JSON query string. It's not trivial to figure out how to build these query strings, and I advise you to spend some time reading over the <a href="http://jmespath.org/tutorial.html">JMESPath tutorial</a> and <a href="http://jmespath.org/examples.html">JMESPath examples</a>.</div>
<div class="p1">
<br /></div>
<div class="p1">
In the example above, the query string is simply <span style="font-family: "courier new" , "courier" , monospace;">"Snapshots[]"</span>, which represents all the snapshots that are present in the AWS account associated with the profile <span style="font-family: "courier new" , "courier" , monospace;">profile1</span>. The default output in our case is JSON, but you can specify <span style="font-family: "courier new" , "courier" , monospace;">--output text</span> at the aws command line if you want to see each snapshot on its own line of text.</div>
<div class="p1">
<br /></div>
<div class="p1">
Let's assume that when you create the snapshots, you specify a description with contains <span style="font-family: "courier new" , "courier" , monospace;">PROD</span> or <span style="font-family: "courier new" , "courier" , monospace;">STAGE</span> for EBS volumes attached to production and stage EC2 instances respectively. If you want to only display snapshots containing the string <span style="font-family: "courier new" , "courier" , monospace;">PROD</span>, you would do:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER--query "Snapshots[?contains(Description, \`PROD\`) == \`true\`]" --output text</span></div>
<div class="p1">
<br /></div>
<div class="p1">
The<span style="font-family: "courier new" , "courier" , monospace;"> Snapshots[] </span>array now contains a condition represented by the question mark <span style="font-family: "courier new" , "courier" , monospace;">?</span>. The condition uses the <span style="font-family: "courier new" , "courier" , monospace;">contains()</span> function included in the JMESPath specification, and is applied against the <span style="font-family: "courier new" , "courier" , monospace;">Description</span> field of each object in the <span style="font-family: "courier new" , "courier" , monospace;">Snapshots[]</span> array, verifying that it contains the string <span style="font-family: "courier new" , "courier" , monospace;">PROD</span>. Note the use of backquotes surrounding the strings <span style="font-family: "courier new" , "courier" , monospace;">PROD</span> and <span style="font-family: "courier new" , "courier" , monospace;">true</span> in the condition. I spent some quality time troubleshooting my queries when I used single or double quotes with no avail. The backquotes also need to be escaped so that the shell doesn't interpret them as commands to be executed.</div>
<div class="p1">
<br /></div>
<div class="p1">
To restrict the PROD snapshots even further, to the ones older than say 7 days ago, you can do something like this:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">DAYS=7</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">TARGET_DATE=`date --date="$DAYS day ago" +%Y-%m-%d`</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER--query "Snapshots[?contains(Description, \`PROD\`) == \`true\`]|[?StartTime < \`$TARGET_DATE\`]" --output text</span></div>
<div class="p1">
<br /></div>
<div class="p1">
Here I used the <span style="font-family: "courier new" , "courier" , monospace;">StartTime</span> field of the objects in the <span style="font-family: "courier new" , "courier" , monospace;">Snapshots[]</span> array and compared it against the target date. In this case, string comparison is good enough for the query to work.</div>
<div class="p1">
<br /></div>
<div class="p1">
In all the examples above, the aws command returned a subset of the Snapshots[] array and displayed all fields for each object in the array. If you wanted to display specific fields, let's say the ID, the start time and the description of each snapshot, you would run:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER</span><span style="font-family: "courier new" , "courier" , monospace;">--query "Snapshots[?contains(Description, \`PROD\`) == \`true\`]</span><span style="font-family: "courier new" , "courier" , monospace;">|[?StartTime < \`$TARGET_DATE\`]</span><span style="font-family: "courier new" , "courier" , monospace;">.[SnapshotId,StartTime,Description]</span><span style="font-family: "courier new" , "courier" , monospace;">" --output text</span></div>
<div class="p1">
<br /></div>
<div class="p1">
To delete old snapshots, you can use the <span style="font-family: "courier new" , "courier" , monospace;">aws ec2 delete-snapshot</span> command, which needs a snapshot ID as a parameter. You could use the command above to list only the <span style="font-family: "courier new" , "courier" , monospace;">SnapshotId</span> for snapshots older than N days, then for each of these IDs, run something like this:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"># aws ec2 delete-snapshot --profile profile1 --snapshot-id $id</span></div>
<div class="p1">
<br /></div>
<div class="p1">
All this is well and good when you run these commands interactively at the shell. However, I had no luck running them out of cron. The backquotes resulted in boto3 syntax errors. I had to do it the hard way, by listing all snapshots first, then going all in with sed and awk:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">aws ec2 describe-snapshots --profile profile1 --owner-id YOUR_AWS_ACCT_NUMBER --output=text --query "Snapshots[].[SnapshotId,StartTime,Description]" > $TMP_SNAPS</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">DAYS=7</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">TARGET_DATE=`date --date="$DAYS day ago" +%Y-%m-%d`</span></div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">cat $TMP_SNAPS | grep PROD | sed 's/T[0-9][0-9]:[0-9][0-9]:[0-9][0-9].000Z//' | awk -v target_date="$TARGET_DATE" '{if ($2 < target_date){print}}' > $TMP_PROD_SNAPS</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">echo PRODUCTION SNAPSHOTS OLDER THAN $DAYS DAYS</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">cat $TMP_PROD_SNAPS</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">for sid in `awk '{print $1}' $TMP_PROD_SNAPS` ; do</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>echo Deleting PROD snapshot $sid</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>aws ec2 delete-snapshot --profile $PROFILE --region $REGION --snapshot-id $sid</span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;">done</span></div>
<div class="p1">
<br /></div>
<div class="p1">
Ugly, but it works out of cron. Hope it helps somebody out there.<br />
<br />
July 14th 2016: I initially forgot to include this very good <a href="http://opensourceconnections.com/blog/2015/07/27/advanced-aws-cli-jmespath-query/">blog post</a> from Joseph Lawson on advanced JMESPath usage with the AWS CLI.</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-59524377283014038552016-07-01T10:11:00.001-07:002016-07-01T10:11:12.094-07:00More tips and tricks for running Gatling in Docker containersThis post is a continuation of my previous one on "<a href="http://agiletesting.blogspot.com/2016/06/running-gatling-load-tests-in-docker.html">Running Gatling tests in Docker containers via Jenkins</a>". As I continued to set up Jenkins jobs to run Gatling tests, I found the need to separate those tests for different environments - development, staging and production. The initial example I showed contained a single setup, which is not suitable for multiple environments.<br />
<br />
Here is my updated Gatling directory structure<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/conf</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/conf/production</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/conf/production/gatling.conf</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/conf/staging</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/conf/staging/gatling.conf</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/Dockerfile</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/results</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/data</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/data/production-urls.csv</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/data/staging-urls.csv</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/simulations</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/simulations/development</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/simulations/production</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/simulations/production/Simulation.scala</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/simulations/staging</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">gatling/user-files/simulations/staging/Simulation.scala</span><br />
<br />
Note that I created a separate directory under <span style="font-family: Courier New, Courier, monospace;">simulations</span><span style="font-family: Arial, Helvetica, sans-serif;"> </span>for each environment (<span style="font-family: Courier New, Courier, monospace;">development</span>, <span style="font-family: Courier New, Courier, monospace;">staging</span>, <span style="font-family: Courier New, Courier, monospace;">production</span>), each with its own simulation files.<br />
<br />
I also created a <span style="font-family: Courier New, Courier, monospace;">data</span> directory under <span style="font-family: Courier New, Courier, monospace;">user-files</span>, because that is the default location for CSV files used by <a href="http://gatling.io/docs/2.0.0-RC2/session/feeder.html">Gatling feeders</a>.<br />
<br />
Most importantly, I created a separate configuration directory (<span style="font-family: Courier New, Courier, monospace;">staging</span>, <span style="font-family: Courier New, Courier, monospace;">production</span>) under <span style="font-family: Courier New, Courier, monospace;">gatling/conf</span>, each directory containing its own customized <span style="font-family: Courier New, Courier, monospace;">gatling.conf</span> file. I started by copying the <a href="https://github.com/gatling/gatling/blob/master/gatling-core/src/main/resources/gatling-defaults.conf">gatling-defaults.conf</a><span id="goog_2090183318"></span><span id="goog_2090183319"></span><a href="https://draft.blogger.com/"></a> file from GitHub to <span style="font-family: Courier New, Courier, monospace;">gatling/conf/staging/gatling.conf</span> and <span style="font-family: Courier New, Courier, monospace;">gatling/conf/production/gatling.conf</span> respectively.<br />
<br />
Here is what I customized in <span style="font-family: Courier New, Courier, monospace;">staging/gatling.conf</span>:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">mute = true # When set to true, don't ask for simulation name nor run description</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">simulations = user-files/simulations/staging</span><br />
<br />
I customized <span style="font-family: Courier New, Courier, monospace;">production/gatling.conf </span>in a similar way:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">mute = true # When set to true, don't ask for simulation name nor run description</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">simulations = user-files/simulations/production</span><br />
<br />
Setting <span style="font-family: Courier New, Courier, monospace;">mute</span> to <span style="font-family: Courier New, Courier, monospace;">true</span> is important because without it, running Gatling in a Docker container was segfaulting while waiting for user input for the simulation ID:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Select simulation id (default is 'gatlingsimulation'). Accepted characters are a-z, A-Z, 0-9, - and _ </span><div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Exception in thread "main" java.lang.NullPointerException</span><div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> at io.gatling.app.Selection$Selector.loop$1(Selection.scala:127)
at io.gatling.app.Selection$Selector.askSimulationId(Selection.scala:135)
at io.gatling.app.Selection$Selector.selection(Selection.scala:50)
at io.gatling.app.Selection$.apply(Selection.scala:33)
at io.gatling.app.Gatling.runIfNecessary(Gatling.scala:75)
at io.gatling.app.Gatling.start(Gatling.scala:65)
at io.gatling.app.Gatling$.start(Gatling.scala:57)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:49)
at io.gatling.app.Gatling$.main(Gatling.scala:43)
at io.gatling.app.Gatling.main(Gatling.scala)<br /></span><br /><div>
The other customization was to point the <span style="font-family: Courier New, Courier, monospace;">simulations</span> attribute to the specific <span style="font-family: Courier New, Courier, monospace;">staging</span> or <span style="font-family: Courier New, Courier, monospace;">production</span> sub-directories.</div>
<div>
<br /></div>
<div>
Since the CSV files containing URLs to be load tested are also environment-specific, I modified the <span style="font-family: Courier New, Courier, monospace;">Simulation.scala</span> files to take this into account. I also added 2 <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span> variables that can be passed at runtime for HTTP basic authentication. Here is the new <span style="font-family: Courier New, Courier, monospace;">Crawl</span> object (compare with the one from my <a href="http://agiletesting.blogspot.com/2016/06/running-gatling-load-tests-in-docker.html">previous post</a>):</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">object Crawl {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> val feeder = csv("staging-urls.csv").random</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> val userName = System.getProperty("username")</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> val userPass = System.getProperty("password")</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> val crawl = exec(feed(feeder)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> .exec(http("${loc}")</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> .get("${loc}").basicAuth(userName, userPass)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ))</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">}</span></div>
</div>
<div>
<br /></div>
<div>
One more thing is needed: to make Gatling use a specific configuration file instead of its default one, which is <span style="font-family: Courier New, Courier, monospace;">conf/gatling.conf</span>. To do that, I set <span style="font-family: Courier New, Courier, monospace;">GATLING_CONF</span> as an <span style="font-family: Courier New, Courier, monospace;">ENV</span> variable in the <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span>, so it can be passed as a '<span style="font-family: Courier New, Courier, monospace;">docker run</span>' command line parameter. Here is the <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span>:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># Gatling is a highly capable load testing tool.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># Documentation: http://gatling.io/docs/2.2.2/</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># Cheat sheet: http://gatling.io/#/cheat-sheet/2.2.2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">FROM java:8-jdk-alpine</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">MAINTAINER Denis Vazhenin</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># working directory for gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">WORKDIR /opt</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># gating version</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ENV GATLING_VERSION 2.2.2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># create directory for gatling install</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN mkdir -p gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># install gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN apk add --update wget && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> mkdir -p /tmp/downloads && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> wget -q -O /tmp/downloads/gatling-$GATLING_VERSION.zip \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/$GATLING_VERSION/gatling-charts-highcharts-bundle-$GATLING_VERSION-bundle.zip && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> mkdir -p /tmp/archive && cd /tmp/archive && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> unzip /tmp/downloads/gatling-$GATLING_VERSION.zip && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> mv /tmp/archive/gatling-charts-highcharts-bundle-$GATLING_VERSION/* /opt/gatling/</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># change context to gatling directory</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">WORKDIR /opt/gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># set directories below to be mountable from host</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">VOLUME ["/opt/gatling/conf", "/opt/gatling/results", "/opt/gatling/user-files"]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># set environment variables</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ENV PATH /opt/gatling/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>ENV GATLING_HOME /opt/gatling</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>ENV GATLING_CONF /opt/gatling/conf</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>ENV JAVA_OPTS ""</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ENTRYPOINT ["gatling.sh"]</span></div>
</div>
<div>
<br /></div>
<div>
Finally, here is how I invoke 'docker run' to tie everything together:</div>
</div>
</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b>docker run --rm -v ${WORKSPACE}/gatling/conf:/opt/gatling/conf -v ${WORKSPACE}/gatling/user-files:/opt/gatling/user-files -v ${WORKSPACE}/gatling/results:/opt/gatling/results -e GATLING_CONF="/opt/gatling/conf/staging" -e JAVA_OPTS="-Dusers=$USERS -Dduration=$DURATION -Dusername=myusername -Dpassword=mypass" /PATH/TO/DOCKER/REGISTRY/gatling</b></span></div>
<div>
<br /></div>
Note the <span style="font-family: Courier New, Courier, monospace;">GATLING_CONF</span> parameter passed with <span style="font-family: Courier New, Courier, monospace;">-e</span> with the value of <span style="font-family: Courier New, Courier, monospace;">/opt/gatling/conf/staging</span>. Also note the <span style="font-family: Courier New, Courier, monospace;">username</span> and <span style="font-family: Courier New, Courier, monospace;">password</span> <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span> parameters.<br /><div>
<br /></div>
<div>
Happy load testing!</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com1tag:blogger.com,1999:blog-9238405.post-10319896048873803872016-06-28T16:16:00.001-07:002016-06-28T16:16:08.476-07:00Running Gatling load tests in Docker containers via JenkinsGatling is a modern load testing tool written in Scala. As part of the <a href="http://agiletesting.blogspot.com/2016/06/running-jenkins-jobs-in-docker.html">Jenkins setup I am in charge of</a>, I wanted to run load tests using Gatling against a collection of pages for a given website. Here are my notes on how I managed to do this.<br />
<br />
<b>Running Gatling as a Docker container locally</b><br />
<br />
There is a Docker image already available on DockerHub, so you can simply pull down the image locally:<br />
<br />
<br />
<div class="p1">
<span class="s1"><span style="font-family: Courier New, Courier, monospace;">$ docker pull denvazh/gatling:2.2.2</span></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
Instructions on how to run a container based on this image are available on <a href="https://github.com/denvazh/gatling">GitHub</a>:</div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span class="s1"><span style="font-family: Courier New, Courier, monospace;">$ docker run -it --rm -v /home/core/gatling/conf:/opt/gatling/conf \</span></span></div>
<div class="p1">
<span class="s1"><span style="font-family: Courier New, Courier, monospace;">-v /home/core/gatling/user-files:/opt/gatling/user-files \</span></span></div>
<div class="p1">
<span class="s1"><span style="font-family: Courier New, Courier, monospace;">-v /home/core/gatling/results:/opt/gatling/results \</span></span></div>
<div class="p1">
</div>
<div class="p1">
<span class="s1"><span style="font-family: Courier New, Courier, monospace;">denvazh/gatling:2.2.2</span></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">Based on these instructions, I created a local directory called <span style="font-family: Courier New, Courier, monospace;">gatling</span>, and under it I created 3 sub-directories: <span style="font-family: Courier New, Courier, monospace;">conf</span>, <span style="font-family: Courier New, Courier, monospace;">results</span> and <span style="font-family: Courier New, Courier, monospace;">user-files</span>. I left the <span style="font-family: Courier New, Courier, monospace;">conf</span> and <span style="font-family: Courier New, Courier, monospace;">results</span> directories empty, and under <span style="font-family: Courier New, Courier, monospace;">user-files</span> I created a <span style="font-family: Courier New, Courier, monospace;">simulations</span> directory containing a Gatling load test scenario written in Scala. I also created a file in the <span style="font-family: Courier New, Courier, monospace;">user-files</span> directory called <span style="font-family: Courier New, Courier, monospace;">urls.csv</span>, containing a header named <span style="font-family: Courier New, Courier, monospace;">loc</span> and a URL per line for each page that I want to load test.</span></div>
<div class="p1">
<br /></div>
<div class="p1">
Assuming the current directory is <span style="font-family: Courier New, Courier, monospace;">gatling</span>, here are examples of these files:</div>
<div class="p1">
<br /></div>
<div class="p1">
<span class="s1"><span style="font-family: Courier New, Courier, monospace;"><b>$ cat user-files/urls.csv</b></span></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">loc</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">https://my.website.com</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">https://my.website.com/category1</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">https://my.website.com/category2/product3</span></div>
<div class="p1">
<br /></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><b>$ cat user-files/simulations/Simulation.scala</b></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">package my.gatling.simulation</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">import io.gatling.core.Predef._</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">import io.gatling.http.Predef._</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">import scala.concurrent.duration._</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">class GatlingSimulation extends Simulation {</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val httpConf = http</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .baseURL("http://127.0.0.1")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .doNotTrackHeader("1")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .acceptLanguageHeader("en-US,en;q=0.5")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .acceptEncodingHeader("gzip, deflate")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .userAgentHeader("Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val scn1 = scenario("Scenario1")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .exec(Crawl.crawl)</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val userCount = Integer.getInteger("users", 1)</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val durationInSeconds = java.lang.Long.getLong("duration", 10L)</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> setUp(</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> scn1.inject(rampUsers(userCount) over (durationInSeconds seconds))</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> ).protocols(httpConf)</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">}</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">object Crawl {</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val feeder = csv("/opt/gatling/user-files/urls.csv").random</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val crawl = exec(feed(feeder)</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .exec(http("${loc}")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> .get("${loc}")</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> ))</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;">}</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">I won't go through the different ways of writing Gatling load tests scenarios here. There are good instructions on the Gatling website -- see the <a href="http://gatling.io/docs/2.2.2/quickstart.html">Quickstart</a> and the <a href="http://gatling.io/docs/2.2.2/advanced_tutorial.html#advanced-tutorial">Advanced Tutorial</a>. What the scenario above does is it reads the file <span style="font-family: Courier New, Courier, monospace;">urls.csv</span> and randomly picks a URL from it, then runs a load test against that URL.</span></div>
<br />I do want to point out 2 variables in the above script:<br />
<br />
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val <b>userCount</b> = Integer.getInteger("<b>users</b>", 1)</span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace;"> val <b>durationInSeconds</b> = java.lang.Long.getLong("<b>duration</b>", 10L)</span></div>
<br />
These variables specify the max number of users we want to ramp up to, and the duration of the ramp-up. They are used in the <span style="font-family: Courier New, Courier, monospace;">inject</span> call:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">scn1.inject(rampUsers(<b>userCount</b>) over (<b>durationInSeconds</b> seconds))</span><br />
<span style="font-family: "Courier New", Courier, monospace;"><br /></span>
The special thing about these 2 variables is that they are read from <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span> by Gatling. So if you have a <span style="font-family: Courier New, Courier, monospace;">-Dusers</span> Java option and a <span style="font-family: Courier New, Courier, monospace;">-Dduration</span> Java option, Gatling will know how to read them and how to set the userCount and durationInSeconds variables accordingly. This is a good thing, because it allows you to specify those numbers outside of Gatling, without hardcoding them in your simulation script. Here is more info on <a href="http://gatling.io/docs/2.0.1/cookbook/passing_parameters.html">passing parameters</a> via the command line to Gatling.<br />
<br />
While pulling the Gatling docker image and running it is the simplest way to run Gatling, I prefer to understand what's going on in that image. I started off by getting the <a href="https://github.com/denvazh/gatling/blob/master/2.2.2/Dockerfile:">Dockerfile</a> from GitHub:<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">$ cat Dockerfile</span><div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># Gatling is a highly capable load testing tool.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">#</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># Documentation: http://gatling.io/docs/2.2.2/</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># Cheat sheet: http://gatling.io/#/cheat-sheet/2.2.2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">FROM java:8-jdk-alpine</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">MAINTAINER Denis Vazhenin</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># working directory for gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">WORKDIR /opt</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># gating version</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ENV GATLING_VERSION 2.2.2</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># create directory for gatling install</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">RUN mkdir -p gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># install gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">RUN apk add --update wget && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> mkdir -p /tmp/downloads && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> wget -q -O /tmp/downloads/gatling-$GATLING_VERSION.zip \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/$GATLING_VERSION/gatling-charts-highcharts-bundle-$GATLING_VERSION-bundle.zip && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> mkdir -p /tmp/archive && cd /tmp/archive && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> unzip /tmp/downloads/gatling-$GATLING_VERSION.zip && \</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> mv /tmp/archive/gatling-charts-highcharts-bundle-$GATLING_VERSION/* /opt/gatling/</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># change context to gatling directory</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">WORKDIR /opt/gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># set directories below to be mountable from host</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">VOLUME ["/opt/gatling/conf", "/opt/gatling/results", "/opt/gatling/user-files"]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># set environment variables</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ENV PATH /opt/gatling/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ENV GATLING_HOME /opt/gatling</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ENTRYPOINT ["gatling.sh"]</span></div>
<div>
<br /></div>
<div>
I then added a way to pass <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span> via an environment variable. I added this line after the <span style="font-family: Courier New, Courier, monospace;">ENV GATLING_HOME</span> line:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ENV JAVA_OPTS ""</span></div>
</div>
<div>
<br /></div>
<div>
I dropped this <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span> in my <span style="font-family: Courier New, Courier, monospace;">gatling</span> directory, then built a local Docker image off of it:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ docker build -t gatling:local .</span></div>
<div>
<br /></div>
I then invoked '<span style="font-family: Courier New, Courier, monospace;">docker run</span>' to launch a container based on this image, using the csv and simulation files from above. The current directory is still gatling.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ docker run --rm -v `pwd`/conf:/opt/gatling/conf -v `pwd`/user-files:/opt/gatling/user-files -v `pwd`/results:/opt/gatling/results -e JAVA_OPTS="-Dusers=10 -Dduration=60" </span><span style="font-family: "Courier New", Courier, monospace;">gatling:local</span><span style="font-family: Courier New, Courier, monospace;"> -s MySimulationName</span><br />
<br />
Note the <span style="font-family: Courier New, Courier, monospace;">-s </span>flag which denotes a simulation name (which can be any string you want). If you don't specify this flag, the <span style="font-family: Courier New, Courier, monospace;">gatling.sh</span> script which is the <span style="font-family: Courier New, Courier, monospace;">ENTRYPOINT</span> in the container will wait for some user input and you will not be able to fully automate your load test.<br />
<br />
Another thing to note is the use of <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span>. In the example above, I pass <span style="font-family: Courier New, Courier, monospace;">-Dusers=10 </span>and <span style="font-family: Courier New, Courier, monospace;">-Dduration=60</span> as the two <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span> parameters. The <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span> variable itself is passed to '<span style="font-family: Courier New, Courier, monospace;">docker run</span>' via the <span style="font-family: Courier New, Courier, monospace;">-e</span> option, which tells Docker to replace the default value for <span style="font-family: Courier New, Courier, monospace;">ENV JAVA_OPTS</span> (which is <span style="font-family: Courier New, Courier, monospace;">""</span>) with the value passed with <span style="font-family: Courier New, Courier, monospace;">-e</span>.<br />
<br />
<b>Running Gatling as a Docker container from Jenkins</b><br />
<br />
Once you have a working Gatling container locally, you can upload the Docker image built above to a private Docker registry. I used a private EC2 Container Registry (ECR). <br />
<br />
I also added the <span style="font-family: Courier New, Courier, monospace;">gatling</span> directory and its sub-directories to a GitHub repository called <span style="font-family: Courier New, Courier, monospace;">devops</span>.<br />
<br />
In Jenkins, I created a new "Freestyle project" job with the following properties:<br />
<br />
<ul>
<li>Parameterized build with 2 string parameters: <span style="font-family: Courier New, Courier, monospace;">USERS</span> (default value 10) and <span style="font-family: Courier New, Courier, monospace;">DURATION</span> in seconds (default value 60)</li>
<li>Git repository - add URL and credentials for the <span style="font-family: Courier New, Courier, monospace;">devops</span> repository which contains the <span style="font-family: Courier New, Courier, monospace;">gatling</span> files</li>
<li>An "Execute shell" build command similar to this one:</li>
</ul>
<span style="font-family: Courier New, Courier, monospace;">docker run --rm -v ${WORKSPACE}/gatling/conf:/opt/gatling/conf -v ${WORKSPACE}/gatling/user-files:/opt/gatling/user-files -v ${WORKSPACE}/gatling/results:/opt/gatling/results -e </span><span style="font-family: "Courier New", Courier, monospace;">JAVA_OPTS="-Dusers=$USERS -Dduration=$DURATION"</span><span style="font-family: "Courier New", Courier, monospace;"> </span><span style="font-family: "Courier New", Courier, monospace;"> /PATH/TO/DOCKER/REGISTRY/gatling -s MyLoadTest </span><br />
<br />
<div>
<br /></div>
<div>
Note that we mount the gatling directories as Docker volumes, similarly to when we ran the Docker container locally, only this time we specify <span style="font-family: Courier New, Courier, monospace;">${WORKSPACE}</span> as the base directory. The 2 string parameters <span style="font-family: Courier New, Courier, monospace;">USERS</span> and <span style="font-family: Courier New, Courier, monospace;">DURATION</span> are passed as variables in <span style="font-family: Courier New, Courier, monospace;">JAVA_OPTS</span>.</div>
<div>
<br /></div>
<div>
A nice thing about running Gatling via Jenkins is that the reports are available in the Workspace directory of the project. If you go to the Gatling project we created in Jenkins, click on Workspace, then on gatling, then results, you should see directories named <span style="font-family: Courier New, Courier, monospace;">gatlingsimulation-TIMESTAMP</span> for each Gatling run. Each of these directories should have an index.html file, which will show you the Gatling report dashboard. Pretty neat.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidsmKd14Hr0zVKyoIn7a5Kxi0E9riZhZkJpUOiXg27iFWpToE9SWjbRYacfVSp_0UuqjGemgghBdjNiuFbnas3qgDoigG_wApIEbJag2kglio7Hg3_KX4IIOc84d_60aNTAro03g/s1600/Screen+Shot+2016-06-28+at+4.15.28+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="136" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidsmKd14Hr0zVKyoIn7a5Kxi0E9riZhZkJpUOiXg27iFWpToE9SWjbRYacfVSp_0UuqjGemgghBdjNiuFbnas3qgDoigG_wApIEbJag2kglio7Hg3_KX4IIOc84d_60aNTAro03g/s320/Screen+Shot+2016-06-28+at+4.15.28+PM.png" width="320" /></a></div>
<div>
<br /></div>
</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com1tag:blogger.com,1999:blog-9238405.post-39323276895201977402016-06-16T16:08:00.001-07:002016-06-16T16:10:02.228-07:00Running Jenkins jobs in Docker containersOne of my main tasks at work is to configure Jenkins to act as a hub for all the deployment and automated testing jobs we run. We use CloudBees Jenkins Enterprise, mostly for its <a href="https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/chapter-rbac.html">Role-Based Access Control plugin</a>, which allows us to create one Jenkins folder per project/application and establish fine grained access control to that folder for groups of users. We also make heavy use of the Jenkins Enterprise <a href="https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/chapter-workflow.html">Pipeline</a> features (which I think are also available these days in the open source version).<br />
<br />
Our Jenkins infrastructure is composed of a master node and several executor nodes which can run jobs in parallel if needed.<br />
<br />
One pattern that my colleague Will Wright and I have decided upon is to run all Jenkins jobs as Docker containers. This way, we only need to install Docker Engine on the master node and the executor nodes. No need to install any project-specific pre-requisites or dependencies on every Jenkins node. All of these dependencies and pre-reqs are instead packaged in the Docker containers. It's a simple but powerful idea, that has worked very well for us. One of the nice things about this pattern is that you can keep adding various types of automated tests. If it can run from the command line, then it can run in a Docker container, which means you can run it from Jenkins!<br />
<br />
I have seen this pattern discussed in multiple places recently, for example in this blog post about "<a href="https://ig.nore.me/2015/09/using-docker-for-a-more-flexible-jenkins/">Using Docker for a more flexible Jenkins</a>".<br />
<br />
Here are some examples of Jenkins jobs that we create for a given project/application:<br />
<ul>
<li>a deployment job that runs Capistrano in its own Docker container, against targets in various environments (development, staging, production); this is a Pipeline script written in Groovy, which can call other jobs below</li>
<li>a Web UI testing job that runs the Selenium Python <a href="http://www.seleniumhq.org/docs/03_webdriver.jsp">WebDriver</a> and drives Firefox in headless mode (see my <a href="http://agiletesting.blogspot.com/2016/02/setting-up-jenkins-to-run-headless.html">previous post</a> on how to do this with Docker)</li>
<li>a JavaScript syntax checking job that runs <a href="http://jshint.com/">JSHint</a> against the application's JS files</li>
<li>an SSL scanner/checker that runs <a href="https://github.com/nabla-c0d3/sslyze">SSLyze</a> against the application endpoints</li>
</ul>
<div>
We also run other types of tasks, such as running an AWS CLI command to perform certain actions, for example to invalidate a CloudFront resource. I am going to show here how we create a Docker image for one of these jobs, how we test it locally, and how we then integrate it in Jenkins.<br />
<br />
I'll use as an example a simple Docker image that installs the AWS CLI package and runs a command when the container is invoked via 'docker run'.</div>
<div>
<br /></div>
<div>
I assume you have a local version of Docker installed. If you are on a Mac, you can use <a href="https://docs.docker.com/engine/installation/mac/">Docker Toolbox</a>, or, if you are lucky and got access to it, you can use the native <a href="https://blog.docker.com/2016/03/docker-for-mac-windows-beta/">Docker for Mac.</a> In any case, I will assume that you have a local directory called <span style="font-family: "courier new" , "courier" , monospace;">awscli</span> with the following <span style="font-family: "courier new" , "courier" , monospace;">Dockerfile</span> in it:</div>
<br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">FROM ubuntu:14.04</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">MAINTAINER You Yourself <you@example.com></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"># disable interactive functions</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">ARG DEBIAN_FRONTEND=noninteractive</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">ENV AWS_ACCESS_KEY_ID=""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">ENV AWS_SECRET_ACCESS_KEY=""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">ENV AWS_COMMAND=""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">RUN apt-get update && \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"> apt-get install -y python-pip && \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"> pip install awscli</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">WORKDIR /root</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: xx-small;">CMD (export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID; export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY; $AWS_COMMAND)</span></div>
</div>
<div>
<br /></div>
<div>
As I mentioned, this simply installs the <span style="font-family: "courier new" , "courier" , monospace;">awscli</span> Python package via <span style="font-family: "courier new" , "courier" , monospace;">pip</span>, then runs a command given as an environment variable when you invoke <span style="font-family: "courier new" , "courier" , monospace;">'docker run</span>'. It also uses two other environment variables that contain the AWS access key ID and secret access key. You don't want to hardcode these secrets in the Dockerfile and have them end up on GitHub.</div>
<div>
<br /></div>
<div>
The next step is to build an image based on this Dockerfile. I'll call the image <span style="font-family: "courier new" , "courier" , monospace;">awscli</span> and I'll tag it as <span style="font-family: "courier new" , "courier" , monospace;">local</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ docker build -t awscli:local .</span><br />
<br />
Then you can run a container based on this image. The command line looks a bit complicated because I am passing (via the <span style="font-family: "courier new" , "courier" , monospace;">-e</span> switch) the 3 environment variables discussed above:<br />
<br />
<br />
<div class="p1">
<span class="s1"><span style="font-family: "courier new" , "courier" , monospace;">$ docker run --rm -e AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY -e AWS_COMMAND='aws cloudfront create-invalidation --distribution-id=abcdef --invalidation-batch Paths={Quantity=1,Items=[/media/*]},CallerReference=my-invalidation-123456' awscli:local</span></span></div>
<div class="p1">
<span class="s1"><br /></span></div>
(where <span style="font-family: "courier new" , "courier" , monospace;">distribution-id</span> needs to be the actual ID of your CloudFront distribution, and <span style="font-family: "courier new" , "courier" , monospace;">CallerReference</span> needs to be unique per invalidation)<br />
<br />
If all goes well, you should see the output of the '<span style="font-family: "courier new" , "courier" , monospace;">aws cloudfront create-invalidation</span>' command.<br />
<br />
In our infrastructure, we have a special GitHub repository where we check in the various folders containing the Dockerfiles and any static files that need to be copied over to the Docker images. When we push the <span style="font-family: "courier new" , "courier" , monospace;">awscli</span> directory to GitHub for example, we have a Jenkins job that will be notified of that commit and that will build the Docker image (similarly to how we did it locally with '<span style="font-family: "courier new" , "courier" , monospace;">docker build</span>'), then it will '<span style="font-family: "courier new" , "courier" , monospace;">docker push</span>' the image to a private AWS ECR repository we have.<br />
<br />
Now let's assume we want to create a Jenkins job that will run this image as a container. First we define 2 secret credentials, specific to the Jenkins folder where we want to create the job (there are also global Jenkins credentials that can apply to all folders). These credentials are of type "<span style="font-family: "courier new" , "courier" , monospace;">Secret text</span>" and contain the AWS access key ID and the AWS secret access key.<br />
<br />
Then we create a new Jenkins job of type "Freestyle project" and call it <span style="font-family: "courier new" , "courier" , monospace;">cloudfront.invalidate</span>. The build for this job is parameterized and contains 2 parameters: <span style="font-family: "courier new" , "courier" , monospace;">CF_ENVIRONMENT</span> which is a drop-down containing the values "Staging" and "Production" referring to the CloudFront distribution we want to invalidate; and <span style="font-family: "courier new" , "courier" , monospace;">CF_RESOURCE</span>, which is a text variable that needs to be set to the resource that needs to be invalidated (e.g. /media/*).<br />
<br />
In the Build Environment section of the Jenkins job, we check "Use secret text(s) or file(s)" and add 2 Bindings, one for the first secret text credential containing the AWS access key ID, which we save in a variable called AWS_ACCESS_KEY_ID, and the other one for the second secret text credential containing the AWS secret access key, which we save in a variable called AWS_SECRET_ACCESS_KEY.<br />
<br />
The Build section for this Jenkins job has a step of type "Execute shell" which uses the parameters and variables defined above and invokes 'docker run' using the path to the Docker image from our private ECR repository:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">DISTRIBUTION_ID=MY_CF_DISTRIBUTION_ID_FOR_STAGING</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">if [ $CF_ENVIRONMENT == "PRODUCTION" ]; then</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> DISTRIBUTION_ID=MY_CF_DISTRIBUTION_ID_FOR_PRODUCTION</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">fi</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">INVALIDATION_ID=jenkins-invalidation-`date +%Y%m%d%H%M%S`</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">COMMAND="aws cloudfront create-invalidation --distribution-id=$DISTRIBUTION_ID --invalidation-batch Paths={Quantity=1,Items=[$CF_RESOURCE]},CallerReference=$INVALIDATION_ID"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">docker run --rm -e AWS_ACCESS_KEY_ID=$ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$SECRET_ACCESS_KEY -e AWS_COMMAND="$COMMAND" MY_PRIVATE_ECR_ID.dkr.ecr.us-west-2.amazonaws.com/awscli</span><br />
<br />
<br />
When this job is run, the Docker image gets pulled down from AWS ECR, then a container based on the image is run and then removed upon completion (that's what <span style="font-family: "courier new" , "courier" , monospace;">--rm</span> does, so that no old containers are left around).<br />
<br />
I'll write another post soon with some more examples of Jenkins jobs that we run as Docker containers to do Selenium test, JSHint testing and SSLyze scanning.<br />
<br />
<br />
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com2tag:blogger.com,1999:blog-9238405.post-27757834343076971082016-05-26T11:00:00.002-07:002016-05-26T11:00:57.089-07:00Setting up AWS CloudFront for MagentoHere are some steps I jotted down for setting up AWS CloudFront as a CDN for the 3 asset directories that are used by Magento installations. I am assuming your Magento application servers are behind an ELB.<br />
<br />
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(66, 66, 66); -webkit-text-stroke-width: initial; color: #424242; font-family: Arial; font-size: 19px; line-height: normal; margin-bottom: 5.3px;">
SSL certificate upload to AWS</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(102, 102, 102); -webkit-text-stroke-width: initial; color: #666666; font-family: Arial; font-size: 16px; line-height: normal; margin-bottom: 5.3px;">
Install aws command line utilities.</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
$ pip install awscli</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(102, 102, 102); -webkit-text-stroke-width: initial; color: #666666; font-family: Arial; font-size: 16px; line-height: normal; margin-bottom: 5.3px;">
Configure AWS credentials</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br style="font-family: -webkit-standard;" /><span style="font-family: -webkit-standard; font-size: small;">Create IAM user and associate it with the IAMFullAccess policy. Run ‘aws configure’ and specify the user’s keys and the region.</span><br style="font-family: -webkit-standard;" /><br style="font-family: -webkit-standard;" /><span style="font-family: -webkit-standard; font-size: small;">Bring SSL key, certificate and intermediate certificate in current directory:</span></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
-rw-r--r-- 1 root root 4795 Apr 11 20:34 gd_bundle-g2-g1.crt</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
-rw-r--r-- 1 root root 1830 Apr 11 20:34 wildcard.mydomain.com.crt</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
-rw------- 1 root root 1675 Apr 11 20:34 wildcard.mydomain.com.key</div>
<div style="-webkit-text-stroke-color: rgb(102, 102, 102); -webkit-text-stroke-width: initial; color: #666666; font-family: Arial; font-size: 16px; line-height: normal; margin-bottom: 5.3px; min-height: 18px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<span style="font-family: -webkit-standard; font-size: small;">Run following script for installing wildcard SSL certificate to be used in staging CloudFront setup:</span></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
$ cat add_ssl_cert_to_iam_for_prod_cloudfront.sh</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
#!/bin/bash</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
aws iam upload-server-certificate --server-certificate-name WILDCARD_MYDOMAIN_COM_FOR_PROD_CF --certificate-body file://wildcard.mydomain.com.crt --private-key file://wildcard.mydomain.com.key --certificate-chain file://gd_bundle-g2-g1.crt --path /cloudfront/prod/</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<span style="font-family: -webkit-standard; font-size: small;">After uploading the SSL certificates, they will be available in drop-downs when configuring CloudFront for SSL.</span></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<span style="font-family: -webkit-standard; font-size: small;"><br /></span></div>
<div style="-webkit-text-stroke-color: rgb(66, 66, 66); -webkit-text-stroke-width: initial; color: #424242; font-family: Arial; font-size: 19px; line-height: normal; margin-bottom: 5.3px;">
Apache Cache-Control headers setup</div>
<ul>
<li>Add these directives (modifying max-age accordingly) in all Apache vhosts, both for port 80 and for port 443</li>
</ul>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|js|css|swf)$"></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
Header set Cache-Control "max-age=604800, public"</div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
</FilesMatch></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: 'Courier New'; font-size: 15px; line-height: normal;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(66, 66, 66); -webkit-text-stroke-width: initial; color: #424242; font-family: Arial; font-size: 19px; line-height: normal; margin-bottom: 5.3px;">
CloudFront setup</div>
<ul>
<li>Origin: prod ELB (mydomain-production-lb-9321962155.us-west-2.elb.amazonaws.com)</li>
<li>Alternate domain name: cdn.mydomain.com\</li>
<li>SSL certificate: ID_OF_CERTIFICATE_UPLOADED_ABOVE</li>
<li>Custom SSL client support: Only Clients that Support Server Name Indication (SNI)</li>
<li>Domain name: eg7ac9k0fa3qwc.cloudfront.net</li>
<li>Behaviors</li>
<ul>
<li>/media/* /skin/* /js/*</li>
<li>Viewer protocol policy: HTTP and HTTPS</li>
<li>Allowed HTTP methods: GET, HEAD</li>
<li>Forward headers: None</li>
<li>Object caching: Use origin cache headers</li>
<li>Forward cookies: None</li>
<li>Forward query strings: Yes</li>
<li>Smooth streaming: No</li>
<li>Restrict viewer access: No</li>
<li>Compress objects automatically: No</li>
</ul>
</ul>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(66, 66, 66); -webkit-text-stroke-width: initial; color: #424242; font-family: Arial; font-size: 19px; line-height: normal; margin-bottom: 5.3px;">
DNS setup<span style="-webkit-text-stroke-color: rgb(0, 0, 0); color: black; font-size: 15px; line-height: normal;"><br />
</span></div>
<ul>
<li>cdn.mydomain.com is a CNAME pointing to the CloudFront domain name above eg7ac9k0fa3qwc.cloudfront.net</li>
</ul>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<span style="-webkit-text-stroke-color: rgb(66, 66, 66); color: #424242; font-family: Arial; font-size: 19px;">Magento setup</span></div>
<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<span style="-webkit-text-stroke-color: rgb(66, 66, 66); color: #424242; font-family: Arial; font-size: 19px;"><br /></span></div>
This depends on the version of Magento you are running (1.x or 2.x), but you want to look for settings for the Base Skin URL, Base Media URL and Base Javascript URL, which are usually under System->Configuration->General-Web. You need to set them to point to the domain name you set up as a CNAME to CloudFront.<br /><br />Base Skin URL: http://cdn.mydomain.com/skin<br />Base Media URL: http://cdn.mydomain.com/media<br />Base Javascript URL: http://cdn.mydomain.com/js<div>
<br />More in-depth Magento-specific instructions for integrating with CloudFront are available <a href="https://www.thirdandgrove.com/integrating-magento-with-amazons-cdn-cloudfront">here</a>.<div style="-webkit-text-stroke-color: rgb(0, 0, 0); -webkit-text-stroke-width: initial; font-family: Times; font-size: 12px; line-height: normal; min-height: 14px;">
<br /></div>
</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-1042322931124522282016-04-15T11:24:00.001-07:002016-04-15T11:24:28.930-07:00LDAP server setup and client authenticationWe recently bought at work a CloudBees Jenkins Enterprise license and I wanted to tie the user accounts to a directory service. I first tried to set up Jenkins authentication via the AWS Directory Service, hoping it will be pretty much like talking to an Active Directory server. That proved to be impossible to set up, at least for me. I also tried to have an LDAP proxy server talking to the AWS Directory Service and have Jenkins authenticate against the LDAP proxy. No dice. I ended up setting up a good old-fashioned LDAP server and managed to get Jenkins working with it. Here are some of my notes.<br />
<br />
<h2>
<span style="font-size: x-large;">OpenLDAP server setup</span></h2>
<br />
I followed this excellent <a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-openldap-and-phpldapadmin-on-an-ubuntu-14-04-server">guide</a> from Digital Ocean. The server was an Ubuntu 14.04 EC2 instance in my case. What follows in terms of the server setup is taken almost verbatim from the DO guide.<br />
<br />
<h3 dir="ltr" style="line-height: 1.38; margin-bottom: 8pt; margin-top: 11pt;">
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Set the hostname</span></span></h3>
<b id="docs-internal-guid-1fb3b3c5-1acf-aa77-475a-382e45e37c7a" style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># hostnamectl set-hostname my-ldap-server</b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Edit /etc/hosts and make sure this entry exists:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b><span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">LOCAL_IP_ADDRESS my-ldap-server.mycompany.com </span><span style="font-size: 15px; white-space: pre-wrap;">my-ldap-server</span></span></b></div>
<b style="font-weight: normal;"><br /></b>
<b style="font-weight: normal;">(it makes a difference that the FQDN is the first entry in the line above!)</b><br />
<b style="font-weight: normal;"><br /></b>
<b style="font-weight: normal;">Make sure the following types of names are returned when you run hostname with different options:</b><br />
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># hostname</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace; font-size: 15px; white-space: pre-wrap;">my-ldap-server</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace; font-size: 15px; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># hostname -f</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">my-ldap-server.mycompany.com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># hostname -d</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">mycompany.com</span><br />
<span style="background-color: transparent; color: black; font-family: "courier new"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Install slapd</span></span></h4>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># apt-get install slapd ldap-utils</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># dpkg-reconfigure slapd</b></span></div>
<br />
(here you specify the LDAP admin password)<br />
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Install the SSL Components</span></span></h4>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># apt-get install gnutls-bin ssl-cert</b></span><br />
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><br /></b></span></div>
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Create the CA Template</span></span></h4>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><b style="font-size: 15px; white-space: pre-wrap;"><br /></b></b></span>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><b style="font-size: 15px; white-space: pre-wrap;"># mkdir /etc/ssl/templates</b></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi /etc/ssl/templates/ca_server.conf</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat /etc/ssl/templates/ca_server.conf</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">cn = LDAP Server CA</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ca</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">cert_signing_key</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Create the LDAP Service Template</span></span></h4>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi /etc/ssl/templates/ldap_server.conf</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat /etc/ssl/templates/ldap_server.conf</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">organization = "My Company"</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">cn = </span><span style="font-size: 15px; white-space: pre-wrap;">my-ldap-server.mycompany.com</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tls_www_server</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">encryption_key</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">signing_key</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">expiration_days = 3650</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Create the CA Key and Certificate</span></span></h4>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># certtool -p --outfile /etc/ssl/private/ca_server.key</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># certtool -s --load-privkey /etc/ssl/private/ca_server.key --template /etc/ssl/templates/ca_server.conf --outfile /etc/ssl/certs/ca_server.pem</b></span></div>
<b style="font-weight: normal;"><br /></b>
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Create the LDAP Service Key and Certificate</span></span></h4>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># certtool -p --sec-param high --outfile /etc/ssl/private/ldap_server.key</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># certtool -c --load-privkey /etc/ssl/private/ldap_server.key --load-ca-certificate /etc/ssl/certs/ca_server.pem --load-ca-privkey /etc/ssl/private/ca_server.key --template /etc/ssl/templates/ldap_server.conf --outfile /etc/ssl/certs/ldap_server.pem</b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h3>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Give OpenLDAP Access to the LDAP Server Key</span></span></h3>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># usermod -aG ssl-cert openldap</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># chown :ssl-cert /etc/ssl/private/ldap_server.key</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># chmod 640 /etc/ssl/private/ldap_server.key</b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h3>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Configure OpenLDAP to Use the Certificate and Keys</span></span></h3>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b><br /></b></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<b>IMPORTANT NOTE</b>: in modern versions of slapd, configuring the server is not done via slapd.conf anymore. Instead, you put together ldif files and run LDAP client utilities such as <span style="font-family: Courier New, Courier, monospace;">ldapmodify</span> against the local server. The Distinguished Name of the entity you want to modify in terms of configuration is generally <span style="font-family: 'Courier New', Courier, monospace; font-size: 15px; white-space: pre-wrap;">dn: cn=config </span>but it can also be the LDAP database <span style="font-family: 'Courier New', Courier, monospace; font-size: 15px; white-space: pre-wrap;">dn: olcDatabase={1}hdb,cn=config.</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><br /></b></span>
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi addcerts.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat addcerts.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: cn=config</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">changetype: modify</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcTLSCACertificateFile</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcTLSCACertificateFile: /etc/ssl/certs/ca_server.pem</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">-</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcTLSCertificateFile</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcTLSCertificateFile: /etc/ssl/certs/ldap_server.pem</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">-</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcTLSCertificateKeyFile</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcTLSCertificateKeyFile: /etc/ssl/private/ldap_server.key</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapmodify -H ldapi:// -Y EXTERNAL -f addcerts.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># service slapd force-reload</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cp /etc/ssl/certs/ca_server.pem /etc/ldap/ca_certs.pem</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi /etc/ldap/ldap.conf</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">* set TLS_CACERT to following:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">TLS_CACERT /etc/ldap/ca_certs.pem</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapwhoami -H ldap:// -x -ZZ</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Anonymous</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h3>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Force Connections to Use TLS</span></span></h3>
<br />
Change <span style="font-family: Courier New, Courier, monospace;">olcSecurity</span> attribute to include '<span style="font-family: Courier New, Courier, monospace;">tls=1</span>':<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi forcetls.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat forcetls.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: olcDatabase={1}hdb,cn=config</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">changetype: modify</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcSecurity</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcSecurity: tls=1</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapmodify -H ldapi:// -Y EXTERNAL -f forcetls.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># service slapd force-reload</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapsearch -H ldap:// -x -b "dc=mycompany,dc=com" -LLL dn</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(</span><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>shouldn’t work</b></span><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapsearch -H ldap:// -x -b "dc=mycompany,dc=com" -LLL -Z dn</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(</span><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>should work</b></span><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Disallow anonymous bind</span></span></h4>
<br />
Create user <span style="font-family: Courier New, Courier, monospace;">binduser</span> to be used for LDAP searches:<b style="font-weight: normal;"><br /></b>
<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi binduser.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat binduser.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: cn=binduser,dc=mycompany,dc=com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: top</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: account</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: posixAccount</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: shadowAccount</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">cn: </span><span style="font-size: 15px; white-space: pre-wrap;">binduser</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">uid: </span><span style="font-size: 15px; white-space: pre-wrap;">binduser</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">uidNumber: 2000</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">gidNumber: 200</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">homeDirectory: /home/</span><span style="font-size: 15px; white-space: pre-wrap;">binduser</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">loginShell: /bin/bash</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">gecos: suser</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">userPassword: {crypt}x</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">shadowLastChange: -1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">shadowMax: -1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">shadowWarning: -1</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapadd -x -W -D "cn=admin,dc=mycompany,dc=com" -Z -f binduser.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Enter LDAP Password:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">adding new entry "cn=binduser,dc=mycompany,dc=com"</span></div>
<b style="font-weight: normal;"><br /></b>
Change <span style="font-family: Courier New, Courier, monospace;">olcDissalows</span> attribute to include <span style="font-family: Courier New, Courier, monospace;">bind_anon</span>:<b style="font-weight: normal;"><br /></b>
<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi disallow_anon_bind.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat disallow_anon_bind.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: cn=config</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">changetype: modify</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcDisallows</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcDisallows: bind_anon</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapmodify -H ldapi:// -Y EXTERNAL -ZZ -f disallow_anon_bind.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># service slapd force-reload</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
Also disable anonymous access to frontend:<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "courier new"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># vi disable_anon_frontend.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat disable_anon_frontend.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: olcDatabase={-1}frontend,cn=config</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">changetype: modify</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcRequires</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcRequires: authc</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># ldapmodify -H ldapi:// -Y EXTERNAL -f disable_anon_frontend.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># service slapd force-reload</b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span style="font-size: large;">Create organizational units and users</span></span></h4>
<br />
Create helper scripts:<br />
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; white-space: pre-wrap;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-weight: 700; white-space: pre-wrap;"># cat add_ldap_ldif.sh</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">#!/bin/bash</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">LDIF=$1</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ldapadd -x -w adminpassword -D "cn=admin,dc=mycompany,dc=com" -Z -f $LDIF</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cat modify_ldap_ldif.sh</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">#!/bin/bash</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">LDIF=$1</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ldapmodify -x -w adminpassword -D "cn=admin,dc=mycompany,dc=com" -Z -f $LDIF</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cat set_ldap_pass.sh</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">#!/bin/bash</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">USER=$1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">PASS=$2</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ldappasswd -s $PASS -w adminpassword -D "cn=admin,dc=mycompany,dc=com" -x "uid=$USER,ou=users,dc=mycompany,dc=com" -Z</span></div>
<b style="font-weight: normal;"><br /></b><b>
Create ‘<span style="font-family: Courier New, Courier, monospace;">mypeople</span>’ organizational unit:</b><b style="font-weight: normal;"><br /></b>
<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cat add_ou_mypeople.ldif</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: ou=mypeople,dc=mycompany,dc=com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectclass: organizationalunit</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ou: users</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">description: all users</span></div>
<b style="font-weight: normal;"><span style="font-family: Courier New, Courier, monospace;"><br /></span></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># ./add_ldap_ldif.sh add_ou_mypeople.ldif</span></div>
<b style="font-weight: normal;"><br /></b><b>
Create '<span style="font-family: Courier New, Courier, monospace;">groups</span>' organizational unit:</b><b style="font-weight: normal;"><br /></b>
<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cat add_ou_groups.ldif</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: ou=groups,dc=mycompany,dc=com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectclass: organizationalunit</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ou: groups</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">description: all groups</span></div>
<span style="font-family: Courier New, Courier, monospace;"><b style="font-weight: normal;"><br /></b>
</span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># ./add_ldap_ldif.sh add_ou_groups.ldif</span></div>
<b style="font-weight: normal;"><br /></b><b>
Create users (note the shadow attributes set to -1, which means they will be ignored):</b><b style="font-weight: normal;"><br /></b>
<br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># cat add_user_myuser.ldif</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: uid=myuser,ou=mypeople,dc=mycompany,dc=com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: top</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: account</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: posixAccount</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">objectClass: shadowAccount</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">cn: </span><span style="font-size: 15px; white-space: pre-wrap;">myuser</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">uid: </span><span style="font-size: 15px; white-space: pre-wrap;">myuser</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">uidNumber: 2001</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">gidNumber: 201</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New, Courier, monospace;"><span style="background-color: transparent; color: black; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">homeDirectory: /home/</span><span style="font-size: 15px; white-space: pre-wrap;">myuser</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">loginShell: /bin/bash</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">gecos: myuser</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">userPassword: {crypt}x</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">shadowLastChange: -1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">shadowMax: -1</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">shadowWarning: -1</span></div>
<b style="font-weight: normal;"><span style="font-family: Courier New, Courier, monospace;"><br /></span></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># ./add_ldap_ldif.sh add_user_myuser.ldif</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># ./set_ldap_pass.sh myuser MYPASS</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-size: large; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Enable LDAPS</span></h4>
<div>
<br /></div>
In<span style="font-family: Courier New, Courier, monospace;"> /etc/default/slapd </span>set:<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>SLAPD_SERVICES="ldap:/// ldaps:/// ldapi:///"</b></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h4>
<span style="background-color: transparent; color: #2d2d2d; font-family: "arial"; font-size: large; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Enable debugging</span></h4>
<b style="font-weight: normal;"><br /></b>
<b style="font-weight: normal;">This was a life saver when it came to troubleshooting connection issues from clients such as Jenkins or other Linux boxes. To enable full debug output, set <span style="font-family: Courier New, Courier, monospace;">olcLogLevel</span> to -1:</b><br />
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># cat enable_debugging.ldif</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dn: cn=config</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">changetype: modify</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">add: olcLogLevel</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">olcLogLevel: -1</span></div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-size: 15px; white-space: pre-wrap;"><span style="font-family: Courier New, Courier, monospace;"><b># ldapadd -H ldapi:// -Y EXTERNAL -f enable_debugging.ldif</b></span></span><br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Courier New, Courier, monospace; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># service slapd force-reload</b></span></div>
<div>
<span style="background-color: transparent; color: black; font-family: "courier new"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="background-color: transparent; color: black; font-family: "courier new"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<h2>
<span style="font-size: x-large;">Configuring Jenkins LDAP authentication</span></h2>
<h4>
<span style="font-size: large;"><br /></span></h4>
<h4>
<span style="font-size: large;">Verify LDAPS connectivity from Jenkins to LDAP server</span></h4>
<br />
In my case, the Jenkins server is in the same VPC and subnet as the LDAP server, so I added an /etc/hosts entry on the Jenkins box pointing to the FQDN of the LDAP server so it can hit its internal IP address:<br />
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>IP_ADDRESS_OF_LDAP_SERVER my-ldap-server.mycompany.com</b></span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;">I verified that port 636 (used by LDAPS) on the LDAP server is reachable from the Jenkins server:</span></div>
<div>
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b># telnet my-ldap-server.mycompany.com 636</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Trying IP_ADDRESS_OF_LDAP_SERVER...</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Connected to </span><span style="font-family: 'Courier New'; font-size: 15px; white-space: pre-wrap;">my-ldap-server.mycompany.com.</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Escape character is '^]'.</span></div>
<b style="font-weight: normal;"><br /></b>
<h4>
<span style="font-size: large;">Set up LDAPS client on Jenkins server (StartTLSdoes not work w/ Jenkins LDAP plugin!)</span></h4>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># apt-get install ldap-utils</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; white-space: pre-wrap;"><br /></span></div>
<b>IMPORTANT</b>: Copy over <span style="font-family: Courier New, Courier, monospace;">/etc/ssl/certs/ca_server.pem</span> from LDAP server as <span style="font-family: Courier New, Courier, monospace;">/etc/ldap/ca_certs.pem</span> on Jenkins server and then:<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># vi /etc/ldap/ldap.conf</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">set:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">TLS_CACERT /etc/ldap/ca_certs.pem</span></div>
<div>
<br /></div>
<h4>
<span style="font-size: large;">Add LDAP certificates to Java keystore used by Jenkins</span></h4>
<b style="font-weight: normal;"><br /></b>
As user jenkins:<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ mkdir .keystore</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cp /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/security/cacerts .keystore/</span></div>
<b style="font-weight: normal;">(you may need to customize the above line in terms of the path to the <span style="font-family: Courier New, Courier, monospace;">cacerts</span> file -- it is the one under your <span style="font-family: Courier New, Courier, monospace;">JAVA_HOME</span>)</b><br />
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ keytool --keystore /var/lib/jenkins/.keystore/cacerts --import --alias my-ldap-server.mycompany.com:636 --file /etc/ldap/ca_certs.pem</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Enter keystore password: </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">changeit</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Owner: CN=LDAP Server CA</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Issuer: CN=LDAP Server CA</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Serial number: 570bddb0</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Valid from: Mon Apr 11 17:24:00 UTC 2016 until: Tue Apr 11 17:24:00 UTC 2017</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Certificate fingerprints:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Courier New;"><span style="font-size: 15px; white-space: pre-wrap;">....</span></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Extensions:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">....</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Trust this certificate? [no]: </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">yes</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Certificate was added to keystore</span></div>
<b style="font-weight: normal;"><br /></b>
In<span style="font-family: Courier New, Courier, monospace;"> /etc/default/jenkins</span>, set <span style="font-family: Courier New, Courier, monospace;">JAVA_ARGS</span> to:<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">JAVA_ARGS="-Djava.awt.headless=true -Djavax.net.ssl.trustStore=/var/lib/jenkins/.keystore/cacerts -Djavax.net.ssl.trustStorePassword=changeit" </span></div>
<b style="font-weight: normal;"><br /></b>
<b style="font-weight: normal;">As root, restart jenkins:</b><br />
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># service jenkins restart</span></div>
<div>
<br /></div>
<h4>
<span style="font-size: large;">Jenkins settings for LDAP plugin</span></h4>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
This took me a while to get right. The trick was to set the <span style="font-family: Courier New, Courier, monospace;"><b>rootDN</b></span> to <span style="font-family: Courier New, Courier, monospace;"><b>dc=mycompany, dc=com</b></span> and the <span style="font-family: Courier New, Courier, monospace;"><b>userSearchBase</b></span> to <span style="font-family: Courier New, Courier, monospace;"><b>ou=mypeople</b></span> (or to whatever name you gave to your users' organizational unit). I also tried to get LDAP groups to work but wasn't very successful.</div>
<b style="font-weight: normal;"><br /></b>
<b>Here is the LDAP section in <span style="font-family: Courier New, Courier, monospace;">/var/lib/jenkins/config.xml</span>:</b><b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <securityRealm class="hudson.security.LDAPSecurityRealm" plugin="ldap@1.11"></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><server>ldaps://my-ldap-server.mycompany.com:636</server></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> <rootDN>dc=mycompany,dc=com</rootDN></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> <inhibitInferRootDN>true</inhibitInferRootDN></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> <userSearchBase>ou=mypeople</userSearchBase></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> <userSearch>uid={0}</userSearch></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b> <groupSearchBase>ou=groups</groupSearchBase>
<groupMembershipStrategy class="jenkins.security.plugins.ldap.FromGroupSearchLDAPGroupMembershipStrategy">
<filter>member={0}</filter>
</groupMembershipStrategy></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b><managerDN>cn=binduser,dc=mycompany,dc=com</managerDN></b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <managerPasswordSecret>JGeIGFZwjipl6hJNefTzCwClRcLqYWEUNmnXlC3AOXI=</managerPasswordSecret></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <disableMailAddressResolver>false</disableMailAddressResolver></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <displayNameAttributeName>displayname</displayNameAttributeName></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <mailAddressAttributeName>mail</mailAddressAttributeName></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <userIdStrategy class="jenkins.model.IdStrategy$CaseInsensitive"/></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> <groupIdStrategy class="jenkins.model.IdStrategy$CaseInsensitive"/></span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </securityRealm></span></div>
<div>
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div>
<br /></div>
At this point, I was able to create users on the LDAP server and have them log in to Jenkins. With CloudBees Jenkins Enterprise, I was also able to use the Role-Based Access Control and Folder plugins in order to create project-specific folders and folder-specific groups specifying various roles. For example, a folder MyProjectNumber1 would have a Developers group defined inside it, as well as an Administrators group and a Readers group. These groups would be associated with fine-grained roles that only allow certain Jenkins operations for each group.<div>
<br /></div>
<div>
I tried to have these groups read by Jenkins from the LDAP server, but was unsuccessful. Instead, I had to populate the folder-specific groups in Jenkins with user names that were at least still defined in LDAP. So that was half a win. Still waiting to see if I can define the groups in LDAP, but for now this is a workaround that works for me.</div>
<div>
<br /></div>
<div>
<h4>
<span style="font-size: x-large;">Allowing users to change their LDAP password</span></h4>
<div>
<br /></div>
<div>
This was again a seemingly easy task but turned out to be pretty complicated. I set up another small EC2 instance to act as a jumpbox for users who want to change their LDAP password.</div>
<div>
<br /></div>
<div>
<div>
The jumpbox is in the same VPC and subnet as the LDAP server, so I added an /etc/hosts entry on the jumpbox pointing to the FQDN of the LDAP server so it can hit its internal IP address:<br />
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;"><b>IP_ADDRESS_OF_LDAP_SERVER my-ldap-server.mycompany.com</b></span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;">I verified that port 636 (used by LDAPS) on the LDAP server is reachable from the jumpbox:</span></div>
<div>
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;"><b># telnet my-ldap-server.mycompany.com 636</b></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;">Trying IP_ADDRESS_OF_LDAP_SERVER...</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;">Connected to </span><span style="font-family: 'Courier New'; font-size: 15px; white-space: pre-wrap;">my-ldap-server.mycompany.com.</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;">Escape character is '^]'.</span></div>
</div>
<div>
<br /></div>
<div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"># apt-get install ldap-utils</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; white-space: pre-wrap;"><br /></span></div>
<b>IMPORTANT</b>: Copy over <span style="font-family: Courier New, Courier, monospace;">/etc/ssl/certs/ca_server.pem</span> from LDAP server as <span style="font-family: Courier New, Courier, monospace;">/etc/ldap/ca_certs.pem</span> on the jumpbox and then:</div>
<div>
<br /><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"># vi /etc/ldap/ldap.conf</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; vertical-align: baseline; white-space: pre-wrap;">set:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">TLS_CACERT /etc/ldap/ca_certs.pem</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: 'Courier New'; font-size: 14.666666666666666px; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
</div>
<div>
Next, I followed this <a href="https://help.ubuntu.com/community/LDAPClientAuthentication">LDAP Client Authentication guide</a> from the Ubuntu documentation.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b># apt-get install ldap-auth-client nscd</b></span></div>
<div>
<br /></div>
<div>
Here I had to answer the setup questions on LDAP server FQDN, admin DN and password, and bind user DN and password. </div>
<div>
<br /></div>
<div>
<b><span style="font-family: Courier New, Courier, monospace;"># auth-client-config -t nss -p lac_ldap</span></b></div>
<div>
<br /></div>
<div>
I edited <span style="font-family: Courier New, Courier, monospace;">/etc/auth-client-config/profile.d/ldap-auth-config</span> and set:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">[lac_ldap]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">nss_passwd=passwd: ldap files</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">nss_group=group: ldap files</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">nss_shadow=shadow: ldap files</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">nss_netgroup=netgroup: nis</span></div>
</div>
<div>
<br /></div>
<div>
I edited <span style="font-family: Courier New, Courier, monospace;">/etc/ldap.conf </span>and made sure the following entries were there:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">base dc=mycompany,dc=com</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;">uri ldaps://my-ldap-server.mycompany.com</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">binddn cn=binduser,mycompany,dc=com</span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">bindpw BINDUSERPASS</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">rootbinddn cn=admin,mycompany,dc=com</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">port 636</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;">ssl on</span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">tls_cacertfile /etc/ldap/ca_certs.pem</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">tls_cacertdir /etc/ssl/certs</span></div>
</div>
<div>
<br /></div>
<div>
I allowed password-based ssh logins to the jumpbox by editing <span style="font-family: Courier New, Courier, monospace;">/etc/ssh/sshd_config</span> and setting:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">PasswordAuthentication yes</span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># service ssh restart</span></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b>IMPORTANT</b>: On the LDAP server, I had to allow users to change their own password by adding this ACL:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b># cat set_userpassword_acl.ldif</b></span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">dn: olcDatabase={1}hdb,cn=config</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">changetype: modify</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">add: olcAccess</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">olcAccess: {0}to attrs=userpassword by dn="cn=admin,dc=mycompany,dc=com" write by self write by anonymous auth by users none</span></div>
</div>
<div>
<br /></div>
<div>
Then:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b># ldapmodify -H ldapi:// -Y EXTERNAL -f set_userpassword_acl.ldif</b></span></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
At this point, users were able to log in via ssh to the jumpbox using a pre-set LDAP password, and change their LDAP password by using the regular Unix '<span style="font-family: Courier New, Courier, monospace;">passwd</span>' command.</div>
<div>
<br /></div>
<div>
I am still fine-tuning the LDAP setup on all fronts: LDAP server, LDAP client jumpbox and Jenkis server. The setup I have so far allows me to have a single sign-on account for users to log in to Jenkins. Some of my next steps is to use the same user LDAP accounts for authentication and access control into MySQL and other services.</div>
<div>
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-20610889726728856252016-04-06T13:53:00.001-07:002016-04-07T22:03:01.422-07:00Joining an EC2 Windows instance to an AWS Directory Service domainI've been struggling with the existing documentation on how to join an EC2 instance running Windows Server 2012 to an AWS Directory Service domain, so I am hastening to jot down some notes on how I got it to work.<br />
<br />
<b>1) Create an AWS Directory Service domain</b><br />
<br />
There is <a href="http://docs.aws.amazon.com/directoryservice/latest/admin-guide/what_is.html#choosing_an_option">good documentation</a> on doing this. I chose the Microsoft Active Directory option.<br />
<br />
A few notes on the creation of an AWS Directory Service:<br />
<br />
<ul>
<li>I created a new VPC with 2 subnets for the Directory Service usage</li>
<li>I made sure each subnet has an Internet gateway associated so that it can be reachable from the outside</li>
</ul>
<div>
During the creation of the Directory Service, you'll be asked to specify an administrator-type user name and password. Make sure you remember what you specified there because you'll need this info in a subsequent step. Also make note of the DNS server IP addresses that were set during the Directory Service creation.</div>
<div>
<br /></div>
<div>
<b>2) Create an IAM role to be associated with the EC2 Windows instance</b></div>
<div>
<br /></div>
<div>
<ul>
<li>the IAM role needs to be associated with the <b>AmazonEC2RoleforSSM</b> and <b>AmazonSSMFullAccess</b> policies</li>
<li>the IAM role also needs to have a trust relationship with <b>ec2.amazonaws.com</b></li>
</ul>
</div>
<div>
<br /></div>
<div>
<b>3) Launch EC2 Windows instance associated with the AWS Directory Service domain</b></div>
<div>
<br /></div>
<div>
I chose the Windows_Server-2012-R2_RTM-English-64Bit-Base-2016.03.09 AMI. In Step 3 of the AWS launch instance wizard ("Configure instance details") I made sure I specified the following:</div>
<div>
<ul>
<li>Network: the VPC created in step 1) above</li>
<li>Subnet: one of the 2 subnets created in step 1) above</li>
<li>Domain join directory: the directory name for the Directory Service created in step 1) above</li>
<li>IAM role: the IAM role created in step 2) above</li>
</ul>
<div>
<b>4) Connect to EC2 Windows instance via RDP</b></div>
</div>
<div>
<br /></div>
<div>
First get the administrator password via the AWS console (you'll need to paste the contents of the private key corresponding to the EC2 key you used when launching the Windows instance). Then connect to the Windows instance as the local administrator user.</div>
<div>
<br /></div>
<div>
Verify that you see the fully qualified domain name of your Directory Service (whatever you indicated in step 1) as the domain of the Windows instance (in Server Manager -> Local Server). If you don't, something went wrong with joining the domain during the instance launch. You can see the system log of that instance in the AWS console by selecting the instance, then going to Actions->Instance Settings->Get System Log. For example, in one of my failed attempts to get all of this working I saw errors related to the IAM role I was using, which at the time didn't have the correct SSM policies attached.</div>
<div>
<br /></div>
<div>
If the Windows instance is correctly joined to the domain, you need to install the Active Directory management tools in order to actually manage the AWS Directory Service. Here is a <a href="https://support.rackspace.com/how-to/installing-active-directory-on-windows-server-2012/">Rackspace article</a> I found with good instructions.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b>5) Log in to the EC2 Windows instance as the domain admin to manage AD</b></div>
<div>
<br /></div>
<div>
After the EC2 Windows instance was rebooted, I managed to log in via RDP as my.aws.directory.fqdn\myusername (where both of these values are the ones chosen in Step 1 above) with the password also chosen in Step 1. At this point I was able to use the Active Directory management tools to add new AD users etc.</div>
<div>
<br /></div>
<div>
Here are some other good resources I found:</div>
<div>
<ul>
<li>AWS documentation on <a href="http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2-join-aws-domain.html">joining an AWS Directory Service domain</a></li>
<li>AWS blog post on <a href="https://blogs.aws.amazon.com/bigdata/post/Tx3J2RL8V6N72G7/Using-LDAP-via-AWS-Directory-Service-to-Access-and-Administer-Your-Hadoop-Enviro">using LDAP via AWS Directory Service</a></li>
</ul>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-38234258611060199792016-02-12T12:29:00.000-08:002016-02-12T13:15:40.005-08:00Running a static website with Hugo on Google Cloud StorageI've played a bit with <a href="https://gohugo.io/">Hugo</a>, the static web site generator written in golang that has been getting a lot of good press lately. At the suggestion of my colleague Warren Runk, I also experimented with hosting the static files generated by Hugo on <a href="https://cloud.google.com/storage/">Google Cloud Storage</a> (GCS). That way there is no need for launching any instances that would serve those files. You can achieve this by using AWS S3 as well of course.<br />
<br />
<h3>
Notes on GCS setup</h3>
<br />
You first need to sign up for a Google Cloud Platform (GCP) account. You get a 30-day free trial with a new account. Once you are logged into the Google Cloud <a href="https://console.cloud.google.com/">console</a>, you need to create a new project. Let's call it <span style="font-family: "courier new" , "courier" , monospace;"><b>my-gcs-hugo-project</b></span>.<br />
<br />
You need to also create a bucket in GCS. If you want to serve your site automatically out of this bucket, you need to give the bucket the same name as your site. Let's assume you call the bucket <span style="font-family: "courier new" , "courier" , monospace;"><b>hugotest.mydomain.com</b></span>. You will have to verify that you own <span style="font-family: "courier new" , "courier" , monospace;">mydomain.com</span> either by creating a special CNAME in the DNS zone file for <span style="font-family: "courier new" , "courier" , monospace;">mydomain.com</span> pointing to google.com, or by adding a special META tag to the HTML file served at <span style="font-family: "courier new" , "courier" , monospace;">hugotest.mydomain.com</span> (you can achieve the latter by temporarily CNAME-ing <span style="font-family: "courier new" , "courier" , monospace;">hugotest</span> to <span style="font-family: "courier new" , "courier" , monospace;">www.mydomain.com</span> and adding the HEAD tag to the home page for <span style="font-family: "courier new" , "courier" , monospace;">www</span>). <br />
<br />
If you need to automate deployments to GCS, it's a good idea to create a GCP <b>Service Account</b>. Click on the 'hamburger' menu in the upper left of the GCP console, then go to Permissions, then Service Accounts. Create a new service account and download its private key in JSON format (the key will be called something like <span style="font-family: "courier new" , "courier" , monospace;"><b>my-gcs-hugo-project-a37b5acd7bc5.json</b></span>.<br />
<br />
Let's say your service account is called <span style="font-family: "courier new" , "courier" , monospace;"><b>my-gcp-service-account1</b></span>. The account will automatically be assigned an email address similar to <span style="font-family: "courier new" , "courier" , monospace;"><b>my-gcp-service-account1@my-gcs-hugo-project.iam.gserviceaccount.com</b>.</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: inherit;">I wanted to be able to deploy the static files generated by Hugo to GCS using Jenkins. So I followed these steps on the Jenkins server as the user running the Jenkins process (user </span><span style="font-family: "courier new" , "courier" , monospace;">jenkins</span><span style="font-family: inherit;"> in my case):</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">1) Installed the Google Cloud SDK</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<div style="font-family: 'Courier New'; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: 'Courier New';">
$ wget https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.tar.gz</div>
<div style="font-family: 'Courier New';">
$ tar xvfz google-cloud-sdk.tar.gz</div>
<div style="font-family: 'Courier New';">
$ cd google-cloud-sdk/</div>
<div style="font-family: 'Courier New';">
$ ./install.sh</div>
<br />
<div style="font-family: 'Courier New';">
- source .bashrc</div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<br /></div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ which gcloud</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>/var/lib/jenkins/google-cloud-sdk/bin/gcloud</b></span><br />
<div style="font-family: 'Courier New'; font-size: 12px;">
<b><br /></b></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<br /></div>
2) Copied the service account's private key <b style="font-family: 'Courier New';"><span style="font-family: "courier new" , "courier" , monospace;">my-gcs-hugo-project-a37b5acd7bc5.json</span></b> to the <span style="font-family: "courier new" , "courier" , monospace;">.ssh</span> directory of the jenkins user.<br />
<br />
3) Activated the service account using the <b><span style="font-family: "courier new" , "courier" , monospace;">gcloud</span></b> command-line utility (still as user jenkins)<br />
<br />
<div style="font-family: 'Courier New';">
<b>$ gcloud auth activate-service-account --key-file .ssh/<span style="font-family: "courier new" , "courier" , monospace;">my-gcs-hugo-project-a37b5acd7bc5.json</span></b></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">Activated service account credentials for: [<span style="font-family: "courier new" , "courier" , monospace;">my-gcp-service-account1@my-gcs-hugo-project.iam.gserviceaccount.com</span>]</span></div>
<div style="font-family: 'Courier New';">
<br /></div>
<span style="font-family: inherit;">4) Set the current GCP project to </span><b style="font-family: 'Courier New', Courier, monospace;">my-gcs-hugo-project</b><br />
<span style="font-family: inherit;"><br /></span>
<br />
<div style="font-family: 'Courier New';">
<b>$ gcloud config set project </b><b style="font-family: 'Courier New', Courier, monospace;">my-gcs-hugo-project</b></div>
<div style="font-family: 'Courier New'; min-height: 14px;">
<br /></div>
<div style="font-family: 'Courier New';">
<b>$ gcloud config list</b></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">Your active configuration is: [default]</span></div>
<div style="font-family: 'Courier New'; min-height: 14px;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">[core]</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">account = <span style="font-family: "courier new" , "courier" , monospace;">my-gcp-service-account1@my-gcs-hugo-project.iam.gserviceaccount.com</span></span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">disable_usage_reporting = True</span></div>
<br />
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">project = <span style="font-family: "courier new" , "courier" , monospace;">my-gcs-hugo-project</span></span></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<br /></div>
5) Configured GCS via the <span style="font-family: "courier new" , "courier" , monospace;"><b>gsutil</b></span> command-line utility (this may actually be redundant since we already configured the project with gcloud, but I leave it here in case you encounter issues with using just gcloud)<br />
<br />
<div style="font-family: 'Courier New';">
<b>$ gsutil config -e</b></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">It looks like you are trying to run "/var/lib/jenkins/google-cloud-sdk/bin/bootstrapping/gsutil.py config".</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">The "config" command is no longer needed with the Cloud SDK.</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">To authenticate, run: gcloud auth login</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">Really run this command? (y/N) y</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">Backing up existing config file "/var/lib/jenkins/.boto" to "/var/lib/jenkins/.boto.bak"...</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">This command will create a boto config file at /var/lib/jenkins/.boto</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">containing your credentials, based on your responses to the following</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">questions.</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">What is the full path to your private key file? <b>/var/lib/jenkins/.ssh/</b></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: x-small;"><b>my-gcs-hugo-project-a37b5acd7bc5.json</b></span></span></div>
<div style="font-family: 'Courier New'; min-height: 14px;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">Please navigate your browser to https://cloud.google.com/console#/project,</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">then find the project you will use, and copy the Project ID string from the</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">second column. Older projects do not have Project ID strings. For such projects,</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">click the project and then copy the Project Number listed under that project.</span></div>
<div style="font-family: 'Courier New'; min-height: 14px;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">What is your project-id? </span><b style="font-family: 'Courier New', Courier, monospace;"><span style="font-size: x-small;">my-gcs-hugo-project</span></b></div>
<div style="font-family: 'Courier New'; min-height: 14px;">
<span style="font-size: x-small;"><br /></span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">Boto config file "/var/lib/jenkins/.boto" created. If you need to use</span></div>
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">a proxy to access the Internet please see the instructions in that</span></div>
<br />
<div style="font-family: 'Courier New';">
<span style="font-size: x-small;">file.</span></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<br /></div>
6) Added the service account created above as an Owner for the bucket <b style="font-family: 'Courier New', Courier, monospace;"><span style="font-size: x-small;">hugotest.mydomain.com</span></b><br />
<div>
<b style="font-family: 'Courier New', Courier, monospace;"><br /></b></div>
7) Copied a test file from the local file system of the Jenkins server to the bucket <b style="font-family: 'Courier New', Courier, monospace;"><span style="font-size: x-small;">hugotest.mydomain.com</span></b> (still logged in as user jenkins), then listed all files in the bucket, then removed the test file<br />
<div>
<br /></div>
<div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<b>$ gsutil cp test.go gs://hugotest.mydomain.com/</b></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
Copying file://test.go [Content-Type=application/octet-stream]...</div>
<div style="font-family: 'Courier New'; font-size: 12px;">
Uploading gs://hugotest.mydomain.com/test.go: 951 B/951 B</div>
<div style="font-family: 'Courier New'; font-size: 12px; min-height: 14px;">
<br /></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<b>$ gsutil ls gs://hugotest.mydomain.com/</b></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
gs://hugotest.mydomain.com/test.go</div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<br /></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<b>$ gsutil rm gs://hugotest.mydomain.com/test.go</b></div>
<div style="font-family: 'Courier New'; font-size: 12px;">
Removing gs://hugotest.mydomain.com/test.go...</div>
</div>
<div>
<br /></div>
<div>
8) Created a Jenkins job for uploading all static files for a given website to GCS</div>
<div>
<br /></div>
<div>
Assuming all these static files are checked in to GitHub, the Jenkins job will first check them out, then do something like this (where <span style="font-family: "courier new" , "courier" , monospace;">TARGET</span> is the value selected from a Jenkins multiple-choice dropdown for this job):</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">BUCKETNAME=$TARGET</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># upload all filee and disable caching (for testing purposes)</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>gsutil -h "Cache-Control:private" cp -r * gs://$BUCKETNAME/</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># set read permissions for allUsers</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">for file in `find . -type f`; do</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> # remove first dot from file name</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> file=${file#"."}</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> <b>gsutil acl ch -u allUsers:R gs://${BUCKETNAME}${file}</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">done</span></div>
</div>
<div>
<br /></div>
<div>
The first <span style="font-family: "courier new" , "courier" , monospace;">gsutil</span> command does a recursive copy (<span style="font-family: "courier new" , "courier" , monospace;">cp -r *</span>) of all files to the bucket. This will preserve the directory structure of the website. For testing purposes, the gsutil command also sets the <span style="font-family: "courier new" , "courier" , monospace;">Cache-Control</span> header on all files to <span style="font-family: "courier new" , "courier" , monospace;">private</span>, which tells browsers not to cache the files.</div>
<div>
<br /></div>
<div>
The second <span style="font-family: "courier new" , "courier" , monospace;">gsutil</span> command is executed for each object in the bucket, and it sets the ACL on that object so that the object has Read (<span style="font-family: "courier new" , "courier" , monospace;">R</span>) permissions for <span style="font-family: "courier new" , "courier" , monospace;">allUsers</span> (by default only owners and other specifically assigned users have Read permissions). This is because we want to serve a public website out of our GCS bucket.</div>
<div>
<br /></div>
<div>
At this point, you should be able to hit <span style="font-family: "courier new" , "courier" , monospace;">hugotest.mydomain.com</span> in a browser and see your static site in all its glory.</div>
<div>
<br /></div>
<h3>
Notes on Hugo setup</h3>
<div>
<br /></div>
<div>
I've only dabbled in Hugo in the last couple of weeks, so these are very introductory-type notes.</div>
<div>
<br /></div>
<div>
<b>Installing Hugo on OSX and creating a new Hugo site</b></div>
<div>
<br /></div>
<div>
<div style="font-family: 'Courier New';">
$ brew update && brew install hugo</div>
<div style="font-family: 'Courier New';">
$ mkdir hugo-sites</div>
<div style="font-family: 'Courier New';">
$ cd hugo-sites</div>
<div style="font-family: 'Courier New';">
$ <b>hugo new site hugotest.mydomain.com</b></div>
<div style="font-family: 'Courier New';">
$ git clone --recursive https://github.com/spf13/hugoThemes themes</div>
<div style="font-family: 'Courier New';">
$ cd hugotest.mydomain.com</div>
<div style="font-family: 'Courier New';">
$ ln -s ../themes .</div>
<div style="font-family: 'Courier New'; font-size: 12px;">
<br /></div>
</div>
<div>
At this point you have a skeleton directory structure created by Hugo (via the <span style="font-family: "courier new" , "courier" , monospace;">hugo new site</span> command) under the directory <span style="font-family: "courier new" , "courier" , monospace;">hugotest.mydomain.com:</span></div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ ls</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">archetypes config.toml content data layouts static themes</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
(note that we symlinked the themes directory into the <span style="font-family: "courier new" , "courier" , monospace;">hugotest.mydomain.com</span> directory to avoid duplication)<br />
<div>
<br /></div>
<div>
<b>Configuring your Hugo site and choosing a theme</b></div>
<div>
<br /></div>
<div>
One file you will need to pay a lot of attention to is the site configuration file <span style="font-family: "courier new" , "courier" , monospace;">config.toml</span>. The default content of this file is deceptively simple:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ cat config.toml</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">baseurl = "http://replace-this-with-your-hugo-site.com/"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">languageCode = "en-us"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">title = "My New Hugo Site"</span></div>
</div>
<div>
<br /></div>
<div>
Before you do anything more, you need to decide on a theme for your site. Browse the <a href="http://themes.gohugo.io/">Hugo Themes</a> page and find something you like. Let's assume you choose the <a href="http://themes.gohugo.io/casper/">Casper</a> theme. You will need to become familiar with the customizations that the theme offers. Here are some customizations I made in config.toml, going by the examples on the Casper theme web page:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ cat config.toml</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>baseurl = "http://hugotest.mydomain.com/"</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>languageCode = "en-us"</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>title = "My Speedy Test Site"</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>newContentEditor = "vim"</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>theme = "casper"</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">canonifyurls = true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">[params]</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> description = "Serving static sites at the speed of light"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> cover = "images/header.jpg"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> logo = "images/mylogo.png"</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> # set true if you are not proud of using Hugo (true will hide the footer note "Proudly published with HUGO.....")</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hideHUGOSupport = false</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># author = "Valère JEANTET"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># authorlocation = "Paris, France"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># authorwebsite = "http://vjeantet.fr"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># bio= "my bio"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># googleAnalyticsUserID = "UA-79101-12"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># # Optional RSS-Link, if not provided it defaults to the standard index.xml</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># RSSLink = "http://feeds.feedburner.com/..."</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># githubName = "vjeantet"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"># twitterName = "vjeantet"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> # facebookName = ""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> # linkedinName = ""</span></div>
</div>
<div>
<br /></div>
<div>
I left most of the Casper-specific options commented out and only specified a cover image, a logo and a description. </div>
<div>
<br /></div>
<div>
<b>Creating a new page</b></div>
<div>
<br /></div>
<div>
If you want blog-style posts to appear on your home page, create a new page with Hugo under a directory called <span style="font-family: "courier new" , "courier" , monospace;">post</span> (some themes want this directory to be named <span style="font-family: "courier new" , "courier" , monospace;">post</span> and others want it <span style="font-family: "courier new" , "courier" , monospace;">posts</span>, so check what the theme expects). </div>
<div>
<br /></div>
<div>
Let's assume you want to create a page caled <span style="font-family: "courier new" , "courier" , monospace;">hello-world.md</span> (I haven't even mentioned this so far, but Hugo deals by default with <a href="https://daringfireball.net/projects/markdown/">Markdown</a> pages, so you will need to brush up a bit on our Markdown skills). You would run:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">$ hugo new post/hello-world.md</span></div>
<div>
<br /></div>
<div>
This creates the <span style="font-family: "courier new" , "courier" , monospace;">post</span> directory under the <span style="font-family: "courier new" , "courier" , monospace;">content</span> directory, creates a file called <span style="font-family: "courier new" , "courier" , monospace;">hello-world.md</span> in <span style="font-family: "courier new" , "courier" , monospace;">content/post,</span> and opens up the file for editing in the editor you specified as the value for newContentEditor in config.toml (vim in my case). The default contents of the md file are specific to the theme you used. For Casper, here is what I get by default:</div>
<div>
<br /></div>
<div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">+++</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">author = ""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">comments = true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">date = "2016-02-12T11:54:32-08:00"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">draft = false</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">image = ""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">menu = ""</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">share = true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">slug = "post-title"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">tags = ["tag1", "tag2"]</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">title = "hello world"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">+++</span></div>
</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Now add some content to that file and save it. Note that the <span style="font-family: "courier new" , "courier" , monospace;">draft</span> property is set to <span style="font-family: "courier new" , "courier" , monospace;">false</span> by the Casper theme. Other themes set it to <span style="font-family: "courier new" , "courier" , monospace;">true</span>, in which case it would not be published by Hugo by default. The slug property is set by Casper to <span style="font-family: "courier new" , "courier" , monospace;">"post-title"</span> by default. I changed it to <span style="font-family: "courier new" , "courier" , monospace;">"hello-world"</span>. I also changed the tags list to only contain one tag I called <span style="font-family: "courier new" , "courier" , monospace;">"blog"</span>.</div>
<div>
<br /></div>
<div>
At this point, you can run the <span style="font-family: "courier new" , "courier" , monospace;">hugo</span> command by itself, and it will take the files it finds under content, static, and its other subdirectories, turn them into html/js/css/font files and save it in a directory called <span style="font-family: "courier new" , "courier" , monospace;">public</span>:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ hugo</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 draft content</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 future content</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 pages created</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">3 paginator pages created</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 tags created</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 categories created</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">in 55 ms</span></div>
</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ find public</b></span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/404.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/css</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/css/nav.css</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/css/screen.css</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts/example.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts/genericons.css</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts/Genericons.eot</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts/Genericons.svg</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts/Genericons.ttf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/fonts/Genericons.woff</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/index.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/index.xml</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/js</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/js/index.js</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/js/jquery.fitvids.js</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/js/jquery.js</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/page</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/page/1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/page/1/index.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/hello-world</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/hello-world/index.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/index.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/index.xml</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/page</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/page/1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/post/page/1/index.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/sitemap.xml</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags/blog</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags/blog/index.html</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags/blog/index.xml</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags/blog/page</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags/blog/page/1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">public/tags/blog/page/1/index.html</span></div>
</div>
</div>
<div>
<br /></div>
<div>
That's quite a number of files and directories created by hugo. Most of it is boilerplate coming from the theme. Our <span style="font-family: "courier new" , "courier" , monospace;">hello-world.md</span> file was turned into a directory called <span style="font-family: "courier new" , "courier" , monospace;">hello-world</span> under <span style="font-family: "courier new" , "courier" , monospace;">public/post</span>, with an <span style="font-family: "courier new" , "courier" , monospace;">index.html</span> file dropped in it. Note that the Casper theme names the <span style="font-family: "courier new" , "courier" , monospace;">hello-world</span> directory after the <span style="font-family: "courier new" , "courier" , monospace;">slug</span> property in the <span style="font-family: "courier new" , "courier" , monospace;">hello-world.md</span> file.</div>
<div>
<br /></div>
<div>
<b>Serving the site locally with Hugo</b></div>
<div>
<br /></div>
<div>
Hugo makes it very easy to check your site locally. Just run<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ hugo server</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 draft content</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 future content</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 pages created</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">3 paginator pages created</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 tags created</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 categories created</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">in 35 ms</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Watching for changes in /Users/grig.gheorghiu/mycode/hugo-sites/hugotest.mydomain.com/{data,content,layouts,static,themes}</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Serving pages from memory</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Press Ctrl+C to stop</span><br />
<br />
<br />
Now if you browse to <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:1313</span> you should see something similar to this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1XAotQX2W7Ni84LxfLtmRFHoo687oW_ujxn-IcJGdPU-f_d6lmwChQj-hatCREXHMh3ti_SmuUDSP49n98mVY9cfxu0CEtBmZHcf4jx-6S5dUYsuQeKbqLRs8xSFZ0Dsugog_Jw/s1600/Screen+Shot+2016-02-12+at+12.18.10+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1XAotQX2W7Ni84LxfLtmRFHoo687oW_ujxn-IcJGdPU-f_d6lmwChQj-hatCREXHMh3ti_SmuUDSP49n98mVY9cfxu0CEtBmZHcf4jx-6S5dUYsuQeKbqLRs8xSFZ0Dsugog_Jw/s320/Screen+Shot+2016-02-12+at+12.18.10+PM.png" width="320" /></a></div>
<br />
Not bad for a few minutes of work.<br />
<br />
For other types of content, such as static pages not displayed on the home page, you can create Markdown files in a <span style="font-family: "courier new" , "courier" , monospace;">pages</span> directory:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ hugo new pages/static1.md</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+++</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">author = ""</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">comments = true</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">date = "2016-02-12T12:24:26-08:00"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">draft = false</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">image = ""</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">menu = "main"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">share = true</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">slug = "static1"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">tags = ["tag1", "tag2"]</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">title = "static1"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+++</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Static page 1.</span><br />
<br />
Note that the menu property value is "main" in this case. This tells the Casper theme to create a link to this page in the main drop-down menu available on the home page.<br />
<br />
If you run hugo server again, you should see something the menu available in the upper right corner, and a link to static1 when you click on the menu:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3ljB9noFzAbPv1x7JYLeFG_s7BsX3uK-3mTrBkMmZWupofhybJGWM3rkj8vV3P9_U0r7CwArZ157qJg4PpR4_ORtXMG6rRhyzrPuvOFes2_Tk29_BYZrTT5o4_aW8xHy-EeZIRw/s1600/Screen+Shot+2016-02-12+at+12.26.51+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="125" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3ljB9noFzAbPv1x7JYLeFG_s7BsX3uK-3mTrBkMmZWupofhybJGWM3rkj8vV3P9_U0r7CwArZ157qJg4PpR4_ORtXMG6rRhyzrPuvOFes2_Tk29_BYZrTT5o4_aW8xHy-EeZIRw/s320/Screen+Shot+2016-02-12+at+12.26.51+PM.png" width="320" /></a></div>
<br />
<br />
<br />
To deploy your site to GCS, S3 or regular servers, you need to upload the files and directories under the <span style="font-family: Courier New, Courier, monospace;">public</span> directory. It's that simple.<br />
<br />
I'll stop here with my Hugo notes. DigitalOcean has a great tutorial on <a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-use-hugo-a-static-site-generator-on-ubuntu-14-04">installing and running Hugo on Ubuntu 14.04</a>.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-30200596864501302722016-02-11T14:09:00.001-08:002016-02-11T14:09:26.717-08:00Some notes on Ansible playbooks and rolesSome quick notes I jotted down while documenting our Ansible setup. Maybe they will be helpful for people new to Ansible.<br />
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Ansible playbooks and roles</span></h2>
<b id="docs-internal-guid-7d309269-d25d-877d-d057-e60cc4d4909d" style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Playbooks are YAML files that specify which roles are applied to hosts of certain type. </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Example: </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api-servers.yml</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cat api-servers.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">---</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts: api</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> sudo: yes</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> roles:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - base</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - tuning</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - postfix</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - monitoring</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - nginx</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - api</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - logstash-forwarder</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This says that for each host in the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group we will run tasks defined in the roles listed above.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Example of a role</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">base</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> role is one that (in our case) is applied to all hosts. Here is its directory/file structure:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/defaults</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/defaults/main.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/newrelic</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/newrelic/newrelic-sysmond_2.0.2.111_amd64.deb</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/pubkeys</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/pubkeys/id_rsa.pub.jenkins</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/rsyslog</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/rsyslog/50-default.conf</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/rsyslog/60-papertrail.conf</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/rsyslog/papertrail-bundle.pem</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/sudoers.d</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/files/sudoers.d/10-admin-users</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/handlers</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/handlers/main.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/meta</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/meta/main.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/README.md</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/tasks</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/tasks/install.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/tasks/main.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/tasks/newrelic.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/tasks/papertrail.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/tasks/users.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/templates</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/templates/hostname.j2</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/templates/nrsysmond.cfg.j2</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/vars</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 12px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">roles/base/vars/main.yml</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">An Ansible role has the following important sub-directories:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>defaults</b></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - contains the main.yml file which defines default values for variables used throughout other role files; note that the role’s files are checked in to GitHub, so these values shouldn’t contain secrets such as passwords, API keys etc. For those types of variables, use </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">group_vars</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> or </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">host_vars</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> files which will be discussed below.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>files</b></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - contains static files that are copied over by ansible tasks to remote hosts</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>handlers</b></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - contains the main.yml file which defines actions such as stopping/starting/restarting services such as nginx, rsyslog etc.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>meta</b></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - metadata about the role; things like author, description etc.</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">tasks</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - the meat and potatoes of ansible, contains one or more files that specify the actions to be taken on the host that is being configured; the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">main.yml</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> file contains all the other files that get executed</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Here are 2 examples of task files, one for configuring rsyslog to send logs to Papertrail and the other for installing the newrelic agent:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cat tasks/papertrail.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- name: copy papertrail pem certificate file to /etc</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> copy: ></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> src=rsyslog/{{item}}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> dest=/etc/{{item}}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> with_items:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - papertrail-bundle.pem</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- name: copy rsyslog config files for papertrail integration</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> copy: ></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> src=rsyslog/{{item}}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> dest=/etc/rsyslog.d/{{item}}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> with_items:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - 50-default.conf</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - 60-papertrail.conf</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> notify:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - restart rsyslog</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cat tasks/newrelic.yml</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- name: copy newrelic debian package</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> copy: ></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> src=newrelic/{{newrelic_deb_pkg}}</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> dest=/opt/{{newrelic_deb_pkg}}</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- name: install newrelic debian package</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> apt: deb=/opt/{{newrelic_deb_pkg}}</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- name: configure newrelic with proper license key</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> template: ></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> src=nrsysmond.cfg.j2</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> dest=/etc/newrelic/nrsysmond.cfg</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> owner=newrelic</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group=newrelic</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> mode=0640</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> notify:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - restart newrelic</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>templates</b></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - contains Jinja2 templates with variables that get their values from </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">defaults/main.yml</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> or from </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">group_vars</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> or </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">host_vars</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> files. One special variable that we use (and is not defined in these files, but instead is predefined by Ansible) is </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">inventory_hostname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> which points to the hostname of the target being configured. For example, here is the template for a hostname file which will be dropped into </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">/etc/hostname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> on the target:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cat roles/base/templates/hostname.j2</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">{{ inventory_hostname }}</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Once you have a playbook and a role, there are a few more files you need to take care of:</span></div>
<b style="font-weight: normal;"><br /></b>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - this is an INI-type file which defines groups of hosts. For example the following snippet of this file defines 2 groups called </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> and </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">magento</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
</li>
</ul>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">[api]</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api01 ansible_ssh_host=api01.mydomain.co</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api02 ansible_ssh_host=api02.mydomain.co</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">[magento]</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">mgto ansible_ssh_host=mgto.mydomain.co</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api-servers.yml</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> playbook file referenced at the beginning of this document sets the hosts variable to the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group, so all Ansible tasks will get run against the hosts included in that group. In the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> file above, these hosts are </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api01</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> and </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api02</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<b style="font-weight: normal;"><br /></b>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">group_vars/somegroupname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - this is where variables with ‘secret’ values get defined for a specific group called </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">somegroupname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. The </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">group_vars</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> directory is not checked into GitHub. </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">somegroupname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> needs to exactly correspond to the group defined in </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
</li>
</ul>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Example:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cat group_vars/api</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ses_smtp_endpoint: email-smtp.us-west-2.amazonaws.com</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ses_smtp_port: 587</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ses_smtp_username: some_username</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">ses_smtp_password: some_password</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">datadog_api_key: some_api_key</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-indent: 36pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. . . other variables (DB credentials etc)</span></div>
<b style="font-weight: normal;"><br /></b><br />
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">host_vars/somehostname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> - this is where variables with ‘secret’ values get defined for a specific host called </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">somehostname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. The </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">host_vars</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> directory is not checked into GitHub. </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">somehostname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> needs to exactly correspond to a host defined in </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
</li>
</ul>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Example:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ cat host_vars/api02</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 13.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">insert_sample_data: false</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-left: 36pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This overrides the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">insert_sample_data</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> variable and sets it to </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">false</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> only for the host called </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api02</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. This could also be used for differentiating between a DB master and slave for example. </span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Tying it all together</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">First you need to have ansible installed on your local machine. I used:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ pip install ansible</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">To execute a playbook for a given hosts file against all api server, you would run:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ ansible-playbook -i hosts/myhosts api-servers.yml</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The name that ties together the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> file, the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api-servers.yml</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> file and the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">group_vars/groupname</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> file is in this case </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. </span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">You need to make sure you have the desired values for that group in these 3 files:</span></div>
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: make sure you have the desired hosts under the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">[api]</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api-server.yml</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: make sure you have the desired roles for hosts in the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">group_vars/api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: make sure you have the desired values for variables that will be applied to the hosts in the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">api</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group</span></div>
</li>
</ul>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Launching a new api instance in EC2</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I blogged about this </span><a href="http://agiletesting.blogspot.com/2015/06/automatically-launching-and-configuring.html" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">here</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Updating an existing api instance</span></h2>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Make sure the instance hostname is the only hostname in the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">[api]</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> group in the </span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">hosts/myhosts</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> file. Then run:</span></div>
<b style="font-weight: normal;"><br /></b>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">$ ansible-playbook -i hosts/myhosts api-servers.yml</span></div>
<br /><br />Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-59439090957077619562016-02-09T16:09:00.000-08:002016-02-09T16:20:34.073-08:00Setting up a mailinator-like test mail server with postfix and MailHogThe purpose of this exercise is to set up a mailinator-style mail server under our control. If you haven't used mailinator, it's a free service which provides an easy way to test sending email to random recipients. If you send mail to somebody189@mailinator.com and then go to<br />
<a href="https://mailinator.com/inbox.jsp?to=somebody189">https://mailinator.com/inbox.jsp?to=somebody189</a>, you will see a mailbox associated with that user, and any incoming email messages destined for that user. It's a handy way to also sign up for services that send confirmation emails.<br />
<div>
<br /></div>
<div>
I have been playing with MailHog, which is a mail server written in Golang for exactly the same purpose as mailinator. In fact, MailHog can happily intercept ANY recipient at ANY mail domain, provided it is set up properly. In my case, I didn't want to expose MailHog on port 25 externally, because that is a recipe for spam. Instead, I wanted to set up a regular postfix server for mydomain.com, then set up a catch-all user which will receive mail destined for anyuser@maildomain.com, and finally send all that mail to MailHog via procmail. Quite a few moving parts, but I got it to work and I am hastening to jot down my notes before I forget how I did it.</div>
<div>
<br /></div>
<div>
The nice thing about MailHog is that it provides a Web UI where you can eyeball the email messages you sent, including in raw format, and it also provides a JSON API which allows you to list messages and search for specific terms within messages. This last feature is very useful for end-to-end testing of your application's email sending capabilities.<br />
<br />
I set up everything on a Google Cloud Engine instance running Ubuntu 14.04.</div>
<div>
<ul>
<li>instance name: mailhog-mydomain-com</li>
<li>DNS/IP: mailhog.mydomain.com / 1.2.3.4</li>
</ul>
<b><span style="font-size: large;">Install go 1.5.3 from source</span></b><br />
<br />
First install the binaries for go 1.4, then compile go 1.5.</div>
<div>
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># apt-get update<br /># apt-get install build-essential git mercurial bazaar unzip<br /># cd /root<br /># wget https://storage.googleapis.com/golang/go1.4.3.linux-amd64.tar.gz<br /># tar xvfz go1.4.3.linux-amd64.tar.gz<br /># mv go go1.4<br /># git clone https://go.googlesource.com/go<br /># cd go<br /># git checkout go1.5.3<br /># cd src<br /># ./all.bash<br /><br /># mkdir /opt/gocode </span></div>
<div>
Edit <span style="font-family: "courier new" , "courier" , monospace;">/root/.bashrc</span> and add:</div>
<div>
<br />
<span style="font-family: "courier new" , "courier" , monospace;">export GOPATH=/opt/gocode<br />export PATH=$PATH:/root/go/bin:$GOPATH/bin</span><br />
<br />
then source <span style="font-family: "courier new" , "courier" , monospace;">/root/.bashrc.<br /><br /># go version<br />go version go1.5.3 linux/amd64</span><br />
<br />
<br />
<b><span style="font-size: large;">Set up postfix</span></b><br />
<br />
<b>Install postfix and mailutils</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># apt-get install postfix mailutils</span><br />
<br />
- specified System mail name as <span style="font-family: "courier new" , "courier" , monospace;">mydomain.com</span><br />
<br />
<b>Set up catch-all mail user (local Unix user)</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># adduser catchall</span></div>
<div>
<br /></div>
<div>
Edit <span style="font-family: "courier new" , "courier" , monospace;">/etc/aliases</span> and replace content with the lines below.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># cat /etc/aliases<br /><br /># See man 5 aliases for format<br />mailer-daemon: postmaster<br />postmaster: root<br />nobody: root<br />hostmaster: root<br />usenet: root<br />news: root<br />webmaster: root<br />www: root<br />ftp: root<br />abuse: root<br />root: catchall</span><br />
<br />
Run:</div>
<div>
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># newaliases</span></div>
<div>
Edit <span style="font-family: "courier new" , "courier" , monospace;">/etc/postfix/main.cf</span> and add lines:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">luser_relay = catchall<br />local_recipient_maps =</span><br />
<br />
Restart postfix:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># service postfix restart</span><br />
<br />
Use Google Cloud Platform Web UI to add firewall rule called <span style="font-family: "courier new" , "courier" , monospace;">allow-smtp</span> for the “default” network associated with the <span style="font-family: "courier new" , "courier" , monospace;">mailhog-mydomain-com</span> instance. The rule allows incoming traffic from everywhere to port tcp:25.</div>
<div>
<br />
<b>Set up DNS</b><br />
<br />
Add A record for mailhog.mydomain.com pointing to 1.2.3.4.<br />
<br />
Add MX record for catchallpayments.com pointing to mailhog.mydomain.com.<br />
<br />
<b>Test the incoming mail setup</b><br />
<br />
Send mail to <a href="mailto:boom@boompayments.org">catchall@mydomain.com</a> from gmail. <br />
<br />
Run mail utility on GCE instance as user catchall:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">catchall@mailhog-mydomain-com:~$ mail<br /><br />"/var/mail/catchall": 1 message 1 new<br /><br />>N 1 Some Body Tue Feb 9 00:23 52/2595 test from gmail<br /><br />? 1<br /><br />Return-Path: <some.body@gmail.com><br /><b>X-Original-To: catchall@mydomain.com<br />Delivered-To: catchall@mydomain.com</b></span><br />
<br />
<br />
Send mail to random user which doesn’t exist locally <a href="mailto:boom333@boompayments.org">catchall333@mydomain.com</a> and verify that user catchall receives it:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">catchall@mailhog-mydomain-com:~$ mail<br /><br />"/var/mail/catchall": 1 message 1 new<br /><br />>N 1 Some Body Tue Feb 9 18:32 52/2702 test 3<br /><br />? 1<br /><br />Return-Path: <some.body@gmail.com><br /><b>X-Original-To: catchall333@mydomain.com<br />Delivered-To: catchall333@mydomain.com</b></span><br />
<br />
<b><span style="font-size: large;">Install and configure MailHog</span></b><br />
<br />
<b>Get MailHog </b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># go get github.com/mailhog/MailHog </span></div>
<div>
- this will drop several binaries in /opt/gocode/bin, including <span style="font-family: "courier new" , "courier" , monospace;">mhsendmail</span> and <span style="font-family: "courier new" , "courier" , monospace;">MailHog</span> (for reference, the code for mhsendmail is <a href="https://github.com/mailhog/mhsendmail">here</a>)<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># which MailHog<br />/opt/gocode/bin/MailHog<br /><br /># which mhsendmail<br />/opt/gocode/bin/mhsendmail</span><br />
<b>Configure HTTP basic authentication</b><br />
<br />
MailHog supports HTTP basic authentication via a file similar to .htpasswd. It uses bcrypt for password (see more details <a href="https://github.com/mailhog/MailHog/blob/master/docs/Auth.md">here</a>). The MailHog binary can also generate passwords with bcrypt.<br />
<br />
I created a password with MailHog:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># MailHog bcrypt somepassword<br />somebcryptoutput</span><br />
<br />
Then I created a file called .mailhogrc in /root and specified a user called <span style="font-family: "courier new" , "courier" , monospace;">mailhogapi</span> with the password generated above:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># cat /root/.mailhogrc<br />mailhogapi:somebcryptoutput</span><br />
<br />
<b>Create upstart init file for MailHog</b><br />
<br />
I specified the port MailHog listens on (I chose the same port as its default which is 1025) and the filed used for HTTP basic auth.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># cat /etc/init/mailhog.conf<br /># MailHog Test SMTP Server (Upstart unit)<br />description "MailHog Test SMTP Server"<br />start on (local-filesystems and net-device-up IFACE!=lo)<br />stop on runlevel [06]<br /><br />exec /opt/gocode/bin/MailHog -smtp-bind-addr 0.0.0.0:1025 -auth-file /root/.mailhogrc<br />respawn<br />respawn limit 10 10<br />kill timeout 10</span><br />
See more command line options for MailHog in <a href="https://github.com/mailhog/MailHog/blob/master/docs/CONFIG.md">this doc</a>.<br />
<br />
<b>Start mailhog service</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># start mailhog<br />mailhog start/running, process 25458<br /><br /># ps -efd|grep Mail<br />root 7782 1 0 22:04 ? 00:00:00 /opt/gocode/bin/MailHog -smtp-bind-addr 0.0.0.0:1025 -auth-file /root/.mailhogrc</span><br />
<br />
At this point MailHog is listening for SMTP messages on port 1025. It also provides a Web UI on default UI port 8025 and a JSON API also on port 8025.<br />
<br />
<b><span style="font-size: large;">Install procmail and configure it for user catchall</span></b><br />
<br />
This is so messages addressed to user catchall (which again is our catch-all user) can get processed by a script via procmail.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"># apt-get install procmail</span><br />
<br />
Add this line to <span style="font-family: "courier new" , "courier" , monospace;">/etc/postfix/main.cf</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">mailbox_command = /usr/bin/procmail -a "$EXTENSION" DEFAULT=$HOME/Maildir/ MAILDIR=$HOME/Maildir</span><br />
<br />
(this will send all messages to procmail instead of individual user mailboxes)<br />
<br />
Then su as user catchall and create .procmailrc file in its home directory:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">catchall@mailhog-mydomain-com:~$ cat .procmailrc<br />:0<br />| /opt/gocode/bin/mhsendmail --smtp-addr="localhost:1025"</span><br />
<br />
This tells procmail to pipe the incoming mail message to mhsendmail, which will format it properly and pass it to port 1025, where MailHog is listening.<br />
<br />
<span style="font-size: large;"><b>Test end-to-end</b></span><br />
<br />
Use Google Cloud Platform Web UI to add firewall rule called <span style="font-family: "courier new" , "courier" , monospace;">allow-mailhog-ui</span> for the “default” network associated with the <span style="font-family: "courier new" , "courier" , monospace;">mailhog-mydomain-com</span> instance. The rule allows incoming traffic from everywhere to tcp:8025 (where the MailHog UI server listens). It’s OK to allow traffic to port 8025 from everywhere because it is protected via HTTP basic auth.<br />
<br />
The MailHog UI is at <a href="http://mx.boompayments.org:8025/">http://mailhog.mydomain.com:8025</a><br />
<br />
Any email sent to <a href="mailto:xyz@boompayments.org">xyz@mydomain.com</a> should appear in the MailHog Inbox.<br />
<br />
By default, MailHog stores incoming messages in memory. Restarting MailHog (via ‘restart mailhog’ at the cmdline) will remove all messages.<br />
<br />
MailHog also supports MongoDB as a persistent storage backend for incoming messages (exercise left to the reader.)<br />
<br />
<b>Use the MailHog JSON API to verify messages</b><br />
<br />
List all messages:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ curl --user 'mailhogapi:somepassword' -H "Content-Type: application/json" "<a href="http://mx.boompayments.org:8025/api/v2/messages">http://mailhog.mydomain.com:8025/api/v2/messages</a>"</span><br />
<br />
Search messages for specific terms (for example for the recipient’s email):<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ curl -i --user 'mailhogapi:somepassword' -H "Content-Type: application/json" "<a href="http://mx.boompayments.org:8025/api/v2/search?kind=containing&query=test1%40boompayments.org">http://mailhog.mydomain.com:8025/api/v2/search?kind=containing&query=test1%40mydomain.com</a>"</span><br />
<br />
See the MailHog API v2 docs <a href="https://github.com/mailhog/MailHog/tree/master/docs/APIv2">here</a>. <br />
<br />
That's it, hope it makes your email sending testing more fun!<br />
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com1tag:blogger.com,1999:blog-9238405.post-63742637509352222162016-02-05T16:40:00.001-08:002016-02-05T16:40:15.302-08:00Setting up Jenkins to run headless Selenium tests in Docker containersThis is the third post in a series on running headless Selenium WebDriver tests. Here are the first two posts:<br />
<ol>
<li><a href="http://agiletesting.blogspot.com/2016/01/running-selenium-webdriver-tests-using.html">Running Selenium WebDriver tests using Firefox headless mode on Ubuntu</a></li>
<li><a href="http://agiletesting.blogspot.com/2016/01/running-headless-selenium-webdriver.html">Running headless Selenium WebDriver tests in Docker containers</a></li>
</ol>
<div>
In this post I will show how to add the final piece to this workflow, namely how to fully automate the execution of Selenium-based WebDriver tests running Firefox in headless mode in Docker containers. I will use Jenkins for this example, but the same applies to other continuous integration systems.</div>
<div>
<br /></div>
<div>
1) Install docker-engine on the server running Jenkins (I covered this in my post #2 above)</div>
<div>
<br /></div>
<div>
2) Add the jenkins user to the docker group so that Jenkins can run the docker command-line tool in order to communicate with the docker daemon. Remember to restart Jenkins after doing this.</div>
<div>
<br /></div>
<div>
3) Go through the rest of the workflow in my post above ("<a href="http://agiletesting.blogspot.com/2016/01/running-headless-selenium-webdriver.html">Running headless Selenium WebDriver tests in Docker containers</a>") and make sure you can run all the commands in that post from the command line of the server running Jenkins.</div>
<div>
<br /></div>
<div>
4) Create a directory structure for your Selenium WebDriver tests (mine are written in Python). </div>
<div>
<br /></div>
<div>
I have a directory called <span style="font-family: Courier New, Courier, monospace;">selenium-docker</span> which contains a directory called <span style="font-family: Courier New, Courier, monospace;">tests</span>, under which I put all my Python WebDriver tests named <span style="font-family: Courier New, Courier, monospace;">sel_wd_*.py</span>. I also have a simple shell script I named <span style="font-family: Courier New, Courier, monospace;">run_selenium_tests.sh</span> which does the following:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#!/bin/bash</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">TARGET=$1 # e.g. someotherdomain.example.com (if not specified, the default is somedomain.example.com)</span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">for f in `ls tests/sel_wd_*.py`; do</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> echo Running $f against $TARGET</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> python $f $TARGET</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">done</span></div>
</div>
<div>
<br /></div>
<div>
My <span style="font-family: Courier New, Courier, monospace;">selenium-docker</span> directory also contains the <span style="font-family: Courier New, Courier, monospace;">xvfb.init</span> file I need for starting up Xvfb in the container, and finally it contains this <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span>:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">FROM ubuntu:trusty</span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN echo "deb http://ppa.launchpad.net/mozillateam/firefox-next/ubuntu trusty main" > /etc/apt/sources.list.d//mozillateam-firefox-next-trusty.list</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE49EC21</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN apt-get update</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN apt-get install -y firefox xvfb python-pip</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN pip install selenium</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN mkdir -p /root/selenium/tests</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ADD tests /root/selenium/tests</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ADD run_all_selenium_tests.sh /root/selenium</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ADD xvfb.init /etc/init.d/xvfb</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN chmod +x /etc/init.d/xvfb</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RUN update-rc.d xvfb defaults</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">ENV TARGET=somedomain.example.com</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CMD (service xvfb start; export DISPLAY=:10; cd /root/selenium; ./run_all_selenium_tests.sh $TARGET)</span></div>
</div>
<div>
<br /></div>
<div>
I explained what this <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span> achieves in the 2nd post referenced above. The <span style="font-family: Courier New, Courier, monospace;">ADD</span> instructions will copy all the files in the <span style="font-family: Courier New, Courier, monospace;">tests</span> directory to the directory called <span style="font-family: Courier New, Courier, monospace;">/root/selenium/tests</span>, and will copy <span style="font-family: Courier New, Courier, monospace;">run_all_selenium_tests.sh</span> to <span style="font-family: Courier New, Courier, monospace;">/root/selenium</span>. The <span style="font-family: Courier New, Courier, monospace;">ENV</span> variable <span style="font-family: Courier New, Courier, monospace;">TARGET</span> represents the URL against which we want to run our Selenium tests. It is set by default to <span style="font-family: Courier New, Courier, monospace;">somedomain.example.com</span>, and is used as the first argument when running <span style="font-family: Courier New, Courier, monospace;">run_all_selenium_tests.sh</span> in the <span style="font-family: Courier New, Courier, monospace;">CMD</span> instruction.</div>
<div>
<br /></div>
<div>
At this point, I checked in the <span style="font-family: Courier New, Courier, monospace;">selenium-docker</span> directory and all files and directories under it into a Github repository I will call '<span style="font-family: Courier New, Courier, monospace;">devops</span>'.</div>
<div>
<br /></div>
<div>
5) Create a new Jenkins project (I usually create a New Item and copy it from an existing project).</div>
<div>
<br /></div>
<div>
I specified that the build is parameterized and I indicated a choice parameter called <span style="font-family: Courier New, Courier, monospace;">TARGET_HOST</span> with a few host/domain names that I want to test. I also specified Git as the Source Code Management type, and I indicated the URL of the <span style="font-family: Courier New, Courier, monospace;">devops</span> repository on Github. Most of the action of course happens in the Jenkins build step, which in my case is of type "Execute shell". Here it is:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">#!/bin/bash</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">set +e</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">IMAGE_NAME=selenium-wd:v1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">cd $WORKSPACE/selenium-docker</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># build the image out of the Dockerfile in the current directory</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/usr/bin/docker build -t $IMAGE_NAME .</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># run a container based on the image</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CONTAINER_ID=`/usr/bin/docker run -d -e "TARGET=$TARGET_HOST" $IMAGE_NAME`</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">echo CONTAINER_ID=$CONTAINER_ID</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># while the container is still running, sleep and check logs; repeat every 40 sec</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">while [ $? -eq 0 ];</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">do</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> sleep 40</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> /usr/bin/docker logs $CONTAINER_ID</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> /usr/bin/docker ps | grep $IMAGE_NAME</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">done</span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"># docker logs sends errors to stderr so we need to save its output to a file first</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/usr/bin/docker logs $CONTAINER_ID > d.out 2>&1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"># remove the container so they don't keep accumulating</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">docker rm $CONTAINER_ID</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"># mark jenkins build as failed if log output contains FAILED</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">grep "FAILED" d.out</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">if [[ $? -eq 0 ]]; then </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> rm d.out </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> exit 1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">else</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> rm d.out</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> exit 0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">fi</span></div>
</div>
<div>
<br /></div>
<div>
Some notes:</div>
<div>
<ul>
<li>it is recommended that you specify <span style="font-family: Courier New, Courier, monospace;"><b>#!/bin/bash</b></span> as the 1st line of your script, to make sure that bash is the shell that is being used</li>
<li>use <span style="font-family: Courier New, Courier, monospace;"><b>set +e</b></span> if you want the Jenkins shell script to continue after hitting a non-zero return code (the default behavior is for the script to stop on the first line it encounters an error and for the build to be marked as failed; subsequent lines won't get executed, resulting in much pulling of hair)</li>
<li>the Jenkins script will build a new image every time it runs, so that we make sure we have updated Selenium scripts in place</li>
<li>when running the container via <span style="font-family: Courier New, Courier, monospace;"><b>docker run</b></span>, we specify <span style="font-family: 'Courier New', Courier, monospace;"><b>-e "TARGET=$TARGET_HOST"</b></span> as an extra command line argument. This will override the <span style="font-family: Courier New, Courier, monospace;">ENV</span> variable named <span style="font-family: Courier New, Courier, monospace;">TARGET</span> in the <span style="font-family: Courier New, Courier, monospace;">Dockerfile</span> with the value received from the Jenkins multiple choice dropdown for <span style="font-family: Courier New, Courier, monospace;">TARGET_HOST</span></li>
<li>the main part of the shell script stays in a while loop that checks for the return code of "<span style="font-family: 'Courier New', Courier, monospace;"><b>/usr/bin/docker ps | grep $IMAGE_NAME</b></span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">". </span>This is so we wait for all the Selenium tests to finish, at which point <span style="font-family: Courier New, Courier, monospace;">docker ps</span> will not show the container running anymore (you can still see the container by running <span style="font-family: Courier New, Courier, monospace;">docker ps -a</span>)</li>
<li>once the tests finish, we save the stdout and stderr of the docker logs command for our container to a file (this is so we capture both stdout and stderr; at first I tried something like <span style="font-family: Courier New, Courier, monospace;">docker logs $CONTAINER_ID | grep FAILED</span> but this was never successful, because it was grep-ing against stdout, and errors are sent to stderr)</li>
<li>we grep the file (<span style="font-family: Courier New, Courier, monospace;">d.out</span>) for the string <span style="font-family: Courier New, Courier, monospace;">FAILED</span> and if we find it, we exit with code 1, i.e. unsuccessful as far as Jenkins is concerned. If we don't find it, we exit successfully with code 0.</li>
</ul>
</div>
<div>
<br /></div>
<div>
<br /></div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com0tag:blogger.com,1999:blog-9238405.post-85516325033950752872016-01-08T15:24:00.002-08:002016-01-10T15:56:56.020-08:00Running headless Selenium WebDriver tests in Docker containersIn my <a href="http://agiletesting.blogspot.com/2016/01/running-selenium-webdriver-tests-using.html">previous post</a>, I showed how to install firefox in headless mode on an Ubuntu box and how to use Xvfb to allow Selenium WebDriver scripts to run against firefox in headless mode.<br />
<br />
Here I want to show how run each Selenium test suite in a Docker container, so that the suite gets access to its own firefox browser. This makes it easy to parallelize the test runs, and thus allows you to load test your Web infrastructure with real-life test cases.<br />
<br />
<b>Install docker-engine on Ubuntu 14.04</b><br />
<br />
<div>
We import the dockerproject.org signing key and apt repo into our apt repositories, then we install the linux-image-extra and docker-engine packages.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"># apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D<br /># echo “deb <a href="https://apt.dockerproject.org/repo">https://apt.dockerproject.org/repo</a> ubuntu-trusty main” > /etc/apt/sources.list.d/docker.list<br /># apt-get update<br /># apt-get install linux-image-extra-$(uname -r)<br /># apt-get install docker-engine</span><br />
<br />
<b>Start the docker service and verify that it is operational</b><br />
<br />
<div>
Installing docker-engine actually starts up docker as well, but to start the service you do:<br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;"># service docker start</span></b><br />
<br />
To verify that the docker service is operational, run a container based on the public “hello-world” Docker image:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><b># docker run hello-world</b><br />Unable to find image ‘hello-world:latest’ locally<br />latest: Pulling from library/hello-world<br />b901d36b6f2f: Pull complete<br />0a6ba66e537a: Pull complete<br />Digest: sha256:8be990ef2aeb16dbcb9271ddfe2610fa6658d13f6dfb8bc72074cc1ca36966a7<br />Status: Downloaded newer image for hello-world:latest<br />Hello from Docker.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">This message shows that your installation appears to be working correctly.<br />To generate this message, Docker took the following steps:<br />1. The Docker client contacted the Docker daemon.<br />2. The Docker daemon pulled the “hello-world” image from the Docker Hub.<br />3. The Docker daemon created a new container from that image which runs the<br />executable that produces the output you are currently reading.<br />4. The Docker daemon streamed that output to the Docker client, which sent it<br />to your terminal.<br />To try something more ambitious, you can run an Ubuntu container with:<br />$ docker run -it ubuntu bash<br />Share images, automate workflows, and more with a free Docker Hub account:<br /><a href="https://hub.docker.com/">https://hub.docker.com</a><br />For more examples and ideas, visit:<br /><a href="https://docs.docker.com/userguide/">https://docs.docker.com/userguide/</a></span><br />
<br />
<b> Pull the ubuntu:trusty public Docker image
</b></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span>
<span style="font-family: Courier New, Courier, monospace;"><b># docker pull ubuntu:trusty</b><br />trusty: Pulling from library/ubuntu<br />fcee8bcfe180: Pull complete<br />4cdc0cbc1936: Pull complete<br />d9e545b90db8: Pull complete<br />c4bea91afef3: Pull complete<br />Digest: sha256:3a7f4c0573b303f7a36b20ead6190bd81cabf323fc62c77d52fb8fa3e9f7edfe<br />Status: Downloaded newer image for ubuntu:trusty</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><b># docker images</b><br />REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE<br />ubuntu trusty c4bea91afef3 3 days ago 187.9 MB<br />hello-world latest 0a6ba66e537a 12 weeks ago 960 B</span><br />
<br />
<b>Build custom Docker image for headless Selenium WebDriver testing</b></div>
<div>
<br />
I created a directory called selwd on my host Ubuntu 14.04 box, and in that directory I created this Dockerfile:<br />
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>FROM</b> ubuntu:trusty<br /><b>RUN</b> echo “deb <a href="http://ppa.launchpad.net/mozillateam/firefox-next/ubuntu">http://ppa.launchpad.net/mozillateam/firefox-next/ubuntu</a> trusty main” > /etc/apt/sources.list.d//mozillateam-firefox-next-trusty.list </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE49EC21 </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> apt-get update </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> apt-get install -y firefox xvfb python-pip </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> pip install selenium </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> mkdir -p /root/selenium_wd_tests </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>ADD</b> sel_wd_new_user.py /root/selenium_wd_tests </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>ADD</b> xvfb.init /etc/init.d/xvfb </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> chmod +x /etc/init.d/xvfb </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>RUN</b> update-rc.d xvfb defaults<br /><b>CMD</b> (service xvfb start; export DISPLAY=:10; python /root/selenium_wd_tests/sel_wd_new_user.py)</span><br />
<br />
This Dockerfile tells docker, via the FROM instruction, to create an image based on the ubuntu:trusty image that we pulled before (if we hadn’t pulled it, it would be pulled the first time our image was built).</div>
<div>
The various RUN instructions specify commands to be run at build time. The above instructions add the Firefox Beta repository and key to the apt repositories inside the image, then install firefox, xvfb and python-pip. Then they install the selenium Python package via pip and create a directory structure for the Selenium tests.<br />
<br />
The ADD instructions copy local files to the image. In my case, I copy one Selenium WebDriver Python script, and an init.d-type file for starting Xvfb as a service (by default it starts in the foreground, which is not something I want inside a Docker container).<br />
<br />
The last two RUN instructions make the /etc/init.d/xvfb script executable and run update-rc.d to install it as a service. The xvfb script is the usual init.d wrapper around a command, in my case this command:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">PROG=”/usr/bin/Xvfb” </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">PROG_OPTIONS=”:10 -ac”</span><br />
<br />
Here is a <a href="https://gist.github.com/griggheo/d943e33ea4d419e343aa">gist for the xvfb.init script</a> for reference.<br />
<br />
Finally, the CMD instruction specifies what gets executed when a container based on this image starts up (assuming no other commands are given in the ‘docker run’ command-line for this container). The CMD instruction in the Dockerfile above starts up the xvfb service (which connects to DISPLAY 10 as specified in the xvfb init script), sets the DISPLAY environment variable to 10, then runs the Selenium WebDriver script sel_wd_new_user.py, which will launch firefox in headless mode and execute its commands against it.<br />
<br />
Here’s the official documentation for <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile</a> instructions.</div>
<div>
To build a Docker image based on this Dockerfile, run:<br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;"># docker build -t selwd:v1 .</span></b><br />
<br />
selwd is the name of the image and v1 is a tag associated with this name. The dot . tells docker to look for a Dockerfile in the current directory.<br />
<br />
The build process will take a while intially because it will install all the dependencies necessary for the packages we are installing with apt. Every time you make a modification to the Dockerfile, you need to run ‘docker build’ again, but subsequent runs will be much faster.<br />
<br />
<b>Run Docker containers based on the custom image</b></div>
<div>
<br />
At this point, we are ready to run Docker containers based on the selwd image we created above.<br />
<br />
Here’s how to run a single container:<br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;"># docker run --rm selwd:v1</span></b></div>
<div>
<br />
In this format, the command specified in the CMD instruction inside the Dockerfile will get executed, then the container will stop. This is exactly what we need: we run our Selenium WebDriver tests against headless firefox, inside their own container isolated from any other container.<br />
<br />
The output of the ‘docker run’ command above is:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">Starting : X Virtual Frame Buffer </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Ran 1 test in 40.441s</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">OK</span><br />
(or a traceback if the Selenium test encountered an error)<br />
<br />
Note that we also specified the rm flag to ‘docker run’ so that the container gets removed once it stops — otherwise these short-lived containers will be kept around and will pile up, as you can see for yourself if you run:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><b># docker ps -a</b><br />CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">6c9673e59585 selwd:v1 “/bin/bash” 5 minutes ago Exited (130) 5 seconds ago modest_mccarthy
980651e1b167 selwd:v1 “/bin/sh -c ‘(service” 9 minutes ago Exited (0) 8 minutes ago stupefied_turing </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">4a9b2f4c8c28 selwd:v1 “/bin/sh -c ‘(service” 13 minutes ago Exited (0) 12 minutes ago nostalgic_ride </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">9f1fa953c83b selwd:v1 “/bin/sh -c ‘(service” 13 minutes ago Exited (0) 12 minutes ago admiring_ride </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">c15b180832f6 selwd:v1 “/bin/sh -c ‘(service” 13 minutes ago Exited (0) 12 minutes ago jovial_booth
.....etc</span><br />
<br />
If you do have large numbers of containers that you want to remove in one go, use this command:<br />
<br />
<b><span style="font-family: Courier New, Courier, monospace;"># docker rm `docker ps -aq`</span></b></div>
<div>
For troubleshooting purposes, we can run a container in interactive mode (with the -i and -t flags) and specify a shell command to be executed on startup, which will override the CMD instruction in the Dockerfile:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><b># docker run -it selwd:v1 /bin/bash </b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">root@6c9673e59585:/#</span>
<br />
At the bash prompt, you can run the shell commands specified by the Dockerfile CMD instruction in order to see interactively what is going on. The <a href="https://docs.docker.com/engine/reference/run/">official ‘docker run’ documentation</a> has lots of details.<br />
<br />
One other thing I found useful for troubleshooting Selenium WebDriver scripts running against headless firefox was to have the scripts take screenshots during their execution with the save_screenshot command:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">driver.save_screenshot(“before_place_order.png”)<br /># Click Place Order
driver.find_element_by_xpath("//*[<a href="http://twitter.com/id">@id</a>='order_submit_button']").click()<br />driver.save_screenshot(“after_place_order.png”)</span><br />
<br />
I then inspected the PNG files to see what was going on.<br />
<br />
<b>Running multiple Docker containers for load testing</b><br />
<br />
Because our Selenium WebDriver tests run isolated in their own Docker container, it enables us to run N containers in parallel to do a poor man’s load testing of our site.</div>
<div>
We’ll use the -d option to ‘docker run’ to run each container in ‘detached’ mode. Here is a bash script that launches COUNT Docker containers, where COUNT is the 1st command line argument, or 2 by default:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">#!/bin/bash<br />COUNT=$1 </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">if [ -z “$COUNT” ]; then </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> COUNT=2 </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">fi<br />for i in `seq 1 $COUNT`; do </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> docker run -d selwd:v1 </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">done</span><br />
<br />
The output of the script consists in a list of container IDs, one for each container that was launched.<br />
<br />
Note that if you launch a container in detached mode with -d, you can’t specify the rm flag to have the container removed automatically when it stops. You will need to periodically clean up your containers with the command I referenced above (docker rm `docker ps -aq`).<br />
<br />
To inspect the output of the Selenium scripts in the containers that were launched, first get the container IDs:
</div>
<div>
<b><span style="font-family: Courier New, Courier, monospace;"><br /></span></b>
<b><span style="font-family: Courier New, Courier, monospace;"># docker ps -a </span></b></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>6fb931689c03</b> selwd:v1 “/bin/sh -c ‘(service” About an hour ago Exited (0) About an hour ago grave_northcutt </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">1b82ef59ad46 selwd:v1 “/bin/sh -c ‘(service” About an hour ago Exited (0) About an hour ago admiring_fermat</span><br />
<br />
Then run ‘docker logs <container_id>’ to see the output for a specific container:</div>
<div>
<b><span style="font-family: Courier New, Courier, monospace;"><br /></span></b>
<b><span style="font-family: Courier New, Courier, monospace;"># docker logs 6fb931689c03 </span></b></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Ran 1 test in 68.436s</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br />OK </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Starting : X Virtual Frame Buffer</span><br />
Have fun load testing your site!</div>
</div>
Grig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.com2