CoreOS Demo

Kubernetes Step-by-step

Preliminary work

1. Create KMS CMK
2. Using ARN: arn:aws:kms:us-east-1:437443400885:key/819a0470-5371-4217-942e-86abd5e3c979
3. Create IAM user in with admin access

My Kubernetes start instructions

1. Install kube-aws
2. Clone kubedemo and cd into it
3. git clone https://github.com/klsetzer/kubedemo.git
4. cd kubedemo
5. Configure environment and create helpers
6. source env_kube.sh
7. cluster_name frodo
8. Create Kubernetes cluster
9. ./kube-creator
10. Check cluster status (How long does this take to stabilize?)
11. kubectl cluster-info
12. May take a few minutes for cluster to finish configuring
13. Start kube proxy
14. kubectl start proxy &
15. Launch dashboard
16. kubectl create -f kubernetes/cluster/addons/dashboard
17. Connect to proxy URL
18. URL: http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
19. Start guestbook/all-in-one
20. kubectl create -f kubernetes/examples/guestbook/all-in-one/guestbook-all-in-one.yaml
21. kubectl describe service frontend
22. How long does it take this to become available? (~5m?)

Talk about navigation/status commands

Demo cleanup

1. cd $KUBEDEMO_HOME/<cluster_name>
2. kube-aws destroy
3. kill kubectl proxy process

TODO

1. Create in multiple AZs
2. Integrate with VPC
3. Integrate with datadog
4. Create CFN template for VPC, subnets, and route tables
5. Need jenkins for this
6. Get web based services
7. HA
8. Loadbalacner
9. Autoscaling
10. Understand Cloud-Config
11. Understand discovery service
12. For new clusters
13. Spot pricing
14. https://gist.github.com/danieldreier/e5685e77f9727bf93b18
15. http://jake.ai/coreos-and-spot-instances-just-for-funzies/
16. Cheaper demo

CoreOS + Fleet Industrialization

1. Launching without using the bootstrap discover service
2. Running fleetctl from dev workstation instead of logging into cluster https://coreos.com/fleet/docs/latest/using-the-client.html#remote-fleet-access

I Did

1. Installed CoreOS cluster from CFN launch button on coreos website

• https://coreos.com/os/docs/latest/booting-on-ec2.html

1. uninstalled boot2docker
2. installed OS X Docker Toolbox: https://docs.docker.com/engine/installation/mac/
3. Ran docker "hello world" test: $ docker run hello-world
4. Installed fleetctl: $ brew install fleetctl
5. Experimented with fleet: https://coreos.com/fleet/docs/latest/launching-containers-fleet.html
6. Found this error: ip-10-16-183-155 bin # fleetctl list-machines Error retrieving list of active machines: googleapi: Error 503: fleet server unable to communicate with etcd
7. Tried starting etcd: ip-10-16-183-155 bin # /bin/etcd [etcd] May 30 17:09:08.487 WARNING | Using the directory ip-10-16-183-155.ec2.internal.etcd as the etcd curation directory because a directory was not specified. [etcd] May 30 17:09:08.487 CRITICAL | Unable to create path: mkdir ip-10-16-183-155.ec2.internal.etcd: read-only file system
8. Tried restarting with customized CFN template. Got new error: The specified instance type can only be used in a VPC. A subnet ID or network interface ID is required to carry out the request. Launching EC2 instance failed. This was because t2.* instance types can only be used in a VPC. Modified CFN template to include SubnetIds. Create VPC and subnets through web console. Got error: Launching a new EC2 instance. Status Reason: The parameter groupName cannot be used with the parameter subnet. Launching EC2 instance failed.
9. Fixed CFN networking issues by changing CFN template to GroupIds instead of GroupNames.
10. Now 'etcdctl cluster-health' and 'fleetctl list-machines' show a healthy cluster'
11. Specifically had to add a default (Internet) route to the subnets in the coreos-demo-vpc
12. Had to add a call to get a new discovery service endpoint for each new stack
13. Figured out how to run fleetctl from local workstation
14. Submitted unit hello.service with 'fleetctl submit units/hello.service'
15. Checked with fleetctl list-unit-files
16. Started hello.service: fleetctl start hello.service
17. Experimenting with services: https://coreos.com/fleet/docs/latest/using-the-client.html

Kubernetes

1. Worked through https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html
2. got gpg keys
3. downloaded kube-aws and installed in /usr/local
4. Created an KMS key 1. aws kms --region= create-key --description="kube-aws assets"
5. Download kubectl from curl -O https://storage.googleapis.com/kubernetes-release/release/v1.2.4/bin/darwin/amd64/kubectl
6. kubectl --kubeconfig=kubeconfig get nodes
7. The certificate management is complex. PKI integration will be interesting.
8. Service load balancing seems too simple: "Services are automatically configured to load balance traffic to pods matching the label query. A random algorithm is used and is currently the only option. Session affinity can be configured to send traffic to pods by client IP." (https://coreos.com/kubernetes/docs/latest/services.html)
9. Start dashboard: kubectl create -f cluster/addons/dashboard 1. run 'kubectl proxy' in a term window 1. Go to 'http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard' in a browser
10. Fire up guestbook demo 1. Works with kubectl: kubectl create -f examples/guestbook/all-in-one/guestbook-all-in-one.yaml 1. Delete with kubectl delete -f examples/guestbook/all-in-one/guestbook-all-in-one.yaml 1. kubectl get svc
11. Tried deploying through the dashboard 1. Failed with Node didn't have enough resource: CPU, requested: 100, used: 960

Getting into docker

• Run 'Docker Quickstart Terminal' after installing Docker Toolbox

Logging into CoreOS instances

1. eval $(ssh-agent)
2. ssh-add ~/.ssh/lc-us-east-1.pem
3. ssh -A core@ec2-dns-name

Running fleetctl commands from the local workstation

FLEETCTL_TUNNEL=54.175.226.58:22 fleetctl list-units

Good docs:

Kubernetes User Guide: http://kubernetes.io/docs/user-guide/ Calico: http://docs.projectcalico.org/en/latest/index.html#