Detailed Deployment Instructions

Once you have a dev machine created and accessible, you have two options:

• Deployment option #1:  Deploy stack from DMC repos.
• Deployment option #2:  Deploy a stack from your own code base.

Creation of a dev machine from your local machine

1. clone dmc dev repo

2. go to sub folder

$ cd dmcdeploy_dev

3. switch branch to devmakina

$ git checkout devmakina 

3a. (Optional only If coming here from Seed Machine Instructions)
Make keys directory if not there

$ mkdir ~/keys

4. Create and download the AWS .pem key files you wish to use for your deployments.

--Key for the seed machine

--Key for dev machine

--Key for any other machines dev may launch

In EC2 console of AWS under keypairs create as many key pairs as needed and download them locally. By convention they are stored in ~/keys

(Optional only If coming here from Seed Machine Instructions) 

If you are deploying from an aws seed machine then you must upload to this seed machine the keys that you would like it to use when deploying subsequent machines

--Upload a .pem file  to seed machine. From your local machine (the machine you use to connect to aws, not the aws seed or dev machines).

$ scp -i  seedmachinekey.pem remotekey.pem ec2-user@ip.of.seed.machine:/home/ec2-user/keys

(Note seedmachinekey.pem is the name of the key from creating the instance in aws, remotekey is a name you choose for the file and will use in the terraform.tfvars file below.) Confirm that the remote pem is on your seed machine by finding its name listed with the following command entered in the seed machine.

$ ls ~/keys

4a. edit terraform.tfvars

$ cd dmcdeploy_dev/

$ nano terraform.tfvars

   -- input your aws credentials on the first two lines
   -- edit key_name to the name of the remote key you just uploaded ( no .pem)
   -- edit the key_full_path to " ~/keys/remotekeyname.pem"

NOTE
--- Make sure that the keys you have entered match the region where you wish to deploy to. Keys are only valid in the region they were created.
 
-- edit the stackPrefix  

-- to lauch the alpha release  

    -- change release = "alpha"

-- to launch latest nightly  

    -- change release = "hot"

-- edit the stackPrefix to something meaningful, this will be the prefix label for your new instances in AWS

-- edit the aws_region ('us-east-1' or 'us-west-2') as appropriate

5. launch devmachine

--uncomment the dev.tf file if needed by removing the comment characters on the first and last line

$ nano dev.tf

--comment out any other machines that may be there

6. Install terraform if you have not already done so

Instructions at:

https://digitalmfgcommons.atlassian.net/wiki/display/DMC/DMCDeploy

$ cd /tmp

$ wget https://releases.hashicorp.com/terraform/0.6.8/terraform_0.6.8_linux_amd64.zip

$ mkdir ~/tf

$ unzip terraform_0.6.8_linux_amd64.zip -d ~/tf

Make sure the path variables are properly set

$ echo "export PATH=$PATH:/home/ec2-user/tf" >> ~/.bashrc

Reload your bashrc

$ source ~/.bashrc

Verify install with:

$ terraform -v 

6a. launch dev instance

-- to create the plan in a file called a, and check you are launching the infrastructure you think you are, execute

$ terraform plan -out a

-- to actually launch that infrastructure from the plan created in the previous step, execute

$ terraform apply a 

----> At this point your local machine or the Seed Machine will have deployed a Dev Machine on aws. You will then use the Dev Machine to do further work.

$ terraform destroy

7. ssh into dev instance

-- to find out what the ip of the dev machine use the aws console to check instances and find the one with the correct tag or use

$ terraform show

-- read it from the line

aws_instance.dev:

...

...

public_ip = 52.23.214.212

...

...

-- if amazon complains that key restrictions are too open

$ cd ~/keys

$ chmod 400 <keyname>.pem 

-- now, ssh into dev instance

$ ssh -i "key_used_in_terraformvars.pem" ec2-user@ip.of.dev.machine 

-- accept the key fingerprint

yes

-----> At this point you are logged into the Dev machine you have just created. The seed machine/ local laptop is no longer needed.

To speed up your development you may now want to create a image of the dev machine in whatever region you want and launch that and use it for development thereby not needing to repeat many steps listed above.

In the future a dev machine may be simply running and multiple developers will only need to ssh into it and not recreate it.

As part of the security infrastructure developers interact with the DMC from the Dev Machines.

We want to be transparent with the tools and technology we use which is why we document and expose our processes instead of simply handing off opaque development environments.

The reason for using a seed machine is that this machine controls the setup of the dev machines. As the DMC is a rapidly evolving project the dev machines will need to be updated periodically. The seed machines can be used to give developers a personalized dev machine without those developers caring for/about the dev ops that created their instances.

8. (optional) switch to better shell  

$ zsh 

$ export PATH=$PATH:/home/ec2-user/tf 

$ alias tf='terraform'

Creation of a dev machine from a seed AWS machine

OPTION 1:  Use prebuilt seed image

Go to the EC2 Dashboard and launch a custom AMI image using the steps below. 

For west:

West_Seed_1203      ami-45a6b424  << linux machines

West_Seed_w_1203      coming soon  << windows machines

For east:

East_Seed_1203      ami-34f3be5e  << linux machine

East_Seed_w_1203     coming soon  << windows machines

Steps to accomplish AWS launch from custom image
--Ec2 Console click launch instance.
--My AMI's section search for appropriate machine.
--Resize as appropriate (M4.large)
--Tag your instance with a memorable name.
--Choose the key-pair you wish and launch.

This instance will have the same configuration that you would get in Option 2 below.

Setup your private key on the machine from which you will access the Amazon instance. For PC users, putty is a good tool to access the Amazon instance. If you are on a corporate network behind a firewall, consult with local administrators to determine how to access AWS instances. Configure putty to use your private key by specifying the file in the Putty Configuration under Connection -> SSH -> Auth -> Private key file for authentication. Login to the new instance with ssh (putty). The username is ec2-user.

After logging in you can verify the configuration with: 

$ terraform -v

$ git --version

Always a good idea to update your machine.

$ sudo yum update -y

You can now follow the same instructions described in Deployment from a Local Machine. To create your personal dev Machine that you will use to launch your stack or further develop.



OPTION 2:  Create your own seed machine


1. Create an aws linux machine following the aws instructions. This machine can be in any region you want and of any size you choose.

2. SSH into the created machine.

3. Install the needed tools

-- git

$ sudo yum install -y git

-- terraform

Install Terraform

Instructions at:

https://digitalmfgcommons.atlassian.net/wiki/display/DMC/DMCDeploy

$ cd /tmp

$ wget https://releases.hashicorp.com/terraform/0.6.8/terraform_0.6.8_linux_amd64.zip

$ mkdir ~/tf

$ unzip terraform_0.6.8_linux_amd64.zip -d ~/tf

Make sure the path variables are properly set

$ echo "export PATH=$PATH:/home/ec2-user/tf" >> ~/.bashrc

Reload your bashrc

$ source ~/.bashrc

Verify install with:

$ terraform -v 

You can now follow the same instructions described in Deployment from a Local Machine. To create your personal dev Machine that you will use to launch your stack or further develop.

Deployment Option I:  Deploy stack from DMC repos  

The stack that this process deploys reflects the data found in the Digital Manufacturing repositories found at:
https://bitbucket.org/DigitalMfgCommons/

The purpose of this is to demonstrate the deployment of the stack from our stable repositories. It focuses on only the infrastructure deployment segment of the software cycle. This process is not applicable if you wish to alter the code as it will ALWAYS deploy from the DMC repositories.  If you wish to make code changes and see them reflected in your deployment see Option II of this document  

1. Pick dev or deploy branch will use dev for demo

$ cd dmcdeploy_dev/  

2. Will only deploy front end to prove it works

$ nano front.tf  

--remove the first and last line comments

--remove the dependency line:

depends_on = ["aws_instance.rest"]

--remove the setting of rest environment variable :

echo 'export Restip=${aws_instance.rest.private_ip}' >> ~/.bashrc

3. edit terraform.tfvars  

-- input your aws credentials on the first two lines

-- upload a key you wish to deploy with to this dev machine    

-- command used          

$ scp -i  mykey.pem somefile.txt ec2-user@my.ec2.id.amazonaws.com:/home/ec2-user/keys

-- edit key_name to the name of the remote key you just uploaded ( no .pem)

-- edit the key_full_path to " ~/keys/key_name.pem"

-- edit the stackPrefix  

-- to launch the alpha release    

-- edit release = "alpha"

-- to launch latest nightly    

-- release = "hot"    

4. Create the stack from the official repos of the DMC

$ terraform plan -out a

Confirm that terraform will create the infrastructure that you want, then run:

$ terraform apply a

5. To prove that you successfully launched the front end process, run:

$ terraform show

From the listing, determine the public IP of the aws_instance.front process.

Open that IP in any browser.  You should see the DMC user interface.    

6. To destroy the infrastructure:

$ terraform destroy 

Deployment Option II:  Deploy a stack from your own code base  

Create a Dev Machine that allows you to edit the code and deploy your changes.  This process demonstrates mutable infrastructure. The purpose of this method is to show how one can deploy and develop for the DMC.

The production infrastructure builds are automated using bamboo and integrated into a code delivery pipeline. We will expose more of our build and deployment tools in the near future.

This demonstrates how one may accomplish steps 2-6 manually and create infrastructure.

1. Go into the dmcdeploy_dev

$ cd ~/dmcdeploy_dev  

2. Switch branches to the devmakina branch

$ git checkout devmakina    

3. Pull in dev repos as needed   -- in our example will pull in the front end repo  

$ cd ~ 

$ git clone https://bitbucket.org/DigitalMfgCommons/frontend.git    

4. Make your code changes

-- in our example will change the website footer to have a :)  

$ cd /home/ec2-user/frontend/app/templates/common/footer 

$ nano footer-tpl.html  

-- change a Lorem Ipsun line to  :)

-- save and exit    

5. Rebuild the app  

a. install the build tools on your dev system  

$ cd ~/dmcdeploy_dev  

$ nano build_tools.sh  

-- Each infrastructure component has a different build process

-- Edit the build_tools.sh to only install the tools you need to save time and space   

$ ./build_tools.sh  

FRONT END BUILD PROCESS  

1. In the FrontEnd repository, run the commands below.  

cd ~/frontend

INITIAL TOOLING INSTALL (only to be done once)

$ npm install

$ bower install

(answer 'n' to question when running bower install)

Make your code changes, if you haven't done so already.  

TO BUILD

$ gulp build  

-- output of this process can be found in dist folder

Commit your changes to the frontend locally

$ cd ~/frontend

-- forcibly add the dist folder to the commit

$ git add dist -f

$ git add -A 

$ git commit -m "your message"  

6. Deploy Infrastructure that reflects your changes  

$ cd ~/dmcdeploy_dev  

Will only deploy front end to prove it works

$ nano front.tf  

--remove the first and last line comments

--remove the dependency line

--remove the setting of rest env val  

7.  Start the git daemon on the dev machine  

$ cd ~

$ git daemon --base-path=. --export-all --reuseaddr --informative-errors --verbose &  

-- should see "Ready to rumble"  if all went well   

8. Edit terraform.tfvars

$ cd dmcdeploy_dev/

$ nano terraform.tfvars  

-- input your aws credentials on the first two lines

-- upload a key you wish to deploy with to this dev machine    

-- command used         

(quick shortcut env | grep MYip -- will show you your public ip )

From the machine that has the keyfile you wish to use.  

$ scp -i  mykey.pem somefile.txt ec2-user@my.ec2.id.amazonaws.com:/home/ec2-user/keys

verify the key is there

$ ls ~/keys/en  

-- edit key_name to the name of the remote key you just uploaded ( no .pem)

-- edit the key_full_path to " ~/keys/key_name.pem"

-- edit the stackPrefix  

-- to launch the alpha release    

    -- edit release = "alpha"

-- to launch latest nightly    

    -- edit release = "hot"  

9. Deploy new infrastructure  

-- Comment out (Add /* to first line and */ to last line) dev.tf if needed as will not be deploying dev machine

$ nano dev.tf

Run 'terraform plan' and confirm that you are actually deploying the infrastructure that you intend:

$ terraform plan

To actually deploy the infrastructure, run:

$ ./deployFromDev.sh   

10. To prove that you successfully launched the front end process, run:

$ terraform show

From the listing, determine the public IP of the aws_instance.front process.

Open that IP in any browser.  You should see the DMC user interface.    

11. To destroy the infrastructure 

$ terraform destroy 

    Once you have deployed your infrastructure further changes to your stack need only build and committed to your dev machine. To deploy those changes you ssh into the stack and execute updateMe.sh. This will update the code-base without having to relaunch the infrastructure.