Sceptre Tool

Sceptre is a tool to drive AWS CloudFormation. It automates the mundane, repetitive and error-prone tasks, enabling you to concentrate on building better infrastructure.

Features

• Code reuse by separating a Stack's template and its configuration
• Support for templates written in JSON, YAML, Jinja2 or Python DSLs such as Troposphere
• Dependency resolution by passing of Stack outputs to parameters of dependent Stacks
• Stack Group support by bundling related Stacks into logical groups (e.g. dev and prod)
• Stack Group-level commands, such as creating multiple Stacks with a single command
• Fast, highly parallelised builds
• Built in support for working with Stacks in multiple AWS accounts and regions
• Infrastructure visibility with meta-operations such as Stack querying protection
• Support for inserting dynamic values in templates via customisable Resolvers
• Support for running arbitrary code as Hooks before/after Stack builds

Benefits

• Utilises cloud-native Infrastructure as Code engines (CloudFormation)
• You do not need to manage state
• Simple templates using popular templating syntax - Yaml & Jinja
• Powerful flexibility using a mature programming language - Python
• Easy to integrate as part of a CI/CD pipeline by using Hooks
• Simple CLI and API
• Unopinionated - Sceptre does not force a specific project structure

Install

Using pip

$ pip install sceptre

More information on installing sceptre can be found in our Installation Guide

Example

Sceptre organises Stacks into "Stack Groups". Each Stack is represented by a YAML configuration file stored in a directory which represents the Stack Group. Here, we have two Stacks, vpc and subnets, in a Stack Group named dev:

$ tree

.

├── config

│   └── dev

│        ├── config.yaml

│        ├── subnets.yaml

│        └── vpc.yaml

└── templates

    ├── subnets.py

    └── vpc.py

We can create a Stack with the create command. This vpc Stack contains a VPC.

$ sceptre create dev/vpc.yaml

 

dev/vpc - Creating stack dev/vpc

VirtualPrivateCloud AWS::EC2::VPC CREATE_IN_PROGRESS

dev/vpc VirtualPrivateCloud AWS::EC2::VPC CREATE_COMPLETE

dev/vpc sceptre-demo-dev-vpc AWS::CloudFormation::Stack CREATE_COMPLETE

The subnets Stack contains a subnet which must be created in the VPC. To do this, we need to pass the VPC ID, which is exposed as a Stack output of the vpc Stack, to a parameter of the subnets Stack. Sceptre automatically resolves this dependency for us.

$ sceptre create dev/subnets.yaml

dev/subnets - Creating stack

dev/subnets Subnet AWS::EC2::Subnet CREATE_IN_PROGRESS

dev/subnets Subnet AWS::EC2::Subnet CREATE_COMPLETE

dev/subnets sceptre-demo-dev-subnets AWS::CloudFormation::Stack CREATE_COMPLETE

Sceptre implements meta-operations, which allow us to find out information about our Stacks:

$ sceptre list resources dev/subnets.yaml

 

- LogicalResourceId: Subnet

  PhysicalResourceId: subnet-445e6e32

  dev/vpc:

- LogicalResourceId: VirtualPrivateCloud

  PhysicalResourceId: vpc-c4715da0

Sceptre provides Stack Group level commands. This one deletes the whole dev Stack Group. The subnet exists within the vpc, so it must be deleted first. Sceptre handles this automatically:

$ sceptre delete dev

 

Deleting stack

dev/subnets Subnet AWS::EC2::Subnet DELETE_IN_PROGRESS

dev/subnets - Stack deleted

dev/vpc Deleting stack

dev/vpc VirtualPrivateCloud AWS::EC2::VPC DELETE_IN_PROGRESS

dev/vpc - Stack deleted

Note: Deleting Stacks will only delete a given Stack, or the Stacks that are directly in a given StackGroup. By default Stack dependencies that are external to the StackGroup are not deleted.

Sceptre can also handle cross Stack Group dependencies, take the following example project:

$ tree

.

├── config

│   ├── dev

│   │   ├── network

│   │   │   └── vpc.yaml

│   │   ├── users

│   │   │   └── iam.yaml

│   │   ├── compute

│   │   │   └── ec2.yaml

│   │   └── config.yaml

│   └── staging

│       └── eu

│           ├── config.yaml

│           └── stack.yaml

├── hooks

│   └── stack.py

├── templates

│   ├── network.json

│   ├── iam.json

│   ├── ec2.json

│   └── stack.json

└── vars

    ├── dev.yaml

    └── staging.yaml

In this project staging/eu/stack.yaml has a dependency on the output of dev/users/iam.yaml. If you wanted to create the Stack staging/eu/stack.yaml, Sceptre will resolve all of it's dependencies, including dev/users/iam.yaml, before attempting to create the Stack.

Usage

Sceptre can be used from the CLI, or imported as a Python package.

CLI

Usage: sceptre [OPTIONS] COMMAND [ARGS]...

 

  Sceptre is a tool to manage your cloud native infrastructure deployments.

 

Options:

  --version              Show the version and exit.

  --debug                Turn on debug logging.

  --dir TEXT             Specify sceptre directory.

  --output [yaml|json]   The formatting style for command output.

  --no-colour            Turn off output colouring.

  --var TEXT             A variable to template into config files.

  --var-file FILENAME    A YAML file of variables to template into config

                         files.

  --ignore-dependencies  Ignore dependencies when executing command.

  --help                 Show this message and exit.

 

Commands:

  create         Creates a stack or a change set.

  delete         Deletes a stack or a change set.

  describe       Commands for describing attributes of stacks.

  estimate-cost  Estimates the cost of the template.

  execute        Executes a Change Set.

  generate       Prints the template.

  launch         Launch a Stack or StackGroup.

  list           Commands for listing attributes of stacks.

  new            Commands for initialising Sceptre projects.

  set-policy     Sets Stack policy.

  status         Print status of stack or stack_group.

  update         Update a stack.

  validate       Validates the template.

Python

Using Sceptre as a Python module is very straightforward. You need to create a SceptreContext, which tells Sceptre where your project path is and which path you want to execute on, we call this the "command path".

After you have created a SceptreContext you need to pass this into a SceptrePlan. On instantiation the SceptrePlan will handle all the required steps to make sure the action you wish to take on the command path are resolved.

After you have instantiated a SceptrePlan you can access all the actions you can take on a Stack, such as validate(), launch(), list() and delete().

from sceptre.context import SceptreContext

from sceptre.plan.plan import SceptrePlan

 

context = SceptreContext("/path/to/project", "command_path")

plan = SceptrePlan(context)

plan.launch()

 

Kubernetes Concepts

Kubernetes Architecture:

Kubernetes components

Kubernetes Master

Kubernetes API

Scheduler

Controll Manager 

etcd

Kubernetes workernode

Kubelet

Kube-proxy

Docker Runtime

Pods

Container

Creating infra with Terraform and VSphere

Terraform used to created infra from many providers, let us one example with VSphere

we will export the path for terraform log file as per given commands

Step1) export TF_LOG_PATH=./terraform.log

export TF_LOG=TRACE

Creating tf file which has following contents

Step 2) create myfirst.tf file with following contents.

# Configure the VMware vSphere Provider

provider “vsphere” {

user           = “${var.vsphere_user}”

 password       = “${var.vsphere_password}”

vsphere_server = “${var.vsphere_server}”

allow_unverified_ssl = “true”

}

# Create a folder

resource “vsphere_folder” “Backend” {

path = “VirtualMachines”

datacenter = “datacenter1”

}

resource “vsphere_virtual_machine” “My_Vm” {

name   = “My_Vm”

vcpu   = 1

memory = 1024

datacenter = “datacenter1″

resource_pool = “mypool”

folder = “${vsphere_folder.backend.path}”

 network_interface {

label = “VM Network”

 }

disk {

datastore = “datastore1”

vmdk = “/test2/test2”   

 }

}

First section defines provider with credentials, based on your understanding you can seperate this file, like writing provider.tf file, but now write only one file.

Second section creates one folder “VMs” under “datacenter1”.

Third section actually creates a VM. resource_pool defines from where resources like cpu and memory should be assigned.  Network interface is the existing network.

Fourth section defines vmdk or template path ( use any one of ).

Step 3) Create variables.tf file , to define variables

————————————————————

variable “vsphere_user” {}

variable “vsphere_password” {}

variable “vsphere_server” {}

————————————————————

Here, variables are defined that are being used in myfirst.tf file.

Step 4) Create terraform.tfvars file where you have placed values for the variables.

—————————————————————-

vsphere_user = “root”

vsphere_password = “mypswd”

vsphere_server = “IpAddress”

—————————————————————-

In this file, provide the values for the defined variables. “terraform.tfvars”  is the default name for variables file. If you used any other file name, then you need to pass it while applying the terraform.

Step 5) Run from that directory where all these files are placed.

$ terraform plan

$ terraform apply

$ terraform show

AWS-SAA Certifications Questions and Answers

1) Amazon Glacier is designed for?

    Ans: Amazon Glacier is designed for infrequent access data, may be it will take it 3 to 5 hours to retrieve the data from the Glacier.

It is very family with its lifecycle, Glacier can store infrequent data and can be deleted after some point of time.

2) Which of the following correctly applies to changing the DB subnet group of your DB instance?
Ans:
An existing DB subnet group can be updated to add more subnets for existing availability Zones.
Removing subnets from existing subnet group can cause unavailbility
Updating an existing DB subnet group of deployed db instance is not currently allowed.
Explicitly changing the DB subnet group of depoyed DB instances is not currently allowed.

3) If you want to build your own payment application in AWS?
Ans:
Amazon AWS DevPay
Amazon AWS FPS

4) Which of the following should be referred to if you want to map Amazon Elastic Block Store to an Amazon EC2 instance for AWS Cloud formation resources?
Ans:
Reference the logical ID's of both the block stores and the instance

5) After creating the instance in AWS getting the error messages "Network error: connection timeout" or  "Error connecting to [instance], reason:-> connection timed out:"
Ans: Verify the private key file corresponds to the Amaxon key pair assigned to launch
Verify that you are entering correct username or not

6) Auto-scaling can not do?
Auto scaling can not increase instance size of EC2-instance
Autoscaling scale horizontly
Autoscaling can not do anything with respect to RDS instance

All about S3?
Ans: S3 stands for Simple storage service

S3 is cloud file storage service more akin to DropBox or GoogleDrive
S3 is very useful for storing the static data eg: you treat whatever data in the form of file
S3 is very useful when we are working full stack application which has layers, if you would like to
move the data from one layer to another layer, it will save the some time we can call it as intermediate state.
S3 namespace is valid global, it is available for all the regions.
S3 and Glacier works hand in hand because later is advance version or  enhancement of S3
Even we can lock the data in Amazon glacier access control with vault lock polocies to protect the data from the deletion.
In such way we can trigger after some point of time or expiry you can move the data or archival data to Glacier.

7) Amazon S3 bucket prevent and recover from accidental data loss?
Ans: object versioning and Multi factor authentication.


8) All about EBs? 
EBS is only for Single Region not across regions
Group of EBS is known as placement group.

9) What is Cloud Formation?

Cloud Formation is template which will allow us to scale or optimize the AWS resources.

10) what are the different types of block store in AWS?
Ans:
Instance block store
Elastic block store 

11) Amazon offers a 5 different options for Database?
RDS: Relational Database service
DynamoDB: is Amazon NoSQL database.
RedShift: Amazon Relational database for warehouse solution

12) Subnets are not distributed to across Zone, It should be limited to one AV Zone it self.

Subnet and AV are one-to-one communication.


13) A customer has established an AWS Direct Connect connection to AWS. The link is up and routes 
are being advertised from the customer's end, however the customer is unable to connect from 
EC2 instances inside its VPC to servers residing in its datacenter. 
Which of the following options provide a viable solution to remedy this situation? 

Ans:
 Add a route to the route table with an iPsec VPN connection as the target
 Enable route propagation to the customer gateway (CGW).

14) What does the following command do with respect to the Amazon Ec2 security groups?
ec2-create-group CreateSecurityGroup 

Ans:
Creates a new security group for use with your account.

15) if you want to launch Amazon Elastic Compute cloud(Ec2) instances and assign each instance a 
predetermined private IP address you should?

Ans:
You should launch instance in Virtual Private Cloud.

16) Why do you make subnets?
Ans:
If there is network with large no.of hosts will be very difficult to mange and tedious, 
so divide this network into subnets and manage the hosts.

One to one connectivity between subnet and avilability Zone

17) What is Cloud Trail?
Ans:
AWS Cloud trail has been designed to track and monitor API calls.
Cloud Trail is enabled for region level,
Cloud Trail logs are delivered to single Amazon S3 for aggregation.
Cloud Trail and Region--one-to-one communication

18)  Keys are associated with Regions?
Ans:
One to one combination between Master key and Region.

19) What is MFA in AWS?
Ans:
MFA stands for Multi factor Authentication, which is very useful and extra layer on your user name and password. 
Authentication code from MFA device.

20) When should i prefer IOPS over standard RDS Storage?
Ans:
When you are running batch oriented  jobs, then prefer IOPS


21) Which option meets the requirements for captioning and analyzing this data?
Ans:
Use Amazon Kinesis Streams to collect and process large streams of data records in real time.

22) What happens when you create a topic on SNS?
Ans:
when you create a SNS, ARN(Amazon Resource Name) will be created.

23) what happens to the I/O operation while you take a database snapshot?
Ans:
I/O operation to the db are suspended for a few minutes while backup is in progress.

24) All About EBS Volumes?
Ans:
EBS volumes persist independently from the running life of an EC2
we can attach an EBS volume to more than one EC2 instance( but please remember not at same time).
To view info about an Amazon EBS open the EC2 console at volumes

25)Response time for premium support?
Ans: 1 hour

26)What is oracle SQL developer?
Ans: A graphical java tool distributed without cost by oracle.

27) where SQL Server stores logins and passwords?
Ans: SQL Server stores logins and passwords in master database

28)What dynamo DB store from the following?
Ans:
Web session
Huge Data
Json

29) All about security groups?
Ans:
you can change the rules of the security group of the running instance
you can assign multiple security group to an instance(but remember max is 3)
you can delete rules from an existing security groups.

30) All about Auto scaling?
Ans:
It can add instances when CPU utilization is above threshold
It can remove the instances when CPU utilization is below threshold
It can maintain fixed number of running instances

31) To improve the performance of t2.small, what are steps you can take?
To increase the instance size, have to add array of EBS volumes.

32) Company looking for relasing all unused Elastic IP Address that incur charges?
Ans: 
when it allocated and associated with running instance

33) EBS backed instances are C4, M4, and T2

34) which route must be added to your instances which are in the subnet?
Ans:
Destination :0.0.0.0/0 and target is internal gateway.

35) authentication of SQS queue?
Ans:
Access Key id and request Signature 
X509 certificate

36) AWS Direct connect?
Ans:
Allows you to establish a direct network connection from your data center to AWS.

37) Cloud Front expiration time?
Ans:
An Expiration can be check how often to check for an updation version of file
By default each object expires after being in an edge location after 24 hours.

38)Distribute content to end users?
Ans:
Cloud Front

DevOps trainings and Job Support

Devops is very popular buzzword in the current IT industry and It offers lot of benefits to the entire delivery process of the organizations

Devops is cutting edge technology
Devops changes the way to look at the IT organization infrastructure.



We Offer DevOps trainings and Job Support.

Please contact for trainings and job support email: scrmtel@gmail.com and contact no:   7780199676

Chef-Cookbooks

For Django

yum_repository 'epel' do
   action :create
end


######  Global Install django through pip
package "python-pip" do
action :install
end

package "django" do
action :install
end

######  Set up directories to create a virtual environment
execute "Start project and create virtual environment" do
  command "mkdir /root/project; cd /root/project"
end

For Memcached cookbook:
------------------------------

package "memcached" do
  action :upgrade
end

##### 2: Make sure memcached is running
service "memcached" do
 enabled true
 running true
 action [:enable, :start]
end

-------------

yum_repository 'epel' do

   action :create

end

execute "update package" do

 command "yum update"
 ignore_failure true
 action :nothing
end.run_action(:run)

###### Set up Postgresql with psycopg2 and python django required packages #######
%w{postgresql postgresql-server postgresql-contrib python-django python-pip python-psycopg2}.each do |pkg|
package pkg do
     action :install
  end
end

######  Make sure postgresql service working and running fine
execute "Start Postgres daemon and enable it to start on boot" do
  command "service postgresql initdb; service postgresql start; chkconfig postgresql on"
end