AWS by Harsha - part2

 Class 27: RDS

Benefits

The main benefit of Amazon RDS is that it helps organizations deal with the complexity of managing large relational databases. Other benefits include the following:Ease of use. Admins don't need to learn specific database management tools. They also can manage multiple database instances using the management console. RDS is compatible with database engines that users may already be familiar with, such as MySQL and Oracle And it automates manual backup and recovery processes.
Cost-effectiveness. According to AWS, customers only pay for what they use. Also, the time spent maintaining instances is reduced, because maintenance tasks, such as backups and patching, are automated.
The use of read replicas routes read-heavy traffic away from the main database instance, reducing the workload on that one instance.
RDS splits up compute and storage so admins can scale them independently.
Drawbacks

Some downsides of using Amazon RDS include the following:Lack of root access. Because it is a managed service, users do not have root access to the server running RDS. RDS restricts access for certain procedures to those with advanced privileges.
Downtime. Systems must go offline for some patching and scaling procedures. The timing on these processes varies. With scaling, compute resources need a few minutes downtime on average.

• We can create a read replica within same region or in another region as well.The cost of same region(same or different availabilty zone is free) only costs when we create read replica in other region
• The use case of read replica is when your reporting application to perform analytics operations on database then if you run it on production dababase then it will impact the performance of db so we create a read replica and perform operations on that
• we cannot perform update,insert and delete operations on readreplicas
• Multi AZ and disaster recovery, It will create a standalone secondary instance and when primary instance is down it will promote other instance as primary
• We can take snapshots and restore them when ever required, manually.
• We can even take automated backups

Class 29:(Redshift,Elastic cache,Document DB)

Amazon Redshift:

Amazon Redshift is a fully managed data warehouse service in the cloud. Its datasets range from 100s of gigabytes to a petabyte. The initial process to create a data warehouse is to launch a set of compute resources called nodes, which are organized into groups called cluster. After that you can process your queries.

Steps for creation of cluster and connecting to it using SQL workbench can be found here

Lab Activity:

• Created Redhat openshift cluster
• Connected it using SQLworkbench, For the first time we use admin credentials to connect and will need to create additional users and handover that to other teams to perform queries
• To analyze data dowloaded sample data and uploaded them to s3 buckets, while importing data from s3 buckets to Amazon redshift we need to provide access and secretkey(for authentication)
• Performed sql queries to get data information
• Used powerbi tool to visualize data(This is not responsibily of devops engineer but for fun we have done it)
  
  
  
  
  

Elastic cache:

Amazon ElastiCache is a cache-as-a-service developed and distributed by Amazon Web Services (AWS). It comes as a fully managed solution that has the ability to deploy, manage, and scale a distributed in-memory cache environment in the cloud.

Amazon describes it as a service that allows you to easily create, operate, and scale open-source compatible in-memory data stores within the cloud. Simply, this means that it eliminates the complexity associated with setting up and managing a distributed cache environment.

The system itself is built to boost the performance of web-based applications by reducing the database load through the quick retrieval of data from high throughput and low latency in-memory data stores. As such, you can think of it as a high-performance caching system that facilitates demanding web applications requiring a quick response.

That said, it’s worth noting that Amazon ElastiCache relies on two different caching engines — Memcached and Redis.

 

More info here

DocumentDB:

Amazon DocumentDB is a NoSQL JSON document database service with a limited degree of compatibility with MongoDB.

DocumentDB is not based on the MongoDB server. Rather it emulates the MongoDB API, and runs on top of Amazon’s Aurora backend platform. This creates significant architectural constraints, functionality limitations, and broken compatibility.

DocumentDB claims to support the MongoDB 4.0 API, which implies that it is at parity with MongoDB v4.0, released back in June 2018. In actual fact the DocumentDB 4.0 feature set still closely resembles early MongoDB 3.0 and 3.2, released in 2015, and compatibility testing reveals it fails 66% of the MongoDB API correctness tests. Applications written for MongoDB will need to be re-written to work with Amazon DocumentDB.

More info: here

Class 30(cloud watch):

Lab activity 1:(To test cpu utilization)

1.created alarm to monitor cpu utilization if it reaches 50% of its utilization

2. created simple notification service(Under Application Integration), We need to create a topic and that topic needs to be assigned to subsciption.We have differnt options for subscription.We have choosen mail.

3. Created a ec2 instance and created alarm for that(this can be done from the instance on monitoring tab or from cloud watch)

4.Installed htop(to monitor system resources cpu,memory and disk) and also installed stress tool for testing stress
5. Intentionally increased cpu utilization by making use of stress and observed that alarm was triggered and we got the mail as well

Lab activity 2(To test memory and disk utilization)

1.In harsha videos he used perl scripts to monitor memory and disk utilization, but aws introduced new features withou relying on perl scripts.check it out here

lab activity 3(To collect application logs)

Class 31&32(Cloud Formation templates)

aws docs: here and anatomy link here

• we can update the stack policy by uploding new file 
• pay attaention while writing templates, if you want to launch ec2 instances acrooss multiple regions then don't hard code ami, it will change from region to region

3 main topics:

1. Drift

2. stack sets

3. stack policies

1.Drift - If we deploy anything using cloud formation templates and some one changed any of the configuration manually for a resource(i.e without editing stacks).We can create a drift and those can be can be detected.who made those changes can be seen in cloud trail.

Eg: If we create a create a ec2 insance using cloud formation template and if some one accidentally added tags to that ec2 instances manually from ui then those changes can be seen in drift(Even we can created sns(simple noification service) for that which is no covered in video)

One strange thing we observed while testing drift is it only detects changes which are performed on the resources defined in templates but not to the new resources. 

Eg: I've created a new ebs volume and attached to that ec2 instance thn no drift was created.Since ebs resource was not defined in template

more info: here

2.stack sets:

One line explanation with picture here

Lab acivity:

We deployed stack onto 2 different accounts using one main accounts.

• Before proceeding we need to create few roles( AWSCloudFormationStackSetAdministrationRole on controller aws account and AWSCloudFormationStackSetExecutionRole role on target accounts), details here
• Make sure to add 2 target aws accounts into same organizaion unit, we need to create a stackset in orginsation account and need to specify ou id while creating stack sets
• This is not widely used, but good to know.
  

3. stack policies

When you create a stack, all update actions are allowed on all resources. By default, anyone with stack update permissions can update all of the resources in the stack. During an update, some resources might require an interruption or be completely replaced, resulting in new physical IDs or completely new storage. You can prevent stack resources from being unintentionally updated or deleted during a stack update by using a stack policy. A stack policy is a JSON document that defines the update actions that can be performed on designated resources.

More info with examples: here

Lab activity:

We applied a stack policy to ec2 instance and vpc(refer file) and then tried updating stack i.e editing aws templae json file and uploaded updated file.It promted error.

Class 33(cloud formation part-3: nested stacks,designer,import resources)

Nested stacks:

In the cloud formation template file(json or yaml), you can define multiple stacks, just upload file to s3 and specify template file, you can define as many stacks as needed.

designer:

Using diagram you can view pictorically the template.It can be used to prepare a plan but most people don't use for that purpose as it is confusing.

I generally use that to get finalized desiggn of my template in pictorial way and to convert json to yaml and viceversa

import resources:

Lets say you created some resources using stack and then later you manually created ec2 instance and you wanted to manage that ec2 instance as part of previously created stack, you can import the resource using this option.

Lab Activity: We created a VPC wih 2 subnets and 1 ec2 instance using in one of the subnets using template file.

Later we created another ec2 instance and then we want to manage taht instance as part of previously created stack so that when we delete stack this resource also gets deleted. We included ued import resources option here, refer video.

Class 34:(Terraform basics,aws config, service catalog etc)

Terraform basics:

Terraform is not truely works on every public cloud with same config.terraform can be used on all clouds but we need to change configuration files based on cloud provider we are working with

cloud trail vs aws config:

A good way to think of it is that AWS Config will tell you what your resource state is now or what it was at a specific point in the past whereas CloudTrail will tell you when specific events in the form of API calls have taken place.

We will get detailed information in case of aws config

The aws config will take some time to populate results, we need to wait.

Lab activity: Enabled aws config for ec2 instance type and then created ec2 instance and observed the output, later we also changed configuration of ec2 instance(i. e added additional security groups,added tags and elastic ip) and observed the ouput.

aws 

WS Service Catalog supports the following types of users:

• Catalog administrators (administrators) – Manage a catalog of products (applications and services), organizing them into portfolios and granting access to end users. Catalog administrators prepare AWS CloudFormation templates, configure constraints, and manage IAM roles for products to provide for advanced resource management.

• End users – Receive AWS credentials from their IT department or manager and use the AWS Management Console to launch products to which they have been granted access. Sometimes referred to as simply users, end users may be granted different permissions depending on your operational requirements. For example, a user may have the maximum permission level (to launch and manage all of the resources required by the products they use) or only permission to use particular service features.

Lab activity:

• Created a product, by using basic cloud formaion template which we used earlier to deploy vpc,subnets and ec2 instances
• Created a portfolio and attach the product created above to that portfolio.
• Created a user and attached that user to same portfolio, user should have password.
• Tried to login using user & password and now he can see the nice ui to create vpc,subnets and ec2 instance under catalog 

Class 35: