Data Persistance
a. Persistent Data Store: Glacier, RDS
b. Transient Data Store: SQS, SNS
c. Ephemeral Data Store: EC2 Instance Store, Memcached

IOPS vs Throughput
a. IOPS: measure of how fast we can read and write to a device
b. Throughput: measure of how much data can be moved at a time

Consistency Models
a. ACID: Atomic (all or nothing); Consistent (must be valid), Isolated (can't mess with one another), Durable (Completed transaction must stick around)
b. BASE: Basic Availability (values availability even if stale), soft-state (might not be instantly consistent across stores), eventual consistency (will achieve consistency at some point)

S3
a. an Object Store
b. Maximum object size is 5TB; largest object in a single PUT is 5 TB
c. multi-part uploads is recommended if larger than 100MB
d. Consistency: read-after-write consistency for PUTs of new objects; HEAD or GET requests of the key before an object exists will result in eventual consistency; s3 offers eventual consistency for overwrite PUTs and DELETEs; updates to a single key are atomic.
e. S3 Security: user-based (IAM policies) -> resource based (bucket policy)-> resource based (object ACL); optional MFA before delete or changing the version state
f. versioning: new version each write, enable "roll-back" and "un-delete" capabilities; old versions count as billable size until they are permanently deleted; integrated with Lifecycle Management (optimize storage cost; adhere to data retention policies; keep s3 volumes well-maintained)
g. Cross-Region Replication: security, compliance, latency
h. Analytics: Data Lake Concept (Athena, Redshift Spectrum, QuickSight); IoT Streaming Data Repo (Kinesis Firehose); Machine Learning and AI Storage (Rekognition, Lex, MXNet); Storage Class Analysis (S3 Management Analytics)
i. S3 Encription at Rest (SSE-S3 (AES-256); SSE-C (AES-256 your own); SSE-KMS; Client-Side (your own local encryption)
j. Transfer Acceleration: Speed up data uploads using Cloud Front in reverse
k. Requester Pays: the requester rather than the bucket owner pays for requests and data transfer
l. Tags: assign tags to objects for use in costing, billing, security, etc.
m. Events: trigger notification to SNS, SQS or Lambda when certain events happen in your bucket
n. Static Web Hosting
o. BitTorrent: Use the BitTorrent protocol to retrieve any publicly available object by automatically generating
a .torrent file

Glacier
a. Cheap, slow to respond, seldom accessed
b. Used by AWS Storage Gateway Virtual Tape Library
c. Integrated with S3 via Lifecycle Management
d. Faster retrieval speed options if you pay more (still archive option)
e. Glacier Vault: IAM manages acces; Glacier Vault Lock manages policies (e.g. no deletes or MFA, immutable); Archive (File, zip, tar, etc. Max size 40TB, Immutable)
f. Glacier Vault Lock: you can initiate it and then decide whether to abort or complete it within 24 hours

EBS ("virtual hard drives" can only be used with EC2 and Tied to a single AZ, variety of optimized choices for IOPS, Throughput and Cost, Snapshots are great)
a. Compared with Instance (Instance - temporary; ideal for caches, buffers, work areas; dta goes away when EC2 is stopped or terminated)
b. Amazon EBS Snapshots (Cost-effective and easy backup strategy, share data sets with other users or accounts, migrate a system to a new AZ or region, converted unencrypted volume to an encrypted volume; incremental snapshot)
c. Schedule snapshots from volumes or instance every X hours by creating Snapshot Lifecycle Policy; retention rules to remove stale snapshots

EFS
a. Implementation of NFS file share
b. Elastic storage capacity, and pay for only what you use (in contrast to EBS)
c. Multi-AZ metadata and data storage
d. Configure mount-points in one or many AZs
e. can be mounted from on-premises systems (security concern though)
f. alternatively, use Amazon DataSync
g. 3x more expensive than EBS and 20x more expensive than S3

Amazon Storage Gateway
a. VM that run on-premises with VMWare or Hyper V or via a specially configured Dell hardware appliance
b. Provides local storage resources backed by S3 and Glacier
c. Often used in disaster recovery preparedness to sync to AWS
d. Useful in cloud migrations
e. modes

Storage Gateway Modes

Amazon WorkDocs
a. Secure, fully managed file collaboration service
b. Can integrate with AD for SSO
c. Web, mobile and native clients (no Linux client)
d. HIPAA, PCI DSS and ISO compliance requirements
e. Available SDK for creating complementary apps

Database on EC2
a. Run any database with full control and ultimate flexibility
b. Must manage everything like backups, redundancy, patching, scale
c. Good option if you require a database not yet supported by RDS, such as IBM DB2 or SAP HANA
d. Good option if it is not feasible to migrate to AWS-managed database

RDS (managed database option for MySQL, Maria, PostgreSQL, Microsoft SQL Server, Oracle and MySQL-compatible Aurora)
a. Best for structured, relational data store needs
b. Aims to be drop-in replacement for existing on-prem instances of same databases
c. Automated backups and patching in customer-defined maintenance windows
d. Push-button scaling, replication and redundancy
e. RDS anti-patterns

RDS anti-patterns

f. multi-AZ RDS
g. Read-replicas service regional (non-transactional database does not support replication)
h. Sync Replication (multi-az, between master and standby) vs Async Replication (read-replica, second/min delay)
i. One AZ fails, standy-by in another AZ assumes role of master, read replicas keep on keeping on
j. Whole region failed, read replica promoted to Stand-Alone (single-AZ), single AZ reconfigured to Multi-AZ

Dynamo DB
a. Managed, multi-AZ noSQL data store with cross-region replication option
b. defaults to eventual consistency reads but can request strongly consistent read via SDK parameter
c. Priced on throughput, rather than compute
d. Provision read and write capacity in anticipation of need
e. Auto scale capacity adjust per configured min/max levels
f. On-Demand capacity for flexible capacity at a small premium cost
g. Achieve ACID compliance with DynamoDB transactions
h. Partition key: A simple primary key which must be unique, to create an internal hash mapping
i. A composite primary key: a partition key + sort key, can have occurrences of the same partition key so long as the sort key is different
j. Secondary indexes (there is a limit to the number of indexes and attributes per index; it takes up storage space as well)

Secondary indexes

Secondary indexe use cases

k. Attribute Projections (like view in traditional database, not more than 20 attributes across all indexes)

Global Secondary Index use cases

Sample Data

Sparse Indexes

Use case 1

Use case 2

Redshift

Redshift

Data Lake
a. Query raw data without extensive pre-processing
b. Lessen time from data collection to data value
c. Identify correlations between disparate data sets

Data lake example

Neptune
a. Fully-managed graph database
b. Supports open graph APIs for both Gremlin and SPARQL

Elasticache
a. Fully managed implementation of two popular in-memory data stores - Redis and Memcached
b. Push-button scalability for memory, writes and reads
c. In Memory key/value store - not persistent in the traditional sense
d. Use cases

Elasticache use cases

e. Memcached vs Redis

Memcached vs Redis

Amazon Athena: SQL Engine overliad on S3 base on Presto; Query raw data objects as they sit in an S3 bucket; Use or convert your data to Parquet format if possible for a big performance jump; Similar in concept to Redshift but Athena does not need to perform joins with other data sources while Redshit Spectrum want to join S3 data with exsiting RedShift tables or create union products

Amazon Quantum Ledger Database
a. Based on blockchain concepts
b. Provides an immutable and transparent journal as a service without having to setup and maintain an entire blockchain framework
c. Centralized design allows for higher performance and scalability
d. Append-only concept where each record contributes to the integrity of the chain

Amazon Managed Blockchain
a. Fully managed blockchain framework supporting open source frameworks of Hyperledger Fabric and Ethereum
b. Distributed consensus-based concept consisting of a network, members, nodes and potentially applications
c. Uses the Amazon QLDB ordering service to maintain complete history of all transactions

Amazon Timestream Database
a. Fully managed database service specifically built for storing and analyzing time-series data
b. Alternatively to DynamoDB or RedShift and includes some built-in analytics like interpoloation and smoothing
c. Use cases: industrial machinery; sensor networks and equipment telemetry

DocumentDB (MongaDB compatibility)

Document DB

Elastic Search

ES

Database Options

How to choose database options

Storage options

Storage options

Pro Tips:
a. Use archiving and backup as the pilot for AWS business case
b. Make use of the S3 endpoints within your VPC
c. Learn how to properly secure your S3 bucket
d. Encrypt, Encrypt, Encrypt
e. Consider Aurora for your production MySQL/Maria or PostgreSQL needs
f. Consider NoSQL if you don't need relational database features
g. Database on EC2 cost less on the surface than RDS, but remember to factor in management (backup, patching, OS-level hardening)
h. There can be a performance hit when RDS backups run if you have only a single AZ instance

Questions

Gaming DL

Answers

b.

Gov data storage

Network Protocols

Network Protocols

Ephemeral Ports

Ephemeral Ports

TCP example

UDP example

Reserve IP Addresses

Reserve IP Addresses for 10.0.0.0/24

The Physical to Logical assignment of AZ's is done at the account level. AZ with the same name may refer to different physical AZ in a different account.

AWS Managed VPN

VPN Overview

AWS VPN Architecture

Redundant connection

Direct Connect

AWS Direct Connect

AWS Direct Connect Architecture

Direct Connect + VPN

Direct Connect + VPN

Direct Connect + VPN Architecture

VPN CloudHub (MPLS)

CloudHub

CloudHub Architecture

Software VPN (unmanaged VPN)

Software VPN

Software VPN Architecture

Transit VPC

Transit VPC

Transit VPC Architecture

VPC to VPC Connectivity
a. VPC Peering

VPC Peering

VPC Peering Architecture

b. AWS PrivateLink

AWS PrivateLink

VPC Endpoints

S3 example without public interface

Internet Gateways: horizontally scaled, redundant and highly available component that allows communication between your VPC and the Internet; No availablility risk or bandwidth constraints; If your subnet is associated with a route to the Internet, then it is a public subnet; Support IPv4and IPV5.
Use case: provide route table target for Internet-bound traffic; perform NAT for instances with public IP addresses (not for instances with prviate IP's only)

Egress-Only Internet Gateway (only for IPv6)
a. IPv6 addresses are globally unique and are therefore public by default
b. Provides outbound Interenet access for IPv6 addressed instances
c. Prevents inbound access to those IPv6 instances
d. Must create a custom route for ::/0 to the Egress-Only Internet Gateway
e. Use Egress-only Internet Gateway instead of NAT for IPv6

NAT Instance: EC2 instance from a special AWS-provided AMI; translate traffic from many private IP instance to a single public IP and back; doesn't allow public internet initiated connection into private instances; not supported for IPv6 (use Egress-Only Gateway instead); NAT instance must live on a public subnet with route to Internet Gateway; Private instances in private subnet must have route to the NAT instance, usually the default route destination of 0.0.0.0/0

NAT Gateway: fully managed NAT service; must be created in a public subnet; uses an Elstatic IP for public IP for the life of the Gateway; Private instances in private subnet must have route to the NAT instance, usually the default route destination of 0.0.0.0/0; Created in specified AZ with redundancy in that zone; For multi-AZ redundancy, create NAT Gateways in each AZ with routes for private subnets to use the local Gateway; Up to 5Gbps bandwidth that can scale up to 45 Gbps; Cannot use a NAT Gateway to access VPC peering, VPN or Direct Connect, so be sure to include specific routes to those in your route table

NAT Gateway vs NAT Instance

NAT Gateway vs NAT Instance

VPC Routing
a. Routing tables: VPC have an implicit router and main routing table; you can modify the main routing table or create new tables; each route table contains a local route for the CIDR block; most specific route for an adress wins

Routing table example

b. BGP: propagates info about network to allow for dynamic routing; required for direct connect and optional for VPN; alternative of not using BGP with AWS VPC is static routes; AWS supports BGP community tagging as a way to control traffic scope and route preference; required TCP port 179 + ephemeral ports; autonomous system number (ASN) = unique endpoint identifier; weighting is local to the router and higher weight is preferred path for outbound traffic

BGP Example

Route 53 Routing (Register domain names, check the health of your domain resources, route internet traffic for your domain)
a. Route 53 Routing Policies

Route 53 Routing Policies

b. Route 53 is a global service

ELB Routing
a. Distribute inbound connections to one or many backend endpoints
b. Three different options: Application Load Balancer (Layer 7); Network Load Balancer (Layer 4); Classic Load Balancer (Layer 4 or Layer 7)
c. Can be used for public/private workloads
d. Consume IP addresses within a VPC subnet for scaling

Load Balancer overview

Load Balancer Comparison

Network Load Balancer Example

Path based routing example

i. Stick Sessions (important feature for web application)

Stick Sessions example

Enhanced Networking
a. Generally used for High Performance Computing use-cases
b. Uses single root I/O virtualization (SR-IOV) to deliver higher performance than traditional virtualised network interfaces
c. Might have to install drvier if other than Amazon Linux HVM AMI
d. Intel 82599 VF Interface (10 Gbps) vs Elastic Network Adapter (25 Gbps)

Placement Groups

Placement Groups

Placement Groups Demo

Placement Groups Partition

CloudFront: Distributed connect delivery service for simple static asset caching up to 4k live and on-demand video streaming; integrated with Amazon Certificate Manager and supports SNI (server name indication): allow clients to choose which server it will connect if there are multiple servers share the same IP address

Slow connection between VPC (note that internet gateway does not have a bandwidth limit)

Challenge 1

Distribute web application traffic (session sticky, application layer)

Challenge 2

Three popular authentication/authorization methods

SAML 2.0 vs OAuth 2.0 vs OpenID Connect

AWS Tools for Account Management
a. AWS Organisations
b. Service Control Policies (sub-account inherited parent account's policies)
c. Tagging
d. Resource Groups
e. Consolidated Billing

Account Structure
a. Identity Account Structure

Identity Account Structure

b. Loggin Account Structure

Logging Account Structure

Publishing Account Structure

Information Security Account Structure

e. Central IT Account Structure

Central IT Account Structure

Multiple Account Example

Consolidated Billing

Consolidated Security

AWS Directory Services

AWS Directory Service Options

AD Connector vs Simple AD

AD Connector vs Simple AD

Credential and Access Management

Context

Example

Token vending machine concept

Token vending machine

AWS Secrets Manager

AWS Secret Manager

Encryption
a. Encryption at Rest: data is encrypted where it is stored such as on EBS, on S3, in an RDS database, or in an SQS queue waiting to be processed
b. Encryption in Transit: data is encrypted as it flows through a network or process, such as SSL/TLS for HTTPS, or with IPSec for VPN connections

Key management service (KMS)
a. Key storage, management and auditing
b. Tightly integrated into Many AWS service

DDoS

Mitigate DDoS

Intruder Detection and Prevention

Intruder Detection and Prevention Methods

IDS and IPS example

Cloud Watch vs Cloud Trail

Cloud Watch vs Cloud Trail