#AWS

AWS DVA-C01 Certification Study Notes

49 mins read

Post's image

Table of contents:

Last modified on FEB 28, 2021

AWS Notes

This article contains my study notes for the AWS Certified Developer - Associate DVA-C01 exam. This is by no means a complete guide to passing the exam, and I would NOT recommend studying solely based on this notes. Please take a look at the services FAQs and key concepts, because as of right now, this list only contains some specific details about the services, and doesn’t contain an introduction to them. I would also suggest looking at the official certification page, and study all the suggested whitepapers and services there.

Feel free to contact me if you have any suggestions, criticisms or contributions, as I would like this list to be a collaborative effort.

General

  • Exponential backoff is useful for Throttling exceptions (retrying after usually twice the time of the last request).
    • On soft limits, you can contact AWS support or use the API to request limit increases, but usually this isn’t the answer on the exam.
  • Timeouts usually indicate poorly configured security groups.
  • Deployment policies:
    • Canary: In a canary release deployment, total traffic is separated at random into a production release and a canary release with a preconfigured ratio.
    • All at once: Every instance is updated at the same time. (fast, free, has downtime) (rollback = repeat)
    • Rolling: Update batches of instances. (slow, free, no downtime) (rollback is difficult)
    • Rolling with aditional batches: Spin up an extra instance group so that capacity isn’t affected. (slow, pay extra instances, no downtime) (rollback is difficult)
    • Immutable: Spin up another set of instances in the same environment. (slow, pay extra set, no downtime) (rollback is easy - terminate new instances)
    • Blue/Green: Spin up another set of instances in another environment, then swap them. (slow, pay extra set, no downtime) (rollback is easy - swap environments)
  • Policy Types:
    • Identity-based policies: Attach managed and inline policies to IAM identities. (users, groups to which users belong, or roles)
    • Resource-based policies: Attach inline policies to resources. (S3 bucket policies and IAM role trust policies)
    • Permissions boundaries: Use a managed policy as the permissions boundary for an IAM entity (user or role).
    • Organizations SCPs – Use an AWS Organizations service control policy (SCP) to define the maximum permissions for account members of an organization or organizational unit (OU).
    • Access control lists (ACLs): Use ACLs to control which principals in other accounts can access the resource to which the ACL is attached, the only policy type that does not use the JSON policy document structure.
    • Session policies: Pass advanced session policies when you use the AWS CLI or AWS API to assume a role or a federated user. Limit the permissions that the role or user’s identity-based policies grant to the session.
  • The correct way of reusing SSH keys in your AWS Regions:
    • Generate a public SSH key (.pub) file from the private SSH key (.pem) file.
    • Set the AWS Region you wish to import to.
    • Import the public SSH key into the new Region.
  • The default region (if none is selected) is us-east-1 (N. Virginia).
  • Region/AZ names follow a certain convention. Taking us-east-1e as an example:
    • “us-east-1” is the region name.
    • “e” is the availability zone.

Other services

  • Using AWS Systems Manager, you can group resources, like Amazon EC2 instances, Amazon S3 buckets, or Amazon RDS instances, by application, view operational data for monitoring and troubleshooting, and take action on your groups of resources.
  • With Amazon Simple Workflow Service you create decider programs and with Step Functions, you define state machines.
  • Secrets manager costs more than SSM Parameter Store, but supports built-in secrets rotation, and every parameter MUST be encrypted.
  • AWS requires approximately 5 weeks of usage data to generate budget forecasts.
  • AWS Security Token Service (STS) is a web service that enables you to request temporary, limited-privilege credentials for AWS Identity and Access Management (IAM) users or for users that you authenticate (federated users).
    • Credentials that are created by using account credentials can range from 900 seconds (15 minutes) up to a maximum of 3,600 seconds (1 hour), with a default of 1 hour.
  • You can decrypt authorization issues using STS DecodeAuthorizationMessage.
  • Amazon Inspector is an automated security assessment service that helps improve the security and compliance of applications deployed on AWS.
  • AWS AppSync is prefered over Cognito Sync.
    • It extends Cognito’s capabilities by allowing multiple users to synchronize and collaborate in real time on shared data.
    • AWS AppSync allows users to build scalable applications, including those requiring real-time updates, on a range of data sources, including Amazon DynamoDB.
    • Uses GraphQL to make it easy for applications to get the exact data they need.
  • You can use Dry Run option on the CLI to test if you have the required permission for the action.
    • If you have the required permissions, the error response is DryRunOperation, otherwise, it is UnauthorizedOperation.

Identity and Access Management (IAM)

  • You can’t use resource-based policies or IAM roles to delegate access across different partitions (e.g. aws and aws-cn)
  • Trust policies are the only resource-based policy that IAM supports.
    • Defines which principal entities (accounts, users, roles, and federated users) can assume the role.
  • Roles need Trust Policies to allow services to assume them (using the AssumeRole STS API).
    • For cross-accounts, you modify the trust policy of the destination account’s role to allow the execution role of the origin account to assume it, then add the AssumeRole API call permission to the origin account’s execution role.
  • EC2 instances assume the role they receive through meta-data calls, not through AWS CLI.
  • IAM is used as a certificate manager only when you must support HTTPS connections in a Region that is not supported by ACM.
  • A permissions boundary is an advanced feature for using a managed policy to set the maximum permissions that an identity-based policy can grant to an IAM entity.
    • An entity’s permissions boundary allows it to perform only the actions that are allowed by both its identity-based policies and its permissions boundaries.
  • You need to activate IAM user access to the Billing and Cost Management Console for all the users that need access (you still need the IAM policy).
  • IAM Access Analyzer helps you identify the resources in your organization and accounts, such as Amazon S3 buckets or IAM roles, that are shared with an external entity.
  • IAM Access Advisor reports the last-used timestamp that represents when a role was last used to make an AWS request.
  • You can use IAM policy variables so that the policy snippet can be made generic for all IAM users.
  • A policy’s Statement needs to have the following policy elements: Effect (Allow/Deny), Actions (specific action or actions that will be allowed or denied) and Resources (object or objects that the statement covers).
    • Conditions (conditions for when a policy is in effect) are optional.
    • You can use NotAction/NotResource, and Principal/NotPrincipal to define the principal that is allowed or denied access to a resource.
  • You can use IAM policy generator to generate policies.
  • Explicit denies are evaluated before explicit allows.

AWS Organizations

  • Master account can create a trail that will log events for all accounts in the organization.
    • Member accounts can see the trail but can’t modify or delete it. By default, they can NOT view the log files in the bucket.

Elastic Container Service (ECS)

  • Terminating container instances on the STOPPED state can lead to synchronization issues (it may continue to appear as a resource in the ECS cluster).
  • For dynamic port mappings, use ECS with target groups on ELBs and leave the Host Port set to 0/blank (random).
  • When using shared memory, place 2 containers into a single task-definition.
  • Fargate launch-type is used for serverless containers.
  • In the /etc/ecs/ecs.config file you have to configure the parameter ECS_CLUSTER=’your_cluster_name’ to register the container instance with a cluster named ‘your_cluster_name’.
  • For X-Ray integration, you can:
    • Run the X-Ray container as a Daemon: Each ECS instance will have it’s own X-Ray container.
    • Run the X-Ray container as a Side Car: Each ECS container will have a X-Ray sidecar attached to it. (useful for Fargate)
  • On /etc/ecs/ecs.config (full list here):
    • ECS_ENABLE_TASK_IAM_ROLE: Must be set to allow ECS tasks to use IAM roles.
    • ECS_CLUSTER: This refers to the ECS cluster that the ECS agent should check into.
    • ECS_RESERVED_PORTS: An array of ports that should be marked as unavailable for scheduling on this container instance.
    • AWS_DEFAULT_REGION: The region to be used in API requests.

Amazon Elastic Compute Cloud (EC2)

  • Billed by second, with a minimum of 60s
  • Instance use case classes:
    • General purpose: Balance of compute, memory and networking resources. (A,T,M)
    • Compute Optimized: Compute bound applications that benefit from high performance processors. (C)
    • Memory Optimized: Deliver fast performance for workloads that process large data sets in memory. (R,X,High Memory,z)
    • Accelerated Computing: Use hardware accelerators or co-processors to perform functions. (P,Inf,G,F)
    • Storage Optimized: Designed for workloads that require high, sequential read and write access to very large data sets on local storage. (I,D,H)
  • On-Demand instances is the most known billing method, billed on usage.
  • Spot instances are for when you don’t mind losing instances, essentially bidding for instances.
    • You can specify that EC2 should do one of the following when it interrupts a Spot Instance: Stop, Hibernate, Terminate (default)
  • Reserved instances (Zonal/Regional) are useful when you will need instances for a very long time.
    • Convertible instances are nice if you need to change the instance type, but it costs more.
    • Scheduled reserved instances are capacity reservations that recur on a daily, weekly, or monthly basis.
    • Only Zonal reserved instances provide capacity reservation, Regional ones only provide discounts.
  • Dedicated instances guarantee they will run on a hardware dedicated for you (multiple instances from your own account can run on the same host, but won’t share the hardware with other hosts)
  • Dedicated hosts allow you to have control over the placement of instances, and will give you access to visibility of sockets, cores and host ID (useful for conformity or BYOL use cases).
  • Burstable instances will be very slow if you run out of Burst Credits.
    • You can get unlimited burst credits for a higher price.
  • User Data scripts are only executed during the boot cycle (when you first launch an instance), and by default as the root user.
  • You can query meta-data at http://169.254.169.254/latest/meta-data
    • ami-id, hostname, public-ipv4, iam role, etc…
  • Syntax to enable detailed monitoring for an existing instance: aws ec2 monitor-instances –instance-ids i-1234567890abcdef0
  • Syntax to enable detailed monitoring for a new instance: aws ec2 run-instances –image-id ami-09092360 –monitoring Enabled=true
  • With detailed monitoring data is available in 1-minute periods on CloudWatch Metrics for an additional charge.
    • With basic monitoring, data is available automatically in 5-minute periods at no charge.
    • If you need data in smaller periods, create a high-resolution custom metric and push the data using a script triggered every X seconds.
  • EBS-optimized instances enable EC2 instances to fully use the IOPS provisioned on an EBS volume.
  • A cluster placement group provides low-latency networking between all instances in the cluster.
  • You configure monitoring using a launch configuration or template. Monitoring is enabled whenever an instance is launched, either basic monitoring (5-minute granularity) or detailed monitoring (1-minute granularity).
    • By default, basic monitoring is enabled when you create a launch template or when you use the AWS Management Console to create a launch configuration.
    • Detailed monitoring is enabled by default when you create a launch configuration using the AWS CLI or an SDK.

Auto Scaling Groups (ASG)

  • You can change the health check type of the ASG from EC2 to ELB, so that failed ELB health checks are able to terminate unhealthy instances.
  • You can have target tracking scaling policies using:
    • ASGAverageCPUUtilization: Average CPU utilization of the Auto Scaling group.
    • ASGAverageNetworkIn: Average number of bytes received on all network interfaces by the Auto Scaling group.
    • ASGAverageNetworkOut: Average number of bytes sent out on all network interfaces by the Auto Scaling group.
    • ALBRequestCountPerTarget: Number of requests completed per target in an Application Load Balancer target group.
  • Number of requests per minute is a custom metric you need to create, as it’s not readily available in CloudWatch.
  • To have access of average RAM usage on ASGs you need to use the PutMetricData API.
  • ASGs can have instances on multiple AZs of a single region.
  • For Auto Scaling groups in a VPC, the EC2 instances are launched in subnets.

Elastic Container Registry (ECR)

  • When logging in, CLI v1 and v2 are different
    • AWS CLI v1 returns a command that needs to be executed. ($(aws ecr get-login –no-include-email))
    • AWS CLI v2 uses pipes to log in. (aws ecr get-login-password –region region | docker login –username AWS –password-stdin aws_account_id.dkr.ecr.region.amazonaws.com)

ElastiCache

  • You can implement two types of caching strategies:
    • Lazy loading: Read first from cache, if the value isn’t there you load from the source than save on the cache.
    • Write-Through: Updates data on the cache whenever data is written to the source.
  • Global Datastore on ElastiCache for Redis provides fully managed, fast, reliable and secure cross-region replication.
    • Up to two other cross-region replica clusters.
  • ElastiCache for Redis has an Enhanced Engine: More usable memory, Improved synchronization, Smoother failovers.
  • If you want to make the application tier stateless and outsource the session information, you should use ElastiCache.
  • You can leverage ElastiCache for Redis with cluster mode enabled to enhance reliability and availability with the primary benefit of horizontal scaling.
    • Allows a configuration with replication, if you can’t easily recreate your data.

CodeDeploy

  • CodeDeploy rolls back deployments by redeploying a previously deployed revision of an application as a new deployment.
    • The last known working version of the application is deployed with a new deployment ID.
  • Rollbacks are always deployed first on the failed instances.
  • You need to install the CodeDeploy Agent on the machines.
  • You can run it on a VPC but the CodeDeploy Agent needs to access public endpoints.
  • Free for EC2/Lambda, paid by instance update if on-premise.
  • Settings are in a file in the root directory called “appspec.yml” or “appspec.json”.
  • CodeDeploy Hooks are found in AppSpec to manage lifecycle events.
  • Lifecycle events hooks (for EC2/on-premise, full list here)
    1. Start (managed by CodeDeploy)
    2. BeforeBlockTraffic: Run tasks on instances before they are deregistered from a load balancer.
    3. BlockTraffic: Internet traffic is blocked from accessing instances that are currently serving traffic. (managed by CodeDeploy)
    4. AfterBlockTraffic: Run tasks on instances after they are deregistered from a load balancer.
    5. ApplicationStop: Gracefully stop the application or remove currently installed packages in preparation for a deployment.
    6. DownloadBundle: CodeDeploy agent copies the application revision files to a temporary location (managed by CodeDeploy)
    7. BeforeInstall: Preinstall tasks, such as decrypting files and creating a backup of the current version.
    8. Install: CodeDeploy agent copies the revision files from the temporary location to the final destination folder. (managed by CodeDeploy)
    9. AfterInstall: Use for tasks such as configuring your application or changing file permissions.
    10. ApplicationStart: Restart services that were stopped during ApplicationStop.
    11. ValidateService: Verify the deployment was completed successfully.
    12. BeforeAllowTraffic: Run tasks on instances before they are registered with a load balancer.
    13. AllowTraffic: Internet traffic is allowed to access instances after a deployment. (managed by CodeDeploy)
    14. AfterAllowTraffic: Run tasks on instances after they are registered with a load balancer.
    15. End (managed by CodeDeploy)
  • You can use the :max_revisions: option in the agent configuration file to specify the number of application revisions to archive by entering any positive integer.
    • CodeDeploy also archives the log files for those revisions. All others are deleted, except for the log file of the last successful deployment.

CodeBuild

  • You can noticeably reduce your build time by caching dependencies in S3.
  • To quickly troubleshoot the issue to see which commands or settings located in the BuildSpec file are causing an issue, use CodeBuild locally using CodeBuild agent.
  • Settings are in a file in the root directory called “buildspec.yml”
  • Artifacts are encrypted using KMS at rest and using Sigv4 + SHA-256 certificates in transit.
  • CodeBuild scales automatically, you do not have to do anything for scaling or for parallel builds.

CodeCommit

  • Repositories are automatically encrypted at rest.

CodePipeline

  • You can use Amazon CloudWatch Events to detect and react to changes in the state of a pipeline, stage, or action.

X-Ray

  • X-Ray sampling is useful to reduce overheads and costs, but isn’t useful for debugging.
    • By default, the X-Ray SDK records the first request each second, and five percent of any additional requests.
  • Annotations are key-value pairs with string, number, or Boolean values. Annotations are indexed for use with filter expressions.
  • Metadata are key-value pairs that can have values of any type, including objects and lists, but are not indexed for use with filter expressions.
  • The computing resources running your application logic send data about their work as segments. A segment provides the resource’s name, details about the request, and details about the work done.
  • X-Ray can also track requests flowing through applications or services across multiple AWS Regions.
  • You MUST run the X-Ray agent in order to have logs. The Execution role must have access to X-Ray.
  • Keeps service graph data for 30 days, processes data in 30 second increments.
  • AWS_XRAY_DAEMON_ADDRESS environment variable sets the host and port of the X-Ray daemon listener.

Simple Storage Service (S3)

  • Keys are the locations of objects, they are composed of bucket ARN + prefix + object name + suffix
    • There are no folders, but the UI simplifies it so it seems they exist.
  • If versioning isn’t enabled, two events on the same file may result in only one log entry (the other event is lost).
  • NEVER push S3 access logging into the same bucket, as it can generate a loop and grow your S3 bucket substantially.
  • An object owner can share objects by creating a pre-signed URL, using their own security credentials, to grant time-limited permission to download the objects.
  • Storage classes:
    • Standard (no retrieval fee)
    • Intelligent tiering (no retrieval fee) (min 30 days)
    • IA (min 30 days, 128kb minimum charge)
    • One-zone IA (min 30 days, 128kb minimum charge)
    • Glacier (min 90 days, 40kb minimum charge)
    • Glacier Deep Archive (min 180 days, 40kb minimum charge)
  • Retrieval from Glacier can be:
    • Expedited (1 to 5 minutes)
    • Standard (3 to 5 hours)
    • Bulk (5 to 12 hours)
  • Retrieval from Glacier Deep Archive can be:
    • Standard (12 hours)
    • Bulk (48 hours)
  • Lifecycle rules to auto move classes / delete objects.
  • On versioned objects, deletes are just markers.
    • MFA Delete on object versions requires root account to be setup, can’t be setup on console.
  • Use CloudFront to have HTTPS on static hosting buckets.
  • Query logs without downloading from S3 using Athena.
  • CloudTrail tracks by default only track bucket-level actions. To track object-level actions you must enable S3 data events.
    • Records all API actions on S3 Objects and receive detailed information such as the AWS account of the caller, IAM user role of the caller, time of the API call, IP address of the API, and other details.
  • Encryption
    • Client-side encryption: The whole process is done by the client
    • SSE-KMS (x-amz-server-side-encryption: aws:kms): Server-side encryption using KMS CMS. (Can throttle the bucket if KMS is throttled)
    • SSE-S3 (x-amz-server-side-encryption: AES256): Server-side encryption managed by S3. You don’t have access to the keys.
    • SSE-C (x-amz-server-side​-encryption​-customer-algorithm: AES256): Server-side encryption where the user sends the key with the objects.
  • S3 object locks enables you to store objects with a Write Once Read Many (WORM) model.
  • You can put objects as big as 5TB, but the maximum size of a single PUT request is 5GB.
    • You must use multipart upload for big files, although it is recommended after 100MB.
  • Pagination uses –max-items and –starting-token.
    • You can use the –page-size option to specify that the AWS CLI requests a smaller number of items from each call to the AWS service. The CLI still retrieves the full list but performs a larger number of service API calls in the background and retrieves a smaller number of items with each call.
  • Until an update is fully propagated, Amazon S3 might return outdated data (data consistency model).

Key Management Service (KMS)

  • Maximum file size is 4KB. (Encrypt / Decrypt)
    • You can use envelope encryption (GenerateDataKey/GenerateDataKeyWithoutPlaintext) to encrypt the files yourself, and send the Data Key bundled with the object where it’s needed.
  • The soft limit is 10000 requests (you can increase it by request if you are getting throttled too often).

Elastic Block Store (EBS)

  • EBS Volumes are AZ locked.
  • Volume Types:
    • gp2: General purpose SSD (1GB-16TB, burst IOPS to 3000, 3IOPS per GB, max 16000)
    • io1 and io2: Provisioned IOPS SSD (4GB-16TB, provisioned IOPS, max 64000 if nitro else 32000, min 100)
    • st1: Throughput optimized HDD (HDD, 500GB-16TB, max throughput of 500MB/s)
    • sc1: Cold HDD (infrequently accessed, 500GB-16TB, max IOPS of 250, max 250MB/s)
    • Magnetic (standard, a previous-generation type) supporting up to 90MiB/sec and 200 IOPS.
  • Provisioned IOPS instances must have a maximum ratio of 50:1 for IOPS:GB.
  • Instance stores are way faster, but very expensive.
    • The high IOPS comes because they are phisically connected to the instances.
  • Multi-Attach is avaiable (for io1 and io2) up to 16 instances.
    • FEB 14, 2020: you support for multi-attaching Provisioned IOPS (io1). (Only at creation time: can’t be modified, opposed to io2)
    • Enabled during creation (except for io2, you can enable it later).
    • No aditional charges
  • Supports encryption in-flight and at rest (using KMS).
  • Encryption by default is a Region-specific setting. If you enable it for a Region, you cannot disable it for individual volumes or snapshots in that Region.
  • A volume restored from an encrypted snapshot, or a copy of an encrypted snapshot, is always encrypted.
    • To encrypt an EBS volume, you need to create a snapshot and then encrypt the snapshot (enabling encryption while copying the snapshot).
    • You cannot remove encryption from an encrypted volume or snapshot.
    • If you need to unencrypt a volume, use the dd command to move the data from the original, encrypted volume to a new, unencrypted volume.
  • AWS CloudTrail event logs for ‘CreateVolume’ aren’t available for EBS volumes created during an Amazon Elastic Compute Cloud (Amazon EC2) launch.

Relational Database Service (RDS)

  • AWS-managed relational databases.
  • PostgreSQL, MySQL, MariaDB, Oracle, Microsoft SQL Server, Aurora.
  • Backed by EBS: gp2 or io1.
  • Instance types are General Purpose (T,M) or Memory Optimized (R,X,Z).
  • Automated backups + snapshots for point in time recovery.
    • Automated backups are limited to a single AWS Region while manual snapshots and Read Replicas are supported across multiple Regions.
  • Encrypting an un-encrypted DB:
    1. Create a snapshot
    2. Copy the snapshot and enable encryption
    3. Restore the DB from snapshot
    4. Migrate applications
  • DB instances that are encrypted can’t be modified to disable encryption.
    • You can export old data from encrypted instance to new one.
  • On MySQL, Aurora and Postgres, you can use IAM database authentication (token granted by IAM & RDS API).
    • Each token has a lifetime of 15 minutes. You don’t need to store user credentials in the database, because authentication is managed externally using IAM.
  • Multi-AZ is useful for disaster recovery (availability)
  • Read replicas are useful for speed, asynchronous replication. (Only one master instance, replicas can be promoted to their own DB)
  • RDS integrates with CloudWatch Metrics, and you can enable Enhanced Monitoring (real time metrics for the operating system (OS) that your DB instance runs on).
    • Performance Insights is a database performance tuning and monitoring feature that helps you quickly assess the load on your database, and determine when and where to take action.
  • RDS Proxy allows applications to pool and share connections established with the database, improving database efficiency and application scalability.
    • Database credentials, authentication, and access can be managed through integration with AWS Secrets Manager and AWS Identity and Access Management (IAM).
    • Only for MySQL/Postgres compatible RDBMS.
  • You can enable automatic backups but as of now, the retention period is 0 to 35 days.
    • If you want backups retained for more days, create a cron event in CloudWatch, which triggers an AWS Lambda function that triggers a database snapshot.

CloudFront

  • Geo Restrictions (Whitelists or Blacklists)
  • Files are cached (via headers, cookies, query string) for a TTL (0s to 1 year). (great only for static content)
    • Cache-Control: max-age=0 header makes cache invalidation (needs proper IAM roles).
  • Cache invalidations cost money (CreateInvalidation API).
  • Viewer Protocol Policy:
    • Redirect HTTP to HTTPS
    • HTTPS only
    • HTTP and HTTPS
  • Origin Protocol Policy:
    • HTTPS only
    • Match viewer
  • Signed URL/Cookies to distribute premium content.
    • You define expiration, IP Ranges and Trusted Signers
    • Signed URLs are for individual resources, cookies are for resource groups.
  • Only the root account can create CloudFront key pairs (used to create signed URLs).
  • Lambda@Edge is a feature of Amazon CloudFront that lets you run lambda code closer to users of your application, running your code in response to events generated by the Amazon CloudFront content delivery network (CDN).
    • When CloudFront receives a request from a viewer (viewer request)
    • Before CloudFront forwards a request to the origin (origin request)
    • When CloudFront receives a response from the origin (origin response)
    • Before CloudFront returns the response to the viewer (viewer response)
  • Default logs are stored in S3, and there are no aditional costs.
  • Real-time logs are sent to Kinesis Data Streams for an aditional cost.
  • Origin shields makes ONLY a centralized cache able to request the origin.

Elastic File System (EFS)

  • ~3x more expensive than gp2 EBS volumes.
  • Works with EC2 on multi-AZ (it is a regional service).
  • You can use AWS DataSync to automatically copy files between an existing file system and a new EFS file system.
  • Uses NFSv4.1 POSIX-compatible file system.
  • You can define security groups.
  • Pay per use (measured in “GB-Month,” which are added up at the end of the month).
  • General purpose (default) or Max I/O.
  • Standard storage tier or EFS-IA (can be moved automatically using life cycles, only for files bigger than 128kb) to have costs on file retrieval but lower cost to store.
  • You can optionally select the Provisioned Throughput mode and provision the throughput of your file system (default is Bursting Throughput).

Aurora

  • Aurora is PostgreSQL (3x speed) or MySQL (5x speed) compatible.
  • Storage automatically grows by 10GB increments, max 64TB.
  • Up to 15 read replicas instead of 5.
  • Instantaneous failover.
  • Makes 6 copies of data across 3 AZs.
    • 4 out of 6 needed for writes.
    • 3 out of 6 needed for reads.
  • Support for cross-region replication.
  • Aurora Serverless
    • Automated instantiation and auto-scaling
    • For infrequent/intermittent/unpredictable workloads
    • Pay per second
  • Aurora Global Database (better than cross-region read replicas)
    • 1 primary region
    • 5 secondary read only regions
    • 16 read replicas per secondary region
    • Promoting another region has Recovery Time Objective (RTO) of less then a minute
    • Replication lag is less than 1 second.

Route 53

  • Routing policies
    • Simple: just return every record.
    • Weighted: return records by percentages (useful for blue/green deployments).
    • Latency: return by minimum latency.
    • Failover: return a second group if the first has no healthy targets.
    • Geolocation: queries a location DB based on the requester’s IP and routes them accordingly.
    • Multi Value: returns up to 8 healthy records.
  • Change TTL to update your routes faster.
  • Aliases are records that internally get mapped to AWS resources.
    • CNAME records can’t be the domain’s apex (root), Aliases can.
  • Sends logs to CloudWatch Logs.
  • Route 53 Resolver automatically answers DNS queries for local VPC domain names for EC2 instances.
    • For all other domain names, Resolver performs recursive lookups against public name servers.

Elastic Load Balancer (ELB)

  • Never resolve the IP of a load balancer as it can change with time. You should always use the DNS name.
  • Cross-zone load balancing is always enabled on ALB, NLB + GLB disabled by default (can be enabled), CLB enabled by default on console, disabled by default on API or CLI.
  • You can get a lot of traffic on a single instance if stickyness is enabled.
  • SSL termination is useful to remove TLS processing costs from EC2 units.
  • Authentication can be done natively using Cognito User Pools.
  • When you use an HTTP/HTTPS load balancer, the IP address of the client is in the X-Forwarded-For request header.
    • X-Forwarded-Port and X-Forwarded-Proto identify the client port and protocol (HTTP/HTTPS).
    • On NLBs the source IP of the client is preserved and provided to the application.
  • Instances of specific capacity types aren’t equally distributed across AZs (so that lower-capacity instances aren’t having too many outstanding requests).
    • It’s a best practice to use similar instance types and configurations to reduce the likelihood of capacity gaps and traffic imbalances.
    • A traffic imbalance might also occur if you have instances of similar capacities running on different Amazon Machine Images (AMIs).
  • If no target groups are associated, you’ll get a 503 error - Service Unavaiable.
  • You can use ALB access logs for detailed logging into S3 (time the request was received, the client’s IP address, latencies, request paths, and server responses).
  • Except on CLBs (charged for each hour and for each GB of data transferred), ELB is charged by Load Balancer Capacity Units (LCU, NLCU and GLCU).
    • New connections: Number of newly established connections per second. Typically, many requests are sent per connection. (25/s)
    • Active connections: Number of active connections per minute. (3000/m)
    • Processed bytes: The number of bytes processed by the load balancer in Gigabytes (GB) for HTTP(S) requests and responses. (1GB/h or 0.4GB/h for Lambda)
    • Rule evaluations: It is the product of number of rules processed by your load balancer and the request rate. The first 10 processed rules are free (Rule evaluations = Request rate * (Number of rules processed - 10 free rules). (1000/s)
    • Only charged on the dimension with the highest usage.
  • UDP traffic on NLB selects a target using a flow hash algorithm based on the protocol, source IP address, source port, destination IP address, and destination port (5 tuples).

API Gateway

  • Authentication using Resource Policies, Standard IAM roles and policies, IAM tags, Lambda authorizers or Cognito User Pools.
  • HTTP API, REST API, WebSocket API and REST API (VPC).
  • Usage plans when associated with API Keys can throttle requests for SaaS use cases.
  • You can import APIs with Swagger or OpenAPI.
  • Use Stage variables to dynamically point to lambda aliases.
    • Every time you update an API, you must redeploy the API to an existing stage or to a new stage.
  • Mapping Templates provides you with the flexibility to perform many-to-one parameter mappings.
  • You can use lambda for authentication of 3rd party authorization using Lambda custom authorizers.
    • Client passes a bearer token or request parameter.
    • The Lambda function authenticates the caller.
    • If the call succeeds, the Lambda function grants access by returning an output object containing at least an IAM policy and a principal identifier.
    • API Gateway evaluates the policy (can generate a 403 ACCESS_DENIED).
  • You can use Sigv4 signing for IAM authentication.
    1. Create a canonical request.
    2. Use the canonical request and additional metadata to create a string for signing.
    3. Derive a signing key from your AWS secret access key. Then use the signing key, and the string from the previous step, to create a signature.
    4. Add the resulting signature to the HTTP request in a header or as a query string parameter.
  • You can attach endpoint policies to interface VPC endpoints to improve the security of private APIs.
  • When your resources receive requests from a domain you want to restrict servicing requests, you must disable CORS for selected methods on the resource.

Elastic Beanstalk (EB)

  • Environment types:
    • load balanced
    • single instance
  • Web environment tier to serve HTTP(S) requests.
  • Worker environment tier to process events.
    • For a worker environment, you need a cron.yaml file to define the cron jobs and do repetitive tasks.
  • Cross-acount beanstalk environment migrations are done as follows:
    1. Create a saved configurated in the origin account, and download it locally from S3.
    2. Make account-specific parameter changes, upload the changed template to S3 in destination account.
    3. Create an application from ‘Saved Configurations’ in the destination account.
  • You can use the EB CLI to more efficiently make operations on EB.
  • EB is backed up by CloudFormation.
  • Beanstalk configuration files follow the naming convention of: .ebextensions/.config
  • You can decouple a RDS DB instance by safeguarding it from deletion (Enable deletion protection).
    • You can also make a snapshot of the database before it gets deleted.
  • You can avoid hitting the application version limit by applying an application version lifecycle policy to your applications.

DynamoDB

  • About DynamoDB WCU + RCU
    • 1 WCU for 1 writes of 1kb / sec
    • 1 RCU for 1 writes of 4kb / sec IF STRONGLY CONSISTENT
    • 1 RCU for 2 writes of 4kb / sec IF EVENTUALLY CONSISTENT
    • 1 extra RCU / WRU for transactional operations. (may add latency)
    • Batch operations rounds the size of each item to the next 1KB boundary (e.g. 500 bytes becomes 1KB).
  • Deleting a table and re-creating it is better than deleting each item individually.
  • DynamoDB has two built-in backup methods (On-demand, Point-in-time recovery) that write to Amazon S3, but you will not have access to the S3 buckets of these backups.
    • To create backups that you can download locally, use AWS DataPipeline, Amazon EMR or AWS Glue.
  • WCUs and RCUs are distributed equally across partitions.
    • In 2018, Amazon introduced Amazon DynamoDB adaptive capacity, which alleviates this issue by allowing the allocation of RCUs and WCUs to be more dynamic between partitions (enabled automatically).
    • Adaptive capacity works by automatically and instantly increasing throughput capacity for partitions that receive more traffic.
  • DynamoDB guarantees Optimistic concurrency using conditional writes.
  • Use global tables for globally distributed users
  • Amazon DynamoDB Time to Live (TTL) allows you to define a per-item timestamp to determine when an item is no longer needed.
    • The attribute’s value is a timestamp in Unix epoch time format in seconds.
  • Recommendations:
    • Use DynamoDB Accelerator (DAX) for read-only patterns, and it can help solving hot partitions.
    • Add random numbers or digits from a predetermined range to the partition key for write-heavy use cases.
    • Don’t use sequences or unique IDs generated by the legacy DB engine as the partition key.
  • There are two formats of partition keys:
    • Partition key: Simple primary key.
    • Partition key and sort key: Referred to as a composite primary key, this type of key is composed of two attributes.
  • When creating a secondary index:
    • If you use the same partition key, you can use a Local Secondary Index (LSI)
    • If you want to use a different partition key, you can use a Global Secondary Index (GSI, creates a whole new table)
  • GSI tables can throttle the main table if they run out of WCUs or RCUs.
  • A single partition can store at most 10 GB of data.
    • DynamoDB splits partitions by sort key if the collection size grows bigger than 10 GB.
  • A single partition can support a maximum of 3000 read capacity units (RCUs) or 1000 write capacity units (WCUs).
    • ProvisionedThroughputExceededException error.
  • A DynamoDB stream is an ordered flow of information about changes to items in a DynamoDB table (stores information in a log for up to 24 hours).
  • Simple API calls:
    • PutItem / DeleteItem / BatchWriteItem (Up to 25 writes, up to 16MB of data written, up to 400KB of data per item)
    • GetItem / BatchGetItem (Up to 100 items, up to 16MB of data)
  • There is a 1-MB limit on items that you can fetch through a single query operation.
    • You can use Limits to evaluate less items.
    • You can use pagination to receive items in batches.
  • Queries are way faster than scans, because they compare directly using the Partition Key (=) and SortKey (=,<,<=,>,>=,Between,Begin)
    • You can use FilterExpression to filter further (done client-side) and ProjectionExpression to receive only specified attributes.
    • Parallel scans makes scans run faster, but can use more RCUs.
  • You can store large objects (larger than 400KB) storing the meta-data of the object uploaded somewhere else (e.g. S3)

Simple Queue Service / Simple Notification Service (SQS/SNS)

  • You can set the message retention period on SQS to a value from 60 seconds to 1,209,600 seconds (14 days) using the SetQueueAttributes action (default is 4 days).
  • SQS delivers AT LEAST ONCE for standard queues and EXACTLY ONCE for FIFO Queues.
    • You cannot convert an existing standard queue to FIFO queue.
    • SQS messages are ordered by MessageGroupId.
    • MessageDeduplicationId makes it so that if a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren’t delivered during the 5-minute deduplication interval.
  • SQS has message-oriented APIs.
  • There are no message limits for messages stored in SQS, but ‘in-flight messages’ (received from a queue by a consumer, but not yet deleted from the queue) do have limits.
    • There can be a maximum of approximately 120,000 in-flight messages.
  • You can specify the maximum number of messages (up to 10) that you want to retrieve.
  • SQS ChangeMessageVisibility to extend the processing time if needed, otherwise message will be delivered to another consumer (Max 12 hours).
    • Default is 30 seconds.
  • Fan Out pattern is useful for delivering SNS notifications into multiple SQS queues.
  • You can use DLQs for messages that couldn’t be processed after X retries, or lambda functions that failed every allowed retry.
  • Long polling is useful when too much calls are being issued to SQS, and no messages are being delivered back.
    • WaitTimeSeconds parameter is the one that defines if the call is Short polling or Long polling (max 20 seconds).
  • Max SQS message size (MaximumMessageSize attribute on SetQueueAttributes API call) is between 1kb and 256kb.
    • You can use S3 and the SQS Extended Client Library for Java for storing and consuming messages up to 2 GB.
  • Encrypt at rest using KMS (SQS KMS, Encrypted SNS Topics).
    • You can choose to have SQS encrypt messages stored in both Standard and FIFO queues using an encryption key provided by AWS Key Management Service (KMS).
  • You should use a dead-letter queue (DLQ) with standard queues.
    • DLQs should be used when you applications doesn’t depend on ordering.

Kinesis

  • Consumers use DynamoDB for checkpoints (needs IAM permissions).
  • Provides libraries build for Java.
  • Built in encryption support for KMS using KPL (Kinesis Producer Library) or KCL (Kinesis Consumer Library)
  • Shard partitioning on Kinesis Data Streams are useful for distributed reading
    • One KCL consumer max per shard, to guarantee ordering.
  • Amazon Kinesis Data Firehose is the easiest way to load streaming data into data stores and analytics tools.
  • Amazon Kinesis Data Analytics is the easiest way to analyze streaming data in real-time.
  • Limits 2MB/second/shard output and 1MB/second/shard input.
  • By default, the 2MB/second/shard output is shared between all of the applications consuming data from the stream.
    • You should use enhanced fan-out if you have multiple consumers retrieving data from a stream in parallel (scales with shard #).

Lambda

  • You are charged based on the number of requests for your functions and the duration (rounded by nearest 1 millisecond), the time it takes for your code to execute.
    • DEC 01, 2020: Now rounded by 1ms rather than rounded up to the nearest 100ms (duration).
    • By default, you can have 1000 concurrent executions and 15 minutes of execution time.
  • Lambda Environment total environment variables size must not exceed 4 KB.
    • You can have as much environment variables as you want.
  • You can have a Deployment package (.zip file archive) size of 50 MB (zipped, for direct upload), 250 MB (unzipped, including layers) or 3 MB (console editor).
  • Lambda Destinations are useful for result event processing (failed invocations, successful invocations…)
    • Recommended by AWS instead of DLQs
    • Async invocations: SQS, SNS, Lambda, EventBridge buses
    • Event Source Mapping: SQS, SNS
  • Lambda vertical scaling is done by incrementing the RAM through 64MB increments (up to 10240MB, min 128MB).
    • On DEC 1, 2020 AWS announced that Lambda customers can now provision Lambda functions with a maximum of 10,240 MB (10 GB), no longer 3008MB of memory. Customers now have access to up to 6 vCPUs.
    • This also increments proportional CPU power and other resources.
  • Event Source Mappings are a way to have lambda poll for events (SQS, Kinesis, DynamoDB streams)
    • It uses Long Polling
  • You can use Reserved Concurrency to throttle a lambda function’s concurrency.
    • Useful so that a single function doesn’t throttle all the other functions running on unreserved concurrency.
  • Provisioned concurrency is useful so that you don’t get cold start times on peaks.
    • You can configure Application Auto Scaling to manage provisioned concurrency on a schedule or based on utilization.
  • You can connect a lambda function to a VPC on a private subnet and use Security Groups to access private services.
  • Aliases can ONLY point to version, not other aliases.
    • You can configure weights on 2 versions to make blue/green deployments.
    • CodeDeploy updates can be Linear (grow traffic % each X minutes), Canary (stay at % for X minutes then 100%) or AllAtOnce deployment for aliases traffic shift.
  • Use lambda layers for re-usable dependencies or unsupported platforms.
  • Enabling Lambda X-Ray integration ensures that the X-Ray agent is used in lambda.

CloudWatch

  • To enable the CloudWatch agent to send data from an on-premises server, you must specify the access key and secret key of the IAM user that you created earlier.
  • You can use Anomaly Detection to leverage statistics and machine learning to detect anomalous behaviour.
    • Anomaly detection alarms based on a metric’s expected value.
    • You can choose whether the alarm is triggered when the metric value is above the band of expected values, below the band, or both.
  • CloudWatch ServiceLens ties together CloudWatch metrics and logs, as well as traces from AWS X-Ray to give you a complete view of your applications and their dependencies.
    • Quickly pinpoint performance bottlenecks, isolate root causes of application issues, and determine users impacted.
  • CloudWatch Synthetics allows you to monitor application endpoints more easily.
    • Monitoring of your REST APIs, URLs, and website content, checking for unauthorized changes from phishing, code injection and cross-site scripting.
  • CloudWatch Logs Insights enables you to interactively search and analyze your log data in Amazon CloudWatch Logs.
    • If an issue occurs, you can use it to identify potential causes and validate deployed fixes.
    • Automatically discovers fields in logs from AWS services such as Amazon Route 53, AWS Lambda, AWS CloudTrail, and Amazon VPC, and any application or custom log that emits log events as JSON.
  • EventBridge Buses are equal to CloudWatch Events (but have extra features such as custom event buses)
    • Events indicate changes in your AWS Environment.
    • Rules matches incoming events and routes them to targets for processing (A rule’s target must be in the same region as the rule).
    • Targets process events.
  • You can use CloudWatch integration with S3 to save logs onto an S3 bucket.
  • Metrics to CloudWatch support high-resolution periods down to 1-second.
    • CloudWatch stores it with a resolution of 1 second, and you can read and retrieve it with a period of 1 second, 5 seconds, 10 seconds, 30 seconds, or any multiple of 60 seconds.
    • High resolution metrics alarms can be defined for 10s or 30s.

CloudTrail

  • Supports encryption using SSE-KMS.
    • You need S3 GetObject and KMS Decrypt IAM Permissions.
    • By default, the log files delivered by CloudTrail to your bucket are encrypted.
  • You can send logs to S3 and/or to CloudWatch with CloudWatch integration (CloudTrail assumes a role).
  • You can have a maximum of 5 trails per region (one is free).
  • CloudTrail is always on, and logs are kept in it for 90 days.
  • Data events are also known as data plane operations and are often high-volume activities.
  • Management events are also known as control plane operations.
  • CloudTrail insights detects abnormal activity, it raises an event through dashboard views in the CloudTrail console, delivers the event to your Amazon Simple Storage Service (Amazon S3) bucket, and sends the event to Amazon CloudWatch Events.
  • You can use Java’s CloudTrail Processing Library for event downloading and analysis.

Virtual Private Cloud (VPC)

  • VPC Flow Logs is a feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC.
  • Traffic mirroring is useful for analysis and offers a more precise insight compared to VPC Flow Logs.
  • To grant internet access to your resources, the resource’s associated VPC must have a NAT gateway (or NAT instance) in a public subnet.
  • VPC Peering is used to connect with other VPCs (IP ranges can’t overlap).
  • PrivateLink is used to connect with you own services.
  • Internet Gateways are used to connect public subnets to the internet.
    • The network access control lists (ACLs) that are associated with the subnet must have rules to allow inbound and outbound traffic on port 80 and port 443.
    • The route table in the instance’s subnet should have a route defined to the Internet Gateway.
  • Basic options:
    • Single public subnet only.
    • Public and private subnets.
    • Public and private subnets and AWS Site-to-Site VPN access.
    • A private subnet only and AWS Site-to-Site VPN access.
  • Access Control Lists (ACLs) are subnet-specific and allow/deny traffic (its stateless).
    • Security Groups are stateful: Any changes applied to an incoming rule will be automatically applied to the outgoing rule. (e.g. if you allow an incoming port 80 request, the outgoing port 80 request will be automatically allowed)

Step Functions

  • Activities are a way to associate code running somewhere (known as an activity worker) with a specific task in a state machine.
    • For example, an activity worker can be an application running on an Amazon EC2 instance or an AWS Lambda function.
  • If you need external signals or child processes, you should use Simple Workflow Service (SWF).
  • Standard workflows
    • Maximum duration of 1 year
    • Over 2.000 execution start rate per second.
    • Over 4.000 state transition rate per second.
    • Priced by state transition.
    • Can be inspected by CloudWatch logs or by the StepFunctions API, or visually debugged through the console.
    • Exactly-once execution.
  • Express workflows
    • Maximum duration of 5 minutes
    • Over 100.000 execution start rate per second.
    • Nearly unlimited state transition rate.
    • Priced by the number of executions, their duration and memory consumption.
    • Can only be inspected by CloudWatch logs.
    • At-least-once execution.

Serverless Application Model (SAM)

  • SAM is backed up by CloudFormation, using the Transform field.
    • It supports the following resources:
    • AWS::Serverless::Api: Creates a collection of Amazon API Gateway resources and methods that can be invoked through HTTPS endpoints.
    • AWS::Serverless::Application: Embeds a serverless application from the AWS Serverless Application Repository or from an Amazon S3 bucket as a nested application.
    • AWS::Serverless::Function: Creates an AWS Lambda function, an AWS Identity and Access Management (IAM) execution role, and event source mappings that trigger the function.
    • AWS::Serverless::HttpApi: Creates an Amazon API Gateway HTTP API, which enables you to create RESTful APIs with lower latency and lower costs than REST APIs.
    • AWS::Serverless::LayerVersion: Creates a Lambda LayerVersion that contains library or runtime code needed by a Lambda Function.
    • AWS::Serverless::SimpleTable: Creates a DynamoDB table with a single attribute primary key. It is useful when data only needs to be accessed via a primary key.
    • AWS::Serverless::StateMachine: Creates an AWS Step Functions state machine, which you can use to orchestrate AWS Lambda functions and other AWS resources to form complex and robust workflows.
  • You can use the SAM CLI to simplify calls to the cloudformation section on AWS CLI.

CloudFormation (CF)

  • You should use the aws cloudformation validate-template command or ValidateTemplate action for the AWS CLI or AWS CloudFormation API.
  • A stack policy is a JSON document that describes what update actions can be performed on designated resources.
  • The “cloudformation package” command packages the local artifacts (local paths) that your AWS CloudFormation template references.
  • The “cloudformation deploy” command deploys the specified AWS CloudFormation template by creating and then executing a changeset.
  • Exported output values need to be unique in a single region.
    • You can only delete a stack if nothing is referencing it.
  • Parameters is the only field that can’t have conditions.
    • With constraints, you can describe allowed input values so that AWS CloudFormation catches any invalid values before creating a stack.
  • Standard parameter types are (full list here):
    • String: A literal string.
    • Number: An integer or float.
    • List: An array of integers or floats that are separated by commas.
    • CommaDelimitedList: An array of literal strings that are separated by commas.
    • AWS-Specific Parameter Types: AWS values such as Amazon EC2 key pair names and VPC IDs. (e.g. AWS::EC2::KeyPair::KeyName)
    • SSM Parameter Types: Parameters that correspond to existing parameters in Systems Manager Parameter Store. (e.g. AWS::SSM::Parameter::Name)
  • Useful intrinsic functions (full list here) :
    • !Join: Appends a set of values into a single value, separated by the specified delimiter.
    • !Sub: Substitutes variables in an input string with values that you specify.
    • !Ref: Returns the value of the specified parameter or resource.
    • !GetAtt: Pass in the logical name of the resource and attribute name to reference an atribute after creation.
    • !FindInMap: Pass the Map name, Top level key and Second level key to retrieve a value from the Mappings section.
    • !ImportValue: Returns the value of an output exported by another stack.
  • Pseudo parameters are parameters that are predefined by AWS CloudFormation.
    • AWS::AccountId: AWS account ID of the account in which the stack is being created.
    • AWS::NotificationARNs: Returns the list of notification Amazon Resource Names (ARNs) for the current stack.
    • AWS::NoValue: Removes the corresponding resource property when specified as a return value in the Fn::If intrinsic function.
    • AWS::Partition: Returns the partition that the resource is in. (aws, aws-cn or aws-us-gov)
    • AWS::Region: Returns a string representing the AWS Region in which the encompassing resource is being created.
    • AWS::StackId: Returns the ID of the stack.
    • AWS::StackName: Returns the name of the stack.
    • AWS::URLSuffix: Returns the suffix for a domain. (The suffix is typically amazonaws.com, but might differ by region. For example, the suffix for the China (Beijing) region is amazonaws.com.cn)
  • You can use a DeletionPolicy on resources such as AWS::EC2::Volume, AWS::RDS::DBInstance, and AWS::Redshift::Cluster (reference here).
    • Delete deletion policy deletes the resource and all its content if applicable during stack deletion.
    • Snapshot deletion policy creates a snapshot before deleting.
    • Retain deletion policy keeps the resource running after stack deletion.
  • You can use AWS::Lambda::Function with code inline for Node.js and Python as long as there are no dependencies for your code.

References

THE NOTES ARE PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE NOTES OR THE USE OR OTHER DEALINGS IN THE NOTES.