AWS Virtual Private Cloud
A virtual private cloud (VPC) is a virtual network that closely resembles a traditional network that you’d operate in your own data center, with the benefits of using the scalable infrastructure of AWS. VPC is a logical data center within an AWS Region. When you create an account, a default VPC is created for you in each Region. For example, when you create an EC2 instance in this account, AWS will put it inside the default VPC. When you check the description of this EC2 instance, you will see the VPC ID, Subnet ID, public and private IP address. All subnets under default VPC have a route out to the internet. If you delete this default VPC, you need to contact Amazon to restore it back. In this article, I will focus on custom VPC. In the custom VPC, you have complete control over your virtual networking environment e.g. selection of your own IP address range, the creation of subnets, and configuration of route tables and network gateways. Let’s take a look at how custom VPC processes.
List of Certifications
- VPC Overview
- Custom VPC Components
- VPC Connectivity Options
- VPC High Availability and Disaster Recovery
- Clean Up
I will use the diagram from Building a Modular and Scalable Virtual Network Architecture with Amazon VPC: Quick Start Reference Deployment to explain VPC.
The quick start has a CloudFormation template to setup VPC with associated networking resources. The template creates a Multi-AZ, multi-subnet VPC infrastructure with managed NAT gateways in the public subnet for each Availability Zone in US West Region. This is a great example with four AZs to support production and disaster recovery environment. To review the template: download the template, login your AWS console, open CloudFormation service, click Design Template (1 in the diagram), then open the download one (2 in the diagram). You can review the template under JSON. You can customize this CloudFormation. I will write another article on CloudFormation.
The diagram looks complicated at first. You may have questions such as how to setup Elastic Load Balancer (ELB) and Route 53 to support high availability (HA) and disaster recovery (DR). Let’s create this custom VPC first then we will go through all components under VPC to understand how it works. Before the end of this article, I will share with you how to setup Elastic Load Balancer and Route 53 to support HA and DR in VPC.
The following steps will create this quick start custom VPC in us-west Oregon region:
Step 1: Make sure you switch to the Oregon region. If you didn’t have any Key Pair in this region, then create Key Pair for the custom VPC. Go to EC2 console, choose Key Pairs then Create Key Pair.
Step 2: Click deploy now, then click Next button
Step 3: In Specify details page, there are three items need to be set up. Availability Zones: pick up AZs e.g. us-west-2a, us-west-2b. If you pick up two AZs then put 2 into Number of Availability Zones. Select Key pair name from Step 1. Click Next to Options page. On the Options page, keep everything as default for this setup. When you’re done, click Next button.
Step 4: Click Next in the Review. Monitor the status of the stack. When the status is CREATE_COMPLETE, the stack is ready.
Go to VPC dashboard, you will see the following components under VPC. There are 2 VPCs: one is default VPC created by AWS on each region. the other one is our sample one. Let’s take a look at each component.
Custom VPC Components
The definition of VPC from Amazon: A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. A subnet is a range of IP addresses in your VPC. You can easily customize the network configuration for your Amazon VPC. For example, you can create a public-facing subnet for your web servers that have access to the Internet, and place your backend systems such as databases or application servers in a private-facing subnet with no Internet access. You can leverage multiple layers of security, including security groups and network access control lists, to help control access to Amazon EC2 instances in each subnet. Additionally, you can create a Hardware Virtual Private Network (VPN) connection between your corporate data center and your VPC and leverage the AWS Cloud as an extension of your corporate data center. You can enable Flow Logs for Custom VPC to see all traffic.
All VPCs will be associated with an IPv4 Classless Inter-Domain Routing(CIDR) block e.g. 10.0.0.0/16 netmask in the diagram which allows 2^16 (65536) IP address range with five IP addresses (0, 1, 2, 3, and 255) are reserved by AWS. For example, in this case, the following addresses are reserved by AWS: 10.0.0.0 Network address, 10.0.0.1 VPC router, 10.0.0.2 DNS server, 10.0.0.3 future use, 10.0.0.255 network broadcast address (not support in VPC). Allowed CIDR block size is between /28 netmask (2^(32-28=4) total 16 available IP addresses) and /16 netmask (2^(32-16=16) total 65536 IP addresses)
A subnet is always associated with a single AZ and cannot span multiple AZs. But an AZ can host multiple subnets e.g. in this example 1 public subnet and 2 private subnets. Each subnet in a VPC is associated with an IPv4 CIDR block that is a subset of the /16 CIDR block of its VPC.
|VPC||10.0.0.0/16 (65536 IP addresses)|
|Availability Zone 1||10.0.0.0/18 (Less than VPC’s total IP addresses)|
|Private subnet A||10.0.0.0/19 (Less than AZ’s total IP address: 8187 IP addresses)|
|Public Subnet||10.0.128.0/20 (4091 IP addresses)|
Only one Internet Gateway per VPC to communicate with the Internet. An Internet gateway is horizontally-scaled, redundant, and highly available. It imposes no bandwidth constraints. An Internet gateway is a fully managed AWS service that serves two purposes:
- provides a target in the VPC route tables for Internet-routable traffic
- performs bi-direction source and destination network address translation (NAT) for instances that have been assigned public IP or Elastic IP addresses.
NAT Gateway or NAT instance (just one EC2 instance) is deployed to the public subnet. It enables the instances in a private subnet to connect to the Internet or other AWS services but prevents the Internet from initiating connections with the instances. NAT instance needs to add Elastic IP. But NAT Gateway is automatically assigned a public IP. The difference between Internet Gateway and NAT Gateway is the Internet Gateway connects to the internet for public subnet and NAT Gateway connects to the internet or other services for private subnet. So two NAT Gateways are created for two private subnets.
You may also hear Private Gateway. VPC uses a virtual private gateway to grant instances secure access to a user’s corporate network via VPN or direct connect links. Please review it in VPC Connectivity Options.
A Route Table contains a set of rules, called routes, that are used to determine where network traffic is directed. Each subnet in your VPC must be associated with a routing table; the table controls the routing for the subnet. A subnet can only be associated with one route table at a time, but you can associate multiple subnets with the same route table. When you’re creating VPCs, do not modify the default route table. Add your custom rules by creating a custom route table. If you modify the default route, it will affect all instances. Instead, create a new route table for customization. In the below diagram, Public Subnets Route Table is the default route table with two targets: local and Internet Gateway.
In the below diagram, two route tables are created for each private subnet with three targets: local, NAT Gateway(or NAT instance), VPC endpoint for S3.
VPC endpoints provide a secure, reliable connection to Amazon S3 without requiring an Internet gateway, a NAT device, or a virtual private gateway. With these endpoints, you can access S3 resources from the private subnet.
AWS provides security mechanisms for your instances in the form of Network access control lists (ACLs) and Security Groups. NACLs act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level. Security groups act as a firewall for associated EC2 instances, controlling both inbound and outbound traffic at the instance level. If you want to learn more on cloud security in other AWS services, please go to my post AWS Cloud Security.
Here is the difference between them:
|Security Group||Network ACL|
|Operates at the instance level (first layer of defense)||Operates at the subnet level (second layer of defense)|
|Supports allow rules only||Supports allow rules and deny rules|
|Is stateful: Return traffic is automatically allowed, regardless of any rules||Is stateless: Return traffic must be explicitly allowed by rules|
|We evaluate all rules before deciding whether to allow traffic||We process rules in number order when deciding whether to allow traffic. Lower order rules take effect in case of conflict with higher order rules.|
|Applies to an instance only if someone specifies the security group when launching the instance, or associates the security group with the instance later on||Automatically applies to all instances in the subnets it’s associated with (backup layer of defense, so you don’t have to rely on someone specifying the security group)|
Custom VPC comes with a default security group. Security group won’t span in VPC e.g. created a security group for other services. Any instance not associated with another security group during launch is associated with the default security group e.g. private subnet with a default security group and create a custom security group for a public subnet. The following is an example security group. The outbound rule is a default rule that allows all outbound communication to anywhere — you do not need to add this rule yourself.
|Source IP||Protocol||Port Range||Comments|
|0.0.0.0/0||TCP||80||Allows inbound HTTP access from any IPv4 address.|
|0.0.0.0/0||TCP||443||Allows inbound HTTPS access from any IPv4 address.|
|Public IPv4 address range of your home network||TCP||22||Allows inbound SSH access from your home network to a Linux/UNIX instance.|
|Public IPv4 address range of your home network||TCP||3389||Allows inbound RDP access from your home network to a Windows instance.|
|Destination IP||Protocol||Port Range||Comments|
|0.0.0.0/0||All||All||The default outbound rule that allows all outbound IPv4 communication.|
Custom VPC automatically comes with a modifiable default network ACL. By default, it allows all inbound and outbound IPv4 traffic and, if applicable, IPv6 traffic. The following is an example default network ACL for a VPC that supports IPv4 only.
|Rule #||Type||Protocol||Port Range||Source||Allow/Deny|
|100||All IPv4 traffic||All||All||0.0.0.0/0||ALLOW|
|*||All IPv4 traffic||All||All||0.0.0.0/0||DENY|
|Rule #||Type||Protocol||Port Range||Destination||Allow/Deny|
|100||All IPv4 traffic||all||all||0.0.0.0/0||ALLOW|
|*||All IPv4 traffic||all||all||0.0.0.0/0||DENY|
You can create a custom network ACL (e.g. Click Create Network ACL to create Custom NACL in the diagram) and associate it with a subnet. By default, each custom network ACL denies all inbound and outbound traffic until you add rules. If you don’t explicitly associate a subnet with a network ACL, the subnet is automatically associated with the default network ACL.
The following sample NACLs need to be manually added into Custom NACL:
|Rule #||Type||Protocol||Port Range||Source||Allow/Deny||Comments|
|100||HTTP||TCP||80||0.0.0.0/0||ALLOW||Allows inbound HTTP traffic from any IPv4 address.|
|110||HTTPS||TCP||443||0.0.0.0/0||ALLOW||Allows inbound HTTPS traffic from any IPv4 address.|
|120||SSH||TCP||22||192.0.2.0/24||ALLOW||Allows inbound SSH traffic from your home network’s public IPv4 address range (over the Internet gateway).|
|130||RDP||TCP||3389||192.0.2.0/24||ALLOW||Allows inbound RDP traffic to the web servers from your home network’s public IPv4 address range (over the Internet gateway).|
|140||Custom TCP||TCP||32768-65535||0.0.0.0/0||ALLOW||Allows inbound return IPv4 traffic from the Internet (that is, for requests that originate in the subnet).
This range is an example only. For more information about how to select the appropriate ephemeral port range, see Ephemeral Ports.
|*||All traffic||All||All||0.0.0.0/0||DENY||Denies all inbound IPv4 traffic not already handled by a preceding rule (not modifiable).|
|Rule #||Type||Protocol||Port Range||Destination||Allow/Deny||Comments|
|100||HTTP||TCP||80||0.0.0.0/0||ALLOW||Allows outbound IPv4 HTTP traffic from the subnet to the Internet.|
|110||HTTPS||TCP||443||0.0.0.0/0||ALLOW||Allows outbound IPv4 HTTPS traffic from the subnet to the Internet.|
|120||Custom TCP||TCP||32768-65535||0.0.0.0/0||ALLOW||Allows outbound IPv4 responses to clients on the Internet (for example, serving web pages to people visiting the web servers in the subnet).
This range is an example only. For more information about how to select the appropriate ephemeral port range, see Ephemeral Ports.
|*||All traffic||All||All||0.0.0.0/0||DENY||Denies all outbound IPv4 traffic not already handled by a preceding rule (not modifiable).|
Amazon VPC provides customers with several options for connecting their AWS virtual networks with other remote networks. Three categories of connectivity options are mentioned in Amazon Virtual Private Cloud Connectivity Options whitepaper: Network-to-Amazon VPC Connectivity Options, Amazon VPC-to-Amazon VPC Connectivity Options, and Internal User-to-Amazon VPC Connectivity Options. I think the AWS Direct Connect Plus VPN solution is a good choice for AWS VPC with the customer’s on-premise environment. It establishes a private, encrypted connection from your remote network to Amazon VPC with leveraging AWS Direct Connect. This combination provides an IPsec-encrypted private connection that also reduces network costs, increases bandwidth throughput, and provides a more consistent network experience than internet-based VPN connections.
VPC Peering is a new concept for us. A VPC peering connection is a networking connection between two VPCs that enables routing using each VPC’s private IP addresses as if they were in the same network. This is the AWS recommended method for connecting VPCs. VPC peering connections can be created between your own VPCs or with a VPC in another AWS account. Ensure that the address space does not overlap. In the VPC navigation pane, choose Peering Connections, Create Peering Connection to create VPC Peering.
Elastic Load Balancing makes it easy to create an internet-facing entry point into your VPC or to route request traffic between tiers of your application within your VPC. You can assign security groups to your load balancer to control which ports are open to a list of allowed sources. Because Elastic Load Balancing is integrated with your VPC, all of your existing Network Access Control Lists (ACLs) and Routing Tables continue to provide additional network controls.
After we created custom VPC, add EC2 instances for each subnet. In step 3 (as below diagram), select custom VPC and subnet. In step 6, select custom Security Group.
Then go to EC2 navigation pane, select Elastic Load Blancer. Select custom VPC, select two Availability Zones.
When you create a load balancer in your VPC, you can specify whether the load balancer is internet-facing (default) or internal in the Scheme. After you are done ELB, go to Route 53 setup this ELB as alias target.
After we are done the custom VPC, we should clean it up to avoid any cost. Since custom VPC is created from CloudFromation, you can go to CloudFormation select the stack. Then click Delete Stack. It will take some time to remove the VPC.
If you have any issue to remove VPC, for example, I created EC2 and ELB instance which were not in CloudFormation. You can’t remove VPC if you have associated EC2 is running. In this case, go to EC2 console, select the EC2 instance, choose Stop action. After it stopped, then select Terminate action. Now go back to VPC console, select the custom VPC, then choose Delete VPC action.
You can see in the above diagram, it deleted objects (e.g. Subnets, Security Groups, Internet Gateways etc.) associated with this VPC in this region but not Key Pair. Because Key Pair is on the region level. If you created one Key Pair for other services, you don’t need to worry to lose the Key Pair during VPC clean up.
- Hands-on with DynamoDB
- AWS Data Warehouse – Build with Redshift and QuickSight
- AWS Relational Database Solution: Hands-on with AWS RDS
- Which is Right Hadoop Solution for You?
- Apache Hadoop Ecosystem Cheat Sheet
- Data Storage for Big Data: Aurora, Redshift or Hadoop?
- AWS Kinesis Data Streams vs. Kinesis Data Firehose
- Streaming Platforms: Apache Kafka vs. AWS Kinesis
- AWS Machine Learning on AWS Redshift Data
- Why Use AWS Redshift Spectrum with Data Lake
- How to Design AWS DynamoDB Data Modeling
- When Should Use Amazon DynamoDB Accelerator (AWS DAX)?
- Web Application with Aurora Serverless Cluster
- Top IT Certifications for 2018
- How I Passed AWS CSAA in 3 Months
- How to Pass AWS Certified Big Data Specialty
- AWS Elastic Beanstalk or AWS Elastic Container Service for Kubernetes (AWS EKS)
- How to Use AWS CodeStar to Manage Lambda Java Project from Source to Test Locally