I tried out the new AWS PrivateLink feature "Tunnel Endpoints"
This page has been translated by machine translation. View original
Introduction
On September 18, 2026, tunnel endpoints were added to AWS PrivateLink.
In this article, assuming a use case where monitoring or file distribution infrastructure needs to access a large number of instances in another account, we shared a CIDR-type Resource Configuration via AWS RAM between 2 accounts belonging to different Organizations, and attempted access via a tunnel endpoint.
What is a Tunnel Endpoint?
According to the official documentation, it is an endpoint that privately tunnels to a shared network segment (a list of CIDR ranges) from another VPC or account. On the source side, a mechanism is set up to run a TUN device and a GENEVE relay, which accesses IP addresses within the shared CIDR range. The relay can be run on the source EC2 itself, or placed as a dedicated relay EC2.
On the destination side, you create a Resource Gateway and a CIDR-type Resource Configuration, and on the source side, you create a tunnel endpoint. Packets from the source reach the destination via the following path:
Source EC2 → TUN device → GENEVE relay → Tunnel Endpoint ENI → Resource Gateway → Destination EC2
The destination port for GENEVE packets is UDP 6081, the VNI is fixed at 0, and the inner packet is an L3 IP packet.
Note that connections can only be initiated from the source side. The destination side cannot initiate new connections to the source side; only responses to communications from the source are returned.
Verification Configuration
| Account | Resource | Value |
|---|---|---|
| Destination (Account A) | VPC | 10.0.0.0/16 |
| Destination (Account A) | Resource Gateway subnet | 10.0.2.0/24 (apne1-az4), 10.0.3.0/24 (apne1-az1) |
| Destination (Account A) | EC2 (EC2-A) | 10.0.0.152, python3 -m http.server 8080, Amazon Linux 2023 |
| Source (Account B) | VPC | 10.1.0.0/16 |
| Source (Account B) | Tunnel endpoint subnet | 10.1.2.0/24 (apne1-az4), 10.1.3.0/24 (apne1-az1) |
| Source (Account B) | EC2 | 10.1.0.200, Amazon Linux 2023 |
Destination Side Setup (Account A)
The Resource Gateway was created in two private subnets of the destination VPC. The AZs where the Resource Gateway is placed become the AZs where the source side can place tunnel endpoints.
Resource Gateway creation command
aws vpc-lattice create-resource-gateway \
--region ap-northeast-1 \
--name <name> \
--vpc-identifier <vpc-id> \
--subnet-ids <dst-prv-az4-id> <dst-prv-az1-id> \
--security-group-ids <rgw-sg-id>
The range to share is specified in a CIDR-type Resource Configuration.
aws vpc-lattice create-resource-configuration \
--region ap-northeast-1 \
--name <name> \
--type CIDR \
--resource-gateway-identifier <rgw-id> \
--protocol TCP_UDP \
--port-ranges "1-65535" \
--resource-configuration-definition \
"cidrResource={cidrRanges=[\"<CIDR>\"]}"
In this case, --protocol was created with TCP_UDP. Although application communication is TCP only, DNS queries flow as inner UDP, so TCP_UDP is specified to allow them through.
For connectivity verification, --port-ranges in the Resource Configuration was set to 1-65535, and the Resource Gateway's SG also allowed all traffic. The --port-ranges of the Resource Configuration and the Resource Gateway's SG are controls at separate layers, and in production use, it is recommended to restrict both to only the necessary ports.
To use the Resource Configuration in another account, share it via AWS RAM. In cases like this one, where the source account and destination account belong to different AWS Organizations, you need to allow external principals when creating the share, and the source account must accept the invitation.
# Destination side: Create RAM share
aws ram create-resource-share \
--region ap-northeast-1 \
--name <share-name> \
--resource-arns <rcfg-arn> \
--principals <src-account-id> \
--allow-external-principals
# Source side: Accept invitation
aws ram accept-resource-share-invitation \
--region ap-northeast-1 \
--resource-share-invitation-arn <invitation-arn>
Security Group Design
In this configuration, the outer GENEVE communication and the inner application communication are each controlled by separate SGs. SGs apply to communication between ENIs on the VPC; the processing by the TUN device and GENEVE relay inside the source EC2 is completed within the host and is therefore not subject to SGs.
[Inside source EC2 host] ← Outside SG scope
App → tun0 → GENEVE relay
↓ (sent via UDP 6081)
────────────────────────────────
[Communication on VPC] ← SG applies from here
Outer: GENEVE (UDP 6081)
Source EC2 ENI [SG: outbound UDP 6081]
→ Tunnel Endpoint ENI [SG: inbound UDP 6081]
Inner: Application communication (e.g., TCP 8080)
Resource Gateway ENI [SG: outbound TCP 8080]
→ Destination EC2 ENI [SG: inbound TCP 8080]
The Resource Gateway's SG is stateful and has rules for both directions, but in this path, the outbound rule (Resource Gateway → destination resource) is the one that applies. When we said "allow all traffic" earlier, it meant we did not restrict the outbound side either.
From the destination EC2's perspective, the source of traffic is the IP of the Resource Gateway's ENI (10.0.2.85), not the source EC2 (10.1.0.200).
The Resource Gateway has an ENI for each subnet (AZ) it is placed in, so if you allow only a specific IP as the source, connections that use a different IP may fail even if some connections succeed.
You can specify the SG associated with the Resource Gateway as the source in the destination EC2's SG. If not using SG references, you can specify the CIDR of the subnets where the Resource Gateway is placed (in this case, 10.0.2.0/24 and 10.0.3.0/24).
Source Side Setup (Account B)
Once the RAM invitation is accepted, the shared Resource Configuration becomes visible in the source account. The tunnel endpoint is created as a Tunnel type VPC endpoint, specifying the ARN of the Resource Configuration created in the destination account in --resource-configuration-arn.
aws ec2 create-vpc-endpoint \
--vpc-endpoint-type Tunnel \
--vpc-id <vpc-id> \
--subnet-ids <src-prv-az4-id> <src-prv-az1-id> \
--security-group-ids <tep-sg-id> \
--resource-configuration-arn arn:aws:vpc-lattice:<region>:<dst-account-id>:resourceconfiguration/<rcfg-id>
The subnets specified must be in the same AZ as the subnets where the Resource Gateway is placed. Since the mapping between AZ names and physical AZs differs per account, use AZ IDs (in this case, apne1-az4 / apne1-az1) for cross-account matching.
The tunnel endpoint's SG requires a rule to allow inbound UDP 6081 from the source EC2's CIDR. In this case, UDP 6081 from 10.1.0.0/16 was allowed.
GENEVE Relay
On the source EC2, a TUN device and a user-space relay were run. The relay is passed the primary IP of the source EC2 (in this case, 10.1.0.200) as the source IP. The roles of the relay are as follows:
- Create
tun0 - Route traffic to the destination VPC's
10.0.0.0/16and the DNS resolver's169.254.168.253/32viatun0 - Attach an 8-byte GENEVE header (VNI=0, Protocol Type
0x0800) to L3 packets read fromtun0, and send them via UDP 6081 to the tunnel endpoint's ENI - Strip the GENEVE header from packets received on UDP 6081, and write them back to
tun0
Since the relay writes return packets back to tun0, the kernel's TCP stack can be used as-is. No changes to the application side are needed, and curl or dig can be run directly.
The L2 overlay device created with Linux's ip link add type geneve uses Protocol Type 0x6558 (Transparent Ethernet Bridging) and carries Ethernet frames as payload. Since the tunnel endpoint expects 0x0800 (IPv4) with the IP packet directly as the payload, it cannot be used as-is. In actual testing, ip link add type geneve did not achieve connectivity, and HTTP 200 was only confirmed using a custom relay with 0x0800 specified.
geneve_tun.py (full GENEVE relay script)
#!/usr/bin/env python3
"""
TUN device + GENEVE relay for AWS PrivateLink Tunnel Endpoint (CIDR type).
- Reads L3 IP packets from tun0, wraps in GENEVE (VNI=0, proto=0x0800), sends UDP to tunnel EP
- Receives UDP from tunnel EP, strips GENEVE header, writes inner IP to tun0
Usage (run as root):
python3 geneve_tun.py <tunnel_ep_ip> <local_src_ip>
local_src_ip: source IP of this EC2 (its primary private IP, e.g. 10.1.0.200)
Example:
python3 geneve_tun.py 10.1.2.24 10.1.0.200
"""
import sys, os, struct, socket, fcntl, select
TUNNEL_PORT = 6081
GENEVE_HDR = struct.pack('!BBH', 0, 0, 0x0800) + b'\x00\x00\x00\x00' # 8 bytes, VNI=0
# Linux TUN/TAP constants
TUNSETIFF = 0x400454ca
IFF_TUN = 0x0001
IFF_NO_PI = 0x1000
def open_tun(name='tun0'):
tun = open('/dev/net/tun', 'r+b', buffering=0)
ifr = struct.pack('16sH', name.encode(), IFF_TUN | IFF_NO_PI)
fcntl.ioctl(tun, TUNSETIFF, ifr)
return tun
def setup_route(tun_name, src_ip):
os.system(f'ip link set {tun_name} up')
os.system(f'ip route add 10.0.0.0/16 dev {tun_name} src {src_ip}')
os.system(f'ip route add 169.254.168.253/32 dev {tun_name} src {src_ip}')
print(f'Routes added: 10.0.0.0/16 and 169.254.168.253/32 -> {tun_name}')
def teardown_route(tun_name):
os.system(f'ip route del 10.0.0.0/16 dev {tun_name} 2>/dev/null')
os.system(f'ip route del 169.254.168.253/32 dev {tun_name} 2>/dev/null')
os.system(f'ip link set {tun_name} down 2>/dev/null')
def main():
if len(sys.argv) < 3:
print(f'Usage: {sys.argv[0]} <tunnel_ep_ip> <local_src_ip>')
sys.exit(1)
tunnel_ep = sys.argv[1]
src_ip = sys.argv[2]
tun_name = 'tun0'
print(f'Tunnel EP: {tunnel_ep}:{TUNNEL_PORT}')
print(f'Source IP: {src_ip}')
tun = open_tun(tun_name)
print(f'Opened {tun_name}')
setup_route(tun_name, src_ip)
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp.bind(('0.0.0.0', TUNNEL_PORT))
print(f'Listening on UDP :{TUNNEL_PORT}')
print('Relay running. Ctrl+C to stop.')
try:
while True:
rlist, _, _ = select.select([tun, udp], [], [], 1.0)
for fd in rlist:
if fd is tun:
pkt = tun.read(65536)
if pkt:
udp.sendto(GENEVE_HDR + pkt, (tunnel_ep, TUNNEL_PORT))
elif fd is udp:
data, addr = udp.recvfrom(65536)
if len(data) >= 8:
inner = data[8:] # strip 8-byte GENEVE header
tun.write(inner)
except KeyboardInterrupt:
print('\nStopping relay...')
finally:
teardown_route(tun_name)
tun.close()
udp.close()
if __name__ == '__main__':
main()
Connectivity Verification
curl -v was executed from the source EC2 to the destination EC2.
curl -v http://10.0.0.152:8080/
< HTTP/1.0 200 OK
< Server: SimpleHTTP/0.6 Python/3.9.25
Below is the tcpdump output captured on the destination EC2 at the same time.
04:19:40 ens5 In IP 10.0.2.85.12907 > 10.0.0.152.8080: Flags [S] (SYN)
04:19:40 ens5 Out IP 10.0.0.152.8080 > 10.0.2.85.12907: Flags [P.] length 157: HTTP/1.0 200 OK
The source is the IP of the Resource Gateway's ENI (10.0.2.85), not the source EC2 that initiated the request (10.1.0.200).
Name Resolution via the Destination VPC's Resolver
A Route 53 private hosted zone internal.pl-test.local was created in the destination VPC and associated with it, and an A record for provider-ec2.internal.pl-test.local was registered with 10.0.0.152. Below is the result of querying the destination VPC's resolver from the source EC2.
$ dig +short @169.254.168.253 provider-ec2.internal.pl-test.local A
10.0.0.152
When querying the destination VPC's DNS resolver via the tunnel endpoint, specify 169.254.168.253 as the nameserver. This is a dedicated address defined in the official documentation, and differs from the standard VPC DNS resolver at 169.254.169.253.
DNS queries are also sent encapsulated in GENEVE. To pass this inner UDP, the Resource Configuration was created with TCP_UDP as described earlier.
CIDR Range Verification
Tunnel endpoints were prepared with 10.0.0.0/16 and 10.0.0.0/24 specified respectively in the Resource Configuration, and curl was executed against 10.0.0.152 and 10.0.1.218. The only difference between the two sets is the CIDR specified in the Resource Configuration.
curl -s -o /dev/null -w '%{http_code}' --max-time 5 http://<host>:8080/
/16 EP → 10.0.0.152:8080 (within CIDR) 200 ✅ Connected as expected
/16 EP → 10.0.1.218:8080 (within CIDR, control) 200 ✅ Connected as expected
/24 EP → 10.0.0.152:8080 (within CIDR) 200 ✅ Connected as expected
/24 EP → 10.0.1.218:8080 (outside CIDR) 000 ✅ Connection blocked as expected (timeout)
✅ indicates the result was as expected (HTTP code 000 is the value returned by curl on timeout, meaning connection to an address outside the CIDR was blocked here). 10.0.1.218 is a control host that falls within the /16 CIDR but outside the /24 CIDR. This confirms that the same destination can either connect or not connect based solely on the CIDR specified.
HTTP status code 200 was confirmed only for addresses within the CIDR range specified in the Resource Configuration.
Constraints and Notes
CIDR-type Resource Configuration is exclusive to tunnel endpoints. It cannot be accessed from resource endpoints, nor can it be associated with a service network.
Private DNS names for the tunnel endpoint itself are not supported. As described above, querying the destination VPC's resolver for name resolution is possible, but this is separate from the Private DNS feature on the endpoint itself (the feature that assigns a custom DNS name as with interface endpoints), and the latter is not provided.
Pricing
Hourly charges and data processing charges for the tunnel endpoint are incurred by the source account. The destination account incurs data processing charges for the Resource Gateway. Additionally, if a GENEVE relay is run on an EC2 or similar on the source side, the compute costs for that EC2, as well as monitoring costs if using CloudWatch or similar, are also borne by the source. For unit prices and the latest pricing structure, refer to the AWS PrivateLink pricing page.
Summary
Tunnel endpoints have been added to AWS PrivateLink. For cross-account monitoring or file distribution, the destination places a Resource Gateway in the VPC containing the target resources, and the source EC2 configures a TUN device and GENEVE relay, enabling access to destinations permitted at the CIDR level. Even if monitored targets increase or decrease dynamically, access is possible without individual registration as long as they fall within the CIDR range.
With traditional PrivateLink, an NLB is placed on the destination side to expose the service. With tunnel endpoints, on the other hand, a GENEVE encapsulation/decapsulation mechanism is required on the source side. Since the relay must either be distributed to each source or operated as a shared relay with availability and routing considerations, it is recommended to evaluate the network implementation and operational burden on the source side when adopting this approach.
Related Articles
