I tried out the new AWS PrivateLink feature "Tunnel Endpoints"
This page has been translated by machine translation. View original
Introduction
On September 18, 2026, tunnel endpoints were added to AWS PrivateLink.
This time, assuming a use case of accessing many instances in another account from a monitoring or file distribution infrastructure, we shared a CIDR-type Resource Configuration between 2 accounts in different Organizations using AWS RAM, and attempted access via a tunnel endpoint.
What is a Tunnel Endpoint?
According to the official documentation, it is an endpoint that privately tunnels to a shared network segment (a list of CIDR ranges) from another VPC or account. On the source side, a mechanism running a TUN device and a GENEVE relay is set up to access IP addresses within the shared CIDR range. The relay can run on the source EC2 itself or be placed as a dedicated relay EC2.
A Resource Gateway and a CIDR-type Resource Configuration are created on the destination side, and a tunnel endpoint is created on the source side. Packets from the source reach the destination via the following path:
Source EC2 → TUN device → GENEVE relay → Tunnel Endpoint ENI → Resource Gateway → Destination EC2
The destination port for GENEVE packets is UDP 6081, the VNI is fixed at 0, and the inner packet is an L3 IP packet.
Note that connections can only be initiated from the source side. The destination side cannot initiate new connections to the source side; only responses to communications from the source are returned.
Verification Configuration
| Account | Resource | Value |
|---|---|---|
| Destination (Account A) | VPC | 10.0.0.0/16 |
| Destination (Account A) | Resource Gateway subnet | 10.0.2.0/24 (apne1-az4), 10.0.3.0/24 (apne1-az1) |
| Destination (Account A) | EC2 (EC2-A) | 10.0.0.152, python3 -m http.server 8080, Amazon Linux 2023 |
| Source (Account B) | VPC | 10.1.0.0/16 |
| Source (Account B) | Tunnel endpoint subnet | 10.1.2.0/24 (apne1-az4), 10.1.3.0/24 (apne1-az1) |
| Source (Account B) | EC2 | 10.1.0.200, Amazon Linux 2023 |
Destination Side Setup (Account A)
The Resource Gateway was created in two private subnets of the destination VPC. The AZs where the Resource Gateway is placed become the AZs where the tunnel endpoint can be placed on the source side.
Resource Gateway creation command
aws vpc-lattice create-resource-gateway \
--region ap-northeast-1 \
--name <name> \
--vpc-identifier <vpc-id> \
--subnet-ids <dst-prv-az4-id> <dst-prv-az1-id> \
--security-group-ids <rgw-sg-id>
The range to share is specified with a CIDR-type Resource Configuration.
aws vpc-lattice create-resource-configuration \
--region ap-northeast-1 \
--name <name> \
--type CIDR \
--resource-gateway-identifier <rgw-id> \
--protocol TCP_UDP \
--port-ranges "1-65535" \
--resource-configuration-definition \
"cidrResource={cidrRanges=[\"<CIDR>\"]}"
This time, --protocol was created with TCP_UDP. Although application communication is TCP only, DNS queries flow as inner UDP, so TCP_UDP is specified to allow this traffic.
For connectivity verification, --port-ranges of the Resource Configuration was set to 1-65535, and all traffic was also allowed in the Resource Gateway's SG. The --port-ranges of the Resource Configuration and the Resource Gateway's SG are controls at separate layers, and in production it is recommended to limit both to only the necessary ports.
To use the Resource Configuration in another account, share it via AWS RAM. When the source account and destination account belong to different AWS Organizations, as in this case, you need to allow external principals when creating the share, and the source account needs to accept the invitation.
# Destination side: Create RAM share
aws ram create-resource-share \
--region ap-northeast-1 \
--name <share-name> \
--resource-arns <rcfg-arn> \
--principals <src-account-id> \
--allow-external-principals
# Source side: Accept invitation
aws ram accept-resource-share-invitation \
--region ap-northeast-1 \
--resource-share-invitation-arn <invitation-arn>
Security Group Design
In this configuration, the outer GENEVE communication and the inner application communication are each controlled by different SGs. SGs apply to communication between ENIs on the VPC; the processing by the TUN device and GENEVE relay inside the source EC2 is completed within the host and is therefore not subject to SGs.
[Inside source EC2 host] ← Outside SG scope
App → tun0 → GENEVE relay
↓ (sent out via UDP 6081)
────────────────────────────────
[Communication on VPC] ← SG applies from here
Outer: GENEVE (UDP 6081)
Source EC2 ENI [SG: outbound UDP 6081]
→ Tunnel Endpoint ENI [SG: inbound UDP 6081]
Inner: Application communication (e.g., TCP 8080)
Resource Gateway ENI [SG: outbound TCP 8080]
→ Destination EC2 ENI [SG: inbound TCP 8080]
The Resource Gateway's SG is stateful and has rules in both directions, but the rule that applies in this path is the outbound side (Resource Gateway → destination resource). The earlier "allow all traffic" means that we did not restrict this outbound traffic either.
From the destination EC2's perspective, the source of traffic is the IP of the Resource Gateway's ENI (10.0.2.85), not the source EC2 (10.1.0.200) in the source account.
The destination EC2's SG can specify the SG associated with the Resource Gateway as the source. If not using SG references, you can specify the CIDR of the subnet where the Resource Gateway is placed (in this case, 10.0.2.0/24 and 10.0.3.0/24).
Source Side Setup (Account B)
Once the RAM invitation is accepted, the shared Resource Configuration becomes visible in the source account. The tunnel endpoint is created as a Tunnel type VPC endpoint, and the ARN of the Resource Configuration created in the destination account is specified in --resource-configuration-arn.
aws ec2 create-vpc-endpoint \
--vpc-endpoint-type Tunnel \
--vpc-id <vpc-id> \
--subnet-ids <src-prv-az4-id> <src-prv-az1-id> \
--security-group-ids <tep-sg-id> \
--resource-configuration-arn arn:aws:vpc-lattice:<region>:<dst-account-id>:resourceconfiguration/<rcfg-id>
The specified subnets must be placed in the same AZ as the subnets where the Resource Gateway is placed. Since the mapping between AZ names and physical AZs differs per account, use AZ IDs (in this case, apne1-az4 / apne1-az1) to match them across accounts.
The tunnel endpoint's SG needs a rule allowing inbound UDP 6081 from the source EC2's CIDR. In this case, UDP 6081 from 10.1.0.0/16 was allowed.
GENEVE Relay
On the source EC2, a TUN device and a user-space relay were run. The relay is given the primary IP of the source EC2 (10.1.0.200 in this case) as the source IP. The relay's roles are as follows:
- Create
tun0 - Route traffic to the destination VPC's
10.0.0.0/16and the DNS resolver's169.254.168.253/32throughtun0 - Add an 8-byte GENEVE header (VNI=0, Protocol Type
0x0800) to L3 packets read fromtun0and send them via UDP 6081 to the tunnel endpoint's ENI - Strip the GENEVE header from packets received via UDP 6081 and write them back to
tun0
Since the relay also writes return packets back to tun0, the kernel's TCP stack can be used as-is. No changes are needed on the application side, and curl or dig can be run directly.
The L2 overlay device created with Linux's ip link add type geneve uses Protocol Type 0x6558 (Transparent Ethernet Bridging) and carries Ethernet frames as payload. What the tunnel endpoint expects is 0x0800 (IPv4), which carries IP packets directly as payload, so it cannot be used as-is. In practice, ip link add type geneve did not establish connectivity, and HTTP 200 was only confirmed with a custom relay specifying 0x0800.
geneve_tun.py (full GENEVE relay script)
#!/usr/bin/env python3
"""
TUN device + GENEVE relay for AWS PrivateLink Tunnel Endpoint (CIDR type).
- Reads L3 IP packets from tun0, wraps in GENEVE (VNI=0, proto=0x0800), sends UDP to tunnel EP
- Receives UDP from tunnel EP, strips GENEVE header, writes inner IP to tun0
Usage (run as root):
python3 geneve_tun.py <tunnel_ep_ip> <local_src_ip>
local_src_ip: source IP of this EC2 (its primary private IP, e.g. 10.1.0.200)
Example:
python3 geneve_tun.py 10.1.2.24 10.1.0.200
"""
import sys, os, struct, socket, fcntl, select
TUNNEL_PORT = 6081
GENEVE_HDR = struct.pack('!BBH', 0, 0, 0x0800) + b'\x00\x00\x00\x00' # 8 bytes, VNI=0
# Linux TUN/TAP constants
TUNSETIFF = 0x400454ca
IFF_TUN = 0x0001
IFF_NO_PI = 0x1000
def open_tun(name='tun0'):
tun = open('/dev/net/tun', 'r+b', buffering=0)
ifr = struct.pack('16sH', name.encode(), IFF_TUN | IFF_NO_PI)
fcntl.ioctl(tun, TUNSETIFF, ifr)
return tun
def setup_route(tun_name, src_ip):
os.system(f'ip link set {tun_name} up')
os.system(f'ip route add 10.0.0.0/16 dev {tun_name} src {src_ip}')
os.system(f'ip route add 169.254.168.253/32 dev {tun_name} src {src_ip}')
print(f'Routes added: 10.0.0.0/16 and 169.254.168.253/32 -> {tun_name}')
def teardown_route(tun_name):
os.system(f'ip route del 10.0.0.0/16 dev {tun_name} 2>/dev/null')
os.system(f'ip route del 169.254.168.253/32 dev {tun_name} 2>/dev/null')
os.system(f'ip link set {tun_name} down 2>/dev/null')
def main():
if len(sys.argv) < 3:
print(f'Usage: {sys.argv[0]} <tunnel_ep_ip> <local_src_ip>')
sys.exit(1)
tunnel_ep = sys.argv[1]
src_ip = sys.argv[2]
tun_name = 'tun0'
print(f'Tunnel EP: {tunnel_ep}:{TUNNEL_PORT}')
print(f'Source IP: {src_ip}')
tun = open_tun(tun_name)
print(f'Opened {tun_name}')
setup_route(tun_name, src_ip)
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp.bind(('0.0.0.0', TUNNEL_PORT))
print(f'Listening on UDP :{TUNNEL_PORT}')
print('Relay running. Ctrl+C to stop.')
try:
while True:
rlist, _, _ = select.select([tun, udp], [], [], 1.0)
for fd in rlist:
if fd is tun:
pkt = tun.read(65536)
if pkt:
udp.sendto(GENEVE_HDR + pkt, (tunnel_ep, TUNNEL_PORT))
elif fd is udp:
data, addr = udp.recvfrom(65536)
if len(data) >= 8:
inner = data[8:] # strip 8-byte GENEVE header
tun.write(inner)
except KeyboardInterrupt:
print('\nStopping relay...')
finally:
teardown_route(tun_name)
tun.close()
udp.close()
if __name__ == '__main__':
main()
Connectivity Verification
curl -v was executed from the source EC2 to the destination EC2.
curl -v http://10.0.0.152:8080/
< HTTP/1.0 200 OK
< Server: SimpleHTTP/0.6 Python/3.9.25
The following is tcpdump output captured simultaneously on the destination EC2.
04:19:40 ens5 In IP 10.0.2.85.12907 > 10.0.0.152.8080: Flags [S] (SYN)
04:19:40 ens5 Out IP 10.0.0.152.8080 > 10.0.2.85.12907: Flags [P.] length 157: HTTP/1.0 200 OK
The source is the IP of the Resource Gateway's ENI (10.0.2.85), not the source EC2 (10.1.0.200) in the originating account.
Name Resolution Using the Destination VPC's Resolver
A Route 53 private hosted zone internal.pl-test.local was created in the destination VPC and associated with it, and an A record for provider-ec2.internal.pl-test.local pointing to 10.0.0.152 was registered. The following is the result of querying the destination VPC's resolver from the source EC2.
$ dig +short @169.254.168.253 provider-ec2.internal.pl-test.local A
10.0.0.152
When querying the destination VPC's DNS resolver via the tunnel endpoint, specify 169.254.168.253 as the name server. This is a dedicated address defined in the official documentation and is different from the standard VPC DNS resolver at 169.254.169.253.
DNS queries are also sent encapsulated in GENEVE. To allow this inner UDP traffic, the Resource Configuration was created with TCP_UDP as described earlier.
CIDR Range Verification
Tunnel endpoints with 10.0.0.0/16 and 10.0.0.0/24 respectively specified in the Resource Configuration were prepared, and curl was executed against 10.0.0.152 and 10.0.1.218. The only difference between the two is the CIDR specified in the Resource Configuration.
curl -s -o /dev/null -w '%{http_code}' --max-time 5 http://<host>:8080/
/16 EP → 10.0.0.152:8080 (within CIDR) 200 ✅ Connected as expected
/16 EP → 10.0.1.218:8080 (within CIDR, control) 200 ✅ Connected as expected
/24 EP → 10.0.0.152:8080 (within CIDR) 200 ✅ Connected as expected
/24 EP → 10.0.1.218:8080 (outside CIDR) 000 ✅ Connection blocked as expected (timeout)
✅ indicates the result was as expected (HTTP code 000 is the value returned by curl on timeout, meaning connection to an address outside the CIDR was blocked here). 10.0.1.218 is a control host that falls within /16 but outside /24. This confirms that the same destination either connects or does not connect depending solely on the specified CIDR.
HTTP status code 200 was confirmed only for addresses within the CIDR range specified in the Resource Configuration.
Limitations and Notes
CIDR-type Resource Configuration is exclusive to tunnel endpoints. It cannot be accessed from resource endpoints, nor can it be associated with a service network.
Private DNS names for the tunnel endpoint itself are not supported. As described earlier, it is possible to query the destination VPC's resolver for name resolution, but this is separate from the Private DNS feature for the endpoint itself (the feature that assigns a dedicated DNS name as with interface endpoints); the latter is not provided.
Pricing
Hourly charges and data processing charges for the tunnel endpoint are incurred by the source account. The destination account incurs data processing charges for the Resource Gateway. In addition, if a GENEVE relay is run on an EC2 or similar instance on the source side, the compute costs for that EC2 and monitoring costs (e.g., for CloudWatch) are also borne by the source. For unit prices and the latest pricing structure, refer to the AWS PrivateLink pricing page.
Summary
Tunnel endpoints have been added to AWS PrivateLink. For cross-account monitoring or file distribution, by placing a Resource Gateway in the VPC where the target resources reside on the destination side, and configuring a TUN device and GENEVE relay on the source EC2, access to destinations permitted at the CIDR level becomes possible. Even as monitored targets dynamically increase or decrease, they can be accessed without individual registration as long as they fall within the CIDR range.
With traditional PrivateLink, an NLB is placed on the destination side to expose services. With tunnel endpoints, on the other hand, a GENEVE encapsulation/decapsulation mechanism is required on the source side. Since the relay must either be distributed to each source or operated as a shared relay with availability and routing considerations, it is recommended to also evaluate the network implementation and operational burden on the source side when adopting this approach.
