HashiCorp Vault contains every secret in your infrastructure. Database passwords, API keys, certificates - everything. Exposing it to the internet, even with authentication, creates an attack surface we couldn't accept. But here's the thing: engineers need access. They need to configure policies, debug issues, and manage secrets. The challenge is to provide secure access without any public endpoints. We built a solution that keeps Vault completely private while staying accessible to authorized users. This is no rocket science, just three practical layers of defense that actually work.

Series overview:

‍

‍

Layer 1: Internal Network Load Balancer

The first layer ensures Vault never gets a public IP address. We expose Vault through an AWS Network Load Balancer configured as internal-only:

annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"

‍

Why NLB Instead of ALB?

‍

Vault uses TCP traffic on port 8200. Application Load Balancers (ALB) work at Layer 7 (HTTP/HTTPS) and would require additional configuration and overhead. Network Load Balancers operate at Layer 4 (TCP/UDP) - perfect for Vault's Kubernetes integration needs.

‍

Health Check Configuration

‍

The NLB has one listener on port 8200, linked to a target group containing the IPs of the Vault pods. For health checks, we use Vault's /v1/sys/health endpoint. Important detail: In high-availability setups, standby nodes return HTTP 429 (Too Many Requests) on the health endpoint. We configure the target group to consider 429 as healthy - otherwise, only the active node would be in the target group, defeating the purpose of HA.

‍

Layer 2: Private DNS with Route 53

‍

AWS generates DNS names for load balancers like: vault-nlb-internal-1234567890.elb.eu-west-1.amazonaws.com Not exactly user-friendly. And if the load balancer gets recreated, the DNS changes. We solved this with Route 53 private hosted zones:

‍

‍

Now everyone uses https://vault.tooling.internal:8200. Clean, consistent, and it works across kubernetes clusters.

‍

Layer 3: VPN Access for Engineers

With Vault completely private, how do engineers access it from their workstations? AWS Client VPN provides the secure tunnel we need for Kubernetes vault integration.

‍

Certificate-Based Authentication

‍

We chose certificate authentication over Active Directory for simplicity:

‍

‍

The setup uses a simple PKI hierarchy:

‍

‍

Client Certificate Generation

‍

We created a generate-client.sh script that:

Engineers run the script, get their personal VPN config, and import it into their OpenVPN client. Connection established, Vault accessible.

‍

Vault vs AWS Native Solutions

Many teams ask about AWS Secrets Manager or AWS KMS for kubernetes. We chose Vault because:

‍

‍

The Complete Security Picture

This three-layer approach means:

‍

‍

Applications in the Kubernetes cluster access Vault directly through the internal load balancer. Engineers access it through VPN. Nobody accesses it from the public internet. Get the code: Our Vault configuration on GitHub

‍

That's a wrap! You've now seen our complete HashiCorp Vault production architecture - from multi-cluster setup to dynamic credentials to zero-trust access. Ready to implement this in your environment?

‍

FAQ

Q: How does this Vault integration work with Kubernetes?

A: The internal NLB allows pods to access Vault using kubernetes service accounts for authentication, while engineers use VPN for external access.

Q: Can I use AWS external secrets with this setup?

A: Yes, External Secrets Operator works perfectly with this architecture. See part 2 of our series for detailed implementation.

Q: Why not use AWS Secrets Manager for kubernetes secrets?

A: Vault offers superior kubernetes integration, dynamic credentials, and works across cloud providers. It also provides better audit logs and more flexible policies.

Q: Why use a Network Load Balancer instead of an Application Load Balancer for Vault?

A: Vault communicates over TCP on port 8200, noat HTTP/HTTPS. NLBs operate at Layer 4 (TCP/UDP) making them perfect for Vault's TCP traffic. ALBs work at Layer 7 (HTTP/HTTPS) and would add unnecessary overhead.

Q: What happens if the internal NLB gets recreated?

A: That's why we use Route 53 private DNS. The CNAME record vault.tooling.internal always points to the current NLB DNS name. If the NLB changes, we only update one DNS record - all configurations keep working.

Q: Is VPN the only way to access Vault externally?

A: For our zero-trust approach, yes. Alternatives like bastion hosts or transit gateways exist, but VPN with certificate-based authentication provides the best balance of security and manageability.

Q: Why mark HTTP 429 as healthy in the health checks?

A: In Vault's high-availability setup, standby nodes return 429 (Too Many Requests) on health checks. They're actually healthy and ready to take over if needed. Only accepting 200 would remove standby nodes from the pool.

Q: Can this setup work with multiple AWS accounts?

A: Yes! You can share the Route 53 private hosted zone across accounts and VPCs. The VPN can also be configured to access resources in multiple accounts through proper networking setup.

‍

‍