What is Antrea

Antrea stands out as a prominent CNI plugin tailored for Kubernetes. It streamlines Kubernetes networking across diverse clouds and operating systems, leveraging the capabilities of Open vSwitch. By integrating Kubernetes controller patterns, Antrea simplifies deployment, operations, and troubleshooting processes. Moreover, it provides diagnostics that are compatible with popular tools for network operators. Thanks to the flexibility of Open vSwitch, Antrea can accommodate advanced networking scenarios, such as kernel bypass and network service mesh.

Why I chose Antrea

For a comprehensive overview and detailed information, you can visit its official website. In this article, I’ll delve into my personal experience with Antrea and explain why I chose it.

Addressing Security Concerns with the Egress Feature

One feature of Antrea that particularly intrigued me was the Egress feature. It addressed several security concerns I had. Given that Kubernetes launches pods on numerous worker nodes, and these nodes are elastic (meaning they can be dynamically added or removed based on system load), managing firewall rules becomes a challenge. Here are some potential solutions:

1. Bind a pod to a specific worker node using an affinity rule.
2. Maintain a static number of worker nodes indefinitely.
3. Open the subnet of the worker nodes or the entire DHCP range they use.

While the first solution is viable if the worker node is stable and crash-proof, the second is ideal for consistent traffic. The third solution is generally effective, but it might raise eyebrows when explaining to a firewall auditor why an entire subnet needs to be open.

Comparison with Calico

Calico offers the EgressGW feature, but it’s restricted to the paid version, so I haven’t had the opportunity to test it. Antrea introduced this feature in its V1.0 release (currently at v1.14.x) and provides an option to set a static SNAT for the pods.

Configuration Guide

Setting up ExternalIPPool in Angrea

apiVersion: crd.antrea.io/v1alpha2
kind: ExternalIPPool
metadata:
  name: prod-external-ip-pool
spec:
  ipRanges:
  - start: 10.10.0.2
    end: 10.10.0.10
  - cidr: 10.10.1.0/28
  nodeSelector:
    matchLabels:
      network-role: egress-gateway

In the configuration above, we established an externalIPPool and selected nodes with the label network-role=egress-gateway. If you prefer not to specify a node, you can use an empty dictionary for nodeSelector ({}).

Creating Egress Resources

apiVersion: crd.antrea.io/v1alpha2
kind: Egress
metadata:
  name: egress-prod-web
spec:
  appliedTo:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: prod
    podSelector:
      matchLabels:
        app: web
  externalIPPool: external-ip-pool
---
apiVersion: crd.antrea.io/v1alpha2
kind: Egress
metadata:
  name: egress-staging-web
spec:
  appliedTo:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: staging
    podSelector:
      matchLabels:
        app: web
  externalIPPool: external-ip-pool

In the configurations above, we deployed two Egress resources for the ‘prod’ and ‘staging’ namespaces. These egress resources are then designated to their respective Egress Nodes.

# kubectl get egress
NAME                 EGRESSIP       AGE   NODE
egress-prod-web      10.10.0.11     1m    node-4
egress-staging-web   10.10.0.12     1m    node-6

Conclusion

EgressPool operates based on podSelector, offering flexibility in determining which pods will utilize the Egress. This feature simplifies firewall rule considerations.