flannel-migration: flush leftover Flannel iptables chains on node cleanup

[stitrace]

Description

When the flannel-migration controller migrates a node to Calico it:

• removes the Flannel daemonset, and
• deletes the Flannel network devices on the node (flannel.<vni>, and cni0 for non-Canal flannel) via the per-node remove-flannel pod in removeFlannelNetworkAndInstallDummyCalicoCNI.

It does not remove the iptables chains Flannel installs: FLANNEL-POSTRTG (nat) and FLANNEL-FWD (filter). These are programmed by flanneld at runtime and are not cleaned up when the daemonset/devices are removed, so they persist after an otherwise successful migration.

Why it matters

FLANNEL-POSTRTG contains:

ip saddr <podCIDR> ip daddr != 224.0.0.0/4  masquerade

with RETURN exceptions only for the node-local subnet. After migration this rule keeps masquerading cross-node pod-to-pod traffic to the source node's VXLAN tunnel IP (<block>.0).

This is invisible until a NetworkPolicy is applied. Once it is, the SNAT'd source address no longer matches pod-selector / same-namespace rules, so Felix's default-deny drops the traffic. Symptom: after a clean migration, cross-node connections to any pod covered by a NetworkPolicy silently time out, while same-node traffic keeps working (Flannel's own RETURN rules exempt the node-local subnet). Calico's own cali-nat-outgoing correctly does not masquerade pod-to-pod, so it's purely the leftover Flannel chain.

Reproduction

1. Migrate a flannel cluster to Calico with this controller.
2. nft list chain ip nat POSTROUTING on any node still shows jump FLANNEL-POSTRTG (non-zero counters).
3. Apply a NetworkPolicy selecting some pods.
4. Cross-node traffic to those pods times out; a listener sees the source as the sender node's <block>.0 tunnel IP instead of the client pod IP.

Fix

Flush FLANNEL-POSTRTG and FLANNEL-FWD as part of the existing per-node remove-flannel step, so Calico becomes the sole owner of pod masquerading. The cleanup:

• runs in the same privileged calico/node pod that already deletes the devices;
• cleans both the legacy and nft iptables backends (either may hold the rules), skipping missing binaries;
• is fully idempotent (safe if the chains are already gone).

Testing

• Manually verified on a live migrated cluster that FLANNEL-POSTRTG persists post-migration, SNATs cross-node pod traffic to the tunnel IP, and breaks NetworkPolicy; removing the chains restores cross-node policy traffic with no other changes.
• gofmt-clean. I was not able to run the flannel-migration FV suite locally; happy to iterate.

Release Note

flannel-migration: remove leftover Flannel iptables chains (FLANNEL-POSTRTG/FLANNEL-FWD) during node migration so they no longer masquerade cross-node pod traffic and break NetworkPolicy after migrating to Calico.

[CLAassistant]

All committers have signed the CLA.

[ctauchen]

@projectcalico/core-maintainers Would you have a look also at the corresponding doc ticket? tigera/docs#2768

[caseydavenport]

Sorry for the long delay on this - it LGTM

I'd generally want to see an FV test of some sort, but it seems we don't really have a good framework in place to add that to at the moment so we'll defer that for now!

[caseydavenport]

/sem-approve