Skip to content

Add opt-in network-ready taint support - #5072

Open
caseydavenport wants to merge 1 commit into
tigera:masterfrom
caseydavenport:casey-network-ready-taint
Open

Add opt-in network-ready taint support#5072
caseydavenport wants to merge 1 commit into
tigera:masterfrom
caseydavenport:casey-network-ready-taint

Conversation

@caseydavenport

Copy link
Copy Markdown
Member

Companion to projectcalico/calico#11985. Adds an opt-in feature that taints nodes until Calico networking is ready, so workloads aren't scheduled onto a node before Calico can service them.

When networkReadyTaint is set to Enabled on the Installation, the operator:

  • Installs a MutatingAdmissionPolicy that taints new nodes with node.projectcalico.org/network-not-ready:NoSchedule at creation. The policy ships in the embedded admission set, and the controller only includes it in the desired set when the feature is enabled, so turning it off uninstalls it.
  • Tolerates the taint on Typha, which is host-networked and needs to schedule so calico-node can come up and clear the taint. kube-controllers does not tolerate it, since it needs pod networking.
  • Turns on taint management in calico-node and kube-controllers, which remove the taint once networking is ready and re-add it if it fails.

Off by default. The admission-time taint needs Kubernetes 1.32+ for MutatingAdmissionPolicy; on older clusters the policy is skipped and the taint is managed reactively by calico-node and kube-controllers.

Depends on projectcalico/calico#11985 for the calico-node and kube-controllers behavior and the admission policy source. The nodetaint policy files are copied into the embedded set by hand here for now; the real sync pulls them from calico/calico-private once that PR merges.

Related: projectcalico/calico#11331

Adds an opt-in option to taint nodes until Calico networking is ready, so that workloads are not scheduled onto a node before Calico can service them.

Companion to the calico-side change (projectcalico/calico#11985). When
Installation.spec.networkReadyTaint is Enabled, the operator:

- Installs a MutatingAdmissionPolicy that taints new nodes with
  node.projectcalico.org/network-not-ready:NoSchedule. The policy ships in the
  embedded admission set; the core controller filters it out of the desired set
  unless the toggle is on, so turning the toggle off uninstalls it.
- Tolerates the taint on Typha, which is host-networked and must schedule so
  calico-node can come up and clear the taint. kube-controllers does not
  tolerate it, since it needs pod networking.
- Sets CALICO_MANAGE_NETWORK_READY_TAINT on calico-node and kube-controllers so
  they manage the taint at runtime.

The nodetaint.* policy files are copied in by hand for now; the real sync pulls
them from calico/calico-private api/admission once that change merges.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants