Uncordon the node after upgrade

Previous logic was relying on the node to recreate the node API object and, as a side-effect, uncordon itself. A change went in that no longer ensures the node recreates itself, so the bug in this logic was exposed.

Uncordon the node after upgrade
231fe793 · Kris · 1ed1cf6e · 231fe793
Commit 231fe793 authored May 09, 2018 by Kris
Hide whitespace changes
Inline Side-by-side

Showing with 12 additions and 6 deletions

upgrade.sh cluster/gce/upgrade.sh +12 -6

No files found.
--- a/cluster/gce/upgrade.sh
+++ b/cluster/gce/upgrade.sh
@@ -342,15 +342,21 @@ function do-single-node-upgrade() {
    sleep 1
  done
-  # Wait for the node to not have SchedulingDisabled=True and also to have
+  # Uncordon the node.
-  # Ready=True.
+  echo "== Uncordon ${instance}. == " >&2
+  local uncordon_rc
+  "${KUBE_ROOT}/cluster/kubectl.sh" uncordon "${instance}" \
+    && uncordon_rc=$? || uncordon_rc=$?
+  if [[ "${uncordon_rc}" != 0 ]]; then
+    echo "== FAILED to uncordon ${instance} =="
+    return ${uncordon_rc}
+  fi
+  # Wait for the node to have Ready=True.
  echo "== Waiting for ${instance} to become ready. ==" >&2
  while true; do
-    local cordoned=$("${KUBE_ROOT}/cluster/kubectl.sh" get node "${instance}" --output='jsonpath={.status.conditions[?(@.type == "SchedulingDisabled")].status}')
    local ready=$("${KUBE_ROOT}/cluster/kubectl.sh" get node "${instance}" --output='jsonpath={.status.conditions[?(@.type == "Ready")].status}')
-    if [[ "${cordoned}" == 'True' ]]; then
+    if [[ "${ready}" != 'True' ]]; then
-      echo "Node ${instance} is still not ready: SchedulingDisabled=${ready}"
-    elif [[ "${ready}" != 'True' ]]; then
      echo "Node ${instance} is still not ready: Ready=${ready}"
    else
      echo "Node ${instance} Ready=${ready}"