The loop should use 'continue' not 'break', otherwise removeFinalizer()
not only removes "orphaningFinalizer" from its finalizers list but
also removes others.