Automatic merge from submit-queue (batch tested with PRs 50306, 49624)

simplify logic around LB deletion for servicecontroller

**What this PR does / why we need it**: simplify logic around LB deletio

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**: discovered when aswering to #50299

**Release note**:

```release-note
None
```