Automatic merge from submit-queue (batch tested with PRs 60214, 58762, 59898, 59897, 60204). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove ClientSetForVersion & ClientConfigForVersion from factory

**What this PR does / why we need it**:
This is further cleaning the factory.

**Special notes for your reviewer**:
/assign @deads2k 

**Release note**:
```release-note
NONE
```