• Kubernetes Submit Queue's avatar
    Merge pull request #55168 from nikhita/customresources-subresources · 6e856480
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    apiextensions: add subresources for custom resources
    
    Fixes #38113
    Fixes #58778
    
    **Related**:
    - Proposal: https://github.com/kubernetes/community/pull/913
    - For custom resources to work with `kubectl scale`: https://github.com/kubernetes/kubernetes/pull/58283
    
    **Add types**:
    
    - Add `CustomResourceSubResources` type to CRD.
        - Fix proto generation for `CustomResourceSubResourceStatus`: https://github.com/kubernetes/kubernetes/pull/55970.
    - Add feature gate for `CustomResourceSubResources`.
        - Update CRD strategy: if feature gate is disabled, this feature is dropped (i.e. set to `nil`).
    - Add validation for `CustomResourceSubResources`:
        - `SpecReplicasPath` should not be empty and should be a valid json path under `.spec`. If there is no value under the given path in the CustomResource, the `/scale` subresource will return an error on GET.
        - `StatusReplicasPath` should not be empty and should be a valid json path under `.status`. If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource will default to 0.
        - If present, `LabelSelectorPath` should be a valid json path. If there is no value under `LabelSelectorPath` in the CustomResource, the status label selector value in the `/scale` subresource will default to the empty string.
        - `ScaleGroupVersion` should be `autoscaling/v1`.
        - If `CustomResourceSubResources` is enabled, only `properties` is allowed under the root schema for CRD validation.
    
    **Add status and scale subresources**:
    
    - Use helper functions from `apimachinery/pkg/apis/meta/v1/unstructured/helpers.go`.
        - Improve error handling: https://github.com/kubernetes/kubernetes/pull/56563, https://github.com/kubernetes/kubernetes/pull/58215.
    - Introduce Registry interface for storage.
    - Update storage:
        - Introduce `CustomResourceStorage` which acts as storage for the custom resource and its status and scale subresources. Note: storage for status and scale is only enabled when the feature gate is enabled _and_ the respective fields are enabled in the CRD.
        - Introduce `StatusREST` and its `New()`, `Get()` and `Update()` methods.
        - Introduce `ScaleREST` and its `New()`, `Get()` and `Update()` methods.
            - Get and Update use the json paths from the CRD and use it to return an `autoscaling/v1.Scale` object.
    - Update strategy:
        - In `PrepareForCreate`,
             - Clear `.status`.
             - Set `.metadata.generation` = 1
        - In `PrepareForUpdate`,
             - Do not update `.status`.
                 - If both the old and new objects have `.status` and it is changed, set it back to its old value.
                 - If the old object has a `.status` but the new object doesn't, set it to the old value.
                 - If old object did not have a `.status` but the new object does, delete it.
             - Increment generation if spec changes i.e. in the following cases:
                 - If both the old and new objects had `.spec` and it changed.
                 - If the old object did not have `.spec` but the new object does.
                 - If the old object had a `.spec` but the new object doesn't.
         - In `Validate` and `ValidateUpdate`,
            - ensure that values at `specReplicasPath` and `statusReplicasPath` are >=0 and < maxInt32.
            - make sure there are no errors in getting the value at all the paths.
        - Introduce `statusStrategy` with its methods.
            - In `PrepareForUpdate`:
                - Do not update `.spec`.
                    - If both the old and new objects have `.spec` and it is changed, set it back to its old value.
                    - If the old object has a `.spec` but the new object doesn't, set it to the old value.
                    - If old object did not have a `.spec` but the new object does, delete it.
                 - Do not update `.metadata`.
            - In `ValidateStatusUpdate`:
                - For CRD validation, validate only under `.status`.
                - Validate value at `statusReplicasPath` as above. If `labelSelectorPath` is a path under `.status`, then validate it as well.
    - Plug into the custom resource handler:
        - Store all three storage - customResource, status and scale in `crdInfo`.
        - Use the storage as per the subresource in the request.
        - Use the validator as per the subresource (for status, only use the schema for `status`, if present).
        - Serve the endpoint as per the subresource - see `serveResource`, `serveStatus` and `serveScale`.
    - Update discovery by adding the `/status` and `/scale` resources, if enabled.
    
    **Add tests**:
    
    - Add unit tests in `etcd_test.go`.
    - Add integration tests.
        - In `subresources_test.go`, use the [polymporphic scale client](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/client-go/scale) to get and update `Scale`.
        -  Add a test to check everything works fine with yaml in `yaml_test.go`.
    
    **Release note**:
    
    ```release-note
    `/status` and `/scale` subresources are added for custom resources.
    ```
    6e856480
Name
Last commit
Last update
..
boilerplate Loading commit data...
cmd/teststale Loading commit data...
e2e-internal Loading commit data...
gen-swagger-doc Loading commit data...
jenkins Loading commit data...
lib Loading commit data...
make-rules Loading commit data...
testdata Loading commit data...
verify-flags Loading commit data...
.golint_failures Loading commit data...
BUILD Loading commit data...
OWNERS Loading commit data...
autogenerated_placeholder.txt Loading commit data...
benchmark-go.sh Loading commit data...
build-cross.sh Loading commit data...
build-go.sh Loading commit data...
build-ui.sh Loading commit data...
cherry_pick_pull.sh Loading commit data...
dev-build-and-push.sh Loading commit data...
dev-build-and-up.sh Loading commit data...
dev-push-hyperkube.sh Loading commit data...
e2e-node-test.sh Loading commit data...
e2e.go Loading commit data...
e2e_test.go Loading commit data...
generate-bindata.sh Loading commit data...
generate-docs.sh Loading commit data...
get-build.sh Loading commit data...
ginkgo-e2e.sh Loading commit data...
godep-restore.sh Loading commit data...
godep-save.sh Loading commit data...
grab-profiles.sh Loading commit data...
import-restrictions.yaml Loading commit data...
install-etcd.sh Loading commit data...
list-feature-tests.sh Loading commit data...
local-up-cluster.sh Loading commit data...
lookup_pull.py Loading commit data...
print-workspace-status.sh Loading commit data...
run-in-gopath.sh Loading commit data...
test-cmd.sh Loading commit data...
test-go.sh Loading commit data...
test-integration.sh Loading commit data...
test-update-storage-objects.sh Loading commit data...
update-all.sh Loading commit data...
update-api-reference-docs.sh Loading commit data...
update-bazel.sh Loading commit data...
update-cloudprovider-gce.sh Loading commit data...
update-codegen.sh Loading commit data...
update-generated-device-plugin-dockerized.sh Loading commit data...
update-generated-device-plugin.sh Loading commit data...
update-generated-docs.sh Loading commit data...
update-generated-kms-dockerized.sh Loading commit data...
update-generated-kms.sh Loading commit data...
update-generated-protobuf-dockerized.sh Loading commit data...
update-generated-protobuf.sh Loading commit data...
update-generated-runtime-dockerized.sh Loading commit data...
update-generated-runtime.sh Loading commit data...
update-generated-swagger-docs.sh Loading commit data...
update-godep-licenses.sh Loading commit data...
update-gofmt.sh Loading commit data...
update-openapi-spec.sh Loading commit data...
update-staging-godeps-dockerized.sh Loading commit data...
update-staging-godeps.sh Loading commit data...
update-swagger-spec.sh Loading commit data...
update-translations.sh Loading commit data...
update_owners.py Loading commit data...
verify-all.sh Loading commit data...
verify-api-groups.sh Loading commit data...
verify-api-reference-docs.sh Loading commit data...
verify-bazel.sh Loading commit data...
verify-boilerplate.sh Loading commit data...
verify-cli-conventions.sh Loading commit data...
verify-cloudprovider-gce.sh Loading commit data...
verify-codegen.sh Loading commit data...
verify-description.sh Loading commit data...
verify-flags-underscore.py Loading commit data...
verify-generated-device-plugin.sh Loading commit data...
verify-generated-docs.sh Loading commit data...
verify-generated-files-remake.sh Loading commit data...
verify-generated-kms.sh Loading commit data...
verify-generated-protobuf.sh Loading commit data...
verify-generated-runtime.sh Loading commit data...
verify-generated-swagger-docs.sh Loading commit data...
verify-godep-licenses.sh Loading commit data...
verify-godeps.sh Loading commit data...
verify-gofmt.sh Loading commit data...
verify-golint.sh Loading commit data...
verify-govet.sh Loading commit data...
verify-import-boss.sh Loading commit data...
verify-imports.sh Loading commit data...
verify-linkcheck.sh Loading commit data...
verify-no-vendor-cycles.sh Loading commit data...
verify-openapi-spec.sh Loading commit data...
verify-pkg-names.sh Loading commit data...
verify-readonly-packages.sh Loading commit data...
verify-staging-godeps.sh Loading commit data...
verify-swagger-spec.sh Loading commit data...
verify-symbols.sh Loading commit data...
verify-test-images.sh Loading commit data...
verify-test-owners.sh Loading commit data...