Automatic merge from submit-queue

Wrong format is used with single quotes in log

Wrong format is used with single quotes in log, should use ""

Automatic merge from submit-queue

Allow empty value for env in kubectl run

Automatic merge from submit-queue

Update the Pod Resource Management Proposal with Implementation status

This is tied to the upstream issue #27204 for adding pod level cgroups into Kubernetes.

@vishh @derekwaynecarr @Random-Liu  PTAL.

Automatic merge from submit-queue

Federation - common libs - cluster lifecycle handlers in FederatedInformer

Adds cluster lifecycle handlers to FederatedInformer to execute some actions when a cluster becomes available or unavailable.

ref: #29347

cc: @nikhiljindal @wojtek-t @kubernetes/sig-cluster-federation

Fixes #28734

Automatic merge from submit-queue

Fix PodAffinity test

Automatic merge from submit-queue

Cleanup k8s script noise with a verbosity concept

Fixes https://github.com/kubernetes/kubernetes/issues/30109

The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.

    V=2 kube::log::info foo bar

Would only print "foo bar" if $KUBE_VERBOSE >= 2.

Examples:

Default verbosity (1 for make commands):
```
$ make kubelet
+++ [0804 17:23:32] Generating bindata:
    /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0804 17:23:37] Building the toolchain targets:
    k8s.io/kubernetes/hack/cmd/teststale
+++ [0804 17:23:37] Building go targets for linux/amd64:
    cmd/kubelet
# k8s.io/kubernetes/pkg/kubelet
pkg/kubelet/kubelet.go:247: undefined: a
make: *** [kubelet] Error 1
```

Extra verbose (5, comparable to previous levels):
<details>
```
$ make kubelet KUBE_VERBOSE=5
I0804 17:31:05.083395    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/trace.go:151:30: cannot use (traceBufHeader literal) (value of type traceBufHeader) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.083503    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/trace.go:151:7: array length 64 << 10 - unsafe.Sizeof((traceBufHeader literal)) (value of type uintptr) must be constant
I0804 17:31:05.083600    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mgcwork.go:269:37: cannot use (workbufhdr literal) (value of type workbufhdr) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.083654    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mgcwork.go:269:7: array length (_WorkbufSize - unsafe.Sizeof((workbufhdr literal))) / sys.PtrSize (value of type uintptr) must be constant
I0804 17:31:05.084006    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:28: cannot use (hchan literal) (value of type hchan) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.084040    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:66: cannot use (hchan literal) (value of type hchan) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.084076    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:14: unsafe.Sizeof((hchan literal)) + uintptr(-int(unsafe.Sizeof((hchan literal))) & (maxAlign - 1)) (value of type uintptr) is not constant
I0804 17:31:05.085536    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:80:31: cannot use (struct{b bmap; v int64} literal).v (value of type int64) as unsafe.ArbitraryType value in argument to unsafe.Offsetof
I0804 17:31:05.085567    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:80:15: unsafe.Offsetof((struct{b bmap; v int64} literal).v) (value of type uintptr) is not constant
I0804 17:31:05.085788    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:1053:45: cannot convert &zeroinitial (value of type *[1024]byte) to unsafe.Pointer
I0804 17:31:05.086995    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mfinal.go:20:65: cannot use (finalizer literal) (value of type finalizer) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.087031    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mfinal.go:20:11: array length (_FinBlockSize - 2 * sys.PtrSize - 2 * 4) / unsafe.Sizeof((finalizer literal)) (value of type uintptr) must be constant
I0804 17:31:05.087957    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mstats.go:170:39: cannot use memstats.by_size (variable of type [67]struct{size uint32; nmalloc uint64; nfree uint64}) as unsafe.ArbitraryType value in argument to unsafe.Offsetof
I0804 17:31:05.087999    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mstats.go:170:76: cannot use memstats.by_size[0] (variable of type struct{size uint32; nmalloc uint64; nfree uint64}) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.088483    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/panic.go:118:34: cannot use (_defer literal) (value of type _defer) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.088510    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/panic.go:118:20: unsafe.Sizeof((_defer literal)) (value of type uintptr) is not constant
I0804 17:31:05.089812    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/sema.go:42:42: cannot use (semaRoot literal) (value of type semaRoot) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.089845    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/sema.go:42:8: array length sys.CacheLineSize - unsafe.Sizeof((semaRoot literal)) (value of type uintptr) must be constant
I0804 17:31:05.094634    2601 parse.go:307] type checking encountered some errors in "runtime", but ignoring.
I0804 17:31:05.875185    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/davecgh/go-spew/spew/bypass.go:33:26: cannot use (*byte)(nil) (value of type *byte) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.875234    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/davecgh/go-spew/spew/bypass.go:33:12: unsafe.Sizeof((*byte)(nil)) (value of type uintptr) is not constant
I0804 17:31:05.875838    2601 parse.go:307] type checking encountered some errors in "github.com/davecgh/go-spew/spew", but ignoring.
I0804 17:31:05.897216    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/crypto/cipher/xor.go:12:36: cannot use uintptr(0) (constant 0 of type uintptr) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.897261    2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/crypto/cipher/xor.go:12:18: int(unsafe.Sizeof(uintptr(0))) (value of type int) is not constant
I0804 17:31:05.897360    2601 parse.go:307] type checking encountered some errors in "crypto/cipher", but ignoring.
I0804 17:31:06.400904    2601 conversion.go:227] considering pkg "k8s.io/kubernetes/federation/apis/core/v1"
I0804 17:31:06.401138    2601 conversion.go:243]   tags: ["k8s.io/kubernetes/federation/apis/core"]
I0804 17:31:06.427408    2601 conversion.go:283]   no viable conversions, not generating for this package
I0804 17:31:06.427508    2601 main.go:73] Completed successfully.
Go version: go version go1.6.2 linux/amd64
+++ [0804 17:31:06] Generating bindata:
    /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
Generated bindata file : 11536 ../../..//test/e2e/generated/bindata.go lines of lovely automated artifacts
+++ [0804 17:31:12] Building the toolchain targets:
    k8s.io/kubernetes/hack/cmd/teststale
+++ [0804 17:31:12] Building go targets for linux/amd64:
    cmd/kubelet
# k8s.io/kubernetes/pkg/kubelet
pkg/kubelet/kubelet.go:247: undefined: a
!!! Error in /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:506
  'go install "${goflags[@]:+${goflags[@]}}" -gcflags "${gogcflags}" -ldflags "${goldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 2
Call stack:
  1: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:506 kube::golang::build_binaries_for_platform(...)
  2: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:692 kube::golang::build_binaries(...)
  3: hack/make-rules/build.sh:27 main(...)
Exiting with status 1
!!! Error in /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:596
  '( kube::golang::setup_env; echo "Go version: $(go version)"; local host_platform; host_platform=$(kube::golang::host_platform); local goflags goldflags gogcflags; eval "goflags=(${KUBE_GOFLAGS:-})"; goldflags="${KUBE_GOLDFLAGS:-} $(kube::version::ldflags)"; gogcflags="${KUBE_GOGCFLAGS:-}"; local use_go_build; local -a targets=(); local arg; readonly BINDATAS=("${KUBE_ROOT}/test/e2e/framework/gobindata_util.go"); kube::log::status "Generating bindata:" "${BINDATAS[@]}"; for bindata in ${BINDATAS[@]};
do
    if [[ -f $bindata ]]; then
        go generate "${bindata}";
    fi;
done; for arg in "$@";
do
    if [[ "${arg}" == "--use_go_build" ]]; then
        use_go_build=true;
    else
        if [[ "${arg}" == -* ]]; then
            goflags+=("${arg}");
        else
            targets+=("${arg}");
        fi;
    fi;
done; if [[ ${#targets[@]} -eq 0 ]]; then
    targets=("${KUBE_ALL_TARGETS[@]}");
fi; local -a platforms=(${KUBE_BUILD_PLATFORMS:-}); if [[ ${#platforms[@]} -eq 0 ]]; then
    platforms=("${host_platform}");
fi; local binaries; binaries=($(kube::golang::binaries_from_targets "${targets[@]}")); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then
    local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -ge ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then
        kube::log::status "Multiple platforms requested and available ${gigs}G >= threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true;
    else
        kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false;
    fi;
fi; kube::golang::build_kube_toolchain; if [[ "${parallel}" == "true" ]]; then
    kube::log::status "Building go targets for ${platforms[@]} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
    do
        ( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: go build started"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-}; kube::log::status "${platform}: go build finished" ) &> "/tmp//${platform//\//_}.build" &
    done; local fails=0; for job in $(jobs -p);
    do
        wait ${job} || let "fails+=1";
    done; for platform in "${platforms[@]}";
    do
        cat "/tmp//${platform//\//_}.build";
    done; exit ${fails};
else
    for platform in "${platforms[@]}";
    do
        kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-};
    done;
fi )' exited with status 1
Call stack:
  1: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:596 kube::golang::build_binaries(...)
  2: hack/make-rules/build.sh:27 main(...)
Exiting with status 1
make: *** [kubelet] Error 1
```
</details>

Remaining work: Add a verbosity label to more log messages.

/cc @kubernetes/sig-api-machinery @kubernetes/contributor-experience 

Automatic merge from submit-queue

Moving image gc to pkg/kubelet/images

Automatic merge from submit-queue

replace improper create quota alias

`q` is not a shortname for `resourcequota`, so it shouldn't be an alias for `kubectl create resourcequota`.

@sttts ptal

Automatic merge from submit-queue

AWS/GCE: Rework use of master name

* Add a pillar for `hostname` (because even if there's a good Salt function for it, I don't trust it to return the short hostname)
* Move `INITIAL_ETCD_CLUSTER` to just the GCE turn-up
* Remove `master_name`, which isn't needed

Automatic merge from submit-queue

Node E2E: Move the node name initialization to first function of SynchronizedBeforeEach

Currently, we start e2e services in the first function of `SynchronizedBeforeEach` to make sure that we only start them once even we are running test in parallel test nodes.

However, e2e services require `NodeName`, but we initialize `NodeName` in the second function.

This PR moved the initialization logic into the first function, and shared the node name with all test nodes via the `SharedContext`.

Automatic merge from submit-queue

Remove use of /usr/share/google/safe_format_and_mount script from GCI

This script was provided by Google Compute Image Packages
(https://github.com/GoogleCloudPlatform/compute-image-packages) and
has been deprecated since over an year. Newer version of the package
doesn't include it at all. So remove its usage and instead follow
recommended instructions from
https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting.

cc @kubernetes/goog-image 

Automatic merge from submit-queue

Add density (batch pods creation latency and resource) and resource performance tests to `test-e2e-node' built for Linux only

This PR adds `+build linux' to density_test.go, resource_usage.go and resource_collector.go to last PR #29764. 

#29764 fails build because it depends on cgroup which can not be built for os other than Linux.

Automatic merge from submit-queue

make etcd makefile work on OSX

Automatic merge from submit-queue

E2E & NodeE2E: Move host_path, downwardapi_volume and empty_dir into common directory.

This is the second part of #29494.

For #29081.
Based on #29092, #29806.

The first commit is squash of all dependent commits. Please only review the second commit.

The second PR is only 20 lines of change.

@vishh @timstclair 

Automatic merge from submit-queue

Node E2E: Change the node e2e junit file name to junit_{image-name}{test-node-number}.xml

Fixes https://github.com/kubernetes/kubernetes/issues/30103.

Reuse the `report-prefix` in e2e test framework. Now the junit file will be like: `junit_{image-name}{test-node-number}.xml`.

Mark P2 to fix the test result.

/cc @rmmh 

Automatic merge from submit-queue

add asserts for RecognizingDecoder and update protobuf serializer

Ensure that serializers are asserting that they implement the `RecognizingDecoder` interface and update the protobuf serializer to add the `unknown` parameter.

Manifested in: https://github.com/openshift/origin/issues/9739

@smarterclayton @deads2k 

The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.

    V=2 kube::log::info foo bar

Would only print "foo bar" if $KUBE_VERBOSE >= 2.

Automatic merge from submit-queue

Clean up items from moving local cluster setup guides

Automatic merge from submit-queue

make reousrce prefix consistent with other registries

Storage class registry was added recently in #29694.
In other registries, resource prefix was changed to a more generic way using RESTOptions.ResourcePrefix.

This PR is an attempt to make both consistent.

Automatic merge from submit-queue

Remove kubelet dependency on uname

Let's avoid exec'ing stuff we don't have to.

Related to #26093

Automatic merge from submit-queue

[GarbageCollector] Fix kubectl rolling-update to work with GC

This changes the order of the [Rename()](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/rolling_updater.go#L532) function. After the change, Rename() first deletes the old RC and orphans its pods, then creates the new RC, which will then have a chance to adopt the orphaned pods.

This also fixes the "should support rolling-update to same image" [test](https://github.com/kubernetes/kubernetes/blob/master/test/e2e/kubectl.go#L915) when the garbage collector is on.

Here is the detailed explanation on why the test would have failed:
`kubectl rolling-update` will [rename](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/rolling_updater.go#L532-L546) the RC. It first creates the an identical RC (including spec.selectors) with the new name, then it deletes the existing RC. When GC is turned on, the newly created RC cannot adopt the existing pod, because it has a controllerRef pointing to the exising RC, so the new RC will create new pods and expect to see the creation. However, the new RC and the old RC have the same selector, so sometimes the old RC, instead of the new RC, has its [expectation lowered](https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/replication/replication_controller.go#L346-L362), the new RC's expectation will stuck forever. The e2e test then times out when executing `kubectl delete newRC`, because there is the new RC will not scale down as its expectation is not fulfilled.

A side-note, we should fix [rm.getPodController()](https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/replication/replication_controller.go#L346) to respect pod's controllerref, that will prevent similar bugs.

Also note that an old version `kubectl rolling-update` will not work with the GC. We cannot fix that.

Automatic merge from submit-queue

optimize lock of runtimeState stuct

use sync.RWMutex may be high effective for concorrent read

Automatic merge from submit-queue

Move NamespaceLifecycle to use shared informers

This was a follow-up to https://github.com/kubernetes/kubernetes/pull/29634

Moves the `NamespaceLifecycle` plug-in to a shared infomer cache.

/cc @kubernetes/rh-cluster-infra @deads2k @hodovska 

Automatic merge from submit-queue

federation: Adding secret API

Adding secret API to federation-apiserver and updating the federation client to include secrets

Automatic merge from submit-queue

ScheduledJob kubectl

@erictune the last part (kubectl) of the ScheduledJob, as usual builds on top of previous PRs, so only last 2 commits matter (the kubectl one and storage leftovers).

```release-note
* Introducing ScheduledJobs as described in [the proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/scheduledjob.md) as part of `batch/v2alpha1` version (experimental feature).
```
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Node disk pressure should induce image gc

If the node reports disk pressure, prior to evicting pods, the node should clean up unused images.

Automatic merge from submit-queue

fixed regex error, which replace 2 line in template file

fixes #30074 

Automatic merge from submit-queue

add alias for createconfigmap

add Aliases for cmd create configmap.
this is a supplement for [https://github.com/kubernetes/kubernetes/pull/29652]

Automatic merge from submit-queue

optimize podKiller for reading channel

Reading kl.podKillingCh should check whether ok first, then to process data

Automatic merge from submit-queue

oidc authentication plugin: don't trim issuer URLs with trailing slashes

The issuer URL passed to the plugin must identically match the issuer
URL returned by OpenID Connect discovery. However, the plugin currently
trims all trailing slashes from issuer URLs, causing a mismatch. Since
the go-oidc client already handles this case correctly, don't trim the
path.

Closes #29749

cc @hanikesn @kubernetes/sig-auth 

Automatic merge from submit-queue

make the removing pods log standardized