• k8s-merge-robot's avatar
    Merge pull request #28143 from xiangpengzhao/service-node-port-range · ff281525
    k8s-merge-robot authored
    Automatic merge from submit-queue
    
    Should set default value for --service-node-port-range flag before verifying
    
    For the flag `--service-node-port-range` of kube-apiserver, we know that it defaults to `30000-32767` if not specified. But if we only pass the flag `--kubernetes-service-node-port` with a valid value between `30000-32767` when starting kube-apiserver, a fatal error will occurs as the last below. It means that service port range is not 30000-32767 but empty. The log is from code [DefaultAndValidateRunOptions-->ValidateRunOptions-->verifyServiceNodePort](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L580) where the flags are verified. 
    
    After tracing the apiserver related code, we can find the call stack:
    ```
    func main() {
    	......
    	s := options.NewAPIServer()
    	......
    	app.Run(s)
    	......
    }
    ```
    In the `app.Run`, it calls [genericapiserver.DefaultAndValidateRunOptions(s.ServerRunOptions)](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L80). But the `--kubernetes-service-node-port` hasn't been defaulted before there, so it's empty. It's then defaulted in `app.Run`-->[master.New](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L276)-->[genericapiserver.New](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/master/master.go#L179)-->[setDefaults](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L338)-->[defaultServiceNodePortRange](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L281).
    
    So, we have to set default value for `--kubernetes-service-node-port` in [NewServerRunOptions](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/options/server_run_options.go#L105), as is done for `--secure-port` and/or `--insecure-port`. The `NewServerRunOptions` will be called in options.[NewAPIServer](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/options/options.go#L50)().
    
    Hope that I have described the issue clearly. Thanks!
    
    ```
    root@vm:~# kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001 &
    [2] 24629
    root@vm:~# F0627 23:46:37.308726   24629 genericapiserver.go:580] Kubernetes service port range  doesn't contain 30001
    goroutine 1 [running]:
    k8s.io/kubernetes/vendor/github.com/golang/glog.stacks(0x44f2500, 0x0, 0x0, 0x0)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:766 +0xb8
    k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).output(0x44d2020, 0xc800000003, 0xc820238000, 0x438c73b, 0x13, 0x244, 0x0)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:717 +0x259
    k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).printf(0x44d2020, 0xc800000003, 0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:655 +0x1d4
    k8s.io/kubernetes/vendor/github.com/golang/glog.Fatalf(0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:1145 +0x5d
    k8s.io/kubernetes/pkg/genericapiserver.verifyServiceNodePort(0xc8202a8400)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:580 +0x1d5
    k8s.io/kubernetes/pkg/genericapiserver.ValidateRunOptions(0xc8202a8400)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:605 +0x2f
    k8s.io/kubernetes/pkg/genericapiserver.DefaultAndValidateRunOptions(0xc8202a8400)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:612 +0x4e
    k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc820224fc0, 0x0, 0x0)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:80 +0x70
    main.main()
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/apiserver.go:50 +0x121
    
    [2]+  Exit 255                kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001
    root@vm:~#
    
    ```
    ff281525
Name
Last commit
Last update
..
admission Loading commit data...
api Loading commit data...
apimachinery Loading commit data...
apis Loading commit data...
apiserver Loading commit data...
auth Loading commit data...
capabilities Loading commit data...
client Loading commit data...
cloudprovider Loading commit data...
controller Loading commit data...
conversion Loading commit data...
credentialprovider Loading commit data...
dns Loading commit data...
fieldpath Loading commit data...
fields Loading commit data...
genericapiserver Loading commit data...
healthz Loading commit data...
httplog Loading commit data...
hyperkube Loading commit data...
kubectl Loading commit data...
kubelet Loading commit data...
kubemark Loading commit data...
labels Loading commit data...
master Loading commit data...
metrics Loading commit data...
probe Loading commit data...
proxy Loading commit data...
quota Loading commit data...
registry Loading commit data...
runtime Loading commit data...
security Loading commit data...
securitycontext Loading commit data...
serviceaccount Loading commit data...
ssh Loading commit data...
storage Loading commit data...
types Loading commit data...
ui Loading commit data...
util Loading commit data...
version Loading commit data...
volume Loading commit data...
watch Loading commit data...
OWNERS Loading commit data...