• k8s-merge-robot's avatar
    Merge pull request #28143 from xiangpengzhao/service-node-port-range · ff281525
    k8s-merge-robot authored
    Automatic merge from submit-queue
    
    Should set default value for --service-node-port-range flag before verifying
    
    For the flag `--service-node-port-range` of kube-apiserver, we know that it defaults to `30000-32767` if not specified. But if we only pass the flag `--kubernetes-service-node-port` with a valid value between `30000-32767` when starting kube-apiserver, a fatal error will occurs as the last below. It means that service port range is not 30000-32767 but empty. The log is from code [DefaultAndValidateRunOptions-->ValidateRunOptions-->verifyServiceNodePort](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L580) where the flags are verified. 
    
    After tracing the apiserver related code, we can find the call stack:
    ```
    func main() {
    	......
    	s := options.NewAPIServer()
    	......
    	app.Run(s)
    	......
    }
    ```
    In the `app.Run`, it calls [genericapiserver.DefaultAndValidateRunOptions(s.ServerRunOptions)](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L80). But the `--kubernetes-service-node-port` hasn't been defaulted before there, so it's empty. It's then defaulted in `app.Run`-->[master.New](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L276)-->[genericapiserver.New](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/master/master.go#L179)-->[setDefaults](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L338)-->[defaultServiceNodePortRange](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L281).
    
    So, we have to set default value for `--kubernetes-service-node-port` in [NewServerRunOptions](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/options/server_run_options.go#L105), as is done for `--secure-port` and/or `--insecure-port`. The `NewServerRunOptions` will be called in options.[NewAPIServer](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/options/options.go#L50)().
    
    Hope that I have described the issue clearly. Thanks!
    
    ```
    root@vm:~# kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001 &
    [2] 24629
    root@vm:~# F0627 23:46:37.308726   24629 genericapiserver.go:580] Kubernetes service port range  doesn't contain 30001
    goroutine 1 [running]:
    k8s.io/kubernetes/vendor/github.com/golang/glog.stacks(0x44f2500, 0x0, 0x0, 0x0)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:766 +0xb8
    k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).output(0x44d2020, 0xc800000003, 0xc820238000, 0x438c73b, 0x13, 0x244, 0x0)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:717 +0x259
    k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).printf(0x44d2020, 0xc800000003, 0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:655 +0x1d4
    k8s.io/kubernetes/vendor/github.com/golang/glog.Fatalf(0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:1145 +0x5d
    k8s.io/kubernetes/pkg/genericapiserver.verifyServiceNodePort(0xc8202a8400)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:580 +0x1d5
    k8s.io/kubernetes/pkg/genericapiserver.ValidateRunOptions(0xc8202a8400)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:605 +0x2f
    k8s.io/kubernetes/pkg/genericapiserver.DefaultAndValidateRunOptions(0xc8202a8400)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:612 +0x4e
    k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc820224fc0, 0x0, 0x0)
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:80 +0x70
    main.main()
    	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/apiserver.go:50 +0x121
    
    [2]+  Exit 255                kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001
    root@vm:~#
    
    ```
    ff281525
Name
Last commit
Last update
..
options Loading commit data...
OWNERS Loading commit data...
default_storage_factory_builder.go Loading commit data...
default_storage_factory_builder_test.go Loading commit data...
doc.go Loading commit data...
genericapiserver.go Loading commit data...
genericapiserver_test.go Loading commit data...
resource_config.go Loading commit data...
resource_config_test.go Loading commit data...
resource_encoding_config.go Loading commit data...
server_run_options_test.go Loading commit data...
storage_factory.go Loading commit data...
storage_factory_test.go Loading commit data...
tunneler.go Loading commit data...
tunneler_test.go Loading commit data...