• Kubernetes Submit Queue's avatar
    Merge pull request #30787 from jbeda/rsync · 63139f93
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue
    
    Speed up dockerized builds
    
    This PR speeds up dockerized builds.  First, we make sure that we are as incremental as possible.  The bigger change is that now we use rsync to move sources into the container and get data back out.
    
    To do yet:
    * [x] Add a random password to rsync.  This is 128bit MD4, but it is better than nothing.
    * [x] Lock down rsync to only come from the host.
    * [x] Deal with remote docker engines -- this should be necessary for docker-machine on the mac.
    * [x] Allow users to specify the port for the rsync daemon.  Perhaps randomize this or let docker pick an ephemeral port and detect the port?
    * [x] Copy back generated files so that users can check them in.  This is done for `zz_generated.*` files generated by `make generated_files` 
      * [x] This should include generated proto files so that we can remove the hack-o-rama that is `hack/hack/update-*-dockerized.sh` 
    * [x] Start "versioning" the build container and the data container so that the CI system doesn't have to be manually kicked.
    * [x] Get some benchmarks to qualify how much faster.
    
    This replaces #28518 and is related to #30600.
    
    cc @thockin @spxtr @david-mcmahon @MHBauer 
    
    Benchmarks by running `make clean ; sync ; time bash -xc 'time build/make-build-image.sh ; time sync ; time build/run.sh make ; time sync; time build/run.sh make'` on a GCE n1-standard-8 with PD-SSD.
    
    | setup | build image | sync | first build | sync | second build | total |
    |-------|-------------|----- |----------|------|--------------|------|
    | baseline | 0m11.420s | 0m0.812s | 7m2.353s | 0m42.380s | 7m8.381s | 15m5.348s |
    | this pr | 0m10.977s | 0m15.168s | 7m31.096s | 1m55.692s | 0m16.514s | 10m9.449s |
    63139f93
Name
Last commit
Last update
..
admin Loading commit data...
api-reference Loading commit data...
design Loading commit data...
devel Loading commit data...
getting-started-guides Loading commit data...
images Loading commit data...
man/man1 Loading commit data...
proposals Loading commit data...
user-guide Loading commit data...
yaml/kubectl Loading commit data...
OWNERS Loading commit data...
README.md Loading commit data...
api.md Loading commit data...
reporting-security-issues.md Loading commit data...
roadmap.md Loading commit data...
troubleshooting.md Loading commit data...
warning.png Loading commit data...
whatisk8s.md Loading commit data...