环境

  1. 主机环境
    cat /etc/os-release
    	NAME="Ubuntu"
    	VERSION="18.04.1 LTS (Bionic Beaver)"
    	ID=ubuntu
    	ID_LIKE=debian
    	PRETTY_NAME="Ubuntu 18.04.1 LTS"
    	VERSION_ID="18.04"
    	HOME_URL="https://www.ubuntu.com/"
    	SUPPORT_URL="https://help.ubuntu.com/"
    	BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
    	PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
    	VERSION_CODENAME=bionic
    	UBUNTU_CODENAME=bionic
    
  2. Docker环境
    docker version
    	Client:
    	 Version:           18.09.7
    	 API version:       1.39
    	 Go version:        go1.10.1
    	 Git commit:        2d0083d
    	 Built:             Fri Aug 16 14:20:06 2019
    	 OS/Arch:           linux/amd64
    	 Experimental:      false
    	
    	Server:
    	 Engine:
    	  Version:          18.09.7
    	  API version:      1.39 (minimum version 1.12)
    	  Go version:       go1.10.1
    	  Git commit:       2d0083d
    	  Built:            Wed Aug 14 19:41:23 2019
    	  OS/Arch:          linux/amd64
    	  Experimental:     false
    
  3. Golang
    go version
    	go version go1.13.12 linux/amd64
    
  4. k8s版本
    	分支: master
    	最后的commit id: 43fbe17dc682b703b0b663935a797244378f0aaf
    

编译

  1. 参考资料: https://github.com/kubernetes/kubernetes/blob/98814409bab818d942e3c341d76eb594f12675a5/build/ README.md
    https://xuxinkun.github.io/2016/08/05/build-kubernetes/

  2. 执行命令:在kubernetes顶层目录执行 build/run.sh make,发生错误:

    Step 1/16 : FROM us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1
    Get https://us.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    
  3. 安装本地docker registry(这里需要本地的registry,只需要本地docker有这个镜像就可以了)

    • Deploy a registry server
    • 先在香港节点的服务器上拉下镜像: us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1, 然后docker save -o kube-cross.tar us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1
    • 从香港节点的服务器用Filezilla拉取到本地windows
    • 再用Filezilla传到虚拟机,然后docker load -i kube-cross.tar
    • 把镜像推送到本地的registry
      • docker tag us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1 localhost:5000/my-kube-cross:v1.14.6-1
      • docker push localhost:5000/my-kube-cross:v1.14.6-1
  4. 修改源文件

    • vi ./build/build-image/Dockerfile
      -FROM us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:KUBE_BUILD_IMAGE_CROSS_TAG
      +FROM localhost:5000/my-kube-cross:KUBE_BUILD_IMAGE_CROSS_TAG
      
  5. 再次运行: build/run.sh make,多撑了会

    root@ubuntu18:~/gomod/kubernetes# build/run.sh make
    +++ [0723 14:40:47] Verifying Prerequisites....
    +++ [0723 14:41:04] Building Docker image kube-build:build-939996438d-5-v1.14.6-1
    +++ [0723 14:42:04] Keeping container registry
    +++ [0723 14:42:04] Keeping container registry
    +++ [0723 14:42:04] Keeping container registry
    +++ [0723 14:42:04] Keeping image kube-build:build-939996438d-5-v1.14.6-1
    +++ [0723 14:42:04] Creating data container kube-build-data-939996438d-5-v1.14.6-1
    +++ [0723 14:42:27] Syncing sources to container
    +++ [0723 14:42:27] Stopping any currently running rsyncd container
    +++ [0723 14:42:27] Starting rsyncd container
    +++ [0723 14:42:28] Running rsync
    ERROR: password file must not be other-accessible
    rsync error: syntax or usage error (code 1) at authenticate.c(196) [sender=3.1.2]
    !!! [0723 14:42:28] Call tree:
    !!! [0723 14:42:28]  1: build/../build/common.sh:706 kube::build::rsync(...)
    !!! [0723 14:42:28]  2: build/../build/common.sh:444 kube::build::sync_to_container(...)
    !!! [0723 14:42:28]  3: build/run.sh:31 kube::build::build_image(...)
    !!! Error in build/../build/common.sh:686
      Error in build/../build/common.sh:686. '((i<5-1))' exited with status 1
    Call stack:
      1: build/../build/common.sh:686 kube::build::rsync(...)
      2: build/../build/common.sh:706 kube::build::sync_to_container(...)
      3: build/../build/common.sh:444 kube::build::build_image(...)
      4: build/run.sh:31 main(...)
    Exiting with status 1
    
  6. 上面的ERROR: password file must not be other-accessible找了好半天的原因是因为,我是从windows宿主机通过mount把kubernetes的源文件挂载到linux的虚拟机导致的, 解决办法就是拷贝源代码文件到另一个位置不要让源文件被2个系统共享

  7. 然后又出现如下错误:

    Makefile:1: *** missing separator.  Stop.
    !!! [0724 14:29:54] Call tree:
    !!! [0724 14:29:54]  1: build/../build/common.sh:527 kube::build::run_build_command_ex(...)
    !!! [0724 14:29:54]  2: build/run.sh:39 kube::build::run_build_command(...)
    !!! Error in build/../build/common.sh:604
      Error in build/../build/common.sh:604. '((i<4-1))' exited with status 2
    Call stack:
      1: build/../build/common.sh:604 kube::build::run_build_command_ex(...)
      2: build/../build/common.sh:527 kube::build::run_build_command(...)
      3: build/run.sh:39 main(...)
    Exiting with status 1
    
  8. 还是因为被windows系统打开过导致文件格式出现问题,重新下载kubernetes源码并直接copy到linux系统上,不再被windows系统染指

  9. 执行build/run.sh make, 报磁盘空间不足,清理磁盘空间

  10. 再次执行build/run.sh make,报错:

    /usr/local/go/pkg/tool/linux_amd64/link: signal: killed
    !!! [0724 16:22:59] Call tree:
    !!! [0724 16:22:59]  1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:709 kube::golang::build_some_binaries(...)
    !!! [0724 16:22:59]  2: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:864 kube::golang::build_binaries_for_platform(...)
    !!! [0724 16:22:59]  3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
    !!! Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:673
      Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:673. 'go install "${build_args[@]}" "$@"' exited with status 1
    Call stack:
      1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:673 kube::golang::build_some_binaries(...)
      2: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:709 kube::golang::build_binaries_for_platform(...)
      3: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:864 kube::golang::build_binaries(...)
      4: hack/make-rules/build.sh:27 main(...)
    Exiting with status 1
    !!! [0724 16:22:59] Call tree:
    !!! [0724 16:22:59]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
    !!! Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:865
      Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:865. '( kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform "${platform}" )' exited with status 1
    Call stack:
      1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:865 kube::golang::build_binaries(...)
      2: hack/make-rules/build.sh:27 main(...)
    Exiting with status 1
    !!! [0724 16:22:59] Call tree:
    !!! [0724 16:22:59]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
    !!! Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:868
      Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:868. '( kube::golang::setup_env; V=2 kube::log::info "Go version: $(go version)"; local host_platform; host_platform=$(kube::golang::host_platform); local goflags goldflags goasmflags gogcflags gotags; goldflags="${GOLDFLAGS=-s -w -buildid=} $(kube::version::ldflags)"; goasmflags="-trimpath=${KUBE_ROOT}"; gogcflags="${GOGCFLAGS:-} -trimpath=${KUBE_ROOT}"; gotags="selinux,$(echo "${GOFLAGS:-}" | sed -e 's|.*-tags=\([^-]*\).*|\1|')"; local -a targets=(); local arg; for arg in "$@";
    do
        if [[ "${arg}" == -* ]]; then
            goflags+=("${arg}");
        else
            targets+=("${arg}");
        fi;
    done; if [[ ${#targets[@]} -eq 0 ]]; then
        targets=("${KUBE_ALL_TARGETS[@]}");
    fi; local -a platforms; IFS=" " read -ra platforms <<< "${KUBE_BUILD_PLATFORMS:-}"; if [[ ${#platforms[@]} -eq 0 ]]; then
        platforms=("${host_platform}");
    fi; local -a binaries; while IFS="" read -r binary; do
        binaries+=("$binary");
    done < <(kube::golang::binaries_from_targets "${targets[@]}"); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then
        local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -ge ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then
            kube::log::status "Multiple platforms requested and available ${gigs}G >= threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true;
        else
            kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false;
        fi;
    fi; if [[ "${parallel}" == "true" ]]; then
        kube::log::status "Building go targets for {${platforms[*]}} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
        do
            ( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: build started"; kube::golang::build_binaries_for_platform "${platform}"; kube::log::status "${platform}: build finished" ) &> "/tmp//${platform//\//_}.build" &
        done; local fails=0; for job in $(jobs -p);
        do
            wait "${job}" || (( fails+=1 ));
        done; for platform in "${platforms[@]}";
        do
            cat "/tmp//${platform//\//_}.build";
        done; exit ${fails};
    else
        for platform in "${platforms[@]}";
        do
            kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; ( kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform "${platform}" );
        done;
    fi )' exited with status 1
    Call stack:
      1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:868 kube::golang::build_binaries(...)
      2: hack/make-rules/build.sh:27 main(...)
    Exiting with status 1
    make: *** [Makefile:93: all] Error 1
    !!! [0724 16:23:01] Call tree:
    !!! [0724 16:23:01]  1: build/../build/common.sh:527 kube::build::run_build_command_ex(...)
    !!! [0724 16:23:01]  2: build/run.sh:39 kube::build::run_build_command(...)
    !!! Error in build/../build/common.sh:603
      Error in build/../build/common.sh:603. '((i<4-1))' exited with status 2
    Call stack:
      1: build/../build/common.sh:603 kube::build::run_build_command_ex(...)
      2: build/../build/common.sh:527 kube::build::run_build_command(...)
      3: build/run.sh:39 main(...)
    Exiting with status 1
    
  11. 发现系统负载特别大, 调高了虚拟机的配置,升到 8核8G

  12. 再次执行build/run.sh make,好像成功了,生成的文件在_output目录下

    Go version: go version go1.14.6 linux/amd64
    +++ [0724 16:55:59] Building go targets for linux/amd64:
        cmd/kube-proxy
        cmd/kube-apiserver
        cmd/kube-controller-manager
        cmd/kubelet
        cmd/kubeadm
        cmd/kube-scheduler
        vendor/k8s.io/kube-aggregator
        vendor/k8s.io/apiextensions-apiserver
        cluster/gce/gci/mounter
        cmd/kubectl
        cmd/gendocs
        cmd/genkubedocs
        cmd/genman
        cmd/genyaml
        cmd/genswaggertypedocs
        cmd/linkcheck
        vendor/github.com/onsi/ginkgo/ginkgo
        test/e2e/e2e.test
        cluster/images/conformance/go-runner
        cmd/kubemark
        vendor/github.com/onsi/ginkgo/ginkgo
        test/e2e_node/e2e_node.test
    Env for linux/amd64: GOOS=linux GOARCH=amd64 GOROOT=/usr/local/go CGO_ENABLED= CC=
    Coverage is disabled.
    Coverage is disabled.
    +++ [0724 17:03:03] Placing binaries
    +++ [0724 17:03:22] Syncing out of container
    +++ [0724 17:03:22] Stopping any currently running rsyncd container
    +++ [0724 17:03:22] Starting rsyncd container
    +++ [0724 17:03:24] Running rsync
    +++ [0724 17:03:33] Stopping any currently running rsyncd container
    
  13. _output目录结构

    root@ubuntu18:~/kubernetes/_output# tree
    .
    ├── dockerized
    │   └── bin
    │       ├── go-bindata
    │       └── linux
    │           └── amd64
    │               ├── apiextensions-apiserver
    │               ├── conversion-gen
    │               ├── deepcopy-gen
    │               ├── defaulter-gen
    │               ├── e2e_node.test
    │               ├── e2e.test
    │               ├── gendocs
    │               ├── genkubedocs
    │               ├── genman
    │               ├── genswaggertypedocs
    │               ├── genyaml
    │               ├── ginkgo
    │               ├── go2make
    │               ├── go-bindata
    │               ├── go-runner
    │               ├── kubeadm
    │               ├── kube-aggregator
    │               ├── kube-apiserver
    │               ├── kube-controller-manager
    │               ├── kubectl
    │               ├── kubelet
    │               ├── kubemark
    │               ├── kube-proxy
    │               ├── kube-scheduler
    │               ├── linkcheck
    │               ├── mounter
    │               ├── openapi-gen
    │               └── prerelease-lifecycle-gen
    └── images
        └── kube-build:build-f1967cbefe-5-v1.14.6-1
            ├── Dockerfile
            ├── localtime
            ├── rsyncd.password
            └── rsyncd.sh
    
    1. 只编译某个组件: bash build/run.sh hack/build-go.sh cmd/kubelet, 建议使用: bash build/run.sh make WHAT='cmd/kubelet'
    2. 总结:
    • 编译的虚拟机配置要高: 至少8核8G
    • 不要通过mount的方式在windows宿主机和编译的linux虚拟机之间共享kubernets源码,会导致rsync出问题
    • 不要用windows打开任何文件,可能导致Makefile出现tab键的问题
    • 编译时要用到的docker镜像通过别的方式下载下来,放到本地docker就可以了,用不着本地启动registry
Logo

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐