踩坑之Docker容器启动Jupyter能找到Julia内核,转Singularity后不行解决方案。也适用于Docker根本就找不着内核的。

1. 首先构建Docker镜像,Dockerfile如下

FROM julia:1.11.2
WORKDIR /workspace

# 2. 基础工具
RUN apt-get update && apt-get install -y \
    git wget vim net-tools iputils-ping ca-certificates curl \
    && rm -rf /var/lib/apt/lists/*

# 3. 复制 Project.toml(没有 Manifest,保证重新生成)
COPY Project.toml ./

# 4. 安装 Julia 依赖
RUN julia --project=/workspace -e "using Pkg; Pkg.instantiate()"

# 5. 安装 Miniconda 到 /opt/conda(清华源,覆盖已存在目录)
ENV CONDA_DIR=/opt/conda
RUN mkdir -p $CONDA_DIR && chmod -R a+rwx $CONDA_DIR && \
    curl -L "https://mirrors.tuna.tsinghua.edu.cn/github-release/conda-forge/miniforge/LatestRelease/Miniforge3-25.3.1-0-Linux-x86_64.sh" \
    -o /tmp/miniforge.sh && \
    bash /tmp/miniforge.sh -b -p $CONDA_DIR -u && \
    rm /tmp/miniforge.sh

# 6. 配置环境变量
ENV PATH="$CONDA_DIR/bin:$PATH"
ENV JULIA_CONDA_EXE="$CONDA_DIR/bin/conda"

# 7. 配置 Conda 清华镜像
RUN echo "channels:\n  - defaults\nshow_channel_urls: true\ndefault_channels:\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r\n  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2\ncustom_channels:\n  conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n  msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n  bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n  menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n  pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n  simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud" \
    > /root/.condarc

# 8. 安装 Jupyter
RUN conda install -y jupyter

COPY . .
# 9. 构建 IJulia 并预编译
RUN julia --project=/workspace -e "using Pkg; Pkg.add(\"IJulia\"); Pkg.build(\"IJulia\"); Pkg.precompile()"

# 9.1 明确注册 Julia Kernel 到 Jupyter
RUN julia --project=/workspace -e 'using IJulia; installkernel("Julia", env=Dict("JULIA_PROJECT"=>"/workspace"), prefix="/usr/local/share/jupyter")'

# 11. 将本地包注册到当前环境(生成 Manifest)
RUN julia --project=/workspace -e "using Pkg; if isdir(\"src/GTFConv\"); Pkg.develop(path=\"src/GTFConv\"); end; Pkg.precompile()"

# 12. 默认启动 Jupyter Lab
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

2. 构建好镜像之后docker save为tar包

 docker save [镜像名:tag] > [镜像包名].tar

3.从tar包构建一下singularity容器

sudo singularity build --fakeroot --fix-perms [singularity容器的名字,自己起一个].sif docker-archive://./[镜像包名].tar

4.欸接下来重点来了,我上面Docker镜像启动之后这个Julia内核时完全没问题的,但是现在你启动Singularity却发现Julia内核找不到,这时候我们需要打个补丁把需要root权限的IJulia安装路径拷贝出来,之后才能用非root普通用户愉快地访问内核。我觉得这属于Docker转Singularity还未能修复的bug.

cat > final-wrapper.def <<'EOF'
Bootstrap: localimage
From: /tmp/julia-conda-final.sif

%post
    # 把 root 家的 kernel 强行搬到全局,所有人都看得见
    mkdir -p /usr/local/share/jupyter/kernels
    cp -r 【改为你实际Docker安装Julia内核的路径】 /usr/local/share/jupyter/kernels/ 2>/dev/null || true
    chmod -R 755 /usr/local/share/jupyter/kernels 2>/dev/null || true
    
    # 再保险起见重新注册一次(新版 IJulia 兼容写法:prefix 是位置参数)
    julia --project=/workspace -e '
        using IJulia
        installkernel(
            "Julia",  # 显示名
            env = Dict("JULIA_PROJECT" => "/workspace"),
            specname = "julia-1.11",  # 内核规格名
            "/usr/local/share/jupyter"  # prefix 作为位置参数
        )
        println("Julia kernel successfully registered to /usr/local/share/jupyter")
    '

%runscript
    exec jupyter lab --ip=0.0.0.0 --port=8889 --no-browser --allow-root --notebook-dir=/workspace "$@"
EOF

【关于如何找到Julia安装的位置,‘sudo docker run -it --rm 容器名 bash’进入容器,find /opt/conda /usr/local /root -name "kernel.json" 2>/dev/null | grep -i julia看一下Julia内核安装再哪里了,比如输出如下】

/root/.local/share/jupyter/kernels/julia-1.11/kernel.json

【那你就应该把上面那条wrapper当中的源地址改为/root/.local/share/jupyter/kernels/julia*】

5. 再build一遍singularity容器

sudo singularity build [Singularity容器名].sif final-wrapper.def

6.启动容器

singularity run PATH/TO/YOUR/FINAL/SIFFILE

更多推荐