Docker镜像转Singularity Jupyter注册Julia内核
·
踩坑之Docker容器启动Jupyter能找到Julia内核,转Singularity后不行解决方案。也适用于Docker根本就找不着内核的。
1. 首先构建Docker镜像,Dockerfile如下
FROM julia:1.11.2
WORKDIR /workspace
# 2. 基础工具
RUN apt-get update && apt-get install -y \
git wget vim net-tools iputils-ping ca-certificates curl \
&& rm -rf /var/lib/apt/lists/*
# 3. 复制 Project.toml(没有 Manifest,保证重新生成)
COPY Project.toml ./
# 4. 安装 Julia 依赖
RUN julia --project=/workspace -e "using Pkg; Pkg.instantiate()"
# 5. 安装 Miniconda 到 /opt/conda(清华源,覆盖已存在目录)
ENV CONDA_DIR=/opt/conda
RUN mkdir -p $CONDA_DIR && chmod -R a+rwx $CONDA_DIR && \
curl -L "https://mirrors.tuna.tsinghua.edu.cn/github-release/conda-forge/miniforge/LatestRelease/Miniforge3-25.3.1-0-Linux-x86_64.sh" \
-o /tmp/miniforge.sh && \
bash /tmp/miniforge.sh -b -p $CONDA_DIR -u && \
rm /tmp/miniforge.sh
# 6. 配置环境变量
ENV PATH="$CONDA_DIR/bin:$PATH"
ENV JULIA_CONDA_EXE="$CONDA_DIR/bin/conda"
# 7. 配置 Conda 清华镜像
RUN echo "channels:\n - defaults\nshow_channel_urls: true\ndefault_channels:\n - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main\n - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r\n - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2\ncustom_channels:\n conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud\n simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud" \
> /root/.condarc
# 8. 安装 Jupyter
RUN conda install -y jupyter
COPY . .
# 9. 构建 IJulia 并预编译
RUN julia --project=/workspace -e "using Pkg; Pkg.add(\"IJulia\"); Pkg.build(\"IJulia\"); Pkg.precompile()"
# 9.1 明确注册 Julia Kernel 到 Jupyter
RUN julia --project=/workspace -e 'using IJulia; installkernel("Julia", env=Dict("JULIA_PROJECT"=>"/workspace"), prefix="/usr/local/share/jupyter")'
# 11. 将本地包注册到当前环境(生成 Manifest)
RUN julia --project=/workspace -e "using Pkg; if isdir(\"src/GTFConv\"); Pkg.develop(path=\"src/GTFConv\"); end; Pkg.precompile()"
# 12. 默认启动 Jupyter Lab
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]
2. 构建好镜像之后docker save为tar包
docker save [镜像名:tag] > [镜像包名].tar
3.从tar包构建一下singularity容器
sudo singularity build --fakeroot --fix-perms [singularity容器的名字,自己起一个].sif docker-archive://./[镜像包名].tar
4.欸接下来重点来了,我上面Docker镜像启动之后这个Julia内核时完全没问题的,但是现在你启动Singularity却发现Julia内核找不到,这时候我们需要打个补丁把需要root权限的IJulia安装路径拷贝出来,之后才能用非root普通用户愉快地访问内核。我觉得这属于Docker转Singularity还未能修复的bug.
cat > final-wrapper.def <<'EOF'
Bootstrap: localimage
From: /tmp/julia-conda-final.sif
%post
# 把 root 家的 kernel 强行搬到全局,所有人都看得见
mkdir -p /usr/local/share/jupyter/kernels
cp -r 【改为你实际Docker安装Julia内核的路径】 /usr/local/share/jupyter/kernels/ 2>/dev/null || true
chmod -R 755 /usr/local/share/jupyter/kernels 2>/dev/null || true
# 再保险起见重新注册一次(新版 IJulia 兼容写法:prefix 是位置参数)
julia --project=/workspace -e '
using IJulia
installkernel(
"Julia", # 显示名
env = Dict("JULIA_PROJECT" => "/workspace"),
specname = "julia-1.11", # 内核规格名
"/usr/local/share/jupyter" # prefix 作为位置参数
)
println("Julia kernel successfully registered to /usr/local/share/jupyter")
'
%runscript
exec jupyter lab --ip=0.0.0.0 --port=8889 --no-browser --allow-root --notebook-dir=/workspace "$@"
EOF
【关于如何找到Julia安装的位置,‘sudo docker run -it --rm 容器名 bash’进入容器,find /opt/conda /usr/local /root -name "kernel.json" 2>/dev/null | grep -i julia看一下Julia内核安装再哪里了,比如输出如下】
/root/.local/share/jupyter/kernels/julia-1.11/kernel.json
【那你就应该把上面那条wrapper当中的源地址改为/root/.local/share/jupyter/kernels/julia*】
5. 再build一遍singularity容器
sudo singularity build [Singularity容器名].sif final-wrapper.def
6.启动容器
singularity run PATH/TO/YOUR/FINAL/SIFFILE
更多推荐
所有评论(0)