Until recently, if I needed to target a specific processor architecture with native runtime dependencies, I would build a dedicated Docker image targeting the platform. Although I was aware of Docker multi architecture support, building one for some reason sounded like it would take more effort.

The objective of this post is to build a reusable multi-architecture Docker image containing native OpenCVSharpExtern library as well as required OpenCV libraries and additional runtime specific dependencies for amd64, arm32 and arm64 processor architectures.

Once the library image is built, then a .Net application will be built to demonstrate it's ability to run on different devices / processor architectures as below:

• amd64
  • Windows 11 Desktop with Docker WSL2
  • Windows 10 Laptop with Docker WSL2 (bootcamp)
  • macOS 12 Monterey Laptop
• Raspberry Pi 3
  • 64 Bit Ubuntu 22.04
  • 32 Bit Raspberry Pi OS Lite (Debian bullseye)
• Raspberry Pi 4
  • 64 Bit Ubuntu 22.04
  • 32 Bit Raspberry Pi OS Lite (Debian bullseye)
• PINE A64-LTS
  • 64 bit Armbian Linux 4.19.59-sunxi64 (Debian Buster)

OpenCV and OpenCvSharp

OpenCV is an open source, cross platform computer vision and machine learning library that supports a range of platforms utilising generic CPU support as well as high performance GPU support for certain algorithms.

OpenCvSharp provides a wrapper for OpenCV for .Net platform. Using this library we can build .Net applications using languages such as C# and run them on different platforms provided platform specific runtime dependencies are available in our target environment.

Native interop in .Net

There are times a .Net application might need to utilise functionality from a dynamic native library (dll / so) and one of the options .Net developers have is platform invoke functionality (pinvoke). This has been a possibility since the early days of .Net.

To use OpenCvSharp, we need native bindings (OpenCvSharpExtern.(dll/so)) to direct the method calls to OpenCV library via dynamic linking at runtime.

The following diagram illustrates how pinvoke allows us to use SIFT.Create(); to call native OpenCV library function at runtime thanks to OpenCvSharp bindings.

Docker Multi Architecture Support

Docker multi architecture support provides the facilities to build an publish an image with specific builds for each target processor architecture using a single command / Dockerfile.

To be able to build an image supporting multiple processor architectures, the following steps are necessary:

• Select a base image with multiple architecture support.
• If dealing with native libraries, provide a conditional build script for the each processor architecture.
• Enable experimental Docker futures:
• Create a new builder to enable multi architecture support on the build machine.

# create a new builder and set it active
docker buildx create --name mybuilder --use

• Build and push images build using the multi processor architecture.

# build the image for amd64, arm64 and arm32 architectures and push
docker buildx build \
     --push \
     --platform linux/arm/v7 \
     --tag syamaner/opencvsharp-build:${build_number} \
     ./opencv-sharp

Base images and supported processor architectures

Most base images already support multiple architecture such as .Net runtimes and sdks.

When in doubt, it is possible to use the docker buildx imagetools inspect command to verify if your base image supports the required architectures:

docker buildx imagetools inspect mcr.microsoft.com/dotnet/runtime:6.0-alpine

The results will look like the following:

Challenges with distributing native bindings via NuGet

Looking into OpenCvSharp4 NuGet packages, there is already a native binding package for ARM devices named OpenCvSharp4.runtime.linux-arm.

The challenge with distributing such bindings via NuGet is that given compiling native libraries require specific versioned dependencies for the target platform, often these packages will not work out of the box due to missing / mismatching dependencies. This will lead to manually validating the dependencies and installing / resolving them on the target machine which is not fun. In addition, given devices like Raspberry Pi it can run 32 bit as well as 64 bit Operating systems, this can also cause further complications. An example issue can be seen at OpenCvSharp repositroy.

Using Docker multi-arch support for native dependencies instead.

For native bindings / dependncies we can skip NuGet and instead create a docker image where we build these bindings for our target platforms (arm32, arm64, amd64). This can also allow certain compiler optimisations as we know our target architectures.

The next section will demonstrate this approach for building the OpenCV dependencies and then a .Net application utilising this at build time.

Step 1: Building OpenCV, OpenCvSharpExtern in a custom image

This step is only needed to be repeated when a new release of OpenCV or OpenCvSharp is available. The Dockerfile looks as the following:

FROM debian:bullseye-slim AS build-native-env
ARG TARGETPLATFORM
ENV DEBIAN_FRONTEND=noninteractive
# 4.5.5: released 25 Dec 2021
ENV OPENCV_VERSION=4.5.5
# 4.5.3.20211228: released 28 Dec 2021
ENV OPENCVSHARP_VERSION=4.5.3.20211228

WORKDIR /
# install dependencies required for building OpenCV and OpenCvSharpExtern 
RUN apt-get update && apt-get -y install --no-install-recommends \
      # details omitted \
      libgdiplus 

# Get OpenCV and opencv-contrib sources using the specified release.
RUN wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
    unzip ${OPENCV_VERSION}.zip && \
    rm ${OPENCV_VERSION}.zip && \
    mv opencv-${OPENCV_VERSION} opencv
RUN wget https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.zip && \
    unzip ${OPENCV_VERSION}.zip && \
    rm ${OPENCV_VERSION}.zip && \
    mv opencv_contrib-${OPENCV_VERSION} opencv_contrib

# configure and build OpenCV optionally specifying architecture related cmake options.
RUN if [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
        ADDITIONAL_FLAGS='' ; \
    elif [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
        ADDITIONAL_FLAGS='-D ENABLE_NEON=ON -D CPU_BASELINE=NEON ' ; \
    elif [ "$TARGETPLATFORM" = "linux/arm/v7" ]; then \
        ADDITIONAL_FLAGS='-D CPU_BASELINE=NEON -D ENABLE_NEON=ON ' ; \
    fi && cd opencv && mkdir build && cd build && \
    cmake $ADDITIONAL_FLAGS \
    # additional flags omitted for clarity \
    && make -j$(nproc) \
    && make install \
    && ldconfig

# Download OpenCvSharp to build OpenCvSharpExtern native library
RUN git clone https://github.com/shimat/opencvsharp.git
RUN cd opencvsharp && git fetch --all --tags --prune && git checkout ${OPENCVSHARP_VERSION}

WORKDIR /opencvsharp/src
RUN mkdir /opencvsharp/make \
    && cd /opencvsharp/make \
    && cmake -D CMAKE_INSTALL_PREFIX=/opencvsharp/make /opencvsharp/src \
    && make -j$(nproc) \
    && make install \
    && cp /opencvsharp/make/OpenCvSharpExtern/libOpenCvSharpExtern.so /usr/lib/ \
    && ldconfig

# Copy the library and dependencies to /artifacts (to be used by images consuming this build)
# cpld.sh will copy the library we specify (./libOpenCvSharpExtern.so) and any dependencies
#    to the /artifacts directory. This is useful for sharing the library with other images
#    consuming this build.
# credits: Hemanth.HM -> https://h3manth.com/content/copying-shared-library-dependencies 
WORKDIR /opencvsharp/make/OpenCvSharpExtern
COPY cpld.sh .
RUN chmod +x cpld.sh && \
    mkdir /artifacts && \
    ./cpld.sh ./libOpenCvSharpExtern.so /artifacts/ 
RUN cp ./libOpenCvSharpExtern.so /artifacts/ 

# Publish the artefacts using a clean image
FROM debian:bullseye-slim AS final

RUN mkdir /artifacts
COPY --from=build-native-env /artifacts/ /artifacts

WORKDIR /artifacts

Step 2: A simple benchmark application consuming the native library image

In order to test compatibility with the target devices / platforms, a basic benchmark application has been built using BenchmarkDotNet

While benchmarking is not the goal, the following benchmarks are executed:

• SIFT vs SURF feature extraction
• Matching the features using FlannBasedMatcher vs BFMatcher classes provided by OpenCV.

Our objective is to ensure the application runs successfully in any combination of hardware and Operating Systems listed above.

The code is available at docker-multi-arch-opencvsharp repository.

The application is built using the following Dockerfile:

ARG OPENCV_SHARP_BUILD_TAG=2
ARG SDK_VERSION=6.0.202-bullseye-slim-amd64
ARG RUNTIME_VERSION=6.0.4-bullseye-slim

FROM syamaner/opencvsharp-build:$OPENCV_SHARP_BUILD_TAG AS opencv

# Given we are building a .Net application, the build does not have to be in the target architecture.
# Reference: https://github.com/dotnet/dotnet-docker/issues/1537#issuecomment-755351628
FROM mcr.microsoft.com/dotnet/sdk:$SDK_VERSION as build

ARG TARGETPLATFORM
WORKDIR /src
COPY . .

# Select the correct RID for the target architecture.
# run dotnet publish as usual and pass the RID.
RUN if [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
        RID=linux-x64 ; \
    elif [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
        RID=linux-arm64 ; \
    elif [ "$TARGETPLATFORM" = "linux/arm/v7" ]; then \
        RID=linux-arm ; \
    fi && \
    dotnet publish -c release -o /app -r $RID --self-contained false

# Copy the application as well as native dependencies to the final stage and build the final image without any unnecessary files.
FROM mcr.microsoft.com/dotnet/runtime:$RUNTIME_VERSION as final

WORKDIR /app

# Copy opencv sharp native binding and runtime dependencies.
COPY --from=opencv /artifacts/ /usr/lib/ 
RUN ldconfig

COPY --from=build /app/ /app/  

ENTRYPOINT [ "dotnet", "/app/OpenCVSharpBenchmarkApp.dll" ]

Next Steps

The next step will be building an application that will scan a directory for photographs and then return a list of unique photographs as well as photos that are potentially duplicates of these using the algorithms and OpenCVSharp native libraries built in this post.

Conclusion

Docker multi architecture support simplifies building applications that will be running on different processor architectures. Once an image is built and published with multiple processor architecture, we can then consume this image on the target platform by using the same image:tag across all platforms and let docker pull the correct architecture for current machine.

As an example the following command will work on amd64, arm64 and arm32 devices as long as there is a Docker installation:

docker run -it -v $(PWD)/reports/:/app/BenchmarkDotNet.Artifacts/ syamaner/opencvsharp-bench:1

It will pull the image with matching architecture and perform the benchmarks and then store the reports in the ./reports directory.

Benchmark results

The objective of the benchmark application was to verify whether or not newly built multi-arch Docker images would run successfully on different devices / processor architectures. So far it has been working consistently on various Linux distributions.

The numbers below are not necessarily accurate as there could be throttling due to temperature as well as low power from the power adapter and need to be viewed with caution.