Skip to content

Install RabbitMQ from conda on arm64 #482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
danielhollas wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Install RabbitMQ from conda on arm64 #482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"default": "15"
},
"RMQ_VERSION": {
"default": "3.9.13"
"default": "3.10.25"
},
"AIIDA_VERSION": {
"default": "2.8.0"
Expand Down
33 changes: 5 additions & 28 deletions stack/base-with-services/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,41 +8,18 @@ WORKDIR /opt/

ARG PGSQL_VERSION
ARG RMQ_VERSION
ARG TARGETARCH

# Location of the Postgresql DB
# This variable is automatically picked up by initdb and pg_ctl
# WARNING: If you change this, you have to change it in full-stack as well!
ENV PGDATA=/home/${NB_USER}/.postgresql

# Install RabbitMQ and PostgreSQL in a dedicated conda environment.
#
# RabbitMQ is currently not available on conda-forge for arm64, see:
# https://github.com/conda-forge/rabbitmq-server-feedstock/issues/67If
# Instead we need install erlang via apt and RabbitMQ as a "Generic Unix Build", see:
# https://www.rabbitmq.com/install-generic-unix.html
# Note that this version must be compatible with system's erlang version.
# Currently installed Erlang version is 23.3, so the latest supported RMQ version is 3.9.21
# https://www.rabbitmq.com/docs/which-erlang#old-timers
# Note that system erlang from arm64 is already installed in the base image,
# together with other APT dependencies to save build time.
RUN if [ "$TARGETARCH" = "amd64" ]; then \
mamba create -p /opt/conda/envs/aiida-core-services --yes \
postgresql=${PGSQL_VERSION} \
rabbitmq-server=${RMQ_VERSION} && \
mamba clean --all -f -y && \
fix-permissions "${CONDA_DIR}"; \
elif [ "$TARGETARCH" = "arm64" ]; then \
mamba create -p /opt/conda/envs/aiida-core-services --yes \
postgresql=${PGSQL_VERSION} && \
mamba clean --all -f -y && \
wget -c https://github.com/rabbitmq/rabbitmq-server/releases/download/v${RMQ_VERSION}/rabbitmq-server-generic-unix-${RMQ_VERSION}.tar.xz && \
tar -xf rabbitmq-server-generic-unix-${RMQ_VERSION}.tar.xz && \
rm rabbitmq-server-generic-unix-${RMQ_VERSION}.tar.xz && \
mv rabbitmq_server-${RMQ_VERSION} /opt/conda/envs/aiida-core-services/rabbitmq_server && \
ln -sf /opt/conda/envs/aiida-core-services/rabbitmq_server/sbin/* /opt/conda/envs/aiida-core-services/bin/ && \
fix-permissions "${CONDA_DIR}"; \
fi
RUN mamba create -p /opt/conda/envs/aiida-core-services --yes \
postgresql=${PGSQL_VERSION} \
rabbitmq-server=${RMQ_VERSION} && \
mamba clean --all -f -y && \
fix-permissions "${CONDA_DIR}"
Comment on lines +18 to +22
Copy link
Copy Markdown
Member

@unkcpz unkcpz Jul 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!


# Configure AiiDA profile.
COPY config-quick-setup.yaml .
Expand Down
3 changes: 0 additions & 3 deletions stack/base-with-services/before-notebook.d/30_start-rabbitmq.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,4 @@ set -emx
# Fix issue where the erlang cookie permissions are corrupted.
chmod 400 "/home/${NB_USER}/.erlang.cookie" || echo "erlang cookie not created yet."

# NOTE: In arm64 build, rabbitmq is not installed via conda,
# but the following incantation still works since
# rabbitmq-server is available globally.
mamba run -n aiida-core-services rabbitmq-server -detached
11 changes: 3 additions & 8 deletions stack/base-with-services/configure_rabbitmq.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,9 @@
#!/bin/bash
set -emx

RMQ_ETC_DIR_ARM64="/opt/conda/envs/aiida-core-services/rabbitmq_server/etc/rabbitmq"
RMQ_ETC_DIR_AMD64="/opt/conda/envs/aiida-core-services/etc/rabbitmq"
if [[ -d $RMQ_ETC_DIR_ARM64 ]]; then
RMQ_ETC_DIR="$RMQ_ETC_DIR_ARM64"
elif [[ -d $RMQ_ETC_DIR_AMD64 ]]; then
RMQ_ETC_DIR="$RMQ_ETC_DIR_AMD64"
else
echo "ERROR: Could not find RabbitMQ etc directory"
RMQ_ETC_DIR="/opt/conda/envs/aiida-core-services/etc/rabbitmq"
if [[ ! -d $RMQ_ETC_DIR ]]; then
echo "ERROR: directory '$RMQ_ETC_DIR' does not exist!"
exit 1
fi

Expand Down
7 changes: 3 additions & 4 deletions stack/base/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,11 @@ USER root
# bc: needed to compute the resources for computer setup
ENV EXTRA_APT_PACKAGES="curl povray rsync build-essential bc"

# For ARM64 we need to install erlang as it is not available on conda-forge
# (this is needed later as rabbitmq dependency in base-with-services image,
# but we install it here so that we don't have to invoke apt multiple times.
# For ARM64 we also install few more packages for compiling from source,
# which is more often needed on ARM64.
ARG TARGETARCH
RUN if [ "$TARGETARCH" = "arm64" ]; then \
EXTRA_APT_PACKAGES="erlang libhdf5-serial-dev pkg-config ${EXTRA_APT_PACKAGES}"; \
EXTRA_APT_PACKAGES="libhdf5-serial-dev pkg-config ${EXTRA_APT_PACKAGES}"; \
fi;\
apt-get update --yes && \
apt-get install --yes --no-install-recommends ${EXTRA_APT_PACKAGES} && \
Expand Down
Loading