DeepChem M1 Mac Support

#1

Hi all,
I am working on creating a script for building DeepChem on M1 Mac. This script should bootstrap without any requirements. Note that the ordering of elements in this script is important:

#!/bin/zsh
set -ex
cd ..
if \[ "$1" = "-n" \]; then
    rm -rf /private/var/folders/w2/*/T/deepchem* || true
    cd "$(mktemp -d -t deepchem)"
    curl https://github.com/conda-forge/miniforge/releases/download/4.12.0-0/miniforge3-`uname`-`uname -m`.sh -L -O
	chmod +x ./miniforge3-$(uname)-$(uname -m).sh
	zsh ./miniforge3-`uname`-`uname -m`.sh -b -f
fi
conda init
source ~/miniforge3/etc/profile.d/conda.sh
brew install hdf5 swig
export SDKROOT=`xcrun --show-sdk-path`
conda create --name venv python=3.9 -y
conda activate venv
conda clean --all -y
pip3 cache purge
conda install -c conda-forge rust pdbfixer mdtraj -y
conda install pymatgen -y
conda install -c anaconda pip h5py cython pkgconfig -y
conda install -c apple tensorflow-deps --force-reinstall -y
pip3 install vina --global-option=build_ext --global-option="-L/$(brew --cellar boost)" --global-option="-L/$(brew --cellar boost-python3)"
pip3 install joblib pandas scikit-learn scipy rdkit-pypi flaky optax dm-haiku biopython
pip3 install --no-binary :all: numpy
pip3 install install transformers
conda install -c pytorch pytorch -y
conda install -c pytest pytest -y
pip3 install tensorflow-macos --force-reinstall
pip3 install tensorflow-metal --force-reinstall
which python3
python -m pip install -e git+https:////github.com/deepchem/deepchem.git#egg=deepchem --force-reinstall
python -m pytest src/deepchem/deepchem/models/tests/ --lf
ls .

Be aware that this also installs globally some brew packages. Please if anyone else who has an M1 Mac, if they are interested, I would like to make sure that this works on any machine, not just mine. I am also working on getting it to work with PyTorch. Make sure to run it with the -n flag the first time to download and reinstall miniforge3. I have some of the tests working, but they are quite slow. However–they are definitely using the GPU. The main issues seem to stem from lack of support in tensorflow-metal for certain optimizers.

1 Like
DeepChem Minutes 6/9/2022
Deepchem on M1 MacBook
#2

Why are the brew packages needed? Both hdf5 and swig are available from conda. Also see https://github.com/deepchem/deepchem/issues/2871 for more issues about running on M1 Macs. The lack of DGL is the biggest problem. It looks like you managed to get around the JAX problem I ran into.

#3

Hi, the brew packages are still needed for boost and boost-python3. Here’s an updated build script. Could you try running this on your machine and let me know if it installs correctly? (Some of the tests should pass, but not all). Particularly I’m not sure about the install locations, and I wanna make sure it doesn’t bork an existing installation (I do not have an existing installation of conda).

#!/bin/zsh
set -ex
cd ..
if \[ "$1" = "-n" \]; then
    rm -rf /private/var/folders/w2/*/T/deepchem* || true
    cd "$(mktemp -d -t deepchem)"
    curl https://github.com/conda-forge/miniforge/releases/download/4.12.0-0/miniforge3-`uname`-`uname -m`.sh -L -O
	chmod +x ./miniforge3-$(uname)-$(uname -m).sh
	zsh ./miniforge3-`uname`-`uname -m`.sh -b -f
fi
conda init
source ~/miniforge3/etc/profile.d/conda.sh
export SDKROOT=`xcrun --show-sdk-path`
conda create --name venv python=3.9 -y
conda activate venv
conda clean --all -y
pip3 cache purge
conda install -c conda-forge rust pdbfixer mdtraj -y
conda install pymatgen -y
conda install -c anaconda pip h5py cython pkgconfig -y
conda install -c apple tensorflow-deps --force-reinstall -y
pip3 install vina --global-option=build_ext --global-option="-L/$(brew install boost || brew --cellar boost)" --global-option="-L/$(brew install boost-python3 || brew --cellar boost-python3)"
pip3 install joblib pandas scikit-learn scipy rdkit-pypi flaky optax dm-haiku biopython
pip3 install --no-binary :all: numpy
pip3 install install transformers
conda install -c pytorch pytorch torchvision -y
conda install -c pytest pytest -y
conda install -c anaconda hdf5 swig -y
pip3 install tensorflow-macos --force-reinstall
pip3 install tensorflow-metal --force-reinstall
which python3
python -m pip install -e git+https:////github.com/deepchem/deepchem.git#egg=deepchem --force-reinstall
python -m pytest src/deepchem/deepchem/models/tests/ -k tensorflow
ls .
#4

Ok for some reason now my script is not working as of that latest version. I think that removing brew install hdf5 swig messed up my installation. So if the second one doesn’t work, please try the first.

#5 
#!/bin/zsh
set -ex
cd ..
if \[ "$1" = "-n" \]; then
    rm -rf /private/var/folders/w2/*/T/deepchem* || true
    cd "$(mktemp -d -t deepchem)"
    curl https://github.com/conda-forge/miniforge/releases/download/4.12.0-0/miniforge3-`uname`-`uname -m`.sh -L -O
	chmod +x ./miniforge3-$(uname)-$(uname -m).sh
	zsh ./miniforge3-`uname`-`uname -m`.sh -b -f
fi
conda init
source ~/miniforge3/etc/profile.d/conda.sh
export SDKROOT=`xcrun --show-sdk-path`
conda create --name venv python=3.9 -y
conda activate venv
conda clean --all -y
pip3 cache purge
conda install -c conda-forge rust pdbfixer mdtraj -y
conda install pymatgen -y
conda install -c anaconda pip cython pkgconfig -y
conda install -c apple tensorflow-deps --force-reinstall -y
brew install cmake hdf5 swig
git clone --recurse-submodules https://github.com/dmlc/dgl.git || true
cd dgl
mkdir build
cd build
export MACOSX_DEPLOYMENT_TARGET=10.14
cmake -DUSE_OPENMP=off -DCMAKE_APPLE_SILICON_PROCESSOR=arm64 -DCMAKE_C_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DCMAKE_CXX_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DUSE_AVX=OFF -DUSE_LIBXSMM=OFF ..
make -j4
cd ../python
python setup.py develop --no-deps
cd ..
pip3 install tutorials/requirements.txt
pip3 install vina --global-option=build_ext --global-option="-L/$(brew install boost || brew --cellar boost)" --global-option="-L/$(brew install boost-python3 || brew --cellar boost-python3)"
pip3 install joblib pandas scikit-learn scipy rdkit-pypi flaky optax dm-haiku biopython
pip3 install --no-binary :all: numpy
pip3 install install transformers
conda install -c pytorch pytorch torchvision -y
which python3
python -m pip install -e git+https:////github.com/deepchem/deepchem.git#egg=deepchem --force-reinstall
CPATH=`brew --prefix` SDKROOT=`xcrun --show-sdk-path` conda install -c pytest pytest hdf5 h5py -y
CPATH=`brew --prefix` SDKROOT=`xcrun --show-sdk-path` pip3 install tensorflow-macos --force-reinstall
CPATH=`brew --prefix` SDKROOT=`xcrun --show-sdk-path` pip3 install tensorflow-metal --force-reinstall
python -m pytest src/deepchem/deepchem/models/tests/ -k tensorflow
ls .

This is an updated version that also builds dgl. In addition to restoring homebrew hdf5 and swig.

#6

Boost is also available from conda.

#7

Hi,
That is great that is available in conda as well. However, does conda have the brew --prefix functionality? I ran into issues trying to install hdf5 with conda, and I’m inclined to believe it will be the same with this as we are leveraging the C header files. Please post your current working build script, and I will integrate the changes.