It is indeed a frustrating step for Machine Learning researchers to setup their development environment. Sometimes we need to spin-up a fresh machine to start our work. There is always a dependency hell, you have to track each and every dependency in the hierarchy and install then in order on your bare bone machine. I developed a project on GitHub which automates all the works you need to get done to have a basic deep learning machine. Bash scripts are added for tasks such as installing CUDA and cuDNN, and TensorFlow on top of them. You can also build and install TensorFlow from source code.
Please note that more scripts for other versions of Ubuntu, CUDA, cuNN, and other deep learning frameworks such as PyTorch will be coming soon. I am also working setting up the same environments on docker containers.
- Ubuntu 16.04 installed (automated installation for newer version of Ubuntu will be coming soon)
- Nvidia compatible GPU
- Ubuntu 16.04 virtual-machine on Google Cloud Platform
- Nvidia Tesla K80 GPU
- Python 3.5
List of bash scripts in the project:
# Automatically installs GPU driver, CUDA 9.0, cuDNN 7.0, and Python packages
# This is useful when you want to install other deep learning framework or a specific version on top of CUDA and cuDNN basic_deeplearning_machine.sh
# Automatically installs TensorFlow on top of CUDA and cuDNN (Python packages are included)
# Automatically builds TensorFlow from source (Python packages are included)
1. Clone/download this project on your machine (or remote virtual machine) 2. Run the following command in the command line to make any of the scripts executable
chmod +x [script-name].sh
3.Run the following commmand to execute the bash script
Note: If you choose to run the first two scripts, there is no need for user interaction. They will install CUDA, cuDNN, and/or Compiled Tensorflow,and Jupyter Notebook automatically.
4. “cuda_with_tensorflow_building_from_source.sh” asks the user for TensorFlow configuration before the compilation starts.
- Install bazel to compile TensorFlow => in this step you will be asked to accept the terms and condition for JDK.
While installing Java to instal Bazel you will be promted to accept terms and conditions.
Hit enter to accept the License terms.
Select Yes and hit enter to accept Oracle binary code License terms.
- Building TensorFlow from source. => in this step you will be asked to set different parameters in TensofFlow (“./configure”) before starting the compilation.
Before TensorFlow compilation gets started, you will be prompted to set the configuration (add or remove of supporting of other frameworks). This could effect the compilation time.
1- you will be asked to specify the location of Python on your machine.
on most machines: /usr/bin/python or /usr/bin/python3.5 or etc
2- then you will be asked to specifiy the path to python libraries.
default is: /usr/local/lib/python3.5/dist-packages
3- After you specify that, you will be asked to choose for your TensorFlow to support or do not support many FrameWorks and Services.
For more information visit TensorFlow installation Page here: TensorFlow
EXAMPLE: (If you hit enter you will choose the default values.)
4– Make sure you choose the right version of CUDA and cuDNN.
5- Then you will be asked to choose NCCL version. Since, V-2.2 is not installed. Choose V-1.3.5 or google how to install V-2.2. The latest version contains performance improvement on multiple GPU jobs.
6- After you finished step-5 the compilation will start. It could take about few minutes to a couple of hours depending on how fast is your CPU, RAM, and hard drive.
7- Then our bash script will install TensorFlow for Python3.
8- If our bash script ran successfully, we should see the version of the TensorFlow installed on the screen.
Congratulations! Your machine is now ready for your Machine Learning research or building your product.
Author: Alireza Darbehani — Machine Learning Engineer
Follow me @ Republic of AI:
or my personal LinkedIn: