ModuleNotFoundError: No module named ‘torch’ for OneAPI AI Toolkit

If you are using OneAPI Environment, and if you are having this issue

ModuleNotFoundError: No module named 'torch'

Here are some steps, you may wish to use to troubleshoot.

Make sure you activated the oneAPI environment using below command

% source /usr/local/intel/oneapi/2021.3/setvars.sh

:: initializing oneAPI environment ...
   -bash: BASH_VERSION = 4.2.46(2)-release
:: clck -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: inspector -- latest
:: intelpython -- latest
:: ipp -- latest
:: itac -- latest
:: LPOT -- latest
:: mkl -- latest
:: modelzoo -- latest
:: mpi -- latest
:: pytorch -- latest
:: tbb -- latest
:: tensorflow -- latest
:: oneAPI environment initialized ::

You might want to check the conda environment

% conda info --envs

# conda environments:
#
myenv                    /myhome/melvin/.conda/envs/myenv
myfsl                    /myhome/melvin/.conda/envs/myfsl
base                  *  /usr/local/intel/oneapi/2021.3/intelpython/latest
2021.3.0                 /usr/local/intel/oneapi/2021.3/intelpython/latest/envs/2021.3.0
myoneapi                 /usr/local/intel/oneapi/2021.3/intelpython/latest/envs/myoneapi
pytorch                  /usr/local/intel/oneapi/2021.3/intelpython/latest/envs/pytorch
pytorch-1.8.0            /usr/local/intel/oneapi/2021.3/intelpython/latest/envs/pytorch-1.8.0
tensorflow               /usr/local/intel/oneapi/2021.3/intelpython/latest/envs/tensorflow
tensorflow-2.5.0         /usr/local/intel/oneapi/2021.3/intelpython/latest/envs/tensorflow-2.5.0
                         /usr/local/intel/oneapi/2021.3/pytorch/1.8.0
                         /usr/local/intel/oneapi/2021.3/tensorflow/2.5.0

Activate Pytorch

% conda activate pytorch
% python
% (pytorch-1.8.0) [user1@node1 ~]$ python
Python 3.7.10 (default, Jun  4 2021, 06:52:02)
[GCC 9.3.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch

If you are still having the error “ModuleNotFoundError: No module named ‘torch’ “

You may want to install directly if you have root access

% conda install pytorch torchvision cpuonly -c pytorch

If not, you may want to create a private environment similar to Creating Virtual Environment with Python using venv

References:

Python [Errno 13] Permission denied

Issues

If you are facing issues like this when using python libraries like “queue = multiproccesing.Queue()”, you may face this issue

Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/intel/2020/intelpython3/lib/python3.7/multiprocessing/context.py", line 102, in Queue
return Queue(maxsize, ctx=self.get_context())
File "/usr/local/intel/2020/intelpython3/lib/python3.7/multiprocessing/queues.py", line 42, in __init__
self._rlock = ctx.Lock()
File "/usr/local/intel/2020/intelpython3/lib/python3.7/multiprocessing/context.py", line 67, in Lock
return Lock(ctx=self.get_context())
File "/usr/local/intel/2020/intelpython3/lib/python3.7/multiprocessing/synchronize.py", line 162, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/usr/local/intel/2020/intelpython3/lib/python3.7/multiprocessing/synchronize.py", line 59, in __init__
unlink_now)
PermissionError: [Errno 13] Permission denied

When executing the code with root privilege, it was working fine, but a normal user doesn’t have permission to access shared memory.

Resolution:

You can counter-check the issue by checking /dev/shm

% ls -ld /dev/shm

Change Permission to 777

% chmod 777 /dev/shm

Turn on the sticky bit

% chmod +t /dev/shm
% ls -ld /dev/shm
drwxrwxrwt 4 root root 520 Mar 5 13:32 /dev/shm

Encountering Error when pip install TensorToolbox

I’m using Python-3.8.7¬† When I do a pip install for TensorFlowbox with Intel Optimized Toolbox, I received errors.

% pip install TensorFlowbox

But it failed with its SpectralToolbox Dependencies.

.....
.....

Building wheels for collected packages: SpectralToolbox, orthpol-light
Building wheel for SpectralToolbox (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/python/intel/2017u3/intelpython3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-p5qjlor0/spectraltoolbox/setup.py'"'"'; __file__='"'"'/tmp/pip-install-p5qjlor0/spectraltoolbox/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-4k6ars4d
cwd: /tmp/pip-install-p5qjlor0/spectraltoolbox/
Complete output (56 lines):

Somehow the later version of Python3 has issues with SpectralToolbox and TensorToolbox. To compile TensorToolbox, you have to go back to earlier version of Python 3. I chose Python-3.6.9 (https://www.python.org/downloads/release/python-369/).

And it works.

Perquisites:

openmpi-3.1.4
gnu-6.5
m4-1.4.18
gmp-6.1.0
mpfr-3.1.4
mpc-1.0.3
isl-0.18
gsl-2.1

 

Compile

% tar -zxvf Python-3.6.9
% cd Python-3.6.9
% ./configure --prefix=/usr/local/python-3.6.9 --enable-optimizations
% make -j 16
% make install

Installing TensorToolbox

% pip install numpy scipy matplotlib
% pip install mpi4py
% pip install TensorToolbox

For more information, see TensorToolbox-1.0.22 (https://pypi.org/project/TensorToolbox/#description)

 

Test Failed… does not support .pth files

If you are doing a setup.py with specific directories

python setup.py install --prefix=/home/user1

If you are getting a PythonPath Error something like this.

TEST FAILED: /home/user1/lib/python3.7/site-packages/ does NOT support .pth files error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not on PYTHONPATH and which Python does not read ".pth" files from. 
The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was: /home/user1/lib/python3.7/site-packages/

and your PYTHONPATH environment variable currently contains:

You can solve like by putting in your .bashrc.

export PYTHONPATH="${PYTHONPATH}:/home/user1/lib/python3.7/site-packages/"
source ~/.bashrc