Path problems are some of the most common and annoying problems machine learning engineers face, especially when frequently switching between operating systems. There are so many different ways to have path problems that no post could cover them all, but in this post, I’ll try to provide some background on possible issues and how to resolve them.

Table of Contents

• System Path
  • Viewing Your Path
  • Adding to Your Path
• Python Path
  • Viewing your Python Path
    • Python
  • Adding to your PYTHONPATH
  • Finding your Python Interpreter
  • Jupyter Notebook problems
  • Other

The first thing you need to realize is that there are multiple different things that are at times called a “path”. The two main ones that Python programmers will run into are the system path and the Python path.

System Path

The first step is being able to find out what’s on your path. You can do this by looking at the $PATH environmental variable. You can do this from the command line, but the exact command depends on which operation system you’re using.

Viewing Your Path

In Windows, it’s not as simple to view your environmental variables because it depends on what terminal emulator you’re using. If you are using ConEmu or Cmder, you can

echo %PATH%

Again, the resulting text can be hard to read so to make it easier you can: echo %PATH:;=&echo.%

However, if you’re using Windows PowerShell, you’ll have to:

echo $Env:PATH

Windows PowerShell is the default terminal in VSCode, so this is what you’ll need to use there as well.

Adding to Your Path

If you want to temporarily add to your path:

set PATH="%PATH%;C:\path\to\directory\"

If you want to permanently add to your path: setx path "%PATH%;C:\path\to\directory\"

Python Path

If you can’t import a module you wrote, it’s probably because that location is missing from your path. But it’s your Python path, not your system path, that it’s missing from.

import my_module
ModuleNotFoundError: No module named 'my_module'

Viewing your Python Path

On Windows, if you echo $PYTHONPATH you get nothing, but you can use Python to see what’s in your path

Python

When inside Python you can print(sys.path) to see your path:

['C:\\Users\\Julius\\Documents\\GitHub\\ResNetFromScratch', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\python37.zip', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\DLLs', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\lib', 'C:\\Users\\Julius\\anaconda3\\envs\\tf', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\lib\\site-packages', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\lib\\site-packages\\win32', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\lib\\site-packages\\win32\\lib', 'C:\\Users\\Julius\\anaconda3\\envs\\tf\\lib\\site-packages\\Pythonwin']

Adding to your PYTHONPATH

To permanently add to your PYTHONPATH on Windows:

1. Press the Windows key and type “environment”
2. Click on “Environment Variables” in the Advanced tab of System Properties
3. Under User variables, click “New” and add your desired path

You can test it from the command line:

python -c "import sys; print(sys.path)"

Remember that your paths are hierarchical on Windows. If C:\Users\Julius\AppData\Local\Microsoft\WindowsApps appears before your Python path, Windows will try that first and may redirect you to the Windows Store. You can reorder the entries to move the one you want higher in the list.

If you’re using Conda, make sure to add both the Conda and Python directories to your path. You can find them by opening a Conda prompt and running where conda and where python.

To temporarily set PYTHONPATH from the command line:

set PYTHONPATH=C:\path\to\my_project\src
python -c "import sys; print(sys.path)"

Note: If you change the environment variables you’ll need to restart VSCode or Jupyter Notebooks. Once you restart you’ll see the addition in sys.path.

Finding your Python Interpreter

From within Python, you can do:

import sys
sys.executable

Jupyter Notebook problems

Let’s say you set your variable and can see it in PYTHONPATH.

You run import sys; print(sys.path) from a command line and it’s there, but when you run the same command in a Jupyter Notebook it’s not there. That’s because Jupyter has its own PATH variable called JUPYTER_PATH.

If you need to quickly add to it, you can start a notebook with:

import sys
sys.path.append('C:\\Users\\Julius\\Documents\\GitHub\\cv_dataclass\\src')

Other

If you want to know more about how the Python sys.path is initialized, there’s a detailed StackOverflow answer.