Develop Reference

Custom modules are not found when calling python script in console VS pycharm

I have a python3 script that I am calling in terminal; I do not use Python prefix to run it, since I did add #!/usr/local/bin/python3 in my script (I have python3 from brew, on OSX).
The interesting thing is that if I run the script in terminal, I get an import error because one of my custom module hasn't been found. If I run the same script in pycharm, it works fine.
I assume Python launch and read all the various path that I use for modules in the same way, in both pycharm and terminal, but it seems not the case. How do I set up my scripts so the modules are found, independently from their path?
I may run the same script from other machines too, so I want to be prepared and do the right thing from the start.
EDIT
I am running pycharm on OSX; Python3 is installed via Brew, but the symlink is in /usr/local/bin.
My script is running from a folder inside my home directory, so
/Users/tester/git/python_test_app/main/base/app_main.py
The custom modules are in the same folder of the main py script, but one level above: /Users/tester/git/python_test_app/main/pyutils.py
The import statement from app_main.py is
import main.pyutils as utilities
This is the stack trace that I get when running the script:
Traceback (most recent call last):
File "main/base/app_main.py", line 13, in <module>
import main.pyutils as utilities
ModuleNotFoundError: No module named 'main'
EDIT 2 and solution
Thanks to The answers, I was able to figure out that the issue is related to how Pycharm handle projects. Basically it somehow save the path where the project is; so calling an import will result in the project folder being parsed, and that's why it works fine from Pycharm.
In Python, unless PYTHONPATH has the path to my project or other modules that I wrote, it won't be able to find them, hence, raise the error.
FIX:
in my main module that I use to run the application, I did retrieve the path of the file; which I know being one level below the modules I need; so I can explicitly add the folder to the current sys.path. This will end up making possible for me to call the import successfully.
import sys
current_dir = os.path.dirname(__file__)
sys.path.insert(0, , current_dir)
The only downside is that every file and resource that I use in my project, has to be directly referred by full path; so I have to pass the current_dir around the various files in the project.

PyCharm has project interpreter settings. Verify these are the same as your system Python. Go to:
File menu
Settings
Project: <project name>
Project Interpreter
View the path to the Python executable/binary being used by the project in PyCharm and verify it matches what your system is calling (e.g., which python3)
Alternatively, it may be that you declared your sources root within PyCharm and the system cannot properly run the module as it exists in the path you're running it from (especially if inside a package). You can get around this using the -m parameter and calling it from Python.
You can also try running it from the Terminal inside PyCharm and see what it adds to the path before initializing the shell session (you can sometimes see this in your Run configurations also). If you are referring to modules not installed via pip / into the Python path but rather loaded into your project path, then this may be the culprit.

On PyCharm, next to the green "RUN" arrow press the box and then press edit configurations (see image)
There you'll have Working Directory - that path is where PyCharm is running that script from (without errors).
Try running it from the terminal within that path - that should solve your import errors.

in IntelliJ, showing "No module named xxx", but "xxx" is actually installed in my system

My IntelliJ version is 15.0.3, and have python plugin installed.
And when I open a python file in IntelliJ it's like below
Situation here is like below:
when I import these flagged modules in terminal, everything works fine.
running this python file in IntelliJ, is also fine
It's only the red underlying warning annoying me.
I tried this one, this one and this one, but none works for me.
Could anyone please tell me how to get rid of it? Thanks a lot.

You have to add the site-packages path of your interpreter.
For that you have to:
go to Project Structure
choose Global Libraries
choose your Python interpreter
press + at the upper left corner
choose the site-package path of your interpreter
choose "Classes"
also add your python interpreter root directory in the same way.

I have similar problems: cannot import manually installed module. I try to add classpath in Mac environment parameters and intellij SDKs, but it doesn't work. My final solution is to add the classpath of the module to Run/Debug Configurations:
Open Run/Debug Configurations and select your unittest class:
Run --> Edit Configurations...
Add your module's classpath to Environment variables:

Cannot Import MySQLdb in PyCharm, runs fine in terminal.

I have problems trying to run a script which imports MySQLdb within PyCharm.
Running the script from terminal works just fine while running within PyCharm fails with
ImportError: No module named MySQLdb
I have tried this thread and it helped making things work in the terminal.
Trying to set the environment variables in the IDE though does not seem to work.
In PyCharm Run Config I set the environment variables
DYLD_LIBRARY_PATH - /usr/local/mysql/lib/libmysqlclient.18.dylib
PATH - /usr/local/mysql/lib/
but I still get the ImportError.

As #Dilettant pointed me to https://stackoverflow.com/a/34992894/1989141,
I realised there are two different places I was supposed to set the Python interpreter in PyCharm.
The first is in the main preferences (as in phil’s solution in the link) and allows me to specify a path to a local folder for mysqldb module.
The second is in the edit configuration settings for the script I want to run. I noticed the interpreters were different.
The script was set to run with the Python 2.7.6 /System/Library/Frameworks/...... version (which I believe is the OSx pre-installed version).
In the terminal and in PyCharm general settings I am using Python 2.7.9 in /usr/local/bin/python.
Matching the script interpreter to the one in the main settings (for which I manyally added mysqldb folder) solves the ImportError. Also I removed the environment variables I set up in the Edit Configuration as they are not needed.
Hope this helps. Thanks to both #Dilettant and #julivico for their suggestions.

Python works in PyCharm but not from terminal

I recently figured out how to import modules for unittesting in python. As a solution to this, I use:
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from Dev.test import someclass
This works fine while running in PyCharm and I get the expected output. However, when I run from terminal I run into an error:
ImportError: No module named Dev.test
I have the init files where they are supposed to be but I'm lost as to why this is working in PyCharm but not from the terminal. I have not changed my path or anything in PyCharm as this code is supposed to be able to run with minimal modifications on other machines. Any idea as to why this is happening and what I might be able to do to fix it?
My folder structure is as follows
-Current
-Dev
-__init__.py
-test
- __init__.py
-someclass.py
-Tests
-__init__.py
-someunittest.py
I have tried running someunittest from the main folder as well as with a complete path but it only works in PyCharm

sys.path.append(os.getcwd()[:os.getcwd().index('Dev')])
I added this to my imports and it seems to have solved the problem. However, this doesn't seem like it would be the right way to do it; it will do for now.

When running a script from within PyCharm, it runs it in an environment with PYTHONPATH set to the list of all the folders that are marked "Sources Root" (with a blue folder icon) in the project explorer.
Outside of PyCharm, PYTHONPATH is not normally set. The first entry in sys.path refers to the current working directory where the script was run from. As long as you run your script with your terminal's working directory as the folder containing Dev, it should be able to find the Dev.test module, regardless of the extra entry added to sys.path.
Once you get the working directory correct, you should be able to remove the sys.path hack.

What #codewarrior has said about the PyCharm setting its own PYTHONPATH is correct. But sys.path didn't have my current working directory. So to get around this problem, I updated my PYTHONPATH (or you can edit sys.path).
Setting PYTHONPATH
export PYTHONPATH=$PYTHONPATH:`pwd` (OR your project root directory)
Updating sys.path
import sys
sys.path.insert(0,'<project directory>') OR
sys.path.append('<project directory>')
You can use insert/append based on the order in which you want your project to be searched.
HTH.

Pycharm uses a virtual environment. When you try run your program in your terminal this enviroment isn't active.
You need to build or upload your enviroment Pycharm with your libraries.
cd to the directory project and write in terminal:
source venv/bin/activate

I too have had this issue - and the PYTHONPATH setting set by PyCharm did seem to be the issue.
My alternative (as I was nearly finished writing the code) was to generate a setup.py - and install the classes/structure in my local virtual Python environment.

I would recommend trying out $ pip install . in your source directory. This will install your own packages for your project.

To add to similar answers here, PyCharm is doing some extra config for you before running your script. If adding your sources root to PYTHONPATH doesn't work then examine your run configuration in PyCharm for the script in question, there will likely be some more behind the scenes magic at play.

I had similar problem. I think the problem is that Pycharm modifies PYTHONPATH so before running your script:
cd to the file where python file resides
run export PYTHONPATH=.
run the script
You can also create "main" python file where you set the python path and then call the other modules

Pydev, eclipse and pythonpath problem

I've installed pydev to my eclipse 3.5.2. Everything was working smoothly, create projects, execute, test, autocomplete.
But then I realized that importing modules from /usr/lib/pymodules/python2.6, such as django, causes error "Unresolved import: xxxx". Of course, PYTHONPATH SYSTEM includes the directories I want. What's more, inside package explorer i can c the modules under "System Libs".
I just can't import them :S. Is this a bug? Or I just missing something.
Thanks.

In eclipse you can add django folder in you python path.
Window->Preferences-> PyDev-> Interpreters->Python Interpreter -> Lirararies -> New Folder
And browse till the parent folder of modules you are searching.

If you're using virtualenv you should setup an interpreter using the python build inside.
ie., default python interpreter for th project will be /usr/bin/python
but change it to something like "{project name} python" and point it to your virtual env path. In my case it's ~/.virtualenvs/acme/bin/python

It seems like some sort of cache issue in PyDev... in which case you could try to remove the interpreter, add it again and restart Eclipse.