I'm writing python classes in a package using VScode. I want to add this package's parent directory to the Python path when using the VScode Terminal so I can import the package (regardless of the directory of the file that's being run).
I've tried a .env file and Workspace settings without success.
Ultimately I want to run doctests on the classes using the terminal, and for that the terminal needs to be able import the package.
.env File
I have a Workspace. I first tried adding a .env file as follows (note I'm using Python 3 Anaconda on Windows):
PYTHONPATH=C:\\MyPython;${PYTHONPATH}
(I've tried single & double back slashes and forward slashes, nothing works).
When I run a script (test.py) in the terminal containing this:
print(os.environ.get('PYTHONPATH'))
I just get back None.
I did try setting a system wide PYTHONPATH environment variable in Windows, which then shows up, but the C:\MyPython is not added to it. I don't want to have to add/change the system PYTHONPATH every time I open a different Workspace!
Workspace Settings
I then tried adding Workspace settings in the MyProject.code-workspace file:
{
"folders": [
{
"path": "C:\\MyPython"
}
],
"settings": {
"terminal.integrated.cwd": "C:\\MyPython",
"terminal.integrated.env.windows": "C:\\MyPython"
}
}
Again this didn't work.
File Structure
My file structure is as follows:
C:\MyPython
.env
MyProject.code-workspace
test.py
Pkg\
__init__.py
Class1.py
Class2.py
If I use the green triangle button ("Run File in Python Terminal") to run test.py then that file's directory (C:\MyPython) gets added to sys.path and everything works (e.g. import Pkg.Class1 works).
However if I run Class2.py (which includes import Pkg.Class1 in the code and doctest) then instead the directory C:\MyPython\Pkg gets added to sys.path and it can't find and import the Pkg package and the doctest fails.
As such I want to add C:\MyPython to the python path, regardless of the directory of the file that is being run.
Of course, I could just add all tests to test.py and run that, but I really just want to run the doctests in the class I'm working on, rather than have to run all the tests every time (and flip to another file to do it).
It seems like this should be easy, but I can't get it to work!
Any ideas?
PS: I haven't included the .py code for the test or classes since it's irrelevant to the problem, it's the import that fails. Ultimately I can see that the required directory appears neither in sys.path nor in os.environ.get('PYTHONPATH') and that's why the import fails.
So two things. One, Python is not designed for you to execute files contained within packages, so you're somewhat going against its design trying to make this work. It would be better to do something like use Python's -m flag to do what you want, e.g. python -m Pkg.Class2. That way you don't have to manipulate your paths to run a module contained in your package.
Two, you were very close in getting what you were after with your settings, but you accidentally used a string instead of an object for specifying your environment variables for your terminal. What you wanted was:
"terminal.integrated.env.windows": {"PYTHONPATH": "C:\\MyPython"}
Related
I have a python3 script that I am calling in terminal; I do not use Python prefix to run it, since I did add #!/usr/local/bin/python3 in my script (I have python3 from brew, on OSX).
The interesting thing is that if I run the script in terminal, I get an import error because one of my custom module hasn't been found. If I run the same script in pycharm, it works fine.
I assume Python launch and read all the various path that I use for modules in the same way, in both pycharm and terminal, but it seems not the case. How do I set up my scripts so the modules are found, independently from their path?
I may run the same script from other machines too, so I want to be prepared and do the right thing from the start.
EDIT
I am running pycharm on OSX; Python3 is installed via Brew, but the symlink is in /usr/local/bin.
My script is running from a folder inside my home directory, so
/Users/tester/git/python_test_app/main/base/app_main.py
The custom modules are in the same folder of the main py script, but one level above: /Users/tester/git/python_test_app/main/pyutils.py
The import statement from app_main.py is
import main.pyutils as utilities
This is the stack trace that I get when running the script:
Traceback (most recent call last):
File "main/base/app_main.py", line 13, in <module>
import main.pyutils as utilities
ModuleNotFoundError: No module named 'main'
EDIT 2 and solution
Thanks to The answers, I was able to figure out that the issue is related to how Pycharm handle projects. Basically it somehow save the path where the project is; so calling an import will result in the project folder being parsed, and that's why it works fine from Pycharm.
In Python, unless PYTHONPATH has the path to my project or other modules that I wrote, it won't be able to find them, hence, raise the error.
FIX:
in my main module that I use to run the application, I did retrieve the path of the file; which I know being one level below the modules I need; so I can explicitly add the folder to the current sys.path. This will end up making possible for me to call the import successfully.
import sys
current_dir = os.path.dirname(__file__)
sys.path.insert(0, , current_dir)
The only downside is that every file and resource that I use in my project, has to be directly referred by full path; so I have to pass the current_dir around the various files in the project.
PyCharm has project interpreter settings. Verify these are the same as your system Python. Go to:
File menu
Settings
Project: <project name>
Project Interpreter
View the path to the Python executable/binary being used by the project in PyCharm and verify it matches what your system is calling (e.g., which python3)
Alternatively, it may be that you declared your sources root within PyCharm and the system cannot properly run the module as it exists in the path you're running it from (especially if inside a package). You can get around this using the -m parameter and calling it from Python.
You can also try running it from the Terminal inside PyCharm and see what it adds to the path before initializing the shell session (you can sometimes see this in your Run configurations also). If you are referring to modules not installed via pip / into the Python path but rather loaded into your project path, then this may be the culprit.
On PyCharm, next to the green "RUN" arrow press the box and then press edit configurations (see image)
There you'll have Working Directory - that path is where PyCharm is running that script from (without errors).
Try running it from the terminal within that path - that should solve your import errors.
I recently figured out how to import modules for unittesting in python. As a solution to this, I use:
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from Dev.test import someclass
This works fine while running in PyCharm and I get the expected output. However, when I run from terminal I run into an error:
ImportError: No module named Dev.test
I have the init files where they are supposed to be but I'm lost as to why this is working in PyCharm but not from the terminal. I have not changed my path or anything in PyCharm as this code is supposed to be able to run with minimal modifications on other machines. Any idea as to why this is happening and what I might be able to do to fix it?
My folder structure is as follows
-Current
-Dev
-__init__.py
-test
- __init__.py
-someclass.py
-Tests
-__init__.py
-someunittest.py
I have tried running someunittest from the main folder as well as with a complete path but it only works in PyCharm
sys.path.append(os.getcwd()[:os.getcwd().index('Dev')])
I added this to my imports and it seems to have solved the problem. However, this doesn't seem like it would be the right way to do it; it will do for now.
When running a script from within PyCharm, it runs it in an environment with PYTHONPATH set to the list of all the folders that are marked "Sources Root" (with a blue folder icon) in the project explorer.
Outside of PyCharm, PYTHONPATH is not normally set. The first entry in sys.path refers to the current working directory where the script was run from. As long as you run your script with your terminal's working directory as the folder containing Dev, it should be able to find the Dev.test module, regardless of the extra entry added to sys.path.
Once you get the working directory correct, you should be able to remove the sys.path hack.
What #codewarrior has said about the PyCharm setting its own PYTHONPATH is correct. But sys.path didn't have my current working directory. So to get around this problem, I updated my PYTHONPATH (or you can edit sys.path).
Setting PYTHONPATH
export PYTHONPATH=$PYTHONPATH:`pwd` (OR your project root directory)
Updating sys.path
import sys
sys.path.insert(0,'<project directory>') OR
sys.path.append('<project directory>')
You can use insert/append based on the order in which you want your project to be searched.
HTH.
Pycharm uses a virtual environment. When you try run your program in your terminal this enviroment isn't active.
You need to build or upload your enviroment Pycharm with your libraries.
cd to the directory project and write in terminal:
source venv/bin/activate
I too have had this issue - and the PYTHONPATH setting set by PyCharm did seem to be the issue.
My alternative (as I was nearly finished writing the code) was to generate a setup.py - and install the classes/structure in my local virtual Python environment.
I would recommend trying out $ pip install . in your source directory. This will install your own packages for your project.
To add to similar answers here, PyCharm is doing some extra config for you before running your script. If adding your sources root to PYTHONPATH doesn't work then examine your run configuration in PyCharm for the script in question, there will likely be some more behind the scenes magic at play.
I had similar problem. I think the problem is that Pycharm modifies PYTHONPATH so before running your script:
cd to the file where python file resides
run export PYTHONPATH=.
run the script
You can also create "main" python file where you set the python path and then call the other modules
Up to this point I have organized my projects in such a way that everything I'm working on is in the same folder, so to play around/debug I have just launched Python from that folder like this:
C:\Users\Username\Dropbox\Projects\MyShinyProject>Python
>>>
However, I want to start organizing things better. I have created some "Utilities" classes that I expect I'll use over and over again. So they should be in their own folder.
So now, say I have a Projects folder (in Windows) with lots of subfolders of things I have been working on:
Projects
Sandbox
Sandbox1
Sandbox2
MyUtilities
__init__.py
Utility1.py
MyShinyProject
__init__.py
ImportantClass.py
I would like to be able to go into the command prompt and use classes/functions from both the MyUtilities folder and from the MyShinyProject folder. However, if I launch Python from inside MyShinyProject, I don't have access to MyUtilities (or vice versa). I've tried doing a relative import like this:
>>>import ..MyUtilities.Utility1
But that doesn't work:
import ..MyUtilities.Utility1
^
SyntaxError
If it matters: I don't use an IDE. I just use Notepad++ and the command prompt. Also, I added the __init__.py files to the folders because I read somewhere you're supposed to do that when you make modules, but I don't understand how to get all of this working correctly, or if I'm even close to doing it right.
I also tried adding my Projects folder to the PATH variable in the Windows environment table, but that doesn't seem to work. Even after adding it importing doesn't work, and when I do this:
import sys
for x in sys.path:
print(x)
...the folder I added to PATH does not appear (I tried adding it to the beginning and the end).
How can I use several of my user created modules together using the command prompt to import them?
Assuming you have __init__.py in your Projects folder, in the console you can do this:
import sys
sys.path.append("C:\Users\Username\Dropbox\Projects")
import Projects.MyUtilities.Utility1
Or if you want to add your desired directory permanently to the python path, you can append your directory to the value of the environment variable called PYTHONPATH.
I have an exercise for a course in Python I have to complete. I have saved my methods/defs in one file and just need to figure out how to run it. I was hoping you could explain to me how to import files (I know the syntax"import filename"). When ever I do this I get an error. How do I change the file path of the import to the file on my desktop? I am using a mac and running IDLE 2.7.3
If the files are in the same directory as that file you can just use
import <filename> #(without the <>)
However, if you are referring to the files in a separate directory use imp
import imp
module = imp.load_source('module.name', '/path/to/file.py')
module.SomeClass()
On Mac OS X, there are two basic ways to launch IDLE. One is by double-clicking on an IDLE icon from the Applications folder in the Finder. (I'll call that IDLE.app) The second is to launch IDLE from a terminal session shell with something like:
idle2.7
I'll call that bin/idle because it refers to file in one of your system's bin directories.
When launching bin/idle, IDLE will inherit the current working directory of the shell and will add that directory to the front of the list of directories Python searches for imports. You can examine that list of directories in the IDLE shell window:
import sys
sys.path
When you launch IDLE.app, however, it is not associated with any shell so there is no opportunity to tell IDLE which working directory to use. In this case, IDLE.app uses your Documents folder/directory as its working directory: in shell notation, ~/Documents, also spelled /Users/your_login_name/Documents.
You can manually manipulate sys.path in your Python program to add another directory for imports:
import sys
sys.path.insert(0, '/path/to/directory')
Another way to manipulate sys.path is to use the PYTHONPATH environment variable. But, again, when using IDLE.app, there is no (easy) way to pass environment variables to it, unlike with bin/idle
So if you want to use IDLE.app, the simplest approach is to put into your Documents directory all of the files and packages directories that you want to be able to import. Otherwise, use a terminal session, set the desired working directory and/or PYTHONPATH variable and launch bin/idle. So something like:
cd ~/MyProject
export PYTHONPATH=~/"AnotherPackageDirectory"
idle2.7
Yet another approach is to install your modules in the default locations that Python searches and use Distutils or easy_install or pip. But that's something to learn about later when you have a finished project.
I am pretty new to Python. Currently I am trying out PyCharm and I am encountering some weird behavior that I can't explain when I run tests.
The project I am currently working on is located in a folder called PythonPlayground. This folder contains some subdirectories. Every folder contains a init.py file. Some of the folders contain nosetest tests.
When I run the tests with the nosetest runner from the command line inside the project directory, I have to put "PythonPlayground" in front of all my local imports. E.g. when importing the module called "model" in the folder "ui" I have to import it like this:
from PythonPlayground.ui.model import *
But when I run the tests from inside Pycharm, I have to remove the leading "PythonPlayground" again, otherwise the tests don't work. Like this:
from ui.model import *
I am also trying out the mock framework, and for some reason this framework always needs the complete name of the module (including "PythonPlayground"). It doesn't matter whether I run the tests from command line or from inside PyCharm:
with patch('PythonPlayground.ui.models.User') as mock:
Could somebody explain the difference in behavior to me? And what is the correct behavior?
I think it happens because PyCharm have its own "copy" of interpreter which have its own version of sys paths where you project's root set to one level lower the PythonPlayground dir.
And you could find preferences of interpreter in PyCharm fro your project and set proper top level.
ps. I have same problems but in Eclipse + pydev