I am pretty new to Python. Currently I am trying out PyCharm and I am encountering some weird behavior that I can't explain when I run tests.
The project I am currently working on is located in a folder called PythonPlayground. This folder contains some subdirectories. Every folder contains a init.py file. Some of the folders contain nosetest tests.
When I run the tests with the nosetest runner from the command line inside the project directory, I have to put "PythonPlayground" in front of all my local imports. E.g. when importing the module called "model" in the folder "ui" I have to import it like this:
from PythonPlayground.ui.model import *
But when I run the tests from inside Pycharm, I have to remove the leading "PythonPlayground" again, otherwise the tests don't work. Like this:
from ui.model import *
I am also trying out the mock framework, and for some reason this framework always needs the complete name of the module (including "PythonPlayground"). It doesn't matter whether I run the tests from command line or from inside PyCharm:
with patch('PythonPlayground.ui.models.User') as mock:
Could somebody explain the difference in behavior to me? And what is the correct behavior?
I think it happens because PyCharm have its own "copy" of interpreter which have its own version of sys paths where you project's root set to one level lower the PythonPlayground dir.
And you could find preferences of interpreter in PyCharm fro your project and set proper top level.
ps. I have same problems but in Eclipse + pydev
Related
Within a python package, what is the generally accept workflow for debugging and testing ?
Currently I have a test folder that sits at the same level as the project src folder. I want the test files (unit tests and others) to call the latest developed Python code in the project in the same-level folder. However for this I need to do some things such as adding folders to the path and relative imports to get it to work. But this seems to break for other project developers.
The alternative is to install the code locally using pip install. But this is a hassle to do every time I want to test.
Can someone please recommend a workflow that is safe and efficient for testing within a project. If the first of these is preferred, what should I do regarding imports and path to get the test code to call the local project code in a reliable way that will work for everyone ?
However for this I need to do some things such as adding folders to the path and relative imports to get it to work. But this seems to break for other project developers.
Doing the following, does not break the project AFAIK.
For me the following layout works (see also here):
project
|--src # directory for all of your code
|--test # directory for tests
...
Then I have the following line of code (before importing from src) in each test .py file:
import sys
sys.path.append("./src")
Finally, I execute the tests from the project directory.
EDIT
When using pytest, you can actually move the path-append-statement into your conftest.py file. That way you don't have to add it to each test .py file.
A possible trick is to make the test folder a package by adding an __init__.py file into it:
import sys
import os.path
sys.path.append(os.path.join(os.path.dirname(os.path.dirname(__file__)), 'src')
then assuming the src folder contains a a.py file, you can import it into the test_a.py file from test as simply as import a provided you start the test from the folder containing test as (assuming unittest):
python -m unittest test.test_a
The good news if you follow that pattern is that you can run the full list of tests with
python -m unittest discover
because the magic of discover will find the test package and all test*.py files under it.
Recently I switched computers and re-downloaded my python and pycharm, I tried activating my saved projects on the new computer, but I ran into problem, pycharm doesn't recognize the parent folder of some of the files, and mistakes it for a module.
The folder is called: "Enrichment_extra_stuff", and when I try to import file in that folder, from another file in that folder, it seems like pycharm recognize it, because it shows me all of the files inside it, but when I run the code, I get the error ModuleNotFoundError: No module named 'Enrichment_extra_stuff'.
Also weirdly, when I try doing explict import and just write import fr to import the file fr, then pycharm shows an error but when I run it, it works like it should.
I tried digging a bit on pycharm, but got confused and didn't found anything, my python interpreter version is 3.8 and I program on windows if that helps.
A folder (or better a directory) is not seen as a module, unless you put __init__.py file in it. This could be empty, or it should be the module content. This is Python: a module is either a file, or a directory with __init__.py
The second part is only for Pycharm: PyCharm is created to handle large projects, and often your program is not in the base (root) directory of your project, but in one (or more) subdirectories (e.g. src). So you should explicitly tell PyCharm which are the root directories. So, go to the project structure panel (the panel with the files, usually on left side), go to your "root" directory, and set with right mouse click on your base source directory: on the pop-up menu select Mark directory as, and then select Source Root.
I have a python3 script that I am calling in terminal; I do not use Python prefix to run it, since I did add #!/usr/local/bin/python3 in my script (I have python3 from brew, on OSX).
The interesting thing is that if I run the script in terminal, I get an import error because one of my custom module hasn't been found. If I run the same script in pycharm, it works fine.
I assume Python launch and read all the various path that I use for modules in the same way, in both pycharm and terminal, but it seems not the case. How do I set up my scripts so the modules are found, independently from their path?
I may run the same script from other machines too, so I want to be prepared and do the right thing from the start.
EDIT
I am running pycharm on OSX; Python3 is installed via Brew, but the symlink is in /usr/local/bin.
My script is running from a folder inside my home directory, so
/Users/tester/git/python_test_app/main/base/app_main.py
The custom modules are in the same folder of the main py script, but one level above: /Users/tester/git/python_test_app/main/pyutils.py
The import statement from app_main.py is
import main.pyutils as utilities
This is the stack trace that I get when running the script:
Traceback (most recent call last):
File "main/base/app_main.py", line 13, in <module>
import main.pyutils as utilities
ModuleNotFoundError: No module named 'main'
EDIT 2 and solution
Thanks to The answers, I was able to figure out that the issue is related to how Pycharm handle projects. Basically it somehow save the path where the project is; so calling an import will result in the project folder being parsed, and that's why it works fine from Pycharm.
In Python, unless PYTHONPATH has the path to my project or other modules that I wrote, it won't be able to find them, hence, raise the error.
FIX:
in my main module that I use to run the application, I did retrieve the path of the file; which I know being one level below the modules I need; so I can explicitly add the folder to the current sys.path. This will end up making possible for me to call the import successfully.
import sys
current_dir = os.path.dirname(__file__)
sys.path.insert(0, , current_dir)
The only downside is that every file and resource that I use in my project, has to be directly referred by full path; so I have to pass the current_dir around the various files in the project.
PyCharm has project interpreter settings. Verify these are the same as your system Python. Go to:
File menu
Settings
Project: <project name>
Project Interpreter
View the path to the Python executable/binary being used by the project in PyCharm and verify it matches what your system is calling (e.g., which python3)
Alternatively, it may be that you declared your sources root within PyCharm and the system cannot properly run the module as it exists in the path you're running it from (especially if inside a package). You can get around this using the -m parameter and calling it from Python.
You can also try running it from the Terminal inside PyCharm and see what it adds to the path before initializing the shell session (you can sometimes see this in your Run configurations also). If you are referring to modules not installed via pip / into the Python path but rather loaded into your project path, then this may be the culprit.
On PyCharm, next to the green "RUN" arrow press the box and then press edit configurations (see image)
There you'll have Working Directory - that path is where PyCharm is running that script from (without errors).
Try running it from the terminal within that path - that should solve your import errors.
Eclipse can run a python project rather than just one .py file. Is it possible to run an entire project from Python 3.x shell. I looked into it a little, but I didn't really find a way. I tried just running the .py file with the main using exec(open('bla/blah/projMain.py')) like you would any python file. All of my modules (including the main) is in one package, but when I ran the main I got a no module named 'blah' (the package it is in). Also, as a side note there is in fact aninit.pyand even apycache' directory.
Maybe I didn't structure it correctly with Eclipse (or rather maybe Eclipse didn't structure it properly), but Eclipse can run it, so how can I with a Python 3.4.1 shell? Do I have to put something in __init__.py, perhaps, and then run that file?
It depends on what your file looks like -- if you have an if __name__ == "__main__": do_whatever(), then an import will not do_whatever() because the name of the imported module will not be "__main__". (It will be whatever the name of the module is).
However, if it is just a script without the conditional, you can just import the module and it will be run. Python needs to know where the module is though, so if it is not in your path, you will have to use relative imports, as documented here.
Based on current information, I would suggest you to run it this way in OSX
1) Bring up the Terminal app
2) cd to the location where bla lives
3) run python bla/blah/projMain.py
Show us stacktrace if the above failed.
I have a pydev project opened in eclipse.
The project's packages nesting is as follows:
my-package:
my-sub-package:
foo
In this project I am using an import from another project I created.
The other project's nesting is:
my-package:
my-sub-package:
bar
So the 2 main packages have the same names.
When trying to import:
from my-package.my-sub-package import bar I get ImportError: No module named bar.
I guess that happens because eclipse is searching in the current project and when it doesn't find bar, it doesn't look for it in dist-utils (where I pip installed the project I'm trying to import from).
(When trying to do the same import from a regular python opened from the same location, the import works perfectly fine - so it's something in eclipse).
Is there a way to tell eclipse to continue searching for the module in all folders in the pythonpath (so it can reach dist-utils), even when the packages have the same name and it didn't find the right package in the current project?
Thanks a lot.
Where are you running the regular python from exactly?
If you have
src-root:
my-package:
__init__.py
my-sub-package:
__init__.py
foo.py
and you run the regular python from within src-root then you will get the same result ie that it will not work but if you run it anywhere else it probably will work as you say.
If you can't rename any of the packages the only thing is to remove the current scr-root from the 'Source Folder' in your eclipse PyDev options but not would mean you would never access the foo my-package again.
See http://docs.python.org/2/library/sys.html#sys.path