Develop Reference

VS Code - Configure Run Python File In Terminal

I have a python environment called nem that I created using Conda, however in VS Code when I right click and select 'Run Python File in Terminal' I get the below error. BUT I know this package is not an issue as I can open up a new windows terminal outside of VS Code and run the commands below and everything works great.
activate nem
python index.py
I have also had issues where the output in the terminal is being suppressed even though the application is running. I'm not sure what the root cause of this is, and whether its directly related to Conda. However does anyone know of a solution to this? Or how to configure VS Code's 'Run Python File in Terminal' to just use the commands above.
Windows PowerShell Copyright (C) Microsoft Corporation. All rights
reserved.
PS C:\Users\nikhil\Git Repos\natatorium-energy-model> conda
activate nem PS C:\Users\nikhil\Git
Repos\natatorium-energy-model> &
C:/Users/nikhil/AppData/Local/Continuum/anaconda3/envs/nem/python.exe
"c:/Users/nikhil/Git Repos/natatorium-energy-model/index.py"
Traceback (most recent call last): File "c:/Users/nikhil/Git
Repos/natatorium-energy-model/index.py", line 8, in
from apps import analyse File "c:\Users\nikhil\Git Repos\natatorium-energy-model\apps\analyse.py", line 17, in
import pyarrow.parquet as pq File "C:\Users\nikhil\AppData\Local\Continuum\anaconda3\envs\nem\lib\site-packages\pyarrow__init__.py",
line 49, in
from pyarrow.lib import cpu_count, set_cpu_count ImportError: DLL load failed: The specified module could not be found. PS
C:\Users\nikhil\Git Repos\natatorium-energy-model>

In the bottom left corner, change the python interpreter to the one in your environment.

ModuleNotFoundError error with PyCharm project folder recs

I am working on a project in PyCharm. The project has the following structure:
/projectRoot/
folder1/
somecode.py
utils/
__init__.py
myutils1.py
I'd want to know how I can do an import such that the import works when running the code in the pyCharm console in an interactive manner, as well as when running the code using the
python somecode.py
command in the terminal.
Currently I do:
from utils.myutils1.py import myClass
But command line I get the error:
File "somecode.py", line 10, in
from utils.myutils1 import myClass ModuleNotFoundError: No module named 'utils'
and on PyCharm:
Traceback (most recent call last): File
"/home/ubuntu/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py",
line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in
from utils.myutils1 import myClass ModuleNotFoundError: No module named 'utils'
Any recommendations on the proper folder structure for modules within a project, and how to import them properly?
Thanks!

To explain the answer I recreated that project structure you had
/projectRoot/
folder1/
somecode.py
utils/
__init__.py
myutils1.py
somecode.py
from utils.myutils1 import myclass
if __name__ == "__main__":
print(myclass)
myutils1.py
myclass="tarun"
Running them from pycharm works without any issues, but running them from terminal will produce below error
File "somecode.py", line XX, in <module>
from utils.myutils1 import myclass
ModuleNotFoundError: No module named 'utils'
The issue is that Pycharm does few things for you because which you don't realize why it is not working in the terminal. So before telling you what you need to, I will tell you two things that PyCharm does on its own.
Python Console
When you launch a Python Console from Pycharm, there is some code that gets executed, using preferences.
As you can see there are two options
[X] Add content roots to PYTHONPATH
[ ] Add source roots to PYTHONPATH
And then a starting script as well. So what this does is that it adds the root of your project to python's path. Which is controlled by two main ways sys.path and PYTHONPATH environment variable
If I run the below code in Python Console
>>> import sys
>>> sys.path
['/Applications/PyCharm.app/Contents/helpers/pydev',
'/Applications/PyCharm.app/Contents/helpers/pydev',
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python27.zip',
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7', ....
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7/site-packages',
'/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27']
As you can see '/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27' is added to the Python terminal.
Python Configurations
When you configure to RUN in code using Pycharm, you have similar two options.
We can change the code of our somecode.py to below
import os
print (os.environ['PYTHONPATH'])
import sys
print (sys.path)
/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27
['/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27/folder1',
'/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27', ....,
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7/site-packages']
From the output we can see that PYTHONPATH is set to current project folder.
Running from terminal
Now let's run the somecode.py from terminal with the modifications we made.
$ python somecode.py
Traceback (most recent call last):
File "somecode.py", line 2, in <module>
print (os.environ['PYTHONPATH'])
File "/Users/tarun.lalwani/.virtualenvs/folderstructure27/bin/../lib/python2.7/UserDict.py", line 40, in __getitem__
raise KeyError(key)
KeyError: 'PYTHONPATH'
So that indicates there is no PYTHONPATH when we ran it in terminal. Let us run it again by removing the print(os.environ['PYTHONPATH']) code. You will get the below output
['/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27/folder1', ...
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7/site-packages']
Traceback (most recent call last):
File "somecode.py", line 7, in <module>
from utils.myutils1 import myclass
ImportError: No module named utils.myutils1
As you can see folder1 is added to sys.path because it is the folder containing somecode.py, but the root folder has not been added. The fix in terminal is simple, which is to set the root directory path in PYTHONPATH.
PYTHONPATH=`pwd`/.. python somcode.py
And now the code will work from terminal also.
But the way they work are different from Python Console.
IMPORTANT NOTE:
Python Console using PyCharm on remote interpreter.
If running the python console using the remote interpreter option pycharm will fail. This is because it will append the path of the local PC and not the path of the remote server.
In order to fix this problem one has to add a mapping between the local PC directory and the remote server path.

You can use utils package inside folder1 folder:
Then the code will work either way:
from utils.myutils1 import myClass

Similar error here and this appears to be working for me:
Make sure Project Interpreter is set to, for example: C:\Python36\python.exe (in my case) and not a copy somewhere or another.
'File > Settings > Project ____ > Project Interpreter'
Or, long story short, if that route isn't cooperating, can also try finding workspace.xml and manually change SDK_HOME before starting PyCharm:
<option name="SDK_HOME" value="C:\Python36\python.exe" />

Why can I not import Caffe in PyCharm but can import it in terminal?

I want to import Caffe. I can import it in terminal but not in PyCharm.
I have tried some suggestions like adding include /usr/local/cuda-7.0/lib64 to /user/etc/ld.so.conf file but still it can not import this module. However, I think this is not a good solution as I am using the CPU mode only.
I am using Linux Mint.
The output for sys.path in PyCharm terminal is:
>>> sys.path
['',
'/home/user/anaconda2/lib/python27.zip',
'/home/user/anaconda2/lib/python2.7',
'/home/user/anaconda2/lib/python2.7/plat-linux2',
'/home/user/anaconda2/lib/python2.7/lib-tk',
'/home/user/anaconda2/lib/python2.7/lib-old',
'/home/user/anaconda2/lib/python2.7/lib-dynload',
'/home/user/anaconda2/lib/python2.7/site-packages',
'/home/user/anaconda2/lib/python2.7/site-packages/Sphinx-1.4.1-y2.7.egg',
'/home/user/anaconda2/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg']
>>>
and when I run sys.path in PyCharm itself, I get:
['/opt/pycharm-community-2016.2.3/helpers/pydev',
'/home/user/',
'/opt/pycharm-community-2016.2.3/helpers/pydev',
'/home/user/anaconda2/lib/python27.zip',
'/home/user/anaconda2/lib/python2.7',
'/home/user/anaconda2/lib/python2.7/plat-linux2',
'/home/user/anaconda2/lib/python2.7/lib-tk',
'/home/user/anaconda2/lib/python2.7/lib-old',
'/home/user/anaconda2/lib/python2.7/lib-dynload',
'/home/user/anaconda2/lib/python2.7/site-packages',
'/home/user/anaconda2/lib/python2.7/site-packages/Sphinx-1.4.1-py2.7.egg',
'/home/user/anaconda2/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg',
'/home/user/anaconda2/lib/python2.7/site-packages/IPython/extensions',
'/home/user/']
which is not exactly the same as the time I ran it in terminal.
moreover, as I run the import caffe in PyCharm the error is as bellow:
/home/user/anaconda2/bin/python /home/user/important_commands.py
Traceback (most recent call last):
File "/home/user/important_commands.py", line 11, in <module>
import caffe
ImportError: No module named caffe
Process finished with exit code 1

This solution worked for me. I think that the problem is that pycharm doesn´t charge the libraries from the bashrc.
Open Pycharm
Go to File --> settings --> project interpreter
Open the bar with all the possible interpreters and press show all.
Click the last button of the options (Brown bottom).
Add the python path (/home/user/caffe/python)

I installed caffe using pycharm terminal too but it did not work. Finally I added sys.path.extend([/home/user/caffe-master/python]) to python consule and meanwhile I wrote the following in my code.
import sys
sys.path.append("/home/user/caffe-master/python/")
import caffe
and it worked!!!

You need to add this same path under you interpreters path.
Settings -> Project interpreter ->click the cogwheel next to interpreter -> More -> click on icon that says 'Show paths for interpreter' -> add path -> Chaos Solved.

I solved the problem by adding caffe in the project interpreter. Just use the + on the right side for the list of available packages. Search for caffe and click on Install Package.

Error when running a python script

I am trying to run a script using an interface created with tkinter. I have a button that executes a script which code is:
subprocess.call("python3 " + PATH_TO_SCRIPTS + "main.py 1 &", shell=True)
However, when this button is pressed I am getting the following error.
Traceback (most recent call last):
File "/home/m//PycharmProjects/ROSAutonomousFlight/catkin_ws/src/ardrone_numeric_method_controller/scripts/main.py", line 17, in <module>
from controller import *
File "/home/m/PycharmProjects/ROSAutonomousFlight/catkin_ws/src/ardrone_numeric_method_controller/scripts/controller.py", line 5, in <module>
import rospy
It says that the module rospy does not exist, but when I run
import rospy
using python or python3 it is imported successfully. What can I do to solve this issue? I am using Ubuntu.

The comments to your question are mostly about Python, but I guess it is more of a ROS issue.
You don't have to set-up your PYTHONPATH manually to find rospy but you have to source the setup.bash of your catkin workspace (otherwise none of the ROS tools is found).
Usually this is done by adding something like
source ~/catkin_ws/devel/setup.bash
to .bashrc. This works fine for everything that is run in a terminal.
I don't know how you start your script but as it provides a graphical interface you probably just run it by double-clicking it in the file browser? If you indeed do so, the script is not run in a terminal and therefore can't find the ROS modules. Run the script from a terminal (in which the setup.bash has been sourced) and it should work.

Pycharm cannot import my modules while executing (the directory is present in the sources list and I can execute it just fine from the command line)

It's throwing me this error:
/usr/bin/python -u /opt/pycharm-community-4.5.1/helpers/pydev/pydev_run_in_console.py 58137 38816 /path/to/my/module.py
/usr/lib/python/site-packages/IPython/external/path.py:32: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import sys, warnings, os, fnmatch, glob, shutil, codecs, md5
/usr/lib/python/site-packages/IPython/iplib.py:58: DeprecationWarning: the sets module is deprecated
from sets import Set
Running /path/to/my/module.py
Traceback (most recent call last):
File "/opt/pycharm-community-4.5.1/helpers/pydev/pydev_run_in_console.py", line 69, in <module>
globals = run_file(file, None, None)
File "/opt/pycharm-community-4.5.1/helpers/pydev/pydev_run_in_console.py", line 29, in run_file
pydev_imports.execfile(file, globals, locals) # execute the script
File "/path/to/my/module.py", line 13, in <module>
import my.module.name
ImportError: No module named my.module.name
Process finished with exit code 1
Couldn't connect to console process.
If I run it on the BASH terminal, it's executing just fine. It was actually executing fine from PyCharm too, but I'm not sure what changed and I suddenly started seeing this happening. Also, it doesn't show any errors about the missing module in the editor, which suggests that at least the editor can see those packages in the path where it searches. PYTHONPATH also has this directory.

Removing PYTHONPATH from the environment variables for the run fixed the issue.
Steps to fix:
Run -> Run...
Edit Configurations
Select the configuration for the run that's affected
Click on button labeled ... next to Environment Variables text field
Remove PYTHONPATH from the list of environment variables displayed there
I still don't understand what happened, but inspecting this variable before removing it showed the same list of paths added several times. That seemed very odd, so I removed this variable and tried, and it worked!