I am trying to run a python pipeline through Mac OS Terminal. I am using Python 2.7.15 because newer versions of Python gave me error messages with this file. I keep getting the following error message:
$ python ../../../scripts/gemmaPipeline.py ../../../data/all.fixGL.Q10.ap.bi.MAF5.vcf.gz outfile Results
Traceback (most recent call last): File
"../../../scripts/gemmaPipeline.py", line 5, in
import utils, vcf, genomes100, gemma ImportError: No module named genomes100
The modules listed in the import command are present under the "scripts" folder in a subfolder "modules". For each of the four modules, I have a .py and .pyc file in this subfolder. I cannot get Python to find this path.
I have tried running the same script within a virtualenv but to no avail. I even tried moving the "modules" subfolder to the Python "site-packages" folder in hopes that it would work but failed.
Finally, I tried including an init.py file (blank) in the "script" folder, but that didn't work either.
I am an utter novice and am pretty much learning as I go.Please help.
Related
I know there has been similar problems, but unfortunately most of them are related to errors with pyperclip itself instead of the batch file, which i suspect is where the problem stems from.
Below is an MRE of my Python script:
#! python3 -> Do I have to use my version(3.8)?
# pw.py - An insecure password locker program.
import sys, pyperclip
#do something with the module
And my batch file pw.bat:
#py.exe C:\Users\KEVIN\PycharmProjects\atbs_exercise\pw.py %*
#pause
I am running python 3.8 on windows 10. I imported the pyperclip module in my python script pw.py and ran the file via pw.bat, and this in turn gives me this error:
Traceback (most recent call last):
File "C:\Users\KEVIN\PycharmProjects\atbs_exercise\pw.py", line 7, in <module>
import sys, pyperclip
ModuleNotFoundError: No module named 'pyperclip'
Press any key to continue . . .
Which shouldn't happen as I have installed pyperclip on the project using pip, and the script itself runs just fine in pycharm. What am I missing?
EDIT: I forgot to mention that I am using pycharm. So the thing is that pycharm had also installed python.exe in the project folder. And as the module pyperclip is only installed to that folder, the python.exe used in the bat must point to the one in the project folder.
i don't know why are you using py.exe. when running commands from a batch file or cmd .you should use python.exe.obviously you would need to add python to add for doing so.instead of adding py.exe to path,add python in system variable Path which is somewhere present in C:\Users\[username]\AppData\Local\Programs\Python\(your path might be diffrernt).you can add python in Path by following this post
after adding python to path just use the following batch-file:
#echo off
python path-to-your-py-file\filename.py
I wrote a script in Spyder IDE, but in order for clients to use it, I would like to make it easier to run it then opening Anaconda and then Spyder and running it from there.
The reason I used Spyder was because it allowed me to use many modules that otherwise wouldn't be allowed on company stations.
I researched a lot but could not find a way to do this.
I was thinking of enveloping the script in a batch file and running it. I tried updating the python script to look for modules in the Anaconda default directories:
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\python37.zip)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\DLLs)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\lib)
sys.path.insert(1, C:\\ProgramData\\Anaconda3)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\lib\\site-packages)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\lib\\site-packages\\win32)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\lib\\site-packages\\win32\\lib)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\lib\\site-packages\\Pythonwin)
sys.path.insert(1, C:\\ProgramData\\Anaconda3\\lib\\site-packages\\IPython\\extensions)
When running the script in Windows cmd, I got these errors when importing pandas module, so there is more to using Spyder then just accessing those modules (I couldn't figure out what):
Traceback (most recent call last):
File "script.py", line 32, in <module>
import pandas as pd
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\__init__.py", line 13, in <module>
__import__(dependency)
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\__init__.py", line 110, in <module>
import warnings
File "C:\ProgramData\Anaconda3\lib\warnings.py", line 494
f"coroutine '{coro.__qualname__}' was never awaited\n"
Another approach I tried was using pyinstaller, but that is not available for us and I cannot install it from the website.
Is there a way of running the python script and using all the modules offered by Anaconda without going through Spyder?
Have you tried py2exe? It's a great tool that does exactly what you're asking for.
Also look into cython. It's a lot easier to generate an executable from C code, and you'll get a performance boost as well.
Also try pyinstaller. It'll give you both an executable and settings for that, like icon or no-console. first go to the directory where your .py file is and run the below command:
pyinstaller script.py --onefile
Before this, you'll need to install that python module:
python* -m pip install pyinstaller
Where * is your specific python version, if you have multiple installed.
The first command will give you dist folder, here is where your executable is placed. and the script.spec file is your settings.
i am working on a long project in Django and suddenly i faced this error:
import asyncio
File "C:\Users\AppData\Local\Programs\Python\Python38\lib\asyncio\__init__.py", line 17, in <module>
from .streams import *
ModuleNotFoundError: No module named 'asyncio.streams'
I tried to instal it using : pip install asyncio, But again it gave some errors :
Traceback (most recent call last):
File "c:\users\appdata\local\programs\python\python38\lib\shutil.py", line 587, in _rmtree_unsafe
with os.scandir(path) as scandir_it:
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\AppData\\Local\\Temp\\pip-req-tracker-ua43f6rj'
please Tell me how can i fix it . I am Using Python 3.8.1
Please reinstall python3.8.1 for windows 10
Packages and modules in python can be imported when they appear in the PYTHONPATH environement variable or in the sys.path list.
On each PYTHONPATH expanding, will ends up in the sys.path list.
so make sure that, the instalation of your asyncio was successeful and its path exist in the sys.path list. how you check? see below:
in comand line or in script.py type:
import sys
p_asyn=r'C:\Users\AppData\Local\Programs\Python\Python38\lib\asyncio'
if p_asyn not in sys.path:
print('asyncio not in the sys path list! :-(')
sys.path.append(p_asyn)
its better to append new paths to the PYTHONPATH rather than editing the sys.path.
Anywa, if the module you are trying to import (asuncio) is a built in lib/module you should check if the pyhton installation was successful and the path environement variable edited correctly and has all paths that should be.
in windows make sure that python installation dir appears in the path. if not use the set command,
set PYTHONPATH=%PYTHONPATH%;C:\My_python_lib
Python usually stores its library (and thereby your site-packages folder) in the installation directory. So, if you had installed Python to C:\Python\, the default library would reside in C:\Python\Lib\ and third-party modules should be stored in C:\Python\Lib\site-packages.
This attended to windows users.
for further reading, enter the link here
using_py2
using_py3
I am working on a project in PyCharm. The project has the following structure:
/projectRoot/
folder1/
somecode.py
utils/
__init__.py
myutils1.py
I'd want to know how I can do an import such that the import works when running the code in the pyCharm console in an interactive manner, as well as when running the code using the
python somecode.py
command in the terminal.
Currently I do:
from utils.myutils1.py import myClass
But command line I get the error:
File "somecode.py", line 10, in
from utils.myutils1 import myClass ModuleNotFoundError: No module named 'utils'
and on PyCharm:
Traceback (most recent call last): File
"/home/ubuntu/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py",
line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in
from utils.myutils1 import myClass ModuleNotFoundError: No module named 'utils'
Any recommendations on the proper folder structure for modules within a project, and how to import them properly?
Thanks!
To explain the answer I recreated that project structure you had
/projectRoot/
folder1/
somecode.py
utils/
__init__.py
myutils1.py
somecode.py
from utils.myutils1 import myclass
if __name__ == "__main__":
print(myclass)
myutils1.py
myclass="tarun"
Running them from pycharm works without any issues, but running them from terminal will produce below error
File "somecode.py", line XX, in <module>
from utils.myutils1 import myclass
ModuleNotFoundError: No module named 'utils'
The issue is that Pycharm does few things for you because which you don't realize why it is not working in the terminal. So before telling you what you need to, I will tell you two things that PyCharm does on its own.
Python Console
When you launch a Python Console from Pycharm, there is some code that gets executed, using preferences.
As you can see there are two options
[X] Add content roots to PYTHONPATH
[ ] Add source roots to PYTHONPATH
And then a starting script as well. So what this does is that it adds the root of your project to python's path. Which is controlled by two main ways sys.path and PYTHONPATH environment variable
If I run the below code in Python Console
>>> import sys
>>> sys.path
['/Applications/PyCharm.app/Contents/helpers/pydev',
'/Applications/PyCharm.app/Contents/helpers/pydev',
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python27.zip',
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7', ....
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7/site-packages',
'/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27']
As you can see '/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27' is added to the Python terminal.
Python Configurations
When you configure to RUN in code using Pycharm, you have similar two options.
We can change the code of our somecode.py to below
import os
print (os.environ['PYTHONPATH'])
import sys
print (sys.path)
/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27
['/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27/folder1',
'/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27', ....,
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7/site-packages']
From the output we can see that PYTHONPATH is set to current project folder.
Running from terminal
Now let's run the somecode.py from terminal with the modifications we made.
$ python somecode.py
Traceback (most recent call last):
File "somecode.py", line 2, in <module>
print (os.environ['PYTHONPATH'])
File "/Users/tarun.lalwani/.virtualenvs/folderstructure27/bin/../lib/python2.7/UserDict.py", line 40, in __getitem__
raise KeyError(key)
KeyError: 'PYTHONPATH'
So that indicates there is no PYTHONPATH when we ran it in terminal. Let us run it again by removing the print(os.environ['PYTHONPATH']) code. You will get the below output
['/Users/tarun.lalwani/Desktop/payu/projects/folderstructure27/folder1', ...
'/Users/tarun.lalwani/.virtualenvs/folderstructure27/lib/python2.7/site-packages']
Traceback (most recent call last):
File "somecode.py", line 7, in <module>
from utils.myutils1 import myclass
ImportError: No module named utils.myutils1
As you can see folder1 is added to sys.path because it is the folder containing somecode.py, but the root folder has not been added. The fix in terminal is simple, which is to set the root directory path in PYTHONPATH.
PYTHONPATH=`pwd`/.. python somcode.py
And now the code will work from terminal also.
But the way they work are different from Python Console.
IMPORTANT NOTE:
Python Console using PyCharm on remote interpreter.
If running the python console using the remote interpreter option pycharm will fail. This is because it will append the path of the local PC and not the path of the remote server.
In order to fix this problem one has to add a mapping between the local PC directory and the remote server path.
You can use utils package inside folder1 folder:
Then the code will work either way:
from utils.myutils1 import myClass
Similar error here and this appears to be working for me:
Make sure Project Interpreter is set to, for example: C:\Python36\python.exe (in my case) and not a copy somewhere or another.
'File > Settings > Project ____ > Project Interpreter'
Or, long story short, if that route isn't cooperating, can also try finding workspace.xml and manually change SDK_HOME before starting PyCharm:
<option name="SDK_HOME" value="C:\Python36\python.exe" />
I have been at this for a day and have found no solution. My python project is very levels deep and some of the modules have to be run independently (no -m flag, no relative imports). For this reason we decided to add the root folder to the PYTHONPATH. I am running OSX 10.11.3. Everything had been running smoothly until yesterday. I could run independent modules from the terminal with no issues in both Python 2.7 and 3.5. Yesterday, without modifying my PYTHONPATH or any other environment setting, running any of these independent modules from the terminal now gives me import errors in both Python 2 and 3.
Here is my working tree:
/Users/sintrafico/Documents/code/central_maestra
- reports_server.py
- api_server.py
- sintrafico
- sql
- SQLConnection
- api
- incident
- csv (not package)
- independent_module
- tests
I was running my tests yesterday with coverage with no problem, but now I can't.
Python 2 python csv/independent_module.py gives error:
Traceback (most recent call last):
File "csv/independent_module.py", line 13, in <module>
from sintrafico.sql import SQLConnection
ImportError: No module named sintrafico.sql
Python 3 python3 csv/independent_module.py gives error:
Traceback (most recent call last):
File "csv/create_bemobile_csv.py", line 13, in <module>
from sintrafico.sql import SQLConnection
ImportError: No module named 'sintrafico'
From within the incident folder, yesterday this command was working fine: coverage run -m unittest discover. Now all the tests fail because ImportError: No module named 'reports_server'.
Since that started happening yesterday, I have been messing with my PYTHONPATH. I left it blank and it didn't work (as expected), I have added the path now several ways:
export PYTHONPATH=“${PYTHONPATH}:/Users/sintrafico/Documents/code/central_maestra"
export PYTHONPATH=“/Users/sintrafico/Documents/code/central_maestra:${PYTHONPATH}"
export PYTHONPATH=“${PYTHONPATH}:/Users/sintrafico/Documents/code/central_maestra/"
export PYTHONPATH=“/Users/sintrafico/Documents/code/central_maestra:${PYTHONPATH}"
And also setting it without appending the PYTHONPATH since it was empty to begin with. I restarted my computer several times with no success. Also note, I have a .bash_profile and .bashrc with the following contents:
# Setting PATH for Python 3.5
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}"
export PATH
PYTHONPATH=“/Users/sintrafico/Documents/code/central_maestra:${PYTHONPATH}“
export PYTHONPATH
And lastly, what bugs me the most is that if I run the tests inside PyCharm they all run and pass, I get no import errors at all.
Have you tried setting the path without the quotes and braces? You are setting the python path as a string representation of the path instead of a list of paths