(Edit: I made a typo when writing this question: I put quotation mark around "pyparsing" in script. Thanks #dswdsyd)
When running a python script, there's only python standard library on target machine. When a package is needed, I have to install it first. For example, When I tried to install pyparsing and import it:
subprocess.call([sys.executable, "-m", "pip", "install", "pyparsing"])
import pyparsing
I got error:
ModuleNotFoundError: No module named 'pyparsing'
So how to install and import a package in the same python script?
[Update:] In the second run of the script, the package can be imported. Strange.
Essentially you are passing pyparsing as a variable instead of a string, to fix this change pyparsing to "pyparsing"
try this:
import subprocess
import sys
subprocess.call([sys.executable, "-m", "pip", "install", "pyparsing"])
import pyparsing
Problem solved by importlib.invalidate_caches(). According to importlib's documentation:
importlib.invalidate_caches()
Invalidate the internal caches of
finders stored at sys.meta_path. If a finder implements
invalidate_caches() then it will be called to perform the
invalidation. This function should be called if any modules are
created/installed while your program is running to guarantee all
finders will notice the new module’s existence.
Related
I have the following, very simple python code in a lambda function:
from sklearn.externals import joblib
import praw
import datetime
from operator import attrgetter
import sys
def handler_name(event, context):
return "I am a cat dog and i meow."
I have also done pip installs for scikit-learn, praw, datetime, numpy and scipy from within a python 2.7 virtualenv. I then compressed my .py file along with everything in my virtualenv's /lib/python2.7/site-packages folder into a zip and uploaded it to AWS lambda. Unfortunately when I run the code I get the following error:
Unable to import module 'mainLambda': /var/task/sklearn/__check_build/_check_build.so: invalid ELF header
___________________________________________________________________________
Contents of /var/task/sklearn/__check_build:
setup.py _check_build.so __init__.pyc
__init__.py setup.pyc
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.
If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.
If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
Obviously the issue is with sk-learn. I have no idea what though. It could be a versioning issue but I downloaded all the libraries from within a virtulenv and chose a python2.7 lambda function. Any idea? I am stumped!
My goal is to create a python script that loops over cells of an excel document. This is my python script called reader.py, and it works just fine.
import xlrd
import os
exceldoc = raw_input("Enter the path to the doc [C:\\folder\\file.xlsx]: ")
wb = xlrd.open_workbook(exceldoc,'rb')
.... some code....
The problem I'm encountering is attempting to use py2exe to create an executable so this script can be used elsewhere.
Here is my setup.py file:
from distutils.core import setup
import py2exe
import sys
from glob import glob
setup(name='Excel Document Checker',console=['reader.py'])
I run the following command: python setup.py py2exe
It appears to run fine; it creates the dist folder that has my reader.exe file, but near the end of the command I get the following:
The following modules appear to be missing
['cElementTree', 'elementtree.ElementTree']
I did some searching online, and tried the recommendations here Re: Error: Element Tree not found, this changing my setup.py file:
from distutils.core import setup
import py2exe
import sys
from glob import glob
options={
"py2exe":{"unbuffered": True,"optimize": 2,
'includes':['xml.etree.ElementPath', 'xml.etree.ElementTree', 'xml.etree.cElementTree'],
"packages": ["elementtree", "xml"]}}
setup(name='Excel Document Checker',options = options,console=['reader.py'])
I'm now getting an error:
ImportError: No module named elementtree
I'm sort of at an impasse here. Any help or guidance is greatly appreciate.
Just some information - I'm running Python 2.6 on a 32 bit system.
You explicitly told setup.py to depend on a package named elementtree here:
"packages": ["elementtree", "xml"]}}
There is no such package in the stdlib. There's xml.etree, but obviously that's the same name.
The example you found is apparently designed for someone who has installed the third-party package elementtree, which is necessary if you need features added after Python 2.6's version of xml.etree, or if you need to work with Python 1.5-2.4, but not if you just want to use Python 2.6's version. (And anyway, if you do need the third-party package… then you have to install it or it won't work, obviously.)
So, just don't do that, and that error will go away.
Also, if your code—or the code you import (e.g., xlrd) is using xml.etree.cElementTree, then, as the py2exe FAQ says, you must also import xml.etree.ElementTree before using it to get it working. (And you also may need to specify it manually as a dependency.)
You presumably don't want to change all the third-party modules you're using… but I believe that making sure to import xml.etree.ElementTree before importing any of those third-party modules works fine.
As part of my build system, I am using a modified version of a Python package (cogapp). I don't want to install the package because:
I've modified the package and don't want to worry about collision with unmodified versions which may already be installed.
It's nicer if the users of the build system don't need to install extra packages.
However, I'm having problems with using the package if it's not installed. If it is installed, I can run:
python -m cogapp <additional args>
and everything runs as intended.
The package has a __main__.py script:
import sys
from cogapp import Cog
sys.exit(Cog().main(sys.argv))
I tried running this directly, e.g.:
python -m <path>/__main__ <additional_args>
But I get the error:
...
/__main__.py", line 3, in <module>
from cogapp import Cog
ImportError: No module named cogapp
This is probably related to the error I get if I run __init__.py:
from .cogapp import *
The error is:
from .cogapp import *
ValueError: Attempted relative import in non-package
How can I run the package as a package?
EDIT:
I found a fix by removing all the relative imports from cogapp, and removing the -m, i.e. not running as a module. In this instance it's not too bad because it's a small package with only a single directory. However I'm interested in how this should be done in future. There's lots of stuff written around this subject, but no clear answers!
Here's the solution I've come to.
Disclaimer: you are actually installing the package but to a different path than the standard one.
$ mkdir newhome
$ python setup.py install --home=./newhome
$ PYTHONPATH=$PWD/newhome/lib/python <COMMAND_NEEDING_THAT_PACKAGE>
how can i import other external libraries in web2py? is it possible to
load up libs in the static file?
can somebody give me an example?
thanks
peter
If the library is shipped with python, then you can just use import as you would do in regular python script. You can place your import statements into your models, controllers and views, as well as your own python modules (stored in modules folder). For example, I often use traceback module to log stack traces in my controllers:
import traceback
def myaction():
try:
...
except Exception as exc:
logging.error(traceback.format_exc())
return dict(error=str(exc))
If the library is not shipped with python (for example, pyodbc), then you will have to install that library (using distutils or easy_install or pip) so that python can find it and run web2py from the source code: python web2py.py. Then you will be able to use regular import statement as described above. Before you do this make sure you installed the library properly: run python interpreter and type "import library_name". If you don't get any errors you are good to go.
If you have a third party python module or package, you can place it to modules folder and import it as shown below:
mymodule = local_import('module_name')
You can also force web2py to reload the module every time local_import is executed by setting reload option:
mymodule = local_import('module_name', reload=True)
See http://web2py.com/book/default/section/4/18?search=site-packages for more information.
In web2py you import external library as you normally do in Python
import module_name
or
from module_name import object_name
I am not sure what you mean by "in the static file"
I need to import the multiprocessing module in Python 2.5.
I've followed the instructions here exactly: http://code.google.com/p/python-multiprocessing/wiki/Install
make and make test run without errors. I've also edited $PYTHONPATH to include the directory where the package is installed.
But 'import multiprocessing' still says: "ImportError: no module named multiprocessing".
What am I doing wrong? Is there some step missing from these instructions? I haven't installed a Python module before.
Navigate to the directory containing the package then type:
python setup.py install
This info was contained in the INSTALL.txt file.
http://code.google.com/p/python-multiprocessing/source/browse/trunk/INSTALL.txt
perhaps you can try:
import sys
sys.path.append('/path/to/processingdotpylibs/')
import processing