Im trying to import a module using exec statement but it fails,
code.py
def test(jobname):
print jobname
exec ('import ' + jobname)
if __name__ = '__main__':
test('c:/python27/test1.py')
Error:
Syntax error:
import:c:\python27 est1.py
You probably mean execfile(jobname). And import does not work with filenames. It works with package names. Any good tutorial would cover that. Another issue would be the \t being interpreted as a tab character, but here it is not the case because you are uaing forward slash not baclslash...
Somehow, I think you must be calling
test('c:\python27\test1.py')
instead of
test('c:/python27/test1.py')
The backslash in front of the t is being interpreted as a tab character. Thus the error
import:c:\python27 est1.py
Notice the missing t.
Secondly, the import command expects a module name, not a path. For importing, use __import__ not exec or execfile. execfile has been removed from Python3, so for future compatibilty, you may not want to use it in Python2. exec can be used instead, but there are problems with using exec.
Assuming c:\python27 is in your PYTHONPATH, you could
do something like this:
def test(jobname):
print jobname
__import__(jobname)
if __name__ == '__main__':
test('test1')
def test(jobname):
print jobname
a = jobname.split('/')
b = "/".join(a[0:-1])
c = a[-1][0:-3]
sys.path.append(b)
exec ('import ' + c)
if __name__ = '__main__':
test('c:/python27/test1.py')
Try this code. Your path must be added to sys.path() variable.
Im trying to import a module using exec statement
Don't do that.
First, do you really need to import a module programmatically? If you tell us what you're actually trying to accomplish, I'm willing to bet we can find the square hole for your square page, instead of teaching you how to force it into a round hole.
If you do ever need to do this, use the imp module; that's what it's for.
Especially if you want to import a module by path instead of by module name, which is impossible to do with the import statement (and exec isn't going to help you with that).
Here's an example:
import imp
def test(jobname):
print jobname
while open(jobname, 'r') as f:
job = imp.load_module('test', f, jobname, ('.py', 'U', 1))
Of course this doesn't do the same thing that import test1 would do if it were on your sys.path. The module will be at sys.modules['test'] instead of sys.modules['test1'], and in local variable job instead of global variable test1, and it'll reload instead of doing nothing if you've already loaded it. But anyone who has a good reason for doing this kind of thing had better know how to deal with all of those issues.
Related
When writing throwaway scripts it's often needed to load a configuration file, image, or some such thing from the same directory as the script. Preferably this should continue to work correctly regardless of the directory the script is executed from, so we may not want to simply rely on the current working directory.
Something like this works fine if defined within the same file you're using it from:
from os.path import abspath, dirname, join
def prepend_script_directory(s):
here = dirname(abspath(__file__))
return join(here, s)
It's not desirable to copy-paste or rewrite this same function into every module, but there's a problem: if you move it into a separate library, and import as a function, __file__ is now referencing some other module and the results are incorrect.
We could perhaps use this instead, but it seems like the sys.argv may not be reliable either.
def prepend_script_directory(s):
here = dirname(abspath(sys.argv[0]))
return join(here, s)
How to write prepend_script_directory robustly and correctly?
I would personally just os.chdir into the script's directory whenever I execute it. It is just:
import os
os.chdir(os.path.split(__file__)[0])
However if you did want to refactor this thing into a library, you are in essence wanting a function that is aware of its caller's state. You thus have to make it
prepend_script_directory(__file__, blah)
If you just wanted to write
prepend_script_directory(blah)
you'd have to do cpython-specific tricks with stack frames:
import inspect
def getCallerModule():
# gets globals of module called from, and prints out __file__ global
print(inspect.currentframe().f_back.f_globals['__file__'])
I think the reason it doesn't smell right is that $PYTHONPATH (or sys.path) is the proper general mechanism to use.
You want pkg_resources
import pkg_resources
foo_fname = pkg_resources.resource_filename(__name__, "foo.txt")
I have a codebase where I'm cleaning up some messy decisions by the previous developer. Frequently, he has done something like:
from scipy import *
from numpy import *
...This, of course, pollutes the name space and makes it difficult to tell where an attribute in the module is originally from.
Is there any way to have Python analyze and fix this for me? Has anyone made a utility for this? If not, how might a utility like this be made?
I think PurityLake's and Martijn Pieters's assisted-manual solutions are probably the best way to go. But it's not impossible to do this programmatically.
First, you need to get a list of all names that existing in the module's dictionary that might be used in the code. I'm assuming your code isn't directly calling any dunder functions, etc.
Then, you need to iterate through them, using inspect.getmodule() to find out which module each object was originally defined in. And I'm assuming that you're not using anything that's been doubly from foo import *-ed. Make a list of all of the names that were defined in the numpy and scipy modules.
Now you can take that output and just replace each foo with numpy.foo.
So, putting it together, something like this:
for modname in sys.argv[1:]:
with open(modname + '.py') as srcfile:
src = srcfile.read()
src = src.replace('from numpy import *', 'import numpy')
src = src.replace('from scipy import *', 'import scipy')
mod = __import__(modname)
for name in dir(mod):
original_mod = inspect.getmodule(getattr(mod, name))
if original_mod.__name__ == 'numpy':
src = src.replace(name, 'numpy.'+name)
elif original_mod.__name__ == 'scipy':
src = src.replace(name, 'scipy.'+name)
with open(modname + '.tmp') as dstfile:
dstfile.write(src)
os.rename(modname + '.py', modname + '.bak')
os.rename(modname + '.tmp', modname + '.py')
If either of the assumptions is wrong, it's not hard to change the code. Also, you might want to use tempfile.NamedTemporaryFile and other improvements to make sure you don't accidentally overwrite things with temporary files. (I just didn't want to deal with the headache of writing something cross-platform; if you're not running on Windows, it's easy.) And add in some error handling, obviously, and probably some reporting.
Yes. Remove the imports and run a linter on the module.
I recommend using flake8, although it may also create a lot of noise about style errors.
Merely removing the imports and trying to run the code is probably not going to be enough, as many name errors won't be raised until you run just the right line of code with just the right input. A linter will instead analyze the code by parsing and will detect potential NameErrors without having to run the code.
This all presumes that there are no reliable unit tests, or that the tests do not provide enough coverage.
In this case, where there are multiple from module import * lines, it gets a little more painful in that you need to figure out for each and every missing name what module supplied that name. That will require manual work, but you can simply import the module in a python interpreter and test if the missing name is defined on that module:
>>> import scipy, numpy
>>> 'loadtxt' in dir(numpy)
True
You do need to take into account that in this specific case, that there is overlap between the numpy and scipy modules; for any name defined in both modules, the module imported last wins.
Note that leaving any from module import * line in place means the linter will not be able to detect what names might raise NameErrors!
I've now made a small utility for doing this which I call 'dedazzler'. It will find lines that are 'from module import *', and then expand the 'dir' of the target modules, replacing the lines.
After running it, you still need to run a linter. Here's the particularly interesting part of the code:
import re
star_match = re.compile('from\s(?P<module>[\.\w]+)\simport\s[*]')
now = str(time.time())
error = lambda x: sys.stderr.write(x + '\n')
def replace_imports(lines):
"""
Iterates through lines in a Python file, looks for 'from module import *'
statements, and attempts to fix them.
"""
for line_num, line in enumerate(lines):
match = star_match.search(line)
if match:
newline = import_generator(match.groupdict()['module'])
if newline:
lines[line_num] = newline
return lines
def import_generator(modulename):
try:
prop_depth = modulename.split('.')[1:]
namespace = __import__(modulename)
for prop in prop_depth:
namespace = getattr(namespace, prop)
except ImportError:
error("Couldn't import module '%s'!" % modulename)
return
directory = [ name for name in dir(namespace) if not name.startswith('_') ]
return "from %s import %s\n"% (modulename, ', '.join(directory))
I'm maintaining this in a more useful stand-alone utility form here:
https://github.com/USGM/dedazzler/
ok, this is what i think you could do, break the program. remove the imports and notice the errors that are made. Then import only the modules that you want, this may take a while but this is the only way I know of doing this, I will be happily surprised if someone does know of a tool to help
EDIT:
ah yes, a linter, I hadn't thought of that.
Can a python module detect if has been imported with import module or from module import *? Something like
if __something__=='something':
print 'Directly imported with "import ' + __name__ + '"'
else:
print 'Imported with "from ' + __name__ + ' import *"'
Thank you.
No, it's not possible to detect this from within the module's code. Upon the first import, the module body is executed and a new module object is inserted in sys.modules. Only after this, the requested names are inserted into the namespace of the importing module.
Upon later imports, the module body isn't even executed. So if a module is first imported as
import module
and a second time as
from module import name
it has no chance to do anything at all during the second import. In particular, it cannot check how it is imported.
While Svens answer is probably the correct one, and this might seem a bit obvious, It is what I was really looking for when I stumbled upon this question.
This module will at least know that you passed an input argument to it. While allows unit testing of just this specific script without the unit test being performed in the module that imported it.
import sys
def myfunction(blah):
return "something like: " + blah
noargs=len(sys.argv)
if noargs>1:
for i in range(noargs-1):
print myfunction(sys.argv[i+1])
However, It doesn't really help you, Emilio, if you have no input arguments. : )
I've searched and this seems to be a simple question without a simple answer.
I have the file a/b/c.py which would be called with python -m a.b.c. I would like to obtain the value a.b.c in the module level.
USAGE = u'''\
Usage:
python -m %s -h
''' % (what_do_i_put_here,)
So when I receive the -h option, I display the USAGE without the need to actually write down the actual value in each and every script.
Do I really need to go through inspect to get the desired value?
Thanks.
EDIT: As said, there are answers (I've searched), but not simple answers. Either use inspect, use of traceback, or manipulate __file__ and __package__ and do some substring to get the answer. But nothing as simple as if I had a class in the module, I could just use myClass.__module__ and I would get the answer I want. The use of __name__ is (unfortunately) useless as it's always "__main__".
Also, this is in python 2.6 and I cannot use any other versions.
This works for me:
__loader__.fullname
Also if I do python -m b.c from a\ I get 'b.c' as expected.
Not entirely sure what the __loader__ attribute is so let me know if this is no good.
edit: It comes from PEP 302: http://www.python.org/dev/peps/pep-0302/
Interesting snippets from the link:
The load_module() method has a few responsibilities that it must
fulfill before it runs any code:
...
It should add an __loader__ attribute to the module, set to the
loader object. This is mostly for introspection, but can be used
for importer-specific extras, for example getting data associated
with an importer.
So it looks like it should work fine in all cases.
I think you're actually looking for the __name__ special variable. From the Python documentation:
Within a module, the module’s name (as a string) is available as the value of the global variable __name__.
If you run a file directly, this name will __main__. However, if you're in a module (as in the case where you're using the -m flag, or any other import), it will be the complete name of the module.
When run with -m, sys.path[0] contains the full path to the module. You could use that to build the name.
source: http://docs.python.org/using/cmdline.html#command-line
Another option may be the __package__ built in variable which is available within modules.
Number of options are there to get the path/name of the current module.
First be familiar with the use of __file__ in Python, Click here to see the usage.
It holds the name of currently loaded module.
Check/Try the following code, it will work on both Python2 & Python3.
» module_names.py
import os
print (__file__)
print (os.path.abspath(__file__))
print (os.path.realpath(__file__))
Output on MAC OS X:
MacBook-Pro-2:practice admin$ python module_names.py
module_names.py
/Users/admin/projects/Python/python-the-snake/practice/module_names.py
/Users/admin/projects/Python/python-the-snake/practice/module_names.py
So here we got the name of current module name and its absolute path.
The only way is to do path manipulation with os.getcwd(), os.path, file and whatnot, as you mentioned.
Actually, it could be a good patch to implement for optparse / argparse (which currently replace "%prog" in the usage string with os.path.basename(sys.argv[0]) -- you are using optparse, right? -- ), i.e. another special string like %module.
Why does nobody mentioned the .__module__?
When doing a self.__module__ you will get the module path.
You can also do this outside of the class:
Class A:
self.__module__ # gets module.filename
def get_module():
A.__module__ # also gets module.filename
One liner But OS dependent
it does not work in interpreter! since file is meaningless there in the interpreter and is not defined.
does not require os module to be imported.
modulename=__file__.split("\\")[-1].split('.')[0]
Explanation:
X:\apple\pythonabc.py | will output pythonabc.py
select the last element after splitting with slashes, then select the first element by splitting it with dot '.'. because first step gives module.py, second step gives 'module' only. __file__ is a unique variable and returns the filepath of current module.
Comment any flaws or has any other pitfalls.
you should hardcode a.b.c in your help, if you distribute the package as such then that's the way to call it regardless of where a is located in the filesystem, as long as it's on the PYTHONPATH it'll be imported.
Note: Solved. It turned out that I was importing a previous version of the same module.
It is easy to find similar topics on StackOverflow, where someone ran into a NameError. But most of the questions deal with specific modules and the solution is often to update the module.
In my case, I am trying to import a function from a module that I wrote myself. The module is named InfraPy, and it is definitely on sys.path. One particular function (called listToText) in InfraPy returns a NameError, but only when I try to import it into another script. Inside InfraPy, under if __name__=='__main__':, the listToText function works just fine. From InfraPy I can import other functions with no problems. Including from InfraPy import * in my script does not return any errors until I try to use the listToText function.
How can this occur?
How can importing one particular function return a NameError, while importing all the other functions in the same module works fine?
Using python 2.6 on MacOSX 10.6, also encountered the same error running the script on Windows 7, using IronPython 2.6 for .NET 4.0
Thanks.
If there are other details you think would be helpful in solving this, I'd be happy to provide them.
As requested, here is the function definition inside of InfraPy:
def listToText(inputList, folder=None, outputName='list.txt'):
'''
Creates a text file from a list (with each list item on a separate line). May be placed in any given folder, but will otherwise be created in the working directory of the python interpreter.
'''
fname = outputName
if folder != None:
fname = folder+'/'+fname
f = open(fname, 'w')
for file in inputList:
f.write(file+'\n')
f.close()
This function is defined above and outside of if __name__=='__main__':
I've tried moving InfraPy around in relation to the script. The most baffling situation is that when InfraPy is in the same folder as the script, and I import using from InfraPy import listToText, I receive this error: NameError: name listToText is not defined. Again, the other functions import fine, they are all defined outside of if __name__=='__main__': in InfraPy.
This could happen if the module has __all__ defined
Alternatively there could be another version of the module in your path that is getting imported instead of the one you are expecting
Is the NameError about listToText or is it something inside the function causing the exception?
In addition the __all__ variable gnibbler mentioned you could also have a problem with a InfraPy.pyc file lying around somewhere.
I'd recommend putting a import pdb;pdb.set_trace() first in the InfraPy.py file to make sure you are in the right file, and step through the definition of InfraPy.py to see what is happening. If you don't get a breakpoint, you are importing another file than you think.
You can also dir(InfraPy) after importing it, and check which file you are actually importing with InfraPy.__file__.
Can't think of any more import debugging hints right now. ;-)