Develop Reference

Python module importing with sys.path and os.path issue

I spent some time researching this and I just cannot work this out in my head.
I run a program in its own directory home/program/core/main.py
In main.py I try and import a module called my_module.py thats located in a different directory, say home/program/modules/my_module.py
In main.py this is how I append to sys.path so the program can be run on anyone's machine (hopefully).
import os.path
import sys
# This should give the path to home/program
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..'))
# Which it does when checking with
print os.path.join(os.path.abspath(os.path.dirname(__file__), '..')
# So now sys.path knows the location of where modules directory is, it should work right?
import modules.my_module # <----RAISES ImportError WHY?
However if I simply do:
sys.path.append('home/program/modules')
import my_module
It all works fine. But this is not ideal as it now depends on the fact that the program must exist under home/program.

that's because modules isn't a valid python package, probably because it doesn't contain any __init__.py file (You cannot traverse directories with import without them being marked with __init__.py)
So either add an empty __init__.py file or just add the path up to modules so your first snippet is equivalent to the second one:
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','modules'))
import my_module
note that you can also import the module by giving the full path to it, using advanced import features: How to import a module given the full path?

Although the answer can be found here, for convenience and completeness here is a quick solution:
import importlib
dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")
Now you can directly use the namespace of the imported module, like this:
a = module.myvar
b = module.myfunc(a)

Find the absolute path of an imported module

How can I get the absolute path of an imported module?

As the other answers have said, you can use __file__. However, note that this won't give the full path if the other module is in the same directory as the program. So to be safe, do something like this:
>>> import os
>>> import math
>>> os.path.abspath(math.__file__)
'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/math.so'
Here's an example with a module I made called checkIP to illustrate why you need to get the abspath (checkIP.py is in the current directory):
>>> import os
>>> import checkIP
>>> os.path.abspath(checkIP.__file__)
'/Users/Matthew/Programs/checkIP.py'
>>> checkIP.__file__
'checkIP.py'

If it is a module within your PYTHONPATH directory tree (and reachable by placing a __init__.py in its directory and parent directories), then call its path attribute.
>>>import sample_module
>>>sample_module.__path__
['/absolute/path/to/sample/module']

You can try:
import os
print os.__file__
to see where the module is located.

How to import module when module name has a '-' dash or hyphen in it?

I want to import foo-bar.py, this works:
foobar = __import__("foo-bar")
This does not:
from "foo-bar" import *
My question: Is there any way that I can use the above format i.e., from "foo-bar" import * to import a module that has a - in it?

Starting from Python 3.1, you can use importlib :
import importlib
foobar = importlib.import_module("foo-bar")
( https://docs.python.org/3/library/importlib.html )

In Python 2, you can't. foo-bar is not an identifier. rename the file to foo_bar.py
It's possible since Python 3.1+, see Julien's answer.
If import is not your goal (as in: you don't care what happens with sys.modules, you don't need it to import itself), just getting all of the file's globals into your own scope, you can use execfile
# contents of foo-bar.py
baz = 'quux'
>>> execfile('foo-bar.py')
>>> baz
'quux'
>>>

Solution: If you can't rename the module to match Python naming conventions, create a new module to act as an intermediary:
New module foo_proxy.py:
tmp = __import__('foo-bar')
globals().update(vars(tmp))
Module doing the import main.py:
from foo_proxy import *

If you can't rename the original file, you could also use a symlink:
ln -s foo-bar.py foo_bar.py
Then you can just do:
from foo_bar import *

Like other said you can't use a - in python naming, there are many workarounds, one such workaround which would be useful if you had to add multiple modules from a path is using sys.path
For example if your structure is like this:
foo-bar
├── barfoo.py
└── __init__.py
import sys
sys.path.append('foo-bar')
import barfoo

This was my scenario: I have a python library cloned in a git submodule which has a dash in its name:
|- python-my-lib
| `- mylib.py
`- my-script.py
It took me a long time to figure out the equivalent of:
# Do NOT use this!
sys.path.insert(1, './my-lib')
from mylib import MyClass
Appending the path is not an option, as it would only work if you run the script within the same directory. If you do /home/user/bin/my-script.py, this will fail.
This is the solution:
import importlib
mylib_module = importlib.import_module("python-my-lib.mylib")
MyClass = mylib_module.MyClass
Feel free to further improve this solution, if you know a simpler solution.

Python has issues with dash -. So use importlib instead.
You can run your test scripts just like this -
# importlib, because python has issues with dash '-' in module names
import importlib
img2txt = importlib.import_module("img2txt-textextractor")
event_with_txt = {...}
event_with_no_txt = {...}
def test_no_text():
response = img2txt.handler(event=event_with_txt, context='')
assert response["body"] == '"Detect Me If You Can. "'
def test_detected_text():
response = img2txt.handler(event=event_with_no_txt, context='')
assert response["body"] == '"unable to find anything"'
Name your test code as test_someName.py. To run, from the same directory on terminal type -
pytest

in Python 3.6
I had the same problem "invalid syntax" when directly
import 'jaro-winkler' as jw
said
"No module named 'jaro-winkler'" when using:
jw = __import__('jaro-winkler')
and importlib.import_module() same.
finally i use pip uninstall the jaro-winkler module...just FYI

Getting a Python Modules Dierctory from Inside Itself

I have a Python module and I'd like to get that modules directory from inside itself. I want to do this because I have some files that I'd like to reference relative to the module.

First you need to get a reference to the module inside itself.
mod = sys.__modules__[__name__]
Then you can use __file__ to get to the module file.
mod.__file__
Its directory is a dirname of that.

As you are inside the module all you need is this:
import os
path_to_this_module = os.path.dirname(__file__)
However, if the module in question is actually your programs entry point, then __file__ will only be the name of the file and you'll need to expand the path:
import os
path_to_this_module = os.path.dirname(os.path.abspath(__file__))

I think this is what you are looking for:
import <module>
import os
print os.path.dirname(<module>.__file__)

You should be using pkg_resources for this, the resource* family of functions do just about everything you need without having to muck about with the filesystem.
import pkg_resources
data = pkg_resources.resource_string(__name__, "some_file")

Access to aliased functions in a Python module

I am consolidating many shell-like operations into a single module. I would then like to be able to do:
pyscript.py:
from shell import *
basename("/path/to/file.ext")
and the shell.py module contains:
shell.py:
from os.path import basename
The problem is that functions imported to the shell module are not available, since the import statement only includes functions, classes, and globals defined in the "shell" module, not ones that are imported.
Is there any way to get around this?

Are you sure you're not just using the wrong syntax? This works for me in 2.6.
from shell import *
basename("/path/to/file.ext")
shell.py:
from os.path import basename

It's not a bug, it's a feature :-)
If you import m1 in a module m2 and then import m2 into another module, it will only import things from m2, not from m1. This is to prevent namespace pollution.
You could do this:
shell.py:
import os.path
basename = os.path.basename
Then, in pyscript.py, you can do this:
from shell import * # warning: bad style!
basename(...)