Develop Reference

Python module importing with sys.path and os.path issue

I spent some time researching this and I just cannot work this out in my head.
I run a program in its own directory home/program/core/main.py
In main.py I try and import a module called my_module.py thats located in a different directory, say home/program/modules/my_module.py
In main.py this is how I append to sys.path so the program can be run on anyone's machine (hopefully).
import os.path
import sys
# This should give the path to home/program
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..'))
# Which it does when checking with
print os.path.join(os.path.abspath(os.path.dirname(__file__), '..')
# So now sys.path knows the location of where modules directory is, it should work right?
import modules.my_module # <----RAISES ImportError WHY?
However if I simply do:
sys.path.append('home/program/modules')
import my_module
It all works fine. But this is not ideal as it now depends on the fact that the program must exist under home/program.

that's because modules isn't a valid python package, probably because it doesn't contain any __init__.py file (You cannot traverse directories with import without them being marked with __init__.py)
So either add an empty __init__.py file or just add the path up to modules so your first snippet is equivalent to the second one:
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','modules'))
import my_module
note that you can also import the module by giving the full path to it, using advanced import features: How to import a module given the full path?

Although the answer can be found here, for convenience and completeness here is a quick solution:
import importlib
dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")
Now you can directly use the namespace of the imported module, like this:
a = module.myvar
b = module.myfunc(a)

Confused about os module

I am confused about os module
I have learned that use import to import module.
We have file os.py.
Do we have file os.path.py?
os.path.abspath()
I have read the official Python document.
I did not fully understand what it meant.
I try os.path.abspath("xxx")
I found even the file did not exist, it still returns the path.
ex: os.path.abspath("fadsffefsfgg")
then return
'C:\\Users\\user-t\\fadsffefsfgg'

>>> os.path
<module 'ntpath' from 'C:\\Python34\\lib\\ntpath.py'>
so no, os.path.py doesn't exist. In os module there's probably something like:
import ntpath as path
now the path is os.path.
For your second question, os.path.abspath just prepends the current path to any path if it's not absolute.
It's not a file operation, it's a string operation. It doesn't even check if the file exists, and that's for the best: imagine that you want to create such a file...

Run Python codes on subdirectories

How do I run python codes (.py) on subdirectories from the main folder?
What is the easiest way to do this?
I tried:
os.chdir("path") #path = path to subdirectory
import abc #abc = module on subdirectory
Error:
ImportError: No module named abc

I believe you want to import abc into your current module, even though they're located on different folders. Depending on your python, there are different ways to do this:
Python2.x
import imp
abc = imp.load_source('abc', '/path/to/abc.py')
Python 3.4
from importlib.machinery import SourceFileLoader
abc = SourceFileLoader('abc', '/path/to/abc.py').load_module()
In either case, abc will be imported for use as usual.
>>> abc
<module 'abc' from '/path/to/abc.py'>
This is cleaner because it does not involve polluting your sys.path.

Take a look at this
import sys
sys.path
sys.path.append('/path/to/the/example_file.py')
import example_file

well, just do it
import sys
sys.path
sys.path.append('/path/to/the/example_file1.py')
sys.path.append('/path/to/the/example_file2.py')
sys.path.append('/path/to/the/example_file3.py')
import example_file1
import example_file2
import example_file3

Adding python module to python path, afterwards importing it does fail, how come?

Okay, I have somewhere on a network drive a module which I want to use in my script (which is as well on the network drive).
I added that module to the python path:
new_path = '..\\..\\..\\ABC\\DEF\\1.0\\GHI'
new_path = os.path.realpath(new_path)
sys.path.append(new_path)
Afterwards I can see that it was proper added:
for p in sys.path:
print p
But when I try to import it, it fails:
import GHI
The folder that contains GHI does have a __init__.py and I also tried to only put '..\\..\\..\\ABC\\DEF\\1.0' into the python path. This seems to be an easy failure, but I don't see it. :/

You should add the dir where the module resides to sys.path, not the path to the module (or module package dir) itself.
Try:
import os, pprint
new_path = os.path.abspath(r'..\..\..\ABC\DEF\1.0')
assert os.path.isdir(new_path), 'The dir does not exist!')
sys.path.append(new_path)
pprint sys.path
import GHI

Getting a Python Modules Dierctory from Inside Itself

I have a Python module and I'd like to get that modules directory from inside itself. I want to do this because I have some files that I'd like to reference relative to the module.

First you need to get a reference to the module inside itself.
mod = sys.__modules__[__name__]
Then you can use __file__ to get to the module file.
mod.__file__
Its directory is a dirname of that.

As you are inside the module all you need is this:
import os
path_to_this_module = os.path.dirname(__file__)
However, if the module in question is actually your programs entry point, then __file__ will only be the name of the file and you'll need to expand the path:
import os
path_to_this_module = os.path.dirname(os.path.abspath(__file__))

I think this is what you are looking for:
import <module>
import os
print os.path.dirname(<module>.__file__)

You should be using pkg_resources for this, the resource* family of functions do just about everything you need without having to muck about with the filesystem.
import pkg_resources
data = pkg_resources.resource_string(__name__, "some_file")