Develop Reference

is there any point in using relative paths in Python import statement?

I have a Python package called Util. It includes a bunch of files. Here is the include statements on top of one of the files in there:
from config_util import ConfigUtil #ConfigUtil is a class inside the config_util module
import error_helper as eh
This works fine when I run my unit tests.
When I install Util in a virtual environment in another package everything breaks. I will need to change the import statements to
from Util.config_util import ConfigUtil
from Util import error_helper as eh
and then everything works as before. So is there any point in using the first form or is it safe to say that it is better practice to always use the second form?
If there is no point in using the first form, then why is it allowed?

Just wrong:
from config_util import ConfigUtil
import error_helper as eh
It will only work if you happen to be in the directory Util, so that the imports resolve in the current working directory. Or you have messed with sys.path using some bad hack.
Right (using absolute imports):
from Util.config_util import ConfigUtil
import Util.error_helper as eh
Also right (using relative imports):
from .config_util import ConfigUtil
import .error_helper as eh
There is no particular advantage to using relative imports, only a couple of minor things I can think of:
Saves a few bytes in the source file (so what / who cares?)
Enables you to rename the top level without editing import statements in source code (...but how often do you do that?)

For your practical problems, maybe this answer can help you.
Regarding your direct question: there's not a lot to it, but they let you move files and rename containing directories more easily. You may also prefer relative imports for stylistic reasons; I sure do.
The semantics are the same if the paths are correct. If your module is foo.bar, then from foo.bar.baz import Baz and from .baz import Baz are the same. If they don't do the same, then you're likely calling your Python file as a script (python foo/bar.py), in which case it will be module __main__ instead of foo.bar.

Unable to split up large Python module

In Python 2.7, I'm getting
'module' has no attribute
, and/or
'name' is not defined
errors when I try to split up a large python file.
(I have already read similar posts and the Python modules documentation)
Say you have a python file that is structured like this:
<imports>
<50 global variables defined>
<100 lengthy functions that each use most or all of the globals
defined above, and also call each other>
<main() that calls some of the functions and uses the globals>
So I can easily categorize groups of functions together, create a python file for each group, and put them there. The problem is whenever I try to call any of them from the main python file, I get the errors listed above. I think the problem is related to circular dependencies. Since all of the functions rely on the globals, and each other, they are circularly dependent.
If I have main_file.py, group_of_functions_1.py, and group_of_functions_2.py,
main_file.py will have:
import group_of_functions_1.py
import group_of_functions_2.py
and group_of_functions_1.py will have
import main_file.py
import group_of_functions_2.py
and group_of_functions_2.py will have
import main_file.py
import group_of_functions_1.py
Regardless of whether I use "import package_x" or "from package_x import *" the problem remains.
If I take the route of getting rid of the globals, then most of the functions will have 50 parameters they will be passing around which then also need to be returned
What is the right way to clean this up?
(I have already read similar posts and the Python modules documentation)

One of the sources of your errors is likely the following:
import group_of_functions_1.py
import group_of_functions_2.py
When importing, you don't add .py to the end of the module name. Do this instead:
import group_of_functions_1
import group_of_functions_2

import module_name Vs __import__('module_name')

I am writing a python module and I am using many imports of other different modules.
I am bit confused that whether I should import all the necessary dependent modules in the opening of the file or shall I do it when necessary.
I also wanted to know the implications of both.
I come from C++ back ground so I am really thrilled with this feature and does not see any reason of not using __import__(), importing the modules only when needed inside my function.
Kindly throw some light on this.

To write less code, import a module at the first lines of the script, e.g.:
#File1.py
import os
#use os somewhere:
os.path.chdir(some_dir)
...
...
#use os somewhere else, you don't need to "import os" everywhere
os.environ.update(some_dict)
While sometimes you may need to import a module locally (e.g., in a function):
abc=3
def foo():
from some_module import abc #import inside foo avoids you from naming conflicts
abc(...) #call the function, nothing to do with the variable "abc" outside "foo"
Don't worry about the time consumption when calling foo() multiple times, since import statements loads modules/functions only one time. Once a module/function is imported, the object is stored in dictionary sys.modules, which is a lookup table for speedup when running the same import statement.
As #bruno desthuilliers mentioned, importing insede functions may not be that pythonic, it violates PEP8, here's a discussion I found, you should stick to importing at the top of the file most of the time.

First, __import__ isn't usually needed anywhere. It's main purpose is to support dynamic importing of things that you don't know ahead of time (think plug-ins). You can easily use the import statement inside your function:
import sys
def foo():
import this
if __name__ == "__main__":
print sys.version_info
foo()
The main advantage to importing everything up-front is that it is most customary. That's where people reading your code will go to see if something is imported or not. Also, you don't need to write import os in every function that uses os. The main downsides of this approach are that:
you can get yourself into unresolvable import loops (A imports B which imports A)
that you pull everything into memory even if you aren't going to use it.
The second problem isn't typically an issue -- very rarely do you notice the performance or memory impact of an import.
If you run into the first problem, it's likely a symptom of poorly grouped code and the common stuff should be factored into a new module C which both A and B can use.

Firstly, it's a violation of PEP8 using imports inside functions.
Calling import it's an expensive call EVEN if the module is already loaded, so if your function is gonna being called many times this will not compensate the performance gain.
Also when you call "import test" python do this:
dataFile = __ import__('test')
The only downside of imports at the top of file it's the namespace that get polluted very fast depending on complexity of the file, but if your file it's too complex it's a signal of bad design.

Proper way to reload a python module from the console

I'm debugging from the python console and would like to reload a module every time I make a change so I don't have to exit the console and re-enter it. I'm doing:
>>> from project.model.user import *
>>> reload(user)
but I receive:
>>>NameError: name 'user' is not defined
What is the proper way to reload the entire user class? Is there a better way to do this, perhaps auto-updating while debugging?
Thanks.

As asked, the best you can do is
>>> from project.models.user import *
>>> import project # get module reference for reload
>>> reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
it would be better and cleaner if you used the user module directly, rather than doing import * (which is almost never the right way to do it). Then it would just be
>>> from project.models import user
>>> reload(user)
This would do what you want. But, it's not very nice. If you really need to reload modules so often, I've got to ask: why?
My suspicion (backed up by previous experience with people asking similar questions) is that you're testing your module. There are lots of ways to test a module out, and doing it by hand in the interactive interpreter is among the worst ways. Save one of your sessions to a file and use doctest, for a quick fix. Alternatively, write it out as a program and use python -i. The only really great solution, though, is using the unittest module.
If that's not it, hopefully it's something better, not worse. There's really no good use of reload (in fact, it's removed in 3.x). It doesn't work effectively-- you might reload a module but leave leftovers from previous versions. It doesn't even work on all kinds of modules-- extension modules will not reload properly, or sometimes even break horribly, when reloaded.
The context of using it in the interactive interpreter doesn't leave a lot of choices as to what you are doing, and what the real best solution would be. Outside it, sometimes people used reload() to implement plugins etc. This is dangerous at best, and can frequently be done differently using either exec (ah the evil territory we find ourselves in), or a segregated process.

For python3.4+, reload has been moved to the importlib module. you can use importlib.reload(). You can refer to this post.
>>> import importlib
>>> import project # get module reference for reload
>>> importlib.reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
For python3 versions before 3.4, the module to import is imp (instead of importlib)

IPython can reload modules before executing every new line:
%load_ext autoreload
%autoreload 2
Where %autoreload 2reloads "all modules (except those excluded by %aimport) every time before executing the Python code typed."
See the docs:
https://ipython.org/ipython-doc/3/config/extensions/autoreload.html

You can't use reload() in a effective way.
Python does not provide an effective support for reloading or unloading of previously imported
modules; module references makes it impractical to reload a module because references could exist in many places of your program.
Python 3 has removed reload() feature entirely.

Unfortunately you've got to use:
>>> from project.model import user
>>> reload(user)
I don't know off the top of my head of something which will automatically reload modules at the interactive prompt… But I don't see any reason one shouldn't exist (in fact, it wouldn't be too hard to implement, either…)
Now, you could do something like this:
from types import ModuleType
import sys
_reload_builtin = reload
def reload(thing):
if isinstance(thing, ModuleType):
_reload_builtin(thing)
elif hasattr(thing, '__module__') and thing.__module__:
module = sys.modules[thing.__module__]
_reload_builtin(module)
else:
raise TypeError, "reload() argument must be a module or have an __module__"

You could also try twisted.python.rebuild.rebuild.

from test_reload import add_test
where test_reload is a module, and add_test is a function
if you changed the function add_test, of course you need to reload this function.
then you can do this:
import imp
imp.reload(test_reload)
from test_reload import add_test
this will refresh the function add_test.
so you need to add
imp.reload(test_reload)
from test_reload import add_test --add this line in your code

As of Python 3.4 you can use importlib.reload(module)
>>> from importlib import reload
>>> from project.model import user
>>> reload(user)

python refresh/reload

This is a very basic question - but I haven't been able to find an answer by searching online.
I am using python to control ArcGIS, and I have a simple python script, that calls some pre-written code.
However, when I make a change to the pre-written code, it does not appear to result in any change. I import this module, and have tried refreshing it, but nothing happens.
I've even moved the file it calls to another location, and the script still works fine. One thing I did yesterday was I added the folder where all my python files are to the sys path (using sys.append('path') ), and I wonder if that made a difference.
Thanks in advance, and sorry for the sloppy terminology.

It's unclear what you mean with "refresh", but the normal behavior of Python is that you need to restart the software for it to take a new look on a Python module and reread it.
If your changes isn't taken care of even after restart, then this is due to one of two errors:
The timestamp on the pyc-file is incorrect and some time in the future.
You are actually editing the wrong file.
You can with reload re-read a file even without restarting the software with the reload() command. Note that any variable pointing to anything in the module will need to get reimported after the reload. Something like this:
import themodule
from themodule import AClass
reload(themodule)
from themodule import AClass

One way to do this is to call reload.
Example: Here is the contents of foo.py:
def bar():
return 1
In an interactive session, I can do:
>>> import foo
>>> foo.bar()
1
Then in another window, I can change foo.py to:
def bar():
return "Hello"
Back in the interactive session, calling foo.bar() still returns 1, until I do:
>>> reload(foo)
<module 'foo' from 'foo.py'>
>>> foo.bar()
'Hello'
Calling reload is one way to ensure that your module is up-to-date even if the file on disk has changed. It's not necessarily the most efficient (you might be better off checking the last modification time on the file or using something like pyinotify before you reload), but it's certainly quick to implement.
One reason that Python doesn't read from the source module every time is that loading a module is (relatively) expensive -- what if you had a 300kb module and you were just using a single constant from the file? Python loads a module once and keeps it in memory, until you reload it.

If you are running in an IPython shell, then there are some magic commands that exist.
The IPython docs cover this feature called the autoreload extension.
Originally, I found this solution from Jonathan March's blog posting on this very subject (see point 3 from that link).
Basically all you have to do is the following, and changes you make are reflected automatically after you save:
In [1]: %load_ext autoreload
In [2]: %autoreload 2
In [3]: Import MODULE
In [4]: my_class = Module.class()
my_class.printham()
Out[4]: ham
In [5]: #make changes to printham and save
In [6]: my_class.printham()
Out[6]: hamlet

I used the following when importing all objects from within a module to ensure web2py was using my current code:
import buttons
import table
reload(buttons)
reload(table)
from buttons import *
from table import *

I'm not really sure that is what you mean, so don't hesitate to correct me. You are importing a module - let's call it mymodule.py - in your program, but when you change its contents, you don't see the difference?
Python will not look for changes in mymodule.py each time it is used, it will load it a first time, compile it to bytecode and keep it internally. It will normally also save the compiled bytecode (mymodule.pyc). The next time you will start your program, it will check if mymodule.py is more recent than mymodule.pyc, and recompile it if necessary.
If you need to, you can reload the module explicitly:
import mymodule
[... some code ...]
if userAskedForRefresh:
reload(mymodule)
Of course, it is more complicated than that and you may have side-effects depending on what you do with your program regarding the other module, for example if variables depends on classes defined in mymodule.
Alternatively, you could use the execfile function (or exec(), eval(), compile())

I had the exact same issue creating a geoprocessing script for ArcGIS 10.2. I had a python toolbox script, a tool script and then a common script. I have a parameter for Dev/Test/Prod in the tool that would control which version of the code was run. Dev would run the code in the dev folder, test from test folder and prod from prod folder. Changes to the common dev script would not run when the tool was run from ArcCatalog. Closing ArcCatalog made no difference. Even though I selected Dev or Test it would always run from the prod folder.
Adding reload(myCommonModule) to the tool script resolved this issue.

The cases will be different for different versions of python.
Following shows an example of python 3.4 version or above:
hello import hello_world
#Calls hello_world function
hello_world()
HI !!
#Now changes are done and reload option is needed
import importlib
importlib.reload(hello)
hello_world()
How are you?
For earlier python versions like 2.x, use inbuilt reload function as stated above.
Better is to use ipython3 as it provides autoreload feature.