Develop Reference

Clean importing of python submodules

I have an directory like this
test.py
foo/__init__.py
foo/bar.py
and in the bar.py a simple function:
def say_stuff()
print("Stuff")
And want to call it from the Test.py like this:
import foo
foo.bar.say_stuff()
Then i get the following error:
AttributeError: module 'foo' has no attribute 'bar'
I know to at least fix that error by editing the __init __.py
from .bar import *
But then both of those approaches work:
foo.bar.say_stuff()
foo.say_stuff()
Now my question, how do i get only the foo.bar.say_stuff() to work.
I dont want to load every thing into foo, already beacuse i want clean auto_complete suggestions from my IDE.
Is there any source you can recommend to properly learn pythons import system. All the tutaorals seem too basic and always run into unexpected behavior.

You just need to change from:
from .bar import *
to:
import foo.bar
This behavior is described here:
https://docs.python.org/3/reference/import.html#submodules

A file import is working when I run the file from inside the module, but not when I run the file by importing the module from outside

My directory structure:
test.py
module/
importer.py
importee.py
__init__.py
So in my directory, I have test.py, then another directory which has been initialized as a module. Within that module, there is a file importer.py which imports a file importee.py. In order to test whether the import works, I made a simple function in importee.py and tried using it in importer.py (i.e. I ran importer.py directly); it worked just fine.
But when I go into test.py and have the import statement from module import * and try to run that (without any other code), it gives an error which traces back to the import statement in importer.py, saying No module named 'importee'
If it matters, the __init__.py in the module directory has the __all__ function specified properly.
I like to think this isn't a duplicate despite there being similarly titled posts, such as this or this or this or this; I've been searching for hours and still have no idea what could be causing this.
Any ideas? Any help is greatly appreciated.
Edit: content of the four files:
init.py
__ all __ = ["importee", "importer"]
importee.py
def example():
print("hey")
importer.py
from importee import *
example()
test.py
from module import *
When I run importer.py I get no errors, but when I run test.py I get a error which traces back to the first line of importer.py saying that No module named 'importee' found, even though I don't get that error when running importer.py directly...

The following runs and prints "hey" regardless of if you run python test.py from root or python importer.py from module.
You can learn more about relative imports from PEP 328. You can learn more about init here, but the important thing I want you to take away from this is that __init__ runs when you import from module.
Furthermore defining __all__ overrides identifiers that begin with _, and since you aren't using them I don't actually know that it would have any effect.
# test.py
from module import *
# module/__init__.py
from .importer import *
# module/importee.py
def example():
print("hey")
# module/importer.py
from .importee import *
example()

Defining and using a decorator function in __init__.py

EDIT: Solved! Solution on the bottom of this post. tl;dr: I should have been using relative imports, and launching python with the correct flag.
So I'm trying to get used to some python stuff (version 2.7.3—it's what's installed on the work PC), and I've got a little project to teach me some python stuff. I've got a module defined like so:
curses_graphics
| __init__.py
| graphicsobject.py
| go_test.py
(code to follow)
I've got a decorator defined in __init__, and I'm trying to use it on methods defined in a class in graphicsobject.py that I'm trying to attach the decorator to, and I'm testing its functionality in go_test.
When I run go_test (just being in the directory and calling "python go_test.py"), I get a no package found error. Same happens if I rename go_test to __main__.py and try to run the package from its parent directory. If I try to run the go_test without importing the package, it doesn't recognise my function.
How can I reference a function defined in __init__.py from within the same package? Is it wrong to try and import while within the package?
Thanks!
__init__.py:
import curses
import logging
#Define logging stuff
gLog = logging.getLogger("cg_log")
gLog.basicConfig(filename="log.log", level=logging.DEBUG)
# Logging decorators for this library
def debug_announce(module_func):
log=logging.getLogger("cg_log")
log.info("Curses Graphics Lib:Entering function:"+module_func.__name__)
module_func()
log.info("Curses Graphics Lib:Left function:"+module_func.__name__)
go_test.py
#debug_announce
def logTester():
print("A test method")
logTester()
logger.debug("Curses initialization")
screen=curses.initscr()
curses.start_color()
screen.keypad(True)
logger.debug("Initializing a green block filled with red '.'s")
block = GraphicsOBject.blankline(0, 0, 3, curses.COLOR_GREEN, curses.COLOR_RED, 20, '.')
logger.debug("Drawing the block.")
block.draw(screen)
logger.debug("Pausing execution with a getch")
screen.getch()
logger.debug("Cleaning up curses")
screen.keypad(False)
curses.endwin()
os.system('stty sane')
I can include graphicsobject.py, though I suspect that would be clutter here, as the issue occurs on the first line of go_test.py
Thanks, everyone!
EDIT:
I'm attaching a capture of the errors reported. In the first error, I've added "from curses_graphics import debug_announce" and in the second the code doesn't have that line.
Errors with and without debug_announce import
EDIT:
Some further searching led me to relative imports. For anyone who has my issue, you use those to import something defined in your own module. In my case, I appended "from . import debug_announce" to the head of my go_test.py file. I attempted to run it, and received the error “Attempted relative import in non-package”.
Some further searching led me to this question:
How to fix "Attempted relative import in non-package" even with __init__.py
Which told me that it wasn't attempting to run this program as a package. This meant the "__init__.py" was never running, as the package wouldn't be imported. Further, since I was inside the package, attempting to import the package (i.e. "import curses_graphics") would result in it searching inside curses_graphics... for curses_graphics.
To run this correctly, as the linked question implies, I need to go to the parent directory of curses_graphics and execute it with "python -m curses_graphics.go_test". This runs the package with its inits and such and run the script of go_test.
(And, FWIW, my code had other issues. Don't take this as an example of how to use curses or logging :P)

Python + PyCharm File Structure issue: AttributeError: 'module' object has no attribute 'X'

I'm having the following file structure:
main.py
Crypto
GetGenerators.py
Utils
RecHash.py
ToInteger.py
Utils.py
GetGenerators.py looks like this:
import unittest
import os, sys
import gmpy2
from gmpy2 import mpz
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from Utils.Utils import AssertInt, AssertClass
from Utils.ToInteger import ToInteger
from Utils.RecHash import RecHash
def GetGenerators(n):
AssertInt(n)
assert n >= 0, "n must be greater than or equal 0"
generators = []
# ... irrelevant code...
return generators
class GetGeneratorsTest(unittest.TestCase):
def testGetGenerators(self):
self.assertEqual(len(GetGenerators(50)), 50)
if __name__ == '__main__':
unittest.main()
When I'm using the function GetGenerators from inside main.py, it works fine.
However, when I'm running the GetGenerators.py UnitTests by rightclicking the file, "Run Unittests in GetGenerators.py", I'm getting the following error:
File "C:\Program Files (x86)\JetBrains\PyCharm 2016.3.2\helpers\pycharm\nose_helper\util.py", line 70, in resolve_name
obj = getattr(obj, part)
AttributeError: 'module' object has no attribute 'GetGenerators'
I suppose it has something to do with the structure of my files, but I don't see the problem.

I haven't had your exact problem before, but I think I've had one like it. When I use PyCharm, I find that if open and use files that I've created in a project in PyCharm, then everything works fine. I can import them, can run them; no problems. The problems I run into (which are similar to yours) are when I open a file that was not created within a PyCharm project. I can't import them, and sometimes can't even run them correctly. Maybe it's just me being stupid or maybe a real bug with PyCharm, but whatever the case is. It might be worth (if you haven't already), create a project in PyCharm and copy and paste the file contents into files you create within PyCharm. For some reason, that has worked for me in the past.

So I've ran into a similar problem with PyCharm 2022.2.2 and this solution didn't help me. Instead, what worked was checking my code to make sure I didn't have any object named 'module' defined anywhere, plus I changed some of the documents like "face_landmarks.py" and "face_recognition.py" into "landmarks.py" to avoid confusion when calling a similar line with face_recognition package in python.
I've also tried marking the project folder as a Namespace package. However, as I've done several things at once, I'm not sure if this had any impact. The problem was resolved, but the issue with file structure is there for PyCharm even 6 years later.

Python mock.patch doesn't patch the correct import

Code
def test_get_network_info(self):
with open(dirname(abspath(__file__)) + '/files/fake_network_info.txt', 'r') as mock_network_info:
with patch('subprocess.check_output', Mock(return_value=mock_network_info.read())):
self.assertEqual('192.168.1.100', get_network_info()[0])
self.assertEqual('255.255.255.0', get_network_info()[1])
self.assertEqual('192.168.1.0', get_network_info()[2])
Error
======================================================================
ERROR: test_get_network_info (tests.test_tools.ToolsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tim/Documents/overseer/app/tests/test_tools.py", line 21, in test_get_network_info
with patch('subprocess.check_output', Mock(return_value=mock_network_info.read())):
File "/usr/local/lib/python2.7/dist-packages/mock.py", line 1268, in __enter__
original, local = self.get_original()
File "/usr/local/lib/python2.7/dist-packages/mock.py", line 1242, in get_original
"%s does not have the attribute %r" % (target, name)
AttributeError: <module 'subprocess' from '/usr/local/lib/python2.7/dist-packages/twill/other_packages/subprocess.pyc'> does not have the attribute 'check_output'
What I understand
My understanding of the problem is that mock is trying to mock twill's subprocess module instead of the python one.
Questions
Am I doing something wrong ?
How can I specify that I want to patch the python subprocess module and not the twill's one ? (that may have been imported earlier in the test suite)**
Is there another way to patch the subprocess module ?
What I tried
I tried with patch('tools.subprocess.check_output', ...
Doesn't work.
I tired to use a decorator ...
Doesn't work either
I tired to patch directly the subprocess module subprocess.check_output = Mock( ...
Works but it's not good since it doesn't undo the patching.
Some more informations
If I run just this test and no other tests, it works because twill's subprocess module never got imported. But as soon as I run a test using twill, the above test will fail.
Here is the twill's version of subprocess wich looks like it has been copy pasted from an old version of python. It doesn't have any check_output function and that's why the test fails.
Twill's package comes from the Flask-Testing plugin which I use extensively. I submitted an issue on github here.
I hope someone from the lovely python community can help. :)

See my comment up there, due to bad practices in twill, the proper way would be to either fix twill, which may take some work, or move away to something else, but since you now heavily depend on Flask-Testing, it's not a cheap move either.
So this leaves us with a dirty trick: make sure to import subprocess anywhere before twill is imported. Internally, this will add a reference to the right subprocess module in sys.modules. Once a module is loaded, all subsequents import won't look anymore in sys.path but just use the reference already cached in sys.modules.
Unfortunately this is maybe not the end of the problem. Apparently twill uses a patched version of subprocess for some reason ; and those patches won't be available for him, since the plain built-in subprocess will be loaded instead. It's very likely it'll crash or behave in an unexpected way. If it's the case, well ... back to the suggestions above.