Develop Reference

How to solve Django app name conflict with external library

I made a big mistake by creating an app inside my Django project called requests that happened a long time ago and the system has already been running for years. now I need to use the requests library and it is being imported like import requests as mentioned in the documentation ... and of course whenever I do this import it imports my app instead of the library .. so how to solve this?

You can try importing the requests/__init__.py file directly as shown in docs: https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly.
Example:
import sys
import importlib.util
module_name = 'requests'
# declare the full path to requests/__init__.py file below
module_path = '/path/to/virtualenv/site-packages/requests/__init__.py'
spec = importlib.util.spec_from_file_location(module_name, module_path)
requests = importlib.util.module_from_spec(spec)
sys.modules[module_name] = requests
spec.loader.exec_module(requests)
print(requests.post) # should not raise error

Why do *import ...* and *from x import y* idioms behave so differently here?

I have a package structure like this:
- src
- src/main.py
- src/package1
- src/package1/__init__.py
- src/package1/module1.py
- src/package1/module2.py
... where module2 is a subclass of module1, and therefore module1 gets referenced by an absolute import path in module2.py.
That is, in src/package1/module2.py:
from package1.module1 import SomeClassFromModule1
The problem occurs in the main.py script:
## here the imports
def main():
# create an instance of the child class in Module2
if __name__ == "__main__":
main()
Option 1 works. That is, in src/main.py:
from package1.module2 import SomeClassFromModule2
some_name = SomeClassFromModule2()
Option 2 does not work. That is, in src/main.py:
import package1.module2.SomeClassFromModule2
some_name = package1.module2.SomeClassFromModule2()
... causes the following error.
ModuleNotFoundError: No module named 'package1.module2.SomeClassFromModule2'; 'package1.module2' is not a package
So why is there this difference between the import and from ... import idiom?
Would be glad for some clarification.

import x keyword brings all the methods and class from x in the the file it is being called.
from x import y this brings a specific method or class('y' is a method or class) from that .py file ('x' is the file here) instead of bringing all the methods it has.
In your case when you import package1.module2 the SomeClassForModule2() is being already imported and hence you need not write import package1.module2.SomeClassFromModule2
here I guess you want to access a class, so you need to create a object in order to access it.
hope this helped you

After some test, I think you cannot import a function or class by using import your_module.your_class.
It's all about package, module, function and class:
# import module
>>>import os
<module 'os' from ooxx>
#use module of module (a litte weird)
>>>os.path
<module 'posixpath' from ooxx>
#import module of module (a litte weird)
>>>import os.path
#use function
>>>os.path.dirname
<function posixpath.dirname(p)>
# you cannot import a function (or class) by using 'import your.module.func'
# 'import ooxx' always get a module or package.
>>>import os.path.dirname
ModuleNotFoundError
No module named 'os.path.dirname'; 'os.path' is not a package
# instead of it, using 'from your_module import your_function_or_class'
>>>from os.path import dirname
<function posixpath.dirname(p)>

How to import library modules to my namespace from another module?

I have written this function inside my utils module
def megaimport(library_name):
skip_modules = ["sys", "os", "dir_path"]
exec(f"import {library_name}")
for module in dir(eval(library_name)):
if module.startswith("__") == False and module not in skip_modules:
exec(f"from {library_name}.{module} import *", globals(), locals())
What I am trying to achieve is to import all functions to my namespace from any library with the following structure:
library:
- module1.py
- function_1_1
- function_1_2
- module2.py
- function_2_1
- function_2_2
just by running in my notebook.ipynb:
from utils import megaimport
megaimport("library")
function_1_2()
function_2_1()
... etc
The idea is to dynamically call from library.module1 import *for every module in a library.
When I do this the function runs without errors but I when I call the functions they have not been imported properly. However, if I define the function in my code and run it it works as expected.
How can I import all the functions into my current namespace?
I am aware that importing everything like this is bad practice but for me it is convenient when working with jupyter notebooks.
My notebook.ipynb and utils.py live in the same directory.

Ok, after trying for a while I realised that I should pass globals() as a parameter to the function and pass it over to the exec in order to make it work in my namespace.
The function now looks like this:
import importlib
def megaimport(globals_dict, library_name):
skip_modules = ["sys", "os", "dir_path"]
mod = importlib.import_module(library)
for module in dir(mod):
if module.startswith("__") == False and module not in skip_modules:
exec(f"from {library_name}.{module} import *", globals_dict)
And it can be called just by:
from utils import megaimport
megaimport(globals(),"library")

How to import module without class in Python?

I have 4 files in my project:
project/__init__.py
project/app.py
project/mod_x.py
project/mod_y.py
In mod_x.py I have a class (e.g. ModX)
In mod_y.py I have just one function.
I import modules from app.py as follows:
from .mod_x import ModX
import .mod_y
I get an error:
ImportError: No module named 'mod_y'
Before I created init.py I didn't have that kind of problems (of course, I dont put "." before module name).
How to import module which doesn't have the class inside in Python3 with init.py file inside the current directory?

Relative imports are only available for from...import syntax.
You could import that function this way:
from .mod_y import FUNCTION_NAME
Module could be imported this way:
from . import mod_y

Import custom modules on IPython.parallel engines with sync_imports()

I've been playing around with IPython.parallel and I wanted to use some custom modules of my own, but haven't been able to do it as explained on the cookbook using dview.sync_imports(). The only thing that has worked for me was something like
def my_parallel_func(args):
import sys
sys.path.append('/path/to/my/module')
import my_module
#and all the rest
and then in the main just to
if __name__=='__main__':
#set up dview...
dview.map( my_parallel_func, my_args )
The correct way to do this would in my opinion be something like
with dview.sync_imports():
import sys
sys.path.append('/path/to/my/module')
import my_module
but this throws an error saying there is no module named my_module.
So, what is the right way of doing it using dview.sync_imports()??

The problem is that you're changing the PYTHONPATH just in the local process running the Client, and not in the remote processes running in the ipcluster.
You can observe this behaviour if you run the next piece of code:
from IPython.parallel import Client
rc = Client()
dview = rc[:]
with dview.sync_imports():
import sys
sys.path[:] = ['something']
def parallel(x):
import sys
return sys.path
print 'Local: ', sys.path
print 'Remote: ', dview.map_sync(parallel, range(1))
Basically all the modules that you want to use with sync_imports must already be in the PYTHONPATH.
If it's not in the PYTHONPATH then you must add it to the path in the function that you execute remotely, and then import the module in the function.