My directory looks like this :
- HttpExample:
- __init__.py
- DBConnection.py
- getLatlong.py
I want to import DBConnection and import getLatlong in __init__.py. There is no error in my __init__.py until I run it, I received :
System.Private.CoreLib: Exception while executing function: Functions.HttpExample. System.Private.CoreLib: Result: Failure
Exception: ModuleNotFoundError: No module named 'getLatlong'
I'm trying to use function in getLatlong to use the information input by user from __init__.py to getLatlong. Below is the code:
__init__.py :
from getLatlong import result
from DBConnection import blobService, container_name, account_key, file_path
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
section = req.params.get('section')
bound = req.params.get('bound')
km_location = req.params.get('km_location')
location = req.params.get('location')
if not section:
try:
req_body = req.get_json()
except ValueError:
pass
else:
section = req_body.get('section')
if section and bound and km_location:
result(section, km_location, bound, location).getResult() #HERE
return func.HttpResponse(f"Hello {section}, {bound}!")
#elif section is None or bound is None or km_location is None:
# return func.HttpResponse("All values are mandatory!")
I am also receiving compile error at getLatlong to import DBConnection to this class. The following values will pass to getLatlong.py. The code :
from DBConnection import blobService, container_name, account_key, file_path #Another import error here says : Unable to import DBConnection
class result:
def __init__(self, section, bound, km_location, location):
self.section = section
self.bound = bound
self.km_location = km_location
self.location = location
def getResult(self):
print(self.section)
print(self.bound)
print(self.km_location)
print(self.location)
I've tried every way to import these files before I lost my mind..
You get these errors, because Python does not know where to look for the files you want to import. Depending on which Python version you are using, I see three ways to solve this:
You could add HttpExample to your PYTHONPATH and than your imports should work as you have them currently.
Another way would be to use the sys module and append the path to HttpExample, e.g.
import sys
sys.path.append('PATH/TO/HttpExample')
But you would have to do this in all files, where you want to import something from the parent folder.
Or you use relative imports, which have been available since Python 2.5 (See PEP238). Those are only available in modules, but as you have your __init__.py file, it should work. For relative imports you are using dots . to tell Python where to look for the import. One dot . tells Python to look for the desired import in the parent folder. You could also use .. to go up two levels. But one level should be enough in your case.
So in your case changing your code to this, should solve your problem.
In __init.py__:
from .getLatlong import result
from .DBConnection import blobService, container_name, account_key, file_path
In getLangLong.py:
from .DBConnection import blobService, container_name, account_key, file_path
You could try from __app__.HttpExample import getLatlong.
There is a document about how to import module in the Shared Code folder. Check this doc:Folder structure.
It says Shared code should be kept in a separate folder in __app__. And in my test this could work for me.
Related
I have a Python project in which I have the following folder structure:
> root
> download_module
> __init__.py
> downloadProcess.py
> sharedFunctions.py
> someHelper.py
> useSharedFunction.py
The download_module/__init__.py has the following code:
from .sharedFunctions import stringArgumentToDate
from .downloadProcess import downloadProcessMethod
The sharedFunctions.py file contains the following function:
def stringArgumentToDate(arg):
dateformat = "%m/%d/%Y"
date = None
if arg.isnumeric():
date = datetime.fromtimestamp(int(arg))
if date == None:
date = datetime.strptime(arg, dateformat)
return date
Then on the useSharedFunction.py I try to import the shared function and use it like this.
from download_module import stringArgumentToDate
from download_module import downloadProcessMethod
def main():
arg = '03/14/2022'
dateArg = stringArgumentToDate(arg)
if __name__ == '__main__':
main()
When I try to run this by using python3 useSharedFunction.py I got the following error:
Traceback (most recent call last):
File "useSharedFunction.py", line 4, in <module>
from download_module import stringArgumentToDate
File "/Users/jacobo/Documents/project/download_module/__init__.py", line 2, in <module>
from .download_module import downloadAndProcessMethod
File "/Users/jacobo/Documents/project/download_module/downloadProcess.py", line 10, in <module>
from sharedFunctions import stringArgumentToDate, otherFunction
ModuleNotFoundError: No module named 'sharedFunctions'
I do believe the error is in downloadProcess since at the beggining of the file we got this import:
from sharedFunctions import stringArgumentToDate, otherFunction
from someHelper import Helper
Which refers to sibling files.
However I'm unsure what will be a proper fix to allow to run the downloadProcess.py main independently but also, being able to call it one of its method from a root or any other file out of the module.
Consider this structure:
┬ module
| ├ __init__.py
| ├ importing_submodule.py
| └ some_submodule.py
├ __main__.py
├ some_submodule.py
└ module_in_parent_dir.py
with content:
__main__.py
import module
/module/__init__.py
from . import importing_submodule
/module/importing_submodule.py
from some_submodule import SomeClass
/module/some_submodule.py
print("you imported from module")
class SomeClass:
pass
/some_submodule.py
print("you imported from root")
class SomeClass:
pass
/module_in_parent_dir.py
class SomeOtherClass:
pass
How sibling import works
(skip this section if you know already)
Now lets run __main__.py and it will say "you imported from root".
But if we change code a bit..
/module/importing_submodule.py
from module.some_submodule import SomeClass
It now says "You imported from module" as we wanted, probably with scary red line in IDE saying "Unresolved reference" if you didn't config working directory in IDE.
How this happen is simple: script root(Current working directory) is decided by main script(first script that's running), and python uses namespaces.
Python's import system uses 2 import method, and for convenience let's call it absolute import and relative import.
Absolute import: Import from dir listed in sys.path and current working directory
Relative import: Import relative to the very script that called import
And what decide the behavior is whether we use . at start of module name or not.
Since we imported by from some_submodule without preceeding dot, python take it as 'Absolute import'(the term we decided earlier).
And then when we also specified module name like from module.some_submodule python looks for module in path list or in current working directory.
Of course, this is never a good idea; script root can change via calls like os.chdir() then submodules inside module folder may get lost.
Therefore, the best practices for sibling import is using relative import inside module folder.
/module/importing_submodule.py
from .some_submodule import SomeClass
Making script that work in both way
To make submodule import it's siblings when running as main script, yet still work as submodule when imported by other script, then use try - except and look for ImportError.
For importing_submodule.py as an example:
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
Importing modules from parent directory is a bit more tricky.
Since submodule is now main script, relative import to parent level directory doesn't work.
So we need to add the parent directory to sys.path, when the script is running as main script.
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
# now since we don't have parent package, we just append the path.
from sys import path
import pathlib
path.append(pathlib.Path(__file__).parent.parent.as_posix())
print("Importing module_in_parent_dir from sys.path")
else:
print("Importing module_in_parent_dir from working directory")
# Now either case we have parent directory of `module_in_parent_dir`
# in working dir or path, we can import it
# might need to suppress false IDE warning this case.
# noinspection PyUnresolvedReferences
from module_in_parent_dir import SomeOtherClass
Output:
"C:\Program Files\Python310\python.exe" .../module/importing_module.py
you imported from module
Importing module_in_parent_dir from sys.path
Process finished with exit code 0
"C:\Program Files\Python310\python.exe" .../__main__.py
you imported from module
Importing module_in_parent_dir from working directory
Process finished with exit code 0
I came across this python class ResourcesMoveInfo for moving resources(Azure Images) from one subscription to another with Azure python SDK.
But it's failing when I use it like below:
Pattern 1
reference from https://buildmedia.readthedocs.org/media/pdf/azure-sdk-for-python/v1.0.3/azure-sdk-for-python.pdf
Usage:
metadata = azure.mgmt.resource.resourcemanagement.ResourcesMoveInfo(resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
Error:
AttributeError: module 'azure.mgmt.resource' has no attribute 'resourcemanagement'
Pattern 2
reference from - https://learn.microsoft.com/en-us/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.v2019_07_01.models.resourcesmoveinfo?view=azure-python
Usage:
metadata = azure.mgmt.resource.resources.v2020_06_01.models.ResourcesMoveInfo(resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
Error:
AttributeError: module 'azure.mgmt.resource.resources' has no attribute 'v2020_06_01'
Any help on this requirement/issue would be helpful. Thanks!
Adding code snippet here:
import sys
import os
import time
from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.resource import ResourceManagementClient
import azure.mgmt.resource
#from azure.mgmt.resource.resources.v2020_06_01.models import ResourcesMoveInfo
from azure.identity import ClientSecretCredential
from cred_wrapper import CredentialWrapper
class Move():
def __init__(self):
self.nonprod_subscription_id = "abc"
self.prod_subscription_id = "def"
self.credential = ClientSecretCredential(
client_id= os.environ["ARM_CLIENT_ID"],
client_secret= os.environ["ARM_CLIENT_SECRET"],
tenant_id= os.environ["ARM_TENANT_ID"]
)
#resource client for nonprod
self.sp = CredentialWrapper(self.credential)
self.resource_client = ResourceManagementClient(self.sp,self.nonprod_subscription_id)
self.resource_group = "imgs-rg"
def getresourceids(self):
resource_ids = list(resource.id for resource in self.resource_client.resources.list_by_resource_group("{0}".format(self.resource_group)) if resource.id.find("latest")>=0)
return resource_ids
def getresourcenames(self):
resource_names = list(resource.name for resource in self.resource_client.resources.list_by_resource_group("{0}".format(self.resource_group)) if resource.id.find("latest")>=0)
return resource_names
def deleteoldimages(self,name):
#resource client id for prod
rc = ResourceManagementClient(self.sp,self.prod_subscription_id)
for resource in rc.resources.list_by_resource_group("{0}".format(self.resource_group)):
if resource.name == name:
#2019-12-01 is the latest api_version supported for deleting the resource type "image"
rc.resources.begin_delete_by_id(resource.id,"2020-06-01")
print("deleted {0}".format(resource.name))
def moveimages(self):
rnames = self.getresourcenames()
for rname in rnames:
print(rname)
#self.deleteoldimages(rname)
time.sleep(10)
rids = self.getresourceids()
rid = list(rids[0:])
print(rid)
metadata = azure.mgmt.resource.resources.v2020_06_01.models.ResourcesMoveInfo(resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
#moving resources in the rid from nonprod subscription to prod subscription under the resource group avrc-imgs-rg
if rid != []:
print("moving {0}".format(rid))
print(self.resource_client.resources.move_resources(source_resource_group_name="{0}".format(self.resource_group),parameters=metadata))
#self.resource_client.resources.move_resources(source_resource_group_name="{0}".format(self.resource_group),resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
#self.resource_client.resources.begin_move_resources(source_resource_group_name="{0}".format(self.resource_group),parameters=metadata)
if __name__ == "__main__":
Move().moveimages()
From your inputs we can see that the code looks fine. From your error messages, the problem is with importing the modules.
Basically when we import a module few submodules will get installed along with and few will not. This will depend on the version of the package, to understand which modules are involved in a specific version we need to check for version-releases in official documentation.
In your case, looks like some resource modules were missing, if you could see the entire error-trace, there will be a path with sitepackages in our local. Try to find that package and its subfolder(Modules) and compare them with Azure SDK for Python under Resource module, you can find this here.
In such situation we need to explicitly add those sub modules under our package. In your case you can simple download the packaged code from Git link which I gave and can merge in your local folder.
I'm trying to test a tool I'm building which uses some jMetalPy functionality. I had/have a previous version working but I am now trying to refactor out some external dependencies (such as the aforementioned jMetalPy).
Project Code & Structure
Here is a minimalist structure of my project.
MyToolDirectory
¦--/MyTool
¦----/__init__.py
¦----/_jmetal
¦------/__init__.py
¦------/core
¦--------/quality_indicator.py
¦----/core
¦------/__init__.py
¦------/run_manager.py
¦----/tests
¦------/__init__.py
¦------/test_run_manager.py
The _jmetal directory is to remove external dependency on the jMetalPy package - and I have copied only the necessary packages/modules that I need.
Minimal contents of run_manager.py
# MyTool\core\run_manager.py
import jmetal
# from jmetal.core.quality_indicators import HyperVolume # old working version
class RunManager:
def __init__(self):
pass
#staticmethod
def calculate_hypervolume(front, ref_point):
if front is None or len(front) < 1:
return 0.
hv = jmetal.core.quality_indicator.HyperVolume(ref_point)
# hv = HyperVolume(ref_point)
hypervolume = hv.compute(front)
return hypervolume
Minimal contents of test_run_manager.py
# MyTool\tests\test_run_manager.py
import unittest
from unittest.mock import MagicMock, Mock, patch
from MyTool import core
class RunManagerTest(unittest.TestCase):
def setUp(self):
self.rm = core.RunManager()
def test_calculate_hypervolume(self):
ref_points = [0.0, 57.5]
front = [None, None]
# with patch('MyTool.core.run_manager.HyperVolume') as mock_HV: # old working version
with patch('MyTool.core.run_manager.jmetal.core.quality_indicator.HyperVolume') as mock_HV:
mock_HV.return_value = MagicMock()
res = self.rm.calculate_hypervolume(front, ref_points)
mock_HV.assert_called_with(ref_points)
mock_HV().compute.assert_called_with(front)
Main Question
When I run a test with the code as-is, I get this error message:
E ModuleNotFoundError: No module named 'MyTool.core.run_manager.jmetal'; 'MyTool.core.run_manager' is not a package
But when I change it to:
with patch('MyTool.core.run_manager.jmetal.core') as mock_core:
mock_HV = mock_core.quality_indicator.HyperVolume
mock_HV.return_value = MagicMock()
res = self.rm.calculate_hypervolume(front, ref_points)
mock_HV.assert_called_with(ref_points)
mock_HV().compute.assert_called_with(front)
... now the test passes. What gives?!
Why can't (or rather, how can) I surgically patch the exact class I want (i.e., HyperVolume) without patching out an entire sub-package as well? Is there a way around this? There may be code in jmetal.core that needs to run normally.
Is the reason this isn't working only because there is no from . import quality_indicator statement in jMetalPy's jmetal\core\__init__.py ?
Because even with patch('MyTool.core.run_manager.jmetal.core.quality_indicator) throws:
E AttributeError: <module 'jmetal.core' from 'path\\to\\venv\\lib\\site-packages\\jmetal\\core\\__init__.py'> does not have the attribute 'quality_indicator'
Or is there something I'm doing wrong?
In the case that it is just about adding those import statements, I could do that in my _jmetal sub-package, but I was hoping to let the user default to their own jMetalPy installation if they already had one by adding this to MyTool\__init__.py:
try:
import jmetal
except ModuleNotFoundError:
from . import _jmetal as jmetal
and then replacing all instances of import jmetal with from MyTool import jmetal. However, I'd run into the same problem all over again.
I feel that there is some core concept I am not grasping. Thanks for the help.
I have three files under the same directory, namely, main.py, Newtester.py, and fileUtility.py. In Newtester.py there is a class named Function. In main.py, there are the following codes:
from file.py import *
...
def main():
...
funcs = parseFuncSpec(funcInputFile)
parseFuncSpec is defined in fileUtilities.py as:
some code to import Newtester.py
def parseFuncSpec(fName):
curFunc = function(funcName, numTest, [], score)
Regardless of what I put in import Newtester.py, I always get an error saying "Function" (the class defined in the file "Newtester.py") is not defined. Following Python: How to import other Python files, I have attempted
import Newtester
__import__("Newtester")
exec("Newtester.py")
exec("Newtester")
import importlib
importlib.__import__("Newtester")
os.system("Newtester.py")
But none of them seemed to work. Any advice is appreciated. See https://github.com/r2dong/unitTesting if you are interested in seeing the complete files.
It's because you are not using it correctly
well when you use import statement like below only Newstester file is imported
import Newtester
hence instead of using parseFuncSpec() directly you have to use it as Newtester.parseFuncSpec()
or to use parseFuncSpec() directly you need to use below import statement:
from Newtester import parseFuncSpec
Im trying to import files on Flask app in base of url route. I started to coding python few days ago so i havent idea if i doing it well. I write this on :
#app.route('/<file>')
def call(file):
__import__('controller.'+file)
hello = Example('Hello world')
return hello.msg
And i have other file called example.py into a controller folder that contains this:
class Example:
def __init__(self, msg):
self.msg = msg
So i start from terminal the app and i try to enter to localhost:5000/example.
Im trying to show in screen Hello world but give me the next error:
NameError: global name 'Example' is not defined
Thanks for all!
__import__ returns the newly imported module; names from that module are not added to your globals, so you need to get the Example class as an attribute from the returned module:
module = __import__('controller.'+file)
hello = module.Example('Hello world')
__import__ is rather low-level, you probably want to use importlib.import_module() instead:
import importlib
module = importlib.import_module('controller.'+file)
hello = module.Example('Hello world')
If you need to dynamically get the classname too, use getattr():
class_name = 'Example'
hello_class = getattr(module, class_name)
hello = hello_class('Hello world')
The Werkzeug package (used by Flask) offers a helpful functions here: werkzeug.utils.import_string() imports an object dynamically:
from werkzeug.utils import import_string
object_name = 'controller.{}:Example'.format(file)
hello_class = import_string(object_name)
This encapsulates the above process.
You'll need to be extremely careful with accepting names from web requests and using those as module names. Please do sanitise the file argument and only allow alphanumerics to prevent relative imports from being used.
You could use the werkzeug.utils.find_modules() function to limit the possible values for file here:
from werkzeug.utils import find_modules, import_string
module_name = 'controller.{}'.format(file)
if module_name not in set(find_modules('controller')):
abort(404) # no such module in the controller package
hello_class = import_string(module_name + ':Example')
I think you might not add the directory to the file, add the following code into the previous python program
# Add another directory
import sys
sys.path.insert(0, '/your_directory')
from Example import Example
There are two ways for you to do imports in Python:
import example
e = example.Example('hello world')
or
from example import Example
e = Example('hello world')