I made a big mistake by creating an app inside my Django project called requests that happened a long time ago and the system has already been running for years. now I need to use the requests library and it is being imported like import requests as mentioned in the documentation ... and of course whenever I do this import it imports my app instead of the library .. so how to solve this?
You can try importing the requests/__init__.py file directly as shown in docs: https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly.
Example:
import sys
import importlib.util
module_name = 'requests'
# declare the full path to requests/__init__.py file below
module_path = '/path/to/virtualenv/site-packages/requests/__init__.py'
spec = importlib.util.spec_from_file_location(module_name, module_path)
requests = importlib.util.module_from_spec(spec)
sys.modules[module_name] = requests
spec.loader.exec_module(requests)
print(requests.post) # should not raise error
Related
My goal is to import code into three separate Flask servers. It's not going well. I am on python 3.10.4. I have read perhaps 10 different posts that say things like "put a __init__.py file in your folders" which I have done.
For context I'm not exactly new to Python but I've never learned the importing/module system properly.
I have three Flask servers that run scraping operations on different (but similar) websites. I need them to be separate for various reasons. Anyway, all three need to run the same procedure of getting an IP for a proxy from my proxy provider. For this I have some code:
# we don't need the details here so I snip it to save space
def get_proxy_ip(choice):
r = requests.get(download_list, headers={"Authorization": "Token " + token})
selected_proxy_ip = r.json()["results"][choice]["proxy_address"]
selected_proxy_port = r.json()["results"][choice]["port"]
print(selected_proxy_ip)
return selected_proxy_ip, selected_proxy_port
I want to use this function across all 3 of my Flask servers. Here are some various ways I've tried to import the code into one of the Flask servers:
scrapers/rentCanada/app.py
import requests
from flask import Flask, request, make_response
print("cats")
app = Flask(__name__)
print(__name__, __package__)
# from ..shared.ipgetter import get_proxy_ip
# from ..shared.checker import check_public_ip
# from scrapers.shared.ipgetter import get_proxy_ip
# from scrapers.shared.checker import check_public_ip
import shared.ipgetter as ipgetter
import shared.checker as checker
None of them work.
import shared.ipgetter as ipgetter yields:
cats
__main__ None
Traceback (most recent call last):
File "/home/rlm/Code/canadaAps/scrapers/rentCanada/app.py", line 10, in <module>
import shared.ipgetter as ipgetter
ModuleNotFoundError: No module named 'shared'
ModuleNotFoundError: No module named 'scrapers' yields: ModuleNotFoundError: No module named 'scrapers'
from ..shared.ipgetter import get_proxy_ip yields: ImportError: attempted relative import with no known parent package
At this point you need to see my folder structure.
/scrapers
..__init__.py
..setup.py
../rentCanada
.....__init__.py
.....app.py
../rentFaster
.....__init__.py
.....app.py
../rentSeeker
.....__init__.py
.....app.py
../shared
.....__init__.py
.....ipgetter.py
.....checker.py
I need to be able to use any of the app.py files as entry points.
I also tried setup.py with this:
from setuptools import setup, find_packages
setup(
name = 'tools',
packages = find_packages(),
)
followed by python setup.py install but that didn't make a "tools" import available in app.py like I wanted.
As a final note I suspect someone will tell me to use a blueprint. To me those look like a tool I'd use if I was adding a route. I'm not sure they're right for a simple function, but maybe I'm wrong.
My solution for now is to run Flask with python rentCanada/app.py from the /scrapers folder and use this code
import sys
from pathlib import Path
sys.path.append(str(Path(__file__).parent.parent)) # necessary so util folder is available
import requests
from flask import Flask, request, make_response
print("cats")
app = Flask(__name__)
print(__name__, __package__)
from util.ipgetter import get_proxy_ip
from util.checker import check_public_ip
So the program appends the app.py file folder's parent folder to the path. That makes the util folder (which used to be shared but had a naming conflict) available within the app.py file.
I came across this python class ResourcesMoveInfo for moving resources(Azure Images) from one subscription to another with Azure python SDK.
But it's failing when I use it like below:
Pattern 1
reference from https://buildmedia.readthedocs.org/media/pdf/azure-sdk-for-python/v1.0.3/azure-sdk-for-python.pdf
Usage:
metadata = azure.mgmt.resource.resourcemanagement.ResourcesMoveInfo(resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
Error:
AttributeError: module 'azure.mgmt.resource' has no attribute 'resourcemanagement'
Pattern 2
reference from - https://learn.microsoft.com/en-us/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.v2019_07_01.models.resourcesmoveinfo?view=azure-python
Usage:
metadata = azure.mgmt.resource.resources.v2020_06_01.models.ResourcesMoveInfo(resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
Error:
AttributeError: module 'azure.mgmt.resource.resources' has no attribute 'v2020_06_01'
Any help on this requirement/issue would be helpful. Thanks!
Adding code snippet here:
import sys
import os
import time
from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.resource import ResourceManagementClient
import azure.mgmt.resource
#from azure.mgmt.resource.resources.v2020_06_01.models import ResourcesMoveInfo
from azure.identity import ClientSecretCredential
from cred_wrapper import CredentialWrapper
class Move():
def __init__(self):
self.nonprod_subscription_id = "abc"
self.prod_subscription_id = "def"
self.credential = ClientSecretCredential(
client_id= os.environ["ARM_CLIENT_ID"],
client_secret= os.environ["ARM_CLIENT_SECRET"],
tenant_id= os.environ["ARM_TENANT_ID"]
)
#resource client for nonprod
self.sp = CredentialWrapper(self.credential)
self.resource_client = ResourceManagementClient(self.sp,self.nonprod_subscription_id)
self.resource_group = "imgs-rg"
def getresourceids(self):
resource_ids = list(resource.id for resource in self.resource_client.resources.list_by_resource_group("{0}".format(self.resource_group)) if resource.id.find("latest")>=0)
return resource_ids
def getresourcenames(self):
resource_names = list(resource.name for resource in self.resource_client.resources.list_by_resource_group("{0}".format(self.resource_group)) if resource.id.find("latest")>=0)
return resource_names
def deleteoldimages(self,name):
#resource client id for prod
rc = ResourceManagementClient(self.sp,self.prod_subscription_id)
for resource in rc.resources.list_by_resource_group("{0}".format(self.resource_group)):
if resource.name == name:
#2019-12-01 is the latest api_version supported for deleting the resource type "image"
rc.resources.begin_delete_by_id(resource.id,"2020-06-01")
print("deleted {0}".format(resource.name))
def moveimages(self):
rnames = self.getresourcenames()
for rname in rnames:
print(rname)
#self.deleteoldimages(rname)
time.sleep(10)
rids = self.getresourceids()
rid = list(rids[0:])
print(rid)
metadata = azure.mgmt.resource.resources.v2020_06_01.models.ResourcesMoveInfo(resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
#moving resources in the rid from nonprod subscription to prod subscription under the resource group avrc-imgs-rg
if rid != []:
print("moving {0}".format(rid))
print(self.resource_client.resources.move_resources(source_resource_group_name="{0}".format(self.resource_group),parameters=metadata))
#self.resource_client.resources.move_resources(source_resource_group_name="{0}".format(self.resource_group),resources=rid,target_resource_group='/subscriptions/{0}/resourceGroups/{1}'.format(self.prod_subscription_id,self.resource_group))
#self.resource_client.resources.begin_move_resources(source_resource_group_name="{0}".format(self.resource_group),parameters=metadata)
if __name__ == "__main__":
Move().moveimages()
From your inputs we can see that the code looks fine. From your error messages, the problem is with importing the modules.
Basically when we import a module few submodules will get installed along with and few will not. This will depend on the version of the package, to understand which modules are involved in a specific version we need to check for version-releases in official documentation.
In your case, looks like some resource modules were missing, if you could see the entire error-trace, there will be a path with sitepackages in our local. Try to find that package and its subfolder(Modules) and compare them with Azure SDK for Python under Resource module, you can find this here.
In such situation we need to explicitly add those sub modules under our package. In your case you can simple download the packaged code from Git link which I gave and can merge in your local folder.
I am setting up a package using distutils.
I need to allow access to a module that is built during the set-up process and is located in ./build/temp.linux-x86_64-3.6. I do this by including the
include_dirs=["./build/temp.linux-x86_64-3.6"]
when adding the extension to the distutils Configuration.
My question is there a way of setting this using a wildcard such as:
include_dirs=["./build/temp.linux*"]
as when I try this it fails, citing error:
Nonexistent include directory ‘build/temp.linux*’ [-Wmissing-include-dirs]
The reason I want this is that the build folder will be named differently depending on the system. Alternatively if anyone knows a way of figuring out what this temp build folder will be called that would also work.
The way I have got around this problem is as follows:
def return_major_minor_python():
import sys
return str(sys.version_info[0])+"."+str(sys.version_info[1])
def return_include_dir():
from distutils.util import get_platform
return get_platform()+'-'+return_major_minor_python()
Then when calling config.add_extension() using:
include_dirs=['build/temp.' + return_include_dir()]
So the whole process for adding a f90wrapped, f2py extension to a python package is:
def setup_fort_ext(args,parent_package='',top_path=''):
from numpy.distutils.misc_util import Configuration
from os.path import join
import sys
config = Configuration('',parent_package,top_path)
fort_src = [join('PackageName/','fortran_source.f90')]
config.add_library('_fortran_source', sources=fort_src,
extra_f90_compile_args = [ args["compile_args"]],
extra_link_args=[args["link_args"]])
sources = [join('PackageName','f90wrap_fortran_source.f90')]
config.add_extension(name='_fortran_source',
sources=sources,
extra_f90_compile_args = [ args["compile_args"]],
extra_link_args=[args["link_args"]],
libraries=['_tort'],
include_dirs=['build/temp.' + return_include_dir()])
return config
if __name__ == '__main__':
import sys
import subprocess
import os
install_numpy() #installs numpy
install_dependencies() #calls to pip to install any requirements
from numpy.distutils.core import setup
config = {'name':'PackageName',
'version':__version__,
'project_description':'Some Package description',
'description':'Some package Description',
'long_description': open('README.txt').read(),
'long_description_content_type':'text/markdown',
'author':'Your name here',
'author_email':'your email here',
'url':'link to git repo here',
'python_requires':'>=3.3',
'packages':['PackageName'],
'package_dir':{'PackageName':'PackageName'},
'package_data':{'PackageName':['*so*']},
'name': 'PackageName'
}
config_fort = setup_fort_ext(args,parent_package='PackageName',top_path='')
config2 = dict(config,**config_fort.todict())
setup(**config2)
where the source fortran_source.f90 is wrapped beforehand and the resulting wrapped source file (f90wrap_fortran_source.f90) is included as a library, and subsequently compiled by f2py.
args in the above is just a dict with the any linking or compile args you wish to pass through.
Im trying to import files on Flask app in base of url route. I started to coding python few days ago so i havent idea if i doing it well. I write this on :
#app.route('/<file>')
def call(file):
__import__('controller.'+file)
hello = Example('Hello world')
return hello.msg
And i have other file called example.py into a controller folder that contains this:
class Example:
def __init__(self, msg):
self.msg = msg
So i start from terminal the app and i try to enter to localhost:5000/example.
Im trying to show in screen Hello world but give me the next error:
NameError: global name 'Example' is not defined
Thanks for all!
__import__ returns the newly imported module; names from that module are not added to your globals, so you need to get the Example class as an attribute from the returned module:
module = __import__('controller.'+file)
hello = module.Example('Hello world')
__import__ is rather low-level, you probably want to use importlib.import_module() instead:
import importlib
module = importlib.import_module('controller.'+file)
hello = module.Example('Hello world')
If you need to dynamically get the classname too, use getattr():
class_name = 'Example'
hello_class = getattr(module, class_name)
hello = hello_class('Hello world')
The Werkzeug package (used by Flask) offers a helpful functions here: werkzeug.utils.import_string() imports an object dynamically:
from werkzeug.utils import import_string
object_name = 'controller.{}:Example'.format(file)
hello_class = import_string(object_name)
This encapsulates the above process.
You'll need to be extremely careful with accepting names from web requests and using those as module names. Please do sanitise the file argument and only allow alphanumerics to prevent relative imports from being used.
You could use the werkzeug.utils.find_modules() function to limit the possible values for file here:
from werkzeug.utils import find_modules, import_string
module_name = 'controller.{}'.format(file)
if module_name not in set(find_modules('controller')):
abort(404) # no such module in the controller package
hello_class = import_string(module_name + ':Example')
I think you might not add the directory to the file, add the following code into the previous python program
# Add another directory
import sys
sys.path.insert(0, '/your_directory')
from Example import Example
There are two ways for you to do imports in Python:
import example
e = example.Example('hello world')
or
from example import Example
e = Example('hello world')
I am using python2.5. I need to import a module dynamically and then reload on change. How do I achieve this.
I tried as in below example - does not work:
import imp
modfile = 'mymod_info.py'
modname = 'mymod'
modhandle = imp.load_source (modname, modfile)
reload (modhandle)
Static import and reload works (I am not looking for this):
import mymod_info as mymod
reload (mymod)
Thanks in advance
If the module was already initialized, imp.load_source will initialize the module again. So instead of reload, just call
modhandle = imp.load_source(modname, modfile)