I'm triggering an Azure Function that fetches some data and writes it to a SQL db. The file Function works perfectly locally, but when I deploy it to Azure I keep getting the following error:
Result: Failure Exception: ModuleNotFoundError: No module named '__main__'. Troubleshooting
Guide: https://aka.ms/functions-modulenotfound Stack: File "/azure-functions-
host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 301, in
_handle__function_load_request func = loader.load_function( File "/azure-functions-
host/workers/python/3.9/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call
raise extend_exception_message(e, message) File "/azure-functions-
host/workers/python/3.9/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call
return func(*args, **kwargs) File "/azure-functions-
host/workers/python/3.9/LINUX/X64/azure_functions_worker/loader.py", line 83, in load_function
mod = importlib.import_module(fullmodname) File
"/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module return
_bootstrap._gcd_import(name[level:], package, level) File
"/home/site/wwwroot/jobinfo/__init__.py", line 7, in <module> from .auth_and_get import * File
"/home/site/wwwroot/jobinfo/auth_and_get.py", line 9, in <module> load_dotenv() File
"/home/site/wwwroot/.python_packages/lib/site-packages/dotenv/main.py", line 317, in
load_dotenv f = dotenv_path or stream or find_dotenv() File
"/home/site/wwwroot/.python_packages/lib/site-packages/dotenv/main.py", line 265, in
find_dotenv if usecwd or _is_interactive() or getattr(sys, 'frozen', False): File
"/home/site/wwwroot/.python_packages/lib/site-packages/dotenv/main.py", line 262, in
_is_interactive main = __import__('__main__', None, None, fromlist=['__file__'])
In my Function file __init__.py I import a module auth_and_get.py where I've made methods, which I wanna call inside my Azure Function, and I think that something's wrong with my import, but I've tried fixing it without luck.
My current __init__.py file is
from __future__ import absolute_import
import datetime
import logging
import azure.functions as func
from .auth_and_get import *
def main(mytimer: func.TimerRequest) -> None:
utc_timestamp = datetime.datetime.utcnow().replace(
tzinfo=datetime.timezone.utc).isoformat()
if mytimer.past_due:
logging.info('The timer is past due!')
authenticate()
users()
to_db()
logging.info('Python timer trigger function ran at %s', utc_timestamp)
and I suspect that it's the following statment that fails: from .auth_and_get import *. I've tried removing the dot such that from auth_and_get import *, but then the module becomes unresolveable.
The structure is
ProjectFolder/
| - .venv
| - .vscode
|jobinfo/
| | - __pycache__
| | - __init__.py
| | - auth_and_get.py
| | - function.json
| | - sample.dat
| - .funcignore
| - host.json
| - local.settings.json
| - proxies.json
| - requirements.txt
QUESTION
Why do I get the Failure Exception: ModuleNotFoundError: No module named '__main__' error?
So after a variety of attempts, I found a solution by adding
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__))))
to the __init__.py file.
Have you set startup command on Azure Portal for your App Service?
I mean Settings->Configuration->General Settings and then in the field "Startup Command" you should have something like:
gunicorn --bind=0.0.0.0 --timeout 600 --chdir jobinfo __init__:<app object>
Of course you can set another command (like non-Gunicorn), but it is an example based on Azure docs.
Azure by default looks for main module named app.py or application.py.
It's explained here: https://learn.microsoft.com/en-us/azure/app-service/configure-language-python#customize-startup-command
ModuleNotFoundError: No module named '__main__' means that Azure looks for module named __main__ instead of __init__.
Related
I have a directory structure as follows for my DAGs:
.
├── __init__.py
├── _base.py
├── dag1.py
├── dag2.py
├── dag3.py
├── ...
When I run PyTest to fill my DagBag, I get the following error for each DAG file
ERROR - Failed to import: dags/dag2.py
Traceback (most recent call last):
File "~/code/myproject/venv/lib/python3.7/site-packages/airflow/models/dagbag.py", line 339, in parse
loader.exec_module(new_module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "dags/dag2.py", line 12, in <module>
from _base import start_here, task_factory
ModuleNotFoundError: No module named '_base'
My _base.py is a helper file that holds common task declarations like:
def task_factory(
operator_func,
base_env_vars=[],
op_kwargs={},
):
def build_airflow_task(task_id=None, extra_env_vars=[], **kwargs):
task = operator_func(
task_id=task_id,
name=task_id,
env_vars=[*base_env_vars, *extra_env_vars],
**op_kwargs,
**kwargs,
)
return task
return build_airflow_task
In each DAG file, I do the following:
import os
from airflow import DAG
from airflow.kubernetes.secret import Secret
from airflow.providers.google.cloud.operators.kubernetes_engine import (
GKEStartPodOperator,
)
from airflow.utils.helpers import chain
from kubernetes.client import models as k8s
from _base import start_here, finish, dummy_task, task_factory
I am not sure how to instrument Airflow/PyTest to find _base.py for tests, but in my actual deployment (Google Cloud Composer), my DAG files are able to find and import from _base.py without issue.
My test file looks like this:
import sys
sys.path.insert(0, "..")
from airflow.models import DagBag
def test_dagbag_compiles():
dags = DagBag("dags", include_examples=False)
assert len(dags.import_errors) == 0
The best solution is adding your module to python path:
PYTHONPATH=/path/to/module pytest ...
I have a script with the following folder structure:
myfolder/
myscript.py
utils.py
mymodels/
mymodel.py
utils.py
within the myscript.py I call:
from utils import funca, funcb, funcc
from mymodel.utils import DataLoader
from mymodels.mymodel import *
But mymodel itself import loads a yolov5 Modell from torch.hub and throws the following error message:
File "/home/vitouser/.local/lib/python3.10/site-packages/torch/hub.py", line 540, in load
model = _load_local(repo_or_dir, model, *args, **kwargs)
File "/home/vitouser/.local/lib/python3.10/site-packages/torch/hub.py", line 569, in _load_local
model = entry(*args, **kwargs)
File "/home/vitouser/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 83, in custom
return _create(path, autoshape=autoshape, verbose=_verbose, device=device)
File "/home/vitouser/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 33, in _create
from models.common import AutoShape, DetectMultiBackend
File "/home/vitouser/.cache/torch/hub/ultralytics_yolov5_master/models/common.py", line 28, in <module>
from utils import TryExcept
ImportError: cannot import name 'TryExcept' from 'utils' (/myfolder/utils.py)
I am guessing the extensive use of "utils" is causing some form of issue. Since Python cant find the TryExcept in my own /myfolder/utils.py.
How do I get rid of this problem.
Just calling my mymodel.py file from the shell using python does not cause any issues. So I am guessing it must be the import of the my utils file.
I am running nosetests across multiple files and getting an error relating to the importing of a specific file, well I'm not actually sure what the error is related to, I think it is either something up with the import or something up with the patching of it. The error itself looks like:
(I'm getting one of these errors for each test function that uses an #patch decorator)
Error
Traceback (most recent call last):
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/unittest2/case.py", line 67, in testPartExecutor
yield
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/unittest2/case.py", line 625, in run
testMethod()
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/mock/mock.py", line 1297, in patched
arg = patching.__enter__()
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/mock/mock.py", line 1353, in __enter__
self.target = self.getter()
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/mock/mock.py", line 1523, in <lambda>
getter = lambda: _importer(target)
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/mock/mock.py", line 1210, in _importer
thing = _dot_lookup(thing, comp, import_path)
File "/home/user/Documents/venvs/migration/local/lib/python2.7/site-packages/mock/mock.py", line 1200, in _dot_lookup
return getattr(thing, comp)
AttributeError: 'module' object has no attribute 'utils'
The package structure looks like this:
my_package
- my_module
- __init__.py
- utils.py
- other.py
- tests
- test_utils.py
- test_other.py
The nosetests command:
nosetests -e unit --with-coverage --cover-package=my_package --cover-erase --cover-xml --with-xunit tests --nocapture
So the weird thing is, if I run nosetests only on the utils test class itself, it runs fine, all imports work and all patches work, no errors, all tests pass.
Here's what the test_utils.py file looks like:
from my_module.utils import *
class TestBusinessProcess(unittest2.TestCase):
#patch('my_module.utils.something')
def test_some_utils_function(self, something_mock):
# test implementation..
# this function will throw:
# AttributeError: 'module' object has no attribute 'utils'
# when running whole tests folder and not on individual test file
pass
#patch('my_module.utils.something_else')
def test_some_other_utils_function(self, something_else_mock):
# test implementation..
# same as above
pass
An example of a test in the other test file that has no issues when ran either way:
from my_module.other import *
class TestBusinessProcess(unittest2.TestCase):
#patch('my_module.other.something')
def test_some_function(self, something_mock):
# test implementation..
# no issues!
pass
#patch('my_module.other.something_else')
def test_some_other_function(self, something_else_mock):
# test implementation..
# no issues!
pass
Any help greatly appreciated.
I still have no idea what was wrong, but it seemed to be something to do with the importing.
Anyway, this workaround solved the problem, but not sure why exactly.
The __init__.py in my_module was empty initially. Then I edited it to expose the individual functions of utils.py:
__init__.py
from utils import test_some_utils_function, test_some_other_utils_function
__all__ = [
"test_some_utils_function",
"test_some_other_utils_function"
]
I'm currently working through LearnPythonTheHardWay and have reached Exercise 48 which details Nosetests. I am able to perform a unit testing as long as all of the code is in a single python.py file. However if I include other files as part of a program, i.e. use import and then attempt to nosetest such a project I am getting an error, as follows:
======================================================================
ERROR: Failure: ImportError (No module named 'temp')
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/usr/local/lib/python3.4/dist-packages/nose/loader.py", line 414, in loadTestsFromName ## ##
addr.filename, addr.module)
File "/usr/local/lib/python3.4/dist-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/usr/local/lib/python3.4/dist-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/usr/lib/python3.4/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/usr/lib/python3.4/imp.py", line 171, in load_source
module = methods.load()
File "", line 1220, in load
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1471, in exec_module
File "", line 321, in _call_with_frames_removed
File "/home/user/LEARNPYTHONTHEHARDWAY/ex48/tests/scanner_tests.py", line 6, in
from ex48.scanner import lexicon
File "/home/user/LEARNPYTHONTHEHARDWAY/ex48/ex48/scanner.py", line 6, in
import temp
ImportError: No module named 'temp'
Ran 1 test in 0.028s
FAILED (errors=1)
The structure of my project directories are as follows:
ex48/
ex48/
scanner.py
temp.py
__pycache__/
tests/
__init__.py
scanner_tests.py
Screenshot of my directory::
Screen shot of files themselves::
My scanner_tests.py file is as follows:
from nose.tools import *
from ex48.scanner import lexicon
from ex48 import temp
def test_directions():
assert_equal(lexicon.scan("north"),[('direction','north')])
result = lexicon.scan("north south east")
assert_equal(result, [('direction', 'north'),
('direction', 'south'),
('direction', 'east')])
My scanner.py file is as follows:
import temp
class lexicon:
def scan(val):
if(val == "north"):
return [('direction', 'north')]
else:
return [('direction', 'north'),
('direction', 'south'),
('direction', 'east')]
runner = temp.temp("hello")
And finally my temp.py file is as follows:
class temp(object):
def __init__(self,name):
self.name = name
def run(self):
print "Your name is; %s" % self.name
runner.run()
My question is how to overcome the ImportError: No Module named 'temp' because it seems as if I have imported the temp.py file in both the scanner.py file and the scanner_tests.py file but nose does not seem to be able to import it when it runs. Nosetests works fine when its just the single scanner.py file but not when importing. Is there a special syntax for importing into a unit test for nose? The script also works fine when run and imports properly at the command line.
*Note: I'm running python off a limited account off an online server so some admin privileges are not available.
**Note below are entirely different screenshots from another project with the exact same error:
Directory Layout:
Game.py:
Otherpy.py - the imported file:
the Nose test script file:
And finally the nosetests importerror:
Everything needs to be with respect to your execution point. You are running your nose command from the root of ex48, therefore all your imports need to be with respect to that location.
Therefore, in game.py you should be importing with respect to ex48. Therefore:
from ex48.otherpy import House
The same logic should be applied to your example referencing the temp folder.
from ex48.temp import temp
The only solution I have found is to post the following to the top of the main file:
try:
# This handles imports when running .py files from inside app directory
from file_to_import.py import class_instance
except:
# This handles imports when running nosetests from top-level (above app)
# directory
from directory_containing_app_files.file_to_import import class_instance
I am supremely interested in an alternative solution.
I am currently developing wep.py application. This is my web application which is binded with web.py and wsgi.
root/main.py
import web
import sys
import imp
import os
sys.path.append(os.path.dirname(__file__))
#from module import module
from exam import exam
urls = (
'/exam', 'exam'
)
application = web.application(urls, globals(), autoreload = True).wsgifunc()
My application has an abstract class called module in module.py in root directory and its purpose is to be inherited by modules.
root/module.py
class module:
def fetchURL(self, url):
# ...
return content
The lower level module called "exam" would inherits module
root/exam/init.py
from module import module
class exam(module):
def getResults(self):
# error occurs here
self.fetchURL('math.json')
When I call the parent method, web.py raises an exception
WalkerError: ('unexpected node type', 339)
Environment: Python 2.5
How can I resolve the problem? Thanks
// EDIT 03 July 10:22 GMT+0
The stack trace is as follows
mod_wsgi (pid=1028): Exception occurred processing WSGI script 'D:/py/labs_library/index.py'.
Traceback (most recent call last):
File "D:\csvn\Python25\lib\site-packages\web\application.py", line 277, in wsgi
result = self.handle_with_processors()
File "D:\csvn\Python25\lib\site-packages\web\application.py", line 247, in handle_with_processors
return process(self.processors)
File "D:\csvn\Python25\lib\site-packages\web\application.py", line 244, in process
raise self.internalerror()
File "D:\csvn\Python25\lib\site-packages\web\application.py", line 467, in internalerror
return debugerror.debugerror()
File "D:\csvn\Python25\lib\site-packages\web\debugerror.py", line 305, in debugerror
return web._InternalError(djangoerror())
File "D:\csvn\Python25\lib\site-packages\web\debugerror.py", line 290, in djangoerror
djangoerror_r = Template(djangoerror_t, filename=__file__, filter=websafe)
File "D:\csvn\Python25\lib\site-packages\web\template.py", line 845, in __init__
code = self.compile_template(text, filename)
File "D:\csvn\Python25\lib\site-packages\web\template.py", line 924, in compile_template
ast = compiler.parse(code)
File "D:\csvn\Python25\lib\compiler\transformer.py", line 51, in parse
return Transformer().parsesuite(buf)
File "D:\csvn\Python25\lib\compiler\transformer.py", line 128, in parsesuite
return self.transform(parser.suite(text))
File "D:\csvn\Python25\lib\compiler\transformer.py", line 124, in transform
return self.compile_node(tree)
File "D:\csvn\Python25\lib\compiler\transformer.py", line 167, in compile_node
raise WalkerError, ('unexpected node type', n)
WalkerError: ('unexpected node type', 339)
If it is possible I would like to turn off the template functionality as I use python only for JSON output for mobile app.
if you create python module you should add __init__.py in top of your hierarchy:
dvedit/
__init__.py
clipview.py
filters/
__init__.py
it means that in every directory which will be imported via from ... import ... should have __init__.py file.
further info available: http://wiki.cython.org/PackageHierarchy