I am trying to get started with using nosegae, however I run into the issue that I can't seem to get it to pass even the simplest of cases when using django.
when running without the --without-sandbox flag both the following tests fail
def test_import_django ():
import django
def test_import_django_http ():
import django.http
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\nose-1.1.2-py2.7.egg\nose\case.py", line 1
97, in runTest
self.test(*self.arg)
File "C:\Users\User\Desktop\TDD_GAE\myproj\tests.py", line 2, in test_import_d
jango
import django
File "C:\Python27\lib\site-packages\nosegae-0.1.9-py2.7.egg\nosegae.py", line
207, in find_module
return super(HookMixin, self).find_module(fullname, path)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\de
v_appserver.py", line 1505, in Decorate
return func(self, *args, **kwargs)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\de
v_appserver.py", line 1998, in find_module
search_path)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\de
v_appserver.py", line 1505, in Decorate
return func(self, *args, **kwargs)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\de
v_appserver.py", line 2119, in FindModuleRestricted
result = self.FindPathHook(submodule, submodule_fullname, path_entry)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\de
v_appserver.py", line 2219, in FindPathHook
return self._imp.find_module(submodule, [path_entry])
Howevere if I do use --without-sandbox at least the first test passes
myproj.tests.test_import_django ... ok
myproj.tests.test_import_django_http ... ERROR
======================================================================
ERROR: myproj.tests.test_import_django_http
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\nose-1.1.2-py2.7.egg\nose\case.py", line 1
97, in runTest
self.test(*self.arg)
File "C:\Users\User\Desktop\TDD_GAE\myproj\tests.py", line 5, in test_import_d
jango_http
import django.http
File "C:\Program Files (x86)\Google\google_appengine\lib\django_1_2\django\htt
p\__init__.py", line 9, in <module>
from mod_python.util import parse_qsl
File "C:\Python27\lib\site-packages\nosegae-0.1.9-py2.7.egg\nosegae.py", line
199, in find_module
mod_path = self.find_mod_path(fullname)
File "C:\Python27\lib\site-packages\nosegae-0.1.9-py2.7.egg\nosegae.py", line
251, in find_mod_path
_sf, path, _desc= self._imp.find_module(top, None)
AttributeError: 'str' object has no attribute 'find_module'
Has anyone encountered and know how I can go about past this?
Edit
It seems that the issue is recursive imports
def test_import_pdb ():
import pdb
pdb.set_trace ()
part of the stack trace is
File "C:\Python27\lib\pdb.py", line 72, in __init__
import readline
notice that an import in __init__ of django.http is also part of the stack trace
Read https://docs.djangoproject.com/en/dev/topics/testing/ about Django testing.
As I know it's better to use unittest or doctest shipped with django as it have several improvements for django-specific testing like form field output testing and some database features. Hovewer it's not essential and if you want to continue using nose - think you missed django environment setup:
from django.test.utils import setup_test_environment
setup_test_environment()
This lines needed to run your tests outside of ./manage.py --test
UPD
Yeah my previous thought's were wrong. So I just digged into sources of nose and nose-gae, and what I think - check HardenedModulesHook definition in your nose version, cause in trunk of nose I've found following:
class HardenedModulesHook(object):
...
def __init__(self,
module_dict,
imp_module=imp,
os_module=os,
dummy_thread_module=dummy_thread,
pickle_module=pickle):
...
That gives following - when noseGAE plugin begin() method is executed -> there self._install_hook(dev_appserver.HardenedModulesHook) is called which declares mixed-hook class and creates it's instance like self.hook = Hook(sys.modules, self._path). <- There is HardenedModulesHook.__init__ called with second argument as mystic '_path' however in NOSE this argument should be 'imp' module by default -> That makes an exception you've got:
_sf, path, _desc= self._imp.find_module(top, None)
AttributeError: 'str' object has no attribute 'find_module'
So I think it might be a problem with nose-gae :(
Related
I am trying to use marshmallow_dataclass in a python (3.9) test in pycharm to try it (marshmallow_dataclass) out and get an error.
Not sure what I am doing wrong. I have installed marshmallow_dataclass (8.1.0)
I have so far been successful in using marshmallow (3.8.0) and marshmallow_oneofschema (2.1.0) so far
(new to python but not programming)
Code is:
import marshmallow_dataclass
import unittest
class Tests(unittest.TestCase):
def test_serialize(self):
print("Hello")
Error is:
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.2.3\plugins\python-ce\helpers\pycharm\_jb_unittest_runner.py", line 35, in <module>
sys.exit(main(argv=args, module=None, testRunner=unittestpy.TeamcityTestRunner, buffer=not JB_DISABLE_BUFFERING))
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\unittest\main.py", line 100, in __init__
self.parseArgs(argv)
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\unittest\main.py", line 147, in parseArgs
self.createTests()
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\unittest\main.py", line 158, in createTests
self.test = self.testLoader.loadTestsFromNames(self.testNames,
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\unittest\loader.py", line 220, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\unittest\loader.py", line 220, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\unittest\loader.py", line 154, in loadTestsFromName
module = __import__(module_name)
File "C:\xxxx\dev\repos\python\solar\pvoutput-publisher\test\test_serialization\test_delme.py", line 1, in <module>
import marshmallow_dataclass
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\marshmallow_dataclass\__init__.py", line 59, in <module>
import typing_inspect
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\typing_inspect.py", line 25, in <module>
from typing_extensions import Literal
File "C:\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\typing_extensions.py", line 2084, in <module>
def TypeAlias(self, parameters):
TypeError: __init__() missing 1 required positional argument: 'doc'```
I hit similar error on python 3.9 alpha release (3.9.0a5), I am not sure the version of typing_extensions.py in your environment (maybe it is v3.7.4.3 ?).
The root cause is that :
the function TypeAlias() is decorated directly with the class _TypeAliasForm, which is subclass of typing._SpecialForm in python stardard library
typing._SpecialForm.__init__() in your python environment has extra argument doc , since the class _TypeAliasForm is a decorator for TypeAlias(), Python couldn't provide extra argument doc when instantiating _TypeAliasForm on decorating process.
In short, your typing_extensions.py is out of sync with your python interpreter, see the source code for detail:
TypeAlias() and _TypeAliasForm class .
typing._SpecialForm.__init__() in python 3.9.0a5
typing._SpecialForm.__init__() in python 3.9.0a6
It seems that the issue happens from python 3.9.0a1 to 3.9.0a5, so the solutions are :
upgrade your python to final release version
if you are not allowed to upgrade python for some other reasons, maybe you can monkey-patch typing._SpecialForm , here is the example for monkey-patching :
from typing import _SpecialForm
origin_getitem = _SpecialForm.__getitem__
def patch_init(self, name, doc=None):
if doc is None and callable(name):
if hasattr(name, '__doc__'):
doc = name.__doc__
self._name = name
self._doc = doc
def patch_getitem(self, parameters):
item = None
if callable(self._name):
item = self._name(self=self, parameters=parameters)
else:
item = origin_getitem(self=self, parameters=parameters)
return item
if not hasattr(_SpecialForm.__init__, 'patched'):
_SpecialForm.__init__ = patch_init
setattr(_SpecialForm.__init__, 'patched', True)
if not hasattr(_SpecialForm.__getitem__, 'patched'):
_SpecialForm.__getitem__ = patch_getitem
setattr(_SpecialForm.__getitem__, 'patched', True)
I am trying to load a resource which is present in my project using pkg_resources but it just throws me an exception saying that it quote "Can't perform this operation for loaders without 'get_data()'". I am not sure if I am doing something wrong here, or if pkg_resources is somehow broken on python 3.3. I am using python 3.3.3 to be exact. Here is the code I am trying to execute
>>> import pkg_resources
>>> data = pkg_resources.resource_stream('configgenerator', 'schema_rules.yml')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/users/frank/workspace/configgenerator/env/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 931, in resource_stream
self, resource_name
File "/home/users/frank/workspace/configgenerator/env/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 1207, in get_resource_stream
return StringIO(self.get_resource_string(manager, resource_name))
File "/home/users/frank/workspace/configgenerator/env/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 1210, in get_resource_string
return self._get(self._fn(self.module_path, resource_name))
File "/home/users/frank/workspace/configgenerator/env/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 1289, in _get
"Can't perform this operation for loaders without 'get_data()'"
NotImplementedError: Can't perform this operation for loaders without 'get_data()'
>>>
Does anyone has an idea on how to fix this?
This can happen for a number of reasons, but the the most common is that the package is not an importable module because there is no __init__.py.
I am trying to Use PyUnit within PyDev for the first time. I created a unittest.py module. When I did run as -> Python unit-test, I got the following error:
Finding files... done.
Importing test modules ... done.
======================================================================
Traceback (most recent call last):
File "/Applications/eclipse/plugins/org.python.pydev_2.7.5.2013052819/pysrc/runfiles.py", line 163, in
main()
File "/Applications/eclipse/plugins/org.python.pydev_2.7.5.2013052819/pysrc/runfiles.py", line 77, in main
pydev_runfiles.main(configuration)
File "/Applications/eclipse/plugins/org.python.pydev_2.7.5.2013052819/pysrc/pydev_runfiles.py", line 761, in main
PydevTestRunner(configuration).run_tests()
File "/Applications/eclipse/plugins/org.python.pydev_2.7.5.2013052819/pysrc/pydev_runfiles.py", line 747, in run_tests
runner.run(test_suite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 158, in run
result.printErrors()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 108, in printErrors
self.printErrorList('ERROR', self.errors)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 114, in printErrorList
self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 46, in getDescription
return '\n'.join((str(test), doc_first_line))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 1060, in __str__
self._testFunc.__name__)
AttributeError: 'str' object has no attribute '__name__'
My unit test is just the PyDev-generated default:
import unittest
class Test(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def testName(self):
pass
print "hello test"
if __name__ == "__main__":
#import sys;sys.argv = ['', 'Test.testName']
unittest.main()
I expected it to print hello test. What am I missing?
Ok, I realized your problem, after having a similar issue for several minutes. You need to rename your file. Name it anything other than unittest.py. This causes the compiler to confuse between your module and the unittest module that you are importing.
Second, even after you change the name of your file, you may run into the same error. That is caused by the unittest.pyc file not being removed from your project. Rename the file, delete the previously generated unittest.pyc file, and the test should run just fine.
I'm trying to reload module that's already loaded within sys.modules, but somehow, reload() doesn't seem to work.
(compile()+load would work although, still I can't use that since it's code reload through twisted)
for loader, module_name, is_pkg in pkgutil.walk_packages(main_module.__path__):
if(not module_name in sys.modules):
module = loader.find_module(module_name).load_module(module_name)
else:
module = sys.modules.get(module_name)
#this is unhelpful
#sys.path.append(module.__path__)
#this works
#imp.find_module(module.__name__, module.__path__)
#this doesn't
reload(module)
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/celery/execute/trace.py", line 36, in trace
return cls(states.SUCCESS, retval=fun(*args, **kwargs))
File "/usr/lib64/python2.7/site-packages/celery/app/task/__init__.py", line 232, in __call__
return self.run(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/celery/app/__init__.py", line 172, in run
return fun(*args, **kwargs)
File "/home/myuser/Dropbox/job-pipe/job_pipe/apps/tasks/crawl.py", line 71, in crawl
crawler = CrawlerScript()
File "/home/myuser/Dropbox/job-pipe/job_pipe/apps/tasks/crawl.py", line 37, in __init__
reload(module)
ImportError: No module named example
What's the proper way to reload? I thought it was caused by find_module, though it seems to work properly, and error is not helpful.
Thanks.
Once you have loaded a module, to reload it you must do
module = reload(module)
I am trying to run Pyramid with Jinja2 using new Python 2.7 runtime in threadsafe mode and GAE 1.6.0 pre-release SDK. I've made modifications to my app as outlined here, i.e. I've set runtime: python27, threadsafe: true in app.yaml and got rid of main() function. When I generate response by myself it works fine, but when I try to bring jinja2 into the equation, I get the following exception:
ERROR 2011-11-07 00:10:34,356 wsgi.py:170]
Traceback (most recent call last):
File "/gae/google/appengine/runtime/wsgi.py", line 168, in Handle
[...]
File "/myapp/source/myapp-tip/main.py", line 29, in <module>
config.include('pyramid_jinja2')
File "/myapp/source/myapp-tip/lib/dist/pyramid/config/__init__.py", line 616, in include
c(configurator)
File "lib/dist/pyramid_jinja2/__init__.py", line 390, in includeme
_get_or_build_default_environment(config.registry)
File "/lib/dist/pyramid_jinja2/__init__.py", line 217, in _get_or_build_default_environment
_setup_environment(registry)
File "/lib/dist/pyramid_jinja2/__init__.py", line 253, in _setup_environment
package = _caller_package(('pyramid_jinja2', 'jinja2', 'pyramid.config'))
File "/lib/dist/pyramid_jinja2/__init__.py", line 136, in caller_package
for t in self.inspect.stack():
File "/usr/lib/python2.7/inspect.py", line 1056, in stack
return getouterframes(sys._getframe(1), context)
File "/usr/lib/python2.7/inspect.py", line 1034, in getouterframes
framelist.append((frame,) + getframeinfo(frame, context))
File "/usr/lib/python2.7/inspect.py", line 1009, in getframeinfo
lines, lnum = findsource(frame)
File "/usr/lib/python2.7/inspect.py", line 534, in findsource
module = getmodule(object, file)
File "/usr/lib/python2.7/inspect.py", line 506, in getmodule
main = sys.modules['__main__']
KeyError: '__main__'
I tried to mess around a bit with pyramid_jinja2 code to work around this issue, only to be left with another exception:
ERROR 2011-11-04 12:06:38,720 wsgi.py:170]
Traceback (most recent call last):
File "/gae/google/appengine/runtime/wsgi.py", line 168, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
[...]
File "/myapp/source/myapp-tip/main.py", line 29, in <module>
config.add_jinja2_search_path("templates")
File "/myapp/source/myapp-tip/lib/dist/pyramid/config/util.py", line 28, in wrapper
result = wrapped(self, *arg, **kw)
File "/lib/dist/pyramid_jinja2/__init__.py", line 311, in add_jinja2_search_path
env.loader.searchpath.append(abspath_from_resource_spec(d))
File "/myapp/source/myapp-tip/lib/dist/pyramid/asset.py", line 38, in abspath_from_asset_spec
return pkg_resources.resource_filename(pname, filename)
File "/myapp/source/myapp-tip/pkg_resources.py", line 840, in resource_filename
return get_provider(package_or_requirement).get_resource_filename(
File "/myapp/source/myapp-tip/pkg_resources.py", line 160, in get_provider
__import__(moduleOrReq)
File "/gae/google/appengine/tools/dev_appserver_import_hook.py", line 640, in Decorate
return func(self, *args, **kwargs)
File "/gae/google/appengine/tools/dev_appserver_import_hook.py", line 1756, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
File "/gae/google/appengine/tools/dev_appserver_import_hook.py", line 640, in Decorate
return func(self, *args, **kwargs)
File "/gae/google/appengine/tools/dev_appserver_import_hook.py", line 1628, in FindAndLoadModule
description)
File "/gae/google/appengine/tools/dev_appserver_import_hook.py", line 640, in Decorate
return func(self, *args, **kwargs)
File "/gae/google/appengine/tools/dev_appserver_import_hook.py", line 1571, in LoadModuleRestricted
description)
ImportError: Cannot re-init internal module __main__
I'd be happy if anybody could shed some light on what pyramid is trying to do under the hood. Judging by the latter stack trace it seems it's trying to resolve an asset, but why is it trying to reload __main__? I'm not even sure my problem is caused by pyramid or GAE.
Thanks for any insight on this issue.
I'm not familiar with pyramid, but the problem really does seem to be with this line:
config.include('pyramid_jinja2')
Whatever that config thing is, it seems to be doing some dynamic import magic.
Don't do that.
The app engine environment doesn't handle imports the way you would in normal python. Step through that line with a debugger and you'll wind up in the replacement version of the import system, which you'll soon see, only implements a small part of what real python does.
If possible, just use a normal import statement... Otherwise, you're going to have to dig into config.include and get it to play nice with the restricted importing features on GAE.
I managed to make it work using Pyramid 1.3's AssetResolver. First attempt is here. Just not sure what the lifetime/scope of the resolver should be in this case, I will figure it out later.
In pyramid_jinja2/__init__.py add the following code before _get_or_build_default_environment()
class VirtualModule(object):
def __init__(self,name):
import sys
sys.modules[name]=self
def __getattr__(self,name):
return globals()[name]
VirtualModule("__main__")
def _get_or_build_default_environment(registry):
(http://www.inductiveautomation.com/forum/viewtopic.php?f=70&p=36917)