I'm trying to insert an instance of a model into an ndb database.
It keeps on giving me the "Model is not immutable" error.
I have tried with different model names, but still the same error.
class User(ndb.Model):
username = ndb.StringProperty()
email = ndb.StringProperty()
lwr_username = ndb.ComputedProperty(lambda self: self.username.lower())
lwr_email = ndb.ComputedProperty(lambda self: self.email.lower())
Here's my insert code:
entity = User()
entity.email = ""
entity.username = "bob"
logging.info(entity)
#Commit data asynchronously
entities = [entity]
futures = ndb.put_multi_async(entities)
#Build Response whilst database is committing
response = {
}
#Wait for commits to finish
ndb.Future.wait_all(entities)
This is the full stack trace
Model is not immutable
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~myapp-api/1.377037445874907069/v1/handler/userHandler.py", line 11, in post
responseCode, response = UserService.create(locale, json.loads(self.request.body), **kwargs)
File "/base/data/home/apps/s~myapp-api/1.377037445874907069/v1/service/userService.py", line 37, in create
ndb.Future.wait_all(entities)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 345, in wait_all
waiting_on = set(futures)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3017, in __hash__
raise TypeError('Model is not immutable')
TypeError: Model is not immutable
You need to use the futures list, not entities to wait for the async process to complete:
ndb.Future.wait_all(futures)
The wait_all() function stores these in a set() object, and sets require the contents to be hashable. Since mutable objects cannot be stored in sets or dictionaries, Google engineers added an explicit TypeError to the Model.__hash__ method, which is what you see raised here.
Related
I am attempting to make use of the google app engine to have a login system for a site I am building using Webapp2 in python. Upon attempting to access the root page however, I receive this error
Traceback (most recent call last):
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/local/lib/python2.7/site-packages/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/local/lib/python2.7/site-packages/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/local/lib/python2.7/site-packages/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/local/lib/python2.7/site-packages/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/local/lib/python2.7/site-packages/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/local/lib/python2.7/site-packages/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/media/Data2/programming/Media-Management/src/server/main_server.py", line 199, in get
user = users.get_current_user()
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/lib/python2.7/site-packages/google_appengine/google/appengine/api/users.py", line 311, in get_current_user
return User()
File "/home/allen/.local/share/virtualenvs/Media-Management--7fPVcRX/lib/python2.7/site-packages/google_appengine/google/appengine/api/users.py", line 102, in __init__
assert _auth_domain
AssertionError
The code section that it errors on is as follows:
class LoginHandler(webapp2.RequestHandler):
def get(self):
user = users.get_current_user()
myPage = """
<html>
<body>
<div>
<h2>Google App Engine Login - Python Web app</h2>
<h3>Welcome, {0}. This is a sample page!</h3>
<b>Click here to {2}</b>
</div>
</body>
</html>
"""
if user:
myData = myPage.format(user.nickname(), users.create_logout_url("/"),'logout' )
else:
myData = myPage.format('Guest', users.create_login_url("/"),'login' )
return myData
As Dan Cornilescu commented, setting the following environment variable fixes this
os.environ["AUTH_DOMAIN"] = 'gmail.com'
My application experiences over quota issues and I would like to handle such cases properly in my code. The limit is reached just for Datastore Read Operations, but I get TypeError: ConjunctionNode() requires at least one node exception when try to read the data from the memcache - entries = memcache.get('mykey').
Why this exception happens?
Exception details:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1519, in __call__
response = self._internal_error(e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~myapp/1.373233284460557570/myapp.py", line 595, in get
entries = memcache.get('mykey')
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 559, in get
results = rpc.get_result()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 612, in get_result
return self.__get_result_hook(self)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 624, in __get_hook
self._do_unpickle)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 271, in _decode_value
return do_unpickle(value)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 401, in _do_unpickle
return unpickler.load()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 550, in __new__
raise TypeError('ConjunctionNode() requires at least one node.')
TypeError: ConjunctionNode() requires at least one node.
Upd. here is what I save to memcache:
entries = MyModel.query()
entries = entries.fetch(keys_only=True)
entries = random.sample(entries, 10)
entries = [list_key.get() for list_key in entries]
memcache.set('mykey', entries, 60*60*24)
Upd2. Online memcache viewer shows the following value stored (just first part is shown below):
Type: Object
..cgoogle.appengine.ext.ndb.query.Query.q.).q.}q.(U._Query__projectionq.NU._Query__filtersq.cgoogle.appengine.ext.ndb.query.ConjunctionNode.q.).q.}q.U._ConjunctionNode__nodesq.]q.(cgoogle.appengine.ext.ndb.query.FilterNode.q.).q.}q.(U._Filt
I believe that this happens simply because you are trying to put a pickled Query object into memcache, but Query objects are not pickle-able.
What I do instead in this case is to convert the Query to a list prior to putting it into the cache. If your code doesn't care about the distinction (that is, if it's not calling .filter() or something like that) then it'll probably work fine for you too.
I have a simple Python program that uses GAE, and I am using GAE's memcache module to make sure that I do not screw up when updating the cache:
from google.appengine.api import memcache
class NewPost(BlogHandler):
def get(self):
self.render("newpost.html")
def post(self):
#update cache for the front page
val, unique = memcache.gets(FRONT_PAGE_KEY)
for p in val:
logging.warning(p)
Now this code should run without problem, but instead when I use the method post, it blows up:
Traceback (most recent call last):
File "/home/pedro/google_appengine/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 1519, in __call__
response = self._internal_error(e)
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/home/pedro/google_appengine/lib/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/home/pedro/google_appengine/hw6/blog.py", line 172, in post
val, unique = memcache.gets(FRONT_PAGE_KEY)
AttributeError: 'module' object has no attribute 'gets'
INFO 2014-05-08 13:36:49,525 module.py:639] default: "POST /blog/newpost HTTP/1.1" 500 -
This makes no sense at all, specially because I know that memcache has a method called gets(key):
https://developers.google.com/appengine/docs/python/memcache/
Based on my research on stackoverflow I found this discussion:
App Engine Python: AttributeError: 'module' object has no attribute 'Stock'
And so I have flushed my cache and deleted all my local DB's content, but I am still getting the error.
What am I doing wrong?
You're mistaken: the memcache module does not have a gets function. See the documentation.
gets is a method of the memcache Client object: again, see the docs.
I have a problem, I want to access the data which are stocked in my google cloud storage but I have an error and I don't know where it comes from.
Here is my code :
DECORATOR = oauth2decorator_from_clientsecrets(
CLIENT_SECRETS,
scope=[
'https://www.googleapis.com/auth/devstorage.read_only',
],
message=MISSING_CLIENT_SECRETS_MESSAGE)
http = DECORATOR.http()
service_cloud = build("storage", "v1beta1")
list_response = service_cloud.objects().list(bucket="directory_structure").execute(http=http)
params = {'directory_list':list_response['items']}
return self.render_template('directoryChoice.html', **params)
and the error message I receive is:
Traceback (most recent call last):
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__ rv = self.router.dispatch(request, response)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher return route.handler_adapter(request, response)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__ return handler.dispatch()
File "/base/data/home/apps/s~jba-gae-boilerplate/dev.366111306063368728/boilerplate/lib/basehandler.py", line 162, in dispatch webapp2.RequestHandler.dispatch(self)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch return self.handle_exception(e, self.app.debug)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch return method(*args, **kwargs)
File "/base/data/home/apps/s~jba-gae-boilerplate/dev.366111306063368728/web/lib/oauth2client/appengine.py", line 469, in check_oauth return method(request_handler, *args, **kwargs)
File "/base/data/home/apps/s~jba-gae-boilerplate/dev.366111306063368728/web/handlers.py", line 190, in get list_response = service_cloud.objects().list(bucket="jba_directory").execute(http=http)
File "/base/data/home/apps/s~jba-gae-boilerplate/dev.366111306063368728/web/lib/oauth2client/util.py", line 120, in positional_wrapper return wrapped(*args, **kwargs)
File "/base/data/home/apps/s~jba-gae-boilerplate/dev.366111306063368728/web/lib/apiclient/http.py", line 678, in execute raise HttpError(resp, content, uri=self.uri)
HttpError:
One thing which looks a bit odd:
In your code you're listing the directory_structure bucket, but the traceback references jba_directory. There is a mis-match of some sort. Perhaps in a wrapping decorator or other code above the snippet you included here?
I'm trying to implement the code that Guido posted as an answer to the question Avoiding Memcache 1M limit of values. Everything seems(?) to be working when I first load the page and add values to memcache, but when I re-load it and retrieve values from the memcache I get a strange error:
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 194, in Handle
for chunk in result:
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/appstats/recording.py", line 1036, in appstats_wsgi_wrapper
result = app(environ, appstats_start_response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1519, in __call__
response = self._internal_error(e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/appengineurlhere/mypythoncode.py", line 87, in get
entitylist = retrieve("entitylist")
File "/appengineurlhere/mypythoncode.py", line 53, in retrieve
return pickle.loads(serialized)
File "/base/python27_runtime/python27_dist/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
File "/base/python27_runtime/python27_dist/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
KeyError: ':'
Here's the store/retrieve code:
def store(key, value, chunksize=750000):
serialized = pickle.dumps(value, 2)
values = {}
for i in xrange(0, len(serialized), chunksize):
values['%s.%s' % (key, i//chunksize)] = serialized[i : i+chunksize]
memcache.set_multi(values)
def retrieve(key):
result = memcache.get_multi(['%s.%s' % (key, i) for i in xrange(32)])
serialized = ''.join([v for v in result.values() if v is not None])
return pickle.loads(serialized)
And this is how I'm using it:
try:
entitylist = retrieve("entitylist")
except EOFError:
entitylist = MyModel.all().fetch(None)
store("entitylist", entitylist)
Other oddities:
I don't see this on my development server; just the production App Engine site. (Though the data on my development server is slightly different; I believe it all fits into the standard 1MB memcache size, where the production data is larger.)
When I search for "entitylist" on both my development and production admin panel, App Engine tells me "no such key". Yet based on the total size of the memcache shown (this is the only place I've implemented memcache), it absolutely appears that something is getting cached.
Can anyone please point me in the direction of something I should look at or fix?
Is it possible that you have some old instances of MyModel that don't match with the current definition of MyModel? You could get errors while pickling if there are unrecognized or missing attributes.