Is there a function in Google App Engine to test if a string is valid 'string key' prior to calling memcache.get(key) without using db.get() or db.get_by_key_name() first?
In my case the key is being passed from the user's get request:
obj = memcache.get(self.request.get("obj"))
Somehow I'd like to know if that string is a valid key string without calling the db first, which would defeat the purpose of using memcache.
That is probably the most efficient (and practical) way to determine if the key string is valid. The code is obviously performing that test for you before it attempts to retrieve the entity from memcache/datastore. Even better, Google will update that code if necessary.
try:
obj = memcache.get(self.request.get("obj"))
except BadKeyError:
# give a friendly error message here
Also, consider switching to ndb. Performing a get() on a key automatically uses two levels of cache, local and memcache. You don't need to write separate code for memcache.
A db module key sent to a client should pass through str(the_key) which gives you an URL safe encoded key. Your templating environment etc.. will do this for you just by rendering the key into a template.
On passing the key back from a client, you should recreate the key with
key = db.Key(encoded=self.request.get("obj"))
At this point it could fail with something like
BadKeyError: Invalid string key "thebadkeystring"=.
If not you have a valid key
obj = memcache.get(self.request.get("obj")) won't actually raise BadKeyError because at that point you are just working with a string, and you just get None returned or a value.
So at that point all you know is you have a key missing.
However you need to use the memcache.get(self.request.get("obj")) to get the object from memcache, as a db.Key instance is not a valid memcache key.
So you will be constructing a key to validate the key string at this point. Of course if the memcache get fails then you can use the just created key to fetch the object with db.get(key)
Any object is a valid key, provided that the object can be serialized using pickle. If pickle.dumps(key) succeeds, then you shouldn't get a BadKeyError.
Related
Problem:
I added a value to redis using Python code, and when I try to query is using Laravel
Redis::get('key_name') it returns null.
Redis::keys('*') returns values created using Laravel but not Python
Redis::scan('*') returns all values even those created using Python
Research:
https://stackoverflow.com/a/56569380/9530790, noted this may be a database number issue, however that is not the case. Since I'm using the same database in the Python code as well as the Laravel code, (database 0)
https://stackoverflow.com/a/65278346/9530790, noted the same issue as above.
Question:
Why is keys('*') not returning the key but scan('*') is and how can I get the value if get('key_name') is returning null?
Laravel: 7.30.4
Python: 3.8.3
Redis: 6.0
Laravel adds a prefix to all keys created. That prefix is defined in the redis config in database.php.
'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_database_'),
I haven't looked at the source code yet but most likely when laravel looks for keys it expects the prefix and appends that to what you passed to keys or get. So if you passed keys('key_name') it'll search for prefix_key_name which is why get returned null and keys didn't return my key created via Python as opposed to the one created via Laravel. I guess scan works a little differently and returns all keys regardless of its prefix.
If you set the default of your prefix to null ('prefix' => env('REDIS_PREFIX', null) then your key will be returned.
Using get and append the prefix, like this Redis::get('prefix_key_name') doesn't work.
I'm using Pyrebase to access my Firebase database. My database is currently structured like so:
- users
- 12345
name: "Kevin"
company: "Nike"
Where 12345 is the user's id, and the company is the company that the user belongs to. I'm currently trying to get all the users that belong to Nike. According to the Pyrebase docs, doing something like this should work:
db.child("users").order_by_child("company").equal_to("Nike").get().val()
but I'm getting the error "error" : "orderBy must be a valid JSON encoded path". Does anyone know why this might be the case?
There is something wrong with the Pyrebase library. Here's a link to the problem.
The solution is to add these lines of code in your app.
# Temporarily replace quote function
def noquote(s):
return s
pyrebase.pyrebase.quote = noquote
I managed to fix this problem, since I'm also using rest api to connect with my firebase realtime database. I'll demonstrate where the error lies with examples:
When I don't put wrap the orderBy value (child, key, etc) and other queries parameters with commas, retrofit (which I'm using) gives me error/bad request.
Here's the error/bad request url:
https://yourfirebaseprojecturl.com/Users.json?orderBy=username&startAt=lifeofkevin
See, both the orderBy value and startAt value, in this case, username and lifeofkevin, are not wrapped with commas, like this "username" and "lifeofkevin", so it will return orderBy must be a valid JSON encoded path.
In order to work, I need to wrap my orderBy and other query parameters, with commas, so that Firebase returns the data, you want to work with.
Here's the second example, the correct one:
https://yourfirebaseprojecturl.com/Users.json?orderBy="username"&startAt="gang"
Now notice, the difference? Both values of orderBy and startAt are wrapped with commas so now they'll return the data you want to work with.
I've created a program to take a users predetermined unique identifier, hash it, and store it in a dictionary mapping to the user's name. I later receive the unique identifier, rehash it, and can look up the user's name.
I've come to a problem where an individual's 9 digit unique ID hash()'s to the same number as somebody else. This has occurred after gathering data for about 40 users.
Is there a common work around to this? I believe this is different than just using a hashmap, because if I create a bucket for the hashed ID, I won't be able to tell who the user was (whether it be the first item in the bucket or second).
Edit:
id = raw_input()
hashed_id = hash(id)
if not dictionary.has_key(hashed_id):
name = raw_input()
dictionary[hashed_id] = name
check_in_user(dictionary[hashed_id])
I have never seen hash() used for this. hash() should be used for data structures as a shorthand for the entire object, such as keys in the internal implementation of dictionaries.
I would suggest using a UUID (universally unique identifier) for your users instead.
import uuid
uuid.uuid4()
# UUID('d36b850c-2433-42c6-9252-6371ea3d33c2')
You'll be very hard pressed to get a collision out of UUIDs.
After fetching an entity from the datastore, I'd like to save its key into memcache and also pass it as part of a url parameter for a task for referencing it later.
However, since Key is a composite item, you can't forward it as is and when I try to re-construct the key, the values aren't identical.
What's the best approach for passing of a key to reference the entity at a later time?
entity_key = feed_entity.key()
logging.info(entity_key) # produces a string like key value
# would like to save a way to reference the key later
memcache.set(entity_key.id_or_name(), some_piece_of_data);
# Will produce the error:
# Key must be a string instance, received datastore_types.Key.from_path
# (u'FeedEntity', u'My_Entity_Name', u'FeedEntity', 2L, _app=u'dev~test_app')
reconstructed_key = Key.from_path('FeedEntity', 'My_Entity_Name', 'FeedEntity', entity_key.id_or_name());
logging.info(reconstructed_key)
# Not the same value as entity_key
params = {"key": entity_key_string_value} # this would be ideal
task = taskqueue.Task(url='/feed_entity/list', params=params).add(queue_name="feed-gopher")
See http://code.google.com/appengine/docs/python/datastore/keyclass.html#Key
A key can be encoded to a string by passing the Key object to str() (or calling the object's __str__() method). A string-encoded key is an opaque value using characters safe for including in URLs. The string-encoded key can be converted back to a Key object by passing it to the Key constructor (the encoded argument).
I want to be able to check if a key_name for my model exists in the datastore.
My code goes:
t=MyModel.get_by_key_name(c)
if t==None:
#key_name does not exist
I don't need the object, so is there a way (which would be faster and cost less resource) to check if the object exist without returning it? I only know the key name, not the key.
You can't avoid get_by_key_name() or key-related equivalents to check if a key exists. Your code is fine.
The API talks about Model.all(keys_only=False) returning all the key names when keys_only is set to True
Look at the query that is fired for this, and then you can write a query similar to this but just for your object and see if any row is fetched or not.