SOLVED: see my own answer for an explanation.
I am writing an API to help me maintain my database when needed.
One function of my API is for updating a specific geoDjango Point field of all the concerned objects, thanks to a dedicated method of my model (which is pretty useless rergading to EDIT 1, see below).
The issue I have been facing with is the following: nothing seems to be saved to the database when calling that method on several objects, even if a call to save method seems to work while browsing the objects.
See EDIT 2, I figured out this issue was specific to geoDjango Point objects (everything else is being updated as expected).
Here below is my model:
models.py
class Location(geoModels.Model):
coord_field = geoModels.PointField(blank=True, null=True)
valTest = models.CharField(max_length=100, default="tagada") # only used to understand why coord_field can't be updated properly
def update_coord_field(self):
# some stuff is being done
self.coord_field = Point(
newLong,
newLat,
srid=3857 # I am trying to update that value
)
# nothing more
newLong and newLat are being generated thanks to Google Maps API.
As Google Maps seems to be working with SRID 3857 while GeoDjango uses SRID 4326 by default when creating a new Point, I want to force SRID of self.coord_field to SRID 3857 (I have been facing incoherent distance calculus, so I am trying to play with SRID to solve that).
Now here is my script :
script.py
from MyApp import models as MyAppModels
def update_all_coord_field():
for location in MyAppModels.Location.objects.all():
print(" old SRID : ", location.coord_field.srid)
location.update_coord_field()
location.save()
print(" new SRID : ", location.coord_field.srid)
update_all_coord_field()
But when trying to execute my script again (or simply just trying to use my new valued attribute), I can only observe that nothing was saved to the db (it still print the old value of location.coord_field.srid for all Locationobjects).
How my script is being executed:
python manage.py shell
exec(script.py)
If I call twice my script, here is the following traceback (only one object to make thinks understandable):
exec(script.py)
old SRID : 4326
new SRID : 3857 # so apparently object has been updated...
exec(script.py)
old SRID: 4326 # ...but this line proves that the updated value has not been pushed to the database
new SRID: 3857
Do you have any idea why is this not working?
FIRST EDIT: #JohnMoutafis pointed out to me there was already a function to change SRID of Point objects: transform.
Location.objects.get(pk=pk).transform(3857) # define a new value of SRID
It is usefull, yet it does not solve my issue! Please notice I did not update my code here with that transform method.
SECOND EDIT: I've been through a lot of pain to understand that it is not the save method which is not working! Actually, if I try to update "casual" data field of MyAppModel, everything works fine and is updated as expected. The real issue is only about my coord_field attribute which can't be updated, and I can't figure out why.
In my script I tried on the one hand what I explained above, and on the other hand I also tried the following to update manually (and not through a custom method of my model) coord_field:
MyAppModels.Location.objects.filter(pk=pk).update(
geoCoord=Point(location.geoCoord.x, location.geoCoord.y, srid=3857),
valTest="meat"
)
As a result, if I execute again my script, I see that valTest has been properly updated (nice!), but geoCoord still is the old value! :-( With the wrong SRID and everything. So, I think I really have a trouble updating geoDjango Point object in particular, and will update the title of my question as a consequence of that!
Based on this gis.stackexchange post, I will assume that you are declaring each point initially without an explicit srid, therefore GeoDajngo assumes that you must have declared it on 4326.
You need to declare every point in your database explicitly.
Here is a script to help you do that with your existing points.
for item in Location.objects.all():
item.coord_field = Point(item.coord_field.x, item.coord_field.y, srid=4326)
item.save()
You can use transform instead of your custom method to change the srid of your Points:
Location.objects.all().transform(3857)
EDIT due to comment:
Since clarification provided by #KrazyMax on the comments, he needs to calculate the spherical distance. I have made a post about that here: How to calculate 3D distance (including altitude) between two points in GeoDjango
SOLVED: my conlusion is that once you specified SRID of one Point, you can not change it later. Even the transform method won't change SRID, but will only make some conversion in place, without altering original SRID of your Point. Actually, I think that when I write:
coord_field = geoModels.PointField(blank=True, null=True)
the default 4826 SRID is already set once for all to the appropriate column of the database (and only this SRID can then be used for this column). So, even when I write:
self.coord_field = Point(newLong, newLat, srid=newSRID)
I won't be able to associate another SRID to my Point. Some calculus is going to be computed, but at the end the Point attribute will still be affected of the initial SRID.
To be clear: I am not sure 100% about those conclusions (I did not find any appropriate doc), but deleting coord_field column then setting coord_field as follows:
coord_field = geoModels.PointField(srid=3857, blank=True, null=True)
..did the trick. If anyone has got some clean answer for me, I would still appreciate to read it!
Related
I want to subscribe to vehicles in my simulation. However, I find it very difficult to understand which variables I can use for this, as the documentation does not include this information.
traci.vehicle.subscribe(veh_id, [
tc.VAR_SPEED,
tc.VAR_ACCELERATION,
tc.VAR_EMERGENCY_DECEL,
# tc.VAR_ROUTE,
tc.VAR_POSITION,
# tc.VAR_FOLLOWER,
tc.VAR_NEXT_TLS
])
The issue is, that tc.VAR_ROUTE causes this error in the terminal:
traci.exceptions.TraCIException: Could not add subscription. Get Vehicle Variable: unsupported variable 0x57 specified
and tc.VAR_FOLLOWER causes this error in SUMO:
Error: Storage::readChar(): invalid position
Quitting (on error).
Why is that? Also I do not quite understand how to learn more about the different constants. For example, which ones can I use to subscribe to vehicles?
When I look into the file under traci/constants.py, there are different types of variables.
starting with CMD_ (the comments call these "command")
starting with RESPONSE_ (the comments call these "response")
starting with VAR_ (the comments say these are "Variable types (for CMD_GET_*_VARIABLE)")
stop flags, departure flags, and many more
Also, the comments sometimes say something like: (get: ...; set: ...)
e.g. here:
# position (2D) (get: vehicle, poi, inductionloop, lane area detector; set: poi)
VAR_POSITION = 0x42
What does that mean, I know that I can get the subscription results from subscribing to these constants, but how could I possibly set them?
My main question is if someone can please explain how these constants are structured and which ones I can use for subscribing to vehicles.
You are right this is not well documented. I added a ticket: https://github.com/eclipse/sumo/issues/8579
The answer to your direct question is that everything which is listed in the value retrieval section of the docs (which should be equivalent to having a "get" in the constants file comment) should be "subscribable".
VAR_ROUTE is not, because it is only used to set a route using the edge list but there is no equivalent getter. You can only subscribe to the route ID for instance. VAR_FOLLOWER in turn needs a parameter when subscribing, so you need to give that as an additional argument. How to do that unfortunately depends on your SUMO version.
For SUMO 1.9 and later the call looks like this:
traci.vehicle.subscribe(vehID, (tc.VAR_FOLLOWER,), begin, end, {tc.VAR_FOLLOWER: ("d", dist)})
I am trying to introduce a mod/mixin for a problem. In particular I am focusing here on a SpeechRecognitionProblem. I intend to modify this problem and therefore I seek to do the following:
class SpeechRecognitionProblemMod(speech_recognition.SpeechRecognitionProblem):
def hparams(self, defaults, model_hparams):
SpeechRecognitionProblem.hparams(self, defaults, model_hparams)
vocab_size = self.feature_encoders(model_hparams.data_dir)['targets'].vocab_size
p = defaults
p.vocab_size['targets'] = vocab_size
def feature_encoders(self, data_dir):
# ...
So this one does not do much. It calls the hparams() function from the base class and then changes some values.
Now, there are already some ready-to-go problems e.g. Libri Speech:
#registry.register_problem()
class Librispeech(speech_recognition.SpeechRecognitionProblem):
# ..
However, in order to apply my modifications I am doing this:
#registry.register_problem()
class LibrispeechMod(SpeechRecognitionProblemMod, Librispeech):
# ..
This should, if I am not mistaken, overwrite everything (with identical signatures) in Librispeech and instead call functions of SpeechRecognitionProblemMod.
Since I was able to train a model with this code I am assuming that it's working as intended so far.
Now here comes the my problem:
After training I want to serialize the model. This usually works. However, it does not with my mod and I actually know why:
At a certain point hparams() gets called. Debugging to that point will show me the following:
self # {LibrispeechMod}
self.hparams # <bound method SpeechRecognitionProblem.hparams of ..>
self.feature_encoders # <bound method SpeechRecognitionProblemMod.feature_encoders of ..>
self.hparams should be <bound method SpeechRecognitionProblemMod.hparams of ..>! It would seem that for some reason hparams() of SpeechRecognitionProblem gets called directly instead of SpeechRecognitionProblemMod. But please note that it's the correct type for feature_encoders()!
The thing is that I know this is working during training. I can see that the hyper-paramaters (hparams) are applied accordingly simply because the model's graph node names change through my modifications.
There is one specialty I need to point out. tensor2tensor allows to dynamically load a t2t_usr_dir, which are additional python modules which get loaded by import_usr_dir. I make use of that function in my serialization script as well:
if usr_dir:
logging.info('Loading user dir %s' % usr_dir)
import_usr_dir(usr_dir)
This could be the only culprit I can see at the moment although I would not be able to tell why this may cause the problem.
If anybody sees something I do not I'd be glad to get a hint what I'm doing wrong here.
So what is the error you're getting?
For the sake of completeness, this is the result of the wrong hparams() method being called:
NotFoundError (see above for traceback): Restoring from checkpoint failed.
Key transformer/symbol_modality_256_256/softmax/weights_0 not found in checkpoint
symbol_modality_256_256 is wrong. It should be symbol_modality_<vocab-size>_256 where <vocab-size> is a vocabulary size which gets set in SpeechRecognitionProblemMod.hparams.
So, this weird behavior came from the fact that I was remote debugging and that the source files of the usr_dir were not correctly synchronized. Everything works as intended but the source files where not matching.
Case closed.
I am extending product.template, and I've added a field called uom_class. When I change this field when editing a product, I'd like to clear what is entered in the Many2One field called "Unit of Measure" (uom_id in product.template). I am doing this because I am also changing the domain for the uom_id, and the existing selection ("Units" by default) will probably not be in that domain.
I've tried the following, as suggested for earlier versions, but it did not work.
#api.onchange('uom_class')
def onchange_uom_class(self):
# [...] not shown: previously set new domain for uom_id
result['value'] ={'uom_id':False}
return result
I've seen other posts suggest I need to add assign it an empty product.uom record, but I have no idea how to do that. Any help would be greatly appreciated.
Well, I figured this one out.
I just declared
uom_id = False
For some reason, returning the domain works, but not returning the value. Either that, or I just have no idea what I'm doing and I'm returning the value wrong... which is entirely possible.
I find myself stuck on this problem, and repeated Googling, checking SO, and reading numerous docs has not helped me get the right answer, so I hope this isn't a bad question.
One entity I want to create is an event taking place during a convention. I'm giving it the property start_time = ndb.TimeProperty(). I also have a property date = messages.DateProperty(), and I'd like to keep the two discrete (in other words, not using DateTimeProperty).
When a user enters information to create an event, I want to specify defaults for any fields they do not enter at creation and I'd like to set the default time as midnight, but I can't seem to format it correctly so the service accepts it (constant 503 Service Unavailable response when I try it using the API explorer).
Right now I've set things up like this (some unnecessary details removed):
event_defaults = {...
...
"start_time": 0000,
...
}
and then I try looping over my default values to enter them into a dictionary which I'll use to .put() the info on the server.
data = {field.name: getattr(request, field.name) for field in request.all_fields()
for default in event_defaults:
if data[default] in (None, []):
data[default] = event_defaults[default]
setattr(request, default, event_defaults[default])
In the logs, I see the error Encountered unexpected error from ProtoRPC method implementation: BadValueError (Expected time, got 0). I have also tried using the time and datetime modules, but I must be using them incorrectly, because I still receive errors.
I suppose I could work around this problem by using ndb.StringProperty() instead, and just deal with strings, but then I'd feel like I would be missing out on a chance to learn more about how GAE and NDB work (all of this is for a project on udacity.com, so learning is certainly the point).
So, how can I structure my default time properly for midnight? Sorry for the wall of text.
Link to code on github. The conference.py file contains the code I'm having the trouble with, and models.py contains my definitions for the entities I'm working with.
Update: I'm a dummy. I had my model class using a TimeProperty() and the corresponding message class using a StringField(), but I was never making the proper conversion between expected types. That's why I could never seem to give it the right thing, but it expected two different things at different points in the code. Issue resolved.
TimeProperty expects a datetime.time value
import datetime
event_defaults = {...
...
"start_time": datetime.time(),
...
}
More in the docs: https://cloud.google.com/appengine/docs/python/ndb/entity-property-reference#Date_and_Time
Use the datetime() library to convert it into a valid ndb time property value
if data['time']:
data['time'] = datetime.strptime(data['time'][:10], "%H:%M").time()
else:
data['time'] = datetime.datetime.now().time()
ps: Don't forget to change data['time'] with your field name
This is something I have been trying to solve for 3 days now and I just can't get my head around it why this is not working.
I have a method that creates a new version of an Object. It used to work, that you would pass in the sou obj. and this would be the source from which a new version is created. You can also pass in a destination, which is not really important in this example. Now I wanted to add locking to this method as we want to add multiple users. So I want to be sure that I always have the most current object from which I create a new one. So I added a line that would just get the newest object. If there is no newer object in the database it would be the same anyway.
def createRevision(request, what, sou, destination=None, ignore = [], **args):
...
if "initial" not in args.keys():
source = get_object_or_404(BaseItem, ppk=sou.ppk, project=sou.project, current=True)
print "------------"
print source == sou
print "------------"
# This outputs True
else:
source = sou
further down in the method I do something like
source.current = False
source.save()
Basically the idea is that I pass in BaseItem and if I don't specify the "initial" keyword then I get the current item from that project with the same ppk (Which is a special random pk in conduction with current). I do this just to be on the save side, that I really have the most current object. And if it is the initial version I just use that one, as there can not be another version.
So now the problem is, that everything works fine if I use sou in this method. I can save it etc .. but as soon as I use source and initial is not in the args it just doesn't save it. The print statement tells me they are the same. Everything I print after the save tells me it has been saved but it just doesn't do it.
source.current = False
source.save()
print "SAVED !!!!"
print source.pk
print source.current
rofl = get_object_or_404(BaseItem, pk=source.pk, project=sou.project)
print rofl.pk
print source.current
outputs the same pk and the same current value but somehow it is not properly saved. As soon as I look into django admin or do a select current = True.
I really don't know what to do anymore.
Why does it work without a problem if I pass in the object into the method but starts to fail when I get the exact same object in the method?
Of course I call the method with the same object:
x = get_object_or_404(BaseItem, ppk=sou.ppk, project=sou.project, current=True)
createRevision(request, "", x)
Thank you pztrick for the hint with the caches. I finally solved it. So the problem was that I was doing:
x = get_object_or_404(BaseItem, ppk=sou.ppk, project=sou.project, current=True)
createRevision(request, "", x)
# .... loads of lines of code
unlock(x)
unlock is a method I wrote that just sets a timestamp so I know no other user is editing it. So now the problem was that I was saving x in createRevision with all the correct data but of course unlock(x) still had a reference to an "old" not updated object and of course was saving it again. Hence it was overwriting my changes in createRevision.
Thank you again to everyone who helped with this.
I think you may be running afoul of model manager caching which is intended to limit database queries. However, by invoking the .all() method on the model manager you force it to hit the databse again.
So, try this: Replace your argument from the BaseItem class to the model manager's .all() QuerySet:
source = get_object_or_404(BaseItem.objects.all(), ppk=sou.ppk, project=sou.project, current=True)
# ...
rofl = get_object_or_404(BaseItem.objects.all(), pk=source.pk, project=sou.project)
get_object_or_404 supports mode classes, model managers, or QuerySets as the first parameter so this is valid.