Develop Reference

Python Heroku Scheduler DB commit doesnt change values - DetachedInstanceError

I am using following code in the free heroku scheduler add-on to send emails to certain users. After the email was sent a value in the DB must be changed, to be more precise:
setattr(user, "stats_email_sent", True)
Somehow the db_session.commit() is executed but doesnt save the new value. Here is the code:
all_users = User.query.all()
for user in all_users:
if user.stats_email_sent is False and user.number_of_rooms > 0:
if date.today() <= user.end_offer_date and date.today() >= user.end_offer_date - relativedelta(days=10):
print user.id, user.email, user.number_of_rooms, user.bezahlt
if user.bezahlt is True:
with app.app_context():
print "app context true", user.id, user.email, user.number_of_rooms, user.bezahlt
html = render_template('stats_email_once.html', usersname=user.username)
subject = u"Update"
setattr(user, "stats_email_sent", True)
#send_email(user.email, subject, html, None)
else:
with app.app_context():
print "app context false", user.id, user.email, user.number_of_rooms, user.bezahlt
html = render_template('stats_email_once.html', usersname=user.username)
subject = u"Update"
setattr(user, "stats_email_sent", True)
#send_email(user.email, subject, html, None)
print "executing commit"
db_session.commit()
I tryed moving db_session.commit() right after setattr then it will work, but only for one user (for the first user).
setattr(user, "stats_email_sent", True)
db_session.commit()
And give me this in the logs:
sqlalchemy.orm.exc.DetachedInstanceError: Instance <User at 0x7f5c04d462d0> is not bound to a Session; attribute refresh operation cannot proceed
I also found some topics on detaching an instance, but I don't need to detach here anything, or?
EDIT:
I tryed now also adding db_session.expire_on_commit = False. This sadly had no effect.
I also looked on the bulk updates, which also didn't worked.
I even tryed to ignore and pass the DetachedInstanceError.
I cant believe that updating multiple rows at once is such an issue. I am running out of ideas here. Any help is appriciated. It either has no effect or is run into DetachedInstanceError.
EDIT
I solved the issue, had yesterday a similar one. I assume the query and its variables were consumed, thats why it didnt worked.
To solve this I created a list client_id_list = [] and appended all the ids of users which got the email and where the value needs to be changed.
Then I created a completely new query and all in all run the same code with the same logic, but the variables here were not consumed I guess? I wont answer the question again, because I am not sure whether this is true or not, here is the code which I appended to the code above which changes the values:
all_users_again = User.query.all()
for user in all_users_again:
if user.id in client_id_list:
setattr(user, "stats_email_sent", True)
db_session.commit()

Session data not being stored during testing in Django

I am currently writing tests for our project, and I ran into an issue. We have this section of a view, which will redirect the user back to the page where they came from including an error message (that's being stored in the session):
if request.GET.get('error_code'):
"""
Something went wrong or the call was cancelled
"""
errorCode = request.GET.get('error_code')
if errorCode == 4201:
request.session['errormessage'] = _('Action cancelled by the user')
return HttpResponseRedirect('/socialMedia/manageAccessToken')
Once the HttpResponseRedirect kicks in, the first thing that the new view does is scan the session, to see if any error messages are stored in the session. If there are, we place them in a dictionary and then delete it from the session:
def manageAccessToken(request):
"""
View that handles all things related to the access tokens for Facebook,
Twitter and Linkedin.
"""
contextDict = {}
try:
contextDict['errormessage'] = request.session['errormessage']
contextDict['successmessage'] = request.session['successmessage']
del request.session['errormessage']
del request.session['successmessage']
except KeyError:
pass
We should now have the error message in a dictionary, but after printing the dictionary the error message is not there. I also printed the session just before the HttpResponseRedirect, but the session is an empty dictionary there as well.
This is the test:
class oauthCallbacks(TestCase):
"""
Class to test the different oauth callbacks
"""
def setUp(self):
self.user = User.objects.create(
email='test#django.com'
)
self.c = Client()
def test_oauthCallbackFacebookErrorCode(self):
"""
Tests the Facebook oauth callback view
This call contains an error code, so we will be redirected to the
manage accesstoken page. We check if we get the error message
"""
self.c.force_login(self.user)
response = self.c.get('/socialMedia/oauthCallbackFacebook/',
data={'error_code': 4201},
follow=True,
)
self.assertEqual('Action cancelled by the user', response.context['errormessage'])
It looks like the session can not be accessed or written to directly from the views during testing. I can, however, access a value in the session by manually setting it in the test by using the following bit of code:
session = self.c.session
session['errormessage'] = 'This is an error message'
session.save()
This is however not what I want, because I need the session to be set by the view as there are many different error messages in the entire view. Does anyone know how to solve this? Thanks in advance!

After taking a closer look I found the issue, it is in the view itself:
errorCode = request.GET.get('error_code')
if errorCode == 4201:
request.session['errormessage'] = _('Action cancelled by the user')
The errorCode variable is a string, and I was comparing it to an integer. I fixed it by changing the second line to:
if int(errorCode) == 4201:

Django Test Client does not create database entries

I'm creating unit tests for my views using Django's built-in Test Client to create mock requests.
The view I'm calling should create an object in the database. However, when I query the database from within the test method the object isn't there - it either hasn't been created or has been discarded on returning from the view.
Here's the view:
def apply_to_cmp(request, campaign_id):
""" Creates a new Application to 'campaign_id' for request.user """
campaign = Campaign.objects.get(pk = campaign_id)
if not Application.objects\
.filter(campaign = campaign, user = request.user)\
.exists():
application = Application(**{'campaign' : campaign,
'user' : request.user})
application.save()
return HttpResponseRedirect(request.META.get('HTTP_REFERER'))
This is the test that calls it:
def test_create_campaign_app(self):
""" Calls the method apply_to_cmp in .views """
c = Client()
c.login(username = self.username, password = self.password)
url = '/campaign/' + self.campaign.id + '/apply/'
response = c.get(url)
# Check whether request was successful (should return 302: redirect)
self.assertEqual(response.status_code, 302)
# Verify that an Application object was created
app_count = Application.objects\
.filter(user = self.user, campaign = self.campaign)\
.count()
self.assertEqual(app_count, 1)
This is the output from the running the test:
Traceback (most recent call last):
File "/test_views.py", line 40, in test_create_campaign_app
self.assertEqual(app_count, 1)
AssertionError: 0 != 1
The method apply_to_cmp is definitely being called, since response.status_code == 302, but still the Application object is not created. What am I doing wrong?
Edit: Solution
Client.login failed because the login system was not properly initialised in the setUp method. I fixed this by calling call_command('loaddata', 'initial_data.json') with initial_data.json containing the setup for the login system. Also, HttpResponseRedirect(request.META.get('HTTP_REFERER')) didn't work for obvious reasons. I changed that bit to
if request.META.get('HTTP_REFERER'):
return HttpResponseRedirect(request.META.get('HTTP_REFERER'))
return HttpResponse()
And therefore the test to
self.assertEqual(response.status_code, 200)
Thanks for your help!

Nothing stands out as particularly wrong with your code - but clearly either your test case or the code your are testing is not working the way you think. It is now time to question your assumptions.
The method apply_to_cmp is definitely being called, since response.status_code == 302
This is your first assumption, and it may not be correct. You might get a better picture of what is happening if you examine other details in the response object. For example, check the response.redirect_chain and confirm that it actually redirects where you expect it to:
response = c.get(url, follow=True)
self.assertEqual(response.redirect_chain, [<expected output here>])
What about other details? I can't see where self.username and self.password are defined from the code you provided. Are you 100% sure that your test code to login worked? c.login() returns 'True' or 'False' to indicate if the login was successful. In my test cases, I like to confirm that the login succeeds.
login_success = c.login(username = self.username, password = self.password)
self.assertTrue(login_success)
You can also be a bit more general. You find nothing if you check Application.objects.filter(user=self.user, campaign=self.campaign), but what about checking Application.objects.all()? You know that a specific item isn't in your database, but do you know what is stored in the database (if anything at all) in the test code at that time? Do you expect other items in there? Check to confirm that what you expect is true.
I think you can solve this one, but you'll need to be a bit more aggressive in your analysis of your test case, rather than just seeing that your app_count variable doesn't equal 1. Examine your response object, put in some debug statements, and question every assumption.

First of all, if you are subclassing from django.test.TestCase, please take in consideration the fact that each test is wrapped into transactions (official docs).
Then, you can add db logging to your project to see whether there was a hit to the database or not (official docs).
And finally be sure that you're using correct lookups at this line: filter(user = self.user, campaign = self.campaign)

Python Django Multiple Database Committing Objects with Foreign Key Relations

I am using Django with multiple databases. I have a 'preview' database that takes messages uploaded by the user and these have to be previewed by an admin and 'accepted', at which point they are committed to the 'default' production database. The following view is supposed to do that, but I'm getting an error. Each newSentenceModel has a Foreign Key to each newMessageSegment, and each newMessageSegment has a Foreign Key to each Message. I want to move each item to the new database if the admin accepts the content, and then delete the old entries in the preview database. Please help! Thanks -
Here is the error:
instance is on database "preview", value is on database "default"
The Error message occurs at this line:
newMessageSegment.msg = newMessage # Setup the foreign key to the msg itself
Here is the View function:
## This view is used when the admin approves content and clicks on the "accept content" button when reviewing
## a recent upload - it saves the data to the production database
def accept_content(request, msg_id=None):
if msg_id == None: # If for some reason we got a None, then it's not a valid page to accept so redirect home
return HttpResponseRedirect("/") # Redirect home
msgList = Message.objects.using('preview').all() # Get all Messages
msgSegmentList = MessageSegment.objects.using('preview').all() # Get all MessageSegment Objects
sentenceModels = SentenceModel.objects.using('preview').all() # Get all SentenceModels
for msgs in msgList: # Iterate all msgs
if int(msgs.id) != int(msg_id): # Don't care if it is not the msg needing review
continue # Short Circuit
msgPrimaryKey = msgs.pk # Extract the primary key from this msg to restore later
msgs.pk = None # Erase the primary key so we can migrate databases properly
newMessage = msgs # This is the msg to transfer to the new one
newMessage.save(using='default') # Save the item to the production database
for msgSegment in msgSegmentList: # Iterate all msg segments for this msg
if msgSegment.msg_id == msgPrimaryKey: # Check the foreign keys on the msg segment to msg connection
newMessageSegment = msgSegment # Define a new msg segment
msgSegment.pk = None # Erase the primary key so we can assign it properly
newMessageSegment.pk = None # Erase the primary key so we can assign it properly
newMessageSegment.msg = newMessage # Setup the foreign key to the msg itself
newMessageSegment.save(using='default') # Save the item to the production database
for sentenceModel in sentenceModels: # Iterate all sentences for this msg segment
if sentenceModel.msg_segment_id == msgSegment.id: # Determine which sentences are for this msg segment
newSentenceModel = sentenceModel # Define the newSentenceModel
newSentenceModel.msg_segment = newMessageSegment # Setup the foreign key to the msg segment
newSentenceModel.save(using='default') # Save the item to the production database
sentenceModel.delete(using='preview') # Delete the item from the review database
msgSegment.delete(using='preview') # Delete the item from the review database
msgs.pk = msgPrimaryKey # Restore the key so we can delete it properly
msgs.delete(using='preview') # Delete the item from the review database
return HttpResponseRedirect("/")

Django remembers which database the object was saved with, so each newMessageSegment is still affiliated with the preview database until you save it to default and it correctly disallows the cross-database FK assignment. This is untested, but it might work to assign to the underlying msg_id field instead:
newMessageSegment.msg_id = newMessage.id
Failing that, you could create a new copy of newMessageSegment rather than just creating a new reference to it. I think you could automate that by iterating over msgSegment._meta.fields, but I might be overlooking a subtlety of inheritance or something. And any many-to-many fields would be a pain.
Or, if you just want to hack it, edit the internal object tracking it. I wouldn't generally recommend that but it's going to be changed when you save anyway.
newMessageSegment._state.db = "default"
newMessageSegment.msg = newMessage
newMessageSegment.save(using="default")

Picture property format using run with friends fb app

I am editing the Runwithfriends FB sample app to build one of my own. It was working fine and I was making a few changes here and there. I took a break from it for about a fortnight but now when I try to access the app, I get a strange python error:
C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\ROOT\app\main.py in init_facebook(self=<main.RecentRunsHandler object>)
316 user_id=facebook.user_id, friends=friends,
317 access_token=facebook.access_token, name=me[u'name'],
=> 318 email=me.get(u'email'), picture=me[u'picture'])
319 user.put()
320 except KeyError, ex:
<..some portion clipped..>
class 'google.appengine.api.datastore_errors.BadValueError'>: Property picture must be a str or unicode instance, not a dict
args = ('Property picture must be a str or unicode instance, not a dict',)
message = 'Property picture must be a str or unicode instance, not a dict'"
I know this is a very generic error but its pointing to the following code. This code has always been there and I have never touched it. I really do not know where else to look now - I have searched exhaustively but couldnt find a clue. Sorry, if this is still too broad but I would be glad if you can tell me what other info can I provide to debug this :-(
def init_facebook(self):
"""Sets up the request specific Facebook and User instance"""
facebook = Facebook()
user = None
# initial facebook request comes in as a POST with a signed_request
if u'signed_request' in self.request.POST:
facebook.load_signed_request(self.request.get('signed_request'))
# we reset the method to GET because a request from facebook with a
# signed_request uses POST for security reasons, despite it
# actually being a GET. in webapp causes loss of request.POST data.
self.request.method = u'GET'
self.set_cookie(
'u', facebook.user_cookie, datetime.timedelta(minutes=1440))
elif 'u' in self.request.cookies:
facebook.load_signed_request(self.request.cookies.get('u'))
# try to load or create a user object
if facebook.user_id:
user = User.get_by_key_name(facebook.user_id)
if user:
# update stored access_token
if facebook.access_token and \
facebook.access_token != user.access_token:
user.access_token = facebook.access_token
user.put()
# refresh data if we failed in doing so after a realtime ping
if user.dirty:
user.refresh_data()
# restore stored access_token if necessary
if not facebook.access_token:
facebook.access_token = user.access_token
if not user and facebook.access_token:
me = facebook.api(u'/me', {u'fields': _USER_FIELDS})
try:
friends = [user[u'id'] for user in me[u'friends'][u'data']]
user = User(key_name=facebook.user_id,
user_id=facebook.user_id, friends=friends,
access_token=facebook.access_token, name=me[u'name'],
email=me.get(u'email'), picture=me[u'picture'])
user.put()
except KeyError, ex:
pass # ignore if can't get the minimum fields
self.facebook = facebook
self.user = user

Might have to do with the October 2012 Breaking Changes, quote:
/picture connection will return a dictionary when a callback is specified
We will start returning a dictionary containing the fields url, height, width, and is_silhouette when accessing the /picture connection for an object and specifying a callback property. Currently we just return the picture URL as a string.
So at this point in your code, where you are currently using picture=me[u'picture'], try accessing the url property of the picture dictionary instead. (If it has one; I can’t tell you for sure if this is applicable, since I don’t know if your code would be considered as having specified a callback property.)
If my assumption is correct, you could also enable the migration as described in the roadmap; but that will only make your app work in the old way until Oct. 3rd, so probably better to try and fix it right away.

This is the way to get the picture:
picture=me[u'picture'][u'data'][u'url']