Develop Reference

Handling JSON registries with MongoDB

I have created a Flask service and I'm having a trouble on parsing some JSON values which are stored in MongoDB. I know how to parse every object which has a single value expect the address parameter which has more values inside (e.g "address":[{"street":"Jardine Place","city":"Lowgap","postcode":...)
JSON file:
{"name":"Morton Fitzgerald","email":"mortonfitzgerald#ontagene.com","yearOfBirth":{"$numberInt":"1997"},"address":[{"street":"Jardine Place","city":"Lowgap","postcode":{"$numberInt":"18330"}}]}
{"name":"Dorthy Cobb","email":"dorthycobb#ontagene.com","yearOfBirth":{"$numberInt":"1994"}}
In the python script, I'm trying to find a student based on the email, which I'm giving to the body at Postman. After that, if a student with an address is found I need to return a message with every value inside the "address" except the "city". With the following code, I'm taking some results but from them, I need to print out only the "street" and "postcode".
Python function:
#app.route('/getStudentAddress', methods=['GET'])
def get_student_address():
# Request JSON data
data = None
try:
data = json.loads(request.data)
uuid = request.headers.get('authorization')
except Exception as e:
return Response("bad json content",status=500,mimetype='application/json')
if data == None:
return Response("bad request",status=500,mimetype='application/json')
if not "email" in data:
return Response("Information incomplete",status=500,mimetype="application/json")
if (is_session_valid(uuid)):
student = students.find_one({"email":data['email']})
if(student != None):
student = students.find_one({"address":student['address']})
if(student != None):
student = {'name':student["name"],'address':student["address"]}
return Response(json.dumps(student), status=200, mimetype='application/json')
else:
return Response("Not-found", status=400, mimetype='application/json')
else:
return Response("Not authenticated user", status=401, mimetype='application/json')
Results:
{"name": "Morton Fitzgerald", "address": [{"street": "Jardine Place", "city": "Lowgap", "postcode": 18330}]}
Error:
If I choose a registry like {"name":"Dorthy Cobb","email":"dorthycobb#ontagene.com","yearOfBirth":{"$numberInt":"1994"}} which doesn't have an address I'm taking an error of KeyError: 'address'
Any thoughts?

You might consider adding a "try to fetch" function that checks to see if the key exists and provides a default value:
def tryfetch( container, key, default=None ):
return container[key] if key in container else default
That way, you can write things like::
address = tryfetch( student, "address" )
if address:
addr = tryfetch(address[0], "street", "NA") + tryfetch(address[0], "postcode", "NA")

im trying to make an api in flask that takes in 2 inputs

ive got an api that takes in an id
http://127.0.0.1:5000/api/v1/resources/books?id=u3qR4Ps4TbATrg97
looks like that
what im trying to do after that is add something to the end of the url, for example
http://127.0.0.1:5000/api/v1/resources/books?id=u3qR4Ps4TbATrg97uid=something
im not 100% sure if this is possible
# Create some test data for our catalog in the form of a list of dictionaries.
books = [
{'id': 'u3qR4Ps4TbATrg97',
'uid': 'what',
'title': 'A Fire Upon the Deep',
'author': 'Vernor Vinge',
'first_sentence': 'The coldsleep itself was dreamless.',
'year_published': '1992'}
]
#app.route('/api/v1/resources/books', methods=['GET'])
def api_id():
# Check if an ID was provided as part of the URL.
# If ID is provided, assign it to a variable.
# If no ID is provided, display an error in the browser.
if 'id' and 'uid' in request.args:
id = str(request.args['id'])
uid = str(request.args['uid'])
else:
return "Error: No id field provided. Please specify an id."
results = []
for book in books:
if book['id'] == id:
results.append(book)
if book['uid'] == uid:
results.append(book)
this is what i have so far, mostly copy pasted from here
thats no the whole file just the important bits i can think of

You can add two inputs inside the GET query like this
http://127.0.0.1:5000/api/v1/resources/books?id=u3qR4Ps4TbATrg97&uid=something
Just put an & in between!

Use request.args.get method to get parameters from your url. Also add & to your URL as a parameter separator.
from flask import Flask, request
app = Flask(__name__)
#app.route('/api/v1/resources/books')
def books():
id_ = request.args.get('id')
uid = request.args.get('uid')
return f'id: {id_}, uid: {uid}'
app.run()
Open http://127.0.0.1:5000/api/v1/resources/books?id=u3qR4Ps4TbATrg97&uid=something
in browser and you'll get:
id: u3qR4Ps4TbATrg97, uid: something

Multiple parameters|arguments are passed with & character. ?params1=5&params2=3. For your example: http://127.0.0.1:5000/api/v1/resources/books?id=u3qR4Ps4TbATrg97&uid=what. For the code, I would do:
from flask import Flask, request, jsonify, make_response
app = Flask(__name__)
# Create some test data for our catalog in the form of a list of dictionaries.
books = [
{
"id": "u3qR4Ps4TbATrg97",
"uid": "what",
"title": "A Fire Upon the Deep",
"author": "Vernor Vinge",
"first_sentence": "The coldsleep itself was dreamless.",
"year_published": "1992",
}
]
#app.route("/api/v1/resources/books", methods=["GET"])
def api_id():
# Check if an ID was provided as part of the URL.
# If ID is provided, assign it to a variable.
# If no ID is provided, display an error in the browser.
if set(["id","uid"]).intersection(set(request.args)):
id_ = str(request.args["id"])
uid = str(request.args["uid"])
else:
return make_response(
jsonify({"message": "Error: No id field provided. Please specify an id."}),
400,
)
results = []
for book in books:
if book["id"] == id_:
results.append(book)
if book["uid"] == uid:
results.append(book)
response = make_response(
jsonify({"message": results}),
200,
)
response.headers["Content-Type"] = "application/json"
return response
This would return status code 400 if no match and 200 when match

TypeError with get_or_insert

Here is my code.
import webapp2
import json
from google.appengine.ext import ndb
class Email(ndb.Model):
email = ndb.StringProperty()
subscribed = ndb.BooleanProperty()
#staticmethod
def create(email):
ekey = ndb.Key("Email", email)
entity = Email.get_or_insert(ekey)
if entity.email: ###
# This email already exists
return None
entity.email = email
entity.subscribed = True
entity.put()
return entity
class Subscribe(webapp2.RequestHandler):
def post(self):
add = Email.create(self.request.get('email'))
success = add is not None
self.response.headers['Content-Type'] = 'application/json'
obj = {
'success': success
}
self.response.out.write(json.dumps(obj))
app = webapp2.WSGIApplication([
webapp2.Route(r'/newsletter/new', Subscribe),
], debug=True)
Here is my error.
File "/Users/nick/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 3524, in _get_or_insert_async
raise TypeError('name must be a string; received %r' % name) TypeError: name must be a string; received Key('Email', 'test#test.com')
What am I missing?

The error is caused by passing ekey (which is an ndb.Key) as arg to get_or_insert() (which expects a string):
ekey = ndb.Key("Email", email)
entity = Email.get_or_insert(ekey)
Since it appears you want to use the user's email as a unique key ID you should directly pass the email string to get_or_insert():
entity = Email.get_or_insert(email)

How can I retrieve the email address from a Google Account and store it in my app's user profile when my user logs in using gae-simpleauth?

I'm attempting to build a very simple user permissions system with webapp2's auth library. I'm using gae-simpleauth to log users in with their Google account. I'm hoping to compare the user's email address to a list of permitted email addresses to determine if a user has access to a resource, but I'm not clear on how to get the email address from the Google account into the account on my app. Users are currently able to log in, but the email address doesn't seem to be something simpleauth adds to their account by default.
How can I retrieve the email address from Google and store it in my app's user profile using gae-simpleauth?
My implementation of gae-simpleauth is nearly identical to the example with the addition of the get_user_and_flags function which fetches the logged in user and sets the admin flag if the user's email is in a list in secrets.py. Unfortunately, that doesn't work because user doesn't have an email attribute.
# -*- coding: utf-8 -*-
import logging, secrets, webapp2
from google.appengine.api import users
from webapp2_extras import auth, sessions, jinja2
from jinja2.runtime import TemplateNotFound
from lib.simpleauth import SimpleAuthHandler
def get_user_and_flags(self):
"""Returns the current user and permission flags for that user"""
flags = {}
user = None
if self.logged_in:
user = self.current_user
flags = {
'admin': user.email in secrets.ADMIN_USERS,
}
return user, flags
def simpleauth_login_required(handler_method):
"""A decorator to require that a user be logged in to access a handler.
To use it, decorate your get() method like this:
#simpleauth_login_required
def get(self):
user = self.current_user
self.response.out.write('Hello, ' + user.name())
"""
def check_login(self, *args, **kwargs):
if self.request.method != 'GET':
self.abort(400, detail='The login_required decorator '
'can only be used for GET requests.')
if self.logged_in:
handler_method(self, *args, **kwargs)
else:
self.session['original_url'] = self.request.url.encode('ascii', 'ignore')
self.redirect('/login/')
return check_login
class BaseRequestHandler(webapp2.RequestHandler):
def dispatch(self):
# Get a session store for this request.
self.session_store = sessions.get_store(request=self.request)
try:
# Dispatch the request.
webapp2.RequestHandler.dispatch(self)
finally:
# Save all sessions.
self.session_store.save_sessions(self.response)
#webapp2.cached_property
def jinja2(self):
"""Returns a Jinja2 renderer cached in the app registry"""
return jinja2.get_jinja2(app=self.app)
#webapp2.cached_property
def session(self):
"""Returns a session using the default cookie key"""
return self.session_store.get_session()
#webapp2.cached_property
def auth(self):
return auth.get_auth()
#webapp2.cached_property
def current_user(self):
"""Returns currently logged in user"""
user_dict = self.auth.get_user_by_session()
return self.auth.store.user_model.get_by_id(user_dict['user_id'])
#webapp2.cached_property
def logged_in(self):
"""Returns true if a user is currently logged in, false otherwise"""
return self.auth.get_user_by_session() is not None
def render(self, template_name, template_vars={}):
# Preset values for the template
values = {
'url_for': self.uri_for,
'logged_in': self.logged_in,
'flashes': self.session.get_flashes()
}
# Add manually supplied template values
values.update(template_vars)
# read the template or 404.html
try:
self.response.write(self.jinja2.render_template(template_name, **values))
except TemplateNotFound:
self.abort(404)
def head(self, *args):
"""Head is used by Twitter. If not there the tweet button shows 0"""
pass
class ProfileHandler(BaseRequestHandler):
def get(self):
"""Handles GET /profile"""
if self.logged_in:
self.render('profile.html', {
'user': self.current_user,
'session': self.auth.get_user_by_session()
})
else:
self.redirect('/')
class AuthHandler(BaseRequestHandler, SimpleAuthHandler):
"""Authentication handler for OAuth 2.0, 1.0(a) and OpenID."""
# Enable optional OAuth 2.0 CSRF guard
OAUTH2_CSRF_STATE = True
USER_ATTRS = {
'facebook' : {
'id' : lambda id: ('avatar_url',
'http://graph.facebook.com/{0}/picture?type=large'.format(id)),
'name' : 'name',
'link' : 'link'
},
'google' : {
'picture': 'avatar_url',
'name' : 'name',
'link' : 'link'
},
'windows_live': {
'avatar_url': 'avatar_url',
'name' : 'name',
'link' : 'link'
},
'twitter' : {
'profile_image_url': 'avatar_url',
'screen_name' : 'name',
'link' : 'link'
},
'linkedin' : {
'picture-url' : 'avatar_url',
'first-name' : 'name',
'public-profile-url': 'link'
},
'foursquare' : {
'photo' : lambda photo: ('avatar_url', photo.get('prefix') + '100x100' + photo.get('suffix')),
'firstName': 'firstName',
'lastName' : 'lastName',
'contact' : lambda contact: ('email',contact.get('email')),
'id' : lambda id: ('link', 'http://foursquare.com/user/{0}'.format(id))
},
'openid' : {
'id' : lambda id: ('avatar_url', '/img/missing-avatar.png'),
'nickname': 'name',
'email' : 'link'
}
}
def _on_signin(self, data, auth_info, provider):
"""Callback whenever a new or existing user is logging in.
data is a user info dictionary.
auth_info contains access token or oauth token and secret.
"""
auth_id = '%s:%s' % (provider, data['id'])
logging.info('Looking for a user with id %s', auth_id)
user = self.auth.store.user_model.get_by_auth_id(auth_id)
_attrs = self._to_user_model_attrs(data, self.USER_ATTRS[provider])
if user:
logging.info('Found existing user to log in')
# Existing users might've changed their profile data so we update our
# local model anyway. This might result in quite inefficient usage
# of the Datastore, but we do this anyway for demo purposes.
#
# In a real app you could compare _attrs with user's properties fetched
# from the datastore and update local user in case something's changed.
user.populate(**_attrs)
user.put()
self.auth.set_session(
self.auth.store.user_to_dict(user))
else:
# check whether there's a user currently logged in
# then, create a new user if nobody's signed in,
# otherwise add this auth_id to currently logged in user.
if self.logged_in:
logging.info('Updating currently logged in user')
u = self.current_user
u.populate(**_attrs)
# The following will also do u.put(). Though, in a real app
# you might want to check the result, which is
# (boolean, info) tuple where boolean == True indicates success
# See webapp2_extras.appengine.auth.models.User for details.
u.add_auth_id(auth_id)
else:
logging.info('Creating a brand new user')
ok, user = self.auth.store.user_model.create_user(auth_id, **_attrs)
if ok:
self.auth.set_session(self.auth.store.user_to_dict(user))
# Remember auth data during redirect, just for this demo. You wouldn't
# normally do this.
self.session.add_flash(data, 'data - from _on_signin(...)')
self.session.add_flash(auth_info, 'auth_info - from _on_signin(...)')
# Go to the last page viewed
target = str(self.session['original_url'])
self.redirect(target)
def logout(self):
self.auth.unset_session()
self.redirect('/')
def handle_exception(self, exception, debug):
logging.error(exception)
self.render('error.html', {'exception': exception})
def _callback_uri_for(self, provider):
return self.uri_for('auth_callback', provider=provider, _full=True)
def _get_consumer_info_for(self, provider):
"""Returns a tuple (key, secret) for auth init requests."""
return secrets.AUTH_CONFIG[provider]
def _to_user_model_attrs(self, data, attrs_map):
"""Get the needed information from the provider dataset."""
user_attrs = {}
for k, v in attrs_map.iteritems():
attr = (v, data.get(k)) if isinstance(v, str) else v(data.get(k))
user_attrs.setdefault(*attr)
return user_attrs

Hope this help( I have same probblem )
First change in secrets.py in line:
'google': (GOOGLE_APP_ID, GOOGLE_APP_SECRET, 'https://www.googleapis.com/auth/userinfo.profile'),
to
'google': (GOOGLE_APP_ID,GOOGLE_APP_SECRET, 'https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email'),
and in auth.py change USER_ATTRS =
{
...
'google' : {
'picture': 'avatar_url',
'email' : 'email', <-- new attr
'name' : 'name',
'link' : 'link'
},
}

Since your question includes no code snippet, I can only guess at what you have done so far. In light of that, the below code should work:
from google.appengine.api import users
user = users.get_current_user()
email = user.email()

Following the idea of nguyên, I add customize also the "_to_user_model_attrs" method.
Here my piece of code:
def _to_user_model_attrs(self, data, attrs_map):
"""Get the needed information from the provider dataset."""
user_attrs = {}
for k, v in attrs_map.iteritems():
if v =="email":
attr = (v, data.get(k)[0].get('value'))
else:
attr = (v, data.get(k)) if isinstance(v, str) else v(data.get(k))
user_attrs.setdefault(*attr)
return user_attrs
It works for me!

There seem to be several methods of authentication, and mix-matching does not work. If you havne't already, make sure you read through this, https://developers.google.com/appengine/articles/auth
There are a few sections that might be relative depending on what else you have been doing with Google.
Changes from the Google Apps account transition
Configuring Google Apps to Authenticate on Appspot

Django: Transferring/Accessing forms full error message

On a website that I'm working on for my school, the user enters their school email and password, and if they have registered they log in. If not, a second part of the log in is revealed asking for a pen name and to confirm the password. Because of this, and my convoluted amateur Django programming, I have a list of errors named er. For instants, when the program tests whether the email is a school one, it might add to the er list "school email only". I am also using two form classes as well. The page uses ajax to call this function, which uses plain html instead of JSON because of the sites small size.
In the forms.py file I have:
class log_in(forms.Form):
username = forms.EmailField(error_messages= {'required': "Email Field is required,", 'invalid' : "Invalid Email Address."})
password = forms.CharField(help_text = 'Password Invalid')
class new_user(forms.Form):
username = forms.EmailField(error_messages = {'required': "Email Field is required,", 'invalid' : "Invalid Email Address."})
password = forms.CharField(required=True)
password2 = forms.CharField(required=True)
pen_name = forms.CharField(max_length=30, min_length=3, error_messages = {'required': "Pen Name is required", 'max_length': "Pen Name must be less than 30 characters", 'min_length': "Pen Name must be more than 3 characters"})
The problem is that I want to transfer the full error message that I specified in the error_message argument to the er list.
This is my views.py file
def user_log_in(request):
er = []
user_pass = log_in(request.POST)
if user_pass.is_valid(): # If it is valid at all
cleaned_info = user_pass.cleaned_data
email_bbn = cleaned_info['username'].split("#")
if 'bbns.org' in email_bbn: # Check if BBN email address
user_object = User.objects.filter(email = cleaned_info['username'])
if user_object.exists():
logged_in_user = auth.authenticate(username=cleaned_info['username'], password=cleaned_info['password'])
#add in is_active
if logged_in_user is not None: #If password is right
if user_object[0].get_profile().activated:
auth.login(request, logged_in_user)
return HttpResponseRedirect("")
else:
return HttpResponse("not_act")
else:
er.append("Incorrect Password")
else: # If new user
new_user_pass = new_user(request.POST)
if new_user_pass.is_valid():
cleaned_info_new = new_user_pass.cleaned_data
if cleaned_info_new['password'] == cleaned_info_new['password2']:
msg = "In order to activate your account at Knights of the Round Table, please click on this link:"
try:
send_mail('Activate', msg, 'michaelrgoldfine#gmail.com', [cleaned_info_new['username']], fail_silently=False)
new_user_object = User.objects.create_user(
username=cleaned_info_new['username'],
password=cleaned_info_new['password'],
email=cleaned_info_new['username']
)
new_user_profile = new_user_object.get_profile()
new_user_profile.pen_name = cleaned_info_new['pen_name']
new_user_profile.activated = False;
new_user_profile.save()
return HttpResponse("not_act")
except:
er.append("Error Sending Email")
else:
er.append('Passwords are not the same')
elif "TN" in request.POST: #If open but not filled in
print "TN"
er.append(new_user_pass.pen_name.error_messages)
else: # if new user field
print "n_usr"
return HttpResponse('n_usr')
else:
er.append("BBN email addresses only")
else:
for e in user_pass.errors:
er.append(e)
errors_template = Template("{% for e in errors %}<li>{{ e }}</li> {% endfor %}")
errors_html = errors_template.render(Context({'errors':er}))
return HttpResponse(errors_html)
I try to accsess the errors twice. Once, on the else you see right at the end with a for loop, and two elses up from that on elif 'TN'... The last one just returns the field thats invalid (so i get user_name or pen_name). The other one says that the form has no object pen_name or whatever I use it for.

It would be better to add errors to the actual form. Forms have an _errors dict attached to them that contain all the errors generated by the form. "Non-field errors" (errors that don't directly relate to a particular field or that relate to multiple fields) go in form._errors['__all__']. All field-specific errors go into the key of the field's name. So, errors for a foo field would go in form._errors['foo'].
Now, the list of errors for each item in the _errors dict is actually an ErrorList type, not a standard list. So, to add errors to the form you do:
from django.forms.util import ErrorList
form._errors.setdefault('foo', ErrorList()).append('Some error here')
Or, to add the error to non-field errors:
form._errors.setdefault('__all__', ErrorList()).append('Some error here')
Then, when your form renders, the errors will all fall naturally where they should, just like any normal validation error.

The array probably looks like error[i].field[i].error, so you're just calling the fieldname and not the error message. Call e.error in your Template() function.