I'm trying to output results of my mysql query to JSON.
I have problem with serializing datetime.datetime field, so I wrote small function to do that:
def date_handler(obj):
if hasattr(obj, 'isoformat'):
return obj.isoformat()
else:
return obj
and then in main code I'm just running:
products_json = []
for code in best_matching_codes:
cur = db.cursor()
query = "SELECT * FROM %s WHERE code LIKE '%s'" % (PRODUCTS_TABLE_NAME, product_code)
cur.execute(query)
columns = [desc[0] for desc in cur.description]
rows = cur.fetchall()
for row in rows:
products_json.append(dict((k,v) for k,v in zip(columns,row)))
return json.dumps(products_json, default = date_handler)
However, since I wrote date_handler function, I'm getting "ValueError: Circular reference detected"
127.0.0.1 - - [10/Jan/2013 00:42:18] "GET /1/product?code=9571%2F702 HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/flask/app.py", line 1701, in __call__
return self.wsgi_app(environ, start_response)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1689, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Library/Python/2.7/site-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/pisarzp/Desktop/SusyChoosy/susyAPI/test1.py", line 69, in product_search
return json.dumps(products_json, default = date_handler)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
ValueError: Circular reference detected
What did I break? Is there a better way to serialize output to JSON?
The function you pass as the default argument will only be called for objects that are not natively serializable by the json module. It must return a serializable object, or raise a TypeError.
Your version returns the same object you were passed if it's not of the one type you're fixing (dates). That is causing the circular reference error (which is misleading, since the circle is between one object and itself after being processed by date_handler).
You can start to fix this by changing date_handler to raise an exception in its else block. That will still probably fail, but you can probably then find out what object it is that is in your data structure causing the problem using code like this:
def date_handler(obj):
if hasattr(obj, 'isoformat'):
return obj.isoformat()
else:
raise TypeError(
"Unserializable object {} of type {}".format(obj, type(obj))
)
Instead of raising the TypeError yourself, you should relay the call to JSONEncoder's default-method:
def date_handler(obj):
if hasattr(obj, 'isoformat'):
return obj.isoformat()
else:
json.JSONEncoder.default(self,obj)
This will also raise TypeError and is a better practice, it allows for JSONEncoder to try and encode the type your method can't.
json.dumps(obj, default=method_name)
"method_name" function must be return a serialize object.
def method_name(obj):
data = {
'__class__': obj.__class__.__name__,
'__module__': obj.__module__
}
data.update(obj.__dict__)
return data
Related
I'm developing an application with python and Flask to automate sending emails.
When I run this application locally, there is no problem, but after deploying the application in Azure, this error appears
TypeError: Object of type NoSectionError is not JSON serializable
The function that is bursting the error is this, when I try to return the dictionary variable
#app.route('/send_email', methods=['POST'])
def get_parameters():
"""
It receives a JSON object, prints it, and then calls another function that sends an email
:return: The result of the function send_email_to_customer
"""
user_input = request.json
print("Conteudo ", user_input)
try:
user = get_config_data(iten_title='EMAIL_LOGIN', iten='email')
psw = get_config_data(iten_title='EMAIL_LOGIN', iten='password')
id_meeting = user_input['id']
list_of_recipients = user_input['recipients']
creator_name = user_input['creator_name']
date_list = user_input['meeting_day']
subject = f'Melhor data para a reuniao {id_meeting}'
url = "https://easy-meeting.azurewebsites.net/external_url?meeting_day="+",".join(date_list)
email_server = create_email_server(user, psw)
for recipients_dict in list_of_recipients:
email_msg = create_email_body(user, recipients_dict, creator_name, subject, url)
run(email_server, email_msg)
dictionary = {"Status":"Sucesso ao enviar o email"}
except Exception as email_error:
dictionary = {"Status":"Erro ao enviar o email", "Error Message": email_error}
print(email_error)
dictionary = jsonify(dictionary)
return dictionary
full error code
2022-11-26T02:26:55.182403211Z [2022-11-26 02:26:55,162] ERROR in app: Exception on /send_email [POST]
2022-11-26T02:26:55.182438711Z Traceback (most recent call last):
2022-11-26T02:26:55.182445810Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/app.py", line 2525, in wsgi_app
2022-11-26T02:26:55.182480710Z response = self.full_dispatch_request()
2022-11-26T02:26:55.182501610Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/app.py", line 1822, in full_dispatch_request
2022-11-26T02:26:55.182505610Z rv = self.handle_user_exception(e)
2022-11-26T02:26:55.182509810Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/app.py", line 1820, in full_dispatch_request
2022-11-26T02:26:55.182513410Z rv = self.dispatch_request()
2022-11-26T02:26:55.182516910Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/app.py", line 1796, in dispatch_request
2022-11-26T02:26:55.182520410Z return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2022-11-26T02:26:55.182524010Z File "/tmp/8dacf551bd5f657/api_run.py", line 55, in get_parameters
2022-11-26T02:26:55.182527510Z dictionary = jsonify(dictionary)
2022-11-26T02:26:55.182531010Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/json/__init__.py", line 342, in jsonify
2022-11-26T02:26:55.182534510Z return current_app.json.response(*args, **kwargs)
2022-11-26T02:26:55.182538010Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/json/provider.py", line 309, in response
2022-11-26T02:26:55.182541510Z f"{self.dumps(obj, **dump_args)}\n", mimetype=mimetype
2022-11-26T02:26:55.182545110Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/json/provider.py", line 230, in dumps
2022-11-26T02:26:55.182548610Z return json.dumps(obj, **kwargs)
2022-11-26T02:26:55.182551910Z File "/opt/python/3.10.4/lib/python3.10/json/__init__.py", line 238, in dumps
2022-11-26T02:26:55.182555410Z **kw).encode(obj)
2022-11-26T02:26:55.182558710Z File "/opt/python/3.10.4/lib/python3.10/json/encoder.py", line 199, in encode
2022-11-26T02:26:55.182562210Z chunks = self.iterencode(o, _one_shot=True)
2022-11-26T02:26:55.182565610Z File "/opt/python/3.10.4/lib/python3.10/json/encoder.py", line 257, in iterencode
2022-11-26T02:26:55.182569210Z return _iterencode(o, 0)
2022-11-26T02:26:55.182572510Z File "/tmp/8dacf551bd5f657/antenv/lib/python3.10/site-packages/flask/json/provider.py", line 122, in _default
2022-11-26T02:26:55.182576010Z raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")
2022-11-26T02:26:55.182579609Z TypeEr
I already tried to add the jsonify flask function, but it didn't work
The goal is to make the dictionary variable just like local
enter image description here
I am trying to fetch data from form in Flask:
app.py
#app.route('/newProjectCreation/', methods=['GET', 'POST'])
def newProjectCreation():
try:
if request.method == 'POST':
# Data Fetch from Models Form
name = request.form['modelName']
modelDescription = request.form['modelDescription']
inputType = request.form.getlist['inputType[]']
outputType = request.form.getlist['outputType[]']
model = request.files['model']
modelLanguage = request.form['modelLanguage']
HTML
It is a simple form (not Flask-WTF form) and the fields "inputType" and "outputType" are supposed to return array as they are dynamically input fields.
Error:
Traceback (most recent call last):
File "/home/adiagarwal/cad/flaskapp-docker/flaskapp/app.py", line 106, in newProjectCreation
inputType = request.form.getlist['inputType[]']
TypeError: 'method' object is not subscriptable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/adiagarwal/cad/env/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/adiagarwal/cad/flaskapp-docker/flaskapp/app.py", line 129, in newProjectCreation
return {{e}}
TypeError: unhashable type: 'set'
I don't have prior experience in js and I am learning, if possible please simplify the response.
Thank You in advance.
Ok, I was focused on a big problem, but the problem is so simple that I did not realize it: Replace all [] with (). See below.
If you [] it assumes as dict and you are returting a list or set.
#app.route('/newProjectCreation/', methods=['GET', 'POST'])
def newProjectCreation():
try:
if request.method == 'POST':
# Data Fetch from Models Form
name = request.form('modelName')
modelDescription = request.form('modelDescription')
inputType = request.form.getlist('inputType[]')
outputType = request.form.getlist('outputType[]')
model = request.files('model')
modelLanguage = request.form('modelLanguage')
I built my api with flask, connexion and swagger UI. I defined my apispec with openapi. I created swagger python server stub from swagger editor where you can find sample project on github. when I deseialize request JSON data for validation, util.py will be used, but I have following error:
erorr
Traceback (most recent call last):
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\_compat.py", line 39, in reraise
raise value
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\connexion\decorators\decorator.py", line 48, in wrapper
response = function(request)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\connexion\decorators\security.py", line 327, in wrapper
return function(request)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\connexion\decorators\uri_parsing.py", line 144, in wrapper
response = function(request)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\connexion\decorators\validation.py", line 184, in wrapper
response = function(request)
File "C:\Users\kim\AppData\Local\Programs\Python\Python37\lib\site-packages\connexion\decorators\parameter.py", line 121, in wrapper
return function(**kwargs)
File "C:\Users\kim\codegen_server\openapi_server\controllers\default_controller.py", line 74, in immunomatch_ed_post
body = ImmunomatchEdInput.from_dict(connexion.request.get_json()) # noqa: E501
File "C:\Users\kim\codegen_server\openapi_server\models\immunomatch_ed_input.py", line 62, in from_dict
return util.deserialize_model(dikt, cls)
File "C:\Users\kim\codegen_server\openapi_server\util.py", line 111, in deserialize_model
setattr(instance, attr, _deserialize(value, attr_type))
File "C:\Users\kim\codegen_server\openapi_server\util.py", line 26, in _deserialize
elif type(klass) == typing.GenericMeta:
AttributeError: module 'typing' has no attribute 'GenericMeta'
error message said typing module has no attribute 'GenericMeta', which I don't understand why. can anyone point me out why this error happened? Is that because of typing module version error or what? any possible thought to get rid of this error? any thought? thanks
update: code that I tried:
import six
import typing
def _deserialize(data, klass):
"""Deserializes dict, list, str into an object.
:param data: dict, list or str.
:param klass: class literal, or string of class name.
:return: object.
"""
if data is None:
return None
if klass in six.integer_types or klass in (float, str, bool):
return _deserialize_primitive(data, klass)
elif klass == object:
return _deserialize_object(data)
elif klass == datetime.date:
return deserialize_date(data)
elif klass == datetime.datetime:
return deserialize_datetime(data)
elif type(klass) == typing.GenericMeta:
if klass.__extra__ == list:
return _deserialize_list(data, klass.__args__[0])
if klass.__extra__ == dict:
return _deserialize_dict(data, klass.__args__[1])
else:
return deserialize_model(data, klass)
error said this line elif type(klass) == typing.GenericMeta: is not passed. any idea?
you might change your code to something like this:
elif hasattr(klass, '__origin__'):
if klass.__origin__ == list:
return _deserialize_list(data, klass.__args__[0])
if klass.__origin__ == dict:
return _deserialize_dict(data, klass.__args__[1])
I tried to add a list to an attribute but when I tried to save the db it raises the exception below. Is there something wrong with my code? I previously modified the models to add new attribute saved_jadwal_ids. I thought it caused no problem as I've tried to set value to it before doing jadwal.save() and I could print the value. Do I actually need to make migrations or something? Thank you.
jadwal = Jadwal.objects(id=jadwal_id).first()
eventIds = []
# courses = data['jadwals']
courses = getCourses()
for course in courses:
id = insert_event(calendar, course)
eventIds.append(id)
jadwal.saved_jadwal_ids = eventIds
jadwal.save()
Jadwal.py:
class Jadwal(Document):
user_id = ReferenceField('User')
jadwals = ListField(EmbeddedDocumentField(JadwalData))
created_at = DateTimeField(default=datetime.now)
saved_jadwal_ids = ListField(StringField)
Error:
Traceback (most recent call last):
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask\app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask\app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask\app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask\_compat.py", line 35, in reraise
raise value
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask\app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\flask\app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\Anisha\RISTEK BISMILLAH\sunjadv2-server\app.py", line 631, in add_jadwal_to_calendar
jadwal.save()
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\mongoengine\document.py", line 362, in save
self.validate(clean=clean)
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\mongoengine\base\document.py", line 377, in validate
field._validate(value)
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\mongoengine\base\fields.py", line 234, in _validate
self.validate(value, **kwargs)
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\mongoengine\fields.py", line 857, in validate
super(ListField, self).validate(value)
File "c:\users\anisha\appdata\local\programs\python\python37\lib\site-packages\mongoengine\base\fields.py", line 431, in validate
self.field._validate(v)
TypeError: _validate() missing 1 required positional argument: 'value'
Try replacing:
saved_jadwal_ids = ListField(StringField)
with:
saved_jadwal_ids = ListField(StringField())
I could reproduce and fix the error with this snippet:
from mongoengine import Document, StringField, ListField
class Example(Document):
meta = {'collection': 'examples'}
saved_jadwal_ids = ListField(StringField) # should be StringField()
e = Example()
e.saved_jadwal_ids = ['not','ok']
e.save()
TypeError: _validate() missing 1 required positional argument: 'value'
I'm working with League of Legends API, and I'm trying to get Ranked Datas from JSON file. But, if the player is not Level 30, he doesn't have his file.
So here
def getRankedData(region, ID, APIkey):
URL = "https://" + region + ".api.pvp.net/api/lol/" + region + "/v2.5/league/by-summoner/" + ID + "/entry?api_key=" + APIkey
response = requests.get(URL)
return response.json()
It won't get JSON file, because it doesn't exist. How do i can do, that if the URL doesn't exist and doesn't have the JSON file, it returns the string.
Here, I'm returning the datas to HTML page. But this isn't work too.
region = request.form['region']
summonerName = request.form['summonerName']
APIkey = "45afde27-b628-473f-9a94-feec8eb86094"
types = request.form['types']
responseJSON = getData(region, summonerName, APIkey)
ID = responseJSON[summonerName]['id']
ID = str(ID)
responseJSON2 = getRankedData(region, ID, APIkey)
if not responseJSON2:
divisionName = "Unranked"
else:
divisionName = responseJSON2[ID][0]['name']
responseJSON3 = getChallengerPlayers(region, str(types), APIkey)
challengerPlayers = responseJSON3['entries'][0]['wins']
#print challengerPlayers
return render_template('form_action.html', ID = ID, divisionName = divisionName, challengerPlayers = challengerPlayers)
I'm getting this error:
Traceback (most recent call last)
File "C:\Python27\lib\site-packages\flask\app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "C:\Python27\lib\site-packages\flask\app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "C:\Python27\lib\site-packages\flask\app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "C:\Python27\lib\site-packages\flask\app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "C:\Python27\lib\site-packages\flask\app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Python27\lib\site-packages\flask\app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Python27\lib\site-packages\flask\app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Python27\lib\site-packages\flask\app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\Hanisek\Documents\Visual Studio 2015\Projects\FlaskWebProject2\FlaskWebProject2\FlaskWebProject2\views.py", line 53, in hello
responseJSON2 = getRankedData(region, ID, APIkey)
File "C:\Users\Hanisek\Documents\Visual Studio 2015\Projects\FlaskWebProject2\FlaskWebProject2\FlaskWebProject2\views.py", line 21, in getRankedData
Open an interactive python shell in this framereturn response.json()
File "C:\Python27\lib\requests\models.py", line 805, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
dont know a ton about LOL but is there a reason that you cant have your program use and if/then statement to check the level of the player and then only check for the json file if the player is a high enough level to have one?
You can try checking if the URL exists, JSON is proper format or if the page throws 40X status codes
try:
r = requests.get("URL")
assert r.status_code < 400
return r.json()
except (ValueError, ConnectionError, AssertionError):
return ''
It's always a good idea to check what the api url is returning by running it in your browser. I'm guessing that it's returning a 404 error because the information doesn't exist.
In that case, I recommend checking to see if there is a 404 error before proceeding with the JSON parsing.
Request has a function called status_code that will return the 404 error if there is one.
Example code:
r = request.get("API STRING")
if r.status_code != 404:
r.json()