Develop Reference

Flask SQLAlchemy can't insert emoji to MySQL

I'm using Python 2.7 and flask framework with flask-sqlalchemy module.
I always get the following exception when trying to insert : Exception Type: OperationalError. Exception Value: (1366, "Incorrect string value: \xF09...
I already set MySQL database, table and corresponding column to utf8mb4_general_ci and I can insert emoji string using terminal.
Flask's app config already contains app.config['MYSQL_DATABASE_CHARSET'] = 'utf8mb4', however it doesn't help at all and I still get the exception.
Any help is appreciated

maybe this will someone in the future:
all i did was edit the sql connection in my config file:
SQLALCHEMY_DATABASE_URI = 'mysql://user:password#localhost/database?charset=utf8mb4'
this way i'm able to store emojis without any altering of the database or tables.
source: https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database/page/13%0A Comment #322

It works for me:
import pickle
data = request.get_json().get("data")
data = pickle.dumps(data)
Then you can input the "data" to the database .
You can send the "data" like "😢" ...Whatever emoji you like.
Next time, when you get the "data" from the database :
you should :
data = pickle.loads(data)
then you can get "data" as "😢"

Add config file main file and set set 'charset' => 'utf8mb4'
you have to edit field in which you want to store emoji and set collation as utf8mb4_unicode_ci

Make sure to use a proper Python Unicode object, like the ones created with the u"..." literal. In other words, the type of your object should be unicode not str:
>>> type('ą')
<type 'str'>
>>> type(u'ą')
<type 'unicode'>
Please note that this only applies to Python 2, in Python 3 all string literals are Unicode by default.

Can't find django auth user with PostgreSQL

from django.contrib.auth.models import User as DjangoUser
class Ward(models.Model):
user = models.ForeignKey(DjangoUser, related_name='wards')
group = models.ForeignKey(Group, related_name='wards')
This is my django model and I use this filter.
Group.objects.filter(wards__user=_user).all()
I used this code in sqlite3, it works well.
But, it doesn't work in PostgreSQL.
operator does not exist: character varying = integer
LINE 1: ...rchive_ward"."group_id" ) WHERE "archive_ward"."user_id" = 1
I think it is caused by user_id field in archive_ward tables.
I found this field's data type is character.varying(20).
What can I do for this code?

Try removing the user table in the database and adding it again.
create a new one from scratch. Syncing database again will work..
or else You can do like this Way raw_query
You cannot compare an integer with a varchar. PostgreSQL is strict and does not do any magic typecasting for you. I'm guessing SQLServer does typecasting automagically (which is a bad thing).
If you want to compare these two different beasts, you will have to cast one to the other using the casting syntax ::

The Postgres error means you're comparing an integer to a string:
operator does not exist: character varying = integer
You could change the database model so user_id is of an integer type. Or you could cast the integer to string in Python:
Group.objects.filter(wards__user=str(_user)).all()

django migrate has error: Specify a USING expression to perform the conversion

I change my model field from Charfiled() to GenericIPAddressField()
ip = models.GenericIPAddressField()
and use django 1.7 migrate
./manage.py makemigrations core
./manage.py migrate
But there is error:
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "ip" cannot be cast automatically to type inet
HINT: Specify a USING expression to perform the conversion.
I try this,but not work:
ALTER TABLE core_message ALTER COLUMN ip TYPE inet USING (ip::inet);
error:
ERROR: invalid input syntax for type inet: ""
What can I do now?
Please help me Thank you!

one quick fix will be to drop and create the field:
delete the migration what is changing the field type.
delete/comment the field ip
make migrations
get back/uncomment the field ip with the new field type
make migrations
migrate
I did this in production and restored the data with a previous csv backup and an python script of a few lines a code.

Django error: conversion from bytes to Decimal is not supported

So I have the following model
class Stock(models.Model):
name = models.CharField(max_length=50)
unit_measure = models.CharField(max_length=10)
unit_price = models.DecimalField(max_digits=10, decimal_places=2)
When I try to add an instance of that model in Django's admin site, it gives me the following error
(<class 'TypeError'>, TypeError('conversion from bytes to Decimal is not supported',))
Exception Location: /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/PyMySQL-0.5-py3.3.egg/pymysql/connections.py in defaulterrorhandler, line 209
But the data was inserted into the table successfully as I look up my database using phpmyadmin.
I am using Django1.5+Python3.3+MySQL5.5+PyMySQL
Anybody have ideas about what has gone wrong here?

After 11 months I hope the original poster found a workaround (better than switching to python 2).
The OP did not list his db connection string, but maybe he was using the "use_unicode=0" setting for his connection?
I was, and I hit the same type conversion error recently. Seems like it should be possible to convert from a byte string to a Decimal, maybe that is on someone's todo list :), but until then I can share what worked around the problem for me:
When connecting to mysql (through pymysql and python 3.4.1) set the charset=utf8 property (assuming you want that property, which you probably should) but do NOT set the use_unicode=0 property. I set that property on the advice of the current (0.9) sqlalchemy docs which said it would be "much faster." Faster but broken ain't an improvement :(. Maybe that advice was intended only for python2.x users? It's a bit confusing given how pymysql tries to be hot-swappable MySqlDB for python 3.x, but python's unicode & string handling has changed between 2.x and 3.x so...
Without diving deep into pymysql, I assume that with python3 "use_unicode" means that char fields are returned as python native (unicode) strings rather than "byte strings", with contents encoded as utf8. Set "use_unicode=0" and you get byte strings and thus the TypeError.
Anway, this works for me; hope this helps someone else who sees this error.

I had the same problem in SQLite 3, the solution I have found was mentioned in a Book (https://books.google.de/books?id=eLKdDwAAQBAJ&lpg=PA394&ots=xBGSOLY4Ue&dq=python%20sqlite%20byte%20to%20decimal&hl=de&pg=PA394#v=onepage&q&f=false)
def convert_decimal(bytes)
return decimal.Decimal(bytes.decode())

Django MongoDB Engine InvalidID

When using the python shell with the following:
>>> User.objects.get(pk=1)
I get the following error:
InvalidId: AutoField (default primary key) values must be strings
representing an ObjectId on MongoDB (got u'1' instead)
A possible solution to this problem, which did not work for me, may be found here: http://django-mongodb.org/troubleshooting.html
I'm wondering if anybody else has come across this problem and how were you able to fix it?

MongoDB doesn't use integer primary keys.