I have MS Access DB file (.accdb), and inside of file stored photos of persons as attachments. In table constructor I see only one field "photos" with type "Attachment". Actually there three hidden fields with names: photos.FileData, photos.FileName, photos.FileType. For parsing these fields I created following class:
class Person:
__tablename__ = 'persons'
name = Column(String(255), name='name')
photos_data = Column(String, name='photos.FileData', quote=False)
....
If I try to get all attributes of Person in same time, as following:
persons = session.query(Person)
I get error in following generated piece of SQL statement:
SELECT ... [persons].photos.FileData AS persons_photos.FileData ...;
As you can see there dot sign present in alias, which raises ODBC error. I can avoid such behavior to request FileData as separate value:
persons = session.query(Person.photos_data.label('photos_data'))
Or I can use raw SQL without aliases. But This is not normal ORM way that I need, because I have to manually construct Persons object each time after request to DB.
Is it possible to set own label to Column during its declaration or even disable label for selected column?
I saw this great answer, but seems this is not applicable to me. Below statement doesn't work properly:
photos_data = Column(String, name='photos.FileData', quote=False).label('photos_data')
Related
I'm using flask-sqlalchemy to store users and posts. The entire database is stored in this users.sqlite3 file. Let's say this is my user class:
class User(db.Model, UserMixin):
id = db.Column("id", db.Integer, primary_key=True)
name = db.Column(db.String(100))
email = db.Column(db.String(100), unique=True)
password = db.Column(db.String(100))
status = db.Column(db.String(100))
about = db.Column(db.String(500))
Now let's say I wanted to add another column like a favorite number or something. I would have to add number = db.Column(db.Integer()). But then it won't work because the file is already generated and now I'm saying that there's another column that doesn't exist in there. So I would have to delete all the data in that file and start with an empty database every time I want to update it.
Is there anyway to get around this? Could I do something to just make it so that those other values were empty when I added them in?
This is called a database migration. You basically need 2 steps-
Update your model file to the new state (add your column).
Run a "migration" - which is a one time change to update the EXISTING database to match the new model (in this case, it would be UPDATE TABLE ADD COLUMN ...)
sqlalchemy provides the alembic package to help with migrations, you just need to generate them and ensure they get run on each deploy to keep your database schema up-to-date with your code.
I am newbie in python/sqlachemy world. I am trying to fetch data from PostgreSQL using sqlachemy in flask, where I need to fetch only selected column instead of streaming all the column from database through network. One of the approach is using session (like below) which is working fine,
session.query(User.user_id, User.name).all()
But for some reason I would like to stick with Model.query method instead of using sessions. So I ended up to use something like below,
User.query.options(load_only(User.user_id, User.name)).all()
Above code snippet doesn't filters the selected column, instead it gives back all the column. It looks like sqlachemy doesn't respects the load_only arguments. Why is that behaviour and any solution to achieve my use case in Model.query instead of using sessions?
My user model looks like below,
class User(db.Model):
__tablename__ = 'user_info'
user_id = Column(String(250), primary_key=True)
name = Column(String(250))
email = Column(String(250))
address = Column(String(512))
Version Info
Python - 3.7
sqlachemy - 1.3.11
Edit 1: Though I added load_only attributes, It is generating the following query,
SELECT user.user_id AS user_user_id, user.name AS user_name, user.email AS user_email, user.address AS user_address FROM user_info
I'm actually building a little utils which aims to take flat csv/excel file and populate a target database on MS Access - as I'm working on a Mac, i'm developping it using Postgres...
So I developped a part which deals with messy input (csv/excel) forms (several heading, etc) but that's not my issue at the moment.
On the other hand, I made my Database model using SQLAlchemy Declarative Base API.
I'm facing issue when importing data in some tables:
- Split flat record to several objects
- Check (SELECT) if the record doesn't exists yet based on uniqueness contraints
- If it doesn't exists I create object else I use the existing one
- Propagate keys information to related object
For some tables I'm using the auto_increment arguments but sometimes the record has its own ID (in input file) so I should it for insert/select in my tables and sometimes no ID so I have to create a new technical Id for my table.
Example: I have a record with for primary key -obsr25644- and sometimes nothing so I use a default value created with uuid.
So below the stacktrace when doing selectoperation on a my table. The same error occurs when working on existing data - obsr25644 - and generated uuid - 'a8098c1a-f86e-11da-bd1a-00112444be1e'
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidTextRepresentation) **invalid input syntax for integer**: "obsr25644"
LINE 3: WHERE "Location"."Id_observer" = 'obsr25644'
As you can see below, "Location"."Id_observer" is declared as String(255). I don't understand why the error is related to 'integer'.
[SQL: SELECT "Location"."Id_location" AS "Location_Id_location", [...], "Location"."Id_observer" AS "Location_Id_observer",
FROM "Location"
WHERE "Location"."Id_observer" = %(Id_observer_1)s
LIMIT %(param_1)s]
[parameters: {'Id_observer_1': 'obsr25644', 'param_1': 1}]
class LocationModel(UniqueMixin, Base):
__tablename__ = 'Location'
# Primary key
Id_location = Column(Integer, primary_key=True, autoincrement=True)
[...]
Id_observer = Column(String(255), ForeignKey('Observer.Id_observer'))
observer = relationship("ObserverModel", load_on_pending=True, back_populates="location")
class ObserverModel(UniqueMixin, Base):
__tablename__ = 'Observer'
# Primary key
Id_observer = Column(String(255), primary_key=True, default=UniqueMixin.unique_hash())
[...]
# Relationship
location = relationship("LocationModel", load_on_pending=True, back_populates="observer")
Note :UniqueMixin.unique_hash() returns uuid.uuid4().hex
I'm trying to build a system that allows users to make 'projects'. These projects have a fairly simple syntax, they just need an ID, a name, optionally a description and the participants.
Because I want users to be able to add or remove other users from the project without having to input the entire list of users again, I want to make use of a string or array or some such method instead of a string.
However, I'm stuck with trying to input it. I initially tried a regular list, but SQLalchemy didn't accept that. After a google search, it appears that's not possible? Unless I simply haven't come upon it.
I am now trying to use an Array instead. My current code looks like this:
class DBProject(db.Model):
__tablename__ = 'project'
project_id = db.Column(db.Integer, primary_key=True)
project_name = db.Column(db.String(128))
project_description = db.Column(db.String(255), nullable=True)
project_participants = db.Column(db.ARRAY(db.Integer))
But this gives the error: in _compiler_dispatch raise exc.UnsupportedCompilationError(visitor, cls)
sqlalchemy.exc.CompileError: (in table 'project', column 'project_participants'): Compiler can't render element of type
Before, I tried leaving (db.Integer) out or replacing it with just Integer (because I had seen this with other people with similar problems) like this:
project_participants = db.Column(db.ARRAY(Integer))
Which gives the error that 'Integer' is not defined or, in case of leaving it out altogether, this error:
TypeError: init() missing 1 required positional argument:
'item_type'
I'm hoping to add the array to the database so that I can use it to append new users or delete users without having to make the system's user input all allowed users all over again when he just wants to add or delete one.
First i recommend you strongly to save your participants data in an additional table.
You can add a m:n relation between your DBProject-Table and your Participants-Table.
Anything else would be against any best practice use of databases.
Saving your participants as an Array in your table makes it impossible or at least very uncomfortable to filter by participants in a SQL-query.
But if you have a good reason to ignore that recommendation you can use pickle to make SQLAlchemy transform your array into a string while writing into your database.
class DBProject(db.Model):
__tablename__ = 'project'
project_id = db.Column(db.Integer, primary_key=True)
project_name = db.Column(db.String(128))
project_description = db.Column(db.String(255), nullable=True)
project_participants = db.Column(db.PickleType, nullable=True)
Using that construct you can basicalliy put any object (if not exceeding a database specific maximum size) into a database field.
Save data:
dbproject_object = DBProject()
dbproject_object.name = "a_name"
dbproject_object.participants = ["you","him","another one"]
session.add(dbproject_object)
session.commit()
Read Data:
participants_array = db.session.query(DBProject).filter(name == "a_name").one().participants
Result:
participants_array : ["you","him","another one"]
I need to iterate and delete all records of my datastore. I am using Google App engine Launcher to test it on local host. How to do it?
When I am trying to delete all recors in Person model that way:
qObj = Person.all()
db.delete(qObj)
I am getting error BadValueError: Property y must be a str or unicode instance, not a long
I guess there is conflict in Model data types.
class Person(db.Model):
name = db.StringProperty()
x = db.StringProperty()
y = db.StringProperty()
group = db.StringProperty()
The field y = db.StringProperty() previously was y = db.IntegerProperty().
At this moment I need to flush all db records. How can I do that?
Is there is an opportunity to delete local file which stores all db records?
The GQL language can only be used to retrieve entities or key (cf. http://code.google.com/appengine/docs/python/datastore/gqlreference.html)
You'll have to do this:
persons = Person.all()
for p in persons:
p.delete()
Regarding the error BadValueError: Property y must be a str or unicode instance, not a long, you'll have to modify all the data (from integer to string) in the database to resolve the conflict.
It seems that you want to delete everything, so another solution would be to just go to the datastore administration page - http://localhost:8080/_ah/admin on localhost or via https://appengine.google.com/ - and remove everything.
You might find this useful: http://code.google.com/appengine/articles/update_schema.html
If you have a variable that stores a record on the database then you can simply use delete().
That is, say you have an Entity called Persons, you can do:
personToDelete = db.GqlQuery("SELECT * FROM Persons WHERE name='Joe'");
person = personToDelete[0];
person.delete();
You do also have to import the database library, but I'm assuming you do that anyway given that you're clearly using the database.
Just to share a helpful tips on the already accepted answer.
You can do the following with db.delete:
persons = Person.all()
d = []
for p in persons:
d.append(p)
db.delete(d)
This saves a lot of db operations.