I'm using flask-sqlalchemy to store users and posts. The entire database is stored in this users.sqlite3 file. Let's say this is my user class:
class User(db.Model, UserMixin):
id = db.Column("id", db.Integer, primary_key=True)
name = db.Column(db.String(100))
email = db.Column(db.String(100), unique=True)
password = db.Column(db.String(100))
status = db.Column(db.String(100))
about = db.Column(db.String(500))
Now let's say I wanted to add another column like a favorite number or something. I would have to add number = db.Column(db.Integer()). But then it won't work because the file is already generated and now I'm saying that there's another column that doesn't exist in there. So I would have to delete all the data in that file and start with an empty database every time I want to update it.
Is there anyway to get around this? Could I do something to just make it so that those other values were empty when I added them in?
This is called a database migration. You basically need 2 steps-
Update your model file to the new state (add your column).
Run a "migration" - which is a one time change to update the EXISTING database to match the new model (in this case, it would be UPDATE TABLE ADD COLUMN ...)
sqlalchemy provides the alembic package to help with migrations, you just need to generate them and ensure they get run on each deploy to keep your database schema up-to-date with your code.
Related
I have MS Access DB file (.accdb), and inside of file stored photos of persons as attachments. In table constructor I see only one field "photos" with type "Attachment". Actually there three hidden fields with names: photos.FileData, photos.FileName, photos.FileType. For parsing these fields I created following class:
class Person:
__tablename__ = 'persons'
name = Column(String(255), name='name')
photos_data = Column(String, name='photos.FileData', quote=False)
....
If I try to get all attributes of Person in same time, as following:
persons = session.query(Person)
I get error in following generated piece of SQL statement:
SELECT ... [persons].photos.FileData AS persons_photos.FileData ...;
As you can see there dot sign present in alias, which raises ODBC error. I can avoid such behavior to request FileData as separate value:
persons = session.query(Person.photos_data.label('photos_data'))
Or I can use raw SQL without aliases. But This is not normal ORM way that I need, because I have to manually construct Persons object each time after request to DB.
Is it possible to set own label to Column during its declaration or even disable label for selected column?
I saw this great answer, but seems this is not applicable to me. Below statement doesn't work properly:
photos_data = Column(String, name='photos.FileData', quote=False).label('photos_data')
I am newbie in python/sqlachemy world. I am trying to fetch data from PostgreSQL using sqlachemy in flask, where I need to fetch only selected column instead of streaming all the column from database through network. One of the approach is using session (like below) which is working fine,
session.query(User.user_id, User.name).all()
But for some reason I would like to stick with Model.query method instead of using sessions. So I ended up to use something like below,
User.query.options(load_only(User.user_id, User.name)).all()
Above code snippet doesn't filters the selected column, instead it gives back all the column. It looks like sqlachemy doesn't respects the load_only arguments. Why is that behaviour and any solution to achieve my use case in Model.query instead of using sessions?
My user model looks like below,
class User(db.Model):
__tablename__ = 'user_info'
user_id = Column(String(250), primary_key=True)
name = Column(String(250))
email = Column(String(250))
address = Column(String(512))
Version Info
Python - 3.7
sqlachemy - 1.3.11
Edit 1: Though I added load_only attributes, It is generating the following query,
SELECT user.user_id AS user_user_id, user.name AS user_name, user.email AS user_email, user.address AS user_address FROM user_info
I'm actually building a little utils which aims to take flat csv/excel file and populate a target database on MS Access - as I'm working on a Mac, i'm developping it using Postgres...
So I developped a part which deals with messy input (csv/excel) forms (several heading, etc) but that's not my issue at the moment.
On the other hand, I made my Database model using SQLAlchemy Declarative Base API.
I'm facing issue when importing data in some tables:
- Split flat record to several objects
- Check (SELECT) if the record doesn't exists yet based on uniqueness contraints
- If it doesn't exists I create object else I use the existing one
- Propagate keys information to related object
For some tables I'm using the auto_increment arguments but sometimes the record has its own ID (in input file) so I should it for insert/select in my tables and sometimes no ID so I have to create a new technical Id for my table.
Example: I have a record with for primary key -obsr25644- and sometimes nothing so I use a default value created with uuid.
So below the stacktrace when doing selectoperation on a my table. The same error occurs when working on existing data - obsr25644 - and generated uuid - 'a8098c1a-f86e-11da-bd1a-00112444be1e'
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidTextRepresentation) **invalid input syntax for integer**: "obsr25644"
LINE 3: WHERE "Location"."Id_observer" = 'obsr25644'
As you can see below, "Location"."Id_observer" is declared as String(255). I don't understand why the error is related to 'integer'.
[SQL: SELECT "Location"."Id_location" AS "Location_Id_location", [...], "Location"."Id_observer" AS "Location_Id_observer",
FROM "Location"
WHERE "Location"."Id_observer" = %(Id_observer_1)s
LIMIT %(param_1)s]
[parameters: {'Id_observer_1': 'obsr25644', 'param_1': 1}]
class LocationModel(UniqueMixin, Base):
__tablename__ = 'Location'
# Primary key
Id_location = Column(Integer, primary_key=True, autoincrement=True)
[...]
Id_observer = Column(String(255), ForeignKey('Observer.Id_observer'))
observer = relationship("ObserverModel", load_on_pending=True, back_populates="location")
class ObserverModel(UniqueMixin, Base):
__tablename__ = 'Observer'
# Primary key
Id_observer = Column(String(255), primary_key=True, default=UniqueMixin.unique_hash())
[...]
# Relationship
location = relationship("LocationModel", load_on_pending=True, back_populates="observer")
Note :UniqueMixin.unique_hash() returns uuid.uuid4().hex
I'm trying to build a system that allows users to make 'projects'. These projects have a fairly simple syntax, they just need an ID, a name, optionally a description and the participants.
Because I want users to be able to add or remove other users from the project without having to input the entire list of users again, I want to make use of a string or array or some such method instead of a string.
However, I'm stuck with trying to input it. I initially tried a regular list, but SQLalchemy didn't accept that. After a google search, it appears that's not possible? Unless I simply haven't come upon it.
I am now trying to use an Array instead. My current code looks like this:
class DBProject(db.Model):
__tablename__ = 'project'
project_id = db.Column(db.Integer, primary_key=True)
project_name = db.Column(db.String(128))
project_description = db.Column(db.String(255), nullable=True)
project_participants = db.Column(db.ARRAY(db.Integer))
But this gives the error: in _compiler_dispatch raise exc.UnsupportedCompilationError(visitor, cls)
sqlalchemy.exc.CompileError: (in table 'project', column 'project_participants'): Compiler can't render element of type
Before, I tried leaving (db.Integer) out or replacing it with just Integer (because I had seen this with other people with similar problems) like this:
project_participants = db.Column(db.ARRAY(Integer))
Which gives the error that 'Integer' is not defined or, in case of leaving it out altogether, this error:
TypeError: init() missing 1 required positional argument:
'item_type'
I'm hoping to add the array to the database so that I can use it to append new users or delete users without having to make the system's user input all allowed users all over again when he just wants to add or delete one.
First i recommend you strongly to save your participants data in an additional table.
You can add a m:n relation between your DBProject-Table and your Participants-Table.
Anything else would be against any best practice use of databases.
Saving your participants as an Array in your table makes it impossible or at least very uncomfortable to filter by participants in a SQL-query.
But if you have a good reason to ignore that recommendation you can use pickle to make SQLAlchemy transform your array into a string while writing into your database.
class DBProject(db.Model):
__tablename__ = 'project'
project_id = db.Column(db.Integer, primary_key=True)
project_name = db.Column(db.String(128))
project_description = db.Column(db.String(255), nullable=True)
project_participants = db.Column(db.PickleType, nullable=True)
Using that construct you can basicalliy put any object (if not exceeding a database specific maximum size) into a database field.
Save data:
dbproject_object = DBProject()
dbproject_object.name = "a_name"
dbproject_object.participants = ["you","him","another one"]
session.add(dbproject_object)
session.commit()
Read Data:
participants_array = db.session.query(DBProject).filter(name == "a_name").one().participants
Result:
participants_array : ["you","him","another one"]
I have a python function that scrapes some data from a few different websites and I want to save that data into my database only if a certain condition is met. Namely, the scraped data should only be saved if the combination of the location and date field is unique
So in my view I have a new location variable and and date variable and essentially I just need to test this combination of values against what's already in the database. If this combination is unique, then save it. If it's not, then do nothing.
class Speech(models.Model):
location = models.ForeignKey(Location)
speaker = models.CharField(max_lenth=100)
date = models.DateField
I'm pretty new to django so I'm just not sure how to go about executing this sort of database query.
You want a combination of two things. First, you want a inner Meta class to enforce the uniqueness in the database:
class Speech(models.Model):
location = models.ForeignKey(Location)
speaker = models.CharField(max_length=100)
date = models.DateField()
class Meta:
unique_together = ('location', 'date')
Then, when you're doing your data manipulation in your view, you want the get_or_create method of the default model manager:
speech, new = Speech.objects.get_or_create(
location=my_location_string,
date=my_datetime_variable,
)
if new:
speech.speaker = my_speaker_string
speech.save()
I hope that gets you started. As always, you know your needs better than I do, so don't blindly copy this example, but adapt it to your needs.
Documentation:
unique_together
get_or_create