I have a MySQL table with a JSON column.
I am able to write into this JSON field following either answer to this question,
but I have not yet found a solution to be able to update the json field with the same method.
The solution I tried:
out_df.to_sql('my_table', my_sql_alchemy_engine, if_exists='append')
can go 2 ways:
if there are no unique keys in my table, the rows are simply added at the end of the table, with the JSON field containing only the values I wanted to make an update with
If there are unique fields, i get an IntegrityError
Is there any way to make that work, or should I find another solution to update my JSON fields ?
Related
Say I have an id column that is saved as ids JSON NOT NULL using SQLAlchemy, and now I want to delete an id from this column. I'd like to do several things at once:
query only the rows who have this specific ID
delete this ID from all rows it appears in
a bonus, if possible - delete the row if the ID list is now empty.
For the query, something like this:
db.query(models.X).filter(id in list(models.X.ids)) should work.
now, I'd rather avoid iterating over each query and then send an update request as it can be multiple rows. Is there any elegant way to do this?
Thanks!
For the search and remove remove part you can use json_remove function (from SQLLite built-in functions)
from sqlalchemy import func
db.query(models.X).update({'ids': func.json_remove(models.X.ids,f'$[{TARGET_ID}]') })
Here replace TARGET_ID by the targeted id.
Now this will update the row 'silently' (wether or not this id is present in the array).
If you want to first check if target id is in the column: you can query first all rows containing the target id with json_extract query (calling .all() method and then remove those ids with an .update() call.
But this will cost you double amount of queries (less performant).
For the delete part, you can use the json_array_length built-in function
from sqlalchemy import func
db.query(models.X).filter(func.json_array_length(models.X.ids) == 0).delete()
FYI : Not sure that you can do both in one query, and even if possible, I would not do it for clean syntax, logging and monitoring reasons.
I am working in Python, using Pandas to pull data from a TSV, convert it to a data frame, then syncing that data frame to a temp table in postgres using df.to_sql. That process works great.
However, once that table exists, I want to move all the rows from that table to the permanent table. The two tables will always be identical. The permanent table has a unique index, so if the id already exists it should update the row instead.
Here is my attempt to upsert all rows from one table to another:
WITH moved_rows AS (
DELETE FROM my_table_tmp a
RETURNING a.*
)
INSERT INTO my_table
SELECT * FROM moved_rows
ON CONFLICT ("unique_id") DO
UPDATE SET
Field_A = excluded.Field_A,
Field_B = excluded.Field_B,
Field_C = excluded.Field_C
Unfortunately, when I run this, I get this error:
psycopg2.errors.UndefinedColumn: column excluded.field_a does not exist
LINE 10: Field_A = excluded.Field_A,
^
HINT: Perhaps you meant to reference the column "excluded.Field_A".
But in fact, that column does exist. What am I missing here? I've tried removing Field_A from the set and then I get the same error with Field_B.
Answering my own question here - the issue is that Postgres ignores capitalization unless it's quoted.
This was not clear in the example I posted because I obscured the naming of the fields I was working with. I've updated them now to show the issue.
In order to fix this you need to wrap your field names with double quotes. E.g. "Field_A" = excluded."Field_A"
I'm trying to work with Databases using Django's ORM.
I'm looking to retrieve a value from a specific column in a database and then convert it from a QuerySet type to a Int or String so I can then work with this data.
So my question is, how can I convert a QuerySet to a usable data type?
EDIT:
I'm currently building a calorie tracker.
I want to grab the column values for "protein", "fat" and "carbs" from my MacroGoal database. There is only one row in the database and therefore only one value in each column.
I know I can do this using data = MacroGoal.objects.all()
and it gives me :
<QuerySet [<MacroGoal: ctracker Fat: 45 Protein: 45 Carbs: 45>]>
I now want to be able to use those values from each column(or key in this instance). I need to convert each column value to an integer.
How can be this done?
Please refer to the django ORM docs, they are very thorough: https://docs.djangoproject.com/en/3.1/topics/db/queries/
If there is only ever going to be one row in the database you can do:
obj = MacroGoal.objects.first()
This gets the first row out of the database, and then to get specific information from the object you would do something like
obj.<field_name> and that would give you the value stored in the database, so if you have a field called protein in your model you would do obj.protein to get the value stored in the protein column.
i am creating a database using SQLAlchemy and I Need to do Migration to my data as i am using df_sql function for converting my csv into dataframe and then to tables in sqlalchemy. As i do this i need to do Migration to add new column and values inside it and assign Primary and foreign key Features. I saw someting related to Alembic and flask but am not sure how to upgrade it as also am working on Jupyter. Any ideas of how i can update delete and assign keys to my tables would be very helpful. Done until the table creation.
metadata.tables.keys()
dict_keys(['table1', 'table2'])
I also tried directly to create a temp table and copy ist values and assinging Primary key but am getting error with my column names as it has Special characters so i cant create duplicate too. Rename property too doesnt work
Column: date
Column: time_stamp
Column: timeslices[5].profilerDataProcess[8]_C0[us]
Column: timeslices[4].profilerDataProcess[54]_C0[us]
Column: timeslices[4]profilerDataProcess[50]_C0[us]
Column: timeslices[4].profilerDataProcess[49]_C0[us]
Column: timeslices[0].profilerDataProcess[14]_C0[us]
I'm using Peewee with Postgres database. I want to know how to update multiple records in a tabel at once?
We can perform this update in SQL using these commands, and I'm looking for a Peewee equivalent approach.
Yes, you can use the insert_many() function:
Insert multiple rows at once. The rows parameter must be an iterable
that yields dictionaries. As with insert(), fields that are not
specified in the dictionary will use their default value, if one
exists.
Example:
usernames = ['charlie', 'huey', 'peewee', 'mickey']
row_dicts = ({'username': username} for username in usernames)
# Insert 4 new rows.
User.insert_many(row_dicts).execute()
More details at: http://docs.peewee-orm.com/en/latest/peewee/api.html#Model.insert_many
ORMs usually dose not support bulk update and you have to use custom SQL, you can see samples in this link (db.excute_sql)