Postgresql sessions from psql and psycopg2 - python

I am running two terminal sessions, in the first one I've opened psql, and in the second one ipython with psycopg2 imported.
I'm connected to the same db in both sessions. When I update a table through ipython/psycopg2, psql session queries won't reflect the updates (i.e. I add a row in a table via psycopg2, and psql still fetches no rows).
What am I doing wrong?

Probably, after executing update you didn't execute commit() (it makes the changes to the database persistent) on the connection object.
See the first example in the docs http://initd.org/psycopg/docs/usage.html

Related

Pyodbc can create/alter tables, but I can't see them in SSMS

Pyodbc is correctly connecting to the same db. When I run
SELECT name FROM sys.databases;
SELECT name FROM master.dbo.sysdatabases;
I get the list of all the DBs I can see in MSSQLSMS.
When I look at my Event Profiler in SSMS, I can see that Pyodbc is executing code actions on the same database in the same server as I look at with SSMS. I see my create table statements, select statements, that I'm running in Python with Pyodbc, executing on my SQL server.
So why can I not see the tables I've created in SSMS? Why, when I run the same queries in SSMS, do I not see the table I've created using Pyodbc?
I am extremely confused. Pyodbc appears to be connecting to my local SQL server correctly, and executing SQL code on it, but I'm not able to view the results using SSMS. I can find the table with Pyodbc, and Pyodbc and SSMS are both telling me they're looking at the same places, but SSMS can't see anything Pyodbc has done.
EDIT : Solved
conn.autocommit=True is required for Pyodbc to make permanent changes.
SQL Server allows some DDL statements (e.g., CREATE TABLE) to be executed inside a transaction. Therefore we also have to remember to commit() those changes if we haven't specified autocommit=True on the Connection.

Python SQLAlchemy commit to two different dbs one MSSQL and other PostgreSQL

I am trying to commit to two different dbs, one hosted on MSSQL and the other PostgreSQL. I have two different session objects. I know I can do the following,
session1.add(record) // MSSQL session
session1.commit()
session2.add(record) // PostgreSQL session
session2.commit()
But, I am trying keep then in sync, so either both successful or both fail (if one of them fails don't commit to other). I would appreciate any help or thoughts.
You need to use the distributed transaction coordinator to create a distributed transaction.
There is an old saying: A man who has one watch knows what time it is, a man who has two is never sure.

How to drop table and recreate in amazon RDS with Elasticbeanstalk?

My database on Amazon currently has only a little data in it (I am making a web app but it is still in development) and I am looking to delete it, make changes to the schema, and put it back up again. The past few times I have done this, I have completely recreated my elasticbeanstalk app, but there seems like there is a better way. On my local machine, I will take the following steps:
"dropdb databasename" and then "createdb databasename"
python manage.py makemigrations
python manage.py migrate
Is there something like this that I can do on amazon to delete my database and put it back online again without deleting the entire application? When I tried just deleting the RDS instance a while ago and making a new one, I was having problems with elasticbeanstalk.
The easiest way to accomplish this is to SSH to one of your EC2 instances, that has acccess to the RDS DB, and then connect to the DB from there. Make sure that your python scripts can read your app configuration to access the configured DB, or add arguments for DB hostname. To drop and create your DB, you must just add the necessary arguments to connect to the DB. For example:
$ createdb -h <RDS endpoint> -U <user> -W ebdb
You can also create a RDS snapshot when the DB is empty, and use the RDS instance actions Restore to Point in Time or Migrate Latest Snapshot.
I had the same problem and came up with a workaround. In your python code just add and run the following method when deploying your app the next time:
FOR SQLALCHEMY AFTER VERSION 2.0
from sqlalchemy import create_engine, text
tables = ["table1_name", "table2_name"] # the names of the tables you want to delte
engine = create_engine("sqlite:///example.db") # here you create your engine
def delete_tables(tables):
for table in tables:
sql = text(f"DROP TABLE IF EXISTS {table} CASCADE;") # CASCADE deltes the tables even if they had some connections to other tables
with engine.connect() as connection:
with connection.begin():
connection.execute(sql)
delete_tables(tables) # Comment this line out after running it once.
FOR SQLALCHEMY BEFORE VERSION 2 (I guess)
def delete_tables(tables):
for table in tables:
engine.execute(f"DROP TABLE IF EXISTS {table} CASCADE;")
delete_tables(tables) # Comment this line out after running it once.
After you deployed and ran this code 1 time, all your tables will be deleted.
IMPORTANT: Delete or comment out this code after that, otherwise you will delete all your tables every time when you deploy your code to AWS

Update and deploy PostgreSQL schema to Heroku

I have a PostgreSQL schema that resides in a schema.sql file that gets run each time a database connection is initiated in Python. It looks something like:
CREATE TABLE IF NOT EXISTS users (
id SERIAL PRIMARY KEY,
facebook_id TEXT NOT NULL,
name TEXT NOT NULL,
access_token TEXT,
created TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
);
The app is deployed on Heroku, using their PostgreSQL and everything works as expected.
Now, what if I want to change a bit the structure of my users table? How can I do this the easiest and the best way? I thought of writing an ALTER... line in schema.sql for each change I want to produce in the database, but I don't think this is the best approach, since after some time the schema file will be full of ALTERs and it will slow down my app.
What's the indicated way to deploy changes made to a database?
Running a hard-coded script on each connection is not a great way to handle schema management.
You need to either manage the schema manually, or use a full-fledged tool that keeps a schema version identifier in the database, checks that, and applies a script to upgrade to the next schema version if it's different to the latest one. Rails calls this "migrations" and it kind-of works. If you're using Django it has schema management too.
If you're not using a framework like that, I suggest just writing your own schema upgrade scripts. Add a "schema_version" table with a single row. SELECT it when the app first starts after a redeploy and if it's lower than the current version the app knows about, apply the update script(s) in order, eg schema_1_to_2, schema_2_to_3, etc.
I don't recommend doing this on connect, do it on app start, or better, as a special maintenance command. If you do it on every connection you'll have multiple connections trying to make the same changes and you'll land up with duplicated columns and all sorts of other mess.
I support several django apps on heroku with Postgres. I just connect via PgAdmin and run my scripts when changes are required. I don't see any need for running a script every time a connection is made.

Problem in insertion from python script in mysql database with innondb engine

I am facing a problem where I am trying to add data from a python script to mysql database with InnonDB engine, it works fine with myisam engine of the mysql database. But the problem with the myisam engine is that it doesn't support foreign keys so I'll have to add extra code each place where I want to insert/delete records in database.
Does anyone know why InnonDB doesn't work with python scripts and possible solutions for this problem ??
InnoDB is transactional. You need to call connection.commit() after inserts/deletes/updates.
Edit: you can call connection.autocommit(True) to turn on autocommit.
Python DB API disables autocommit by default
Pasted from google (first page, 2nd result)
MySQL :: MySQL 5.0 Reference Manual :: 13.2.8 The InnoDB ...
By default, MySQL starts the session for each new connection with autocommit ...
dev.mysql.com/.../innodb-transaction-model.html
However
Apparently Python starts MySQL in NON-autocommit mode, see:
http://www.kitebird.com/articles/pydbapi.html
From the article:
The connection object commit() method commits any outstanding changes in the current transaction to make them permanent in the database. In DB-API, connections begin with autocommit mode disabled, so you must call commit() before disconnecting or changes may be lost.
Bummer, dunno how to override that and I don't want to lead you astray by guessing.
I would suggest opening a new question titled:
How to enable autocommit mode in MySQL python DB-API?
Good luck.

Categories

Resources