I need to use multiple connections in my python test code. But the problem I'm facing is that the second connection does not see statements executed in the first one. As far as I understand autocommit should be ON by default.
Here is the code
import testing.postgresql
from sqlalchemy import create_engine
def test_simple():
with testing.postgresql.Postgresql() as postgresql:
try:
engine = create_engine(postgresql.url())
conn = engine.connect().connection
with conn.cursor() as cur:
cur.execute("""CREATE TABLE country (id integer, name text);
INSERT INTO country(id, name) VALUES (1, 'Mali');
INSERT INTO country(id, name) VALUES (2, 'Congo');
""")
# OK
cur.execute('select * from country')
countries = cur.fetchall()
print(str(countries))
# ERROR psycopg2.ProgrammingError: relation "country" does not exist
conn1 = engine.connect().connection
with conn1.cursor() as cur1:
cur1.execute('select * from country')
countries1 = cur1.fetchall()
print(str(countries1))
finally:
conn.close()
conn1.close()
How can I use multiple connections in my test?
Related
I'm aware that the best way to prevent sql injection is to write Python queries of this form (or similar):
query = 'SELECT %s %s from TABLE'
fields = ['ID', 'NAME']
cur.execute(query, fields)
The above will work for a single query, but what if we want to do a UNION of 2 SQL commands? I've set this up via sqlite3 for sake of repeatability, though technically I'm using pymysql. Looks as follows:
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT, VALUE varchar(255))'
query2 = 'CREATE TABLE DUMMy2(ID int AUTO INCREMENT, VALUE varchar(255)'
try:
cur.execute(query)
cur.execute(query2)
except:
print('Already made table!')
tnames = ['DUMMY1', 'DUMMY2']
sqlcmds = []
for i in range(0,2):
query = 'SELECT %s FROM {}'.format(tnames[i])
sqlcmds.append(query)
fields = ['VALUE', 'VALUE']
sqlcmd = ' UNION '.join(sqlcmds)
cur.execute(sqlcmd, valid_fields)
When I run this, I get a sqlite Operational Error:
sqlite3.OperationalError: near "%": syntax error
I've validated the query prints as expected with this output:
INSERT INTO DUMMY VALUES(%s) UNION INSERT INTO DUMMY VALUES(%s)
All looks good there. What is the issue with the string substitutions here? I can confirm that running a query with direct string substitution works fine. I've tried it with both selects and inserts.
EDIT: I'm aware there are multiple ways to do this with executemany and a few other. I need to do this with UNION for the purposes I'm using this for because this is a very, very simplified example fo the operational code I'm using
The code below executes few INSERTS at once
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT NOT NULL, VALUE varchar(255))'
try:
cur.execute(query)
except:
print('Already made table!')
valid_fields = [('ya dummy',), ('stupid test example',)]
cur.executemany('INSERT INTO DUMMY (VALUE) VALUES (?)',valid_fields)
firstly apologies for the basic question, just starting off with Python.
I have the following code:
import sqlite3
conn = sqlite3.connect("test.sqb")
cursor = conn.cursor()
sql = "SELECT * FROM report WHERE type LIKE 'C%'"
cursor.execute(sql)
data = cursor.fetchall()
for row in data:
print (row[0])
cursor.execute("UPDATE report SET route='ABCDE'")
conn.commit()
conn.close()
Why is it updating all records and not just the filtered records from sql query, even though the print (row[0]) just shows the filtered records.
Many thanks.
What's actually happening is you are running this query for each record returned from the SELECT query.
UPDATE report SET route='ABCDE'
If you only want to update route where type starts with C add the criteria to the UPDATE query and execute it once.
import sqlite3
conn = sqlite3.connect("test.sqb")
cursor = conn.cursor()
sql = "SELECT * FROM report WHERE type LIKE 'C%'"
cursor.execute(sql)
data = cursor.fetchall()
cursor.execute("UPDATE report SET route='ABCDE' WHERE type LIKE 'C%'")
conn.commit()
conn.close()
I have an issue to run my SQL queries on a Postgres ElephantSql hosted:
This is my code to connect (except dynamo, user, password which are replaced by XXX
DATABASE_URL = 'postgres://YYYY:ZZZZ#drona.db.elephantsql.com:5432/YYYY'
# ---------------------------- CONNECT ELEPHANT DB
def ElephantConnect():
up.uses_netloc.append("postgres")
url = up.urlparse(DATABASE_URL)
conn = psycopg2.connect(dbname='YYYY',
user='YYYY',
password='ZZZZ',
host='drona.db.elephantsql.com',
port='5432'
)
cursor = conn.cursor()
# cursor.execute("CREATE TABLE notes(id integer primary key, body text, title text);")
#conn.commit()
# conn.close()
return conn
this code seems to connect well to db
My issue is when I want to delete a table:
def update(df, table_name, deleteYes= 'Yes'):
conn = ElephantConnect()
db = create_engine(DATABASE_URL)
cursor =conn.cursor()
if deleteYes == 'Yes': # delete
queryCount = "SELECT count(*) FROM {};".format(table_name)
queryDelete = "DELETE FROM {};".format(table_name)
count = db.execute(queryCount)
rows_before = count.fetchone()[0]
try:
db.execute(queryDelete)
logging.info('Deleted {} rows into table {}'.format(rows_before, table_name))
except:
logging.info('Deleted error into table {}'.format(table_name))
else:
pass
It seems when I run db.execute(queryDelete), it goes to the exception.
I have no message of error. But the query with count data is working...
thanks
I think that the reason for the error is because there are foreign keys against the table. In order to be sure, assign the exception into a variable and print it:
except Exception as ex:
print(ex)
By the way, if you want to quickly delete all of the rows from a table then
It will be much more efficient to truncate the table instead of deleting all the rows:
truncate table table_name
Delete is more useful when you want to delete rows under some conditions:
delete from table_name where ...
I have this code
import MySQLdb
db = MySQLdb.connect("localhost", "root", "password", "db_name")
cursor = db.cursor()
cursor.execute("INSERT INTO directedEdges (`originLbl`, `targetLbl`) VALUES
('user1#enron.com', 'user2#enron.com' )")
data = cursor.fetchone()
print data
but when I execute this script the output is None and and I can't insert the values in the db. Why ?
In a first moment I thought it was a problem whit db connection, but if I execute
import MySQLdb
db = MySQLdb.connect("localhost", "root", "password", "db_name")
cursor = db.cursor()
cursor.execute("SELECT * FROM directedEdges")
data = cursor.fetchone()
print data
I see the content of the table directedEdges.
Thanks
You issued the cursor.fetchone() command immediately after inserting into the database. You don't have any queried data like that. You need to have queried some data before using fetchone(). Try this:
import MySQLdb
db = MySQLdb.connect("localhost", "root", "password", "db_name")
cursor = db.cursor()
cursor.execute("INSERT INTO directedEdges (`originLbl`, `targetLbl`) VALUES
('user1#enron.com', 'user2#enron.com' )")
# Commit your insert
db.commit()
# Query for data
cursor.execute("SELECT * FROM directedEdges")
data = cursor.fetchone()
print data
I have a MySQL Table named TBLTEST with two columns ID and qSQL. Each qSQL has SQL queries in it.
I have another table FACTRESTTBL.
There are 10 rows in the table TBLTEST.
For example, On TBLTEST lets take id =4 and qSQL ="select id, city, state from ABC".
How can I insert into the FACTRESTTBL from TBLTEST using python, may be using dictionary?
Thx!
You can use MySQLdb for Python.
Sample code (you'll need to debug it as I have no way of running it here):
#!/usr/bin/python
import MySQLdb
# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
# Select qSQL with id=4.
cursor.execute("SELECT qSQL FROM TBLTEST WHERE id = 4")
# Fetch a single row using fetchone() method.
results = cursor.fetchone()
qSQL = results[0]
cursor.execute(qSQL)
# Fetch all the rows in a list of lists.
qSQLresults = cursor.fetchall()
for row in qSQLresults:
id = row[0]
city = row[1]
#SQL query to INSERT a record into the table FACTRESTTBL.
cursor.execute('''INSERT into FACTRESTTBL (id, city)
values (%s, %s)''',
(id, city))
# Commit your changes in the database
db.commit()
# disconnect from server
db.close()