I have an issue to run my SQL queries on a Postgres ElephantSql hosted:
This is my code to connect (except dynamo, user, password which are replaced by XXX
DATABASE_URL = 'postgres://YYYY:ZZZZ#drona.db.elephantsql.com:5432/YYYY'
# ---------------------------- CONNECT ELEPHANT DB
def ElephantConnect():
up.uses_netloc.append("postgres")
url = up.urlparse(DATABASE_URL)
conn = psycopg2.connect(dbname='YYYY',
user='YYYY',
password='ZZZZ',
host='drona.db.elephantsql.com',
port='5432'
)
cursor = conn.cursor()
# cursor.execute("CREATE TABLE notes(id integer primary key, body text, title text);")
#conn.commit()
# conn.close()
return conn
this code seems to connect well to db
My issue is when I want to delete a table:
def update(df, table_name, deleteYes= 'Yes'):
conn = ElephantConnect()
db = create_engine(DATABASE_URL)
cursor =conn.cursor()
if deleteYes == 'Yes': # delete
queryCount = "SELECT count(*) FROM {};".format(table_name)
queryDelete = "DELETE FROM {};".format(table_name)
count = db.execute(queryCount)
rows_before = count.fetchone()[0]
try:
db.execute(queryDelete)
logging.info('Deleted {} rows into table {}'.format(rows_before, table_name))
except:
logging.info('Deleted error into table {}'.format(table_name))
else:
pass
It seems when I run db.execute(queryDelete), it goes to the exception.
I have no message of error. But the query with count data is working...
thanks
I think that the reason for the error is because there are foreign keys against the table. In order to be sure, assign the exception into a variable and print it:
except Exception as ex:
print(ex)
By the way, if you want to quickly delete all of the rows from a table then
It will be much more efficient to truncate the table instead of deleting all the rows:
truncate table table_name
Delete is more useful when you want to delete rows under some conditions:
delete from table_name where ...
Related
i wrote i python code that requsts an api and collect data existed on this api in a dataframe named df2
i created a mysql database named speed_data
the code bellow allowed me to add the data existed in my dataframe into my SQLdatabase :
try:
conn = msql.connect(host='localhost',database='speed_data' ,user='root',
password='')
if conn.is_connected():
cursor = conn.cursor()
cursor.execute("select database();")
record = cursor.fetchone()
print("You're connected to database: ", record)
cursor.execute("DROP TABLE IF EXISTS speed_cord;")
print('creating table.....')
cursor.execute("CREATE TABLE speed_cord(id varchar(20),vitesse_med varchar(20),vitesse_max varchar(20),distance varchar(20),temps timestamp,PRIMARY KEY(id)) ")
print("table is created....")
for i,row in df2.iterrows():
sql = "INSERT INTO speed_data.speed_cord VALUES (%s,%s,%s,%s,%s)"
cursor.execute(sql, tuple(row))
print("Record inserted")
# the connection is not auto committed by default, so we must commit to save our changes
conn.commit()
except Error as e:
print("Error while connecting to MySQL", e)
now when i request the api evry time it returns new rows in my datafram df2 and i need to add only these new rows to my table speed_cord without dropping the first data
how can i do that?
I'm aware that the best way to prevent sql injection is to write Python queries of this form (or similar):
query = 'SELECT %s %s from TABLE'
fields = ['ID', 'NAME']
cur.execute(query, fields)
The above will work for a single query, but what if we want to do a UNION of 2 SQL commands? I've set this up via sqlite3 for sake of repeatability, though technically I'm using pymysql. Looks as follows:
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT, VALUE varchar(255))'
query2 = 'CREATE TABLE DUMMy2(ID int AUTO INCREMENT, VALUE varchar(255)'
try:
cur.execute(query)
cur.execute(query2)
except:
print('Already made table!')
tnames = ['DUMMY1', 'DUMMY2']
sqlcmds = []
for i in range(0,2):
query = 'SELECT %s FROM {}'.format(tnames[i])
sqlcmds.append(query)
fields = ['VALUE', 'VALUE']
sqlcmd = ' UNION '.join(sqlcmds)
cur.execute(sqlcmd, valid_fields)
When I run this, I get a sqlite Operational Error:
sqlite3.OperationalError: near "%": syntax error
I've validated the query prints as expected with this output:
INSERT INTO DUMMY VALUES(%s) UNION INSERT INTO DUMMY VALUES(%s)
All looks good there. What is the issue with the string substitutions here? I can confirm that running a query with direct string substitution works fine. I've tried it with both selects and inserts.
EDIT: I'm aware there are multiple ways to do this with executemany and a few other. I need to do this with UNION for the purposes I'm using this for because this is a very, very simplified example fo the operational code I'm using
The code below executes few INSERTS at once
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT NOT NULL, VALUE varchar(255))'
try:
cur.execute(query)
except:
print('Already made table!')
valid_fields = [('ya dummy',), ('stupid test example',)]
cur.executemany('INSERT INTO DUMMY (VALUE) VALUES (?)',valid_fields)
firstly apologies for the basic question, just starting off with Python.
I have the following code:
import sqlite3
conn = sqlite3.connect("test.sqb")
cursor = conn.cursor()
sql = "SELECT * FROM report WHERE type LIKE 'C%'"
cursor.execute(sql)
data = cursor.fetchall()
for row in data:
print (row[0])
cursor.execute("UPDATE report SET route='ABCDE'")
conn.commit()
conn.close()
Why is it updating all records and not just the filtered records from sql query, even though the print (row[0]) just shows the filtered records.
Many thanks.
What's actually happening is you are running this query for each record returned from the SELECT query.
UPDATE report SET route='ABCDE'
If you only want to update route where type starts with C add the criteria to the UPDATE query and execute it once.
import sqlite3
conn = sqlite3.connect("test.sqb")
cursor = conn.cursor()
sql = "SELECT * FROM report WHERE type LIKE 'C%'"
cursor.execute(sql)
data = cursor.fetchall()
cursor.execute("UPDATE report SET route='ABCDE' WHERE type LIKE 'C%'")
conn.commit()
conn.close()
I need to use multiple connections in my python test code. But the problem I'm facing is that the second connection does not see statements executed in the first one. As far as I understand autocommit should be ON by default.
Here is the code
import testing.postgresql
from sqlalchemy import create_engine
def test_simple():
with testing.postgresql.Postgresql() as postgresql:
try:
engine = create_engine(postgresql.url())
conn = engine.connect().connection
with conn.cursor() as cur:
cur.execute("""CREATE TABLE country (id integer, name text);
INSERT INTO country(id, name) VALUES (1, 'Mali');
INSERT INTO country(id, name) VALUES (2, 'Congo');
""")
# OK
cur.execute('select * from country')
countries = cur.fetchall()
print(str(countries))
# ERROR psycopg2.ProgrammingError: relation "country" does not exist
conn1 = engine.connect().connection
with conn1.cursor() as cur1:
cur1.execute('select * from country')
countries1 = cur1.fetchall()
print(str(countries1))
finally:
conn.close()
conn1.close()
How can I use multiple connections in my test?
Okay here is my code. Hopefully you can help me. I am using the MySQL lib called MySQLdb.
def createNick(self, user, nick):
try: # TRY STATEMENT HERE SO THE NICK CAN BE RECREATED
db = m.connect("host", "user", "password", "database")
cur = db.cursor()
cur.execute("CREATE TABLE nick_%s(name TEXT NOT NULL)" % user.lower())
cur.execute('INSERT INTO nick_%s(name) VALUES("%s")' % (user.lower(), nick))
db.commit()
except:
db = m.connect("host", "user", "password", "database")
cur = db.cursor()
cur.execute("DROP TABLE nick_%s" % user.lower())
cur.execute("CREATE TABLE nick_%s(name TEXT NOT NULL)" % user.lower())
cur.execute('INSERT INTO nick_%s(name) VALUES("%s")' % (user.lower(), nick))
db.commit()
def getNick(user):
db = m.connect("host", "user", "password", "database")
cur = db.cursor()
cur.execute("SELECT * FROM nick_%s" % user.lower())
nick = [nick[0] for nick in cur.fetchall()]
try: # TRY STATEMENT HERE JUST INCASE USER DID NOT MAKE ONE
return nick
except:
return user
self.createNick("username","<font color='#FFFF'>nickname</font>")
print self.getNick("username")
output: <font color=#FFF>nickname</font>
My problem is, every time I call the function it won't phrase the HTML correctly. I tried everything, can you help?
I wouldn't do that, I would use a specific data type for XML so that I save HTML to a datatype of type XML. Or I would not save HTML and insert the markup with a controller, or make my own midddleware language since it is usually not adviseable to save HTML in a database table.