I'm trying to add info into a record from two different files, I'm trying to achieve this by opening the first file and adding to a record, then opening the second and updating that record.
with open('C:/Users/ELITEBOOK/documents/github/chatbot/chatbot/bot/human_text.txt', 'r') as table2, open('C:/Users/ELITEBOOK/documents/github/chatbot/chatbot/bot/robo_text.txt','r') as table3:
for line in table2.readlines():
message_text = line
#for robo_line in table3.readlines():
message_intent = ''
message_entities = ''
test = 'hello'
#cursor.execute('START TRANSACTION')
cursor.execute("INSERT INTO conversation (text) VALUES ('%s')" % line)
cnx.commit()
#cursor.execute((line))
for robo_line in table3.readlines():
#message_reply = robo_line
cursor.execute("UPDATE conversation SET reply = '%s' WHERE text = %s " % (robo_line, line))
#cursor.execute(robo_line)
cnx.commit()
I am receiving a Unknown column 'start' in 'where clause' error, "start" is just the string from the first line in my second text file. I'm using string formatters right now because otherwise I get a syntax error, this code is only being used to update the DB once, not in production.
You need to put quotes around the value, since it's a string, just like you did for the string you're setting reply to.
cursor.execute("UPDATE conversation SET reply = '%s' WHERE text = '%s' " % (robo_line, line))
But it would be better to use a prepared statement rather than string formatting, to prevent SQL injection. Then you don't put quotes around placeholders, cursor.execute replaces them safely.
cursor.execute("UPDATE conversation SET reply = %s WHERE text = %s ", (robo_line, line))
Also, your looping is wrong. You don't want to loop through the entire table3 for every line in table2, you just want to read both files in parallel. See Read two textfile line by line simultaneously -python
with open('C:/Users/ELITEBOOK/documents/github/chatbot/chatbot/bot/human_text.txt', 'r') as table2, open('C:/Users/ELITEBOOK/documents/github/chatbot/chatbot/bot/robo_text.txt','r') as table3:
for line, robo_line in zip(table2, table3):
message_text = line
message_intent = ''
message_entities = ''
test = 'hello'
#cursor.execute('START TRANSACTION')
cursor.execute("INSERT INTO conversation (text, reply) VALUES (%s, %s)", (line, robo_line))
cnx.commit()
Related
I am trying to insert data into my Postgres Database, I am able to insert some data which is something else but not the actual data
This is my data generator and data sender into my query execution function(
(Python-Falsk)
def runid():
cmdStr = 'cd /root/Dart/log; ls -rt DartRunner*.log | tail -1 '
ret = execCmdLocal(cmdStr)
logName = ret[1].strip()
runId = ""
print('The DartRunner Log generated is: %s'%logName)
with open('/root/Dart/log/' + logName, "r") as fd:
for line in fd:
if 'runId' in line:
runId = line.split()[-1]
print('Run Id: %s'%runId)
break
print (runId) # output : Run Id: 180628-22
post_runid(runId) # output is given in below link
return jsonify({"run_id": runId})
This my database(postgres) execution method:
(Python)
def post_runid(run_id):
query = "insert into runid(runid) values(%s)"
cur.execute(query %run_id)
conn.commit()
My output looks something like this:
The above two rows are manually inserted by me but the below two rows are executed from the code, the below two rows must be same as the above ones but for some reason they are not as the original data but being generated in series
Changing %s to '%s' in the query fixed the problem.
def post_runid(run_id):
query = "insert into runid(runid) values('%s')"
cur.execute(query, (run_id,))
conn.commit()
I've printed the output of my "payload" which I want to save to the MySQL database:
('B01MTOV8IP', '40462', '23.95', 'n/a', 'Usually ships in 24 hours',
'https://www.amazon.com/reviews/iframe?akid=AKIAIDCPAFSAQICDTFNQ&alinkCode=xm2&asin=B01MTOV8IP&atag=reakenture-20&exp=2017-08-25T17%3A27%3A37Z&v=2&sig=3zbBXVo4cQAJueFeVeo%252F%252FejvaUOmvuwAtfB4EfMyDiU%253D', 'CHG-GSTWL')
There seems to be something wrong with the way I am formatting it before I pass it to connect.
try:
selling_price = product.price_and_currency
selling_price_v = selling_price[0]#type
print selling_price_v
except Exception as e:
selling_price = "n/a"
conn = MySQLdb.connect(host="clabadmin.cfcudy1fdz8o.us-east-1.rds.amazonaws.com", user="", passwd="", db="")
payload =[
asin,
bsr,
str(selling_price_v),
str(listing_price_v),
# availability_type,
availability,
reviews,
sku]
print payload
# conn = sqlite3.connect('skubsr.db')
c = conn.cursor()
c.execute("""UPDATE webservice_bsr
SET
AISN = %s,
Best_Sellers_Rank = %s,
selling_price = %s,
price = %s,
# availability_type = %s,
availability = %s,
reviews = %s
WHERE ItemSKU = %s""", payload)
conn.commit()
I get the following error:
Traceback (most recent call last):
File "/home/trackstarz/clabReportScraper/bsrimport.py", line 907, in <module>
WHERE ItemSKU = %s""", payload)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 187, in execute
query = query % tuple([db.literal(item) for item in args])
TypeError: not enough arguments for format string
[Finished in 3.1s with exit code 1]
# is only used to indicate a comment when used inside Python code. In your query, it is inside the query string, and so is not parsed as a comment identifier, but as part of the query.
If you delete it, you are left with 8 %s and only 7 items inside payload.
I believe the problem is you have multiple %s string indicators in your execute string but are only giving it a single item (in this case a list) which it doesn't know it should break down into multiple values.
Try using some of the suggestions in this post to get your desired effect.
Using Python String Formatting with Lists
I scripted in Python SQL calls to my MonetDB server (which I verify is running, of course). When I print the calls instead of calling them, the commands look OK, but if I run the original script, it does not crash, it does use the CPU and memory, but nothing is changed in the database, not even the first line is executed. Why?
The Python script looks like this:
# script to merge tables in MonetDB
import re
from monetdb import mapi
server = mapi.Server()
server.connect(hostname="localhost", port=50000, username="monetdb", password="monetdb", database="dbfarm", language="sql")
def tablemerge(stub,yearlist):
for year in yearlist:
# server.cmd('ALTER TABLE %s_%d ADD COLUMN "year" INTEGER DEFAULT %d;' % (stub,year,year))
print 'ALTER TABLE %s_%d ADD COLUMN "year" INTEGER DEFAULT %d;' % (stub,year,year)
newstub = re.sub(r'sys.ds_chocker_lev_', r'', stub)
if year == yearlist[0]:
unioncall = 'CREATE TABLE %s AS SELECT * FROM %s_%d ' % (newstub,stub,year)
else:
unioncall += 'UNION ALL SELECT * FROM %s_%d ' % (stub,year)
unioncall += ';'
server.cmd(unioncall)
# print unioncall
for year in yearlist:
server.cmd('DROP TABLE %s_%d;' % (stub,year))
# print 'DROP TABLE %s_%d;' % (stub,year)
print '%s done.' % stub
for stub in ['civandr']:
tablemerge('sys.ds_chocker_lev_%s' % stub,xrange(1998,2013))
E.g. the first call would be:
ALTER TABLE sys.ds_chocker_lev_civandr_1998 ADD COLUMN "year" INTEGER DEFAULT 1998;
But not even this happens. There is no year column in the table.
Or could I run the script in the console with more output than what I print myself?
Do commit! By default, the autocommit parameter is set to False. You can either do:
server.connect(hostname="localhost", port=50000, username="monetdb", password="monetdb", database="dbfarm", language="sql", autocommit=True)
or simply run:
connection.commit()
connection = monetdb.sql.connect(username=username,password=password,hostname=hostname,port=port,database=database)
cursor = connection.cursor()
cursor.execute('create table test (id int, name varchar(50));')
connection.commit()
I am working on a learning how to execute SQL in python (I know SQL, not Python).
I have an external sql file. It creates and inserts data into three tables 'Zookeeper', 'Handles', 'Animal'.
Then I have a series of queries to run off the tables. The below queries are in the zookeeper.sql file that I load in at the top of the python script. Example for the first two are:
--1.1
SELECT ANAME,zookeepid
FROM ANIMAL, HANDLES
WHERE AID=ANIMALID;
--1.2
SELECT ZNAME, SUM(TIMETOFEED)
FROM ZOOKEEPER, ANIMAL, HANDLES
WHERE AID=ANIMALID AND ZOOKEEPID=ZID
GROUP BY zookeeper.zname;
These all execute fine in SQL. Now I need to execute them from within Python. I have been given and completed code to read in the file. Then execute all the queries in the loop.
The 1.1 and 1.2 is where I am getting confused. I believe in the loop this is the line where I should put in something to run the first and then second query.
result = c.execute("SELECT * FROM %s;" % table);
but what? I think I am missing something very obvious. I think what is throwing me off is % table. In query 1.1 and 1.2, I am not creating a table, but rather looking for a query result.
My entire python code is below.
import sqlite3
from sqlite3 import OperationalError
conn = sqlite3.connect('csc455_HW3.db')
c = conn.cursor()
# Open and read the file as a single buffer
fd = open('ZooDatabase.sql', 'r')
sqlFile = fd.read()
fd.close()
# all SQL commands (split on ';')
sqlCommands = sqlFile.split(';')
# Execute every command from the input file
for command in sqlCommands:
# This will skip and report errors
# For example, if the tables do not yet exist, this will skip over
# the DROP TABLE commands
try:
c.execute(command)
except OperationalError, msg:
print "Command skipped: ", msg
# For each of the 3 tables, query the database and print the contents
for table in ['ZooKeeper', 'Animal', 'Handles']:
**# Plug in the name of the table into SELECT * query
result = c.execute("SELECT * FROM %s;" % table);**
# Get all rows.
rows = result.fetchall();
# \n represents an end-of-line
print "\n--- TABLE ", table, "\n"
# This will print the name of the columns, padding each name up
# to 22 characters. Note that comma at the end prevents new lines
for desc in result.description:
print desc[0].rjust(22, ' '),
# End the line with column names
print ""
for row in rows:
for value in row:
# Print each value, padding it up with ' ' to 22 characters on the right
print str(value).rjust(22, ' '),
# End the values from the row
print ""
c.close()
conn.close()
Your code already contains a beautiful way to execute all statements from a specified sql file
# Open and read the file as a single buffer
fd = open('ZooDatabase.sql', 'r')
sqlFile = fd.read()
fd.close()
# all SQL commands (split on ';')
sqlCommands = sqlFile.split(';')
# Execute every command from the input file
for command in sqlCommands:
# This will skip and report errors
# For example, if the tables do not yet exist, this will skip over
# the DROP TABLE commands
try:
c.execute(command)
except OperationalError, msg:
print("Command skipped: ", msg)
Wrap this in a function and you can reuse it.
def executeScriptsFromFile(filename):
# Open and read the file as a single buffer
fd = open(filename, 'r')
sqlFile = fd.read()
fd.close()
# all SQL commands (split on ';')
sqlCommands = sqlFile.split(';')
# Execute every command from the input file
for command in sqlCommands:
# This will skip and report errors
# For example, if the tables do not yet exist, this will skip over
# the DROP TABLE commands
try:
c.execute(command)
except OperationalError, msg:
print("Command skipped: ", msg)
To use it
executeScriptsFromFile('zookeeper.sql')
You said you were confused by
result = c.execute("SELECT * FROM %s;" % table);
In Python, you can add stuff to a string by using something called string formatting.
You have a string "Some string with %s" with %s, that's a placeholder for something else. To replace the placeholder, you add % ("what you want to replace it with") after your string
ex:
a = "Hi, my name is %s and I have a %s hat" % ("Azeirah", "cool")
print(a)
>>> Hi, my name is Azeirah and I have a Cool hat
Bit of a childish example, but it should be clear.
Now, what
result = c.execute("SELECT * FROM %s;" % table);
means, is it replaces %s with the value of the table variable.
(created in)
for table in ['ZooKeeper', 'Animal', 'Handles']:
# for loop example
for fruit in ["apple", "pear", "orange"]:
print(fruit)
>>> apple
>>> pear
>>> orange
If you have any additional questions, poke me.
A very simple way to read an external script into an sqlite database in python is using executescript():
import sqlite3
conn = sqlite3.connect('csc455_HW3.db')
with open('ZooDatabase.sql', 'r') as sql_file:
conn.executescript(sql_file.read())
conn.close()
First make sure that a table exists if not, create a table then follow the steps.
import sqlite3
from sqlite3 import OperationalError
conn = sqlite3.connect('Client_DB.db')
c = conn.cursor()
def execute_sqlfile(filename):
c.execute("CREATE TABLE clients_parameters (adress text, ie text)")
#
fd = open(filename, 'r')
sqlFile = fd.readlines()
fd.close()
lvalues = [tuple(v.split(';')) for v in sqlFile[1:] ]
try:
#print(command)
c.executemany("INSERT INTO clients_parameters VALUES (?, ?)", lvalues)
except OperationalError as msg:
print ("Command skipped: ", msg)
execute_sqlfile('clients.sql')
print(c.rowcount)
according me, it is not possible
solution:
import .sql file on mysql server
after
import mysql.connector
import pandas as pd
and then you use .sql file by convert to dataframe
I get the error when running this code:
import sqlite3
user_name = raw_input("Please enter the name: ")
user_email = raw_input("Please enter the email: ")
db = sqlite3.connect("customer")
cursor=db.cursor()
sql = """INSERT INTO customer
(name, email) VALUES (?,?);,
(user_name, user_email)"""
cursor.execute(sql)
Why is this happening?
While the other posters are correct about your statement formatting you are receiving this particular error because you are attempting to perform multiple statements in one query (notice the ; in your query which separates statements).
From Python sqlite3 docs:
"execute() will only execute a single SQL statement. If you try to execute more than one
statement with it, it will raise a Warning. Use executescript() if you want to execute
multiple SQL statements with one call."
https://docs.python.org/2/library/sqlite3.html
Now your statement will not execute properly even if you use executescript() because there are other issues with the way it is formatted (see other posted answers). But the error you are receiving is specifically because of your multiple statements. I am posting this answer for others that may have wandered here after searching for that error.
Use executescript instead of execute
execute() will only execute a single SQL statement. If you try to execute more than one statement with it, it will raise a Warning. Use executescript() if you want to execute multiple SQL statements with one call.
https://docs.python.org/2/library/sqlite3.html#sqlite3.Cursor.execute
You have a ;, in the middle of the query string - that is an invalid syntax. Pass a dictionary as a second argument to execute if you want to use a named parameter binding.
sql = "INSERT INTO customer (name, email) VALUES (:name, :email)"
cursor.execute(sql, {'name':user_name, 'email':user_email})
Try this:
sql = """INSERT INTO customer
(name, email) VALUES (?,?)"""
cursor.execute(sql, (user_name, user_email))
import sqlite3
def DB():
List = {"Name":"Omar", "Age":"33"}
columns = ', '.join("" + str(x).replace('/', '_') + "" for x in List.keys())
values = ', '.join("'" + str(x).replace('/', '_') + "'" for x in List.values())
sql_qry = "INSERT INTO %s ( %s ) values (?,?) ; ( %s )" % ('Table Name', columns, values)
conn = sqlite3.connect("DBname.db")
curr = conn.cursor()
# curr.execute("""create table if not exists TestTable(
# Name text,
# Age text
# )""")
# print columns
# print values
# print sql
# sql = 'INSERT INTO yell (Name , Age) values (%s, %s)'
curr.execute(sql_qry)
DB()