I'm planning to insert data to bellow CF that has compound keys.
CREATE TABLE event_attend (
event_id int,
event_type varchar,
event_user_id int,
PRIMARY KEY (event_id, event_type) #compound keys...
);
But I can't insert data to this CF from python using cql.
(http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/)
import cql
connection = cql.connect(host, port, keyspace)
cursor = connection.cursor()
cursor.execute("INSERT INTO event_attend (event_id, event_type, event_user_id) VALUES (1, 'test', 2)", dict({}) )
I get the following traceback:
Traceback (most recent call last):
File "./v2_initial.py", line 153, in <module>
db2cass.execute()
File "./v2_initial.py", line 134, in execute
cscursor.execute("insert into event_attend (event_id, event_type, event_user_id ) values (1, 'test', 2)", dict({}))
File "/usr/local/pythonbrew/pythons/Python-2.7.2/lib/python2.7/site-packages/cql-1.4.0-py2.7.egg/cql/cursor.py", line 80, in execute
response = self.get_response(prepared_q, cl)
File "/usr/local/pythonbrew/pythons/Python-2.7.2/lib/python2.7/site-packages/cql-1.4.0-py2.7.egg/cql/thrifteries.py", line 80, in get_response
return self.handle_cql_execution_errors(doquery, compressed_q, compress)
File "/usr/local/pythonbrew/pythons/Python-2.7.2/lib/python2.7/site-packages/cql-1.4.0-py2.7.egg/cql/thrifteries.py", line 98, in handle_cql_execution_errors
raise cql.ProgrammingError("Bad Request: %s" % ire.why)
cql.apivalues.ProgrammingError: Bad Request: unable to make int from 'event_user_id'
What am I doing wrong?
It looks like you are trying to follow the example in:
http://pypi.python.org/pypi/cql/1.4.0
import cql
con = cql.connect(host, port, keyspace)
cursor = con.cursor()
cursor.execute("CQL QUERY", dict(kw='Foo', kw2='Bar', kwn='etc...'))
However, if you only need to insert one row (like in your question), just drop the empty dict() parameter.
Also, since you are using composite keys, make sure you use CQL3
http://www.datastax.com/dev/blog/whats-new-in-cql-3-0
connection = cql.connect('localhost:9160', cql_version='3.0.0')
The following code should work (just adapt it to localhost if needed):
import cql
con = cql.connect('172.24.24.24', 9160, keyspace, cql_version='3.0.0')
print ("Connected!")
cursor = con.cursor()
CQLString = "INSERT INTO event_attend (event_id, event_type, event_user_id) VALUES (131, 'Party', 3156);"
cursor.execute(CQLString)
For python 2.7, 3.3, 3.4, 3.5, and 3.6 for installation you can use
$ pip install cassandra-driver
And in python:
import cassandra
Documentation can be found under https://datastax.github.io/python-driver/getting_started.html#passing-parameters-to-cql-queries
Related
I have been trying to extracting a sql table using cx_oracle and saving it as pandas dataframe using the following script:
import cx_Oracle
import pandas as pd
id = 1234
connection = cx_Oracle.connect(user="user", password='pwd',dsn="dsn")
# Obtain a cursor
cursor = connection.cursor()
# Execute the query
query = """select * from table where id= {id}"""
my_sql =cursor.execute(query.format(id=id))
df_sql = pd.read_sql(my_sql, connection)
I am able to connect to the database but I am unable to save it as pandas dataframe. How do I do that? I get the following error :
File "file/to/path.py", line 38, in file
df_sql = pd.read_sql(my_sql, connection)
File "C:\file/to/path\venv\lib\site-packages\pandas\io\sql.py", line 495, in read_sql
return pandas_sql.read_query(
File "File/to/path\venv\lib\site-packages\pandas\io\sql.py", line 1771, in read_query
cursor = self.execute(*args)
File "File/to/path\venv\lib\site-packages\pandas\io\sql.py", line 1737, in execute
raise ex from exc
pandas.io.sql.DatabaseError: Execution failed on sql '<cx_Oracle.Cursor on <cx_Oracle.Connection to dsn>>': expecting string or bytes object
The first argument to the pd.read_sql should be the query (if I'm not mistaken). You are parsing a cursor object. Try replace my_sql in pd.read_sql with query i.e
pd.read_sql(query.format(id=id))
or use the cursor object i.e
df = pd.DataFrame(my_sql.fetchall())
Note, fetchall() does only return the data i.e not the header, which can be obtained using cursor.description (see the SO answer here )
Thank you for reading. I have some experience with SQL, very new to python.
In the below code, i am accessing 2 databases in python 2.7
The connections work. I can query a tables that has a serial #s for devices in one statement with no issue. I then want to query a table which name matches that serial number in another database, pulling the latest value of the "Stamp" field. All of this works when i explictly name the table ccnbsc00000001, but when using variable subsitution, it fails.
When the variable currentdevice is substituted, extras characters are included. When i print that variable, those character are not present in that output. here is the code, and the error result at the bottom
#!/usr/bin/python
### Imports
import datetime
import mysql.connector
#Connect to heartbeat results database
hb_db = mysql.connector.connect(
host="localhost",
user="otheruser",
passwd="******",
database="active_devices"
)
#Connect to heartbeat results database
device_Settings_db = mysql.connector.connect(
host="localhost",
user="otheruser",
passwd="******",
database="active_devices"
)
device_settings_cursor = device_settings_db.cursor()
hb_cursor = hb_db.cursor()
## Get deviuce serial#
device_settings_cursor.execute('select device_serial from devices')
active_devices = device_settings_cursor.fetchall()
print ("these are the current devices:")
print (active_devices)
for device in active_devices:
currentdevice = device[0]
print(currentdevice)
print ("SELECT MAX(stamp) FROM (%s)" , (currentdevice,) )
hb_cursor.execute('SELECT MAX(stamp) FROM (%s)' , (currentdevice,) )
laststamp = hb_cursor.fetchone
laststamp = laststamp[0]
print("Last time stamp is:")
print(laststamp)
*
Output of print(active_devices)
[(u'ccnbsc00000001',), (u'ccnbsc00000002',)]
output of print(currentdevice)
ccnbsc00000001
(This is the correct output/value)
but I get this error in the SQL query that implies it has kept the surrounding characters ' and ')
Traceback (most recent call last):
File "./hb_notify.py", line 61, in <module>
hb_cursor.execute('SELECT MAX(stamp) FROM (%s)' , (currentccn,) )
File "/usr/lib/python2.7/site-packages/mysql/connector/cursor.py", line 551, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 490, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 395, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your **SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''ccnbsc00000001')' at line 1**
Python MySQL libraries commonly insert quotation marks when you pass string arguments to them as arguments, because usually you do actually want those quotation marks. This is why you're seeing quotation marks.
The fix here is easy: instead of passing those values as arguments to your cursor, you can just insert those values directly into the string like you would if it were any other Python string. Like so:
hb_cursor.execute('SELECT MAX(stamp) FROM {0}'.format(currentdevice))
Python string arguments will remove quotes around a string, MySQL cursor arguments will keep the quotes.
I am trying to take a data from a log file in cvs format, open the log file and inserting row by row into mysql. I am getting an error like this:
ERROR Traceback (most recent call last): File "/Users/alex/PycharmProjects/PA_REPORTING/padb_populate.py", line 26, in VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)', row) File "/Users/alex/anaconda/lib/python2.7/site-packages/MySQLdb/cursors.py", line 187, in execute query = query % tuple([db.literal(item) for item in args]) TypeError: not all arguments converted during string formatting.
import csv
import MySQLdb
mydb = MySQLdb.connect(host='192.168.56.103',
user='user',
passwd='pass',
db='palogdb')
cursor = mydb.cursor()
csv_data = csv.reader(file('/tmp/PALOG_DEMODATA-100.csv'))
for row in csv_data:
cursor.execute('INSERT INTO palogdb(RECEIVE_TIME,SERIAL,TYPE,SUBTYPE,COL1,TIME_GENERATED,SRC,DST,NATSRC,NATDST,RULE,\
SRCUSR,DSTUSR,APP,VSYS1,FROM,TO,INBOUND_IF,OUTBOUND_IF,LOGSET,COL2,SESSIONID,COL3,REPEATCNT,SOURCEPORT,NATSPORT,NATDPORT, \
FLAGS,PROTO,ACTION,BYTES,BYTES_SENT,BYTES_RECEIVED,PACKETS,START,ELAPSED,CATEGORY,COL4,SEQNO,ACTIONFLAGS,SRCLOC,DSTLOC,NONE, \
PKTS_SENT,PKTS_RECEIVED,SESSION_END_REASON) \
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)', row)
#close the connection to the database.
mydb.commit()
cursor.close()
Is it possible, that you don't have enough data in row for all your %s's? Maybe your row is interpreted as one value, and thus only the first %s is expanded? Try *row to expand the vector to values.
To debug, you could try to build the string passed to execute by some other method, e.g.
sql_string = 'INSERT ... VALUES ({}, {}, {})'.format(*row)
and print it. If you get such an error, you can check, whether the generated string looks reasonable...
I'm trying to write a script using pythong and the mysql-connector library. The script should connect to the mysql server do a "SHOW DATABASES LIKE 'pdns_%' and then using the results returned by the query use each database and then run another query while using that database.
Here is the code
import datetime
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect (user='user', password='thepassword',
host='mysql.server.com',buffered=True)
cursor = cnx.cursor()
query = ("show databases like 'pdns_%'")
cursor.execute(query)
databases = query
for (databases) in cursor:
cursor.execute("USE %s",(databases[0],))
hitcounts = ("SELECT Monthname(hitdatetime) AS 'Month', Count(hitdatetime) AS 'Hits' WHERE hitdatetime >= Date_add(Last_day(Date_sub(Curdate(), interval 4 month)), interval 1 day) AND hitdatetime < Date_add(Last_day(Date_sub(Curdate(), interval 1 month)), interval 1 day) GROUP BY Monthname(hitdatetime) ORDER BY Month(hitdatetime)")
cursor.execute(hitcounts)
print(hitcounts)
cursor.close()
cnx.close()
When running the script it stops with the following error'd output
Traceback (most recent call last):
File "./mysql-test.py", line 18, in <module>
cursor.execute("USE %s",(databases[0],))
File "/usr/lib/python2.6/site-packages/mysql/connector/cursor.py", line 491, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 635, in cmd_query
statement))
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 553, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''pdns_382'' at line 1
Based on the error I'm guessing there is an issue with how its doing the datbase name from the first query. Any pointers in the correct direction would be very helpful as I'm very much a beginner. Thank you very much.
Alas, the two-args form of execute does not support "meta" parameters, such as names of databases, tables, or fields (roughly, think of identifiers you wouldn't quote if writing the query out manually). So, the failing statement:
cursor.execute("USE %s",(databases[0],))
needs to be re-coded as:
cursor.execute("USE %s" % (databases[0],))
i.e, the single arg form of execute, with a string interpolation. Fortunately, this particular case does not expose you to SQL injection risks, since you're only interpolating DB names coming right from the DB engine.
I have prepared a stored procedure that runs fine if I make the call from a MySQL console. But when running in python with mysql.connector controller, performs the insertion procedure correctly.
However, the result does not bring with fectchall() because the following error:
File "/home/sis1/prueba/prueba.py", line 16, in <module>
reg=conn.fetchall()
File "/usr/lib/pymodules/python2.7/mysql/connector/cursor.py", line 551, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
InterfaceError: No result set to fetch from.`
Here's the stored procedure:
DROP PROCEDURE IF EXISTS pr_prueba;
CREATE DEFINER = rooter#localhost PROCEDURE pr_prueba(IN p_emp tinyint,OUT mensaje varchar(50),OUT registros integer)
BEGIN
DECLARE numreg INT (10);
DECLARE tabla VARCHAR (30);
DECLARE emp TINYINT(2);
SET #tabla = CONCAT("emp",p_emp,".usuario");
SET #emp = CAST(p_emp AS UNSIGNED);
SET #sql_text = CONCAT("INSERT INTO ",#tabla," ( name, lastname ) (SELECT UPPER(name), UPPER(lastname) FROM tablas GROUP BY tablas.operador);");
PREPARE stmt FROM #sql_text;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #mensaje="OK";
SET #sql_text = CONCAT("SELECT COUNT(*) INTO #numreg FROM ",#tabla,";");
PREPARE stmt FROM #sql_text;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #registros=#numreg;
SELECT #mensaje as mensaje, #registros as registros;
END ;
Here's python code:
import sys
import mysql.connector
if (__name__=='__main__'):
db = mysql.connector.connect(host="192.168.1.1",user="de",passwd="de2",database="dbim" )
conn = db.cursor()
args=(1,"",0)
conn.callproc("pr_prueba",args)
reg=conn.fetchall()
try:
db.commit()
except:
db.rollback()
print "error"
conn.close()
db.close()
I found the problem myself. I had to change the line:
reg=conn.fetchall()
for this:
for reg in conn.next_proc_resultset():
pass
I do not know if it is the best solution but it works