When using django shell: python manage.py shell
import datetime
from django.utils import timezone
DE = datetime.datetime(1970, 1, 1, tzinfo=timezone.utc)
cursor.execute("insert into monitor_regionserver (cluster_id, task_id, name, last_attempt_time, load, numberOfRegions, numberOfRequests, memStoreSizeMB, storefileSizeMB, readRequestsCount, writeRequestsCount, readRequestsCountPerSec, writeRequestsCountPerSec, replication_last_attempt_time, replicationMetrics) values (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)", [30L, 484L, '', DE, 0, 0, 0, 0, 0, 0, 0, 0, 0, DE, ''])
The cursor command looks like this (text-wrapped):
cursor.execute("insert into monitor_regionserver (cluster_id, task_id, name, last_attempt_time, load, numberOfRegions, numberOfRequests, memStoreSizeMB, storefileSizeMB, readRequestsCount, writeRequestsCount, readRequestsCountPerSec, writeRequestsCountPerSec, replication_last_attempt_time, replicationMetrics) values (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)", [30L, 484L, '', DE, 0, 0, 0, 0, 0, 0, 0, 0, 0, DE, ''])
I always get the error:
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'load, numberOfRegions, numberOfRequests, memStoreSizeMB, storefileSizeMB, readRe' at line 1")
I'm confused with this problem, any idea to share?
load is a keyword in mysql see here
try to rename the column
Related
I'm trying to insert some values into my vertica database using the vertica_python module:
data = {'SalesNo': ['12345', '678910'],
'ProductID': ['12345_2021-10-21_08:51:22', '678910_2021-10-21_10:27:03'],
'StoreID': ['6d522936e240cd64e1cf9176c5bfdff3bfe8146a345ff2', 'a7274d507d443c752be66b2851415138d75bd913d4949e'],
'PurchaseTime': ['2021-10-21 08:51:22.846000', '2021-10-21 10:44:06.218000'],
'Date': ['2021-10-21', '2021-10-21'],
'StoreNumber': ['0', '1'],
'PurchaseValue': ['348.0', '4893.23']
}
dataset = pd.DataFrame(data)
column = dataset.columns
n = len(column)
SQL_insert = f"INSERT INTO table_name ({','.join(list(column))}) VALUES ({' ?,'*(n-1)} ?);"
valueTuplelst = []
for i,row in dataset.iterrows():
valuelist = list(map(lambda x: str(x), row))
valuelist = [None if element == "None" else element for element in valuelist]
valuetup = tuple(valuelist)
valueTuplelst.append(valuetup)
connection.cursor().executemany(SQL_insert, valueTuplelst)
The equivalent SQL statement is
INSERT INTO table_name (SalesNo,ProductID,StoreID,PurchaseTime,Date,StoreNumber,PurchaseValue) VALUES ('12345', '12345_2021-10-21_08:51:22', '6d522936e240cd64e1cf9176c5bfdff3bfe8146a345ff2', '2021-10-21 08:51:22.846000', '2021-10-21', '0', '348.0'), ('678910', '678910_2021-10-21_10:27:03', 'a7274d507d443c752be66b2851415138d75bd913d4949e', '2021-10-21 10:44:06.218000', '2021-10-21', '1', '4893.23')
which works perfectly in SQL when I execute it.
However I get the error
File "C:\tools\Anaconda3\lib\site-packages\vertica_python\vertica\cursor.py", line 576, in format_operation_with_parameters
operation = operation % tuple(tlist)
TypeError: not all arguments converted during string formatting
I can't seem to figure out why I get this error as I convert all of my data to string format. Any idea where I'm going wrong?
My guess is that you should use %s and not ? as placeholder in your string:
SQL_insert = f"INSERT INTO table_name ({','.join(list(column))}) VALUES ({' %s,'*(n-1)} %s);"
Then the output string will be 'INSERT INTO table_name (SalesNo,ProductID,StoreID,PurchaseTime,Date,StoreNumber,PurchaseValue) VALUES ( %s, %s, %s, %s, %s, %s, %s);', which is compatible with operation = operation % tuple(tlist) replacement
I'm trying to implement a few simple SQL insert statements with python-mariadb-connector but cannot figure out what I'm doing wrong.
The database looks like this:
SET FOREIGN_KEY_CHECKS = false;
CREATE OR REPLACE TABLE `forums` (
`id` int(10) unsigned NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE OR REPLACE TABLE `accounts` (
`id` int(10) unsigned NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE OR REPLACE TABLE `posts` (
`id` int(10) unsigned NOT NULL,
`forum_id` int(10) unsigned NOT NULL,
`account_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`id`),
KEY `posts_forum_fk` (`forum_id`),
KEY `posts_account_fk` (`account_id`),
CONSTRAINT `posts_account_fk` FOREIGN KEY (`account_id`) REFERENCES `accounts` (`id`),
CONSTRAINT `posts_forum_fk` FOREIGN KEY (`forum_id`) REFERENCES `forums` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE OR REPLACE TABLE `comments` (
`id` int(10) unsigned NOT NULL,
`post_id` int(10) unsigned NOT NULL,
`account_id` int(10) unsigned NOT NULL,
`parent_id` int(10) unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `comments_post_fk` (`post_id`),
KEY `comments_account_fk` (`account_id`),
-- KEY `comments_comments_fk` (`parent_id`),
-- CONSTRAINT `comments_comments_fk` FOREIGN KEY (`parent_id`) REFERENCES `comments` (`id`),
CONSTRAINT `comments_account_fk` FOREIGN KEY (`account_id`) REFERENCES `accounts` (`id`),
CONSTRAINT `comments_post_fk` FOREIGN KEY (`post_id`) REFERENCES `posts` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
SET FOREIGN_KEY_CHECKS = true;
The code for inserting data looks like this:
import mariadb
config = {
"user": "db_user_name",
"password": "db_passwd",
"host": "db_host",
"port": 3306,
"database": "db_name"
}
if __name__ == '__main__':
with mariadb.connect(**config) as conn:
cur = conn.cursor()
cur.executemany(
"INSERT INTO `forums` (`id`) VALUES (?)",
[(1,), (2,), (3,)]
)
cur.executemany(
"INSERT INTO `accounts` (`id`) VALUES (?)",
[(1,), (2,), (3,), (4,)]
)
cur.executemany(
"INSERT INTO `posts` (`id`, `forum_id`, `account_id`) VALUES (?, ?, ?)",
[(6, 3, 1)]
)
cur.executemany(
"INSERT INTO `comments` (`id`, `post_id`, `account_id`, `parent_id`) VALUES (?, ?, ?, ?)",
[(1, 6, 1, None), (2, 6, 2, 1)]
) # exception happens here
When executing this, I get the following error:
Traceback (most recent call last):
File ".../db_test.py", line 28, in <module>
cur.executemany(
mariadb.DatabaseError.DataError: Invalid parameter type at row 2, column 4
Im not sure how executemany is implemented but I think it should do something similar to the following SQL-query:
INSERT INTO `forums` (`id`) VALUES (1), (2), (3);
INSERT INTO `accounts` (`id`) VALUES (1), (2), (3), (4);
INSERT INTO `posts` (`id`, `forum_id`, `account_id`) VALUES (6, 3, 1);
INSERT INTO `comments` (`id`, `post_id`, `account_id`, `parent_id`)
VALUES (1, 6, 1, NULL), (2, 6, 2, 1);
which works just fine for me...
Is it a bug or am I doing something wrong here?
It took me a while
cur.executemany(
"INSERT INTO `comments` (`id`, `post_id`, `account_id`, `parent_id`) VALUES (?, ?, ?, ?)",
[(1, 6, 1, None), (2, 6, 2, 1)]
)
But in your comment table, you have this constraint
CONSTRAINT `comments_comments_fk` FOREIGN KEY (`parent_id`)
REFERENCES `comments` (`id`),
Before you can enter (2, 6, 2, 1) the tuple (1, 6, 1, None) has to already commit in the database and in a bulk insert the commit is made after all inserts are in the database, but the first tuple isn't at the time not there
so if you make this, both rows will appear in the database(i also committed all other tables after bulk insert):
MariaDB
mycursor.execute(
"INSERT INTO `comments` (`id`, `post_id`, `account_id`,
`parent_id`) VALUES (?, ?, ?, ?)",
(1, 6, 1,None ))
mydb.commit()
mycursor.executemany(
"INSERT INTO `comments` (`id`, `post_id`, `account_id`, `parent_id`) VALUES (?, ?, ?, ?)",
[ (2, 6, 2, 1)])
mydb.commit()
MySQL
sql = """INSERT INTO forums (id) VALUES (%s)"""
val = [("1",), ("2",), ("3",)]
mycursor.executemany(sql,val)
mydb.commit()
mycursor.executemany(
"INSERT INTO accounts (id) VALUES (%s)",
[(1,), (2,), (3,), (4,)]
)
mydb.commit()
mycursor.executemany(
"INSERT INTO posts (id, forum_id, account_id) VALUES (%s, %s, %s)",
[(6, 3, 1)]
)
mydb.commit()
mycursor.execute(
"INSERT INTO `comments` (`id`, `post_id`, `account_id`, `parent_id`) VALUES (%s, %s, %s, %s)",
(1, 6, 1,None ))
mydb.commit()
mycursor.executemany(
"INSERT INTO `comments` (`id`, `post_id`, `account_id`, `parent_id`) VALUES (%s, %s, %s, %s)",
[ (2, 6, 2, 1)])
mydb.commit()
I use the following query:
cur.executemany(sql, data)
conn.commit()
Where data is:
[('ON-LOOK', 'ONLOOK KOREA', 'master#on-look.co.kr', 'Entertainment', '1.287', 'USK: All ages', 'https://www.google.com/url', 'https://ama.com', 'December 13, 2016', '//lh3.googleusercontent.com/bZrL9zRTXujVSFsNS_vD8AomApZeP2xgI1vGTsOz_zvoVKOz5T9pk7nFB3ONXXf04rqj=w300', '100 - 500', '4.7', '6', '4.3 and up')]
And sql is:
sql = """insert into apps_global (app_name, app_developer, app_email, app_category, app_version, app_age, app_website, app_url, app_developer_url, app_updated, app_image, app_installations, app_score, app_score_count, app_android \
) values ('%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s')"""
When I try to execute query I get error:
ProgrammingError(1064, "1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ON-LOOK'', ''ONLOOK KOREA'', ''master#on-look.co.kr'', ''Entertainment'', ''1.28' at line 1", u'42000'))
How to fix it?
Here is a query I have that runs fine in MySQL workbench with the included sample values and works fine if I manually plug in the values in the code, but fails when I use the values as parameters. Any ideas?
Python Code:
print player
cur.execute("""
INSERT INTO scoredata
(gameid, playerid, starter, pos, min, fgm, fga, tpm, tpa, ftm, fta, oreb, reb, ast, stl, blk, tos, pf, pts)
VALUES
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);
"""), (player[0],
int(player[20]),
int(player[19]),
player[3],
int(player[4]),
int(player[5]),
int(player[6]),
int(player[7]),
int(player[8]),
int(player[9]),
int(player[10]),
int(player[11]),
int(player[12]),
int(player[13]),
int(player[14]),
int(player[15]),
int(player[16]),
int(player[17]),
int(player[18]) )
db.commit()
Error message:
['330060130', 103, 'Roy Devyn Marble', 'G-F', '28', '4', '9', '3', '6', '3', '3', '0', '2', '1', '0', '0', '0', '1', '14', 1, 1391]
Traceback (most recent call last):
File "C:\Users\jcaine\workspace\BasketballStats\src\BasketballStats\basketballstats.py", line 350, in <module>
insert_game_data('20130106', '20130106')
File "C:\Users\jcaine\workspace\BasketballStats\src\BasketballStats\basketballstats.py", line 284, in insert_game_data
"""), (player[0], int(player[20]), int(player[19]), player[3], int(player[4]), int(player[5]), int(player[6]), int(player[7]), int(player[8]), int(player[9]), int(player[10]), int(player[11]), int(player[12]), int(player[13]), int(player[14]), int(player[15]), int(player[16]), int(player[17]), int(player[18]) )
File "c:\users\jcaine\appdata\local\temp\easy_install-7_fysp\MySQL_python-1.2.3-py2.7-win32.egg.tmp\MySQLdb\cursors.py", line 174, in execute
File "c:\users\jcaine\appdata\local\temp\easy_install-7_fysp\MySQL_python-1.2.3-py2.7-win32.egg.tmp\MySQLdb\connections.py", line 36, in defaulterrorhandler
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)' at line 4")
MySQL scoredata Table Columns:
gameid varchar
playerid int
starter int
pos varchar
min int
fgm int
fga int
tpm int
tpa int
ftm int
fta int
oreb int
reb int
ast int
stl int
blk int
tos int
pf int
pts int
MySQL Code that runs fine in Workbench:
INSERT INTO scoredata (gameid, playerid, starter, pos, min, fgm, fga, tpm,
tpa, ftm, fta, oreb, reb, ast, stl, blk, tos, pf, pts)
VALUES ('3300601300', 1391, 1, 'G-F', 28, 4, 9, 3, 6, 3, 3, 0, 2, 1, 0, 0, 0, 1, 14)
You're not passing data to the execute call. Note the closing brace in your example.
cur.execute("""
INSERT INTO scoredata
(gameid, playerid, starter, pos, min, fgm, fga, tpm, tpa, ftm, fta, oreb, reb, ast, stl, blk, tos, pf, pts)
VALUES
(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);
""")//*Remove me*
, (player[0],
int(player[20]),
int(player[19]),
player[3],
int(player[4]),
int(player[5]),
int(player[6]),
int(player[7]),
int(player[8]),
int(player[9]),
int(player[10]),
int(player[11]),
int(player[12]),
int(player[13]),
int(player[14]),
int(player[15]),
int(player[16]),
int(player[17]),
int(player[18]) )
db.commit()
import mysql.connector
cnx = mysql.connector.connect(user=user, password=password,host=host,database=database)
cursor = cnx.cursor()
add_employee = ("INSERT INTO tbl_gps "
"(Locating, MainPower, Acc, PowerOff, Alarm,Speed,Direction,Latitude,Longitude,DateTime,IOState,OilState,MainID) "
"VALUES (%s, %s, %s, %s, %s,% s, %s, %s, %s, %s, %s, %s, (SELECT MainID From tbl_device WHERE DeviceID = %s))")
data_employee = {
'Locating': 1,
'MainPower': 1,
'Acc': 1,
'PowerOff': 1,
'Alarm': 0x80,
'Speed': 1,
'Direction': 300,
'Latitude': 10.00,
'Longtitude': -20.00,
'DateTime': '2013/2/8 01:00:00',
'IOState': 0,
'OilState': 0,
'MainID': 'NR09533'
}
cursor.execute(add_employee, data_employee)
cnx.close()
when i debug get the following error:
AttributeError: function 'inet_pton' not found
what wrong with my code?please help to fix it,thanks you very much !
the connector are downlaoded from Mysql official web site,
and i m using Python 3.3
i have found this in google, this is a Mysql Bug?because i m using Window XP.
http://bugs.mysql.com/bug.php?id=68065
Problem Solved after i download the Connector from http://pypi.python.org/pypi/mysql-connector-python, the mysql official web's connector can't use
oh, that hits the point, i came across the same problem. thanks, it really works.
it seems the privous version(1.0.7) is ok, and v1.0.8 is bad.
ps. the real download link is:
http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-1.0.7.zip
the installation tutorial is in dir 'doc'.