I was working on a straightforward SQLAlchemy Core (Python 3.x) count unique query using the following code:
table_object = sqlalchemy.Table(table_name, metadata, autoload=True, autoload_with=engine, schema=schema)
agg_fields = [get_column_correct_case(table_object, col) for col in agg_fields]
agg_col_obj = [table_object.c[col] for col in agg_fields]
agg_query = sqlalchemy.select(agg_col_obj).select_from(table_object)\
.group_by(*agg_fields).count()
engine.scalar(agg_query)
While I believe this should be valid SQLAlchemy Core code, its execution returned a PYODBC error:
(pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]
Incorrect syntax near ')'. (102) (SQLExecDirectW);
[42000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Statement(s) could not be prepared. (8180)")
The SQL query generated by this code which failed to execute properly was:
[SQL: SELECT count([ACCOUNT]) AS tbl_row_count
FROM (SELECT dbo.history.[ACCOUNT] AS [ACCOUNT], dbo.history.[RTXNNBR] AS [RTXNNBR]
FROM dbo.history
GROUP BY dbo.history.[ACCOUNT], dbo.history.[RTXNNBR])]
Running the query directly in SQL Server, it appears to be failing because the nested select is missing an alias. Is this a bug? If not, how do I fix this code?
I figured out the answer. I was surprised it was necessary, but this can be resolved by giving the query to be counted an alias.
So
agg_query = sqlalchemy.select(agg_col_obj).select_from(table_object)\
.group_by(*agg_fields).count()
becomes:
agg_query = sqlalchemy.select(agg_col_obj).select_from(table_object)\
.group_by(*agg_fields).alias('a').count()
which now works.
Related
I am trying to execute below SQL query by introducing 2 variables in Python but getting error,
Original Query:
ALTER DATABASE db1 MODIFY (SERVICE_OBJECTIVE = 'DW300');
I want to use variables for db1 and 'DW300'. I have tried below statement in Python and got error.
dwu = 'DW100'
sqlpool = 'db1'
cursor.execute("""ALTER DATABASE ? MODIFY (SERVICE_OBJECTIVE = ?) """, (sqlpool, dwu))
Error: pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Incorrect syntax near '#P1'. (102) (SQLExecDirectW)")
I use sqlite and it accepts this answe.
try the use of fstrings
f"""Alter Database {sqlpool} modify (service objective = {dwu}) """
I am trying to save some data from file to database using pyodbc bulk insert.
I am getting a lot of these exceptions:
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Driver 17
for SQL Server][SQL Server]Bulk load data conversion error (type
mismatch or invalid character for the specified codepage) for row
90411, column 3 (ARTIST_NAME).
I guess it could be caused by some speical chars like %, ó, ś. I could not even point all of them cause there is over 10 milions lines in a file. Is there any way to get rid of this exception?
My bulk insert looks like this:
BULK INSERT someSchema.SomeTable FROM 'filePath'
WITH (
FIELDTERMINATOR='',
ROWTERMINATOR='\n'
);
I tried to add CODEPAGE = '65001' to this command but then I got
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Keyword or statement option 'CODEPAGE' is not supported on the 'Linux' platform. (16202) (SQLExecDirectW)")
I am running a query to MS SQL Server through pyodbc:
conn = pyodbc.connect('DRIVER={ODBC Driver 11 for SQL Server};SERVER=some_server, some_port;DATABASE=MAP;UID='xxx';PWD='xxx')
cursor = conn.cursor()
query = """SELECT v_MAMain.MAID, v_MAMain.startDate, v_MAMain.endDate, v_MAMain.fromTime, v_MAMain.toTime, v_MAMain.StatusDesc, v_MAMain.name, v_Event_Statistic.TotalGuestsAtEvent, v_Event_Statistic.PlannedGuestsAtEvent, v_Event_Statistic.City, v_Event_Statistic.Location, v_Event_Statistic.Host, v_Event_Statistic.EventSpeakers, v_MAMain.Owner, v_Event_Statistic.PlannedHoursProjectOwner, v_Event_Statistic.EventManagerName, v_Event_Statistic.PlannedHoursEventManager, v_Event_Statistic.CreditSuisseEventManager, v_MAMain.CostCenter, v_MAMain.[Project Year], v_MAMain.category, MADetails.NameOfEventStaff1, MADetails.PlannedWorkDurationEventStaff1, MADetails.NameOfEventStaff2, MADetails.PlannedWorkDurationEventStaff2, MADetails.NameOfEventStaff3, MADetails.PlannedWorkDurationEventStaff3, MADetails.NameOfEventStaff4, MADetails.PlannedWorkDurationEventStaff4, MADetails.NameOfEventStaff5, MADetails.PlannedWorkDurationEventStaff5, v_Event_Statistic.KPIEventQuality, v_Event_Statistic.KPIBusinessFit
FROM (v_MAMain INNER JOIN v_Event_Statistic ON v_MAMain.MAID = v_Event_Statistic.ProjectID) INNER JOIN MADetails ON v_MAMain.MAID = MADetails.MAID
WHERE (((v_MAMain.StatusDesc)<>"Deleted") AND ((v_MAMain.CostCenter)="0892 / 932" Or (v_MAMain.CostCenter)="0897 / 951") AND ((v_MAMain.[Project Year])=2019) AND ((v_MAMain.category)="Event"));"""
cursor.execute(query)
I get the following error:
pyodbc.ProgrammingError: ('42S22', "[42S22] [Microsoft][ODBC Driver
11 for SQL Server][SQL Server]Invalid column name 'Deleted'. (207)
(SQLExecDirectW); [42S22] [Microsoft][ODBC Driver 11 for SQL
Server][SQL Server]Invalid column name '0892 / 932'. (207); [42S22]
[Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Invalid column
name '0897 / 951'. (207); [42S22] [Microsoft][ODBC Driver 11 for SQL
Server][SQL Server]Invalid column name 'Event'. (207)")
It seems that the column criteria: Deleted, Event etc. are being read as column names instead. The same query without the WHERE clause will work fine.
The query is designed in Access and the auto-generated SQL is copy-pasted from Access to my script.
Do I need to correct the SQL somehow or is it something else causing the error?
Change double quotes to single quotes around constant values. In SQL Server string in double quote is considered a column name.
Sometimes I get this error randomly when I try to insert data in to the database. I fetch the data with using request.get and parsing the JSON data.
This is the error that I get:
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17
for SQL Server][SQL Server]Incorrect syntax near 's'. (102)
(SQLExecDirectW); [42000] [Microsoft][ODBC Driver 17 for SQL
Server][SQL Server]Unclosed quotation mark after the character string
')'. (105)")
This is my code:
for product in parsed['products']:
cursor.execute("INSERT INTO dbo.producten (productid, datum_toegevoegd, naam, prijs_excl, prijs_incl, gewicht) VALUES ('%s','%s','%s','%s','%s','%s')" %(product['id'],product['created_at'], product['nl']['title'],product['price_excl'],product['price_incl'],product['weight']))
You must not use string interpolation for SQL queries. The db-api will do correct parameter substitution for you - replace that % with a comma.
cursor.execute('SELECT.... ', (product['id'],product['created_at'...))
# ^
cursor.execute('SELECT.... product = ?', (value_for_product))
works for python 3.^
I have created a Dataset containing the various attributes of tweets extracted from Tweepy, the Twitter API. I am trying to insert values from various columns of that Dataset into a database created in SQL Server, by typecasting each attribute into a list. I know this is probably THE worst way to go about this, but once I have the problem solved I'll definitely be improving this code.
connection = pyodbc.connect('DRIVER={SQL Server};Server=***;Database=Tweets;User Id=***;Password=***;')
cursor = connection.cursor()
ID_list = list(dataset['ID'])
text_list = list(dataset['Text'])
fav_count_list = list(dataset['Favourite_Counter'])
retweet_count_list = list(dataset['Retweet_Counter'])
timestamp_list = list(dataset['Timestamp'])
location_list = list(dataset['Location'])
for i in range(1,len(ID_list),1):
cursor.execute('INSERT INTO Tweet(ID,Text,Favourite_Counter,Retweet_Counter,Timestamp,Location) VALUES(%?,%?,%?,%?,%?,%?)',(tweetID_list[i],text_list[i],fav_count_list[i],retweet_count_list[i],timestamp_list[i],location_list[i]))
connection.commit()
cursor.close()
connection.close()
Once I execute this code block, I get the following error:
---------------------------------------------------------------------------
ProgrammingError Traceback (most recent call last)
<ipython-input-62-a61238cf4963> in <module>()
11
12 for i in range(1,len(ID_list),1):
---> 13 cursor.execute('INSERT INTO Tweet(ID,Text,Favourite_Counter,Retweet_Counter,Timestamp,Location) VALUES(%?,%?,%?,%?,%?,%?)',(tweetID_list[i],text_list[i],fav_count_list[i],retweet_count_list[i],timestamp_list[i],location_list[i]))
14 connection.commit()
15
ProgrammingError: ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near '#P1'. (102) (SQLExecDirectW); [42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared. (8180)")
I've been trying to solve this for a good part of 2 hours, with not much progress. Can anyone please help me and give me a good solution for solving this error?
You are using ODBC for connecting to the database. Timestamp is an ODBC reserved word (see here).
Try escaping it, using [Timestamp] (or change the name to something safer).