This question already has an answer here:
Postgres error after updating TimescaleDB on Ubuntu: file not found
(1 answer)
Closed 2 years ago.
I was trying to hook up the sqlalchemy with my underlying postgresql, which uses the timescaledb extension. All queries work fine when I try them from the psql terminal client. But when I try to use python & sqlalchemy to do it, it keeps throwing me an error.
Here's the very basic code snippet that I try to test it with:
engine = create_engine('postgres://usr:pwd#localhost:5432/postgres', echo=True)
engine.execute('select 1;')
And it always shows the following the error message:
File "/home/usr/.local/share/virtualenvs/redbird-lRSbFM0t/lib/python3.6/site-packages/psycopg2/extras.py", line 917, in get_oids
""" % typarray)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not access file "timescaledb-0.9.0": No such file or directory
The connection to the db is fine, otherwise it won't know the db is using timescaledb.
Any one has any insights?
UPDATE: I try to use psycopg2 directly. It basically gives the same error. DB is connected successfully, but timescaledb-0.9.0 cannot be accessed.
Here's the code snippted
conn_string = "host='localhost' dbname='db' user='usr' password='pwd'"
print("Connecting to database\n ->%s " % (conn_string))
conn = psycopg2.connect(conn_string)
cursor = conn.cursor()
print("Connected!\n")
cursor.execute("\dx")
records = cursor.fetchall()
Here's the exact same error message:
Connecting to database
Connected!
Traceback (most recent call last):
File "/home/usr/Workspace/somepath/web/model/model.py", line 21, in <module>
cursor.execute("\dx")
psycopg2.OperationalError: could not access file "timescaledb-0.9.0": No such file or directory
This seems very similar to my issue.
I guess you also updated to a new version of Timescale? The thing is: After each update of the timescale package you don't just have to make sure the library is preloaded (as the warning says on the command line) - you also have to upgrade each database that uses the extension manually via psql.
See my own answer to my issue for the steps.
--
This snipplet works for me:
#! /usr/bin/python
# -*- coding: utf-8 -*-
import psycopg2
# Connect to an existing database.
conn = psycopg2.connect(dbname='my-db-name',
user='postgres',
password='super-secret',
host='localhost',
port='5432')
# Open a cursor to perform database operations.
cur = conn.cursor()
# Query the database and obtain data as Python objects.
cur.execute('SELECT * FROM my-table-name LIMIT 100 ;')
# Print results.
results = cur.fetchall()
for result in results:
print(result)
# Close communication with the database.
cur.close()
conn.close()
Using the cursor to executue psql commands also doesn't work for me. I don't think it is supposed to. But what works reliably is doing SQL:
# Check if the database has the timescaledb extension installed.
# This is about the same as xecuting '\dx' on psql.
cur.execute('SELECT * from pg_extension;')
Related
I am using Python with spyder and would like to directly connect to the our postgres datawarehouse to directly query data from there for further analysis.
This is what I found under the following link:
[https://www.psycopg.org/docs/]
import psycopg
Connect to your postgres DB
conn = psycopg.connect("dbname=*** user=***")
Open a cursor to perform database operations
cur = conn.cursor()
Execute a query
cur.execute("SELECT * FROM my_data")
Retrieve query results
records = cur.fetchall()
However, I get the following error message:
runfile('C:/Users/.../.spyder-py3/untitled2.py', wdir='C:/Users/.../.spyder-py3')
Traceback (most recent call last):
File "C:\Users....spyder-py3\untitled2.py", line 9, in
import psycopg
ModuleNotFoundError: No module named 'psycopg'
https://www.psycopg.org/docs/ -> this is what you are looking for.
Next step might be using pandas read_sql() to load the sql table into to your variable exploirer if it is not too big.
Does this help?
Posting this in the Spyder Editor actually worked to establish a connection with the PostgreSQL Database:
import psycopg2
connection = psycopg2.connect(user="...",
password="...",
host="...",
port="...",
database="...")
cursor = connection.cursor() cursor.execute("SELECT * FROM data.table25 limit 5")
After running the above command you can check for your rows with:
rows=cur.fetchall()
I am trying to run MySQL commands through Python source code and have it display in my web browser.
When I compile the code in an editor (Komodo Edit) it complies and returns the expected set of rows, but when I try to view in a browser the source code is displayed, not the query result I seek...
Here's what I have...
import mysql.connector
conn = None
try:
conn = mysql.connector.Connect(host="localhost", user="jump", password="secret")
print('Connected to MySQL!')
except Exception as ex:
print('cannot connect to MySQL : exception : ' + str(ex))
cursor = conn.cursor()
cursor.close()
print('terminating connection with MySQL')
conn.close()
...I'm pretty sure the source is good, and that somewhere I missed a configuration step or some such.
I can get my command prompt to open the source file and display the query results, so I've got a connection between Python and MySQL server, which I verified through the MySQL workbench also.
Thanks for any help y'all can give!
-M
This is my first question here. So, I am sorry if it is repeated or the formatting is off. I searched through other questions and the error is common but appears on multiple situations.
I have a very simple python code where I want to execute a procedure in MSSQL from pyodbc.
import pyodbc
conn = pyodbc.connect(r'DSN=myDSN')
cursor = conn.cursor()
query = r'{call myproc}'
cursor.execute(query)
I am using call instead of exec after reading that ODBC uses call for executing procedures in MSSQL.
The error I am getting is the following:
Traceback (most recent call last):
File "myscript.py", line 26, in <module>
cursor.execute(query)
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][SQL Server Native Client 11.0][SQL Server]The current transaction has aborted, and any pending changes have been rolled back. Cause: A transaction in a rollback-only state was not explicitly rolled back before a DDL, DML or SELECT statement. (111233) (SQLExecDirectW)')
Thanks for the help
In case someone is having the same issue. I was able to find out what the problems was. When you open a connection with DSN, the autocommit is set to False. For some reason, this should be True for the code to work (this depends largely on what I was doing on MSSQL).
import pyodbc
conn = pyodbc.connect(r'DSN=myDSN', autocommit=True)
cursor = conn.cursor()
query = r'{call myproc}'
cursor.execute(query)
This runs well!
Here are two examples on how you can execute a stored proc in MS SQL Server through pyodbc:
Passing a NULL, and the VARCHAR 'tallen' as positional parameter variables:
cursor.execute('EXEC usp_get_user_data ?, ?', None, 'flipperpa')
Passing two VARCHAR values as named parameter variables:
cursor.execute('EXEC usp_get_user_data #user_full_name = ?, #user_username = ?', 'flip', 'flipperpa')
Then to loop through the returned data:
rows = cursor.fetchall()
for row in rows:
# Do stuff
print(row.user_id)
Good luck!
I had issue with executing the Stored procedure using the SQL server 2008. The first step I did is to go to control panel > Administrative tools > Data Sources (ODBC) > add the driver
The only change I made in my python code is use "{call procdev_2017}". I got an error when I tried using exec instead of call
import pyodbc
conn = pyodbc.connect(driver='{SQL Server Native Client 11.0}', server='XXXXXXX', database='XXX',
trusted_connection='yes', autocommit=True)
bepcur = conn.cursor()
ipcur.execute("{call procdev_2017}")
I have case :
import pymysql
conn = pymysql.connect(host='127.0.0.1', unix_socket='/opt/lampp/var/mysql/mysql.sock', user='root', passwd=None, db='test')
cur = conn.cursor()
cur.execute("test < /mypath/test.sql")
cur.close()
conn.close()
I always get error :
1064 , "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'test < /mypath/test.sql' at line 1"
I tried to use source and it still failed. Did you know why?
Thank you.
Your error message says that the MySQL server can't understand
test < /mypath/test.sql' at line 1
If you're a long time *nix user, it seems intuitive that you should be able to use commands like this to pass various sorts of data streams to various programs. But that's not the way the Python sql API (or most language-specific) sql APIs works.
You need to pass a valid SQL query to the execute() method in the API, so the API can pass it to the database server. A vaild query will be something like INSERT or CREATE TABLE.
Look, the server might be on a different host machine, so telling the server to read from /mypath/test.sql is very likely a meaningless instruction to that server. Even if it did understand it, it might say File test.sql not found.
The mysql(1) command line client software package can read commands from files. Is that what you want?
>>> import MySQLdb
>>> db = MySQLdb.connect(host = 'demodb', user = 'root', passwd = 'root', db = 'mydb')
>>> cur = db.cursor()
>>> cur.execute('select * from mytable')
>>> rows = cur.fetchall()
Install MySQL-Python package to use MySQLdb.
I have a database in my server called ex.
So in my python script (where sqlite3 is imported) I will try to insert values...
try:
# Initial setup runs one time
connect = sqlite3.connect('ex')
cursor = connect.cursor()
except Exception as e:
error_string = e
# Insert query into the cursor
cursor.execute("INSERT INTO student_urls (user_name,assignment,url,created_at) \
values(?,?,?,date('now'))" ,(user_name,assignment, url))
But the cursor.execute line is giving me issues... In particular I get a 500 internal server error. Any ideas?
Specify the full path of your SQL database, as with sqlite3.connect('/var/www/ex.db'). Also, ensure that the application has sufficient read/write permissions. I encountered an identical problem when hosting my Flask app on a server, and had to both use a full path and chmod -R my app directory.