Can anyone please guide me in how to run .hql queries using Python. Tried Hiveutils lib but its not present in the dev environment. Any other way to execute the queries?
You can use PyHive: PyHive is a collection of Python DB-API and SQLAlchemy interfaces for Presto and Hive.
Example:
from pyhive import hive
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10')
print(cursor.fetchone())
print(cursor.fetchall())
You can try using pyhs2, try working like the below given example
import pyhs2
with pyhs2.connect(host='localhost',
port=10000,
authMechanism="PLAIN",
user='your_user',
password='your_password',
database='your_default_db') as conn:
with conn.cursor() as cur:
print cur.getDatabases()
cur.execute("select * from table")
#Return info from query
print cur.getSchema()
Related
I am currently developing a program in python that interacts with multiple database. I am using pyodbc to connect, and execute queries. One of the database is an azure database. I noticed sometimes the sent data is not updated in the database although the program run successfully and no error was thrown. Is there any practices that i should follow to make sure this doesn't happen or is this related to my code or db connection issue? I am a beginner. Would appreciate everyone's help thank you!
Also is the .commit() line should be run after every sql run?
The program should be updating a row of data in the database based on a condition, this particular query sometimes doesn't take effect, but no error was thrown. I also executed multiple queries after that, no issue was found for the next queries. It is successfully executed.
the query is a simple query which is
UPDATE DraftVReg SET VRStatus = 'Potential Duplicate Found' WHERE RowID = ?
I tried to reproduce your scenario on my end and was able to update the SQL row in the Azure SQL DB with Pyodbc module.
Yes, Its very necessary to use
conn.commit
to commit your changes inside a database after you perform operations such as Update or Insert inside Azure SQL DB programmatically.
1) Fetch Data with Select statement.
I was able to fetch the Table’s data successfully with Select * from ‘Tablename’ query inside pyodbc code before I try UPDATE statement.
import pyodbc
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};''SERVER=tcp:sqlservernamesql.database.windows.net,1433;''DATABASE=databasename; UID=siliconuser;PWD=Password;')
#conn.commit()
cursor = conn.cursor()
cursor.execute('Select * FROM StudentReviews')
#conn.commit()
for i in cursor:
print(i)
cursor.close()
conn.close()
Result:-
2) UPDATE the rows require conn.commit()
Code :-
import pyodbc
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};''SERVER=tcp:siliconserversql.database.windows.net,1433;''DATABASE=silicondb; UID=userid; PWD=Password;')
cursor = conn.cursor()
#cursor.execute('Select * FROM StudentReviews')
cursor.execute("UPDATE StudentReviews SET ReviewTime = ('7') WHERE ReviewText = ('SQL DB')")
conn.commit()
cursor.close()
conn.close()
Result:-
Update statement Executed successfully and the Table Row was updated in Azure SQL, Refer Below :-
3) With autocommit=true
Thank you #Gord thompson for the comment and suggestion!
Code :-
import pyodbc
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};''SERVER=tcp:siliconserversql.database.windows.net,1433;''DATABASE=silicondb; UID=username; PWD=Password;', autocommit=True)
#conn.commit()
cursor = conn.cursor()
cursor.execute("UPDATE StudentReviews SET ReviewTime = ('8') WHERE ReviewText = ('SQL DB')")
cursor.close()
conn.close()
Results :- With autocommit=true, You do not need to add conn.commit everytime you update the SQL DB.
I am creating a program that uses VS Code and MySQL Workbench 8.0 together. I am stuck and do not know how to connect the two software together
I also have to be able to upload records into a table that is stored in MySQL Workbench from the Python program that uses variables.
Please tell me if their are any details missing.
Thank you.
For connection:
I have researched on Google and have been unable to find an answer. I have found that I have to install certain packages and use the connect method. However, I do not know the parameters of the connect function.
For uploading data into table:
I have found that I have to create a cursor to somehow upload the data to the table, but am unsusre of the full details.
There are many packages in python that can connect to the mysql database, here we take pymysql as an example.
Install pymysql
pip install PyMySQL
I have already installed, so the prompt package already exists.
Sample code, query and insert data
import pymysql
con = pymysql.Connect(
host='localhost',
port=3306,
user='root',
password='123456',
db='test',
charset='utf8'
)
cur = con.cursor()
sql1 = 'select * from student'
cur.execute(sql1)
data = cur.fetchall()
cur.close()
con.close()
for i in data:
print(str(i))
Add an insert data statement, and re-query after inserting data.
import pymysql
con = pymysql.Connect(
host='localhost',
port=3306,
user='root',
password='123456',
db='test',
charset='utf8'
)
cur = con.cursor()
sql2 = 'insert into student values("002","jerry","W");'
cur.execute(sql2)
sql1 = 'select * from student'
cur.execute(sql1)
data = cur.fetchall()
con.commit()
cur.close()
con.close()
for i in data:
print(str(i))
I want to connect to MySql database using Python through PythonAnywhere, without creating a Flask/Django application.
I have seemingly managed to connect through MySQLdb, using the code below, but I do not receive a response when I run the code. Any solutions?
import MySQLdb
db = MySQLdb.connect(
host = "myuser.mysql.pythonanywhere-services.com",
user = "myuser",
passwd = XXX,
db = "myuser$db_name"
)
cursor = db.cursor()
cursor.execute("SELECT * FROM table_name")
for x in cursor:
print(x)
cursor.close()
db.close()
You retrieve all rows in the table, without error.
cursor.execute("SELECT * FROM table_name")
for x in cursor:
print(x)
Yet you see no output. This is normal for a table that contains zero rows.
Consider doing one or more INSERTs, and a COMMIT,
prior to the query.
I am using Python with spyder and would like to directly connect to the our postgres datawarehouse to directly query data from there for further analysis.
This is what I found under the following link:
[https://www.psycopg.org/docs/]
import psycopg
Connect to your postgres DB
conn = psycopg.connect("dbname=*** user=***")
Open a cursor to perform database operations
cur = conn.cursor()
Execute a query
cur.execute("SELECT * FROM my_data")
Retrieve query results
records = cur.fetchall()
However, I get the following error message:
runfile('C:/Users/.../.spyder-py3/untitled2.py', wdir='C:/Users/.../.spyder-py3')
Traceback (most recent call last):
File "C:\Users....spyder-py3\untitled2.py", line 9, in
import psycopg
ModuleNotFoundError: No module named 'psycopg'
https://www.psycopg.org/docs/ -> this is what you are looking for.
Next step might be using pandas read_sql() to load the sql table into to your variable exploirer if it is not too big.
Does this help?
Posting this in the Spyder Editor actually worked to establish a connection with the PostgreSQL Database:
import psycopg2
connection = psycopg2.connect(user="...",
password="...",
host="...",
port="...",
database="...")
cursor = connection.cursor() cursor.execute("SELECT * FROM data.table25 limit 5")
After running the above command you can check for your rows with:
rows=cur.fetchall()
sqlplus sys/Oracle_1#pdborcl as sysdba;
i'm using this command to connect to Oracle 12c from Command Prompt.
How can i connect to the db using cx_Oracle. I'm new to Oracle DB.
I think this is the equivalent of the sqlplus command line that you posted:
import cx_Oracle
connect_string = "sys/Oracle_1#pdborcl"
con = cx_Oracle.connect(connect_string,mode=cx_Oracle.SYSDBA)
I tried it with a non-container database and not with a pdb so I can't verify that it would work with a pdb. You may not want to connect as sys as sysdba unless you know that you need that level of security.
Bobby
You can find the documentation here cx_Oracle docs
To query the database, use the below algorithm
import cx_Oracle
dsn = cx_Oracle.makedsn(host, port, sid)
connection = cx_Oracle.connect(dsn,mode = cx_Oracle.SYSDBA)
query = "SELECT * FROM MYTABLE"
cursor = connection.cursor()
cursor.execute(query)
resultSet=cursor.fetchall()
connection.close()
The above code works to fetch data from MYTABLE connecting to the above dsn.
Better to go through cx_Oracle docs.