Running impala-shell from jupyter notebook - python

I am trying to run impala from jupyter-notebook as follows--
from impala.dbapi import connect
conn = connect(host='xx-xx-xx.xx.com',
port=21000,
auth_mechanism="PLAIN",
user='xxxx',
password='xxxx',
)
cursor = conn.cursor()
But I am getting this error
TTransportException Traceback (most recent call last)
<ipython-input-28-6c858acffc1b> in <module>
.
.
TTransportException: Bad status: 3 (b'Unsupported mechanism type PLAIN')
After trying out many things which did not work I thought to run impala command as python subprocess.run but I am getting no output (return code 1)--
r = subprocess.run(['impala-shell', '-q', "select xxx...xxx"],
stdout=subprocess.PIPE)
print(r.stdout.decode()) # returncode=1
Also !impala-shell in jupyter giving this error
File "/opt/xxxx/xxxx/xxxx/impala-shell/impala_shell.py", line 262
print "Query options (defaults shown in []):"
^
SyntaxError: invalid syntax
Can anyone let me know the problem and guide me on how to resolve this?

Related

issues running a face_recogition file using editors and cmd

When I run the code below in jupyter, it is excellent, I get the results I want.
real = fr.load_image_file("image.jpg")
unknown = fr.load_image_file("image2.jpg")
# encodings
real_encoding = fr.face_encodings(real)[0]
unk = fr.face_encodings(unknown)[0]
result = fr.compare_faces([real_encoding], unk)
print(result)
but when I copy and paste the same code to an editor(atom the one I am using), and then run it using command prompt, I get an error as shown below
Traceback (most recent call last):
File "face_recognizer.py", line 13, in <module>
real_encoding = fr.face_encodings(real)[0]
IndexError: list index out of range
I activated the environment in which I was running the cmd.
Can someone advise me where I maybe going wrong?
There was no face captured in the photo.
That is why it throws an exception.

viewer error when running the stokesCavity.py example

Running Manjaro stable with python-3.9 and python-fipy-3.4.2.1-1.
Just got started with FiPy, ultimately interested in writing single and two-phase flow code. Naturally I tried to run examples/flow/stokesCavity.py (stripped down from all rst text) with: python stokesCavity.py and it throws the following error:
Traceback (most recent call last):
File "/home/zbinkz/HGST/Projects/Python/fipy/examples/flow/stokesCavity.py", line 117, in <module>
viewer = Viewer(vars=(pressure, xVelocity, yVelocity, velocity),
File "/usr/lib/python3.9/site-packages/fipy/viewers/__init__.py", line 130, in Viewer
raise ImportError("Failed to import a viewer: %s" % str(errors))
ImportError: Failed to import a viewer: ["matplotlib: True is not a valid value for orientation; supported values are None, 'vertical', 'horizontal'", "mayavi: No module named 'enthought'"]
I tinkered with different values for FIPY_VIEWER in the viewer command at line 117 reported above but still get the same error. At this very early stage with FiPy I'm clueless, anybody know how to fix this?
Thanks :)
In this line, change
... viewer = Viewer(vars=(pressure, xVelocity, yVelocity, velocity),
... xmin=0., xmax=1., ymin=0., ymax=1., colorbar=True)
to
... viewer = Viewer(vars=(pressure, xVelocity, yVelocity, velocity),
... xmin=0., xmax=1., ymin=0., ymax=1., colorbar='vertical')
I've filed a ticket to correct this.

Getting traceback on pymysql with script

Got a simple script that is given below. When trying to run the script, I get traceback error that I cannot quite figure out. The error and script are below. I am running python 3.4 as is required. Not sure what is going on exactly. Any help is appreciated. It says its a syntax error on import... yet its the only import.
Traceback (most recent call last):
File "script.py", line 1, in
import pymysql
File "C:\Python34\lib\site-packages\pymysql_init_.py", line 59, in
from . import connections # noqa: E402
File "C:\Python34\lib\site-packages\pymysql\connections.py", line 206
):
^
SyntaxError: invalid syntax
import pymysql
myConnection = pymysql.connect(host='localhost', user = 'root', password='******', database='accidents')
cur = myConnection.cursor()
cur.execute('SELECT vtype FROM vehicle_type WHERE vtype LIKE "%otorcycle%";')
cycleList = cur.fetchall()
selectSQL = ('''
SELECT t.vtype, a.accident_severity
FROM accidents_2016 AS a
JOIN vehicles_2016 AS v ON a.accident_index = v.Accident_Index
JOIN vehicle_type AS t ON v.Vehicle_Type = t.vcode
WHERE t.vtype LIKE %s
ORDER BY a.accident_severity;
''')
insertSQL = ('''
INSERT INTO accident_medians VALUES (%s, %s);
''')
for cycle in cycleList:
cur.execute(selectSQL,cycle[0])
accidents = cur.fetchall()
quotient, remainder = divmod(len(accidents),2)
if remainder:
med_sev = accidents[quotient][1]
else:
med_sev = (accidents[quotient][1] + accidents[quotient+2][1])/2
print('Finding median for',cycle[0])
cur.execute(insertSQL,(cycle[0],med_sev))
myConnection.commit()
myConnection.close()
This is from connections.py. Which seems to be the standard install file.
def __init__(
self,
*,
user=None, # The first four arguments is based on DB-API 2.0 recommendation.
password="",
host=None,
database=None,
unix_socket=None,
port=0,
charset="",
sql_mode=None,
read_default_file=None,
conv=None,
use_unicode=True,
client_flag=0,
cursorclass=Cursor,
init_command=None,
connect_timeout=10,
read_default_group=None,
autocommit=False,
local_infile=False,
max_allowed_packet=16 * 1024 * 1024,
defer_connect=False,
auth_plugin_map=None,
read_timeout=None,
write_timeout=None,
bind_address=None,
binary_prefix=False,
program_name=None,
server_public_key=None,
ssl=None,
ssl_ca=None,
ssl_cert=None,
ssl_disabled=None,
ssl_key=None,
ssl_verify_cert=None,
ssl_verify_identity=None,
compress=None, # not supported
named_pipe=None, # not supported
passwd=None, # deprecated
db=None, # deprecated
): -- line 206
I had a similar issue when running a script using python 3.5.2. I followed the suggestion here https://github.com/miguelgrinberg/microblog/issues/282#issuecomment-776937071 and downgraded my pymysql, which solved the problem
pip uninstall pymysql
pip install -Iv pymysql==0.9.3

How do I get the mac-address table of a Juniper router using PyEZ? It throws an error when running

I have created a simple python script to get the vpls mac table from a rouer using an RPC command. Hower it throws an error when running. Does anybody know what I am doing wrong?
root#ubuntu:~# cat vpls3.py
#!/usr/bin/python3
from jnpr.junos import Device
from lxml import etree
import jxmlease
username='lab'
password='lab'
dev = Device(host='10.85.164.172', user=username, password=password, normalize=True)
dev.open()
#invoke the RPC command
sw = dev.rpc.get-vpls-mac-table()
print(etree.tostring(sw, encoding='unicode'))
root#ubuntu:~#
Below is the error:
root#ubuntu:~# python vpls3.py
Traceback (most recent call last):
File "vpls3.py", line 13, in <module>
sw = dev.rpc.get-vpls-mac-table()
NameError: name 'vpls' is not defined
root#ubuntu:~#
I have also tried below script:
root#ubuntu:~# cat test1.py
from jnpr.junos import Device
from lxml import etree
# Set device information with IP-address, login user and passwort
dev = Device( user='lab', host='10.85.164.172', password='lab')
# Connect to the device
dev.open()
# Get MACs
macs = dev.rpc.get-vpls-mac-table(normalize=True)
# Print response of device
print (etree.tostring(macs))
# Close the connection
dev.close()
Same error:
root#ubuntu:~# python test1.py
Traceback (most recent call last):
File "test1.py", line 11, in
macs = dev.rpc.get-vpls-mac-table(normalize=True)
NameError: name 'vpls' is not defined
root#ubuntu:~#
JunOS RPCs translate to Pyez using underscore:
change it to:
sw = dev.rpc.get_vpls_mac_table()
Thanks! its working now with sw = dev.rpc.get_vpls_mac_table() .

Error when importing CSV to postgres with python and psycopg2

I try to COPY a CSV file from a folder to a postgres table using python and psycopg2 and I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
psycopg2.ProgrammingError: must be superuser to COPY to or from a file
HINT: Anyone can COPY to stdout or from stdin. psql's \copy command also works for anyone.
I also tried to run it through the python environment as:
constr = "dbname='db_name' user='user' host='localhost' password='pass'"
conn = psycopg2.connect(constr)
cur = conn.cursor()
sqlstr = "COPY test_2 FROM '/tmp/tmpJopiUG/downloaded_xls.csv' DELIMITER ',' CSV;"
cur.execute(sqlstr)
I still get the above error. I tried \copy command but this works only in psql. What is the alternative in order to be able to execute this through my python script?
EDITED
After having a look in the link provided by #Ilja Everilä I tried this:
cur.copy_from('/tmp/tmpJopiUG/downloaded_xls.csv', 'test_copy')
I get an error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: argument 1 must have both .read() and .readline() methods
How do I give these methods?
Try using cursor.copy_expert():
constr = "dbname='db_name' user='user' host='localhost' password='pass'"
conn = psycopg2.connect(constr)
cur = conn.cursor()
sqlstr = "COPY test_2 FROM STDIN DELIMITER ',' CSV"
with open('/tmp/tmpJopiUG/downloaded_xls.csv') as f:
cur.copy_expert(sqlstr, f)
conn.commit()
You have to open the file in python and pass it to psycopg, which then forwards it to postgres' stdin. Since you're using the CSV argument to COPY, you have to use the expert version in which you pass the COPY statement yourself.
You can also use copy_from. See the code below
with open('/tmp/tmpJopiUG/downloaded_xls.csv') as f:
cur.copy_from(f, table_name,sep=',')
conn.commit()

Categories

Resources