Develop Reference

ciscoaxl api python errors

I am starting out working on looking into using Python for some Cisco CUCM automation. I found the plugin ciscoaxl here, I installed it and programed the following script:
from ciscoaxl import axl
cucm = "10.10.20.1"
username = "axlusr"
password = "password1"
version = "12.5"
ucm = axl(username, password, cucm, version)
for phone in ucm.get_phones():
print(phone.name)
I am connected to Cisco's DevNET Sandbox and all the login and configuration for the AXL user appear to be correct, however I get the following output when I attempt to run the script:
Traceback (most recent call last):
File "%home%\AppData\Local\Programs\Python\Python39\axl-test.py", line 7, in <module>
for phone in ucm.get_phones():
File "%home%\AppData\Local\Programs\Python\Python39\lib\site-packages\ciscoaxl\axl.py", line 1877, in get_phones
for each in inner(skip):
File "%home%\AppData\Local\Programs\Python\Python39\lib\site-packages\ciscoaxl\axl.py", line 1869, in inner
res = self.client.listPhone(
File "%home%\AppData\Local\Programs\Python\Python39\lib\site-packages\zeep\proxy.py", line 40, in __call__
return self._proxy._binding.send(
File "%home%\AppData\Local\Programs\Python\Python39\lib\site-packages\zeep\wsdl\bindings\soap.py", line 130, in send
return self.process_reply(client, operation_obj, response)
File "%home%\AppData\Local\Programs\Python\Python39\lib\site-packages\zeep\wsdl\bindings\soap.py", line 195, in process_reply
return self.process_error(doc, operation)
File "%home%\AppData\Local\Programs\Python\Python39\lib\site-packages\zeep\wsdl\bindings\soap.py", line 283, in process_error
raise Fault(
zeep.exceptions.Fault: Unknown fault occured
I have run it on Windows10 in an IDLE enviornment, from the Linux-Subsystem (Ubuntu 20.04) via python and ipython3.

After some additional research this is a know issue with CUCM 12.5. It should be fixed in CU1 see here: https://github.com/mvantellingen/python-zeep/issues/989

I still receive this error on 12.5.1.12900-115, but I receive it when I don't have the appropriate permissions. Fixing user permissions for AXL access resolves it.

"Private key is missing or invalid. It should be service " when making BigQuery call with credential

I'm following this data prediction using Cloud ML Engine with scikit-learn tutorial for GCP AI Platforms. I tried to make an API call to BigQuery with:
def query_to_dataframe(query):
import pandas as pd
import pkgutil
privatekey = pkgutil.get_data('trainer', 'privatekey.json')
print(privatekey[:200])
return pd.read_gbq(query,
project_id=PROJECT,
dialect='standard',
private_key=privatekey)
but got the following error:
Traceback (most recent call last):
[...]
TypeError: a bytes-like object is required, not 'str'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/.local/lib/python3.7/site-packages/trainer/task.py", line 66, in <module>
arguments['numTrees']
File "/root/.local/lib/python3.7/site-packages/trainer/model.py", line 119, in train_and_evaluate
train_df, eval_df = create_dataframes(frac)
File "/root/.local/lib/python3.7/site-packages/trainer/model.py", line 95, in create_dataframes
train_df = query_to_dataframe(train_query)
File "/root/.local/lib/python3.7/site-packages/trainer/model.py", line 82, in query_to_dataframe
private_key=privatekey)
File "/usr/local/lib/python3.7/dist-packages/pandas/io/gbq.py", line 149, in read_gbq
credentials=credentials, verbose=verbose, private_key=private_key)
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 846, in read_gbq
dialect=dialect, auth_local_webserver=auth_local_webserver)
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 184, in __init__
self.credentials = self.get_credentials()
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 193, in get_credentials
return self.get_service_account_credentials()
File "/root/.local/lib/python3.7/site-packages/pandas_gbq/gbq.py", line 413, in get_service_account_credentials
"Private key is missing or invalid. It should be service "
pandas_gbq.gbq.InvalidPrivateKeyFormat: Private key is missing or invalid. It should be service account private key JSON (file path or string contents) with at least two keys: 'client_email' and 'private_key'. Can be obtained from: https://console.developers.google.com/permissions/serviceaccounts
When the package runs in local environment, the private key loads fine, but when submitted as a ml-engine training job, the error occurs. Note that the private key fails to load only when I use GCP RUNTIME_VERSION="1.15" and PYTHON_VERSION="3.7", but can load with no problem when I use PYTHON_VERSION="2.7".
In case it's useful, the structure of my package is:
/babyweight
- setup.py
- trainer
- __init__.py
- model.py
- privatekey.json
- task.py
I'm not sure if the problem is due to a bug in Python, or where I placed privatekey.json.

I was able to solve the problem after I changed read_gbq's attribute for reading BigQuery access key from private_keys to credentials, as recommended by #rmesteves, and as shown here. I then set the value as the absolute path to privatekey.json, as shown here. Now the job is able to run without error.
Note: I only encountered this problem with Python 3+, but not with Python 2.7. I'm not sure why. It could possibly be due to the implementation of read_gbq.

How do I pull change records from ServiceNow for a specific system?

I'm trying to automate patching and need to find a way to pull change records from ServiceNow for specific servers and find out if a change window is open for that server.
I tried doing this in Ansible but so far I've had no luck. I'm wondering if there is a better way to do this? Maybe through Python or just something simple with using curl and bash? I've heard of some people having luck with rest APIs but I know very little about it.
Here is an example of my playbook using the snow_record module:
- hosts: localhost
connection: local
gather_facts: no
ignore_errors: True
tasks:
- name: Grab a user record
snow_record:
username: testuser
password: somepass
instance: myinstance
state: present
number: CHG9999
table: change_request
lookup_field: cmdb_ci.name=someservername
Error output:
The full traceback is:
Traceback (most recent call last):
File "/root/.ansible/tmp/ansible-tmp-1553017980.16-56687342300902/AnsiballZ_snow_record.py", line 113, in <module>
_ansiballz_main()
File "/root/.ansible/tmp/ansible-tmp-1553017980.16-56687342300902/AnsiballZ_snow_record.py", line 105, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/root/.ansible/tmp/ansible-tmp-1553017980.16-56687342300902/AnsiballZ_snow_record.py", line 48, in invoke_module
imp.load_module('__main__', mod, module, MOD_DESC)
File "/tmp/ansible_snow_record_payload_0p5jn8/__main__.py", line 332, in <module>
File "/tmp/ansible_snow_record_payload_0p5jn8/__main__.py", line 328, in main
File "/tmp/ansible_snow_record_payload_0p5jn8/__main__.py", line 317, in run_module
AttributeError: 'module' object has no attribute 'UnexpectedResponse'
fatal: [localhost]: FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File \"/root/.ansible/tmp/ansible-tmp-1553017980.16-56687342300902/AnsiballZ_snow_record.py\", line 113, in <module>\n _ansiballz_main()\n File \"/root/.ansible/tmp/ansible-tmp-1553017980.16-56687342300902/AnsiballZ_snow_record.py\", line 105, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/root/.ansible/tmp/ansible-tmp-1553017980.16-56687342300902/AnsiballZ_snow_record.py\", line 48, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/tmp/ansible_snow_record_payload_0p5jn8/__main__.py\", line 332, in <module>\n File \"/tmp/ansible_snow_record_payload_0p5jn8/__main__.py\", line 328, in main\n File \"/tmp/ansible_snow_record_payload_0p5jn8/__main__.py\", line 317, in run_module\nAttributeError: 'module' object has no attribute 'UnexpectedResponse'\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
I tried snow_record_find. A module I found here.
Playbook:
- hosts: mytest
tasks:
- name: Grab a user record
snow_record_find:
username: testuser
password: somepass
instance: someinstance
#state: present
#number: null??
table: change_request
query:
AND:
equals:
active: "True"
type: "standard"
u_change_stage: "80"
contains:
u_template: "MY-Template"
Error output from that:
fatal: []: FAILED! => {
"changed": false,
"module_stderr": "Shared connection to closed.\r\n",
"module_stdout": "Traceback (most recent call last):\r\n File \"/root/.ansible/tmp/ansible-tmp-1553019300.21-10858374496170/AnsiballZ_snow_record_find.py\", line 113, in <module>\r\n _ansiballz_main()\r\n File \"/root/.ansible/tmp/ansible-tmp-1553019300.21-10858374496170/AnsiballZ_snow_record_find.py\", line 105, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File \"/root/.ansible/tmp/ansible-tmp-1553019300.21-10858374496170/AnsiballZ_snow_record_find.py\", line 48, in invoke_module\r\n imp.load_module('__main__', mod, module, MOD_DESC)\r\n File \"/tmp/ansible_snow_record_find_payload_j9P62r/__main__.py\", line 260, in <module>\r\n File \"/tmp/ansible_snow_record_find_payload_j9P62r/__main__.py\", line 257, in main\r\n File \"/tmp/ansible_snow_record_find_payload_j9P62r/__main__.py\", line 249, in run_module\r\n File \"/usr/lib/python2.7/site-packages/pysnow/legacy_request.py\", line 91, in _all_inner\r\n yield self._get_content(response)\r\n File \"/usr/lib/python2.7/site-packages/pysnow/legacy_request.py\", line 332, in _get_content\r\n server_error['summary'], server_error['details']\r\npysnow.legacy_exceptions.UnexpectedResponse: Unexpected HTTP GET response code. Expected 200, got 401\r\n",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",

Unfortunately, what you're attempting is not easy. There is not an existing API in ServiceNow that will return the next available maintenance window for a given Configuration Item (CI / Server).
You could reverse engineer the Maintenance schedule functionality and create your own Scripted API on the ServiceNow platform to complete, but that is likely not worth the effort.
Just FYI, you can use the REST API Explorer in ServiceNow to get code snippets for various languages, Python included.
And finally, there's some good REST Web Service tutorials available as a part of ServiceNow's Developer program, through which you're also able to get a free dedicated ServiceNow instance to play with.

Hive Server 2 error on python connect with hiveserver2

I am using
Centos , Python2.7 , hive 2.1 ,Hadoop 2.7.2 ,pyHive
here is code
from pyhive import hive
from TCLIService.ttypes import TOperationState
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)
#status = cursor.poll().operationState
#while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNI$
# logs = cursor.fetch_logs()
# for message in logs:
# print message
# If needed, an asynchronous query can be cancelled at any time with:
# cursor.cancel()
# status = cursor.poll().operationState
#print cursor.fetchall()
when I run python /usr/local/py/test5.py in terminal its showing ....
Traceback (most recent call last):
File "/usr/local/py/test5.py", line 3, in <module>
cursor = hive.connect('localhost').cursor()
File "/usr/local/lib/python2.7/site-packages/pyhive/hive.py", line 63, in connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/pyhive/hive.py", line 104, in __init__
self._transport.open()
File "/usr/local/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 80, in open
status, payload = self._recv_sasl_message()
File "/usr/local/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 98, in _recv_sasl_message
header = read_all_compat(self._trans, 5)
File "/usr/local/lib/python2.7/site-packages/thrift_sasl/six.py", line 31, in <lambda>
read_all_compat = lambda trans, sz: trans.readAll(sz)
File "/usr/local/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 120, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
Hive server error log showing after this...
ERROR [HiveServer2-Handler-Pool: Thread-41]: server.TThreadPoolServer (:()) - Thrift error occur$
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Also I had tried pyhs2 getting same error
what was going wrong ?
Thanks

I hava solved this error with these versions :
Centos 7 , Python2.7 , hive 2.1 ,Hadoop 2.7.3 and java java 1.7.0_91
using impyla
its working for me.

thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
I had the same problem and solved it by
Start hiveserver2 (how to start hiveserver2)
Modify your port. The default port is 10000 or you can change it to follow your hive_site.xml

PyHive is Hanging on Connection -- Thrift_sasl appears to be the issue

from pyhive import hive
import thrift_sasl
connection = hive.Connection(host='myhost', port=10000, database='local')
#hangs here
from sqlalchemy import create_engine
engine = create_engine('hive://myhost:10000/local')
logs = Table('mytable', MetaData(bind=engine), autoload=True)
#also hangs here
Both of these snippets will hang for me.
Hitting ctrl+c stops the execution here:
^CTraceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/apps/Python/lib/python2.7/site-packages/pyhive/hive.py", line 86, in __init__
self._transport.open()
File "thrift_sasl.py", line 74, in open
status, payload = self._recv_sasl_message()
File "thrift_sasl.py", line 92, in _recv_sasl_message
header = self._trans.readAll(5)
File "/apps/Python/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz - have)
File "/apps/Python/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 105, in read
buff = self.handle.recv(sz)
KeyboardInterrupt
I am using Hive 0.12 and HiveServer2. I can connect to it using the python Hive library provided with Hadoop (.../hive/lib/py) but cannot do so with pyhive, which uses thrift_sasl.
Some people not using the thrift_sasl module suggest turning off SASL support in hive-site.xml via:
<property><name>hive.server2.authentication</name><value>NOSASL</value></property>
However after trying this the code still hanged with the same stack trace when I issued a KeyboardInterrupt.