ERROR:py4j.java_gateway:There was an exception while executing the Python Proxy on the Python Side.
Traceback (most recent call last):
File "/databricks/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 2442, in _call_proxy
return_value = getattr(self.pool[obj_id], method)(*params)
File "/local_disk0/tmp/1662349653615-0/PythonShell.py", line 1528, in getNewDatasetInfosJson
return self.shell.user_ns.get_new_dataframe_infos_json()
File "/local_disk0/tmp/1662349653615-0/PythonShell.py", line 685, in get_new_dataframe_infos_json
for idx, (df_name, df) in enumerate(self.new_dataframes.items()):
RuntimeError: OrderedDict mutated during iteration
I am getting this error when i am running databricks notebook
Related
I am trying to run a click.confirm() command. It works when I run the file stand-alone, but when I call the function from another module I get a SystemError
Traceback (most recent call last):
File "/home/usr/.local/lib/python3.6/site-packages/click/_compat.py", line 108, in __getattr__
return getattr(self._stream, name)
File "/home/usr/.vscode-server/extensions/ms-python.python-2022.6.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_io.py", line 59, in __getattr__
raise AttributeError(name)
AttributeError: closed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/usr/.local/lib/python3.6/site-packages/click/termui.py", line 231, in confirm
echo(prompt.rstrip(" "), nl=False, err=err)
File "/home/usr/.local/lib/python3.6/site-packages/click/utils.py", line 298, in echo
file.write(out) # type: ignore
SystemError: <built-in method write of _NonClosingTextIOWrapper object at 0x7f20e85ea048> returned a result with an error set
I run my program on a computer over SSH using the vscode extension
I am trying to create connection to Hive hosted in HDInsight cluster through my python script and getting below error-
Traceback (most recent call last):
File "ClassLoader.java", line 357, in java.lang.ClassLoader.loadClass
File "Launcher.java", line 349, in sun.misc.Launcher$AppClassLoader.loadClass
File "ClassLoader.java", line 424, in java.lang.ClassLoader.loadClass
File "URLClassLoader.java", line 382, in java.net.URLClassLoader.findClass
java.lang.ClassNotFoundException: java.lang.ClassNotFoundException: org.apache.thrift.transport.TTransportException
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "org.jpype.JPypeContext.java", line 330, in org.jpype.JPypeContext.callMethod
File "Method.java", line 498, in java.lang.reflect.Method.invoke
File "DelegatingMethodAccessorImpl.java", line 43, in sun.reflect.DelegatingMethodAccessorImpl.invoke
File "NativeMethodAccessorImpl.java", line 62, in sun.reflect.NativeMethodAccessorImpl.invoke
File "NativeMethodAccessorImpl.java", line -2, in sun.reflect.NativeMethodAccessorImpl.invoke0
File "DriverManager.java", line 247, in java.sql.DriverManager.getConnection
File "DriverManager.java", line 664, in java.sql.DriverManager.getConnection
File "HiveDriver.java", line 105, in org.apache.hive.jdbc.HiveDriver.connect
Exception: Java Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "s.py", line 5, in <module>
"/root/jdbc/hive-jdbc-1.2.1000.2.6.5.3009-43.jar")
File "/usr/local/lib64/python3.6/site-packages/jaydebeapi/__init__.py", line 412, in connect
jconn = _jdbc_connect(jclassname, url, driver_args, jars, libs)
File "/usr/local/lib64/python3.6/site-packages/jaydebeapi/__init__.py", line 230, in _jdbc_connect_jpype
return jpype.java.sql.DriverManager.getConnection(url, *dargs)
java.lang.NoClassDefFoundError: java.lang.NoClassDefFoundError: org/apache/thrift/transport/TTransportException
My Script is -
import jaydebeapi
conn = jaydebeapi.connect("org.apache.hive.jdbc.HiveDriver",
"jdbc:hive2://10.20.30.40:10001/default;transportMode=http;ssl=false;httpPath=/hive2",
["username", "password"],
"/root/jdbc/hive-jdbc-1.2.1000.2.6.5.3009-43.jar")
I have exported CLASSPATH wil all jar files.
The error is java.lang.ClassNotFoundException: java.lang.ClassNotFoundException which specifies that the execution is not able to find the jar /root/jdbc/hive-jdbc-1.2.1000.2.6.5.3009-43.jar.I believe it's only placed in a host from where you are executing the code. I would suggest placing the jar file in the same directory structure in all the nodes in the cluster and have a check on permissions so that the user executing the job has access to that path.
I am trying to set up the ChirpSDK, but every time I configure and run the code, I get this error:
Traceback (most recent call last):
File "test.py", line 3, in <module>
chirp = ChirpSDK()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/chirpsdk/chirpsdk.py", line 395, in __init__
self.read_chirprc(block)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/chirpsdk/chirpsdk.py", line 501, in read_chirprc
raise IOError('Could not find a ~/.chirprc file')
OSError: Could not find a ~/.chirprc file
Exception ignored in: <function ChirpSDK.__del__ at 0x10fa31af0>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/chirpsdk/chirpsdk.py", line 422, in __del__
self.close()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/chirpsdk/chirpsdk.py", line 470, in close
if self._sdk:
AttributeError: 'ChirpSDK' object has no attribute '_sdk'
I realize that the error is saying that my .chirprc file is not being recognized, but I have no idea how to remedy this. I created a .chirprc file in my /Users/username/ path, and named it c.chirprc (as the Chirp getting started article suggests), but I am still getting this error. Is there another part that I am missing? Am I reading the instructions wrong?
Thanks
The Chirp configuration file should be placed at /Users/<username>/.chirprc on macOS.
If you run ls -l ~/.chirprc in the terminal, do you get any results? If it displays no such file or directory then you have not created the file correctly.
I'm trying to create an xlsm file using xlwings. Or openpyxl if not possible with xlwings.
I'm on Mac so I can't use PyWin32.
I'm running the following python code:
import xlwings as xw
b = xw.Book()
b.save('test_book2.xlsm')
When I run that code I receive the message:
When I click 'yes' to that message I receive the following error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/aeosa/appscript/reference.py", line 482, in __call__
return self.AS_appdata.target().event(self._code, params, atts, codecs=self.AS_appdata).send(timeout, sendflags)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/aeosa/aem/aemsend.py", line 92, in send
raise EventError(errornum, errormsg, eventresult)
aem.aemsend.EventError: Command failed: Parameter error. (-50)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/matthewbell/Desktop/test.py", line 4, in <module>
b.save('test_book2.xlsm')
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/xlwings/main.py", line 704, in save
return self.impl.save(path)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/xlwings/_xlmac.py", line 244, in save
self.xl.save_workbook_as(filename=hfs_path, overwrite=True)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/aeosa/appscript/reference.py", line 518, in __call__
raise CommandError(self, (args, kargs), e, self.AS_appdata) from e
appscript.reference.CommandError: Command failed:
OSERROR: -50
MESSAGE: Parameter error.
COMMAND: app(pid=6198).workbooks['Book1'].save_workbook_as(filename='Macintosh HD:Users:matthewbell:Desktop:test_book2.xlsm', overwrite=True)
What can I do to create an xlsm file?
I have a Pandas dataframe and want to compute bigrams with the following code:
from nltk import bigrams
df['tweet_bigrams'] = df['tweet_tokenized'].apply(lambda x: list(bigrams(x)))
It was working fine in Jupyter. However, when I tried to run it on Linux terminal, I keep receiving the following error:
Traceback (most recent call last):
File "/usr/licensed/anaconda3/5.3.1/lib/python3.7/site-packages/nltk/util.py", line 468, in ngrams
history.append(next(sequence))
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "url_tweet_feature_extraction.py", line 143, in <module>
df['tweet_bigrams'] = df['tweet_tokenized'].apply(lambda x: list(bigrams(x)))
File "/usr/licensed/anaconda3/5.3.1/lib/python3.7/site-packages/pandas/core/series.py", line 3194, in apply
mapped = lib.map_infer(values, f, convert=convert_dtype)
File "pandas/_libs/src/inference.pyx", line 1472, in pandas._libs.lib.map_infer
File "url_tweet_feature_extraction.py", line 143, in <lambda>
df['tweet_bigrams'] = df['tweet_tokenized'].apply(lambda x: list(bigrams(x)))
File "/usr/licensed/anaconda3/5.3.1/lib/python3.7/site-packages/nltk/util.py", line 491, in bigrams
for item in ngrams(sequence, 2, **kwargs):
RuntimeError: generator raised StopIteration
Any idea on how to resolve this?
Update your NLTK. You need version 3.4 (or higher, for future readers). Old versions relied on StopIteration handling that changed in Python 3.7.