"Spark 2.4.3 S3 MultiObjectDeleteException " in sparklyr - python

I am really struggling with this error which I keep getting whenever I try to do the following . And I tried all the suggestions provided in the old links as well , but in vain.
spark_write_csv(final_data_3,path = "mypath",header = T,mode = "overwrite",infer_schema=F)
This is the error I get
Spark 2.4.0 S3 MultiObjectDeleteException
I was reading through old answers for similar problem and it suggested it should below configuration in settings to avoid this error .
I added this and I also used true in place of false in script , still am getting same error .
hconf %>% sparklyr::invoke("set","fs.s3a.multiobjectdelete.enable","false")
Any suggestions to address this error ?

Related

Python : "TypeError","evalue":"'list' object is not callable"

I have python code which keeps giving below error. To the surprise it keeps coming in one environment (QA) but does not happen in another (DEV) inspite of using exact same code and input file
"TypeError","evalue":"'list' object is not callable"
I have below lines related to python list in my code -
country_list = list(input_df.select('country').distinct().toPandas()['country'])
country_list = list(filter(None, country_list))
country_code = str(country_list).strip('[]')
Which looks fine to me and works in DEV environment without any issue. What could be the possible reason for this. Can this be something to with python versions?
Thanks in advance

JupyterLab keeps throwing Name is not defined error

I'm a Python beginner and am trying out JupyterLab. I tried a simple line of code but keep getting an error a NameError message: name 'val' is not defined. I'm using JupyterLab 2.1.3 on a localhost (Win 10 Enterprise). Any help?
In the picture, the [] beside your code cell indicates that the cell defining val = ... has not been executed. If you run the cell, [] would change to [2] based on your picture.
Simply put, run the cell above first before running the cell you are having error in now.
Or, you can simply combine them:
val = 4+2+4
val
Run this and it will give you what you want.
Thank you everyone for helping. I found out that my original line of code was written in Raw mode that's why it showed an error. Setting it to Code mode fixed it. Sorry I didn't capture the entire screen.

create vm on opennebula with python api

As the title says, i am trying to create a vm on opennebula with the python api. (this is my first time working with one and i have zero clue of anythings, so plz be not too hard with me)
#!/usr/bin/python3
import oca
client = oca.Client('username:passwd', 'ip:2633/RPC2')
oca.VirtualMachine.allocate(client, 'test')
#'test' is the name of the template i want to use
it gives me this error:
oca.exceptions.OpenNebulaException: [VirtualMachineAllocate] Error allocating a new virtual machine. Parse error: syntax error, unexpected $end, expecting EQUAL or EQUAL_EMPTY at line 1, columns 1:5
anyone an idea how to fix this?
thanks in advance.
ok.. i am stupid and have searched only after 4h trying in the right place --> the git repository and the 'issues' tab.
vm_templ = oca.VmTemplatePool(client)
vm_templ.info()
for templ in vm_templ:
if templ.name == "Ubuntu 14.04 - KVM":
templ.instantiate("oca-test")
that is a way you can create a vm of an existing template via python.
(the connection part is missing, but that is trivial).
i don't delete this question, perhaps somebody out there will have a hard time too with this.

Tableau SDK TableException (40200)

Issue: Error being thrown: tableausdk.Exceptions.TableauException: TableauException (40200): The system cannot find the path specified.
- OS::mkdir(CreateDirectory path="C:\PATH\Tableau-SDK\tdetmp2A0E0E5E")
I am attempting to to create a tableau extract from oracle data using python and the tableauSDK.
The code seems to run correctly if the extract already exists. (although the produced tde is unreadable)
According to the Tableau community I should be able to create an extract from any source data without the extract already existing...
Any idea on why this is occuring?
tde_path = r'C:\PATH\test.tde'
tde_file = Extract(path=tde_path) ## ERROR Thrown here
The reason now seems obvious...
The error had the answer :
OS::mkdir(CreateDirectory path="C:\PATH\Tableau-SDK\tdetmp2A0E0E5E")
To solve the issue :
The Directory C:\PATH\Tableau-SDK\ did not exist.
Created the Directory and the code ran without error.

pandas read_csv working only as root user

I am reading a csv file using pandas. It works fine if I run script as root user. But when I try to run it with different user it does not read data and gives:
error : KeyError: 'no item named 0'
it appears at:
dt = pd.read_csv('rt.csv', header=None).fillna('').set_index(0).to_dict()[1]
Btw, I am working on Ubuntu 12.02 and using anaconda, which is installed in root user and other user as well (which is giving error)
Please help.
You like have different pandas versions installed as user and root.
I get the same error with version 0.16.2 when I use the wrong delimiter.
Have a look at your data in rt.csv.
For example, this would work for a whitespace-delimited file:
dt = pd.read_csv('rt.csv', header=None,
delim_whitespace=True).fillna('').set_index(0).to_dict()[1]
Check the file and adapt the delimiter accordingly.

Categories

Resources