Develop Reference

Recursive view error while using spark 3.2.0 version

This happens while replacing temp view using below code block
latest_data.createOrReplaceGlobalTempView("e_test")
ideally this command should replace the view if e_test already exists instead it is throwing
"Recursive view `global_temp`.`e_test` detected (cycle: `global_temp`.`e_test` -> `global_temp`.`e_test`)" error..
Can someone help me with this
codeblock

It works with 3.0.1 but looks like the latest version has a bug

XLWings - app.display_alerts = False not suppressing bad formula error message

I am using Python 3.8.3 and XLWings 0.19.5. I am trying to write a try...catch while running excel equations directly in a workbook, but equations with certain errors are causing the application to hang. I think this is due to a message box popping up.
import xlwings as xw
appExcel = xw.apps.add()
appExcel.display_alerts = False
appExcel.screen_updating = False
wbEquation = xw.Book()
wbEquation.sheets.add(name='Calculate')
wsEquation = wbEquation.sheets['Calculate']
badFormula = "=A1+(A2+A3"
try:
wsEquation.range('B1').formula = badFormula
except Exception:
appExcel.quit()
raise Exception("There was an error when running the equation.")
With display_alerts=False I am not sure why it is getting hung on the wsEquation.range('B1').formula = badFormula line, I believe due to a There is a problem with a formula error. On the older version of XLWings 0.10.0 I was running it does not get hung up and moves past the error message when I try and execute this equation.
*Edit: So I tested to see which version exactly breaks it and I was able to keep it working through 0.11.5. Upgrading to 0.11.6 broke it. I do not see anything in the release notes between the two versions suggesting why that would be.

So for right now I am fixing by rolling back to 0.11.5:
pip install "XLWings==0.11.5"
I have put in a bug report: link and it was marked as a bug so I think they will be working on it.

bingads V13 report request fails in python sdk

I try to download a bingads report using python SDK, but I keep getting an error says: "Type not found: 'Aggregation'" after submitting a report request. I've tried all 4 options mentioned in the following link:
https://github.com/BingAds/BingAds-Python-SDK/blob/master/examples/v13/report_requests.py
Authentication process prior to request works just fine.
I execute the following:
report_request = get_report_request(authorization_data.account_id)
reporting_download_parameters = ReportingDownloadParameters(
report_request=report_request,
result_file_directory=FILE_DIRECTORY,
result_file_name=RESULT_FILE_NAME,
overwrite_result_file=True, # Set this value true if you want to overwrite the same file.
timeout_in_milliseconds=TIMEOUT_IN_MILLISECONDS
)
output_status_message("-----\nAwaiting download_report...")
download_report(reporting_download_parameters)
after a careful debugging, it seems that the program fails when trying to execute a command within "reporting_service_manager.py". Here is workflow:
download_report(self, download_parameters):
report_file_path = self.download_file(download_parameters)
then:
download_file(self, download_parameters):
operation = self.submit_download(download_parameters.report_request)
then:
submit_download(self, report_request):
self.normalize_request(report_request)
response = self.service_client.SubmitGenerateReport(report_request)
SubmitGenerateReport starts a sequence of events ending with a call to "_SeviceCall.init" function within "service_client.py", returning an exception "Type not found: 'Aggregation'"
try:
response = self.service_client.soap_client.service.__getattr__(self.name)(*args, **kwargs)
return response
except Exception as ex:
if need_to_refresh_token is False \
and self.service_client.refresh_oauth_tokens_automatically \
and self.service_client._is_expired_token_exception(ex):
need_to_refresh_token = True
else:
raise ex
Can anyone shed some light? .
Thanks

Please be sure to set Aggregation e.g., as shown here.
aggregation = 'Daily'
If the report type does not use aggregation, you can set Aggregation=None.
Does this help?

This may be a bit late 2 months after the fact but maybe this will help someone else. I had the same error (though I suppose it may not be the same issue). It does look like you did what I did (and I'm sure others will as well): copy-paste the Microsoft example code and tried to run it only to find that it didn't work.
I spent quite some time trying to debug the issue and it looked to me like the XML wasn't being searched correctly. I was using suds-py3 for the script at the time so I tried suds-community and everything just worked after that.
I also re-read the Bing Ads API walkthrough for getting started again and found that they recommend suds-jurko instead.
Long story short: If you want to use the bingads API don't use suds-py3, use either suds-community (which I can confirm works for everything I've used the API for) or suds-jurko (which is the one recommended by Microsoft).

Error Message when trying to delete features in Feature Service Python API 1.7 for ArcGIS

I am trying to delete features using the delete_features method on the FeatureLayer Object and I keep getting the following error: "This SqlTransaction has completed; it is no longer usable."
The code is below. The error message seems to populate in the last line where="OBJECTID >=0", but I'm not a 100 sure if this is the problem. Unfortunately I'm not very good at programming.
gis = arcgis.GIS("http://gfcgis.maps.arcgis.com", "UserName", "Password")
feature_layer_item = gis.content.search(FeatureLayer, item_type = 'Feature Service')[0]
flayers = feature_layer_item.layers
flayer = flayers[0]
flayer.delete_features(where="OBJECTID >= 0", rollback_on_failure=True)
Any help would be greatly appreciated.
Michael

This sounds like a zombie transaction. Check with your DBA if there's a query, most likely a Stored Procedure, which is being called when your code runs. This message usually shows up when the application code tries to do a commit on the DB after the SP has already committed.
That's the SQL Transaction which has already completed.

Come to find out, it was a simple syntax error causing the issue. I didn't put quotations around 'True' for the rollback_on_failure parameter.

How to catch specific Postgres exceptions in Python psycopg2

Default psycopg2 error messages are too broad. Most of the time it simply throws:
psycopg2.OperationalError
without any additional information. So it is hard to guess, what was the real reason of the error - either incorrect user credentials, or simply the fact the server is not running. So, I need some more appropriate error handling, like error codes in pymysql library.
I've seen this page, but it does not help. When I do
except Exception as err:
print(err.pgcode)
it always prints None. And as for errorcodes it is simply undefined. I tried to import it, but failed. So, I need some help.

For anyone looking a quick answer:
Short Answer
import traceback # Just to show the full traceback
from psycopg2 import errors
InFailedSqlTransaction = errors.lookup('25P02')
try:
feed = self._create_feed(data)
except InFailedSqlTransaction:
traceback.print_exc()
self._cr.rollback()
pass # Continue / throw ...
Long Answer
Go to psycopg2 -> erros.py you will find the lookup function lookup(code).
Go to psycopg2\_psycopg\__init__.py in sqlstate_errors you will find all codes to add as string into the lookup function.

This is not an answer to the question but merely my thoughts that don't fit in a comment. This is a scenario in which I think setting pgcode in the exception object would be helpful but, unfortunately, it is not the case.
I'm implementing a pg_isready-like Python script to test if a Postgres service is running on the given address and port (e.g., localhost and 49136). The address and port may or may not be used by any other program.
pg_isready internally calls internal_ping(). Pay attention to the comment of Here begins the interesting part of "ping": determine the cause...:
/*
* Here begins the interesting part of "ping": determine the cause of the
* failure in sufficient detail to decide what to return. We do not want
* to report that the server is not up just because we didn't have a valid
* password, for example. In fact, any sort of authentication request
* implies the server is up. (We need this check since the libpq side of
* things might have pulled the plug on the connection before getting an
* error as such from the postmaster.)
*/
if (conn->auth_req_received)
return PQPING_OK;
So pg_isready also uses the fact that an error of invalid authentication on connection means the connection itself is already successful, so the service is up, too. Therefore, I can implement it as follows:
ready = True
try:
psycopg2.connect(
host=address,
port=port,
password=password,
connect_timeout=timeout,
)
except psycopg2.OperationalError as ex:
ready = ("fe_sendauth: no password supplied" in except_msg)
However, when the exception psycopg2.OperationalError is caught, ex.pgcode is None. Therefore, I can't use the error codes to compare and see if the exception is about authentication/authorization. I'll have to check if the exception has a certain message (as #dhke pointed out in the comment), which I think is kind of fragile because the error message may be changed in the future release but the error codes are much less likely to be changed, I think.