Develop Reference

Python client for elasticsearch 8.5

I used to connect elasticsearch 7 self managed cluster using following code.
from elasticsearch import Elasticsearch,RequestsHttpConnection
es = Elasticsearch(['hostname'], timeout=1000,http_auth=('user_name', 'password'),use_ssl=True,verify_certs=True,connection_class=RequestsHttpConnection,scheme="https",port=9200)
After updating the Elasticsearch to 8.5 most of the parameters were invalid. Need a help to figure out the correct way to connect elastic cluster in elastic search 8.5.

In Elasticsearch 8.X, there have been significant changes in the Elasticsearch API.
Now, in the Elasticsearch 8.X, the scheme and port need to be included explicitly as part of the hostname, scheme://hostname:port e.g.(https://localhost:9200)
The http_auth should be updated to basic_auth instead. You can have a look at all the additional available options here. So the new snippet to connect would be something like -
es = Elasticsearch(['https://hostname:port'], timeout=1000 ,basic_auth=('user_name', 'password'),verify_certs=True)
There are significant changes in the requests/responses while doing querying as well, so I would suggest giving this a read.
If you are doing migration to elasticsearch 8.X from 7.X, for short-term, there is another workaround as well which would not require any code changes and just setting the ELASTIC_CLIENT_APIVERSIONING=1 env variable in your python application.
Enable compatibility mode and upgrade Elasticsearch
Upgrade your Elasticsearch client to 7.16:
$ python -m pip install --upgrade 'elasticsearch>=7.16,<8
If you have an existing application enable the compatibility mode by setting ELASTIC_CLIENT_APIVERSIONING=1 environment variable. This will instruct the Elasticsearch server to accept and respond with 7.x-compatibile requests and responses.
https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/migration.html#migration-compat-mode

Is there any way to import Azure Blob Services Package in Snowpark

I've created the python script which read the excel file from azure blob storage and parse into multiple CSV files. The script is working fine in python. But I try to run the same file using snowpark it throws the error as "Modulenotfound" for azure blob service package

First thing to check if the package is supported:
Displaying Available Packages
You can display all packages available and their version information by querying the PACKAGES view in the Information Schema.
select * from information_schema.packages where language = 'python';
Second, accessing network or file system is restricted:
Downloading Data On Demand From Data Science Libraries
However, on-demand downloading does not work with Python UDFs due to Snowflake security constraints, which disable some capabilities, such as network access and writing to files.
To work around this issue, download the data to your local environment and then provide it to the UDF via a Snowflake stage.
Following Good Security Practices
Network access.
Because your code cannot access the network directly or indirectly, you cannot use the code in the Snowflake Python Connector to access the database. Your UDF cannot itself act as a client of Snowflake.

PowerBI Service Refresh with python script not working

A very quick question.
I have built out a PBI Dashboard which runs some python script in the query editor. It works fine on my desktop but when uploaded to PBI Service it errors when the dataset is refreshed.
Data source error: Unable to refresh the model (id=853989) because it references an unsupported data source.
I have read online that since 2019 (I think) PowerBI service now supports python scripting?
Thanks

Data Pipeline using SQl and Python

I need to create a data pipeline using Python. I want to connect with MySql in Python and read the tables in dataframes, perform pre-processing and then load the data back to Mysql Db. I was able to connect to the MySql Db using mysql connector and then pre-process the dataframes. However, I'm not able to load these dataframes from Python back to Mysql. Error: ValueError: unknown type str96 python.
Please help me with methods to complete this task.
I'm new to programming. Any help will be greatly appreciated. Thanks!

It is a bug and has been fixed in version 1.1.3.
upgrade pandas package
pip3 install --upgrade pandas

How to run a python script on images present in firebase?

I have a image in my firebase acccount and I want to run a python script on this image and get a result back.
Can someone suggest me a simple way to do this?
I tried hosting the python file in heroku. Fetching the image from firebase to heroku and running the python script would be an overhead.
Is there a simpler way to run the python script in firebase itself?

There are a handful of Python wrappers for Firebase but some have not been updated in awhile. Try this Getting Started with Pyrebase, then try this tutorial for Getting started on Heroku with Python.