In my Azure Function, I have specified an Environment Variable/App Setting for a database connection string. I can use the Environment Variable when I run the Function locally on my Azure Data Science Virtual Machine using VS Code and Python.
However, when I deploy the Function to Azure, I get an error: KeyValue is None, meaning that it cannot find the Environment Variable for the connection string. See error:
Exception while executing function: Functions.matchmodel Result: Failure
Exception: KeyError: 'CONNECTIONSTRINGS:PDMPDBCONNECTIONSTRING'
Stack: File "/azure-functions
host/workers/python/3.7/LINUX/X64/azure_functions_worker/dispatcher.py", line 315, in
_handle__invocation_request self.__run_sync_func, invocation_id, fi.func, args)
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/dispatcher.py",
line 434, in __run_sync_func
return func(**params)
File "/home/site/wwwroot/matchmodel/__init__.py", line 116, in main
File "/home/site/wwwroot/matchmodel/production/dataload.py", line 28, in query_dev_database
setting = os.environ["CONNECTIONSTRINGS:PDMPDBCONNECTIONSTRING"]
File "/usr/local/lib/python3.7/os.py", line 679, in __getitem__
raise KeyError(key) from None'
I have tried the following solutions:
Added "CONNECTIONSTRINGS" to specify the Environment Variable in the Python script (which made it work locally)
setting = os.environ["CONNECTIONSTRINGS:PDMPDBCONNECTIONSTRING"]
Used logging.info(os.environ) to output my Environment Variables in the console. My connection string is listed.
Added the connection string as Application Setting in the Azure Function portal.
Added the Connection String as Connection Strings in the Azure Function portal.
Does anyone have any other solutions that I can try?
Actually you are almost get the right the connection string, however you use the wrong prepended string. Further more detailed information you could refer to this doc:Configure connection strings.
Which string to use it depends on which type you choose, like in my test I use a custom type. Then I should use os.environ['CUSTOMCONNSTR_testconnectionstring'] to get the value.
From the doc you could find there are following types:
SQL Server: SQLCONNSTR_
MySQL: MYSQLCONNSTR_
SQL Database: SQLAZURECONNSTR_
Custom: CUSTOMCONNSTR_
I was struggling with this and found out this solution:
import os
setting = os.getenv("mysetting")
In Functions application settings, such as service connection strings, are exposed as environment variables during execution. You can access these settings by declaring import os and then using
setting = os.environ["mysetting"]
As Alex said, try to remove CONNECTIONSTRINGS: from the name of the environment variable. In azure portal just add mysetting in application settings as keyName.
I figured out the issue with help from George and selected George's answer as the correct answer.
I changed the code to os.environ["SQLCONNSTR_PDMPDBCONNECTIONSTRING"] but also I had been deploying the package from VS Code to Azure via the following code using Azure Command Line Interface (Azure CLI). This code overwrites Azure App Settings with the local.settings.json.
func azure functionapp publish <MY_FUNCTION_NAME> --publish-local-settings -i --overwrite-settings -y
I think that was causing changes to the type of database that I had specified in Azure App Settings (SQL Server), so when I had previously tried os.environ["SQLCONNSTR_PDMPDBCONNECTIONSTRING"] it didn't work because I was overwriting my Azure settings using my local.settings.json, which did not specify a database type.
I finally got it to work by deploying using the VS Code Extension called Azure Functions (Azure Functions: Deploy to Function App). This retains the settings that I had created in Azure App Services.
Related
I'm deploying this code https://github.com/misscoded/webinar-bolt-python-nov-2020 on my PC, but I am getting the error as shown below. I tried to remove the App initialization but still got the same error.
The code:
app = App(
signing_secret=os.environ.get('SLACK_SIGNING_SECRET'),
token=os.environ.get("SLACK_BOT_TOKEN"),
)
The error:
line 10, in <module>
app = App(
File "C:\Users\Ruba\AppData\Local\Programs\Python\Python38-32\lib\site-packages\slack_bolt\app\app.py", line 208, in __init__
self._init_middleware_list()
File "C:\Users\Ruba\AppData\Local\Programs\Python\Python38-32\lib\site-packages\slack_bolt\app\app.py", line 232, in _init_middleware_list
raise BoltError(error_token_required())
slack_bolt.error.BoltError: Either an env variable `SLACK_BOT_TOKEN` or `token` argument in the constructor is required.
Depending on your version of Windows, check your environment variables (Control Panel -> Advanced System Settings -> Environment Variables) and ensure the system is picking them up. From the error message, it looks like it's not reading them, but I may be wrong.
Also, familiarize yourself with Slack's Bolt Framework. It has something called Socket Mode which makes local development so much easier and quicker.
I am trying to retrieve findings from Google Security Center using the Python API. I have installed the python libraries, set up a service account, generated a key and when I try to get the finding or any client functions I get the following error:
Traceback (most recent call last):
File "./find.py", line 12, in <module>
finding_result_iterator = client.list_findings(all_sources)
File "/usr/local/lib/python3.6/site-packages/google/cloud/securitycenter_v1/gapic/security_center_client.py", line 1532, in list_findings
self.transport.list_findings,
AttributeError: 'str' object has no attribute 'list_findings'
I am using the code example from here:
https://cloud.google.com/security-command-center/docs/how-to-api-list-findings
Using Python 3.6, I have the json key file in the client create and my organization id. Any idea why I can't get any client functions to work?
I am passing the API Key in as the argument for authentication like this
client = securitycenter.SecurityCenterClient("gcp-sc.json")
If you have a file called gcp-sc.json with the Google credential data, either
set the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to that path, then initialize the client without configuration (SecurityCenterClient()), it'll pick that up
or if you need to explicitly name the file, SecurityCenterClient.from_service_account_json('gcp-sc.json') ought to do the trick.
You can also pass in a custom credentials object (see the docs) as SecurityCenterClient(credentials=...)
I am trying to connect to a SQL Anywhere database via python. I have created the DSN and I can use command prompt to connect to the database using dbisql - c "DNS=myDSN". When I try to connect through python using con = sqlanydb.connect(DSN= "myDSN")
I get
`Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
con = sqlanydb.connect(DSN= "RPS Integration")
File "C:\Python27\lib\site-packages\sqlanydb.py", line 522, in connect
return Connection(args, kwargs)
File "C:\Python27\lib\site-packages\sqlanydb.py", line 538, in __init__
parent = Connection.cls_parent = Root("PYTHON")
File "C:\Python27\lib\site-packages\sqlanydb.py", line 464, in __init__
'libdbcapi_r.dylib')
File "C:\Python27\lib\site-packages\sqlanydb.py", line 456, in load_library
raise InterfaceError("Could not load dbcapi. Tried: " + ','.join(map(str, names)))
InterfaceError: (u'Could not load dbcapi. Tried: None,dbcapi.dll,libdbcapi_r.so,libdbcapi_r.dylib', 0)`
I was able to resolve the problem. I was never able to use the sqlanydb.connect. I ended up using pyodbc. So my final connection string was con = pydobc.connect(dsn="myDSN"). I think that sqlanydb is only fully functional with sqlanydb 17 and I was using a previous version.
Depending on how Sybase is locally installed, it could also be that the Python module really cannot find the (correct) library when looking up dbcapi.dll in your PATH. A quick fix for that is to create a new SQLANY_API_DLL environment variable (that's the initial None in the names the error message says it tried using; it takes priority over all the others) containing the correct path, usually something like %SQLANY16%\Bin64\dbcapi.dll depending on what version you have installed (Sybase usually creates an environment variable pointing to its installation folder on a per version basis).
One way to encounter the backtrace shown by the OP is running 64-bit Python interpreter, which is unable to load the 32-bit dbcapi.dll. Try launching with "py -X-32" to force a 32-bit engine, or reinstall using a 32-bit python engine.
Unfortunately sqlanydb's code that tries to be smart about finding dbcapi also ends up swallowing the exceptions thrown during loading. The sqlanydb author probably assumed that failure to load implies file not found, which isn't always the case.
I was stoked in this same problem, both windows system32 and 64.
I'm using Python 3.9 and I found that since Python 3.8 the cdll.LoadLibrary function does no longer search PATH to find library files.
To fix this, I created the enviroment variable SQLANY_API_DLL pointing to the route of sqlanywhere installation, in my case version 17, bin32 or bin64 depending on your system.
I have written a bit of python code to interact with an Openstack instance; using the shade library.
The call
myinstance = shade.openstack_cloud(cloud='mycloud', **auth_data)
works fine on my local Ubuntu installation; but fails on our "backend" servers (running RHEL 7.2).
File "mystuff/core.py", line 248, in _create_connection
myinstance = shade.openstack_cloud(cloud='mycloud', **auth_data)
File "/usr/local/lib/python3.5/site-packages/shade-1.20.0-py3.5.egg/shade/init.py", line 106, in openstack_cloud
return OpenStackCloud(cloud_config=cloud_config, strict=strict)
File "/usr/local/lib/python3.5/site-packages/shade-1.20.0-py3.5.egg/shade/openstackcloud.py", line 312, in init
self._local_ipv6 = _utils.localhost_supports_ipv6()
File "/usr/local/lib/python3.5/site-packages/shade-1.20.0-py3.5.egg/shade/_utils.py", line 254, in localhost_supports_ipv6
return netifaces.AF_INET6 in netifaces.gateways()['default']
AttributeError: module 'netifaces' has no attribute 'AF_INET6'
The admin for that system tells me that IPv6 is not enabled there; maybe that explains the fail. I did some research, but couldn't find anything to prevent the failure.
Any thoughts are welcome.
Update: I edited my clouds.yml; and it looks like this:
# openstack/shade config file
# required to connect provisioning using the shade module
client:
force_ipv4: true
clouds:
mycloud:
auth:
user_domain_name: xxx
auth_url: 'someurl'
region_name: RegionOne
I also tried export OS_FORCE_IPV4=True - but the error message is still there.
If you go through the OpenStack os-client-config documentation, there they have mentioned about IPV6 related issue.
IPv6 is the future, and you should always use it if your cloud
supports it and if your local network supports it. Both of those are
easily detectable and all friendly software should do the right thing.
However, sometimes you might exist in a location where you have an
IPv6 stack, but something evil has caused it to not actually function.
In that case, there is a config option you can set to unbreak you
force_ipv4, or OS_FORCE_IPV4 boolean environment variable.
So, using these boolean config you can force appropriate network protocol. Adding below lines to your clouds.yaml file
client:
force_ipv4: true
will force IPV4 and hope it will resolve your issue.
Edit by OP: unfortunately the above doesn't help; fixed it by reworking shade-1.20.0-py3.5.egg/shade/_utils.py: I changed the return statement
return netifaces.AF_INET6 in netifaces.gateways()['default']`
to a simple
return False
And stuff is working. Of course, this is just a workaround; but a bug report was filed as well.
I've used boto to interact with S3 with with no problems, but now I'm attempting to connect to the AWS Support API to pull back info on open tickets, trusted advisor results, etc. It seems that the boto library has different connect methods for each AWS service? For example, with S3 it is:
conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
According to the boto docs, the following should work to connect to AWS Support API:
>>> from boto.support.connection import SupportConnection
>>> conn = SupportConnection('<aws access key>', '<aws secret key>')
However, there are a few problems I see after digging through the source code. First, boto.support.connection doesn't actually exist. boto.connection does, but it doesn't contain a class SupportConnection. boto.support.layer1 exists, and DOES have the class SupportConnection, but it doesn't accept key arguments as the docs suggest. Instead it takes 1 argument - an AWSQueryConnection object. That class is defined in boto.connection. AWSQueryConnection takes 1 argument - an AWSAuthConnection object, class also defined in boto.connection. Lastly, AWSAuthConnection takes a generic object, with requirements defined in init as:
class AWSAuthConnection(object):
def __init__(self, host, aws_access_key_id=None,
aws_secret_access_key=None,
is_secure=True, port=None, proxy=None, proxy_port=None,
proxy_user=None, proxy_pass=None, debug=0,
https_connection_factory=None, path='/',
provider='aws', security_token=None,
suppress_consec_slashes=True,
validate_certs=True, profile_name=None):
So, for kicks, I tried creating an AWSAuthConnection by passing keys, followed by AWSQueryConnection(awsauth), followed by SupportConnection(awsquery), with no luck. This was inside a script.
Last item of interest is that, with my keys defined in a .boto file in my home directory, and running python interpreter from the command line, I can make a direct import and call to SupportConnection() (no arguments) and it works. It clearly is picking up my keys from the .boto file and consuming them but I haven't analyzed every line of source code to understand how, and frankly, I'm hoping to avoid doing that.
Long story short, I'm hoping someone has some familiarity with boto and connecting to AWS API's other than S3 (the bulk of material that exists via google) to help me troubleshoot further.
This should work:
import boto.support
conn = boto.support.connect_to_region('us-east-1')
This assumes you have credentials in your boto config file or in an IAM Role. If you want to pass explicit credentials, do this:
import boto.support
conn = boto.support.connect_to_region('us-east-1', aws_access_key_id="<access key>", aws_secret_access_key="<secret key>")
This basic incantation should work for all services in all regions. Just import the correct module (e.g. boto.support or boto.ec2 or boto.s3 or whatever) and then call it's connect_to_region method, supplying the name of the region you want as a parameter.