I have written an azure function, currently the azure function is acting as Webhook Consumer. The job of the function is to read Webhook events and save into azure storage.
I am using an HTTP trigger template to get the job done. I am able to receive the events from the Webhook, but when I try to write to azure storage it is giving me below error.
I tried the option mentioned in this post, but no luck still getting the same error.
System.InvalidOperationException: Storage account connection string 'AzureWebJobs<AzureStorageAccountName>' does not exist. Make sure that it is a defined App Setting.
Below is my function.json file
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "blob",
"name": "outputblob",
"path": "test/demo",
"direction": "out",
"connection": "<AzureStorageAccountName>"
}
]
}
init.py
import logging
import azure.functions as func
def main(req: func.HttpRequest,outputblob: func.Out[str]) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
name = 'some_name'
if not name:
try:
req_body = 'req_body_test'#req.get_json()
except ValueError:
pass
else:
name = 'name'#req_body.get('name')
print(str(req.get_json()))
outputblob.set(str(req.get_json()))
Please make sure you have already add the connection string to the local.settings.json on local or configuration settings on azure.
Please test below code and settings files:
__init__.py
import logging
import azure.functions as func
def main(req: func.HttpRequest,outputblob: func.Out[func.InputStream]) -> func.HttpResponse:
outputblob.set("this is a test.")
return func.HttpResponse(
"Test.",
status_code=200
)
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"name": "outputblob",
"type": "blob",
"path": "test/demo",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
}
]
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "",
"FUNCTIONS_WORKER_RUNTIME": "python",
"MyStorageConnectionAppSetting":"DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net"
}
}
On azure:
Related
So I have created a function app and now I am trying to create an API that connects to my Blob Storage and "Posts" the content within the container
import azure.functions as func
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
name = req.params.get('name')
if not name:
try:
req_body = req.get_json()
except ValueError:
pass
else:
name = req_body.get('name')
if name:
return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")
else:
return func.HttpResponse(
"This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
status_code=200
)
Thank you Anupam Chand. Posting your suggestions as an answer to help other community members.
The Output Binding allows you to connect to the blob storage using azure function.
Below is the sample code in how to connect to the Blob storage using output and input bindings.
{
"bindings": [
{
"queueName": "myqueue-items",
"connection": "MyStorageConnectionAppSetting",
"name": "queuemsg",
"type": "queueTrigger",
"direction": "in"
},
{
"name": "inputblob",
"type": "blob",
"dataType": "binary",
"path": "samples-workitems/{queueTrigger}",
"connection": "MyStorageConnectionAppSetting",
"direction": "in"
},
{
"name": "outputblob",
"type": "blob",
"dataType": "binary",
"path": "samples-workitems/{queueTrigger}-Copy",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
}
],
"disabled": false,
"scriptFile": "__init__.py"
}
For related information check Azure Blob output bindings.
Is it possible to access the values of function.json binding expressions from within a Python Azure function?
This is what I want to do:
function.json:
{
"scriptFile": "__init__.py",
"disabled": false,
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "samples-workitems/{folder1}/{folder2}/",
"connection":"MyStorageAccountAppSetting"
}
]
}
__init__.py:
import logging
import azure.functions as func
def main(myblob: func.InputStream):
logging.info('Python blob folder 1: %s', myblob.folder1)
logging.info('Python blob folder 2: %s', myblob.folder2)
This Example makes it look like it should work, but in reality it seems like the name variable is hard coded.
To use the return value of a function as the value of an output binding, the name property of the binding should be set to $return in function.json.
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "req",
"direction": "in",
"type": "httpTrigger",
"authLevel": "anonymous"
},
{
"name": "msg",
"direction": "out",
"type": "queue",
"queueName": "outqueue",
"connection": "AzureWebJobsStorage"
},
{
"name": "$return",
"direction": "out",
"type": "http"
}
]
}
For further information check Binding expressions
Is there any chance to read a bunch of messages in my python Azure Function (using the serviceBusTrigger)?
The question is fair simple as it is. I did many tries but it seems that the Azure Function Framework after reading the messages coming from Azure Service Bus, calls the trigger function one message at time, no matter how great the prefetch value is or any parameter is set.
I would like to handle the entire fetch of messages, because when you have to handle million of messages, one at time or many at time makes a difference.
This is my configuration:
function.json
{
"scriptFile": "main.py",
"entryPoint": "function_handler",
"bindings": [{
"name": "msg",
"type": "serviceBusTrigger",
"direction": "in",
"topicName": "topic-name",
"subscriptionName": "topic-subscription-name"
}]
}
main.py
import azure.functions as func
import json
def function_handler(msg: func.ServiceBusMessage):
print(str(msg))
host.json
{
"version": "2.0",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
},
"extensions": {
"serviceBus": {
"prefetchCount": 100,
"messageHandlerOptions": {
"autoComplete": true,
"maxConcurrentCalls": 6,
"maxAutoRenewDuration": "00:05:00"
}
}
}
}
I think you may be need a async service bus trigger:
__init__.py
import logging
import asyncio
import azure.functions as func
async def hello():
await asyncio.sleep(2)
async def main(msg: func.ServiceBusMessage):
logging.info('Python ServiceBus queue trigger processed message: %s',
msg.get_body().decode('utf-8'))
await asyncio.gather(hello(), hello())
logging.info(msg.get_body().decode('utf-8')+" is been done.")
host.json
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
},
"extensions": {"serviceBus": {"messageHandlerOptions": {"maxConcurrentCalls": 5}}}
}
And I can get:
Below is the code,
import logging
import json
import urllib.request
import urllib.parse
import azure.functions as func
def main(myblob: func.InputStream):
logging.info(f"Python blob trigger function processed blob \n"
f"Name: {myblob.name}\n"
f"Blob Size: {myblob.length} bytes")
response = urllib.request.urlopen("http://example.com:5000/processing")
return {
'statusCode': 200,
'body': json.dumps(response.read().decode('utf-8'))
}
Error: Result: Failure Exception: RuntimeError: function 'abc' without a $return binding returned a non-None value Stack: File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/dispatcher.py", line 341, in _handle__invocation_request f'function {fi.name!r} without a $return binding '. The same code works in lambda.. Please help me in debugging in azure functions.
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "sourcemetadata/{name}",
"connection": "AzureWebJobsStorage"
}
]
}
In the Azure function, if you use return in the function app code, it means that you want to use output binding. But you do not define it in function.json. Please define it. For more details, please refer to here and here
For example
I use process blob with blob trigger and send message to azure queue with queue output binding
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "test/{name}.csv",
"connection": "AzureWebJobsStorage"
},
{
"name": "$return",
"direction": "out",
"type": "queue",
"queueName": "outqueue",
"connection": "AzureWebJobsStorage"
}
]
}
Code
async def main(myblob: func.InputStream) :
logging.info(f"Python blob trigger function processed blob \n"
f"Name: {myblob.name}\n")
return "OK"
I am trying to setup access to blob storage using a python function app but the file name is received from a post request not preset. The http trigger part works but i'm having trouble accessing files in my blob storage. This is my json:
{
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"post",
"get"
]
},
{
"name": "inputblob",
"type": "blob",
"path": "sites/{httpTrigger}",
"connection": "STORAGE",
"direction": "in"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
],
"disabled": false
}
I saw an example (https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob#input---configuration) using a queue trigger but when i do do something similar using http i get 'No value for named parameter 'httpTrigger''. My issue is that i don't know how to reflect a variable that is assigned in my python code in my path. When i do this container/{variable} i get a nullreference exception. This is my python code:
import os
import json
import sys
import logging
import azure.functions as func
_AZURE_FUNCTION_DEFAULT_METHOD = "GET"
_AZURE_FUNCTION_HTTP_INPUT_ENV_NAME = "req"
_AZURE_FUNCTION_HTTP_OUTPUT_ENV_NAME = "res"
_REQ_PREFIX = "REQ_"
def write_http_response(status, response):
output = open(os.environ[_AZURE_FUNCTION_HTTP_OUTPUT_ENV_NAME], 'w')
output.write(json.dumps(response))
env = os.environ
postreqdata = json.loads(open(env['req']).read())
print ('site: ' + postreqdata['site'])
site = postreqdata['site']+'.xlsx'
input_file = open(os.environ['inputBlob'], 'r')
clear_text = input_file.read()
input_file.close()
print("Content in the blob file: '{0}'".format(clear_text))
# Get HTTP METHOD
http_method = env['REQ_METHOD'] if 'REQ_METHOD' in env else
_AZURE_FUNCTION_DEFAULT_METHOD
print("HTTP METHOD => {}".format(http_method))
# Get QUERY STRING
req_url = env['REQ_HEADERS_X-ORIGINAL-URL'] if 'REQ_HEADERS_X-ORIGINAL-URL'
in env else ''
urlparts =req_url.split('?')
query_string = urlparts[1] if len(urlparts) == 2 else ''
print("QUERY STRING => {}".format(query_string))
if http_method.lower() == 'post':
request_body = open(env[_AZURE_FUNCTION_HTTP_INPUT_ENV_NAME], "r").read()
print("REQUEST BODY => {}".format(request_body))
write_http_response(200, site)
note: i have made my connection string successfully ( i think) and i am new to azure and using the portal only
This looks like an older version of function apps. In the new version, you can actually use the request handler to do all this work for you. I just started working in azure functions and if you want to access a file in blob storage, all you have to do is pass in the filename parameters in the form of http query, and use that query param name as the binding variable.
Ex:
def main(req: func.HttpRequest, inputblob: func.InputStream):
input_file_content = input_blob.read()
and in your binding you give
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"type": "blob",
"direction":"in",
"name": "inputblob",
"path": "upload/{filename}",
"connection": "AzureWebJobsStorage"
}
]
}
and you simply call the api with the query parameters filename
http://localhost:7071/api/HttpTriggerFileUpload?filename=file.ext
You can take a look at this