I am trying to setup access to blob storage using a python function app but the file name is received from a post request not preset. The http trigger part works but i'm having trouble accessing files in my blob storage. This is my json:
{
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"post",
"get"
]
},
{
"name": "inputblob",
"type": "blob",
"path": "sites/{httpTrigger}",
"connection": "STORAGE",
"direction": "in"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
],
"disabled": false
}
I saw an example (https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob#input---configuration) using a queue trigger but when i do do something similar using http i get 'No value for named parameter 'httpTrigger''. My issue is that i don't know how to reflect a variable that is assigned in my python code in my path. When i do this container/{variable} i get a nullreference exception. This is my python code:
import os
import json
import sys
import logging
import azure.functions as func
_AZURE_FUNCTION_DEFAULT_METHOD = "GET"
_AZURE_FUNCTION_HTTP_INPUT_ENV_NAME = "req"
_AZURE_FUNCTION_HTTP_OUTPUT_ENV_NAME = "res"
_REQ_PREFIX = "REQ_"
def write_http_response(status, response):
output = open(os.environ[_AZURE_FUNCTION_HTTP_OUTPUT_ENV_NAME], 'w')
output.write(json.dumps(response))
env = os.environ
postreqdata = json.loads(open(env['req']).read())
print ('site: ' + postreqdata['site'])
site = postreqdata['site']+'.xlsx'
input_file = open(os.environ['inputBlob'], 'r')
clear_text = input_file.read()
input_file.close()
print("Content in the blob file: '{0}'".format(clear_text))
# Get HTTP METHOD
http_method = env['REQ_METHOD'] if 'REQ_METHOD' in env else
_AZURE_FUNCTION_DEFAULT_METHOD
print("HTTP METHOD => {}".format(http_method))
# Get QUERY STRING
req_url = env['REQ_HEADERS_X-ORIGINAL-URL'] if 'REQ_HEADERS_X-ORIGINAL-URL'
in env else ''
urlparts =req_url.split('?')
query_string = urlparts[1] if len(urlparts) == 2 else ''
print("QUERY STRING => {}".format(query_string))
if http_method.lower() == 'post':
request_body = open(env[_AZURE_FUNCTION_HTTP_INPUT_ENV_NAME], "r").read()
print("REQUEST BODY => {}".format(request_body))
write_http_response(200, site)
note: i have made my connection string successfully ( i think) and i am new to azure and using the portal only
This looks like an older version of function apps. In the new version, you can actually use the request handler to do all this work for you. I just started working in azure functions and if you want to access a file in blob storage, all you have to do is pass in the filename parameters in the form of http query, and use that query param name as the binding variable.
Ex:
def main(req: func.HttpRequest, inputblob: func.InputStream):
input_file_content = input_blob.read()
and in your binding you give
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"type": "blob",
"direction":"in",
"name": "inputblob",
"path": "upload/{filename}",
"connection": "AzureWebJobsStorage"
}
]
}
and you simply call the api with the query parameters filename
http://localhost:7071/api/HttpTriggerFileUpload?filename=file.ext
You can take a look at this
Related
So I have created a function app and now I am trying to create an API that connects to my Blob Storage and "Posts" the content within the container
import azure.functions as func
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
name = req.params.get('name')
if not name:
try:
req_body = req.get_json()
except ValueError:
pass
else:
name = req_body.get('name')
if name:
return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")
else:
return func.HttpResponse(
"This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
status_code=200
)
Thank you Anupam Chand. Posting your suggestions as an answer to help other community members.
The Output Binding allows you to connect to the blob storage using azure function.
Below is the sample code in how to connect to the Blob storage using output and input bindings.
{
"bindings": [
{
"queueName": "myqueue-items",
"connection": "MyStorageConnectionAppSetting",
"name": "queuemsg",
"type": "queueTrigger",
"direction": "in"
},
{
"name": "inputblob",
"type": "blob",
"dataType": "binary",
"path": "samples-workitems/{queueTrigger}",
"connection": "MyStorageConnectionAppSetting",
"direction": "in"
},
{
"name": "outputblob",
"type": "blob",
"dataType": "binary",
"path": "samples-workitems/{queueTrigger}-Copy",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
}
],
"disabled": false,
"scriptFile": "__init__.py"
}
For related information check Azure Blob output bindings.
I have written an azure function, currently the azure function is acting as Webhook Consumer. The job of the function is to read Webhook events and save into azure storage.
I am using an HTTP trigger template to get the job done. I am able to receive the events from the Webhook, but when I try to write to azure storage it is giving me below error.
I tried the option mentioned in this post, but no luck still getting the same error.
System.InvalidOperationException: Storage account connection string 'AzureWebJobs<AzureStorageAccountName>' does not exist. Make sure that it is a defined App Setting.
Below is my function.json file
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "blob",
"name": "outputblob",
"path": "test/demo",
"direction": "out",
"connection": "<AzureStorageAccountName>"
}
]
}
init.py
import logging
import azure.functions as func
def main(req: func.HttpRequest,outputblob: func.Out[str]) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
name = 'some_name'
if not name:
try:
req_body = 'req_body_test'#req.get_json()
except ValueError:
pass
else:
name = 'name'#req_body.get('name')
print(str(req.get_json()))
outputblob.set(str(req.get_json()))
Please make sure you have already add the connection string to the local.settings.json on local or configuration settings on azure.
Please test below code and settings files:
__init__.py
import logging
import azure.functions as func
def main(req: func.HttpRequest,outputblob: func.Out[func.InputStream]) -> func.HttpResponse:
outputblob.set("this is a test.")
return func.HttpResponse(
"Test.",
status_code=200
)
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"name": "outputblob",
"type": "blob",
"path": "test/demo",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
}
]
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "",
"FUNCTIONS_WORKER_RUNTIME": "python",
"MyStorageConnectionAppSetting":"DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net"
}
}
On azure:
Below is the code,
import logging
import json
import urllib.request
import urllib.parse
import azure.functions as func
def main(myblob: func.InputStream):
logging.info(f"Python blob trigger function processed blob \n"
f"Name: {myblob.name}\n"
f"Blob Size: {myblob.length} bytes")
response = urllib.request.urlopen("http://example.com:5000/processing")
return {
'statusCode': 200,
'body': json.dumps(response.read().decode('utf-8'))
}
Error: Result: Failure Exception: RuntimeError: function 'abc' without a $return binding returned a non-None value Stack: File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/dispatcher.py", line 341, in _handle__invocation_request f'function {fi.name!r} without a $return binding '. The same code works in lambda.. Please help me in debugging in azure functions.
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "sourcemetadata/{name}",
"connection": "AzureWebJobsStorage"
}
]
}
In the Azure function, if you use return in the function app code, it means that you want to use output binding. But you do not define it in function.json. Please define it. For more details, please refer to here and here
For example
I use process blob with blob trigger and send message to azure queue with queue output binding
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "test/{name}.csv",
"connection": "AzureWebJobsStorage"
},
{
"name": "$return",
"direction": "out",
"type": "queue",
"queueName": "outqueue",
"connection": "AzureWebJobsStorage"
}
]
}
Code
async def main(myblob: func.InputStream) :
logging.info(f"Python blob trigger function processed blob \n"
f"Name: {myblob.name}\n")
return "OK"
I have a python function with a blob input binding. The blob in question contains a parquet file. Ultimately I want to read the bound blob into a pandas dataframe but I am unsure of the correct way to do this.
I have verified that the binding is correctly set up and I've been able to successfully read a plain text file. I am happy that the integrity of the parquet file is fine as I have been able to read it using the example provided here: https://arrow.apache.org/docs/python/parquet.html#reading-a-parquet-file-from-azure-blob-storage
The following code shows what I am trying to do:
import logging
import io
import azure.functions as func
import pyarrow.parquet as pq
def main(req: func.HttpRequest, inputblob: func.InputStream) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
# Create a bytestream to hold blob content
byte_stream = io.BytesIO()
byte_stream.write(inputblob.read())
df = pq.read_table(source=byte_stream).to_pandas()
I get the following error message:
pyarrow.lib.ArrowIOError: Couldn't deserialize thrift: TProtocolException: Invalid data
The following is my function.json file:
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"name": "inputblob",
"type": "blob",
"path": "<container>/file.parquet",
"connection": "AzureWebJobsStorage",
"direction": "in"
}
]
}
My host.json file:
{
"version": "2.0",
"functionTimeout": "00:10:00",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
}
}
I have been working on the same problem, and this solution worked for me.
__ini_.py file:
from io import BytesIO
import azure.functions as func
def main(blobTrigger: func.InputStream):
# Read the blob as bytes
blob_bytes = blobTrigger.read()
blob_to_read = BytesIO(blob_bytes)
df = pd.read_parquet(blob_to_read, engine='pyarrow')
print("Length of the parquet file:" + str(len(df.index)))
When using azure functions with blob storage output bindings, how do you get the created blob's name, path or URL? I want to save this to a DB once it saved it.
Im using Python, but any example will do:
blob = open(os.environ['outputBlob'], 'wb')
blob.write(attachment.get_payload(decode=True))
print blob.name # this is not the correct name, but actually the temp file name I think
blob.close()
For C# :
As discussed at https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob#input you can use below type of blob bindings
Below is example of binding json file and the code. I am returning outpuBlob.Uri in http return to get the primary location path of Blob.
Bindings:-
{
"bindings": [
{
"authLevel": "function",
"name": "req",
"type": "httpTrigger",
"direction": "in",
"methods": [
"get",
"post"
]
},
{
"name": "$return",
"type": "http",
"direction": "out"
},
{
"type": "blob",
"name": "outputBlob",
"path": "outcontainer/{rand-guid}",
"connection": "AzureWebJobsDashboard",
"direction": "inout"
}
],
"disabled": false
}
Function Code (C#):
#r "Microsoft.WindowsAzure.Storage"
using System.Net;
using Microsoft.WindowsAzure.Storage.Blob;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log,CloudBlockBlob outputBlob)
{
log.Info("C# HTTP trigger function processed a request.");
// parse query parameter
string name = req.GetQueryNameValuePairs()
.FirstOrDefault(q => string.Compare(q.Key, "name", true) == 0)
.Value;
if (name == null)
{
// Get request body
dynamic data = await req.Content.ReadAsAsync<object>();
name = data?.name;
}
return name == null
? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a name on the query string or in the request body")
: req.CreateResponse(HttpStatusCode.OK, "Hello " + outputBlob.Uri);
}
Take a look at the environment variables in your Python script's process space. You should have one that contains the path of the blob binding.