Azure function blob storage file name - python

When using azure functions with blob storage output bindings, how do you get the created blob's name, path or URL? I want to save this to a DB once it saved it.
Im using Python, but any example will do:
blob = open(os.environ['outputBlob'], 'wb')
blob.write(attachment.get_payload(decode=True))
print blob.name # this is not the correct name, but actually the temp file name I think
blob.close()

For C# :
As discussed at https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob#input you can use below type of blob bindings
Below is example of binding json file and the code. I am returning outpuBlob.Uri in http return to get the primary location path of Blob.
Bindings:-
{
"bindings": [
{
"authLevel": "function",
"name": "req",
"type": "httpTrigger",
"direction": "in",
"methods": [
"get",
"post"
]
},
{
"name": "$return",
"type": "http",
"direction": "out"
},
{
"type": "blob",
"name": "outputBlob",
"path": "outcontainer/{rand-guid}",
"connection": "AzureWebJobsDashboard",
"direction": "inout"
}
],
"disabled": false
}
Function Code (C#):
#r "Microsoft.WindowsAzure.Storage"
using System.Net;
using Microsoft.WindowsAzure.Storage.Blob;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log,CloudBlockBlob outputBlob)
{
log.Info("C# HTTP trigger function processed a request.");
// parse query parameter
string name = req.GetQueryNameValuePairs()
.FirstOrDefault(q => string.Compare(q.Key, "name", true) == 0)
.Value;
if (name == null)
{
// Get request body
dynamic data = await req.Content.ReadAsAsync<object>();
name = data?.name;
}
return name == null
? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a name on the query string or in the request body")
: req.CreateResponse(HttpStatusCode.OK, "Hello " + outputBlob.Uri);
}

Take a look at the environment variables in your Python script's process space. You should have one that contains the path of the blob binding.

Related

SNS always sends default message instead of the specific protocol message that is given

Here is the code
message = {
"default":"Sample fallback message",
"http":{
"data":[
{
"type":"articles",
"id":"1",
"attributes":{
"title":"JSON:API paints my bikeshed!",
"body":"The shortest article. Ever.",
"created":"2015-05-22T14:56:29.000Z",
"updated":"2015-05-22T14:56:28.000Z"
}
}
]
}
}
message_as_json = json.dumps(message)
response = sns_client.publish(TopicArn = "arn:aws:sns:us-east-1:MY-ARN",
Message = message_as_json, MessageStructure = "json")
print(response)
To test, I used ngrok to connect the localhost (which runs my flask app) to the web and created a http subscription.
When I publish the message I can only see default message in that.
Any idea why this happens?
You need to specify the value of the http key as a simple JSON string value according to the AWS boto3 docs:
Keys in the JSON object that correspond to supported transport
protocols must have simple JSON string values.
Non-string values will cause the key to be ignored.
import json
import boto3
sns_client = boto3.client("sns")
message = {
"default": "Sample fallback message",
"http": json.dumps(
{
"data": [
{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON:API paints my bikeshed!",
"body": "The shortest article. Ever.",
"created": "2015-05-22T14:56:29.000Z",
"updated": "2015-05-22T14:56:28.000Z",
},
}
]
}
),
}
response = sns_client.publish(
TopicArn="arn:aws:sns:us-east-1:MY-ARN", Message=json.dumps(message), MessageStructure="json"
)

How do I create an Azure API that connects to my Blob Storage?

So I have created a function app and now I am trying to create an API that connects to my Blob Storage and "Posts" the content within the container
import azure.functions as func
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
name = req.params.get('name')
if not name:
try:
req_body = req.get_json()
except ValueError:
pass
else:
name = req_body.get('name')
if name:
return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")
else:
return func.HttpResponse(
"This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
status_code=200
)
Thank you Anupam Chand. Posting your suggestions as an answer to help other community members.
The Output Binding allows you to connect to the blob storage using azure function.
Below is the sample code in how to connect to the Blob storage using output and input bindings.
{
"bindings": [
{
"queueName": "myqueue-items",
"connection": "MyStorageConnectionAppSetting",
"name": "queuemsg",
"type": "queueTrigger",
"direction": "in"
},
{
"name": "inputblob",
"type": "blob",
"dataType": "binary",
"path": "samples-workitems/{queueTrigger}",
"connection": "MyStorageConnectionAppSetting",
"direction": "in"
},
{
"name": "outputblob",
"type": "blob",
"dataType": "binary",
"path": "samples-workitems/{queueTrigger}-Copy",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
}
],
"disabled": false,
"scriptFile": "__init__.py"
}
For related information check Azure Blob output bindings.

Azure Functions (Python) blob ouput binding. How to set name when name is only part of the input message

I have an Azure Functions (Python 3) function that takes a message from a Service Bus queue and creates a Blob in a container as a result.
The function trigger is the Sevice Bus message. This message is a JSON object with several properties, one of which is the blob name.
The docs suggest something like this in the bindings:
{
"name": "outputblob",
"type": "blob",
"path": "samples-workitems/{queueTrigger}-Copy",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
}
But this suggest that the triggering message contains just the blob name. I can not make the message solely the blob name as I require the other attributes in the message to determine what to do / what data to put in the blob.
Is there any way to use the output bindings that will resolve this for me?
Thanks.
Yes, this could be done. You could just set the input and output binding path with the json value from the trigger json data. The below is my function.json. Use service bus trigger get the input blob name and output blob name, then write the input blob to the output blob. You could also set the container name with this way.
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "msg",
"type": "serviceBusTrigger",
"direction": "in",
"queueName": "myqueue",
"connection": "servicebuscon"
},
{
"name": "inputblob",
"type": "blob",
"path": "inputcontainer/{blobname}",
"connection": "AzureWebJobsStorage",
"direction": "in"
},
{
"name": "outputblob",
"type": "blob",
"path": "outputcontainer/{outblobname}",
"connection": "AzureWebJobsStorage",
"direction": "out"
}
]
}
And the below is the function code.
import logging
import azure.functions as func
import json, os
def main(msg: func.ServiceBusMessage,inputblob: func.InputStream,outputblob: func.Out[bytes]) -> func.InputStream:
logging.info('Python ServiceBus queue trigger processed message: %s',
msg.get_body().decode('utf-8'))
jsonData= json.loads(inputblob.read())
logging.info(jsonData)
outputblob.set(str(jsonData))
And I set the service bus message like below message.
Here is the result pic. You could find the input blob json data shown in the console and I check the container the output blob is created.

azure httptrigger blob storage using Python

I am trying to setup access to blob storage using a python function app but the file name is received from a post request not preset. The http trigger part works but i'm having trouble accessing files in my blob storage. This is my json:
{
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"post",
"get"
]
},
{
"name": "inputblob",
"type": "blob",
"path": "sites/{httpTrigger}",
"connection": "STORAGE",
"direction": "in"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
],
"disabled": false
}
I saw an example (https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob#input---configuration) using a queue trigger but when i do do something similar using http i get 'No value for named parameter 'httpTrigger''. My issue is that i don't know how to reflect a variable that is assigned in my python code in my path. When i do this container/{variable} i get a nullreference exception. This is my python code:
import os
import json
import sys
import logging
import azure.functions as func
_AZURE_FUNCTION_DEFAULT_METHOD = "GET"
_AZURE_FUNCTION_HTTP_INPUT_ENV_NAME = "req"
_AZURE_FUNCTION_HTTP_OUTPUT_ENV_NAME = "res"
_REQ_PREFIX = "REQ_"
def write_http_response(status, response):
output = open(os.environ[_AZURE_FUNCTION_HTTP_OUTPUT_ENV_NAME], 'w')
output.write(json.dumps(response))
env = os.environ
postreqdata = json.loads(open(env['req']).read())
print ('site: ' + postreqdata['site'])
site = postreqdata['site']+'.xlsx'
input_file = open(os.environ['inputBlob'], 'r')
clear_text = input_file.read()
input_file.close()
print("Content in the blob file: '{0}'".format(clear_text))
# Get HTTP METHOD
http_method = env['REQ_METHOD'] if 'REQ_METHOD' in env else
_AZURE_FUNCTION_DEFAULT_METHOD
print("HTTP METHOD => {}".format(http_method))
# Get QUERY STRING
req_url = env['REQ_HEADERS_X-ORIGINAL-URL'] if 'REQ_HEADERS_X-ORIGINAL-URL'
in env else ''
urlparts =req_url.split('?')
query_string = urlparts[1] if len(urlparts) == 2 else ''
print("QUERY STRING => {}".format(query_string))
if http_method.lower() == 'post':
request_body = open(env[_AZURE_FUNCTION_HTTP_INPUT_ENV_NAME], "r").read()
print("REQUEST BODY => {}".format(request_body))
write_http_response(200, site)
note: i have made my connection string successfully ( i think) and i am new to azure and using the portal only
This looks like an older version of function apps. In the new version, you can actually use the request handler to do all this work for you. I just started working in azure functions and if you want to access a file in blob storage, all you have to do is pass in the filename parameters in the form of http query, and use that query param name as the binding variable.
Ex:
def main(req: func.HttpRequest, inputblob: func.InputStream):
input_file_content = input_blob.read()
and in your binding you give
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"type": "blob",
"direction":"in",
"name": "inputblob",
"path": "upload/{filename}",
"connection": "AzureWebJobsStorage"
}
]
}
and you simply call the api with the query parameters filename
http://localhost:7071/api/HttpTriggerFileUpload?filename=file.ext
You can take a look at this

Create File Version with Autodesk API

Following on from my file upload question, I am getting stuck at Step 5 in the documentation. I get a 400 error suggesting some of the data I am sending is incorrect, but I think I have matched the code exactly.
def create_version_for_file(self, file_name, project_id, folder_id, object_id):
url = '{}data/v1/projects/{}/items'.format(self.DOMAIN, project_id)
logger.info('Starting version create at %s for file_name %s, folder %s, object %s',
url, file_name, folder_id, object_id)
data = {
"jsonapi": {"version": "1.0"},
"data": {
"type": "items",
"attributes": {
"displayName": file_name,
"extension": {
"type": "items:autodesk.core:File",
"version": "1.0"
}
},
"relationships": {
"tip": {
"data": {
"type": "versions",
"id": "1"
}
},
"parent": {
"data": {
"type": "folders",
"id": folder_id
}
}
}
},
"included": [
{
"type": "versions",
"id": "1",
"attributes": {
"name": file_name,
"extension": {
"type": "versions:autodesk.core:File",
"version": "1.0"
}
},
"relationships": {
"storage": {
"data": {
"type": "objects",
"id": object_id
}
}
}
}
]
}
response = self.session.post(url, json=data, headers={
'content-type': 'application/vnd.api+json',
'accept': 'application/vnd.api+json'
})
if response.status_code != status.HTTP_201_CREATED:
logger.warn('Version create for %s failed with status %s: %s', file_name, response.status_code,
response.content)
return None
return json.loads(response.content)
However, the request always fails like so:
Upload succeeded for README.md 2017-10-12 16:53:15,943
Starting version create at https://developer.api.autodesk.com/data/v1/projects/b.f19577f2-c4da-428f-9625-bb53bf434cca/items for file_name README.md, folder urn:adsk.wipprod:fs.folder:co.Hx1ePxPtS1e0P-Ib9qudyQ, object urn:adsk.objects:os.object:3a06e38e-4cac-4ffc-981f-0e5c4e4078aab.f19577f2-c4da-428f-9625-bb53bf434cca/d14c3591-d339-4e62-907c-6f0c8b58b743.md
Version create for README.md failed with status 400: {"jsonapi":{"version":"1.0"},"errors":[{"id":"bfbf0a93-c92a-47af-9ce7-a6af48594e44","status":"400","code":"BAD_INPUT","title":"One or more input values in the request were bad","detail":"Request input is invalid for this operation."}]}
Sample values for all of the variables are off to the right in the logs above.
This might be setup correctly but maybe you have your storage location created in the incorrect folder. From the tutorial I suggested in the forum post, you need to navigate down one level to avoid the creation of the storage location on the Root folder of BIM 360 Docs. Try going back on your steps and do the followed suggested on the tutorial. Focus on step number 4
Authorization Web Flow (This will return us the code we need to obtain our oauth token)
Rest call to the Authentication API to obtain a 3 legged Token GET call to obtain detail of which Hubs do we have access in BIM 360 Docs (Registration of APP required for BIM 360 API access)
GET call to find the project that has your resource
GET call to find the folder where the upload will happen (Plans, Project File, Drawings).
4.1 Extra step can include the access to a sub-folder.
POST call to create a storage location in the previously defined folder
PUT call to upload file to the storage location
POST call to create the first version of the uploaded file.
Check BIM 360 Docs to see your recently uploaded file.
From this forum response, the tutorial is out-of-date and you should use autodesk.bim360:File (but not autodedsk.bim360:File as the typo suggests) in place of autodesk.core:File. There is a more recent example here..
It's still not working but at least my error moved on to The urn must be an unassigned urn prepared by the create storage endpoint, by the same user.

Categories

Resources