I have a model endpoint running in Azure Kubernetes Service and I am not using Django or Flask. I am sending local png files to score as follows:
import base64
import json
import cv2
import requests
img_path = 'C:/path/to/exampleImage.png'
link = aks_service.scoring_uri
api_key = aks_service.get_keys()[0]
def send2score(img_path, score_url, api_key):
headers = {'Content-Type': 'application/json',
'Authorization': ('Bearer ' + api_key)
}
img = cv2.imread(img_path)
string = base64.b64encode(cv2.imencode('.png', img)[1]).decode()
dict = {
'img': string
}
jsonimg2 = json.dumps(dict, ensure_ascii=False, indent=4)
resp = requests.post(url=link, data=jsonimg2, headers=headers)
print(resp.text)
send2score(img_path=img_path, score_url=link, api_key=api_key)
My question is: how can I get the file name (exampleImage.png) in the score script in Azure Kubernetes after I do the request.post? Please no Django or Flask specific methods
Bonus question: Feel free to suggest improvements in the way I am uploading the data (send2score function), this function is working, I get the score back, I just can't get the file name in the score script. Thank you!
According to your code, you send your image as base64 string. It cannot contain the file name. I think you need to define a parameter to store the file name in request body. Besides, you also can post the file as a Multipart-Encoded File with the requests module.
For example
send file
import requests
import magic
import os
url = ''
path = "D:/sample/image/faces.jpg"
mime = magic.Magic(mime=True)
headers = {
'Authorization': ('Bearer ' + 'cdff')
}
files = {'file': (os.path.basename(path), open(path, 'rb'), mime.from_file(path), {'Expires': '0'})}
res = requests.post(url, files=files, headers=headers)
print(res.content.decode('utf-8'))
My backend
from http.server import BaseHTTPRequestHandler, HTTPServer
import cgi
hostName =
hostPort =
class MyServer(BaseHTTPRequestHandler):
def do_POST(self):
try:
// use cgi to read file
form = cgi.FieldStorage(fp=self.rfile, headers=self.headers, environ={
'REQUEST_METHOD': 'POST', 'CONTENT_TYPE': self.headers['Content-Type'], })
file = form.list[0]
data =file.file.read()
#process data
.............
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(bytes(
f"<html><head></head><body><h2>fileName : {file.filename}</h2>/html>", "utf-8"))
except Exception as e:
httperror = e.httperror if hasattr(e, "httperror") else 500
self.send_error(httperror, str(e)) # Send an error response
myServer = HTTPServer((hostName, hostPort), MyServer)
myServer.serve_forever()
I was overcomplicating things, I realized that I am sending the encoded image as a json in a dictionary. I can include other information, like the file name, in the dictionary:
import base64
import json
import cv2
import requests
img_path = 'C:/path/to/exampleImage.png'
link = aks_service.scoring_uri
api_key = aks_service.get_keys()[0]
def send2score(img_path, score_url, api_key):
headers = {'Content-Type': 'application/json',
'Authorization': ('Bearer ' + api_key)
}
img = cv2.imread(img_path)
string = base64.b64encode(cv2.imencode('.png', img)[1]).decode()
dict = {
'imgname': os.path.basename(img_path),
'img': string
}
jsonimg2 = json.dumps(dict, ensure_ascii=False, indent=4)
resp = requests.post(url=link, data=jsonimg2, headers=headers)
print(resp.text)
send2score(img_path=img_path, score_url=link, api_key=api_key)
And I can get the image and file name in the score script:
# Lots of code before
response = json.loads(path)
string = response['img']
jpg_original = base64.b64decode(string) # decode
jpg_as_np = np.frombuffer(jpg_original, dtype=np.uint8)
img0 = cv2.imdecode(jpg_as_np, flags=1) # image
img0name = response['imgname'] # file name
# Lots of code after
Related
I have a yaml file : file.yaml structured as follows :
index:
- uid: "uid"
name: "name"
headline: "headline"
overview: "overview"
features: "features"
instructions: "instructions"
callback_url: "https://some-url.com/params"
edit_url: "https://edit-url/params"
uninstall_hook: "https://uninstall-url/params"
svg:
screenshot1:
screenshot2:
screenshot3:
I have to upload those informations to an api endpoint by performing a PUT request. I managed to do it first using the register.py following script that I just run python register.py:
import json
import requests
from pathlib import Path
import base64
import yaml
BASE_URL = "https://url.com" # API Host
FILE_FOLDER = Path.cwd() # Current working directory
if __name__ == "__main__":
public_key = <public_key>
private_key = <private_key>
auth_key = "{}:{}".format(public_key, private_key).encode("utf-8")
encodedKey = base64.b64encode(auth_key).decode("utf-8")
headers = {"Authorization": f"Basic {encodedKey}", "Content-type": "application/json"}
def update_app_info():
infos_file = FILE_FOLDER / "file.yaml"
with open(infos_file) as infos_file_data:
yamlcontent = yaml.safe_load(infos_file_data) # Parse file.yaml and produce a dictionary of it
file_infos = yamlcontent["index"][0] # retrieve actual configuration informations
response = requests.put(
f"{BASE_URL}/path/to/api_endpoint/{public_key}", data=json.dumps(file_infos), headers=headers
)
print(response)
print(response.json())
update_app_info()
That gives a 202 success response.
As you may observe, I tried to get content of the yaml file as a dicitonary and send that in data. I proceeded that way regarding format of data at GET https://url.com/path/to/api_endpoint (mock example for illustration...) . Having the dictionary file_infos seemed more appropriate and gets me a success response. Sending directly the file itself or 'infos_file_data' gave me some errors I got over with the above script.
The issue is when I update svg, screenshot1, screenshot2 & screenshot3 so that file.yaml is now :
index:
- uid: "uid"
name: "name"
headline: "headline"
overview: "overview"
features: "features"
instructions: "instructions"
callback_url: "https://some-url.com/params"
edit_url: "https://edit-url/params"
uninstall_hook: "https://uninstall-url/params"
svg: "icon.svg"
screenshot1: "screenshot1.png"
screenshot2: "screenshot2.png"
screenshot3: "screenshot3.png"
That gives now :
<Response [400]>
{'error': {'message': {'svg': ['The submitted data was not a file. Check the encoding type on the form.'], 'screenshot1': ['The submitted data was not a file. Check the encoding type on the form.'], 'screenshot2': ['The submitted data was not a file. Check the encoding type on the form.'], 'screenshot3': ['The submitted data was not a file. Check the encoding type on the form.']}, 'code': 400}}
I've done multiple searches (1 , 2 , 3 , 4 , 5...) but their application and few other errors, eventually get me to this :
import base64
import json
from pathlib import Path
import requests
import yaml
from requests_toolbelt.multipart.encoder import MultipartEncoder
BASE_URL = "https://url.com" # API Host
FILE_FOLDER = Path.cwd() # Current working directory
if __name__ == "__main__":
public_key = <public_key>
private_key = <private_key>
auth_key = "{}:{}".format(public_key, private_key).encode("utf-8")
encodedKey = base64.b64encode(auth_key).decode("utf-8")
def update_app_info():
infos_file = FILE_FOLDER / "file.yaml"
with open(infos_file) as infos_file_data:
yamlcontent = yaml.safe_load(infos_file_data) # Parse file.yaml and produce a dictionary of it
file_infos = yamlcontent["index"][0] # retrieve actual configuration informations
m = MultipartEncoder(fields=file_infos)
#print(m.content_type)
headers = {
"Authorization": f"Basic {encodedKey}",
"Content-Type": m.content_type,
}
response = requests.put(
f"{BASE_URL}/path/to/api_endpoint/{public_key}",
data=json.dumps(file_infos),
headers=headers
)
print(response)
print(response.json())
update_app_info()
That is also giving me the 202 success response but the file svg, screenshot1, screenshot2 & screenshot3 fields are not updated.
I'll share more informations where needed. Your help is very welcome.
I've got additional resources that helped.
As I was trying to solve my issue, I found this. It happens I didn't wrote files part as it should, plus I was having data as a JSON string. That causes a ValueError: Data must not be a string. error. This was useful to get it fixed.
Now, for what it's worth, here's the working script :
import base64
from pathlib import Path
import requests
import yaml
BASE_URL = "https://url.com" # API Host
FILE_FOLDER = Path.cwd() # Current working directory
if __name__ == "__main__":
public_key = <public_key>
private_key = <private_key>
auth_key = "{}:{}".format(public_key, private_key).encode("utf-8")
encodedKey = base64.b64encode(auth_key).decode("utf-8")
def update_app_info():
infos_file = FILE_FOLDER / "file.yaml"
with open(infos_file) as infos_file_data:
yamlcontent = yaml.safe_load(infos_file_data) # Parse file.yaml and produce a dictionary of it
if "index" in yamlcontent:
file_infos = yamlcontent["index"][0] # retrieve actual configuration informations
headers = {
"Authorization": f"Basic {encodedKey}",
}
files = {
"svg": open("icon.svg", "rb"),
"screenshot1": open("screenshot1.png", "rb"),
"screenshot2": open("screenshot2.png", "rb"),
"screenshot3": open("screenshot3.png", "rb"),
}
response = requests.put(
f"{BASE_URL}/path/to/api_endpoint/{public_key}", data=file_infos, files=files, headers=headers
)
print("\n", response)
print("\n", response.headers)
print("\n", response.json())
update_app_info()
I am trying to connect to an api for which I was provided with link, certificate(.p12) and subscription key.
Having some issue while giving the certificate details. I am trying in the following 2 ways:
1.
import json
from requests_pkcs12 import get,post
url = 'https://....'
pkcs12_filename = '<certificate file path>'
pkcs12_password = '<certificate password>'
headers = {
# Request headers
'Subscription-Key': '<subscription key>',}
response = post(url, headers=headers, verify=False, pkcs12_filename=pkcs12_filename,pkcs12_password=pkcs12_password)
print(response.status_code)
import http.client, urllib.request, urllib.parse, urllib.error, base64
#file = "<certificate path>"
headers = {
'Subscription-Key': '<subscriptionkey>',
#'cert' : crypto.load_pkcs12(open(file, 'rb').read(), "<password>").get_certificate(),
'file' : '<certificate path>',
'password':'<certificate password>',}
params = urllib.parse.urlencode({
})
conn = http.client.HTTPSConnection('<api main link>')
conn.request("GET", "<api remaining link>" , params, headers)
response = conn.getresponse()
data = response.read()
print("Status: {} and reason: {}".format(response.status, response.reason))
conn.close()
I am new to API concept. Will someone help me over this?
Refered to this link: How to use .p12 certificate to authenticate rest api
But didn't get what i need to put in data variable
This question already has answers here:
Python file upload from url using requests library
(3 answers)
Closed 5 years ago.
Is there a chance to upload file by API endpoint which is taking multipart/form-data as a content-type having only URL of that file?
Rule:
Download the whole file into memory and then upload by this endpoint is no option (There is no guarantee that the box will ever be big enough to hold a temporary file).
Question:
I want to stream file in chunks from one server (GET) to another (multipart/form-data POST). Is this possible? How to achieve that?
Flow:
file_server <-GET- my_script.py -POST-> upload server
here is a simple example of downloading into memory (RAM) option (but it's against the rule):
from io import BytesIO
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder
file_url = 'https://www.sysaid.com/wp-content/uploads/features/itam/image-banner-asset.png'
requested_file_response = requests.get(file_url, stream=True)
TOKEN_PAYLOAD = {
'grant_type': 'password',
'client_id': '#########',
'client_secret': '#########',
'username': '#########',
'password': '#########'
}
def get_token():
response = requests.post(
'https://upload_server/oauth/token',
params=TOKEN_PAYLOAD)
response_data = response.json()
token = response_data.get('access_token')
if not token:
print("token error!")
return token
token = get_token()
file_object = BytesIO()
file_object.write(requested_file_response.content)
# Form conctent
multipart_data = MultipartEncoder(
fields={
'--': (
'test.png',
file_object # AttributeError: 'generator' object has no attribute 'encode' when I try to pass generator here.
),
'id': '2217',
'fileFieldDefId': '4258',
}
)
# Create headers
headers = {
"Authorization": "Bearer {}".format(token),
'Content-Type': multipart_data.content_type
}
session = requests.Session()
response = session.post(
'https://upload_server/multipartUpdate',
headers=headers,
data=multipart_data,
)
the answer is in a file like object creation for stream purposes
Thank You very much for any help. Cheers!
If I read requests_toolbelt source code right than it requires not only a ability to .read() the file (which we could get just by passing requests.get(..., stream=True).raw), but also that there is someway to determine how much data is left in the stream.
Assuming you are CONFIDENT that you always have a valid content-length header, this would be the solution I would suggest:
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder
file_url = 'https://www.sysaid.com/wp-content/uploads/features/itam/image-banner-asset.png'
target = 'http://localhost:5000/test'
class PinocchioFile:
"""I wish I was a real file"""
def __init__(self, url):
self.req = requests.get(url, stream=True)
length = self.req.headers.get('content-length')
self.len = None if length is None else int(length)
self._raw = self.req.raw
def read(self, chunk_size):
chunk = self._raw.read(chunk_size) or b''
self.len -= len(chunk)
if not chunk:
self.len = 0
return chunk
multipart_data = MultipartEncoder(
fields={
'--': (
'test.png',
PinocchioFile(file_url),
),
'id': '2217',
'fileFieldDefId': '4258',
}
)
# Create headers
headers = {
'Content-Type': multipart_data.content_type
}
response = requests.post(
target,
data=multipart_data,
headers=headers,
)
How can I send a file from my local computer to hipchat using a python API? I am currently using Hypchat but it is not well documented.
Here is my code so far:
import hypchat
hc = hypchat.HypChat("myKey")
room = hc.get_room('bigRoom')
I'm not sure how to proceed. I tried other methods such as this one but I keep getting the error:
[ERROR] HipChat file failed: '405 Client Error: Method Not Allowed for url: https://api.hipchat.com/v2/room/bigRoom/share/file'
This code allows me to send any file to a hipchat room:
# do this:
# pip install requests_toolbelt
from os import path
from sys import exit, stderr
from requests import post
from requests_toolbelt import MultipartEncoder
class MultipartRelatedEncoder(MultipartEncoder):
"""A multipart/related encoder"""
#property
def content_type(self):
return str('multipart/related; boundary={0}'.format(self.boundary_value))
def _iter_fields(self):
# change content-disposition from form-data to attachment
for field in super(MultipartRelatedEncoder, self)._iter_fields():
content_type = field.headers['Content-Type']
field.make_multipart(content_disposition = 'attachment',
content_type = content_type)
yield field
def hipchat_file(token, room, filepath, host='api.hipchat.com'):
if not path.isfile(filepath):
raise ValueError("File '{0}' does not exist".format(filepath))
url = "https://{0}/v2/room/{1}/share/file".format(host, room)
headers = {'Content-type': 'multipart/related; boundary=boundary123456'}
headers['Authorization'] = "Bearer " + token
m = MultipartRelatedEncoder(fields={'metadata' : (None, '', 'application/json; charset=UTF-8'),
'file' : (path.basename(filepath), open(filepath, 'rb'), 'text/csv')})
headers['Content-type'] = m.content_type
r = post(url, data=m, headers=headers)
if __name__ == '__main__:
my_token = <my token>
my_room = <room name>
my_file = <filepath>
try:
hipchat_file(my_token, my_room, my_file)
except Exception as e:
msg = "[ERROR] HipChat file failed: '{0}'".format(e)
print(msg, file=stderr)
exit(1)
Shout out to #Martijn Pieters
I am trying to upload a local .csv file to a hipchat room using the requests and email libraries and the HipChat API. This is the code that I am using:
import os
import re
import sys
import requests
from email.mime.multipart import MIMEMultipart
from email.mime.multipart import MIMEBase
from email import encoders
def hipchat_file(token, room, filepath, host='api.hipchat.com'):
""" Send file to a HipChat room via API version 2
Parameters
----------
token : str
HipChat API version 2 compatible token - must be token for active user
room: str
Name or API ID of the room to notify
filepath: str
Full path of file to be sent
host: str, optional
Host to connect to, defaults to api.hipchat.com
"""
if not os.path.isfile(filepath): raise ValueError("File '{0}' does not exist".format(filepath))
url = "https://{0}/v2/room/{1}/share/file".format(host, room)
headers = {'Authorization': 'Bearer {}'.format(token),
'Accept-Charset': 'UTF-8',
'Content-Type': 'multipart/related'}
related = MIMEMultipart('related')
part = MIMEBase('application', "octet-stream")
part.set_payload(open(filepath, "rb").read())
encoders.encode_base64(part)
part.add_header('Content-Disposition', 'attachment; filename="{}"'.format(os.path.basename(filepath)))
related.attach(part)
raw_headers, body = related.as_string().split('\n\n', 1)
boundary = re.search('boundary="([^"]*)"', raw_headers).group(1)
headers['Content-Type'] = 'multipart/related; boundary="{}"'.format(boundary)
# r = requests.post(url, data = body, headers = headers)
r = requests.put(url, data = body, headers = headers)
r.raise_for_status()
print('done')
my_token = 'f0rh4r4mb3'
my_room = 'test'
my_file = r"L:\data\data\0\my_file.csv"
my_message = 'Check out this cool file' # optional
try:
hipchat_file(my_token, my_room, my_file)
except Exception as e:
msg = "[ERROR] HipChat file failed: '{0}'".format(e)
print(msg, file=sys.stderr)
sys.exit(1)
But I keep receiving the error code:
'405 Client Error: Method Not Allowed for url: https://api.hipchat.com/v2/room/test/share/file'
And when I go to that link I see this:
{
"error": {
"code": 405,
"message": "<p>The method is not allowed for the requested URL.</p>",
"type": "Method Not Allowed"
}
}
I think I've treied everything I could, including changing post to get and to put, but nothing worked. What can I try?