I have the following code of a GET method which takes a photo which is stored in a blob type field in MySql and return. I want to return it to the client inside a JSON string in a type it can display the image in an angularjs application.
def GET(self,r):
user_data = CC.get_data(query) # holds the content of the blob field.
print type(user_data) # prints <type 'str'>
data = {'name': 'test',
'photo': user_data}
return json.dump(data)
This gives,
UnicodeDecodeError: 'utf8' codec can't decode byte 0x89 in position 0:
invalid start byte
I have found in some websites its better to send as photo as byte array.
Im using web.py python framework.
whats the best way to do this?
To prevent data loss, the best thing you can do to send binary data is encode as base64.
import base64
def GET(self,r):
user_data = CC.get_data(query) # holds the content of the blob field.
data = {'name': 'test',
'photo': base64.b64encode(user_data)}
return json.dump(data)
However, sending binary data over JSON is really not recommended, specially in web. You can send a URL to download the photo, for example.
First how to work with blob:
http://www.mysqltutorial.org/python-mysql-blob/
To send your image to your template:
def imageResponseView(...) .... return response
bind url to this view
show image using <img src = "//url_to_that_view" />
From: I have an image's HTTPResponse. How do I display it in a Django template?
Related
Attempting to send an image to the client browser - the following code works perfectly - please assume that I am receiving the correct base64.b64encoded byte stream:
def get_user_image(usr_rec):
with open("config.json",) as config:
data = json.load(config)
avatar = data["PATHS"]["AVATAR"]
file_ext = os.path.splitext(usrdata["avatar"])[1] if usrdata["avatar"] else ".png"
file_fmt = ".jpeg" if file_ext.upper()==".JPG" else file_ext
path = "{}/{}{}".format(avatar,usr_rec["recordid"],file_ext)
img = Image.open(path,mode="r")
barr = io.BytesIO()
img.save(barr,format="{}".format(file_fmt[1:].upper()))
return b64encode(barr.getvalue())
def format_record (usr_rec):
avatar = get_user_image(usr_rec["recordid"])
return jsonify({"avatar": str(avatar)})
On my development box. Move it to a flask production running under gunicorn and I get JSON serialization errors: TypeError: Object of type bytes is not JSON serializable I am also getting this error for Decimal types as well.
How can I get around this issue?
This is because your get_user_image function returns a stream of bytes and not a string, so you have to cast the bytes read into a string: get_user_image(usr_rec["recordid"]).decode("utf-8"). The same happens for the object of type Decimal.
The jsonify function only serializes objects of type string, as you can also see here and here
I am attempting to send an image to a Flask API. So far, I am using base64 to encode the image to be sent as a string. On the server side, I am receiving that string and decoding it, and attempting to write over a file on the server side. The code runs, but the resulting JPG is not viewable, and it shows "It looks like we don't support this format." I have also attempted saving the file as other photo file formats. It is a jpg that I convert to string so that is why I am saving as jpg.
Here is my client side code:
with open(filename, "rb") as img:
string = base64.b64encode(img.read())
print(type(string))
print(string)
name = 'John Doe'
EmpID = 1
company = 1
def test():
api_url = "http://192.168.8.13:5000/register-new?emp_name=%s&company_id=%s&emp_id=%s&user_photo=%s" % (name, company, EmpID, string)
response = requests.post(url= api_url)
assert response.status_code == 200
And here is the server side code for receiving the photo.
photo = request.args.get('user_photo')
photo1 = photo.replace(" ", "")
f = (base64.b64decode(photo1))
a = io.BytesIO()
with open("compare.jpg", "wb") as file:
file.write(f)
If you really want to upload this as base64 data, I suggest putting this as JSON in the post body, rather than as a GET parameter.
On the client side, open the file like this:
with open(filename, "rb") as img:
string = base64.b64encode(img.read()).decode('utf-8')
Then in your test function, take that image_data string out of the URL, and use the request.post argument json to pass this across with the correct Content Type. You could consider sending the other variables by adding them to the dictionary passed with this arg:
def test():
api_url = "http://192.168.8.13:5000/register-new?emp_name=%s&company_id=%s&emp_id=%s" % (name, company, EmpID)
response = requests.post(url= api_url, json={'user_photo':string})
Then on the backend, grab this with Flask's request.get_json function and initialise the BytesIO object, before writing the data to that, and finally writing to a file:
#app.route('/register-new', methods=['POST'])
def register_new():
photo = request.get_json()['user_photo']
photo_data = base64.b64decode(photo)
with open("compare.jpg", "wb") as file:
file.write(photo_data)
With a test image this works correctly, as confirmed by the Linux file command:
$ file compare.jpg
compare.jpg: JPEG image data, baseline, precision 8, 500x750, components 3
I'm trying to send one image to my Lambda Function using Python just to test for one project, but Postman is giving me one error and I don't know how to solve it.
My code is simply to detect if I have some data in the key "image" and return some message. I'm using Postman to send the POST request, I clicked in the Body tab, selected the form-data option and I wrote image for the key and selected the image file from my computer (the image size is 27 kb). This is the code in my Lambda Function:
def lambda_handler(event, context):
if event['image']:
return {
"Message": 'Everything went ok'
}
And this is the error message that I'm receiving from Postman:
{ "message": "Could not parse request body into json: Unexpected
character ('-' (code 45)) in numeric value: expected digit (0-9) to
follow minus sign, for valid numeric value\n at [Source:
(byte[])"----------------------------137965576541301454606184\r\nContent-Disposition: form-data; name="image"; filename="TestImage.png"\r\nContent-Type:
image/png\r\n\r\n�PNG\r\n\n ... }
To solve that problem, I needed to set my Camera to convert the image to base64 and then upload it to the server.
In the server, I convert it again and then work with it as I want. Base64 is a group of binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation.
So, you will convert your image to string and then sending it, it was the best way that I found to upload my images.
I was struggling with this. I was using Postman, getting UnidentifiedImageError. The below worked.
Posting the Image:
data = open('x.jpg','rb').read()
data = base64.b64encode(data).decode("utf8")
r = requests.post('url',data=data)
Processing on the function side
def lambda_handler(event, context):
image_bytes = event['body'].encode('utf-8')
img_b64dec = base64.b64decode(image_bytes)
img_byteIO = BytesIO(img_b64dec)
image = Image.open(img_byteIO)
I am downloading file from Hadoop to Django backend and storing the file using the code below:
import shutil
import requests
url = 'http://112.138.0.12:9870/webhdfs/v1/user/username/1.jpg?op=OPEN&user.name=username'
response = requests.get(url, stream=True)
with open('img.png', 'wb') as out_file:
shutil.copyfileobj(response.raw, out_file)
del response
I don't need to store the file in backend local system since I want to send this file to Angular 5 frontend where user will save this file in their local system. I'm getting the following error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position
0: invalid start byte.
Can someone suggest me what would be the right way to do download large files in a short time?
DJANGO:
views.py:
class DownloadFileView(GenericAPIView):
serializer_class = UserNameSerializer
def get(self, request):
key = request.META.get('HTTP_AUTHORIZATION').split()[1]
user_id = Token.objects.get(key=key).user_id
user_name = User.objects.get(id=user_id).username
response = download_files(user_name)
return Response(response)
def download_files(user_name):
response = requests.get('http://112.138.0.12:9870/webhdfs/v1/user/' + user_name + '/1.jpg?op=OPEN&user.name=username', stream=True)
return response.raw
ANGULAR:
DownloadFile(){
this.userService.DownloadFiles().subscribe((data : any) => {
const blob = new Blob([data], { type: 'application/octet-stream'});
fileUrl = this.sanitizer.bypassSecurityTrustResourceUrl(window.URL.createObjectURL(blob));
}
}
DownloadFiles() {
this.token = localStorage.getItem('userToken')
var reqHeader = new HttpHeaders({ 'Content-Type': 'application/octet-stream', 'Authorization': 'token ' + this.token });
console.log(reqHeader)
return this.http.get(this.rootURL + 'download/', { headers: reqHeader});
}
To begin with your unicode error, it's because:
HttpResponse.init(content='', content_type=None, status=200,
reason=None, charset=None)
Instantiates an HttpResponse
object with the given page content and content type.
content should be an iterator or a string. If it’s an iterator, it
should return strings, and those strings will be joined together to
form the content of the response. If it is not an iterator or a
string, it will be converted to a string when accessed.
I do believe django is having trouble converting the binary data in the file to string. A more common approach when dealing with file downloads is:
response = HttpResponse(content_type="application/jpeg")
response.write(binary_data)
This works because there is a call to make_bytes behind the scenes which handles the binary data correctly.
Having said that, this is not the most efficient way to go about it. Your web app makes a request to a remote server using requests, and then passes that onto the client. Why not get your angular code to fetch the data directly from the end point?
Can't do that because you want authentication you say? Ok, How about checking the authentiation and then sending an HttpResponseDirect like this:
return HttpResponseRedirect('http://112.138.0.12:9870/webhdfs/v1/user/' + user_name + '/1.jpg?op=OPEN&user.name=username')
I am using redis with Lua to fetch sessiondata of any admin from Django project .In Django project sessiondata is encoded into base64 form.
sessiondata value is :
session_data = "NzlmZjZmNWQxMGIzNTQzMDZhNDZjNzJiZGQ4OWZiY2NjNDg0NDVlZTqAAn1xAShVEl9hdXRoX3VzZXJfYmFja2VuZHECVSlkamFuZ28uY29udHJpYi5hdXRoLmJhY2tlbmRzLk1vZGVsQmFja2VuZHEDVQ1fYXV0aF91c2VyX2lkcQSKAgEKdS4="
My Lua code to decode session data is
-- decode base64 code fetch from django session data
local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
local function dec(data)
data = string.gsub(data, '[^'..b..'=]', '')
return (data:gsub('.', function(x)
if (x == '=') then return '' end
local r,f='',(b:find(x)-1)
for i=6,1,-1 do r=r..(f%2^i-f%2^(i-1)>0 and '1' or '0') end
return r;
end):gsub('%d%d%d?%d?%d?%d?%d?%d?', function(x)
if (#x ~= 8) then return '' end
local c=0
for i=1,8 do c=c+(x:sub(i,i)=='1' and 2^(8-i) or 0) end
return string.char(c)
end))
end
when I am running print(dec(session_data)) I am getting
output = 79ff6f5d10b354306a46c72bdd89fbccc48445ee:�}q(U_auth_user_backendqU)django.contrib.auth.backends.ModelBackendqU _auth_user_idq� u.
while output suppose to be
{'_auth_user_id': 2561L, '_auth_user_backend': 'django.contrib.auth.backends.ModelBackend'}
like Django we had.
Please let me know what I am doing wrong .
A quick google search yields this blog post.
This is the (abridged, nonverifying) python code he has to do the decoding of sessiondata:
def decode(session_data, secret_key, class_name='SessionStore'):
encoded_data = base64.b64decode(session_data)
utoken, pickled = encoded_data.split(b':', 1)
return pickle.loads(pickled)
In other words, your expectations are wrong. The un-base64-ed data contains a checksum hash (79ff6...445ee) followed by a : followed by serialized (via pickle) python data (�}q(U_auth..._user_idq� u.).
If you really want to understand how to decode python picked data, see PEP 307.
If you are using django 1.5.3+ you can use json serializer so that you don't have to try to decode python pickles in lua ;) Json serializer is default in django 1.6+.