Develop Reference

Django convert image to webp

I have service in my Django project's app, that upload images, and I need to convert all images to webp to optimize further work with these files on the frontend side.
Draft of _convert_to_webp method:
# imports
from pathlib import Path
from django.core.files import temp as tempfile
from django.core.files.uploadedfile import InMemoryUploadedFile
from PIL import Image
# some service class
...
def _convert_to_webp(self, f_object: InMemoryUploadedFile):
new_file_name = str(Path(f_object._name).with_suffix('.webp'))
temp_file = tempfile.NamedTemporaryFile(suffix='.temp.webp')
# FIXME: on other OS may cause FileNotFoundError
with open(temp_file 'wb') as f:
for line in f_object.file.readlines():
... # will it works good?
new_file = ...
new_f_object = InMemoryUploadedFile(
new_file,
f_object.field_name,
new_file_name,
f_object.content_type,
f_object.size,
f_object.charset,
f_object.content_type_extra
)
return new_file_name, new_f_object
...
f_object is InMemoryUploadedFile instance from POST request body (Django automatically create it).
My idea is to create a temporary file, write data from f_object.file.readlines() to it, open this file with PIL.Image.open and save with format="webp". Is this idea a good one or there is another way to make file converting?

I found a pretty clean way to do this using the django-resized package.
After pip installing, I just needed to swap out the imageField for a ResizedImageField
img = ResizedImageField(force_format="WEBP", quality=75, upload_to="post_imgs/")
All image uploads are automatically converted to .webp!

The solution was pretty simple. PIL.Image can be opened using file instance, so I just opened it using f_object.file and then saved it in BytesIO instance with optimization and compression.
Correctly working code:
# imports
from pathlib import Path
from django.core.files.uploadedfile import InMemoryUploadedFile
from PIL import Image
# some service class
...
def _convert_to_webp(self, f_object: InMemoryUploadedFile):
suffix = Path(f_object._name).suffix
if suffix == ".webp":
return f_object._name, f_object
new_file_name = str(Path(f_object._name).with_suffix('.webp'))
image = Image.open(f_object.file)
thumb_io = io.BytesIO()
image.save(thumb_io, 'webp', optimize=True, quality=95)
new_f_object = InMemoryUploadedFile(
thumb_io,
f_object.field_name,
new_file_name,
f_object.content_type,
f_object.size,
f_object.charset,
f_object.content_type_extra
)
return new_file_name, new_f_object
95% was chosen as balanced parameter. There was very bad quality with quality=80 or quality=90.

How to load a .sigmf-meta file as JSON?

I'm trying to load metadata from a file with a SigMF specification as a JSON using python. Here's my code so far:
import json
f_path = "test.sigmf-meta"
sigmf_meta_f = open(f_path,)
sigmf_data = json.load(sigmf_meta_f)
for i in sigmf_data['sample_text']:
print(i)
sigmf_meta_f.close()
This doesn't seem to work for some reason.
When I change the file extension from "sigmf-meta" to "json", it works perfectly, but I need to be able to load these SigMF files without having to change all of their extensions.

Are you sure you changed the extension of the file in its properties to sigmf-meta?
I just tried it it worked just fine, you may have not changed the extension but just the name so its test.sigmf-meta.json and there's no directory of test.sigmf-meta.

As per GNU Radio / SigMF: how to read sigmf-data file? #114
import json
import numpy as np
with open("myrecord.sigmf-meta", "r") as f:
md = json.loads(f.read())
if md["global"]["dtype"] == "cf32_le":
samples = np.memmap("myrecord.sigmf-data", mode="r", dtype=np.complex64)
elif md["global"]["dtype"] == "ci16_le":
samples = np.memmap("myrecord.sigmf-data", mode="r", dtype=np.int16)
# Convert samples to float if you want...

Unable to read image data when converting from PDF to Image

I am trying to convert the PDF to Image to proceed further with the Tesseract. It works when I convert using cmd:
magick convert a.pdf b.png
But doesn't work when I try to do the same using Python:
from wand.image import Image
with Image (filename='a.pdf') as img:
img.save(filename = 'sample.png')`
The error I get is:
unable to read image data D:/Users/UserName/AppData/Local/Temp/magick-4908Cq41DDA5FxlX1 # error/pnm.c/ReadPNMImage/1346
I have also installed ghostscipt but the error is still there.
EDIT:
I took the code provided in the reply below and modified it to read all the pages. The original issue is still there and the code below uses pdf2image:
from pdf2image import convert_from_path
import os
pdf_dir = "D:/Users/UserName/Desktop/scraping"
for pdf_file in os.listdir(pdf_dir):
if pdf_file.endswith(".pdf"):
pages = convert_from_path(pdf_file, 300)
pdf_name = pdf_file[:-4]
for page in pages:
page.save("%s-page%d.jpg" % (pdf_name, pages.index(page)), "JPEG")

Instead of using wand.image, you can use pdf2image. Install it like this:
pip install pdf2image
Here is a code that loops through every page in the PDF, finally converting them to JPEG:
import os
import tempfile
from pdf2image import convert_from_path
filename = 'target.pdf'
with tempfile.TemporaryDirectory() as path:
images_from_path = convert_from_path(filename, output_folder=path, last_page=1, first_page =0)
base_filename = os.path.splitext(os.path.basename(filename))[0] + '.jpg'
save_dir = 'dir'
for page in images_from_path:
page.save(os.path.join(save_dir, base_filename), 'JPEG')

How to handle "OSError: Cannot seek back after getting firstbytes" when trying to extract jpg files into numpy array

I'm trying to collect all images in a zip file into a numpy array.
Always getting an error:
OSError: Cannot seek back after getting firstbytes!
import urllib.request
import os
import zipfile
import scipy
import numpy as np
import pandas as pd
import glob
import imageio
from os.path import splitext
url = 'https://github.com/yoavram/Sign-Language/raw/master/Dataset.zip'
filename = '../data/sign-lang'
if not os.path.exists('../data'):
os.mkdir('../data')
if not os.path.exists(filename):
urllib.request.urlretrieve(url, filename)
zf = zipfile.ZipFile(filename)
for file in zf.namelist():
basename,extension = splitext(file)
if extension == '.jpg':
with zf.open(file) as img_file:
img = imageio.read(img_file)
Help?

I had a similar problem for the longest time.
The problem is that imageio.read() expects bytes but is being provided with a file-type object.
To fix this, simply read the bytes from the file.
img = imageio.read(img_file.read())
Also, if you want numpy arrays you should use imageio.imread()

reduce time of save encodings face recognition

i have a dataset and i am saving the result of the encoding of the images for fave recognition in pickle object.
i would like to add new images or delete images in database and when i do it then prior images that exist in the database are stored in dataset_faces.dat and only for new images encode_faces.py be done.
I want to reduce the time to save the encoding in the encoding.pickle.
Otherwise, a lot of time should be spent even adding a new image.
encode_faces.py
import face_recognition
import numpy as np
import os
import pickle
known_person = []
known_image= []
known_face_encoding=[]
for file in os.listdir("Imagefolder"):
#Extracting person name from the image filename eg:Abhilash.jpg
known_person.append(str(file).replace(".jpg", ""))
file=os.path.join("Imagefolder", file)
known_image = face_recognition.load_image_file(file)
known_face_encoding.append(face_recognition.face_encodings(known_image)[0])
with open('dataset_faces.dat', 'wb') as f:
pickle.dump(known_face_encoding, f,pickle.HIGHEST_PROTOCOL)
with open('dataset_fac.dat', 'wb') as d:
pickle.dump(known_person, d)
print(known_face_encoding)
print(known_person)