I just want to add new facial encoding to the pickle file. I've tried the following the method but it's not working.
creating of the pickle file
import face_recognition
import pickle
all_face_encodings = {}
img1 = face_recognition.load_image_file("ex.jpg")
all_face_encodings["ex"] = face_recognition.face_encodings(img1)[0]
img2 = face_recognition.load_image_file("ex2.jpg")
all_face_encodings["ex2"] = face_recognition.face_encodings(img2)[0]
with open('dataset_faces.dat', 'wb') as f:
pickle.dump(all_face_encodings, f)
Appending new data to the pickle file.
import pickle
img3 = face_recognition.load_image_file("ex3.jpg")
all_face_encodings["ex3"] = face_recognition.face_encodings(img3)[0]
with open('dataset_faces.dat', 'wb') as f:
pickle.dump(img3,f)
pickle.dump(all_face_encodings["ex3"],f)
But It's not working. Is there a way to append it?
I guess your steps should be:
Load old pickle-data to memory to <all_face_encodings> dictionary;
Add a new encodings to the dictionary.
Dump the whole dictionary to pickle file again.
Related
I would like to combine two pickle files into a new onethat both come from face recognition. I have a main actors.pkl file that one holds a lot of actors, now I would like to update this file but not using the imageset update methode. So I am creating a new picklefile actorupdate.pkl in this set are the registered faces not in actors.pkl
I am able to open / read both files and they both work perfect (recognition wise) so how would I be able to join both files into a new one (or append actorupdate.pkl to actors.pkl)
import pickle
data = pickle.loads(open("encodings/actors.pkl", "rb").read())
data2 = pickle.loads(open("encodings/actorupdate.pkl", "rb").read())
verify dataset (names):
data['names'] = ['ben_afflek', 'ben_afflek', 'ben_afflek', 'ben_afflek', 'ben_afflek', 'ben_afflek', 'ben_afflek', 'ben_afflek', ........
data2['names'] = ['Gal_Gadot', 'Gal_Gadot', 'Gal_Gadot', 'Gal_Gadot', 'Gal_Gadot', 'Gal_Gadot']
trying to use this methode:
filename="picklefile"
with open(filename, 'wb') as fp:
pickle.dump(data ,fp)
pickle.dump(data2 ,fp)
but in that case it writes a new file to my disk but the information from data2 is not there, it became just a copy of data and I would like to use the file directly not create a new pickle file on my disk.
You can simply append second list into first one and then just dumps it
I found the following worked (had to loop over the contents of the data2):
data1 = pickle.load(open('encodings/actors.pkl', 'rb'))
data2 = pickle.loads(open("encodings/actorupdate.pkl", "rb").read())
new_encodings = data1['encodings']
new_names = data1['names']
for encode in data2['encodings']:
new_encodings.append(encode)
for name in data2['names']:
new_names.append(name)
data = {"encodings": new_encodings, "names": new_names}
with open(data1 , 'wb') as fp:
pickle.dump(data, fp)
data now holds both pickle files and allows to detect the faces from both pickle files, the file data1 itself is overwritten with the updated dataset (data2) on disk.
From what I understand, the only way to edit an object in a pickle file is to unpickle each object, edit the desired object, and repickle everything back into the original file.
This is what I tried doing:
pickleWrite = open(fileName, 'wb')
pickleRead = open(fileName, 'rb')
#unpickle objects and put it in dataList
dataList = list()
try:
while True:
dataList.append(pickle.load(pickleRead))
except EOFError:
pass
#change desired pickle object
dataList[0] = some change
#clear pickle file
pickleWrite.truncate(0)
#repickle each item in data list
for data in dataList:
pickle.dump(data, fileName)
For some reason, this makes the pickle file have some large number of unknown symbols at the front of the file making it unpickleable.
Error when we try to unpickle:
_pickle.UnpicklingError: invalid load key, '\x00'.
I would suggest avoiding to create multiple opened connection to the same file like this.
Instead, you can try:
# Read the contents
with open(filename, 'rb') as file:
dataList = pickle.load(file)
# something to dataList
# Overwrite the picle file
with open(filename, 'wb') as file:
pickle.dump(dataList, file)
i have a dataset and i am saving the result of the encoding of the images for fave recognition in pickle object.
i would like to add new images or delete images in database and when i do it then prior images that exist in the database are stored in dataset_faces.dat and only for new images encode_faces.py be done.
I want to reduce the time to save the encoding in the encoding.pickle.
Otherwise, a lot of time should be spent even adding a new image.
encode_faces.py
import face_recognition
import numpy as np
import os
import pickle
known_person = []
known_image= []
known_face_encoding=[]
for file in os.listdir("Imagefolder"):
#Extracting person name from the image filename eg:Abhilash.jpg
known_person.append(str(file).replace(".jpg", ""))
file=os.path.join("Imagefolder", file)
known_image = face_recognition.load_image_file(file)
known_face_encoding.append(face_recognition.face_encodings(known_image)[0])
with open('dataset_faces.dat', 'wb') as f:
pickle.dump(known_face_encoding, f,pickle.HIGHEST_PROTOCOL)
with open('dataset_fac.dat', 'wb') as d:
pickle.dump(known_person, d)
print(known_face_encoding)
print(known_person)
I need to append many binary files in one binary file. All my binary files are saved i one folder:
file1.bin
file2.bin
...
For that I try by using this code:
import numpy as np
import glob
import os
Power_Result_File_Path ="/home/Deep_Learning_Based_Attack/Test.bin"
Folder_path =r'/home/Deep_Learning_Based_Attack/Test_Folder/'
os.chdir(Folder_path)
npfiles= glob.glob("*.bin")
loadedFiles = [np.load(bf) for bf in binfiles]
PowerArray=np.concatenate(loadedFiles, axis=0)
np.save(Power_Result_File_Path, PowerArray)
It gives me this error:
"Failed to interpret file %s as a pickle" % repr(file))
OSError: Failed to interpret file 'file.bin' as a pickle
My problem is how to concatenate binary file it is not about anaylysing every file indenpendently.
Taking your question literally: Brute raw data concatenation
files = ['my_file1', 'my_file2']
out_data = b''
for fn in files:
with open(fn, 'rb') as fp:
out_data += fp.read()
with open('the_concatenation_of_all', 'wb') as fp:
fp.write(out_data)
Comment about your example
You seem to be interpreting the files as saved numpy arrays (i.e. saved via np.save()). The error, however, tells me that you didn't save those files via numpy (because it fails decoding them). Numpy uses pickle to save and load, so if you try to open a random non-pickle file with np.load the call will throw an error.
for file in files:
async with aiofiles.open(file, mode='rb') as f:
contents = await f.read()
if file == files[0]:
write_mode = 'wb' # overwrite file
else:
write_mode = 'ab' # append to end of file
async with aiofiles.open(output_file), write_mode) as f:
await f.write(contents)
I want to encode an image into a string using the base64 module. I've ran into a problem though. How do I specify the image I want to be encoded? I tried using the directory to the image, but that simply leads to the directory being encoded. I want the actual image file to be encoded.
EDIT
I tried this snippet:
with open("C:\Python26\seriph1.BMP", "rb") as f:
data12 = f.read()
UU = data12.encode("base64")
UUU = base64.b64decode(UU)
print UUU
self.image = ImageTk.PhotoImage(Image.open(UUU))
but I get the following error:
Traceback (most recent call last):
File "<string>", line 245, in run_nodebug
File "C:\Python26\GUI1.2.9.py", line 473, in <module>
app = simpleapp_tk(None)
File "C:\Python26\GUI1.2.9.py", line 14, in __init__
self.initialize()
File "C:\Python26\GUI1.2.9.py", line 431, in initialize
self.image = ImageTk.PhotoImage(Image.open(UUU))
File "C:\Python26\lib\site-packages\PIL\Image.py", line 1952, in open
fp = __builtin__.open(fp, "rb")
TypeError: file() argument 1 must be encoded string without NULL bytes, not str
What am I doing wrong?
I'm not sure I understand your question. I assume you are doing something along the lines of:
import base64
with open("yourfile.ext", "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
You have to open the file first of course, and read its contents - you cannot simply pass the path to the encode function.
Edit:
Ok, here is an update after you have edited your original question.
First of all, remember to use raw strings (prefix the string with 'r') when using path delimiters on Windows, to prevent accidentally hitting an escape character. Second, PIL's Image.open either accepts a filename, or a file-like (that is, the object has to provide read, seek and tell methods).
That being said, you can use cStringIO to create such an object from a memory buffer:
import cStringIO
import PIL.Image
# assume data contains your decoded image
file_like = cStringIO.StringIO(data)
img = PIL.Image.open(file_like)
img.show()
The first answer will print a string with prefix b'.
That means your string will be like this b'your_string' To solve this issue please add the following line of code.
encoded_string= base64.b64encode(img_file.read())
print(encoded_string.decode('utf-8'))
I have experienced this while converting Image to Base64 string. You can take a look at how I removed that from there also. Link is here Image to base64 string and fix 'b from prefix
import base64
from PIL import Image
from io import BytesIO
with open("image.jpg", "rb") as image_file:
data = base64.b64encode(image_file.read())
im = Image.open(BytesIO(base64.b64decode(data)))
im.save('image1.png', 'PNG')
Borrowing from what Ivo van der Wijk and gnibbler have developed earlier, this is a dynamic solution
import cStringIO
import PIL.Image
image_data = None
def imagetopy(image, output_file):
with open(image, 'rb') as fin:
image_data = fin.read()
with open(output_file, 'w') as fout:
fout.write('image_data = '+ repr(image_data))
def pytoimage(pyfile):
pymodule = __import__(pyfile)
img = PIL.Image.open(cStringIO.StringIO(pymodule.image_data))
img.show()
if __name__ == '__main__':
imagetopy('spot.png', 'wishes.py')
pytoimage('wishes')
You can then decide to compile the output image file with Cython to make it cool. With this method, you can bundle all your graphics into one module.
As I said in your previous question, there is no need to base64 encode the string, it will only make the program slower. Just use the repr
>>> with open("images/image.gif", "rb") as fin:
... image_data=fin.read()
...
>>> with open("image.py","wb") as fout:
... fout.write("image_data="+repr(image_data))
...
Now the image is stored as a variable called image_data in a file called image.py
Start a fresh interpreter and import the image_data
>>> from image import image_data
>>>
Its work for me
import base64
import requests
# Getting image in bytes
response = requests.get("image_url")
# image encoding
encoded_image = base64.b64encode(response.content)
# image decoding and without it's won't work due to some '\xff' error
decoded_image= base64.b64decode(encoded_image)