I'm working on a script, which builds an image, combines it with another image and saves it locally as an 8-bit BMP-file.
The image is then read by a ESP32 microcontroller, but the problem is that due to memorylimitations, the allowed file size is somewhat limited.
As a consequence, I made a BMP decoder for the ESP32, which supports RLE. In theory, the allowed number of bytes can still be exceeded, but only text and simple icons are to be read, so it will most likely never happen.
It uses Pillow for image processing, which now supports RLE-compression from version 9.1.0
https://github.com/python-pillow/Pillow/blob/main/docs/handbook/image-file-formats.rst
Pillow reads and writes Windows and OS/2 BMP files containing 1, L, P,
or RGB data. 16-colour images are read as P images. Support for
reading 8-bit run-length encoding was added in Pillow 9.1.0. Support
for reading 4-bit run-length encoding was added in Pillow 9.3.0.
Here's the part of the code, that combines two existing images into a new one and saves them:
img_buf = io.BytesIO() # Start converting from Matplotlib to PIL
# Supported: eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff, webp
plt.savefig(img_buf, format='png', transparent=True)
graph = Image.open(img_buf)
# Create empty, 8-bit canvas
new_image = Image.new('P',(600,448), (255,255,255)) # P = 8-bit indexed
new_image.paste(img,(0,0)) # Insert image 1 into canvas
new_image.paste(graph,(0,200)) # Insert image 2 into canvas at y:200
new_image.save("../output/priceeast.bmp", compression=1) # According to the docs, 1 = RLE
It saves the image, alright, but not RLE-encoded and I can't work out, how to enable it... or is RLE only supported when reading BMP, not saving?
UPDATE:
I added this line below:
subprocess.call('magick ../output/priceeast.png -type palette -compress RLE ../output/priceeast.bmp ', shell=True)
Pillow does not support writing BMP files with compression, which can be determined by investigating the source. BmpImagePlugin._write:
# bitmap info header
fp.write(
o32(header) # info header size
+ o32(im.size[0]) # width
+ o32(im.size[1]) # height
+ o16(1) # planes
+ o16(bits) # depth
+ o32(0) # compression (0=uncompressed)
+ o32(image) # size of bitmap
+ o32(ppm[0]) # resolution
+ o32(ppm[1]) # resolution
+ o32(colors) # colors used
+ o32(colors) # colors important
)
We can see here that the compression field in the output file header is hard-coded to none, indicating that compression is not supported when writing a file.
If you would like a work-around, ImageMagick can convert any image format to 8-bit RLE BMP like this:
magick INPUTIMAGE.xxx -type palette -compress RLE result.bmp # where XXX is PNG, JPG, TIFF, GIF, TGA etc
Check the result with exiftool like this:
exiftool -filename -filesize -compression result.bmp
File Name : result.bmp
File Size : 1168 bytes
Compression : 8-Bit RLE
Note that there are Python bindings to ImageMagick via wand, so you can achieve the same effect in Python like this:
#!/usr/bin/env python3
# You may need this before running on macOS
# export MAGICK_HOME=/opt/homebrew
from wand.image import Image
# Load image , or create pseudo image
with Image(filename='gradient:red-blue') as img:
img.type = 'palette'
img.compression = 'rle'
img.save(filename='result.bmp')
Related
Sorry for my english but it's not my first language.
I would like to create a program that:
Transform a jpeg or png image into an array (very important: I would like an array composed only of the values that the pixels of the image have and not metadata or other information. Where I can select each specific pixel of the image).
Save this array in a txt file.
Transform this array composed of only the pixel values of the image back into jpg or png image and save it in a file.
Requests:
Is the array I created with the program I wrote composed only of the pixel values of the image? is there also metadata or other information?
Is this a valid way to remove metadata from an image?
Is this a valid way to create the array representing that image pixel by pixel?
Is this a valid way to convert png images to jpeg or jpeg to png?
Thank you!
This is the program I created, any opinion?
import numpy as np
from PIL import Image
import sys
img_data = Image.open("imagea.jpeg")
img_arr = np.array(img_data)
np.set_printoptions(threshold=sys.maxsize)
print(img_arr.shape)
new_img = Image.fromarray(img_arr)
new_img.save("imageb.jpeg")
print("Image saved!")
file = open("file1.txt", "w+")
content = str(img_arr)
file.write(content)
file.close()
print("Finished!")
Loading an image and converting it to a Numpy array is a perfectly legitimate way of discarding all metadata including:
EXIF data, copyright data,
IPTC and XMP data,
ICC colour profile data
You can tell it's all gone by thinking about the Numpy array you hold and its dimensions and data type.
Note that you need to be careful with PNG palette images and images with an alpha channel.
Note that you can achieve this more simply on the command-line with ImageMagick using:
magick mogrify -strip IMAGE.JPG
Or with exiftool.
Note that you can achieve this by using a format that doesn't support metadata, such as NetPBM, with extension .ppm e.g.:
magick INPUT.JPG -strip -compress none RESULT.PPM # gives P3/plain ASCII file
magick INPUT.JPG -strip RESULT.PPM # gives P6/binary file
You can also read/write PPM files with PIL.
I created a greyscale image like this
def create_new_image(size, luminance):
width, height = size
black_frame = int(luminance) * np.ones((width, height, 1), dtype=np.uint8)
return black_frame
Where luminance is element of [0, 255]
I have saved the image using imageio
def save_image(image, output_path):
imageio.imwrite(output_path, image)
Where the output_path is something like /valid_path/img.jpg
Now I want to load my grayscale image back:
img = imageio.imread(file, format ='jpg')
But what I get is a syntax error.
raise SyntaxError("not a JPEG file")
File "<string>", line None
SyntaxError: not a JPEG file
If I don't specify the format, I get another error.
"Could not find a format to read the specified file in %s mode" % modename
ValueError: Could not find a format to read the specified file in single-image mode
Why?
Thanks
You can try :
def save_image(image, output_path):
imageio.imwrite(output_path, format= "jpg", image)
to explicitly state that it is a jpg file.
JPEG files (compressed images) start with an image marker that always contains the marker code hex values FF D8 FF. It does not have a length of the file embedded, thus we need to find JPEG trailer, which is FF D9.
See the documentation using the link at this page.
As en example, opening a jpeg image with a hexadecimal viewer (for example Hex Viewer), you should see something like this:
Solution: In other words, try to add the header to the file before saving it as JPEG, you should solve your problem.
The page with the API's documentation can be found here. Following the doc, you should locate the right instruction that makes you specify the format for saving (as point out by #Meto in the answer).
Concluding: the solution is just specifying the format when you physically write the image in the hard disk:
imageio.imwrite(uri, im, format=None, **kwargs)
in your case format=jpg.
Moreover,
imageio.show_formats()
Show a nicely formatted list of available formats.
Concluding, just try to replace
imageio.imwrite(output_path, image)
with
imageio.imwrite(output_path, image, format ='jpg' )
Please note that the solution is always the same in every answer. I have just added what happens specifying a format (i.e., just writes the right header).
You need to make sure if your file is really saved as a JPG file.
On Linux/Mac you can use file command to verify that.
For example, below command confirms fireside.jpg is a JPEG file:
# file fireside.jpg
fireside.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 2048x1365, components 3
If the file is not saved as JPG, try specifying file format="jpg" as
imageio.imwrite(output_path, image, format ='jpg')
This works (Verified in jupyter notebook)
import numpy as np
import imageio
def create_new_image(size, luminance):
width, height = size
black_frame = int(luminance) * np.ones((width, height, 1), dtype=np.uint8)
return black_frame
def save_image(image, output_path):
imageio.imwrite(output_path, image)
img = create_new_image((256, 256), 125)
save_image(img, "test.jpg")
img1 = imageio.imread("test.jpg", format ='jpg')
Given the short, 5 page PDF file (attached at the bottom), and the following python code to convert to a multi-page TIFF:
from wand.image import Image
with Image(filename='5-page-pdf.pdf', resolution=200) as img:
img.type = "grayscale"
img.format = "tiff"
img.compression = "lzw"
img.save(filename="test.tiff")
results in a TIFF file that has pages 2-4 as what appears to be black text on a dark-grey (or maybe transparent) background. Other image processing libraries cannot open the file or render it.
Converting the same PDF with ImageMagick, which Wand uses, works just fine
convert -density 200 5-page-pdf.pdf -type grayscale -compress lzw 5-page-pdf.tiff
this produces a file that does work with other imaging libraries and looks correct in a TIFF viewer.
I've tried removing the alpha channel, I've tried setting the background color to 'White', and a few other things, to no avail. The TIFF that comes out of Wand is always garbled. If it's doable in ImageMagick it should be doable in Wand, right? What parameter or setting am I missing?
Original PDF
Wand Produced TIFF
Looks like setting the img.alpha_channel property is not propagating across the pages.
Try this workaround
from wand.api import library
from wand.image import Image
with Image(filename="5-page-pdf.pdf", resolution=200) as img:
img.type = 'grayscale'
img.compression = "lzw"
# Manually iterate over all page, and turn off alpha channel.
library.MagickResetIterator(img.wand)
for idx in range(library.MagickGetNumberImages(img.wand)):
library.MagickSetIteratorIndex(img.wand, idx)
img.alpha_channel = 'off'
img.save(filename="test.tiff")
What I'm actually doing is saving images in a tiff file, using imageio.mimwrite(). But in my script, I open and close the file several times, so it erases existing images before saving news images. I want to keep existing images in the tiff file, and just add new images without erasing the previous ones. I did not find anything in the documentation which can helps me.
I'm actually using this :
imageio.mimwrite("example.tiff", image, format=".tiff")
image is an array which contains arrays of integers, each array representing an image.
This code opens example.tiff, erase existing images (if they exist), and write news images. But I want to add like open("file.txt", "a") does.
I made three differently sized TIFF images with ImageMagick like this for testing:
convert -size 640x480 xc:green green.tif
convert -size 1024x768 xc:blue blue.tif
convert -size 400x100 gradient:cyan-yellow gradient.tif
Then I used the tool tiffcp which is distributed with the TIFF library and the -a option to append the blue and gradient image to the green one like this:
tiffcp -a blue.tif gradient.tif green.tif
If I then check the contents of green.tiff with ImageMagick identify, I see it looks correct:
magick identify green.tif
green.tif[0] TIFF 640x480 640x480+0+0 16-bit sRGB 6.49355MiB 0.000u 0:00.000
green.tif[1] TIFF 1024x768 1024x768+0+0 16-bit sRGB 0.000u 0:00.000
green.tif[1] TIFF 400x100 400x100+0+0 16-bit sRGB 0.000u 0:00.000
And if I preview the file, all three images are there with the correct sizes and colours:
So, I am suggesting you consider using subprocess.run() to shell out to tiffcp.
With tifffile writing one page at a time (in the example a CTYX multipage), you can just write straight from a n-D array if you have enough RAM for it with tifffile.imwrite(filename,array)
https://pypi.org/project/tifffile/ :
import tifffile as tf
with tf.TiffWriter("filenametest.tiff",
#bigtiff=True,
#If you want to add on top of an existing tiff file (slower) uncomment below
#append = True,
imagej=False,) as tif:
for time in range(rgb.shape[1]):
tif.save(rgb[:,time,:,:].,
#compress= 3,
photometric='minisblack',
metadata= None,
contiguous=False,
)
tif.close()
With python-bioformats:
https://pythonhosted.org/python-bioformats/
bioformats.write_image(pathname, pixels, pixel_type, c=0, z=0, t=0, size_c=1, size_z=1, size_t=1, channel_names=None)[source]
if you have:
4 time points with
3 color zstacks (11 Z's)
with XY at 1024 pixels.
with independent numpy arrays being [3,11,1024,1024] (one for each timepoint),
16bits,
and named: a,b,c,d.
This should do the trick
import bioformats as bf
import numpy as np
#do something here to load a,b,c, and d
i=0
for t in [a,b,c,d]:
for c in range(3):
for z in range(11):
#t here is the numpy array
bf.write_image('/path/to/file.tiff'
, t
, bf.PT_UINT16
, c = c, z = z , t = i, size_c= 3, size_z=11, size_t=4
)
i+=1
I am working with python to convert an RGBA tiff to an RGB tiff with a white background.
I am using the library ImageMagick and GDAL.
My code looks like that:
def add_background_to_rgba_geotiff(source, destination):
convert_rgba_to_rgb_tif(source, destination)
add_metadata_to_new_geotiff_file(source, destination)
def convert_rgba_to_rgb_tif(source, destination):
# work also with BigTiff
command = ' '.join(['convert', quote(source),
'-background', 'white',
'-alpha', 'background',
'-alpha', 'off', quote(destination)])
shell_command.execute_and_log_outputs(command, shell=True)
def add_metadata_to_new_geotiff_file(source, destination):
RGBA_tif = gdal.Open(source, gdalconst.GA_ReadOnly)
RGB_tif = gdal.Open(destination, gdalconst.GA_Update)
RGB_tif.SetMetadata(RGBA_tif.GetMetadata())
RGB_tif.SetGeoTransform(RGBA_tif.GetGeoTransform())
RGB_tif.SetProjection(RGBA_tif.GetProjection())
del (RGBA_tif)
del (RGB_tif)
def execute_and_log_outputs(command, silence_errors=False, **kwargs):
shell_process = execute_async(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
**kwargs)
(out, err) = shell_process.communicate()
When I try to run my code with a tiff everything happens well but when I want to apply that on a BigTIFF (>4GB) it fails with this error:
TIFFWriteDirectoryTagData: Maximum TIFF file size exceeded.
Does someone know how to use BigTiff with ImageMagick?
Or could it be possible to do that with GDAL? For the moment when I try with GDAL, I only achieved to have a black background.
Thanks for your help.
If you check the available formats that ImageMagick knows, you will see a TIFF64:
identify -list format | grep -i tiff
EPT EPT rw- Encapsulated PostScript with TIFF preview
EPT2 EPT rw- Encapsulated PostScript Level II with TIFF preview
EPT3 EPT rw+ Encapsulated PostScript Level III with TIFF preview
GROUP4* TIFF rw- Raw CCITT Group4
PTIF* TIFF rw+ Pyramid encoded TIFF
TIFF* TIFF rw+ Tagged Image File Format (LIBTIFF, Version 4.0.9)
TIFF64* TIFF rw- Tagged Image File Format (64-bit) (LIBTIFF, Version 4.0.9)
If you try to make a TIFF like this it fails:
convert -size 20000x50000 xc:red +noise random a.tif
convert: Maximum TIFF file size exceeded. `TIFFAppendToStrip' # error/tiff.c/TIFFErrors/652.
So, force the TIFF64 delegate by prepending it to the output filename to get a BigTIFF like this:
convert -size 20000x50000 xc:red +noise random TIFF64:a.tif
and it works.