I was wondering if there is a way to iteratively write png files using PyPNG (as in, row-by-row, or chunk-by-chunk) without providing the png.Writer() with the entire data. I've looked through the documentation, but all the writer methods require all rows of the PNG at once (uses too much memory).
Thanks in advance!
If you supply an iterator, then PyPNG will use it. Full example:
#!/usr/bin/env python
import random
import png
def iter_rows():
H = 4096
for i in xrange(H):
yield [random.randrange(i+1) for _ in range(4096)]
img = png.from_array(iter_rows(), mode="L;12",
info=dict(size=(4096,4096)))
img.save("big.png")
Related
I need to perform a function on images in less than 1 second. I have a problem on a 1000x1000 image that, just to load it as a matrix in the program, takes 1 second.
The function I use to load it is as follows:
import png
def load(fname):
with open(fname, mode='rb') as f:
reader = png.Reader(file=f)
w, h, png_img, _ = reader.asRGB8()
img = []
for line in png_img:
l = []
for i in range(0, len(line), 3):
l+=[(line[i], line[i+1], line[i+2])]
img+=[l]
return img
How can I modify it in such a way that, when opening the image, it takes a little more than a few milliseconds?
IMPORTANT NOTE: I cannot import other functions outside of this (this is a university exercise and therefore there are rules -.-). So I have to get one myself
you can use PIL to do this for you, it's highly optimized and fast
from PIL import Image
def load(path):
return Image.open(path)
Appending to a list is inherently slow - read about Shlemiel the painter’s algorithm. You can replace it with a generator expression and slicing.
for line in png_img:
img += list(zip(line[0::3], line[1::3], line[2::3])
I'm not sure it is remotely possible to run a python script that opens a file, etc. in just a few ms. On my computer, the simplest program takes several 10ms
Without knowing more about the specifics of your problem and the reasons for your constraint, it is hard to answer. You should consider what you are trying to do, in the context of the way your program really works, and then formulate a strategy to achieve your goal.
The total context here is, you're asking the computer to:
run python, load your code and interpret it
load any modules you want to use
find your image file and read it from disk
give those bytes some meaning as an image abstraction - parse, etc these bytes
do some kind of transform or "work" on the image
export your result in some way
You need to figure out which of those steps is it that really needs to be lightning fast. After that, maybe someone can make a suggestion.
I am looking to convert a xml file to an image (ideally a png file) using a python script. I have not found much from my online research. I am trying to use PIL. From this post on StackOverflow I was able to find this code:
from PIL import Image
import ImageFont, ImageDraw
image = Image.new("RGBA", (288,432), (255,255,255))
usr_font = ImageFont.truetype("resources/HelveticaNeueLight.ttf", 25)
d_usr = ImageDraw.Draw(image)
d_usr = d_usr.text((105,280), "MYTEXT",(0,0,0), font=usr_font)
But I do not quite understand what's happening. I tried to replace "MYTEXT" with the actual xml file content and it did not work.
I am basically looking for any solution (ideally using PIL, but it can be another module for python). I came close using imgkit:
import imgkit
imgkit.from_file('example_IN.xml','example_OUT.png')
which returns a png file. The resolution of the image is terrible though, and it lies within a very large white rectangle. I may be missing something. I know you can modify options for imgkit, but I have no idea what modifications to bring, even after checking the documentation. Any help would be deeply appreciated.
Thank you so much!
Best regards.
I had a go in pyvips:
#!/usr/bin/env python3
import sys
import pyvips
from xml.sax.saxutils import escape
# load first arg as a string
txt = open(sys.argv[1], "r").read()
# pyvips allows pango markup in strings -- you can write stuff like
# text("hello <i>sailor!</i>")
# so we need to escape < > & in the text file
txt = escape(txt)
img = pyvips.Image.text(txt)
# save to second arg
img.write_to_file(sys.argv[2])
You can run it like this:
./txt2img.py vari.ws x.png
To make this:
It's pretty quick -- that took 300ms to run on this modest laptop.
The text method has a lot of options if you want higher res, to change the alignment, wrap lines at some limit, change the font, etc. etc.
https://libvips.github.io/libvips/API/current/libvips-create.html#vips-text
The solution suggested above by jcuppit using pyvips definitely works and is quick. I found another solution to make my previous code above work using imgkit (it is slower, I am giving it here just for reference): the resolution of the output image was bad. If this happens, width and height can be changed in the options (this is an easy fix I had missed):
import imgkit
options = {
'width' : 600,
'height' : 600
}
imgkit.from_file('example_IN.xml','example_OUT.png', options=options)
And that will convert a xml file into a png file as well.
I am trying to saving a large amount of images. I want to save them in a format that costs as less disk memory as possible. I have tested with HDF5 and cPickle in python. Surprisingly, I found out that the data files generated by PyTables and cPickle have much larger sizes than the folder that contains the same amount of images.
My code is here:
import cv2
import copy
import cPickle as pickle
import tables
import numpy as np
image = cv2.imread("aloel.jpg")
images = []
for i in xrange(1000):
images.append(copy.deepcopy(image))
images = np.asarray(images, dtype=np.uint8)
hdf5_path = "img.hdf5"
filters = tables.Filters(complevel=5, complib='blosc')
with tables.open_file(hdf5_path, mode='w', filters=filters) as hdf5_file:
data_storage = hdf5_file.create_array(hdf5_file.root, 'data', obj=images)
with open('img.pickle', 'wb') as f:
pickle.dump(images, f, protocol=pickle.HIGHEST_PROTOCOL)
The folder that contains 1000 copies of aloel.jpg consumes 61.5 MB, but the img.hdf5 and img.pickle are both 1.3GB in size.
I wonder why this occurs? If this is the case, does it mean that it would be better to directly save image data into individual image file rather than save them into a pickle file or hdf5 file?
Update:
your problem is that compression is not applied at all, because first you need to have chunking, which can be achieved by replacing "create_array" with "create_carray". Then, apply "zlib" with complevel 5 and you should see already some improvement. For this particular case, of course, it makes sense to set chunking also along the repeated data axis, so if you add something like chunkshape=[100,100,100,3] to the create_carray command, you should see a major change.
Jpeg is highly efficient lossy compression algorithm. Blosc is optimised for speed, and pickle is not compressed at all by default. There are other options for HDF5 available, take a look at https://support.hdfgroup.org/services/filters.html and I believe you can find a method that is close enough for original jpeg.
I am trying to read raw image data from a cr2 (canon raw image file). I want to read the data only (no header, etc.) pre-processed if possible (i.e pre-bayer/the most native unprocessed data) and store it in a numpy array. I have tried a bunch of libraries such as opencv, rawkit, rawpy but nothing seems to work correctly.
Any suggestion on how I should do this? What I should use? I have tried a bunch of things.
Thank you
Since libraw/dcraw can read cr2, it should be easy to do. With rawpy:
#!/usr/bin/env python
import rawpy
raw = rawpy.imread("/some/path.cr2")
bayer = raw.raw_image # with border
bayer_visible = raw.raw_image_visible # just visible area
Both bayer and bayer_visible are then a 2D numpy array.
You can use rawkit to get this data, however, you won't be able to use the actual rawkit module (which provides higher level APIs for dealing with Raw images). Instead, you'll want to use mostly the libraw module which allows you to access the underlying LibRaw APIs.
It's hard to tell exactly what you want from this question, but I'm going to assume the following: Raw bayer data, including the "masked" border pixels (which aren't displayed, but are used to calculate various things about the image). Something like the following (completely untested) script will allow you to get what you want:
#!/usr/bin/env python
import ctypes
from rawkit.raw import Raw
with Raw(filename="some_file.CR2") as raw:
raw.unpack()
# For more information, see the LibRaw docs:
# http://www.libraw.org/docs/API-datastruct-eng.html#libraw_rawdata_t
rawdata = raw.data.contents.rawdata
data_size = rawdata.sizes.raw_height * rawdata.sizes.raw_width
data_pointer = ctypes.cast(
rawdata.raw_image,
ctypes.POINTER(ctypes.c_ushort * data_size)
)
data = data_pointer.contents
# Grab the first few pixels for demonstration purposes...
for i in range(5):
print('Pixel {}: {}'.format(i, data[i]))
There's a good chance that I'm misunderstanding something and the size is off, in which case this will segfault eventually, but this isn't something I've tried to make LibRaw do before.
More information can be found in this question on the LibRaw forums, or in the LibRaw struct docs.
Storing in a numpy array I leave as an excersize for the user, or for a follow up answer (I have no experience with numpy).
I have a .tar file containing several hundreds of pictures (.png). I need to process them via opencv.
I am wondering whether - for efficiency reasons - it is possible to process them without passing by the disc. In other, words I want to read the pictures from the memory stream related to the tar file.
Consider for instance
import tarfile
import cv2
tar0 = tarfile.open('mytar.tar')
im = cv2.imread( tar0.extractfile('fname.png').read() )
The last line doesn't work as imread expects a file name rather than a stream.
Consider that this way of reading directly from the tar stream can be achieved e.g. for text (see e.g. this SO question).
Any suggestion to open the stream with the correct png encoding?
Untarring to ramdisk is of course an option, although I was looking for something more cachable.
Thanks to the suggestion of #abarry and this SO answer I managed to find the answer.
Consider the following
def get_np_array_from_tar_object(tar_extractfl):
'''converts a buffer from a tar file in np.array'''
return np.asarray(
bytearray(tar_extractfl.read())
, dtype=np.uint8)
tar0 = tarfile.open('mytar.tar')
im0 = cv2.imdecode(
get_np_array_from_tar_object(tar0.extractfile('fname.png'))
, 0 )
Perhaps use imdecode with a buffer coming out of the tar file? I haven't tried it but seems promising.