Given a tensorflow event file, how can I extract images corresponding to a specific tag, and then save them to disk in a common format e.g. .png?
You could extract the images like so. The output format may depend on how the image is encoded in the summary, so the resulting write to disk may need to use another format besides .png
import os
import scipy.misc
import tensorflow as tf
def save_images_from_event(fn, tag, output_dir='./'):
assert(os.path.isdir(output_dir))
image_str = tf.placeholder(tf.string)
im_tf = tf.image.decode_image(image_str)
sess = tf.InteractiveSession()
with sess.as_default():
count = 0
for e in tf.train.summary_iterator(fn):
for v in e.summary.value:
if v.tag == tag:
im = im_tf.eval({image_str: v.image.encoded_image_string})
output_fn = os.path.realpath('{}/image_{:05d}.png'.format(output_dir, count))
print("Saving '{}'".format(output_fn))
scipy.misc.imsave(output_fn, im)
count += 1
And then an example invocation may look like:
save_images_from_event('path/to/event/file', 'tag0')
Note that this assumes the event file is fully written -- in the case that it's not, some error handling is probably necessary.
For those who can also do without code, there is an elegant way in the Tensorboard UI.
In the upper left corner, select the checkbox Show data download links
In the lower left corner, select the download icon which lets you download a svg file.
In the lower right corner, select data download links for the raw data, handy for those who want to do more sophisticated data analysis or data visualization
If you are using TensorFlow 2, this works nicely
from collections import defaultdict, namedtuple
from typing import List
import tensorflow as tf
TensorBoardImage = namedtuple("TensorBoardImage", ["topic", "image", "cnt"])
def extract_images_from_event(event_filename: str, image_tags: List[str]):
topic_counter = defaultdict(lambda: 0)
serialized_examples = tf.data.TFRecordDataset(event_filename)
for serialized_example in serialized_examples:
event = event_pb2.Event.FromString(serialized_example.numpy())
for v in event.summary.value:
if v.tag in image_tags:
if v.HasField('tensor'): # event for images using tensor field
s = v.tensor.string_val[2] # first elements are W and H
tf_img = tf.image.decode_image(s) # [H, W, C]
np_img = tf_img.numpy()
topic_counter[v.tag] += 1
cnt = topic_counter[v.tag]
tbi = TensorBoardImage(topic=v.tag, image=np_img, cnt=cnt)
yield tbi
Although, 'v' has an image field, it is empty.
I used
tf.summary.image("topic", img)
to add the images to the event file.
Related
New to python, new to OpenCV, which I'm gonna use for my master-thesis, and already got some problems using the VideoCapture object of OpenCV.
Situation:
I got 2 folders containing corresponding images (taken with RGB and infrared cameras). I want to display them sibe by side in a Window using a while-loop. The problem arises, when there are some images missing from one of the image-sequences (Due to problems while recording or whatever, I don't really know but that should be of no importance). My idea was to use the bool-returnvalue of the .read() function to check wheather there is a frame to be read and if not, replace the image by a black one. This is what I did:
Code:
import cv2
import numpy as np
pathRGB = "Bilder/RGB"
pathIR = "Bilder/IR"
# the paths to the folders containing the images
capRGB = cv2.VideoCapture(pathRGB + "/frame_%06d.jpg")
capIR = cv2.VideoCapture(pathIR + "/frame_%06d.jpg")
# setting up the VideoCapture-elements with the according format
shapeRGB = capRGB.read()[1].shape
shapeIR = capIR.read()[1].shape
# get the shape of the first image in each folder to later create the black
# dummy-image
dtypeRGB = capRGB.read()[1].dtype
dtypeIR = capIR.read()[1].dtype
# get the type of the first image in each folder to later create the black
# dummy-image
if (capRGB.isOpened() is False):
print("Error opening RGB images")
if (capIR.isOpened() is False):
print("Error opening IR images")
cv2.namedWindow("frames", cv2.WINDOW_NORMAL)
while capRGB.isOpened() and capIR.isOpened() is True:
retRGB, imgRGB = capRGB.read()
retIR, imgIR = capIR.read()
# read both images
if retRGB is True and retIR is False:
imgIR = np.zeros(shapeIR, dtype=dtypeIR)
# if there is no IR image, crate a dummy one
if retIR is True and retRGB is False:
imgRGB = np.zeros(shapeRGB, dtype=dtypeRGB)
# if there is no RGB image, crate a dummy one
if retRGB is False and retIR is False:
break
imgCombined = np.hstack((imgRGB, imgIR))
# put both images together
cv2.imshow("frames", imgCombined)
k = cv2.waitKey(1)
if k == ord("q"):
break
capRGB.release()
capIR.release()
cv2.destroyAllWindows()
Problem:
From my understanding, the problem arises as capIR.read() attempts to read a missing image (in my case the 527th) and instead of just returning false/None it attempts to read the same image over and over again. Up to the missing frame, everything works fine, the right "IR" image even turns black but then the videoplayback begins to slow down and while i still can close the window by pressing 'q', spyder IDE freezes and if I wait "too long" i even have to shut it down. Console gives out "[image2 # 000002a7af8f0480] Could not open file : Bilder/IR/frame_000527.jpg" over and over again, so much that i can't scroll to the top.
I guess what I'm asking is: Is there any way to make the .read() function just attempt 1 read and after it fails continue with the next frame?
Best regards and thank you very much in advance!
Simulated for testing with different files and directory names.
Will retrieve the largest frame number from both directories and afterwards iterate over all frame numbers for reading the files from both directories.
import os
import cv2
import re
import glob
image_dir1 = 'test1'
image_dir2 = 'test2'
# retrieve all frames in both directories
frames_cap1 = glob.glob(os.path.join(image_dir1, "frame_*.jpg"))
frames_cap2 = glob.glob(os.path.join(image_dir2, "frame_*.jpg"))
# sort inscending
frames_cap1.sort()
frames_cap2.sort()
# retrieve last frame No for both directories
last_frame_cap1 = frames_cap1[-1]
last_frame_cap2 = frames_cap2[-1]
# extract integer counter as a group
# modify regex to match file name if required
match_cap1 = re.search('frame_(\d+).jpg', last_frame_cap1)
match_cap2 = re.search('frame_(\d+).jpg', last_frame_cap2)
last_frame_no_cap1 = int(match_cap1.group(1))
last_frame_no_cap2 = int(match_cap2.group(1))
# retrieve max frame No
max_frame_no = max(last_frame_no_cap1, last_frame_no_cap2)
for i in range(max_frame_no + 1):
# adapt formatting of frame number to digit count in file name
# here: 6 digits with leading zeros
image_path_cap1 = os.path.join(image_dir1, f"frame_{i:06d}.jpg")
image_path_cap2 = os.path.join(image_dir2, f"frame_{i:06d}.jpg")
if not os.path.isfile(image_path_cap1):
print(f"handle missing file: '{image_path_cap1}'")
# ...
else:
img1 = cv2.imread(image_path_cap1)
# …
if not os.path.isfile(image_path_cap2):
print(f"handle missing file: '{image_path_cap2}'")
# ...
else:
img2 = cv2.imread(image_path_cap2)
# …
# …
Assuming that the images in directory1 have the same names as directory2 images, but we know that some image may not be present in both directories...
import glob,os,cv2
path1 = "folder1/"
path2 = "folder2/"
#change directory to path1
os.chdir(path1)
l1 = glob.glob("*.jpg") #get a list of images names
os.chdir("../") #go one directory up
blackimg = cv2.imread("blackimg.jpg")
for fname in l1:
#check if image1 exists , then read it . otherwise im1 = blackimg
if os.path.isfile(path1+fname):
im1=cv2.imread(path1+fname)
else:
im1=blackimg
#check if image2 exists , then read it . otherwise im2 = blackimg
if os.path.isfile(path2+fname):
im2=cv2.imread(path2+fname)
else:
im2=blackimg
imgCombined = np.hstack((im1, im2))
cv2.imshow("Combined", imgCombined)
print("press any key to continue, q to exit")
k = cv2.waitKey(0)
if k == ord("q"):break
cv2.destroyAllWindows()
I'm writing a script in python using Open Cascade Technology (using the pyOCCT package for Anaconda) to import STEP files, defeature them procedurally and re-export them. I want to preserve the product hierarchy, names and colours as much as possible. Currently the script can import STEP files, simplify all of the geometry while roughly preserving the hierarchy and re-export the step file. The problem is no matter how I approach the problem, I can't manage to make it preserve the colours of the STEP file in a few particular cases.
Here's the model I pass in to the script:
And here's the result of the simplification:
In this case, the simplification has worked correctly but the colours of some of the bodies were not preserved. The common thread is that the bodies that loose their colours are children of products which only have other bodies as their children (ie: they don't contain sub-products).
This seems to be related to the way that Open Cascade imports STEP files which are translated as follows:
Alright, now for some code:
from OCCT.STEPControl import STEPControl_Reader, STEPControl_Writer, STEPControl_AsIs
from OCCT.BRepAlgoAPI import BRepAlgoAPI_Defeaturing
from OCCT.TopAbs import TopAbs_FACE, TopAbs_SHAPE, TopAbs_COMPOUND
from OCCT.TopExp import TopExp_Explorer
from OCCT.ShapeFix import ShapeFix_Shape
from OCCT.GProp import GProp_GProps
from OCCT.BRepGProp import BRepGProp
from OCCT.TopoDS import TopoDS
from OCCT.TopTools import TopTools_ListOfShape
from OCCT.BRep import BRep_Tool
from OCCT.Quantity import Quantity_ColorRGBA
from OCCT.ShapeBuild import ShapeBuild_ReShape
from OCCT.STEPCAFControl import STEPCAFControl_Reader, STEPCAFControl_Writer
from OCCT.XCAFApp import XCAFApp_Application
from OCCT.XCAFDoc import XCAFDoc_DocumentTool, XCAFDoc_ColorGen, XCAFDoc_ColorSurf
from OCCT.XmlXCAFDrivers import XmlXCAFDrivers
from OCCT.TCollection import TCollection_ExtendedString
from OCCT.TDF import TDF_LabelSequence
from OCCT.TDataStd import TDataStd_Name
from OCCT.TDocStd import TDocStd_Document
from OCCT.TNaming import TNaming_NamedShape
from OCCT.Interface import Interface_Static
# DBG
def export_step(shape, path):
writer = STEPControl_Writer()
writer.Transfer( shape, STEPControl_AsIs )
writer.Write(path)
# DBG
def print_shape_type(label, shapeTool):
if shapeTool.IsFree_(label):
print("Free")
if shapeTool.IsShape_(label):
print("Shape")
if shapeTool.IsSimpleShape_(label):
print("SimpleShape")
if shapeTool.IsReference_(label):
print("Reference")
if shapeTool.IsAssembly_(label):
print("Assembly")
if shapeTool.IsComponent_(label):
print("Component")
if shapeTool.IsCompound_(label):
print("Compound")
if shapeTool.IsSubShape_(label):
print("SubShape")
# Returns a ListOfShape containing the faces to be removed in the defeaturing
# NOTE: For concisness I've simplified this algorithm and as such it *MAY* not produce exactly
# the same output as shown in the screenshots but should still do SOME simplification
def select_faces(shape):
exp = TopExp_Explorer(shape, TopAbs_FACE)
selection = TopTools_ListOfShape()
nfaces = 0
while exp.More():
rgb = None
s = exp.Current()
exp.Next()
nfaces += 1
face = TopoDS.Face_(s)
gprops = GProp_GProps()
BRepGProp.SurfaceProperties_(face, gprops)
area = gprops.Mass()
surf = BRep_Tool.Surface_(face)
if area < 150:
selection.Append(face)
#log(f"\t\tRemoving face with area: {area}")
return selection, nfaces
# Performs the defeaturing
def simplify(shape):
defeaturer = BRepAlgoAPI_Defeaturing()
defeaturer.SetShape(shape)
sel = select_faces(shape)
if sel[0].Extent() == 0:
return shape
defeaturer.AddFacesToRemove(sel[0])
defeaturer.SetRunParallel(True)
defeaturer.SetToFillHistory(False)
defeaturer.Build()
if (not defeaturer.IsDone()):
return shape# TODO: Handle errors
return defeaturer.Shape()
# Given the label of an entity it finds it's displayed colour. If the entity has no defined colour the parents are searched for defined colours as well.
def find_color(label, colorTool):
col = Quantity_ColorRGBA()
status = False
while not status and label != None:
try:
status = colorTool.GetColor(label, XCAFDoc_ColorSurf, col)
except:
break
label = label.Father()
return (col.GetRGB().Red(), col.GetRGB().Green(), col.GetRGB().Blue(), col.Alpha(), status, col)
# Finds all child shapes and simplifies them recursively. Returns true if there were any subshapes.
# For now this assumes all shapes passed into this are translated as "SimpleShape".
# "Assembly" entities should be skipped as we don't need to touch them, "Compound" entities should work with this as well, though the behaviour is untested.
# Use the print_shape_type(shapeLabel, shapeTool) method to identify a shape.
def simplify_subshapes(shapeLabel, shapeTool, colorTool, set_colours=None):
labels = TDF_LabelSequence()
shapeTool.GetSubShapes_(shapeLabel, labels)
#print_shape_type(shapeLabel, shapeTool)
#print(f"{shapeTool.GetShape_(shapeLabel).ShapeType()}")
cols = {}
for i in range(1, labels.Length()+1):
label = labels.Value(i)
currShape = shapeTool.GetShape_(label)
print(f"\t{currShape.ShapeType()}")
if currShape.ShapeType() == TopAbs_COMPOUND:
# This code path should never be taken as far as I understand
simplify_subshapes(label, shapeTool, colorTool, set_colours)
else:
''' See the comment at the bottom of the main loop for an explanation of the function of this block
col = find_color(label, colorTool)
#print(f"{name} RGBA: {col[0]:.5f} {col[1]:.5f} {col[2]:.5f} {col[3]:.5f} defined={col[4]}")
cols[label.Tag()] = col
if set_colours != None:
colorTool.SetColor(label, set_colours[label.Tag()][5], XCAFDoc_ColorSurf)'''
# Doing both of these things seems to result in colours being reset but the geometry doesn't get replaced
nshape = simplify(currShape)
shapeTool.SetShape(label, nshape) # This doesn't work
return labels.Length() > 0, cols
# Set up XCaf Document
app = XCAFApp_Application.GetApplication_()
fmt = TCollection_ExtendedString('MDTV-XCAF')
doc = TDocStd_Document(fmt)
app.InitDocument(doc)
shapeTool = XCAFDoc_DocumentTool.ShapeTool_(doc.Main())
colorTool = XCAFDoc_DocumentTool.ColorTool_(doc.Main())
# Import the step file
reader = STEPCAFControl_Reader()
reader.SetNameMode(True)
reader.SetColorMode(True)
Interface_Static.SetIVal_("read.stepcaf.subshapes.name", 1) # Tells the importer to import subshape names
reader.ReadFile("testcolours.step")
reader.Transfer(doc)
labels = TDF_LabelSequence()
shapeTool.GetShapes(labels)
# Simplify each shape that was imported
for i in range(1, labels.Length()+1):
label = labels.Value(i)
shape = shapeTool.GetShape_(label)
# Assemblies are just made of other shapes, so we'll skip this and simplify them individually...
if shapeTool.IsAssembly_(label):
continue
# This function call here is meant to be the fix for the bug described.
# The idea was to check if the TopoDS_Shape we're looking at is a COMPOUND and if so we would simplify and call SetShape()
# on each of the sub-shapes instead in an attempt to preserve the colours stored in the sub-shape's labels.
#status, loadedCols = simplify_subshapes(label, shapeTool, colorTool)
#if status:
#continue
shape = simplify(shape)
shapeTool.SetShape(label, shape)
# The code gets a bit messy here because this was another attempt at fixing the problem by building a dictionary of colours
# before the shapes were simplified and then resetting the colours of each subshape after simplification.
# This didn't work either.
# But the idea was to call this function once to generate the dictionary, then simplify, then call it again passing in the dictionary so it could be re-applied.
#if status:
# simplify_subshapes(label, shapeTool, colorTool, loadedCols)
shapeTool.UpdateAssemblies()
# Re-export
writer = STEPCAFControl_Writer()
Interface_Static.SetIVal_("write.step.assembly", 2)
Interface_Static.SetIVal_("write.stepcaf.subshapes.name", 1)
writer.Transfer(doc, STEPControl_AsIs)
writer.Write("testcolours-simplified.step")
There's a lot of stuff here for a minimum reproducible example but the general flow of the program is that we import the step file:
reader.ReadFile("testcolours.step")
reader.Transfer(doc)
Then we iterate through each label in the file (essentially every node in the tree):
labels = TDF_LabelSequence()
shapeTool.GetShapes(labels)
# Simplify each shape that was imported
for i in range(1, labels.Length()+1):
label = labels.Value(i)
shape = shapeTool.GetShape_(label)
We skip any labels marked as assemblies since they contain children and we only want to simplify individual bodies. We then call simplify(shape) which performs the simplification and returns a new shape, we then call shapeTool.SetShape() to bind the new shape to the old label.
The thing that doesn't work here is that as explained, Component3 and Component4 don't get marked as Assemblies and are treated as SimpleShapes and when they are simplified as one shape, the colours are lost.
One solution I attempted was to call a method simplify_subshapes() which would iterate through each of the subshapes, and do the same thing as the main loop, simplifying them and then calling SetShape(). This ended up being even worse as it resulted in those bodies not being simplified at all but still loosing their colours.
I also attempted to use the simplify_subshapes() method to make a dictionary of all the colours of the subshapes, then simplify the COMPOUND shape and then call the same method again to this time re-apply the colours to the subshapes using the dictionary (the code for this is commented out with an explanation as to what it did).
col = find_color(label, colorTool)
#print(f"{name} RGBA: {col[0]:.5f} {col[1]:.5f} {col[2]:.5f} {col[3]:.5f} defined={col[4]}")
cols[label.Tag()] = col
if set_colours != None:
colorTool.SetColor(label, set_colours[label.Tag()][5], XCAFDoc_ColorSurf)
As far as I see it the issue could be resolved either by getting open cascade to import Component3 and Component4 as Assemblies OR by finding a way to make SetShape() work as intended on subshapes.
Here's a link to the test file:
testcolours.step
I'm trying to load files to TensorFlow Dataset where some files might be missing (in which case I want to replace these with zeroes).
The structure of directories that I'm trying to read data from is as follows:
|-data
|---sensor_A
|-----1.dat
|-----2.dat
|-----3.dat
|---sensor_B
|-----1.dat
|-----2.dat
|-----3.dat
.dat files are .csv files with spacebar as a separator. The content of every file is a single, multi-row observation where the number of columns is constant (say 4) and the number of rows is unknown (timeseries data).
I've successfully managed to read every sensor data to a separate TensorFlow Dataset with the following code:
import os
import tensorflow as tf
tf.enable_eager_execution()
data_root_dir = "data"
modalities_to_use = ["sensor_A", "sensor_B"]
timestamps = [1, 2, 3]
for mod_idx, modality in enumerate(modalities_to_use):
# Will produce: ['data/sensor_A/1.dat', 'data/sensor_A/2.dat', 'data/sensor_A/3.dat']
filenames = [os.path.join(data_root_dir, modality, str(timestamp) + ".dat") for timestamp in timestamps]
dataset = tf.data.Dataset.from_tensor_slices((filenames,))
def _parse_function_internal(filename):
number_of_columns = 4
single_observation = tf.read_file(filename)
# Tokenise every value so we can cast these to floats later.
single_observation = tf.string_split([single_observation], sep='\r\n ').values
single_observation = tf.reshape(single_observation, (-1, number_of_columns))
single_observation = tf.strings.to_number(single_observation, tf.float32)
return filename, single_observation
dataset = dataset.map(_parse_function_internal)
print('Result:')
for el in dataset:
try:
# Filename
print(el[0])
# Parsed file content
print(el[1])
except tf.errors.OutOfRangeError:
break
which successfully prints out content of all three files for every sensor.
My problem is that some timestamps in the dataset might be missing. For instance if file 1.dat in sensor_A directory will be missing I'm getting this error:
tensorflow.python.framework.errors_impl.NotFoundError: NewRandomAccessFile failed to Create/Open: mock_data\sensor_A\1.dat : The system cannot find the file specified.
; No such file or directory
[[{{node ReadFile}}]] [Op:IteratorGetNextSync]
which is thrown in this line:
for el in dataset:
What I've tried to do is to surround the call to tf.read_file() function with try block but obviously it doesn't work as the error is not thrown when tf.read_file() is called, but when the value is fetched from the dataset. Later I want to pass this dataset to a Keras model so I can't just surround it with try block. Is there any workaround? Is that even supported?
Thanks!
I managed to solve the problem, sharing the solution just in case someone else will be struggling with it as well. I had to use additional list of booleans specifying whether the file actually exist and pass it into the mapper. Then using tf.cond() function we decide whether to read the file or mock the data with zeroes (or any other logic).
import os
import tensorflow as tf
tf.enable_eager_execution()
data_root_dir = "data"
modalities_to_use = ["sensor_A", "sensor_B"]
timestamps = [1, 2, 3]
for mod_idx, modality in enumerate(modalities_to_use):
# Will produce: ['data/sensor_A/1.dat', 'data/sensor_A/2.dat', 'data/sensor_A/3.dat']
filenames = [os.path.join(data_root_dir, modality, str(timestamp) + ".dat") for timestamp in timestamps]
files_exist = [os.path.isfile(filename) for filename in filenames]
dataset = tf.data.Dataset.from_tensor_slices((filenames, files_exist))
def _parse_function_internal(filename, file_exist):
number_of_columns = 4
single_observation = tf.cond(file_exist, lambda: tf.read_file(filename), lambda: ' '.join(['0.0'] * number_of_columns))
# Tokenise every value so we can cast these to floats later.
single_observation = tf.string_split([single_observation], sep='\r\n ').values
single_observation = tf.reshape(single_observation, (-1, number_of_columns))
single_observation = tf.strings.to_number(single_observation, tf.float32)
return filename, single_observation
dataset = dataset.map(_parse_function_internal)
print('Result:')
for el in dataset:
try:
# Filename
print(el[0])
# Parsed file content
print(el[1])
except tf.errors.OutOfRangeError:
break
I am staring playing with tensorflow. I am facing the following problem. I am trying to run an example to do image recognition based on the Stanford Dog Dataset.I am stuck in the step of converting the image and label in TRFRECORDS files.
In the image dataset folder there are 120 sub-folders, one for each breed (label).
If I run the code below with just on sub-folder in run fine (Actually I didn't tried to read the trfrecord file). But If I include a second sub-folder the process kills the python kernel process.
Here is the code I am running
import glob
import tensorflow as tf
from itertools import groupby
from collections import defaultdict
image_filenames = glob.glob(r'C:\Users\Administrator\Documents\Tensorflow\images\n02*\*.jpg')
training_dataset = defaultdict(list)
testing_dataset = defaultdict(list)
# Split up the filename into its breed and corresponding filename. The breed is found by taking the directo
image_filename_with_breed =map(lambda filename: (filename.split("\\")[6], filename), image_filenames)
# Group each image by the breed which is the 0th element in the tuple returned above
for dog_breed, breed_images in groupby(image_filename_with_breed, lambda x: x[0]):
# Enumerate each breed's image and send ~20% of the images to a testing set
for i, breed_image in enumerate(breed_images):
if i % 5 == 0:
testing_dataset[dog_breed].append(breed_image[1])
else:
training_dataset[dog_breed].append(breed_image[1])
# Check that each breed includes at least 18% of the images for testing
breed_training_count = len(training_dataset[dog_breed])
breed_testing_count = len(testing_dataset[dog_breed])
assert round(breed_testing_count / (breed_training_count + breed_testing_count), 2) > 0.18,'Not enough testing data'
sess = tf.Session()
def write_records_file(dataset, record_location):
"""
Fill a TFRecords file with the images found in `dataset` and include their category.
Parameters
----------
dataset : dict(list)
Dictionary with each key being a label for the list of image filenames of its value.
record_location : str
Location to store the TFRecord output.
"""
writer = None
# Enumerating the dataset because the current index is used to breakup the files if they get over 100
# images to avoid a slowdown in writing.
current_index = 0
for breed, images_filenames in dataset.items():
for image_filename in images_filenames:
print(image_filename)
if current_index % 100 == 0:
if writer:
writer.close()
record_filename = "{record_location}-{current_index}.tfrecords".format(
record_location=record_location,
current_index=current_index)
print(record_filename)
writer = tf.python_io.TFRecordWriter(record_filename)
current_index += 1
image_file = tf.read_file(image_filename)
# In ImageNet dogs, there are a few images which TensorFlow doesn't recognize as JPEGs. This
# try/catch will ignore those images.
try:
image = tf.image.decode_jpeg(image_file)
except:
print(image_filename)
continue
# Converting to grayscale saves processing and memory but isn't required.
grayscale_image = tf.image.rgb_to_grayscale(image)
resized_image = tf.image.resize_images(grayscale_image, (250, 151))
# tf.cast is used here because the resized images are floats but haven't been converted into
# image floats where an RGB value is between [0,1).
image_bytes = sess.run(tf.cast(resized_image, tf.uint8)).tobytes()
# Instead of using the label as a string, it'd be more efficient to turn it into either an
# integer index or a one-hot encoded rank one tensor.
# https://en.wikipedia.org/wiki/One-hot
image_label = breed.encode("utf-8")
example = tf.train.Example(features=tf.train.Features(feature={
'label': tf.train.Feature(bytes_list=tf.train.BytesList(value=[image_label])),
'image': tf.train.Feature(bytes_list=tf.train.BytesList(value=[image_bytes]))
}))
writer.write(example.SerializeToString())
writer.close()
write_records_file(testing_dataset, r'C:\Users\Administrator\Documents\Tensorflow\TRF\testing_images')
write_records_file(training_dataset, r'C:\Users\Administrator\Documents\Tensorflow\TRF\training_images')
I monitored the memory usage and running the script does not seems to consume to much memory. I tried this in two Virtual Machines. One with Ubuntu and the other on with Windows 2000.
Does anyone have a idea?
Thanks!
I found the problem. It was the writer.close() statement that was incorrectly idented. I should be idented in the first for loop but I idented it in the second loop.
How can I copy slide?
I created a template slide and I need to copy it and edit shapes of each copy separately.
Or how I can add my template slide to presentation.slide_layouts?
This is what I found on GitHub, and it works for me. I did change a couple of things for my project. You will need to import six and copy. I am using pptx-6.10
def duplicate_slide(pres, index):
template = pres.slides[index]
try:
blank_slide_layout = pres.slide_layouts[12]
except:
blank_slide_layout = pres.slide_layouts[len(pres.slide_layouts)]
copied_slide = pres.slides.add_slide(blank_slide_layout)
for shp in template.shapes:
el = shp.element
newel = copy.deepcopy(el)
copied_slide.shapes._spTree.insert_element_before(newel, 'p:extLst')
for _, value in six.iteritems(template.part.rels):
# Make sure we don't copy a notesSlide relation as that won't exist
if "notesSlide" not in value.reltype:
copied_slide.part.rels.add_relationship(
value.reltype,
value._target,
value.rId
)
return copied_slide
Then you can create the copy with passing in your presentation and the slide index of your template:
copied_slide = duplicate_slide(pres, 4)
I am still working on editing the shapes from the copied slide, once I am further along in my project I can update
I wanted to present my workaround to copy slides. I use a template ppt and populate it. I know before populating the slides which slides of the template need to be copied and how often. What I then do is copying the slides and saving the new ppt with the copied slides. After saving I can open the ppt with the copied slides and use pptx to populate the slides.
import win32com.client
ppt_instance = win32com.client.Dispatch('PowerPoint.Application')
#open the powerpoint presentation headless in background
read_only = True
has_title = False
window = False
prs = ppt_instance.Presentations.open('path/ppt.pptx',read_only,has_title,window)
nr_slide = 1
insert_index = 1
prs.Slides(nr_slide).Copy()
prs.Slides.Paste(Index=insert_index)
prs.SaveAs('path/new_ppt.pptx')
prs.Close()
#kills ppt_instance
ppt_instance.Quit()
del ppt_instance
In this case the firste slide would be copied of the presentation and inserted after the first slide of the same presentation.
Hope this helps some of you!
Since I also found another usecase for the code shared by #d_bergeron, I just wanted to share it here.
In my case, I wanted to copy a slide from another presentation into the one I generated with python-pptx:
As argument I pass in the Presentation() object I created using python-pptx (prs = Presenation()).
from pptx import Presentation
import copy
def copy_slide_from_external_prs(prs):
# copy from external presentation all objects into the existing presentation
external_pres = Presentation("PATH/TO/PRES/TO/IMPORT/from.pptx")
# specify the slide you want to copy the contents from
ext_slide = external_pres.slides[0]
# Define the layout you want to use from your generated pptx
SLD_LAYOUT = 5
slide_layout = prs.slide_layouts[SLD_LAYOUT]
# create now slide, to copy contents to
curr_slide = prs.slides.add_slide(slide_layout)
# now copy contents from external slide, but do not copy slide properties
# e.g. slide layouts, etc., because these would produce errors, as diplicate
# entries might be generated
for shp in ext_slide.shapes:
el = shp.element
newel = copy.deepcopy(el)
curr_slide.shapes._spTree.insert_element_before(newel, 'p:extLst')
return prs
I am mainly posting it here, since I was looking for a way to copy an external slide into my presentation and ended up in this thread.
I was using n00by0815's answer and it worked great until I had to copy images. Here is my adapted version that handles images. This code creates a local copy of the image then adds it to the slide. I'm sure there's a cleaner way, but this works.
Here is another way to copy each slide onto a single PPTX slide for an entire presentation, and then you can use LibreOffice to convert each individual powerpoint into an image:
def get_slide_count(prs):
""" Get the number of slides in PPTX presentation """
slidecount = 0
for slide in prs.slides:
slidecount += 1
return slidecount
def delete_slide(prs, slide):
""" Delete a slide out of a powerpoint presentation"""
id_dict = { slide.id: [i, slide.rId] for i,slide in enumerate(prs.slides._sldIdLst) }
slide_id = slide.slide_id
prs.part.drop_rel(id_dict[slide_id][1])
del prs.slides._sldIdLst[id_dict[slide_id][0]]
def get_single_slide_pres(prs, slidetokeep):
for idx, slide in enumerate(prs.slides):
if idx < slidetokeep:
delete_slide(prs, slide)
elif (idx > slidetokeep):
delete_slide(prs, slide)
prs.save(str(slidetokeep + 1) + ".pptx")
pptxfilepath = "test.pptx"
prs = Presentation(pptxfilepath)
slidecount = get_slide_count(prs)
for i in range(slidecount):
prs_backup = Presentation(pptxfilepath)
get_single_slide_pres(prs_backup, i)
prs_backup = None
I edited #n00by0815 solution and came up with very elegant code, which also can copy images without errors:
# ATTENTNION: PPTX PACKAGE RUNS ONLY ON CERTAINS VERSION OF PYTHON (https://python-pptx.readthedocs.io/en/latest/user/install.html)
from pptx import Presentation
from pptx.util import Pt
from pptx.enum.text import PP_ALIGN
import copy
import os
DIR_PATH = os.path.dirname(os.path.realpath(__file__))
#modeled on https://stackoverflow.com/a/56074651/20159015 and https://stackoverflow.com/a/62921848/20159015
#this for some reason doesnt copy text properties (font size, alignment etc.)
def SlideCopyFromPasteInto(copyFromPres, slideIndex, pasteIntoPres):
# specify the slide you want to copy the contents from
slide_to_copy = copyFromPres.slides[slideIndex]
# Define the layout you want to use from your generated pptx
slide_layout = pasteIntoPres.slide_layouts.get_by_name("Blank") # names of layouts can be found here under step 3: https://www.geeksforgeeks.org/how-to-change-slide-layout-in-ms-powerpoint/
# it is important for slide_layout to be blank since you dont want these "Write your title here" or something like that textboxes
# alternative: slide_layout = pasteIntoPres.slide_layouts[copyFromPres.slide_layouts.index(slide_to_copy.slide_layout)]
# create now slide, to copy contents to
new_slide = pasteIntoPres.slides.add_slide(slide_layout)
# create images dict
imgDict = {}
# now copy contents from external slide, but do not copy slide properties
# e.g. slide layouts, etc., because these would produce errors, as diplicate
# entries might be generated
for shp in slide_to_copy.shapes:
if 'Picture' in shp.name:
# save image
with open(shp.name+'.jpg', 'wb') as f:
f.write(shp.image.blob)
# add image to dict
imgDict[shp.name+'.jpg'] = [shp.left, shp.top, shp.width, shp.height]
else:
# create copy of elem
el = shp.element
newel = copy.deepcopy(el)
# add elem to shape tree
new_slide.shapes._spTree.insert_element_before(newel, 'p:extLst')
# things added first will be covered by things added last => since I want pictures to be in foreground, I will add them after others elements
# you can change this if you want
# add pictures
for k, v in imgDict.items():
new_slide.shapes.add_picture(k, v[0], v[1], v[2], v[3])
os.remove(k)
return new_slide # this returns slide so you can instantly work with it when it is pasted in presentation
templatePres = Presentation(f"{DIR_PATH}/template.pptx")
outputPres = Presentation()
outputPres.slide_height, outputPres.slide_width = templatePres.slide_height, templatePres.slide_width
# this can sometimes cause problems. Alternative:
# outputPres = Presentation(f"{DIR_PATH}/template.pptx") and now delete all slides to have empty presentation
# if you just want to copy and paste slide:
SlideCopyFromPasteInto(templatePres,0,outputPres)
# if you want to edit slide that was just pasted in presentation:
pastedSlide = SlideCopyFromPasteInto(templatePres,0,outputPres)
pastedSlide.shapes.title.text = "My very cool title"
for shape in pastedSlide.shapes:
if not(shape.has_text_frame): continue
# easiest ways to edit text fields is to put some identifying text in them
if shape.text_frame.text == "personName": # there is a text field with "personName" written into it
shape.text_frame.text = "Brian"
if shape.text_frame.text == "personSalary":
shape.text_frame.text = str(brianSalary)
# stylizing text need to be done after you change it
shape.text_frame.paragraphs[0].font.size = Pt(80)
shape.text_frame.paragraphs[0].alignment = PP_ALIGN.CENTER
outputPres.save(f'{DIR_PATH}/output.pptx')
Sorry for the delay, I was moved to another project. I was able to complete my ppt project using multiple template slides and copying them. At the end of building the presentation I delete the templates. To grab the shapes you will need to iterate through the slide.shapes and find the name of the shape that you are looking for. Once you have this returned you can then edit the shape as needed. I have added a version of the add_text function that I use to populate shape.text_frame.
def find_shape_by_name(shapes, name):
for shape in shapes:
if shape.name == name:
return shape
return None
def add_text(shape, text, alignment=None):
if alignment:
shape.vertical_anchor = alignment
tf = shape.text_frame
tf.clear()
run = tf.paragraphs[0].add_run()
run.text = text if text else ''
To find the shape "slide_title".
slide_title = find_shape_by_name(slide.shapes,'slide_title')
To add text to the shape.
add_text(slide_title,'TEST SLIDE')
Please let me know if you need any other assistance.