I am trying to solve Cartpole problem using screen pixels of the Gym environment. for having the screen pixels I use
img = env.render(mode="rgb_array")
but the problem is that during learning it's score can raise up to more than 1000. But cartpole game should end after gaining 500 scores. why this does not happen?
Another problem is that by using the above line of code for acquiring the screen pixels the evironment simulation is rendered and it slows down the training. How can I prevent the rendering to speed up the training process?
Also I use the following function to crop the received image around the cartpole location and during the training I use two images and substract them as state. Is it the right way to do it?
resize = T.Compose([T.ToPILImage(),T.Resize((40,40)),T.ToTensor()])
def get_center(imgnp):
temp = 0
screen_cropped = imgnp[168:301,:,:]
for i in screen_cropped[131,:,1]:
if i!= 0:
temp = temp +1
else:
return temp+25#((temp+30)/600)*4.8*2 - 4.8
return temp
def CroppImage(sc):
width = 30
center = get_center(sc)
if center >= 570 and center < 600:
width = 600 - center
elif center <= 30:
width = center
left = center-width
right = left + 2*width
if center>= 600:
width = 70
right = 600
try:
sc_crop = sc[171:290,left :right,:]#left :right
except:
print("---------- here is the problem -------------")
sc_crop = sc_crop.transpose((2, 0, 1))
sc_crop = torch.from_numpy(sc_crop.copy())
try:
sc_crop = resize(sc_crop)
except:
print("some problem has been occured")
print(sc_crop.shape)
return sc_crop.unsqueeze(0)
here you can see the output of the crop function
output of crop funtion
Related
I am trying to read all the touching pixels with the same color in a image.
For that I use reccursive functions. When I check one pixel, I look on the right, left, top and bottom if the pixel close to it is the same color. If it is I add it to an array otherwise I don't.
The code is as follow:
vimport tkinter as tk
from PIL import Image
import sys
sys.setrecursionlimit(200000)
## WINDOWS
# to launch in debug mode
imgToDraw = Image.open('assets-test\\smile-face.png')
# to launch normaly
# imgToDraw = Image.open('..\\assets-test\\smile-face.png')
## LINUX
# imgToDraw = Image.open('../assets-test/smile-face.png')
imgPixels = imgToDraw.load()
imgWidth = imgToDraw.size[0]
imgHeight = imgToDraw.size[1]
# an element is a part of the image, it's a bunch of pixels with approximately the same color
# and each pixel touch at least one other pixel of the same element
elements = [];
isPixelChecked = [[ False for y in range( imgWidth ) ] for x in range( imgHeight )]
# min tolerable difference between two colors to consider them the same
# the higher the value is the more colors will be considered the same
COLOR_TOLERANCE = 10
reccursionCount = 0
class Element:
def __init__(self, color):
self.pixels = [];
self.color = color;
def addPixel(self, pixel):
self.pixels.append(pixel);
class Pixel:
def __init__(self, x, y, color):
self.x = x # x position of the pixel
self.y = y # y position of the pixel
self.color = color # color is a tuple (r,g,b)
def cutImageInElements():
global element
completeElement(element.pixels)
def completeElement(elemPixels):
global reccursionCount
global isPixelChecked
reccursionCount += 1
nbPixels = len(elemPixels);
xIndex = elemPixels[nbPixels - 1].x
yIndex = elemPixels[nbPixels - 1].y
xRightIdx = elemPixels[nbPixels - 1].x + 1
xLeftIdx = elemPixels[nbPixels - 1].x - 1
yBottomIdx = elemPixels[nbPixels - 1].y + 1
yTopIdx = elemPixels[nbPixels - 1].y - 1
isPixelChecked[xIndex][yIndex] = True
if((xRightIdx < imgWidth) and isPixelChecked[xRightIdx][yIndex] == False):
if(isColorAlmostSame(imgPixels[elemPixels[0].x, elemPixels[0].y], imgPixels[xRightIdx, yIndex])):
pixelAppended = Pixel(xRightIdx, yIndex, imgPixels[xRightIdx, yIndex])
elemPixels.append(pixelAppended)
completeElement(elemPixels)
if((xLeftIdx >= 0) and isPixelChecked[xLeftIdx][yIndex] == False):
if(isColorAlmostSame(imgPixels[elemPixels[0].x, elemPixels[0].y], imgPixels[xLeftIdx, yIndex])):
pixelAppended = Pixel(xLeftIdx, yIndex, imgPixels[xLeftIdx, yIndex])
elemPixels.append(pixelAppended)
completeElement(elemPixels)
if((yBottomIdx < imgHeight) and isPixelChecked[xIndex][yBottomIdx] == False):
if(isColorAlmostSame(imgPixels[elemPixels[0].x, elemPixels[0].y], imgPixels[xIndex, yBottomIdx])):
pixelAppended = Pixel(xIndex, yBottomIdx, imgPixels[xIndex, yBottomIdx])
elemPixels.append(pixelAppended)
completeElement(elemPixels)
if((yTopIdx >= 0) and isPixelChecked[xIndex][yTopIdx] == False):
if(isColorAlmostSame(imgPixels[elemPixels[0].x, elemPixels[0].y], imgPixels[xIndex, yTopIdx])):
pixelAppended = Pixel(xIndex, yTopIdx, imgPixels[xIndex, yTopIdx])
elemPixels.append(pixelAppended)
completeElement(elemPixels)
def isColorAlmostSame(pixel1, pixel2):
redDiff = abs(pixel1[0] - pixel2[0])
greenDiff = abs(pixel1[1] - pixel2[1])
blueDiff = abs(pixel1[2] - pixel2[2])
if(redDiff < COLOR_TOLERANCE and greenDiff < COLOR_TOLERANCE and blueDiff < COLOR_TOLERANCE):
return True
else:
return False
def printPixelsArr(pixelsArr):
for x in range(0, len(pixelsArr)):
print(pixelsArr[x].x, pixelsArr[x].y, pixelsArr[x].color)
if __name__ == '__main__':
pixel = Pixel(0, 0, imgPixels[0, 0]);
element = Element(pixel.color);
element.addPixel(pixel);
cutImageInElements();
print("NbReccursive call: ", reccursionCount)
This code works for small images of size 100x100 but crashes with an image of 400x400 with the error "terminated by signal SIGSEGV (Address boundary error)" when I launch the program on wsl2. When I run the program on cmd or powershell it just crashes but with no error code/msg.
I cannot understand why it would work with some size of images and not others. I can only think that the memory runs out or something but in the task manager the program uses almost no memory.
Not sure why that's failing, but that much recursion in Python isn't a great idea. I'd suggest reading about tail recursion that other languages use to make some recursive algorithms consume constant stack space. Note that your algorithm is not tail recursive, so this optimisation wouldn't help even if Python supported it.
I hacked together the following flood fill implementation. It uses Numpy so that it's only 10x slower than Pillow's ImageDraw.floodfill.
import numpy as np
def floodfill(im, row, col, threshold):
similar = np.mean(np.abs(im - im[row, col]), 2) < threshold
mask = np.zeros_like(similar)
mask[row, col] = 1
m2 = mask.copy()
while True:
m2[:,:] = mask
m2[1:,:] |= mask[:-1]
m2[:-1,:] |= mask[1:]
m2[:,1:] |= mask[:,:-1]
m2[:,:-1] |= mask[:,1:]
m2 &= similar
if np.all(m2 == mask):
return mask
mask[:,:] = m2
As an example of using this, you could do;
import requests
from io import BytesIO
res = requests.get("https://picsum.photos/300")
res.raise_for_status()
src = Image.open(BytesIO(res.content))
mask = floodfill(np.array(src, int), 10, 10, 40)
where the random image I got and the output mask are:
I cant blit images in normal speed if the images in array.
#brife review
In my code i defined 10 images as a variable(x1-x10)
those 100 images relevant for specific class (object.draw_function()), and will be bliting in main loop according specific condtions.
in the object.draw_function() all the images are saved in lst "images_lst" = [img1,img2,img3,,,,img10]
and bliting from that array according rulles.
i noticed that if the len of the array is higher then 4,5 , the loop FPS is slower. and i dont understand why ? the loading images is outside the loop.
Code example:
#loading images
img1 = pygame.image.load(r'images\game_background1\img1.jpg')
img2 = pygame.image.load(r'images\game_background1\img2.jpg')
img3 = pygame.image.load(r'images\game_background1\img3.jpg')
'
'
'
img10= pygame.image.load(r'images\game_background1\img10.png')
#define font and space size
space_width,space_height = pysical_tm_img.get_width(),pysical_tm_img.get_height()
font0_0 = pygame.font.SysFont(pygame.font.get_fonts()[0],12)
font0_02 = pygame.font.SysFont(pygame.font.get_fonts()[0],17)
# define class
class EXAMPLE():
def __init__(self,x,y,text,power,energy,range):
self.rect = pygame.Rect(x,y,100,10)
self.text = text
self.power = power
self.energy = energy
self.range = range
def draw_func(self,surface):
img_lst = [img1,text1,img2,text2,img3,text3......img10,text10]
for i,img in enumerate(img_lst):
if i % 2 == 0 :
img_rect = img.get_rect(center=(self.rect.x +20 + (i *space_width*2),self.rect.top + space_height))
surface.blit(img,img_rect)
else:
img_rect = img.get_rect(center=(self.rect.x +20 + space_width + ((i-1) *space_width*2),self.rect.top + space_height))
surface.blit(img,img_rect)
#main loop
while True:
if somthing:
object1 = EXAMPLE(10,10,"abc",100,50,10)
object1.draw_func(screen)
elif somthing else:
object3 = EXAMPLE(10,10,"abc",100,50,10)
object3.draw_func(screen)
pygame.display.update()
clock.tick(60)
I dont understand whats wrong and why i cant append more images to my images list without reducing runtime.
Another question that not much relevant to this code but rellevant to runtime. if this code without the main loop is in file number 1 and in this file i import pygame and init the font only.
pygame.font.init()
and in file num 2 where the main loop is running i import pygame and init pygame
pygame.init()
is it reduce my proggram runtime?
Ensure that the image Surface has the same format as the display Surface. Use convert() (or convert_alpha()) to create a Surface that has the same pixel format. This improves performance when the image is blit on the display, because the formats are compatible and blit does not need to perform an implicit transformation.
e.g.:
img1 = pygame.image.load(r'images\game_background1\img1.jpg').convert()
I'm working on a simple project using pygame zero and so far I've been able to display a group of images to form a very simple animation. I rendered the video out into a .png sequence at 60 fps. It seems like pygame zero is rendering them out a bit faster than that and I was just wondering if there was a way to lock the FPS to 60 so everything would render as I'd expect it to. I have some sound files that I'd like to sync up with the image sequence, so having them render at a constant FPS would be very helpful.
I've also noticed that the sound keeps looping after it's played, so I tried to stop it after it's played, but the sound cuts off at the end because the animation seems to be finishing too early.
Here's the code I have so far:
import pgzrun
WIDTH = 480
HEIGHT = 360
# boot1 graphics
boot1 = Actor('boot1_1')
boot1.frame = 1
boot1.active = True
# boot2 graphics
boot2 = Actor('boot2_1')
boot2.frame = 1
boot2.active = False
# overlay
overlay = Actor("overlay_a")
def update_boot1():
if boot1.active:
boot1.x = WIDTH/2
boot1.image = "boot1_{}".format(boot1.frame)
boot1.frame += 1
else:
boot1.x = 1000
if boot1.frame > 59:
#boot1.frame = 1
boot2.active = True
update_boot2()
boot1.active = False
def update_boot2():
if boot2.active:
boot2.x = WIDTH/2
sounds.boot1.play()
boot2.image = "boot2_{}".format(boot2.frame)
boot2.frame += 1
else:
boot2.x = 1000
if boot2.frame > 233:
boot2.frame = 233
boot2.active = False
sounds.boot1.stop()
def draw():
screen.clear
screen.fill((0, 75, 0))
boot2.draw()
boot1.draw()
overlay.draw()
# running the animation
def update(dt):
update_boot1()
update_boot2()
pgzrun.go()
I also haven't found a way to "unload" the image sequences, or make them not visible when I'm done with them, so I just throw them off to the side with ".x = 1000".
in your running loop, just use:
just use clock.tick(60)#or whatever fps you want
I'm using following code to add watermark to animated GIF images. My problem is that all GIF frames except the first one have incorrect colors in result. Would you know how to fix the color of frames? Thank you.
def add_watermark(in_file, watermark_file, watermark_position, watermark_ratio, out_file, quality=85):
img = Image.open(in_file)
watermark_layer = Image.new('RGBA', img.size, (0,0,0,0))
watermark_img = Image.open(watermark_file).convert('RGBA')
watermark_img.thumbnail((img.size[0]/watermark_ratio, 1000), Image.ANTIALIAS)
alpha = watermark_img.split()[3]
alpha = ImageEnhance.Brightness(alpha).enhance(0.95)
watermark_img.putalpha(alpha)
watermark_layer.paste(watermark_img, count_watermark_position(img, watermark_img, watermark_position))
frames = images2gif.readGifFromPIL(img, False)
frames_out = []
for frame in frames:
frames_out.append(Image.composite(watermark_layer, frame, watermark_layer))
images2gif.writeGif(out_file, frames_out, duration=0.5)
To complete example, i provide also code of helper function:
def count_watermark_position(img, watermark, position):
if position == 'right_bottom':
return img.size[0] - watermark.size[0], img.size[1] - watermark.size[1]
if position == 'center':
return (img.size[0] - watermark.size[0])/2, (img.size[1] - watermark.size[1])/2
if position == 'left_bottom':
return 0, img.size[1] - watermark.size[1]
if position == 'left_top':
return 0, 0
if position == 'right_top':
return img.size[0] - watermark.size[0], 0
raise AttributeError('Invalid position')
Source code of images2gif I 've used - I modified it a little bit to make it work with pillow. See comment at the begining of source code.
Is there an easy way to do this?
In Python, I created a script to get a "square box" from an image...based on the center.
However, that killed some of my brain cells. Is there an easy way to do this for a 3x2 (3 width, 2 height), based on the center?
This is my "square box" script, but I don't feel like modifying it for the 3x2.
def letterbox(f,thumb_w=None,thumb_h=None):
try:
im = Image.open(StringIO(f))
imagex = int(im.size[0])
imagey = int(im.size[1])
thumb_size = (thumb_w,thumb_h) #what if it is too small!?? fix it.
if imagex > imagey:
setlen = imagey
left = (imagex - setlen)/2
top = 0
height = setlen
width = setlen
if imagey > imagex:
setlen = imagex
left = 0
top = (imagey - setlen)/2
heigth = setlen
width = setlen
if imagex == imagey:
left = 0
top = 0
height = imagey
width = imagex
box = (left,top,left+width,top+height)
im = im.crop(box)
#im.thumbnail(thumb_size,Image.ANTIALIAS)
new_file = StringIO()
im.save(new_file,'JPEG')
new_file.seek(0)
except Exception, e:
pass
return new_file
Is there a script online that can do what I need?
Use an aspect ratio which is defined as imagex/imagey, so you use 3/2 for 3:2, 16/9 for 16:9 etc.
def letterbox(f,aspect_ratio=1):
try:
im = Image.open(StringIO(f))
imagex = int(im.size[0])
imagey = int(im.size[1])
width = min(imagex, imagey*aspect_ratio)
height = min(imagex/aspect_ratio, imagey)
left =(imagex - width)/2
top = (imagey - height)/2
box = (left,top,left+width,top+height)
im = im.crop(box)
new_file = StringIO()
im.save(new_file,'JPEG')
new_file.seek(0)
except Exception, e:
pass
return new_file
You might want to check for roundoff errors at some point, but otherwise this does it.
First of all, the name of your function is misleading because what it does is not letterbox images (regardless of the specific height/width [aspect ratio] involved) -- so here it's been renamed aspectcrop() to better describe what it does.
def aspectcrop(f, ratio=1.0):
try:
im = Image.open(StringIO(f))
imagewidth,imageheight = im.size
# determine the dimensions of a crop area
# which is no wider or taller than the image
if int(imagewidth*ratio) > imageheight:
cropheight,cropwidth = imageheight,int(imageheight/ratio)
else:
cropwidth,cropheight = imagewidth,int(imagewidth*ratio)
# center the crop area on the image (dx and/or dy will be zero)
dx,dy = (imagewidth-cropwidth)/2,(imageheight-cropheight)/2
# crop, save, and return image data
im = im.crop((dx,dy,cropwidth+dx,cropheight+dy))
new_file = StringIO()
im.save(new_file,'JPEG')
new_file.seek(0)
except Exception, e:
new_file = None # prevent exception on return
pass
return new_file
If any aspect ratio argument you pass to it is not naturally a whole number, make sure it's in floating point, as in 3./2. not 3/2. The default (1.0) could have been integer, but I explicitly made it floating point as a reminder. It might be better to pass it in as two separate integers and compute the ratio internally.
Lastly, I noticed that your example script had some visible traces of what looks like an attempt to create a thumbnail of the image, so my answer to the related question What's the simplest way in Python to resize an image to a given bounded area? might also be of interest (and could probably be integrated into this function without too much trouble).