I’m doing some testing activity which requires me to capture screenshot of applications/DB etc and save it to a document. The whole activity has more than 50 screenshots. Is there a way in Python through which I can take a screenshot with windows shortcut key( eg; CTRL ALT shift C) and it appends the image to a document file. I believe the python program should be running in background like nohup in Unix.
For storing screen captures in Word using a hotkey, you can use a combination of libraries.
Use win32gui to open Word
Use python-docx to update the document and save
Use PyAutoGUI to do the screen capture
Use keyboard to listen for the hotkey
For this script to work, you will need to create the Word document before running the script.
# Need these libraries
# pip install keyboard
# pip install PyAutoGUI
# pip install python-docx
# pip install win32gui
import keyboard
import pyautogui
from docx import Document
from docx.shared import Inches
import win32gui
from PIL import ImageGrab
shotfile = "C:/tmp/shot.png" # temporary image storage
docxfile = "C:/tmp/shots.docx" # main document
hotkey = 'ctrl+shift+q' # use this combination anytime while script is running
def do_cap():
try:
print ('Storing capture...')
hwnd = win32gui.GetForegroundWindow() # active window
bbox = win32gui.GetWindowRect(hwnd) # bounding rectangle
# capture screen
shot = pyautogui.screenshot(region=bbox) # take screenshot, active app
# shot = pyautogui.screenshot() # take screenshot full screen
shot.save(shotfile) # save screenshot
# append to document. Doc must exist.
doc = Document(docxfile) # open document
doc.add_picture(shotfile, width=Inches(7)) # add image, 7 inches wide
doc.save(docxfile) # update document
print ('Done capture.')
except Exception as e: # allow program to keep running
print("Capture Error:", e)
keyboard.add_hotkey(hotkey, do_cap) # set hot keys
print("Started. Waiting for", hotkey)
keyboard.wait() # Block forever
Related
I'm using pyautogui to take few screenshots per second and save them.
pyautogui save the screenshot without the cursor, can I take the screenshot with it by default (without edit the photo)?
This is the code I'm using:
import time
import pyautogui
def takeScreenshot(count):
myScreenshot = pyautogui.screenshot()
myScreenshot.save(r'imgs\screenshot_'+str(count)+'.png')
time.sleep(0.2)
takeScreenshot(count+1)
takeScreenshot(0)
Using the module pywhatkit you can send messages on WhatsApp,
I used the script:
import pywhatkit as w
w.sendwhatmsg("xxxxxxxx", " this is a generated msg",9,26)
x is the number
The problem is, all it does is load the message in WhatsApp's textbox, it does not send. Am I missing something?
I just had the same issue, and I uninstalled and installed again pywhatkit with the following commands:
pip uninstall pywhatkit
pip install pywhatkit
I know this question is old, hope it will help someone with the same problem
import pywhatkit as w
import time
import pyautogui
import keyboard as k
w.sendwhatmsg("your number", 'hi', 8, 38)
pyautogui.click(1050, 950)
time.sleep(2)
k.press_and_release('enter')
by pyautogui.click you can adjust cursor to message box and then it will click on it
and after that with keyboard you can click enter as simple as that
you must increase the waiting time to more than or equal to 30 seconds, it happens generally due to slow internet. write it like this:
import pywhatkit as py
py.sendwhatmsg("+91xxxxxxxxxx", "hello", 13, 12, 32)
This seems to be a bug in Pywhatkit: https://github.com/Ankit404butfound/PyWhatKit/issues/20
the module currently only supports 1 screen, if you are using multiple screens and the new whatsapp web is currently opening on a separate window from where your code is running is going to give you that error. It is a known issue therefore if you are going to use more than one screen (who isn't) ensure your editor of choice (ie: google chrome and visual studio) are on the same screen.
After debugging sendwhatmsg function is doing the following things
opening your default browser using this query web.open(f"https://web.whatsapp.com/send?phone={phone_no}&text={quote(message)}")
clicking here pg.click(core.WIDTH / 2, core.HEIGHT / 2)
and pressing enter pg.press("enter")
The main problem is that this 2 variables core.WIDTH and core.HEIGHT gives you the resolution of your screen, not the resolution of the tab just opened in the browser. So if the browser tab its not maximized you might end up clicking somewhere else. So please make sure that your browser its maximized when opening.
The problem is that the window in which pywhatkit opens Whatsapp Web is not selected. When it opens Whatsapp Web, waits for a few seconds and clicks 'enter', the 'enter' is not processed by the browser window. Add the following code to your program:
# Import Libraries
import pywhatkit
import pyautogui
from tkinter import *
win = Tk() # Some Tkinter stuff
screen_width = win.winfo_screenwidth() # Gets the resolution (width) of your monitor
screen_height= win.winfo_screenheight() # Gets the resolution (height) of your monitor
print(screen_width, screen_height) # prints your monitor's resolution
pywhatkit.sendwhatmsg("+91xxxxxxxxxx", "Enter Message", 0, 0) # Sends the message
pyautogui.moveTo(screen_width * 0.694, screen_height* 0.964) # Moves the cursor the the message bar in Whatsapp
pyautogui.click() # Clicks the bar
pyautogui.press('enter') # Sends the message
No matter what screen you are using, pyautogui will always send your message except if Whatsapp updates its UI. In case this happens, then check where the message bar is located through pyautogui and whatever values you get, divide it by your screen's resolution. This way, it will work on any monitor.
you can do it like this :
import pywhatkit as kit
kit.sendwhatmsg('+966********','hi from Python Watsapp Bot',10,56)
I'm trying to write a python script which would allow me to paste large pieces of code. Now I can get the system to paste text from a variable. However the code I want to paste is long and has a lot of special characters in it. Is there any way I can get around this easily?
Here is the code I have at present.
import pyautogui
from tkinter import Tk
root = Tk()
root.withdraw()
result = 'Text to Paste'
x, y = pyautogui.position()
pyautogui.click(x, y)
pyautogui.typewrite(result)
import pyautogui, time, pyperclip # pyperclip used for copy and paste
var1 = ("words block here") # you replace this with large words
pyautogui.click(x=399, y=600) # click where you want to paste "var1" value
time.sleep(1) # delay
pyperclip.copy(var1) # variable that stores your pasting block
time.sleep(1) # delay
pyautogui.hotkey("ctrl", "v") # and paste it with pyautogui.
The problem
Im trying to capture my desktop with OpenCV and have Tesseract OCR find text and set it as a variable, for example, if I was going to play a game and have the capturing frame over a resource amount, I want it to print that and use it. A perfect example of this is a video by Micheal Reeves
where whenever he loses health in a game it shows it and sends it to his Bluetooth enabled airsoft gun to shoot him. So far I have this:
# imports
from PIL import ImageGrab
from PIL import Image
import numpy as np
import pytesseract
import argparse
import cv2
import os
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter("output.avi", fourcc, 5.0, (1366, 768))
while(True):
x = 760
y = 968
ox = 50
oy = 22
# screen capture
img = ImageGrab.grab(bbox=(x, y, x + ox, y + oy))
img_np = np.array(img)
frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2RGB)
cv2.imshow("Screen", frame)
out.write(frame)
if cv2.waitKey(1) == 0:
break
out.release()
cv2.destroyAllWindows()
it captures real-time and displays it in a window but I have no clue how to make it recognise the text every frame and output it.
any help?
It's fairly simple to grab the screen and pass it to tesseract for OCRing.
The PIL (pillow) library can grab the frames easily on MacOS and Windows. However, this feature has only recently been added for Linux, so the code below works around it not existing. (I'm on Ubuntu 19.10 and my Pillow does not support it).
Essentially the user starts the program with screen-region rectangle co-ordinates. The main loop continually grabs this area of the screen, feeding it to Tesseract. If Tesseract finds any non-whitespace text in that image, it is written to stdout.
Note that this is not a proper Real Time system. There is no guarantee of timeliness, each frame takes as long as it takes. Your machine might get 60 FPS or it might get 6. This will also be greatly influenced by the size of the rectangle your ask it to monitor.
#! /usr/bin/env python3
import sys
import pytesseract
from PIL import Image
# Import ImageGrab if possible, might fail on Linux
try:
from PIL import ImageGrab
use_grab = True
except Exception as ex:
# Some older versions of pillow don't support ImageGrab on Linux
# In which case we will use XLib
if ( sys.platform == 'linux' ):
from Xlib import display, X
use_grab = False
else:
raise ex
def screenGrab( rect ):
""" Given a rectangle, return a PIL Image of that part of the screen.
Handles a Linux installation with and older Pillow by falling-back
to using XLib """
global use_grab
x, y, width, height = rect
if ( use_grab ):
image = PIL.ImageGrab.grab( bbox=[ x, y, x+width, y+height ] )
else:
# ImageGrab can be missing under Linux
dsp = display.Display()
root = dsp.screen().root
raw_image = root.get_image( x, y, width, height, X.ZPixmap, 0xffffffff )
image = Image.frombuffer( "RGB", ( width, height ), raw_image.data, "raw", "BGRX", 0, 1 )
# DEBUG image.save( '/tmp/screen_grab.png', 'PNG' )
return image
### Do some rudimentary command line argument handling
### So the user can speicify the area of the screen to watch
if ( __name__ == "__main__" ):
EXE = sys.argv[0]
del( sys.argv[0] )
# EDIT: catch zero-args
if ( len( sys.argv ) != 4 or sys.argv[0] in ( '--help', '-h', '-?', '/?' ) ): # some minor help
sys.stderr.write( EXE + ": monitors section of screen for text\n" )
sys.stderr.write( EXE + ": Give x, y, width, height as arguments\n" )
sys.exit( 1 )
# TODO - add error checking
x = int( sys.argv[0] )
y = int( sys.argv[1] )
width = int( sys.argv[2] )
height = int( sys.argv[3] )
# Area of screen to monitor
screen_rect = [ x, y, width, height ]
print( EXE + ": watching " + str( screen_rect ) )
### Loop forever, monitoring the user-specified rectangle of the screen
while ( True ):
image = screenGrab( screen_rect ) # Grab the area of the screen
text = pytesseract.image_to_string( image ) # OCR the image
# IF the OCR found anything, write it to stdout.
text = text.strip()
if ( len( text ) > 0 ):
print( text )
This answer was cobbled together from various other answers on SO.
If you use this answer for anything regularly, it would be worth adding a rate-limiter to save some CPU. It could probably sleep for half a second every loop.
Tesseract is a single-use command-line application using files for input and output, meaning every OCR call creates a new process and initializes a new Tesseract engine, which includes reading multi-megabyte data files from disk. Its suitability as a real-time OCR engine will depend on the exact use case—more pixels requires more time—and which parameters are provided to tune the OCR engine. Some experimentation may ultimately be required to tune the engine to the exact scenario, but also expect the time required to OCR for a frame may exceed the frame time and a reduction in the frequency of OCR execution may be required, i.e. performing OCR at 10-20 FPS rather than 60+ FPS the game may be running at.
In my experience, a reasonably complex document in a 2200x1700px image can take anywhere from 0.5s to 2s using the english fast model with 4 cores (the default) on an aging CPU, however this "complex document" represents the worst-case scenario and makes no assumptions on the structure of the text being recognized. For many scenarios, such as extracting data from a game screen, assumptions can be made to implement a few optimizations and speed up OCR:
Reduce the size of the input image. When extracting specific information from the screen, crop the grabbed screen image as much as possible to only that information. If you're trying to extract a value like health, crop the image around just the health value.
Use the "fast" trained models to improve speed at the cost of accuracy. You can use the -l option to specify different models and the --testdata-dir option to specify the directory containing your model files. You can download multiple models and rename the files to "eng_fast.traineddata", "eng_best.traineddata", etc.
Use the --psm parameter to prevent page segmentation not required for your scenario. --psm 7 may be the best option for singular pieces of information, but play around with different values and find which works best.
Restrict the allowed character set if you know which characters will be used, such as if you're only looking for numerics, by changing the whitelist configuration value: -c tessedit_char_whitelist='1234567890'.
pytesseract is the best way to get started with implementing Tesseract, and the library can handle image input directly (although it saves the image to a file before passing to Tesseract) and pass the resulting text back using image_to_string(...).
import pytesseract
# Capture frame...
# If the frame requires cropping:
frame = frame[y:y + h, x:x + w]
# Perform OCR
text = pytesseract.image_to_string(frame, lang="eng_fast" config="--psm 7")
# Process the result
health = int(text)
Alright, I was having the same issue as you so I did some research into it and I'm sure that I found the solution! First, you will need these libraries:
cv2
pytesseract
Pillow(PIL)
numpy
Installation:
To install cv2, simply use this in a command line/command prompt: pip install opencv-python
Installing pytesseract is a little bit harder as you also need to pre-install Tesseract which is the program that actually does the ocr reading. First, follow this tutorial on how to install Tesseract. After that, in a command line/command prompt just use the command: pip install pytesseract
If you don't install this right you will get an error using the ocr
To install Pillow use the following command in a command-line/command prompt: python -m pip install --upgrade Pillow or python3 -m pip install --upgrade Pillow. The one that uses python works for me
To install NumPy, use the following command in a command-line/command prompt: pip install numpy. Thought it's usually already installed in most python libraries.
Code:
This code was made by me and as of right now it works how I want it to and similar to the effect that Michal had. It will take the top left of your screen, take a recorded image of it and show a window display of the image it's currently using OCR to read. Then in the console, it is printing out the text that it read on the screen.
# OCR Screen Scanner
# By Dornu Inene
# Libraries that you show have all installed
import cv2
import numpy as np
import pytesseract
# We only need the ImageGrab class from PIL
from PIL import ImageGrab
# Run forever unless you press Esc
while True:
# This instance will generate an image from
# the point of (115, 143) and (569, 283) in format of (x, y)
cap = ImageGrab.grab(bbox=(115, 143, 569, 283))
# For us to use cv2.imshow we need to convert the image into a numpy array
cap_arr = np.array(cap)
# This isn't really needed for getting the text from a window but
# It will show the image that it is reading it from
# cv2.imshow() shows a window display and it is using the image that we got
# use array as input to image
cv2.imshow("", cap_arr)
# Read the image that was grabbed from ImageGrab.grab using pytesseract.image_to_string
# This is the main thing that will collect the text information from that specific area of the window
text = pytesseract.image_to_string(cap)
# This just removes spaces from the beginning and ends of text
# and makes the the it reads more clean
text = text.strip()
# If any text was translated from the image, print it
if len(text) > 0:
print(text)
# This line will break the while loop when you press Esc
if cv2.waitKey(1) == 27:
break
# This will make sure all windows created from cv2 is destroyed
cv2.destroyAllWindows()
I hope this helped you with what you were looking for, it sure did help me!
I have currently written an python script that takes input from the user's video camera with the following code:
import numpy as np
import cv2
import cv
import datetime
import math
import time
import os
cap = cv2.VideoCapture(0)
def init():
ret, imgBase = cap.read()
print(type(imgBase))`
The above code is completely functional and it does get input from the user. However, instead of getting input from the user's web cam, I want to get input from user's screen. I need something like cap = cv2.ScreenCapture(0) I understand that that doesn't work, but I need something that does a similar function so I can read and capture an image from the user screen. In essence, I need to get a screenshot of the monitor/user's screen.
Only works on Mac:
import os
os.system('screencapture out.jpg')
You can read the screenshot into memory using default python functions.
For further info type man screencapture in a terminal or check this link: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/screencapture.1.html.