I am trying to simulate a mouse left click on a window (Samsung Flow) but unfortunately it is not working. When I try on a google chrome window it works, but I have tried on Paint and on Samsung Flow but it does not work. Here is my code:
import win32api
import win32con
import time
import random
def enumHandler(hwnd, lParam):
if win32gui.IsWindowVisible(hwnd):
# if 'Stack Overflow' in win32gui.GetWindowText(hwnd):
if 'Samsung Flow' in win32gui.GetWindowText(hwnd):
print(win32gui.GetWindowText(hwnd))
for _ in range(50):
l_param = win32api.MAKELONG(random.randint(10, 500), random.randint(10, 500))
win32gui.PostMessage(hwnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, l_param)
time.sleep(0.1)
win32gui.PostMessage(hwnd, win32con.WM_LBUTTONUP, win32con.MK_LBUTTON, l_param)
time.sleep(0.1)
win32gui.EnumWindows(enumHandler, None)
I know that it detects the window as I am printing the detected text, but I do not know why it only work on the Chrome window.
Update
I have tried the following code, to search two windows notepad and browser:
import win32gui
import win32api
import win32con
import time
def enumHandler(hwnd, lParam):
if win32gui.IsWindowVisible(hwnd):
# if 'Notepad' in win32gui.GetWindowText(hwnd):
if 'Stack Overflow' in win32gui.GetWindowText(hwnd):
print(win32gui.GetWindowText(hwnd))
win32gui.SetForegroundWindow(hwnd)
win32api.SendMessage(hwnd, win32con.WM_CHAR, ord("c"), 0)
time.sleep(0.1)
win32gui.EnumWindows(enumHandler, None)
And the result was:
when using on the browser (searching for Stack Overflow window), the window came to foreground and printed the letter c
when using the notepad the window came to foreground but the letter was not printed! and I have no idea why.
Well, I found the problem.
With the example trying to send the letter C to notepad and the chrome browser I assumed that the first hwnd is the right one BUT in some cases, you have to interact with a child window. A window may have more than one child window, so, I will post a code here where you can find the window to interact with.
import win32gui
import win32con
import win32api
import time
def send_char(hwnd, lparam):
s = win32gui.GetWindowText(hwnd)
print("child_hwnd: %d txt: %s" % (hwnd, s))
win32api.PostMessage(hwnd, win32con.WM_CHAR, ord('c'), 0)
time.sleep(5)
return 1
def main():
main_app = 'Untitled - Notepad'
hwnd = win32gui.FindWindow(None, main_app)
if hwnd:
win32gui.EnumChildWindows(hwnd, send_char, None)
main()
With that you can find the child window that you should send the messages to (the code print the window id and name, send the character and wait 5 seconds, so when you notice the character on your window just get the last printed window id and use it instead of the parent window).
I hope it help someone facing the same problem.
Related
I'm looking to create a bot for a game I play, where he also created a bot by another company where he does things in background mode and without moving the user's cursor, so he tried to use their same method, in this case, a " bot vision" and click background on it
import win32gui, win32api, win32con
def click_game_window():
game_window_name = 'GAME'
game_hwnd = win32gui.FindWindow(None, game_window_name)
click(game_hwnd)
def click(hwnd):
lParam = win32api.MAKELONG(935, 821)
win32api.SendMessage(hwnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, lParam)
win32api.SendMessage(hwnd, win32con.WM_LBUTTONUP, None, lParam)
so I did that, but it is not clicking in the game, I tested it in other windows like NOTEPAD, but it still didn't work, anyone can help me, PLEASE?
I can click on the window, but it doesn't move my character, or interact with anything in game. I've tried moving the mouse around, i've tried doing keyboard inputs, full screen, windowed, etc. I've also tried using screenshots with pyautogui, but no luck. The game i'm trying to use it with was initially released in 2000. Non coding wise i've tried running it as admin, running in windows xp sp 2-3 compatibility mode, disabling desktop composition, etc.
win32api code:
import win32api, win32con
import time
def click(x,y):
win32api.SetCursorPos((x,y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)
# click(573, 841)
# time.sleep(1)
# click(289, 342)
# time.sleep(1)
time.sleep(5)
click(319, 399)
x = win32api.GetCursorPos()
print(x)
error:
win32api.SetCursorPos((x,y)) pywintypes.error: (0, 'SetCursorPos', 'No error message is available')
pyautogui/pydirect input:
import pyautogui
import pydirectinput as p
import time
icon = pyautogui.locateCenterOnScreen('./icon.png', confidence=0.9)
p.click(icon[0], icon[1])
time.sleep(2)
p.press('enter')
this code doesn't throw an error, it completes normally without actually clicking in the game window
First, make sure you are running your script as admin, sometimes if you don't Windows will prevent mouse movement.
Also, try doing this:
def click(x,y):
win32api.SetCursorPos((x, y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN, 0, 0)
time.sleep(.01)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP, 0, 0
You have to give it a little bit of time to click or else Python will do it too quickly and the game won't register it.
I have seen a few methods, but none of the work.
import win32gui, win32process, win32con
import os
windowList = []
win32gui.EnumWindows(lambda hwnd, windowList: windowList.append((win32gui.GetWindowText(hwnd),hwnd)), windowList)
cmdWindow = [i for i in windowList if "c:\python26\python.exe" in i[0].lower()]
win32gui.SetWindowPos(cmdWindow[0][1],win32con.HWND_TOPMOST,0,0,100,100,0) #100,100 is the size of the window
'''
This one doens't work because the range exceeds the list.
'''
import win32gui
import win32con
hwnd = win32gui.GetForegroundWindow()
win32gui.SetWindowPos(hwnd,win32con.HWND_TOPMOST,100,100,200,200,0)
This one works as in it stays on top, but not the window I want.
Is there a line I could add so it knows which window I want? Or is there another way to approach this?
Use win32gui.FindWindow
hwnd = win32gui.FindWindow(None, 'Untitled - Notepad')
win32gui.SetWindowPos(hwnd,win32con.HWND_TOPMOST,100,100,200,200,0)
import win32gui
import time
def windowEnumerationHandler(hwnd, top_windows):
top_windows.append((hwnd, win32gui.GetWindowText(hwnd)))
if __name__ == "__main__":
top_windows = []
win32gui.EnumWindows(windowEnumerationHandler, top_windows)
for i in top_windows:
print(i)
if "zoom" in i[1].lower():
print(i, 'is found')
while True:
win32gui.ShowWindow(i[0],5)
win32gui.SetForegroundWindow(i[0])
time.sleep(1)
I've heard that zoom monitors whether the window is not in focus for more than 30 seconds, so I've been working on a way to repetitively throw it to the front while I work on other projects. The problem is the code raises an exception
0, 'SetForegroundWindow', 'No error message is available'
and the window just flashes yellow. Same problem with chrome as well. Would appreciate some help here :)
I had the same problem while I was trying to SetForegroundWindow(hwnd). The icon on the taskbar was just flashing, but the program stayed in the background. As you can read here:
https://learn.microsoft.com/zh-cn/windows/win32/api/winuser/nf-winuser-setforegroundwindow?redirectedfrom=MSDN
"An application cannot force a window to the foreground while the user is working with another window. Instead, Windows flashes the taskbar button of the window to notify the user."
For me helped:
import win32gui, win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys('%')
win32gui.SetForegroundWindow(hwnd)
I'm trying to listen for mouse click events on task bar app icons for currently running apps in windows using python. In the example below, I use python to initiate the internet explorer application. After the app initiates, I then use the FindWindow windows API function to get the window ID/handle.
Is there a way to listen for mouse clicks on the internet explorer task bar icon by the user using windows API or some other method? I need to print a message when the IE icon in the task bar is clicked.
Here is the code I am working with currently:
from win32gui import GetWindowText, GetForegroundWindow, FindWindow
import win32api, win32gui
import win32con
import sys, os
os.system('"C:\Program Files (x86)\Internet Explorer\iexplore.exe"') #open internet explorer application.. this will make the internet explorer icon appear in the taskbar..
window_id = FindWindow(0, 'Internet Explorer') #find the newly opened IE window's id/handle
print (window_id)
class test:
def __init__(self, **kwargs):
super(test, self).__init__(**kwargs)
self.oldWndProc = win32gui.SetWindowLong(window_id, win32con.GWL_WNDPROC, self.MyWndProc)
self.msgdict = {}
for name in dir(win32con):
if name.startswith("WM_"):
value = getattr(win32con, name)
self.msgdict[value] = name
print (self.msgdict) #I can access all the win32 window events using this dictionary..
#listener to track win32 window events related to the newly opened internet explorer window..
#how do I track when the internet explorer icon is CLICKED in the task bar??
def MyWndProc(self, hWnd, msg, wParam, lParam):
print (self.msgdict.get(msg)) #none of the window events print here..
return win32gui.CallWindowProc(self.oldWndProc, hWnd, msg, wParam, lParam)
t = test()
Try to use UI Automation.
The window class name of the taskbar is MSTaskListWClass:
And you can also see the UIA element and its BoundingRectangle from inspect.exe:
You could first get the click event on desktop and then get the task icon's IUIAutomationElement, then use GetClickablePoint to determine whether the click is on the element.
Thanks to Drake Wu pointing me in the right direction, I was able to use python to get UI Automation info about the element that is clicked and its parent. Using that information, I can tell that an app icon in the taskbar is clicked when the ControlType of the element clicked is 'ButtonControl' and the class name of its parent is 'MSTaskListWClass'. You can figure out which application was clicked by looking at the Automation ID of the clicked element.
You need to pip install uiautomation for this example. I had to add a few additional functions/methods to the uiautomation.py file because it was missing functions that were needed.
Here are the methods that I added to the uiautomation.py file to make it more useful for this example (see where it says NEW CODE HERE):
def ControlFromPoint(x: int, y: int) -> Control:
"""
Call IUIAutomation ElementFromPoint x,y. May return None if mouse is over cmd's title bar icon.
Return `Control` subclass or None.
"""
element = _AutomationClient.instance().IUIAutomation.ElementFromPoint(ctypes.wintypes.POINT(x, y))
return Control.CreateControlFromElement(element)
#NEW CODE HERE
def GetElementFromPoint(x: int, y: int) -> Control: #gets the element from a point (x, y)
element = _AutomationClient.instance().IUIAutomation.ElementFromPoint(ctypes.wintypes.POINT(x, y))
return element
#NEW CODE HERE
def GetParentElement(element): #gets the parent of an element input
parent_element = _AutomationClient.instance().ViewWalker.GetParentElement(element)
return parent_element
#NEW CODE HERE
def GetElementInfo(element): #gets the property information about an element
element_info = Control.CreateControlFromElement(element)
return element_info
I then utilized these newly added uiautomation methods in the main example below:
import sys, os
from pynput import mouse
from pynput.mouse import Controller
from pynput.mouse import Listener
import uiautomation
from uiautomation import GetElementFromPoint, GetParentElement, GetElementInfo #these are newly added methods to make this example work
mouse2 = mouse.Controller()
class test:
last_mouse_pos_x = ''
last_mouse_pos_y = ''
def __init__(self, **kwargs):
super(test, self).__init__(**kwargs)
self.mouse_listener() #start mouse listener
def mouse_listener(self):
global element_clicked
while True:
if mouse2.position[0] != test.last_mouse_pos_x or mouse2.position[1] != test.last_mouse_pos_y:
element_clicked = uiautomation.GetElementFromPoint(mouse2.position[0], mouse2.position[1]) #returns the element that you clicked (this has to run here or it errors)
test.last_mouse_pos_x = mouse2.position[0]
test.last_mouse_pos_y = mouse2.position[1]
def on_click(x, y, button, pressed):
if pressed:
try:
element_clicked_info = GetElementInfo(element_clicked) #returns info about the element you clicked
parent_of_element_clicked = uiautomation.GetParentElement(element_clicked) #returns the parent of the element that you clicked
parent_of_element_clicked_info = GetElementInfo(parent_of_element_clicked) #returns info about the parent of the element you clicked
#print info about the element you clicked and its parent
print ("ELEMENT CLICKED INFO: " + str(element_clicked_info))
print ("PARENT ELEMENT INFO: " + str(parent_of_element_clicked_info))
except:
pass #skip over any errors
#ANALYSIS: If ControlType of element clicked = 'ButtonControl' AND class name of its parent = 'MSTaskListWClass', then an app icon in the taskbar was clicked!
listener = mouse.Listener(on_click=on_click)
listener.start()
t = test()