I am developing a code using pyautogui where i detect an event in the window "A" and them, with alt+tab, execute an action in the window "B". I would like to know if it is possible to do this, but with the window "A" minimized, so it doesnt have to alt+tab all the time (the solution doesnt necessary has to be with pyautogui).
If you intended to use the locate function with pyautogui, whether or not it's LocateAll or just locate, it has to see that same matching image on screen first then you can adjust the confident to your needs. What you're asking is really hard to answer without knowing what you wanted to do, for example if you wanna do that on browser then you can replace it by selenium for example, for normal window can just make a script to let it switch tab for you then let pyautogui detect and do it job
Related
I have a script that is used to login in to a company-made application and click the right buttons like "continue", or "ok", etc. to perform a certain process. However, I have had to use screenshots of these buttons to click in order to do this using pyautogui. Is there any package or way to automate this process without using images. Maybe it can detect the text of the button and click it. I do not have identifiers for the buttons available and no access to the code/info behind the application. Let me know if you have any ideas. Thanks!
I have a few questions that may be helpful:
Does the layout of the buttons change? If it's always the same you can just program the correct locations and timing and not worry about reading the screen.
If you really have to read the screen, look into optical character recognition (ocr).
Is the application keyboard accessible? If so, using Tab and Enter to activate the buttons is simpler than controlling the mouse. Also, if it was made by superstars you can use find (ctrl-F) to search for the text on the buttons and go to them.
This answer is pretty vague, but I can only be as specific as the question asked.
A python evdev device has a .grab() function that prevents other processes from getting input events on the device. Is there any way to limit this to specific events from a device?
For my example, if I .grab() a pen input device that has pressure sensitivity and tilt and 2 click buttons on the side, how would I 'grab' ONLY the 2 click buttons but let the rest of the input (the tip, pressure sensitivity and tilt) be caught by the rest of the system as normal?
One of my pen buttons is normally a right click mouse event. I want to make it do something else but it still pops up the right click menu so I'm trying to figure out how to stop that.
I tried doing the grab and ungrab when the event occurs. Like event > grab > do my stuff > ungrab. But that is obviously too late and the OS still pops up the menu.
I tried doing the full grab, then in the event loop if it is a button press do my stuff, otherwise create a UInput event injection by just passing the event back to the system. This was a bit of a tangled mess. Permissions are required. When I finally got past that, the movement was offset and the pressure/tilt wasn't working... I think it is something to do with the DigiMend driver that actually makes that stuff work and/or xinput settings I have to pass to calibrate the tablet. But I'm not interested in writing all the pressure/tilt functionality from scratch or anything like that, so I need the DigiMend stuff to work as normal. So I gave up on this idea for now.
The only other thought I had was figure out why the OS defaults to the behavior it does and see if I can just manually disable the actions (i.e. Why does it think that button is a right mouse click and make it think that button is nothing instead.)
So I guess this is a 3 level question.
Can I achieve the the grab functionality on select events instead of the device as a whole?
If the passthrough idea was better, is there a way to achieve this without having to do any permission modifications and be able to pass the exact event (i.e. no offset and such that I experienced?)
If evdev does not have this ability or it'd be easier to do in another way, like disabling the defaults for the pen in the OS somehow, I am open to suggestions. I am using Kubuntu 20.04 if that helps.
Any help would be appreciated, let me know if more info is needed, thanks in advance!
I ended up going with #3 and using xinput. Figured I'd put up this answer for now in case others come across this and want to do something similar.
The workaround was actually kind of simple. I just use xinput to remap the 2 buttons. So evdev doesn't have to grab at all. Just disable those buttons and everything goes normally except those, which I listen for with evdev.
xinput set-button-map {} 1 0 0 4 5 6 7
My device has 7 buttons and are normally mapped 1-7. Which are all mouse equivalents of left click, middle click, right click, etc...
By using that string and passing the device ID in for the {} I just run that with subprocess first. And voila, no more right click menu. And I can use evdev to map the events to whatever I want.
what i am trying to do:
i trying to use the code below to locate and give me a X and Y position of the image
pyautogui.locateCenterOnScreen('accept.png')
after locating the image, i am trying to use pywinauto* to click on the image location in the background.
form.click(button='left', pressed='', coords=(pyautogui.locateCenterOnScreen('accept.png')), double=False, absolute=False)
problem:
nothing seems to happen.. i dont understand why. i checked on the pywinauto* and pyautogui "cheat sheets" it seems okay
someone please enlighten me
The correct spelling is "pywinauto". Method .click_input(...) with the same params is what you need as it moves the cursor and performs realistic click.
Method .click(...) silently sends window messages like WM_LBUTTONDOWN and WM_LBUTTONUP which may not work for some UI elements if they don't handle it.
I have a program in Python that starts another executable. Some automated operations need to be done in the ribbon of this open executable, so I use pyautogui to do so.
First the ribbon needs to be ‘active’, so I click on the left most part.
Then I need to use the arrows to change the ribbon menu selection (two times to the left).
Then I need to hit enter to open the correct menu. (going from 'File' to 'Scripting')
The code I’m using for this is:
import pyautogui
pyautogui.click(x=0, y=30)
pyautogui.press(['left', 'left']) #this part does not work here
pyautogui.hotkey('enter')
Somehow, the click and enter do work, but the arrow-keys don’t work. I can use the physical arrow-keys to change the menu selection, but this code doesn’t perform these actions somehow.
Does someone know what is wrong here and how to solve this?
Best regards,
Ganesh
EDIT:
I tried to open both the program and the script with admin right, but that still didn't work. Somehow, the 'enter' and everything else works, except for the arrows.
Ganesh, it might not work with pyautogui as the program/ interface you're using might simply not register the key. To use the arrow keys or other special keys like 'enter' I would suggest using pydirectinput
First, install the library if not already
pip install pydirectinput
Then you can rewrite your code as
import pyautogui
import pydirectinput
pyautogui.click(x=0, y=30)
pydirectinput.press(['left', 'left']) #this is the updated line of code
pyautogui.hotkey('enter')
In the app I'm trying to automate there is an error window that could pop-up in literally any moment of application work. Then I need to perform some actions like logging this event and stop next automation steps. So I need to catch this moment. How can I do it?
This window always has the same properties (auto_id) so I can describe it before it exist. I can think of using something like a timer which will always check for existence of this window (with the help of .exists(), I guess). But maybe there is a better way of doing it?