I am using pyautogui to automate some mouse clicking in a script that I created.
This script needs to perform one click to log into a program, nothing else (besides other non-clicking stuff). The idea is that the script will run on a remote Win10 PC via a task scheduler every day at, lets say, at 11PM.
My script works perfectly when run on my own PC but I am running into issues with how to handle the remote RDP resolution to correctly identify the mouse position to click. Furthermore, the script will run on a remote PC that might not have "active user" logged in and I am not even sure if this mouse clicking approach will work. How to identify correctly the "resolution" of the remote PC?
Could you help me?
Thanks.
Pyautogui can be used to interact with remote system the same way a human can by opening the RDP or VNC window identifying elements inside of it and then directly interacting with them. This works but you're now expecting two systems to be in sufficiently consistent state for the automation to work. This does have the advantage that you don't need to know the resolution of the remote system just what the interactable elements look like when rendered through the RDP client on your system.
Related
I want to automate a very old software (drafix) that can only be run in Windows XP, on a remote desktop.
Would it be possible to write a pywinauto script that would directly interact with the software controls, without the need to send mouse/keyboard clicks on the GUI, and then minimize the Remote Desktop screen or even lock the PC?
I need to open the program, loop through a list of file names, open each one of them and save as a different format.
I did it with some basic GUI automation - clicking on buttons if necessary, sending keyboard shortcuts and entering the file names form a list. But it isn't very reliable, and I would like to minimize the Remote Desktop window and use my PC instead of having it in the foreground.
From what I have read here, it should be possible with some workarounds, and I would need to install an older version of Python (any idea which one?) on the remote desktop to write and run the script there.
What you can do is just copying your script to remote machine and run it there. Regarding minimized or closed RDP there are few workarounds and tricks described in the Remote Execution Guide which summarizes few StackOverflow answers.
There is other way that you can control gui application on remote Desktop with pywinauto. You can use rpyc or Remote Python Call. that is a python library for rpc(remote procedure call)
https://rpyc.readthedocs.io/en/latest/
your remote computer is as server and your host is as client in rpyc.
first you you need to run rpyc_classic.py in remote computer after connect to it and use pywinauto library. for example i use rpyc for start microsip on remote computer with pywinauto.
ip of remote computer is 192.168.222.222.
import rpyc
ip = "192.168.222.222"
conn = rpyc.classic.connect(ip)
conn.execute("from pywinauto import Application")
conn.execute(r"Application().start(r'C:\Program Files (x86)\MicroSIP\microsip.exe')")
I have a sikuli script which does the Siebel Tools incremental Compilation Task on a Windows Box from Jenkins.
The problem is: When i run the task i always have to open the RDC window open for the task to be executed.As soon i minimize the window the script fails.
Hence if the Sikuli script is running i will not able to use my local system for any other task.
I found some posts on launchpad.net.
this is one reference: https://answers.launchpad.net/sikuli/+question/213636
But it did not work.
Can anyone help on this.
This solution has helped me out:
Close all open Remote Desktop sessions.
Launch the Registry editor (regedit.exe).
Navigate to one of the following Registry keys, depending on whether you wish to modify the Remote Desktop settings only for the current user or for all users on the computer:
HKEY_CURRENT_USER\Software\Microsoft\Terminal Server Client
HKEY_LOCAL_MACHINE\Software\Microsoft\Terminal Server Client
Create a DWORD value named RemoteDesktop_SuppressWhenMinimized and set it to 2.
Close the Registry Editor.
After these steps are done RDC when minimized will not be suppressed and sikuli will work.
Answer taken from:
https://support.smartbear.com/testcomplete/docs/testing-with/running/via-rdp/in-minimized-window.html
Other solution is to use VNC connection instead of RDC. RealVNC for example.
You setup VNC server on machine that sikuli scripts are running on and connect from your other machine. You should even be able to close the connection window and scripts should still be able to run.
You could just keep the RDC session running in the background. Just don't minimize it. I am doing the same with my tests.
I prepared a script which does some operation on a web page, recognize colors and simulate mouse click and keyboard send keys while running. It works fine.
The problem is that I want it to run on a virtual machine (I'm trying greencloudvps) and it works fine as long as have my eyes on it.
As soon as I close the "remote desktop connection" tool, the script crashes, because (I guess) there is no more a monitor where to move the mouse.
I use the following Python code:
win32api.SetCursorPos((x,y))
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
Is there any solution?
UPDATE:i tried to run my script also on Amazon virtual machine but i got the same problem. Which could be the solution?
Basically I need to click with mouse, read pixel colors and send skeys with keyboard automatically on a virtual machine even when I don’t witness the session, using python
So I am trying to set up a Continuous integration environment using Jenkins.
One of the build step requires a series of mouse actions/movements to accomplish a task in Excel. I have already written a python script using the ctypes library to do this.
The script works perfectly fine if I run it either through Jenkins or on the server itself when I am actively logged in to the server using remote desktop connection, but as soon as I minimize/close the connection and then run the script from Jenkins, it seems the mouse events never get executed. Is there something I can add to the script to make this work? Thanks for any help you can provide.
Is there a way to automate an old DOS application (16-bit, probably needs an emulator such as DOSBox) from Python (on Windows)? I would like to send keys and strings to the application, detect updates to the DOS "screen" and get the application output.
It would be even better if the DOS application could run "hidden", i.e., not showing in the taskbar.
Note: It is not a game, it is one of those old application where you are given menus with press 1 for something, press 2 for something else, etc... then it asks for some input numbers, and then it shows some results. It is a pure console application.
Note2: It doesn't need to use DOSBox necessarily... could be done with other emulators such as Bochs
I'm not familiar with DosBOX or whether it has an external API. However, for Sun VirtualBox there is a python API, so if it is OK to run DOS on a VM, you could easily use the VirtualBox Python API to control & automate the application you run on the DOS.
You can download the VirtualBox SDK here