Develop Reference

Python pyautogui press-function arrows does not work in ribbon

I have a program in Python that starts another executable. Some automated operations need to be done in the ribbon of this open executable, so I use pyautogui to do so.
First the ribbon needs to be ‘active’, so I click on the left most part.
Then I need to use the arrows to change the ribbon menu selection (two times to the left).
Then I need to hit enter to open the correct menu. (going from 'File' to 'Scripting')
The code I’m using for this is:
import pyautogui
pyautogui.click(x=0, y=30)
pyautogui.press(['left', 'left']) #this part does not work here
pyautogui.hotkey('enter')
Somehow, the click and enter do work, but the arrow-keys don’t work. I can use the physical arrow-keys to change the menu selection, but this code doesn’t perform these actions somehow.
Does someone know what is wrong here and how to solve this?
Best regards,
Ganesh
EDIT:
I tried to open both the program and the script with admin right, but that still didn't work. Somehow, the 'enter' and everything else works, except for the arrows.

Ganesh, it might not work with pyautogui as the program/ interface you're using might simply not register the key. To use the arrow keys or other special keys like 'enter' I would suggest using pydirectinput
First, install the library if not already
pip install pydirectinput
Then you can rewrite your code as
import pyautogui
import pydirectinput
pyautogui.click(x=0, y=30)
pydirectinput.press(['left', 'left']) #this is the updated line of code
pyautogui.hotkey('enter')

Shift and mouse click

I am trying to hold shift key and at the same time click with the mouse. I searched the internet and found a stackoverflow question about this. This is the post: Pyautogui - Need to hold shift and click
Also the code in this post was working for like three times!
Then suddenly it stopped working. It is really strange. I also tried it with pynput. Here is my post: Holding shift key + mouse click
It seems like holding shift and clicking the mouse are working seperately.
However, together it seems not to work
This is the code:
import pyautogui
import time
time.sleep(2)
pyautogui.keyDown('shift')
pyautogui.click()
pyautogui.keyUp('shift')
I am running on windows OS

It looks working fine?
Maybe use more keydown('shift') will make it

I will suggest you not to use pyautogui.
Its easy and simple with other modules. Install keyboard(for controlling keyboard) and mouse(for controlling mouse)
Here is a sample code that does what you want:
import keyboard, mouse #< Importing the required modules
keyboard.press("shift") #< Presses and holds the key
mouse.click("left") #< Makes Left click
keyboard.release("shift") #< Releases the held key

How to control the mouse in Minecraft using Python?

All in all, I'm trying to programmatically -and externally- control the Minecraft player's orientation.
No APIs, no Java mods to the game environment
Typically this requires the movement of the mouse, but every single mouse movement simulating python3 library that I've tried doesn't move the player's head in-game. Each library does something different, too.
For example, pyautogui doesn't do anything until you move the mouse manually after the script has finished. Doing this will jerk the player's view to where the program supposedly moved it to, before continuing to follow your current mouse movements. This happens for both mouse commands.
import pyautogui
pyautogui.moveTo(500, 500)
pyautogui.moveRel(100, 100)
The pynput library had the same weird result as pyautogui:
from pynput.mouse import Controller
mouse = Controller()
mouse.position = (100, 200)
mouse.move(200, -100)
Quartz doesn't do anything at all:
import Quartz
class Mouse():
down = [Quartz.kCGEventLeftMouseDown, Quartz.kCGEventRightMouseDown, Quartz.kCGEventOtherMouseDown]
up = [Quartz.kCGEventLeftMouseUp, Quartz.kCGEventRightMouseUp, Quartz.kCGEventOtherMouseUp]
[LEFT, RIGHT, OTHER] = [0, 1, 2]
def click_pos(self, x, y, button=LEFT):
self.move(x, y)
self.click(button)
def to_relative(self, x, y):
curr_pos = Quartz.CGEventGetLocation( Quartz.CGEventCreate(None) )
x += curr_pos.x;
y += curr_pos.y;
mouse = Mouse()
mouse.to_relative(200, 200)
And the python mouse library is outdated: the error showed that it will only run on Darwin (I'm on macOS High Sierra). I was sad on learning this because of the description on the Github page. It says "Global event hook on all mice devices (captures events regardless of focus)". I then thought that, somehow, Minecraft was sucking up all the simulated mouse movements on it's own. Either way, I'm not using the right interface for this game, and I need something that can bypass Minecraft's interesting mouse controls to get the movement that I want.
I even tried using mouse keys (mac's mouse-moving accessibility feature that lets you control the mouse with only keys) along with pyautogui.
import pyautogui # with mouse keys on
import time
# mouse keys is an accessibility feature on mac that controls the mouse with the keyboard
print("[ALERT]: Make sure mouse keys is on! (press option 5 times if shortcut is enabled)")
pyautogui.keyDown('8') # up in mouse keys
time.sleep(5)
pyautogui.keyUp('8')
I wasn't particularly surprised that the last one didn't work, but I think I'm running out of ways to try and bypass whatever is making Minecraft not take my python-mouse input. At this point, I'm pretty sure that there must be some distinction in the kind of input that I'm giving the computer. Minecraft as a program doesn't use the mouse like other programs do, and python mice don't control the mouse like other sources do, so there is a disconnect.
I'm on my macOS High Sierra running Minecraft in both fullscreen and windowed mode, trying everything I can to get this to function properly. I'll start the test script (python 3.6) in PyCharm, change windows (or window focus) to Minecraft (with adequate delay time in-program), and then witness what happens. Mouse clicking, keyboard presses, and even hotkeys that involve the command and escape keys all work fine in Minecraft with pyautogui, so I'm not worried about those at all. It's literally just the mouse movement that's not doing anything.
First of all, is this the right place to ask this question? Is there anything else to try, or is there something crucial that I'm missing, that would allow my mouse input to be responded to correctly?

I am trying to do the same thing, and I got mine to move the view in Minecraft (Java edition).
What worked for me was to use pynput with relative mouse commands. It also needed 'Raw Input' to be off in the Minecraft settings. [Esc -> Options... -> Controls... -> Mouse Settings... -> Raw input: OFF]
import pynput
mouse = pynput.mouse.Controller()
mouse.move(10, 10)
Also, here's the beginnings to a smooth movement of the mouse if anyone wants it:
def move_smooth(xm, ym, t):
for i in range(t):
if i < t/2:
h = i
else:
h = t - i
mouse.move(h*xm, h*ym)
time.sleep(1/60)
move_smooth(2, 2, 40)
Now, onto trying to make the keyboard work :P

I managed to make it work with the mouse library. Instead of using mouse.move(x,y,absolute,duration) I used mouse._os_mouse.move_to(x,y) and mouse._os_mouse.move_relative(x,y). Take into account that if you want a smooth effect you'll have to implement it yourself using something like time.sleep(s).

This question has also bothered me for a while and I finally found a solution to this question.(macOS Monterey 12.2, M1 chip 2020)
I have tried multiple python library and macro apps which none worked and rawinput also didn't help. However, pointer control in system preferences works with Minecraft mouse movement. The mouse keys option allows you to control mouse movement with keystrokes,
Where to find Mouse Keys in system preferences
System Preferences -> Accessibility -> Pointer control(under Motor) -> Enable Mouse Keys
so I tried to simulate the keyboard to move the mouse. For some reason, none of the python library or macro apps will work with mouse keys except for apple script. So I wrote an apple script and embedded it in python, which worked moving the mouse in Minecraft when the mouse key option is on.
The mouse keys option will allow you to use the numpad keys to control you mouse.
https://eastmanreference.com/complete-list-of-applescript-key-codes
This link will tell you the key code for the keystroke you want to press.
import os
cmd=""" osascript -e '
repeat 500 times
tell application "System Events" to key code 88 --right
end repeat'
"""
os.system(cmd)
In the script key code 88 matches numpad key 6, which will move your cursor right, change the key code to customize where the cursor will move.

I am in a similar situation to you. I was also unable to find a way to register my mouse movements in games such as minecraft.
However, I learned that you can use Java and the built-in robot library to achieve the mouse movement as desired. I don't know if you are set on python but if not it's worth checking out.

In the Minecraft Options... go to Controls... go to Mouse Settings... and there turn off Raw Input.
That should do it (you're on mac tho and I don't know if this setting is on
the mac version of minecraft)

I was struggling with this too and I found the following findings (tested on Ubuntu 20.04):
If you activate RawInput, you can use mouse.position=(dx, dy) which (although it is intended to be absolute) causes relative motion of the head! And it also works with negative values for dx and dy.
This is much faster and more accurate. You can for instance chain
mouse.position = (0, -100000) # turn head all the way up
mouse.position = (0, 600) # turn head exactly to horizon level
The horizon is 600 "pitch-pixels" down from upright, when mouse sensitivity is set to 100%. You don't need any delay between the calls and they happen so fast that in the vast majority of cases, the player does not seem to look up in between, but looks to the horizon right away. When I used the relative motion with raw input off, I needed sleeps in between and the speed was also capped at some value.
The only issue is, that you need to switch between this hacky way of controlling the mouse and the normal way using mouse.move(dx, dy) or with actually absolute screen coordinates mouse.position=(x, y) when you are on the crafting screen / in a menu. But I found that if you also create a pynput mouse listener, you will see that Minecraft sets the mouse position to the center of its window everytime you enter a menu and it does it another two times when you go back to the game. So I use these as triggers.

Try running python as admin and run the game in windowed mode. Pyautogui should work then.

Media Play/Pause Simulation

My keyboard contains a row of buttons for various non-standard keyboard tasks. These keys contain such functions as modifying the volume, playing or pausing, and skipping tracks. How can I simulate a basic play/pause with Python? I am on Windows, by the way.

I would use pywin32. Bundled with the installation is a large number of API-docs (usually placed at something like C:\Python32\Lib\site-packages.) It essentially wraps a lot of stuff in the Win32-library which is used for many low-levels tasks in Windows.
After installing it you could use the wrapper for keybd_event.
You could also use SendInput instead of keybd_event but it doesn't seem to be wrapped by PyWin32. SendMessage is also an option but more cumbersome.
You'll need to look up the virtual scan code for those special buttons, since I doubt the char-to-code mapping functions will help you here. You can find the reference here.
Then it is a simple matter of calling the function. The snippet below pauses Chuck Berry on my computer.
>>> import win32api
>>> VK_MEDIA_PLAY_PAUSE = 0xB3
>>> hwcode = win32api.MapVirtualKey(VK_MEDIA_PLAY_PAUSE, 0)
>>> hwcode
34
>>> win32api.keybd_event(VK_MEDIA_PLAY_PAUSE, hwcode)
MapVirtualKey gives us the hardware scan code which keybd_event needs (or more likely, the keyboard driver.)
Note that all this is snapped up by the keyboard driver, so you don't really have any control where the keystrokes are sent. With SendMessage you can send them to a specific window. It usually doesn't matter with media keys since those are intercepted by music players and such.

This was not working for me on Windows 10 64, all recent updates installed. I needed this (the 3rd parameter 2) before it'd work.
win32api.keybd_event(0xB3,0,2,0)
didn't bother looking into why it works, but threw it out there since this and other similar posts had me frustrated as for some reason not working on my PC.
This successfully paused/played Amazon Music on my PC.

You can use pyautogui. This library contains a lot of keyboard and mouse simulations.
To install run pip install pyautogui in cmd.
In order to simulate a play/pause keypress, you should use pyautogui.press("playpause").
Check out their docs at https://pyautogui.readthedocs.io/en/latest/keyboard.html to see the list of the supported keys and some other keyboard functions.

Simulate Mouse Clicks on Python

I'm currently in the process of making my Nintendo Wiimote (Kinda sad actually) to work with my computer as a mouse. I've managed to make the nunchuk's stick control actually move the mouse up and down, left and right on the screen! This was so exciting. Now I'm stuck.
I want to left/right click on things via python when I press A, When I went to do a search, All it came up with was tkinter?
So my question is, What do I call to make python left/right click on the desktop, and if it's possible, maybe provide a snippet?
Thank you for your help!
NOTE: I guess I forgot to mention that this is for Linux.

You can use PyMouse which has now merged with PyUserInput. I installed it via pip:
apt-get install python-pip
pip install pymouse
In some cases it used the cursor and in others it simulated mouse events without the cursor.
from pymouse import PyMouse
m = PyMouse()
m.position() #gets mouse current position coordinates
m.move(x,y)
m.click(x,y) #the third argument "1" represents the mouse button
m.press(x,y) #mouse button press
m.release(x,y) #mouse button release
You can also specify which mouse button you want used. Ex left button:
m.click(x,y,1)
Keep in mind, on Linux it requires Xlib.

The evdev package provides bindings to parts of the input handling subsystem in Linux. It also happens to include a pythonic interface to uinput.
Example of sending a relative motion event and a left mouse click with evdev:
from evdev import UInput, ecodes as e
capabilities = {
e.EV_REL : (e.REL_X, e.REL_Y),
e.EV_KEY : (e.BTN_LEFT, e.BTN_RIGHT),
}
with UInput(capabilities) as ui:
ui.write(e.EV_REL, e.REL_X, 10)
ui.write(e.EV_REL, e.REL_Y, 10)
ui.write(e.EV_KEY, e.BTN_LEFT, 1)
ui.syn()

PyAutoGui works superb.. Thanks to Al Sweigart...
An example of mine...
import pyautogui
pyautogui.FAILSAFE = False
for x in range(555, 899):
pyautogui.moveTo(x, x)

Open your terminal and goto cd /usr/share/pyshared/twisted/protocols/mice
may this __init__.py mouseman.py python script will work for you,check them out.

You can install the PyAutoGUI GUI automation module from PyPI (run pip install pyautogui) and then call the pyautogui.click() to click on a certain X and Y coordinates of the screen:
>>> import pyautogui
>>> pyautogui.click(50, 100)
>>> pyautogui.moveTo(200, 200)
PyAutoGUI works on Windows, Mac, and Linux, and on Python 2 and 3. It also can emulate the keyboard, do mouse drags, take screenshots, and do simple image recognition of the screenshots.
Full docs are at https://pyautogui.readthedocs.org/

I didn't see this mentioned, so here it goes - there is also python-dogtail; see:
Automated GUI testing with Dogtail | Red Hat
Testing/Automation/DogtailTutorial - Ubuntu Wiki
It requires "Enable assistive technologies" in the Gnome Desktop - but can in principle obtain e.g. names of GUI buttons of an application, and allow virtual clicks on them (rather than via x/y coordinates).