I am creating a small app for myself to be able to show my keyboard inputs and show them in a Joystick layout, like this:
This itself, works perfectly fine while the Pygame windows is focused, my problem is, i can't have this focused all the time, in fact it will never have focus because either way i will be using OBS to stream or i will be using my emulator, and pygame doesn't detect inputs that are out of the window. Is there any way to make python or pygame read all the input made to the computer??? I am hitting a wall here. Thanks in advance!!
There's an SDL environment variable for this! Simply setting it will enable background input.
import os
os.environ["SDL_JOYSTICK_ALLOW_BACKGROUND_EVENTS"] = "1"
Add that to the top of the sample in the joystick docs and try it out.
Tested with pygame 2.1.2, sdl 2.0.18, Windows 10 (21H2), Xbox One like controller
ref:
https://wiki.libsdl.org/SDL_HINT_JOYSTICK_ALLOW_BACKGROUND_EVENTS
https://wiki.libsdl.org/CategoryHints
As per two comments directly above, IN order to keep focus on the pygame window (effectively keep the cursor within the bounds of the pygame window until a I was able to make this work with the following code:
pygame.event.set_grab(True) # Keeps the cursor within the pygame window
Combine this code with a way to quit the program with a key press such as the ESCAPE key (since it will be impossible to close the window by moving your cursor to the frame of the window to close it that way by pygame.QUIT):
run = True
while run:
for event in pygame.event.get():
if event.type == pygame.QUIT or (event.type == KEYDOWN and event.key == K_ESCAPE):
run = False
Use pygame.event.set_grab(True)
When your program runs in a windowed environment, it will share the mouse and keyboard devices with other applications that have focus. If your program sets the event grab to True, it will lock all input into your program.
It is best to not always grab the input, since it prevents the user from doing other things on their system.
You will need another way to exit though, since you will not be able to move the mouse out of the display window.
Related
I was experimenting with pygame and noticed it raised a VideoExpose event when I press alt+tab and the window is fullscreen. when I switch press alt+tab again, everything on the screen is moved to the bottom left.
I know that this is supposed to mean that 'portions of the window must be redrawn', but how am I supposed to redraw them and what why does pygame even have this event in the first place?
If you are writing a program to use the windowing Event Model, the windowing environment sends the program events to notify it of environmental changes - window resize, mouse move, need to re-paint, etc.
Not handling these events will cause your application to be considered "non responsive" by the environment. Here on SO, there's about one question a week with PyGame and exactly this issue.
When working with PyGame re-drawing event handling may seem superfluous as the majority of PyGame games redraw the entire screen every frame, e.g.: 60 FPS. But if unnecessary, this method is a complete waste of resources (CPU-time, electricity, etc.) It is quite simple though, so good for beginners.
Say you were writing a card game like Solitaire... the screen updates only when interacting with the user. In terms of CPU, it's doing nothing 99.9% of the time while the user contemplates their next move. In this case, the program could be written to only re-draw the screen when necessary. When is it necessary? When the player gives input, or the program receives a pygame.VIDEOEXPOSE event from the widowing environment.
If your program is redrawing the window constantly, you can simply ignore the message. If not, when receiving the message call whatever block of code is normally used to render the window. The expose message may come with the region of the screen that needs to be re-drawn, in this case a really good application would only update that section of the display.
All in all, I'm trying to programmatically -and externally- control the Minecraft player's orientation.
No APIs, no Java mods to the game environment
Typically this requires the movement of the mouse, but every single mouse movement simulating python3 library that I've tried doesn't move the player's head in-game. Each library does something different, too.
For example, pyautogui doesn't do anything until you move the mouse manually after the script has finished. Doing this will jerk the player's view to where the program supposedly moved it to, before continuing to follow your current mouse movements. This happens for both mouse commands.
import pyautogui
pyautogui.moveTo(500, 500)
pyautogui.moveRel(100, 100)
The pynput library had the same weird result as pyautogui:
from pynput.mouse import Controller
mouse = Controller()
mouse.position = (100, 200)
mouse.move(200, -100)
Quartz doesn't do anything at all:
import Quartz
class Mouse():
down = [Quartz.kCGEventLeftMouseDown, Quartz.kCGEventRightMouseDown, Quartz.kCGEventOtherMouseDown]
up = [Quartz.kCGEventLeftMouseUp, Quartz.kCGEventRightMouseUp, Quartz.kCGEventOtherMouseUp]
[LEFT, RIGHT, OTHER] = [0, 1, 2]
def click_pos(self, x, y, button=LEFT):
self.move(x, y)
self.click(button)
def to_relative(self, x, y):
curr_pos = Quartz.CGEventGetLocation( Quartz.CGEventCreate(None) )
x += curr_pos.x;
y += curr_pos.y;
mouse = Mouse()
mouse.to_relative(200, 200)
And the python mouse library is outdated: the error showed that it will only run on Darwin (I'm on macOS High Sierra). I was sad on learning this because of the description on the Github page. It says "Global event hook on all mice devices (captures events regardless of focus)". I then thought that, somehow, Minecraft was sucking up all the simulated mouse movements on it's own. Either way, I'm not using the right interface for this game, and I need something that can bypass Minecraft's interesting mouse controls to get the movement that I want.
I even tried using mouse keys (mac's mouse-moving accessibility feature that lets you control the mouse with only keys) along with pyautogui.
import pyautogui # with mouse keys on
import time
# mouse keys is an accessibility feature on mac that controls the mouse with the keyboard
print("[ALERT]: Make sure mouse keys is on! (press option 5 times if shortcut is enabled)")
pyautogui.keyDown('8') # up in mouse keys
time.sleep(5)
pyautogui.keyUp('8')
I wasn't particularly surprised that the last one didn't work, but I think I'm running out of ways to try and bypass whatever is making Minecraft not take my python-mouse input. At this point, I'm pretty sure that there must be some distinction in the kind of input that I'm giving the computer. Minecraft as a program doesn't use the mouse like other programs do, and python mice don't control the mouse like other sources do, so there is a disconnect.
I'm on my macOS High Sierra running Minecraft in both fullscreen and windowed mode, trying everything I can to get this to function properly. I'll start the test script (python 3.6) in PyCharm, change windows (or window focus) to Minecraft (with adequate delay time in-program), and then witness what happens. Mouse clicking, keyboard presses, and even hotkeys that involve the command and escape keys all work fine in Minecraft with pyautogui, so I'm not worried about those at all. It's literally just the mouse movement that's not doing anything.
First of all, is this the right place to ask this question? Is there anything else to try, or is there something crucial that I'm missing, that would allow my mouse input to be responded to correctly?
I am trying to do the same thing, and I got mine to move the view in Minecraft (Java edition).
What worked for me was to use pynput with relative mouse commands. It also needed 'Raw Input' to be off in the Minecraft settings. [Esc -> Options... -> Controls... -> Mouse Settings... -> Raw input: OFF]
import pynput
mouse = pynput.mouse.Controller()
mouse.move(10, 10)
Also, here's the beginnings to a smooth movement of the mouse if anyone wants it:
def move_smooth(xm, ym, t):
for i in range(t):
if i < t/2:
h = i
else:
h = t - i
mouse.move(h*xm, h*ym)
time.sleep(1/60)
move_smooth(2, 2, 40)
Now, onto trying to make the keyboard work :P
I managed to make it work with the mouse library. Instead of using mouse.move(x,y,absolute,duration) I used mouse._os_mouse.move_to(x,y) and mouse._os_mouse.move_relative(x,y). Take into account that if you want a smooth effect you'll have to implement it yourself using something like time.sleep(s).
This question has also bothered me for a while and I finally found a solution to this question.(macOS Monterey 12.2, M1 chip 2020)
I have tried multiple python library and macro apps which none worked and rawinput also didn't help. However, pointer control in system preferences works with Minecraft mouse movement. The mouse keys option allows you to control mouse movement with keystrokes,
Where to find Mouse Keys in system preferences
System Preferences -> Accessibility -> Pointer control(under Motor) -> Enable Mouse Keys
so I tried to simulate the keyboard to move the mouse. For some reason, none of the python library or macro apps will work with mouse keys except for apple script. So I wrote an apple script and embedded it in python, which worked moving the mouse in Minecraft when the mouse key option is on.
The mouse keys option will allow you to use the numpad keys to control you mouse.
https://eastmanreference.com/complete-list-of-applescript-key-codes
This link will tell you the key code for the keystroke you want to press.
import os
cmd=""" osascript -e '
repeat 500 times
tell application "System Events" to key code 88 --right
end repeat'
"""
os.system(cmd)
In the script key code 88 matches numpad key 6, which will move your cursor right, change the key code to customize where the cursor will move.
I am in a similar situation to you. I was also unable to find a way to register my mouse movements in games such as minecraft.
However, I learned that you can use Java and the built-in robot library to achieve the mouse movement as desired. I don't know if you are set on python but if not it's worth checking out.
In the Minecraft Options... go to Controls... go to Mouse Settings... and there turn off Raw Input.
That should do it (you're on mac tho and I don't know if this setting is on
the mac version of minecraft)
I was struggling with this too and I found the following findings (tested on Ubuntu 20.04):
If you activate RawInput, you can use mouse.position=(dx, dy) which (although it is intended to be absolute) causes relative motion of the head! And it also works with negative values for dx and dy.
This is much faster and more accurate. You can for instance chain
mouse.position = (0, -100000) # turn head all the way up
mouse.position = (0, 600) # turn head exactly to horizon level
The horizon is 600 "pitch-pixels" down from upright, when mouse sensitivity is set to 100%. You don't need any delay between the calls and they happen so fast that in the vast majority of cases, the player does not seem to look up in between, but looks to the horizon right away. When I used the relative motion with raw input off, I needed sleeps in between and the speed was also capped at some value.
The only issue is, that you need to switch between this hacky way of controlling the mouse and the normal way using mouse.move(dx, dy) or with actually absolute screen coordinates mouse.position=(x, y) when you are on the crafting screen / in a menu. But I found that if you also create a pynput mouse listener, you will see that Minecraft sets the mouse position to the center of its window everytime you enter a menu and it does it another two times when you go back to the game. So I use these as triggers.
Try running python as admin and run the game in windowed mode. Pyautogui should work then.
When the pygame window closes, I want to immediately stop execution; that's the end of the program. I have looked up and down the internet and every problem I come across is something like "Did you try pygame.quit()" or "You never break out of your infinite loop". That isn't my problem. Here's my code:
#!/usr/bin/env python3
import pygame
from sys import exit as sysExit
pygame.init()
screen = pygame.display.set_mode((512,512))
screen.fill((255,255,255))
pygame.display.update()
while True:
for event in pygame.event.get():
if event.type == pygame.QUIT:
pygame.display.update()
pygame.display.quit()
pygame.quit()
exit(0)
sysExit(0)
#None of these work, code under them is unreachable but the process hangs
As the comment and question title say, this process hangs. I honestly don't what else I could do to tell python I'm done, and I've tried every combination of these quit functions but nothing works. The window will close, but the program never ends. In my searching I've found people have issues with something called "IDLE" which I do not have.
Here's what I do have (my system): Linux Mint Cinnamon 18, 32-bit (i686), using python version 3.5.2 (no IDE, just running from terminal)
EDIT - Apparently there's some confusion about what I'm talking about when I say the program "hangs", so here's some screenshots of the issue in action. This first one shows you that the above code is, in fact, usable to reproduce the problem.
Now all I do is run this script, and it just gives me a blank window like you'd expect.
Now I click the "x" button on the window, and the window disappears, but my terminal is left like this. The screen capture won't show you the blinking cursor, but you'll notice there's no new prompt.
I've so far been resorting to doing this to get the thing to stop:
throw it in the background, force kill the job (SIGTERM gets ignored btw) and foreground it again and now I have a prompt.
check pygame.init() -> (numpass, numfails), then loading each module manually (yes, each module has an init() and quit() method). I would suspect the mixed or music module to be the problem (due to os/driver/hardware issues).
The pygame irc channel helped me figure this out. Specifically, this answer comes from user DR0ID_ who walked me through some debugging I should've thought of in the first place.
First of all, once I'm done with this answer I'll edit the question, but I have to apologize for leaving something out initially: I'm doing all this on a Virtualbox VM. By walking through the different modules initialized by pygame.init(), I discovered that the one that actually hangs it is pygame.mixer. No word yet from the pygame devs, but the working theory is that this is related to Virtualbox sound drivers and/or their interface with the host drivers. Basically, if you're on a virtual machine and don't need to play sounds, the workaround is to only initialize the modules you need. Since in my example I only need pygame.display, the solution is easy:
#!/usr/bin/env python3
import pygame
pygame.display.init()
screen = pygame.display.set_mode((512, 512))
screen.fill((255, 255, 255))
pygame.display.update()
while True:
for event in pygame.event.get():
if event.type == pygame.QUIT:
pygame.display.quit()
exit()
I am just on my first steps using pygame on python 3.5 on mac osx 10.11.1. I think i have properly installed pygame because when i run
import pygame
it accepts it. I am running some tests on basic pygame use and i have a problem with pygame.display.set_mode
This is the code:
import pygame
pygame.init()
screen = pygame.display
screen.set_mode((720,480))
it runs ok without any bugs but the pygame screen that opens (different from the IDLE screen) it freezes. The cursor becomes this spinning rainbow thing.
Excuse me if this is a really dumb question but i'm really new at this and i have been searching all day and can't find something similar.
Thank you for your time
What you need to write is an event loop and a quit event, as the user cannot close the window.
For the event loop I would do something like:
running = True
while running: # This would start the event loop
for event in pygame.event.get():
if event.type == pygame.QUIT: # This would be a quit event.
running = False # So the user can close the program
screen.fill(0,0,0) # This fills the screen with black colour.
pygame.display.flip() # This "flips" the display so that it shows something
pygame.quit()
I hope this helps! The event loop is just something that keeps the program running, without it the code would look quite confusing so it's best to have one.
Is there anyway to use pygame to get input from the console, rather than having to display a separate window to get input? I'm using pygame to track how long the keys on the keyboard are pressed.
The following code doesn't work (this is just a minimal example, it doesn't actually keep track of time elapsed):
pygame.init()
while 1:
for event in pygame.event.get():
if event.type == pygame.KEYDOWN:
print event.key, 'pressed'
It doesn't look like any pygame event is being raised. If I add
screen = pygame.display.set_mode((640, 480))
After pygame.init()
then the event is raised, but I have that ghastly window I don't want to deal with.
To explain why I don't want the window, I envision this application being a command-line utility, so I can't have that. Is there any functional reason preventing pygame from running on the command-line?
Thanks!
EDIT: I speculated that the problem was pygame.init(), and that I only needed to initialize the key and event modules. According to http://www.pygame.org/docs/tut/ImportInit.html I should have been able to call pygame.key.init()pygame.event.init() but it didn't work.
Pygame is designed for making (graphical) games, so it only captures key presses when there is a window displayed. As Ignacio said in his answer, reading from the command line and from another window are very different things.
If you want to create a command line application, try curses:
http://docs.python.org/library/curses.html
Unfortunately, it only works on Linux and Mac OS X.
Console input comes in via stdin, which pygame is not prepared to handle. There is no reliable way to get press/release events via stdin since it's dependent on the terminal sending it keypresses.
If you just make the window really small using
screen = pygame.display.set_mode((1, 1))
you can't see it. So you are clicked into the window but you don't notice.
If you click anywhere of course it stops working. You have to click on the pygame window icon for it to work again.
If you just plain don't want a window of any kind at all, you can use PyHook. If you just want a console application, get user input with the built-in Python command "raw_input(...)".
Try pygame.display.iconify().
This will hide the pygame screen, and you will still be able to detect keypresses.