Finding position of viewport in Selenium Python? - python

I need to find the position of the top left corner of the viewport in Selenium. I know we can use drive_get_window_position() to get the window position, but I need just the position of the part that renders. Is that possible?
The use case is that I am attempting to use Selenium to get the coordinates of an element, and need to move the cursor to it. Since Selenium only moves the virtual mouse and not the visible cursor, I am using pyautogui to move the cursor to absolute coordinates.
However because the browser position affects where the absolute coordinates should be, I need to know where the viewport is so I can offset appropriately. Using the get_window_position() works for x coordinates but not y coordinates due to the various menus and toolbars.


location_once_scrolled_into_view¶
THIS PROPERTY MAY CHANGE WITHOUT WARNING. Use this to discover where
on the screen an element is so that we can click it. This method
should cause the element to be scrolled into view.
https://www.selenium.dev/selenium/docs/api/py/webdriver_remote/selenium.webdriver.remote.webelement.html#module-selenium.webdriver.remote.webelement
Returns the top lefthand corner location on the screen, or None if the element is not visible.
Eg:
e = driver.find_element_by_xpath('//html')
print(e.location_once_scrolled_into_view)

Related

Getting coordinate of elements on my laptop screen

I am making a fun python program to automate couple clicks for me. I want to use pyAutoGui library provided by python to make this program. I am struggled to find the coordinates of elements that I want to click. Are there any ways that I can find coordinates of my elements?

You can use pyautogui.mouseInfo().
MouseInfo is an application to display the XY position and RGB color information of the pixel currently under the mouse. Works on Python 2 and 3. This is useful for GUI automation planning.
The full documentation is at https://mouseinfo.readthedocs.io/en/latest/

Point Position (for Windows) is a simple tool that lets you pick the coordinates for any point on your screen (using X,Y axis). Simply point one of the four corner arrows at the spot on your screen that you want to define and click the button to display the X/Y coordinates.

Python - TKinter - Moving shape on canvas with a mouse

I need to find a way to code the following
1.there's a geometry object that contains an array of points that are then drawn on a canvas widget (got this covered)
2.when you left click on the canvas it checks if you clicked withing a certain margin of an existing point and if that's true a point in the array is selected (got this covered in terms of searching for the point and selecting it)
3.Once selected the point will follow the mouse until the mouse button is released.
Using the Motion event on it's own doesn't seem to work as it seems the function is called over and over while the button is pressed. So I'd need to trigger the search function when the button is pressed them the move function when the button is held.
I'd be grateful for pointers.

Thanks to Dan Getz I did the following:
-bind the point selection to select point and store index in self.selectedPoint
-bind move function to using the self.selectedPoint to indicate the selected point in the array, then passing the events x,y coords to the array as new coords for the selected point
-bind the clearSelected function to to set the self.selectedPoint to -1 thus clearing the selection
The problem I'm still having is that when moving the point I update the screen while the mouse is held which produces quite a bit of flickering. I'm wondering if there's anything I can do to prevent that.

How to get select area coordinate with python?

I would like to know if there is a simple way to get coordinates of an area selected with the mouse on screen?
Imagine, I have a small gui, clicking a "select" button then I draw a select area on my screen and it returns the top-left / bottom-right coordinate of my selected area.
And also which kind of gui should I use to be multi platform compatible?
wxPython / wkTinker, any other?
Thanks to point me on the right direction.

The first question depends on the second, but usually you have something like a mousePressed and a mouseRelease event which both provide coordinates (see here for Tkinter). A selected rectangle is defined by the coordinates of those two events.
The second question is rather subjective and also depends on what you want to do exactly. But another option would be PyQt.

Mouse click command does not work on certain GUIs

The script below works perfectly when I want to click, for example, the Start button on Windows, but when I try to click a button in a certain GUI program it does not have any effect.
Is it possible that this program has disabled virtual mouse clicks?
If so, can I circumvent this somehow?
import win32api, win32con
Po=win32api.GetCursorPos()
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,Po[0],Po[1],0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,Po[0],Po[1],0,0)

mouse_event (and SendInput, which is the preferred API to use for input) have a couple of tricky bits, it's a good idea to
read the MSDN page for mouse_event fully and carefully before using it - pay attention to the small print: in particular, the x and y values are not pixels, so you can't just put in values you get from GetCursorPos.
It just happens that 0,0 is the bottom-left corner, so points near the bottom left will be roughly in the same area of the screen, but the further away from that you get, the more the pixel values diverge from the actual units that this API uses: so it can appear to work for positions near the start button (assuming it's in the bottom left of the screen), but for other values, it may appear to be clicking somewhere else, which sounds similar to what you are seeing.
From MSDN:
dx [in]
Type: DWORD
The mouse's absolute position along the x-axis or its amount of motion since the last mouse event was generated, depending on the setting of MOUSEEVENTF_ABSOLUTE. Absolute data is specified as the mouse's actual x-coordinate; relative data is specified as the number of mickeys moved. A mickey is the amount that a mouse has to move for it to report that it has moved.
So first of all, you need the MOUTEVENTF_ABSOLUTE flag. But that's not all:
Remarks
...
If MOUSEEVENTF_ABSOLUTE value is specified, dx and dy contain normalized absolute coordinates between 0 and 65,535. The event procedure maps these coordinates onto the display surface. Coordinate (0,0) maps onto the upper-left corner of the display surface, (65535,65535) maps onto the lower-right corner.
...so you'll need to scale your target coordinates appropriately before passing them to this API.

Mouse events generated by programs have an "injected" property that an app can filter if they want - for example MMO clients often filter those to avoid bots
Those zeros in the mouse_event let you set some properties, you might want to research if those will let you overcome the injected flag, although I can't find a way immediately

below code worked for me
pyautogui.mouseDown(fromx, fromy)
time.sleep(2)
pyautogui.mouseUp(tox, toy)
pyautogui.mouseUp(1000,400)

The Tkinter Text Widget Insertion Cursor

Is it possible to get the x y coordinates of the insertion cursor in a Tkinter Text widget? I'm trying to make a popup menu that pops up next to the insertion cursor.

The bbox method can be used to get the bounding box of an index. You can use that to get the position relative to the window. You can use the winfo method to get the x/y of the window.
Typically popups appear next to the mouse rather than the insertion point (though typically, right-clicking first sets the insertion point and then displays a menu).

Categories

Resources