I'm trying to read the text from a popup window.
The title is always the same. I've managed to identify the hwnd and get the title with the code below, but I can't figure out how to read the contents.
import time
import win32gui, win32con
windows = []
def _MyCallback( hwnd, extra ):
extra.append(hwnd)
win32gui.EnumWindows(_MyCallback, windows)
while True:
window = win32gui.GetForegroundWindow()
title = win32gui.GetWindowText(window)
if title == 'Errors occurred': print 'error window'
time.sleep(1)
Here's the working version:
import time
import win32gui
while True:
window = win32gui.GetForegroundWindow()
title = win32gui.GetWindowText(window)
if title == 'Errors occurred':
control = win32gui.FindWindowEx(window, 0, "static", None)
print 'text: ', win32gui.GetWindowText(control)
time.sleep(1)
You will only be able to read this text programmatically if it is contained in a windowed control. You can easily check this with Spy++. Many GUI frameworks don't use windowed controls for their child controls, or only use windowed controls for some children.
If it is a windowed control then you can identify it by calling GetWindow() and walking the child structure (obviously you need to use the win32gui equivalent).
I don't have access to the framework or the error dialog you are using, so I can only say in general what you want.
You need the FindWindowEx function, and use it to find a control whose class name is 'static' (or whatever the class name of the control is). I imagine this would be the line:
control = win32gui.FindWindowEx(window, 0, "Static", 0)
That returns the handle to the control, and you can then use GetWindowText on that to get the text.
Related
I'm trying to listen for mouse click events on task bar app icons for currently running apps in windows using python. In the example below, I use python to initiate the internet explorer application. After the app initiates, I then use the FindWindow windows API function to get the window ID/handle.
Is there a way to listen for mouse clicks on the internet explorer task bar icon by the user using windows API or some other method? I need to print a message when the IE icon in the task bar is clicked.
Here is the code I am working with currently:
from win32gui import GetWindowText, GetForegroundWindow, FindWindow
import win32api, win32gui
import win32con
import sys, os
os.system('"C:\Program Files (x86)\Internet Explorer\iexplore.exe"') #open internet explorer application.. this will make the internet explorer icon appear in the taskbar..
window_id = FindWindow(0, 'Internet Explorer') #find the newly opened IE window's id/handle
print (window_id)
class test:
def __init__(self, **kwargs):
super(test, self).__init__(**kwargs)
self.oldWndProc = win32gui.SetWindowLong(window_id, win32con.GWL_WNDPROC, self.MyWndProc)
self.msgdict = {}
for name in dir(win32con):
if name.startswith("WM_"):
value = getattr(win32con, name)
self.msgdict[value] = name
print (self.msgdict) #I can access all the win32 window events using this dictionary..
#listener to track win32 window events related to the newly opened internet explorer window..
#how do I track when the internet explorer icon is CLICKED in the task bar??
def MyWndProc(self, hWnd, msg, wParam, lParam):
print (self.msgdict.get(msg)) #none of the window events print here..
return win32gui.CallWindowProc(self.oldWndProc, hWnd, msg, wParam, lParam)
t = test()
Try to use UI Automation.
The window class name of the taskbar is MSTaskListWClass:
And you can also see the UIA element and its BoundingRectangle from inspect.exe:
You could first get the click event on desktop and then get the task icon's IUIAutomationElement, then use GetClickablePoint to determine whether the click is on the element.
Thanks to Drake Wu pointing me in the right direction, I was able to use python to get UI Automation info about the element that is clicked and its parent. Using that information, I can tell that an app icon in the taskbar is clicked when the ControlType of the element clicked is 'ButtonControl' and the class name of its parent is 'MSTaskListWClass'. You can figure out which application was clicked by looking at the Automation ID of the clicked element.
You need to pip install uiautomation for this example. I had to add a few additional functions/methods to the uiautomation.py file because it was missing functions that were needed.
Here are the methods that I added to the uiautomation.py file to make it more useful for this example (see where it says NEW CODE HERE):
def ControlFromPoint(x: int, y: int) -> Control:
"""
Call IUIAutomation ElementFromPoint x,y. May return None if mouse is over cmd's title bar icon.
Return `Control` subclass or None.
"""
element = _AutomationClient.instance().IUIAutomation.ElementFromPoint(ctypes.wintypes.POINT(x, y))
return Control.CreateControlFromElement(element)
#NEW CODE HERE
def GetElementFromPoint(x: int, y: int) -> Control: #gets the element from a point (x, y)
element = _AutomationClient.instance().IUIAutomation.ElementFromPoint(ctypes.wintypes.POINT(x, y))
return element
#NEW CODE HERE
def GetParentElement(element): #gets the parent of an element input
parent_element = _AutomationClient.instance().ViewWalker.GetParentElement(element)
return parent_element
#NEW CODE HERE
def GetElementInfo(element): #gets the property information about an element
element_info = Control.CreateControlFromElement(element)
return element_info
I then utilized these newly added uiautomation methods in the main example below:
import sys, os
from pynput import mouse
from pynput.mouse import Controller
from pynput.mouse import Listener
import uiautomation
from uiautomation import GetElementFromPoint, GetParentElement, GetElementInfo #these are newly added methods to make this example work
mouse2 = mouse.Controller()
class test:
last_mouse_pos_x = ''
last_mouse_pos_y = ''
def __init__(self, **kwargs):
super(test, self).__init__(**kwargs)
self.mouse_listener() #start mouse listener
def mouse_listener(self):
global element_clicked
while True:
if mouse2.position[0] != test.last_mouse_pos_x or mouse2.position[1] != test.last_mouse_pos_y:
element_clicked = uiautomation.GetElementFromPoint(mouse2.position[0], mouse2.position[1]) #returns the element that you clicked (this has to run here or it errors)
test.last_mouse_pos_x = mouse2.position[0]
test.last_mouse_pos_y = mouse2.position[1]
def on_click(x, y, button, pressed):
if pressed:
try:
element_clicked_info = GetElementInfo(element_clicked) #returns info about the element you clicked
parent_of_element_clicked = uiautomation.GetParentElement(element_clicked) #returns the parent of the element that you clicked
parent_of_element_clicked_info = GetElementInfo(parent_of_element_clicked) #returns info about the parent of the element you clicked
#print info about the element you clicked and its parent
print ("ELEMENT CLICKED INFO: " + str(element_clicked_info))
print ("PARENT ELEMENT INFO: " + str(parent_of_element_clicked_info))
except:
pass #skip over any errors
#ANALYSIS: If ControlType of element clicked = 'ButtonControl' AND class name of its parent = 'MSTaskListWClass', then an app icon in the taskbar was clicked!
listener = mouse.Listener(on_click=on_click)
listener.start()
t = test()
Here is my sample code. How do I get the html source code of the current page. It only prints 'GString at 0x8875130' . How to convert it to real text contains html?
from gi.repository import WebKit
from gi.repository import Gtk, Gdk
def get_source(webobj, frame):
print "loading..."
x = web.get_main_frame().get_data_source().get_data()
print x
win = Gtk.Window()
web = WebKit.WebView()
web.open("http://google.com")
web.connect("load-finished", get_source)
win.add(web)
win.show_all()
Gtk.main()
print x.str
Data is available as .str member of GLib.String object. For further details try help(GLib.String) on python prompt after importing libraries.
#Before you can use the require_version() method from gi, you need to import the gi module.
import gi
#Specify versions to import from the repository.
gi.require_version('Gtk','3.0')
gi.require_version('WebKit','3.0')
#Import the modules that will give us a Graphical User Interface (GUI) and a WebKit Browser.
from gi.repository import Gtk,WebKit
#Define your function to handle the WebKit's "load-finished" event. The webobj is a reference to the WebKit that triggered the event. The frame is which frame triggered the event (useful if the loaded page has multiple frames like a frameset.
def ShowSource(webobj,frame):
#What you have printed is what results from this line. This line returns a reference to an object, so when you print it's return value, a description is all Python knows to print.
SourceCodeStringObject=frame.get_data_source().get_data()
#You can get the text the object is carrying from it's "str" member property like I do below.
SourceCodeStringText=SourceCodeStringObject.str
#Send the source code string text to the output stream.
print(SourceCodeStringText)
#Create Window object.
Window=Gtk.Window()
#Set the text to display in the window's caption.
Window.set_title("Test of Python GTK and WebKit")
#Set the starting window size in pixels.
Window.set_default_size(480,320)
#Create the WebView object.
WebBrowser=WebKit.WebView()
#Tell the WebView object to load a website.
WebBrowser.open("https://stackoverflow.com/questions/24119290/pygtk-webkit-get-source-html")
#Set the event handler for the WebView's "load-finished" event to the function we have above.
WebBrowser.connect("load-finished",ShowSource)
#Add the WebView to the window.
Window.add(WebBrowser)
#Set the handler of the window closing to cause GTK to exit. Without this, GTK will hang when it quits, because it's main loop that we start later will still be running. Gtk.main_quit will stop the main loop for GTK.
Window.connect("delete-event",Gtk.main_quit)
#Display the window.
Window.show_all()
#Start GTK's main loop.
Gtk.main()
This way works for me.
#!/usr/bin/env python
import webkit, gtk
def get_source(webobj, frame):
print "loading..."
x = web.get_main_frame().get_data_source().get_data()
print x
win = gtk.Window()
win.set_position(gtk.WIN_POS_CENTER_ALWAYS)
win.resize(1024,768)
win.connect('destroy', lambda w: gtk.main_quit())
win.set_title('Titulo')
vbox = gtk.VBox(spacing=5)
vbox.set_border_width(5)
web = webkit.WebView()
vbox.pack_start(web, fill=True, expand=True)
web = webkit.WebView()
web.open("http://www.google.co.ve")
web.connect("load-finished", get_source)
browser_settings = web.get_settings()
browser_settings.set_property('user-agent', 'Mozilla/5.0 (X11; Linux i586; rv:31.0) Gecko/20100101 Firefox/31.0')
browser_settings.set_property('enable-default-context-menu', True)
browser_settings.set_property('enable-accelerated-compositing', True)
browser_settings.set_property('enable-file-access-from-file-uris', True)
web.set_settings(browser_settings)
win.add(web)
win.show_all()
gtk.main()
I'm trying to find the whole text that is currently being edited in gedit window. Firstly i tried to find out the current gedit tab that is focused, by using Xlib.display. Now i got an Xlib.display.window object . Now i want to find out the text that is in that particular window/widget using this window object
And my code is like this
import gtk, gobject, Xlib.display
currentFocus=''
def current_focused:
global currentFocus
display = Xlib.display.Display()
window = display.get_input_focus().focus
wmname = window.get_wm_name()
wmclass = window.get_wm_class()
if wmclass is None and wmname is None:
window = window.query_tree().parent
wmname = window.get_wm_name()
if currentFocused!=wmname:
if window.get_wm_class()[0]=='gedit':
print "\nNow you are in : < %s >" % (wmname,)
# Here i have to find the text of the gedit's widget
currentFocused=wmname
return True
gobject.timeout_add(1000, current_focused)
gtk.main()
is there any API to get the text of a specific widget using Xlib.display.window
Please help me. I'm completely new in this area
Thank you
WM_CLASS / WM_NAME properties are only used to provide information to a window manager ( hence WM prefix ) and not usually set on child windows. Check GTK source code if edit widget sets similar properties, but in general it is not possible for external process to read edit control text
I would like to automate the upload of a file to a website using PyQt4's QWebView, but there's a part I can't figure out yet. To upload the file, the website has a button, which opens a dialog from which you are supposed to select the local file. So, these are my questions :) Is there a way to control that dialog once I click the button? Is there a better way to achieve this?
edit
The website is https://maps.google.com/ and I'm uploading a .kml file through My Places > Create Map > Import.
It is possible what you're looking for is QWebPage::chooseFile() (I suppose it depends also on how the website is handling that). Reimplement that and see if it is sufficient. Do whatever you want and return the chosen file path.
EDIT: Now that you provided the link I tested and seems to work.
Ok, to begin let me start with background information and references.
The module that I will be using is pywin32 download here, specifically the win32gui, API reference here.
Now before you can manipulate the dialog you have to "navigate" to the window handle, the following uses win32.FindWindow API Reference here, which looks like this, where the two inputs are the lpclassName in this case #32770 (stands for a dialog) reference here and the lpWindowName which in this case is File Upload,
HWND WINAPI FindWindow(
_In_opt_ LPCTSTR lpClassName,
_In_opt_ LPCTSTR lpWindowName
);
Code to locate file handle:
import win32gui
control = win32gui.FindWindow("#32770", "File Upload")
And it stores the handle, which in my case was 721470.
The next step is locate the handles of the GUI objects in the dialog, i will show an example of the Cancel button. To find the handle, I wil be using FindWindowEx API reference here,
import win32con
import win32api
ButtonHandle = win32gui.FindWindowEx(control, 0, "Button", "Cancel");
win32api.SendMessage(ButtonHandle, win32con.BM_CLICK, 0, 0)
Reference here for the BM_CLICK and here for the SendMessage.
Final code:
import win32gui
import win32api
import win32con
window = win32gui.GetForegroundWindow()
title = win32gui.GetWindowText(window)
control = win32gui.FindWindow("#32770", "File Upload")
ButtonHandle = win32gui.FindWindowEx(control, 0, "Button", "Cancel")
win32api.SendMessage(ButtonHandle, win32con.BM_CLICK, 0, 0)
Another way is to use the watsup.winGuiAuto module, here, example below:
from watsup.winGuiAuto import *
optDialog = findTopWindow(wantedText="File Upload")
CancelButton = findControl(optDialog,wantedClass="Button", wantedText="Cancel")
clickButton(SaveButton)
But i believe the easiest way is to use autoit here, i have used it before in pyqt, to shoot out commands.
Hope this helps!
Additional References (pywin32 versions):
win32gui here
win32api here
Here's a pure PyQt4 demo that more or less reproduces the default implementation:
import sys
from PyQt4 import QtCore, QtGui, QtWebKit
class WebPage(QtWebKit.QWebPage):
def chooseFile(self, frame=None, path=''):
return QtGui.QFileDialog.getOpenFileName(self.parent(), '', path)
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
view = QtWebKit.QWebView()
view.setPage(WebPage(view))
view.load(QtCore.QUrl('https://maps.google.com/'))
view.show()
sys.exit(app.exec_())
Just as some history, I have been using python for about 5 years now and have finally decided to make my first gui app in Glade.
I started with something basic, I have a button, a Gtkentry and gtktextview
This is what I am trying to accomplish:
on button press, take from the text from gtk.entry and have it appended to the gtk.textview
now the main problem I have is that I can not find descent documentation for how to use the widgets, and the examples I find on the Internet reference both a builder variation as well as another variation of glade project which I can only assume has been discontinued. I would like to learn how builder fits into the python / glade collaboration.
my code so far:
import gtk
import pygtk
def onDeleteWindow(self, *args):
Gtk.main_quit(*args)
def hello(button):
text_buffer.set_text(txtinput.get_text())
builder = gtk.Builder()
builder.add_from_file("dagui.glade")
handlers = {
"onDeleteWindow": gtk.main_quit,
"buttondown": hello
}
builder.connect_signals(handlers)
textarea = builder.get_object("textview1")
window = builder.get_object("window1")
txtinput = builder.get_object("entry1")
window.show_all()
gtk.main()
window.show_all()
gtk.main()
now this all works and pressing the button will print what ever is in the gtk.entry but I can not find how to append it to the textview. I also am not sure what to search for to find documentation, I tried "gtk builder gtk.textview" and pygtk build gtk.textview append" and all other variations.
Though knowing how to simply add the text to the text view would be great, having a link to somewhere where I can get in plain english how to use these widgets I would be forever great-full.
Frob the gtk.TextView, you need to get the gtk.TextBuffer by using the textview's buffer property.
From the textbuffer, you need to get the iterator that points to the end of the buffer with the get_end_iter method. With that iterator, and your text, you can use the textbuffer's insert method.
Edit: Since I don't have the dagui.glade file, I couldn't test it, but see the following code:
def hello(button):
global textarea, txtinput
buffer = textarea.get_property('buffer')
i = buffer.get_end_iter()
buffer.insert(i, txtinput.get_text())
# clear the input window after appending the text
txtinput.set_text('')
I figured it out, I have found out the the gtk.textview.get_buffer actually sets the buffer ID and then the textview.set_text(buffer) isall I needed.
here is the full working code, the glade is just a button, an entry and a textview:
#!/usr/bin/python
import os
import sys
import gtk
import pygtk
def onDeleteWindow(self, *args):
Gtk.main_quit(*args)
def hello(button):
textbuffer = textarea.get_buffer()
textbuffer.set_text(txtinput.get_text())
builder = gtk.Builder()
builder.add_from_file("dagui.glade")
handlers = {
"onDeleteWindow": gtk.main_quit,
"buttondown": hello
}
builder.connect_signals(handlers)
textarea = builder.get_object("textview1")
window = builder.get_object("window1")
txtinput = builder.get_object("entry1")
window.show_all()
gtk.main()
Use this to add text :
textarea.set_text('whatever you want')
and this for adding pango markup ( http://goo.gl/94Pkk ) :
textarea.set_markup('<span size="large>Example</span>')
Here's the documentation : http://python-gtk-3-tutorial.readthedocs.org/en/latest/index.html