Develop Reference

Clicking NEXT to Images

I have had some success automating some work navigating a website lately, but I've had a LOT of difficulty trying to get it to work across multiple systems.
Currently, I am using PyAutoGUI to locate an image of text, and then telling Python to click to the RIGHT of that image in order to select the desired text box. Unfortunately, this isn't panning out to other systems even with lowered confidence intervals for the locateOnScreen() method.
I tried looking into Selenium to locate the text, but unfortunately I would have to run this on an existing IE explorer which is currently unsupported.
I was thinking about using OCR, but I'm unsure of how I would tell Python to click NEXT the the text that I am searching for.
Any input on possible solutions would be greatly appreciated!
-MontX
Edit:
Image of what I've been trying to get PyAutoGUI to recognize.
Example of text I've been trying to locate on screen:
Example of kind of icons I've been trying to locate on screen:

pywinauto ribbon menu returns no control identifiers

I'm trying to automate a series of tasks I have to do with a given software. I've managed to "wing it" using keyboard controls, but to extend the complexity of the script I need to access a ribbon menu.
So far, my code is as follows:
windows = pwa.Desktop(backend="win32").windows()
running_windows = [window.window_text() for window in windows]
midas_title = ""
target_title = "Civil"
for window in running_windows:
if target_title in window:
midas_title += window
else:
pass
app = pwa.Application().connect(title=midas_title)
app[midas_title].set_focus()
app.MidasGenMainFrmClass['MIDAS/Civil'].print_control_identifiers()
app.MidasGenMainFrmClass['BCGPRibbonBar:40000000:8:10003:10'].print_control_identifiers()
Judging by the top/left and bottom/right coordinates of the BCGPRibbonBar:40000000:8:10003:10, that seems to be the menu that I want to access so that I can click on the "Results" button in this ribbon (my final goal), but when I do print_control_identifiers() on that the output I get is this:
BCGPRibbonBar:40000000:8:10003:10 - 'MIDAS/Civil' (L0, T0, R1920, B174)
['MIDAS/CivilBCGPRibbonBar:40000000:8:10003:10', 'BCGPRibbonBar:40000000:8:10003:10', 'MIDAS/Civil']
child_window(title="MIDAS/Civil", class_name="BCGPRibbonBar:40000000:8:10003:10")
I'm very new to this, but seems like the ribbon has no children?
Seems like I'm too new to embed images, but link here to what the ribbon menu looks like
And finally, link to the image of the inspector of the "Results" menu where I'd like to click
Hopefully this is sufficient information, but please let me know if anything is missing.
If anyone has any idea of what I might be doing wrong, that would be extremely helpful.

Use backend="uia" instead of backend="win32". Ribbon controls are visible to UIA backend only. Application(backend="uia") is also a must.

Selenium (python) switching to popup panel

I am trying to use Selenium with Python to click on a text field, which opens a pop-up panel, select the text entry area of that popup, and enter text to it.
switch_to_window and switch_to_frame don't seem to be working. In a previous question I asked about Selenium, someone told me to pause the program until the element I need is available. The solution worked for that problem, but not this one, so I'm assuming I have a different issue and I'm too new to Selenium to understand what it is.
This is what the original box I'm trying to click on looks like:
And the Inspect Element for this box:
When that description box is clicked, it should open this window:
And select this element to enter text into:
So in my code I have:
descriptionBox = driver.find_element_by_id('kiadvany_fulszoveg_text')
descriptionBox.click()
That does not error the program, but it also doesn't seem to actually be clicking on that element. To make matters more confusing, I got this to work exactly ONCE, where it opened the correct Description text box as pictured above, but it has since not worked at all even when I try the exact same thing.
The panel's ID is:
As I mentioned, switching to this panel ID using switch_to_frame or switch_to_window was the first thing I tried, but I'm getting a No Such Element error.
Because I saw the description box open correctly once, but never again, I'm assuming that's where the problem is. I wish, the one time it did pop up, that I'd tried to put the text into the field to see if that would work too, but I hadn't gotten there yet at that point, so I don't know if that would have worked.
Thank you in advance to anyone who can help with this!

Try this
descriptionBox = driver.find_element_by_id('kiadvany_fulszoveg_text')
driver.execute_script('arguments[0].click();', descriptionBox)
or
actions = ActionChains(driver)
actions.move_to_element(descriptionBox)
actions.click(descriptionBox)
actions.perform()

Mouse the focus to an object using selenium python

I am automating a website using selenium RC and python 2.7 on Ubuntu Linux. Here is what I need to do:
Go to the site http://borro.com.
Scroll down to the bottom of the page using key down native command
I need to hover the mouse on g +1
read the tool tip
click on the name that appears in the tool tip.
The problem I am having is -- I need the mouse to physically move there, wait for say 2 secs and then read the tool tip and click on the name
The mouse is not physically moving there and I think the focus is lost and it says element xpath not found.

We've solved a lot of our focus issues by sending a blank key to the element so it gets focused. In this case, you'd probably want to send blank key to the tooltip as soon as it appears. I'm familiar with webdriver but not RC, but RC should have something like send_key(element_xpath, " ") as well.
To get the tooltip's xpath, you can use firebug, and in the console, use something like
$x("//*[contains(text(), 'Publicly recommend this as')]")
to make sure this element is found and xpath is correct. I also recommend not using wildcard characters, so once you find the tooltip's xpath, try to replace the * by the actual element type.

Button Xpath and on hover on button xpath are given below
Actions builder = new Actions(driver);
WebElement tagElement = driver.findElement(By.id("button"));
builder.moveToElement(tagElement).build().perform();
/html/body/div/div/table/tbody/tr/td/div
Try:
selenium.mouseOver("mylocator");

Selenium: Testing pop-up windows

I have an issue when trying to test a web application with Selenium/Python. Basically I can't test elements of a pop-up window.
A scenario: I can test all elements for a page. But when I go to click on a button that opens up a small pop up box I can't test the elements on the popup. It's like the pop up isn't in focus or active.
I can test elements on the next page. For example click a button, brings me on to next page, and I can work with elements on the 'next' page. So it the problem seems to be popup specific.
I could post code but to be honest it might confuse at this stage. I may post code in a later post, thanks

There is a property called switch_to
Q: How do I handle pop up windows?
A: WebDriver offers the ability to cope with multiple windows. This is done by using the WebDriver.switch_to.window(knownName) method to switch to a window with a known name.
If the name is not known, you can use WebDriver.window_handles to obtain a list of known windows.
You may pass the handle to switch_to.window(handleName)
For example I used driverName.switchTo.window(driverName.getWindowHandle()) to get a hold of popups for which I didn't want to look for names.
Additional references:
http://code.google.com/p/selenium/wiki/FrequentlyAskedQuestions

For the Selenium RC API, you need to use the SelectWindow command to switch to the pop-up window. The window can be specified either by its name (as specified on the JavaScript window.open() function) or its title. To switch back to the main window, use SelectWindow(None).