Interacting with flash with selenium webdriver Python - python

I am trying to build a tool for automatic scraping of information from a website that uses flash along with Oracle BI tool. The information is presented as we select things and flash is used to render the images.
Since I cannot scrap information from the images themselves (to my current knowledge), I found out that right clicking on the image, open a menu with "switch to table" and clicking again on that, opens a page with the data on a table scrapable from the HTML tree. My problem is that clicking on flash object (image), like:
from selenium.webdriver.support import expected_conditions as EC
elem = WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.XPATH, '(//embed[#type="application/x-shockwave-flash"])[1]')))
menu=ActionChains(browser).move_to_element_with_offset(elem, 61, 61).click().perform()
But then I try to click on the “switch to table” option but I am not being able to do so, since I believe selenium is still clicking on the image below the menu. What I have tried (with different values for offset and using elem and menu elements with no success):
ActionChains(browser).move_to_element_with_offset(elem, 75, 75).click()
There is any way to do this?
The website is: http://www.adrreports.eu/
Steps to get to the image:
Click language -> search for a report -> pick a letter and then a medicine (new tab appears)-> selecting tab named “Number of individual cases for a selected Reaction”(4th on right top) and then try to get the info from the images on the left
Thank you,

well this is awkward.
my idea was pretty much usable, even though is not very elegant.
With the offset:
menu=ActionChains(browser).move_to_element_with_offset(elem, 61, 61).click().perform()
ActionChains(browser).move_to_element_with_offset(elem, 66, 76).click().perform()
I was able to click on the switch table on the flash menu and make the new page appear with the values.
The main problem was to let the page load, and it seems like it needs quite some time.
time.sleep(10)
time sleep made the trick!
This method work, but if anyone has a better idea for extracting information from a flash image or file, I would love to hear!
Thank you.

Related

How to interact with a separate pop-up window/website with Selenium / Chrome Driver in Python?

I am trying to automate a process with Selenium, and am having troubles figuring out how to switch between open windows while the program is running.
After clicking on the button, it opens another website that has a separate url, which is unique each time it is opened. I need to switch Selenium from interacting with the original website to this new popup within the browser, caused by the original website. The new window shows that it is also controlled by Chromedriver with the bit at the top that says "Chrome is being controlled by automated test software." Additionally, the actual website opened will be the same, just the fine print after the '.com/' is different.
How would I go about doing this? Also, how would I switch back? (If this is even possible)
For example:
driver=webdriver.Chrome(service=s)
driver.get("https://originalwebsite.com/")
driver.find_element(By.XPATH, 'buttons-xpath').click()
# (popup opens up now)
# *switch to popup website here*
driver.find_element(By.XPATH, 'button-on-new-website-xpath').click()
driver.find_element(By.XPATH, 'second-button-on-new-website-xpath').click()
# *popup website closes*
# switch back to original website / window
Thanks!
I have tried to use driver.navigate in a variety of ways but generally have no clue what I am doing. Thanks again!
The comment from ALex Break led to the answer.
All I had to do was:
handles = driver.window_handles
driver.switch_to.window(handles[x])
#handles[x] is the index of the list handles that has the handle I want to switch
#to stored in it

Selenium unable to automate click on web element

I should start by saying that I'm not a web developer but I have been using Selenium with Python for a few weeks and think I've gotten the basics down.
What I have is a page with a weekly calendar on it. Image as follows:
What happens is that you can click on any of the coloured boxes which will bring up a register for a class. It features items that you can click on which bring up new information. The problem I have is that I can't click on the items - or, rather, I don't know how to automate the click. I have automated clicking with Selenium before in other situations.
That looks as follows:
As you can see, the calendar still appears in the background. That is to say, the item we click on doesn't take us to a new page but, I suppose, runs some kind of method which shows the register and populates it with data.
My problem is this: I want to automate the process so that I can click on each of these items then scrape the information. In order to do that I need to be able to automate the clicking for each of the items.
So what have I done? When I've done this before, I've searched through the web html for the relevant part and then grabbed the xPath to the element I needed. But here I can't do that. Why not? Well, firstly, I just can't find that element!
Take a look at this close up of the first column:
It's divided into columns, but then I'd expect the clickable area to be an element within that. As you can see, it's not. Furthermore, the clickable area is just the coloured box itself, but if you look closely you can see the element goes outside of that area. I have gone very close with my mouse cursor to see exactly what's clickable, and it definitely is just the coloured box.
So I've not been able to get the element at all.
I thought I might be able to just find out where we went after I clicked the button, but when I got the link address, it just said it was the same page with no differences.
I appreciate I'm asking quite a broad question here, but the problem is that I don't really know where to start. If someone could give me at least that much, I would be grateful. Like if I could just click on each of these one at a time... I've found where the populated data is so I could grab that without a problem.
Well, here's to hoping.
Edit: I should add that there are some JavaScript items (tag type script, type='text/javascript'). I presume that the answer is in there somewhere, but there is a lot of Javascript and I'm not adept at reading it. It's hard for me to tell what script does what. If I could at least figure out what script runs when I click the item then I think I'd be onto something, but I have no idea. Even that would help me.
I had encountered similar problem when scraping for Instagram followers in mobile view where it was floating box when showing the accounts followers name. The approach I took was identifying the name of floating dialog box and clicking the element in it. It might different in your case of html.
Trying looking at this link. Selenium Scroll inside of popup div
Hard to say without the HTML. Maybe try Katalon Recorder (chrome extension) and see if that can detect the xpath for you? It might also be you have to use some kind of javascript to invoke the method for the element

how can I get the value in this page in python selenium

I am trying to get the value from webpage using selenium base on python. However, I do not get the point about how to make it. The value I want to catch is in the red mark side of below picture. Please help me to figure out this. Thank you very much!
Please click here to check the picture
There are few ways (By css,by xpath...) I recommend you reading this page: http://selenium-python.readthedocs.io/locating-elements.html
You also need some understanding about html, css and the DOM but the simplest way to get the value you want is by xpath (But its the slowest)
Open your web browser lets say chrome, right click on the element you want and inspect. It will open developer tools and you will have the html line selected already, right click it again-> copy -> copy xpath
from selenium import webdriver
driver = webdriver.Chrome('/path/to/chromedriver')
driver.get("http://www.web.com")
driver.find_element_by_xpath(the_xpath_you_copied_before_from_dev_tools).text

Find CSS selector for popup menu item for Selenium in Python 3

I'm attempting to make a DeviantART Llamabot for a friend as my first Selenium project with Python 3. I have the bot 99% working except for being able to find the "give llama" button.
The problem seems to be that the menu the button appears under is a popup and I can't just right click and select "copy css selector" in Firefox. As soon as I inspect the element for the give menu the menu closes and the html changes.
I've managed to take a screen shot of a random sample page of the code so I can even see what's there. I've tried learning how CSS selectors work from scratch myself and I've managed to find every nested element EXCEPT the actual items in the list. I've tried looking for Nth child and using the ">" operator. I've attempted searching by class name, name, Xpath, link name, partial link name, nothing has worked.
I've read about this problem inspecting popup elements elsewhere and the suggestions are effectively to write an HTML parser or something to copy the entire html code as it changes and then select it from your copy. I'm not going to do that for this project. It's entirely too much work unless I absolutely have to for some reason.
Honestly at this point I don't even care anymore and I just want someone to outright just tell me what to type in so I can finish this project. This is the screenshot I managed to get. I'm looking for the item highlighted in blue.
Since my code was requested for clarification here it is
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.action_chains import ActionChains
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
Deviant = browser.get("http://www.deviantart.com/random/deviant")
GiveMenu = browser.find_element_by_css_selector(".i47")
GiveMenu.click()
#GiveLlama = browser.find_element_by_css_selector("")`
Everything except the last line works which is why the last line is commented out until I can figure out what to put in there. No matter what I've tried, including the examples provided by the people answering this question so far, I either get a no such element error or an illegal syntax error.
You can use the xpath selector, in chrome you can use rigth click and click on copy > copy Xpath, or the element selected on your inspector is "div.popup2 .blockmenu a.f.givellama"
You should click the "Give" button first, when the pop up dialog opens, inspect the "Give a Llama Badge" element and identify the xpath. Here is a screen shot.
screen shot to locate "Give a Llama Badge" button
You can find that, the xpath to locate "Give a Llama Badge" button(Should click the Give button first to locate this element).
//a[#class='f givellama']
The xpath to locate "Give button"
//a[#href='#give-give-give']/span[text()='Give']
As you pointed out, when the popup dialog opens, if you try to test the xpath in the browser console via "Firepath" or others, when you type "Enter" or click the left mouse, the popup windows is closed. But don't worry about this, since you already identified the xpath locator, you can debug it in your scripts.

How to control Webpage dialog with python

When I try to automatically download a file from some webpage using Python,
I get Webpage Dialog window (I use IE). The window has two buttons, such as 'Continue' and 'Cancel'. I cannot figure out how to click on the Continue Button. The problem is
that I don't know how to control Webpage Dialog with Python. I tried to use
winGuiAuto to find the controls of the window, but it fails to recognize any Button type
controls... An ideas?
Sasha
A clarification of my question:
My purpose is to download stock data from a certain web site. I need to perform it for many stocks so I need python to do it for me in a repetitive way. This specific site exports the data by letting me download it in Excel file by clicking a link. However after clicking the link I get a Web Page dialog box asking me if I am sure that I want to download this file. This Web page dialog is my problem - it is not an html page and it is not a regular windows dialog box. It is something else and I cannot configure how to control it with python. It has two buttons and I need to click on one of them (i.e. Continue). It seems like it is a special kind of window implemented in IE. It is distinguished by its title which looks like this: Webpage Dialog -- Download blalblabla. If I click Continue mannually it opens a regular windows dialog box (open,save,cancel) which i know how to handle with winGuiAuto library. Tried to use this library for the Webpage Dialog window with no luck. Tried to recognize the buttons with Autoit Info tool -no luck either. In fact, maybe these are not buttons, but actually links, however I cannot see the links and there is no source code visible... What I need is someone to tell me what this Web page Dialog box is and how to control it with Python. That was my question.
You can't, and you don't want to. When you ask a question, try explaining what you are trying to achieve, and not just the task immediately before you. You are likely barking down the wrong path. There is some other way of doing what you are trying to do.
The title 'Webpage Dialog' suggests that is a Javascript-generated input box, hence why you can't access it via winGuiAuto. What you're asking directly is unlikely to be possible.
However, making the assumption that what you want to do is just download this data from the site, why are you using the GUI at all? Python provides everything you need to download files from the internet without controlling IE. The process you will want to follow is:
Download the host page
Find the url for your download in the page (if it changes)
Download the file from that url to a local file
In Python this would look something like this:
import urllib,re
f = urllib.urlopen('http://yoursitehere') # Original page where the download button is
html = f.read()
f.close()
m = re.search('/[\'"](.*\.xls)["\']/', html, re.S) # Find file ending .xls in page
if m:
urllib.urlretrieve(m.group(1), 'local_filename.xls') # Retrieve the Excel file
It is better to use selenium Python bindings:
from selenium import webdriver
from selenium.webdriver.common import alert
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
class AlertsManager:
def alertsManager(self,url):
self.url_to_visit=url
self.driver=webdriver.Ie()
self.driver.get(self.url_to_visit)
try:
while WebDriverWait(self.driver,1).until(EC.alert_is_present()):
self.alert=self.driver.switch_to_alert()
self.driver.switch_to_alert().accept()
except TimeoutException:
pass
if __name__=='__main__':
AM=AlertsManager()
url="http://htmlite.com/JS006.php" # This website has 2 popups
AM.alertsManager(url)

Categories

Resources