How do I click all the unopened tabs pages where the value changes when you click tabs? (see image below)
Take the following script, based off of this question with the following approach:
clickMe = wait(driver, 10).until(
EC.element_to_be_clickable((By.XPATH, path + '[%s]' % str(index - 1))))
Works great for the large majority of pages, however I cannot see how it can be used in this case. Any idea how you can get around this issue?
The reason it does not work is because as soon as you click an unopened tab, the number values change. This means the job is trying to click elements that used to exist, but the number values changed when you opened a closed tab. This is better illustrated in the image below.
Say you had //div[#class="KambiBC-collapsible-container KambiBC-mod-event-group-container"][1]
In the picture below, you can see that the above selector will change based upon when the job itself clicks tabs and upon loading the webpage. The webpage has a habit of randomising opened and closed tabs. So this explains why it will click some elements and then completely break or if you add try except, click only some elements and finish.
output:
1
2
3
4
#Stops working
Desired
1
2
3
4
etc,, (Clicks all the pages)
The above images are both:
//div[#class="KambiBC-collapsible-container KambiBC-mod-event-group-container"][1]`
You can see how this will cause issues later on. The image on the right would not be able to locate //div[#class="KambiBC-collapsible-container KambiBC-mod-event-group-container"][11] where a the one on the left could based on the order of tabs clicked.
outerhtml
Rather than using an index to reference these "tabs", you need an alternate way to uniquely identify each one.
One solution is to work with the developers of the site to get them to add an id or name attribute to the "KambiBC-collapsible-container" for each tab.
Another solution, if you're using the Page Object Model approach, would be to create a method on the page that provides a list of the available tabs that you can then reference by the header text. Then you just keep track of which ones you've opened.
Yet another solution would depend on the functionality of the site. You may be able to leverage the fact that if a tab has been opened it looks like the container has a class "KambiBC-expanded". If a tab will stay open until you explicitly close it, you could simply get a list of all of the containers that do not have the "KambiBC-expanded" class and pick one to expand, repeating until there are no more to expand.
Related
I am writing a program that answers questions from a database. I am currently on the last question type (ordering questions). Here is an image of what the question looks like. The user needs to put the elements in the box in order.
As you can see, the bottom of the question gets cut off. This caused errors when trying to click certain elements that were below the page. To try and solve this I scrolled to the bottom of the page but selenium scrolled back to the top before clicking which kept the error in place. And after trying a few other things I decided to try and zoom out on the webpage to see them all at once.
This is the HTML of the list where the <p> tag is what I am trying to click on. Actually, I am trying to right-click but same thing. The rest of the <div>s with the ellipses are the rest of the boxes and are the same as the first one.
newheader = self.driver.find_element(By.XPATH, "//div[contains(#class, 'responses-container')]")
action = ActionChains(self.driver)
action.context_click(WebDriverWait (newheader, 10).until(EC.presence_of_element_located((By.XPATH, "//p[contains(text(), '"+ dictionary_.get(prompt)[r] +"')]")))).perform()
# dictionary.get(prompt)[r] contains the values we are looking for (ovulation, fertilization, etc)
# I have also tried EC.element_to_be_clickable and had the same results
I have tested this on a question that does not overflow the page and it works perfectly. The only time I am getting errors is when I try to zoom out. I have tried refreshing the page before starting the clicking and after zooming out. I have also tried resetting the driver by calling
self.driver.get(self.driver.current_url)
Each of those elements cannot be left-clicked on to select. They can only be right-clicked or tabbed to. Tabbing requires calculations that are complex since each element is being moved dynamically so I chose to right-click instead. It does right-click but in seemingly random places now that it's zoomed out.
Any help is appreciated! Thanks!
https://squidindustries.co/checkout
checkout_cc_number = driver.find_element_by_id("number")
checkout_cc_number.send_keys(card_number)
When I try to input information into the card number field I get an error saying the element could not be located. I tried using time.sleep and driver.implicitly_wait when i first got to the page but both failed. Any ideas?
The element is in a frame (i.e. a webpage within a webpage). Selenium will look for elements in the page it has loaded and not within frames. That's the problem.
To solve this we just need a bit more code, which will tell Selenium to look in the frame.
The example you've given is several pages deep into a shopping cart, so I'm going to use a much more accessible example instead: the mozilla guide to iframes.
Here is some code to open that page and then click the CSS button within the frame:
from selenium import webdriver
import time
browser = webdriver.Chrome()
browser.get(r"https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
time.sleep(5)
browser.switch_to.frame(browser.find_element_by_class_name("interactive"))
css_button = browser.find_element_by_id("css")
css_button.click()
browser.switch_to.default_content()
There are two lines that are important. The first one is:
browser.switch_to.frame(browser.find_element_by_class_name("interactive"))
That finds the frame and then switches to it. Once we have done that, any code that looks for elements will be looking in the frame and not in the page that we navigated to. That is what you need to do to access the number element. In your example the class of the frame is card-fields-iframe, so use that instead of interactive.
The second important line is:
browser.switch_to.default_content()
That reverts the previous line. So now Selenium will be looking for elements within the page that we navigated to. You'll want to do that after interacting with the frame, so that you can continue through the shopping cart.
have you tried getting the input element using the DOM? what happens if you do document.getElementById('number') ?
I ran into the same issue, and with checkouts, as you mentioned, all the iframe class names are the same. What I did was get all the iframes with the same class name as a list:
iframes = driver.find_elements(By.CLASS_NAME, "card-fields-iframe")
I then switched through the iframes referencing each one by its place in the list. Since there are only four fields in the checkout, the list is only 4 elements long, starting with [0].
driver.switch_to.frame(iframes[0])
number = driver.find_element(By.ID, "number")
if number.is_displayed:
number.send_keys("4000300040005000")
driver.switch_to.default_content()
It's important to note that switching back to the default content, using driver.switch_to.default_content(), before switching to the next frame is the only way I was able to make this work. The is_displayed function just checks to see whether the element is on the page or not.
I should start by saying that I'm not a web developer but I have been using Selenium with Python for a few weeks and think I've gotten the basics down.
What I have is a page with a weekly calendar on it. Image as follows:
What happens is that you can click on any of the coloured boxes which will bring up a register for a class. It features items that you can click on which bring up new information. The problem I have is that I can't click on the items - or, rather, I don't know how to automate the click. I have automated clicking with Selenium before in other situations.
That looks as follows:
As you can see, the calendar still appears in the background. That is to say, the item we click on doesn't take us to a new page but, I suppose, runs some kind of method which shows the register and populates it with data.
My problem is this: I want to automate the process so that I can click on each of these items then scrape the information. In order to do that I need to be able to automate the clicking for each of the items.
So what have I done? When I've done this before, I've searched through the web html for the relevant part and then grabbed the xPath to the element I needed. But here I can't do that. Why not? Well, firstly, I just can't find that element!
Take a look at this close up of the first column:
It's divided into columns, but then I'd expect the clickable area to be an element within that. As you can see, it's not. Furthermore, the clickable area is just the coloured box itself, but if you look closely you can see the element goes outside of that area. I have gone very close with my mouse cursor to see exactly what's clickable, and it definitely is just the coloured box.
So I've not been able to get the element at all.
I thought I might be able to just find out where we went after I clicked the button, but when I got the link address, it just said it was the same page with no differences.
I appreciate I'm asking quite a broad question here, but the problem is that I don't really know where to start. If someone could give me at least that much, I would be grateful. Like if I could just click on each of these one at a time... I've found where the populated data is so I could grab that without a problem.
Well, here's to hoping.
Edit: I should add that there are some JavaScript items (tag type script, type='text/javascript'). I presume that the answer is in there somewhere, but there is a lot of Javascript and I'm not adept at reading it. It's hard for me to tell what script does what. If I could at least figure out what script runs when I click the item then I think I'd be onto something, but I have no idea. Even that would help me.
I had encountered similar problem when scraping for Instagram followers in mobile view where it was floating box when showing the accounts followers name. The approach I took was identifying the name of floating dialog box and clicking the element in it. It might different in your case of html.
Trying looking at this link. Selenium Scroll inside of popup div
Hard to say without the HTML. Maybe try Katalon Recorder (chrome extension) and see if that can detect the xpath for you? It might also be you have to use some kind of javascript to invoke the method for the element
I've been creating a tool that plays through an online game using python 2.7 and selenium, and am very stuck on one particular element I need to select.
The UI looks as follows:
1 2 3
a d g
b e h
c f i
The numbers one two and three represent a drop down menu, which when clicked open up the letters. Each option represents a different outcome. The problem is, at the start of each game the positions of both the numbers and letters are randomized. In the code, each button's css selector is labeled as for example "#action-1 > button:nth-child(1)", as in the "first button", but the "first button" will be different every game.
I've tried finding by link text, xpath, and css selector to no success.
If it helps, pressing "copy outer html" gives this:
<button class="ng-binding" ng-click="subBtn($event)" ng-class="{disabled : !state.chapterStart || state.btns.indexOf(btn.action) != -1}" ng-disabled="!state.chapterStart || state.btns.indexOf(btn.action) != -1">Wait and See What They Do</button>
The "Wait and See What They Do" part of it is what the button says and is how you know what to click, but I can't seem to find the element by that.
Copying the xpath results in //*[#id="action-1"]/button, which again isn't really helpful because telling it to click that would just be like saying "click the button that is in the first position", and it changes every time. I've added long time.sleep() commands to test, the page is definitely loading all the way so that is not the issue.
Anyways, I've been trying to figure this one out for way too long and it has me stumped. I'd be very grateful for any input you all can give. Thank you!
If you are looking to locate an element by the text it contains you can use XPath. You stated that you tried XPath but no specifics were given. Did you try the below? It should work given the HTML you provided.
//button[text()='Wait and See What They Do']
To read this XPath... find any descendant // that is a BUTTON tag that has an attribute [] where the text contained in the element is equal to the search string, text()=''.
Another example might be the below which is an alternate way to find your button. The problem with this way is that there may be many buttons on the page with that class so it may not be specific enough.
//button[#class='ng-binding']
I've found (and others have also) that XPaths are generally slower than other location methods. Because of this, I generally prefer By.Id, when available, then By.CssSelector. They are both significantly faster than XPath. I save XPath for things like finding text in an element or finding relative elements (CSS can do this some but is no where near as powerful as XPath).
XPath Examples
I want to crawl the dialogue text in a popup window. The problem is that after I triggered the link the window appears but it seems that the selenium driver cannot handle it automatically as I learned from other questions on this site by entering driver.window_handles.
The source of the trigger:
The value of len(driver.window_handles) is 1. I thought I can get the window element and then get the text via the get_attributes, fortunately I succeeded getting the element by
wd = driver.find_element_by_css_selector('div[node-type="repeat_list"]')
selenium.webdriver.remote.webelement.WebElement (session="f810cbbe-db43-4e8d-b484-664559ec8efc", element="{dd00e689-7991-44e9-85d3-76c69e79218f}")
But the sad thing is I don't know how to get all the stuff out from it since I don't know their attributes.
I'm not certain if it's a dialogue, a front end engineer told me that it looks like an animation. Anyway this is the source snippet:
PS: the browser is Firefox.
I thought it may violate the site's Acceptable Use Policy to crawl then I should hide some information. Sorry.
Once you have your parent element :
wd = driver.find_element_by_css_selector('div[node-type="repeat_list"]')
you can continue calling methods on this object, and in this order reach the children elements, you can use find element_by_xpath, or find element_by_class name, for example:
wd = driver.find_element_by_css_selector('div[node-type="repeat_list"]')
wd.find_element_by_class_name("list_box").find_element_by_class_name("list_ul").find_elements_by_class_name("list_li S_line1 clearfix")
and so on until you reach the desired element down the hierarchy and extract it's content as you wish.
I hope this helps!