itemList = driver.find_elements_by_class_name('site-piclist_info')
print(len(itemList))
for item in itemList:
item.click()
handles = webdriver.window_handles
print(handles)
line1 works and line2 is processed and get the result
but line4 cannot be processed
the list of handls always contain only one handle; see the image.
Let us see whats happening in your code:
itemList = driver.find_elements_by_class_name('site-piclist_info')
print(len(itemList))
With this we have got a list of all the nodes with class='site-piclist_info'
Next:
for item in itemList:
item.click()
Here we are trying to click on each and every node in iterations which are within the list irrespective of whether they are actual links or not. But as you mentioned line4 cannot be processed that is because the first node with class='site-piclist_info' may not be a Link or within an <a> tag that's why invoking click() method doesn't works.
Finally, list of handls always contain only one handle that is the handle of the main window which is opened by the webdriver
A proper solution may be to construct a unique xpath or css which identifies the proper group of elements through find_elements_* method and then save in a list to iterate through it.
I checked the website in your screenshot. You're actually clicking on a list of divs that each contains a link that opens in a new window or tab, depending on browser.
You'll need to click on the links with this class site-piclist_pic_link instead.
Related
I need to find a location on the webpage by the visible text. Then I need to copy the value that is adjacent to it. It makes more sense when you see the screenshot.
So the idea is to find the text "Net PPW" and then grab the value, which for this example is "1.56".
I assume this is possible since they are in the same section of the webpage but I have no clue how to do this or where to start.
One of the simpliest solution is mannually check every p element text on page to get next argument with p tag in result(cause they always stay together).
elements = driver.find_elements(By.TAG_NAME, 'p')
for x in range(len(elements)):
if elements[x].text == "Net PPW"
print(elements[x+1].text)
With use of XPath you can locate the parent element based on it's parent and then locate the parent element, as following:
driver.find_element(By.XPATH, "//div[contains(#class,'justify-content-between')][contains(.,'Net PPW')]//p[contains(#class,'MuiTypography-body1')]").text
I'm very new to programming so apologies in advance if I'm not communicating my issue clearly.
Essentially, using Selenium I have created a list of elements on a webpage by finding all the elements with the same class name I'm looking for.
In this case, I'm finding songs, which have the html class 'item-song' on this website.
On the website, there are lots of clickable options for each listed song . I just want to click the title of the song, which opens a popup modal window in which I edit the note attached to the song, then click save, which closes the popup.
I have successfully been able to do that by using what I guess would be called the title’s XPATH 'relative' to the song class.
songs = driver.find_elements(By.CLASS_NAME, "item-song")
songs[0].find_element(By.XPATH, "div[5]/a").click()
# other code that ends by closing popup
This works, hooray! It also works for any other list index that I put in that line of code.
However, it does not work sequentially, or in a for loop.
i.e.
songs[0].find_element(By.XPATH, "div[5]/a").click()
# other code
time.sleep(5) # to ensure the popup has finished closing
songs[1].find_element(By.XPATH, "div[5]/a").click()
Does not work.
for song in songs:
song.find_element(By.XPATH, "div[5]/a").click()
# other code
time.sleep(5)
continue
Also does not work.
I get a traceback error:
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
After going back to the original page, the song does now say note(1) so I suppose the site has changed slightly. But as far as I can tell, the 'songs' list object and the xpath for the title of the next song should be exactly the same. To verify this, I even tried:
for song in songs:
print(song)
print(songs)
print()
song.find_element(By.XPATH, "div[5]/a").click()
# other code
Sure enough, on the first iteration, print(song) matched the first index of print(songs) and on the second iteration, print(song) matches the second index of print(songs). And print(songs) is identical both times. (Only prints twice as the error happens halfway through the second iteration)
Any help is greatly appreciated, I'm stumped!
---------------------------------
Edit: Of course, it would be easier if my songs list could be all the song titles instead of the class ‘item-song’, that was what I was trying first. However I couldn’t find anything common between the titles in the HTML that would let me use find_elements to just get the song title element, as each song has a different title, and there are also other items like videos listed in between each song.
Through the comments, the solution is to use an iterative loop and an xpath.
songs = driver.find_elements(By.CLASS_NAME, "item-song")
for i in range(songs.count):
driver.find_element(By.XPATH, "(//*[#class='item-song'][" + i + "])/div[5]/a").click()
Breaking this down:
this: By.XPATH, "//*[#class='item-song']" is the same as this: By.CLASS_NAME, "item-song". The former is the xpath equivalent of the latter. I did this so we can build a single identification string to the link instead of trying to find elements within elements.
The [" + i + "] is the iteration for the the loop. If you were to print this you'd see (//*[#class='item-song'][1])") then (//*[#class='item-song'][2])"). That [x] is the ordinal identifier - it means the xth instance of the element in the DOM. The brackets around it ensure the entire thing is matched for the next part - you can sometimes get unexpected matches without it.
The last part /div[5]/a is just the original solution. Doing div[5] isn't great. Your link must ALWAYS be inside the 5th div else it will fail - but as i can't see your application I can't comment on another way.
The original approach throws a StaleElementReferenceException because of the way Selenium stores identified elements.
Once you've identified an element by doing driver.find_elements(By.CLASS_NAME, "item-song") Selenium essentially captures a reference to it - it doesn't store the identifier you used. Stick a break point and after you identify an element and you'll see something like this:
That image is from visual studio as I have it hand but you can see it's a GUID on the ID.
Once you change the page that reference is lost.
Repeat the same steps, identify the same object and the ID is unique every time. This is same break point, same element on a second test run:
Page has changed == Selenium can no longer find it == Stale element.
The solution in this answer works because we're not storing an element.
Every action in the loop freshly identifies the element.
..Then add some clever pun about fresh vs stale... ;-)
I have logged onto a third party website where I have a page full of URLs to click. I was able to collect the elements as such:
for i in num_alphabets:
names = browser.find_elements(by=By.CLASS_NAME, value='client-name')
print(len(names)) # gives you the number of clients!!!
So now that I have this list called names, I want to know how I can click through all these elements. For now that would be enough, later on I want to have it store some information after clicking. I tried adding .click() at the end and it obviously won't work as names is a list.
Should I maybe create an array with only the first values of the list and then click through that? browser.click() isn't a valid attribute so I don't think that would work.
Here names is the list of WebElements with value as client-name. So to click on each of those WebElements you have to iterate through the list. Incase the page doesn't changes and the client-name sections are on the same page you can use the following code block:
for name in names:
name.click()
I am learning web scraping using selenium and I've come into an issue when trying to select an attribute inside of a selenium object. I can get the broader data if I just print elems.text inside the loop (this outputs the whole paragraph for each listing) however when I try to access the xpath of the h2 title tag of all the listings inside this broader element, it only appends the first listing to the titles array, whereas I want all of them. I checked the XPATH and they are the same for each listing. How can I get all of the listings instead of just the first one?
titles = []
driver.get("https://www.sellmytimesharenow.com/timeshare/All+Timeshare/vacation/buy-timeshare/")
results = driver.find_elements(By.CLASS_NAME, "results-list")
for elems in results:
print(elems.text) #this prints out full description paragraphs
elem_title = elems.find_element(By.XPATH, '//*[#id="search-page"]/div[3]/div/div/div[2]/div/div[2]/div/a[1]/div/div[1]/div/h2')
titles.append(elem_title.text)
If you aren't limited to accessing the elements by XPATH only, then here is my solution:
results = driver.find_elements(By.CLASS_NAME, "result-box")
for elems in results:
titles.append(elems.text.split("\n")[0])
When you try getting the listings, you use find_elements(By.CLASS_NAME, "results-list"), but on the website, there is only one element with the class name "results-list". This aggregates all the text in this div into one long string and therefore you can't get the heading.
However, there are multiple elements with the class name "result-box", so find_elements will store each as its own item in "results". Because the title of each listing is on the first line, you can split the text of each element by the newline.
I am trying to create code that will click multiple links on a webpage that will inturn download files. However, I have written an XPath that contains all the links. When trying to click I get this error: "click() takes 1 positional argument but 2 were given".
Is there any way to click all links in the desired XPath with .click() function in selenium?
This is what I am trying with 10 files:
browser.find_element_by_xpath("//a[starts-with(#href,'sys_attachment.do') and #class='linked formlink']").click('href')
Is there a better way to download all of these with the XPath? Thanks.
You should use find_elements_by instead of element so you can loop over the list of elements, and click on each one, ex:
clickList = browser.find_elements_by_xpath("//a[starts-with(#href,'sys_attachment.do') and #class='linked formlink']")
for count, elem in enumerate(clickList):
# I am refreshing the list in case the DOM is changed
currentList = browser.find_elements_by_xpath("//a[starts-with(#href,'sys_attachment.do') and #class='linked formlink']")
currentList[count].click()
If you want to find many elements then use find_elemenets_... with s in word elements and you get list with many elements. And then you have to use for-loop to use click() (without arguments) on every element separatelly.
all_items = browser.find_elements_by_xpath("//a[starts-with(#href,'sys_attachment.do') and #class='linked formlink']")
for item in all_items:
item.click()
Problem is when click changes page because then items stop exist and you would have to go back, find again all items and click next element on list - but you have to remeber index of previously clicked item.