Disclaimer: sorry if my explanation is weird. It's my first week learning.
As my first project, I've set out to build a program that counts Instagram posts in a given time frame. Almost everything works, but there is one button standing in my way. (sometimes)
When you open up Instagram with WebDriver, the GDPR notice pops up. This is easy to get rid of since the xpath for this button is always the same. Or less barbaric way:
driver = webdriver.Chrome()
driver.get(ig_link)
driver.find_element_by_xpath('//button[text()="Accept"]').click()
then, I want to scroll down and during it, get URLs of all posts on the page. As I've already mentioned, on some profiles this works just fine. Yet on others, only few recent posts load up and then you are required to press this pesky "Show More Posts from {instagram_tag}"
My idea was to just brute-force it through driver.find_element_by_xpath() but then I found out that on different IG pages, the xpath is different. The button also doesn't have any text directly inside it, so I am not able to use the same approach that I've used with the "Accept" button.
<button class="tCibT qq7_A z4xUb w5S7h">
<div class="Igw0E IwRSH eGOV_ _4EzTm lC6p0 HVWg4">
<div class="_7UhW9 xLCgt qyrsm h_zdq uL8Hv l4b0S ">Show More Posts from choco_afro
</div>
</div>
</button>
My idea is that I check whether the div element under button includes text "Show More Posts" and then click the button, but I don't know how to do that if even possible. Would anybody be so kind and help me with this? :)
I believe, that, this xpath might help you.
//button//div[contains(text(), 'Show More Posts')]/../..
If each IG page has the same HTML design, then this example should help you. Solution: Click Show More Posts Button
driver.find_element(By.XPATH, "//button//div[contains(text(), 'Show More Posts')]/../..").click()
Try this:
btn = driver.find_element_by_xpath('//button[#class = "tCibT qq7_A z4xUb w5S7h"]')
div = btn.find_elements_by_tag_name('div')[1]
if 'Show More Posts' in div.text.strip():
btn.click()
You can use the following to retrieve that button from it's sibling's text by css selector.
//div[text()='Show More Posts from choco_afro']/parent::div/parent::button
Related
I am automating a process using Selenium and python. Right now, I am trying to click on a button in a webpage (sorry I cannot share the link, since it requires credential to login), but there is no way my code can find this button element. I have tried every selector (by id, css selector, xpath, etc.) and done a lot of googling, but no success.
Here is the source content from the web page:
<button onclick="javascript: switchTabs('public');" aria-selected="false" itemcount="-1" type="button" title="Public Reports" dontactassubmit="false" id="public" aria-label="" class="col-xs-12 text-left list-group-item tabbing_class active"> Public Reports </button>
I also added a sleep command before this to make sure the page is fully loaded, but it does not work.
Can anyone help how to select this onclick button?
Please let me know if you need more info.
Edit: you can take a look at this picture to get more insight (https://ibb.co/cYXWkL0). The yellow arrow indicates the button I want to click on.
The element you trying to click is inside an iframe. So, you need to switch driver into the iframe content before accessing elements inside it.
I can't give you a specific code solution since you didn't share a link to that page, even not all that HTML block. You can see solutions for similar questions here or enter link description here. More results can be found with google search
Hey guys I am working on an automating bot for adding connections in LinkedIn, recently I encountered a problem for finding the "Connect" element(button) after searching in the search box and filtering for "People only".
That is the inspect of the relevant part:
<div id="ember374" class="ember-view">
<button id="ember378" class="artdeco-button artdeco-button--2 artdeco-button--secondary ember-view" data-control-name="entity_action_primary" data-control-id="AQT4G2UkQk6LQlbdf6XySw==">
<!---->
<span class="artdeco-button__text">Connect</span></button>
This is my tries for clicking on that "connect" element:
1. CONNECT_BUTTON_SEARCH_PAGE = "//*[contains(#class,'artdeco-button__text') and contains(.,'Connect')]"
2. CONNECT_BUTTON_SEARCH_PAGE = "//*[text()='Connect']"
3. CONNECT_BUTTON_SEARCH_PAGE = "//button[contains(. , ’Connect’)]"
WebDriverWait(self.driver, 5).until(EC.presence_of_element_located(
(By.XPATH, Xpath.CONNECT_BUTTON_SEARCH_PAGE))).click()
I have to mention that it works fine in the "Network" page, but I want it to add connections after searching a key word. So when I'm searching people according to a special key word, in that page I don't find the "Connect" button
I don`t know why but I can not reach for that element, appreciate if someone could help me
Finaly found a (weird) soulution with the help of #arundeepchohan.
First I used this xpath:
"//button/span[text()='Connect']/.."
But then I had to "refresh" the page before trying to click the "connect" button. I don't have an explanation for that to work and why it was necessary to refresh the page, but I used this part of code before the clicking method:
time.sleep(2)
self.driver.get(self.driver.current_url)
Thanks for everyone who tried to help!
Try this xpath :
//button[contains(. , ’Connect’)]
yeah, there are similar question, but upon reading through them I weren't able to find a solution for my problem.
Following situation: I'm tryin to click the "reply" button on "https://charleston.craigslist.org/ctd/d/charleston-2018-nissan-sentra-sedan-4d/7108660907.html" and after executing this click a popups shows up where I shall click another button, but let's start with the first button as the "reply" click itself is very trouble-making.
The reply button has the following X-Path:
'/html/body/section/section/header/div[2]/div/button'
speaking of which the source code is:
<button role="button" class="reply-button js-only" data-href="/__SERVICE_ID__/chs/ctd/7108660907">
reply
</button>
(see code on mentioned website).
However, my approach with Selenium (Python) doesn't work:
reply_button = '/html/body/section/section/header/div[2]/div/button'
driver.get('https://charleston.craigslist.org/ctd/d/charleston-2018-nissan-sentra-sedan-4d/7108660907.html')
driver.find_element_by_xpath(reply_button).click()
Everytime I tried, the website just loads up properly (even with implementing time.sleep(x)) and tries to click the button, but this fails and the website just refreshes - my guess is that they either regocnize the browser being Selenium-controlled, that the click isn't legitimate or that I didn't catch anything right in my code.. Anyone able to help out?
Btw I already tried searching "by_class_name", that didn't work either.
This Xpath: '/html/body/section/section/header/div[2]/div/button' is like when you get a map with instructions like step forward until you see a car then turn left 30° then step forward until you see a tree then hop twice then go to the second house to your right. Not safe to use, avoid such paths. If the page layout changes, your path may become invalid.
Try this:
button = driver.find_element_by_xpath('//*[#class="reply-button js-only"]')
button.click()
Clicking the button opens a "show phone number" popup (which may be located by driver.find_element_by_xpath('//*[#class="show-phone"]')).
Explanation:
If you want proper Xpath, inspect what you want to interact with. The button you want to click is this:
<button role="button" class="reply-button js-only" data-href="/__SERVICE_ID__/chs/ctd/7108660907">
reply
</button>
You can see that it has no "id" tag but it is a button with a specific class. You may copy right away the "class" part -> class="reply-button js-only"
Now you can check if is it unique enough:
driver.find_elements_by_xpath('//*[#class="reply-button js-only"]')
If "find elements" returns a single result, usually you should be OK. You can see that all I did is that I pasted the class inside this: driver.find_elements_by_xpath('//*[# and this ]').
If you need more accuracy, you can specify that it is a button:
driver.find_element_by_xpath('//button[#class="reply-button js-only"]')
Or it is the direct child of the element with class: class="actions-combo", so a more safe path would be:
driver.find_element_by_xpath('//*[#class="actions-combo"]/button[#class="reply-button js-only"]')
This pattern works for all webelement attributes, not just for classes. You could use the role="button" too for more filtering. Look up for Xpath, it is a pretty neat stuff.
I am trying to click a button to show more comments on a discusson thread on http://disqus.com/
The HTML looks like following:
<div class="load-more" data-role="more" style="">
Show more
</div>
I have tried using the xPath from the button like following:
driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")
driver.get(url)
driver.find_element_by_xpath('''//*[#id="posts"]/div[3]/a''').click()
But I get the NoSuchElement exception.
What is the proper way of clicking a button with that type of HTML?
UPDATE: Worked when I switched to a specific iFrame:
driver.switch_to.frame('myID')
Then loaded by class name:
element = driver.find_element_by_class_name('load-more')
element.click()
NOTE: That the click() did not work when it was performed on the same line like driver.find_element_by_class_name('load-more').click()
Xpath should be a last resort for finding elements. For your case, I would try
driver.find_element_by_class_name("load-more__button"));
Using btn load-more__button won't work as compound classNames aren't valid.
If that still doesn't work, Id recommend using css selectors before Xpath.
However, if none of these options work, and your Xpath is still not working, you should use the firebug add on for Firefox to ensure the Xpath you are using is correct.
Here's some of my other answers that may help you get started with Firebug:
https://stackoverflow.com/a/38980488/3537915
https://stackoverflow.com/a/38723782/3537915
https://stackoverflow.com/a/38744577/3537915
This is going to be an easy one for somebody, and turned into a long post for what I think is a simple question.
I am scraping a supplier's site and writing to a csv so that I don't have to manually slog through pages and pages of products. In order to get a product list I run a search with no variables and get the lot. However, if I try to limit my search to in-stock products using a dropdown option my code craps out.
Since this is a 3rd party site I cannot alter the html. Here is a slice of the html from the site (reformatted since it copied very ugly):
<tbody id="adv_search_box">
<tr>
<td class="data-name">Search in category:</td>
<td class="data-input" colspan="2">
<select name="posted_data[categoryid]" class="adv-search-select">
<option value=""> </option>
<option value="257">Hot New Arrivals</option>
<option value="252">In-Stock</option>
...
<button class="button main-button" type="submit" title="Search">
<span class="button-right"><span class="button-left">Search</span></span>
</button>
</td>
</tr>
</tbody>
When I use the following to click the search button, I get everything the supplier has with no regard to status. So this works just fine:
searchButton = driver.find_element_by_xpath('//*[#id="adv_search_box"]/tr[8]/td[2]/button/span/span')
actions = ActionChains(driver)
actions.move_to_element(searchButton)
actions.click(searchButton)
actions.perform()
Then I add code to select the "In-Stock"option in front of the button-click code:
#Select in-stock on search page
InStockSelection = driver.find_element_by_xpath('//*[#id="adv_search_box"]/tr[1]/td[2]/select/option[3]')
actions = ActionChains(driver)
actions.move_to_element(InStockSelection)
actions.click(InStockSelection)
actions.perform()
#Click button
searchButton = driver.find_element_by_xpath('//*[#id="adv_search_box"]/tr[8]/td[2]/button/span/span')
actions = ActionChains(driver)
actions.move_to_element(searchButton)
actions.click(searchButton)
actions.perform()
I get the following error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="adv_search_box"]/tr[8]/td[2]/button/span/span"}
One final note. I realize that this can be coded in a much, much more compact and elegant way. It is coded like this because I'm not going to look at it again until it breaks (i.e. the supplier changes their site), and if it's coded like my six-year old did it I'll be able to figure out what I did years from now.
Thank you all for your assistance.
Edit 1: It appears to me that straightforward xpath solution will not work. In pseudo-coding-babble, I think the program is focused on the drop down table and can't see the rest of the html. If I could refocus back on the page as a whole it ought to work.
Solution: Programming is to me like magic. It is often what you're not looking at that is the bit you need to watch out for. In this case, there was nothing wrong with the code to select the button. The problem was that the code to select the dropdown option wasn't working correctly, and the click at the end was changing the page focus.
The solution was to change the way the option was located and avoid the action chain.
driver.find_element_by_xpath("//select[#name='posted_data[categoryid]']/option[text()='In-Stock']").click()
Assistance was found in this article: Selenium - Python - drop-down menu option value.
Thank you, MikeJRamsey56, for convincing me to ditch the action chain, and thank you dejavu_cmd_delt for taking the time to answer.
i would suggest to use below xpath as it is free of indexing hassle, assuming that there is only one button in all tr
searchButton = driver.find_element_by_xpath('//*[#id="adv_search_box"]//button[#title="Search"]//span[#class="button-left"])