Develop Reference

I cannot find a "button onclick" element using Selenium and Python

I am automating a process using Selenium and python. Right now, I am trying to click on a button in a webpage (sorry I cannot share the link, since it requires credential to login), but there is no way my code can find this button element. I have tried every selector (by id, css selector, xpath, etc.) and done a lot of googling, but no success.
Here is the source content from the web page:
<button onclick="javascript: switchTabs('public');" aria-selected="false" itemcount="-1" type="button" title="Public Reports" dontactassubmit="false" id="public" aria-label="" class="col-xs-12 text-left list-group-item tabbing_class active"> Public Reports </button>
I also added a sleep command before this to make sure the page is fully loaded, but it does not work.
Can anyone help how to select this onclick button?
Please let me know if you need more info.
Edit: you can take a look at this picture to get more insight (https://ibb.co/cYXWkL0). The yellow arrow indicates the button I want to click on.

The element you trying to click is inside an iframe. So, you need to switch driver into the iframe content before accessing elements inside it.
I can't give you a specific code solution since you didn't share a link to that page, even not all that HTML block. You can see solutions for similar questions here or enter link description here. More results can be found with google search

How to find the "See More Posts from..." button? Selenium - Python

Disclaimer: sorry if my explanation is weird. It's my first week learning.
As my first project, I've set out to build a program that counts Instagram posts in a given time frame. Almost everything works, but there is one button standing in my way. (sometimes)
When you open up Instagram with WebDriver, the GDPR notice pops up. This is easy to get rid of since the xpath for this button is always the same. Or less barbaric way:
driver = webdriver.Chrome()
driver.get(ig_link)
driver.find_element_by_xpath('//button[text()="Accept"]').click()
then, I want to scroll down and during it, get URLs of all posts on the page. As I've already mentioned, on some profiles this works just fine. Yet on others, only few recent posts load up and then you are required to press this pesky "Show More Posts from {instagram_tag}"
My idea was to just brute-force it through driver.find_element_by_xpath() but then I found out that on different IG pages, the xpath is different. The button also doesn't have any text directly inside it, so I am not able to use the same approach that I've used with the "Accept" button.
<button class="tCibT qq7_A z4xUb w5S7h">
<div class="Igw0E IwRSH eGOV_ _4EzTm lC6p0 HVWg4">
<div class="_7UhW9 xLCgt qyrsm h_zdq uL8Hv l4b0S ">Show More Posts from choco_afro
</div>
</div>
</button>
My idea is that I check whether the div element under button includes text "Show More Posts" and then click the button, but I don't know how to do that if even possible. Would anybody be so kind and help me with this? :)

I believe, that, this xpath might help you.
//button//div[contains(text(), 'Show More Posts')]/../..
If each IG page has the same HTML design, then this example should help you. Solution: Click Show More Posts Button
driver.find_element(By.XPATH, "//button//div[contains(text(), 'Show More Posts')]/../..").click()

Try this:
btn = driver.find_element_by_xpath('//button[#class = "tCibT qq7_A z4xUb w5S7h"]')
div = btn.find_elements_by_tag_name('div')[1]
if 'Show More Posts' in div.text.strip():
btn.click()

You can use the following to retrieve that button from it's sibling's text by css selector.
//div[text()='Show More Posts from choco_afro']/parent::div/parent::button

Selenium Click JS Button - Python?

yeah, there are similar question, but upon reading through them I weren't able to find a solution for my problem.
Following situation: I'm tryin to click the "reply" button on "https://charleston.craigslist.org/ctd/d/charleston-2018-nissan-sentra-sedan-4d/7108660907.html" and after executing this click a popups shows up where I shall click another button, but let's start with the first button as the "reply" click itself is very trouble-making.
The reply button has the following X-Path:
'/html/body/section/section/header/div[2]/div/button'
speaking of which the source code is:
<button role="button" class="reply-button js-only" data-href="/__SERVICE_ID__/chs/ctd/7108660907">
reply
</button>
(see code on mentioned website).
However, my approach with Selenium (Python) doesn't work:
reply_button = '/html/body/section/section/header/div[2]/div/button'
driver.get('https://charleston.craigslist.org/ctd/d/charleston-2018-nissan-sentra-sedan-4d/7108660907.html')
driver.find_element_by_xpath(reply_button).click()
Everytime I tried, the website just loads up properly (even with implementing time.sleep(x)) and tries to click the button, but this fails and the website just refreshes - my guess is that they either regocnize the browser being Selenium-controlled, that the click isn't legitimate or that I didn't catch anything right in my code.. Anyone able to help out?
Btw I already tried searching "by_class_name", that didn't work either.

This Xpath: '/html/body/section/section/header/div[2]/div/button' is like when you get a map with instructions like step forward until you see a car then turn left 30° then step forward until you see a tree then hop twice then go to the second house to your right. Not safe to use, avoid such paths. If the page layout changes, your path may become invalid.
Try this:
button = driver.find_element_by_xpath('//*[#class="reply-button js-only"]')
button.click()
Clicking the button opens a "show phone number" popup (which may be located by driver.find_element_by_xpath('//*[#class="show-phone"]')).
Explanation:
If you want proper Xpath, inspect what you want to interact with. The button you want to click is this:
<button role="button" class="reply-button js-only" data-href="/__SERVICE_ID__/chs/ctd/7108660907">
reply
</button>
You can see that it has no "id" tag but it is a button with a specific class. You may copy right away the "class" part -> class="reply-button js-only"
Now you can check if is it unique enough:
driver.find_elements_by_xpath('//*[#class="reply-button js-only"]')
If "find elements" returns a single result, usually you should be OK. You can see that all I did is that I pasted the class inside this: driver.find_elements_by_xpath('//*[# and this ]').
If you need more accuracy, you can specify that it is a button:
driver.find_element_by_xpath('//button[#class="reply-button js-only"]')
Or it is the direct child of the element with class: class="actions-combo", so a more safe path would be:
driver.find_element_by_xpath('//*[#class="actions-combo"]/button[#class="reply-button js-only"]')
This pattern works for all webelement attributes, not just for classes. You could use the role="button" too for more filtering. Look up for Xpath, it is a pretty neat stuff.

form, input problems when parsing webpage with selenium

I'm parsing webpages with selenium and beautifulsoup4,
and I have a problem with parsing specific webpage.
I got different html source pages when I actually view html source on that page, and parsing with selenium or bs4.
The difference is existence of form and input.
When I parse that page, I got html with
<form action="" method="post" name="fmove">
<input name="goAction" style="display:none" type="submit"/>
</form>
I can't find what to input or submit.
Please let me understand this problem.
Thanks!

I'm going to concentrate on '[finding] what to input or submit' without touching on wider questions. Even so, what I tell you is not guaranteed to yield answers if code associated with that page does not arrange to fill the form's action attribute and/or some of its input elements with name and value pairs.
First, open the page in the Chrome browser. Use the item in the context menu to 'Inspect' the element on the screen to find the Javascript that finally submits that form. Put a breakpoint on the line in the code where this happens. Now reload the page (F5) and exercise the form. The code should stop at the breakpoint. You should be able to see the properties of the form element, including action and the name-value pairs, in the rightmost portion of the screen where you can copy them for use in your own code.
PS: I really must mention that it's difficult to be sure of a lot of this without knowing what site you're scraping. Good luck!

Interact with webpage Beautifulsoup and python

I'm using Python 2.7 with beautifulsoup and urllib2, I'm trying to scrap this page: angel.co/companies
As you see it shows a list with companies and it ends with a button "More" to show the others. As you click the button, more companies appear to watch and it creates a new tag with the new list of resutls. The button is in this div: <div class="more" data-page="2">More</div> and each time you click it the data-page increases.
I'd like to know if it's possible to scrap this page completely (so it clicks the "More" button each time it arrives to the end). I suppose it is scrapping the css and changing it but I never did so and I haven't found information about this anywhere.

Depending on what you want to do you could use their API for this. If you are not sure what it is and how to use it, try googling around for an answer. Here's one for starters.