Develop Reference

Python/Selenium: Any way to wildcard the end of an xpath? Or search for a specifically formatted piece of an xpath?

I am using python / selenium to archive some posts. They are simple text + images. As the site requires a login, I'm using selenium to access it.
The problem is, the page shows all the posts, and they are only fully readable on clicking a text labeled "read more", which brings up a popup with the full text / images.
So I'm writing a script to scroll the page, click read more, scrape the post, close it, and move on to the next one.
The problem I'm running into, is that each read more button is an identical element:
read more
If I try to loop through them using XPaths, I run into the problem of them being formatted differently as well, for example:
//*[#id="page"]/div[2]/article[10]/div[2]/ul/li/a
//*[#id="page"]/div[2]/article[14]/div[2]/p[3]/a
I tried formatting my loop to just loop through the article numbers, but of course the xpath's terminate differently. Is there a way I can add a wildcard to the back half of my xpaths? Or search just by the article numbers?

/ is used to go for direct child, use // instead to go from <article> to the <a>
//*[#id="page"]/div[2]/article//a[.="read more"]
This will give you a list of elements you can iterate. You might be able to remove the [.="read more"], but it might catch unrelated <a> tags, depends on the rest of the html structure.
You can also try looking for the read more elements directly by text
//a[.="read more"]

I recommend using CSS Selectors over XPaths. CSS Selector provide faster, cleaner and simpler way to deal with these queries.
('a[href^="javascript"]')
This will selects every element whose href attribute value begins with "javascript" which is what you are looking for...
You can learn more about Locating Elements by CSS Selectors in selenium here.
readMore = driver.find_element(By.CSS_SELECTOR, 'a[href^="javascript"]')
And about Locating Hyperlinks by Link Text
readMore_link = driver.find_elements(By.LINK_TEXT, 'javascript')

Is there any way to click on "plain text" using selenium?

Apologies if this question was answered before, I want to click on an area in a browser with plain text using Selenium Webdriver in python
The code I'm using is:
element_plainText = driver.find_elements(By.XPATH, '//*[contains(#class, "WgFkxc")]')
element_plainText.click()
However this is returning "ElementNotInteractableException". Can anyone help me out with this?

Selenium is trying to be helpful here, by telling you why it won't click on the element; ElementNotInteractableException means it thinks that what you're trying to click on isn't clickable.
This usually happens because either:
The element isn't actually visible, or is disabled
Another element is "overlapping" the element, possibly invisibly
You're clicking something Selenium thinks won't do anything, like plain text
There's two things I'd try to get around this. Firstly, Actions. Selenium has an Action API you can use to cause specific UI events to occur. I'd suggest finding the co-ordinates of the text, then making Selenium click those co-ordinates instead of telling it to click the element. Read more about that API here.
Secondly, try clicking it with Javascript, using a Javascript Executor. That can often give you the same outcome as using Selenium directly, without it being so "helpful".

Python Selenium Finding Element With Dynamic URL and Name with Little HTML

Hey everyone! Today I am working with Python 3 and Selenium web driver. I am encountering a problem with finding by element. Here is the HTML:
<a _ngcontent-pxc-c302="" class="name ng-star-inserted" href="/person/20d4a795d3fb43bdbee7e480df27b05b">michele regina</a>
The goal is to click on the first name that appears in a listed column.
The name changes with every page refresh. This comes with two problems in that the link text is the name and it changes, and the link constantly changes as well for each different name except for the part that says /person/
I tried the following:
driver.find_element_by_css_selector("a.name ng-star-inserted").click()
driver.find_element_by_class_name("name ng-star-inserted").click()
Which resulted in an element that is not a clickable error.
I also tried xpath by copying and pasting the entire XPath copied from Google inspector... lol
driver.find_element_by_xpath('/html/body/app-root/app-container/div/div/tc-app-view/div/div/div/ng-component/mat-sidenav-container/mat-sidenav-content/div/div[2]/tc-person-table-container/tc-collapse-expand[1]/section/tc-person-table/table/tbody/tr[1]/td[2]/tc-person-table-cell/a').click()
and it kind of works sometimes, which is weird but I am sure there has to be a better way. Thank you so much in advance! Any help is appreciated! Also, I am very new to Python, I know zero HTML, and I am also newer to stack! Thanks!

From the given full xpath the a tag you are looking for is in table,section,div. Look for an id in those tags and trace the a.

Python Selenium Find if Element with an Attribute Equal to a Certain Value Exists in a Page

I am using Python and Selenium to scrape a web page.
I am trying to find the following 'next page' button in the page:
<button class="pagination__next" aria-label="click to go to the next page" disabled="true" aria-hidden="true"></button>
disabled='true' appears when I'm on the last page of a multi-page embedded javascript generated table.
There may be more buttons on the page and some are disabled at the time I'm looking for this particular button.
So what I'm trying to do is determine if this particular button which has class="pagination__next" with attribute disabled equals 'true' is in driver.page_source
There are plenty of examples around about identifying the particular tag ('button'). But not how to find the button and the button attribute disabled equal to 'true'
I tried various approaches. I think the closes I got was:
driver.find_element_by_css_name('pagination__next[disabled='true'])
but I don't think it worked.
There is an identical question here but OP asks for Java solution. I'm looking for Python.
Guidance please.

find_element_by_xpath() with multiple attributes is what you need, i.e.:
driver.find_element_by_xpath('//*/button[#class="pagination__next"][#disabled="true"]')

Using .find_element_by_xpath with the AND expression:
driver.find_element_by_xpath('//button[#class="pagination__next" and #disabled="true"]')
With the AND expression you can locate element with more than one condition using the attribute value.
xpath-selenium OR & AND expression

Python with Selenium - Cannot find and click this specific element due to randomization of it's location on the site

I've been creating a tool that plays through an online game using python 2.7 and selenium, and am very stuck on one particular element I need to select.
The UI looks as follows:
1 2 3
a d g
b e h
c f i
The numbers one two and three represent a drop down menu, which when clicked open up the letters. Each option represents a different outcome. The problem is, at the start of each game the positions of both the numbers and letters are randomized. In the code, each button's css selector is labeled as for example "#action-1 > button:nth-child(1)", as in the "first button", but the "first button" will be different every game.
I've tried finding by link text, xpath, and css selector to no success.
If it helps, pressing "copy outer html" gives this:
<button class="ng-binding" ng-click="subBtn($event)" ng-class="{disabled : !state.chapterStart || state.btns.indexOf(btn.action) != -1}" ng-disabled="!state.chapterStart || state.btns.indexOf(btn.action) != -1">Wait and See What They Do</button>
The "Wait and See What They Do" part of it is what the button says and is how you know what to click, but I can't seem to find the element by that.
Copying the xpath results in //*[#id="action-1"]/button, which again isn't really helpful because telling it to click that would just be like saying "click the button that is in the first position", and it changes every time. I've added long time.sleep() commands to test, the page is definitely loading all the way so that is not the issue.
Anyways, I've been trying to figure this one out for way too long and it has me stumped. I'd be very grateful for any input you all can give. Thank you!

If you are looking to locate an element by the text it contains you can use XPath. You stated that you tried XPath but no specifics were given. Did you try the below? It should work given the HTML you provided.
//button[text()='Wait and See What They Do']
To read this XPath... find any descendant // that is a BUTTON tag that has an attribute [] where the text contained in the element is equal to the search string, text()=''.
Another example might be the below which is an alternate way to find your button. The problem with this way is that there may be many buttons on the page with that class so it may not be specific enough.
//button[#class='ng-binding']
I've found (and others have also) that XPaths are generally slower than other location methods. Because of this, I generally prefer By.Id, when available, then By.CssSelector. They are both significantly faster than XPath. I save XPath for things like finding text in an element or finding relative elements (CSS can do this some but is no where near as powerful as XPath).
XPath Examples