Pull data from a div class Python Selenium - python

I'm trying to pull a specific number out of a div class in Python Selenium but can't figure out how to do it. I'd want to get the "post_parent" ID 947630 as long as it matches the "post_name" number starting 09007.
I'm looking to do this across multiple "post_name" classes, so I'd feed it something like this: search_text = "0900766b80090cb6", but there will be multiple in the future so it has to read the "post_name" first then pull the "post_parent" if that makes sense.
Appreciate any advice anyone has to offer.
<div class="hidden" id="inline_947631">
<div class="post_title">Interface Converter</div>
<div class="post_name">0900766b80090cb6</div>
<div class="post_author">28</div>
<div class="comment_status">closed</div>
<div class="ping_status">closed</div>
<div class="_status">inherit</div>
<div class="jj">06</div>
<div class="mm">07</div>
<div class="aa">2001</div>
<div class="hh">15</div>
<div class="mn">44</div>
<div class="ss">17</div>
<div class="post_password"></div>
<div class="post_parent">947630</div>
<div class="page_template">default</div>
<div class="tags_input" id="rs-language-code_947631">de</div>
</div>

If you see <div class="post_name">0900766b80090cb6</div> this and <div class="post_parent">947630</div> are siblings nodes to each other.
You can use xpath -> following-sibling like this:
Code:
search_text = "0900766b80090cb6"
post_parent_num = driver.find_element(By.XPATH, f"//div[#class='post_name' and text()='{search_text}']//following-sibling::div[#class='post_parent']").text
print(post_parent_num)
or Using ExplicitWait:
search_text = "0900766b80090cb6"
post_parent_num = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, f"//div[#class='post_name' and text()='{search_text}']//following-sibling::div[#class='post_parent']"))).get_attribute('innerText')
print(post_parent_num)
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Update:
NoSuchElementException:
Please check in the dev tools (Google chrome) if we have unique entry in HTML-DOM or not.
xpath that you should check :
//div[#class='post_name' and text()='0900766b80090cb6']//following-sibling::div[#class='post_parent']
Steps to check:
Press F12 in Chrome -> go to element section -> do a CTRL + F -> then paste the xpath and see, if your desired element is getting highlighted with 1/1 matching node.
If this is unique //div[#class='post_name' and text()='0900766b80090cb6']//following-sibling::div[#class='post_parent'] then you need to check for the below conditions as well.
Check if it's in any iframe/frame/frameset.
Solution: switch to iframe/frame/frameset first and then interact with this web element.
Check if it's in any shadow-root.
Solution: Use driver.execute_script('return document.querySelector to have returned a web element and then operates accordingly.
Make sure that the element is rendered properly before interacting with it. Put some hardcoded delay or Explicit wait and try again.
Solution: time.sleep(5) or
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='post_name' and text()='0900766b80090cb6']//following-sibling::div[#class='post_parent']"))).text
If you have redirected to a new tab/ or new windows and you have not switched to that particular new tab/new window, otherwise you will likely get NoSuchElement exception.
Solution: switch to the relevant window/tab first.
If you have switched to an iframe and the new desired element is not in the same iframe context then first switch to default content and then interact with it.
Solution: switch to default content and then switch to respective iframe.

I don't see any specific relation between "post_parent" ID 947630 and "post_name" number starting 09007. Moreover, the parent <div> is having class="hidden".
However, to pull the specific number you can use either of the following locator strategies:
Using css_selector:
print(driver.find_element(By.CSS_SELECTOR, "div[id^='inline'] div.post_parent").text)
Using xpath:
print(driver.find_element(By.XPATH, "//div[starts-with(#id, 'inline_')]//div[#class='post_parent']").text)
Ideally you need to induce WebDriverWait for the presence_of_element_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
print(WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "div[id^='inline'] div.post_parent"))).text)
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//div[starts-with(#id, 'inline_')]//div[#class='post_parent']"))).text)
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can create a method and use the following xpath to get the post_parent text based on post_name text.
def getPostPatent(postname):
element=driver.find_element(By.XPATH,"//div[#class='post_name' and starts-with(text(),'{}')]/following-sibling::div[#class='post_parent']".format(postname))
print(element.get_attribute("textContent"))
getPostPatent('09007')
This will return value if it is matches the text starts-with('09007')
It seems parent class is hidden you need to use textContent to get the value.

Related

Selenium unable to locate xpath however the xpath is there in the browser

Im not sure if that was quite the best title but Im not sure how else to describe it, I am trying to use selenium to automate 2fa on a website so all I need to do is anwser the phone call and the script will take care of the rest however the button I am trying to get selenium to click keeps showing up as unable to locate even though its always in the same place and never changes here is my code in python
callMe = driver.find_element('xpath', '//*[#id="auth_methods"]/fieldset/div[2]/button')
callMe.click()
sleep(25)
this is one of three buttons that all have the same element information besides the xpaths here are all 3 button elements I am trying to grab the second one
<button tabindex="2" type="submit" class="positive auth-button"><!-- -->Send Me a Push </button>
<button tabindex="2" type="submit" class="positive auth-button"><!-- -->Call Me </button>
<button tabindex="2" type="submit" class="positive auth-button"><!-- -->Text Me </button>
I am not sure how else I can locate the second button besides using the xpath but that is not working and I dont know if I can or how I can search for the button based off the text that is inside it.
Did you try with By ?
from selenium.webdriver.common.by import By
callMe = driver.find_element(By.XPATH, '//*[#id="auth_methods"]/fieldset/div[2]/button')
Try using By.cssselector, get the css selector of the main body html for the button you want.
callMe = driver.find_element(By.css_selector, 'selectorofbodyhtml')
callme.click()
Try below xpath
//button[starts-with(text(),'Call Me')]
The desired element Call Me is a dynamic element, so to click on it you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using XPATH and contains():
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(., 'Call Me')]"))).click()
Using XPATH and starts-with():
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[starts-with(., 'Call Me')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Python Selenium - clicking Send button 'not clickable

I have a python script where I need to click a button. My function is as follows:
def inviteuser():
invitebutton.click()
time.sleep(2.5)
addressbox = driver.find_element_by_xpath('/html/body/div[9]/div/div/div[2]/div/div[1]/div/div/div/div/div[3]/div/div/div[1]')
time.sleep(2.5)
addressbox.send_keys(email)
time.sleep(2.5)
sendbutton = driver.find_element_by_xpath('/html/body/div[8]/div/div/div[3]/div[2]')
sendbutton.click()
When running the script at the button clicking part, I get this message:
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <div class="c-sk-modal_footer_actions"> is not clickable at point (834,677) because another element <div class="ReactModal__Overlay ReactModal__Overlay--after-open c-popover c-popover--z_above_fs c-popover--fade"> obscures it
I tried searching for that div, but the search in the browser could not find it.
I also tried
driver.find_element_by_css_selector('.c-button .c-button--primary .c-button--medium').click()
HTML code of the items
<div class="c-sk-modal_footer_actions">
<button class="c-button c-button--primary c-button--medium c-button--disabled" data-qa="invite-to-workspace-modal-invite-form-send-button" type="button" aria-disabled="true">
"Send"
::after
</button>
</div>
If it helps at all, this is for the invite people box in slack admin portal
EDIT:
So I basically figured out the issue but can't figure out how to fix the issue...
So just using the variable sys.argv[1] puts in the email address, but I need to either press space bar , comma, or enter key after. I can get it to work if I specify what the variable email is (email = "test#test.com" then confirm = " ") and adding a second line addressbox.send_keys(confirm) but if I make the variable what I need it to be so it's called from powershell (sys.argv[1]) It doesn't work. It's like it removes what I entered and only puts what's in the variable "confirm"
Try the execute script method which can have a better chance when getting that error. Also you could use implicitly wait instead of time waits to be more effecient.
driver.implicitly_wait(5)
addressbox = driver.find_element_by_xpath('/html/body/div[9]/div/div/div[2]/div/div[1]/div/div/div/div/div[3]/div/div/div[1]')
driver.execute_script("arguments[0].click();", addressbox)
Generally <div> elements aren't clickable. Presumably you need to crosscheck if the following element is your desired element:
<div class="c-sk-modal_footer_actions">
Where as, the element from your second attempt:
<div class="c-button c-button--primary c-button--medium...">
looks as a perfect clickable candidate.
Solution
The desired element is a React element, so to click() on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".c-button.c-button--primary.c-button--medium"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[#class='c-button c-button--primary c-button--medium']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

How to write a value for input tag created by ion-input using selenium Python

I have ion-input tags in my app and that creates another input tag as a sibling. The sibling input is responsible for any input value.
I want to access that sibling to enter a value using selenium Python (can be using send_keys or using javascript_executor
<ion-input data-cy="email" type="email" debounce="50" value="" class="sc-ion-input-md-h sc-ion-input-md-s md">
<input class="native-input sc-ion-input-md" autocapitalize="off" autocomplete="off"
autocorrect="off" name="ion-input-0" placeholder="" spellcheck="false" type="email"></ion-input>
Everytime I use that data-cy="email" I am getting elementNotFound exception only.
I am using Selenium python.
The element should be unique in nature, based on the HTML that you've shared, can you check this CSS or XPATH
XPATH:
//input[#class='native-input sc-ion-input-md' and #type='email' and#name='ion-input-0']
CSS_SELECTOR:
input[class='native-input sc-ion-input-md'][type='email'][name='ion-input-0']
Please check in the dev tools (Google chrome) if we have unique entry in HTML DOM or not.
Steps to check:
Press F12 in Chrome -> go to element section -> do a CTRL + F -> then paste the xpath/css and see, if your desired element is getting highlighted with 1/1 matching node.
If they happen to be unique in nature you can try with the below code:
Code1:
wait = WebDriverWait(driver, 30)
input = wait.until(EC.visibility_of_element_located((By.XPATH, "//input[#class='native-input sc-ion-input-md' and #type='email' and#name='ion-input-0']")))
input.send_keys('the string that you want to send')
Code2:
wait = WebDriverWait(driver, 30)
input = wait.until(EC.visibility_of_element_located((By.XPATH, "//input[#class='native-input sc-ion-input-md' and #type='email' and#name='ion-input-0']")))
driver.execute_script("arguments[0].setAttribute('value', 'the string that you want to send')", input)
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I don't any such major issues in your code attempts. However the the desired element is a dynamic element, so ideally to send a character sequence to the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "ion-input[data-cy='email'] > input[type='email'][name^='ion-input']"))).send_keys("Sarath#N.com")
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//ion-input[#data-cy='email']/input[#type='email' and starts-with(#name, 'ion-input')]"))).send_keys("Sarath#N.com")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
It was so easy. I have been using either XPATH or CSS_SELECTOR hence always it does not penetrate to the input tag that was created by ion. Instead of the bespoke one, i have used TAGNAME and that did the magic
email = [(By.XPATH, "ion-input[data-cy='email'] input")]
Instead use like this
email = [(By.TAGNAME, "ion-input[data-cy='email']")]

Finding web element of dynamic websites using selenium python

I want to scrape text of few fields on the basis of their web elements (xpath, classes etc).
<div class = myOnlyElement>
<div> ......
<div class = afafasf> ......</div>
<div class = klklkl> ......
<div class = qwqwqwq> ......
<div class = reaction> text i need</div>
</div>
</div>
</div>
</div>
<div class = myElement>
<div> ......
<div class = dfdfdf> ......</div>
<div class = ghgghghg> ......
<div class = erererere> ......
<div class = reaction> text i don't need</div>
</div>
</div>
</div>
</div>
Suppose I have backend of element like this. I find element like:
myelem = driver.find_element_by_classname('myOnlyElement')
Now I only want to pick class "reaction" with text I need.
I am doing like:
myelem.find_element_by_classname('reaction')
if this class is present it captures it, but in some cases it goes for class = "reaction" whose text is "text i don't need"
Hope I have clearly mentioned my question. Can you please help me
my friend, best solution when it comes to this stuff, right click on the webpage, where you see the text. Right click in the DOM inspector and click Copy -> Copy Full XPath value. then you might need to do .text .source to get those values. but try and play around.
To print the text text i need you can use either of the following Locator Strategies:
Using css_selector and get_attribute():
print(driver.find_element_by_css_selector("div.myOnlyElement div.reaction").get_attribute("innerHTML"))
Using xpath and text attribute:
print(driver.find_element_by_xpath("//div[#class='myOnlyElement']//div[#class='reaction']").text)
Ideally, to print the text text i need you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and get_attribute():
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.myOnlyElement div.reaction"))).get_attribute("innerHTML"))
Using XPATH and text attribute:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='myOnlyElement']//div[#class='reaction']"))).text)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
Outro
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

How to click on the angular based checkbox as per the HTML provided through Selenium and Python?

I am trying to click on a checkbox using selenium python. I tried
buttons = driver.find_element_by_xpath("//*[contains(text(), 'Exact')]"); buttons.click()
I keep getting
"ElementNotVisibleException:"
<button type="button" data-ng-class="{iconCheck: event.locationExactness.isExact, inputBox:!event.locationExactness.isExact}" class="link icon locationButton inputBox" data-sfs-callout-visible="relativeExactnesses.length > 1" data-sfs-callout="sfs_-sfsLocationExactness-1-place-callout" data-sfs-callout-focus="sfs_-sfsLocationExactness-1-exact" data-ng-click="updateIsExact(relativeExactnesses.length > 1 ? true : !event.locationExactness.isExact)" data-autoname="NameAPlace_msypn_LocationExactButton"><!--
--><span class="locationLabel ng-binding">Exact</span><!--
--></button>
<<pseudo:before>></<pseudo:before>>
<!--
-->
<span class="locationLabel ng-binding">Exact</span>
<!--
-->
First of all you are locating one element, but are giving it plural name buttons. I would use singular name button.
Second thing is even in your code snippet there is two span which contain Exact. You have to change your locator so, that only one element(which you want to interact with) will be selected via selector.
For example, if you want the first Exact in your code snippet, you can use this xPath:
//button[#data-sfs-callout = 'sfs_-sfsLocationExactness-1-place-callout']/span
Note: as #RajKamal already mentioned, there can be multiple elements on the page with text Exact. You can check this in dev tools by pressing F12.
As per the HTML you have shared the element with text as Exact is a Angular element so to invoke click() on the desired element you have to induce WebDriverWait for the element to be clickable and you can use the following solution:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#class='link icon locationButton inputBox' and #data-autoname='NameAPlace_msypn_LocationExactButton']//span[#class='locationLabel ng-binding'][contains(.,'Exact')]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Categories

Resources