I'm using selenium, with find_element_by_path method to do some web scraping, I have some problem to get a path which change through pages, I know how the path is written, but one of the string within the path change through my loop, I would like to know how can I use regex to solve it.
I have this code for one of the page but when I go through all pages the string "NUMBER" below changes:
browser.find_element_by_xpath(re.compile('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[NUMBER]/div').click()
I want to know if it was possible to use regex in order to say that it has to click whatever the "NUMBER" as long as the rest of the path is the same so I tried this but I'm not sure about the syntax and how to use regex here:
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div['). + re.compile("^[1-9]\d*$") + ']/div').click()
browser.find_element_by_xpath(re.compile('^//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[')).click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[1]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[9]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[4]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[10]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[6]/div').click()
the path evolves more or less in this manner (randomly) but not gradually one by one.
How do I solve this problem?
Welcome to SO.
If you are trying to pass the NUMBER as part of xpath in your loop then you can do the below.
If NUMBER in an integer:
browser.find_element_by_xpath("//*#id='exhibDetail:exhib']/section[3]/div[2]/div/div[2]/div/div/div[%i]/div"%(NUMBER)).click()
If NUMBER is a string
browser.find_element_by_xpath("//*#id='exhibDetail:exhib']/section[3]/div[2]/div/div[2]/div/div/div[%s]/div"%(NUMBER)).click()
I want to know if it was possible to use regex in order to say that it
has to click whatever the "NUMBER" as long as the rest of the path is
the same
If you want to select those div elements disregarding their position (that is what the predicates [1], [2], etc. are testing) then just don't use the predicates at all:
//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div/div
Related
I've just started learning python and I'm using selenium to interact with the web pages.
I have to repeat a specific action until a certain element is present in the HTML code, so I wrote something like this:
\\\\\\\\\\\\\\\\\\\
t = driver.find_element_by_id("i3")
t.send_keys(Codice)
while driver.find_element_by_id("i3")==t:
time.sleep(5)
t.send_keys(Keys.RETURN)
\\\\\\\\\\\\\\\\\\\\\\
But it doesn't work because it doesn' stop at the end so I wanted to know what does the function driver.find_element_by_id("i3") returns when it doesn't find the element.
Also, is there a more appropriate cycle for doing this?
Thank you and sorry for the bad english
Your code is unclear, what I can say here is the while loop to validate the element is presented can be done by
while driver.find_elements_by_id("i3")
Since driver.find_elements_by_id return list of web elements. In case elements found list in not empty and true, in case no elements found the list is empty and is false.
I have a page with a list of cards with information.
The XPATHS of each cards are:
self.automatic_payments_cards_list = (By.XPATH , '//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info')
I'm trying to get the text of a specific elements for every card in the page.
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[1]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[2]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[3]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]
I know that with this code i get all the text on each card
for i in range(len(self.driver.find_elements(*self.automatic_payments_cards_list))):
print(self.driver.find_element(*self.automatic_payments_cards_list)[i].text)
But i don't want to get all the text on the cards, only the text on this specifics XPATHS
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[**X**]**/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]**
Can you guys guide me in finding a solution to this?
The best way to actually achieve this is by using find element_by_xpath on the element.
all_card_els = driver.find_elements_by_xpath('//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info')
for card_el in all_card_els:
specific_el_within_card = card_el.find_element_by_xpath('.//lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]')
The . at the starting of the xpath is essential to make sure that the search is within the selected el, without the . you will always end up getting the first el which matches this xpath on the page. You can now use specific_el_within_card however way you like inside the loop, or append it to an external list.
PS: You can access the text via specific_el_within_card.text() as you mentioned you wanted to extract info for each card.
It is simple.:-) Transfer your xpath as dynamical string and pass it, like you do in a loop for e.g.
parent_locator_String=string1+ iterator + string2
"//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info["+i+"]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]"
I am working on a script that aims "taking all of the entries which are written by users" under a specific title in a website(In this case, the title is "python(programlama dili"). I would like to read the number which shows the current number of pages under this specific title.
The reason behind reading the number of this elements is that number of pages can increase at the time when we run the script due to increasing number of entries by users. Thus, I should take the number which exist within the element via script.
In this case, I need to read "122" as the value and assign it to a int variable . I use Selenium to take all entries and Firefox web driver.
It Would be better if you try to access it using the xpath.
Try and get the value attribute of the element, you've mentioned you can find the element using xpath so you can do the following.
user_count = element.get_attribute('value')
If that gets you the number (as a string) then you can just convert to an int as usual
value = int(user_count)
First pick the selector .last and then you can extract the reference of that. Don't forget to split the reference.
my_val = driver.find_element_by_css_selector(".last [href]").split("p=")[1]
I am a newer in Selenium and use python to build it. Recently, I found a question which want to ask someone who can help me to figure it. The question is the Xpath I want to get is randomly, for example:
'//*[#id="wiki-edit-wikiEdit26"]/div/div/div/div[2]/a[1]'
'//*[#id="wiki-edit-wikiEdit27"]/div/div/div/div[2]/a[1]'
'//*[#id="wiki-edit-wikiEdit28"]/div/div/div/div[2]/a[1]'
These three xpath are used on the same button, but the number after wikiEdit will be changed every time. Therefore, are there any way which can help me to run my script more smoothly? Thank you very much!
Here is my python code:
broswer.find.element_by_xpath('//*[#id="wiki-edit-wikiEdit26"]/div/div/div/div[2]/a[1]') .click()
You can use matches in xpath to do this,
broswer.find.element_by_xpath("//*[matches(#id, '^(wiki-edit-wikiEdit)[0-9]')]/div/div/div/div[2]/a[1]") .click()
so basically that matches the id anything starting wiki-edit-wikiEdit followed by numbers form [0-9]
You just need to format your string..
import random
# Random range 1 - 100
x = random.randint(1,100)
broswer.find.element_by_xpath(f'//*[#id="wiki-edit-wikiEdit{x}"]/div/div/div/div[2]/a[1]') .click()
You can use starts-with or contains
broswer.find.element_by_xpath("//*[contains(#id,'wiki-edit-wikiEdit')]/div/div/div/div[2]/a[1]").click()
Also, using such long xpath's is not recommended. Use css-selectors over xpath.
I am a newbie to Selenium and is implementing selenium-rc with Python client library. I tried traversing through my page's div using xpath(s) elements using the command "sel.get_xpath_count(xpath)".
It gives a count of 20, but when I iterate through every div using for statement and command "sel.get_text(xpath='%s[%d]'%(xpath, i))", but it only finds the first element and give a error on the remaining 19 saying divs not found.
Your second XPath expression is wrong. Programmers trained in C-style languages frequently make this mistake, because they see [...] and think "index into an array", but that's not what brackets do in XPath.
If you use sel.get_xpath_count(something), then you need to use sel.get_text("xpath=(something)[item_number]"). Note the use of parentheses around the original XPath expression in the second use.
The reason behind this is that something[item_count] is short-hand for something AND position() = item_count - thus you wind up adding another predicate to the "something" expression, instead of selecting one of the nodes selected by the expression. (something)[item_count] works because the value of (something) is a list of nodes, and adding a position() = item_count selects the node from the list with the specified position. That's more like a C-style array.