I have the following code in Python/Selenium:
try:
main = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "main"))
)
print(main.text)
except:
driver.quit()
And expecting a print statement but the "try" block seems to fail. I have all the right packages imported and what not but no dice. I am following along with a tutorial and everything has been working fine up to this point. Am happy to post the file thus far if needed. But just wondering why it is failing every pass. Any input is appreciated! Thanks!
EDIT: So I'm just tryna fetch/print the main content of the page given by:
<main id="main" class="site-main" role="main">...</main> ==$0
I have basically zero knowledge of html and I am just trying to follow along with a tutorial so I have no idea why its throwing errors.
Have you tried this:
print(main.get_attribute("innerHTML"))
or
print(main.get_attribute("outerHTML"))
Related
So I am trying to find elements by CSS selector using the class name. One of my code works while the other does not.
Screenshot of the html code that works:
The code for the above screenshot, that works:
driver.get(url)
print('a') # to check if code has run till here
disabled_next = driver.find_elements(By.CSS_SELECTOR, '.page-link.next.disabled')
print(disabled_next)
Screenshot of the html code that does NOT work:
The code for the above screenshot, that does not work:
driver.get(url)
print('a')
enabled_next = driver.find_elements(By.CSS_SELECTOR, '.page-link.next')
print(enabled_next)
Just trying to understand why the second one does not work. It does not even print ('a'). The error I am getting is along the lines of this:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot determine loading status
from unknown error: unexpected command response
(Session info: chrome=103.0.5060.66)
I am using chromedriver103 with chrome version 103. I know there have been some issues relating to it. Could that be why? Or have I used the wrong format for '.page-link.next'.
Also, a semi-related question: How do I grab the text after <a class = "text here"? I would like to just grab "text here". So in my examples above I would love to grab "page-link next disabled" or "page-link next".
Thank you in advance!
This is not related to your code at all, your code should do fine. The error message pertains to the chromedriver version and the current chrome browser you are using. Try to double check the chromedriver and chrome browser you are using for testing.
cannot determine loading status
I am trying to print by ID using Selenium. As far as I can tell, "a" is the tag and "title" is the attribute. See HTML below.
When I run the following code:
print(driver.find_elements(By.TAG_NAME, "a")[0].get_attribute('title'))
I get the output:
Zero Tolerance
So I'm getting the first attribute correctly. When I increment the code above:
print(driver.find_elements(By.TAG_NAME, "a")[1].get_attribute('title'))
My expected output is:
Aaliyah Love
However, I'm just getting blank. No errors. What am I doing incorrectly? Pls don't suggest using xpath or css, I'm trying to learn Selenium tags.
HTML:
<a class=" Link ScenePlayer-ChannelName-Link styles_1lHAYbZZr4 Link ScenePlayer-ChannelName-Link styles_1lHAYbZZr4" href="/en/channel/ztfilms" title="Zero Tolerance" rel="">Zero Tolerance</a>
...
<a class=" Link ActorThumb-ActorImage-Link styles_3dXcTxVCON Link ActorThumb-ActorImage-Link styles_3dXcTxVCON" href="/[RETRACTED]/Aaliyah-Love/63565" title="Aaliyah Love"
Selenium locators are a toolbox and you're saying you only want to use a screwdriver (By.TAG_NAME) for all jobs. We aren't saying that you shouldn't use By.TAG_NAME, we're saying that you should use the right tool for the right job and sometimes (most times) By.TAG_NAME is not the right tool for the job. CSS selectors are WAY more powerful locators because they can search for not only tags but also classes, properties, etc.
It's hard to say for sure what's going on without access to the site/page. It could be that the entire page isn't loaded and you need to add a wait for the page to finish loading (maybe count links expected on the page?). It could be that your locator isn't specific enough and is catching other A tags that don't have a title attribute.
I would start by doing some debugging.
links = driver.find_elements(By.TAG_NAME, "a")
for link in links:
print(link.get_attribute('title'))
What does this print?
If it prints some blank lines sprinkled throughout the actual titles, your locator is probably not specific enough. Try a CSS selector
links = driver.find_elements(By.CSS_SELECTOR, "a[title]")
for link in links:
print(link.get_attribute('title'))
If instead it returns some titles and then nothing but blank lines, the page is probably not fully loaded. Try something like
count = 20 # the number of expected links on the page
link_locator = (By.TAG_NAME, "a")
WebDriverWait(driver, 10).until(lambda wd: len(wd.find_elements(link_locator)) == count)
links = driver.find_elements(link_locator)
for link in links:
print(link.get_attribute('title'))
im new to Python, and everything that i know i learned by myself so i have a huge gap in some things like language structure, it would be awsome if someone could help me.
Im building something with selenium and openpyxl and i face a problem, if it doesnt find an element the code stops there, and thats exactly what im trying to bypass without any success. I need to continue if the element not found.
Heres a piece of my currently code, can someone help me how can i do that?
Thanks
#distin_city varia
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[11]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/div[1]/span[1]"))).click()
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[11]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/input[1]")))
element.send_keys(destin_city[:3])
time.sleep(0.2)
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[11]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/ul[1]/li[1]/div[3]/span[1]/div[1]"))).click()
print("DESTINATION CITY - ",destin_city)
#f_payment varia
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[12]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/div[1]/span[1]/span[1]"))).click()
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[12]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/input[1]")))
element.send_keys(f_payment[:3])
time.sleep(0.2)
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[12]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/ul[1]/li[1]/div[3]/span[1]/div[1]"))).click()
print("FREIGHT PAYMENT TYPE - ",f_payment)
#currency varia
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[13]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/div[1]/span[1]"))).click()
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[13]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/input[1]")))
element.send_keys(currency[:1])
time.sleep(0.2)
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[13]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/ul[1]/li[1]/div[3]/span[1]/div[1]"))).click()
print("CURRENCY - ",currency)
In order to simply continue with the flow in case the element is not found you can use try-except block with pass in except section.
For example if you want to apply that on
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[11]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/div[1]/span[1]"))).click()
You can do this:
try:
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"/html[1]/body[1]/div[1]/div[1]/div[1]/div[2]/form[1]/ivd-form-field[11]/div[1]/div[1]/ivd-field[1]/div[1]/ivd-select-box-field[1]/div[1]/div[1]/div[1]/span[1]"))).click()
except Exception:
pass
What you're looking for is try, except. Say you want one or multiple lines to be checked. If anything failed there you can do something else (Or in your example do nothing and pass)
You wrap your line(s) with try, except.
try:
# This is your code
Do Something 1
Do something 2
except:
# This is your code when any line above fails. You can use `pass` to do nothing
Do something else
SCREENSHOT OF THE HTML
Here is the screenshot of html code with which I am struggling , I want to click on the Smart Watches in the left-Nav and I am using the following code to click on it
driver.implicitly_wait(30)
driver.find_element_by_link_text('Smart Watches').click()
But I am getting the following error and I am clueless why i just cant find it on page
selenium.common.exceptions.NoSuchElementException: Message: no such
element: Unable to locate element: {"method":"link
text","selector":"Smart Watches"} (Session info:
chrome=60.0.3112.113) (Driver info: chromedriver=2.29.461591
(62ebf098771772160f391d75e589dc567915b233),platform=Windows NT
6.2.9200 x86_64)
I have also tried the explicit code and Expected conditions as follows -:
wait = WebDriverWait(driver, 20)
link = wait.until(expected_conditions.presence_of_element_located((By.LINK_TEXT,'"Smart Watches"')))
link.click()
But even its giving me Timeout exception
Here is the link of the page where I am stuck since morning
https://www.kogan.com/au/shop/phones/
I am very new to coding , any help would be helpful !! I just want to know why find_element_by_link_text is not working here , it looks weird to me!!
Thanks in advance
The problem is that when you use find_element_by_link_text(), it must be an exact match to the text contained in the link. In your HTML picture, you can see "Smart Watches" but what you aren't seeing is the SPAN just below but still inside the A is closed. Most likely if you expand it, you will see additional text that you must include if you are going to use find_element_by_link_text().
Another option is find_element_by_partial_link_text() which is more like a contains() instead of equals(). Depending on the page, it may find too many matches. You would have to try and see if it works.
Yet another option is using an XPath. There are a lot of different ways to create an XPath for this depending on exactly what you want.
This is the most general and thus most likely to find unwanted links but it may work. It's pretty much the same as find_element_by_partial_link_text()
//a[contains(.,'Smart Watches')
Other options include
//a[starts-with(.,'Smart Watches')
//li[#data-filter-facet='smart-watches']/a[contains(.,'Smart Watches')
//li[#data-filter-facet='smart-watches']/a[starts-with(.,'Smart Watches')
... and so on...
You can try this way...
Accessing text: find_element_by_xpath["//a[contains(text(), 'Smart Watches')]"].click()
Don't know why it is not wotks but partial link text works. Please see my java code for the same:
WebDriver driver=new FirefoxDriver();
driver.get("https://www.kogan.com/au/shop/phones/");
WebElement watch=driver.findElement(By.partialLinkText("Smart Watch"));
WebDriverWait waitElement=new WebDriverWait(driver, 30);
waitElement.until(ExpectedConditions.elementToBeClickable(watch));
watch.click();
You need to add double quotation marks, as it is in the html code:
driver.find_element_by_link_text('"Smart Watches"').click()
most of the time it might happen that link is taking some more time to load after your page has loaded. rather than using implicit wait use explicit wait.
wait = WebDriverWait(driver, 30)
link = wait.until(expected_conditions.presence_of_element_located((By.LINK_TEXT,"Smart Watches")))
link.click()
it could also be the case that link is inside another frame, in this case you will have to switch to this frame.
I am trying to get user details from each block as given
driver.get("https://www.facebook.com/public/karim-pathan")
wait = WebDriverWait(driver, 10)
li_link = []
for s in driver.find_elements_by_class_name('clearfix'):
print s
print s.find_element_by_css_selector('_8o._8r.lfloat._ohe').get_attribute('href')
print s.find_element_by_tag_name('img').get_attribute('src')
it says:
unable to find element with css selector
any hint appreciable.
Just a mere guess based on assumption that you are not logged in. You are getting exception cause for all class clearfix, element with ._8o._8r.lfloat._ohe does not exists. So your code isn't reaching the required elements. Anyhow, if you are trying to fetch href and img source of results, you need not iterate over all clearfix cause as suggested by #leo.fcx, your css is incorrect, trying the css provided by leo, you can achieve the desired result as:
driver.get("https://www.facebook.com/public/karim-pathan")
for s in driver.find_elements_by_css_selector('._8o._8r.lfloat._ohe'): // there didn't seemed to iterate over each class of clearfix
print s.get_attribute('href')
print s.find_element_by_tag_name('img').get_attribute('src')
P.S. sorry for any syntax, never explored python binding :)
Since you are using all class-names that the element applies, adding a . to the beginning of your CSS selector should fix it.
Try this:
s.find_element_by_css_selector('._8o._8r.lfloat._ohe')
instead of:
s.find_element_by_css_selector('_8o._8r.lfloat._ohe')
Adding to what #leo.fcx pointed about the selector, wait for search results to become visible:
wait.until(EC.visibility_of_element_located((By.ID, "all_search_results")))