Thats my code, I was changing, fixing, trying another modules, but still can not get a respond with my div text extracted.
import selenium
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.helloworld.com/')
element = driver.find_element_by_id('main')
WebElement = driver.findElement(By.xpath("//div[#class='main']"));
webElement.getText();
I was trying with bs4 package but there is a big problem, becouse the data what i want is possible to get only when i am loged in on the website, and in bs4 respond was like from guest account, without login in.
Here is a Traceback what i got usint that code with Selenium:
Traceback (most recent call last):
File "D:/Python27/get text value div.py", line 8, in <module>
WebElement = driver.findElement(By.xpath("//div[#class='main']"));
AttributeError: 'WebDriver' object has no attribute 'findElement'
Aftter a small fix, I am using this:
import selenium
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.helloworld.com/')
element = driver.find_element_by_id('main')
main_text = element.text
The respond after print element.text in shell is:
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
element.text
File "D:\Python27\lib\selenium\webdriver\remote\webelement.py", line 50, in text
return self._execute(Command.GET_ELEMENT_TEXT)['value']
File "D:\Python27\lib\selenium\webdriver\remote\webelement.py", line 228, in _execute
return self._parent.execute(command, params)
File "D:\Python27\lib\selenium\webdriver\remote\webdriver.py", line 165, in execute
self.error_handler.check_response(response)
File "D:\Python27\lib\selenium\webdriver\remote\errorhandler.py", line 152, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: u'\'[JavaScript Error: "a is null" {file: "file:///c:/dokume~1/tomek/lokale%20einstellungen/temp/tmpupvgr2/extensions/fxdriver#googlecode.com/components/command_processor.js" line: 7623}]\' when calling method: [nsICommandProcessor::execute]'
Where did you get the following from? Looks like Java to me.
WebElement = driver.findElement(By.xpath("//div[#class='main']"));
webElement.getText();
Try:
import selenium
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.helloworld.com/')
element = driver.find_element_by_id('main')
print element.text #There's no text under div main, what would you expect?
footer = driver.find_element_by_id('footer')
print footer.text
# Should print out "Copyright ©2013 helloworld.com. All Rights Reserved. About Us | Privacy Policy "
If the information you want is inside a div with an id of botloc then you need to grab that element.
import selenium
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.helloworld.com/')
bot_location = driver.find_element_by_id('botloc').text
print bot_location
Related
I want to organize my urls for the testing project, so I tried to create a separate class or dictionary to store urls, but whenever I tried to call a "driver.get(pages.url)" method I get an error. Although if you pass the url from a variable or a string directly, everything works.
I can't understand where is the problem. Could someone help?
The code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
class Pages:
basic_url:str = 'google.com'
translate:str = 'translate.' + basic_url
maps:str = 'www.' + basic_url + '/maps'
pages = Pages()
print(pages.translate)
print(type(pages.translate))
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=Service(executable_path=ChromeDriverManager().install()),options=chrome_options)
wait = WebDriverWait(driver, 60)
driver.maximize_window()
driver.get(pages.translate)
The error:
'Traceback (most recent call last):
File "E:\Projects\Python\portal_UI_testing_automation\ptn.py", line 23, in <module>
driver.get(pages.translate)
File "E:\portal_UI_testing_automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 440, in get
self.execute(Command.GET, {'url': url})
File "E:\portal_UI_testing_automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 428, in execute
self.error_handler.check_response(response)
File "E:\portal_UI_testing_automation\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument
(Session info: chrome=106.0.5249.119)'
Print your url before passing it to driver.get to make sure its valid.
As some of the comments suggest try adding http/https.
I think you are missing https:// in the URL
This is the correct answer. Thanks Sighil, and others.
New to Python Selenium.
I am trying to create an script to login to my home router and press the button restart.
Running to error, when trying to login to the router, can some on guide on my mistake here.
below is the code and also attaching the .screenshot
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
driver_service = Service(executable_path="C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=driver_service)
PASSWORD = 'testtes'
login_page = 'http://192.168.2.1/login.html'
driver.get(login_page)
driver.find_element_by_xpath("//input[#placeholder='Password']").send_keys(PASSWORD)
Below is the error I am getting.
Traceback (most recent call last):
File "C:\Users\admin\Desktop\pyhton\index.py", line 14, in
driver.find_element_by_xpath("//input[#placeholder='Password']").send_keys(PASSWORD)
AttributeError: 'WebDriver' object has no attribute 'find_element_by_xpath'
getting this error now.
Traceback (most recent call last):
File "C:\Users\admin\Desktop\pyhton\index.py", line 13, in
driver.find_element(By.XPATH, "//input[#placeholder='Password']").send_keys(PASSWORD)
File "C:\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 856, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "C:\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//input[#placeholder='Password']"}
Probably you are using Selenium 4. if so, find_element_by_xpath and all the others find_element_by_* methods are not supported by Selenium 4, you have to use the new syntax and add an essential import, as following:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
driver_service = Service(executable_path="C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=driver_service)
PASSWORD = 'testtes'
login_page = 'http://192.168.2.1/login.html'
driver.get(login_page)
driver.find_element(By.XPATH, "//input[#placeholder='Password']").send_keys(PASSWORD)
Try this:
from selenium.webdriver.common.by import By
driver.find_element(By.XPATH, "//input[#placeholder='Password']").send_keys(PASSWORD)
Currently, I'm trying to scrape data from a Website (https://account.kaspersky.com/). Before I can read the data I need to login to the website.
But for some reason, it is not working. I read through the internet to get it to work, but unfortunately, I wasn't able to solve the issue.
import mechanicalsoup
import csv
import xlsxwriter
from time import sleep
# create stateful browser
browser = mechanicalsoup.StatefulBrowser(
soup_config={'features': 'lxml'},
raise_on_404=True,
user_agent='MyBot/0.1: mysite.example.com/bot_info',
)
# use browser to open link
browser.open("https://account.kaspersky.com/")
sleep(2)
# check url
print(browser.get_url())
# get first form available
form = browser.select_form()
browser.submit()
# check url
print(browser.get_url())
The script always ends at the selct_form() method. No matter what I try I always get the same error. Even when I specify it.
Traceback (most recent call last):
File "c:\Python\Cloud.Kaspersky\read_from_website_mechanicalsoup.py", line 23, in <module>
form = browser.select_form()
File "C:\Users\dw.FROMMEDV\AppData\Local\Programs\Python\Python39\lib\site-packages\
mechanicalsoup\stateful_browser.py", line 220, in select_form
raise LinkNotFoundError()
mechanicalsoup.utils.LinkNotFoundError
After a few hours of trying, I wanted to try a different tool. Selenium. But I have kind of the same problem here as well. But here I can't submit the login. Selenium can't find the button.
from selenium import webdriver
import time
driver = webdriver.Firefox()
driver.get("https://account.kaspersky.com/")
file = open('login.txt', 'r')
username_file = file.readline()
password_file = file.readline()
time.sleep(1)
username = driver.find_element_by_id("EMail")
username.clear()
username.send_keys(username_file)
time.sleep(1)
password = driver.find_element_by_name("Password")
password.clear()
password.send_keys(password_file)
time.sleep(2)
driver.find_element_by_class_name("assets-button primary").click()
Is it possible that this website is protected or something? Or is anyone seeing my issue?
Here the Error Message with Selenium:
Traceback (most recent call last):
File "c:\Python\Cloud.Kaspersky\read_from_website_selenium.py", line 26,
in
<module>
driver.find_element_by_class_name("assets-button primary").click()
File "C:\Users\dw.FROMMEDV\AppData\Local\Programs\Python\Python39
\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 564, in
find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Users\dw.FROMMEDV\AppData\Local\Programs\Python\Python39
\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in
find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\dw.FROMMEDV\AppData\Local\Programs\Python\Python39
\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\dw.FROMMEDV\AppData\Local\Programs\Python\Python39
\lib\site-
packages\selenium\webdriver\remote\errorhandler.py", line 242, in
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to
locate
element: .assets-button primary
Try using the following identifiers. The last line is the error using Java instead of Python and multiple class names instead of singular.
driver.find_element_by_class_name("assets-button").click()
driver.find_element_by_xpath("//button[#class='assets-button primary']").click()
You could also try
from selenium.webdriver.common.keys import Keys
password.send_keys(Keys.ENTER)
i am trying to post on facebook wall using selenium in python. I am able to login but after login it cant find class name of status box which i copied from browser
here is my code-
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
user_name = "email"
password = "password"
msg = "hi i am new here"
driver = webdriver.Firefox()
driver.get("https://www.facebook.com")
element = driver.find_element_by_id("email")
element.send_keys(user_name)
element = driver.find_element_by_id("pass")
element.send_keys(password)
element.send_keys(Keys.RETURN)
time.sleep(5)
post_box = driver.find_element_by_class_name("a8c37x1j ni8dbmo4 stjgntxs l9j0dhe7")
post_box.click()
time.sleep(5)
post_box.send_keys(msg)
the snapshot of code i copied from browser is attached as image here
here is error i recived-
Traceback (most recent call last):
File "C:/Users/rosha/Desktop/facebook bot.py", line 17, in <module>
post_box = driver.find_element_by_class_name("a8c37x1j ni8dbmo4 stjgntxs l9j0dhe7")
File "C:\ProgramData\Anaconda3\envs\facebook bot.py\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 564, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\ProgramData\Anaconda3\envs\facebook bot.py\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\ProgramData\Anaconda3\envs\facebook bot.py\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\ProgramData\Anaconda3\envs\facebook bot.py\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: .a8c37x1j ni8dbmo4 stjgntxs l9j0dhe7
try to find element by Xpath for example:
driver.find_element(By.XPATH, '//button[text()="Some text"]')
to find the xpath from the browser, just right click on something in the webpage and press inspect after that right click, a menu will appear, navigate to copy then another menu will appear, press copy fullpath.
check this https://selenium-python.readthedocs.io/locating-elements.html
The problem is that driver.find_element_by_class_name() can be used for one class, and not multiple classes as you have: a8c37x1j ni8dbmo4 stjgntxs l9j0dhe7 which are multiple classes separated by spaces.
Refer to the solution suggested here, it suggests using find_elements_by_xpath or find_element_by_css_selector.
I am using selenium bindings for python to download all links on a page that contain a string "VS". The problem is that the second item in the list is not a valid web page (returns 404 error), also if I manually click on the broken link it returns:
error.html - 404 error page does not exist.
And if I run the following code it raises an error.
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import StaleElementReferenceException, NoSuchElementException, NoSuchWindowException
# To prevent download dialog
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2) # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/msword, application/vnd.ms-powerpoint')
driver = webdriver.Firefox(profile)
driver.get("http://www.SOME_URL.com/")
links = driver.find_elements_by_partial_link_text("VS")
for link in links:
url = link.get_attribute("href")
try:
driver.get(url)
except StaleElementReferenceException:
pass
The error:
Traceback (most recent call last):
File "C:\Users\lskrinjar\Dropbox\work\preracun\src\web_data_mining\get_files_from_web.py", line 79, in <module>
url = link.get_attribute("href")
File "C:\Python27\lib\site-packages\selenium-2.44.0-py2.7.egg\selenium\webdriver\remote\webelement.py", line 93, in get_attribute
resp = self._execute(Command.GET_ELEMENT_ATTRIBUTE, {'name': name})
File "C:\Python27\lib\site-packages\selenium-2.44.0-py2.7.egg\selenium\webdriver\remote\webelement.py", line 385, in _execute
return self._parent.execute(command, params)
File "C:\Python27\lib\site-packages\selenium-2.44.0-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium-2.44.0-py2.7.egg\selenium\webdriver\remote\errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: Element not found in the cache - perhaps the page has changed since it was looked up
Stacktrace:
at fxdriver.cache.getElementAt (resource://fxdriver/modules/web-element-cache.js:8329:1)
at Utils.getElementAt (file:///c:/users/lskrin~1/appdata/local/temp/tmppsn7tm/extensions/fxdriver#googlecode.com/components/command-processor.js:7922:10)
at WebElement.getElementAttribute (file:///c:/users/lskrin~1/appdata/local/temp/tmppsn7tm/extensions/fxdriver#googlecode.com/components/command-processor.js:11107:31)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/lskrin~1/appdata/local/temp/tmppsn7tm/extensions/fxdriver#googlecode.com/components/command-processor.js:11635:16)
at fxdriver.Timer.prototype.setTimeout/<.notify (file:///c:/users/lskrin~1/appdata/local/temp/tmppsn7tm/extensions/fxdriver#googlecode.com/components/command-processor.js:548:5)
The links list is a list of elements on the webpage. Once you navigate away from that page, the elements no longer exists (you have a list where each element points to a no-longer-existing web element). Instead of referring to the list of elements, you should use a list of URL strings:
list_of_links = []
links = driver.find_elements_by_partial_link_text("VS")
for link in links:
list_of_links.append(link.get_attribute("href"))
for string_link in list_of_links:
try:
driver.get(string_link)
except StaleElementReferenceException:
pass