Selenium doesnt fetch url (3 lines of python) - python

Using chrome 76.0.3809.132 with chromedriver 76.0.3809.126 the testing browser opens but throws a traceback on the driver.get. It isn't able to fetch the page.
Tried couple of arguments, and tried looking for an older version of Chrome with an older chrome driver, which seems impossible.
from selenium import webdriver
driver = webdriver.Chrome("C:/chromedriver/chromedriver.exe")
driver.get("www.google.com")
Output:
DevTools listening on ws://127.0.0.1:57577/devtools/browser/7494ca24-60fb-4b14-9aed-4f25232b94bd
Traceback (most recent call last):
File "slidingwindow.py", line 5, in <module>
driver.get("www.google.com")
File "C:\projects\python\harvester\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "C:\projects\python\harvester\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\projects\python\harvester\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument (Session info: chrome=76.0.3809.132)

Try to insert https://:
from selenium import webdriver
driver = webdriver.Chrome(executable_path="chromedriver.exe")
driver.get("https://www.google.com")

Related

Unable to display website on chrome when ran the program

I had recently started learning automation with python using selenium. I had installed chromedriver but when i run the code chrome displays no output. My code:
from selenium import webdriver
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("www.wikipedia.com")
Output
Output :
C:\Users\Admin\PycharmProjects\untitled\venv\Scripts\python.exe C:/Users/Admin/PycharmProjects/untitled/1.py
Traceback (most recent call last):
File "C:/Users/Admin/PycharmProjects/untitled/1.py", line 5, in <module>
driver.get("www.wikipedia.com")
File "C:\Users\Admin\PycharmProjects\untitled\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "C:\Users\Admin\PycharmProjects\untitled\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Admin\PycharmProjects\untitled\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument
(Session info: chrome=88.0.4324.96)
Please help as I'm unable to proceed.
Thanks!
URL should contain scheme. Simply replace
"www.wikipedia.com"
with
"https://www.wikipedia.com"

Selenium error on Ubuntu EC2: "stale element reference: element is not attached to the page document"

Hello I have a simple python script just here :
#!/usr/bin/python3
# -*- coding: utf­-8 ­-*-
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.headless = True
options.add_argument("window-size=1400,800")
driver = webdriver.Chrome(options=options)
driver.get('https://www.coteur.com/cotes-foot.php')
#Store url associated with the soccer games
url_links = []
for i in driver.find_elements_by_xpath('//a[contains(#href, "match/cotes-")]'):
url_links.append(i.get_attribute('href'))
print(len(url_links), '\n')
print(url_links, '\n')
This script run perfectly on my Ubuntu PC but when I try to run it from a ec2 ubuntu server It does not work.
Here is my EC2 config:
DNS public (IPv4) : ec2-3-21-128-62.us-east-2.compute.amazonaws.com
ID d'AMI : ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200611 (ami-0a63f96e85105c6d3)
I indicate that google chrome and chromedriver are installed on my EC2 server.
When I run the script from EC2 I have this error message:
Traceback (most recent call last):
File "./test.py", line 17, in <module>
url_links.append(i.get_attribute('href'))
File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 141, in get_attribute
self, name)
File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
'args': converted_args})['value']
File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=83.0.4103.116)

[Python][Selenium]Unable to use webdriver with Chrome

I'm a beginner in Python and Selenium, and I dont know what is the error in my code or environment...
# encoding: utf-8
import time
from selenium import webdriver
from bs4 import BeautifulSoup
driver = webdriver.Chrome(executable_path=r'C:/Python27/Scripts/chromedriver')
time.sleep(3)
driver.get('https://www.google.com.tw/')
for i in range(10):
driver.execute_script('window.scrollTo(0, document.body.scrollHeight);')
time.sleep(1)
print ("Scrolling...")
driver.close()
And here is error messages...
C:\Python27\python.exe D:/PythonPratice/test.py
Traceback (most recent call last):
File "D:/PythonPratice/test.py", line 9, in
driver.get('https://www.google.com.tw/')
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 250, in get
self.execute(Command.GET, {'url': url})
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 238, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 193, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Runtime.executionContextCreated has invalid 'context': {"auxData":{"frameId":"2172.1","isDefault":true},"id":1,"name":"","origin":"://"}
(Session info: chrome=56.0.2924.87)
(Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 SP1 x86_64)
Process finished with exit code 1
Much thanks!
Try to download latest version of chromedriver and put it to C:/Python27/Scripts/ instead of outdated one
you should provide full path of chromedriver.exe, see the following:
driver = webdriver.Chrome(executable_path=r'C:/Python27/Scripts/chromedriver.exe')

Selenium chromedriver 2.25 TimeoutException cannot determine loading status

I am using python3 on mac os and I have updated chrome, chromedriver and selenium to the latest version. I am getting a TimeoutException, the browser opens correctly but it freezes.
>>> from selenium import webdriver
>>> driver = webdriver.Chrome()
# opens browser with blank page
>>> driver.get('http://example.com')
# gets first page OK and then driver literally flashes once
>>> driver.get('http://stackoverflow.com')
>>>
# Cursor loading forever... until TimeoutException
This error is thrown:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 248, in get
self.execute(Command.GET, {'url': url})
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: cannot determine loading status
from timeout: Timed out receiving message from renderer: -0.003
(Session info: chrome=54.0.2840.71)
(Driver info: chromedriver=2.25.426935 (820a95b0b81d33e42712f9198c215f703412e1a1),platform=Mac OS X 10.12.0 x86_64)
I'm using: Python 3.5.2 ,Chrome 54.0, chromedriver 2.25, selenium 3.0.1
I've tried other versions of chromedriver with no success, also I did not find any solution to this online. Thanks.
EDIT:
Still getting the error from above + getting a new error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 92, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 179, in start_session
response = self.execute(Command.NEW_SESSION, capabilities)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: session not created exception
from unknown error: bad inspector message: {"method":"Page.loadEventFired","params":{"timestamp":14220,088073}}
(Session info: chrome=54.0.2840.71)
(Driver info: chromedriver=2.25.426935 (820a95b0b81d33e42712f9198c215f703412e1a1),platform=Mac OS X 10.12.0 x86_64)
selenium.common.exceptions.TimeoutException: Message: timeout: cannot determine loading status
from timeout: Timed out receiving message from renderer: -0.003
Issue817:It looks like this issue has been logged as a bug for Selenium.
Someone has answered to get rid from this issue by using the --dns-prefetch-disable option of chrome.
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--dns-prefetch-disable')
driver = Chrome(chrome_options=chrome_options)
If issue still exists follow this thread may be it solves your problem
Set
env LANG=en_US.UTF-8 ./chromedriver
from : https://bugs.chromium.org/p/chromedriver/issues/detail?id=1552

Selenium error At the end of the page

I have written this code using selenium python so parse a webpage(dynamically loading) so
that it goes till the end and then stop when "load more" button is not clickable anymore.
wait=WebDriverWait(driver,50)
wait.until(EC.element_to_be_clickable((By.ID, 'bottomPager'))) while
EC.element_to_be_clickable((By.ID,'bottomPager')):
driver.find_element_by_xpath('.//div[#id="bottomPager"]').click()
if not driver.find_element_by_id('bottomPager').is_enabled():
break
wait.until(EC.element_to_be_clickable((By.ID,'bottomPager')))
but I am getting the error when it reaches the end of the page
Traceback (most recent call last): File "python_org_search2.py", line 24, in <module>
driver.find_element_by_xpath('.//div[#id="bottomPager"]').click() File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 60, in click
self._execute(Command.CLICK_ELEMENT) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 370, in _execute
return self._parent.execute(command, params) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 166, in execute
self.error_handler.check_response(response) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: u'unknown error: Element is not clickable at point (643, 628)\n (Session info: chrome=34.0.1847.137)\n (Driver info: chromedriver=2.9.248304,platform=Linux 3.13.0-24-generic x86_64)'
What are the changes required in code so as to rectify it ?
In my experience, this exception only occurs when using the Chrome driver with Selenium. Can you use Firefox instead? (I believe Firefox is the primary intended browser for webdriver invocations via Selenium.)
driver = webdriver.Firefox()
That should clear up this exception.
(You can get portable versions of Firefox at this site.)

Categories

Resources