Can't use Selenium with for loop - python

I am trying to write a program with for loop and for that I need to declare Selenium webdriver before the loop.
Each time I am trying to do it I get an error. Like the loop doesn't recognize driver.get part.
Here is the code:
from selenium import webdriver
import time
import csv
with open('smlxl.csv', 'r') as myFile:
myReader = csv.reader(myFile)
myList = list(myReader)
driver = webdriver.Chrome(executable_path='C:\Program Files (x86)\chromedriver.exe')
for n in range(1,1000,4):
barcode = (myList[n][24])
url = ("https://www.google.com/search?q=" + barcode)
driver.get(url)
print(n)
n = (n+4)
time.sleep(3)
driver.close()
The error I get:
C:\Users\DELL\PycharmProjects\pythonProject3\erro.py:10: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(executable_path='C:\Program Files (x86)\chromedriver.exe')
1
Traceback (most recent call last):
File "C:\Users\DELL\PycharmProjects\pythonProject3\erro.py", line 16, in <module>
driver.get(url)
File "C:\Users\DELL\PycharmProjects\pythonProject3\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 447, in get
self.execute(Command.GET, {'url': url})
File "C:\Users\DELL\PycharmProjects\pythonProject3\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 435, in execute
self.error_handler.check_response(response)
File "C:\Users\DELL\PycharmProjects\pythonProject3\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSessionIdException: Message: invalid session id
Stacktrace:
Backtrace:
Ordinal0 [0x00326463+2188387]
Ordinal0 [0x002BE461+1762401]
Ordinal0 [0x001D3C40+801856]
Ordinal0 [0x001F68BD+944317]
Ordinal0 [0x001F7AD6+948950]
GetHandleVerifier [0x005C71F2+2712546]
GetHandleVerifier [0x005B886D+2652765]
GetHandleVerifier [0x003B002A+520730]
GetHandleVerifier [0x003AEE06+516086]
Ordinal0 [0x002C468B+1787531]
Ordinal0 [0x002C8E88+1805960]
Ordinal0 [0x002C8F75+1806197]
Ordinal0 [0x002D1DF1+1842673]
BaseThreadInitThunk [0x7587FA29+25]
RtlGetAppContainerNamedObjectPath [0x770D7A9E+286]
RtlGetAppContainerNamedObjectPath [0x770D7A6E+238]
Process finished with exit code 1
Thank you for helping

Had to place driver.close() outside the for loop.

These error message...
selenium.common.exceptions.InvalidSessionIdException: Message: invalid session id
...implies that the previous session wasn't successfully killed/removed/eliminated.
Solution
You need to replace:
driver.close()
with:
driver.quit()
References
You can find a couple of relevant detailed discussions in:
selenium.common.exceptions.WebDriverException: Message: invalid session id using Selenium with ChromeDriver and Chrome through Python

Related

python scrollIntoView to the element error

I am trying to get names of teams by ID. With action of to move focus to element I get element names to list and text files. At some point web page reloads, and screen freezes then it stopes focus move and does not take team names to the list or text file neither.
I even tried time.sleep(3) it could not get any team name data any.
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from datetime import datetime
driver = webdriver.Chrome(r"C:\Users\Admin\Downloads\chromedriver_win32 (1)\chromedriver.exe")
driver.get("https://www.nba.com/schedule?pd=false&region=1")
driver.implicitly_wait(5)
element_to_click=driver.find_element(By.ID,"onetrust-accept-btn-handler") #.click()
element_to_click.click()
element_to_save=driver.find_element(By.XPATH,"//div/div/div/div/h4")
f=open('new_result_file00.txt','w')#before optional read=write mode was ,r+,
f.write(element_to_save.text)
f.write("\n")
f.write(str(datetime.today()))
myList=[]
myList.append(1)
elements_to_save=driver.find_elements(By.XPATH,"//*[#data-id='nba:schedule:main:team:link']")
i=1
for element in elements_to_save:
driver.execute_script("arguments[0].scrollIntoView();", element)
try:
f.write(element.text)
myList.append(element.text)
except Exception as e:
print("err",i)
i=i+1
f.write(" \n ")
f.write(str(datetime.today()))
f.close()
error TraceBack:
err 1
Traceback (most recent call last):
File "C:\pythonPro\w_crawl\w01_nba.py", line 23, in <module>
driver.execute_script("arguments[0].scrollIntoView();", element)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 491, in execute_script
return self.execute(command, {
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 428, in execute
self.error_handler.check_response(response)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=105.0.5195.127)
Stacktrace:
Backtrace:
Ordinal0 [0x004FDF13+2219795]
Ordinal0 [0x00492841+1779777]
Ordinal0 [0x003A423D+803389]
Ordinal0 [0x003A6D04+814340]
Ordinal0 [0x003A6BC2+814018]
Ordinal0 [0x003A755F+816479]
Ordinal0 [0x003FFC1B+1178651]
Ordinal0 [0x003EE7FC+1107964]
Ordinal0 [0x003FF192+1175954]
Ordinal0 [0x003EE616+1107478]
Ordinal0 [0x003C7F89+950153]
Ordinal0 [0x003C8F56+954198]
GetHandleVerifier [0x007F2CB2+3040210]
GetHandleVerifier [0x007E2BB4+2974420]
GetHandleVerifier [0x00596A0A+565546]
GetHandleVerifier [0x00595680+560544]
Ordinal0 [0x00499A5C+1808988]
Ordinal0 [0x0049E3A8+1827752]
Ordinal0 [0x0049E495+1827989]
Ordinal0 [0x004A80A4+1867940]
BaseThreadInitThunk [0x75B8FA29+25]
RtlGetAppContainerNamedObjectPath [0x77357B5E+286]
RtlGetAppContainerNamedObjectPath [0x77357B2E+238]
Process finished with exit code 1
I added expected conditions and waitwebdriver package and put 10sec to wait until element loads (!with 5 sec error ) everything flied off
wait = WebDriverWait(driver, 10)
elements_to_save=wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//*[#data-id='nba:schedule:main:team:link']")))

Script which works on PC1 doesn't work on PC2

a friend of mine wrote me a piece of code which does a web automation for me which im running on a VPS.
On VPS 1 the script has been running for days without a problem.
Now I wanted to do the same on a different VPS.
Its an older windows version of win 10 I don't know if that plays a role*
Chrome version on VPS 1 is 102
If I try to install chrome on VPS 2 it updates automatically to 103 so I tried downloaded version 102 over this link : https://www.slimjet.com/chrome/google-chrome-old-version.php but it still auto updates to 103 so i ended up running the 103 chrome driver instead of the 102 on VPS1
For the rest I installed everything completely the same on VPS 1 on VPS2
Selenium version 4.1.5
But now all it does it that it opens the browser with the link but it doesn't do any actions and I don't know what the error means.
I will put the code in here (some things censored because of personal information) + the error I get
import schedule
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
exec_times = ["13:17"]
wert = "5"
id = "ID: CENSORED"
def do_purchase():
PATH = "C:\\Users\\Administrator\\Desktop\\bumping\\chromedriver.exe"
URL = "URL CENSORED"
options = Options()
options.add_argument(
"user-data-dir=C:\\Users\\\Administrator\\AppData\\Local\\Google\\Chrome\\User Data")
driver = webdriver.Chrome(PATH, chrome_options=options)
driver.get(URL)
time.sleep(10)
div = div = driver.find_element_by_xpath(
"//*[contains(text(), '{}')]".format(id))
parent = div.find_element_by_xpath('..')
buttons = parent.find_elements_by_tag_name('button')
button = buttons[0]
hover = ActionChains(driver).move_to_element(parent)
hover.perform()
time.sleep(2)
webdriver.ActionChains(driver).move_to_element(
button).click(button).perform()
time.sleep(3)
inputs = driver.find_elements_by_xpath(
"//*[contains(text(), 'Transfer amount')]")
parent = inputs[0].find_element_by_xpath('..')
input = parent.find_element_by_tag_name('input')
input.clear()
input.send_keys(wert)
confirm_span = driver.find_element_by_xpath(
"//*[contains(text(), 'Confirm')]")
confirm_button = confirm_span.find_element_by_xpath('..')
webdriver.ActionChains(driver).move_to_element(
confirm_button).click(confirm_button).perform()
time.sleep(5)
driver.quit()
def task(time):
do_purchase()
for exec_time in exec_times:
schedule.every().day.at(exec_time).do(task, exec_time)
while True:
schedule.run_pending()
time.sleep(1)
The terminal says the following
PS C:\Users\Administrator> & C:/Users/Administrator/AppData/Local/Programs/Python/Python310/python.exe "c:/Users/Administrator/Desktop/bumping/Client 1.py"
c:\Users\Administrator\Desktop\bumping\Client 1.py:19: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(PATH, chrome_options=options)
c:\Users\Administrator\Desktop\bumping\Client 1.py:19: DeprecationWarning: use options instead of chrome_options
driver = webdriver.Chrome(PATH, chrome_options=options)
DevTools listening on ws://127.0.0.1:64181/devtools/browser/854481e2-f194-46a1-8421-8dab5852169d
Traceback (most recent call last):
File "c:\Users\Administrator\Desktop\bumping\Client 1.py", line 65, in <module>
schedule.run_pending()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\schedule\__init__.py", line 780, in run_pending
default_scheduler.run_pending()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\schedule\__init__.py", line 100, in run_pending
self._run_job(job)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\schedule\__init__.py", line 172, in _run_job
ret = job.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\schedule\__init__.py", line 661, in run
ret = self.job_func()
File "c:\Users\Administrator\Desktop\bumping\Client 1.py", line 57, in task
do_purchase()
File "c:\Users\Administrator\Desktop\bumping\Client 1.py", line 20, in do_purchase
driver.get(URL)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 442, in get
self.execute(Command.GET, {'url': url})
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 430, in execute
self.error_handler.check_response(response)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot determine loading status
from unknown error: unexpected command response
(Session info: chrome=103.0.5060.53)
Stacktrace:
Backtrace:
Ordinal0 [0x0060D953+2414931]
Ordinal0 [0x0059F5E1+1963489]
Ordinal0 [0x0048C6B8+837304]
Ordinal0 [0x0047EB34+781108]
Ordinal0 [0x0047E06A+778346]
Ordinal0 [0x0047D646+775750]
Ordinal0 [0x0047C565+771429]
Ordinal0 [0x0047CB68+772968]
Ordinal0 [0x0048785F+817247]
Ordinal0 [0x0049255B+861531]
Ordinal0 [0x00494F70+872304]
Ordinal0 [0x0047CE36+773686]
Ordinal0 [0x00492197+860567]
Ordinal0 [0x004E4B55+1198933]
Ordinal0 [0x004D42B6+1131190]
Ordinal0 [0x004AE860+976992]
Ordinal0 [0x004AF756+980822]
GetHandleVerifier [0x0087CC62+2510274]
GetHandleVerifier [0x0086F760+2455744]
GetHandleVerifier [0x0069EABA+551962]
GetHandleVerifier [0x0069D916+547446]
Ordinal0 [0x005A5F3B+1990459]
Ordinal0 [0x005AA898+2009240]
Ordinal0 [0x005AA985+2009477]
Ordinal0 [0x005B3AD1+2046673]
BaseThreadInitThunk [0x750D62C4+36]
RtlSubscribeWnfStateChangeNotification [0x77E20969+1081]
RtlSubscribeWnfStateChangeNotification [0x77E20934+1028]
Help would be hugely appreciated

When i try to scrape this website with selenium and python

When i try and scrape the website it just throws some errors
I think it may have something to do with my webdriver but idk
I am trying to get this data so i can put it in a spreadsheet to get some cool staticstics
from selenium import webdriver
url = 'https://prosettings.net/cs-go-pro-settings-gear-list/'
driver = webdriver.Chrome(executable_path="C:\WebDrivers\chromedriver.exe")
driver.get(url)
names = driver.find_elements_by_class_name(" column-player")
for name in names:
title = name.find_element_by_xpath('.//a').text
print(name)
Here are the error i get in terminal
d:\downloads\PythonScraping\Test.py:5: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(executable_path="C:\WebDrivers\chromedriver.exe")
DevTools listening on ws://127.0.0.1:53131/devtools/browser/73ca0453-352e-47a0-a98a-fb539150d6f9
d:\downloads\PythonScraping\Test.py:8: DeprecationWarning: find_elements_by_* commands are deprecated. Please use find_elements() instead
names = driver.find_elements_by_class_name(" column-player")
Traceback (most recent call last):
File "d:\downloads\PythonScraping\Test.py", line 8, in <module>
names = driver.find_elements_by_class_name(" column-player")
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 783, in
find_elements_by_class_name
return self.find_elements(by=By.CLASS_NAME, value=name)
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 1279, in find_elements
return self.execute(Command.FIND_ELEMENTS, {
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in
execute
self.error_handler.check_response(response)
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\errorhandler.py", line 247,
in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
(Session info: chrome=98.0.4758.102)
Stacktrace:
Backtrace:
Ordinal0 [0x00EF69A3+2582947]
Ordinal0 [0x00E8A6D1+2139857]
Ordinal0 [0x00D83A98+1063576]
Ordinal0 [0x00D862B7+1073847]
Ordinal0 [0x00D8617E+1073534]
Ordinal0 [0x00D863F0+1074160]
Ordinal0 [0x00DAFCB2+1244338]
Ordinal0 [0x00DB013B+1245499]
Ordinal0 [0x00DD9F8C+1417100]
Ordinal0 [0x00DC8594+1344916]
Ordinal0 [0x00DD834A+1409866]
Ordinal0 [0x00DC8366+1344358]
Ordinal0 [0x00DA5176+1200502]
Ordinal0 [0x00DA6066+1204326]
GetHandleVerifier [0x0109BE02+1675858]
GetHandleVerifier [0x0115036C+2414524]
GetHandleVerifier [0x00F8BB01+560977]
GetHandleVerifier [0x00F8A8D3+556323]
Ordinal0 [0x00E9020E+2163214]
Ordinal0 [0x00E95078+2183288]
Ordinal0 [0x00E951C0+2183616]
Ordinal0 [0x00E9EE1C+2223644]
BaseThreadInitThunk [0x7586FA29+25]
RtlGetAppContainerNamedObjectPath [0x77957A9E+286]
RtlGetAppContainerNamedObjectPath [0x77957A6E+238]
There are 2 problems here:
Instead of
names = driver.find_elements_by_class_name(" column-player")
it should be
names = driver.find_elements_by_class_name("column-player")
(I know, there are spaces before column-player class name there, but you still should not put them inside the locator)
2) You should add a delay to access these elements only after the page have been completely loaded.
This should work better:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = 'https://prosettings.net/cs-go-pro-settings-gear-list/'
driver = webdriver.Chrome(executable_path="C:\WebDrivers\chromedriver.exe")
wait = WebDriverWait(driver, 20)
driver.get(url)
#wait for at least 1 element visibility
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".column-player")))
#short additional wait for all the other elements to complete loading
time.sleep(0.5)
names = driver.find_elements_by_class_name("column-player")
for name in names:
title = name.find_element_by_xpath('.//a').text
print(name)

What does Backtrace: Ordinal0 [0x00723AB3+2505395] in stack trace error means while using Selenium and ChromeDriver

I'm relatively new to coding and kinda unfamiliar with stacktrace errors.
I have been given this error, but im not sure which part of my code do i fix? Did the error come
up because of a NoSuchElementException, or was it due to a timeout error?
I thought that within my code I had dealt with how the program will face the nosuchelement exception. Basically, my code searches for slots that are available and will disable the back button when a slot becomes available. However, I am met with other sorts of error such as connection time out when the webpage has been idle for too long. Thus, my mission now is to filter out the different problems i am faced with so that i can deal with the issues respectively. I have added the expected conditions as the filter to differentiate the errors i am faced with
Code:
no_slot = True
while no_slot == 1:
for i in range(60):
search_button = browser.find_element(By.NAME, "btnSearch")
search_button.click()
try:
noslot_back_button = browser.find_element(By.CSS_SELECTOR, 'input[name="btnBack"]')
except NoSuchElementException:
if EC.presence_of_element_located(browser.find_element(By.XPATH, '//td[#class="title"]')):
# Proceed with booking (make sure that it is at the booking page (implementation)
slot_radio_info = browser.find_element(By.NAME, 'slot')
slot_id = slot_radio_info.get_attribute('id')
radio_button = browser.find_element(By.XPATH, '//input[#id="' + str(slot_id) + '"]')
radio_button.click()
slot_submit_button = browser.find_element(By.CSS_SELECTOR, 'input[value="Submit"]')
WebDriverWait(EC.element_to_be_clickable(slot_submit_button))
slot_submit_button.click()
double_cfm_button = browser.find_element(By.CSS_SELECTOR, 'input[value="Confirm"]')
double_cfm_button.click()
elif not EC.title_contains('BBDC: Booking For TP Driving Simulator Lesson'):
print("Connection timed out or something else happened")
else:
print("unaccounted condition")
Error received:
Traceback (most recent call last):
File "C:/Users/Jonathan/PycharmProjects/Riddle/BBDC.py", line 83, in <module>
noslot_back_button = browser.find_element(By.CSS_SELECTOR, 'input[name="btnBack"]')
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1238, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
self.error_handler.check_response(response)
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[name="btnBack"]"}
(Session info: chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00723AB3+2505395]
Ordinal0 [0x006BAE41+2076225]
Ordinal0 [0x005C2498+1057944]
Ordinal0 [0x005ECB74+1231732]
Ordinal0 [0x00616D92+1404306]
Ordinal0 [0x00605A2A+1333802]
Ordinal0 [0x00615168+1397096]
Ordinal0 [0x006058BB+1333435]
Ordinal0 [0x005E23E4+1188836]
Ordinal0 [0x005E323F+1192511]
GetHandleVerifier [0x008ACB36+1554566]
GetHandleVerifier [0x00954A0C+2242396]
GetHandleVerifier [0x007B0E0B+523099]
GetHandleVerifier [0x007AFEB0+519168]
Ordinal0 [0x006C02FD+2097917]
Ordinal0 [0x006C4388+2114440]
Ordinal0 [0x006C44C2+2114754]
Ordinal0 [0x006CE041+2154561]
BaseThreadInitThunk [0x7739FA29+25]
RtlGetAppContainerNamedObjectPath [0x77987A9E+286]
RtlGetAppContainerNamedObjectPath [0x77987A6E+238]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Jonathan\PyCharm\plugins\python-ce\helpers\pydev\pydevd.py", line 1483, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Users\Jonathan\PyCharm\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Jonathan/PycharmProjects/Riddle/BBDC.py", line 86, in <module>
if EC.presence_of_element_located(browser.find_element(By.XPATH, '//td[#class="title"]')):
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1238, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
self.error_handler.check_response(response)
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//td[#class="title"]"}
(Session info: chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00723AB3+2505395]
Ordinal0 [0x006BAE41+2076225]
Ordinal0 [0x005C2498+1057944]
Ordinal0 [0x005ECB74+1231732]
Ordinal0 [0x00616D92+1404306]
Ordinal0 [0x00605A2A+1333802]
Ordinal0 [0x00615168+1397096]
Ordinal0 [0x006058BB+1333435]
Ordinal0 [0x005E23E4+1188836]
Ordinal0 [0x005E323F+1192511]
GetHandleVerifier [0x008ACB36+1554566]
GetHandleVerifier [0x00954A0C+2242396]
GetHandleVerifier [0x007B0E0B+523099]
GetHandleVerifier [0x007AFEB0+519168]
Ordinal0 [0x006C02FD+2097917]
Ordinal0 [0x006C4388+2114440]
Ordinal0 [0x006C44C2+2114754]
Ordinal0 [0x006CE041+2154561]
BaseThreadInitThunk [0x7739FA29+25]
RtlGetAppContainerNamedObjectPath [0x77987A9E+286]
RtlGetAppContainerNamedObjectPath [0x77987A6E+238]
python-BaseException
Traceback (most recent call last):
File "C:\Users\Jonathan\PyCharm\plugins\python-ce\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 292, in _on_run
r = self.sock.recv(1024)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
Process finished with exit code 1
Selenium is unable to locate the element
noslot_back_button = browser.find_element(By.CSS_SELECTOR, 'input[name="btnBack"]')
most likely a typo or maybe you forgot to name the css selector.
This error message...
Traceback (most recent call last):
.
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[name="btnBack"]"}
(Session info: chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00723AB3+2505395]
Ordinal0 [0x006BAE41+2076225]
Ordinal0 [0x005C2498+1057944]
...implies that NoSuchElementException was raised while locating the elements.
Addressing NoSuchElementException
Without the relevant HTML of the element it would be difficult to address if the Locator Strategy identifies the element uniquely within the HTML DOM.
Additionally, though you are using chrome=95.0.4638.69 but your ChromeDriver version isn't in sync with the Chrome browser version. Hence you see the Backtrace.
Solution
Ensure that:
ChromeDriver is updated to current ChromeDriver v95.0 level.

Implicit delay with Selenium

I am practicing with selenium. I tried this page scraping with beautifulsoup, but as the page gets some time to load, I tried it with selenium with adding a delay time. I have tried drive.delay and timeout too. But nothing works fine with me. I get same alike error with all the tries, and I have attached my latest try and the error I get with it.
This is my ty:
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
url = 'https://www.checkatrade.com/search?page=1&categoryId=1164&location=London'
driver =webdriver.Chrome(executable_path='D:\Web_Scraping\chromedriver_win32\chromedriver.exe')
driver.get(url)
driver.implicitly_wait(20)
items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')
for item in items:
company = item.find_element_by_xpath('//*[#id="1052063"]/div/div[2]/div[2]/a/div[1]/h2').text
print(company)
But it throws me this error, which I can not figure out.
DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver =webdriver.Chrome(executable_path='D:\Web_Scraping\chromedriver_win32\chromedriver.exe')
DevTools listening on ws://127.0.0.1:1090/devtools/browser/9ed2753e-5d6e-4382-a9ac-a0ed2fbb8379
d:/Web_Scraping/seleniumtest.py:21: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')
[5536:25384:1014/115806.197:ERROR:chrome_browser_main_extra_parts_metrics.cc(228)] crbug.com/1216328: Checking Bluetooth availability started. Please report if there is no report that this ends.
[5536:25384:1014/115806.198:ERROR:chrome_browser_main_extra_parts_metrics.cc(231)] crbug.com/1216328: Checking Bluetooth availability ended.
[5536:25384:1014/115806.199:ERROR:chrome_browser_main_extra_parts_metrics.cc(234)] crbug.com/1216328: Checking default browser status started. Please report if there is no report that this ends.
[5536:23016:1014/115806.200:ERROR:device_event_log_impl.cc(214)] [11:58:06.200] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[5536:25384:1014/115806.733:ERROR:chrome_browser_main_extra_parts_metrics.cc(238)] crbug.com/1216328: Checking default browser status ended.
Traceback (most recent call last):
File "d:/Web_Scraping/seleniumtest.py", line 21, in <module>
items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 754, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1238, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
self.error_handler.check_response(response)
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU"}
(Session info: chrome=94.0.4606.81)
Stacktrace:
Backtrace:
Ordinal0 [0x0107BDE3+2473443]
Ordinal0 [0x01016661+2057825]
Ordinal0 [0x00F22438+1057848]
Ordinal0 [0x00F4C7DB+1230811]
Ordinal0 [0x00F76712+1402642]
Ordinal0 [0x00F6571A+1333018]
Ordinal0 [0x00F74ABF+1395391]
Ordinal0 [0x00F655AB+1332651]
Ordinal0 [0x00F42104+1188100]
Ordinal0 [0x00F42F59+1191769]
GetHandleVerifier [0x01202266+1549718]
GetHandleVerifier [0x012AD4A7+2250711]
GetHandleVerifier [0x0110718B+521403]
GetHandleVerifier [0x01106229+517465]
Ordinal0 [0x0101B79D+2078621]
Ordinal0 [0x0101FB58+2095960]
Ordinal0 [0x0101FC92+2096274]
Ordinal0 [0x01029541+2135361]
BaseThreadInitThunk [0x764DFA29+25]
RtlGetAppContainerNamedObjectPath [0x77A77A9E+286]
RtlGetAppContainerNamedObjectPath [0x77A77A6E+238]
What causes this error?
You are using a wrong locator.
find_element_by_class_name method is intended to receive a SINGLE class name while you are passing a multiple class names value to it "find_element_by_class_name".
To pass multiple class names you should use css selector or XPath.
So instead of
items = driver.find_element_by_class_name('sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU')
It could be
items = driver.find_element_by_css_selector('.sc-8keyp7-0.kbPUFb.sc-1a2v1qv-0.gwBNSh.sc-3jeiz1-1.bfuahU')
Or
items = driver.find_element_by_xpath("//div[#class='sc-8keyp7-0 kbPUFb sc-1a2v1qv-0 gwBNSh sc-3jeiz1-1 bfuahU']")

Categories

Resources