Develop Reference

getting TimeoutException when using expected_conditions in heroku

I have a selenium robot that worked perfectly locally but on heroku TimeoutException raises whenever its on a expected_condition (element_to_be_clickable, visibility_of_element_located and presence_of_element_located). Anyone knows how to fix this problem in heroku.
here is an example where I used expected_conditions
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(#class, 'productlistning__btn')]")))
and the chrome arguments that I used in my code:
chrome_options = Options()
chrome_options.binary_location = os.environ.get("GOOGLE_CHROME_BIN")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument("--no-sandbox")
driver = webdriver.Chrome(executable_path=os.environ.get("CHROMEDRIVER_PATH"), options=chrome_options)

I guess you didn't define the screen size for the driver while in headless mode the default screen size is 800,600.
So, to make your Selenium code working try setting the screen size to maximal or 1920,1080. As following:
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
options.add_argument("window-size=1920,1080")
Or
options = Options()
options.add_argument("start-maximized")

I'm trying to use selenium with specific user but I get this error

s=Service(ChromeDriverManager().install())
chrome_options = Options()
options = webdriver.ChromeOptions()
options.binary_location = "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
options.add_experimental_option("excludeSwitches", ["enable-logging"])
options.add_argument('--disable-gpu')
#options.add_argument('--headless')
options.add_argument("user-data-dir=C:\Users\USER\AppData\Local\Google\Chrome\User Data")
options.add_argument('--profile-directory=Profile 828')
options.add_argument("--start-maximized")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
browser = webdriver.Chrome(service=s, options=options)
browser.get('**LINK**')
Tried everything and I want Just use specific browser to use specific extensions
https://i.stack.imgur.com/Noeg6.png
https://i.stack.imgur.com/7OsZ8.png

How to use Firefox with enable-automation disabled?

The code below starts Chrome loading its options which consequently disables "automation", I would like to know how I do the same for firefox?
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches",["enable-automation"])
#driver_path = r'C:\Users\mkdob\AppData\Local\ms-playwright\firefox-1323\firefox\firefox.exe'
driver = webdriver.Chrome(executable_path=r'C:\chromedriver.exe',
chrome_options=options) driver.get('https://google.com')
driver.close()

How to avoid bot detection in Selenium?

I am trying to scrape this website with the following code:
from selenium import webdriver
options = webdriver.ChromeOptions()
driver_path = '/Users/francopiccolo/Utils/chromedriver97'
driver = webdriver.Chrome(executable_path=driver_path, chrome_options=options)
url = 'https://www.zonaprop.com.ar/inmuebles-venta-rosario.html'
driver.get(url)
The problem is it somehow detects a bot and throws an error.
Ideas?

options = Options()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
Try these to remove the detection of the bot.

Accesssing Corsair.com with Python Selenium?

Is it possible to access https://www.corsair.com/ with Selenium in Python without getting blocked by Corsair?
When I try to load the page in Selenium, it keeps giving me this error message:
What I tried to bypass it, is changing the user-agent to a random one, which didn't fix the issue.
This is my code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from fake_useragent import UserAgent
options = webdriver.ChromeOptions()
options.add_argument("window-size=1400,600")
ua = UserAgent()
user_agent = ua.random
print(user_agent)
options.add_argument(f'user-agent={user_agent}')
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=options)
print('Loading Corsair Website ...')
driver.get("https://www.corsair.com/")

There are multiple ways to evade detection of Selenium automation and one of them is to use the following argument:
--disable-blink-features=AutomationControlled.
Code Block:
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
# options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://www.corsair.com/")
driver.save_screenshot("image.png")
Screenshot: