Develop Reference

getting TimeoutException when using expected_conditions in heroku

I have a selenium robot that worked perfectly locally but on heroku TimeoutException raises whenever its on a expected_condition (element_to_be_clickable, visibility_of_element_located and presence_of_element_located). Anyone knows how to fix this problem in heroku.
here is an example where I used expected_conditions
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(#class, 'productlistning__btn')]")))
and the chrome arguments that I used in my code:
chrome_options = Options()
chrome_options.binary_location = os.environ.get("GOOGLE_CHROME_BIN")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument("--no-sandbox")
driver = webdriver.Chrome(executable_path=os.environ.get("CHROMEDRIVER_PATH"), options=chrome_options)

I guess you didn't define the screen size for the driver while in headless mode the default screen size is 800,600.
So, to make your Selenium code working try setting the screen size to maximal or 1920,1080. As following:
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
options.add_argument("window-size=1920,1080")
Or
options = Options()
options.add_argument("start-maximized")

Print dom after Selenium driver.get

I'm used to requests where I can just print the response after I do make a GET request. I find myself unsure if parts of the page are in the resonse or not, particularly when the website uses React or jQuery.
Is there a way I can do the same with Selemium?
Like this?
DRIVER_PATH = '/usr/bin/chromedriver'
options = webdriver.ChromeOptions()
options.add_argument("--headless")
driver = webdriver.Chrome(executable_path=DRIVER_PATH, options=options)
driver.get('example.com')
# Print the DOM
driver.quit()

You are looking for driver.page_source.
from selenium import webdriver
DRIVER_PATH = '/usr/bin/chromedriver'
options = webdriver.ChromeOptions()
options.add_argument("--headless")
driver = webdriver.Chrome(executable_path=DRIVER_PATH, options=options)
driver.get('https://google.com')
# Print the DOM
print(driver.page_source)
driver.quit()

Python Selenium Firefox - how to enable headless-mode as part of a class/object?

I have the following code:
options = Options()
options = options.set_headless( headless=True)
class Sel_Driver():
def __init__(self):
self.driver = webdriver.Firefox(firefox_options=options)
I can then use self.driver.get(url) as part of a method to open urls I feed in. This works - I can feed in and open the URLs, but they don't in headless mode.
(I initially defined the driver as self.driver = webdriver.Firefox(firefox_options=Options().set_headless(headless=True) - but that didn't work, so I tried it as above).
What am I missing? I don't understand why the driver is able to open pages, but the options aren't enabled.

Please try following code :
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(firefox_options=options)

This will work for you for sure. Try it.Please specify the path of the driver. It is for chrome change it to firefox.
from pyvirtualdisplay import Display
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=options, executable_path="C:\\Users\\Username\\Downloads\\chromedriver.exe")
print("Firefox Headless Browser Invoked")
driver.get('https://www.facebook.com/')
jks = driver.find_element_by_id("email").get_attribute("class")
print(jks)
driver.quit()

How to disable chrome notifications popup in python and selenium?

How to disable chrome notifications popup in python and selenium?
I tried:
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)
But then also it shows notification...
I tried the same codes answered
here but then also I
am not able to disable the notifications!

The selenium package has a ChromeOptions class, in which you can add many arguments. One of which is 'disable-notifications'. You can pass that class to the driver class when initializing it.
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('disable-notifications')
driver = webdriver.Chrome('chromedriver.exe', options=chrome_options)

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('disable-notifications')
driver = webdriver.Chrome(executable_path="C:\Users\PycharmProjects\chromedriver_win32\chromedriver.exe", options=chrome_options)
driver.maximize_window()
This works with the right placement of code:
Click here for Image-->ChromeOptions Pycharm Code Image

You can pass disable-notifications to your chrome options.
Here's an example I have using Javascript, should work the same with Python.
var o = new chrome.Options();
o.addArguments('user-data-dir=./chromeprofile');
o.addArguments('disable-infobars');
o.addArguments("disable-notifications");
o.setUserPreferences( { credentials_enable_service: false } );

Disable python chrome driver extensions without lose driver path

I started having issues running a python script which uses selenium and chrome driver, so I want to disable extensions of chrome driver using python without losing the path where the driver is located.
Currently I have the below:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
path_to_Ie = 'C:\\Python34\\ChromeDriver\\ChromeDriver.exe'
browser = webdriver.Chrome(executable_path = path_to_Ie)
url = 'https://wwww.test.com/'
browser.get(url)
and I would like to add the below lines:
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
browser = webdriver.Chrome(chrome_options=chrome_options)
Any idea how I can merge both codes to make it work properly?
Thanks a lot.

Just provide multiple keyword arguments:
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
browser = webdriver.Chrome(executable_path=path_to_Ie, chrome_options=chrome_options)