Develop Reference

Selenium webdriver does not see that site is not safe

Im trying to check if site is safe/does not have any google warnings with selenium webdriver:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.headless = True
options.add_experimental_option( "prefs", {'safebrowsing.enabled':'true'})
driver = webdriver.Chrome("/usr/bin/chromedriver", options=options)
driver.get("https://example.com/")
print(driver.title)
driver.quit()
i know that my scriptwill output title of site but is there any way to get info if site is safe?
When i enter via my chrome browser on any site that is not safe i can see that there is phishing warning, but when i enter via webdriver it says nothing about phishing, 0 warnings, nothing.
Below screenshot of warning im talking about:
All in all my main goal is to get information is site is safe or not via selenium. Is there is any way to do it without using google api?
thanks
v

Loading Selenium user profile in Tor Chrome on Windows

This code works for Windows where it launches Chrome connected via Tor. Keep in mind you have to have Tor browser running beforehand. How can I enable the user-profile and start the browser logged in? I have tried the regular method. I have only 1 profile. Default. Doesn't seem to be working. Any clues?
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
tor_proxy = "127.0.0.1:9150"
chrome_options = Options()
'''chrome_options.add_argument("--test-type")'''
chrome_options.add_argument('--ignore-certificate-errors')
'''chrome_options.add_argument('--disable-extensions')'''
chrome_options.add_argument('disable-infobars')
'''chrome_options.add_argument("--incognito")'''
chrome_options.add_argument('--user-data=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data\\Default')
chrome_options.add_argument('--proxy-server=socks5://%s' % tor_proxy)
driver = webdriver.Chrome(executable_path='C:\\chromedriver.exe', options=chrome_options)
driver.get('https://www.gmail.com')
time.sleep(4)
driver.switch_to.frame(0)
driver.find_element_by_id("introAgreeButton").click()

Use this instead.
chrome_options.add_argument("user-data-dir=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data")
you don't have to specify the profile directory (Default) as it is used by default if nothing is explicitly specified using below code. SO in your case use only the above line of code
chrome_options.add_argument("profile-directory=Profile 1")
Eg:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument(r"user-data-dir=C:\Users\prave\AppData\Local\Google\Chrome\User Data")
driver =webdriver.Chrome("./chromedriver.exe",options=options)
driver.get(url)
Output:

How to initiate a Chromium based Vivaldi browser session using Selenium and Python

I am trying to use the vivaldi browser with Selenium. It is a chromium browser that runs very similar to chrome. I have Selenium working with Firefox (geckodriver), and Google Chrome(chromedriver), but I can't seem to find a way with Vivaldi. Any help would be appreciated.

If the vivaldi binary by default is located at C:\Users\levir\AppData\Local\Vivaldi\Application\vivaldi.exe you can use the following solution:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.binary_location=r'C:\Users\levir\AppData\Local\Vivaldi\Application\vivaldi.exe'
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', options=options)
driver.get('http://google.com/')

For future reference:
to make Vivaldi work with selenium you need to make sure of three things :
The correct version of ChromeDriver
Set selenium's driver to use Vivaldi's binary via webdriver.ChromeOptions()
Make sure you're getting the correct url (don't forget "https://")
All of the above is explained step by step with screenshots in this blog post

The key executable_path will be deprecated in the upcoming releases of Selenium.
This post has the solution. I'm posting a copy of said solution with the path to Vivaldi, where the username is fetched by the script so you don't have to hard code it.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
import os
current_user = os.getlogin()
s = Service(rf"C:\Users\{current_user}\AppData\Local\Vivaldi\Application\vivaldi.exe")
driver = webdriver.Chrome(service=s)
driver.get("http://duckduckgo.com") # or your website of choice

You can use ChromeOptions and supply binary.
from selenium.webdriver.chrome.options import Options
opt = Options()
opt.binary_location = chromium_path//path to chromium binary
driver = webdriver.Chrome(options=opt, executable_path="path_to_chromedriver")

Disable proxy via Python for Selenium with headless Chrome

I'm using Chrome with Selenium in Python 2.7.
I've tried to run Chrome in headless mode but it slows down my tests significantly.
One workaround shall be to disable the proxy settings but I don't know how to do it in python.
This is my code so far:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument('headless')
chrome_options.add_argument('--hide-scrollbars')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('???') # which option?
self.driver = webdriver.Chrome("C:\Python27\Scripts\chromedriver.exe", chrome_options=chrome_options)
Does anyone know how to solve this?

Try this:
chrome_options.add_argument('--no-proxy-server')

Splnter selenium headless with Chrome Canary (python)

I can drive a headless browser using selenium and Chrome Canary.
But I can't get it to work using Splinter.
Thanks in advance.
Here's what works.
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.binary_location = '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary'
driver = webdriver.Chrome(executable_path='/usr/local/bin/chromedriver', chrome_options=chrome_options)
Here's what doesn't work:
from splinter import Browser
executable_path = {'executable_path':'/Applications/Google Chrome Canary/Contents/MacOS/Google Chrome Canary'}
B=Browser('chrome',**executable_path)
Incidentally Splinter DOES work with phantomjs
executable_path = {'executable_path':'/Applications/phantomjs/bin/phantomjs'}
B=Browser('phantomjs',**executable_path )
The error message is
WebDriverException: Message: 'Google Chrome Canary' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
Here's the path, as perceived by os.os.environ['PATH']
/Users/jonschull-MBPR/miniconda2/bin:/Applications/Google Chrome Canary/Contents/MacOS/Google Chrome Canary:/Users/jonschull-MBPR/miniconda2/bin:/Users/jonschull-MBPR/anaconda/bin:/Users/jonschull-MBPR/Downloads/google-cloud-sdk/bin:/opt/local/bin:/opt/local/sbin:/Users/jonschull-MBPR/anaconda/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/chromedrive:/opt/X11/bin:/usr/local/mongodb/bin
And by the way, I've tried escaping "Google\ Chrome\ Canary "

Well, according to issue in splinter's git you just need to upgrade to splinter 0.7.6+.