My goal is to be able to get the main text content of a webpage without anything else.
Firefox do this using with the reader view feature. It seems that Chrome haves this as experimental feature. Despite activating the feature from code, the icon doesn't show up.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service as ChromiumService
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.core.utils import ChromeType
chrome_options = Options()
chrome_options.add_argument("--reader-mode=true")
chrome_options.add_argument("--enable-reader-mode")
driver = webdriver.Chrome(service=ChromiumService(ChromeDriverManager(chrome_type=ChromeType.CHROMIUM).install()), options=chrome_options)
driver
url = "https://www.google.com"
driver.get(url)
print(driver.page_source)
driver.quit()
The command line information chrome://version/ in are showing the parameters :
Command Line
/Applications/Google Chrome.app/Contents/MacOS/Google Chrome --allow-pre-commit-input --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-automation --enable-blink-features=ShadowDOMV0 --enable-logging --enable-reader-mode --log-level=0 --no-first-run --no-service-autorun --password-store=basic --reader-mode=true --remote-debugging-port=0 --test-type=webdriver --use-mock-keychain --user-data-dir=/var/folders/qx/y0sp0wpn5g524st9yn6f3cdm0000gn/T/.com.google.Chrome.L3pFOZ --flag-switches-begin --flag-switches-end
chrome://flags/ doesn't show the reader mode activated. Even if I activate it manually and restart, the reader mode icon is not shown in the binaries as well as Chrome on mac.
Is this feature still exists? If yes, how do I use it from selenium?
I'm unsure what you mean by this in your question:
My goal is to be able to get the main text content of a webpage
without anything else
Also I'm not sure that the search page Google.com has a reader mode.
The following code is able to toggle the Reader Mode on using selenium with a Chrome browser.
from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--disable-popup-blocking")
chrome_options.add_argument('--ignore-certificate-errors')
# reference:
# https://source.chromium.org/chromium/chromium/src/+/main:docs/accessibility/browser/reader_mode.md
chrome_options.add_argument("--enable-features=ReaderMode")
# disable the banner "Chrome is being controlled by automated test software"
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ['enable-automation'])
driver = webdriver.Chrome('/usr/local/bin/chromedriver', options=chrome_options)
url_main = "https://www.cnn.com/us/live-news/baldwin-rust-shooting-charges-decision/index.html"
driver.get(url_main)
sleep(60)
driver.quit()
Here is a screenshot showing the webpage without the reader icon using selenium with a Chrome browser.
Here is a screenshot showing the webpage with the reader icon using selenium with a Chrome browser.'
My development environment is:
----------------------------------------
My system information
----------------------------------------
Platform: macOS
Python: 3.9.0
Selenium: 3.141.0
Chromedriver: 109.0.5414.74
----------------------------------------
Related
I want to open google page with a python code in visual studio editor using the following code :
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
#browser = webdriver.Chrome(ChromeDriverManager().install())
Path = "C:\Program Files (x86)\chromedriver.exe"
browser = webdriver.Chrome(Path)
browser.get('https://www.google.com/')
My problem is the chrome window it opens for 2 or 3 secondes and it close. Did some one face this issue ?
The default framework of visual studio editor would close the Chrome browser, per say any browser client after the last line of your program gets executed.
Hence, once the line of code:
browser.get('https://www.google.com/')
gets executed, the Chrome browser is CLOSED. This is the expected behavior.
Incase you like to keep the Chrome browser OPEN even after the last line of your program gets executed, you can pass the experimental option detach as true as follows:
selenium4 compatible code
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_experimental_option("detach", True)
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get('https://www.google.com/')
webdriver_manager site has a code to start webdriver with brave, but instead of brave it opens it with google chrome. My selenium version 4.6.0 gave the following code for selenium 4 (I also tried the given codes for selenium 3) as it can be seen on the site, but the webdriver still opens with chrome
# selenium 4
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as BraveService
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.core.utils import ChromeType
driver = webdriver.Chrome(service=BraveService(ChromeDriverManager(chrome_type=ChromeType.BRAVE).install()))
driver.get("https://pypi.org/project/webdriver-manager/")
Okay, so you just want to start a driver with Brave browser instead of chrome? Here is how I do that simplely, keep in mind I'm on Mac. You need the binaryPATH to the application.
add_block_ext = "Path to .crx extension"
driverPath = 'chromedriver'
binaryPath = '/Applications/Brave Browser.app/Contents/MacOS/Brave Browser'
options = webdriver.ChromeOptions()
options.binary_location = binaryPath
options.add_extension(add_block_ext)
browser = webdriver.Chrome(executable_path=binaryPath, chrome_options=options)
browser.get("https://www.google.com")
I am writing code to make selenium take over an instance of chrome that has all my bookmarks and stuff. So I created a chrome profile and I have the command
chrome.exe --remote-debugging-port=9222 --user-data-dir='C:\\selenium\\ChromeProfile'
this works when you run it in the python terminal, but I can't just put it into the code.
I have tried using
import os
os.system("chrome.exe --remote-debugging-port=9222 --user-data-dir='C:\\selenium\\ChromeProfile'")
but that returns with
Failed To Create Data Directory: Google Chrome cannot read and write to its data directory: C\selenium\ChromeProfile
Does someone know what I am doing wrong or a command that will run the chrome command to open this specific profile?
My total code so far looks like this
import subprocess
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
os.system("chrome.exe --remote-debugging-port=9222 --user-data-dir='C:\\selenium\\ChromeProfile'")
chrome_options = Options()
chrome_options.add_experimental_option("debuggerAddress", "127.0.0.1:9222")
chrome_driver = "C:\\selenium\\chromedriver.exe"
driver = webdriver.Chrome(chrome_driver, chrome_options=chrome_options)
#Print website title to make sure its connected properly
driver.get('https://google.com')
print(driver.title)
search_bar = driver.find_element_by_name('q')
search_bar.send_keys('test')
Nevermind people, I figured it out. I am posting this here for anyone else in the future who may run into the same issue I did where you want python to open the browser in debug mode on the port.
Here is the code
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--remote-debugging-port=9222")
chrome_options.add_argument('user-data-dir=C:\\selenium\\ChromeProfile')
chrome_driver = "C:\\selenium\\chromedriver.exe"
driver = webdriver.Chrome(chrome_driver, chrome_options=chrome_options)
#Print website title to make sure its connected properly
driver.get('https://google.com')
print(driver.title)
search_bar = driver.find_element_by_name('q')
search_bar.send_keys('test')
I had to add two lines of
chrome_options.add_argument()
for some reason it didn't like when I put them in the same parenthesis.
I hope I help someone in the future.
I was recently experimenting with Selenium in Python and noticed that every time I created a new webdriver object and used webdriver.get(website), the webdriver window would open first (black window that sort of looks like the terminal) and then the actual tab with the website would open. Is there anyway to run the webdriver in the background so only the tab opens without the webdriver window opening?
EDIT: Currently using the Chrome webdriver.
to make chrome run in background you can add headless mode option
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://www.google.com')
print(driver.title)
driver.close()
I am currently trying to code a basic smartmirror for my coding II class in high school with python. One thing I'm trying to do is open new tabs in full screen (using chrome). I currently have it so I can open url's, but I am not getting them in full screen. Any ideas on code I can use to open chrome in full screen?
If you're using selenium, just code like below:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://google.com')
driver.maximize_window()
As suggested, selenium is a good way to accomplish your task.
In order to have it full-screen and not only maximized I would use:
chrome_options.add_argument("--start-fullscreen");
or
chrome_options.add_argument("--kiosk");
First option emulates the F11 pressure and you can exit pressing F11. The second one turns your chrome in "kiosk" mode and you can exit pressing ALT+F4.
Other interesting flags are:
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
Those will remove the top bar exposed by the chrome driver saying it is a dev chrome version.
The complete script is:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
# chrome_options.add_argument("--start-fullscreen");
chrome_options.add_argument("--kiosk");
driver = webdriver.Chrome(executable_path=rel("path/to/chromedriver"),
chrome_options=chrome_options)
driver.get('https://www.google.com')
"path/to/chromedriver" should point to the chrome driver compatible with your chrome version downloaded from here