I have this code that I run from two different files in two different consoles in Spyder. I need to do different tasks with each browser but it does create a second driver, the second instance of the script just use the first browser... How to separate the tasks and have two browser please ? I thought two different consoles was multithreading, this is not the case ?
chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
try:
browser = webdriver.Chrome(options=chrome_options)
except SessionNotCreatedException:
os.environ['WDM_LOG_LEVEL'] = '0'
browser = webdriver.Chrome(ChromeDriverManager().install(), options=chrome
browser.get('http://google.com')
I don't know if it is what you wanted but you can have 2 different browsers with that code for example :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver1 = webdriver.Chrome()
driver2 = webdriver.Chrome()
driver1.get('https://www.google.com/?gws_rd=ssl')
driver2.get('https://stackoverflow.com/')
Hope I helped :)
Hope this solve your problem.
from selenium import webdriver
amount = int(input('[?] amount browsers '))
driver = []
while amount != len(driver):
driver.append(webdriver.Chrome())
# if u want the to make a ctrl+c ctrl+v browser use this down here
for i in range(len(driver)):
driver[i].get('https://www.google.com/')
#else use this
driver[0].get('https://www.google.com/')
driver[1].get('https://stackoverflow.com/')
Related
I am writing code to make selenium take over an instance of chrome that has all my bookmarks and stuff. So I created a chrome profile and I have the command
chrome.exe --remote-debugging-port=9222 --user-data-dir='C:\\selenium\\ChromeProfile'
this works when you run it in the python terminal, but I can't just put it into the code.
I have tried using
import os
os.system("chrome.exe --remote-debugging-port=9222 --user-data-dir='C:\\selenium\\ChromeProfile'")
but that returns with
Failed To Create Data Directory: Google Chrome cannot read and write to its data directory: C\selenium\ChromeProfile
Does someone know what I am doing wrong or a command that will run the chrome command to open this specific profile?
My total code so far looks like this
import subprocess
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
os.system("chrome.exe --remote-debugging-port=9222 --user-data-dir='C:\\selenium\\ChromeProfile'")
chrome_options = Options()
chrome_options.add_experimental_option("debuggerAddress", "127.0.0.1:9222")
chrome_driver = "C:\\selenium\\chromedriver.exe"
driver = webdriver.Chrome(chrome_driver, chrome_options=chrome_options)
#Print website title to make sure its connected properly
driver.get('https://google.com')
print(driver.title)
search_bar = driver.find_element_by_name('q')
search_bar.send_keys('test')
Nevermind people, I figured it out. I am posting this here for anyone else in the future who may run into the same issue I did where you want python to open the browser in debug mode on the port.
Here is the code
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--remote-debugging-port=9222")
chrome_options.add_argument('user-data-dir=C:\\selenium\\ChromeProfile')
chrome_driver = "C:\\selenium\\chromedriver.exe"
driver = webdriver.Chrome(chrome_driver, chrome_options=chrome_options)
#Print website title to make sure its connected properly
driver.get('https://google.com')
print(driver.title)
search_bar = driver.find_element_by_name('q')
search_bar.send_keys('test')
I had to add two lines of
chrome_options.add_argument()
for some reason it didn't like when I put them in the same parenthesis.
I hope I help someone in the future.
I am using selenium and python in order to scrape data on a website.
The problem is I need to manually log in because there is a CAPTCHA after the login.
My question is the following : is there a way to start the program on a page that is already loaded ? (for example, here I would log to the website, solve the CAPTCHA manually, and then launch the program that would scrape the data)
Note: I have already been looking for an answer on SO but did not find it, might have missed it as it seems to be an obvious question.
don't open in headless mode. open in head mode.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
options = Options()
options.headless = False # Set false here
driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("http://google.com/")
print ("Headless Chrome Initialized")
time.sleep(30) # wait 30 seconds, this should give enough time to manually do the capture
# do other code here
driver.quit()
So basically i am working on a python script that loggs into a twitch account and stays there to generate a viewer.
But my main issue is how do i make this work for multiple accounts.
How to hide alle the Windows, and how can i handle multiple selenium windows ?
Is selenium even good for that or is there a other way ?
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--mute-audio")
driver = webdriver.Chrome("D:\Downloads\chromedriver_win32\chromedriver.exe", chrome_options=chrome_options)
driver.minimize_window()
driver.get('https://www.twitch.tv/login')
search_form = driver.find_element_by_id('login-username')
search_form.send_keys('user')
search_form = driver.find_element_by_id('password-input')
search_form.send_keys('password')
search_form.submit()
driver.implicitly_wait(10)
driver.get('https://www.twitch.tv/channel')
You are definitely able to use Selenium and Python to do this. To run multiple accounts, you will have to utilize multi-threading or create multiple driver objects to manage.
Multithreading example from this thread:
from selenium import webdriver
import threading
import time
def test_logic():
driver = webdriver.Firefox()
url = 'https://www.google.co.in'
driver.get(url)
# Implement your test logic
time.sleep(2)
driver.quit()
N = 5 # Number of browsers to spawn
thread_list = list()
# Start test
for i in range(N):
t = threading.Thread(name='Test {}'.format(i), target=test_logic)
t.start()
time.sleep(1)
print t.name + ' started!'
thread_list.append(t)
# Wait for all thre<ads to complete
for thread in thread_list:
thread.join()
print 'Test completed!'
Each driver will have to use a proxy connection to connect to Twitch on separate IP addresses. I suggest using Opera as it has a built in VPN, makes it a lot easier.
Example of Opera and Selenium from this thread:
from selenium import webdriver
from time import sleep
# The profile directory which Opera VPN was enabled manually using the GUI
opera_profile = '/home/user-directory/.config/opera'
options = webdriver.ChromeOptions()
options.add_argument('user-data-dir=' + opera_profile)
driver = webdriver.Opera(options=options)
driver.get('https://whatismyipaddress.com')
sleep(10)
driver.quit()
To hide the console for webdrivers you must run them with the "headless" option.
Headless for chrome driver.
from selenium import webdriver from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
Unfortunately headless is not supported in Opera driver, so you must use Chrome or Firefox for this.
Good luck!
hi you will not be able to create a bot with selenium because even if you manage to connect several accounts on the twitch account, twitch (like youtube) have a system that looks at your IP address and does not increase the number of views if the multiple connection come from the same computer.
I have been trying to open multiple browser windows in internet explorer using webdriver in selenium. Once it reaches the get(url) line, it just halts there and eventually times out. I've added a print line, which does not execute. I've tried various methods and the one below is the Ie version of code I used to open multiple tabs in Chrome. Even if I remove the first 3 lines, it still only goes up to opening google.com. I've looked googled this issue and looked through other posts but nothing has helped. Would really appreciate any advice, thanks!
options = webdriver.IeOptions()
options.add_additional_option("detach", True)
driver = webdriver.Ie(options = options, executable_path=r'blahblah\IEDriverServer.exe')
driver.get("http://google.com")
print("syrfgf")
driver.execute_script("window.open('about:blank', 'tab2');")
driver.switch_to.window("tab2")
driver.get("http://yahoo.com")
You need to replace the url you have provided:
http://google.com
with a proper url as follows:
https://www.google.com/
Which should be represented as per the syntax diagram as follows:
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
driver = webdriver.Chrome()
driver.get("https://github.com")
signin_link = driver.find_element(By.LINK_TEXT, "Sign in")
signin_link.click()
time.sleep(1)
user = driver.find_element(By.ID, "login_field")
user.send_keys("X")
passw = driver.find_element(By.ID, "password")
passw.send_keys("X")
passw.submit()
time.sleep(5)
driver.close()
I had this issue and writing this code seems to have made it work flawlessly. Adjust the sleep time as you want it. Putting my chromedriver.exe into my project folder also helped with some errors
I want to iteratively search for 30+ items through a search button in webpage and scrape the related data.
My search items are stored in a list: vol_list
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome("driver path")
driver.get("web url")
for item in vol_list :
mc_search_box = driver.find_element_by_name("search_str")
mc_search_box.clear()
search_box.send_keys(item)
search_box.send_keys(Keys.RETURN)
After search is complete I will proceed to scrape the data for each item and store in array/list.
Is it possible to repeat this process without opening browser for every item in the loop?
You can't use chrome and other browsers without opening it.
In your case, headless browsers should do the job. Headless browsers simulates browser, but doesn't have GUI.
Try ghost driver/ html unit driver/ NodeJS. Then you will have to modify at least this line with the driver you want to use:
driver = webdriver.Chrome("driver path")
Good luck!
If you're using firefox, you can apply the headless option:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(options=options)
driver.get('your url')