I'm trying to make a bot in order to reset my router easily, so I'm using mechanize for this task.
import mechanize
br = mechanize.Browser()
br.set_handle_robots(False)
response = br.open("http://192.168.0.1/")
br.select_form(nr=0)
br.form['loginUsername']='support'
br.form['loginPassword']='71689637'
response=br.submit()
if(response.read().find("wifi")) != -1:
# ?????
If it finds the string 'wifi', it means the bot has logged in, but here's where I get stuck, because the restart button is in another tab (Another page, I guess that from the same object indicating the new URL it should be able to follow the redirection URL without logging off). However, the button from that tab is, well, a button but not a form.
Picture 1:
Picture 2:
And here's the source:
https://github.com/SharkiPy/Code-stackoverflow/blob/master/Source
Here is a start of code using Selenium, with hidden browser. You just have to add the actions you take when browsing through your router. I hope it can get you started!
import time
from selenium import webdriver
from selenium.common.exceptions import WebDriverException, NoSuchElementException,InvalidElementStateException,ElementNotInteractableException, StaleElementReferenceException, ElementNotVisibleException
from selenium.webdriver.common.keys import Keys
# There may be some unnecessary import above
from selenium.webdriver.chrome.options import Options
options_chrome = webdriver.ChromeOptions()
options_chrome.binary_location = 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe' # PATH to your chrome driver (you can also use firefox or any other browser, but options below will not be exactly the same
prefs = {"profile.default_content_setting_values.notifications" : 2} # disable notification by default
options_chrome.add_experimental_option("prefs",prefs)
#### below are options for headless chrome
#options_chrome.add_argument('headless')
#options_chrome.add_argument("disable-gpu")
#options_chrome.add_argument("--start-maximized")
#options_chrome.add_argument("--no-sandbox")
#options_chrome.add_argument("--disable-setuid-sandbox")
#### You should uncomment these lines when your code will be working
# starting browser :
browser = webdriver.Chrome( options=options_chrome)
# go to the router page :
browser.get("http://192.168.0.1/")
# connect
elem = browser.find_element_by_id("loginUsername")
elem.send_keys('support')
elem = browser.find_element_by_id("loginPassword")
elem.send_keys('71689637')
elem.send_keys(Keys.RETURN)
# here you need to find your button and click it
button = browser.find_element_by_[Whatever works for you]
button.click()
Related
I am learning to use Selenium and my goal is to open zoom through a python program on a Raspberry Pi 4. Upon running the pasted code, the program works as intended; opens zoom in browser, maximizes window, selects and clicks sign in, enters credentials and then presses enter. After log in is attempted I am given "error: Http 401 error". I am guessing this is because zoom is detecting an auto-login and blocking me. First off, am I correct? And if so, is there any way to get around this? Or does zoom block any auto filling of credentials.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
driver=webdriver.Chrome()
driver.get("https://zoom.us")
driver.maximize_window()
elem = driver.find_element(By.XPATH, "//a[contains(text(),'SIGN IN')]").click()
emailField = driver.find_element(By.XPATH, "//input[#id='email']")
emailField.send_keys("email") #"email" replaced with zoom login
passField = driver.find_element(By.XPATH, "//input[#id='password']")
passField.send_keys("password") #"password" replaced with zoom password
passField.send_keys(Keys.RETURN)
I faced the same issue, and was able to get around the bot-detection by using
the undetected-chromedriver package.
replace
from selenium import webdriver
...
driver = webdriver.chrome()
with
import undetected_chromedriver.v2 as ucdriver
...
driver = ucdriver.Chrome()
Add this to your code, right after the libraries import, and it will work:
options = webdriver.ChromeOptions()
options.add_argument('--disable-blink-features=AutomationControlled')
#in executable_path put the path to your chromedriver.exe
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://zoom.us")
Goal is to scrape information from people/users
Here is my code - Trying to get into the url to eventually scrape data from the search.
However when executing the code, it prompts the log in page. This is where im currently stuck
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
productlinks=[]
test1=[]
options = Options()
options.headless = True
driver = webdriver.Chrome(ChromeDriverManager().install())
url = "https://www.linkedin.com/search/results/people/?currentCompany=%5B%221252860%22%5D&geoUrn=%5B%22103644278%22%5D&keywords=sales&origin=FACETED_SEARCH&page=2"
driver.get(url)
time.sleep(3)
username = driver.find_element_by_class_name('login-email')
username.send_keys('example123#gmail.com')
password = driver.find_element_by_class_name('login-password')
password.send_keys('Password123')
log_in_button = driver.find_element_by_class_name('login-submit')
log_in_button.click()
There are 3 methods
Add the login code logic in your code by clicking the login button and send the login credentials using send_keys()
Disable headless by removing options.headless = True and manually login yourself
Since LinkedIn uses cookies to validate session, so you can login once and store up the cookies somewhere else and inject back to your session every time you launch selenium driver.
For getting the cookies,
# Go to the correct domain
driver.get("https://www.example.com")
# get all the cookies from this domain
driver = browser.get_cookies()
# store it somewhere, maybe a text file
or do it manually and copy from Chrome Dev Tools
For restoring the cookies
# Go to the correct domain
driver.get("https://www.example.com")
# get back the cookies
cookies = {‘name’ : ‘foo’, ‘value’ : ‘bar’}
driver.add_cookies(cookies)
Reference: LinkedIn Cookies Policy
I am trying to acces whatsapp web with python without having to scan the QR code everytime I restart the program (because in my normal browser I also dont have to do that). But how can I do that? Where is the data stored that tells whatsapp web to connect to my phone? And how do I save this data and send it to the browser when I rerun the code?
I already tried this because someone told me I should save the cookies:
from selenium import webdriver
import time
browser = None
cookies = None
def init():
browser = webdriver.Firefox(executable_path=r"C:/Users/Pascal/Desktop/geckodriver.exe")
browser.get("https://web.whatsapp.com/")
time.sleep(5) # in this time I scanned the QR to see if there are cookies
cookies = browser.get_cookies()
print(len(cookies))
print(cookies)
init()
Unfortunately there were no cookies..
The output was 0 and [].
How do I fix this probblem?
As mentioned in the answer to this question, pass your Chrome profile to the Chromedriver in order to avoid this problem. You can do it like this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Path") #Path to your chrome profile
driver = webdriver.Chrome(executable_path="C:\\Users\\chromedriver.exe", options=options)
This one works for me, I just created a folder, on the home directory of the script and a little modifications and it works perfectly.
###########
E_PROFILE_PATH = "user-data-dir=C:\Users\Denoh\Documents\Project\WhatBOts\SessionSaver"
##################
This is the Config File that I will import later
##################
The main script starts here
##################
from selenium import webdriver
from config import E_PROFILE_PATH
options = webdriver.ChromeOptions()
options.add_argument(E_PROFILE_PATH)
driver = webdriver.Chrome(executable_path='chromedriver_win32_86.0.4240.22\chromedriver.exe', options=options)
driver.get('https://web.whatsapp.com/')
When selenium opens youtube, I am not signed in and when I try to sign in, it says the following:
"This browser or app may not be secure. Learn more
Try using a different browser. If you’re already using a supported browser, you can refresh your screen and try again to sign in."
Is there any way to sign in
This is the code:
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
browser = webdriver.Chrome("C:\\Users\\Pranav Sandeep\\Downloads\\chromedriver.exe")
browser.get('https://www.youtube.com')
SearchBar = browser.find_element_by_name("search_query").send_keys("Selenium", Keys.ENTER)
Video = browser.find_element_by_id("video-title")
Video.click()
You need to first sign in to google and then you can go to youtube. Use this code below.
Use Seleniumwire with undetected browser v2
Note: put chromedriver in your sys path.
from seleniumwire.undetected_chromedriver.v2 import Chrome, ChromeOptions
import time
options = {}
chrome_options = ChromeOptions()
chrome_options.add_argument('--user-data-dir=hash')
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--incognito")
chrome_options.add_argument("--disable-dev-shm-usage")
# chrome_options.add_argument("--headless")
browser = Chrome(seleniumwire_options=options, options=chrome_options)
browser.get('https://gmail.com')
browser.find_element_by_xpath('//*[#id="identifierId"]').send_keys('your-email')
browser.find_element_by_xpath('//*[#id="identifierNext"]/div/button').click()
time.sleep(5)
browser.find_element_by_xpath('//*[#id="password"]/div[1]/div/div[1]/input').send_keys('you-password')
browser.find_element_by_xpath('//*[#id="passwordNext"]/div/button').click()
time.sleep(5)
browser.get('https://www.youtube.com')
In addition to this, selenium wire has many awesome features, check out Github repository
I'm working on trying to automate a game I want to get ahead in called pokemon vortex and when I login using selenium it works just fine, however when I attempt to load a page that requires a user to be logged in I am sent right back to the login page (I have tried it outside of selenium with the same browser, chrome).
This is what I have
import time
from selenium import webdriver
from random import randint
driver = webdriver.Chrome(r'C:\Program Files (x86)\SeleniumDrivers\chromedriver.exe')
driver.get('https://zeta.pokemon-vortex.com/dashboard/');
time.sleep(5) # Let the user actually see something!
usernameLoc = driver.find_element_by_id('myusername')
passwordLoc = driver.find_element_by_id('mypassword')
usernameLoc.send_keys('mypassword')
passwordLoc.send_keys('12345')
submitButton = driver.find_element_by_id('submit')
submitButton.submit()
time.sleep(3)
driver.get('https://zeta.pokemon-vortex.com/map/10')
time.sleep(10)
I'm using python 3.6+ and I literally just installed selenium today so it's up to date, how do I force selenium to hold onto cookies?
Using a pre-defined user profile might solve your problem. This way your cache will be saved and will not be deleted.
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--user-data-dir=C:/Users/user_name/AppData/Local/Google/Chrome/User Data")
driver = webdriver.Chrome(options=options)
driver.get("xyz.com")