I'm trying to use a Proxy in selenium python, but when i test the code on a web page that show my ip the proxy doesn't change my ip adress.
Here there is the code
import undetected_chromedriver as uc
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
option = Options()
driver = uc.Chrome(options = option)
PROXY = "139.162.214.146"
option.add_argument('--proxy-server=%s' % PROXY)
option.add_argument('--disable-extensions')
option.add_argument('--disable-notifications')
option.add_argument('--no-sandbox')
option.add_argument('--disable-dev-shm-usage')
option.add_argument('--disable-gpu')
option.add_argument('--mute-audio')
option.add_argument('--ignore-certificate-errors')
option.add_argument('--disable-logging')
driver.get("https://www.whatismyip.com/")
input ()
The code should change mi ip but this doesn't appen, i tried to change the proxy but the problem is the same.
I can see multiple issues here. Try using the following code:
import undetected_chromedriver as uc
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
option = Options()
PROXY = "139.162.214.146"
option.add_argument('--proxy-server=%s' % PROXY)
option.add_argument('--disable-extensions')
option.add_argument('--disable-notifications')
option.add_argument('--no-sandbox')
option.add_argument('--disable-dev-shm-usage')
option.add_argument('--disable-gpu')
option.add_argument('--mute-audio')
option.add_argument('--ignore-certificate-errors')
option.add_argument('--disable-logging')
driver = uc.Chrome(options = option)
driver.get("https://www.whatismyip.com/")
input ()
Also note, that you need to specify a port for your proxy as following: PROXY = "139.162.214.146:proxy_port"
Related
I'm using Selenium webdriver to open a webpage and I set up a proxy for the driver to use. The code is listed below:
PATH = "C:\Program Files (x86)\chromedriver.exe"
PROXY = "212.237.16.60:3128" # IP:PORT or HOST:PORT
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument(f'--proxy-server={PROXY}')
proxy = Proxy()
proxy.auto_detect = False
proxy.http_proxy = PROXY
proxy.sslProxy = PROXY
proxy.socks_proxy = PROXY
capabilities = webdriver.DesiredCapabilities.CHROME
proxy.add_to_capabilities(capabilities)
driver = webdriver.Chrome(PATH, chrome_options=chrome_options,desired_capabilities=capabilities)
driver.get("https://whatismyipaddress.com")
The problem is that the web driver is not using the given proxy and it accesses the page with my normal IP. I already tried every type of code I could find on the internet and it didn't work. I also tried to set a proxy directly in my pc settings and when I open a normal chrome page it works fine (it's not a proxy server problem then), but if I open a page with the driver it still uses my normal IP and somehow bypasses the proxy. I also tried changing the proxy settings of the IDE (pycharm) and still it's not working. I'm out of ideas, could someone help me?
This should work.
Code snippet-
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
PROXY = "212.237.16.60:3128"
#add proxy in chrome_options
chrome_options.add_argument(f'--proxy-server={PROXY}')
driver = webdriver.Chrome(PATH,options=chrome_options)
#to check new IP
driver.get("https://api.ipify.org/?format=json")
Note:- chrome_options is deprecated now, you have to use options instead
I have a code for making temp mail automatically but I have a problem.
This is code:
import time
from selenium import webdriver
browser = webdriver.Chrome()
browser.get("https://temp-mail.org/")
time.sleep(10)
browser.close()
The link opens correctly but I can't pass cloudflare.
Also, I see some errors on my console:
Thanks...
Try adding user agent argument in chrome options and set user agent to any value
ops = Options()
ua='me'
ops.add_argument('--user-agent=%s' % ua)
driver=uc.Chrome(executable_path=r"C:\chromedriver.exe",chrome_options=ops)
Alternatively try using undetected-chromedriver
import undetected_chromedriver as uc
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
driver = uc.Chrome(options=options)
driver.get("https://temp-mail.org/")
I try to connect http://username:password#ip:port proxy and options.add_argument(f'--proxy-server=%s' % proxy) not working. Can you help me?..
To connect a proxy through ChromeDriver you can use the following code block:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
PROXY = "ip:port"
options = Options()
options.add_argument('--proxy-server=%s' % PROXY)
driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
This code works for Windows where it launches Chrome connected via Tor. Keep in mind you have to have Tor browser running beforehand. How can I enable the user-profile and start the browser logged in? I have tried the regular method. I have only 1 profile. Default. Doesn't seem to be working. Any clues?
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
tor_proxy = "127.0.0.1:9150"
chrome_options = Options()
'''chrome_options.add_argument("--test-type")'''
chrome_options.add_argument('--ignore-certificate-errors')
'''chrome_options.add_argument('--disable-extensions')'''
chrome_options.add_argument('disable-infobars')
'''chrome_options.add_argument("--incognito")'''
chrome_options.add_argument('--user-data=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data\\Default')
chrome_options.add_argument('--proxy-server=socks5://%s' % tor_proxy)
driver = webdriver.Chrome(executable_path='C:\\chromedriver.exe', options=chrome_options)
driver.get('https://www.gmail.com')
time.sleep(4)
driver.switch_to.frame(0)
driver.find_element_by_id("introAgreeButton").click()
Use this instead.
chrome_options.add_argument("user-data-dir=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data")
you don't have to specify the profile directory (Default) as it is used by default if nothing is explicitly specified using below code. SO in your case use only the above line of code
chrome_options.add_argument("profile-directory=Profile 1")
Eg:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument(r"user-data-dir=C:\Users\prave\AppData\Local\Google\Chrome\User Data")
driver =webdriver.Chrome("./chromedriver.exe",options=options)
driver.get(url)
Output:
The challenge I see is that, through selenium, I am trying to click on a website element (a div with some js attached). The "button" navigates you to another page.
How can I configure the browser to automatically route the requests through a proxy?
My proxy is set up as follows:
http://api.myproxy.com?key=AAA111BBB6&url=http://awebsitetobrowse.com
I am trying to put webdriver (chrome) behind the proxy
from selenium import webdriver
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(chrome_options=options)
where options, so far, is some basic configuration of the browser window size.
I have seen quite some examples (ex1, ex2, ex3) but I somehow fail to find an example that suits my needs.
import os
dir_path = os.path.dirname(os.path.realpath(__file__)) + "\\chromedriver.exe"
PROXY = "http://api.scraperapi.com?api_key=1234&render=true"
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
driver = webdriver.Chrome(executable_path = dir_path, chrome_options=chrome_options)
driver.get("https://stackoverflow.com/questions/11450158/how-do-i-set-proxy-for-chrome-in-python-webdriver")
Though it seems like the Proxy address you are using is not an actual proxy it is an API that returns HTML content of page itself after handling proxies, captcha or any IP blocking. But still for different scenario there can be different solution. some of those are as follow.
Scenario 1
So according to me, you are using this API in the wrong manner if your
api provide the facility to return the response of your visited page through the proxy.
So it should be used directly in 'driver.get()' with
address="http://api.scraperapi.com/?api_key=YOURAPIKEY&url="+url_to_be_visited_via_api
Example code for this would look like:
import os
dir_path = os.path.dirname(os.path.realpath(__file__)) + "\\chromedriver.exe"
APIKEY=1234 #replace with your API Key
apiURL = "http://api.scraperapi.com/?api_key="+APIKEY+"&render=true&url="
visit_url = "https://stackoverflow.com/questions/11450158/how-do-i-set-proxy-for-chrome-in-python-webdriver"
from selenium import webdriver
driver = webdriver.Chrome(executable_path = dir_path)
driver.get(apiURL+visit_url)
Scenario 2
But if you have some API that provides proxy address and login
credentials in response then it can be fudged in chrome options to use
it with chrome itself.
This should be in case if response of api is something like
"PROTOCOL://user:password#proxyserver:proxyport" (In case of authentication)
"PROTOCOL://proxyserver:proxyport" (In case of null authentication)
In both cases PROTOCOL can like HTTP, HTTPS, SOCKS4, SOCKS5 etc.
And that code should look like:
import os
dir_path = os.path.dirname(os.path.realpath(__file__)) + "\\chromedriver.exe"
import requests
proxyapi = "http://api.scraperapi.com?api_key=1234&render=true"
proxy=requests.get(proxyapi).text
visit_url = "https://stackoverflow.com/questions/11450158/how-do-i-set-proxy-for-chrome-in-python-webdriver"
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server='+proxy)
driver = webdriver.Chrome(executable_path = dir_path, chrome_options=chrome_options)
driver.get(visit_url)
Scenario 3
But if you have some API itself is a proxy with null authentication, then it can be fudged in chrome options to use
it with chrome itself.
And that code should look like:
import os
dir_path = os.path.dirname(os.path.realpath(__file__)) + "\\chromedriver.exe"
proxyapi = "http://api.scraperapi.com?api_key=1234&render=true"
visit_url = "https://stackoverflow.com/questions/11450158/how-do-i-set-proxy-for-chrome-in-python-webdriver"
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server='+proxyapi)
driver = webdriver.Chrome(executable_path = dir_path, chrome_options=chrome_options)
driver.get(visit_url)
So the solution can be used as per the different scenario.
Well, after countless of experiments, I have figure out that the thing works with:
apiURL = "http://api.scraperapi.com/?api_key="+APIKEY+"&render=true&url="
while fails miserably with
apiURL = "http://api.scraperapi.com?api_key="+APIKEY+"&render=true&url="
I have to admit my ignorance here: I thought the two should be equivalent