I am using selenuim webdriver to get a page with https scheme. if i go to the page using chrome i get the 'privacy error' and the response is empty.
If i use webdriver to get the page the webdriver stay on the last page it was in. How can i detect the webdriver did not move to the new page ?
I thought about checking if the driver.current_url is same as the url i want but it doesnt feel like the right solution.
Code:
from selenium import webdriver
driver = webdriver.PhantomJS()
driver.get('http://doman.com/1.html') # this return 200 status and content
driver.get('https://domain2.com/2.html') # this return the privacy error
# so the driver is still at the first url
I think i have found a solution after reading support desired_capabilities: acceptSslCerts=True
from selenium import webdriver
service_args = ['--ignore-ssl-errors=true']
driver = webdriver.PhantomJS(service_args=service_args)
driver.get('http://doman.com/1.html')
driver.get('https://domain2.com/2.html') # now it returns 404 as it should
Related
I'm trying to use Selenium (3.141.0) with ChromeDriver (87.0.4280) to access a page. When accessed manually, it brings me to a policy page (different URL) where you have to hit 'Ok' before continuing to the site. Edit This is using Win 10 and I have the folder with the chromedriver on PATH.
When using the following code, I'm able to get to the policy page with the ("--headless") option but without it I get a blank page with 'data:,' in the URL and nothing else loads. I've tried accessing straight from the policy page and the site URL but they both get stuck when the webdriver is created. Am I missing something? I'm open to any suggestions, thanks!
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
driver_path = 'D:\....\chromedriver.exe'
driver = webdriver.Chrome(executable_path= driver_path, options= chrome_options)
driver.get(...) # left out the url
This is the output page I get without using ("--headless")
Funny enough, I realized it was because my Chrome Developer tools had become disabled. Not sure how but when I re-enabled them, it worked perfectly again. Weird.
I am running a script with Selenium that loops over a number of URLs. However, some URLs give a 503 service error. For example the following code gives me such an error.
import requests
import selenium
from selenium import webdriver
url = 'https://destyy.com/q3P5W7'
driver = webdriver.Chrome()
driver.get(url)
response = requests.get(driver.current_url).status_code
I tried refreshing the browser in the case of an error, but then the error persists.
if response == 503:
driver.refresh()
I noticed that if I manually select the url in the address bar of my driver and press enter, the page loads to the desired location. I tried to write this into code but it fails:
from selenium.webdriver.common.keys import Keys
address_bar = driver.find_element_by_xpath('/html/body').send_keys(Keys.CONTROL, "l")
address_bar.sendKeys(Keys.ENTER)
Please help me out. How can I load the page? Thanks in advance,
I am using selenium webdriver to open a page in my website using python.
The page link is : "www.example.com/App/Details.aspx/I=5aM%+8KbCv1o=&T=M30Lr7RtcdR=&H=fRFKse5fKA=="
Since I cannot open this page directly, I have to first login on
www.example.com, which I am doing this way:
from selenium.common.exceptions import TimeoutException
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
try:
driver.set_page_load_timeout(10)
driver.get('http://www.example.com')
except Exception:
print ('time out')
#driver.find_element_by_id("Default").send_keys(Keys.CONTROL +'Escape')
username = driver.find_element_by_id("usrid")
password = driver.find_element_by_id("pswd")
username.send_keys("Sample_user")
password.send_keys("samplepass")
driver.find_element_by_id("submitbtn").click()
After this I am am able to log in successfully,
but when I redirect to the above link using
driver.get("www.example.com/App/Details.aspx/I=5aM%+8KbCv1o=&T=M30Lr7RtcdR=&H=fRFKse5fKA==")
I get error saying please login first. Please suggest a solution for the same.
You sent the data via
password.send_keys
but after that you were to also send enter command using
password.send_keys((Keys.ENTER)
After this you have to have a
sleep(1)
or
WebDriverWait Until(expect...
before you open the next page
I just want to refresh an already opened web page with Selenium.
It always opens a new browser window.
What I'm doing wrong?
from selenium import webdriver
import urllib
import urllib2
driver = webdriver.Firefox()
driver.refresh()
I would suggest binding the driver element search to the tag body and use the refresh command of the browser.
In OSX for example
driver.find_element_by_tag_name('body').send_keys(Keys.COMMAND + 'r')
Documentation on keys here: http://selenium-python.readthedocs.org/en/latest/api.html
Update:
The following code, very similar to your one, works fine for me.
driver = webdriver.Firefox()
driver.get(response.url) #tested in combination with scrapy
time.sleep(3)
driver.refresh()
Are you sure you correctly load the web page with the driver before refreshing it ?
The problem is you are opening the webdriver and then trying to refresh when you have not specified a URL.
All you need to do is get your desired URL before refreshing:
from selenium import webdriver
import urllib
import urllib2
driver = webdriver.Firefox()
driver.get("Your desired URL goes here...")
#now you can refresh the page!
driver.refresh()
The following codes work for me
driver.get(driver.current_url)
sleep(2)
driver.refresh()
I use python 3.7.6, selenium 3.141.0
You are trying to refresh the page before it loads so u can use a sleep function
from time import sleep
sleep(1)
or you can wait for an XPath to load so
WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, xpath goes here)))
For me helped
from selenium import webdriver
import time
driver = webdriver.Firefox()
driver.get("URL")
time.sleep(5)
driver.refresh()
I got mine fixed by adding "browser.refresh()" the for loop or while loop.
You can try any one of the below methods for the same.
Method 1:
driver.findElement(By.name("s")).sendKeys(Keys.F5);
Method 2:
driver.get(driver.getCurrentUrl());
Method3:
driver.navigate().to(driver.getCurrentUrl());
Method4:
driver.findElement(By.name("s")).sendKeys("\uE035");
I am having a strange issue with PhantomJS or may be I am newbie. I am trying to login on NewEgg.com via Selenium by using PhantomJS. I am using Python for it. Issue is, when I use Firefox as a driver it works well but as soon as I set PhantomJS as a driver it does not go to next page hence give message:
Exception Message: u'{"errorMessage":"Unable to find element with id \'UserName\'","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"89","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:55372","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\\"using\\": \\"id\\", \\"sessionId\\": \\"aaff4c40-6aaa-11e4-9cb1-7b8841e74090\\", \\"value\\": \\"UserName\\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/aaff4c40-6aaa-11e4-9cb1-7b8841e74090/element"}}' ; Screenshot: available via screen
The reason I found after taking screenshot that phantom could not navigate the page and script got finished. How do I sort this out? Code Snippet I tried given below:
import requests
from bs4 import BeautifulSoup
from time import sleep
from selenium import webdriver
import datetime
my_username = "user#mail.com"
my_password = "password"
driver = webdriver.PhantomJS('/Setups/phantomjs-1.9.7-macosx/bin/phantomjs')
firefox_profile = webdriver.FirefoxProfile()
#firefox_profile.set_preference('permissions.default.stylesheet', 2)
firefox_profile.set_preference('permissions.default.image', 2)
firefox_profile.set_preference('dom.ipc.plugins.enabled.libflashplayer.so', 'false')
#driver = webdriver.Firefox(firefox_profile)
driver.set_window_size(1120, 550)
driver.get('http://newegg.com')
driver.find_element_by_link_text('Log in or Register').click()
driver.save_screenshot('screen.png')
I even put sleep but it is not making any difference.
I experienced this with PhantomJS when the content type of the second page is not correct. A normal browser would just interpret the content dynamically, but Phantom just dies, silently.