link = "https://www.google.com"
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % str(xxx))
chrome = webdriver.Chrome(chrome_options=chrome_options)
time.sleep(3)
chrome.get(link)
print("po get")
time.sleep(1)
chrome.get(link)
time.sleep(15)
Hello, I have a problem with selenium for a long time and I would like to find a way how to fix it
The problem is that almost everytime I run a script that's opening selenium / even when I use it like
for a test such as :
from selenium import webdriver
chrome = webdriver.Chrome()
chrome.get(https://www.google.com)
It still sometimes doesn't get the website, I thought it's because of how slow selenium opens but even after its nicely open it doesn't get the value, sadly it just gets stuck on an empty browser that had data in an url window... any idea what should I do to fix it?
Okay, after a few hours I decided to give it a little try and change ( "" ) to ( ' ' ) and as I can see it works :D I dont know why it has a problem with string ""
here is edited line of my code :
chrome.get('https://www.google.com')
( I ve tried it with proxies with loop that was starting the webdriver
100 times , and everytime after i've changed it, it passed )
You can use driver.navigate.to("");
Also as I see, You might missing String Double Quotes " " here, chrome.get(https://www.google.com)
I think it's too late for answer, but I want to answer anyway. You may have already solved problem, but if it doesn't work check this out:
Different links navigate you same site or etc. It is not a code error but I want to say that this could cause of selenium.common.exceptions.NoSuchElementException: Message: no such element
You logged a site and if you want to go back login screen, you should log out of your account, otherwise the site will automatically navigate to the home screen because you already logged in.
That's all I wanted to say.
Related
I'm very new and learning web scraping in python by trying to get the search results from the website below after a user types in some information, and then print the results. Everything works great up until the very last 2 lines of this script. When I include them in the script, nothing happens. However, when I remove them and then just try typing them into the shell after the script is done running, they work exactly as I'd intended. Can you think of a reason this is happening? As I'm a beginner I'm also super open if you see a much easier solution. All feedback is welcome. Thank you!
#Setup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import time
#Open Chrome
driver = webdriver.Chrome()
driver.get("https://myutilities.seattle.gov/eportal/#/accountlookup/calendar")
#Wait for site to load
time.sleep(10)
#Click on street address search box
elem = driver.find_element(By.ID, 'sa')
elem.click()
#Get input from the user
addr = input('Please enter part of your address and press enter to search.\n')
#Enter user input into search box
elem.send_keys(addr)
#Get search results
elem = driver.find_element(By.XPATH, ('/html/body/app-root/main/div/div/account-lookup/div/section/div[2]/div[2]/div/div/form/div/ul/li/div/div[1]'))
print(elem.text)
I haven't used Selenium in a while, so I can only point you in the right direction. It seems to me you need to iterate over the individual entries, and print those, as opposed to printing the entire div as one element.
You should remove the parentheses from the xpath expression
You can shorten the xpath expression as follows:
Code:
elems = driver.find_element(By.XPATH, '//*[#class="addressResults"]/div')
for elem in elems:
print(elem.text)
You are using an absolute XPATH, what you should be looking into are relative XPATHs
Something like this should do it:
elems = driver.find_elements(By.XPATH, ("//*[#id='addressResults']/div"))
for elem in elems:
...
I ended up figuring out my problem - I just needed to add in a bit that waits until the search results actually load before proceeding on with the script. tossing in a time.sleep(5) did the trick. Eventually I'll add a bit that checks that an element has loaded before proceeding with the script, but this lets me continue for now. Thanks everyone for your answers!
I hope you all feel well. I tried coding an "gmail creator".
This Project is really important to me, so I would be very
thankfull if someone can help me. If u want, for the guy who
can help me and if this works will get some money on paypal.
Not much but some money. Also sorry for my english I am from germany.
So :
I am struggling for 4.5 Hours now.
It just cant find the element "firstname". I tried via normal way:
driver find element by xpath and doesnt find, also id and many things.
Then I noticed, that there is an iframe on the page, so i though I have to switch to it before.
But now it cant find this iframe.
Infos:
-Python
-Selenium
-Chrome webdriver
THE SITE:
https://www.google.com/intl/de/gmail/about/#
CODE:
def main():
options = webdriver.ChromeOptions()
options.add_argument("--window-size=950,1000")
driver = webdriver.Chrome(options=options)
driver.get("https://www.google.com/intl/de/gmail/about/#")
time.sleep(randint(1, 3))
#click create account
while True:
try:
driver.find_element_by_xpath("/html/body/div[2]/div[1]/div[4]/ul[1]/li[3]/a").click()
break
except:
yeet = 1
#works all well till here
#now I am struggling finding element, firstname element
#firstname_input = driver.find_element_by_xpath("/html/body/div[1]/div[1]/div[2]/div[1]/div[2]/div/div/div[2]/div/div[1]/div/form/span/section/div/div/div[1]/div[1]/div[1]/div/div[1]/div/div[1]/input")
#iframe = driver.find_element_by_xpath("/html/body/iframe")
#iframe = driver.find_element_by_tag_name("iframe")
#driver.switch_to.frame(iframe)
###all didnt work
main()
By clicking on "Create Account" button a new window is opened.
So, you have to switch to that new window.
To do so you should do the following:
new_window = driver.window_handles[1]
driver.switch_to_window(new_window)
Now you can access the "firstname" element and all the others there.
If you wish finally get back to the first window just do it similarly to the above:
initial_window = driver.window_handles[0]
driver.switch_to_window(initial_window)
Also, please never use long absolute locators like "/html/body/div[2]/div[1]/div[4]/ul[1]/li[3]/a.
The 'Create Button' can be located by this //li/a[#data-action="create an account" and (contains(#class,'button'))] XPath
I thought about wanting to know when the next episode of my favorite anime show is gonna be by doing a little bit of web scraping, and searching that specific anime. Then printing out the next episode countdown from the id="nextEpisodeCountDown" in a span tag.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("https://kissanime.ru/")
time.sleep(15)
search = driver.find_element_by_id("keyword")
search.send_keys("Rent a Girlfriend")
search.send_keys(Keys.RETURN)
time.sleep(10)
element = driver.find_element_by_id("nextEpisodeCountDown")
print(element)
I'm not sure exactly what your question is here. You basically just said "I had this idea and here's the code I wrote to do it". I'm gonna go out on a limb and assume you're asking for help making this work because it isn't working as you expected it to? The code you posted works just fine to accomplish the goal you stated with perhaps one issue. I'm guessing the issue you're having is what's being printed out with your code is something like
<selenium.webdriver.remote.webelement.WebElement (session="567cb03b2d0f311ccf81166ff58c62c4", element="92669594-5987-48e7-8cd1-1572a68fc34e")>
and the output you wanted was
06 days 19h:11m
going with this assumption what you need to do is alter your print statement to say print(element.text). When you use element = driver.find_element_by_id("nextEpisodeCountDown") what is returned is that element object and you are looking to print the text of that element thus you need to write element.text. Hope this helps and if not perhaps edit your question to be more clear what it is you're asking.
I've never studied HTML seriously, so probably what I'm going to say is not right.
While I was writing my selenium code, I noticed that some buttons on some webpages does not redirect to other pages, but they change the structure of the the first one. From what I've understood, this happens because there's some JavaScript code that modifies it.
So, when I wanna get some data which is not present on the first page loading, I just have to click the right sequence of button to obtain it, rigth?
The page I wanted to load is https://watch.nba.com/, and what I want to get is the match list of a given day. The fastest path to get it is to open the calendary:
calendary = driver.find_element_by_class_name("calendar-btn")
calendary.click()
and click the selected day:
page = calendary.find_element_by_xpath("//*[contains(text(), '" + time.strftime("%d") + "')]")
page.click()
running this code, I get this error:
selenium.common.exceptions.ElementNotVisibleException
I read somewhere that the problem is that the page is not correctly loaded, or the element is not visible/clickable, so I tried with this:
wait = WebDriverWait(driver, 10)
page = wait.until(EC.visibility_of_element_located((By.XPATH, "//*[contains(text(), '" + time.stfrtime("%d") + "')]")))
page.click()
But this time I get this error:
selenium.common.exceptions.TimeoutException
Can you help me to solve at least one of these two problems?
The reason you are getting such behavior is because this page is loaded with iFrames (I can see 15 on the main page) once you click the calendar button, you will need to switch your context to either be on the defaultContext or a specific iframe where the calendar resides. There is tons of code outthere that shows you how to switch into and out of iframe. Hope this helps.
firebug
console
I have a project that I chose Selenium to open 1-5 links. It's stopping at the 3rd link. I've followed the same methods for the previously successful requests. I've allowed 17 seconds and watched as I can see the page load, before the script continues to run in my console. I'm just not sure why it can't find this link, and I hope it's something I'm simply overlooking...
from selenium import *
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
import csv
import time
username = "xxxxxxx"
password = "xxxxxxx"
driver = webdriver.Firefox()
driver.get("https://tm.login.trendmicro.com/simplesaml/saml2/idp/SSOService.php")
assert "Trend" in driver.title
elem1 = driver.find_element_by_class_name("input_username")
elem2 = driver.find_element_by_class_name("input_password")
elem3 = driver.find_element_by_id("btn_logon")
elem1.send_keys(username)
elem2.send_keys(password)
elem3.send_keys(Keys.RETURN)
time.sleep(7)
assert "No results found." not in driver.page_source
elem4 = driver.find_element_by_css_selector("a.float-right.open-console")
elem4.send_keys(Keys.RETURN)
time.sleep(17)
elem5 = driver.find_element_by_tag_name("a.btn_left")
elem5.send_keys(Keys.RETURN)
Well one of the reasons is elem5 is looking for the element by tag name, but you are passing it a css tag. "a.btn_left" is not an html tag name and so your script will never actually find it, because it simply doesn't exist in the dom.
You either need to find it by css_selector or better yet by Xpath. If you want to make this as reliable possible and more future proof I always try and find elements on a page with at least 2 descriptors using Xpath if possible.
Change this:
elem5 = driver.find_element_by_tag_name("a.btn_left")
To this:
elem5 = driver.find_element_by_css_selector("a.btn_left")
You will almost never use tag_name, mostly because it will always retrieve the first tag you pass to it, so "a" will always find the first link and click it, yours however does not exist.
I wound up solving it with this code. I increased time to 20 secs, believe it or not, I did try the find by css, I actually left the a.btn_left, and cycled through all the elements, and none of them worked, fortunately, I could access by tab and key functions so that worked for now.
time.sleep(20)
driver.get("https://wfbs-svc-nabu.trendmicro.com/wfbs-svc/portal/en/view/cm")
elem5 = driver.find_element_by_link_text("Devices")
elem5.send_keys(Keys.ENTER)