I'm new to selenium, this is my first day of learning it so please go easy on me.
So I want to print umidTokenFromHeader a javascript variable of this website https://member.lazada.co.id/user/login. Here's my code so far
#!/usr/bin/python
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("https://member.lazada.co.id/user/login")
# print (umidTokenFromHeader)
driver.close()
On the Console tab of Chrome Developer Tools, umidTokenFromHeader looks like this:
I found this question Reading JavaScript variables using Selenium WebDriver
but I don't understand Java at all
How do I do this on selenium with python ?
You can use a JavaScript and execute_script() to return the value of umidTokenFromHeader variable. Then you can capture that value in element variable and finally print it.
#!/usr/bin/python
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("https://member.lazada.co.id/user/login")
element = driver.execute_script("return umidTokenFromHeader")
print (element)
driver.close()
Related
I want to click on Select Year dropdown and select a year from it. Go to that page and fetch the HTML.
I've written this piece of code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('C:/Users/SoumyaPandey/Desktop/Galytix/Scrapers/data_ingestion/chromedriver.exe')
driver.get('https://investors.aercap.com/results-and-events/financial-results')
driver.maximize_window()
driver.find_element_by_id('year_filter_chosen').click()
driver.find_element_by_class_name('active-result')
I'm just starting to work with selenium and got no clue how to proceed further.
I tried to look for the next class after clicking on the dropdown. I want to set the attribute value 'data-option-array-index' to 1 first, open the page, get html. Then keep on changing the value of this attribute.
Any help would be much appreciated!!
driver.get('https://investors.aercap.com/results-and-events/financial-results')
elem=driver.find_element_by_css_selector('#year-filter')
driver.execute_script("arguments[0].style.display = 'block';", elem)
selectYear=Select(elem)
selectYear.select_by_index(1)
Simply find the element and use Select after you change it's style to display block to access it's values.
Imports
from selenium.webdriver.support.select import Select
For tag in selenium there's great Class Select, example provided my colleague in the neighbor answer. But there's also a bit easier way to do it as well, like:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('https://investors.aercap.com/results-and-events/financial-results')
el = driver.find_element(By.CSS_SELECTOR, '#year-filter')
time.sleep(3)
driver.execute_script("arguments[0].style.display = 'block';", el)
el.send_keys('2018')
time.sleep(3)
driver.quit()
Currently using python and trying to have selenium click the "About" on google without using id. When I try to use .click() it does not execute, what is wrong with my code? I have looked at many videos and tutorials and it looks correct.
from selenium import webdriver
from time import sleep
browser = webdriver.Safari()
browser.get('http://google.com')
browser.maximize_window()
elm = browser.find_element_by_link_text('About')
browser.implicitly_wait(5)
elm.click()
I think, you can try using find_element_by_xpath.
First you will copy xpath of about link then you can try like below:
from selenium import webdriver
from time import sleep
browser = webdriver.Safari()
browser.get('http://google.com')
browser.maximize_window()
elm = browser.find_element_by_xpath('//*[#id="fsl"]/a[3]')
browser.implicitly_wait(5)
elm.click()
So the issue ended up being safari. For some reason the web driver safari was not allowing me to use .click. I switched to chrome and it worked.
As the title said, I'd like to performa a Google Search using Selenium and then open all results of the first page on separate tabs.
Please have a look at the code, I can't get any further (it's just my 3rd day learning Python)
Thank you for your help !!
Code:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
import pyautogui
query = 'New Search Query'
browser = webdriver.Chrome('/Users/MYUSERNAME/Desktop/Desktop-Files/Chromedriver/chromedriver')
browser.get('http://www.google.com')
search = browser.find_element_by_name('q')
search.send_keys(query)
search.send_keys(Keys.RETURN)
element = browser.find_element_by_class_name('LC20lb')
element.click()
The reason why I imported pyautogui is because I tried simulating a right click and then open in new tab for each result but it was a little confusing :)
Forget about pyautogui as what you want to do can be done in Selenium. Same with most of the rest. You just do not need it. See if this code meets your needs.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
query = 'sins of a solar empire' #my query about a video game
browser = webdriver.Chrome()
browser.get('http://www.google.com')
search = browser.find_element_by_name('q')
search.send_keys(query)
search.send_keys(Keys.RETURN)
links = browser.find_elements_by_class_name('r') #I went on Google Search and found the container class for the link
for link in links:
url = link.find_element_by_tag_name('a').get_attribute("href") #this code extracts the url of the HTML link
browser.execute_script('''window.open("{}","_blank");'''.format(url)) # this code uses Javascript to open a new tab and open the given url in that new tab
print(link.find_element_by_tag_name('a').get_attribute("href"))
I try to find refresing elements (time minute) on the webpage. My code worked only for simple text earlier. Now I use Ctrl+Shift+I and point out my element and "Copy Xpath".
Also, I have Chrome extension "XPath helper" and tried to do that with it one. There is more longer XPath, than in my code below. And it doesn't work too.
Error: NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id....
And, I also tried to use find by class, by tag, by CSS selector.. It only worked by tag and no perfect, on different page.
And I don't even say about print it, sometimes find_element(By.XPATH,'//*[...).text work, sometimes not.
I don't understand, why it work on one page and not on second.. I want to work with find elements by XPath in flash later.
UPDATE Now I retrying code and it work! But still doesn't work on the next webpage.. why it is so changeable? XPath change, when page reload or what? What is the simplest way to get text(refresing) info from flash, opened in chrome browser?
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(r"C:\Users\vishniakov\Desktop\python bj\driver\chromedriver.exe",chrome_options=options)
driver.get("https://www.betfair.com/sport/football/event?eventId=28935432")
print(driver.title)
elem =driver.find_element(By.XPATH,'//*[#id="yui_3_5_0_1_1538670363144_2571"]').text
print(elem)
This will work with assumption you want data of that page not of any specific element:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(chrome_options=options)
driver.get("https://www.betfair.com/sport/football/event?eventId=28935730")
print(driver.title)
elem =driver.find_element(By.CSS_SELECTOR,'.scroller.context-event').text
print(elem)
Assuming you do want spcipic data, you can use the contains() Xpath method... You can read about this here
For your case:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(r"C:\Users\vishniakov\Desktop\python bj\driver\chromedriver.exe",chrome_options=options)
driver.get("https://www.betfair.com/sport/football/event?eventId=28935432")
print(driver.title)
elements =driver.find_elements(By.XPATH,'//*[contains(#id, "yui_3_5_0_1_")]')
print([i.text for i in elements])
You can play around with the contains() if my example didn't work... You must find what part of the id changes and exclude that part of the ID.
Hope this helps you!
I’m working to make web crawler with python by using selenium
Here, I successfully got contents by using chromedriver, but problem occurred when I tried to make
Headless access crawling through PhantomJS. find_element_by_id, or find_element_by_name did not work
Is there any difference between these? Actually I am trying to make this as headless because I want to run this
Code in ubuntu server as a batch job without GUI support.
My script is like as below.
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import re
#driver = webdriver.PhantomJS('/Users/user/Downloads/phantomjs-2.1.1-macosx/bin/phantomjs')
#driver = webdriver.Chrome('/Users/user/Downloads/chromedriver')
driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)
driver.get(url)
driver.implicitly_wait(3)
#here I tried two different find_tag things but both didn’t work
user = driver.find_element(by=By.NAME,value="user:email")
password = driver.find_element_by_id('user_password')