Taking full page screenshot in chrome store SELENIUM PYTHON - python

I'm trying to save a full-page screenshot of a chrome store page, using selenium, and python 3.
I've searched online for different answers and I keep getting only the "header" part, no matter what I try. As if the page doesn't scroll for the next "section".
I tried clicking inside the page to verify it's in focus but that didn't help.
Tried answers with stitching and imported Screenshots and Image.
my current code is:
ob = Screenshot_Clipping.Screenshot()
driver2 = webdriver.Chrome(executable_path=chromedriver)
url = "https://chrome.google.com/webstore/detail/online-game-zone-new-tab/abalcghoakdcaalbfadaacmapphamklh"
driver2.get(url)
img_url = ob.full_Screenshot(driver, save_path=r'.', image_name='Myimage.png')
print(img_url)
print('done')
driver2.close()
driver2.quit()
but that gives me this picture:
What am I doing wrong?

Related

Get the current url when it's not valid with Selenium Python

I'm an beginner learning web scraping with Selenium. Recently I faced the problem that sometimes there are button elements that do not have a "href" attribute with link to the website it leads to. In order to obtain the link or useful information from that link, I need to click on the button and get the current url in the new window using the "current_url" method. However, it doesn't always work, when the new url is not valid. I'm asking for help on the solution.
To give you an example, say one wants to obtain the Spotify link to the song listed on https://www.what-song.com/Tvshow/100242/BoJack-Horseman/e/116712. After clicking on the Spotify button, instead of being directed to spotify web player, I see a new window popping up with this url "spotify:track:6ta5yavnnEfCE4faU0jebM". It's not valid probably due to some errors made by the website, but the identifier "6ta5yavnnEfCE4faU0jebM" is still useful so I want to obtain it.
However, when I try using the "current_url" method, it gives me the original link "https://www.what-song.com/Tvshow/100242/BoJack-Horseman/e/116712", instead of the invalid url. My codes are attached below. Note that I already have a time.sleep.
Specs: MacOS 12.6, chrome and webdriver version 106.something, Python 3.
s = Service('/web_scraping/chromedriver')
driver = webdriver.Chrome(service=s)
wait = WebDriverWait(driver, 3)
driver.get('https://www.what-song.com/Tvshow/100242/BoJack-Horseman/e/116712')
spotify_button_element = driver.find_element("xpath",'/html/body/div/div[2]/main/div[2]/div/div[1]/div[5]/div[1]/div[2]/div/div/div[2]/div/div[1]/button[3]')
driver.execute_script("arguments[0].click();", spotify_button_element)
time.sleep(3)
print(driver.current_url)
Any idea on why this happened and how to fix it? Hugh thanks in advance!
What you could do instead of finding the button to click and opening a new tab is to do the following:
import json
spotify_data_request = driver.find_element("id",'__NEXT_DATA__') # get the data stored in a script tag with id = '__NEXT_DATA__'
temp = json.loads(spotify_data_request.get_attribute('innerHTML')) # convert the string into a dict like object
print(temp['props']['pageProps']['episode']['songs'][0]['song']['spotifyId']) # get the Id attribute that you want instead of having to click the spotify button and retrieve it from the URL

Selenium python iframe

so I want to do an automated testing on an iframe (input image). Here is what I have done so far:
WebDriverWait(driver, 1000).until(EC.frame_to_be_available_and_switch_to_it(driver.find_element("xpath","//div[#id='el_banner_url_upload']//iframe")))
upload_image = driver.find_element("xpath", "//input[#name='images']")
upload_image.send_keys(str(d_image))
driver.switch_to.default_content()
For the flow:
I click on a button inside the iframe
It opens my windows folder, then I select a picture I want to upload
The image uploaded and it gives a preview + a link to the image that is generated automatically by the website is shown on the field below the displayed image
Here is the link to the screenshot of what I'm talking about:
Before uploading the image
After uploading the image
I got 2 problems right now,
After running the script, it works well (didn't give any error on the command). The "upload" button is gone now. But, the image is not shown on the display and there are no link generated. As shown in this screenshot. Tried inspect it, and found this error on the console:
Uncaught TypeError: Cannot read properties of undefined (reading 'showQuality')
at HTMLFormElement.add (pic-embed.js:173:35)
at $.<computed>.<computed>._trigger (jquery.ui.widget.js:489:13)
at File.<anonymous> (jquery.fileupload.js:846:31)
at Function.each (jquery.js:4:5347)
at $.<computed>.<computed>._onAdd (jquery.fileupload.js:840:15)
at $.<computed>.<computed>._onAdd (jquery.ui.widget.js:105:25)
at Object.<anonymous> (jquery.fileupload.js:1016:26)
at c (jquery.js:4:26036)
at Object.add [as done] (jquery.js:4:26346)
at Object.always (jquery.js:4:27212)
The iframe would randomly not refreshed correctly, this too I couldn't figure out how. If this happens, the script couldn't run. Here is the screenshot of when the iframe didn't refresh correctly
Are there any solution to this?
You should use Selenium expected conditions. Don't just find elements and click them, wait for them to be enabled or clickable:
driver.wait.until(ExpectedCondition.element_to_be_clickable((By.XPATH, "myXpath"))).click()
driver.wait.until(ExpectedCondition.element_to_be_clickable((By.CSS, "myCSS"))).click()
When you try to click on an element before it is actually clickable - you are going to get errors.

python selenium . how to switch back to previous window/tab after a download opens up a new tab

The problem is when I download a file through a click(), it opens a new tab/window to initiate the download, then the browser automatically closes the tab/window. However, when I want to access the previous page in which the download link was, I get the error "invalid session id".
I get the error "No such window exception" when using Safari for the automation.
If anyone knows how to deal with this issue I would appreciate all the help I could get. thank you all!
my code is below and the error comes after trying to click file_dl2
attachments = browser.find_element_by_id('sAttachments')
attachments.click()
time.sleep(2)
files = browser.find_element_by_xpath('//*[#id="FileListFI"]/div[1]')
files.click()
file_dl = browser.find_element_by_xpath('//*[#id="ctl00_chpDialog_hrefFileLink"]/img')
file_dl.click()
browser.implicitly_wait(10)
file_dl2 = browser.find_element_by_xpath('//*[#id="ctl01_chpDialog_hrefFileLink"]/img')
file_dl2.click()
....
driver.get("<web-site>")
sleep(1)
....
This fetches the original/previous website is fetched (add this code where you have click() action in role ).
You can append new set of instruction after sleep() function and wrok further.

Selenium doesn't realize page finished loading

I'm writing a trying to scrape some data from the following website:
http://www.b3.com.br/pt_br/market-data-e-indices/servicos-de-dados/market-data/historico/renda-fixa/
It worked as expected for a while, but now it get stuck in loading the page at line 3.
url = 'http://www.b3.com.br/pt_br/market-data-e-indices/servicos-de-dados/market-data/historico/renda-fixa/'
driver = webdriver.Chrome()
driver.get(url)
What is weird is that the page is in fact fully loaded, as I can browse through it without a problem, but chrome keeps showing me a "Connecting..." message in the bottom.
When selenium finally gives up and raises the TimeoutException, the "Connecting..." message dissapears and Chrome understands that the page is in fact fully loaded.
If I try to manually open the link in another tab, it does so in less than a second.
Is there a way I can overide the built in "wait until loaded" and just get to next steps, as everything i need is already loaded?
http://www.b3.com.br/lumis/portal/controller/html/SetLocale.jsp?lumUserLocale=pt_BR
This link loads inifinitely.
Report a bug an ask developers to fix.

Python-Selenium page scraping is not working properly

I am simply trying to open a web page through selenium web-driver. Clicks a button on it, interacts with some elements on second page.. etc..
I heard that selenium is best to work with python for this specified purpose so I wrote my code in it which works very fine at once. But gradually day after day the code which was working absolutely fine before ..just stopped working. Stopped interacting with page elements. Every time throw different errors. I am sick of this selenium behavior. Do anyone know why such so happens? Or can u suggest any good alternatives?
driver = webdriver.Chrome()
driver.get(url)
driver.implicitly_wait(50)
cookie = driver.find_elements_by_xpath("//*[contains(text(), 'Decline')]")
cookie[0].click()
buttons = driver.find_elements_by_xpath("//button[contains(text(), 'Search')]")
buttons[0].click()
driver.implicitly_wait(50)
close = driver.find_elements_by_css_selector("button.close")
close[0].click()
parent = driver.find_elements_by_class_name("job-info")
for link in parent[:19]:
links = link.find_elements_by_tag_name('a')
hyperlink = random.choice(links)
driver.implicitly_wait(150)
driver.find_element_by_link_text(hyperlink.text).click()
driver.close()

Categories

Resources