I am using Selenium + Python to scrape this site. I am interested in getting the information out of the Power BI table, which I can currently do using Beautiful Soup. My problem is that only the cells that are currently in view are loaded on the page. The rest are loaded as the table is scrolled, which can only be done if the mouse is over the table (in other words, this driver.execute_script('arguments[0].scrollIntoView(true);', target) doesn't work) or if the scroll bar is dragged downward. The code I am currently using tries to find the scrollBar element, click-and-hold, and drag it down some number of pixels to load another set of cells.
scrollBar = driver.find_element_by_class_name("scroll-bar-part-bar")
webdriver.ActionChains(driver).move_to_element(scrollBar).click_and_hold(scrollBar).move_by_offset(0,30).release().perform()
It kinda seems like the "mouse" isn't in the correct place when it clicks and holds. Is there a more straight forward was of doing this? I have tried simulating a mouse wheel scroll from this answer, but I am not able to get that to work either.
use:
driver.execute_script("arguments.scrollBy(0,arguments[0].scrollHeight)",scrollBar)
This scrolls the element, you should do scroll on the scroll bar element
Related
I am writing a program that answers questions from a database. I am currently on the last question type (ordering questions). Here is an image of what the question looks like. The user needs to put the elements in the box in order.
As you can see, the bottom of the question gets cut off. This caused errors when trying to click certain elements that were below the page. To try and solve this I scrolled to the bottom of the page but selenium scrolled back to the top before clicking which kept the error in place. And after trying a few other things I decided to try and zoom out on the webpage to see them all at once.
This is the HTML of the list where the <p> tag is what I am trying to click on. Actually, I am trying to right-click but same thing. The rest of the <div>s with the ellipses are the rest of the boxes and are the same as the first one.
newheader = self.driver.find_element(By.XPATH, "//div[contains(#class, 'responses-container')]")
action = ActionChains(self.driver)
action.context_click(WebDriverWait (newheader, 10).until(EC.presence_of_element_located((By.XPATH, "//p[contains(text(), '"+ dictionary_.get(prompt)[r] +"')]")))).perform()
# dictionary.get(prompt)[r] contains the values we are looking for (ovulation, fertilization, etc)
# I have also tried EC.element_to_be_clickable and had the same results
I have tested this on a question that does not overflow the page and it works perfectly. The only time I am getting errors is when I try to zoom out. I have tried refreshing the page before starting the clicking and after zooming out. I have also tried resetting the driver by calling
self.driver.get(self.driver.current_url)
Each of those elements cannot be left-clicked on to select. They can only be right-clicked or tabbed to. Tabbing requires calculations that are complex since each element is being moved dynamically so I chose to right-click instead. It does right-click but in seemingly random places now that it's zoomed out.
Any help is appreciated! Thanks!
I'm trying to automate my spendings&budget, but the website of the bank reveals/hides the element I am looking for, depending on the scroll height.
Example
This is when I am at the top of the page:
And this is when I scrolled a little:
Problem
I am trying to get all elements in this role='transactions-group' but, when I added an option for scrolling:
driver.execute_script("window.scrollTo(0,document.body.scrollHeight)")
transactions = driver.find_elements_by_xpath(
"//div[#role='transactions-group']")
print(str(transactions[0].text))
it just loaded everything, but it continued to find the items that in this case were on the bottom of the page.
So my question is: Can I do some kind of loop or something to fix this problem?
In the my test-app that's built in angular there is a table with no scrollbar containing 10-11 columns. In the default view, there would be 8 fields displayed. To view the rest of fields, the user has to either use the mouse scroll to the right to view the fields/ use keyword arrow buttons. I need to simulate this in selenium webdriver to scroll to the 11th column so that I can perform further validations. I tried to use scrollintoview function in javascript. However, since the element is not displayed without scroll, it does not work. Please note that the html table do not contain any veritical/ horizontal scrollbars.
You can give a try with this.
javaScript = "document.getElementById('any_visible_locator').scrollLeft += -250;"
driver.execute_script(javaScript)
The title is pretty self explanatory. I have a page with an iframe and two columns (left and right)
Left has no scroll bar, while right has a scroll bar with a total of 600px to scroll. I need to screenshot the entirety of the right hand column. It doesn't matter if I get the entire page or not, as long as everything in the right hand column is in the screenshot.
I am using Selenium 2 Webdriver + python
There is workaround by following the steps described in http://www.ramandv.com/blog/angularjs-protractor-selenium-headless-end-end-testing/ and create a gaint virtual display for running your program.
I need to take a screenshot of a particular given dom element including the area inside scroll region.
I tried to take a screen shot of entire web page using selenium and crop the image using Python Imaging Library with the dimensions given by selenium. But I couldnt figure out a way to capture the are under scroll region.
for example I have a class element container in my page and it is height is dynamic based on the content. I need to take screenshot of it entirely. but the resulting image skips the region inside scrollbar and the cropped image results with just the scroll bar in it
Is there any way to do this? Solution using selenium is preferable, if it cannot be done with selenium alternate solution will also do.
You can scrool with driver.execute_script method and then take a screenshot.
I scroll some modal windowns this way with jQuery:
driver.execute_script("$('.ui-window-wrap:visible').scrollTop(document.body.scrollHeight);")