Develop Reference

How to use existing session with Seleniumbase?

I would like seleniumbase to use my existing session when it launches vs starting a clean one.
How do I start selenium using an existing session?
webdriver:
driver = webdriver.Chrome('/snap/bin/chromium.chromedriver', options=option)
UPDATE:
This is marked as a duplicate which it is not.
Dear #TodorMinakov this code here didn't meet my expectations. I wanted whenever I run the python program to select the unique opened window and use selenium over that window. I will appreciate your help. – Youssof H
Same issue here. The referred URL does not grab an existing session.

You can use default chrome profile.
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=PATH") #)PATH is path to your chrome profile
w = webdriver.Chrome(executable_path="C:\\Users\\chromedriver.exe", chrome_options=options)
To find path to your chrome profile data you need to type chrome://version/ into address bar.
For ex. mine is displayed as C:\Users\pc\AppData\Local\Google\Chrome\User Data\Default,
If path will not work, try to remove \Default\ so we end up with only C:\Users\pc\AppData\Local\Google\Chrome\User Data.
Better to copy-paste default profile outside, because there no way to use same session (cookies) in two browsers in same time.

How to change geolocation of chrome selenium driver in Python?

I am trying to trick the chromedriver to make it believe that it is running in a different city. Under normal circumstances, this can easily be done manually as shown in a quick diagram
Then, when a google search is done, the new coordinates are used, and the results that would normally originate from that location are displayed. You can confirm that this worked when you look at the bottom of a Google search page as seen
.
However, Selenium can only control what the browser displays, not the browser in itself. I cannot tell Selenium to automatically click the buttons needed to change the coordinates. I tried the solutions posted here but that is not meant for Python, and even after I tried to adapt the script, nothing seemed to happen.
Is there a browser.execute_script() argument that could work, or is this the wrong way to change the geolocation?

You can do this by importing Selenium DevTools package. Please refer below for complete java code sample:
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
public void geoLocationTest(){
ChromeDriver driver = new ChromeDriver();
Map coordinates = new HashMap()
{{
put("latitude", 50.2334);
put("longitude", 0.2334);
put("accuracy", 1);
}};
driver.executeCdpCommand("Emulation.setGeolocationOverride", coordinates);
driver.get("<your site url>");
}
Reference : Selenium Documentation

Try this code below :
driver.execute_script("window.navigator.geolocation.getCurrentPosition=function(success){"+
"var position = {\"coords\" : {\"latitude\": \"555\",\"longitude\": \"999\"}};"+
"success(position);}");
print(driver.execute_script("var positionStr=\"\";"+
"window.navigator.geolocation.getCurrentPosition(function(pos){positionStr=pos.coords.latitude+\":\"+pos.coords.longitude});"+
"return positionStr;"))

In search for a solution to the same problem I also came across the already postet scripts. While I am yet to find a solution, I assume that the scripts don't work because they do not change the sensors permanently. The sensors are only changed for that one specific call of window.navigator.geolocation.getCurrentPosition.
The website (in this case google) will later call the same function but with the regular (unchanged) geolocation. Happy to hear solutions to permanently change the sensors to then also affect future geolocation requests.

This Can be Done Using Selenium 4.
HashMap<String ,Object> coordinate = new HashMap<String ,Object>();
coordinate.put("latitude", 39.913818);
coordinate.put("longitude", 116.363625);
coordinate.put("accuracy", 1);
((ChromeDriver)driver).executeCdpCommand("Emulation.setGeolocationOverride",coordinate);
driver.navigate().to("URL");

PhantomJS loads much less HTML than other drivers

I'm trying to load one web page and get some elements from it. So the first thing I do is to check the page using "inspect element". When I search for the tags I'm looking for, I can see them (in Chrome).
But when I try to do driver.get(url) and then driver.find_element_by_..., it doesn't find those elements because they aren't in the source code.
I think that it is probably because it doesn't load the whole page but only a part.
Here is an example:
I'm trying to find ads on the web page.
PREPARED_TABOOLA_BLOCK = """//div[contains(#id,'taboola') and not(ancestor::div[contains(#id,'taboola')])]"""
driver = webdriver.PhantomJS(service_args=["--load-images=false"])
# driver = webdriver.Chrome()
driver.maximize_window()
def find_taboola_blocks_selenium(url):
driver.get(url)
taboola_blocks = driver.find_elements_by_xpath(PREPARED_TABOOLA_BLOCK)
return taboola_blocks
print len(find_taboola_blocks_selenium('http://www.breastfeeding-problems.com/breastfeeding-a-sick-baby.html'))
driver.get('http://www.breastfeeding-problems.com/breastfeeding-a-sick-baby.html')
print len(driver.page_source)
OUTPUTS:
Using PhantomJS:
0
85103
Using ChromeDriver:
3
420869
Do you know how to make PhantomJS to load as much Html as possible or any other way to solve this?

Can you compare the request that ChromeDriver is making versus the request you are making in PhantomJS? Since you are only doing GET for the specified url, you may not be including other request parameters that are needed to get the advertisements.
The open() method may give you a better representation of what you are looking for here: http://phantomjs.org/api/webpage/method/open.html

The reason for this is because PhantomJS, by default, renders in a really small window, which makes it load the mobile version of the site. And with the PhantomJSDriver, calling maximizeWindow() (or maximize_window() in python) does absolutely nothing, since there is no rendered window to maximize. You will have to explicitly set the window's render size with:
edit: Below is the Java solution. I'm not entirely sure what the Python solution would be when setting the window size, but it should be similar.
driver.manage().window().setSize(new Dimension(1920, 1200));
edit again: Found the python version:
driver.set_window_size(1920, 1200)
Hope that helps!

PhantomJS 1.x is a really old browser. It only uses SSLv3 (now disabled on most sites) by default and doesn't implement most cutting edge functionality.
Advertisement scripts are usually delivered over HTTPS (SSLv3/TLS) and usually use some obscure feature of JavaScript which is not well tested or simply not implemented in PhantomJS.
If you use PhantomJS < v1.9.8 then you should use those commandline options (service_args): --ignore-ssl-errors=true --ssl-protocol=any.
If iframes or strange cross-domain requests are necessary for the page/ads to work, then add --web-security=false to the service_args.
If this still doesn't solve the problem, then try upgrading to PhantomJS 2.0.0. You might need to compile it yourself on Linux.

Selenium: How to disable image loading with firefox and python?

I have read similar questions and one was supposed to be the answer, but when I tried it, it only gave a partial solution.
I refer to this question: Disable images in Selenium Python
My problem is that I tried the solution and some of the images do not appear, but images that arrive from:
<img href="www.xxx.png">
Are being loaded.
Is there a way to tell firefox/selenium not to get it?
If not, is there a way to discard it from the dom element that I get back, via:
self._browser.get(url)
content=self._browser.page_source
for example by doing some kind of find replace on the dom tree?
The browser configuration is the same browser from the previous question:
firefox_profile = webdriver.FirefoxProfile()
# Disable CSS
firefox_profile.set_preference('permissions.default.stylesheet', 2)
# Disable images
firefox_profile.set_preference('permissions.default.image', 2)
# Disable Flash
firefox_profile.set_preference('dom.ipc.plugins.enabled.libflashplayer.so', 'false')
# Set the modified profile while creating the browser object
self._browser = webdriver.Firefox(firefox_profile=firefox_profile)
Update:
I kept on digging and what I learned is that if I inspect the text document that the selenium/firefox combo did I see that, it didn't bring the images and kept them as links.
But when I did:
self._browser.save_screenshot("info.png")
I got a 24 mega file with all the img links loaded.
Can anyone explain to me this matter?
Thanks!

You can disable images using the following code:
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference('permissions.default.image', 2)
firefox_profile.set_preference('dom.ipc.plugins.enabled.libflashplayer.so', 'false')
driver = webdriver.Firefox(firefox_profile=firefox_profile)
if you need to block some specific url... hm...
I think you need to add string:
127.0.0.1 www.someSpecificUrl.com
to the hosts file before test start and delete it after test finish.

In the latest Firefox versions permissions.default.image can't be changed. To disable the images, either switch to ChromDriver or use alternative extentions as suggested here.

configure RefControl extension using selenium webdriver

i am using this addon with selenium and it starts fine, but how do you go about affecting the settings of addons in selenium?
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox import webdriver
profile = FirefoxProfile('/path/to/profile')
driver = webdriver.WebDriver(firefox_profile = profile)
driver.get("http://localhost/referer.html")
driver.find_element_by_link_text("go there").click()
the problem is that this addon does not do anything unless you configure it first, but there is no documentation, that i can find on this.

General way, as alecxe suggested, "to install and configure the extension first, then use that firefox profile with selenium". This will probably work with any extension.
I also search for configuration in prefs.js (or about:conf) and found out, that it stores the value in "refcontrol.actions" preference. This way you will be able to change the extension behavior in your code. Example:
profile = FirefoxProfile('/path/to/profile')
profile.set_preference("refcontrol.actions", '#DEFAULT=#NORMAL example.com=#3RDPARTY:http://www.referer.com/your/referer');