How to solve NameError:Options is not defined in AWS Lambda? - python

How to solve NameError:Options is not defined in AWS Lambda ?
my attempts
Added selenium module file in awslambda
aws docuemnt refer. but not solve
stackoverflow errormessage search but not solve.
Then, it worked, but the error occurred (cloudwatch).
2021-06-28T12:30:31.892+09:00 START RequestId: 7bb8408e-2b12-4e16-80be-e6f1b0166a60 Version: $LATEST
2021-06-28T12:30:31.892+09:00 Error in Imports
2021-06-28T12:30:31.895+09:00 [ERROR] NameError: name 'Options' is not defined Traceback (most recent call last): File "/var/task/lambda_function.py", line 203, in lambda_handler instance_ = WebDriver() File "/var/task/lambda_function.py", line 25, in __init__ self.options = Options()
selenium sample code
import json
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
def __init__(self):
self.options = Options()
self.options.binary_location = '/opt/headless-chromium'
self.options.add_argument('--headless')
self.options.add_argument('--no-sandbox')
self.options.add_argument('--start-maximized')
self.options.add_argument('--start-fullscreen')
self.options.add_argument('--single-process')
self.options.add_argument('--disable-dev-shm-usage')
def get(self):
driver = Chrome('/opt/chromedriver', options=self.options)
return driver
my goals
Through aws_lambda, I want to collect data through crawling all day long.

Just had a similar issue so perhaps it can help you:
Check if the selenium, chromdriver $ headless-chromium are zipped properly.
so when you upload it as a layer to aws lambda, it has the right pathname for you to import selenium in your lambda code.
you can download the file chrome_headless.zip from: https://github.com/soumilshah1995/Selenium-on-AWS-Lambda-Python3.7
and then zip it again inside that folder, on your local machine:
zip -r chrome_headless.zip chromedriver headless-chromium python
then try to upload it to aws lambda layers, attach it to your function and test it.
all credit goes to Soumil Nitin Shah with his great tutorial:
https://www.youtube.com/watch?v=jWqbYiHudt8

Related

Test desktop windows using Selenium WebDriver init on root

I know that Selenium WebDriver might be mostly for testing web apps but I've seen that there is possibility to handle also desktop windows. I have this piece of code:
self._host = "http://localhost:4723"
options = {
"app": "Root",
'deviceName': 'WindowsPC',
'platformName': 'Windows',
}
self.root = webdriver.Remote(self._host, options)
self.root.find_element_by_accessibility_id("Word")
And I want to grab here the window with Word document and then edit it. I am getting error:
Traceback (most recent call last):
File "Test.py", line 77, in test_python_is_python
self.root.find_element_by_accessibility_id("Word")
AttributeError: 'WebDriver' object has no attribute 'find_element_by_accessibility_id'
Any ideas? Because for me it looks exactly like in example.
Thanks!

Robotframework "Choose file" causes AttributeError: module 'base64' has no attribute 'encodestring' in docker

I'm trying to run a test in a docker container, which runs locally with no issues:
I want to upload a correct.csv file from 'correct' directory
*** Keyword ***
Upload file
[Arguments] ${directory} ${file}
Choose File ${choose_file_input} ${EXECDIR}/Files/${directory}/${file}
** Test case ***
Upload
Upload file correct correct.csv
But when running test in docker I get a FAIL with the AttributeError: module 'base64' has no attribute 'encodestring'. Is it because there is no GUI in docker? or the encoding needs to be fixed? Or eventually maybe there is another solution I can use for uploading files?
15:20:01.250 INFO Sending /App/Files/correct/correct.csv to browser.
15:20:01.251 DEBUG POST http://192.168.1.29:4444/wd/hub/session/4b6d453b394adaaa51bb4149e9ba8678/elements {"using": "xpath", "value": "//div[#id=\"upload\"]//input"}
15:20:01.252 DEBUG Starting new HTTP connection (1): 192.168.1.29:4444
15:20:01.305 DEBUG http://192.168.1.29:4444 "POST /wd/hub/session/4b6d453b394adaaa51bb4149e9ba8678/elements HTTP/1.1" 200 90
15:20:01.305 DEBUG Finished Request
15:20:01.618 FAIL AttributeError: module 'base64' has no attribute 'encodestring'
15:20:01.619 DEBUG Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/SeleniumLibrary/__init__.py", line 490, in run_keyword
return DynamicCore.run_keyword(self, name, args, kwargs)
File "/usr/local/lib/python3.9/site-packages/robotlibcore.py", line 103, in run_keyword
return self.keywords[name](*args, **(kwargs or {}))
File "/usr/local/lib/python3.9/site-packages/SeleniumLibrary/keywords/formelement.py", line 224, in choose_file
self.find_element(locator).send_keys(file_path)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 475, in send_keys
value = self._upload(local_file)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 695, in _upload
content = base64.encodestring(fp.getvalue())
Based on the traceback you have found this issue:
Selenium 3 is incompatible with Python 3.9
This is the fix for the issue: DeprecationWarning of base64.encodestring().
They won't back port this fix:
Thanks for the issue. We won't be releasing another version 3 as we're
heading to finishing off Selenium 4. It is a drop in replacement to
use Selenium 4.0.0.a5 so should work the same. There should not be any
breaking changes.
So you could upgrade selenium to Selenium 4.0.0.a5 or
Downgrade Python to 3.7 for example. I suppose locally you do not run 3.9.
We were running into this issue as well, but going back to an older version of Python was not an option due to incompatibilities with other libraries. If you find yourself in the same spot, you can re-create the alias like so:
import base64
base64.encodestring = base64.encodebytes
In whatever your entry-point is.

Unable to run PyTest-bdd step definition file as it throws index out of range error

Feature file is as below
Feature: Nopcommerce Login
Scenario: login to nopcommerce website
Given nopcommerce page is displayed
When user enters username as admin#yourstore.com
When user enters password as admin
Then user is able to login to nocpmmerce website
step definition python file is as below
from pytest_bdd import scenarios, given, when, then
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pytest
scenarios('../features/NopcommerceLogin.feature')
#pytest.fixture()
def browser():
driver = webdriver.Safari()
yield driver
driver.quit()
#given("nopcommerce page is displayed")
def webpage(browser):
browser.get("http://admin-demo.nopcommerce.com")
#when("user enters username as admin#yourstore.com")
def enter_uname(browser):
browser.find_element_by_id("Email").send_keys("admin#yourstore.com")
#when("user enters password as admin")
def enter_pwd(browser):
browser.find_element_by_id("Password").send_keys("admin")
browser.find_element_by_xpath("/html/body/div[6]/div/div/div/div/div[2]/div[1]/div/form/div[3]/input").click()
#then("user is able to login to nocpmmerce website")
def loginsuccess(browser):
assert browser.current_url == "https://admin-demo.nopcommerce.com/admin/"
when the step_def file is run, the following error message is displayed
Traceback (most recent call last):
File "~/tests/step_defs/test_NopcommerceLogin.py", line 6, in
scenarios('../features/NopcommerceLogin.feature')
File "~/venv/lib/python3.8/site-packages/pytest_bdd/scenario.py", line 343, in scenarios
features_base_dir = get_features_base_dir(module)
File "~/venv/lib/python3.8/site-packages/pytest_bdd/scenario.py", line 295, in get_features_base_dir
return get_from_ini('bdd_features_base_dir', default_base_dir)
File "~/venv/lib/python3.8/site-packages/pytest_bdd/scenario.py", line 303, in get_from_ini
config = CONFIG_STACK[-1]
IndexError: list index out of range
It's not entirely the solution, and I have the same issue in PycharmIDE, but I suggest using terminal and start tests like:
pytest <test_fily.py>
for IDEA solution, still working on it
Change your configurations to use pytest configuration: screenshot
or try running using terminal : pipenv run python -m pytest
I guess the problem is that the pytest is not identifying any executable pytest method's in your step definition file. Please try changing the "scenarios" to "scenario" and add a pytest identifiable method below the same
#scenario('../features/NopcommerceLogin.feature')
def test_login():
pass
This approach always works for me and is based on Pytest-BDD doc.
Make sure that pytest configured in pycham
Enable Pytest for your project
Open the Settings/Preferences | Tools | Python Integrated Tools settings dialog as described in Choosing Your Testing Framework.
In the Default test runner field select pytest.
Click OK to save the settings
#Vova
I found that in the run configuration, the Working directory was incorrectly set to the directory where the steps python file was, instead of the project root directory.
Fixing that made the test run successfully in PyCharm.
I had this issue - it was simply alignment in the feature file, I had a space on the Scenario definition after 'Scenario' and before the colon. When I removed - the error no longer occured.

Issue running Selenium on AWS Lambda

I am currently trying to implement a scraper that will check twice a day for if certain PDFs change names. Unfortunately it requires website manipulation to find the pdfs so the best solution in my mind is a combination of Selenium and AWS Lambda.
To begin I was following this tutorial. I have completed the tutorial but ran into this error from Lambda:
START RequestId: 18637c6d-ea75-40ee-8789-374654700b99 Version: $LATEST
Starting google.com
Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 46, in lambda_handler
driver = webdriver.Chrome(chrome_options=chrome_options)
File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in __init__
self.service.start()
File "/var/task/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
This error was experienced by others and was "resolved" by the author by linking to this stack overflow page. I have tried going through it but all the answers are pertaining to using headless chromium on desktop not AWS lambda.
A couple of changes Ive tried to no avail.
1) Changing the chromedriver and headless-chromium to .exe files
2) Changing this line of code to include the executable_path
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=os.getcwd() + "/bin/chromedriver.exe")
Any help in getting selenium and aws lambda working together would be greatly appreciated.
I had the same issue and it was due to the binary files being in a location that couldn't execute them. Adding a function to move them, then reading them from that location fixed it. See below example which I just got working while researching this error. (Apologies for the messy code.)
import time
import os
from selenium import webdriver
from fake_useragent import UserAgent
import subprocess
import shutil
import time
BIN_DIR = "/tmp/bin"
CURR_BIN_DIR = os.getcwd() + "/bin"
def _init_bin(executable_name):
start = time.clock()
if not os.path.exists(BIN_DIR):
print("Creating bin folder")
os.makedirs(BIN_DIR)
print("Copying binaries for " + executable_name + " in /tmp/bin")
currfile = os.path.join(CURR_BIN_DIR, executable_name)
newfile = os.path.join(BIN_DIR, executable_name)
shutil.copy2(currfile, newfile)
print("Giving new binaries permissions for lambda")
os.chmod(newfile, 0o775)
elapsed = time.clock() - start
print(executable_name + " ready in " + str(elapsed) + "s.")
def handler(event, context):
_init_bin("headless-chromium")
_init_bin("chromedriver")
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--window-size=1280x1696')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--hide-scrollbars')
chrome_options.add_argument('--enable-logging')
chrome_options.add_argument('--log-level=0')
chrome_options.add_argument('--v=99')
chrome_options.add_argument('--single-process')
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.binary_location = "/tmp/bin/headless-chromium"
driver = webdriver.Chrome("/tmp/bin/chromedriver", chrome_options=chrome_options)
driver.get('https://en.wikipedia.org/wiki/Special:Random')
line = driver.find_element_by_class_name('firstHeading').text
print(line)
driver.quit()
return line
I also had the same issue but I have fixed it now. In my case it was the python version was not same on lambda and My Dockerfile.

selenium.common.exceptions.WebDriverException: Message: 'library' executable may have wrong permissions for ChromeDriver

I want to use the chrome webdriver to connect to "https://www.google.com".
below is the code.
from selenium import webdriver
import time
driver = webdriver.Chrome("C:\\Users\\faisal\\library")
driver.set_page_load_timeout(10)
driver.get("https://www.google.com")
driver.find_element_by_name("q").send_keys(" automation by name ")
driver.find_element_by_name("blink").click()
time.sleep(5)
driver.close()
When I run the test, the following error message is displayed.Its a permission problem
C:\Users\faisal\PycharmProjects\firstSeleniumTest2\venv\Scripts\python.exe C:/Users/faisal/PycharmProjects/firstSeleniumTest2/test.py
Traceback (most recent call last):
File "C:\Users\faisal\PycharmProjects\firstSeleniumTest2\venv\lib\site-packages\selenium\webdriver\common\service.py", line 76, in start
stdin=PIPE)
File "C:\Python\lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:\Python\lib\subprocess.py", line 997, in _execute_child
startupinfo)
PermissionError: [WinError 5] Access is denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/faisal/PycharmProjects/firstSeleniumTest2/test.py", line 4, in <module>
driver = webdriver.Chrome("C:\\Users\\faisal\\library")
File "C:\Users\faisal\PycharmProjects\firstSeleniumTest2\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 68, in __init__
self.service.start()
File "C:\Users\faisal\PycharmProjects\firstSeleniumTest2\venv\lib\site-packages\selenium\webdriver\common\service.py", line 88, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'library' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
Process finished with exit code 1
The error says it all :
selenium.common.exceptions.WebDriverException: Message: 'library' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
In you code block you mentioned :
driver = webdriver.Chrome("C:\\Users\\faisal\\library")
The error clearly says your program was considering library as the ChromeDriver binary. Hence the error.
But as per the documentation of selenium.webdriver.chrome.webdriver the call to the WebDriver() is as :
class selenium.webdriver.chrome.webdriver.WebDriver(executable_path='chromedriver', port=0, options=None, service_args=None, desired_capabilities=None, service_log_path=None, chrome_options=None)
So you need to change send the Key executable_path along with the Value as the absolute path within single qoute '' along with the raw (r) switch as follows :
driver = webdriver.Chrome(executable_path=r'C:\Users\faisal\library\chromedriver.exe')
Update
As per the counter question from #Mangohero1 of-coarse executable_path is optional but in case you provide only the absolute path as per the source code provided below the absolute path is considered as the Value to the Key executable_path.
class WebDriver(RemoteWebDriver):
"""
Controls the ChromeDriver and allows you to drive the browser.
You will need to download the ChromeDriver executable from
http://chromedriver.storage.googleapis.com/index.html
"""
def __init__(self, executable_path="chromedriver", port=0,
options=None, service_args=None,
desired_capabilities=None, service_log_path=None,
chrome_options=None):
"""
Creates a new instance of the chrome driver.
Starts the service and then creates new instance of chrome driver.
:Args:
- executable_path - path to the executable. If the default is used it assumes the executable is in the $PATH
C:\Users\faisal\library is not the correct path to chromedriver. Give the actual path to your chromedriver file.
In case of Linux providing permission will solve the problem.
Use
sudo chmod +x chromedriver
driver=webdriver.Chrome("C:\\Users\\SQA Anas\\Downloads\\chromedriver.exe")
Please enter the complete chrome driver path like this:
"C:\Users\SQA Anas\Downloads\chromedriver.exe"
Its works for me :)
The executable_path should have chromedriver added at last:
executable_path='/home/selenium/Linkedin-Automation/chromedriver'
I had to use the following to run on Windows 10 64 bit and 32 bit chromedriver:
driver = webdriver.Chrome(executable_path=r'C:\\Users\\My Name\\Downloads\\chromedriver_win32\\chromedriver.exe')

Categories

Resources