Develop Reference

ModuleNotFoundError: No module named 'pyathena' when running AWS Glue Job

Despite setting the parameter for my Python AWS Glue Job like this:
--additional-python-modules pyathena
I still get the following error when I try and run the job:
ModuleNotFoundError: No module named 'pyathena'
I have also tried the following parameters:
--additional-python-modules pyathena
--pip-install pyathena
--pip-install pyathena==2.23.0

In case someone finds themselves with a similar issue this is what worked for me.
I solved this issue by removing the '--additional-python-modules pyathena' argument and upgrading the AWS Glue Job Python version to 3.9 (where pyathena is automatically installed as part of the default analytics package).
As I understand it, the --additional-python-modules argument does not work with earlier versions of AWS Glue Python.

Grequests on AWS Lambda "Gevent is required" Error

I want to use grequests on AWS Lambda. I created venv and reqs.txt and then pip install in a folder named python, then zipped the python folder and uploaded it to a Lambda Layer. After that, I am facing this error:
Gevent is required for grequests.
I already tried this answer on AWS Linux but nothing changed.
Any solution or advice?
Edit: A solution was written, "make zip process on ec2 server, not get the files from ec2 and zip on your local" and I did so. The error messages changed to:
Unable to import module 'lambda_function': No module named 'grequests'
Edit 2: I followed this guide, and facing a new error :)
Unable to import module 'lambda_function': No module named 'zope.interface'
and zope.interface already installed.

"Unable to import module 'lambda_function': cannot import name 'cygrpc' from 'grpc._cython' (/var/task/grpc/_cython/__init__.py)"

I am working on aws lambda function, i install the package but i got error
Unable to import module 'lambda_function': cannot import name 'cygrpc' from 'grpc._cython' (/var/task/grpc/_cython/init.py)
How to solve this type of error?

Are you trying to import it in code without haveing it installed on the lambda container? If you gonna use none std libs you should have you function and its libs installed locally and when zip it and upload it to Lambda.
Docs:
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-package.html

In addition to an answer marked correct I wanted to mention that for libraries using C-extensions is important that the layer was built on the same type of system as it runs in lambda. For example, if you're trying to build the layer including grpc on MacOS most likely it won't work since lambda is running on Linux. To make the layer working you have to build it in a Linux system (for example, on the virtual machine or in Docker)

Lambda error: no module found. Cryptography.hamtaz.bindings._constant_time

I created a lambda function which upload data to snowflake. I installed a all requirements in folder and zipped along with my main python file. While running in AWS it shows an error:
no module found. Cryptography.hamtaz.bindings._constant_time.
But I have this module at specified path. I don't know why it shows an error. I don't know why the error is arise.
Here is the code:
main(event, context):
import snowflake.connector
cnx = snowflake.connector.connect( user='xxx', password='yyyyy', account='zzzz', database="db Name", schema = "schema Name" )
try:
query = "SELECT * FROM Table_Name"
cnx.cursor().execute(query)
finally:
cnx.close()

I recently encountered the same issue. It turned out my Lambda function runtime was Python 3.8 but the 'cffi' library had been compiled for Python 3.6. I created a new Lambda function with the Python 3.6 runtime and uploaded my deployment package to it and it began working right away.

I faced same issue recently and found it is a problem with windows environment, try to create linux environment, install Python, packages, zip your code with all libraries and then throw back to AWS lambda, hopefully it will work.

i needed to set up a virtualenv for my lambda package to work. i also found pip install snowflake-connector-python did not install some cryptography libraries, although if i navigated to the directory i wanted them to be put in, adding --target . did cause those libraries to get installed.

For python 3.6, when I encountered the error "Unable to import module 'main': No module named '_cffi_backend'" in an AWS Lambda Function, I was able to run mv _cffi_backend.cpython-36m-x86_64-linux-gnu.so _cffi_backend.so in my linux docker image with virtualenv and the issue was resolved. Like mentioned above, some dependencies might be better placed with --target, to get them where you need them

aws - "Unable to import module 'process': /var/task/numpy/core/multiarray.so: invalid ELF header"

Using aws lambda I am receiving the following error when the script is run with the numpy module:
Unable to import module 'process': /var/task/numpy/core/multiarray.so:
invalid ELF header
Is this problem related to numpy itself or numpy specifically on aws lambda. What is an invalid ELF header?
Edit:
I believe this is related to native code execution, as stated in this answer "invalid ELF header" when using the nodejs "ref" module on AWS Lambda

The problem has to do with the multiarray.so file which was complied on my local computer's architecture. Spin up an ec2 instance and create your virtualenv with the necessary dependencies. This will cause it to compile with the correct architecture as used by Aws lambda. Then download your virtualenv from the ec2 instance and use that for lambda.

I had similar error :
/var/task/bcrypt/_bcrypt.so: undefined symbol: PyInt_FromLong
Previous answer didn't seem to work for my Pyhon:3.6 lambda.
I succeeded using this article : https://medium.com/i-like-big-data-and-i-cannot-lie/how-to-create-an-aws-lambda-python-3-6-deployment-package-using-docker-d0e847207dd6