Hi i deployed an application with lambda aws (using Serverless Framework).
but i receive the error when i run a test
START RequestId: 5bcf988a-2064-11e8-8d1e-c5a0129510f Version: $LATEST
Unable to import module 'handler':
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: cannot import name multiarray
I removed the numpy package folder multiple times and reinstalled but, still not working
So do you have any solution for this problem ?
for the code :
import sys
import os
sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)),'env','Lib','site-packages'))
import numpy as np
import pandas as pd
import boto3
import csv
import psycopg2
def hello(event, context):
s3 = boto3.resource('s3')
return "test"
Thank you in advance
I resolved the problem by changing the environnement.
I didn't know that python is not portable.
I developed the lambda function in my Windows environment, but lambda is usually run in a Linux environment in AWS, and that's why my lambda was not working.
So the solution is to change the environnement, and use Linux instead of windows
I compiled my code on MACOSX environment and was caught in the same problem these days, and even I put my code on ec2 and installed all the dependencies in linux or ubuntu environment, the error is the same.
https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html docs here gave me the hint that the current lambda execution environment is Public Amazon Linux AMI version (AMI name: amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2),please pay attention to the specific version of AMI, even you use Amazon Linux as your ec2 instance, the newer or older version can make problem.
So after I changed the version of my ec2 instance, recompiled my code on it and store the zipped files on s3, than the AWS lambda run successfully.
Related
I am working on aws lambda function, i install the package but i got error
Unable to import module 'lambda_function': cannot import name 'cygrpc' from 'grpc._cython' (/var/task/grpc/_cython/init.py)
How to solve this type of error?
Are you trying to import it in code without haveing it installed on the lambda container? If you gonna use none std libs you should have you function and its libs installed locally and when zip it and upload it to Lambda.
Docs:
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-package.html
In addition to an answer marked correct I wanted to mention that for libraries using C-extensions is important that the layer was built on the same type of system as it runs in lambda. For example, if you're trying to build the layer including grpc on MacOS most likely it won't work since lambda is running on Linux. To make the layer working you have to build it in a Linux system (for example, on the virtual machine or in Docker)
I am attempting to import the google-cloud-firestore module in a python script on AWS Lambda. I have the module installed in my virtual environment and upload the code/packages in a zip folder, but receive the following error.
I have successfully imported and executed a script with the requests module using the same approach.
I have attempted to uninstall and reinstall the following modules to my virtual environment and still have no luck: google-cloud-firestore, grpcio, google-cloud-core.
Does anyone have a solution to this problem? What could I be missing here?
This thread gives me the impression that the google-cloud-firestore module cannot be used in a zip file. If this is the case, can I install the module at runtime?
Thank you for any and all advice!
I created a lambda function which upload data to snowflake. I installed a all requirements in folder and zipped along with my main python file. While running in AWS it shows an error:
no module found. Cryptography.hamtaz.bindings._constant_time.
But I have this module at specified path. I don't know why it shows an error. I don't know why the error is arise.
Here is the code:
main(event, context):
import snowflake.connector
cnx = snowflake.connector.connect( user='xxx', password='yyyyy', account='zzzz', database="db Name", schema = "schema Name" )
try:
query = "SELECT * FROM Table_Name"
cnx.cursor().execute(query)
finally:
cnx.close()
I recently encountered the same issue. It turned out my Lambda function runtime was Python 3.8 but the 'cffi' library had been compiled for Python 3.6. I created a new Lambda function with the Python 3.6 runtime and uploaded my deployment package to it and it began working right away.
I faced same issue recently and found it is a problem with windows environment, try to create linux environment, install Python, packages, zip your code with all libraries and then throw back to AWS lambda, hopefully it will work.
i needed to set up a virtualenv for my lambda package to work. i also found pip install snowflake-connector-python did not install some cryptography libraries, although if i navigated to the directory i wanted them to be put in, adding --target . did cause those libraries to get installed.
For python 3.6, when I encountered the error "Unable to import module 'main': No module named '_cffi_backend'" in an AWS Lambda Function, I was able to run mv _cffi_backend.cpython-36m-x86_64-linux-gnu.so _cffi_backend.so in my linux docker image with virtualenv and the issue was resolved. Like mentioned above, some dependencies might be better placed with --target, to get them where you need them
I have followed all the steps in the documentation:
https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
create a directory.
Save all of your Python source files (the .py files) at the root level of this directory.
Install any libraries using pip at the root level of the directory.
Zip the content of the project-dir directory)
But after I uploaded the zip-file to lambda function, I got the error message when I test the script
my code:
import psycopg2
#my code...
the error:
Unable to import module 'myfilemane': No module named 'psycopg2._psycopg'
I don't know where is the suffix '_psycopg' from...
Any help regarding this?
You are using native libraries with lambda. We had this similar problem and here is how we solved it.
Spin a machine with AWS supported AMI that runs your real lambda.
https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html
As this writing, it is,
AMI name: amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2
Full documentation in installing native modules your python lambda.
https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
Install the required modules required for your lambda,
pip install module-name -t /path/to/project-dir
and prepare your package to upload along with the native modules under lambda ami environment.
Hope this helps.
I believe this is caused because psycopg2 needs to be build an compiled with statically linked libraries for Linux. Please reference Using psycopg2 with Lambda to Update Redshift (Python) for more details on this issue. Another [reference][1] of problems of compiling psycopg2 on OSX.
There are a few solutions, but basically it comes down to installing the library on a Linux machine and using that as the Psycopg2 Library in your upload package.
How do we create a pymssql package for lambda. I tried creating it using
pip install pymssql -t . When I run my lambda function it complaints saying that
Unable to import module 'lambda_function': No module named lambda_function
I follow the steps on this link
http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
I have a windows machine
Glad that it worked for you, could you please share the working process for your, me too tried different trial and error steps and ended up with the following one which is working fine in AWS Lambda, I am using PYMSSQL package only.
1) did 'pip install pymssql' on amazon EC2 instance as under the hood Amazon uses Linux AMIs to run their Lambda functions.
2) copied the generated .so files and packaged inside the Lambda deployment package hope this will helps others as well who are searching for the solution.
Hope this will help you further, can you please share what you did to connect to MSSQL server using AWS Lambda.
Below is the folder structure of my lambda deployment package
Windows "pymssql" file not supported in AWS lambda since lambda running on Amazon Linux 2.
So we can get Linux-supported wheel files from the following official "pymssql" website download link. You look for "manylinux_2_24_x86_64" version file with the required python version. At the time of writing python 3.9 is the latest version. So download file will be "pymssql-2.2.2-cp39-cp39-manylinux_2_24_x86_64.whl".
Once download the wheel file execute the below command
pip install {path of downloaded wheel file} -t {target folder to store this package}
Example
pip install pymssql-2.2.2-cp39-cp39-manylinux_2_24_x86_64.whl -t /package
Finally i could do it. It didnt worked with windows packages so used ubuntu to package freetds.so file and it worked.
Along with pymssql try to import cypthon.
Just a quick update for new changes.
There seems to be some issue with python 3.9 AWS lambda, it throws the below error on pymssql==2.2.3.
{
"errorMessage": "Unable to import module 'index': No module named
'pymssql._pymssql'",
"errorType": "Runtime.ImportModuleError",
"stackTrace": []
}
but if I change the python version to 3.8, the error vanishes.