I got a question about python, maven and aws lambda. Basically I am trying to build dependency trees for my repos using the terminal command
mvn dependency:tree
This command is being ran via python using the os library, i.e.
import os
os.system('mvn dependency:tree')
Now comes the issue - I need to run this on AWS Lambda.
Being aware that AWS Lambda is serverless and that the layers of each lambda can only be 250mb, 1) is it possible to run terminal commands via lambda without spinning up any sort of server? and 2) maven usually needs to be installed on a system, thus is it possible, or even viable, to run maven on AWS Lambda?
Any input will be appreciated.
Thanks
is it possible to run terminal commands via lambda without spinning up any sort of server?
Yes, you can run terminal commands in a Lambda function.
maven usually needs to be installed on a system, thus is it possible, or even viable, to run maven on AWS Lambda?
You can create a custom Lambda container image that includes dependencies.
Additional AWS Blog Post: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/
Related
I'm using Lambda container images to package complicated libraries like opencv and pdf2image in Python.
Is there a way to run unit tests against it so I can get a code coverage for tools like Sonar?
With normal code, I could do the following:
python -m unittest -v
But not sure how to do that if the code is inside a container image.
I'm using bitbuckett pipelines as well.
In order to run Unit tests inside a container there are different solutions, and they mostly depends on where you are going to run the image.
Assuming you are going to run the image locally and assuming you are proficient with docker, you could build your image including development dependencies and then you could modify the entrypoint of the image accordingly in order to run your unit test suite. I suggest to bind a path to the local host in order to be able to retrieve possible junit.xml (or any test report) files. This way, you just do:
docker run --entrypoint="/path/to/my/ut/runner" -v /tmp/testout:/container/path myimage
Assuming you want to run the lambda remotely I suggest either to bind it to any APIGateway and to perform a remote API call (maybe create an endpoint just for development purposes that returns the test report), or to use additional tools like Terratest to invoke the lambda remotely without any additional infrastructure (note that this require using Terraform). Finally note that AWS Serverless Application Model documentation gives additional examples on how you could run your tests inside a lambda.
I have some python libraries that I need to invoke in NodeJS on a Lambda function. I need to do this because some of the python functions are doing external API calls and may take a while to finish, and using NodeJS speeds this up quite a lot thanks to promises.
I have read it is possible to crate custom runtimes as Layers but canont find some samples on NodeJS 12 + Python 3.7 for example, so how to do that? is there a list of already published and available runtimes somewhere?
I think the newly announced option to run Docker images in AWS Lambda might be the best solution here.
You can use either the Python or the NodeJS base image provided by Amazon and then install the rest of the required dependencies. Put it into AWS ECR and then run your Lambda using the Docker image.
Check the news article.
I have a Python AWS Lambda running on a Linux, but due to some dependencies, I need it to be deployed on a Windows. I have tried using Python Azure Functions and have successfully deployed it on a Linux as well, but found out they cannot be deployed on Windows. Is it possible to do it with AWS Lambda?
Basically my solution has a few .exe that need to be run by a python library (Tesseract OCR and pytesseract)
AWS Lambda and Azure Functions are considered Function as a Service (FaaS) solutions, where the developer worries about the code and the cloud provider worries about availability, scalability and the platform underneath to run the code.
In that aspect, you can't run any of them on a server. If you need specific Windows dependencies, you must create a Python project as you normally would, install the dependencies and configure the Windows Server, being responsible for infrastructure and OS configurations and management.
I am writing a serverless script using AWS Lambda function (runtime Python 3.5) to connect to a SOAP server, get some data, process that data then update some records in storage.
I have written the script on my local machine where I had to install the 'suds' SOAP client. I have it all working correctly, however AWS doesn't have suds installed and i'm not sure how to get it installed or whether I can.
Has anyone tried writing a soap client in aws lambda using python and if so can they give me some suggestions on how to progress further?
Thanks
Kevin
This is really a more general question about packaging dependencies in a Python AWS Lambda function deployment. This documentation details the process of including your dependencies in your AWS Lambda zip file deployment artifact.
I´ve created a set up using python 3.4 code (as I need the library pyrebase) as well as using Cronjobs.
AWS Lambda would do the job perfect if it wasn´t for the lack of Python 3-support.
Any ideas of other AWS computer services supporting Python and Cronjobs? (or other cloud computing services that would support the project?)
The environment that Lambda runs your scripts in has Python 3 available to it, so it is possible to run Python 3 code via Lambda, but it takes a bit of work (I've not done it myself). Here's a similar question: using Python 3 with AWS lamba
And the a link from the answer in that question with some good info: http://www.cloudtrek.com.au/blog/running-python-3-on-aws-lambda/
Basically you'd have a small chunk of Python 2.7 code with the lambda handler function, which then creates a Python 3 virtualenv in which to run your Python 3 code.
As for cronjobs, you could probably put something together with timers in SWS: http://docs.aws.amazon.com/amazonswf/latest/developerguide/swf-dg-timers.html . If it has to be literal cronjobs (vs. running something on a schedule, regardless of method), you'd need an EC2 instance running cron and calling out to other services.