I need to create an AWS Lambda version of an existing Python 2.7 program written by someone else who has left the company.
Using one function I need to convert as an example:
#!/usr/bin/env python
from aws_common import get_profiles,get_regions
from aws_ips import get_all_public_ips
import sys
def main(cloud_type):
# csv header
output_header = "profile,region,public ip"
profiles = get_profiles(cloud_type)
regions = get_regions(cloud_type)
print output_header
for profile in profiles:
for region in regions:
# public_ips = get_public_ips(profile,region)
public_ips = get_all_public_ips(profile,region)
for aws_ip in public_ips:
print "%s,%s,%s" % (profile,region,aws_ip)
if __name__ == "__main__":
cloud_type = 'commercial'
if sys.argv[1]:
if sys.argv[1] == 'govcloud':
cloud_type = 'govcloud'
main(cloud_type)
I need to know how to create this as an AWS handler with event and context arguments from the code above.
If I could get some pointers on how to do this it would be appreciated.
You can simply start writing python function inside the handler of aws labda.
in handler simply start defining functions and variables and uplaod zip file inside lambda if there is any type of dependency.
you can change the python version in lambda as per if you are using python 2.7.
i would like to suggest server less framework and uplaoding your code to lambda. it's so easy to manage dependency code management from locally.
here you are using aws_common and importing you have to check it is inside aws sdk or not.
you can import aws-sdk and use it
var aws = require('aws-sdk');
exports.handler = function (event, context)
{
}
inside exports handler you can start making for loops in python or goes further
Related
I have situation like this. I am creating a Python package. That Python package needs to use Redis, so I want to allow the user of the package to define the Redis url.
Here's how I attempted to do it:
bin/main.py
from my_package.main import run
from my_package.config import config
basicConfig(filename='logs.log', level=DEBUG)
# the user defines the redis url
config['redis_url'] = 'redis://localhost:6379/0'
run()
my_package/config.py
config = {
"redis_url": None
}
my_package/main.py
from .config import config
def run():
print(config["redis_url"]) # prints None instead of what I want
Unfortunately, it doesn't work. In main.py the value of config["redis_url"] is None instead of the url defined in bin/main.py file. Why is that? How can I make it work?
I could pass the config to the run() function, but then if I run some other function I will need to pass the config to that function as well. I'd like to pass it one time ideally.
I am trying to simplify a workflow that requires several individual scripts to be run. So far, I have been able to write a script that runs the other scripts but I have one issue that I can't seem to resolve. Each of the sub-scripts requires a file path and one argument within the path needs to be changed depending on who runs the scripts. Currently, I have to open each sub-script and manually changing this argument.
Is it possible to set this argument to a variable in the parent script, which can then be passed to the subscripts? Thus, it will only need to be set once and will no longer require it to be updated in each sub-script.
So far I have.....
import os
def driver(path: str):
path_base = path
path_use = os.path.join(path_base, 'docs', 'analysis', 'forecast')
file_cash = os.path.join(path_use, 'cash.py')
file_cap = os.path.join(path_use, 'cap.py')
exec(open(file_cash).read())
exec(open(file_cap).read())
return
if __name__ == '__main__':
driver(path=r'c:\users\[username]')
I would like to set path=r'c:\users\[username]' and then pass that to cash.py and cap.py.
Instead of trying to replicate the behaviour of the import statement, you should directly import these subscripts and pass the values you need them to use as function / method aruments. To import a script from a specific path, you can use importlib.import(), like this:
main.py
import os
def driver(path: str):
path_use = os.path.join(path, 'docs', 'analysis', 'forecast')
file_cash = os.path.join(path_use, 'cash.py')
file_cap = os.path.join(path_use, 'cap.py')
importlib.import(file_cash)
importlib.import(file_cap)
cash.cash("some_arg")
cap.cap("some_other_arg")
if __name__ == '__main__':
driver(path=r'c:\users\[username]')
So on the python sdk for speaker recognition using Microsoft cognitive on the CreateProfile.py I set my subscription key under the variable subscritionKey (note: the value set to the variable on this example isn't my actual product key) But when I place it into the one of the parameters for the function create_profile I get the error...
Exception: Error creating profile: {"error":{"code":"Unspecified","message":"Access denied due to invalid subscription key. Make sure you are subscribed to an API you are trying to call and provide the right key."}}
Is there I can pass my subscritionKey without having to input it constantly through terminal each time?
import IdentificationServiceHttpClientHelper
import sys
subscritionKey = "j23h4i32h4iu3324iu234h233b43"
def create_profile(subscription_key, locale):
"""Creates a profile on the server.
Arguments:
subscription_key -- the subscription key string
locale -- the locale string
"""
helper = IdentificationServiceHttpClientHelper.IdentificationServiceHttpClientHelper(
subscription_key)
creation_response = helper.create_profile(locale)
print('Profile ID = {0}'.format(creation_response.get_profile_id()))
if __name__ == "__main__":
if len(sys.argv) < 2:
print('Usage: python CreateProfile.py <subscription_key>')
print('\t<subscription_key> is the subscription key for the service')
#sys.exit('Error: Incorrect Usage.')
create_profile(subscritionKey, 'en-us')
My guess is that I'm getting issues because I'm passing it as a string :/
Your question is basically how to consume the SDK, right?
The following code works for me: In a file called main.py that is one level out of the folder Identification:
import sys
sys.path.append('./Identification')
from CreateProfile import create_profile
subscriptionKey = "<YOUR-KEY>"
create_profile(subscriptionKey, "en-us")
Running python main.py (with Python 3), that code returns
Profile ID = cf04bf79-xxxx-xxxxx-xxxx
I want to use Luigi to manage workflows in Openstack. I am new to Luigi. For the starter, I just want to authenticate myself to Openstack and then fetch image list, flavor list etc using Luigi. Any help will be appreciable.
I am not good with python but I tried below code. I am also not able to list images. Error: glanceclient.exc.HTTPNotFound: The resource could not be found. (HTTP 404)
import luigi
import os_client_config
import glanceclient.v2.client as glclient
from luigi.mock import MockFile
import sys
import os
def get_credentials():
d = {}
d['username'] = 'X'
d['password'] = 'X'
d['auth_url'] = 'X'
d['tenant_name'] = 'X'
d['endpoint'] = 'X'
return d
class LookupOpenstack(luigi.Task):
d =[]
def requires(self):
pass
def output(self):
gc = glclient.Client(**get_credentials())
images = gc.images.list()
print("images", images)
for i in images:
print(i)
return MockFile("images", mirror_on_stderr=True)
def run(self):
pass
if __name__ == '__main__':
luigi.run(["--local-scheduler"], LookupOpenstack())
The general approach to this is just write python code to perform the tasks you want using the OpenStack API. https://docs.openstack.org/user-guide/sdk.html It looks like the error you are getting is addressed on the OpenStack site. https://ask.openstack.org/en/question/90071/glanceclientexchttpnotfound-the-resource-could-not-be-found-http-404/
You would then just wrap this code in luigi Tasks as appropriate- there's nothing special about doing with this OpenStack, except that you must define the output() of your luigi tasks to match up with an output that indicates the task is done. Right now it looks like the work is being done in the output() method, which should be in the run() method, the output method should just be what to look for to indicate that the run() method is complete so it doesn't run() when required by another task if it is already done.
It's really impossible to say more without understanding more details of your workflow.
I'm trying to deploy a python .zip package as an AWS Lambda
I choose the hello-python Footprint.
I created the 1st lambda with the inline code, after that I tried to change to upload from a development .zip.
The package I used is a .zip contains a single file called hello_python.py with the same code as the default inline code sample, which is shown below:
from __future__ import print_function
import json
print('Loading function')
def lambda_handler(event, context):
#print("Received event: " + json.dumps(event, indent=2))
print("value1 = " + event['key1'])
print("value2 = " + event['key2'])
print("value3 = " + event['key3'])
return event['key1'] # Echo back the first key value
#raise Exception('Something went wrong')
After I click "save and test", nothing happens, but I get this weird red ribbon, but no other substantive error messages. The logs and the run results do not exhibit any change if modifying to source, repackaging and uploading it again.
Lambda functions requires a handler in the format <FILE-NAME-NO-EXTENSION>.<FUNCTION-NAME>. In your case the handler is set to lambda_function.lambda_handler, which is the default value assigned by AWS Lambda). However, you've named your file hello_python.py. Therefore, AWS Lambda is looking for a python file named lambda_function.py and finding nothing.
To fix this either:
Rename your hello_python.py file to lambda_function.py
Modify your lambda function handler to be hello_python.lambda_handler
You can see an example of how this works in the documentation where they create a python function called my_handler() inside the file hello_python.py, and they create a lambda function to call it with the handler hello_python.my_handler.