trying to pass an array list of ec2 instance ids to my python script
#!/usr/bin/env python3
import boto3
import sys
import argparse
instance_id = list(sys.argv[1])
#aws_region = sys.argv[2]
tagname = sys.argv[2]
tagvalue = sys.argv[3]
EC2_RESOURCE = boto3.resource('ec2', region_name='us-west-1')
def tag_instance(instance_id,tagname,tagvalue):
TAGS = [
{
'Key': tagname,
'Value': tagvalue
}
]
instances = EC2_RESOURCE.instances.filter(
InstanceIds=[
instance_id,
],
)
for instance in instances:
instance.create_tags(Tags=TAGS)
print(f'Tags successfully added to the instance {instance.id}')
tag_instance(instance_id,tagname,tagvalue)
but keep getting this error
Invalid type for parameter InstanceIds[0],
value: ['i', '-', '0', '9', '2', 'b', 'b', 'a', 'b', 'a',
'f', '1', 'e', '8', '2', 'f', '5', 'a', '1', ','],
type: <class 'list'>,
valid types: <class 'str'>
any ideas?
Looking at the error, the comma at the end of the instance name strikes me. I assume, you make some incorrect assumptions on how the command line is parsed.
Try the following. Change the following lines in your script:
instance_ids = sys.argv[1].split(",")
and
instances = EC2_RESOURCE.instances.filter(InstanceIds=instance_ids)
Then call the script as:
scriptname "instance1,instance2,instance3" tagname tagvalue
Alternatively, you can move tagname and tagvalue to position 1 and 2 and use position 3.. for the instance names:
tagname = sys.argv[1]
tagvalue = sys.argv[2]
instance_ids = sys.argv[3:]
Then call it:
scriptname tagname tagvalue instance1 instance2 instance3
A different point: Your print statement is indented incorrectly, it will only be executed after the loop. Correct your code as follows:
for instance in instances:
instance.create_tags(Tags=TAGS)
print(f'Tags successfully added to the instance {instance.id}')
Related
I used the python lib OpenSSL.crypto to load a .p12 file, and then I used the hashlib.sha256 to generate a digest of my data. The question is that I cannot get the "signature" I want. Is it because the python lib, crypto.sign(pkey, data, digest) used the lowercase a-z rather than the uppercase A-Z in the hexdigest? The code is shown here.
The signature is different from the one generated by another method of JAVA-RSASHA256.
Java used the digest below: The final String is with upper format A-F.
import java.security.MessageDigest;
char hexDigits[] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
try {
byte[] btInput = content.getBytes("UTF-8");
MessageDigest mdInst = MessageDigest.getInstance("SHA-256");
mdInst.update(btInput);
byte[] md = mdInst.digest();
int j = md.length;
char str[] = new char[j * 2];
int k = 0;
for (int i = 0; i < j; i++) {
byte byte0 = md[i];
str[k++] = hexDigits[byte0 >>> 4 & 0xf];
str[k++] = hexDigits[byte0 & 0xf];
}
return new String(str);
} catch(Exception e) {
return null;
}
While I used the python code to generate the digest below.
from OpenSSL import crypto
from Crypto.Hash import SHA256
data = 'hello world!'
hash_256 = hashlib.sha256()
h_sha256 = hash_256.copy()
h_sha256.update(data.encode('utf-8'))
sha256_str = h_sha256.hexdigest()
print('output:', sha256_str)
#The "print-result" contains only lower-format 'a-f'.
Will crypto.sign(pkey, data, digest) of Python uses the lowercase digest to generate a signature? So the signature is different from that of java because java takes the uppercase?
The following code is how I import my .p12 file to sign a String.
# -*- coding: utf-8 -*-
# load OpenSSL.crypto
from OpenSSL import crypto
import hashlib
psw = b'123456'
p12 = crypto.load_pkcs12(open(r"D:\tasks\private_key.p12", 'rb').read(), psw)
pkey = p12.get_privatekey()
data = 'hello world!'
my_signature = crypto.sign(pkey, str.encode(data), 'sha256')
The class below is driving me crazy. It is stuck on the for loop. I'm unsure why it will not access the last element in self.job_ids. This should be working. Any ideas why this for loop does not work inside the class but it works perfectly find outside the class?
import subprocess
class job_runner():
def __init__(self, user_id='staffner'):
self.job_ids = ['12054807', '12054808', '12054809', '12054810', '12054811', '12054812', '12054813', '10', '100']
self.user_id = user_id
def update_running_jobs(self):
'''
() ->
Remove job ids from self.job_ids which completed
'''
# find currently running jobs
curr_running = self.find_running_jobs()
#iterate over self.job_ids and see if they are in the currently running jobs
working_ids = self.job_ids
print 'start working ids:'
print working_ids
for j_id in working_ids:
# for some reason I can not access the last id in the list
print j_id
if j_id not in curr_running:
self.job_ids.remove(j_id)
print 'job_ids'
print self.job_ids
def find_running_jobs(self):
'''
() -> list running ids
Find what job ids are still running on the high performance cluster
'''
proc = subprocess.Popen(['squeue -u %s --Format=arrayjobid'%(self.user_id)], stdout=subprocess.PIPE, shell=True)
out, err = proc.communicate()
if err == None:
out_list = out.replace('ARRAY_JOB_ID', '').replace(' ', '').split('\n')
# filter out any empty strings
out_list = filter(None, out_list)
return out_list
else:
return False
curr_jobs = job_runner()
curr_jobs.update_running_jobs()
Here's the output (as you can see 100 is never accessed):
start working ids:
['12054807', '12054808', '12054809', '12054810', '12054811', '12054812', '12054813', '10', '100']
12054807
12054808
12054809
12054810
12054811
12054812
12054813
10
job_ids
['12054807', '12054808', '12054809', '12054810', '12054811', '12054812', '12054813', '100']
You should change:
working_ids = self.job_ids
to:
working_ids = self.job_ids[:] # create a copy of job_ids and use it
Explanation: you're modifying the list while iterating it which causes unexpected results, changing the code you'll iterate a copy of the list.
I am trying to implement a stack machine in Python using list as my stack and dictionary as current machine state. But it doesn’t work as planned. After some debugging I found out that when I store machine state into stack, and then change it state, the stored state changes also.
Here is an example demonstrating my problem:
MyStack = []
myState = {}
myState['param'] = '1'
MyStack.append(myState)
myState['param'] = '2'
MyStack.append(myState)
myState['param'] = '3'
MyStack.append(myState)
print(MyStack.pop())
print(MyStack.pop())
print(MyStack.pop())
the result is:
{'param': '3'}
{'param': '3'}
{'param': '3'}
And not
{'param': '3'}
{'param': '2'}
{'param': '1'}
As I would think
Evidently, Python stores in list not copies of my dictionary, but references to it. So, all stacked items in reality are the same myState object. When I’ve got this, I walked the problem around by using dictionary.copy() method like so: MyStack.append(myState.copy()). But it seems a bit unnatural.
So, the question is: which is the most idiomatic way to implement a stack for dictionary type in Python?
Here, you are passing the same myState object inside MyStack.append, which gets updated every time rather than creating a new one. SO, inside the list you are always getting same identical element.
Let's see a data flow of your program.
MyStack = []
myState = {} # An empty dictionary 'myState' created
myState['param'] = '1' # 'mystate' -> {'param':'1'}
MyStack.append(myState) # 'MyStack' -> [mystate] -> [{'param':'1'}]
myState['param'] = '2' # 'mystate' -> {'param':'2'}
MyStack.append(myState) # 'MyStack' -> [mystate,mystate ]
# -> [{'param':'2'},{'param':'2'}]
myState['param'] = '3' # 'mystate' -> {'param':'3'}
MyStack.append(myState) # 'MyStack' -> [mystate,mystate, mystate]
# -> [{'param':'3'},{'param':'3'},{'param':'3'}]
I hope, it is clear to you now.
How to solve this:
Ensure you are creating a new dictionary every time rather than using the same one.
Before myState['param'] = '2' and myState['param'] = '3', just call,
myState = myState.copy()
Code:
MyStack = []
myState = {}
myState['param'] = '1'
MyStack.append(myState)
myState = myState.copy()
myState['param'] = '2'
MyStack.append(myState)
myState = myState.copy()
myState['param'] = '3'
MyStack.append(myState)
print(MyStack.pop())
print(MyStack.pop())
print(MyStack.pop())
Output:
{'param': '3'}
{'param': '2'}
{'param': '1'}
Before this question is immediately marked as a duplicate let me say that I have already tried every solution which were the two questions most relevant to my situation. I would appreciate it if someone could at least look at my particular issue before closing this question if necessary.
I have a finite state machine object called e which is a MCFiniteSM object. At the core of e is a dictionary called state_dict which stores "process" ids ('1', '2', etc) and an associated dictionary which stores more information about each "process". I am running unittests to add processes, changes their states based on given parameters, etc. However, in between function calls in the unittest file the finite state machine seems to be cleared. I have looked at the two questions listed above in order to avoid this and persist changes but no matter what I try the changes to the finite state machine are not persisted. Here is the uniitest file.
from finite_state_machine import MCFiniteSM
from unittest import TestLoader, TestCase, main as unimain
from datetime import datetime
import time, calendar
class MyUnitTest(TestCase):
#classmethod
def setUpClass(cls):
cls.e = MCFiniteSM()
cls.timestamp = datetime.strftime(datetime.fromtimestamp(calendar.timegm(time.gmtime())), '%Y/%m/%d %H:%M:%S')
class TestFSM(MyUnitTest):
#classmethod
def setUpClass(cls):
super(TestFSM, cls).setUpClass()
#e = MCFiniteSM()
#timestamp = datetime.strftime(datetime.fromtimestamp(calendar.timegm(time.gmtime())), '%Y/%m/%d %H:%M:%S')
def test_add_convert_processes(self):
self.e.add_process('1', 'S', self.timestamp, 100, 'start message for process 1')
self.e.add_process('2', 'S', self.timestamp, 200, 'start message for process 2')
self.e.add_process('3', 'S', self.timestamp, 300, 'start message for process 3')
self.e.add_process('4', 'S', self.timestamp, 400, 'start message for process 4')
self.e.add_process('5', 'S', self.timestamp, 500, 'start message for process 5')
self.e.add_process('6', 'S', self.timestamp, 600, 'start message for process 6')
self.assertEqual(self.e.state_dict['1'], {'id':'1', 'message_type': 'S', 'timestamp':self.timestamp, 'state': 'i', 'threshold':100, 'message': 'start message for process 1'})
self.assertEqual(self.e.state_dict['2'], {'id':'2', 'message_type': 'S', 'timestamp':self.timestamp, 'state': 'i', 'threshold':200, 'message': 'start message for process 2'})
self.assertEqual(self.e.state_dict['3'], {'id':'3', 'message_type': 'S', 'timestamp':self.timestamp, 'state': 'i', 'threshold':300, 'message': 'start message for process 3'})
self.assertEqual(self.e.state_dict['4'], {'id':'4', 'message_type': 'S', 'timestamp':self.timestamp, 'state': 'i', 'threshold':400, 'message': 'start message for process 4'})
self.assertEqual(self.e.state_dict['5'], {'id':'5', 'message_type': 'S', 'timestamp':self.timestamp, 'state': 'i', 'threshold':500, 'message': 'start message for process 5'})
self.assertEqual(self.e.state_dict['6'], {'id':'6', 'message_type': 'S', 'timestamp':self.timestamp, 'state': 'i', 'threshold':600, 'message': 'start message for process 6'})
self.e.convert_state('1', 'S')
self.e.convert_state('2', 'S')
self.e.convert_state('3', 'S')
self.e.convert_state('4', 'S')
self.e.convert_state('5', 'S')
self.e.convert_state('6', 'S')
self.assertEqual(self.e.state_dict['2']['state'], 'a')
self.assertEqual(self.e.state_dict['3']['state'], 'a')
self.assertEqual(self.e.state_dict['4']['state'], 'a')
self.e.add_process('2', 'E', self.timestamp, None, 'end message for process 2')
self.e.add_process('3', 'E', self.timestamp, None, 'end message for process 3')
self.e.add_process('4', 'E', self.timestamp, None, 'end message for process 4')
self.assertEqual(self.e.state_dict['2']['state'], 'i')
self.assertEqual(self.e.state_dict['3']['state'], 'i')
self.assertEqual(self.e.state_dict['4']['state'], 'i')
def test_active_start_conversion(self):
print self.e
print 'trying...'
import sys
from StringIO import StringIO
orig = sys.stdout
try:
output = StringIO()
sys.stdout = output
self.e.convert_state('1', 'S')
out = output.getvalue().strip()
test = "Process ID:", '1', "\n" \
"Process Message Type:", 'S', "\n" \
"Process timestamp:", self.timestamp, "\n" \
"Process Threshold:", 100, "\n" \
"Process Message:", 'start message for process 1', "\n" \
"Log Message:", "PROCESS WITH ID 1 SENT MULTIPLE START MESSAGES", "\n"
self.assertEqual(out, test)
finally:
sys.stdout = orig
if __name__ == '__main__':
unimain()
'e' is the variable I want to keep modified between function calls. When I get down to TestFSM.test_active_start_conversion the size of e prints out 0, when it should be 6. The TestFSM.test_add_convert_processes method runs successfully.
The actual error is a key error. Here is the stack trace:
Error
Traceback (most recent call last):
File "/home/Desktop/fsm_unit_tests.py", line 68, in test_active_start_conversion
self.e.convert_state('1', 'S')
File "/home/Desktop/finite_state_machine.py", line 26, in convert_state
cb_id = self.state_dict[id]['id']
KeyError: '1'
The line where the error is thrown is the self.e.convert_state('1', 'S') where I am telling the fsm to change the state of the process with ID '1'. It believes that there is no process with ID 1 and upon printing the size of the fsm it believes the finite state machine is empty. This must be due to the fact that e is not continually maintained but I do not see why.
I have tried converting self.e to self.__class__.e (same with timestamp) as well as just keeping e and timestamp global and calling TestFSM.e and TestFSM.timestamp to persist changes. Both these solutions were listed in the other questions, both of them still produced a Key Error. I tried setting up setUpClass(), but that still produced a key error.
How can I persist e and timestamp?
Test methods are executed in alphabetical order. So test_active_start_conversion is executed before test_add_convert_processes.
However, really your tests should be independent - you should be doing the set up in the actual setUp method, not in a test method.
I have following use of getopt in my code:
opts, args = getopt.getopt(sys.argv[1:], "", ["admit=", "wardName="])
Then I run the code from command line in the following matter:
Test.py --args --admit=1 --wardName="CCU"
When I print the contents of opts, I get the following output:
[('--admit', '1'), ('--wardName', 'CCU')]
The first question is what is the data type of this result? It seems to me like a list of tuples. Is that correct?
The second question - is there any convenient way to work with such tuple pairs (if these ARE tuples)? For example, how can I now say: if admit == 1 then do X?
I thought of converting the tuples to a dictionary, but is that a good practice?
P.S. It shouldn't make any difference this is actually Jython and not pure Python.
The front page of the Python docs describes the Python Library docs as "keep this under your pillow". The page on getopt is at:
http://docs.python.org/2/library/getopt.html
You get two lists back from getopt: That list of tuples that you mentioned, followed by the list of remaining arguments after all the options have been parsed out. Try this:
import getopt
args = ['-a', '-b', '-cfoo', '-d', 'bar', 'a1', 'a2']
opts, args = getopt.getopt(args, "abc:d")
print ("opts=" + str(opts))
print ("args=" + str(args))
optdic = dict(opts) # convert options to dictionary
print ("optdic['-c'] = " + str(optdic['-c']))
output:
opts=[('-a', ''), ('-b', ''), ('-c', 'foo'), ('-d', '')]
args=['bar', 'a1', 'a2']
optdic['-c'] = foo
A convenient way to work with the options and values:
import getopt, sys
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], "", ["admit=", "wardName="])
print opts
except getopt.GetoptError, err:
# print help information and exit:
print str(err)
sys.exit(2)
admitArg = False
wardName = ""
for o, a in opts:
if o == "--admit":
admitArg = a
elif o == "--wardName":
wardName = a
else:
assert False, "unhandled option"
# do something with the argument values
if admitArg:
print "--admit is True"
if wardName == "Foobar":
print "wardName is Foobar!"
if __name__ == "__main__":
main()
Yes, it is a list of tuplea. You can use type() to check.
A dictionary would work fine.
D = dict(opts)
print D['--admit']
print D['--wardName']