So I am creating a Python script with boto to allow the user to prepare to expand their Linux root volume and partition. During the first part of the script, I would like to have a While loop or something similar to make the script not continue until:
a) the instance has been fully stopped
b) the snapshot has been finished creating.
Here are the code snippets for both of these:
Instance:
ins_prog = conn.get_all_instances(instance_ids=src_ins, filters={"instance-state":"stopped"})
while ins_prog == "[]":
print src_ins + " is still shutting down..."
time.sleep(2)
if ins_prog != "[]":
break
Snapshot:
snap_prog = conn.get_all_snapshots(snapshot_ids=snap_id, filters={"progress":"100"})
while snap_prog == "[]":
print snap.update
time.sleep(2)
if snap_prog != "[]":
print "done!"
break
So when calling conn.get_all_instances and conn.get_all_snapshots they return an empty list if the filters show nothing, which is formatted like []. The problem is the While loop does not even run. It's as if it does not recognize [] as the string produced by the get_all functions.
If there is an easier way to do this, please let me know I am at a loss right now ):
Thanks!
Edit: Based on garnaat's help here is the follow up issue.
snap_prog = conn.get_all_snapshots(snapshot_ids=snap.id)[0]
print snap_prog
print snap_prog.id
print snap_prog.volume_id
print snap_prog.status
print snap_prog.progress
print snap_prog.start_time
print snap_prog.owner_id
print snap_prog.owner_alias
print snap_prog.volume_size
print snap_prog.description
print snap_prog.encrypted
Results:
Snapshot:snap-xxx
snap-xxx
vol-xxx
pending
2015-02-12T21:55:40.000Z
xxxx
None
50
Created by expandDong.py at 2015-02-12 21:55:39
False
Note how snap_prog.progress returns null, but snap_prog.status stays as 'pending' when being placed in a While loop.
SOLVED:
MY colleague and I found out how to get the loop for snapshot working.
snap = conn.create_snapshot(src_vol)
while snap.status != 'completed':
snap.update()
print snap.status
time.sleep(5)
if snap.status == 'completed':
print snap.id + ' is complete.'
break
snap.update() call purely updates the variable snap to return the most recent information, where snap.status outputs the "pending" | "completed". I also had an issue with snap.status not showing the correct status of the snapshot according to the console. Apparently there is a significant lagtime between the Console and the SDK call. I had to wait ~4 minutes for the status to update to "completed" when the snapshot was completed in the console.
If I wanted to check the state of a particular instance and wait until that instance reached some state, I would do this:
import time
import boto.ec2
conn = boto.ec2.connect_to_region('us-west-2') # or whatever region you want
instance = conn.get_all_instances(instance_ids=['i-12345678'])[0].instances[0]
while instance.state != 'stopped':
time.sleep(2)
instance.update()
The funny business with the get_all_instances call is necessary because that call returns a Reservation object which, in turn, has an instances attribute that is a list of all matching instances. So, we are taking the first (and only) Reservation in the list and then getting the first (and only) Instance inside the reservation. You should probably but some error checking around that.
The snapshot can be handled in a similar way.
snapshot = conn.get_all_snapshots(snapshot_ids=['snap-12345678'])[0]
while snapshot.status != 'completed':
time.sleep(2)
snapshot.update()
The update() method on both objects queries EC2 for the latest state of the object and updates the local object with that state.
I will try to answer this generally first. So, you are querying a resource for its state. If a certain state is not met, you want to keep on querying/asking/polling the resource, until it is in the state you wish it to be. Obviously this requires you to actually perform the query within your loop. That is, in an abstract sense:
state = resource.query()
while state != desired_state:
time.sleep(T)
state = resource.query()
Think about how and why this works, in general.
Now, regarding your code and question, there are some uncertainties you need to figure out yourself. First of all, I am very sure that conn.get_all_instances() returns an empty list in your case and not actually the string '[]'. That is, your check should be for an empty list instead of for a certain string(*). Checking for an empty list in Python is as easy as not l:
l = give_me_some_list()
if not l:
print "that list is empty."
The other problem in your code is that you expect too much of the language/architecture you are using here. You query a resource and store the result in ins_prog. After that, you keep on checking ins_prog, as if this would "magically" update via some magic process in the background. No, that is not happening! You need to periodically call conn.get_all_instances() in order to get updated information.
(*) This is documented here: http://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.get_all_instances -- the docs explicitly state "Return type: list". Not string.
Related
I created an automatic Twitter bot with a group of friends during my university's Hackathon. As a new Python developer, I came across an issue where certain functions of my driver code were not being reached and never running. That's when I remembered that Python has an interpreter compiler.
We created a folder containing all of our bot's functions and then called all said functions in our main file. Our main file's code looks something like this:
from bot_functions.get_quote import quote_on_enable
from bot_functions.reply import reply_on_enable
from bot_functions.retweet import retweet_on_enable
while True:
quote_on_enable() # Always runs
reply_on_enable() # Sometimes runs
retweet_on_enable() # Code cannot be reached.
Examples of some of our function's code (functions are all in seperate files, but share the same folder:
def quote_on_enable(): # Purpose to run code infinitely.
while True:
try:
quote = get_quote() # Uses helper function above.
tweet = "．．．" + "\n\n" + "Komi-Translation: " + "\n" + quote
print('\n Tweeting: ' + '\n' + tweet)
api.update_status(tweet) # Actually tweeting
print("Next quote in 10 seconds.")
time.sleep(10) # Halts the loop for 'x' amount of time
except Exception as error:
print(error)
break
def reply_on_enable():
bot_id = int(api.verify_credentials().id_str)
mention_id = 1
message = "#{} ．．．"
while True:
mentions = api.mentions_timeline(since_id=mention_id) # Finding mention tweets
for mention in mentions:
print("Mention tweet found")
print(f"{mention.author.screen_name} - {mention.text}")
mention_id = mention.id
# Checking if the mention tweet is not a reply, we are not the author, and
# that the mention tweet contains one of the words in our 'words' list
# so that we can determine if the tweet might be a question.
if mention.in_reply_to_status_id is None and mention.author.id != bot_id:
try:
print("Attempting to reply...")
api.update_status(message.format(mention.author.screen_name), in_reply_to_status_id
id_str, auto_populate_reply_metadata = True)
print("Successfully replied :)")
except Exception as exc:
print(exc)
time.sleep(15) # The bot will only check for mentions every 15 seconds, unless you tweak this value
I believe that since all of our functions are calling while True, we will never be able to leave the function calls. How should I go about solving this issue? Is it possible for me to maintain the format I have going right now, or will I have to end up throwing all the functions into the main file? Thank you!I
The problem is not related to the file where your functions are.
You are first calling the quote_on_enable function and, because of the while True: loop inside, you stay inside until an exception is caught (because, in that case, you are calling a break).
So you call your second function, reply_on_enable, only when it happens. And once you are there, you are stuck inside because there is no way out its infinite while True: loop.
And this is why your third function is never reached.
By removing all my functions While True: loops as well as adding exceptions to all the proper locations, I have fixed this issue.
I am trying to open an Excel file and to take the columns according to the names (headings). If it doesn't find those headings, I want the script to stop and display a message that the column was not found.
I first assign the current numbers to the columns with those names, and then I search for the name in the headings to make sure there were no columns inserted in between that will alter the column numbers. If the number is not the same, it will take whatever column number the name is in.
I have been trying to put an "else" under the "elif" and asking it to print an error and then I put a "break". The problem is that it doesn't stop the script if a column name is not found, it just takes the default assignment. I do understand why that break won't stop the whole script, but I don't know how to put it in order to stop it. The code is a bit more complex than just this, but I hope this will give a good understanding.
I also tried removing the default assignment but then it says that it can't find "i" so it gives me an error.
I would really appreciate some help.
Code:
colTPNumber = 2
colTDNumber = 3
rows = sheet.iter_rows(min_row=1, max_row=1)
firstRow = next(rows)
for i in firstRow:
if str(i.value) == "TP":
colTPNumber = i.column
elif str(i.value) == "TD":
colTDNumber = i.column
else:
print ("Column name not found")
break
break exits the loop, not the whole program.
Use exit() or sys.exit() (don't forget to import sys for the second option) to exit the program
Example:
import sys #unnessary if you're using exit() only
colTPNumber = 2
colTDNumber = 3
rows = sheet.iter_rows(min_row=1, max_row=1)
firstRow = next(rows)
for i in firstRow:
if str(i.value) == "TP":
colTPNumber = i.column
elif str(i.value) == "TD":
colTDNumber = i.column
else:
print ("Column name not found")
#break
sys.exit()
#OR
exit()
Some more info:
You can also use quit()
quit() raises the SystemExit exception behind the scenes.
Furthermore, if you print it, it will give a message:
>>> print (quit)
Use quit() or Ctrl-Z plus Return to exit
This functionality was included to help people who do not know Python. After all, one of the most likely things a newbie will try to exit Python is typing in quit.
Nevertheless, quit should not be used in production code. This is because it only works if the site module is loaded. Instead, this function should only be used in the interpreter.
exit() is an alias for quit (or vice-versa). They exist together simply to make Python more user-friendly.
Furthermore, it too gives a message when printed:
>>> print (exit)
Use exit() or Ctrl-Z plus Return to exit
However, like quit, exit is considered bad to use in production code and should be reserved for use in the interpreter. This is because it too relies on the site module.
sys.exit raises the SystemExit exception in the background. This means that it is the same as quit and exit in that respect.
Unlike those two however, sys.exit() is considered good to use in production code. This is because the sys module will always be there.
os._exit exits the program without calling cleanup handlers, flushing stdio buffers, etc. Thus, it is not a standard way to exit and should only be used in special cases. The most common of these is in the child process(es) created by os.fork.
Note that, of the four methods given, only this one is unique in what it does.
Summed up, all four methods exit the program. However, the first two are considered bad to use in production code and the last is a non-standard, dirty way that is only used in special scenarios.
So, if you want to exit a program normally, go with the third method: sys.exit.
Or, even better in my opinion, you can just do directly what sys.exit does behind the scenes and run:
raise SystemExit
This way, you do not need to import sys first.
However, this choice is simply one on style and is purely up to you
Adapted from: https://www.edureka.co/community/16988/python-exit-commands-why-so-many-and-when-should-each-be-used
More info on sys.exit():
sys.exit([arg])
Exit from Python. This is implemented by raising the SystemExit exception, so cleanup actions specified by finally clauses of try statements are honored, and it is possible to intercept the exit attempt at an outer level.
The optional argument arg can be an integer giving the exit status (defaulting to zero), or another type of object. If it is an integer, zero is considered “successful termination” and any nonzero value is considered “abnormal termination” by shells and the like. Most systems require it to be in the range 0–127, and produce undefined results otherwise. Some systems have a convention for assigning specific meanings to specific exit codes, but these are generally underdeveloped; Unix programs generally use 2 for command line syntax errors and 1 for all other kind of errors. If another type of object is passed, None is equivalent to passing zero, and any other object is printed to stderr and results in an exit code of 1. In particular, sys.exit("some error message") is a quick way to exit a program when an error occurs.
Since exit() ultimately “only” raises an exception, it will only exit the process when called from the main thread, and the exception is not intercepted.
Changed in version 3.6: If an error occurs in the cleanup after the Python interpreter has caught SystemExit (such as an error flushing buffered data in the standard streams), the exit status is changed to 120.
From https://docs.python.org/3/library/sys.html
Give the variables invalid default values, so you can check them after the loop.
Then you can call exit() if they're not set correctly, this will terminate the entire script.
colTPNumber = None
colTDNumber = None
rows = sheet.iter_rows(min_row=1, max_row=1)
firstRow = next(rows)
for i in firstRow:
if str(i.value) == "TP":
colTPNumber = i.column
elif str(i.value) == "TD":
colTDNumber = i.column
if colTPNumber is None or colTDNumber is None:
print("Column name not found")
exit(1)
See Terminating a Python script
Try using exit() instead of break. break ends a loop, but exit() actually exits your program.
This is my signal
def my_signal(instance, action, pk_set, **kwrags):
instance.animal_count = instance.animals.all().count()
instance.save()
if action == 'post_add':
print pk_set #works
print pk_set[0] #no work
The weird thing is, if I just print pk_set I get set([1]) but if I try and get the actual value using [0] I run into:
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
Is there any way to just get the pk_set number? I literally only need the id.
Note: This happens during test cases.
Oddly enough if I do
list(pk_set)[0]
this works
For some reasons doing
list(pk_set)[0]
works but this doesn't:
pk_set[0]
I have a code which creates a snapshot and then checks if it is done, i wrote the following code, but for some reason it doesn't update the state variable and the while loops keep printing the same thing even if the snapshot has been completed
The following is the code :
def call_creater():
regions = ['eu-central-1']
for region in regions:
ec2 = boto3.resource('ec2', region, aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY, )
snapshot = ec2.create_snapshot(VolumeId='vol-f9e7d220', Description='fra01-he-trial-ansible01')
while snapshot.state != 'completed':
print snapshot.state
print "Snapshot under creation"
time.sleep(10)
else:
print "snapshot READY"
OUTPUT:
pending
Snapshot under creation
pending
Snapshot under creation
pending
Snapshot under creation
pending
Snapshot under creation
This just keeps on printing the "Snapshot under creation" even though the snapshot gets completed. The reason for this is, i am not able to update my state variable i believe , please help me out how ?
snapshot = ec2.create_snapshot(VolumeId='vol-f9e7d220', Description='fra01-he-trial-ansible01')
This line is only executed once and at that instance the state is "pending". You have to recheck the state of snapshot variable again inside while loop.
You are not updating snapshot variable anywhere inside your code or while loop.
You will have to do something like this inside your while loop.
snapshot = conn.get_all_snapshots(snapshot_ids=[<YOUR SNAPSHOT ID>])[0]
Check the boto library and how to get the state of snapshot with id.
As Pratik mentioned, your statement is only executing once and never updating. You can update your resource with .load(). Even better though, I recommend you use the waiter. This will handle all the waiting logic for you and return when your snapshot is completed. For that resource, you would use: snapshot.wait_until_completed().
So I'm trying to do this (access a file on a website, read its contents and perform actions until the content says to exit. The precondition being it has to wait x seconds before accessing the website again; to recheck the contents):
perform = True
while(perform):
data = urllib.urlopen('someurl')
data = data.read()
if(data == '0'):
dosomething()
elif(data == '1'):
#1 signifies to exit the loop
perform = False
else:
time.sleep(10)
However this never seems to work. 'Someurl' always has a value. Some Google says that it's something to do with the sleep function. Please help!
this:
import time
while True:
print "fetching"
time.sleep(10)
is the minimal test case and as this always works it can't be a problem with the time.sleep function
1) Have you tried actually checking what data you get back from reading the URL? Like, printing it out or something?
2) If the data does actually match '0' or '1', then you don't get to the else case, so the sleep() doesn't happen. This is bad because it means you try to read the data again immediately. It probably won't have changed, and web servers usually don't like you very much when you ask them for the same URL over and over without pausing (they think you are trying to hack them). Solution: don't put the sleep() call in an else case; just leave it at the same level as the original if. You don't need an else to have valid code.