How to perform these operations atomically?
def setNickName(nick):
oldNick = r.get("user:id:1:nick") # r - instance of redis.Redis()
updated = r.set("user:id:1:nick", nick) if r.hsetnx("user:ref:nick", nick, '1') else False
if updated and oldNick:
r.hdel("user:ref:nick", oldNick)
return True
return False
You can make a lua script and execute it with EVAL command. It will effectively make this whole procedure atomic.
Note that Redis with Lua scripting is not released yet (2.6-rc5), but it's pretty stable already.
You would want to do this within a transaction using watch https://github.com/andymccurdy/redis-py/#pipelines
Let me rewrite your code with pipeline:
def setNickName(nick):
with r.pipeline() as pipe:
while 1:
try:
pipe.watch("user:id:1:nick")
oldNick = pipe.get("user:id:1:nick") # r - instance of redis.Redis()
pipe.multi()
updated = r.set("user:id:1:nick", nick) if r.hsetnx("user:ref:nick", nick, '1') else False
pipe.execute()
if updated and oldNick:
r.hdel("user:ref:nick", oldNick)
return True
print("False")
except redis.WatchError:
continue
Related
minimum working code :
step1_failed = False
try:
print("step #1")
except:
step1_failed = True
print("step #2") # always appening after step #1 but before step #3 regardless of if step #1 failed or not
if not step1_failed:
print("step #3") # only if step #1 execute without error
My question is : is there a better way of doing this that i don't see ?
Ideally without any dummy variable like step1_failed.
I thought that maybe "finally" and "else" was the answers but finally happen after the else and i need to do something before the else statement.
The use case of this is for PyQt5, I want to disconnect a signal and reconnect it after doing something to avoid unwanted recursion.
But I need to reconnect it, only if it was connected at first.
Here is my PyQt5 code to understand why i need this :
def somefunction():
connected_at_first = True # assuming it was connected
try:
lineedit1.textChanged.disconnect(somefunction) # throw a TypeError if it is not connected
except TypeError:
connected_at_first = False # happen only if lineedit1 wasn't connected
lineedit1.setText("always happening !")
# reconnecting lineedit1 if it was connected at the beginning
if connected_at_first:
lineedit1.textChanged.connect(somefunction)
I don't know if there's a cleaner way, but your approach can be wrapped in a context manager.
from contextlib import contextmanager
def tempdisconnect(o, f)
connected = True
try:
o.disconnect(f)
except TypeError:
connected = False
yield
if connected:
o.connect(f)
with tempdisconnect(lineedit1.textChanged, somefunction):
lineedit1.setText("always happening !")
A better API for disconnect would be to return either the function being disconnected (similar to how signal.signal works), or return None. Then tempdisconnect could be written
def tempdisconnect(o, f):
old = o.disconnect(f)
yield
o.connect(old)
This also assumes that o.connect(None) is a no-op, so that it remains unconnected before and after the body of the with statement.
If you want to avoid recursion, you can use blockSignals():
def somefunction():
blocked = lineedit1.blockSignals(True)
lineedit1.setText("always happening !")
lineedit1.blockSignals(blocked)
Otherwise, use a simple flag:
class SomeClass(QtWidgets.QWidget):
signalFlag = False
# ...
def somefunction(self):
if self.signalFlag:
return
self.signalFlag = True
self.lineEdit.setText("always happening !")
self.signalFlag = False
Base on the answers of chepner, I modified his code to be able to remove duplicate connect of the same function and to handle multiple function.
from contextlib import contextmanager
#contextmanager
def tempdisconnect(signal, func):
if not isinstance(func, (tuple, list)):
func = (func,)
connected = [True] * len(func)
for i in range(len(func)):
a = 0
try:
while True:
signal.disconnect(func[i])
a += 1
except TypeError:
if a == 0:
connected[i] = False
yield
if connected != False:
for i in range(len(func)):
if connected[i]:
signal.connect(func[i])
usage :
# Disconnect somefunction (even if it was accidently connected multiple times)
with tempdisconnect(lineEdit1.textChanged, somefunction):
lineEdit1.setText("hello")
or
# Disconnect somefunc1, somefunc2, somefunc3
with tempdisconnect(lineEdit1.textChanged, (somefunc1, somefunc2, somefunc3)):
lineEdit1.setText("hello")
I was looking at this SO question about the Python For Else flow control and I thought I had a portion of code that which is very close to the example given. I think that my code is very close to the example code except that I want the for loop to finish (for logging purposes).
for module_name, class_name in BASIC_PARSERS_TO_RUN:
full_module_name = "parsers." + module_name
parser = getattr(import_module(full_module_name), class_name)(
logger=logger)
parser_data = parser.parse(cluster_path)
if parser_data is None or parser_data == "0":
# Basic None instead of "None" so that json is serialized to null
# instead of "None"
json_data_list.append({module_name: parser_data})
failed_basic_checks = True
else:
json_data_list.append({module_name: str(parser_data)})
# Checking if we have a valid data set.
if failed_basic_checks:
json_data_list.append({"basic_validation_succeeded": False})
return json.dumps(json_data_list)
# We've run into a dataset which isn't valid.
exit(1)
Is there any way to change my for loop to use the for else flow control?
found_obj = None
for obj in objects:
if obj.key == search_key:
found_obj = obj
break
else:
print 'No object found.'
The code as written is just fine; there's no reason to use a for/else structure.
According to the docs, an else after a loop is always executed unless the loop was terminated by a break statement. So if you are not using break statements in the loop, an else clause is unnecessary; you should simply put the relevant code after the loop.
I have a need to poll a MSSQL database to watch the status of a running job. I want to run a status check every X seconds to see if the status=done. I am trying to use the threading module. I have tested the threading module with some simple print statements and it seems to work, but when I try inside my pymssql script it does not.
def watcher_query(cursor):
print 'Watching'
return cursor.execute(""" select *
from some_table' """)
def is_it_done(row):
if row['status'] == 'done':
return row['the_id'], True
else:
return row['the_id'], False
def d(cur):
watcher_query(cur)
for row in cur:
return is_it_done(row)[1]
threading.Timer(100, d).start()
def job_watch(server):
with pymssql.connect(server_info) as conn:
with conn.cursor(as_dict=True) as cur:
is_done = false
while is_done:
is_done = d(cur)
No matter what I set the threading.Timer to I see the 'Watching' statement print constantly. Is there a better way to set the polling timer perhaps?
I have also tried to use Twisted to set up a basic function which makes a function call every X sec until some condition is met. I haven't tried it with MSSQL yet though.
The way your code is written it doesn't seems to be in a working order:
It doesn't compile because of is_done = false,
If fixed to is_done = False, it skips the loop straight away,
Even if the loop is fixed in some reasonable way you never get to call threading.Timer(100, d).start() and don't examine any other rows as you return from d straight away after examining the first row using return is_it_done(row)[1]
It doesn't matter what the actual timed worker method does, prints to console or checks the database, should work just the same with the same timer.
What about something like this:
import threading
def is_it_done():
# get some dummy predictable results
if not hasattr(is_it_done, 'results'):
is_it_done.results = iter([False] * 3)
return next(is_it_done.results, True)
def job_watch():
is_done = False
def d():
is_done = is_it_done()
print('is_done: {}'.format(is_done))
timer = threading.Timer(3, d).start()
d()
job_watch()
I have this class called DecayingSet which is a deque with expiration
class DecayingSet:
def __init__(self, timeout): # timeout in seconds
from collections import deque
self.timeout = timeout
self.d = deque()
self.present = set()
def add(self, thing):
# Return True if `thing` not already in set,
# else return False.
result = thing not in self.present
if result:
self.present.add(thing)
self.d.append((time(), thing))
self.clean()
return result
def clean(self):
# forget stuff added >= `timeout` seconds ago
now = time()
d = self.d
while d and now - d[0][0] >= self.timeout:
_, thing = d.popleft()
self.present.remove(thing)
I'm trying to use it inside a running script, that connects to a streaming api.
The streaming api is returning urls that I am trying to put inside the deque to limit them from entering the next step of the program.
class CustomStreamListener(tweepy.StreamListener):
def on_status(self, status, include_entities=True):
longUrl = status.entities['urls'][0]['expanded_url']
limit = DecayingSet(86400)
l = limit.add(longUrl)
print l
if l == False:
pass
else:
r = requests.get("http://api.some.url/show?url=%s"% longUrl)
When i use this class in an interpreter, everything is good.
But when the script is running, and I repeatedly send in the same url, l returns True every time indicating that the url is not inside the set, when is supposed to be. What gives?
Copying my comment ;-) I think the indentation is screwed up, but it looks like you're creating a brand new limit object every time on_status() is called. Then of course it would always return True: you'd always be starting with an empty limit.
Regardless, change this:
l = limit.add(longUrl)
print l
if l == False:
pass
else:
r = requests.get("http://api.some.url/show?url=%s"% longUrl)
to this:
if limit.add(longUrl):
r = requests.get("http://api.some.url/show?url=%s"% longUrl)
Much easier to follow. It's usually the case that when you're comparing something to a literal True or False, the code can be made more readable.
Edit
i just saw in the interpreter the var assignment is the culprit.
How would I use the same obj?
You could, for example, create the limit object at the module level. Cut and paste ;-)
So I have a long chain of conditions that should be validated to be true. Instead of chaining a long if condition, I tried to be "innovative" and did it this way, which I reckon is more readable. But my question is, is this the optimal way of doing it?
Or is there a pythonic way of doing it? PS: Please respond with an alternative instead of answering "No", thanks!
Here's the code chunk:
def site_exists(site):
"""
returns the sitebean if it exists,
else returns false
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
for site_listing in all_sites:
if site.getId():
#condition check
try:
assert site.getId() == site_listing.getId()
assert site.getName() == site_listing.getName()
assert site.getCustomer().getId() == site_listing.getCustomer().getId()
except AssertionError:
continue
#pass conditions
return site_listing
#no id, so just check for name and customer
else:
#condition check
try:
assert site.getName() == site_listing.getName()
assert site.getCustomer().getId() == site_listing.getCustomer().getId()
except AssertionError:
continue
#pass conditions
site.setId(site_listing.getId())
return site_listing
return False
A simpler approach is to build a tuple of the conditions and compare the tuples:
def site_info(s):
return s.getId(), s.getName(), s.getCustomer().getId()
if site_info(site) == site_info(site_listing):
return site_listing
else:
continue
If you have a lot of conditions, or the conditions are expensive, you can instead create a generator for the conditions, and compare with any or all:
import itertools
def iter_site_info(s):
yield s.getId()
yield s.getName()
yield s.getCustomer().getId()
if all(x==y for (x, y) in itertools.izip(iter_site_info(site), iter_site_info(site_listing)):
return site_listing
else:
continue
I'm not sure whether Jython has any and all, but they're trivial functions to write.
EDIT - any and all appeared in Python 2.5, so Jython should have them.
Using exception for control flow is bad! And it will also kill your performance. Assuming the condition will be true in one out of 10 cases? So 9 exceptions to handle in the worst case - this comes with a huge performance cost.
If you want readability, try:
if (
condition1 and \
condition2 and \
condition3
):
# do something
For the revised version, this should be equivalent:
for site_listing in all_sites:
if site.getName() == site_listing.getName() and site.getCustomer().getId() == site_listing.getCustomer().getId():
if not site.getId():
site.setId(site_listing.getId())
if site.getId() == site_listing.getId():
return site_listing
I would implement an is_same method on site and call it in the for loops.
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
for site_listing in all_sites:
if site_listing.is_same(site, set_id=True):
return site_listing
As for its implementation (that is your real question), I suggest something like:
...
if self.getName() != site.getName(): return False
if self.getCustomer() != site.getCustomer(): return False
...
return True
Use the __eq__ method! If you wrote the class of site yourself, no problem. If it is from a module outside of your control, see this solution, or create a somewhat less elegant wrapping class. For the wrap variant, this would be the class:
class WrappedSite(object):
def __init__(self, site):
self.site = site
def __eq__(self, other):
if site.getId():
if self.site.getId() != other.site.getId():
return False
if self.site.getName() != other.site.getName():
return False
if self.site.getCustomer().getId() != other.site.getCustomer().getId():
return False
return True
Then your big ugly function is reduced to this:
def site_exists(site):
"""
returns the sitebean if it exists,
else returns false
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
wsite = WrappedSite(site)
for site_listing in all_sites:
if wsite == WrappedSite(site_listing):
return site_listing
return False
EDIT Fixed site_exists to return sitebean instead of True.
Remove some code duplication:
def site_exists(site):
"""
returns the sitebean if it exists,
else returns None
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
for site_listing in all_sites:
if (site.getName() == site_listing.getName() and
site.getCustomer().getId() == site_listing.getCustomer().getId()):
if site.getId(): # if id is set; it should be the same
if site.getId() != site_listing.getId(): continue
else: # no id; consider it the same site
site.setId(site_listing.getId()) #XXX side-effect
return site_listing
Note: it is unexpected that site_exists() might modify its argument (via .setId()). Consider to refactor it:
def same_site(site, other):
if site.getId() and site.getId() != other.getId():
# if id is set; it should be the same
return False
return (site.getName() == other.getName() and
site.getCustomer().getId() == other.getCustomer().getId())
def get_site_listing(site):
"""
returns the sitebean corresponding to `site`,
returns None if there is none
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
return next((s for s in all_sites if same_site(site, s)), None)
Note: the code doesn't modify the site object. Use the return value from get_site_listing() instead.
If next() is unavailable then use:
for site_listing in all_sites:
if same_site(site, site_listing):
return site_listing
return None
btw, jython should provide property wrappers for you; so you could write:
def same_site(site, other):
if site.id and site.id != other.id:
# if id is set; it should be the same
return False
return site.name == other.name and site.customer.id == other.customer.id
and site.id = id instead of site.setId(id).
Using an exception handler as a code target is basically a "goto". I don't see anything wrong with it really (despite the ancient furor surrounding "goto considered harmful"), but knowing that it's a goto might give you a different perspective.
Using exception handling for control flow is pythonic in the sense that "it's easier to ask for forgiveness than for permission". However, that doesn't extend to making rules to break just so that you can ask for forgiveness.
Others have provided excellent alternatives. Just wanted to address the principle aspect.