I'm using threading.Timer on python 2.74. This is the relevant piece of code:
__lock = threading.Lock()
def RegeneratePopulationData():
__lock.acquire()
print 'I feel regenerated!'
__lock.release()
def __setRegenerationTimer(firstTime = False):
global __regenerationTimer
now = _getNow().date()
nextRun = datetime(now.year, now.month, now.day + 1, 2, 0)
__regenerationTimer = threading.Timer((nextRun - _getNow()).total_seconds(), __setRegenerationTimer)
print 'Regeneration timer set to run in %s' % (nextRun - _getNow())
print __regenerationTimer.interval
__regenerationTimer.start()
if not firstTime:
RegeneratePopulationData()
def _getNow():
return datetime.now() + timedelta(hours = TIME_DIF)
This cause a TypeError exception to be thrown from threading.py. I've searched the net for hours, and couldn't find a solution.
After changing the code at threading.py (for debugging purposes), around line 349, to:
# than 20 times per second (or the timeout time remaining).
print _time()
print timeout
endtime = _time() + timeout
delay = 0.0005 # 500 us -> initial delay of 1 ms
This is the exception I'm getting:
Regeneration timer set to run in 2:52:12.337000
10332.337
1377634067.66
10332.337
1377634067.66
2013-08-27 21:07:47.663000
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 812, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 1082, in run
self.finished.wait(self.interval)
File "C:\Python27\lib\threading.py", line 622, in wait
self.__cond.wait(timeout)
File "C:\Python27\lib\threading.py", line 350, in wait
endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'datetime.datetime'
I can't figure out why the interval is suddenly turning into datetime!
Can anyone please help?
Your code doesn't show where timeout is coming from. It's not "suddenly becoming" anything. Whatever your code, at some point previously, a datetime.datetime was assigned to timeout. Either eliminate that condition, or handle it appropriately (perhaps by catching the exception).
Update: You're modifying standard library code, which is an awful idea. Directly or indirectly, you're passing in a datetime.datetime. Use pdb to figure out where. Run your code in the python console. When you trigger the exception do import pdb; pdb.pm(), and you'll drop into the debugger at the site of the exception. You can explore from there.
Related
I have a function with the following logic:
def align(val, al): # This is technically imported from a separate file
return (val + (al - 1)) & -al
def compileFont(font, output, audiobank_off=0):
# Get used slot counts
icount = font.instSlotCount()
pcount = font.percSlotCount()
xcount = font.sfxSlotCount()
# Calculate head offset table size
current_pos = (2 + icount) * packspecs.pointerSize()
current_pos = align(current_pos, 16)
To be clear, current_pos is an integer by the last line. When it gets to that last line, though, Python ends up throwing the following exception:
Exception has occurred: TypeError (note: full exception trace is
shown but execution is paused at: <module>)
object of type 'int' has no len()
File "tools/assemble_sound.py", line 311, in compileFont
current_pos = align(current_pos, 16)
File "tools/assemble_sound.py", line 1020, in main
compileFont(font, f, aboff)
File "tools/assemble_sound.py", line 1136, in <module> (Current frame)
main()
I have no idea how len() is coming into play here. I double-checked and the align() function doesn't even call len(). Has anyone ever seen this before? What do I do to troubleshoot this?
Edit: I've added some more of the code and stacktrace to provide additional context. In terms of the types, current_pos is of type int.
The traceback was apparently the right clue. For some reason, when I ran this in VS Code, the exception stacktrace it showed was incomplete (as posted above). When I ran it from the command line, I got this:
Traceback (most recent call last):
File "oot/tools/assemble_sound.py", line 1136, in <module>
main()
File "oot/tools/assemble_sound.py", line 1020, in main
compileFont(font, f, aboff)
File "oot/tools/assemble_sound.py", line 311, in compileFont
current_pos = align(current_pos, 16)
File "~/.local/lib/python3.9/site-packages/makeelf/type/align.py", line 8, in align
full = ceil(len(b) / alignment)
TypeError: object of type 'int' has no len()
This pointed me in the right direction that it was calling the wrong align() method. VS Code couldn't step into align(), so I wasn't able to figure it out until I tried the command line.
I am trying to run a simple function that checks whether a folder has a single .txt file in it every 5 seconds. I am using threading to run the function every 5 seconds. When the function is to be called a second time i get an error "'bool' object is not callable". I am not sure what I dm doing wrong. It was able to run the first time
program code
import threading
import glob
DIR = r'C:\test_folder\dl_file'
def download_wait5(path_to_downloads):
dl_wait2 = True
dlCounter = len(glob.glob1(path_to_downloads,"*.txt"))
if dlCounter == 1:
dl_wait2 = True
print("True")
else:
dl_wait2 = False
print("False")
return dl_wait2
t = threading.Timer(5.0, download_wait5(DIR))
t.start()
Output
True
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\ProgramData\Anaconda3\lib\threading.py", line 1166, in run
self.function(*self.args, **self.kwargs)
TypeError: 'bool' object is not callable
I am not sure wha t I am doing wrong. All I want the function to do is return True or False depending if the folder has a .txt file or not.
I also tried this code below after reading the correct syntax for Threading https://docs.python.org/2/library/threading.html but this time it only runs once and then stops:
t = threading.Timer(1.0, download_wait5, args=[DIR])
t.start()
I wrote this function that handles the "rate limit error" of a Tweepy's cursor in order to keep downloading from Twitter APIs.
def limit_handled(cursor, user):
over = False
while True:
try:
if (over == True):
print "Routine Riattivata, Serviamo il numero:", user
over = False
yield cursor.next()
except tweepy.RateLimitError:
print "Raggiunto Limite, Routine in Pausa"
threading.Event.wait(15*60 + 15)
over = True
except tweepy.TweepError:
print "TweepError"
threading.Event.wait(5)
Since I am using serveral threads to connect I would like to stop each one of them when the RateLimitError error raises and restart them after 15 minutes.
I previously used the function:
time.sleep(x)
But I understood that doesn't work well for threads (the counter do not increase if the thread is not active) so I tried to use:
threading.Event.wait(x)
But it this error raises:
Exception in thread Thread-15:
Traceback (most recent call last):
File "/home/xor/anaconda/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/home/xor/anaconda/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/xor/spyder/algo/HW2/hw2.py", line 117, in work
storeFollowersOnMDB(ids, api, k)
File "/home/xor/spyder/algo/HW2/hw2.py", line 111, in storeFollowersOnMDB
for followersPag in limit_handled(tweepy.Cursor(api.followers_ids, id = user, count=5000).pages(), user):
File "/home/xor/spyder/algo/HW2/hw2.py", line 52, in limit_handled
threading.Event.wait(15*60 + 15)
AttributeError: 'function' object has no attribute 'wait'
How can I "sleep/wait" my threads being sure that they will wake up at the right moment?
Try doing it like this instead:
import threading
dummy_event = threading.Event()
dummy_event.wait(timeout=1)
also try google-ing next time first: Issues with time.sleep and Multithreading in Python
I am getting the following error
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run() File "python/file_download.py", line 30, in run
self._downloadFile(host[0], host[1]) File "python/file_download.py", line 57, in _downloadFile
th.exit() AttributeError: 'Thread' object has no attribute 'exit'
from the code below:
th = threading.Thread(
target=self._fileWriteToDisk,
args=(saveTo, u, file_name),
name="fileWrite_Child_of_%s" % self.getName(),
)
th.setDaemon(False)
th.start()
print "Writing to disk using child: %s " % th.name
th.exit()
There is no need to kill the thread, python threads kill themselves when they are completed.
Abruptly killing a thread is bad practice, you could have resourced that are used by the thread that aren't closed properly. A stop flag is a better way to end a thread, see this for an example.
You should be using the following, instead of hat you have:
th.daemon = False
as th.setDaemon(False) is depreciated.
I have the following code, which when is running inside of a thread (the full code is here - https://github.com/eWizardII/homobabel/blob/master/lovebird.py)
for null in range(0,1):
while True:
try:
with open('C:/Twitter/tweets/user_0_' + str(self.id) + '.json', mode='w') as f:
f.write('[')
threadLock.acquire()
for i, seed in enumerate(Cursor(api.user_timeline,screen_name=self.ip).items(200)):
if i>0:
f.write(", ")
f.write("%s" % (json.dumps(dict(sc=seed.author.statuses_count))))
j = j + 1
threadLock.release()
f.write("]")
except tweepy.TweepError, e:
with open('C:/Twitter/tweets/user_0_' + str(self.id) + '.json', mode='a') as f:
f.write("]")
print "ERROR on " + str(self.ip) + " Reason: ", e
with open('C:/Twitter/errors_0.txt', mode='a') as a_file:
new_ii = "ERROR on " + str(self.ip) + " Reason: " + str(e) + "\n"
a_file.write(new_ii)
break
Now without the thread lock I generate the following error:
Exception in thread Thread-117: Traceback (most recent call last): File "C:\Python27\lib\threading.py", line 530, in __bootstrap_inner
self.run() File "C:/Twitter/homobabel/lovebird.py", line 62, in run
for i, seed in enumerate(Cursor(api.user_timeline,screen_name=self.ip).items(200)): File "build\bdist.win-amd64\egg\tweepy\cursor.py", line 110, in next
self.current_page = self.page_iterator.next() File "build\bdist.win-amd64\egg\tweepy\cursor.py", line 85, in next
items = self.method(page=self.current_page,
*self.args, **self.kargs) File "build\bdist.win-amd64\egg\tweepy\binder.py", line 196, in _call
return method.execute() File "build\bdist.win-amd64\egg\tweepy\binder.py", line 182, in execute
result = self.api.parser.parse(self, resp.read()) File "build\bdist.win-amd64\egg\tweepy\parsers.py", line 75, in parse
result = model.parse_list(method.api, json) File "build\bdist.win-amd64\egg\tweepy\models.py", line 38, in parse_list
results.append(cls.parse(api, obj)) File "build\bdist.win-amd64\egg\tweepy\models.py", line 49, in parse
user = User.parse(api, v) File "build\bdist.win-amd64\egg\tweepy\models.py", line 86, in parse
setattr(user, k, parse_datetime(v)) File "build\bdist.win-amd64\egg\tweepy\utils.py", line 17, in parse_datetime
date = datetime(*(time.strptime(string, '%a %b %d %H:%M:%S +0000 %Y')[0:6])) File "C:\Python27\lib\_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0] File "C:\Python27\lib\_strptime.py", line 300, in _strptime
_TimeRE_cache = TimeRE() File "C:\Python27\lib\_strptime.py", line 188, in __init__
self.locale_time = LocaleTime() File "C:\Python27\lib\_strptime.py", line 77, in __init__
raise ValueError("locale changed during initialization") ValueError: locale changed during initialization
The problem is with thread lock on, each thread runs itself serially basically, and it takes way to long for each loop to run for there to be any advantage to having a thread anymore. So if there isn't a way to get rid of the thread lock, is there a way to have it run the for loop inside of the try statement faster?
According to a previous Answer on StackOverflow, time.strptime is not thread-safe. Unfortunately, the error referenced in that question is different than the error you're experiencing.
Their solution was to call time.strptime prior to initializing any threads, and then subsequent calls to time.strptime in various threads will work.
I think the same solution may work in your situation after reviewing the _strptime and locale standard library modules. I can't be certain it will work since I can't test your code locally, but I thought I'd provide you with a potential solution.
Let me know if this works.
Edit:
I've done a bit more research and the Python standard library is calling setlocale in the locale.h C header file. According to the setlocale documentation, this is not thread-safe and that calls to setlocale should occur before initializing threads as I mentioned previously.
Unfortunately, setlocale is called each time you call time.strptime. So, I suggest the following:
Test out the solution laid out earlier, try calling time.strptime before initializing the threads and remove the locks.
If #1 doesn't work, you'll probably need to roll your own time.strptime function that is thread-safe as mentioned in the Python documentation for the locale module.
The problem you are running into is related to missing thread safety of the used functions and modules.
As you can see here, tweepy is not re-entrant nor thread safe. As you can see here, Python's LocaleTime is not too.
For a multi-threaded application like yours, wrap the tweepy API through your own class that is synchronized (RLock'ed). But do not derive from the tweepy class, make a has-a relationship with a private attribute to tweepy instance.