Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I have this code related with the download, unzip, search and upload to a AWS S3.
I have a function to download, another to unzip, another to make the searches and another function to upload the found files; to finally call all this functions into one function.
The problem it's that at some point (specifically in the last function) the execution gets an error.
The code has 1684 lines of code, and could take even 4 hours to execute.
If an error is found in a function, the try/catches guarantee the final return.
I've tried to call every function sequentially and they work.
If trying to call all the functions, except the last one it still works.
If trying to call the last function (the upload to S3), it works.
I believe it could be related with the RAM
Trust me, it's huge
Basically, my problem was solved through the creation of a ".sh" format script that calls all the functions sequentially.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I run a Python code in which an iterative process is done. Every few minutes an iteration is performed and the results are stored in a file. Currently, after each iteration I have to run another Python script to plot the recent results to monitor the progress. I want to have a dashboard which plots the recent results whenever the results file is updated.
It's not entirely clear what your question is, but it sounds like you want to monitor the output file for changes and plot them when the file is changed.
If you're using Linux (as the tag suggests), then I'd suggest using inotify, which is a Linux API allows you to monitor filesystem events (like file writes!).
There is a Python wrapper around this, also named inotify: https://pypi.org/project/inotify/. You should be able to add a watch on your log file and run your plotting function when it's modified (perhaps by watching for the IN_CLOSE_WRITE event).
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
We have a streaming pipeline reading from pub/sub and writing to bigquery. It wasn't working without adding a window function, because a default global window only fires once and doesn't know when to re-trigger. There is no GroupBy or combine.
We tried to add a beam Window with a trigger, but there are some problems. If we use a globalWindow, it runs really slow and sometimes gives null pointer exceptions. If we use a fixed window, it's fast but but it doesn't seem to acknowledge the pub/sub messages sometimes.
What we'd really want is a pipeline that reads from pub/sub, gets a batch of however many it could get, writes to bigquery, and once everything is written and the pubsub messages are acknowledged, retrigger the read-from-pubsub. Is this possible?
I think you are looking for this. You have a composite trigger named Repeatedly.forever and you can combine it with AfterCount
Something like this where you trigger after 1000 elements read.
Repeatedly.forever(AfterCount(1000))
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Im writting a script in Python that takes as an input a timestamp. For the first execution, it should be something as the now() function. However, for further executions, the input parameter should be the last execution time of the same script. This is being done to avoid getting duplicates results.
Can anyone give me a clue please?
As far as I know, there is no "last executed" attribute for files. Some operating systems have a "last accessed" attribute, but even if that's updated on execution, it would also be updated any time the file was read, which is probably not what you want.
If you need to be able to store information between runs, you'll need to save that data somewhere. You could write it to a file, save it to a database, or write it to a caching service, such as memcached.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I was wandering if anyone was able to help me properly add Tkinker into my existing script?. The Tkinker code works fine outside of the current script as does the script without the Tkinter code, but I am having trouble merging it together. been doing alot of searching and i cant seem to fine the answer i am looking for. Here is the link to my script:
https://github.com/Octane70/Code/blob/master/Garden/Garden_v0.3.py
Line 50 #GUI_window is the code i am trying to add.
Thanks
You need to call root.mainloop() exactly once. You need to convert your while True loop into a function, and at the end of the function you need to call itself again with after. This function should also update the GUI, though you could put that in a separate function if you wish.
You do not want to call your gui_display function more than once. As it stands now, you are creating six new widgets every second. Instead, you want to create them once and then update them every second.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
When I ctrl click a builtin function in my IDE I noticed I usually get sended to an init file which holds the function but it just returns the function again.. It states a vague doc string like 'original footprint unknown'
Where do I find the real functions ?
For example where is print_function
the specific example, print is defined in C, in the bltinmodule.c: specifically.
http://hg.python.org/cpython/file/3.3/Python/bltinmodule.c#l1518
More generally, functions implemented in C have no equivalent to the source file you would read in python; the C code is compiled into binary machine code, and no reference to where that bit of code might have come from is (usually) retained in the result; and even if there was, it's unlikely that you happen to have the source code installed in a place your IDE is likely to find it, unless you built it yourself, with debug symbols, and are running the C executable process in that ide's debugger.
Usually in the same directory where that file is. (Which I can't possibly know.)