Let's say I have a python script called scr.py. Running python scr.py creates a scr.pyc file which is interpreted by Python. Now, let's say I make a change in scr.py while it is running, and then in another terminal window, I run python scr.py again. What happens? Does the original scr.pyc file get overwritten? Are there any problems that might occur? Could you run two slightly different copies of the same file at the same time?
Yes, it will try to overwrite .pyc file with the new version. But this won't affect the first program unless explicit module reload is called, in it because the module is loaded into memory.
OTOH, for example, printing stack for an exception needs reading source file, and, if it's changed, wrong lines will be printed. So this replacing on the fly is recommended only when module is properly reloaded just after this.
Related
Hey everyone!
Just moved over to VScode and dealing with some initial transition problems.
I'm using VScode for Python and have been using the interactive window and debugger. For my python interpreter, I've been selecting Python 3.9.7, which is a part of my Anaconda installation.
I've noticed that when I've been changing and saving my functions in one py file, and then calling the function from another file, that the changes I've made in my code aren't reflected in the code output.
It's worth noting that when the changes are made and saved in a file, and the same file is run, the changes WILL be reflected, so it's purely an issue between files. I reload the functions from the file after I make the changes and save them, so it's not an issue of reloading the function.
To provide some context in the photo, I have different functions in the file "Metric_Functions.py". I'm testing the code using different tests in the file "UnitTestCode.py". However, as I'm running the tests (reloading the functions and running the cell with the specific test), I noticed that when I made updated in the file "Metric_Functions", those changes were not being reflected in the unit test results.
Any help/experience with this kind of issue/suggestions of where to start to look would be really appreciated! Really inexperienced with VScode, so any help would be awesome.
Thanks!
In iPython and Jupyter imported modules persist throughout the session. If a module has already been imported then running the import statement again doesn't do anything at all since the interpreter can see that this module already exists in the namespace.
In order for the changes to external module to be seen in iPython/Jupyter you need to kill/restart the current instance and then run the import again.
I am trying to find ways to make better use of my time while programming.
I have a python script that does some heavy work (it can take hours) to finish. Now, most of the work it does is network related, so i have plenty of cpu resources to spare.
If the script was a C binary executable, it would be fine to git checkout onto a different branch and do extra work, I could even modify the binary in disk as it has been copied to ram, so until it finishes running I won't affect program output.
But python scripts are translated, not compiled. What happens if I start tampering with the source file, can i corrupt the programs output, or is the text file and associated imports copied to RAM, allowing me to tamper with the source with no risk of changing the behaviour of the running program?
In general, if you have a single Python file which you run as a script, you're fine. When you run the file, it is compiled into bytecode which is then executed. You can change the original script at this point and nothing breaks.
However, we can deliberately break it by writing some horrible but legal code like this:
horrible.py:
from time import sleep
sleep(10)
import silly
silly.thing()
silly.py:
def thing():
print("Wow!")
You can run horrible.py and while it is running you can edit silly.py on disk to make it do something else. When silly.py is finally imported, the updated version will be loaded.
A workaround is to put all your imports at the top of the file, which you probably do anyway.
When a python program is run it is compiled (kinda, more like translated) into a .pyc file that is then run by the python interpreter. When you change a file it should NOT affect the code if it is already running.
Here is a related stackoverflow answer. What will happen if I modify a Python script while it's running?
Why not have another working directory where you make your modifications? Is there a lot of ancillary data or something that makes it hard to set up a working directory? I.e. if your working directory is A, git clone A B, and then work in B. When you're done, you can pull the changes back from B to A:
git remote add B ../B
git pull B master
I have a test case for an algorithm, which gives a different result after the first execution.
The test imports the algorithm and the test data from two files.
The first execution returns the correct results and creates a .pyc file for the test data file.
The second and all following executions return incorrect results.
When I delete the test data's .pyc file the next execution returns the correct results again (and creates a new .pyc file again).
When I move the test data into the same file as the test case itself (i.e. avoiding the creation of a .pyc file) the test always passes.
I cannot apply this fix to my full program.
Is this a known issue, is there a fix?
.pyc files contain byte code, which is what the Python interpreter compiles the source to. This code is then executed by Python's virtual machine.
Python's documentation explains the definition like this:
Python is an interpreted language, as opposed to a compiled one,
though the distinction can be blurry because of the presence of the
bytecode compiler. This means that source files can be run directly
without explicitly creating an executable which is then run
The .pyc files are created (and possibly overwritten) only when that python file is imported by some other script. If the import is called, Python checks to see if the .pyc file's internal timestamp matches the corresponding .py file. If it does, it loads the .pyc; if it does not or if the .pyc does not yet exist, Python compiles the .py file into a .pyc and loads it.
One thing I found that changes is the value of file (.pyc vs .py), which tripped me up writing a utility invoking stack traces.
I have a python script which takes a while to finish its executing depending on the passed argument. So if I run them from two terminals with different arguments, do they get their own version of the code? I can't see two .pyc files being generated.
Terminal 1 runs: python prog.py 1000 > out_1000.out
Before the script running on terminal 1 terminate, i start running an another; thus terminal 2 runs: python prog.py 100 > out_100.out
Or basically my question is could they interfere with each other?
If you are writing the output to the same file in disk, then yes, it will be overwritten. However, it seems that you're actually printing to the stdout and then redirect it to a file. So that is not the case here.
Now answer to your question is simple: there is no interaction between two different executions of the same code. When you execute a program or a script OS will load the code to the memory and execute it and subsequent changes to code has nothing to do with the code that is already running. Technically a program that is running is called a process. Also when you run a code on two different terminals there will be two different processes on the OS one for each of them and there is no way for two process to interfere unless you explicitly do that (IPC or inter-process communication) which you are doing here.
So in summary you can run your code simultaneously on different terminals they will be completely independent.
Each Python interpreter process is independent. How the script reacts to itself being run multiple times depends on the exact code in use, but in general they should not interfere.
.pyc file reference http://effbot.org/pyfaq/how-do-i-create-a-pyc-file.htm
Python automatically compiles your script to compiled code, so called
byte code, before running it. When a module is imported for the first
time, or when the source is more recent than the current compiled
file, a .pyc file containing the compiled code will usually be created
in the same directory as the .py file.
If you afraid your code get overwritten due to whatever mistake, you should learn to put your code under VERSION CONTROL. Register github and use git to do that.
bigger sign ">" will send the output to the right handler. It you specify file name, it will push the output to that file name. Even in different terminal, if you run the code inside the same folder, use the ">" point to SAME file name, the file on the right of the ">" definitely get overwrite.
Program SOURCE CODE ARE NOT mutable during execution. Unless you acquire high level program hacking skill.
Each program will run inside its "execution workspace". Unless you make a code that tap into same resources(like change same file,shared reources ), otherwise there is no interference. (except if one exhaust all CPU, Memory resources, the second one will be interfere, but that is other story)
I'm having the same problem on Windows and Linux. I launch any of various python 2.6 shells and run nose.py() to run my test suite. It works fine. However, the second time I run it, and every time thereafter I get exactly the same output, no matter how I change code or test files. My guess is that it's holding onto file references somehow, but even deleting the *.pyc files, I can never get the output of nose.run() to change until I restart the shell, or open another one, whereupon the problem starts again on the second run. I've tried both del nose and reload(nose) to no avail.
Solved* it with some outside help. I wouldn't consider this the proper solution, but by searching through sys.modules for all of my test_modules (which point to *.pyc files) and deling them, nose finally recognizes changes again. I'll have to delete them before each nose.run() call. These must be in-memory versions of the pyc files, as simply deleting them out in the shell wasn't doing it. Good enough for now.
Edit:
*Apparently I didn't entirely solve it. It does seem to work for a bit, and then all of a sudden it won't anymore, and I have to restart my shell. Now I'm even more confused.