I am using IDLE to write a few small sized Python programs. There are two class files - node.py (Node Class) and position.py (Position class). I have my main module code in main.py from which I instantiate Node and Position objects.
What I have noticed is that - when I make a change in node.py or position.py, check the modules and then run them using F5 the changes are not reflected back when I run main.py as long as all the files are open in IDLE. I noticed that I have to manually close all the three .py files and then close IDLE, start over again and run main.py to see the changes made in node.py and position.py.
What is the issue here? Are my environment variables not being set correctly? I have searched SO and online but have not found a satisfactory answer.
[Details: I am using IDLE version 2.7.3 in Ubuntu. All the three .py files and the corresponding .pyc byte code files are in the same directory. This directory is also seen in sys.path]
What was happening is that I needed to use "import node" but I was using "from node import *" from main.py. This was preventing the main.py from linking to the updated node module!
(Sorry for accepting my own answer, but may be someone would also face the same problem later and hence I am uploading the solution)
I used Ctrl+F6 as Matthew Plourde suggested. Using Python 3 and IDLE, I would have my test file and my core file open. I would then fix errors reported by my assertions in my test file in my core file but the test file would not see the changes. The only solution I had was closing and reopening which is a pain. Ctrl+F6 did work. I could not find the reload function in Python3.
I retract this statement. With python3, if you run IDLE a "-n" because of lack of admin rights or some other security reason you need to import imp and use imp.reload(function) for IDLE to pickup any changes to other python files you are editing and recalling. Otherwise python does not detect the changes and will not reload the files.
http://docs.python.org/3.0/library/imp.html#imp.reload
Related
I always structure my repos in the following way.
repo/main.py
repo/scripts/script.py
…
In main.py, I import script.py in the following manner:
from scripts import script as sc
This always works unless I decide to make changes to script.py. After making changes, if I run main.py from shell, it still imports code from the older script.py without the current changes. Until now, what I would then is just create another branch. However, this leads to a lot of branches in the development process. Is there any way to avoid this? How else should I be importing from the scripts directory to avoid this?
Help would be highly appreciated.
UPDATE
From the answers, I can see that I have caused some confusion. What I mean when I say that I run main.py from shell, I mean executing it with python main.py from the terminal. One can think of main.py as a script that does some math and outputs the answer. In doing those math, it imports script.py from scripts that has additional functions that it (main.py) uses. After running main.py N times, if I choose to update script.py, and then I execute main.py again (in the terminal), it imports the old script.py again and does the math with the older code. The answer does not reflect the changes I just made to script.py. Until now, what I have done, when I have had to go through something like this is just create another new branch and literally copy-paste the old files and the newer script.py in the new branch and execute main.py in the shell. It does import the newer script.py then. One more thing I have noticed is that if I just create a new file as say script2.py and then import it in main.py as
from scripts import script2 as sc
it imports script2.py just as it should - it reflects all the changes made to script.py.
There’s no second import statement in main.py.
On the surface this question sounds like we're repeatedly running
$ python main.py,
which quickly executes and exits.
But from the symptom, it must be the case that
we have a long-lived REPL prompt repeatedly executing the main.py code.
The module you're looking for is
importlib.
Do this:
from importlib import reload
from scripts import script as sc
# [do stuff, then edit script.py]
reload(sc)
# [do more stuff, and see the effect of the edit]
What is going on here?
Well, if you repeatedly execute
import scripts.script
import scripts.script
it turns out that the 2nd and subsequent import does nothing.
It consults sys.modules, finds the module has already
been loaded, and reports "cache hit!" instead of doing
the hard work of pulling in that text file.
The purpose of the reload() function is to
do a cache invalidate and repeat the import, so it actually
pulls in python text from the (presumably edited) source file.
Suppose you have a short-lived $ python main.py process
that runs repeatedly, sometimes after a script.py edit.
So the in-memory sys.module cache is not relevant,
having been discarded each time the process exits.
There is another level of caching at work here.
Typically the cPython interpreter will read script.py,
parse, produce bytecode from the parse tree, and write
the bytecode to script.pyc. More than one output cache
directory is possible, depending on system details.
Upon being asked to read script.py, the interpreter
can look to see if the corresponding .pyc bytecode file
is available and fresh, and then declare "cache hit!",
in which case the .py source file is not even read.
Normally this works great, because source file updates
are infrequent (human editing speed), and the freshness
comparison of file timestamps is effective. If there's
something wrong with those timestamps the whole mechanism
won't work properly, and we might fail to notice the
source was recently edited. Perhaps you suffer from this.
First, identify the relevant .pyc bytecode file.
Then, run main, make an edit, delete the .pyc file,
re-run main, and notice that the edit took effect.
Let us know
how it goes.
I'm asking help today because I'm new to Tkinter and Pyinstaller (and python in general) and I'm having troubles with it.
I have a simple app working with sqlite, tkinter and pyinstaller to compile all of this in an executable program, the entrance point of my program is a file named main.py
This file calls all the dependancies (like the sqlite module for python, tkinter and my other files like classes etc...)
I made a very simple interface, with a Hello World in a tkinter label and a button to go to page 2 which displays page2 (also in a label), just to see if I'm capable of making it all run and compile all of these pieces together.
I can run it throught my shell executing it like : python main.py and everything is working fine.
But when I run pyinstaller on my linux machine, and start executing the program, nothing appears, my database.db (sqlite database file) is created but I don't have any interface like when I run it with my shell. The thing is getting even worse on windows where, once I've my .exe it just opens a shell and crash after few seconds, not even creating the database.
What I did is I created a 'log file', in which I write the steps of the program.
As you can see on the following picture, the 2 first prints are wrote in my log file (on linux), so I think it crashes when I try to create the window.
If any of you have an idea on what I do wrong, I would really appreciate help :)
General
From the PyInstaller manual:
Before you attempt to bundle to one file, make sure your app works correctly when bundled to one folder. It is is much easier to diagnose problems in one-folder mode.
As the comments suggested, use a catch-all try/except block to log all exceptions to a file. That is probably the best way to see what is really happening. Make sure that the logfile is created in an existing location where you have the necessary permissions.
I would suggest to take advantage of the built-in logging module instead of creating your own. It can e.g. automatically add from which file a log line was created.
IMHO, it is probable that the failures on Linux and ms-windows have completely different causes. You should probably treat them as different issues.
Linux
When you use single file mode, that file is unpacked into a temporary folder, probably somewhere in /tmp. Some Linux distributions mount the /tmp filesystem with the noexec flag. This is incompatible with PyInstaller.
ms-windows
On windows, there are basically two different Pythons; python.exe and pythonw.exe. Basically it is one of the quirks of windows that this is necessary. The latter is for GUI programs like tkinter programs. A tkinter script should not show a cmd window. So I'm guessing that PyInstaller calls your command with python.exe instead of pythonw.exe. From the manual:
By default the bootloader creates a command-line console (a terminal window in GNU/Linux and Mac OS, a command window in Windows). It gives this window to the Python interpreter for its standard input and output. Your script’s use of print and input() are directed here. Error messages from Python and default logging output also appear in the console window.
An option for Windows and Mac OS is to tell PyInstaller to not provide a console window. The bootloader starts Python with no target for standard output or input. Do this when your script has a graphical interface for user input and can properly report its own diagnostics.
As noted in the CPython tutorial Appendix, for Windows a file extention of .pyw suppresses the console window that normally appears. Likewise, a console window will not be provided when using a myscript.pyw script with PyInstaller.
Also, on windows it can matter which Python distribution you're using. I used to be a fan of Anaconda, but lately I've come to prefer the python.org version because it gives me less headaches. On anaconda Python I had the problem that tkinter programs would not launch without showing a cmd window, whatever I tried. Only switching to python.org Python solved that problem.
I am trying to find ways to make better use of my time while programming.
I have a python script that does some heavy work (it can take hours) to finish. Now, most of the work it does is network related, so i have plenty of cpu resources to spare.
If the script was a C binary executable, it would be fine to git checkout onto a different branch and do extra work, I could even modify the binary in disk as it has been copied to ram, so until it finishes running I won't affect program output.
But python scripts are translated, not compiled. What happens if I start tampering with the source file, can i corrupt the programs output, or is the text file and associated imports copied to RAM, allowing me to tamper with the source with no risk of changing the behaviour of the running program?
In general, if you have a single Python file which you run as a script, you're fine. When you run the file, it is compiled into bytecode which is then executed. You can change the original script at this point and nothing breaks.
However, we can deliberately break it by writing some horrible but legal code like this:
horrible.py:
from time import sleep
sleep(10)
import silly
silly.thing()
silly.py:
def thing():
print("Wow!")
You can run horrible.py and while it is running you can edit silly.py on disk to make it do something else. When silly.py is finally imported, the updated version will be loaded.
A workaround is to put all your imports at the top of the file, which you probably do anyway.
When a python program is run it is compiled (kinda, more like translated) into a .pyc file that is then run by the python interpreter. When you change a file it should NOT affect the code if it is already running.
Here is a related stackoverflow answer. What will happen if I modify a Python script while it's running?
Why not have another working directory where you make your modifications? Is there a lot of ancillary data or something that makes it hard to set up a working directory? I.e. if your working directory is A, git clone A B, and then work in B. When you're done, you can pull the changes back from B to A:
git remote add B ../B
git pull B master
I am using portable python 1.1 with python 2.6.2. The PyScriptor is 1.9.9.6. I open all the files I am working on with PyScriptor. So, I run my main file and an error shows up with code in one of my imported files. I fix it and run the main file again, but the same error shows up. It is as if the imported file is still the old one but PyScriptor is correctly saving the files I edit. Restarting PyScriptor fixes it, but is a pain to do that for every bug. I tested this happens bu adding a print statement that showed up after restarting, and then removing it and still see the print statement.
You can use reload(imported_module_name) in the interactive shell to reload the module before re-running your script. PyScripter does everything in a single Python instance, which makes debugging easier, but also, as you discovered, makes fixing imported files a bit trickier.
You can also completely reinitialize the Remote Engine from the Run menu to get a fresh interpreter.
I have created a little pre-commit hook in python. This hook works like a charm under Linux, but in Windows it keeps telling me:
error: cannot spawn .git/hooks/pre-commit: No such file or directory
I know there have been similar questions here about the same issue and the conclusion seams to be the shebang. My script has this on the very first line:
#!F:\PortableApps\PortablePython3.2\App\python.exe
It's also interesting to note that executing the script simply by writing .git/hooks/pre-commit works wonderful, but as soon as I try to commit, git spits out the above message.
Another interesting thing is, when I convert the encoding from ANSI to UTF-8 (using Notepad++), I get the following error when trying to execute the script:
.git/hooks/pre-commit: Cannot execute binary file
I'm using the following tools:
PortablePython 3.2.1.1
msysgit 1.7.6 (Portable)
I used the proxy-approach to make the python script work under windows (with msysgit). The complete script (with description on how I did it) might be found here: https://gist.github.com/1839424
Here is the important part about making it work under Windows
If you're working with Windows (and "msysgit"), it's a little more complicated. Since "msysgit" seems to have a problem handling the SHEBANG, you'll have to use a little trick to make the script executable (further information on this problem can be found here).
In order to make the script work, you'll want to remove the SHEBANG from the Python script ("pre-commit.py") and use a wrapper bash-script to call the interpreter. This script should look something like this:
#!/bin/sh
python .git/hooks/pre-commit.py
Store this script as a file called "pre-commit" (no file-ending). This assumes that you have Python in your PATH. If you don't, you can also specify the full path to your interpreter-executable.
This script will be called by "git commit" and call the python-script to check for the huge files. The path after the SHEBANG should not be changed, as "msysgit" will remap it automatically. You must specify a path relative to the repo-root for the Python script to be executed (because thats from where the script is called).
Afterwards you'll want to copy both the wrapper-file ("pre-commit") and the Python-script ("pre-commit.py") to your repos ".git/hooks"-directory, personalize the Python-script ("max_file_size" and "git_binary_path") and mark the "pre-commit"-file executable.