Suppose I have a minimal PyInstaller spec file, e.g. the hello_world.spec that is created when I run pyinstaller hello_world.py (also see docs). This spec file has python code but no import statements.
Now suppose I customize this file, e.g. using the Tree and TOC classes, but something is going wrong and I need to find out what.
I am aware of the PyInstaller --log-level options and warn*.txt files, but I would prefer to place some break-points in my IDE and debug the spec file (or maybe just play around with the Tree class in the console). However, debugging does not work out-of-the-box because there are no import statements in the spec file. I can add those, as below, for example:
from PyInstaller.building.build_main import Analysis, PYZ, EXE, COLLECT
from PyInstaller.building.datastruct import TOC, Tree
But then it turns out some configuration info is required, as I keep running into KeyErrors related to CONF. I tried adding those key/value-pairs manually, based on a list of globals from the docs, which appears to work, up to a point, but I cannot help thinking I'm doing something wrong...
import PyInstaller.config
PyInstaller.config.CONF['specnm'] = 'hello_world'
... etc. ...
Could someone tell me what is the right way to do this? Should I just stick with the pyinstaller --log-level approach?
A couple of years later, still no answer, so here's one alternative I came up with:
One approach is to use unittest.mock.Mock to mock the PyInstaller classes that are not relevant for the problem at hand.
For example, my spec file has some custom code that generates some of the values passed into the Analysis class. In order to debug that code, I just mock out all the PyInstaller classes and run the spec file, using runpy.run_path from the standard library, as follows:
from runpy import run_path
from unittest.mock import Mock
mocks = dict(Analysis=Mock(), EXE=Mock(), PYZ=Mock(), COLLECT=Mock())
run_path(path_name="path/to/my.spec", init_globals=mocks)
This is also very useful to extract the values of arguments defined in the spec file. For example, we can extract the name value passed into EXE() as follows:
exe_name = mocks["EXE"].call_args.kwargs["name"]
Found this unresolved thread, I found that putting an old style Python breakpoint in the spec file works with Pyinstaller 4.8:
...
import pdb; pdb.set_trace()
...
Related
I would like to find out where the source code for the mindstorms module is for the Mindstorms Robot Inventor.
At the start of each file there is a starting header of
from Mindstorms import ...
Etc..
That is what I want to find.
I have tryed multiple python methods to file the file path, but they all return ./projects/8472.py
Thanks,
henos
You've likely been using __file__ and similar, correct? Yes, that will give you the currently running file, but since you can't edit the code for the actual Mindstorms code, it's not helpful. Instead, you want to inspect the directory itself, like you would if your regular Python code were accessing a data file elsewhere.
Most of the internal systems are .mpy files (see http://docs.micropython.org/en/v1.12/reference/mpyfiles.html), so directly reading the code from the device is less than optimal. Additionally, this means that many "standard library" packages are missing or incomplete; you can't import pathlib but you can import os, but you can't use os.walk(). Those restrictions make any sort of directory traversal a little more frustrating, but not impossible.
For instance, the file runtime/extensions.music.mpy looks like the following (note: not copy-pasted since the application doesn't let you):
M☐☐☐ ☐☐☐
☐6runtime/extensions/music.py ☐☐"AbstractExtensions*☐☐$abstract_extension☐☐☐☐YT2 ☐☐MusicExtension☐☐4☐☐☐Qc ☐|
☐☐ ☐ ☐☐ ☐☐☐☐ ☐2 ☐☐play_drum2☐☐☐play_noteQc ☐d:
☐☐ *☐☐_call_sync#☐,☐+/-☐☐drumb6☐YQc☐ ☐☐drum_nos☐musicExtension.playDrum☐☐E☐
☐ *☐ #☐,☐+/-☐☐instrumentb*☐☐noteb*☐☐durationb6☐YQc☐ ☐☐☐☐s☐musicExtension.playNote
Sure, you can kind of see what's going on, but it isn't that helpful.
You'll want to use combinations of os.listdir and print here, since the MicroPython implementation doesn't give access to os.walk. Example code to get you started:
import os
import sys
print(os.uname()) # note: doesn't reflect actual OS version, see https://stackoverflow.com/questions/64449448/how-to-import-from-custom-python-modules-on-new-lego-mindstorms-robot-inventor#comment115866177_64508469
print(os.listdir(".")) # ['util', 'projects', 'runtime', ...]
print(os.listdir("runtime/extenstions")) # ['__init__.mpy', 'abstract_extension.mpy', ...]
with open("runtime/extensions/music/mpy", "r") as f:
for line in f:
print(line)
sys.exit()
Again, the lack of copy-paste from the console is rough, so even when you do get to the "show contents of the file on screen" part, it's not that helpful.
One cool thing to note though is that if you load up a scratch program, you can read the code in regular .py. It's about as intelligible as you'd expect, since it uses very low-level calls and abstractions, not the hub.light_matrix.show_image("CLOCK6") that you'd normally write.
I've been getting na error in PyCharm and I can't figure out why I'm getting it:
No tests were found
This is what I have for my point_test.py:
import unittest
import sys
import os
sys.path.insert(0, os.path.abspath('..'))
from ..point import Point
class TestPoint(unittest.TestCase):
def setUp(self):
pass
def xyCheck(self,x,y):
point = Point(x,y)
self.assertEqual(x,point.x)
self.assertEqual(y,point.y)
and this point.py, what I'm trying to test:
import unittest
from .utils import check_coincident, shift_point
class Point(object):
def __init__(self,x,y,mark={}):
self.x = x
self.y = y
self.mark = mark
def patched_coincident(self,point2):
point1 = (self.x,self.y)
return check_coincident(point1,point2)
def patched_shift(self,x_shift,y_shift):
point = (self.x,self.y)
self.x,self,y = shift_point(point,x_shift,y_shift)
Is it something wrong with my run configuration? I looked at this SO post but I'm still utterly confused. My run configuration currently looks like this:
In order to recognize test functions, they must be named test_. In your case, rename xyCheck to test_xyCheck :)
I know it's more than a year since the question was asked, but I had the same issue and that post was the first result in search.
As I understood PyCharm (or Intellij Idea Python plugin) needs your test to meet the following criteria if you want it to be launched when you run all the tests in directory.
test functions should start with "test" (underscore is not necessary)
the file, containing the test should also start with "test". "Test" (with capital T doesn't work in my case
I'm using Intellij IDEA 2016.3.5 with Python plugin
If you want to run you tests with command line
python -m unittest
Then you should add __init__.py to test directory. Python still wants your test function names to start with "test", and you test file name to start with "test", but in case of files it doesn't care if the first "t" is capital or not. TestCase and test_case is equally fine.
One thing that can also cause this problem is if you have not selected the right testing framework in your settings:
settings > Python Integrated Tools > Testing > Default test runner
All my tests are in pytest, but it was set to unit test
Don't use dashes ("-") in your filename. These files were being ignored on my machine. renaming them from project-tests.py to project_tests.py solved the problem.
Another gotcha that has just bitten me.
I had a test file within my test package called test_queue.py which followed all of the above advice, however when I selected "Run UnitTests" in PyCharm the console reported no tests found.
The issue in my case was that I had a non-unit test file in the root of the project also called test_queue.py which had been used for some other purpose early on in the project and forgotten about.
Even though I was specifically selecting the test file in my tests folder, and the path was set to absolutely point at the unit test version of the file, it seems the file in the root of the project was being used.
So, one more thing to check, make sure there are no other files in your project with the same name.
Another issue you may want to check is if you see in the console output "## tests deselected by '-k XXXX'". This means you've added a Keyword to the Run/Debug Configuration, and Pycharm will only run tests whose name contains the keyword.
Adding another answer in the hopes that it helps someone down the road. I've been fighting this problem and just figured out the answer (I think). I originally deleted the standard Django file tests.py out of my application folder. I also created a subdirectory of my project called tests, which contains separate test scripts. In this situation, Pycharm failed to find the tests. I corrected this simply by creating an empty file called tests.py in my application folder.
So:
Make sure you have a file called tests.py in your application director (it can be an empty file)
It seems that a folder called tests, in your project, can contain separate test scripts and Pycharm seems to find and run these.
Here's a picture of the directory structure that's working for me:
I had this exception when running individual tests in a Django 1.8 project in PyCharm 2018.1. I could run all the tests together, but individual tests in one file crashed.
The exception was happening in unittest's loader.py
It was getting an ImportError trying to import test_admin_views.py, though the exception was hiding the details of that error.
To see the details of the ImportError, I opened a Python Console and ran:
import my_app.users.tests.test_admin_views
This gave me:
Traceback (most recent call last):
[...]
File "my_app/my_app/users/tests/model_factories.py", line 42, in <module>
from my_app.applications.tests.model_factories import ApplicationFactory
ImportError: cannot import name ApplicationFactory
I noticed that some of the other factories in that file are imported without using the full path, so I tried adding ApplicationFactory to the relative path import list:
from .model_factories import UserFactory, UserGroupFactory, ApplicationFactory
Which worked!
None of these helped me, but I was able to fix the issue by setting the working directory to the project directory.
I debugged this by first creating a new Django test configuration witj a target of one of the app names with tests.
That ran successfully. No idea why working directory should need to be specified.
I have written some python scripts, and embedded some metadata at the top, i.e.:
__version__ = '0.6.1'
Someone contributed a setup.py, mostly just copying the meta data from the script, so we have:
setup(
(...)
version='0.4.0',
I dislike such duplication a lot, the reason should be obvious above. Updating the meta data is done by people, and not all people know/remember that it needs to be updated at both places. Mistakes are bound to happen. Not to forget the fact that it's also easier to just update the version number one place. I've attempted to deduplicate the meta data by removing it from the code itself, but ... it's actually referenced in the script (it is a script and it accepts --version and --help after all). It's also considered "best practice" to embed such meta data in the script.
I can certainly do some tricks to let setup.py read the meta data from the script. Indeed, it would probably take me less time than to write this question - I more want to know what is considered "best practice". Is it really considered "best practice" to duplicate this information both in setup.py and in the library/script? Does there exist some ready-made cookbook boilerplate code for reading the meta data from the script/package itself? Wouldn't it make sense if the setup method could do this by itself?
You could just import your package and get the metadata from there.
This does assume that your file structure looks like this:
root/
code.py
setup.py
Or
root/
code/
__init__.py
setup.py
And either code.py or __init__.py have the metadata.
import os
import sys
__dir__ = path.abspath(path.dirname(__file__)) # The directory of setup.py, root/
sys.path.insert(0, __dir__) # Will search __dir__ first for the package
try:
import code
finally:
sys.path.pop(0)
setup_args = dict(
name='code',
# ...
)
for metadata in ('version', 'author', 'email', 'license'):
if hasattr(code, '__{}__'.format(metadata)):
setup_args[metadata] = getattr(code, '__{}__'.format(metadata))
setup(**setup_args)
After a bit of research into this, I'll answer myself ...
First of all - apparently there doesn't exist any convention to embed metadata in the script or library itself - I can only guess that it was my own invention to add __author__, __license__ etc into the script.
ATOH, there exists a convention to add __version__ to scripts and libraries. PEP 8 describes that this should be automatically set to the VCS revision number on every commit - though it is commonly set to a symantic version number. PEP 396 was an attempt to clear this up, but was deferred due to a lack of interesst or time.
Hence when having symantic version numbers in the __version__ and also supplying the project with a setup.py, one will either need to update the version number in two places, or do some tricks to get it auto-updated at one place (or possibly let the VCS autoupdate it both places).
As for myeself, I ended up importing the script into the setup.py and reading __version__ from there -
https://github.com/tobixen/calendar-cli/commit/854948ddc057f0ce7cc047b3618e8d5f6a1a292c
I'm trying to copy an entire directory from one locations to another via python every 7 days to essentially make a backup...
The backup folder/tree folder may or may not exist so it needs to create the folder if it doesn't exist, that's why I assumed distutils is better suited over shutil
Note Is it better for me to use batch or some other language for the said job?
The following code:
import distutils
distutils.dir_util.copy_tree("C:\Users\A\Desktop\Test", "C:\Users\A\Desktop\test_new", preserve_mode=1, preserve_times=1, preserve_symlinks=0, update=1, verbose=0, dry_run=0)
Returns:
Traceback (most recent call last):
File "C:\Users\A\Desktop\test.py", line 2, in <module>
distutils.dir_util.copy_tree("C:\Users\A\Desktop\test", "C:\Users\A\Desktop\test2", preserve_mode=1, preserve_times=1, preserve_symlinks=0, update=1, verbose=0, dry_run=0)
AttributeError: 'module' object has no attribute 'dir_util'
What am I doing wrong?
Thanks in advance
- Hyflex
You need to import dir_util specifically to access it's functions:
from distutils import dir_util
If there are other modules in that package that you need, add them to the line, separated by commas. Only import the modules you need.
For Unix/Linux, I suggest 'rsync'.
For windows: xcopy
I've been attempting essentially the same thing to back up what I write on a plethora of virtual machines.
I ran into the same problem you did with distutils. From what I can tell, the Python community is using the distutils module to start standardizing how new modules interface with Python. I think they're still in the thick of it though as everything I've seen relating to it seems more complicated, not less complicated. Hopefully, I'm just seeing all the crazy that happens in the middle of a big change.
But I did figure out how to get it working. To use distutil.dir_util.copytree(),
>>> from distutils import dir_util
>>> dir_util.copy_tree("/home/user/backing_up/temp", "/home/user/backing_up/other")
['/home/user/backing_up/other/stuff.txt'] # Return value indicating success
If you feel like it's worthwhile, you can import distutils.core and make the longer call to distutils.dir_util.copy_tree().
>>> import distutils.core
>>> distutils.dir_util.copy_tree("/home/user/backing_up/temp", "/home/user/backing_up/other")
['/home/user/backing_up/other/stuff.txt'] # Return value indicating success
(I know, I know, there are subtle differences between "import module.submodule" and "from module import submodule" but that's not the intent of the question and so long as you're importing the correct stuff and calling the functions appropriately, it doesn't make a difference.)
Like you, I also explicitly stated that I wanted the default for preserve_mode and preserve_times, but I didn't touch the other variables. Everything worked as expected once I imported and called the function the way it wanted me to.
Now that my back up script works, I realize I should have written it in Bash since I plan on having it run whenever the machine goes to a specific runlevel. I'm using a wrapper instead now, even if I should just re-write it.
I am importing a lot of different scripts, so at the top of my file it gets cluttered with import statements, i.e.:
from somewhere.fileA import ...
from somewhere.fileB import ...
from somewhere.fileC import ...
...
Is there a way to move all of these somewhere else and then all I have to do is import that file instead so it's just one clean import?
I strongly advise against what you want to do. You are doing the global include file mistake again. Although only one module is importing all your modules (as opposed to all modules importing the global one), the remaining point is that if there's a valid reason for all those modules to be collected under a common name, fine. If there's no reason, then they should be kept as separate includes. The reason is documentation. If I open your file, and see only one import, I don't get any information about what is imported and where it comes from. If on the other hand, I have the list of imports, I know at a glance what is needed and what not.
Also, there's another important error I assume you are doing. When you say
from somewhere.fileA import ...
from somewhere.fileB import ...
from somewhere.fileC import ...
I assume you are importing, for example, a class, like this
from somewhere.fileA import MyClass
this is wrong. This alternative solution is much better
from somewhere import fileA
<later>
a=fileA.MyClass()
Why? two reasons: first, namespacing. If you have two modules having a class named MyClass, you would have a clash. Second, documentation. Suppose you use the first option, and I find in your code the following line
a=MyClass()
now I have no idea where this MyClass comes from, and I will have to grep around all your files in order to find it. Having it qualified with the module name allows me to immediately understand where it comes from, and immediately find, via a /search, where stuff coming from the fileA module is used in your program.
Final note: when you say "fileA" you are doing a mistake. There are modules (or packages), not files. Modules map to files, and packages map to directories, but they may also map to egg files, and you may even create a module having no file at all. This is naming of concepts, and it's a lateral issue.
Of course there is; just create a file called myimports.py in the same directory where your main file is and put your imports there. Then you can simply use from myimports import * in your main script.