I would like to find out where the source code for the mindstorms module is for the Mindstorms Robot Inventor.
At the start of each file there is a starting header of
from Mindstorms import ...
Etc..
That is what I want to find.
I have tryed multiple python methods to file the file path, but they all return ./projects/8472.py
Thanks,
henos
You've likely been using __file__ and similar, correct? Yes, that will give you the currently running file, but since you can't edit the code for the actual Mindstorms code, it's not helpful. Instead, you want to inspect the directory itself, like you would if your regular Python code were accessing a data file elsewhere.
Most of the internal systems are .mpy files (see http://docs.micropython.org/en/v1.12/reference/mpyfiles.html), so directly reading the code from the device is less than optimal. Additionally, this means that many "standard library" packages are missing or incomplete; you can't import pathlib but you can import os, but you can't use os.walk(). Those restrictions make any sort of directory traversal a little more frustrating, but not impossible.
For instance, the file runtime/extensions.music.mpy looks like the following (note: not copy-pasted since the application doesn't let you):
M☐☐☐ ☐☐☐
☐6runtime/extensions/music.py ☐☐"AbstractExtensions*☐☐$abstract_extension☐☐☐☐YT2 ☐☐MusicExtension☐☐4☐☐☐Qc ☐|
☐☐ ☐ ☐☐ ☐☐☐☐ ☐2 ☐☐play_drum2☐☐☐play_noteQc ☐d:
☐☐ *☐☐_call_sync#☐,☐+/-☐☐drumb6☐YQc☐ ☐☐drum_nos☐musicExtension.playDrum☐☐E☐
☐ *☐ #☐,☐+/-☐☐instrumentb*☐☐noteb*☐☐durationb6☐YQc☐ ☐☐☐☐s☐musicExtension.playNote
Sure, you can kind of see what's going on, but it isn't that helpful.
You'll want to use combinations of os.listdir and print here, since the MicroPython implementation doesn't give access to os.walk. Example code to get you started:
import os
import sys
print(os.uname()) # note: doesn't reflect actual OS version, see https://stackoverflow.com/questions/64449448/how-to-import-from-custom-python-modules-on-new-lego-mindstorms-robot-inventor#comment115866177_64508469
print(os.listdir(".")) # ['util', 'projects', 'runtime', ...]
print(os.listdir("runtime/extenstions")) # ['__init__.mpy', 'abstract_extension.mpy', ...]
with open("runtime/extensions/music/mpy", "r") as f:
for line in f:
print(line)
sys.exit()
Again, the lack of copy-paste from the console is rough, so even when you do get to the "show contents of the file on screen" part, it's not that helpful.
One cool thing to note though is that if you load up a scratch program, you can read the code in regular .py. It's about as intelligible as you'd expect, since it uses very low-level calls and abstractions, not the hub.light_matrix.show_image("CLOCK6") that you'd normally write.
Related
I have a bytecode document that declares functions and a logo. I also have a .py file where I call the bytecode to output the logo and strings in the functions. How do I go about actually executing the bytecode? I was able to dissemble it and see the assembly code. How can I actually run it?
question.py
import dis
import logo
def work_here():
# execute the bytecode
def main():
work_here()
if __name__ == '__main__':
main()
Try something like:
import dis
code = 'some byte code'
b_code = dis.Bytecode(code)
exec(b.codeobj)
To import a .pyc file, you just do the same thing you do with a .py file: import spam will find an appropriately-placed spam.pyc (or rather, something like __pycache__/spam.cpython-36.pyc) just as it will find an appropriately-placed spam.py. Its top-level code gets run, any functions and classes get defined so you can call them, etc., exactly the same as with a .py file; the only difference is that there isn't source text to show for things like tracebacks or debugger stepping.
If you want to programmatically import a .pyc file by explicit path, or execute one without importing it, you again do the same thing you do with a .py file.
Look at the Examples in importlib. For example:
path = 'bytecoderepo/myfile.pyc'
spec = importlib.util.spec_from_file('myfile', path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
And now, the code in bytecoderepo/myfile.pyc has been executed, and the resulting module is available in the variable mod, but it isn't in sys.modules or stored as a global.
If you actually need to dig into the .pyc format and, e.g., extract the bytecode of some function so you can exec it (or build a function object out of it) without executing the main module code, the details are only documented in the source, and subject to change between Python versions. Start with importlib; being able to (validate and) skip over the header and marshal.loads the body may be as far as you need to learn, but probably not (since ultimately, that's what the module loader already does for you in the sample code above, so if that's not good enough, you need to get deeper into the internals).
I have the following function:
unpack_binaryfunction('third-party/jdk-6u29-linux-i586.bin' , ('/home/user/%s/third-party' % installdir), 'jdk1.6.0_29')
Which uses os.sys to execute a java deployment. The line, combined with the function (Which is unimportant, it just calls some linux statements) works perfectly.
However, this only works if in the 'third-party' folder is specificaly that version of the jdk.
Therefore I need a code that will look at the files in the 'third-party' folder and find one that starts with 'jdk' and fill out the rest of the filename itself.
I am absolutely stuck. Are there any functions or libraries that can help with file searching etc?
To clarify: I need the code to not include the entire: jdk-6u29-linux-i586.bin but to use the jdk-xxxx... that will be in the third-party folder.
This can easily be done using the glob module, and then a bit a string parsing to extract the version.
import glob
import os.path
for path in glob.glob('third-party/jdk-*'):
parent, name = os.path.split(path) # "third-party", "jdk-6u29-linux-i586.bin"
version, update = name.split('-')[1].split('u') # ("6", "29")
unpack_binaryfunction(path, ('/home/user/%s/third-party' % installdir), 'jdk1.{}.0_{}'.format(version, update))
I have written three simple scripts (which I will not post here, as they are part of my dissertation research) that are all in working order.
What I would like to do now is write a "batch-processing" script for them. I have many (read as potentially tens of thousands) of data files on which I want these scripts to act.
My questions about this process are as follows:
What is the most efficient way to go about this sort of thing?
I am relatively new to programming. Is there a simple way to do this, or is this a very complex endeavor?
Before anyone downvotes this question as "unresearched" or whatever negative connotation comes to mind, PLEASE just offer help. I have spent days reading documentation and following leads from Google searches, and it would be most appreciated if a human being could offer some input.
If you just need to have the scripts run, probably a shell script would be the easiest thing.
If you want to stay in Python, the best way would be to have a main() (or somesuch) function in each script (and have each script importable), have the batch script import the subscript and then run its main.
If staying in Python:
- your three scripts must have the .py ending to be importable
- they should either be in Python's search path, or the batch control script can set the path
- they should each have a main function (or whatever name you choose) that will activate that script
For example:
batch_script
import sys
sys.path.insert(0, '/location/of/subscripts')
import first_script
import second_script
import third_script
first_script.main('/location/of/files')
second_script.main('/location/of/files')
third_script.main('/location/of/files')
example sub_script
import os
import sys
import some_other_stuff
SOMETHING_IMPORTANT = 'a value'
def do_frobber(a_file):
...
def main(path_to_files):
all_files = os.listdir(path_to_files)
for file in all_files:
do_frobber(os.path.join(path_to_files, file)
if __name__ == '__main__':
main(sys.argv[1])
This way, your subscript can be run on its own, or called from the main script.
You can write a batch script in python using os.walk() to generate a list of the files and then process them one by one with your existing python programs.
import os, re
for root, dir, file in os.walk(/path/to/files):
for f in file:
if re.match('.*\.dat$', f):
run_existing_script1 root + "/" file
run_existing_script2 root + "/" file
If there are other files in the directory you might want to add a regex to ensure you only process the files you're interested in.
EDIT - added regular expression to ensure only files ending ".dat" are processed.
I am importing a lot of different scripts, so at the top of my file it gets cluttered with import statements, i.e.:
from somewhere.fileA import ...
from somewhere.fileB import ...
from somewhere.fileC import ...
...
Is there a way to move all of these somewhere else and then all I have to do is import that file instead so it's just one clean import?
I strongly advise against what you want to do. You are doing the global include file mistake again. Although only one module is importing all your modules (as opposed to all modules importing the global one), the remaining point is that if there's a valid reason for all those modules to be collected under a common name, fine. If there's no reason, then they should be kept as separate includes. The reason is documentation. If I open your file, and see only one import, I don't get any information about what is imported and where it comes from. If on the other hand, I have the list of imports, I know at a glance what is needed and what not.
Also, there's another important error I assume you are doing. When you say
from somewhere.fileA import ...
from somewhere.fileB import ...
from somewhere.fileC import ...
I assume you are importing, for example, a class, like this
from somewhere.fileA import MyClass
this is wrong. This alternative solution is much better
from somewhere import fileA
<later>
a=fileA.MyClass()
Why? two reasons: first, namespacing. If you have two modules having a class named MyClass, you would have a clash. Second, documentation. Suppose you use the first option, and I find in your code the following line
a=MyClass()
now I have no idea where this MyClass comes from, and I will have to grep around all your files in order to find it. Having it qualified with the module name allows me to immediately understand where it comes from, and immediately find, via a /search, where stuff coming from the fileA module is used in your program.
Final note: when you say "fileA" you are doing a mistake. There are modules (or packages), not files. Modules map to files, and packages map to directories, but they may also map to egg files, and you may even create a module having no file at all. This is naming of concepts, and it's a lateral issue.
Of course there is; just create a file called myimports.py in the same directory where your main file is and put your imports there. Then you can simply use from myimports import * in your main script.
In a program, and obviously being influenced by the way Java does things, I want to read a static file (a log configuration file, actually) from a directory within the interpreter's PYTHONPATH. I know I could do something like:
import foo
a = foo.__path__
conf = open(a[0] + "/logging.conf")
but I don't know if this is the "Pythonic" way of doing things. How could I distribute the logging configuration file in a way that my application does not need to be externally configured to read it?
In general, that's fine, though I'm not sure you want a[0] above (that will just give you the first character of the path), and you should use os.path.join instead of just appending / to be cross-platform compatible. You might consider making the path canonical, i.e. os.path.abspath(os.path.dirname(foo.__path__)). Note that it won't work if __path__ is in a zip file or other import trickery is in use, but I wouldn't worry about that (it's not normal to do so for the main program in Python, unlike Java).
If you do want to support zipped files, there's pkg_resources, but that's somewhat deprecated at this point (there's no corresponding API I could see in the new packaging module).
Here's a snippet based on the link Nix posted upthread but written in a more functional style:
def search_path(pathname_suffix):
cands = [os.path.join(d,pathname_suffix) for d in sys.path]
try:
return filter(os.path.exists, cands)[0]
except IndexError:
return None