rPython and __future__ imports - python

My python script starts with
from __future__ import division
In R I do
library(rPython)
python.load("myscript.py")
I get
File "", line 2 SyntaxError: from future imports must
occur at the beginning of the file

I just bumped into the same problem - apparently python.load() is simply executing the script loaded from the location as if it were a bunch of commands.
I'm not sure if it's wrapped or preceded with some boilerplate code by default somehow, but it seems so. And if you were to catch errors using rPython it would surely be executed within a try... block (given the current code on GitHub at least).
However, using a workaround based on execfile() did the job for me:
python.exec("execfile('myscript.py')")
Another approach is, if there's no need to execute code in the main block, to import the module
python.exec("import myscript")
however, in this slightly more convoluted case, you likely have to deal with path problems, as mentioned e.g. here.
(It would probably be a good idea to let the package maintainers know about this situation, and that it could use something better than a workaround.)

Related

Is there any way to change the name of creator of a module?

I have created a code that automates some kind of task using python.
So for that I imported a module named Pywhatkit.
But after importing when I run the program first I get this(shown in the attached image below) note from creator of Pywhatkit(highlighted part in the attachment below)and then I am able to run my program.
Sometimes this seems unprofessional.
So is there any way too either change the name that appears on running the program or is there any way with which I can remove this part?
Python modules are generally written in python. If you need to make a modification like this, you can usually just look up where the module is installed and change the relevant bit of the code (usually in your python version's site_modules directory, in a folder named the same as the module, where the file __init__.py is the first thing executed, so that's a good place to start if you're blindly searching for something to change).
In this particular case, do the following:
Run the console command pip show pywhatkit to find the location of the installed pywhatkit module. Should be the third-to-last line of that command's output. I'll call this $pwkdir
Open the file $pwkdir/pywhatkit/mainfunctions.py in your text editor of choice
Comment out lines 300 through 304, and save the file.
the cause of the output you're seeing is a straightforward print() call, so removing it is easy and harmless.
I found the location that needs to be commented out by doing the command grep -nr 'Hello from the' $pwkdir/pywhatkit (i.e. searching for any usages of the observed phrase, because the first three words are enough to identify it), and reading the code.
You will probably need to do this again every time you reinstall or update this module to a new version.
Note that there are other places within the module where it prints to console. You may wish to search for and comment out those lines as well, or disable printing to stdout before importing the module for the first time.
Noted, this "Note from the creator" will be removed from the next update. Meanwhile, you can do as suggested in the other answer, you may consider editing the "mainfunctions.py", commenting the last print statement of that file will stop printing the message.

Is it possible to execute indented lines from another file in Python?

Say, I have two files demo.py
# demo.py
from pathlib import Path
for i in range(5):
exec(Path('another_file.txt').read_text())
and another_file.txt (note the indent)
print(i)
Is it possible to make python demo.py run?
N.B. This is useful when using Page (or wxformbuilder or pyqt's designer) to generate a GUI layout where skeletons of callback functioins are automatically generated. The skeletons will have to be modified, at the same time, each iteration overwrites the skeletons -- code snippets will have to be copied back. Anyway, you know what I am talking about if you have used any of Page or wxformbuilder or pyqt's designer.
You can solve the basic problem by removing the indent:
from pathlib import Path
import textwrap
for i in range(5):
exec(textwrap.dedent(Path('another_file.txt').read_text()))
There are still two rather major problems with this:
There are serious security implications here. You're running code without including it in your project. The idea that you can "worry about security and other issues later" will cause you pain at a later point. You'll see similar advice on this site with avoiding SQL injection. That later date may never come, and even if it does, there's a very real chance you won't remember or correctly identify all of the problems. It's absolutely better to avoid the problems in the first place.
Also, with dynamic code like this, you run the very real risk of running into a syntax error with a call stack that shows you nothing about where the code is from. It's not bad for simple cases like this, but as you add more and more complexity to a project like this, you'll likely find you're adding more support to help you debug issues you run into, rather than spending your time adding features.
And, combining these two problems can be fun. It's contrived, but if you changed the for loop to a while loop like this:
i = 0
while i < 5:
exec(textwrap.dedent(Path('another_file.txt').read_text()))
i += 1
And then modified the text file to be this:
print(i)
i += 1
It's trivial to understand why it's no longer operating the 5 times you expect, but as both "sides" of this project get more complex, figuring out the complex interplay between the elements will get much harder.
In short, don't use eval. Future you will thank past you for making your life easier.

Run the main python program from any sub-program file

This is more of a convenience than a real problem, but the project I'm working on has a lot of separate files, and I want to basically be able to run any of those files (that all basically only contain classes) to run the main file.
Now in the middle of writing the first sentence of this question, I tried just importing main.py into each file, and that seemed to work fine and dandy, but I cant help but feeling that:
it might cause problems, and
that I had problems with circular imports before and I am somewhat surprised that nothing came up.
First let me say: this is most likely a bad idea, and it's definitely not at all standard. It will likely lead to confusion and frustration down the road.
However, if you really want to do it, you can put:
if __name__ == "__main__":
from mypackage import main
main.run()
Which, assuming mypackage.main.run() is your main entry point, will let you run any file you want as if it were the main file.
You may still hit issues issues with circular imports, and those will be completely unavoidable, unless mypackage.main doesn't import anything… Which would make it fairly useless :)
As an alternative, you may wish to use a testing framework like doctest or unittest, then configure your IDE to run the unit tests from a hotkey. This way you're automatically building the repeatable tests as you develop your code.

Dangerous Python Keywords?

I am about to get a bunch of python scripts from an untrusted source.
I'd like to be sure that no part of the code can hurt my system, meaning:
(1) the code is not allowed to import ANY MODULE
(2) the code is not allowed to read or write any data, connect to the network etc
(the purpose of each script is to loop through a list, compute some data from input given to it and return the computed value)
before I execute such code, I'd like to have a script 'examine' it and make sure that there's nothing dangerous there that could hurt my system.
I thought of using the following approach: check that the word 'import' is not used (so we are guaranteed that no modules are imported)
yet, it would still be possible for the user (if desired) to write code to read/write files etc (say, using open).
Then here comes the question:
(1) where can I get a 'global' list of python methods (like open)?
(2) Is there some code that I could add to each script that is sent to me (at the top) that would make some 'global' methods invalid for that script (for example, any use of the keyword open would lead to an exception)?
I know that there are some solutions of python sandboxing. but please try to answer this question as I feel this is the more relevant approach for my needs.
EDIT: suppose that I make sure that no import is in the file, and that no possible hurtful methods (such as open, eval, etc) are in it. can I conclude that the file is SAFE? (can you think of any other 'dangerous' ways that built-in methods can be run?)
This point hasn't been made yet, and should be:
You are not going to be able to secure arbitrary Python code.
A VM is the way to go unless you want security issues up the wazoo.
You can still obfuscate import without using eval:
s = '__imp'
s += 'ort__'
f = globals()['__builtins__'].__dict__[s]
** BOOM **
Built-in functions.
Keywords.
Note that you'll need to do things like look for both "file" and "open", as both can open files.
Also, as others have noted, this isn't 100% certain to stop someone determined to insert malacious code.
An approach that should work better than string matching us to use module ast, parse the python code, do your whitelist filtering on the tree (e.g. allow only basic operations), then compile and run the tree.
See this nice example by Andrew Dalke on manipulating ASTs.
built in functions/keywords:
eval
exec
__import__
open
file
input
execfile
print can be dangerous if you have one of those dumb shells that execute code on seeing certain output
stdin
__builtins__
globals() and locals() must be blocked otherwise they can be used to bypass your rules
There's probably tons of others that I didn't think about.
Unfortunately, crap like this is possible...
object().__reduce__()[0].__globals__["__builtins__"]["eval"]("open('/tmp/l0l0l0l0l0l0l','w').write('pwnd')")
So it turns out keywords, import restrictions, and in-scope by default symbols alone are not enough to cover, you need to verify the entire graph...
Use a Virtual Machine instead of running it on a system that you are concerned about.
Without a sandboxed environment, it is impossible to prevent a Python file from doing harm to your system aside from not running it.
It is easy to create a Cryptominer, delete/encrypt/overwrite files, run shell commands, and do general harm to your system.
If you are on Linux, you should be able to use docker to sandbox your code.
For more information, see this GitHub issue: https://github.com/raxod502/python-in-a-box/issues/2.
I did come across this on GitHub, so something like it could be used, but that has a lot of limits.
Another approach would be to create another Python file which parses the original one, removes the bad code, and runs the file. However, that would still be hit-and-miss.

Embedded Python - Blocking operations in time module

I'm developing my own Python code interpreter using the Python C API, as described in the Python documentation. I've taken a look on the Python source code and I tried to follow the same steps that are carried out in the standard interpreter when executing a py file. These steps (sequence of C API function calls) are basically:
PyRun_AnyFileExFlags()
PyRun_SimpleFileExFlags()
PyRun_FileExFlags()
PyArena_New()
PyParser_ASTFromFile()
run_mod()
PyAST_Compile()
PyEval_EvalCode()
PyEval_EvalCodeEx()
PyThreadState_GET()
PyFrame_New()
PyEval_EvalFrameEx()
The only difference in my code is that I do manually the AST compilation, frame creation, etc. and then I call PyEval_EvalFrame.
With this, I am able to execute an arbitrary .py file with my program, as if it were the normal Python interpreter. My problem comes when the code that my program is executing makes use of the time module: all time module operations get blocked in the GIL! For example, if the Python code calls time.sleep(1), this call is blocked and never gets executed.
Obviously I am doing something wrong that blocks the GIL (and therefore blocks the time module) but I dont know how to correct it. The last statement in my code where I have control is in PyEval_EvalFrameEx, and from that point on, everything runs "as in regular Python interpreter", I think.
Anybody had a similar problem? What am I doing wrong, so that I block the time module?
Hope somebody can help me...
Thanks for your time. Best regards,
R.
You need to provide more detail.
How does your interpreter's behavior differ from the standard interpreter?
If you just want to run arbitrary source files, why are you not calling one of the higher level interfaces, like PyRun_SimpleFile? Did your code call Py_Initialize?

Categories

Resources