If I were to make, for example, a program that takes as input a sorting algorithm and determines empirically whether or not the algorithm is partially correct, how would I convert the string input to an executable program? I have read other threads in which it was suggested using exec or eval, but all answers recommended against using this method due to security risks. Is there a way to create such a program that does not involve converting a string to executable code? Or will it inherently be a risky program no matter the implementation? Lastly, is there another programming language that would be a better alternative to define such a program?
Executing Arbitrary Code
No matter what language you choose, if you read code from the user and execute that code, it will be dangerous. No ifs, ands, or buts. You notice the same caveats to Python's exec and eval also are noted for Javascript, PHP, and many other languages.
Safely Executing Code from a String
There are safe ways to map strings to predefined functions, but there is no safe way to compile/interpret and execute arbitrary code.
One good example is the following on how to safely map functions to a string:
functions = {
'print': print,
'str': str,
'int': int
}
name = input('Choose from the above functions here')
functions.get(name)()
Static Code Analysis
And for the final answer, potentially, but no, as there would be ways of evaluating a sorting algorithm, but they're unlikely to be effective, reproducible, or accurate without compiling the code or at least interpreting it. Static code analysis is difficult, and can only go so far.
One simple example for how difficult static code analysis can be even with a single if statement is the following:
for index, value in enumerate(range(10)):
if index and value - old == 1:
print(value)
old = value
Some libraries that do static code analysis think this code will raise an error (such as Pylint, for example), because old is defined after it is first used, however, since bool(0) will evaluate to False, old actually only ever checked after the first loop, after it is already defined, and so the code runs without issue.
Think of the complexity of inputs, the complexity of outputs, and the number of variations of possible sort algorithms that would all be equivalent. The easiest way to test code is to run it. There are limitations of dynamic code analysis, but with a given input and then comparing it to the desired output, you can get a good idea if the code works as it should, something that is very difficult with merely static analysis.
Related
I typically work with C++ but off late have to program a lot in Python. Coming from a C++ background, I am finding dynamic typing to be very inconvenient when I have to modify an existing codebase. I know I am missing something very basic and hence turning to the stackoverflow community to understand best practices.
Imagine, there is a class with a number of methods and I need to edit an existing method. Now, in C++, I could explicitly see the datatype of every parameter, check out the .h files of the corresponding class if need be and could quickly understand what's happening. In python on the other hand, all I see are some variable names. I am not sure if it is a list or a dictionary or maybe some custom datastructure with its getters and setters. To figure this out, I need to look at some existing usages of this function or run the code with breakpoints and see what kind of datastructure am I getting. I find either methods to be very time consuming. Is there a faster way to resolve this problem? How should I quickly determine what's the datatype of a particular variable?
The general impression is that code is easier to read/write in Python, but I am not finding it very quick to read python code because of lack of types. What am I missing here?
I feel your pain, too! I frequently switch between Python and C++, so paradigm shifting does give me paranoia.
However, I've been readjusting my codes with:
Type Annotations
It doesn't improve runtime performance, but it provides sense of comfort when reading through tens of thousands line of codes. Also, you can run your python programs with this to further verify your type annotations:
mypy
These are the following things i follow:
Comment clearly what is being returned and what is the input in the docstring
Use a debug(or a Flag) variable, which is by default set to False, and keep a if block as follows.
if debug:
print(type(variable))
So, in that way, you would be sure to see what is the type of the variable.
In Python, you can see the data type of any variable by using
type(variable_name)
It will show you data type of that variable. Such as int, bool, str, etc.
Is there any way short of reading and parsing a file to see whether a function uses a for loop vs. recursion to get its answer? Assume it can be only one or the other, that the function doesn't raise, and that the function is guaranteed to end.
There is not general method to prove that a computer program finishes (it is proven that there are programs for which you cannot tell). Fortunately, in practise there are static analysis tools, that can do this for most of small programs.
Also, for most of Python functions it is either obvious what complexity they have (for an educated programmer) or it is mentioned in documentation what algorithm is used.
If you are interested in details, you should read a good book or two about algorithms. You can also take a look at https://cs.stackexchange.com/ , which is a site better suited for general discussion about computational complexity of different algorithms.
Under "basic" conditions, you can at least tell if a function will call itself directly:
def test(x):
return test(x)
test.__code__.co_name in test.__code__.co_names # true and co_name can't be changed
test.__name__ in test.__code__.co_names # true but name can be changed
This checks if the name of the function is in the global names referenced by the function.
I say "basic" because there are a bunch of ways someone can probably get around this. Lambdas don't have a name, so they won't match the global name they were bound to. A function could be renamed. The inner code could reference a different name for the function. The function could call a second function, which is the real recursive function. And on and on...
Basically, there is no way to know for sure that a function is recursive.
I'm a long time Python developer and I really love the dynamic nature of the language, but I wonder if Python would benefit from optional static typing.
Would it be beneficial to be able to apply static typing to the API of a library, and what would the disadvantages of this be?
I quickly sketched up a decorator implementing runtime-static type checking on pastebin and it works like this:
# A TypeError will be thrown if the argument "string" is not a "str" and if
# the returned value is not an "int"
#typed(int, string = str)
def getStringLength(string):
return len(string)
Would it be practical to use a decorator like this on the API-functions of a library? In my point of view type checking is not needed in the internal workings of a domain specific module of a library, but on the connection points between the library and it's client a simple version of design by contract by applying type checking could be useful. Especially as a type of enforced documentation which clearly states to the client of the library what it expects and returns.
Like this example where addObjectToQueue() and isObjectProcessed() are exposed for use by the client and processTheQueueAndDoAdvancedStuff() is an internal library function. I think type checking could be useful on the outward facing functions but would only bloat and restrict the dynamicness and usefulness of python if used on the internal functions.
# some_library_module.py
#typed(int, name = string)
def addObjectToQueue(name):
return random.randint() # Some object id
def processTheQueueAndDoAdvancedStuff(arg_of_library_specific_type)
# Function body here
#typed(bool, object_id = int)
def isObjectProcessed(object_id):
return True
What would the disadvantages of using this technique be?
What would the disadvantages of my naive implementation on pastebin be?
I don't want answers discussing the conversion of Python to a statically typed language, but thoughts about API design-specific pros/cons. (please move this to programmers.stackexchange.com if you consider it not a question)
Personally, I don't find this idea attractive for Python. This is all just my opinion, of course, but for context I'll tell you that Python and Haskell are probably my two favourite programming languages - I like languages at both extreme ends of the static vs dynamic typing spectrum.
I see the main benefits of static typing as follows:
Increased likelihood that your code is correct once the compiler has accepted it; if I know I've threaded my values through all the operations I invoked in such a way that the result type of one always matches the input type of another, and the final result type is the one I wanted, it increases the probability that I've selected the correct operations. This point is of deeply arguable value, since it only really matters if you're not testing very much, which would be bad. But it is true that, when programming in Haskell, when I sit back and say "there, done!" I am actually done a lot of the time, whereas that's almost never true of my Python code.
The compiler automatically points out most of the places that need changing when I make an incompatible change to a data structure or interface (most of the time). Again, tests are still needed to actually be sure you've caught all the implications, but most of the time the compiler's nagging is actually sufficient, in my experience, which deeply simplifies such refactoring; you can go straight from implementing the core of the refactoring to testing that the program still works okay, because the actual work of making all the flow-on changes is almost mechanical.
Efficient implementation. The compiler gets to use all the knowledge it has about types to do optimisation.
Your suggested system doesn't really provide any of these benefits.
Having written a program making use of your library, I still don't know if it contains any type-incorrect uses of your functions until I do extensive testing with full code coverage to see if any execution path contains a bad call.
When I refactor something, I need to go through many many rounds of "run full test suite, look for exception, find where it came from, fix the code" to get anything at all like a static-typing compiler's problem detection.
Python will still be behaving as if those variables could be anything at any time.
And to get even that much, you've sacrificed the flexibility of Python duck-typing; it's not enough that I provide a sufficiently "list-like" object, I have to actually provide a list.
To me, this sort of static typing is the worst of both worlds. The main dynamic typing argument is "you have to test your code anyway, so you may as well use those tests to catch type errors and free yourself from having to work around the type system when it doesn't help you". That may or may not be a good argument with respect to a really good static type system, but it absolutely is a compelling argument with respect to a weak partial static type system that only detects type errors at runtime. I don't think nicer error messages (which is all it really buys you most of the time; a type error not caught at the interface is almost certainly going to throw an exception deeper in the call stack) is worth the loss of flexibility.
I am looking for example where things in python would be easier to program just because it is dynamically typed?
I want to compare it with Haskell type system because its static typing doesn't get in the way like c# or java. Can I program in Haskell as I can in python without static typing being a hindrance?
PS: I am a python user and have played around little bit with ML and Haskell.. ... I hope it is clear now..
Can I program in Haskell as I can in python without static typing being a hindrance
Yes.
To elaborate, I would say the main gotcha will be the use of existential types in Haskell for heterogeneous data structures (regular data structures holding lists of variously typed elements). This often catches OO people used to a top "Object" type. It often catches Lisp/Scheme programmers. But I'm not sure it will matter to a Pythonista.
Try to write some Haskell, and come back when you get a confusing type error.
You should think of static typing as a benefit -- it checks a lot of things for you, and the more you lean on it, the less things you have to test for. In addition, it enables the compiler to make your code much faster.
Well for one you can't create a list containing multiple types of values without wrappers (like to get a list that may contain a string or an int, you'd have to create a list of Either Int String and wrap each item in a Left or a Right).
You also can't define a function that may return multiple types of values (like if someCondition then 1 else "this won't compile"), again, without using wrappers.
Like Chris said, this is one objective question (what can a dynamically typed language do that a statically typed one can't?) and one subjective question (can I use Haskell without static typing being a hindrance). So you're going to get mostly subjective answers, because the first question is not as interesting.
For me, the biggest hindrance was Haskell's IO type, because I had to stop and think about what code does I/O and what code doesn't, and explicitly pass information between the two. Everything else was pretty easy. If you commonly write
if someCondition:
return 1
else:
return "other"
Then you're making your own problems, Python just doesn't stop you from doing it. Haskell will, and that's about the only difference. The only exception is that this is sort of common in Python:
if someErrorCondition:
return None
else:
return NewItem(Success)
You can't do that in Haskell because there is no common None object. But there are easy ways to work around it.
I did find the type errors confusing at first, but I learned to read them in about a week.
I want to echo Don's advice: just try writing some Haskell and come back when you get a confusing type error.
I'm working on a macro system for Python (as discussed here) and one of the things I've been considering are units of measure. Although units of measure could be implemented without macros or via static macros (e.g. defining all your units ahead of time), I'm toying around with the idea of allowing syntax to be extended dynamically at runtime.
To do this, I'm considering using a sort of partial evaluation on the code at compile-time. If parsing fails for a given expression, due to a macro for its syntax not being available, the compiler halts evaluation of the function/block and generates the code it already has with a stub where the unknown expression is. When this stub is hit at runtime, the function is recompiled against the current macro set. If this compilation fails, a parse error would be thrown because execution can't continue. If the compilation succeeds, the new function replaces the old one and execution continues.
The biggest issue I see is that you can't find parse errors until the affected code is run. However, this wouldn't affect many cases, e.g. group operators like [], {}, (), and `` still need to be paired (requirement of my tokenizer/list parser), and top-level syntax like classes and functions wouldn't be affected since their "runtime" is really load time, where the syntax is evaluated and their objects are generated.
Aside from the implementation difficulty and the problem I described above, what problems are there with this idea?
Here are a few possible problems:
You may find it difficult to provide the user with helpful error messages in case of a problem. This seems likely, as any compilation-time syntax error could be just a syntax extension.
Performance hit.
I was trying to find some discussion of the pluses, minuses, and/or implementation of dynamic parsing in Perl 6, but I couldn't find anything appropriate. However, you may find this quote from Nicklaus Wirth (designer of Pascal and other languages) interesting:
The phantasies of computer scientists
in the 1960s knew no bounds. Spurned
by the success of automatic syntax
analysis and parser generation, some
proposed the idea of the flexible, or
at least extensible language. The
notion was that a program would be
preceded by syntactic rules which
would then guide the general parser
while parsing the subsequent program.
A step further: The syntax rules would
not only precede the program, but they
could be interspersed anywhere
throughout the text. For example, if
someone wished to use a particularly
fancy private form of for statement,
he could do so elegantly, even
specifying different variants for the
same concept in different sections of
the same program. The concept that
languages serve to communicate between
humans had been completely blended
out, as apparently everyone could now
define his own language on the fly.
The high hopes, however, were soon
damped by the difficulties encountered
when trying to specify, what these
private constructions should mean. As
a consequence, the intreaguing idea of
extensible languages faded away rather
quickly.
Edit: Here's Perl 6's Synopsis 6: Subroutines, unfortunately in markup form because I couldn't find an updated, formatted version; search within for "macro". Unfortunately, it's not too interesting, but you may find some things relevant, like Perl 6's one-pass parsing rule, or its syntax for abstract syntax trees. The approach Perl 6 takes is that a macro is a function that executes immediately after its arguments are parsed and returns either an AST or a string; Perl 6 continues parsing as if the source actually contained the return value. There is mention of generation of error messages, but they make it seem like if macros return ASTs, you can do alright.
Pushing this one step further, you could do "lazy" parsing and always only parse enough to evaluate the next statement. Like some kind of just-in-time parser. Then syntax errors could become normal runtime errors that just raise a normal Exception that could be handled by surrounding code:
def fun():
not implemented yet
try:
fun()
except:
pass
That would be an interesting effect, but if it's useful or desirable is a different question. Generally it's good to know about errors even if you don't call the code at the moment.
Macros would not be evaluated until control reaches them and naturally the parser would already know all previous definitions. Also the macro definition could maybe even use variables and data that the program has calculated so far (like adding some syntax for all elements in a previously calculated list). But this is probably a bad idea to start writing self-modifying programs for things that could usually be done as well directly in the language. This could get confusing...
In any case you should make sure to parse code only once, and if it is executed a second time use the already parsed expression, so that it doesn't lead to performance problems.
Here are some ideas from my master's thesis, which may or may not be helpful.
The thesis was about robust parsing of natural language.
The main idea: given a context-free grammar for a language, try to parse a given
text (or, in your case, a python program). If parsing failed, you will have a partially generated parse tree. Use the tree structure to suggest new grammar rules that will better cover the parsed text.
I could send you my thesis, but unless you read Hebrew this will probably not be useful.
In a nutshell:
I used a bottom-up chart parser. This type of parser generates edges for productions from the grammar. Each edge is marked with the part of the tree that was consumed. Each edge gets a score according to how close it was to full coverage, for example:
S -> NP . VP
Has a score of one half (We succeeded in covering the NP but not the VP).
The highest-scored edges suggest a new rule (such as X->NP).
In general, a chart parser is less efficient than a common LALR or LL parser (the types usually used for programming languages) - O(n^3) instead of O(n) complexity, but then again you are trying something more complicated than just parsing an existing language.
If you can do something with the idea, I can send you further details.
I believe looking at natural language parsers may give you some other ideas.
Another thing I've considered is making this the default behavior across the board, but allow languages (meaning a set of macros to parse a given language) to throw a parse error at compile-time. Python 2.5 in my system, for example, would do this.
Instead of the stub idea, simply recompile functions that couldn't be handled completely at compile-time when they're executed. This will also make self-modifying code easier, as you can modify the code and recompile it at runtime.
You'll probably need to delimit the bits of input text with unknown syntax, so that the rest of the syntax tree can be resolved, apart from some character sequences nodes which will be expanded later. Depending on your top level syntax, that may be fine.
You may find that the parsing algorithm and the lexer and the interface between them all need updating, which might rule out most compiler creation tools.
(The more usual approach is to use string constants for this purpose, which can be parsed to a little interpreter at run time).
I don't think your approach would work very well. Let's take a simple example written in pseudo-code:
define some syntax M1 with definition D1
if _whatever_:
define M1 to do D2
else:
define M1 to do D3
code that uses M1
So there is one example where, if you allow syntax redefinition at runtime, you have a problem (since by your approach the code that uses M1 would be compiled by definition D1). Note that verifying if syntax redefinition occurs is undecidable. An over-approximation could be computed by some kind of typing system or some other kind of static analysis, but Python is not well known for this :D.
Another thing that bothers me is that your solution does not 'feel' right. I find it evil to store source code you can't parse just because you may be able to parse it at runtime.
Another example that jumps to mind is this:
...function definition fun1 that calls fun2...
define M1 (at runtime)
use M1
...function definition for fun2
Technically, when you use M1, you cannot parse it, so you need to keep the rest of the program (including the function definition of fun2) in source code. When you run the entire program, you'll see a call to fun2 that you cannot call, even if it's defined.