Related
A lot of the languages I know of (like C++, C, Rust, etc.) print multiple error messages at a time. Then why does python print only one error message?
First off, I assume we are talking about syntax errors, i.e. those that can (and should) be detected and reported by the compiler.
It is primarily a design choice. Python is basically built on the notion that everything should be done during runtime. And the compiler is deliberately kept as simple as possible.
Simple and easy to understand, or complex and sophisticated:
Simply put, you have a choice between either using a very simple compiler that is easy to understand and maintain, or you have a complex piece of machinery with sophisticated program analysis and optimisations.
Languages like C, C++, and Rust take their strength from heavily optimising code during compilationg and thus took the second route with highly complex and extremely sophisticated compilers. Dealing with syntax errors is rather one of their less impressive feats.
Python, on the other hand, went to other way. Indeed, in general, it is quite impossible for a Python compiler to predict what exacty a piece of Python code is doing without actually running it—which precludes all the interesting opportunities for optimisation in the first place, and hence a sophisticated compiler would not really make sense, anyway. Keeping Python's compiler simple and rather focus on runtime optimisations is thus the right choice. But it comes with the downside that the compiler simply bails out whenever it discovers an error.
To give a bit more context...
1. Error Recovery
Dealing with errors and recover from a syntax error in a compiler is hard.
Compilers are usually very good at translating (syntactically) correct programs fast and efficiently to machine code that represents the original program. However, if there is a syntax error, it is often impossible for the compiler to guess the original intention of the programmer, and it is therefore not clear what to do with an erroneous piece of code.
Here is a very simple example:
pen color("red")
Obviously, there is something wrong here, but without further context it is impossible to tell, whether the original intent of this line was pen = color("red"), pencolor("red"), pen.color("red") or something else entirely.
If a compiler wants to continue with looking at the rest of the program (and thus discover potentially more syntax errors), it needs a stretegy of how to cope with such situations and recover so as to move on: it needs an error recovery strategy. This might something as simple as just skipping the entire line or individual tokens, but there is no clear-cut "correct" solution to this.
2. Python's Compiler
Python compiles your program one symbol at a time.
Python's current compiler works by looking at one symbol at a time (called a LL(1) compiler). This makes it extremely simple to automatically build the compiler for Python, and it is quite fast and efficient. But it means that there are situations where, despite an "obvious" syntax error, Python happily moves on compiling the program until it is really lost.
Take a look at this example:
x = foo(
y = bar()
if x > y:
As humans, we quickly see the missing closing parenthesis in line 1. However, from the compiler's perspective, this looks rather like a call with a named argument, something like this:
x = foo(y = bar() if x > y else 0)
Accordingly, Python will only notice that something is wrong when it hits the colon in line 3—the first symbol that does not work with its "assumption". But at that point, it is extremely hard to figure out what to do with this piece of code, and how to correctly recover: do you just skip the colon in this case? Or should you go back and correct something earlier on—and if so, how far back are you going?
3. Follow-up Errors
Error recovery can create "ghost" errors.
In the first example above, the compiler could just skip the entire line and move on without any issue. But there are situations, where the choice of how to recover from a syntax errors influences (potentially) everything that follows, as in this example:
deffoo(x):
The intent behind this could either be def foo(x): or simply a call deffoo(x). But this distinction determines how the compiler will look at the code that follows, and either report an indentation error, or perhaps a return outside a function, etc.
The danger of error recovery is that the compiler's guess might actually be wrong, which could lead to a whole series of reported follow-up errors—which might not even be true errors, but rather ghosts created by the compiler's wrong decision.
Bottom-line: getting error recovery and error reporting right is extremely hard. Python's choice to only report the first syntax error it encounters is thus sensible and works for most users and situations just fine.
I have actually written a parser with more sophisticated error detection, which can list all errors it discovers in a Python program. But to my experience, too many of the additional errors beyond the first one are just rubbish, and I therefore always stuck to displaying only the first error in a program.
Difficult to answer the exact why. I cannot look into the head of the developers of the C-python, jython, pypy or others.
Many errors will only be seen at run time as python is an interpreted language without strict typing.
However each file is compiled to byte code if there is no syntax error.
So for syntax errors I cannot give you the reason, as it should have been technically possible. However I never had issues with this as I use tools like pylint or flake8 to check the code for me.
These tools can detect multiple errors and give a lot of warnings about coding style as well.
So I cannot tell you the why, but only what to do to get multiple errors in one go.
These tools can be configured to just display certain kinds of errors.
to install one or the other tool, just type:
pip install flake8 or pip install pylint
Then type just flake8 or pylint in the directory where all your code is located
or type flake8 <filename> or pylint <filename> to check just one file.
Please note, that many IDEs like for example Microsoft Visual Studio Code, Pycharm, and others can be configured to run these tools automatically for you and signal any issues even before you execute your code.
C or C++ are use compiler to compiled and then execute but python is an interpreted language it means that the interpreter read each line and then execute it when the interpreter see error in on line of program it stopped and display the error in other interpreted language like JS it is the same.
I hope your problem solved but if you want to read more you can google "Interpreted and compiled languages" or see this
I have to write a daemon program that constantly runs in the background and performs some simple tasks. The logic is not complicated at all, however it has to run for extended periods of time and be stable.
I think C++ would be a good choice for writing this kind of application, however I'm also considering Python since it's easier to write and test something quickly in it.
The problem that I have with Python is that I'm not sure how its runtime environment is going to behave over extended periods of time. Can it eat up more and more memory because of some GC quirks? Can it crash unexpectedly? I've never written daemons in Python before, so if anyone here did, please share your experience. Thanks!
I've written a number of daemons in Python for my last company. The short answer is, it works just fine. As long as the code itself doesn't have some huge memory bomb, I've never seen any gradual degradation or memory hogging. Be mindful of anything in the global or class scopes, because they'll live on, so use del more liberally than you might normally. Otherwise, like I said, no issues I can personally report.
And in case you're wondering, they ran for months and months (let's say 6 months usually) between routine reboots with zero problems.
Yes it can leak. Yes it can crash unexpectedly. Anything can.
I'd say you're far more likely to end up accidentally leaking in an environment with manual memory management (e.g. C++) than you are with something like Python.
As for crashing unexpectedly, well, chances are an arbitrary lump of Python might be more likely to crash unexpectedly than an arbitrary lump of Java, because the latter benefits from static typing where you can catch a whole load of errors at compile time, that Python with its duck typing and other forms of flexibility.
Realistically, Python sounds a perfectly reasonable choice for what you want to do. Take a look at something like Twisted for a decent engine to build things around, or at least for an idea of structure (your question sounds like some sort of school assignment, so I'm not sure how much freedom of implementation you get)
I've written many things in C/C++ and Perl that are initiated when a LINUX box O.S. boots, launching them using the rc.d.
Also I've written a couple of java and python scripts that are started the same way I've mentioned above, but I needed a little shell-script (.sh file) to launch them and I used rc.5.
Let me tell you that your concerns about their runtime environments are completely valid, you will have to be careful about wich runlevel you'll use... (only from rc.2 to rc.5, because rc.1 and rc.6 are for the System).
If the runlevel is too low, the python runtime might not be up at the time you are launching your program and it could flop. e.g.: In a LAMP Server MySQL and Apache are started in rc.3 where the Network is already available.
I think your best shot is to make your script in python and launch it using a .sh file from rc.5.
Good luck!
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 12 years ago.
I have heard before that many Python developers don't use an IDE like Eclipse because it is unnecessary with a language like Python.
What are the reasons people use to justify this claim?
I'd say the main reason is because Python isn't horribly verbose like, e.g., Java. You don't need an IDE to generate 100s of lines of boilerplate because you don't need 100s of lines of boilerplate in Python. You tend to automate stuff within the language instead of further up the toolchain.
A second reason is that you don't need build process automation b/c there's no build process.
I'm going to risk offending some people and express something that I think a lot of python lovers will agree with: Java is so bloody cumbersome and verbose that one almost needs an IDE like Eclipse just to manage its unwieldy bloat.
With python, the main programming-specific features I want from my editor are syntax highlighting and a jump-to-definition command. Bonus points for a complementary return-from-jump command.
I find Geany does what I need, and is refreshingly light, quick, and stable compared to monster IDEs like Eclipse. For other suggestions, take a look at this question.
I know why you need (can benefit from) a good IDE - Rapid Application Development
Time is money :) And I'd much rather spend my time solving problems than typing every little piece of code in.
Two Main Reasons
Because of the dynamic typing and generally super-powerful functionality, there just isn't much extra typing that the IDE can do for you. Implementing a Java interface in a package is a lot of work and dozens of lines of boilerplate. In Python or Ruby it just doesn't have to be typed in the first place.
Because of the dynamic typing, the fancy editor doesn't have nearly as much information at its fingertips, and so the capability is reduced as well.
So the squeeze is top-down from the reduced need and bottom-up from the reduced editor capability, the net result being a really small application area. The benefit of using a fast and familiar day-to-day editor ends up higher than the benefit of the mostly-pointless IDE.
I suppose it's also possible that the categories are a bit fuzzy now. Vi(1) is the lightest-weight and fastest "plain" editor around, and yet vim(1) can now colorize every language under the sun and TextMate is classified as "lightweight", so all of the categories have really begun to merge a bit.
Python is dynamically typed and the way it handles modules as objects makes it impossible to determine what a name will resolve to at a certain time without actually running the code. Therefore, the 'tab completion' feature of IDEs is pretty useless.
Also, since Python doesn't have a build step, an IDE isn't needed to automate this. You can just fire up python app.py in a terminal and have much more control over how it runs.
It sounds like 'You don't need vehicle to go to work' to me. It might be true or not, depends on how far your workplace is.
IDE's assist in developer productivity and can equally apply to Python. The defining thing about an IDE is the ability to not have to "mode switch" between tasks such as editing, compiling, testing, running and debugging etc.
Python uses dynamic typing and interpreting, rather than compiling.
The interpreter itself will output comprehensive error messages, similar to Perl.
If you look at dynamically typed programming languages in general, you'll find that most of them are not really suitable for IDEs. RAD components (code completion, code generation, code templates, etc) can be included in almost any smart text editor, like Vim, Emacs, Gedit, or SciTE.
I use Vim and Gedit for most of my programming, and I find, that I don't need IDE-ish stuff other than that, what is already included in those text editors. When I program in Java, however, I use Eclipse most of the time, since keeping track of all those parts manually, would be too time consuming. I tend not to use IDE's for my C++ stuff, too, but when the projects grows beyond a certain size, I tend to use either Eclipse (CDT), NetBeans, Code::Blocks, or something like that.
So it's the family of languages itself, that make IDE's unnecessary, but it doesn't mean that working with IDE's with those languages, is bad practice.
Side Note: There's even a Lua environment for Eclipse. Out of all languages, Lua is probably one of the least that needs an IDE...
Well I use an IDE when programming in Python on my computer. Its easier that way . But when on the run or on university's terminal , I prefer terminal .
I'm still fairly new to Python and use an IDE with code completion but find myself rarely needing it, Python does a really good job of not having an uncessarily large number of verbose calls, as dsimcha pointed out above. I find that just using a basic IDE I can work efficiently in it and the fact that the code is a lot less cluttered without having brackets makes it easier to work with files that have a lot of lines of code (something that I found unbearable in PHP due to all its syntax clutter)
As far as #Postman's answer, I'm not sure that having an IDE makes RAD any faster, at least not in the case of python, its such a succinct language, the only thing that it would help in would be code completion, the way you answered it it sounds more like you are hinting at the use of a framework, which I believe is still very important in Python which does make RAD much more possible than otherwise.
The problems is IDEs dont work very well with dynamic languages.
The IDE cannot second guess runtime duck typing so other than some basic syntax checking and displaying the keywords in pretty colours they ar enot much help.
My personal experience is with groovy and eclipse where eclipse is actually pretty annoying. Method completion for a groovy object brings up about 200 posabilties, it constantly insert quotes and brackets exactly where you dont want them and messes up the syntax coloring whenever it encounters a reasonably complex regular expression. I would ditch eclipse except the majority of the code base is in Java where eclipse is useful.
I am basically from the world of C language programming, now delving into the world of scripting languages like Ruby and Python.
I am wondering how to do debugging.
At present the steps I follow is,
I complete a large script,
Comment everything but the portion I
want to check
Execute the script
Though it works, I am not able to debug like how I would do in, say, a VC++ environment or something like that.
My question is, is there any better way of debugging?
Note: I guess it may be a repeated question, if so, please point me to the answer.
Your sequence seems entirely backwards to me. Here's how I do it:
I write a test for the functionality I want.
I start writing the script, executing bits and verifying test results.
I review what I'd done to document and publish.
Specifically, I execute before I complete. It's way too late by then.
There are debuggers, of course, but with good tests and good design, I've almost never needed one.
Here's a screencast on ruby debugging with ruby-debug.
Seems like the problem here is that your environment (Visual Studio) doesn't support these languages, not that these languages don't support debuggers in general.
Perl, Python, and Ruby all have fully-featured debuggers; you can find other IDEs that help you, too. For Ruby, there's RubyMine; for Perl, there's Komodo. And that's just off the top of my head.
There is a nice gentle introduction to the Python debugger here
If you're working with Python then you can find a list of debugging tools here to which I just want to add Eclipse with the Pydev extension, which makes working with breakpoints etc. also very simple.
My question is, is there any better way of debugging?"
Yes.
Your approach, "1. I complete a large script, 2. Comment everything but the portion I want to check, 3. Execute the script" is not really the best way to write any software in any language (sorry, but that's the truth.)
Do not write a large anything. Ever.
Do this.
Decompose your problem into classes of objects.
For each class, write the class by
2a. Outline the class, focus on the external interface, not the implementation details.
2b. Write tests to prove that interface works.
2c. Run the tests. They'll fail, since you only outlined the class.
2d. Fix the class until it passes the test.
2e. At some points, you'll realize your class designs aren't optimal. Refactor your design, assuring your tests still pass.
Now, write your final script. It should be short. All the classes have already been tested.
3a. Outline the script. Indeed, you can usually write the script.
3b. Write some test cases that prove the script works.
3c. Runt the tests. They may pass. You're done.
3d. If the tests don't pass, fix things until they do.
Write many small things. It works out much better in the long run that writing a large thing and commenting parts of it out.
Script languages have no differences compared with other languages in the sense that you still have to break your problems into manageable pieces -- that is, functions. So, instead of testing the whole script after finishing the whole script, I prefer to test those small functions before integrating them. TDD always helps.
There's a SO question on Ruby IDEs here - and searching for "ruby IDE" offers more.
I complete a large script
That's what caught my eye: "complete", to me, means "done", "finished", "released". Whether or not you write tests before writing the functions that pass them, or whether or not you write tests at all (and I recommend that you do) you should not be writing code that can't be run (which is a test in itself) until it's become large. Ruby and Python offer a multitude of ways to write small, individually-testable (or executable) pieces of code, so that you don't have to wait for (?) days before you can run the thing.
I'm building a (Ruby) database translation/transformation script at the moment - it's up to about 1000 lines and still not done. I seldom go more than 5 minutes without running it, or at least running the part on which I'm working. When it breaks (I'm not perfect, it breaks a lot ;-p) I know where the problem must be - in the code I wrote in the last 5 minutes. Progress is pretty fast.
I'm not asserting that IDEs/debuggers have no place: some problems don't surface until a large body of code is released: it can be really useful on occasion to drop the whole thing into a debugging environment to find out what is going on. When third-party libraries and frameworks are involved it can be extremely useful to debug into their code to locate problems (which are usually - but not always - related to faulty understanding of the library function).
You can debug your Python scripts using the included pdb module. If you want a visual debugger, you can download winpdb - don't be put off by that "win" prefix, winpdb is cross-platform.
The debugging method you described is perfect for a static language like C++, but given that the language is so different, the coding methods are similarly different. One of the big very important things in a dynamic language such as Python or Ruby is the interactive toplevel (what you get by typing, say python on the command line). This means that running a part of your program is very easy.
Even if you've written a large program before testing (which is a bad idea), it is hopefully separated into many functions. So, open up your interactive toplevel, do an import thing (for whatever thing happens to be) and then you can easily start testing your functions one by one, just calling them on the toplevel.
Of course, for a more mature project, you probably want to write out an actual test suite, and most languages have a method to do that (in Python, this is doctest and nose, don't know about other languages). At first, though, when you're writing something not particularly formal, just remember a few simple rules of debugging dynamic languages:
Start small. Don't write large programs and test them. Test each function as you write it, at least cursorily.
Use the toplevel. Running small pieces of code in a language like Python is extremely lightweight: fire up the toplevel and run it. Compare with writing a complete program and the compile-running it in, say, C++. Use that fact that you can quickly change the correctness of any function.
Debuggers are handy. But often, so are print statements. If you're only running a single function, debugging with print statements isn't that inconvenient, and also frees you from dragging along an IDE.
There's a lot of good advice here, i recommend going through some best practices:
http://github.com/edgecase/ruby_koans
http://blog.rubybestpractices.com/
http://on-ruby.blogspot.com/2009/01/ruby-best-practices-mini-interview-2.html
(and read Greg Brown's book, it's superb)
You talk about large scripts. A lot of my workflow is working out logic in irb or the python shell, then capturing them into a cascade of small, single-task focused methods, with appropriate tests (not 100% coverage, more focus on edge and corner cases).
http://binstock.blogspot.com/2008/04/perfecting-oos-small-classes-and-short.html
Since Python is a dynamic, interpreted language you don't have to compile your code before running it. Hence, it's very easy to simply write your code, run it, see what problems occur, and fix them. Using hotkeys or macros can make this incredibly quick.
So, because it's so easy to immediately see the output of your program and any errors that may occur, I haven't uses a debugger tool yet. What situations may call for using a real debugger vs. the method I currently use?
I'd like to know before I get into a situation and get frustrated because I don't know how to fix the problem.
In 30 years of programming I've used a debugger exactly 4 times. All four times were to read the core file produced from a C program crashing to locate the traceback information that's buried in there.
I don't think debuggers help much, even in compiled languages. Many people like debuggers, there are some reasons for using them, I'm sure, or people wouldn't lavish such love and care on them.
Here's the point -- software is knowledge capture.
Yes, it does have to run. More importantly, however, software has meaning.
This is not an indictment of your use of a debugger. However, I find that the folks who rely on debugging will sometimes produce really odd-looking code and won't have a good justification for what it means. They can only say "it may be a hack, but it works."
My suggestion on debuggers is "don't bother".
"But, what if I'm totally stumped?" you ask, "should I learn the debugger then?" Totally stumped by what? The language? Python's too simple for utter befuddlement. Some library? Perhaps.
Here's what you do -- with or without a debugger.
You have the source, read it.
You write small tests to exercise the library. Using the interactive shell, if possible. [All the really good libraries seem to show their features using the interactive Python mode -- I strive for this level of tight, clear simplicity.]
You have the source, add print functions.
I use pdb for basic python debugging. Some of the situations I use it are:
When you have a loop iterating over 100,000 entries and want to break at a specific point, it becomes really helpful.(conditional breaks)
Trace the control flow of someone else's code.
Its always better to use a debugger than litter the code with prints.
Normally there can be more than one point of failures resulting in a bug, all are not obvious in the first look. So you look for obvious places, if nothing is wrong there, you move ahead and add some more prints.. debugger can save you time here, you dont need to add the print and run again.
Usually when the error is buried in some function, but I don't know exactly what or where. Either I insert dozens of log.debug() calls and then have to take them back out, or just put in:
import pdb
pdb.set_trace ()
and then run the program. The debugger will launch when it reaches that point and give me a full REPL to poke around in.
Any time you want to inspect the contents of variables that may have caused the error. The only way you can do that is to stop execution and take a look at the stack.
pydev in Eclipse is a pretty good IDE if you are looking for one.
I find it very useful to drop into a debugger in a failing test case.
I add import pdb; pdb.set_trace() just before the failure point of the test. The test runs, building up a potentially quite large context (e.g. importing a database fixture or constructing an HTTP request). When the test reaches the pdb.set_trace() line, it drops into the interactive debugger and I can inspect the context in which the failure occurs with the usual pdb commands looking for clues as to the cause.
You might want to take a look at this other SO post:
Why is debugging better in an IDE?
It's directly relevant to what you're asking about.