If I type-annotate my code properly, and use something like Pylance in my IDE, when I hover over a method or function I get helpful information about that code's 'signature': the variables it expects, their types, and what I can expect from that code in terms of a response. All that is great, and I have come to rely on it in my daily coding activities.
Anyway, I was reviewing some old code of mine, and was trying to make sense of it after 18+ months (I'm sure I'm alone in that, right? lol). And although I had commented the code in question back then, looking at it now I found the comments to actually clutter things up, instead of being very helpful.
So it got me to thinking: what if there were some sort of plugin or code-assistant library that, when I hovered over a section of code, not only gave me the code's signature/type annotation info, but also gave me the documentation for that particular section? I could potentially make the comments much longer and clearer without having to worry about code bloat, or its visual impact.
tldr;
is there a way to make my comments be metadata instead of having to be embedded in the code itself?
Related
Edit #1:
Thanks to the keen insights of #Chaos_is_harmony a bit of deprecated Tensorflow syntax has been discovered:
net.get_shape()
whereas it is now
net.ref()
While this resolves the question that is the topic of this post, it does prompt a new, perhaps related error:
'Reference' object is not iterable
Original post:
Hello hello StackOverflow,
I'm an amateur musician who's not particularly savvy at programming (took a couple Python courses at Uni over a decade ago and haven't gone any further, let alone engaged with it, since), but returned to it today, and have been trying all afternoon, to run Dmitri Ulyanov's "Neural Style Audio Transfer" in order to generate and iterate samples from pre-existing bits of sound design I've done, or to convolve what I've done with what others have done in styles I'm less proficient at. It seemed more or less usable without much coding so I thought appropriate to try. In any case, Dmitri's project's details may be found here: https://github.com/DmitryUlyanov/neural-style-audio-tf
That is what I'm kinda good at, not programming nor assessing code, so I must apologize for my ignorance in advance, both in my description of the issue and my ability to track/implement your responses should I receive any. Nonetheless, there were a few extant issues with running the notebook I was able to resolve by importing "tensorflow.compat.v1 as tf". But it did not fix all the issues.
I am stuck at one error in the "Optimize" portion of the notebook:
AttributeError: 'int' object has no attribute 'value'
Here's how it looks in context (I'm not sure which part of the code is relevant so as to copy it directly in here, I'm sorry...):
https://imgur.com/a/CUMynHM
Here's a video of the notebook being run, cell by cell if that's helpful:
https://streamable.com/tqzmwf
I'm not sure what else might be helpful to mention other than I'm using Python 3 as the kernel (as indicated in the video.) Hopefully I've not given you all too little to go on here, as again, I've sort of jumped into the deep end of things I really know almost nothing about. If it were more than just this one error I would have let it go and not bothered you all; but I also thought it prudent to ask this in posterity for other amateur musicians who may need some guidance in how to use ML tools for music creation and might thus come across this post in the future. All silly justification that aside, let me know if there's anything more I can detail!
Much obliged~!
According to the Tensforflow docs for get_shape(), it says it is a deprecated method and should instead be replaced with ref().
How can I get Geany to autocomplete an object's constraints?
For example, I type:
self.window.set_position(gtk.WIN_
And I want the list of possible constraints to show up such as WIN_POS_NONE and WIN_POS_CENTER etc.
NOTE: CTRL+SPACE or CTRL+SHIFT+SPACE does not show constraints.
Autocompletion works fine for functions and symbols, just not constraints, unless I've used it once already before. This saves me the time of looking at documentation. Sometimes I can partially remember the constraint, and it would be nice to be able to browse the options.
I would basically like it to work like it does in Sublime Text, which is a near-perfect editor for me, but I'm looking for something free/opensource to use.
EDIT: I've also tried Ninja-IDE which can also display constraints, but it locks up sometimes and is not as lightweight as Geany...
EDIT 2: I'm not looking for an alternative to Geany, I'm looking to make this functionality work via a mod or plug-in.
I don't think Geany is capable of doing this. But PyCharm is a lot better IDE for Python. It knows everything about your code, and also has an intelligent code completion, on-the-fly error checking and quick-fixes, easy project navigation, and much more.
There is also a free version ready to download which contains more than enough functionality a common programmer may need.
From what I have been able to find, this is beyond what Geany can do. I asked how to get Geany to do this and I am not looking for any alternatives to Geany, nor am I interested in using anything else. Therefore, this is the accepted answer, unless someone posts a way to make it work in Geany, at that point I will change the accepted answer.
How easy is it to reverse engineer an auto-generated C code? I am working on a Python project and as part of my work, am using Cython to compile the code for speedup purposes.
This does help in terms of speed, yet, I am concerned that where I work, some people would try to "peek" into the code and figure out what it does.
Cython code is basically an auto-generated C. Is it very hard to reverse engineer it?
Are there any recommendations that would make the code safer and reverse-engineering harder to do? (I assume that with enough effort, everything can be reversed engineered).
Okay -- to attempt to answer your question more directly: most auto-generated C code is fairly ugly, so somebody would need to be fairly motivated to reverse engineer it. At the same time, I don't believe I've never looked at what Cython generates, so I'm not sure how it looks.
In addition, a lot of auto-generated code is done in the form of things like state machine tables, that most programmers find fairly difficult to follow even at best. The tendency (in many cases) is to have a generic framework, with tables of data that the framework more or less "interprets" at run-time. This isn't necessarily impossible to follow, but it's enough different from most typical code that most people will give up on it fairly quickly (and if they do much, they'll typically waste a lot of time looking at the framework instead of the data, which is what really matters in cases like this).
I'll repeat, however, that I'm pretty sure I haven't looked at what Cython produces, so I can't say much about it with any real certainty.
There are (or at least used to be) commercial obfuscators intended to make C source code difficult to understand. I suspect the availability of Perl has taken a lot of the market share from them, but if you look you may still be able to find one and use it.
Absent that, it's not terribly difficult to write an obfuscator of your own, but the degree of effectiveness will probably vary with the amount of effort you're willing to put into it. Just systematically renaming any meaningful variable names into things like _ and __ can do quite a bit (e.g., profit = sales - costs; is a lot more meaningful than _ = _I_ - _i_;). Depending on the machine generated code in question, however, this may not really accomplish much -- obfuscating a generic framework may not make much difference in understanding what your code does -- and if they figure out the procedure you're following, they may be able to simply replicate the correct framework code and transplant the pieces specific to your program into the un-obfuscated framework.
You should really take a look at the code that Cython produces. To help with debugging, for example, it copies the complete Python source code into the generated file, marking each source line before generating C code for it. That makes it very easy to find the code section that you are interested in.
A very nice feature is that you can compile your code with the "-a" (annotate) option, and it will spit out an HTML file next to the C file that contains the annotated Python code. When you click on a line, you'll see the C code for that line. As a bonus, it marks lines that do a lot of Python processing in dark yellow, so that you get a simple indicator where to look for potential optimisations.
There's also special gdb support in Cython now, so you can do Cython source level debugging etc.
Ah, I guess I missed the bit that you were talking about the compiled module, whereas I was only referring to the source code that Cython generates. I agree with Jerry that it will be fairly tricky to extract something useful from the compiled module, as long as you keep the gdb support disabled (the default) and strip the debugging symbols. That is because the C compiler will do lots of inlining of helper functions all over the place and apply various low-level code optimisations, thus making it harder to extract the original macro level code patterns. However, you will see named C-API calls to CPython, and you will also see function names from your own code. Cython isn't specifically designed for code obfuscation, quite the opposite. But readable assembly has certainly never been a design goal.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have a source of python program which doesn't have any documentation or comments.
I did tried twice to understand it but most of the times I am losing my track, because there are many files.
What should be the steps to understand that program fully and quickly.
Michael Feathers' "Working Effectively with Legacy Code" is a superb starting point for such endeavors -- not particularly language-dependent (his examples are in several non-python languages, but the techniques and mindset DO extend pretty well to Python and just about any other language).
The key focus is, that you want to understand the code for a reason -- modifying it and/or porting it. So, instrumenting the legacy code -- with batteries and scaffolding of tests and tracing/logging -- is the crucial path on the long, hard slog to understanding and modifying safely and responsibly.
Feathers suggests heuristics and techniques for where to focus your efforts and how to get started when the code is a total mess (hence "legacy") - no docs, or misleading docs (describing something quite different, maybe in subtle ways, from what the code actually DOES), no tests, an untestable-without-refactoring tangle of spaghetti dependencies. This may seem an extreme case but anybody who's spent a long-ish career in programming knows it's actually more common than anyone would like;-).
In past I have used 'Python call graph' to understand the source structure
Use a debugger e.g. pdb to wak thru
the code.
Try to read code again after one day
break, that also helps
I would recommend to generate some documentation with epydoc http://epydoc.sourceforge.net/ . For sure, if no docstring exists, the result will be poor but it will give you at least one view of your application and you'lle be able to navigate in the classes more easily.
Then you can try to document by yourself when you understand something new and then regenerate the docs again. It is never too late to start something.
I hope it helps
You are lucky it's in Python which is easy to read. But it is of course possible to write tricky hard to understand code in Python as well.
The steps are:
Run the software and learn to use it, and understand it's features at least a little bit.
Read though the tests, if any.
Read through the code.
When you encounter code you don't understand, put a debug break there, and step through the code, looking at what it does.
If there aren't any tests, or the test coverage is low, write tests to increase the test coverage. It's a good way to learn the system.
Repeat until you feel you have a vague grip on the code. A vague grip is all you need if you are going to manage the code. You'll get a good grip once you start actually working with the code. For a big system that can take years, so don't try to understand it all first.
There are tools that can help you. As Stephen C says, an IDE is a good idea. I'll explain why:
Many editors analyses the code. This typically gives you code completion, but more importantly in this case, it makes it possible to just just ctrl-click on a variable to see where it comes from. This really speeds things up when you want to understand otehr peoples code.
Also, you need to learn a debugger. You will, in tricky parts of the code, have to step through them in a debugger to see what the code actually do. Pythons pdb works, but many IDE's have integrated debuggers, which make debugging easier.
That's it. Good luck.
I have had to do a lot of this in my job. What works for me may be different to what works for you, but I'll share my experience.
I start by trying to identify the data structures being used and draw diagrams showing the relationships between them. Not necessarily something formal like UML, but a sketch on paper which you understand which allows you to see the overall structure of the data being manipulated by the program. Only once I have some view of the data structures being used do I start to try to understand how the data is being manipulated.
Secondly, for a large body of software, sometimes you need to just attack bite sized pieces at first. You won't get an overall understanding straight away, but if you understand small parts in detail and keep chipping away, eventually all the pieces fall together.
I combine these two approaches, switching between them when I am getting overly frustrated or bored. Regular walks around the block are recommended :) I find this gets me good results in the end.
Good luck!
pyreverse from Logilab and PyNSource from Andy Bulka are helpful too for UML diagram generation.
I'd start with a good python IDE. See the answers for this question.
Enterprise Architect by Sparx Systems is very good at processing a source directory and generating class diagrams. It is not free, but very reasonably priced for what you get. (I am not associated with this company in any way, I've just been a satisfied user of their product for several years.)
I'm currently working on a simple wxPython app that's essentially document based. So far I've been manually implementing the usual open/save/undo/redo etc etc stuff.
It occurred to me that wxPython must have something to help me out and after a bit of searching revealed the docview package.
At this point though I'm just not quite sure how to hook everything up and get things started. Anyone got any good links or hints about places to start?
The docs seems to be a little thin about this and Robin Dunn's wxPython book doesn't really cover this package at all.
You might take a look at the docviewdemo.py from the wxPython Docs and Demos:
on my machine they are located:
C:\Program Files\wxPython2.8 Docs and Demos\samples\pydocview\
C:\Program Files\wxPython2.8 Docs and Demos\samples\docview\
In addition to the ones mentioned, there is quite an extensive example docview/pydocview in the samples\ide. If you want it to run you will have to make a few code corrections (I have submitted a ticket that outlines the fixes at trac.wxwidgets.org #11237). It is pretty complex but I found it handy to figure out how to do some more complex things. For example, samples\ide\activegrid\tools\ProjectEditor.py is built from scratch and has undo support etc rather than just relying on a control that does everything for you already. That way you can see how things are supposed to be done at the detailed level. The documentation is rather useless in that regard.
If you have decided against using docview/pydocview I have a spreadsheet application built on wxPython that you may find useful as an example. While it does not implement a document view framework it does have some characteristics of it and I've implemented an undo/redo system. Check it out at http://www.missioncognition.net/pysheet/ I'm currently working on a pydocview based app so I expect that to be up on my site eventually.