Long story short, a piece of code that I'm working with at work has the line:
from System import System
with a later bit of code of:
desc_ = System()
xmlParser = Parser(desc_.getDocument())
# xmlParser.setEntityBase(self.dtdBase)
for featureXMLfile in featureXmlList.split(","):
print featureXMLfile
xmlParser.parse(featureXMLfile)
feat = desc_.get(featureName)
return feat
Parser is an XML parser in Java (it's included in a different import), but I don't get what the desc_ bit is doing. I mean obviously, it somehow holds the feature that we're trying to pull out, but I don't entirely see where. Is System a standard library in Python or Java, or am I looking at something custom?
Unfortunately, everyone else in my group is out for Christmas Eve vacation, so I can't ask them directly. Thank you for your help. I'm still not horribly familiar with Python.
This isn't from the standard library, so you'll need to check your system (Python has plenty of introspection to help you with that).
You can tell as Python modules in the standard library use lowercase names as per PEP-8, or by searching the library reference.
Note as well that Python has it's own XML parsing tools that will be much nicer to work with in Python than Java's.
Edit: As you have noted in the comments you are using Jython, it seems likely this is Java's System package.
millimoose indicated the correct answer in his comment, but neglected to submit it as an answer, so I'm posting to indicate the correct answer. It was indeed a custom module built by my company. I was able to determine this by typing import System; print(System) into the interpreter.
Related
Struggling to find a Python library of script to tokenize (find specific tokens like function definition names, variable names, keywords etc.).
I have managed to find keywords, whitespaces etc. using something like this but I found it quite a challenge for function/class definition names etc. I was hoping of using a pre-existent script; I explored Pygments with no success. Its lexer seems amazing for what I want but have no idea how to utilize it in Python and to also get positions for each found token.
For example I am looking at doing something like that:
int fac(int n)
{
return (n>1) ? n∗fac(n−1) : 1;
}
from the source code above I would like to get:
function_name: 'fac' at position (x, y)
variable_name: 'n' at position (x, y+8)
EDITED:
Any suggestions will be appreciated since I am in the dark here regarding tokenizations and parsing in C++?
Eli Bendersky is a smart guy, and sometimes active here on SO. He's got a blog post on this issue which I'll refer you directly to: Parsing C++ in Python with Clang.
Because things disappear, here's the takeaway:
Eli Bendersky wrote a C language (not C++) parser in Python, called pycparser. People keep asking him if he's going to add support for C++. He is not. He recommends instead that people use the Python bindings for libclang to get access to "a C API that the Clang team vows to keep relatively stable, allowing the user to examine parsed code at the level of an abstract syntax tree (AST)".
You can find the bindings separately on PyPI here. Note though that you'll have to have clang installed, so you may just want to point your PYTHON_PATH directly at the install location.
You're struggling to find a python library to do what you want because what you want is impossible to do, fundamentally.
I have managed to find keywords, whitespaces etc. using something like this but I found it quite a challenge for function/class definition names etc
You mean like this:
foo = 3
def foo():pass
What is foo? All a tokenizer should/can tell you is that foo is an identifier. It's context tells you whether it's a variable or a function declaration. You need a parser to handle context free grammars. Mathematically, the space of context free grammars is too large for a standard lexer to tackle.
Try a parser: here's one in python
Normally I'd try and provide you links here to distinguish between the topics, but this is too broad to provide a single good link to. If you're interested, start with any standard compiler text. Elsewhere on SE, we see this question pop up as a theoretical question and, in some form, as a famous question about html.
Once you realize that tokenizers are (usually) built (largely) on regular expressions, it becomes more obvious why your task is not going to end happily.
Now that you know the terminology, I think you'll find this SO article useful, which recommends gcc-ml. I don't know how up-to-date it is, but it's the type of program you're looking for.
Greetings!
So contrary to my last question I think I was pursuing the wrong method. Basically I need to be able to compile a Python script without fear of any common leecher to obtain my source code and begin usage of it themselves. Now I know Python is a open source language not meant for being protected but there must be a way?
For example Py2Exe asks you to leave your source code in a plain text file for it then to be compiled. Which is exactly not what I'm wanting. The whole reason for this is that compilation will be happening not just on my own PC. My project is going to be compiled to PE executable form and is intended for public use and I can't have them simply opening and viewing my source.
Can anyone aid me?
You can search for a Python obfuscator or scrumbler but there will be no solution that will protect you intellectual property anyway. If you have developed something completely new go for a patent and try to control it.
In the meantime continue reading at How do I protect python code?.
There's no way to protect anything completely against reverse engineering. Some things you can do:
distribute the .pyc files. This requires slightly more effort to reverse engineer
Use an obfuscator (there aren't a lot of these out there and I haven't found any I would call first class)
Write critical sections (maybe license checks, etc.) in c and call into them
Write critical sections as web services
If you distribute .pyc or py2exe, most people will probably never look at your code. IF someone reverses engineers your code from the byte code, what's the big deal? If they're determined enough, they could just write their own application. Don't worry about it so much and ship your application if it's great enough to be protective of the source.
Well, its not very likely to happen anytime soon - Python simply is an interpreted language that is not readily fit for compilation to native code. What you could do, however, is to use Cython to create a Python extension using Cython's Python-like syntax, compile this to native code and bundle it with a "harmless" application that makes use of your extension module while at the same time making your IP harder to obtain.
You might also try ShedSkin, a native Python to C++ compiler, but you will soon find it very limited.
Still, you might consider if all that is really worth the effort - most of the time it is not.
You could contact the author of Nuitka to see if it could help with distribution of non-readable code.
Edit: Try the Google cache...
i am creating ( researching possibility of ) a highly customizable python client and would like to allow users to actually edit the code in another language to customize the running of program. ( analogous to browser which itself coded in c/c++ and run another language html/js ). so my question is , is there any programming language implemented in pure python which i can see as a reference ( or use directly ? ) -- i need simple language ( simple statements and ifs can do )
edit: sorry if i did not make myself clear but what i want is "a language to customize the running of program" , even though pypi seems a great option, what i am looking for is more simple which i can study and extend myself if need arise. my google searches pointing towards xml based langagues. ( BMEL , XForms etc ).
The question isn't completely clear on scope, but I have a hunch that PyPy, embedding other full languages, and similar solutions might be overkill. It sounds like iamgopal may really be interested in something more like Interpreter Pattern or Little Language.
If the language you want to support is really small (see the Interpreter Pattern link), then hand-coding this yourself in Python won't be too hard. You can write a simple parser (Google around; here's one example), then walk the AST and evaluate user expressions.
However, if you expect this to be used for a long time or by many people, it may be worth throwing a real language at the problem. (I'd recommend Python itself if your users are already familiar with basic Python syntax).
Ren'Py is a modification to Python syntax built on top of Python itself, using the language tools in the stdlib.
For your user's sake, don't use an XML based language - XML is an awful basis for a programming language and your users will hate you for it.
Here is a suggestion. Use a strict subset of Python for your language. Use the compiler module to convert their code into an abstract syntax tree and walk the tree to to validate that the code conforms to your subset before converting the AST into python bytecode.
N.B. I just checked the docs and see that the compiler package is deprecated in 2.6 and removed in Python 3.x. Does anyone know why that is?
Numerous template languages such as Cheetah, Django templates, Genshi, Mako, Mighty might serve as an example.
Why not Python itself? With some care you can use eval to run user code.
One of the good thing about interpreted scripting languages is that you don't need another extra scripting language!
PLY (Python Lex-Yacc)
is something of your interest.
Possibly Common Lisp (or any other Lisp) will be the best choice for that task. Because Lisp make it possible to easily extend host language with powerful macroses and construct DSL (domain specific language).
If all you need is simple if statements and expressions, I'm sure it wouldn't be an awful task to parse each line. Something like
if some flag
activate some feature
deactivate some feature
elif some other flag
activate some feature
activate some feature
else
logout
Just write a class which, while parsing takes the first word, checks if it's "if, elif, else," etc, and if so, check a flag and set a flag saying you either are or are not executing until the next conditional. If it's not a conditional, call a function based on the first keyword that would modify the program state in some way.
The class could store some local execution state (are we in an if statement? If so are we executing this branch?) and have another class containing some global application state (flags that are checkable by if statements, etc).
This is probably the wrong thing to do in your situation (it's very prone to bugs, it's dangerous if you don't treat the data in the scripts correctly), but it's at least a start if you do decide to interpret your own mini-language.
Seriously though, if you try this, be very, very, srs careful. Don't give the scripts any functionality that they don't definitely need, because you are almost certainly opening security holes by doing something like this.
Don't say I didn't warn you.
there is a year old, similar question - but in case there have been changes afoot:
i'm an intermediate c++ programmer just starting out on python, post some online tuts etc i can do some basic pythoneering, but was wondering if there are good places i can look online for simple(ish) --pref console based-- code that i can learn from, ideally with some sort of commentary.
anything come to mind?
thanks
The standard library is an excellent place to the start. It's maintained by the core python team and is of high quality with a lot of interesting idioms. I'd recommend the newer modules since they don't have much backward compatibility cruft and are more representative of the language as it is now. The older ones were written for earlier versions of Python and have some restrictions when it comes to API changes etc.
The list of modules in the standard library is described at http://docs.python.org/library/. You can go through it and decide which one you want to look at (area of interest etc.).
Their sources are viewable at the mercurial repo here http://hg.python.org/cpython/file/d7e85ddb1336/Lib (as of today). These are for the mainline 2.6 release. You can also checkout the repo and browse it on your local machine.
You can also start up your interpreter, import a module (say os) and do a print os.__file__ to see where the source file is if you want to look at the code in your local editor.
ActiveState Recipes is a good source for all kinds of Python scripts. But if you want to learn the basics of Python, you might just want to look at the standard library that ships with Python ("lib" directory").
i came across This The other day, Probably you can learn some python basics and have a laugh too!
Anyways, look at the libs as they said above, they are very useful
If you enjoy riddles:
www.pythonchallenge.com
If you're an intermediate C++ programmer, you're already equipped to handle to programming concepts. I like it because it gives me a reason to learn each part of the language, without being mundane 'Hello World' tasks.
However, some of the riddles are pretty tough and/or unrelated to programming. Either way, doing the first few will probably be enough to get your confidence up with Python syntax.
Here's the situation. The company I work for has quite a bit of existing Tcl code, but some of them want to start using python. It would nice to be able to reuse some of the existing Tcl code, because that's money already spent. Besides, some of the test equipment only has Tcl API's.
So, one of the ways I thought of was using the subprocess module to call into some Tcl scripts.
Is subprocess my best bet?
Has anyone used this fairly new piece of code: Plumage? If so what is your experience (not just for Tk)?
Any other possible ways that I have not considered?
I hope you're ready for this. Standard Python
import Tkinter
tclsh = Tkinter.Tcl()
tclsh.eval("""
proc unknown args {puts "Hello World!"}
}"!dlroW olleH" stup{ sgra nwonknu corp
""")
Edit in Re to comment: Python's tcl interpreter is not aware of other installed tcl components. You can deal with that by adding extensions in the usual way to the tcl python actually uses. Here's a link with some detail
How Tkinter can exploit Tcl/Tk extensions
This can be done.
http://wiki.tcl.tk/13312
Specificly look at the typcl extension.
Typcl is a bit weird... It's a an extension to use Tcl from Python.
It doesn't really require CriTcl and could have been done in standard C.
This code demonstrates using Tcl as shared library, and hooking into it
at run time (Tcl's stubs architecture makes this delightfully simple).
Furthermore, Typcl avoids string conversions where possible (both ways).
I've not used it myself, but SWIG might help you out:
http://www.swig.org/Doc1.1/HTML/Tcl.html