As I understood, PYTHONCASEOK option enables importing modules by matching case insensitively. But, as almost everything in python is case sensitive, why does it has to enable this option enabling more lazy writing.
Any other reason for introducing?
The purpose of PYTHONCASEOK is to enable finding module files on filesystems which are case-insensitive such as FAT, or which behave in a case-insensitive manner from point of view of the programmer, such as NTFS on Windows.
It exists to support code written for case-insensitive filesystems before case-sensitivity became the default behaviour when searching for modules in python 2.1.
The detailed explanation for the change is available in PEP 235
One interesting scenario described in the PEP is that some operating systems - such as OpenVMS - might change the case of a filename when the file is written:
if you create "fiLe", there's no telling
what it's stored as -- but most likely as "FILE" -- and any of the
16 obvious variations on open("FilE") will open it.
so a case-insensitive method of finding the module is a necessity on such as system.
Related
I'm doing a research project on detecting breaking changes from Python library upgrades. One of the steps is to extract the difference between two major versions of the same Python library by using static analysis(Coule be AST-based or not), in order to triage the pattern of change. The detection should not only find the difference from .py files, but also the difference from other project files including config files, resources, etc. Ideally, a scenario like if a .py file moved to another module should also be included. So I have two questions here:
Is there a tool that can do a similar job and also support flexible configuration for analysis?
If not, what will be the best strategy to search for that kind of difference and identify the category of this difference(e.g. variable, function, etc.)
Sorry, this might be a silly question, I'm not coming from a Python background, really running out of thoughts here. Any thoughts, ideas, and inputs are welcome. Thanks in advance.
Just spit balling some ideas here:
I don't think I'd be so concerned about detecting changes in the source files up front. There are a lot of ways to move code around among files without changing the interface to the module. For example you can put all of the code in __init__.py or, you can split it up into any number of files and subdirectories. However, the programmatic interface will stay the same.
Instead, you could use the dir() built-in to detect changes in the public classes and methods in the module. This will work well for libraries that used named arguments, but won't work well for functions which just use def func(*args, **kwargs) (this is why that should be avoided, all you former perl programmers!)
If the module uses the new type hinting, you can really get some mileage out of detecting change in types. If you use some tool that actually parses the python and infers types, that would work as well. I would guess VSCode probably contains such a library that it uses to give context-sensitive help.
So I have Emacs 24.3 and with it comes a quite recent python.el file providing a Python mode for editing.
But I keep reading that there is a python-mode.el on Launchpad, and comparing the two files it jumps out to me that the former is under 4000 lines, while the latter is almost 20000. This suggests that the latter is much more feature-rich.
And I can't find any online feature comparison about them, documentation, or at least a list about the features for each of them. Yep, there is syntax highlighting and embedded interpreter, but what about completion in shell buffer, completion in source file buffer, autoindent, reindent etc.
So what are the important features of these modes? (Or any other Python mode for Emacs which you recommend.) Please provide detailed answers.
I was python-mode.el user once but quit using it a year ago because I felt the way it was developed was not well organized. Here is a list from the note I took at that time. But I need to warn you that almost a year is passed since then so the situation may be changed.
Many copy & paste functions.
Many accidentally working code. For example, not passing around variables but using implicit binding. This produces many compile errors (and will not work if you change it to lexical scope).
Rough granularity of commit. I send a patch, and it was committed with unrelated changes.
One thing I like about python-mode.el is it comes with automated test set (although I've never run it). python.el does not have a test set yet. But I know the author of python.el is writing it now.
While python.el is compact, it does not mean you get poor functionality. It is more like keeping core small and let others to extend it by providing concise API. Same author of python.el wrote python-django.el to extend python.el for django projects. I wrote auto-completion plugin for Python called Jedi.el and advanced IPython plugin called EIN. Both of them have better support for python.el than python-mode.el (well, that's because I don't use python-mode.el, though).
I had a few thing I missed from python-mode.el first, but they are quickly fixed in python.el (Of course, this probably means that I was not using so much functionality in python-mode.el).
what about completion in shell buffer, completion in source file buffer, autoindent, reindent etc.
completion in shell buffer:
It sort of works in both python.el and python-mode.el. But sometimes it does not work if you have bad combination of Emacs version and python(-mode).el version. So probably python.el is safer in this manner.
But if you want better solution, use EIN :)
completion in source file buffer:
Just use Jedi.el :)
autoindent/reindent:
I don't know which one is better in performance-wise. However, keybind for return differs one to the other. In python-mode.el, if you type RET you get autoindent. In python.el, RET does not give you indentation and you should use C-j instead. Actually C-j for newline+indentation is universal behavior in Emacs. So python.el is better if you do programming in other languages.
Being heavily involved in developing python-mode.el last years, my comment probably is biased: Recommend to stay with python.el for Emacs beginners. Also its author deserves credit for some useful approaches.
python-mode.el is designed to boost edits productivity. It makes it easy to run or execute via python2 and python3 or IPython shells in parallel.
It reduces number of needed keystrokes providing tailored commands. It makes edits faster, assists programming by speech, macro-driven input etc.
Supports Python language features not known from python.el currently:
py-up, py-down - travelling nested blocks
Avoid typos fetching forms at point, a clause for example:
py-backward-clause
py-copy-clause
py-down-clause
...
No need to customize when testing different versions:
py-execute-clause-python2
py-execute-clause-python3
py-execute-clause-ipython
...
notion of finer grained parts - py-expression, py-minor-expression
commands running versioned and paralleled (I)Python-executables, no need to re-define the default Python
largely removing the need of an active region marked before, see py-execute-line and a whole bunch more
To get an overview, have a look at the menu. Directory "doc" lists commands.
As the quality of code is raised: a way to compare both modes probably is checking for the bugs listed in http://debbugs.gnu.org/. See for example bug #15510, #16875; or http://lists.gnu.org/archive/html/help-gnu-emacs/2014-04/msg00250.html
Commented at "rough granularity of commit" already: While tkf basically is right looking for smaller pieces, sometimes conditions make me leave the rule. Considerable parts are not written by hand, but by programs residing in directory "devel". They create files used in development branch frist - i.e. components-python-mode. When starting a new feature it's often not obvious if the path chosen is fruitful.
After a hundred would-be-commits or so it still might turn out impossible or not so recommendable. Instead of posting all the meanders, used to keep that experimental branch for several days in these cases and check in when tests passed.
BTW assume tkf refers not to compile-errors --which would be looked for instantly-- but compiler warnings. Unfortunately Emacs mixes warning about backed style preferences with real errors.
I want to allow users to make their own Python "mods" for my game, by placing their scripts in a special folder which the game "scans" for Python modules and imports.
What would be the simplest way to prevent "dangerous" scripts from being imported? I don't want people complaining to me that they used someone's mod and it erased their hard drive.
Things I would like to limit is accessing/modifying/creating any files outside of their folder and connecting to the internet/downloading/sending data. If you can thik of anything else, let me know.
So how can this be done?
Restricted Python seems to able to restrict functionality for code in a clean way and is compatible with python up to 2.7.
http://pypi.python.org/pypi/RestrictedPython/
e.g.
By supplying a different __builtins__ dictionary, we can rule out unsafe operations, such as opening files [...]
The obvious way to do it is to load the module as a string and exec it. This has just as many security risks, but might be easier to block by using custom globals and locals. Have a look at this question - it gives some really good guidance on this. As pointed out in Delnan's comments, this isn't completely secure though.
You could also try this. I haven't used it, but it seems to provide a safe environment for unsafe scripts.
There are some serious shortcomings for sandboxed python execution. aquavitae's answer links to some good discussion on the matter, especially this blog post. Read that first.
There is a kernel of secure execution within cPython. The fundamental idea is to replace the __builtins__ global (Note: not the __builtin__ module), which informs python to turn on some security features; making a handful of attributes on certain objects inaccessible, and removing most of the implementation objects from the interpreter when evaulating that bit of code.
You'll then need to write an actual implementation; in such a way that the protected modules are not the leaked into the sandbox. A fairly tested "file" replacement is provided in the linked blog. Getting a look on that might give you an idea of how involved and complex this problem is.
So now that you have understood that this is a challenge in python; you should take a look at languages with sandbox execution as a core feature, such as Lua, which is very popular in games.
Giving them python execution and trying to limit what they do is asking for trouble. See this SO question for discussion and a pointer to a good article. (You would presumably disable "eval", but it wouldn't make much difference in practice.
My suggestion: Turn the question around. Your goal is to provide them with scripting facilities so they can enhance the game. Find or define an interpreter for a suitable scripting language that has the features you need, and use it to execute their scripts. For example, you could support data persistence in a simple keystore model, without giving them file creation access. Or give them a command to create files but ensure it only accepts a path-less filename. The essential thing is to ensure that there is NO way for them to execute python commands directly.
My project has the ability to run python functions remotely. Doing so requires transmitting modules a given function utilizes. Determining what to send is conducted via a modified modulefinder.
As I modify the modulefinder to support arbitrary path_hooks, I've started to get the impression that path_hooks are not all that popular. Quick google codesearching seems to only show the ZipImporter using them. I've noticed a minor project using it (and even then, its loader doesn't support the PEP 302 extension of get_code, which is needed by the modified modulefinder).
Has anyone come across or created projects that use custom path_hooks to access source code?
Yes, I've coded some path hooks (for one of the obvious purposes: access modules living in other forms of storage besides the filesystem and zipfiles), but never on an open-source project (and actually never needed to support modulefinder in them). What difficulties are you encountering? While I can't share my original code I think I can share the know-how developed with it (though offhand I can't recall any special difficulties -- it has been a while). As for "popular", I guess they will be in direct proportion to the need to site modules "elsewhere" (e.g. in some form of DB), though of course general "usermode file systems" built e.g. using fuse , macfuse and dokan may also allow this (and offer other advantages in terms of generality -- not sure how performance compares).
I don't see that the compilation step is adding any value.
Reading just quickly about .mo files, it is clear that:
It is a machine-readable representation
It is a hash table
Given gettext's function, to lookup strings by keys at runtime, it is reasonable for this lookup to be implemented efficiently.
Also, it is needed for gettext's performance impact to be negligible; else interest to play nice with the international community would be even lower for english-speaking hackers (from all countries, we all speak english, programming languages are in english).
Making the .mo file an already processed representation is good since
The format is well suited for quick lookup (hash table)
The format needs little processing at application launch (custom-representation binary)
Because gettext module in Python follows GNU gettext standards which required to use PO files for do translations by people, and MO files for using by application in runtime. When you're using gettext module you're actually using GNUTranslations class, which name suggests it's implementation of GNU gettext.
But you can provide your own Translations class and load translations from PO files, there is nothing special about it. There is even some implementation of PO files reader in standard Python distribution, see script msgfmt.py in Tools directory.
I think that the compilation is .po -> .mo
The letters PO in .po files means Portable Object, to distinguish it from .mo files, where MO stands for Machine Object.
MO files are meant to be read by programs, and are binary in nature. A few systems already offer tools for creating and handling MO files as part of the Native Language Support coming with the system, but the format of these MO files is often different from system to system, and non-portable.
GNU `gettext' utilities - 1.4 Files Conveying Translations