I know it's possible to choose which Python version to check e.g. in vim.rc as per this SO answer, but can I do it per project? Using e.g. virtualenv or a configuration file?
It all depends on how you define "this is a project".
Several of us have been providing local vimrc plugins where a project definition is "all the files in this directory and its subdirectories". See this answer for more details and alternatives solutions on the subject: Vim: apply settings on files in directory
Note that lately I've been working on a different (and more lightweight) way (in most cases) to specify what a project is: https://github.com/LucHermitte/lh-vim-lib/blob/master/doc/Project.md (This is still highly experimental).
Reading the answer you've linked to... It only speaks about a global variable that permits to tune the behaviour of the plugin. If there was no other way but tuning this global option, you'd have needed to reset this global variable unconditionally in the local vimrc, or on a BufEnter autocommand. Fortunately, syntastic is project aware thanks to buffer local variables -- #lcd047 corrected me on this topic. This means that instead of resetting a global variable, you could instead set a buffer local variable depending on the current directory (or any other heuristic you could define in an autocommand -- without these plugins, you could simply listen for BufNew and BufRead event, but this won't support migration among machines, directories, etc).
Note that my local-vimrc plugin sources the current local vimrc configuration file every time we enter a buffer matching that configuration file. This means that if you don't add an anti-reinclusion guard, b:syntastic_python_python_exec would be reset every time you enter a buffer for which it has been defined. It shouldn't be that problematic here. Note also that I don't know how alternative plugins proceed.
Related
Like I said in the title I don't get what os.getenv("HOME") does in this code. I am following a course on an online site and the tutor was coding an interface with PyQt5 similar to notepad. I searched for an answer but they are a bit too advanced I guess. Also I have no idea what an environment variable is. By the way this is my first question on stack so excuse me for any possible mistakes and insufficient information.
def open_file(self):
file_name=QFileDialog.getOpenFileName(self,"Open File",os.getenv("HOME"))
with open(file_name[0],"r") as file:
self.writing_ar.setText(file.read())
The function above is connected to a button self.open such as self.open.clicked.connect(self.open_file)
And self.writing_ar is a QTextEdit object
In the case of os.getenv('HOME'), it's a UNIX-centric way to get the current user's home directory, which is stored as an environment variable per POSIX specification. A typical home directory location is /Users/yourname on MacOS, or /home/yourname on Linux, or c:\Users\Your Name on Windows -- so that's what this code is trying to look up.
The set of environment variables is effectively a key/value store, mapping strings to other strings, that is copied from any program to other processes it starts; they're thus a way to share configuration and other information between programs (though it only shares information down the tree, propagated only on process creation; changes made by a child process are not seen by its parent; and changes to a parent's environment after a child is started are not seen by the child).
If you want something that works reliably even on Windows, consider os.path.expanduser("~") instead. Thus, your code might become:
file_name = QFileDialog.getOpenFileName(self,
"Open File",
os.path.expanduser("~"))
See also What is the correct cross-platform way to get the home directory in Python?
It basically gets an environment variable for you and cast that onto a python variable.
From the code you shared, there should be a variable defined at the operating system level named HOME.
In Linux, that can be done with
export HOME="something_here"
You can check that this variable has actually been defined by typing
echo "$HOME"
in the terminal.
You can think of the os.getenv() method like it "echoes" the value of that argument onto some variable.
By default, pip logs errors into "~/.pip/pip.log". Pip has an option to change the log path, and I'd like to put the log file somewhere besides ~/.pip so as not to clutter up my home directory. Where should I put it and be XDG base dir compatible?
Right now I'm considering one of these:
$XDG_DATA_HOME (typically $HOME/.local/share)
$XDG_CACHE_HOME (typically $HOME/.cache)
This is, for the moment, unclear.
Different software seem to handle this in different ways (imsettings puts it in $XDG_CACHE_HOME,
profanity in $XDG_DATA_HOME).
Debian, however, has a proposal which I can get behind (emphasis mine):
This is a recurring request/complaint (see this or this) on the xdg-freedesktop mailing list to introduce another directory for state information that does not belong in any of the existing categories (see also home-dir.proposal. Examples for this information are:
history files of shells, repls, anything that uses libreadline
logfiles
state of application windows on exit
recently opened files
last time application was run
emacs: bookmarks, ido last directories, backups, auto-save files, auto-save-list
The above example information is not essential data. However it should still persist on reboots of the system unlike cache data that a user might consider putting in a TMPFS. On the other hand the data is rather volatile and does not make sense to be checked into a VCS. The files are also not the data files that an application works on.
A default folder for a future STATE category might be: $HOME/.local/state
This would effectively introduce another environment variable since $XDG_DATA_HOME usually points to $HOME/.local/share and this hypothetical environment variable ($XDG_STATE_HOME?) would point to $HOME/.local/state
If you really want to adhere to the current standard I would place my log files in $XDG_CACHE_HOME since log files aren't required to run the program.
I need to modify a text file at runtime but restore its original state later (even if the computer crash).
My program runs in regular sessions. Once a session ended, the original state of that file can be changed, but the original state won't change at runtime.
There are several instances of this text file with the same name in several directories. My program runs in each directory (but not in parallel), but depending on the directory content's it does different things. The order of choosing a working directory like this is completely arbitrary.
Since the file's name is the same in each directory, it seems a good idea to store the backed up file in slightly different places (ie. the parent directory name could be appended to the backup target path).
What I do now is backup and restore the file with a self-written class, and also check at startup if the previous backup for the current directory was properly restored.
But my implementation needs serious refactoring, and now I'm interested if there are libraries already implemented for this kind of task.
edit
version control seems like a good idea, but actually it's a bit overkill since it requires network connection and often a server. Other VCS need clients to be installed. I would be happier with a pure-python solution, but at least it should be cross-platform, portable and small enough (<10mb for example).
Why not just do what every unix , mac , window file has done for years -- create a lockfile/working file concept.
When a file is selected for edit:
Check to see if there is an active lock or a crashed backup.
If the file is locked or crashed, give a "recover" option
Otherwise, begin editing the file...
The editing tends to do one or more of a few things:
Copy the original file into a ".%(filename)s.backup"
Create a ".%(filename)s.lock" to prevent others from working on it
When editing is achieved, the lock goes away and the .backup is removed
Sometimes things are slightly reversed, and the original stays in place while a .backup is the active edit; on success the .backup replaces the original
If you crash vi or some other text programs on a linux box, you'll see these files created . note that they usually have a dot(.) prefix so they're normally hidden on the command line. Word/Powerpoint/etc all do similar things.
Implement Version control ... like svn (see pysvn) it should be fast as long as the repo is on the same server... and allows rollbacks to any version of the file... maybe overkill but that will make everything reversible
http://pysvn.tigris.org/docs/pysvn_prog_guide.html
You dont need a server ... you can have local version control and it should be fine...
Git, Subversion or Mercurial is your friend.
I'd like to save my script's data to disk to load next time the script runs. For simplicity, is it a good idea to use os.path.expanduser('~') and save a directory named ".myscript_data" there? It would only need to be read by the script, and to avoid clutter for the user, I'd like it to be hidden. Is it acceptable practice to place hidden ".files" on the users computer?
On Windows, use a subfolder of os.environ['APPDATA']; on Linux, a dot-folder under $HOME is typical.
You can also consider putting your files in a subdirectory of ~/.config*, which is a sort of emerging standard for configuration file placement. See also the XDG basefiles spec.
Not entirely related, but interesting: origin of dotfiles
*(edit) More accurately, os.environ.get('XDG_CONFIG_HOME', os.path.expanduser('~/.config')) as per the XDG spec.
Yes, it is. (I'm assuming you're on linux, right?)
See also this.
Yes, this is standard practice on most Unix systems. For transparency, it's often a good idea to print an informative message like Creating directory .dir to store script state the first time you create the storage location.
If you are expecting to store significant amounts of data, it's a good idea to confirm the location with the user.
This is also the standard place for any additional configuration files for your application.
On Linux, I suggest a file or directory (no dotfile) in os.environ['XDG_CONFIG_HOME'], which is in most cases the directory $HOME/.config. A dotfile in $HOME, however, is also often being used.
I'm writing a web server based on Python which should be able to execute "plugins" so that functionality can be easily extended.
For this I considered the approach to have a number of folders (one for each plugin) and a number of shell/python scripts in there named after predefined names for different events that can occur.
One example is to have an on_pdf_uploaded.py file which is executed when a PDF is uploaded to the server. To do this I would use Python's subprocess tools.
For convenience and security, this would allow me to use Unix environment variables to provide further information and set the working directory (cwd) of the process so that it can access the right files without having to find their location.
Since the plugin code is coming from an untrusted source, I want to make it as secure as possible. My idea was to execute the code in a subprocess, but put it into a chroot jail with a different user, so that it can't access any other resources on the server.
Unfortunately I couldn't find anything about this, and I wouldn't want to rely on the untrusted script to put itself into a jail.
Furthermore, I can't put the main/calling process into a chroot jail either, since plugin code might be executed in multiple processes at the same time while the server is answering other requests.
So here's the question: How can I execute subprocesses/scripts in a chroot jail with minimum privileges to protect the rest of the server from being damaged by faulty, untrusted code?
Thank you!
Perhaps something like this?
# main.py
subprocess.call(["python", "pluginhandler.py", "plugin", env])
Then,
# pluginhandler.py
os.chroot(chrootpath)
os.setgid(gid) # Important! Set GID first! See comments for details.
os.setuid(uid)
os.execle(programpath, arg1, arg2, ..., env)
# or another subprocess call
subprocess.call["python", "plugin", env])
EDIT: Wanted to use fork() but I didn't really understand what it did. Looked it up. New
code!
# main.py
import os,sys
somevar = someimportantdata
pid = os.fork()
if pid:
# this is the parent process... do whatever needs to be done as the parent
else:
# we are the child process... lets do that plugin thing!
os.setgid(gid) # Important! Set GID first! See comments for details.
os.setuid(uid)
os.chroot(chrootpath)
import untrustworthyplugin
untrustworthyplugin.run(somevar)
sys.exit(0)
This was useful and I pretty much just stole that code, so kudos to that guy for a decent example.
After creating your jail you would call os.chroot from your Python source to go into it. But even then, any shared libraries or module files already opened by the interpreter would still be open, and I have no idea what the consequences of closing those files via os.close would be; I've never tried it.
Even if this works, setting up chroot is a big deal so be sure the benefit is worth the price. In the worst case you would have to ensure that the entire Python runtime with all modules you intend to use, as well as all dependent programs and shared libraries and other files from /bin, /lib etc. are available within each jailed filesystem. And of course, doing this won't protect other types of resources, i.e. network destinations, database.
An alternative could be to read in the untrusted code as a string and then exec code in mynamespace where mynamespace is a dictionary defining only the symbols you want to expose to the untrusted code. This would be sort of a "jail" within the Python VM. You might have to parse the source first looking for things like import statements, unless replacing the built-in __import__ function would intercept that (I'm unsure).