Most rails modes for emacs have this kind of functionality. You are in a controller file over a function "kaboosh" in
"app/controller/bla.rb" and with a keyboard shortcut you switch to "app/views/kaboosh.erb" or to app/models/bla.rb".
A similar functionality exists for .c and .h files using ff-find-other-file.
I checked jump.el and findr.el but all seems a little bit complicated.
I tried searching this for django projects (it would have helped a lot) but no luck.
Anyone knows a simple way (etags?) that it's easy to define the connection between current file/function and target file and then bind it to a keyboard shortcut?
Tags is set up well to jump you to the definition of a function. M-. will take you to the first occurrence of a function definition, C-u M-. will take you to the next (and one after that, and after that...). Perhaps the C-u M-. solves some of your problem.
Regarding associations between files, and wanting a rails like interface, it looks like you could pull out the appropriate chunks of code and customize them for python.
Specifically, you'll need the jump package, make a similar jump schema like the rinari-jump-schema (found in rinari.el). The jump schema uses 'ruby-add-log-current-method (found in ruby-mode.el), and you'd just need to customize that to return the current method name for Python syntax.
It doesn't look like anyone has done this for Python yet, you could be the first. I believe those are the only pieces you'll need.
Related
The full explanation of what I want to do and why would take a while to explain. Basically, I want to use a private SSL connection in a publicly distributed application, and not handout my private ssl keys, because that negates the purpose! I.e. I want secure remote database operations which no one can see into - inclusive of the client.
My core question is : How could I make the Python ssl module use data held in memory containing the ssl pem file contents instead of hard file system paths to them?
The constructor for class SSLSocket calls load_verify_locations(ca_certs) and load_cert_chain(certfile, keyfile) which I can't trace into because they are .pyd files. In those black boxes, I presume those files are read into memory. How might I short circuit the process and pass the data directly? (perhaps swapping out the .pyd?...)
Other thoughts I had were: I could use io.StringIO to create a virtual file, and then pass the file descriptor around. I've used that concept with classes that will take a descriptor rather than a path. Unfortunately, these classes aren't designed that way.
Or, maybe use a virtual file system / ram drive? That could be trouble though because I need this to be cross platform. Plus, that would probably negate what I'm trying to do if someone could access those paths from any external program...
I suppose I could keep them as real files, but "hide" them somewhere in the file system.
I can't be the first person to have this issue.
UPDATE
I found the source for the "black boxes"...
https://github.com/python/cpython/blob/master/Modules/_ssl.c
They work as expected. They just read the file contents from the paths, but you have to dig down into the C layer to get to this.
I can write in C, but I've never tried to recompile an underlying Python source. It looks like maybe I should follow the directions here https://devguide.python.org/ to pull the Python repo, and make changes to. I guess I can then submit my update to the Python community to see if they want to make a new standardized feature like I'm describing... Lots of work ahead it seems...
It took some effort, but I did, in fact, solve this in the manner I suggested. I revised the underlying code in the _ssl.c Python module / extension and rebuilt Python as a whole. After figuring out the process for building Python from source, I had to learn the details for how to pass variables between Python and C, and I needed to dig into guts of OpenSSL (over which the Python module is a wrapper).
Fortunately, OpenSSL already has functions for this exact purpose, so it was just a matter of swapping out the how Python is trying to pass file paths into the C, and instead bypassing the file reading process and jumping straight to the implementation of using the ca/cert/key data directly instead.
For the moment, I only did this for Windows. Since I'm ultimately creating a cross platform program, I'll have to repeat the build process for the other platforms I'll support - so that's a hassle. Consider how badly you want this, if you are going to pursue it yourself...
Note that when I rebuilt Python, I didn't use that as my actual Python installation. I just kept it off to the side.
One thing that was really nice about this process was that after that rebuild, all I needed to do was drop the single new _ssl.pyd into my working directory. With that file in place, I could pass my direct cert data. If I removed it, I could pass the normal file paths instead. It will use either the normal Python source, or implicitly use the override if the .pyd file is simply put in the program's directory.
Has someone implemented the CSV-handling for Flyway? It was requested some time ago (Flyway specific migration with csv files). Flyway comments it now as a possibility for the MigrationResolver and MigrationExecutor, but it does not seem to be implemented.
I've tried to do it myself with Flyway 4.2, but I'm not very good with java. I got as far as creating my own jar using the sample and make it accessible to flyway. But how does flyway distinguish when to use the SqlMigrator and when to use my CsvMigrator? I thought I have to register my own prefix/suffix (as the question above writes), but FlywayConfiguration seems to be read-only, at least I did not see any API calls for doing this :(.
How to connect the different Resolvers to the different migration file types? (.sql to the migration using Sql and .csv/.py to the loading of Csv and executing python scripts)
After some shed of tears and blood, it looks like came up with something on this. I can't make the whole code available because it is using proprietary file format, but here's the main ideas:
implement the ConfigurationAware as well, and use the setFlywayConfiguration implementation to catalog the extra files you want to handle (i.e. .csv). This is executed only once during the run.
during this cataloging I could not use the scanner or LoadableResources, there's some Java magic I do not understand. All the classes and methods seem to be available and accessible, even when using .getMethods() runtime... but when trying to actually call them during a run it throws java.lang.NoSuchMethodError and java.lang.NoClassDefFoundError. I've wasted a whole day on this - don't do that, just copy-paste the code from org.flywaydb.core.internal.util.scanner.filesystem.FileSystemScanner.
use Set< String > instead of LoadableResources[], way easier to work with, especially since there's no access to LoadableResources anyway and working with [] was a nightmare.
the python/shell call will go to the execute(). Some tips:
any exception or fawlty exitcode needs to be translated to an SQLException.
the build is enforcing Java 1.6, so new ProcessBuilder(cmd).inheritIO() cannot be used. Look at these solutions: ProcessBuilder: Forwarding stdout and stderr of started processes without blocking the main thread if you want to print the STDOUT/STDERR.
to compile flyway including your custom module, clone the whole flyway repo from git, edit the main pom.xml to include your module as well and use this command to compile: "mvn install -P-CommercialDBTest -P-CommandlinePlatformAssemblies -DskipTests=true" (I found this in another stackoverflow question.)
what I haven't done yet is the checksum part, I don't know yet what that wants.
Is there a emacs plugin which lists all the methods in the module in a side pane.
I am looking for a plugin which has keyboard shortcuts to show/hide all the methods in python module file currently opened.
I suggest using elpy.
C-c C-o runs elpy-occur-definitions which creates a new buffer with a list of all the class and function signatures.
You can navigate the list using C-n and C-p.
The mode works with next-error-follow minor mode.
So C-c C-f inside the buffer enables jumping to the class or function definition selected.
Here's an example of the contents of that buffer:
8 matches for "^ *\(def\|class\) " in buffer: leveling_utils.py
11:def leveling(episodes_with_potential_associations_by_member):
26:def _apply_leveling(sorted_episodes):
41:def _set_non_chronic_associations(episode):
73:def _apply_sub_to_procedural_association(assoc):
94:def _set_chronic_associations(episode):
102:def _set_associations_for_self(episode):
118:def _set_to_actual(association):
122:def _log_actual_associations(member, leveled_episodes):
By the way, it has many other useful features that to me, make it a necessary addition to python-mode.
For the first question, use M-xspeed-bar, like Alex suggested.
For the second, enable hs-minor-mode, M-xhs-minor-mode, and use C-cC-#C-S-h to hide all methods, and C-cC-#C-S-s to show.
You can look to ECB (it's better to take my fork of it, as it's adapted to fresh Emacs & CEDET) - it can display information about source code fetching it from Semantic (CEDET subpackage) or via imenu (for languages that aren't supported by Semantic).
Speedbar (included into GNU Emacs) can also show a list of top level objects, but doesn't show differences between imports, functions of other top-level stuff
For me, the easiest and most convenient method to quickly lookup methods is the command helm-occur (C-x c M-s o). You start typing the name of the method you want to jump to and suggestions start popping in as you type. Then you hit enter to select the one you want and your cursor jumps right there in the code. Helm-occur wasn't strictly written for this purpose, but works quite well that way.
Speedbar is good, and another nice alternative is helm-imenu. I've bind several keys to access it quicky from different contexts and use it most of the time
I am about to get a bunch of python scripts from an untrusted source.
I'd like to be sure that no part of the code can hurt my system, meaning:
(1) the code is not allowed to import ANY MODULE
(2) the code is not allowed to read or write any data, connect to the network etc
(the purpose of each script is to loop through a list, compute some data from input given to it and return the computed value)
before I execute such code, I'd like to have a script 'examine' it and make sure that there's nothing dangerous there that could hurt my system.
I thought of using the following approach: check that the word 'import' is not used (so we are guaranteed that no modules are imported)
yet, it would still be possible for the user (if desired) to write code to read/write files etc (say, using open).
Then here comes the question:
(1) where can I get a 'global' list of python methods (like open)?
(2) Is there some code that I could add to each script that is sent to me (at the top) that would make some 'global' methods invalid for that script (for example, any use of the keyword open would lead to an exception)?
I know that there are some solutions of python sandboxing. but please try to answer this question as I feel this is the more relevant approach for my needs.
EDIT: suppose that I make sure that no import is in the file, and that no possible hurtful methods (such as open, eval, etc) are in it. can I conclude that the file is SAFE? (can you think of any other 'dangerous' ways that built-in methods can be run?)
This point hasn't been made yet, and should be:
You are not going to be able to secure arbitrary Python code.
A VM is the way to go unless you want security issues up the wazoo.
You can still obfuscate import without using eval:
s = '__imp'
s += 'ort__'
f = globals()['__builtins__'].__dict__[s]
** BOOM **
Built-in functions.
Keywords.
Note that you'll need to do things like look for both "file" and "open", as both can open files.
Also, as others have noted, this isn't 100% certain to stop someone determined to insert malacious code.
An approach that should work better than string matching us to use module ast, parse the python code, do your whitelist filtering on the tree (e.g. allow only basic operations), then compile and run the tree.
See this nice example by Andrew Dalke on manipulating ASTs.
built in functions/keywords:
eval
exec
__import__
open
file
input
execfile
print can be dangerous if you have one of those dumb shells that execute code on seeing certain output
stdin
__builtins__
globals() and locals() must be blocked otherwise they can be used to bypass your rules
There's probably tons of others that I didn't think about.
Unfortunately, crap like this is possible...
object().__reduce__()[0].__globals__["__builtins__"]["eval"]("open('/tmp/l0l0l0l0l0l0l','w').write('pwnd')")
So it turns out keywords, import restrictions, and in-scope by default symbols alone are not enough to cover, you need to verify the entire graph...
Use a Virtual Machine instead of running it on a system that you are concerned about.
Without a sandboxed environment, it is impossible to prevent a Python file from doing harm to your system aside from not running it.
It is easy to create a Cryptominer, delete/encrypt/overwrite files, run shell commands, and do general harm to your system.
If you are on Linux, you should be able to use docker to sandbox your code.
For more information, see this GitHub issue: https://github.com/raxod502/python-in-a-box/issues/2.
I did come across this on GitHub, so something like it could be used, but that has a lot of limits.
Another approach would be to create another Python file which parses the original one, removes the bad code, and runs the file. However, that would still be hit-and-miss.
In the nautilus-python bindings, there is a file "nautilus.defs". It contains stanzas like
(define-interface MenuProvider
(in-module "Nautilus")
(c-name "NautilusMenuProvider")
(gtype-id "NAUTILUS_TYPE_MENU_PROVIDER")
)
or
(define-method get_mime_type
(of-object "NautilusFileInfo")
(c-name "nautilus_file_info_get_mime_type")
(return-type "char*")
)
Now I can see what most of these do (eg. that last one means that I can call the method "get_mime_type" on a "FileInfo" object). But I'd like to know: what is this file, exactly (ie. what do I search the web for to find out more info)? Is it a common thing to find in Python/C bindings? What is the format, and where is it documented? What program actually processes it?
(So far, I've managed to glean that it gets transformed into a C source file, and it looks a bit like lisp to me.)
To answer your "What program actually processes it?" question:
From Makefile.in in the src directory, the command that translates the .defs file into C is PYGTK_CODEGEN. To find out what PYGTK_CODEGEN is, look in the top-level configure.in file, which contains these lines:
AC_MSG_CHECKING(for pygtk codegen)
PYGTK_CODEGEN="$PYTHON `$PKG_CONFIG --variable=codegendir pygtk-2.0`/codegen.py"
AC_SUBST(PYGTK_CODEGEN)
AC_MSG_RESULT($PYGTK_CODEGEN)
So the program that processes it is a Python script called codegen.py, that apparently has some link with PyGTK. Now a Google search for PyGTK codegen gives me this link as the first hit, which says:
"PyGTK-Codegen is a system for automatically generating wrappers for interfacing GTK code with Python."
and also gives some examples.
As for: "What is the format, and where is it documented?". As others have said, the code looks a lot like simple Scheme. I couldn't find any documentation at all on codegen on the PyGTK site; this looks like one of those many dark corners of open source that isn't well documented. Your best bet would probably be to download a recent tarball for PyGTK, look through the sources for the codegen.py file and see if the file itself contains sufficient documentation.
All you need to create Python bindings for C code is to use the Python / C API. However, the API can be somewhat repetitive and redundant, and so various forms of automation may be used to create them. For example, you may have heard of swig. The LISP-like (Scheme) code that you see is simply a configuration file for PyGTK-Codegen, which is a similar automation program for creating bindings to Python.