Develop Reference

SimpleFSDirectory not working in python lucene

import sys, os
import lucene
from java.io import File
from org.apache.lucene.analysis.standard import StandardAnalyzer
from org.apache.lucene.index import DirectoryReader
from org.apache.lucene.queryparser.classic import QueryParser
from org.apache.lucene.document import Document, Field
from org.apache.lucene.index import IndexWriter, IndexWriterConfig
from org.apache.lucene.store import SimpleFSDirectory
from org.apache.lucene.util import Version
def index(start, no, dom):
lucene.initVM()
# join base dir and index dir
path = raw_input("Path for index: ")
index_path = File(path)
directory = SimpleFSDirectory(index_path) # the index
I keep having errors with the SimpleFSDirectory, even when I tried other things like directory = SimpleFSDirectory(File(os.path.abspath("paths")))
InvalidArgsError: (, 'init', (,))

As of 5.0, the SimpleFSDirectory ctor no longer takes a File argument, it takes a Path. You can convert to a Path, with File.toPath().
Also, I would recommend you use FSDirectory.open, instead. FSDirectory.open allows lucene to try to choose the best Directory implementation for the current environment, which will generally perform better than a SimpleFSDirectory.
So, something like:
index_path = File(path).toPath()
directory = FSDirectory.open(index_path)

get version of python module.pyd

I load a .pyd file as python module.
Under windows I see a version when I do a right click->Properties->"Details"-Tab
How can I read (in python) the fileversion of this pyd-file?
Exists a function or something in python to read this version?

Using win32api.GetFileVersionInfo:
>>> import win32api
>>> path = r'c:\python27\lib\site-packages\win32\win32api.pyd'
>>> info = win32api.GetFileVersionInfo(path, '\\')
>>> '{}.{}.{}.{}'.format(
... win32api.HIWORD(info['FileVersionMS']),
... win32api.LOWORD(info['FileVersionMS']),
... win32api.HIWORD(info['FileVersionLS']),
... win32api.LOWORD(info['FileVersionLS']))
'2.7.218.0'

Import custom modules on IPython.parallel engines with sync_imports()

I've been playing around with IPython.parallel and I wanted to use some custom modules of my own, but haven't been able to do it as explained on the cookbook using dview.sync_imports(). The only thing that has worked for me was something like
def my_parallel_func(args):
import sys
sys.path.append('/path/to/my/module')
import my_module
#and all the rest
and then in the main just to
if __name__=='__main__':
#set up dview...
dview.map( my_parallel_func, my_args )
The correct way to do this would in my opinion be something like
with dview.sync_imports():
import sys
sys.path.append('/path/to/my/module')
import my_module
but this throws an error saying there is no module named my_module.
So, what is the right way of doing it using dview.sync_imports()??

The problem is that you're changing the PYTHONPATH just in the local process running the Client, and not in the remote processes running in the ipcluster.
You can observe this behaviour if you run the next piece of code:
from IPython.parallel import Client
rc = Client()
dview = rc[:]
with dview.sync_imports():
import sys
sys.path[:] = ['something']
def parallel(x):
import sys
return sys.path
print 'Local: ', sys.path
print 'Remote: ', dview.map_sync(parallel, range(1))
Basically all the modules that you want to use with sync_imports must already be in the PYTHONPATH.
If it's not in the PYTHONPATH then you must add it to the path in the function that you execute remotely, and then import the module in the function.

Opening files with Pydbg while application is running

Using pydbg I'm opening files(ex. c:\\myfile.mnp) within a win32 application(Ex. c:\\myprog.exe) in this way.
dbg = pydbg()
dbg.load("c:\\myprog.exe", "c:\\myfile1.mnp")
If the target application is already running then, is it possible to open a another file(For example c:\myfile2.mnp ) within the same application which is already running without closing that process/apps, using pydbg?

From personal experience, its better to have python start the application, or attach to it while its running.
import pydbg
from pydbg import *
from pydbg.defines import *
import struct
import utils
dbg = pydbg()
pid = ''
name = ''
found_program = False
for (pid, name) in dbg.enumerate_processes():
if name.lower() == "program.exe":
found_program = True
dbg.attach(pid)
if found_program:
dbg.run()
To make python start it:
from os import system
system('start "c:\program.exe"')

How to get filename of the __main__ module in Python?

Suppose I have two modules:
a.py:
import b
print __name__, __file__
b.py:
print __name__, __file__
I run the "a.py" file. This prints:
b C:\path\to\code\b.py
__main__ C:\path\to\code\a.py
Question: how do I obtain the path to the __main__ module ("a.py" in this case) from within the "b.py" library?

import __main__
print(__main__.__file__)

Perhaps this will do the trick:
import sys
from os import path
print(path.abspath(str(sys.modules['__main__'].__file__)))
Note that, for safety, you should check whether the __main__ module has a __file__ attribute. If it's dynamically created, or is just being run in the interactive python console, it won't have a __file__:
python
>>> import sys
>>> print(str(sys.modules['__main__']))
<module '__main__' (built-in)>
>>> print(str(sys.modules['__main__'].__file__))
AttributeError: 'module' object has no attribute '__file__'
A simple hasattr() check will do the trick to guard against scenario 2 if that's a possibility in your app.

The python code below provides additional functionality, including that it works seamlessly with py2exe executables.
I use similar code to like this to find paths relative to the running script, aka __main__. as an added benefit, it works cross-platform including Windows.
import imp
import os
import sys
def main_is_frozen():
return (hasattr(sys, "frozen") or # new py2exe
hasattr(sys, "importers") # old py2exe
or imp.is_frozen("__main__")) # tools/freeze
def get_main_dir():
if main_is_frozen():
# print 'Running from path', os.path.dirname(sys.executable)
return os.path.dirname(sys.executable)
return os.path.dirname(sys.argv[0])
# find path to where we are running
path_to_script=get_main_dir()
# OPTIONAL:
# add the sibling 'lib' dir to our module search path
lib_path = os.path.join(get_main_dir(), os.path.pardir, 'lib')
sys.path.insert(0, lib_path)
# OPTIONAL:
# use info to find relative data files in 'data' subdir
datafile1 = os.path.join(get_main_dir(), 'data', 'file1')
Hopefully the above example code can provide additional insight into how to determine the path to the running script...

Another method would be to use sys.argv[0].
import os
import sys
main_file = os.path.realpath(sys.argv[0]) if sys.argv[0] else None
sys.argv[0] will be an empty string if Python gets start with -c or if checked from the Python console.

import sys, os
def getExecPath():
try:
sFile = os.path.abspath(sys.modules['__main__'].__file__)
except:
sFile = sys.executable
return os.path.dirname(sFile)
This function will work for Python and Cython compiled programs.

I use this:
import __main__
name = __main__.__file__ # this gets file path
lastBar =int(name[::-1].find('/')) # change '/' for linux or '\' for windows
extension = 3 #3 for python file .py
main_file_name=name[::-1][extension:lastBar][::-1] #result