Develop Reference

Resolve dns (edns) with client subnet option in Python

I'm looking for an implementation in Python that would allow me to resolve a DNS address using an extension of DNS (EDNS) "client sub options" . This option allows better DNS-resolution for content delivery systems - and ultimately, faster internet routing. The motivation is better explained here: http://www.afasterinternet.com/howitworks.htm
another name for this is "vandergaast-edns-client-subnet"
an implementation for dig is available here:
https://www.gsic.uva.es/~jnisigl/dig-edns-client-subnet.html
I'm looking for a python implementation that would do the same.

I'm the developer/maintainer of dnspython-clientsubnet. It is designed to be used in your code as an additive to dnspython. I've just released version 2.0.0 (after trying to do what you wanted) which makes everything much easier
pip install clientsubnetoption (works for both Python2 & Python3)
Import clientsubnetoption and dependancies you'll need:
import dns
import clientsubnetoption
Setup your ClientSubnetOption with the information you want:
cso = clientsubnetoption.ClientSubnetOption('1.2.3.4')
Create your DNS packet:
message = dns.message.make_query('google.com', 'A')
Add the edns option:
message.use_edns(options=[cso])
Use message to make your query:
r = dns.query.udp(message, '8.8.8.8')
Option information is now at r.options and there can be multiple, so you may need to iterate through them to find the ClientSubnetOption object.
for options in r.options:
if isinstance(options, ClientSubnetOption):
# do stuff here
pass
The code in clientsubnetoption.py is there to act as a unit test and a testing tool for support of edns-clientsubnet, not because you have to use it that way.

A python implementation exists:
its an extension of dnspython (http://www.dnspython.org/) that can be found here: https://github.com/opendns/dnspython-clientsubnetoption
pip install dnspython
git clone the repo from github
use this command:
python clientsubnetoption.py (name-server) (host to query>) -s (client-ip) -m 32
Note that the repo does not actually print the results. its just a tester, so it just emits "success" or "failure". To get the actual results you'll need to modify the python code to print the response from the DNS server.

Using a TLB-defined interface with Python and COM

I will try to keep this question as tight as possible, but if it seems that I am saying insane things, it is almost certainly because I am ignorant of some key point, so please do correct me.
I am writing a program, in a Windows environment, that will interface with an existing application that has a COM interface to allow 3rd-party software to interact with it.
I have read all of the documentation for this application, and it says that there is a TLB file that defines the functions and data available via COM.
How do I use the TLB file with python? How do I discover the progID of the application so that I can interface with it (this isn't given in the documentation).
I'm pretty lost. I have a fair amount of experience with Python, but I am completely new to developing in a Windows environment. Any help would be enormously helpful. I have been reading all the documentation on win32com, but I still have no clue what to do, as no one addresses -- as far as I have seen -- bringing in a TLB file.

The questions asked is to link the custom TLB file with COM client to be developed in python. I have done a small example code for my COM server developed in C# and same is accessed by python client using "comtypes" package.
The below snippet of code gives:
import comtypes.client as CC
import comtypes
ccHandle = CC.CreateObject("CSharpServer.InterfaceImplementation")
print (ccHandle)
import comtypes.gen.CSharpServer as CS
InterfaceHandle = ccHandle.QueryInterface(CS.IManagedInterface)
print ("output of PrintHi function = ", InterfaceHandle.PrintHi("World"))
The above python script is for the C# COM server code available at http://msdn.microsoft.com/en-us/library/aa645738(v=vs.71).aspx (refer to the File 1: CSharpServer.cs).

OK, it's been a while since I've done this, and I'm not a COM expert by any means. Read the COM chapter from Python Programming on Windows to see how to do this. Follow along with the examples (trying things out against Excel) to get a feel of how things work.
First off, install the PyWin32 Extensions if you haven't already. This is the package that gives you pythonwin.exe and the COM interface modules. Get it from here.
Then you are going to open the "COM Makepy Utility" from PythonWin's Tools menu. Browse through the list of registered COM components (some will be typelibs, others DLLs) until you identify the one you have (you have to do a bit of detective work). Click OK to generate the Python glue code. You will then need to run it again with the -i command-line argmument to generate the boilerplate code so your python script can use this glue. Here's a paraphrase of the O'Reilly example for the Microsoft Excel Object library:
import win32com.client
from win32com.client import gencache
gencache.EnsureModule('{00020813-0000-0000-C000-000000000046}', 0, 1, 2)
earlyBound = win32com.client.Dispatch("Excel.Application")
lateBound = win32com.client.dynamic.Dispatch("Excel.Application")
print earlyBound.ActiveCell()
Using early-bound objects is optional, but it does improve performance.
To find the ProgID is again a bit of detective work, although this answer seems to imply it's going to be hard. Try poking around the HKEY_CLASSES_ROOT hive of the registry with RegEdit to see if you see a ProgID that looks promising.

Detecting symlinks (mklink) on Vista/7 in Python without Pywin32

Currently the buildout recipe collective.recipe.omelette uses junction.exe on all versions of Windows to create symlinks. However junction.exe does not come with Windows by default and most importantly does not support creating symlinks to files (only directories) which causes a problem with quite a few Python packages.
On NT6+ (Vista and 7) there is now the mklink utility that not only comes by default but is also capable of creating symlinks to files as well as directories. I would like to update collective.recipe.omelette to use this if available and have done so except for one otherwise simple feature; detecting whether a file or folder is actually a symlink. Since this is a small buildout recipe, requiring Pywin32 in my opinion is a bit too much (unless setuptools could somehow only download it on Windows?).
Currently on Windows what omelette does is call junction.exe on the folder and then grep the response for "Substitute Name:" but I can't find anything as simple for mklink.
The only method I can think of is to call "dir" in the directory and then to go through the response line by line looking for "<SYMLINK>" and the folder/filename on the same line. Surely there is something better?

See jaraco.windows.filesystem (part of the jaraco.windows package) for extensive examples on symlink operations in Windows without pywin32.

Could you use ctypes to access the various needed functions and structures? this patch, currently under discussion, is intended to add symlink functionality to module os under Vista and Windows 7 -- but it won't be in before Python 2.7 and 3.2, so the wait (and the requirement for the very latest versions when they do eventually come) will likely be too long; a ctypes-based solution might tide you over and the code in the patch shows what it takes in C to do it (and ctypes-based programmed in Python is only a bit harder than the same programming in C).
Unless somebody's already released some other stand-alone utility like junction.exe for this purpose, I don't see other workable approaches (until that patch finally makes it into a future Python's stdlib) beyond using ctypes, Pywin32, or dir, and you've already ruled out the last two of these three options...!-)

On windows junctions and symbolic links have the attribute FILE_ATTRIBUTE_REPARSE_POINT (0x400) for reparse points. If you get the file's attributes, then detect this on?
You could use ctypes (as stated in the other answer) to access Kernel32.dll and GetFileAttributes, and detect this value.

You could leverage the Tcl you have available with Tkinter, as that has a 'file link' command that knows about junctions, unlike Pythons os module.

I have searched widely for a better solution, however python 2.7 just does not have a good solution for this. So eventually, I ended up with this (code below), its admittedly ugly, but it's downright pretty compared to all the ctypes hacks I've seen.
import os, subprocess
def realpath(path):
# not a folder path, ignore
if (not os.path.isdir(path)):
return path
rootpath = os.path.abspath(path)
oneup, foldername = os.path.split(rootpath)
output = subprocess.check_output("dir " + oneup, shell=True)
links = {}
for line in output.splitlines():
pos = line.find("<SYMLINKD>")
if not pos == -1:
link = line[pos+15:].split()
links[link[0]] = link[1].strip("[]")
return links[foldername] if links.has_key(foldername) else rootpath

Mercurial scripting with python

I am trying to get the mercurial revision number/id (it's a hash not a number) programmatically in python.
The reason is that I want to add it to the css/js files on our website like so:
<link rel="stylesheet" href="example.css?{% mercurial_revision "example.css" %}" />
So that whenever a change is made to the stylesheet, it will get a new url and no longer use the old cached version.
OR if you know where to find good documentation for the mercurial python module, that would also be helpful. I can't seem to find it anywhere.
My Solution
I ended up using subprocess to just run a command that gets the hg node. I chose this solution because the api is not guaranteed to stay the same, but the bash interface probably will:
import subprocess
def get_hg_rev(file_path):
pipe = subprocess.Popen(
["hg", "log", "-l", "1", "--template", "{node}", file_path],
stdout=subprocess.PIPE
)
return pipe.stdout.read()
example use:
> path_to_file = "/home/jim/workspace/lgr/pinax/projects/lgr/site_media/base.css"
> get_hg_rev(path_to_file)
'0ed525cf38a7b7f4f1321763d964a39327db97c4'

It's true there's no official API, but you can get an idea about best practices by reading other extensions, particularly those bundled with hg. For this particular problem, I would do something like this:
from mercurial import ui, hg
from mercurial.node import hex
repo = hg.repository('/path/to/repo/root', ui.ui())
fctx = repo.filectx('/path/to/file', 'tip')
hexnode = hex(fctx.node())
Update At some point the parameter order changed, now it's like this:
repo = hg.repository(ui.ui(), '/path/to/repo/root' )

Do you mean this documentation?
Note that, as stated in that page, there is no official API, because they still reserve the right to change it at any time. But you can see the list of changes in the last few versions, it is not very extensive.

An updated, cleaner subprocess version (uses .check_output(), added in Python 2.7/3.1) that I use in my Django settings file for a crude end-to-end deployment check (I dump it into an HTML comment):
import subprocess
HG_REV = subprocess.check_output(['hg', 'id', '--id']).strip()
You could wrap it in a try if you don't want some strange hiccup to prevent startup:
try:
HG_REV = subprocess.check_output(['hg', 'id', '--id']).strip()
except OSError:
HG_REV = "? (Couldn't find hg)"
except subprocess.CalledProcessError as e:
HG_REV = "? (Error {})".format(e.returncode)
except Exception: # don't use bare 'except', mis-catches Ctrl-C and more
# should never have to deal with a hangup
HG_REV = "???"

If you are using Python 2, you want to use hglib.
I don't know what to use if you're using Python 3, sorry. Probably hgapi.
Contents of this answer
Mercurial's APIs
How to use hglib
Why hglib is the best choice for Python 2 users
If you're writing a hook, that discouraged internal interface is awfully convenient
Mercurial's APIs
Mercurial has two official APIs.
The Mercurial command server. You can talk to it from Python 2 using the hglib (wiki, PyPI) package, which is maintained by the Mercurial team.
Mercurial's command-line interface. You can talk to it via subprocess, or hgapi, or somesuch.
How to use hglib
Installation:
pip install python-hglib
Usage:
import hglib
client = hglib.open("/path/to/repo")
commit = client.log("tip")
print commit.author
More usage information on the hglib wiki page.
Why hglib is the best choice for Python 2 users
Because it is maintained by the Mercurial team, and it is what the Mercurial team recommend for interfacing with Mercurial.
From Mercurial's wiki, the following statement on interfacing with Mercurial:
For the vast majority of third party code, the best approach is to use Mercurial's published, documented, and stable API: the command line interface. Alternately, use the CommandServer or the libraries which are based on it to get a fast, stable, language-neutral interface.
From the command server page:
[The command server allows] third-party applications and libraries to communicate with Mercurial over a pipe that eliminates the per-command start-up overhead. Libraries can then encapsulate the command generation and parsing to present a language-appropriate API to these commands.
The Python interface to the Mercurial command-server, as said, is hglib.
The per-command overhead of the command line interface is no joke, by the way. I once built a very small test suite (only about 5 tests) that used hg via subprocess to create, commit by commit, a handful of repos with e.g. merge situations. Throughout the project, the runtime of suite stayed between 5 to 30 seconds, with nearly all time spent in the hg calls.
If you're writing a hook, that discouraged internal interface is awfully convenient
The signature of a Python hook function is like so:
# In the hgrc:
# [hooks]
# preupdate.my_hook = python:/path/to/file.py:my_hook
def my_hook(
ui, repo, hooktype,
... hook-specific args, find them in `hg help config` ...,
**kwargs)
ui and repo are part of the aforementioned discouraged unofficial internal API. The fact that they are right there in your function args makes them terribly convenient to use, such as in this example of a preupdate hook that disallows merges between certain branches.
def check_if_merge_is_allowed(ui, repo, hooktype, parent1, parent2, **kwargs):
from_ = repo[parent2].branch()
to_ = repo[parent1].branch()
...
# return True if the hook fails and the merge should not proceed.
If your hook code is not so important, and you're not publishing it, you might choose to use the discouraged unofficial internal API. If your hook is part of an extension that you're publishing, better use hglib.

give a try to the keyword extension

FWIW to avoid fetching that value on every page/view render, I just have my deploy put it into the settings.py file. Then I can reference settings.REVISION without all the overhead of accessing mercurial and/or another process. Do you ever have this value change w/o reloading your server?

I wanted to do the same thing the OP wanted to do, get hg id -i from a script (get tip revision of the whole REPOSITORY, not of a single FILE in that repo) but I did not want to use popen, and the code from brendan got me started, but wasn't what I wanted.
So I wrote this... Comments/criticism welcome. This gets the tip rev in hex as a string.
from mercurial import ui, hg, revlog
# from mercurial.node import hex # should I have used this?
def getrepohex(reporoot):
repo = hg.repository(ui.ui(), reporoot)
revs = repo.revs('tip')
if len(revs)==1:
return str(repo.changectx(revs[0]))
else:
raise Exception("Internal failure in getrepohex")

Bash alias to Python script -- is it possible?

The particular alias I'm looking to "class up" into a Python script happens to be one that makes use of the cUrl -o (output to file) option. I suppose I could as easily turn it into a BASH function, but someone advised me that I could avoid the quirks and pitfalls of the different versions and "flavors" of BASH by taking my ideas and making them Python scripts.
Coincident with this idea is another notion I had to make a feature of legacy Mac OS (officially known as "OS 9" or "Classic") pertaining to downloads platform-independent: writing the URL to some part of the file visible from one's file navigator {Konqueror, Dolphin, Nautilus, Finder or Explorer}. I know that only a scant few file types support this kind of thing using some other command-line tools (exiv2, wrjpgcom, etc). Which is perfectly fine with me as I only use this alias to download single-page image files such as JPEGs anyways.
I reckon I might as well take full advantage of the power of Python by having the script pass the string which is the source URL of the download (entered by the user and used first by cUrl) to something like exiv2 which could write it to the Comment block, EXIF User Comment block, and (taking as a first and worst example) Windows XP's File Description field. Starting small is sometimes a good way to start.
Hope someone has advice or suggestions.
BZT

The relevant section from the Bash manual states:
Aliases allow a string to be
substituted for a word when it is used
as the first word of a simple command.
So, there should be nothing preventing you from doing e.g.
$ alias geturl="python /some/cool/script.py"
Then you could use it like any other shell command:
$ geturl http://example.com/excitingstuff.jpg
And this would simply call your Python program.

I thought Pycurl might be the answer. Ahh Daniel Sternberg and his innocent presumptions that everybody knows what he does. I asked on the list whether or not pycurl had a "curl -o" analogue, and then asked 'If so: How would one go about coding it/them in a Python script?' His reply was the following:
"curl.setopt(pycurl.WRITEDATA, fp)
possibly combined with:
curl.setopt(pycurl.WRITEFUNCITON, callback) "
...along with Sourceforge links to two revisions of retriever.py. I can barely recall where easy_install put the one I've got; how am I supposed to compare them?
It's pretty apparent this gentleman never had a helpdesk or phone tech support job in the Western Hemisphere, where you have to assume the 'customer' just learned how to use their comb yesterday and be prepared to walk them through everything and anything. One-liners (or three-liners with abstruse links as chasers) don't do it for me.
BZT