Develop Reference

Non-ASCII Environment variables set by Qt cannot be retrieved correctly in Python

Set a environment variable "GAME_FILENAME" using the following code in Qt where args[0] contains a file name alättet.DTA -
qputenv("GAME_FILENAME", qUtf8Printable(args[0])); //args[0]='\temp\alättet.DTA'
When the environment variable is retrieved in Python code with the following code, variable files is as shown here:
One character 'ä' became two. Looking into fBytes, there are 4 bytes corresponding to 'ä'.
files = os.environ.get('GAME_FILENAME')
fBytes = bytes(os.environ.get('GAME_FILENAME'))
Verified on the Qt side by storing the result of qgetenv('GAME_FILENAME') in a QByteArray. The bytes look correct. Also in Python, tried os.environ['GAME_FILENAME']="alättet.DTA", then os.environ.get('GAMRY_FILENAME') returns "alättet.DTA" correctly.
Anything missed here? Thanks.

Using multiple keywords in xattr via _kMDItemUserTags or kMDItemOMUserTags

While reorganizing my images, in anticipation of OSX Mavericks I am writing a script to insert tags into the xattr fields of my image files, so I can search them with Spotlight. (I am also editing the EXIF just to be safe.)
My questions are:
Which attribute is the best to use? _kMDItemUserTags seems to be the OSX version, but kMDItemOMUserTags is already in use by OpenMeta. I would ideally like something that will be Linux and OSX forward compatible.
How do I set multiple tags? Are the comma- or space-delimited or something else?
As an example, using the python xattr module, I am issuing these commands:
xattr.setxattr(FileName, "_kMDItemUserTags", "Name - Sample")
xattr.setxattr(FileName, "kMDItemOMUserTags", "Name,Institution,Sample")
I have also seen mention of these tags: kOMUserTags and kMDItemkeywords but don't know if they are likely to be implemented...
EDIT: Further investigation has shown that for things to be searchable in 10.8,
You need to preface the kMD with com.apple.metadata:
You have to either hex-encode or wrap in a plist.
This python code will generate the tag for kMDItemFinderComment which is searchable in spotlight...
def writexattrs(F,TagList):
""" writexattrs(F,TagList):
writes the list of tags to three xattr field:
'kMDItemFinderComment','_kMDItemUserTags','kMDItemOMUserTags'
This version uses the xattr library """
plistFront = '<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array>'
plistEnd = '</array></plist>'
plistTagString = ''
for Tag in TagList:
plistTagString = plistTagString + '<string>{}</string>'.format(Tag)
TagText = plistFront + plistTagString + plistEnd
OptionalTag = "com.apple.metadata:"
XattrList = ["kMDItemFinderComment","_kMDItemUserTags","kMDItemOMUserTags"]
for Field in XattrList:
xattr.setxattr (F,OptionalTag+Field,TagText.encode('utf8'))
# Equivalent shell command is xattr -w com.apple.metadata:kMDItemFinderComment [PLIST value] [File name]
I could not get it to work recursively on a folder with reliable results.

If you are worried about compatibility you have to set both of the attributes _kMDItemUserTags and kMDItemOMUserTags. I don't think there's a different solution since all the new OS X apps will use the former attribute, while the old apps still use the latter. This is just my speculation, but I guess OpenMeta will eventually be discontinued in favor of the new native API. Looking to the future you can use the _kMDItemUserTags attribute for your new apps/scripts even in Linux environment.
The tags are set as a property list-encoded array of strings as you have figured out. I don't know if it is a requirement but the OS X encodes the property list in the binary format and not in XML as you did.
I adapted your code to use binary property list as attribute values and everything worked. Here's my code. I am using biplist library which you can get with easy_install biplist.
import xattr
import biplist
def write_xattr_tags(file_path, tags):
bpl_tags = biplist.writePlistToString(tags)
optional_tag = "com.apple.metadata:"
map(lambda a: xattr.setxattr(file_path, optional_tag + a, bpl_tags),
["kMDItemFinderComment", "_kMDItemUserTags", "kMDItemOMUserTags"])
Tested with files and directories using tag:<some_tag> in Spotlight.
Hope this helps.
Note: I am using OS X Lion in this answer but it should work on Mavericks without any problems.
Edit: If you want to apply the tags to the contents of a directory it has to be done individually for every file since the xattr python module doesn't have the recursive option.

Having a hard time with wx.Config

My problem is that I wanna make a config file for an application but things aren't as simple as they seamed. So I've seen a tutorial where a config file contained the width and the height and the values that followed the equal sign could been retrieved and used to establish the size of a frame. Been there, done that, and everything worked okey. Strangely enough this seams to work only for some key words, as if I've used in the config file, also a parameter named freq it didn't retrieved its value as it did with the width, height parameters.
for example if I have a piece of code like this:
self.cfg = wx.Config('myconf')
wid = self.cfg.ReadInt('width')
hei = self.cfg.ReadInt('height')
freq = self.cfg.ReadInt('frequency')
print wid, hei, freq
where in myconf
width=400
height=250
frequency=3000
So it displays the height, the width but not the frequency, as for freq it gives only 0, and this happens for any other word I use in the config file. I'm guessing that only certain key words can be used in the config file, so they could be recognized while using wxPython.
If so where could I get a list of those keys I could use in making a configuration file ?

Personally, I would recommend using ConfigParser which comes with Python or perhaps ConfigObj (which I like better). If you're trying to save widget attributes / settings, then you also might want to look at the PersistenManager

What you could do is reading the lines yourself:
F = open('myconf', 'r')
for line in F.read().split('\n'):
switchres = { 'frequency': print(line.split('=')[1]),
# add the rest
}
switchres[line.split('=')[0]]

wxPython is a wrapper (or binding) for wxWidgets that is a C++ library.
So wx.Config or wx.FileConfig is intended for C++ that has no specific way of treating these type of config file (still wx.Config is meant to be portable between platforms).
In Python you already have ConfigParser module, but have in mind:
that some wxPython classes work with wx.Config like wx.FileHistory,
use wx.Config if you want to use the platform specific way of storing configurations (like registry in Windows).

How to determine file's extended attributes and resource forks with their size on Mac OSX?

I had written a small utility for creating xml for any folder structure and comparison of folders via generated xml that supports both win and Mac as platforms. However on Mac, recursively calculating folder size don't adds up to total size. On investigation, it came that it is due to extended attributes and resource forks that were present on certain files.
Can anybody know how can I determine these extended attributes and resource forks and their size preferably in python. Currently, I am using os.path.getsize to determine the size of file and adding files size to determine folder size eventually.

You want the hidden member of a stat result called st_blocks.
>>> s = os.stat('some_file')
>>> s
posix.stat_result(st_mode=33261, st_ino=12583347, st_dev=234881026,
st_nlink=1, st_uid=1000, st_gid=20, st_size=9889973,
st_atime=1301371810, st_mtime=847731600, st_ctime=1301371422)
>>> s.st_size / 1e6 # size of data fork only, in MB
9.889973
>>> x.st_blocks * 512e-6 # total size on disk, in MB
20.758528
The file in question has about 10 MB in the resource fork, which shows up in the result from stat but in a "hidden" attribute. (Bonus points for anyone who knows exactly which file this is.) Note that it is documented in man 2 stat that the st_blocks attribute always measures increments of 512 bytes.
Note: st_size measures the number of bytes of data, but st_blocks measures size on disk including the overhead from partially used blocks. So,
>>> open('file.txt', 'w').write('Hello, world!')
13
>>> s = os.stat('file.txt')
>>> s.st_size
13
>>> s.st_blocks * 512
4096
Now if you do a "Get Info" in the Finder, you'll see that the file has:
Size: 4 KB on disk (13 bytes)

Merely a partial answer ... but to learn the size of resource forks you can simply use the namedfork psuedodirectory
os.path.getsize("<path to file of interest>/..namedfork/rsrc")
Its theoretically possible that other named forks may exist ... but you can't discover a list of available forks.
As to the extended attributes ... what "size" are you interested in? You can use the xattr module to discover their content and thus the length of the key/value pairs.
But if you are interested more in their "on disk" size ... then its worth noting that extended attributes are not stored in some sort of file. They form part of the file metadata (ie just like the name and modified time are metadata) and are stored directly within a B*-tree node, rather than some "file"

Two options:
You could try using subprocess to call the system's "ls" or "du" command, which should be aware of the extended attributes.
or
You could install the xattr package, which can read the resource fork in addition to extended attributes (it's accessed via xattr.XATTR_RESOURCEFORK_NAME. Something like this might work:
import xattr
x = xattr.xattr("/path/to/my/file")
size_ = 0
for attribute in x:
size_ += len(x[attribute])
print size_
You might need to play around a little with the format of the extended attributes, as they're returned as strings but might be binary (?).
If you provide a minimal almost working example of code, I might be able to play with it a little more.

abstracting the conversion between id3 tags, m4a tags, flac tags

I'm looking for a resource in python or bash that will make it easy to take, for example, mp3 file X and m4a file Y and say "copy X's tags to Y".
Python's "mutagen" module is great for manupulating tags in general, but there's no abstract concept of "artist field" that spans different types of tag; I want a library that handles all the fiddly bits and knows fieldname equivalences. For things not all tag systems can express, I'm okay with information being lost or best-guessed.
(Use case: I encode lossless files to mp3, then go use the mp3s for listening. Every month or so, I want to be able to update the 'master' lossless files with whatever tag changes I've made to the mp3s. I'm tired of stubbing my toes on implementation differences among formats.)

I needed this exact thing, and I, too, realized quickly that mutagen is not a distant enough abstraction to do this kind of thing. Fortunately, the authors of mutagen needed it for their media player QuodLibet.
I had to dig through the QuodLibet source to find out how to use it, but once I understood it, I wrote a utility called sequitur which is intended to be a command line equivalent to ExFalso (QuodLibet's tagging component). It uses this abstraction mechanism and provides some added abstraction and functionality.
If you want to check out the source, here's a link to the latest tarball. The package is actually a set of three command line scripts and a module for interfacing with QL. If you want to install the whole thing, you can use:
easy_install QLCLI
One thing to keep in mind about exfalso/quodlibet (and consequently sequitur) is that they actually implement audio metadata properly, which means that all tags support multiple values (unless the file type prohibits it, which there aren't many that do). So, doing something like:
print qllib.AudioFile('foo.mp3')['artist']
Will not output a single string, but will output a list of strings like:
[u'The First Artist', u'The Second Artist']
The way you might use it to copy tags would be something like:
import os.path
import qllib # this is the module that comes with QLCLI
def update_tags(mp3_fn, flac_fn):
mp3 = qllib.AudioFile(mp3_fn)
flac = qllib.AudioFile(flac_fn)
# you can iterate over the tag names
# they will be the same for all file types
for tag_name in mp3:
flac[tag_name] = mp3[tag_name]
flac.write()
mp3_filenames = ['foo.mp3', 'bar.mp3', 'baz.mp3']
for mp3_fn in mp3_filenames:
flac_fn = os.path.splitext(mp3_fn)[0] + '.flac'
if os.path.getmtime(mp3_fn) != os.path.getmtime(flac_fn):
update_tags(mp3_fn, flac_fn)

I have a bash script that does exactly that, atwat-tagger. It supports flac, mp3, ogg and mp4 files.
usage: `atwat-tagger.sh inputfile.mp3 outputfile.ogg`
I know your project is already finished, but somebody who finds this page through a search engine might find it useful.

Here's some example code, a script that I wrote to copy tags between
files using Quod Libet's music format classes (not mutagen's!). To run
it, just do copytags.py src1 dest1 src2 dest2 src3 dest3, and it
will copy the tags in sec1 to dest1 (after deleting any existing tags
on dest1!), and so on. Note the blacklist, which you should tweak to
your own preference. The blacklist will not only prevent certain tags
from being copied, it will also prevent them from being clobbered in
the destination file.
To be clear, Quod Libet's format-agnostic tagging is not a feature of mutagen; it is implemented on top of mutagen. So if you want format-agnostic tagging, you need to use quodlibet.formats.MusicFile to open your files instead of mutagen.File.
Code can now be found here: https://github.com/DarwinAwardWinner/copytags
If you also want to do transcoding at the same time, use this: https://github.com/DarwinAwardWinner/transfercoder
One critical detail for me was that Quod Libet's music format classes
expect QL's configuration to be loaded, hence the config.init line in my
script. Without that, I get all sorts of errors when loading or saving
files.
I have tested this script for copying between flac, ogg, and mp3, with "standard" tags, as well as arbitrary tags. It has worked perfectly so far.
As for the reason that I didn't use QLLib, it didn't work for me. I suspect it was getting the same config-related errors as I was, but was silently ignoring them and simply failing to write tags.

You can just write a simple app with a mapping of each tag name in each format to an "abstract tag" type, and then its easy to convert from one to the other. You don't even have to know all available types - just those that you are interested in.
Seems to me like a weekend-project type of time investment, possibly less. Have fun, and I won't mind taking a peek at your implementation and even using it - if you won't mind releasing it of course :-) .

There's also tagpy, which seems to work well.

Since the other solutions have mostly fallen off the net, here is what I came up, based on the python mediafile library (python3-mediafile in Debian GNU/Linux).
#!/usr/bin/python3
import sys
from mediafile import MediaFile
src = MediaFile (sys.argv [1])
dst = MediaFile (sys.argv [2])
for field in src.fields ():
try:
setattr (dst, field, getattr (src, field))
except:
pass
dst.save ()
Usage: mediafile-mergetags srcfile dstfile
It copies (merges) all tags from srcfile into dstfile, and seems to work properly with flac, opus, mp3 and so on, including copying album art.