I am using pygraphviz to create a large number of graphs for different configurations of data. I have found that no matter what information is put in the graph the program will crash after drawing the 170th graph. There are no error messages generated the program just stops. Is there something that needs to be reset if drawing this many graphs?
I am running Python 3.7 on a Windows 10 machine, Pygraphviz 1.5, and graphviz 2.38
for graph_number in range(200):
config_graph = pygraphviz.AGraph(strict=False, directed=False, compound=True, ranksep='0.2', nodesep='0.2')
# Create Directory
if not os.path.exists('Graph'):
os.makedirs('Graph')
# Draw Graph
print('draw_' + str(graph_number))
config_graph.layout(prog = 'dot')
config_graph.draw('Graph/'+str(graph_number)+'.png')
I was able to constantly reproduce the behavior with:
Python 3.7.6 (pc064 (64bit), then also with pc032)
PyGraphviz 1.5 (that I built - available for download at [GitHub]: CristiFati/Prebuilt-Binaries - Various software built on various platforms. (under PyGraphviz, naturally).
Might also want to check [SO]: Installing pygraphviz on Windows 10 64-bit, Python 3.6 (#CristiFati's answer))
Graphviz 2.42.2 ((pc032) same as #2.)
I suspected an Undefined Behavior somewhere in the code, even if the behavior was precisely the same:
OK for 169 graphs
Crash for 170
Did some debugging (added some print(f) statements in agraph.py, and cgraph.dll (write.c)).
PyGraphviz invokes Graphviz's tools (.exes) for many operations. For that, it uses subprocess.Popen and communicates with the child process via its 3 available streams (stdin, stdout, stderr).
From the beginning I noticed that 170 * 3 = 510 (awfully close to 512 (0x200)), but didn't pay as much attention as I should have until later (mostly because the Python process (running the code below) had no more than ~150 open handles in Task Manager (TM) and also Process Explorer (PE)).
However, a bit of Googleing revealed:
[SO]: Is there a limit on number of open files in Windows (#stackprogrammer's answer) (and from here)
[MS.Learn]: _setmaxstdio (which states (emphasis is mine)):
C run-time I/O now supports up to 8,192 files open simultaneously at the low I/O level. This level includes files opened and accessed using the _open, _read, and _write family of I/O functions. By default, up to 512 files can be open simultaneously at the stream I/O level. This level includes files opened and accessed using the fopen, fgetc, and fputc family of functions. The limit of 512 open files at the stream I/O level can be increased to a maximum of 8,192 by use of the _setmaxstdio function.
[SO]: Python: Which command increases the number of open files on Windows? (#NorthCat's answer)
Below is your code that I modified for debugging and reproducing the error. It needs (for code shortness' sake, as same thing can be achieved via CTypes) the PyWin32 package (python -m pip install pywin32).
code00.py:
#!/usr/bin/env python
import os
import sys
#import time
import pygraphviz as pgv
import win32file as wfile
def handle_graph(idx, dir_name):
graph_name = "draw_{:03d}".format(idx)
graph_args = {
"name": graph_name,
"strict": False,
"directed": False,
"compound": True,
"ranksep": "0.2",
"nodesep": "0.2",
}
graph = pgv.AGraph(**graph_args)
# Draw Graph
img_base_name = graph_name + ".png"
print(" {:s}".format(img_base_name))
graph.layout(prog="dot")
img_full_name = os.path.join(dir_name, img_base_name)
graph.draw(img_full_name)
graph.close() # !!! Has NO (visible) effect, but I think it should be called anyway !!!
def main(*argv):
print("OLD max open files: {:d}".format(wfile._getmaxstdio()))
# 513 is enough for your original code (170 graphs), but you can set it up to 8192
#wfile._setmaxstdio(513) # !!! COMMENT this line to reproduce the crash !!!
print("NEW max open files: {:d}".format(wfile._getmaxstdio()))
dir_name = "Graph"
# Create Directory
if not os.path.isdir(dir_name):
os.makedirs(dir_name)
#ts_global_start = time.time()
start = 0
count = 170
#count = 1
step_sleep = 0.05
for i in range(start, start + count):
#ts_local_start = time.time()
handle_graph(i, dir_name)
#print(" Time: {:.3f}".format(time.time() - ts_local_start))
#time.sleep(step_sleep)
handle_graph(count, dir_name)
#print("Global time: {:.3f}".format(time.time() - ts_global_start - step_sleep * count))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.\n")
sys.exit(rc)
Output:
e:\Work\Dev\StackOverflow\q060876623> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" ./code00.py
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 064bit on win32
OLD max open files: 512
NEW max open files: 513
draw_000.png
draw_001.png
draw_002.png
...
draw_167.png
draw_168.png
draw_169.png
Done.
Conclusions:
Apparently, some file handles (fds) are open, although they are not "seen" by TM or PE (probably they are on a lower level). However I don't know why this happens (is it a MS UCRT bug?), but from what I am concerned, once a child process ends, its streams should be closed, but I don't know how to force it (this would be a proper fix)
Also, the behavior (crash) when attempting to write (not open) to a fd (above the limit), seems a bit strange
As a workaround, the max open fds number can be increased. Based on the following inequality: 3 * (graph_count + 1) <= max_fds, you can get an idea about the numbers. From there, if you set the limit to 8192 (I didn't test this) you should be able handle 2729 graphs (assuming that there are no additional fds opened by the code)
Side notes:
While investigating, I ran into or noticed several adjacent issues, that I tried to fix:
Graphviz:
[GitLab]: graphviz/graphviz - [Issue #1481]: MSB4018 The NativeCodeAnalysis task failed unexpectedly. (merged on 200406)
PyGraphviz:
[GitHub]: pygraphviz/pygraphviz - AGraph Graphviz handle close mechanism (merged on 200720)
There's also an issue open for this behavior (probably the same author): [GitHub]: pygraphviz/pygraphviz - Pygraphviz crashes after drawing 170 graphs
I tried you code and it generated 200 graphs with no problem (I also tried with 2000).
My suggestion is to use these versions of the packages, I installed a conda environment on mac os with python 3.7 :
graphviz 2.40.1 hefbbd9a_2
pygraphviz 1.3 py37h1de35cc_1
Related
I run conda 4.6.3 with python 3.7.2 win32. In python, when I import numpy, i see the RAM usage increase by 80MB. Since I am using multiprocessing, I wonder if this is normal and if there is anyway to avoid this RAM overhead? Please see below all the versions from relevant packages (from conda list):
python...........3.7.2 h8c8aaf0_2
mkl_fft...........1.0.10 py37h14836fe_0
mkl_random..1.0.2 py37h343c172_0
numpy...........1.15.4 py37h19fb1c0_0
numpy-base..1.15.4 py37hc3f5095_0
thanks!
You can't avoid this cost, but it's likely not as bad as it seems. The numpy libraries (a copy of C only libopenblasp, plus all the Python numpy extension modules) occupy over 60 MB on disk, and they're all going to be memory mapped into your Python process on import; adding on all the Python modules and the dynamically allocated memory involved in loading and initializing all of them, and 80 MB of increased reported RAM usage is pretty normal.
That said:
The C libraries and Python extension modules are memory mapped in, but that doesn't actually mean they occupy "real" RAM; if the code paths in a given page aren't exercised, the page will either never be loaded, or will be dropped under memory pressure (not even written to the page file, since it can always reload it from the original DLL).
On UNIX-like systems, when you fork (multiprocessing does this by default everywhere but Windows) that memory is shared between parent and worker processes in copy-on-write mode. Since the code itself is generally not written, the only cost is the page tables themselves (a tiny fraction of the memory they reference), and both parent and child will share that RAM.
Sadly, on Windows, fork isn't an option (unless you're running Ubuntu bash on Windows, in which case it's only barely Windows, effectively Linux), so you'll likely pay more of the memory costs in each process. But even there, libopenblasp, the C library backing large parts of numpy, will be remapped per process, but the OS should properly share that read-only memory across processes (and large parts, if not all, of the Python extension modules as well).
Basically, until this actually causes a problem (and it's unlikely to do so), don't worry about it.
[NumPy]: NumPy
is the fundamental package for scientific computing with Python.
It is a big package, designed to work with large datasets and optimized (primarily) for speed. If you look in its __init__.py (which gets executed when importing it (e.g.: import numpy)), you'll notice that it imports lots of items (packages / modules):
Those items themselves, may import others
Some of them are extension modules (.pyds (.dlls) or .sos) which get loaded into the current process (their dependencies as well)
I've prepared a demo.
code.py:
#!/usr/bin/env python3
import sys
import os
import psutil
#import pprint
def main():
display_text = "This {:s} screenshot was taken. Press <Enter> to continue ... "
pid = os.getpid()
print("Pid: {:d}\n".format(pid))
p = psutil.Process(pid=pid)
mod_names0 = set(k for k in sys.modules)
mi0 = p.memory_info()
input(display_text.format("first"))
import numpy
input(display_text.format("second"))
mi1 = p.memory_info()
for idx, mi in enumerate([mi0, mi1], start=1):
print("\nMemory info ({:d}): {:}".format(idx, mi))
print("\nExtra modules imported by `{:s}` :".format(numpy.__name__))
print(sorted(set(k for k in sys.modules) - mod_names0))
#pprint.pprint({k: v for k, v in sys.modules.items() if k not in mod_names0})
print("\nDone.")
if __name__ == "__main__":
print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
main()
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q054675983]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
Pid: 27160
This first screenshot was taken. Press <Enter> to continue ...
This second screenshot was taken. Press <Enter> to continue ...
Memory info (1): pmem(rss=15491072, vms=8458240, num_page_faults=4149, peak_wset=15495168, wset=15491072, peak_paged_pool=181160, paged_pool=180984, peak_nonpaged_pool=13720, nonpaged_pool=13576, pagefile=8458240, peak_pagefile=8458240, private=8458240)
Memory info (2): pmem(rss=27156480, vms=253882368, num_page_faults=7283, peak_wset=27205632, wset=27156480, peak_paged_pool=272160, paged_pool=272160, peak_nonpaged_pool=21640, nonpaged_pool=21056, pagefile=253882368, peak_pagefile=253972480, private=253882368)
Extra modules imported by `numpy` :
['_ast', '_bisect', '_blake2', '_compat_pickle', '_ctypes', '_decimal', '_hashlib', '_pickle', '_random', '_sha3', '_string', '_struct', 'argparse', 'ast', 'atexit', 'bisect', 'copy', 'ctypes', 'ctypes._endian', 'cython_runtime', 'decimal', 'difflib', 'gc', 'gettext', 'hashlib', 'logging', 'mtrand', 'numbers', 'numpy', 'numpy.__config__', 'numpy._distributor_init', 'numpy._globals', 'numpy._import_tools', 'numpy.add_newdocs', 'numpy.compat', 'numpy.compat._inspect', 'numpy.compat.py3k', 'numpy.core', 'numpy.core._internal', 'numpy.core._methods', 'numpy.core._multiarray_tests', 'numpy.core.arrayprint', 'numpy.core.defchararray', 'numpy.core.einsumfunc', 'numpy.core.fromnumeric', 'numpy.core.function_base', 'numpy.core.getlimits', 'numpy.core.info', 'numpy.core.machar', 'numpy.core.memmap', 'numpy.core.multiarray', 'numpy.core.numeric', 'numpy.core.numerictypes', 'numpy.core.records', 'numpy.core.shape_base', 'numpy.core.umath', 'numpy.ctypeslib', 'numpy.fft', 'numpy.fft.fftpack', 'numpy.fft.fftpack_lite', 'numpy.fft.helper', 'numpy.fft.info', 'numpy.lib', 'numpy.lib._datasource', 'numpy.lib._iotools', 'numpy.lib._version', 'numpy.lib.arraypad', 'numpy.lib.arraysetops', 'numpy.lib.arrayterator', 'numpy.lib.financial', 'numpy.lib.format', 'numpy.lib.function_base', 'numpy.lib.histograms', 'numpy.lib.index_tricks', 'numpy.lib.info', 'numpy.lib.mixins', 'numpy.lib.nanfunctions', 'numpy.lib.npyio', 'numpy.lib.polynomial', 'numpy.lib.scimath', 'numpy.lib.shape_base', 'numpy.lib.stride_tricks', 'numpy.lib.twodim_base', 'numpy.lib.type_check', 'numpy.lib.ufunclike', 'numpy.lib.utils', 'numpy.linalg', 'numpy.linalg._umath_linalg', 'numpy.linalg.info', 'numpy.linalg.lapack_lite', 'numpy.linalg.linalg', 'numpy.ma', 'numpy.ma.core', 'numpy.ma.extras', 'numpy.matrixlib', 'numpy.matrixlib.defmatrix', 'numpy.polynomial', 'numpy.polynomial._polybase', 'numpy.polynomial.chebyshev', 'numpy.polynomial.hermite', 'numpy.polynomial.hermite_e', 'numpy.polynomial.laguerre', 'numpy.polynomial.legendre', 'numpy.polynomial.polynomial', 'numpy.polynomial.polyutils', 'numpy.random', 'numpy.random.info', 'numpy.random.mtrand', 'numpy.testing', 'numpy.testing._private', 'numpy.testing._private.decorators', 'numpy.testing._private.nosetester', 'numpy.testing._private.pytesttester', 'numpy.testing._private.utils', 'numpy.version', 'pathlib', 'pickle', 'pprint', 'random', 'string', 'struct', 'tempfile', 'textwrap', 'unittest', 'unittest.case', 'unittest.loader', 'unittest.main', 'unittest.result', 'unittest.runner', 'unittest.signals', 'unittest.suite', 'unittest.util', 'urllib', 'urllib.parse']
Done.
And the (before and after import) screenshots ([MS.Docs]: Process Explorer):
As a personal remark, I think that ~80 MiB (or whatever the exact amount is), is more than decent for the current "era", which is characterized by ridiculously high amounts of hardware resources, especially in the memories area. Besides, that would probably insignificant, compared to the amount required by the arrays themselves. If it's not the case, you should probably consider moving away from numpy.
There could be a way to reduce the memory footprint, by selectively importing only the modules containing the features that you need (my personal advice is against it), and thus going around __init__.py:
You'd have to be an expert in numpy's internals
Modules must be imported "manually" (by file name), using [Python 3]: importlib - The implementation of import (or alternatives)
Their dependents will be imported / loaded as well (and because of this, I don't know how much free memory you'd gain)
I would like to use Elaphe to generate barcodes.
I am working on a 64-bit windows machine. This is on Windows 7, Python 2.7, I have Elaphe 0.6.0 and Ghostscript 9.10 installed.
When I run the simple example usage, nothing seems to be happening. The barcode does not show up. When I execute _.show(), it hangs but nothing shows up. I have to do a KeyboardInterrupt to get back to the prompt. What viewer is supposed to launch when I do _.show()? I however see a gswin32.exe process in the Windows Task Manager.
Please refer to my Python traceback at http://dpaste.com/hold/1653582/
Is there a way to see the PS code generated? How can I troubleshoot?
Please help.
The object returned by elaphe.barcode is an EpsImageFile (where EPS means Encapsulated PostScript), but after calling barcode it hasn't yet run Ghostscript to convert the code into a bitmap image.
You can dump out the code that it has generated by looking at the fp attribute - there's a lot of it, because it embeds the full PS library code for all the different barcode types it supports. So it's probably best to write it out to a file:
b = el.barcode('qr', 'slamacow')
with open('code.eps') as outfile:
outfile.write(b.fp.getvalue()) # fp is a StringIO instance
In the file you'll see something like this:
%!PS-Adobe-2.0
%%Pages: (attend)
%%Creator: Elaphe powered by barcode.ps
%%BoundingBox: 0 0 42 42
%%LanguageLevel: 2
%%EndComments
% --BEGIN RESOURCE preamble--
... A whole lot of included library ...
% --END ENCODER hibccodablockf--
gsave
0 0 moveto
1.000000 1.000000 scale
<74686973206973206d792064617461>
<>
/qrcode /uk.co.terryburton.bwipp findresource exec
grestore
showpage
If you want to see how PIL or pillow runs Ghostscript so you can try it yourself at the commandline, the key part from the PIL/pillow code is this (from site-packages/PIL/EpsImagePlugin.py, line 84):
# Build ghostscript command
command = ["gs",
"-q", # quiet mode
"-g%dx%d" % size, # set output geometry (pixels)
"-r%d" % (72*scale), # set input DPI (dots per inch)
"-dNOPAUSE -dSAFER", # don't pause between pages, safe mode
"-sDEVICE=ppmraw", # ppm driver
"-sOutputFile=%s" % outfile, # output file
"-c", "%d %d translate" % (-bbox[0], -bbox[1]),
# adjust for image origin
"-f", infile, # input file
]
But on Windows the gs command will be replaced with the path to the executable.
I've made simple code editor using wxPython. File size (python files) is 1.3 KB. But when I create executable using PyInstaller, I get 30 MB file! Is there a way to decrease file size? Btw, I am not importing whole wx library, only components I need (ex from wx import Frame).
Using Linux, Fedora 18 64bit.
wxPython is a big library so when you create an executable, they tend to end up being between 20 and 30 MB. Also note that Python itself is kind of bulky because Python is an interpreted language. So you are also including the Python interpreter when you create the exe.
With py2exe, I have gotten the executable below 10 MB, but it's a pain and doesn't work for all projects. It really depends on what else you are using. You can read about my adventures with py2exe here.
The other way to get it smaller is to use a compression program. That sometimes works and sometimes doesn't.
You can also tell most of these binary creation tools to exclude items. You can try that too.
I shipped a fairly simple wxPython app and it ended up being ~9.8MB.
If you utilize the ArchiveViewer.py script that's part of PyInstaller you can determine what's taking up so much space.
This was with python 2.7.5, without UPX, and excluding these modules:
excludesPassedToAnalysis = ['ssl',
'_ssl',
# coverage uses _socket. :(
#'_socket',
'select',
'pywin',
'unittest',
'win32ui',
'bz2',
'doctest',
'os2emxpath',
'servicemanager',
'xml.parsers.expat',
'sitecustomize',
'tarflie',
'email',
'urllib',
'urllib2',
# This exclude isn't optional in order to get pubsub working
# correctly in wxPython 2.9.3 or later.
'wx.lib.pubsub.autosetuppubsubv1']
# These are removed from a.pure after the Analysis object is created.
excludeEncodings = \
['encodings.base_64_codec',
'encodings.big5',
'encodings.big5hkscs',
'encodings.bz2_codec',
'encodings.cp037',
'encodings.cp1006',
'encodings.cp1026',
'encodings.cp1140',
'encodings.cp1258',
'encodings.cp424',
'encodings.cp437',
'encodings.cp500',
'encodings.cp720',
'encodings.cp737',
'encodings.cp775',
'encodings.cp850',
'encodings.cp852',
'encodings.cp855',
'encodings.cp856',
'encodings.cp857',
'encodings.cp858',
'encodings.cp860',
'encodings.cp861',
'encodings.cp862',
'encodings.cp863',
'encodings.cp864',
'encodings.cp865',
'encodings.cp866',
'encodings.cp869',
'encodings.cp874',
'encodings.cp875',
'encodings.cp932',
'encodings.cp949',
'encodings.cp950',
'encodings.euc_jis_2004',
'encodings.euc_jisx0213',
'encodings.euc_jp',
'encodings.euc_kr',
'encodings.gb18030',
'encodings.gb2312',
'encodings.gbk',
'encodings.hex_codec',
'encodings.hp_roman8',
'encodings.hz',
'encodings.iso2022_jp',
'encodings.iso2022_jp_1',
'encodings.iso2022_jp_2',
'encodings.iso2022_jp_2004',
'encodings.iso2022_jp_3',
'encodings.iso2022_jp_ext',
'encodings.iso2022_kr',
'encodings.iso8859_10',
'encodings.iso8859_11',
'encodings.iso8859_13',
'encodings.iso8859_14',
'encodings.iso8859_15',
'encodings.iso8859_16',
'encodings.iso8859_2',
'encodings.iso8859_3',
'encodings.iso8859_4',
'encodings.iso8859_5',
'encodings.iso8859_6',
'encodings.iso8859_7',
'encodings.iso8859_8',
'encodings.iso8859_9',
'encodings.johab',
'encodings.koi8_r',
'encodings.koi8_u',
'encodings.mac_arabic',
'encodings.mac_centeuro',
'encodings.mac_croatian',
'encodings.mac_cyrillic',
'encodings.mac_farsi',
'encodings.mac_greek',
'encodings.mac_iceland',
'encodings.mac_latin2',
'encodings.mac_roman',
'encodings.mac_romanian',
'encodings.mac_turkish',
'encodings.mbcs',
'encodings.palmos',
'encodings.ptcp154',
'encodings.quopri_codec',
'encodings.raw_unicode_escape',
'encodings.rot_13',
'encodings.shift_jis',
'encodings.shift_jis_2004',
'encodings.shift_jisx0213',
'encodings.string_escape',
'encodings.tis_620',
'encodings.undefined',
'encodings.utf_32',
'encodings.utf_32_be',
'encodings.utf_32_le',
'encodings.utf_7',
'encodings.uu_codec',
'encodings.zlib_codec',]
If one wishes to beep the speaker on Windows, Python 2 apparently provides a useful function: winsound.Beep(). The neat thing about this function is that it takes arguments specifying the exact frequency and duration of the beep. This is exactly what I want to do, except that I don't use Windows. So...
What are the nearest equivalents of winsound.Beep() for Linux and OS X [edit: macOS], bringing in as few dependencies as possible?
Please note that I want to be able to beep the speaker directly, not to play a sound file. Also, I need to be able to control the frequency and duration of the beep, so curses.beep() and print '\a' won't do. Lastly, I am aware that PyGame provides extensive sound capabilities, but given that I don't require any of PyGame's other functionality, that would seem like using a sledgehammer to crack a nut (and anyway, I'm trying to do away with dependencies as far as possible).
[Edited on 9 Feb 2023 to reflect the fact that OS X was renamed macOS a few years after this question was asked]
winsound is only for windows and I could not find any cross platform way to do this, other than print "/a". However, you cannot set the frequency and duration with this.
However, you can try the os.system command to do the same with the system command beep. Here is a snippet, which defines the function playsound in a platform independent way
try:
import winsound
except ImportError:
import os
def playsound(frequency,duration):
#apt-get install beep
os.system('beep -f %s -l %s' % (frequency,duration))
else:
def playsound(frequency,duration):
winsound.Beep(frequency,duration)
For more info, look at this blog
EDIT: You will need to install the beep package on linux to run the beep command. You can install by giving the command
sudo apt-get install beep
I found a potential solution here:
http://bytes.com/topic/python/answers/25217-beeping-under-linux
It involves writing directly to /dev/audio. Not sure how portable it is or if it even works at all - i'm not on a linux machine atm.
def beep(frequency, amplitude, duration):
sample = 8000
half_period = int(sample/frequency/2)
beep = chr(amplitude)*half_period+chr(0)*half_period
beep *= int(duration*frequency)
audio = file('/dev/audio', 'wb')
audio.write(beep)
audio.close()
This works on mac:
import numpy as np
import simpleaudio as sa
def sound(x,z):
frequency = x # Our played note will be 440 Hz
fs = 44100 # 44100 samples per second
seconds = z # Note duration of 3 seconds
# Generate array with seconds*sample_rate steps, ranging between 0 and seconds
t = np.linspace(0, seconds, seconds * fs, False)
# Generate a 440 Hz sine wave
note = np.sin(frequency * t * 2 * np.pi)
# Ensure that highest value is in 16-bit range
audio = note * (2**15 - 1) / np.max(np.abs(note))
# Convert to 16-bit data
audio = audio.astype(np.int16)
# Start playback
play_obj = sa.play_buffer(audio, 1, 2, fs)
# Wait for playback to finish before exiting
play_obj.wait_done()
sound(300,2)
sound(200,1)
The most light-weight cross-platform layer I can see is "PortAudio". This is used by R for instance in their package to wrap platform-specific driver calls into simple play/record of digitized waveforms as an array.
The good folk at M.I.T. produce a Python binding for this, but you will need to include the compiled .dll/.so for this to work. http://people.csail.mit.edu/hubert/pyaudio/
( libao is similar by Xiph the makers of Ogg/Vorbis , a wrapper pyao exists but this seems less widely used )
SoX is an excellent set of cross-platform tools with much more functionality for format conversion and reading files etc..
Using ctypes to make calls from Python to a driver is feasible but very messy, even the simplest legacy WinMM.
I've found 3 methods for Linux:
new method using the Linux evdev API, works with any user in the input group (example source code)
old method using fcntl and /dev/console (requires root priviledges) (example source code)
invoke the beep command directly with subprocess or os.system (slower and must be installed in the system).
See also my tone() function here with all the alternatives.
I'm playing around with pygame, and one thing I'd like to do is reduce the number of frames per second when the computer is on battery power (to lower the CPU usage and extend battery life).
How can I detect, from Python, whether the computer is currently on battery power?
I'm using Python 3.1 on Windows.
If you want to do it without win32api, you can use the built-in ctypes module. I usually run CPython without win32api, so I kinda like these solutions.
It's a tiny bit more work for GetSystemPowerStatus() because you have to define the SYSTEM_POWER_STATUS structure, but not bad.
# Get power status of the system using ctypes to call GetSystemPowerStatus
import ctypes
from ctypes import wintypes
class SYSTEM_POWER_STATUS(ctypes.Structure):
_fields_ = [
('ACLineStatus', wintypes.BYTE),
('BatteryFlag', wintypes.BYTE),
('BatteryLifePercent', wintypes.BYTE),
('Reserved1', wintypes.BYTE),
('BatteryLifeTime', wintypes.DWORD),
('BatteryFullLifeTime', wintypes.DWORD),
]
SYSTEM_POWER_STATUS_P = ctypes.POINTER(SYSTEM_POWER_STATUS)
GetSystemPowerStatus = ctypes.windll.kernel32.GetSystemPowerStatus
GetSystemPowerStatus.argtypes = [SYSTEM_POWER_STATUS_P]
GetSystemPowerStatus.restype = wintypes.BOOL
status = SYSTEM_POWER_STATUS()
if not GetSystemPowerStatus(ctypes.pointer(status)):
raise ctypes.WinError()
print('ACLineStatus', status.ACLineStatus)
print('BatteryFlag', status.BatteryFlag)
print('BatteryLifePercent', status.BatteryLifePercent)
print('BatteryLifeTime', status.BatteryLifeTime)
print('BatteryFullLifeTime', status.BatteryFullLifeTime)
On my system that prints this (basically meaning "desktop, plugged in"):
ACLineStatus 1
BatteryFlag -128
BatteryLifePercent -1
BatteryLifeTime 4294967295
BatteryFullLifeTime 4294967295
The most reliable way to retrieve this information in C is by using GetSystemPowerStatus. If no battery is present ACLineStatus will be set to 128. psutil exposes this information under Linux, Windows and FreeBSD, so to check if battery is present you can do this
>>> import psutil
>>> has_battery = psutil.sensors_battery() is not None
If a battery is present and you want to know whether the power cable is plugged in you can do this:
>>> import psutil
>>> psutil.sensors_battery()
sbattery(percent=99, secsleft=20308, power_plugged=True)
>>> psutil.sensors_battery().power_plugged
True
>>>
It is easy, all you have to do is to call Windows API function GetSystemPowerStatus from Python, probably by importing win32api module.
EDIT: GetSystemPowerStatus() is not yet implemented in win32api as of build 219 (2014-05-04).
A simple method for cross platform power status indication is the 'power' module which you can install with pip
import power
ans = power.PowerManagement().get_providing_power_source_type()
if not ans:
print "plugged into wall socket"
else:
print "on battery"
You can install acpi.From wikipedia
In a computer, the Advanced Configuration and Power Interface provides an open standard that operating systems can use to discover and configure computer hardware components, to perform power management by putting unused components to sleep, and to perform status monitoring.
Then use the subprocess module in python
import subprocess
cmd = 'acpi -b'
# for python 3.7+
p = subprocess.run(cmd.split(), shell=True, capture_output=True)
battery_info, error = p.stdout.decode(), p.stderr.decode()
# for python3.x (x<6)
battery_info = subprocess.check_output(cmd.split(), shell=True).decode('utf-8')
print (battery_info)
[SO]: In Python, how can I detect whether the computer is on battery power? (#BenHoyt's answer) is portable and doesn't require extra packages, but it's negatively impacted (until Python v3.12) by a CTypes (WinTypes) bug.
More details about the bug (and fix, workaround): [SO]: Why ctypes.wintypes.BYTE is signed, but native windows BYTE is unsigned? (#CristiFati's answer).
Anyway, I submitted [GitHub]: mhammond/pywin32 - Add GetSystemPowerStatus wrapper for GetSystemPowerStatus function to be available in Win32API.
Building win32api.pyd locally and overwriting the one from site-packages directory (as I mentioned in the Test section), yields:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q006153860]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]>
[prompt]> :: Power cable unplugged
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test1_pw32\Scripts\python.exe" -c "import win32api as wapi;from pprint import pprint as pp;pp(wapi.GetSystemPowerStatus(), sort_dicts=0);print(\"\nDone.\n\")"
{'ACLineStatus': 0,
'BatteryFlag': 1,
'BatteryLifePercent': 99,
'SystemStatusFlag': 0,
'BatteryLifeTime': 13094,
'BatteryFullLifeTime': 4294967295}
Done.
[prompt]>
[prompt]> :: Plug in power cable
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test1_pw32\Scripts\python.exe" -c "import win32api as wapi;from pprint import pprint as pp;pp(wapi.GetSystemPowerStatus(), sort_dicts=0);print(\"\nDone.\n\")"
{'ACLineStatus': 1,
'BatteryFlag': 1,
'BatteryLifePercent': 100,
'SystemStatusFlag': 0,
'BatteryLifeTime': 4294967295,
'BatteryFullLifeTime': 4294967295}
Done.
Check [SO]: How to change username of job in print queue using python & win32print (#CristiFati's answer) (at the end) for possible ways to benefit from the (above) patch.
Worth mentioning (if [SO]: In Python, how can I detect whether the computer is on battery power? (#GiampaoloRodolà's answer) is not clear enough about it) that [PyPI]: psutil also uses GetSystemPowerStatus in order to retrieve battery information.