Develop Reference

Running minizinc from python with time-limit results in keyboard interrupt

I am solving graph coloring problem using minizinc.
The process run looks loke
mz_output = subprocess.run(["minizinc", "--time-limit", "10000", "--no-intermediate", "-p 4", "graph_coloring.mzn", "data.dzn"],
shell=False, stdout=subprocess.PIPE).stdout.decode('utf-8')
It works fine for some graphs, but for graph with 1000 nodes and ~250000 edges it throws KeyboardInterrupt:
Traceback (most recent call last):
File "submit.py", line 458, in <module>
main(parser.parse_args())
File "submit.py", line 390, in main
results = compute(metadata, args.override)
File "submit.py", line 181, in compute
submission = output(problem.input_file, solver_file)
File "submit.py", line 222, in output
solution = pkg.solve_it(load_input_data(input_file))
File "D:\Coursera\Discrete\Coloring\coloring\solver.py", line 73, in solve_it
shell=False, stdout=subprocess.PIPE).stdout.decode('utf-8')
File "D:\programs\Anaconda\lib\subprocess.py", line 490, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "D:\programs\Anaconda\lib\subprocess.py", line 951, in communicate
stdout = self.stdout.read()
KeyboardInterrupt
I supposed it is connected with size of the graph but it doesn't appear so. First, it works fine in minizinc IDE and gives intermediate solutions in the first seconds. Second, it is expected some time errors or memory errors in case of large size, while KeyboardInterrupt is confusing. How I can resolve that issue?

Python code runs in interactive prompt but not as script

I am trying to interface with an Ocean optics spectrometer using seabreeze (seabreeze) and pyseabreeze (pyseabreeze).
If I enter python from the command prompt (cmd) in windows and enter the following code line by line, it works. However, if I put it in a script (spec_test.py) and try running it from cmd.exe by using 'python spec_test.py', it will not work.
I have searched around a lot for similar problems but none seem to cover the issue I'm having. I am running Windows 7 64 bit, python 3.5.2 installed using anaconda. I also had to install pyusb and libusb to use pyseabreeze.
The code:
import seabreeze
seabreeze.use('pyseabreeze')
import seabreeze.spectrometers as sb
devs = sb.list_devices()
print(devs)
spec = sb.Spectrometer(devs[0])
print(spec.model)
and the error message I get:
[<SeaBreezeDevice USB2000PLUS:FLMS02379>]
Traceback (most recent call last):
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\common.py", line 14, in decorated_func
return func(*args, **kwargs)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\spectrometer.py", line 46, in open
self.open_device(device.handle)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\communication.py", line 37, in open_device
device.set_configuration()
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\core.py", line 869, in set_configuration
self._ctx.managed_set_configuration(self, configuration)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\core.py", line 148, in managed_set_configuration
self.backend.set_configuration(self.handle, cfg.bConfigurationValue)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\backend\libusb0.py", line 493, in set_configuration
_check(_lib.usb_set_configuration(dev_handle, config_value))
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\backend\libusb0.py", line 431, in _check
raise USBError(errmsg, ret)
usb.core.USBError: [Errno None] b'libusb0-dll:err [set_configuration] could not set config 1: win error: The parameter is incorrect.\r\n'
Traceback (most recent call last):
File "<ipython-input-9-ead886eb3666>", line 1, in <module>
runfile('C:/Users/Raman Lab/Python code/Spectrometers/spec_testing.py', wdir='C:/Users/Raman Lab/Python code/Spectrometers')
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 866, in runfile
execfile(filename, namespace)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Raman Lab/Python code/Spectrometers/spec_testing.py", line 7, in <module>
spec = sb.Spectrometer(devs[0])
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\spectrometers.py", line 62, in __init__
self._open_device(device)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\spectrometers.py", line 90, in _open_device
lib.device_open(self._dev)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\wrapper.py", line 81, in device_open
return device.interface.open(device)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\common.py", line 23, in decorated_func
raise SeaBreezeError(msg)
SeaBreezeError: An error occured during opening.
Thanks for any help!
Edit:
For some reason I figured this out, thought it might help describe the issue. If I enter interactive python from cmd.exe and paste the code instead of typing it in manually, I get the same error. This makes me think (most likely I'm wrong) that it is somehow tied up with speed of imports. I tried adding a sleep for a few seconds in between lines 3 and 5, to simulate what happens when I am typing in the prompt, but that didn't help. I hope this was descriptive enough.

Your initial guess was not far off. I had exactly the same problem, however, on Windows 7 32bit with Python 2.7. The example code gchaks linked to was running fine.
When I tried executing the script several times in quick succession, the error message changed and pointed at a empty devices list. I added a delay of 2 seconds after
devs = sb.list_devices()
and that solved the problem. Your code was running just fine on a Linux machine btw.
Another hint: If your code should run once, but then you run into another error message, make sure you close the device connection or dis- and reconnect the spectrometer.

Python Dropbox API - Invalid "cursor" parameter

I've been migrating some Python 2.7.11 code to 3.5.1 after running into trouble with unicode. This was the last straw - since I started using the venv module there's no reason to be on 2.7 just because someone doesn't like 3!
The problem occurs while trying to run a one-way sync (ie. downloading changes only).
Here is the full error message, paths shortened:
Traceback (most recent call last):
File "%SCRIPT%.py", line 209, in <module>
updated_schedules = dbx_sync.One_Way_Sync(config['Dropbox Parameters']['Directory'], config['Dropbox Parameters']['Base Path'])
File "%COMMON_PATH%\modules\dropbox_sync_schedules.py", line 62, in One_Way_Sync
result = client.delta(cursor, base_path)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\client.py", line 569, in delta
return self.rest_client.POST(url, params, headers)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\rest.py", line 322, in POST
return cls.IMPL.POST(*n, **kw)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\rest.py", line 260, in POST
is_json_request=is_json_request)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\rest.py", line 235, in request
raise ErrorResponse(r, r.read())
dropbox.rest.ErrorResponse: [400] 'Invalid "cursor" parameter: u"b\'\'"'
Searching for "invalid cursor parameter" wasn't any help, so I thought I'd come here.

u"b\'\'" is the key here. I just couldn't understand how that representation had ended up being sent as a string.
The issue was in reading the old cursor from a file (which for this example is empty): in Python 2 I had opened the file in mode rb - in Python 3 just r is all that's required, and everything works.
Hurrah!

multiprocessing in iPython: too many files open when I'm not opening any files?

I have an IPython notebook that I was running as multi-threaded and have changed over to try and run on multiple processors. There is a substantial amount of code that I won't quote here unless someone thinks it would be helpful to look at a specific portion of it. The gist of it is that I'm trying to wrap a load balancer around PyOpenCL so I need to feed multiple independent queues-- partly to solve a specific problem (particle simulation) and partly as a learning exercise.
At no point am I intentionally opening any files, yet it's crashing with this error:
Traceback (most recent call last):
File "//anaconda/lib/python3.4/site-packages/IPython/core/interactiveshell.py", line 3035, in run_code
File "<ipython-input-23-d29c87aba720>", line 1, in <module>
co.convergeStep(mags[3])
File "<ipython-input-17-0eac43f0bbbf>", line 62, in convergeStep
evt=self.nlb.qTask(evtList,(b,k),b.forces,k,out=b.forceAccum,out2=k.forceAccum,sum=True)
File "<ipython-input-13-0f26b8ae660e>", line 36, in qTask
t=StagingThread(waitlist,c,self.pool)
File "<ipython-input-10-549b980aea87>", line 7, in __init__
self.killme=mp.Event()
File "//anaconda/lib/python3.4/multiprocessing/context.py", line 91, in Event
File "//anaconda/lib/python3.4/multiprocessing/synchronize.py", line 336, in __init__
File "//anaconda/lib/python3.4/multiprocessing/context.py", line 76, in Condition
File "//anaconda/lib/python3.4/multiprocessing/synchronize.py", line 215, in __init__
File "//anaconda/lib/python3.4/multiprocessing/context.py", line 81, in Semaphore
File "//anaconda/lib/python3.4/multiprocessing/synchronize.py", line 127, in __init__
File "//anaconda/lib/python3.4/multiprocessing/synchronize.py", line 60, in
__init__ OSError: [Errno 24] Too many open files
Is there something opening files in the background that I'm not considering?
It looks like I have 11 processes spawned at the time of failure. I'm running under OS X 10.10.3, Python 3.4.3, IPython 3.1. I am not using multiprocessing.Pool (at least not intentionally). It looks like the failure is happening in Event? There are other exceptions that occur while handling this one, but they're mostly too many files errors and then some traceback failures and other things that look like they're a result of one of the above...

Parallel Python - RuntimeError: Communication pipe read error

I'm using parallel python to run multiple dynamic simulations using a module called OrcFxAPI. The program works perfectly if it is run as a python program on my machine however if I convert it to an exe file using py2exe and then run I get the following error:
Traceback (most recent call last):
File "Analysis.pyc", Line 500, in multiprocessor
File "pp.pyc", Line 342, in __init__
File "pp.pyc", Line 506, in set_ncpus
File "pp.pyc", Line 140, in __init__
File "pp.pyc", Line 152, in start
File "pptransport.pyc", Line 140, in receive
RuntimeError: Communication pipe read error
It is failing at this line in my program:
job_server = pp.Server(ppservers=ppservers)
but I think it might be something to do with the path used to import the OrcFxAPI module when submitting the job:
job = job_server.submit(max_seastate, (gui_vars, case_list, case, line_info, output_vars), (), ("OrcFxAPI",), callback=finished, callbackargs=(case_no, no_of_cases,))