Develop Reference

Compiling Numpy from Source with Intel MKL/Intel C - python

I've been trying to compile numpy from source using Intel MKL and the Intel C/C++ compilers. I have a student license which gives me access to these compilers/MKL. It appears that I built numpy but when I run numpy.test('full') I get these errors:
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1565, in close
fp.close()
OSError: [Errno 28] No space left on device
EException ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba36cd02e8>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1560, in close
self.fp.flush()
OSError: [Errno 28] No space left on device
/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed file <_io.BufferedWriter name='/tmp/tmpgl9796ik.npz'>
outcome.errors.clear()
EException ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba3727f550>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1565, in close
fp.close()
OSError: [Errno 28] No space left on device
EEException ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba329a76a0>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1468, in close
pos1 = self.fp.tell()
File "/usr/lib/python3.4/tempfile.py", line 399, in func_wrapper
return func(*args, **kwargs)
ValueError: I/O operation on closed file
EEEException ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba329a78d0>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1468, in close
pos1 = self.fp.tell()
ValueError: I/O operation on closed file
EF....E.EEException ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba329a7fd0>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1565, in close
fp.close()
OSError: [Errno 28] No space left on device
Exception ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba329a73c8>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1565, in close
fp.close()
OSError: [Errno 28] No space left on device
Exception ignored in: <bound method ZipFile.__del__ of <zipfile.ZipFile object at 0x7fba329a7da0>>
Traceback (most recent call last):
File "/usr/lib/python3.4/zipfile.py", line 1457, in __del__
self.close()
File "/usr/lib/python3.4/zipfile.py", line 1565, in close
fp.close()
OSError: [Errno 28] No space left on device
warnings.warn("Warning: converting a masked element to nan.")
======================================================================
ERROR: test_big_arrays (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/testing/decorators.py", line 146, in skipper_func
return f(*args, **kwargs)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 199, in test_big_arrays
np.savez(tmp, a=a)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: 2147583648 requested and 2021494704 written
======================================================================
ERROR: test_closing_fid (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 282, in test_closing_fid
np.savez(fp, data='LOVELY LOAD')
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_closing_zipfile_after_load (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 309, in test_closing_zipfile_after_load
np.savez(tmp, lab='place holder')
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_format_2_0 (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 538, in write_array
version)
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 319, in _write_array_header
fp.write(header)
OSError: [Errno 28] No space left on device
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 166, in test_format_2_0
self.check_roundtrips(a)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 111, in check_roundtrips
self.roundtrip(a)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 179, in roundtrip
RoundtripTest.roundtrip(self, np.savez, *args, **kwargs)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 92, in roundtrip
save_func(target_file, *arr, **save_kwds)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_mmap (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 92, in roundtrip
save_func(target_file, *arr, **save_kwds)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/testing/decorators.py", line 215, in knownfailer
return f(*args, **kwargs)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 152, in test_mmap
self.roundtrip(a, file_on_disk=True, load_kwds={'mmap_mode': 'r'})
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 179, in roundtrip
RoundtripTest.roundtrip(self, np.savez, *args, **kwargs)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 107, in roundtrip
if not isinstance(arr_reloaded, np.lib.npyio.NpzFile):
UnboundLocalError: local variable 'arr_reloaded' referenced before assignment
======================================================================
ERROR: test_multiple_arrays (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 208, in test_multiple_arrays
self.roundtrip(a, b)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 179, in roundtrip
RoundtripTest.roundtrip(self, np.savez, *args, **kwargs)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 92, in roundtrip
save_func(target_file, *arr, **save_kwds)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_named_arrays (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 214, in test_named_arrays
np.savez(c, file_a=a, file_b=b)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_not_closing_opened_fid (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 255, in test_not_closing_opened_fid
np.savez(fp, data='LOVELY LOAD')
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_record (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 159, in test_record
self.check_roundtrips(a)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 111, in check_roundtrips
self.roundtrip(a)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 179, in roundtrip
RoundtripTest.roundtrip(self, np.savez, *args, **kwargs)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 92, in roundtrip
save_func(target_file, *arr, **save_kwds)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_io.test_gzip_loadtxt
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 1684, in test_gzip_loadtxt
os.write(f, s.read())
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_io.test_npzfile_dict
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 1708, in test_npzfile_dict
np.savez(s, x=x, y=y)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_io.test_load_refcount
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 597, in _savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/python3.4/site-packages/numpy/lib/format.py", line 562, in write_array
array.tofile(fp)
OSError: first argument must be a string or open file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 1735, in test_load_refcount
np.savez(f, [1, 2, 3])
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 534, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3.4/site-packages/numpy/lib/npyio.py", line 603, in _savez
fid.close()
OSError: [Errno 28] No space left on device
======================================================================
ERROR: test_fromtextfile (test_mrecords.TestMRecordsImport)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/ma/tests/test_mrecords.py", line 489, in test_fromtextfile
os.write(tmp_fd, fcontent)
OSError: [Errno 28] No space left on device
======================================================================
FAIL: test_savez_filename_clashes (test_io.TestSavezLoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/numpy/lib/tests/test_io.py", line 245, in test_savez_filename_clashes
raise AssertionError(errors)
AssertionError: [OSError(28, 'No space left on device'), OSError(28, 'No space left on device'), OSError(28, 'No space left on device')]
----------------------------------------------------------------------
Ran 5762 tests in 122.291s
FAILED (KNOWNFAIL=6, SKIP=17, errors=13, failures=1)
Any idea what's going on or where it might be going wrong?

Related

Traceback (most recent call last):
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 399, in wrapper
return cache[key]
~~~~~^^^^^
KeyError: (('/proc',), frozenset())
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 285, in
set_scputimes_ntuple("/proc")
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 401, in wrapper
ret = cache[key] = fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
with open_binary('%s/stat' % procfs_path) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/psutil/_common.py", line 728, in open_binary
return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/proc/stat'
Traceback (most recent call last):
File "/data/data/com.termux/files/home/YouTube-Viewer/youtube_viewer.py", line 33, in
from fake_headers import Headers, browsers
ModuleNotFoundError: No module named 'fake_headers'
What's problem ? I don't know

I am using the most lightweight/simple dask multiprocessing which is the non-cluster local Client:
from distributed import Client
client = Client()
Even so: the first instance of invoking dask.bag.compute() results in the following:
Connected to pydev debugger (build 191.7141.48)
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 383, in _on_run
r = self.sock.recv(1024)
OSError: [Errno 9] Bad file descriptor
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 383, in _on_run
r = self.sock.recv(1024)
OSError: [Errno 9] Bad file descriptor
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 383, in _on_run
r = self.sock.recv(1024)
OSError: [Errno 9] Bad file descriptor
The result is that you can more or less flip a coin on whether the program will proceed or error out with a communication exception. Here is what happens when the flip comes up "tails":
Connected to pydev debugger (build 191.7141.48)
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 383, in _on_run
r = self.sock.recv(1024)
OSError: [Errno 9] Bad file descriptor
Process ForkServerProcess-3:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/distributed/core.py", line 178, in __init__
from .counter import Digest
ImportError: cannot import name 'Digest' from 'distributed.counter' (/usr/local/lib/python3.7/site-packages/distributed/counter.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/distributed/process.py", line 181, in _run
target(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/distributed/nanny.py", line 587, in _run
worker = Worker(*worker_args, **worker_kwargs)
File "/usr/local/lib/python3.7/site-packages/distributed/worker.py", line 552, in __init__
**kwargs
File "/usr/local/lib/python3.7/site-packages/distributed/node.py", line 76, in __init__
io_loop=self.io_loop,
File "/usr/local/lib/python3.7/site-packages/distributed/core.py", line 180, in __init__
self.digests = defaultdict(partial(Digest, loop=self.io_loop))
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.7/site-packages/distributed/utils.py", line 179, in ignoring
yield
SystemError: error return without exception set
distributed.nanny - WARNING - Worker process 20417 exited with status 1
Traceback (most recent call last):
File "_pydevd_frame_eval/pydevd_frame_evaluator_darwin_37_64.pyx", line 95, in _pydevd_frame_eval.pydevd_frame_evaluator_darwin_37_64.get_bytecode_while_frame_eval
KeyError: '/usr/local/lib/python3.7/site-packages/distributed/bokeh/__init__.py'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1758, in <module>
main()
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1752, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1147, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/git/huddl/python/hamspam/enron.py", line 205, in <module>
client = Client()
File "/usr/local/lib/python3.7/site-packages/distributed/client.py", line 712, in __init__
self.start(timeout=timeout)
File "/usr/local/lib/python3.7/site-packages/distributed/client.py", line 858, in start
sync(self.loop, self._start, **kwargs)
File "/usr/local/lib/python3.7/site-packages/distributed/utils.py", line 331, in sync
six.reraise(*error[0])
File "/usr/local/lib/python3.7/site-packages/six.py", line 693, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/distributed/utils.py", line 316, in f
result[0] = yield future
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/usr/local/lib/python3.7/site-packages/distributed/client.py", line 928, in _start
yield self.cluster
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py", line 603, in _wrap_awaitable
return (yield from awaitable.__await__())
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/usr/local/lib/python3.7/site-packages/distributed/deploy/local.py", line 284, in _start
yield [self._start_worker(**self.worker_kwargs) for i in range(n_workers)]
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 501, in callback
result_list.append(f.result())
File "/usr/local/lib/python3.7/site-packages/tornado/gen.py", line 742, in run
yielded = self.gen.send(value)
File "/usr/local/lib/python3.7/site-packages/distributed/deploy/local.py", line 316, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
Any advice on this?
There will be even more issues/complications when trying to use a LocalCluster mode -but that will be saved for a different question.

devignesh#devignesh:~/soup$ pip list
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2556, in version
return self._version
File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
raise AttributeError(attr)
AttributeError: _version
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/pip/_internal/basecommand.py", line 141, in main
status = self.run(options, args)
File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/list.py", line 143, in run
self.output_package_listing(packages, options)
File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/list.py", line 208, in output_package_listing
data, header = format_for_columns(packages, options)
File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/list.py", line 274, in format_for_columns
row = [proj.project_name, proj.version]
File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2561, in version
raise ValueError(tmpl % self.PKG_INFO, self)
ValueError: ("Missing 'Version:' header and/or METADATA file", Unknown [unknown version] (/home/devignesh/.local/lib/python3.5/site-packages))
devignesh#devignesh:~/soup$

I just updated pip and some packages and now I get the following error when running just about any command like show or list (-V works):
Exception:
Traceback (most recent call last):
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2897, in _dep_map
return self.__dep_map
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2691, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2888, in _parsed_pkg_info
return self._pkg_info
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2691, in __getattr__
raise AttributeError(attr)
AttributeError: _pkg_info
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 176, in main
status = self.run(options, args)
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_internal/commands/show.py", line 47, in run
results, list_files=options.files, verbose=options.verbose):
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_internal/commands/show.py", line 137, in print_results
pkg.project_name for pkg in pkg_resources.working_set
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_internal/commands/show.py", line 138, in <listcomp>
if name in [required.name for required in pkg.requires()]
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2635, in requires
dm = self._dep_map
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2899, in _dep_map
self.__dep_map = self._compute_dependencies()
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2908, in _compute_dependencies
for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2890, in _parsed_pkg_info
metadata = self.get_metadata(self.PKG_INFO)
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1410, in get_metadata
value = self._get(self._fn(self.egg_info, name))
File "/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1522, in _get
with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/sterlingbutters/anaconda3/lib/python3.6/site-packages/-ytz-2018.5.dist-info/METADATA'
Is this just my configuration or is there a known bug, etc?
Any ideas on how to fix it? - I can't say I've ever had this problem before

I have installed the pyrouge by following the instruction given in the official site itself.
https://pypi.python.org/pypi/pyrouge
But i'm getting the below error when i try to run "python -m pyrouge.test"
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/bin/pyrouge_convert_plain_text_to_rouge_format", line 4, in <module>
__import__('pkg_resources').run_script('pyrouge==0.1.3', 'pyrouge_convert_plain_text_to_rouge_format')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 743, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1490, in run_script
raise ResolutionError("No script named %r" % script_name)
pkg_resources.ResolutionError: No script named 'pyrouge_convert_plain_text_to_rouge_format'
EEEETraceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/bin/pyrouge_evaluate_plain_text_files", line 4, in <module>
__import__('pkg_resources').run_script('pyrouge==0.1.3', 'pyrouge_evaluate_plain_text_files')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 743, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1490, in run_script
raise ResolutionError("No script named %r" % script_name)
pkg_resources.ResolutionError: No script named 'pyrouge_evaluate_plain_text_files'
EEEEE
======================================================================
ERROR: test_config_file (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 138, in test_config_file
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
======================================================================
ERROR: test_convert_summaries (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 126, in test_convert_summaries
check_output(command.split())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '[u'pyrouge_convert_plain_text_to_rouge_format', u'-i', u'data/SL2003_models_plain_text', u'-o', u'/tmp/tmps6qLvp']' returned non-zero exit status 1
======================================================================
ERROR: test_evaluation (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 151, in test_evaluation
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
======================================================================
ERROR: test_options (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 203, in test_options
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
======================================================================
ERROR: test_paths (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 23, in test_paths
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
======================================================================
ERROR: test_rouge_for_plain_text (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 173, in test_rouge_for_plain_text
pyrouge_output = check_output_clean(pyrouge_command.split())
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 17, in <lambda>
check_output_clean = lambda c: check_output(c).decode("UTF-8").strip()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '[u'pyrouge_evaluate_plain_text_files', u'-m', u'data/models_plain', u'-s', u'data/systems_plain', u'-sfp', u'D(\\d+).M.100.T.A', u'-mfp', u'D#ID#.M.100.T.[A-Z]', u'-id', u'1']' returned non-zero exit status 1
======================================================================
ERROR: test_text_conversion (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 91, in test_text_conversion
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
======================================================================
ERROR: test_write_config (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 197, in test_write_config
check_output(command.split())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
======================================================================
ERROR: test_wrong_model_pattern (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 73, in test_wrong_model_pattern
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
======================================================================
ERROR: test_wrong_system_pattern (pyrouge.tests.Rouge155_test.PyrougeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pyrouge/tests/Rouge155_test.py", line 59, in test_wrong_system_pattern
rouge = Rouge155()
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 88, in __init__
self.__set_rouge_dir(rouge_dir)
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 407, in __set_rouge_dir
self.data_dir = os.path.join(self._home_dir, 'data')
File "/Library/Python/2.7/site-packages/pyrouge/Rouge155.py", line 543, in fset
verify_dir(path, dir_name)
File "/Library/Python/2.7/site-packages/pyrouge/utils/file_utils.py", line 87, in verify_dir
raise Exception(msg)
Exception: Cannot set data directory because the path /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyrouge/data does not exist.
----------------------------------------------------------------------
Ran 10 tests in 0.403s
FAILED (errors=10)
I have ran the command in the sudo mode only. I didn't what do they mean by this "pyrouge_set_rouge_path /absolute/path/to/ROUGE-1.5.5/directory". Does this mean should i need to set/provide installation path of the pyrouge as environment variable? like how we set path for python and all.
I'm really stuck over here, please help me.