Develop Reference

Conda: concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending

I want to install Anaconda on my mac (version 10.9.5).
The command I used:
sh Anaconda3-2020.02-MacOSX-x86_64.sh
Led to this error:
Unpacking payload ...
Traceback (most recent call last):
File "entry_point.py", line 69, in <module>
File "concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists
File "concurrent/futures/_base.py", line 598, in result_iterator
File "concurrent/futures/_base.py", line 435, in result
File "concurrent/futures/_base.py", line 384, in __get_result
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
[1061] Failed to execute script entry_point
After rooting around, I found this suggestion, that I check the hash.
I typed this:
shasum -a 512 /Users/Slowat/Anaconda3-2020.02-MacOSX-x86_64.sh
and the output was:
aa1ed0c40646ba9041abf59c13ce38da1dc51bf15de239b6f966a0b02b4c09c960ae33698c72aa46db41731f8e67938d1972fcb76fa4c5c8081bc0272bb1b535 /Users/Slowat/Anaconda3-2020.02-MacOSX-x86_64.sh
The hash listed here does not match this.
So then I thought it was an issue with downloaded, so i deleted the bash script and the directory that anaconda attempted to make during the failed installation.
I double checked my python version:
localhost:~ Slowat$ python --version
Python 3.7.4
and re-downloaded Anaconda3-2020.02-MacOSX-x86_64.sh from here.
I re-checked the hash:
aa1ed0c40646ba9041abf59c13ce38da1dc51bf15de239b6f966a0b02b4c09c960ae33698c72aa46db41731f8e67938d1972fcb76fa4c5c8081bc0272bb1b535 /Users/Slowat/Anaconda3-2020.02-MacOSX-x86_64.sh
and it's still wrong. Any ideas what I'm doing wrong here (mainly for the original area, checking the hash was just an idea I had).

As marhil95 mentioned in a GitHub issue, you can also get this error if you run out of disk space.

I have discovered that conda cannot be run on mac v. 10.9.5; i first updated the mac to 10.11, and then from 10.11 to 10.13 (you have to do it in those two steps). Then conda installed and ran fine.

I can´t install Anaconda on Linux

When I try to install Anaconda on Linux, I get to this point:
Anaconda3 will now be installed into this location:
/home/jorge/anaconda3
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below
[/home/jorge/anaconda3] >>>
PREFIX=/home/jorge/anaconda3
Unpacking payload ...
Then I receive the following error message:
concurrent.futures.process._RemoteTraceback:
'''
Traceback (most recent call last):
File "concurrent/futures/process.py", line 367, in _queue_management_worker
File "multiprocessing/connection.py", line 251, in recv
TypeError: __init__() missing 1 required positional argument: 'msg'
'''
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "entry_point.py", line 69, in <module>
File "concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists
File "concurrent/futures/_base.py", line 598, in result_iterator
File "concurrent/futures/_base.py", line 435, in result
File "concurrent/futures/_base.py", line 384, in __get_result
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
[1770] Failed to execute script entry_point
What can I do? I was following all the instructions step by step

I had a similar problem where available storage (disk space) was only 1.2 GB. Please make sure that you have enough free storage space to install it. To check available space, type
df -h /home/

Did you verify the integrity of the installer's data? because it is a common error when downloading this corrupted or incomplete
since it is the previous step you have to do to make sure that the file is ok before executing the script.
This post helped me a lot for the first time I installed it.
https://www.digitalocean.com/community/tutorials/how-to-install-anaconda-on-ubuntu-18-04-quickstart

Broken pipe connecting to AWS with Fabric

I've been using Fabric to run setup commands on an EC2 instance during some automated unittests. These tests ran fine for months, and then a few days ago, suddenly started failing with the error:
Traceback (most recent call last):
File "tests.py", line 207, in test_setup
run('./setup_server')
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/network.py", line 682, in host_prompting_wrapper
return func(*args, **kwargs)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/operations.py", line 1091, in run
shell_escape=shell_escape, capture_buffer_size=capture_buffer_size,
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/operations.py", line 934, in _run_command
capture_buffer_size=capture_buffer_size)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/operations.py", line 816, in _execute
worker.raise_if_needed()
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/thread_handling.py", line 26, in raise_if_needed
six.reraise(e[0], e[1], e[2])
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/thread_handling.py", line 13, in wrapper
callable(*args, **kwargs)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/io.py", line 31, in output_loop
OutputLooper(*args, **kwargs).loop()
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/io.py", line 152, in loop
self._flush(end_of_line + "\n")
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/fabric/io.py", line 57, in _flush
self.stream.flush()
IOError: [Errno 32] Broken pipe
The command I'm running is a pip install -r requirements.txt, and it appears to run just fine. If I run the test locally, it completes without error. However, it now fails half-way through when run on AWS every time.
What would cause this? Since it's an IOError, and these can be caused by virtually any kind of minor network interruption, I'm not sure how to diagnose it. If Fabric lost connection temporarily, that would explain it, but it wouldn't explain why it's repeatable. I've re-run this script several dozen times, and it fails each time after initially connecting to the EC2 instance perfectly.
Is there some sort of configuration in Fabric I can change to improve connection error handling?

Canopy is having difficulties updating packages, "AbortedOperationDetected"

I run Canopy version Version: 2.1.3.3542 (64 bit) on Windows 10.
Canopy cant manage to update any package, all results in the same
log output for a numpy update (for example):
Warming up...
Traceback (most recent call last):
File "build\bdist.win-amd64\egg\canopy_dashboard\packman\package_action_worker.py", line 54, in run
File "build\bdist.win-amd64\egg\canopy_dashboard\packman\package_action.py", line 196, in execute
File "build\bdist.win-amd64\egg\canopy_dashboard\packman\packman.py", line 626, in <lambda>
File "build\bdist.win-amd64\egg\canopy_dashboard\packman\packman.py", line 1051, in _install
File "build\bdist.win-amd64\egg\canopy_platform\cpython_packages_manager.py", line 152, in install_packages_prompt
File "build\bdist.win-amd64\egg\canopy_platform\cpython_packages_manager.py", line 137, in _install_packages_prompt
PackagesInstallationError: installation of packages ['numpy 1.11.3-3'] failed. Details below:
Traceback (most recent call last):
File "build\bdist.win-amd64\egg\canopy_platform\edm_api.py", line 64, in wrapper
File "build\bdist.win-amd64\egg\canopy_platform\edm_api.py", line 384, in install_command
File "build\bdist.win-amd64\egg\canopy_platform\edm_api.py", line 414, in _install_packages_command
File "build\bdist.win-amd64\egg\edm\core\packages_manager.py", line 124, in decorator
File "build\bdist.win-amd64\egg\edm\core\packages_manager.py", line 219, in install_command
File "build\bdist.win-amd64\egg\edm\core\packages_manager.py", line 223, in _install_command
File "build\bdist.win-amd64\egg\edm\core\packages_manager.py", line 549, in _compute_fix_aborted_actions
AbortedOperationDetected: Aborted operation detected in environment 'User'
There is absolutly 0 results on google for this error code, so stackexchange is my last resort

It looks like a previous update was force-aborted, possibly leaving the environment corrupted. Assuming that you are using the standard installer, then it should suffice to
reboot your computer
temporarily disable your anti-virus software if possible (or at least disable its more intrusive / slow functionality, such as online checking each of the tens of thousands of package files that Canopy provides),
from the Canopy Tools menu, select Troubleshoot => Reset Python environment

Azure Batch Apps install python packages on startup

We are using Azure Batch Apps which will create multiple VMs which can be used to run our tasks parallelly. We are using python for data fetching tasks.
We have mentioned in the batch apps to install anaconda on the VMs when they start up. Anaconda is installed properly. We have listed out the packages(requirements.txt) we need to install to run the tasks.
pip install -r requirements.txt
Some packages get installed correclty, but some packages result in the following error,
Error [Error 6] The handle is invalid while executing command python setup.py egg_info
Exception:
Traceback (most recent call last):
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\basecommand.py", line 209, in main
status = self.run(options, args)
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\commands\install.py", line 310, in run
wb.build(autobuilding=True)
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\wheel.py", line 748, in build
self.requirement_set.prepare_files(self.finder)
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\req\req_set.py", line 360, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\req\req_set.py", line 591, in _prepare_file
abstract_dist.prep_for_dist()
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\req\req_set.py", line 127, in prep_for_dist
self.req_to_install.run_egg_info()
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\req\req_install.py", line 430, in run_egg_info
command_desc='python setup.py egg_info')
File "C:\user\tasks\shared\anaconda2\lib\site-packages\pip\utils\__init__.py", line 678, in call_subprocess
cwd=cwd, env=env)
File "C:\user\tasks\shared\anaconda2\lib\subprocess.py", line 702, in __init__
errread, errwrite), to_close = self._get_handles(stdin, stdout, stderr)
File "C:\user\tasks\shared\anaconda2\lib\subprocess.py", line 823, in _get_handles
p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
WindowsError: [Error 6] The handle is invalid
When we open the VM and give the same command, all packages get installed correctly.
I just wonder where the issue is.

It seems that the issue was caused by some limits for Azure Batch service, you can see these limits here.
According to the error information, it seems that the installation process needs to fork the subprocess, but the maximum number of tasks per computer node is 4, please see below.