Unable to install dependencies for dataflow template - python

I have a dataflow that works with DirectRunner but when I create a template version of it with DataflowRunner I have this error
| 'Read from BQ Table' >> beam.io.Read(beam.io.BigQuerySource(query=query, use_standard_sql=True))
.\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\io\gcp\bigquery.py:1971: BeamDeprecationWarning: options is deprecated since First stable
release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
.\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\io\gcp\bigquery_file_loads.py:900: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
INFO:apache_beam.runners.portability.stager:Executing command: ['C:\\Users\\PhuongAnhNguenVenefi\\virtualenvs\\nieuwbouw-data\\Scripts\\python.exe', '-m', 'pip', 'download', '--dest', 'C:\\Users\\PHUONG~1\\AppData\\Local\\Temp\\dataflow-requirements-cache', '-r', 'requirements-template.txt', '--exists-action', 'i', '--no-binary', ':all:']
Traceback (most recent call last):
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\utils\processes.py", line 91, in check_output
out = subprocess.check_output(*args, **kwargs)
File ".\anaconda3\lib\subprocess.py", line 411, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File ".\anaconda3\lib\subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\PhuongAnhNguenVenefi\\virtualenvs\\nieuwbouw-data\\Scripts\\python.exe', '-m', 'pip', 'download', '--dest', 'C:\\Users\\PHUONG~1\\AppData\\Local\\Temp\\dataflow-requirements-cache', '-r', 'requirements-template.txt', '--exists-action', 'i', '--no-binary', ':all:']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ".\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File ".\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File ".\Desktop\nieuwbouw-data\am_gcp\automation\template_housetype_am.py", line 288, in <module>
run()
File ".\Desktop\nieuwbouw-data\am_gcp\automation\template_housetype_am.py", line 283, in run
p.run().wait_until_finish()
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\pipeline.py", line 514, in run
return Pipeline.from_runner_api(
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\pipeline.py", line 547, in run
return self.runner.run_pipeline(self, self._options)
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\runners\dataflow\dataflow_runner.py", line 493, in run_pipeline
artifacts=environments.python_sdk_dependencies(options)))
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\transforms\environments.py", line 623, in python_sdk_dependencies
staged_name in stager.Stager.create_job_resources(
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\runners\portability\stager.py", line 177, in create_job_resources
(
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\utils\retry.py", line 236, in wrapper
return fun(*args, **kwargs)
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\runners\portability\stager.py", line 571, in _populate_requirements_cache
processes.check_output(cmd_args, stderr=processes.STDOUT)
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\utils\processes.py", line 96, in check_output
raise RuntimeError( \
RuntimeError: Full traceback: Traceback (most recent call last):
File ".\virtualenvs\nieuwbouw-data\lib\site-packages\apache_beam\utils\processes.py", line 91, in check_output
out = subprocess.check_output(*args, **kwargs)
File ".\anaconda3\lib\subprocess.py", line 411, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File ".\anaconda3\lib\subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\PhuongAnhNguenVenefi\\virtualenvs\\nieuwbouw-data\\Scripts\\python.exe', '-m', 'pip', 'download', '--dest', 'C:\\Users\\PHUONG~1\\AppData\\Local\\Temp\\dataflow-requirements-cache', '-r', 'requirements-template.txt', '--exists-action', 'i', '--no-binary', ':all:']' returned non-zero exit status 1.
Pip install failed for package: -r
The deploying command:
python template_housetype_am.py --runner DataflowRunner --project vf-scrapers --region=europe-west4 --staging_location gs://am_scraper/test --temp_location gs://am_scraper/test --template_location gs://am_scraper/templates/template_housetype_am --experiment=use_beam_bq_sink --requirements_file requirements-template.txt --save_main_session True
The requirements-template.txt file:
apache-beam[gcp]==2.25.0
google-cloud==0.34.0
google-cloud-bigquery==1.28.0
google-cloud-storage==1.33.0
The issue does not come from the template code, I have this error even if the template code only has importing statement:
import argparse
import logging
import re
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
from google.cloud import storage
I tried:
Use setup.py per this suggestion but the setup_file argument is discarded:
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['setup.py', 'True']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['setup.py', 'True']
Not including the requirements file, which successfully create the template but the flow fails because google-cloud-storage is not installed. In other words, specifying dependencies is a must for me. Unless there are other ways to install dependencies on Dataflow.
Deploy the template using cloud shell, which worked. However, I need to deploy it on my local machine.

Related

pip install rpy2==3.2.7 error with code 1

Please anyone help me, i got these when pip install rpy2==3.2.7 :
Collecting wheel
Using cached https://files.pythonhosted.org/packages/8c/23/848298cccf8e40f5bbb59009b32848a4c38f4e7f3364297ab3c3e2e2cd14/wheel-0.34.2-py2.py3-none-any.whl
Installing collected packages: wheel
Successfully installed wheel-0.34.2
(venv) C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50>pip install rpy2==3.2.7
Collecting rpy2==3.2.7
Using cached https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
Complete output from command python setup.py egg_info:
Command "python setup.py egg_info" failed with error code 1 in C:\Users\billpebert\AppData\Local\Temp\pip-wheel-4l7yazyi\cffi\
Traceback (most recent call last):
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\setuptools\installer.py", line 128, in fetch_build_egg
subprocess.check_call(cmd)
File "C:\Users\billpebert\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\Scripts\python.exe', '-m', 'pip', '--disable-pip-versio
n-check', 'wheel', '--no-deps', '-w', 'C:\Users\BILLPE~1\AppData\Local\Temp\tmpvdyfy70j', '--quiet', 'cffi>=1.13.1']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\billpebert\AppData\Local\Temp\pip-install-22ktwibt\rpy2\setup.py", line 147, in <module>
setup(
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\setuptools\__init__.py", line 143, in setup
_install_setup_requires(attrs)
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\setuptools\__init__.py", line 138, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\setuptools\dist.py", line 695, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\pkg_resources\__init__.py", line 781, in resolve
dist = best[req.key] = env.best_match(
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\pkg_resources\__init__.py", line 1066, in best_match
return self.obtain(req, installer)
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\pkg_resources\__init__.py", line 1078, in obtain
return installer(requirement)
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\setuptools\dist.py", line 754, in fetch_build_egg
return fetch_build_egg(self, req)
File "C:\Users\billpebert\PycharmProjects\KlasifikasiKualitasUdaraC50\venv\lib\site-packages\setuptools\installer.py", line 130, in fetch_build_egg
raise DistutilsError(str(e))
distutils.errors.DistutilsError: Command '['C:\\Users\\billpebert\\PycharmProjects\\KlasifikasiKualitasUdaraC50\\venv\\Scripts\\python.exe', '-m', 'pip', '--disable-pip-vers
ion-check', 'wheel', '--no-deps', '-w', 'C:\Users\BILLPE~1\AppData\Local\Temp\tmpvdyfy70j', '--quiet', 'cffi>=1.13.1']' returned non-zero exit status 1.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\billpebert\AppData\Local\Temp\pip-install-22ktwibt\rpy2\
Fixes for Windows only appeared with rpy2-3.3.2. If you pip install rpy2 you will get the latest release (currrently 3.3.3).

install v8js extension to php

I'm trying to install v8js on a local http/ssh server built with Debian 9.3, so I need to install v8 chromium engine first according to this documentation.
But when I'm trying to run the command line tools/dev/v8gen.py -vv x64.release -- is_component_build=true the command sends me this:
################################################################################
/usr/bin/python -u tools/mb/mb.py gen -f infra/mb/mb_config.pyl -m developer_default -b x64.release out.gn/x64.release
Writing """\
is_debug = false
target_cpu = "x64"
""" to /home/dev/v8/out.gn/x64.release/args.gn.
/home/dev/v8/buildtools/linux64/gn gen out.gn/x64.release --check
Traceback (most recent call last):
File "tools/mb/mb.py", line 67, in Main
ret = self.args.func()
File "tools/mb/mb.py", line 317, in CmdGen
return self.RunGNGen(vals)
File "tools/mb/mb.py", line 856, in RunGNGen
ret, _, _ = self.Run(cmd)
File "tools/mb/mb.py", line 1378, in Run
ret, out, err = self.Call(cmd, env=env, buffer_output=buffer_output)
File "tools/mb/mb.py", line 1392, in Call
env=env)
File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1024, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error
Traceback (most recent call last):
File "tools/dev/v8gen.py", line 304, in <module>
sys.exit(gen.main())
File "tools/dev/v8gen.py", line 298, in main
return self._options.func()
File "tools/dev/v8gen.py", line 166, in cmd_gen
gn_outdir,
File "tools/dev/v8gen.py", line 208, in _call_cmd
stderr=subprocess.STDOUT,
File "/usr/lib/python2.7/subprocess.py", line 219, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/usr/bin/python', '-u', 'tools/mb/mb.py', 'gen', '-f', 'infra/mb/mb_config.pyl', '-m', 'developer_default', '-b', 'x64.release', 'out.gn/x64.release']' returned non-zero exit status 1
Obviously I can not continue the following installation steps (I can't use ninja) and I do not really know about Python so a help would be really convenient :)
PS: I saw (I don't remember where) that there is probably minimum RAM (8GB) and Disk space (100GB) requirements to build v8, is that it ...?

pip install language-check not working

I get this error when I do pip install language-check:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/j6/yr6zgb4d1w308jkpc27p2t9w0000gn/T/pip-build-ymwzmypp/language-check/setup.py", line 595, in <module>
sys.exit(main())
File "/private/var/folders/j6/yr6zgb4d1w308jkpc27p2t9w0000gn/T/pip-build-ymwzmypp/language-check/setup.py", line 590, in main
run_setup_hooks(config)
File "/private/var/folders/j6/yr6zgb4d1w308jkpc27p2t9w0000gn/T/pip-build-ymwzmypp/language-check/setup.py", line 561, in run_setup_hooks
language_tool_hook(config)
File "/private/var/folders/j6/yr6zgb4d1w308jkpc27p2t9w0000gn/T/pip-build-ymwzmypp/language-check/setup.py", line 584, in language_tool_hook
download_lt()
File "/private/var/folders/j6/yr6zgb4d1w308jkpc27p2t9w0000gn/T/pip-build-ymwzmypp/language-check/download_lt.py", line 117, in download_lt
version = get_newest_possible_languagetool_version()
File "/private/var/folders/j6/yr6zgb4d1w308jkpc27p2t9w0000gn/T/pip-build-ymwzmypp/language-check/download_lt.py", line 81, in get_newest_possible_languagetool_version
universal_newlines=True)
File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/java', '-version']' returned non-zero exit status 1.
I've looked at this question python pip install error language_check and followed the instructions to install python setup.py install and I get this:
Traceback (most recent call last):
File "setup.py", line 595, in <module>
sys.exit(main())
File "setup.py", line 590, in main
run_setup_hooks(config)
File "setup.py", line 561, in run_setup_hooks
language_tool_hook(config)
File "setup.py", line 584, in language_tool_hook
download_lt()
File "/Users/allyzamarquez/ben/language_check/language-check/download_lt.py", line 117, in download_lt
version = get_newest_possible_languagetool_version()
File "/Users/allyzamarquez/ben/language_check/language-check/download_lt.py", line 81, in get_newest_possible_languagetool_version
universal_newlines=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/usr/bin/java', '-version']' returned non-zero exit status 1
Java 8 must be used until language-check fixes their installer script:
sudo apt install openjdk-8-jdk
sudo update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
pip install language-check

Build OpenCV for iOS fail

I downloaded OpenCV folder from GitHub and followed the ReadMe instruction to build it for ios.
After use python opencv/platforms/ios/build_framework.py ios command I have the following error:
Executing: ['cmake', '-GXcode', '-DAPPLE_FRAMEWORK=ON', '-DCMAKE_INSTALL_PREFIX=install', '-DCMAKE_BUILD_TYPE=Release', '-DIOS_ARCH=armv7', '-DCMAKE_TOOLCHAIN_FILE=/Users/name/Downloads/opencv/platforms/ios/cmake/Toolchains/Toolchain-iPhoneOS_Xcode.cmake', '-DENABLE_NEON=ON', '/Users/name/Downloads/opencv', '-DCMAKE_C_FLAGS=-fembed-bitcode', '-DCMAKE_CXX_FLAGS=-fembed-bitcode'] in /Users/name/Downloads/ios/build/build-armv7-iphoneos
============================================================
ERROR: [Errno 2] No such file or directory
============================================================
Traceback (most recent call last):
File "opencv/platforms/ios/build_framework.py", line 112, in build
self._build(outdir)
File "opencv/platforms/ios/build_framework.py", line 104, in _build
self.buildOne(t[0], t[1], mainBD, cmake_flags)
File "opencv/platforms/ios/build_framework.py", line 186, in buildOne
execute(cmakecmd, cwd = builddir)
File "opencv/platforms/ios/build_framework.py", line 36, in execute
retcode = check_call(cmd, cwd = cwd)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 535, in check_call
retcode = call(*popenargs, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
What I have to do to fix it?
If someone in the future will have the same problem.
This problem was in the master branch, downloading branch 2.4 and adding CMake at PATH the build process go well
I also encountered the same problem.
In the master branch or 3.4 branch or 2.4 branch.
But after installing cmake, it will be fine.
In the 2.4 branch error message will show :
/bin/sh: cmake: command not found

Requirements.txt invalid with conda/pip on elastic beanstalk

Is it possible that it doesnt like the mixing of conda/pip? It says my pip is out of date, but in fact it's at 7.1.2 on my local machine.
I am trying to deploy a flask app with numpy, scipy, etc to aws
AppDeployPreHook/03deploy.py] : Activity execution failed, because: You are using pip version 7.0.3, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Exception:
Traceback (most recent call last):
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/commands/install.py", line 268, in run
wheel_cache
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 287, in populate_requirement_set
wheel_cache=wheel_cache):
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_file.py", line 86, in parse_requirements
for req in req_iter:
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_file.py", line 130, in process_line
wheel_cache=wheel_cache
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 207, in from_line
wheel_cache=wheel_cache)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 66, in __init__
req = pkg_resources.Requirement.parse(req)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2960, in parse
reqs = list(parse_requirements(s))
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2904, in parse_requirements
"version spec")
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in scan_list
raise ValueError(msg, line, "at", line[p:])
ValueError: ('Expected version spec in', '_license=1.1=py27_0', 'at', '=1.1=py27_0')
2015-10-16 01:37:43,439 ERROR Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 2
Traceback (most recent call last):
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
install_dependencies()
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 2 (ElasticBeanstalk::ExternalInvocationError)
caused by: You are using pip version 7.0.3, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Exception:
Traceback (most recent call last):
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/commands/install.py", line 268, in run
wheel_cache
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 287, in populate_requirement_set
wheel_cache=wheel_cache):
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_file.py", line 86, in parse_requirements
for req in req_iter:
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_file.py", line 130, in process_line
wheel_cache=wheel_cache
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 207, in from_line
wheel_cache=wheel_cache)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 66, in __init__
req = pkg_resources.Requirement.parse(req)
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2960, in parse
reqs = list(parse_requirements(s))
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2904, in parse_requirements
"version spec")
File "/opt/python/run/venv/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in scan_list
raise ValueError(msg, line, "at", line[p:])
ValueError: ('Expected version spec in', '_license=1.1=py27_0', 'at', '=1.1=py27_0')
2015-10-16 01:37:43,439 ERROR Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 2
Traceback (most recent call last):
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
install_dependencies()
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 2 (Executor::NonZeroExitStatus)
Here is my requirements file, which it says is invalid:
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-64
_license=1.1=py27_0
abstract-rendering=0.5.1=np19py27_0
alabaster=0.7.3=py27_0
anaconda=2.3.0=np19py27_0
appscript=1.0.1=py27_0
argcomplete=0.8.9=py27_0
astropy=1.0.3=np19py27_0
babel=1.3=py27_0
bcolz=0.9.0=np19py27_0
beautiful-soup=4.3.2=py27_0
binstar=0.11.0=py27_0
bitarray=0.8.1=py27_0
blaze-core=0.8.0=np19py27_0
blz=0.6.2=np19py27_1
bokeh=0.9.0=np19py27_0
boto=2.38.0=py27_0
bottleneck=1.0.0=np19py27_0
cdecimal=2.3=py27_0
certifi=14.05.14=py27_0
cffi=1.1.0=py27_0
clyent=0.3.4=py27_0
colorama=0.3.3=py27_0
configobj=5.0.6=py27_0
cryptography=0.9.1=py27_0
curl=7.43.0=0
cython=0.22.1=py27_0
cytoolz=0.7.3=py27_0
datashape=0.4.5=np19py27_0
decorator=3.4.2=py27_0
docutils=0.12=py27_0
dynd-python=0.6.5=np19py27_0
enum34=1.0.4=py27_0
fastcache=1.0.2=py27_0
flask=0.10.1=py27_1
freetype=2.5.2=2
funcsigs=0.4=py27_0
gevent=1.0.1=py27_0
gevent-websocket=0.9.3=py27_0
greenlet=0.4.7=py27_0
grin=1.2.1=py27_1
h5py=2.5.0=np19py27_3
hdf5=1.8.15.1=1
idna=2.0=py27_0
ipaddress=1.0.7=py27_0
ipython=3.2.0=py27_0
ipython-notebook=3.2.0=py27_0
ipython-qtconsole=3.2.0=py27_0
itsdangerous=0.24=py27_0
jdcal=1.0=py27_0
jedi=0.8.1=py27_0
jinja2=2.7.3=py27_1
jpeg=8d=1
jsonschema=2.4.0=py27_0
launcher=1.0.0=3
libdynd=0.6.5=0
libpng=1.6.17=0
libsodium=0.4.5=0
libtiff=4.0.2=1
libxml2=2.9.2=0
libxslt=1.1.28=2
llvmlite=0.5.0=py27_0
lxml=3.4.4=py27_0
markupsafe=0.23=py27_0
matplotlib=1.4.3=np19py27_2
mistune=0.5.1=py27_1
mock=1.0.1=py27_0
multipledispatch=0.4.7=py27_0
networkx=1.9.1=py27_0
nltk=3.0.3=np19py27_0
node-webkit=0.10.1=0
nose=1.3.7=py27_0
numba=0.19.1=np19py27_0
numexpr=2.4.3=np19py27_0
numpy=1.9.2=py27_0
odo=0.3.2=np19py27_0
openpyxl=1.8.5=py27_0
openssl=1.0.1k=1
pandas=0.16.2=np19py27_0
patsy=0.3.0=np19py27_0
pep8=1.6.2=py27_0
pillow=2.8.2=py27_0
pip=7.1.2=py27_0
ply=3.6=py27_0
psutil=2.2.1=py27_0
ptyprocess=0.4=py27_0
py=1.4.27=py27_0
pyasn1=0.1.7=py27_0
pyaudio=0.2.7=py27_0
pycosat=0.6.1=py27_0
pycparser=2.14=py27_0
pycrypto=2.6.1=py27_0
pycurl=7.19.5.1=py27_2
pyflakes=0.9.2=py27_0
pygments=2.0.2=py27_0
pyopenssl=0.15.1=py27_1
pyparsing=2.0.3=py27_0
pyqt=4.11.3=py27_0
pytables=3.2.0=np19py27_0
pytest=2.7.1=py27_0
python=2.7.10=1
python-dateutil=2.4.2=py27_0
python.app=1.2=py27_4
pytz=2015.4=py27_0
pyyaml=3.11=py27_1
pyzmq=14.7.0=py27_0
qt=4.8.6=3
readline=6.2=2
redis=2.6.9=0
redis-py=2.10.3=py27_0
requests=2.7.0=py27_0
rope=0.9.4=py27_1
runipy=0.1.3=py27_0
scikit-image=0.11.3=np19py27_0
scikit-learn=0.16.1=np19py27_0
scipy=0.15.1=np19py27_0
setuptools=18.4=py27_0
sip=4.16.5=py27_0
six=1.9.0=py27_0
snowballstemmer=1.2.0=py27_0
sockjs-tornado=1.0.1=py27_0
sphinx=1.3.1=py27_0
sphinx_rtd_theme=0.1.7=py27_0
spyder=2.3.5.2=py27_0
spyder-app=2.3.5.2=py27_0
sqlalchemy=1.0.5=py27_0
sqlite=3.8.4.1=1
ssl_match_hostname=3.4.0.2=py27_0
statsmodels=0.6.1=np19py27_0
sympy=0.7.6=py27_0
terminado=0.5=py27_0
tk=8.5.18=0
toolz=0.7.2=py27_0
tornado=4.2=py27_0
ujson=1.33=py27_0
unicodecsv=0.9.4=py27_0
werkzeug=0.10.4=py27_0
wheel=0.26.0=py27_1
xlrd=0.9.3=py27_0
xlsxwriter=0.7.3=py27_0
xlwings=0.3.5=py27_0
xlwt=1.0.0=py27_0
yaml=0.1.6=0
zeromq=4.0.5=0
zlib=1.2.8=0
Your requirements.txt file is invalid (at least for pip). If you look at the header it says;
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-64
You'll have to re-format this list for pip (remove the extra =npXYpyAB_Z blah as #mikko-ohtamaa mentioned, also rename the non-valid conda packages - _licence), or you'll need to work out how to get EBS to work from a conda environment.
Aside, you see a warning like
You are using pip version 7.0.3, however version 7.1.2 is available
it's generally best to assume it's telling the truth. Maybe you might have 7.1.2 locally, but is this script being run locally (or possibly on an EBS instance with an older pip installed...)?

Categories

Resources