libvips cannot install due to wrong Python location - python

I'm trying to install libvips via Brew, but when i install i get this error about Python not being in the right location.
Error message
martins-mbp:~ martinnielsen$ brew install vips
==> Downloading http://www.vips.ecs.soton.ac.uk/supported/7.40/vips-7.40.10.tar.Already downloaded: /Library/Caches/Homebrew/vips-7.40.10.tar.gz
==> ./configure --prefix=/usr/local/Cellar/vips/7.40.10
==> make check
/usr/local/Cellar/gobject-introspection/1.42.0/bin/g-ir-scanner --add-include-path=. --namespace=Vips --nsversion=8.0 --libtool="/bin/sh ../libtool" --include=GObject-2.0 --library=libvips.la --warn-all --verbose --identifier-prefix=Vips --identifier-prefix=vips --symbol-prefix=vips --symbol-prefix=im --symbol-prefix=im_ --cflags-begin -I../libvips/include --cflags-end arithmetic/abs.c arithmetic/add.c arithmetic/arithmetic.c arithmetic/avg.c arithmetic/binary.c arithmetic/boolean.c arithmetic/complex.c arithmetic/deviate.c arithmetic/divide.c arithmetic/getpoint.c arithmetic/hist_find.c arithmetic/hist_find_indexed.c arithmetic/hist_find_ndim.c arithmetic/hough.c arithmetic/hough_circle.c arithmetic/hough_line.c arithmetic/invert.c arithmetic/linear.c arithmetic/math.c arithmetic/math2.c arithmetic/max.c arithmetic/measure.c arithmetic/min.c arithmetic/multiply.c arithmetic/nary.c arithmetic/profile.c arithmetic/project.c arithmetic/relational.c arithmetic/remainder.c arithmetic/round.c arithmetic/sign.c arithmetic/statistic.c arithmetic/stats.c arithmetic/subtract.c arithmetic/sum.c arithmetic/unary.c arithmetic/unaryconst.c cimg/cimg_dispatch.c colour/colour.c colour/colourspace.c colour/dE00.c colour/dE76.c colour/dECMC.c colour/float2rad.c colour/icc_transform.c colour/Lab2LabQ.c colour/Lab2LabS.c colour/Lab2LCh.c colour/Lab2XYZ.c colour/LabQ2Lab.c colour/LabQ2LabS.c colour/LabQ2sRGB.c colour/LabS2Lab.c colour/LabS2LabQ.c colour/LCh2Lab.c colour/LCh2UCS.c colour/rad2float.c colour/scRGB2sRGB.c colour/scRGB2XYZ.c colour/sRGB2scRGB.c colour/UCS2LCh.c colour/XYZ2Lab.c colour/XYZ2scRGB.c colour/XYZ2Yxy.c colour/Yxy2XYZ.c conversion/bandary.c conversion/bandbool.c conversion/bandjoin.c conversion/bandmean.c conversion/bandrank.c conversion/cache.c conversion/cast.c conversion/conversion.c conversion/copy.c conversion/embed.c conversion/extract.c conversion/falsecolour.c conversion/flatten.c conversion/flip.c conversion/gamma.c conversion/grid.c conversion/ifthenelse.c conversion/insert.c conversion/join.c conversion/msb.c conversion/recomb.c conversion/replicate.c conversion/rot.c conversion/rot45.c conversion/scale.c conversion/sequential.c conversion/subsample.c conversion/tilecache.c conversion/wrap.c conversion/zoom.c convolution/compass.c convolution/conv.c convolution/convolution.c convolution/convsep.c convolution/correlation.c convolution/fastcor.c convolution/gaussblur.c convolution/im_aconv.c convolution/im_aconvsep.c convolution/im_conv.c convolution/im_conv_f.c convolution/sharpen.c convolution/spcor.c create/black.c create/buildlut.c create/create.c create/eye.c create/fractsurf.c create/gaussmat.c create/gaussnoise.c create/grey.c create/identity.c create/im_benchmark.c create/invertlut.c create/logmat.c create/mask.c create/mask_butterworth.c create/mask_butterworth_band.c create/mask_butterworth_ring.c create/mask_fractal.c create/mask_gaussian.c create/mask_gaussian_band.c create/mask_gaussian_ring.c create/mask_ideal.c create/mask_ideal_band.c create/mask_ideal_ring.c create/other_dispatch.c create/point.c create/sines.c create/text.c create/tonelut.c create/xyz.c create/zone.c draw/draw.c draw/draw_circle.c draw/draw_flood.c draw/draw_image.c draw/draw_line.c draw/draw_mask.c draw/draw_rect.c draw/draw_smudge.c draw/drawink.c foreign/analyze2vips.c foreign/analyzeload.c foreign/csv.c foreign/csvload.c foreign/csvsave.c foreign/dzsave.c foreign/fits.c foreign/fitsload.c foreign/fitssave.c foreign/foreign.c foreign/jpeg2vips.c foreign/jpegload.c foreign/jpegsave.c foreign/magick2vips.c foreign/magickload.c foreign/matlab.c foreign/matload.c foreign/matrixload.c foreign/matrixsave.c foreign/openexr2vips.c foreign/openexrload.c foreign/openslide2vips.c foreign/openslideload.c foreign/pngload.c foreign/pngsave.c foreign/ppm.c foreign/ppmload.c foreign/ppmsave.c foreign/radiance.c foreign/radload.c foreign/radsave.c foreign/rawload.c foreign/rawsave.c foreign/tiff2vips.c foreign/tiffload.c foreign/tiffsave.c foreign/vips2jpeg.c foreign/vips2tiff.c foreign/vips2webp.c foreign/vipsload.c foreign/vipspng.c foreign/vipssave.c foreign/webp2vips.c foreign/webpload.c foreign/webpsave.c freqfilt/freqfilt.c freqfilt/freqmult.c freqfilt/fwfft.c freqfilt/invfft.c freqfilt/phasecor.c freqfilt/spectrum.c histogram/hist_cum.c histogram/hist_equal.c histogram/hist_ismonotonic.c histogram/hist_local.c histogram/hist_match.c histogram/hist_norm.c histogram/hist_plot.c histogram/hist_unary.c histogram/histogram.c histogram/maplut.c histogram/percent.c histogram/stdif.c iofuncs/base64.c iofuncs/buf.c iofuncs/buffer.c iofuncs/cache.c iofuncs/enumtypes.c iofuncs/error.c iofuncs/gate.c iofuncs/generate.c iofuncs/header.c iofuncs/image.c iofuncs/init.c iofuncs/mapfile.c iofuncs/memory.c iofuncs/object.c iofuncs/operation.c iofuncs/rect.c iofuncs/region.c iofuncs/semaphore.c iofuncs/sink.c iofuncs/sinkdisc.c iofuncs/sinkmemory.c iofuncs/sinkscreen.c iofuncs/system.c iofuncs/threadpool.c iofuncs/type.c iofuncs/util.c iofuncs/vector.c iofuncs/vips.c iofuncs/vipsmarshal.c iofuncs/window.c morphology/countlines.c morphology/hitmiss.c morphology/labelregions.c morphology/morph.c morphology/morphology.c morphology/rank.c mosaicing/global_balance.c mosaicing/im_avgdxdy.c mosaicing/im_chkpair.c mosaicing/im_clinear.c mosaicing/im_improve.c mosaicing/im_initialize.c mosaicing/im_lrcalcon.c mosaicing/im_lrmerge.c mosaicing/im_lrmosaic.c mosaicing/im_remosaic.c mosaicing/im_tbcalcon.c mosaicing/im_tbmerge.c mosaicing/im_tbmosaic.c mosaicing/match.c mosaicing/merge.c mosaicing/mosaic.c mosaicing/mosaic1.c mosaicing/mosaicing.c resample/affine.c resample/interpolate.c resample/quadratic.c resample/resample.c resample/shrink.c resample/similarity.c resample/transform.c video/im_video_test.c video/video_dispatch.c include/vips/basic.h include/vips/vips.h include/vips/object.h include/vips/image.h include/vips/error.h include/vips/foreign.h include/vips/interpolate.h include/vips/header.h include/vips/operation.h include/vips/enumtypes.h include/vips/arithmetic.h include/vips/conversion.h include/vips/type.h libvips.la --output Vips-8.0.gir
/bin/sh: /usr/local/Cellar/gobject-introspection/1.42.0/bin/g-ir-scanner:
/usr/local/opt/python/bin/python: bad interpreter: No such file or directory
make[2]: *** [Vips-8.0.gir] Error 126
make[1]: *** [check-recursive] Error 1
make: *** [check-recursive] Error 1
READ THIS: https://github.com/Homebrew/homebrew/wiki/troubleshooting
If reporting this issue please do so at (not Homebrew/homebrew): https://github.com/homebrew/homebrew-science/issues
I looked for the folder /usr/local/opt/python/bin/python but i don't have it.
If i do which Python in the Terminal i get /usr/bin/Python which i guess should be where the script should look for Python.
Can i somehow alias the correct folder to allow libvips to build?


I fixed the error by installing python via homebrew aswell.
So i did homebrew install python first and then brew install homebrew/science/vips --with-webp --with-graphicsmagick afterwards.

Related

"configure: error: C compiler cannot create executables" When trying to upgrade Python to latest version

New at this so please bare with me. Following these instructions trying update to the latest version of Python
https://www.vultr.com/docs/update-python3-on-centos/
I get all the way to step 2.5 and get the following:
[user1#localhost Python-3.9.6]$ ./configure --enable-optimizations
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for python3.9... no
checking for python3... python3
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... "linux"
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/home/user1/Python-3.9.6':
configure: error: C compiler cannot create executables
See `config.log' for more details
My config.log file
## --------- ##
## Platform. ##
## --------- ##
hostname = localhost.localdomain
uname -m = x86_64
uname -r = 3.10.0-1160.76.1.el7.x86_64
uname -s = Linux
uname -v = #1 SMP Wed Aug 10 16:21:17 UTC 2022
/usr/bin/uname -p = x86_64
/bin/uname -X = unknown
/bin/arch = x86_64
/usr/bin/arch -k = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo = unknown
/bin/machine = unknown
/usr/bin/oslevel = unknown
/bin/universe = unknown
PATH: /opt/rh/devtoolset-8/root/usr/bin
PATH: /usr/local/bin
PATH: /usr/bin
PATH: /usr/local/sbin
PATH: /usr/sbin
PATH: /home/user1/.local/bin
PATH: /home/user1/bin
## ----------- ##
## Core tests. ##
## ----------- ##
configure:2848: checking build system type
configure:2862: result: x86_64-pc-linux-gnu
configure:2882: checking host system type
configure:2895: result: x86_64-pc-linux-gnu
configure:2925: checking for python3.9
configure:2955: result: no
configure:2925: checking for python3
configure:2941: found /usr/bin/python3
configure:2952: result: python3
configure:3046: checking for --enable-universalsdk
configure:3093: result: no
configure:3117: checking for --with-universal-archs
configure:3132: result: no
configure:3288: checking MACHDEP
configure:3339: result: "linux"
configure:3633: checking for gcc
configure:3649: found /opt/rh/devtoolset-8/root/usr/bin/gcc
configure:3660: result: gcc
configure:3889: checking for C compiler version
configure:3898: gcc --version >&5
gcc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
configure:3909: $? = 0
configure:3898: gcc -v >&5
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-8/root/usr --mandir=/opt/rh/devtoolset-8/root/usr/share/man --infodir=/opt/rh/devtoolset-8/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-8.3.1-20190311/obj-x86_64-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)
configure:3909: $? = 0
configure:3898: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3909: $? = 1
configure:3898: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'; did you mean '--version'?
gcc: fatal error: no input files
compilation terminated.
configure:3909: $? = 1
configure:3929: checking whether the C compiler works
configure:3951: gcc conftest.c >&5
/lib/../lib64/crt1.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
configure:3955: $? = 1
configure:3993: result: no
configure: failed program was:
| /* confdefs.h */
| #define _GNU_SOURCE 1
| #define _NETBSD_SOURCE 1
| #define __BSD_VISIBLE 1
| #define _DARWIN_C_SOURCE 1
| #define _PYTHONFRAMEWORK ""
| #define _XOPEN_SOURCE 700
| #define _XOPEN_SOURCE_EXTENDED 1
| #define _POSIX_C_SOURCE 200809L
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:3998: error: in `/home/user1/Python-3.9.6':
configure:4000: error: C compiler cannot create executables
See `config.log' for more details
## ---------------- ##
## Cache variables. ##
## ---------------- ##
ac_cv_build=x86_64-pc-linux-gnu
ac_cv_env_CC_set=
ac_cv_env_CC_value=
ac_cv_env_CFLAGS_set=
ac_cv_env_CFLAGS_value=
ac_cv_env_CPPFLAGS_set=
ac_cv_env_CPPFLAGS_value=
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_LDFLAGS_set=
ac_cv_env_LDFLAGS_value=
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_MACHDEP_set=
ac_cv_env_MACHDEP_value=
ac_cv_env_PKG_CONFIG_LIBDIR_set=
ac_cv_env_PKG_CONFIG_LIBDIR_value=
ac_cv_env_PKG_CONFIG_PATH_set=set
ac_cv_env_PKG_CONFIG_PATH_value=/opt/rh/devtoolset-8/root/usr/lib64/pkgconfig
ac_cv_env_PKG_CONFIG_set=
ac_cv_env_PKG_CONFIG_value=
ac_cv_env_PROFILE_TASK_set=
ac_cv_env_PROFILE_TASK_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_host=x86_64-pc-linux-gnu
ac_cv_prog_PYTHON_FOR_REGEN=python3
ac_cv_prog_ac_ct_CC=gcc
## ----------------- ##
## Output variables. ##
## ----------------- ##
ABIFLAGS=''
ALT_SOABI=''
AR=''
ARCH_RUN_32BIT=''
ARFLAGS=''
BASECFLAGS=''
BASECPPFLAGS=''
BINLIBDEST=''
BLDLIBRARY=''
BLDSHARED=''
BUILDEXEEXT=''
CC='gcc'
CCSHARED=''
CFLAGS=''
CFLAGSFORSHARED=''
CFLAGS_ALIASING=''
CFLAGS_NODIST=''
CONFIGURE_MACOSX_DEPLOYMENT_TARGET=''
CONFIG_ARGS=' '\''--enable-optimizations'\'' '\''PKG_CONFIG_PATH=/opt/rh/devtoolset-8/root/usr/lib64/pkgconfig'\'''
CPP=''
CPPFLAGS=''
CXX=''
DEFS=''
DEF_MAKE_ALL_RULE=''
DEF_MAKE_RULE=''
DFLAGS=''
DLINCLDIR=''
DLLLIBRARY=''
DTRACE=''
DTRACE_HEADERS=''
DTRACE_OBJS=''
DYNLOADFILE=''
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP=''
ENSUREPIP=''
EXEEXT=''
EXPORTSFROM=''
EXPORTSYMS=''
EXPORT_MACOSX_DEPLOYMENT_TARGET='#'
EXT_SUFFIX=''
FRAMEWORKALTINSTALLFIRST=''
FRAMEWORKALTINSTALLLAST=''
FRAMEWORKINSTALLAPPSPREFIX=''
FRAMEWORKINSTALLFIRST=''
FRAMEWORKINSTALLLAST=''
FRAMEWORKPYTHONW=''
FRAMEWORKUNIXTOOLSPREFIX='/usr/local'
GITBRANCH=''
GITTAG=''
GITVERSION=''
GNULD=''
GREP=''
HAS_GIT='no-repository'
HAVE_GETHOSTBYNAME=''
HAVE_GETHOSTBYNAME_R=''
HAVE_GETHOSTBYNAME_R_3_ARG=''
HAVE_GETHOSTBYNAME_R_5_ARG=''
HAVE_GETHOSTBYNAME_R_6_ARG=''
INSTALL_DATA=''
INSTALL_PROGRAM=''
INSTALL_SCRIPT=''
INSTSONAME=''
LDCXXSHARED=''
LDFLAGS=''
LDFLAGS_NODIST=''
LDLIBRARY=''
LDLIBRARYDIR=''
LDSHARED=''
LDVERSION=''
LIBC=''
LIBFFI_INCLUDEDIR=''
LIBM=''
LIBOBJS=''
LIBPL=''
LIBPYTHON=''
LIBRARY=''
LIBS=''
LIBTOOL_CRUFT=''
LINKCC=''
LINKFORSHARED=''
LIPO_32BIT_FLAGS=''
LIPO_INTEL64_FLAGS=''
LLVM_AR=''
LLVM_AR_FOUND=''
LLVM_PROFDATA=''
LLVM_PROF_ERR=''
LLVM_PROF_FILE=''
LLVM_PROF_FOUND=''
LLVM_PROF_MERGER=''
LN=''
LTLIBOBJS=''
MACHDEP='linux'
MACHDEP_OBJS=''
MAINCC=''
MKDIR_P=''
MULTIARCH=''
MULTIARCH_CPPFLAGS=''
NO_AS_NEEDED=''
OBJEXT=''
OPENSSL_INCLUDES=''
OPENSSL_LDFLAGS=''
OPENSSL_LIBS=''
OPT=''
OTHER_LIBTOOL_OPT=''
PACKAGE_BUGREPORT='https://bugs.python.org/'
PACKAGE_NAME='python'
PACKAGE_STRING='python 3.9'
PACKAGE_TARNAME='python'
PACKAGE_URL=''
PACKAGE_VERSION='3.9'
PATH_SEPARATOR=':'
PGO_PROF_GEN_FLAG=''
PGO_PROF_USE_FLAG=''
PKG_CONFIG=''
PKG_CONFIG_LIBDIR=''
PKG_CONFIG_PATH='/opt/rh/devtoolset-8/root/usr/lib64/pkgconfig'
PLATFORM_TRIPLET=''
PLATLIBDIR=''
PROFILE_TASK=''
PY3LIBRARY=''
PYTHONFRAMEWORK=''
PYTHONFRAMEWORKDIR='no-framework'
PYTHONFRAMEWORKIDENTIFIER='org.python.python'
PYTHONFRAMEWORKINSTALLDIR=''
PYTHONFRAMEWORKPREFIX=''
PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E'
PYTHON_FOR_REGEN='python3'
PY_ENABLE_SHARED=''
READELF=''
RUNSHARED=''
SED=''
SHELL='/bin/sh'
SHLIBS=''
SHLIB_SUFFIX=''
SOABI=''
SOVERSION='1.0'
SRCDIRS=''
TCLTK_INCLUDES=''
TCLTK_LIBS=''
THREADHEADERS=''
TRUE=''
TZPATH=''
UNIVERSALSDK=''
UNIVERSAL_ARCH_FLAGS=''
VERSION='3.9'
_PYTHON_HOST_PLATFORM=''
ac_ct_AR=''
ac_ct_CC='gcc'
ac_ct_CXX=''
ac_ct_READELF=''
bindir='${exec_prefix}/bin'
build='x86_64-pc-linux-gnu'
build_alias=''
build_cpu='x86_64'
build_os='linux-gnu'
build_vendor='pc'
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
exec_prefix='NONE'
host='x86_64-pc-linux-gnu'
host_alias=''
host_cpu='x86_64'
host_os='linux-gnu'
host_vendor='pc'
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
oldincludedir='/usr/include'
pdfdir='${docdir}'
prefix='NONE'
program_transform_name='s,x,x,'
psdir='${docdir}'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''
## ----------- ##
## confdefs.h. ##
## ----------- ##
/* confdefs.h */
#define _GNU_SOURCE 1
#define _NETBSD_SOURCE 1
#define __BSD_VISIBLE 1
#define _DARWIN_C_SOURCE 1
#define _PYTHONFRAMEWORK ""
#define _XOPEN_SOURCE 700
#define _XOPEN_SOURCE_EXTENDED 1
#define _POSIX_C_SOURCE 200809L
configure: exit 77
Is there something that I need to install within my Linux (Centos 7) to get this to work?
My current Python Version is 3.6.8

Apache_beam--python --error: the following arguments are required: --output-path

When run my code in python or colaborator the following error, I intall all libraries from apache beam, somebody gives in one moment this error or knows about it..
usage: aaa_users_py.py [-h] [--runner RUNNER] [--streaming] [--resource_hint RESOURCE_HINTS] [--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}] [--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check] [--performance_runtime_type_check]
[--allow_non_deterministic_key_coders] [--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle] [--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_workers DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--dataflow_endpoint DATAFLOW_ENDPOINT] [--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION] [--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL] [--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update] [--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine] [--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT] [--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS] [--enable_hot_key_logging]
[--enable_artifact_caching] [--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT] [--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_workers NUM_WORKERS] [--max_num_workers MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}] [--worker_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB] [--worker_disk_type DISK_TYPE] [--worker_region WORKER_REGION]
[--worker_zone WORKER_ZONE] [--zone ZONE] [--network NETWORK] [--subnetwork SUBNETWORK]
[--worker_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES] [--use_public_ips]
[--no_use_public_ips] [--min_cpu_platform MIN_CPU_PLATFORM] [--dataflow_worker_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE] [--experiment EXPERIMENTS]
[--number_of_worker_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS] [--profile_cpu]
[--profile_memory] [--profile_location PROFILE_LOCATION] [--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE] [--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources] [--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}] [--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES] [--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL] [--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT] [--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE] [--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS] [--sdk_worker_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS] [--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT] [--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT] [--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES] [--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14}] [--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar] [--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR] [--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL] [--spark_version {2,3}] [--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN] [--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL] [--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY] [--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL] [--s3_region_name S3_REGION_NAME] [--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl] [--input-file INPUT_FILE] --output-path OUTPUT_PATH
aaa_users_py.py: error: the following arguments are required: --output-path

It probably means that the pipeline built in this Python script has a required customized pipeline option that includes a field named --output-path. Think it as a "template" to spawn jobs by ETL data from the --input-path to the --output-path, you have to tell the pipeline where to read and write before submitting it as a job.

VSCode Jupyter Interactive Window - show long output

The VSCode Interactive window for Jupyter truncates long output:
import os
dir(os)
['CLD_CONTINUED',
'CLD_DUMPED',
'CLD_EXITED',
'CLD_TRAPPED',
'DirEntry',
'EX_CANTCREAT',
'EX_CONFIG',
'EX_DATAERR',
'EX_IOERR',
'EX_NOHOST',
'EX_NOINPUT',
'EX_NOPERM',
'EX_NOUSER',
'EX_OK',
'EX_OSERR',
'EX_OSFILE',
'EX_PROTOCOL',
'EX_SOFTWARE',
'EX_TEMPFAIL',
'EX_UNAVAILABLE',
'EX_USAGE',
'F_LOCK',
'F_OK',
'F_TEST',
'F_TLOCK',
show more (open the raw output data in a text editor) ...
'waitid',
'waitid_result',
'waitpid',
'walk',
'write',
'writev']
This is nice functionality as it stops commands that generate a lot of output from overwhelming the Interactive window.
How can I see the entire output?
I can click on the "show more" link, but the output is in a JSON format which is difficult to read.
[
{
"metadata": {
"outputType": "execute_result",
"executionCount": 8,
"metadata": {}
},
"outputItems": [
{
"mimeType": "text/plain",
"data": "['CLD_CONTINUED',\n 'CLD_DUMPED',\n 'CLD_EXITED',\n 'CLD_TRAPPED',\n 'DirEntry',\n 'EX_CANTCREAT',\n 'EX_CONFIG',\n 'EX_DATAERR',\n 'EX_IOERR',\n 'EX_NOHOST',\n 'EX_NOINPUT',\n 'EX_NOPERM',\n 'EX_NOUSER',\n 'EX_OK',\n 'EX_OSERR',\n 'EX_OSFILE',\n 'EX_PROTOCOL',\n 'EX_SOFTWARE',\n 'EX_TEMPFAIL',\n 'EX_UNAVAILABLE',\n 'EX_USAGE',\n 'F_LOCK',\n 'F_OK',\n 'F_TEST',\n 'F_TLOCK',\n 'F_ULOCK',\n 'GRND_NONBLOCK',\n 'GRND_RANDOM',\n 'MutableMapping',\n 'NGROUPS_MAX',\n 'O_ACCMODE',\n 'O_APPEND',\n 'O_ASYNC',\n 'O_CLOEXEC',\n 'O_CREAT',\n 'O_DIRECT',\n 'O_DIRECTORY',\n 'O_DSYNC',\n 'O_EXCL',\n 'O_LARGEFILE',\n 'O_NDELAY',\n 'O_NOATIME',\n 'O_NOCTTY',\n 'O_NOFOLLOW',\n 'O_NONBLOCK',\n 'O_PATH',\n 'O_RDONLY',\n 'O_RDWR',\n 'O_RSYNC',\n 'O_SYNC',\n 'O_TMPFILE',\n 'O_TRUNC',\n 'O_WRONLY',\n 'POSIX_FADV_DONTNEED',\n 'POSIX_FADV_NOREUSE',\n 'POSIX_FADV_NORMAL',\n 'POSIX_FADV_RANDOM',\n 'POSIX_FADV_SEQUENTIAL',\n 'POSIX_FADV_WILLNEED',\n 'PRIO_PGRP',\n 'PRIO_PROCESS',\n 'PRIO_USER',\n 'P_ALL',\n 'P_NOWAIT',\n 'P_NOWAITO',\n 'P_PGID',\n 'P_PID',\n 'P_WAIT',\n 'PathLike',\n 'RTLD_DEEPBIND',\n 'RTLD_GLOBAL',\n 'RTLD_LAZY',\n 'RTLD_LOCAL',\n 'RTLD_NODELETE',\n 'RTLD_NOLOAD',\n 'RTLD_NOW',\n 'R_OK',\n 'SCHED_BATCH',\n 'SCHED_FIFO',\n 'SCHED_IDLE',\n 'SCHED_OTHER',\n 'SCHED_RESET_ON_FORK',\n 'SCHED_RR',\n 'SEEK_CUR',\n 'SEEK_DATA',\n 'SEEK_END',\n 'SEEK_HOLE',\n 'SEEK_SET',\n 'ST_APPEND',\n 'ST_MANDLOCK',\n 'ST_NOATIME',\n 'ST_NODEV',\n 'ST_NODIRATIME',\n 'ST_NOEXEC',\n 'ST_NOSUID',\n 'ST_RDONLY',\n 'ST_RELATIME',\n 'ST_SYNCHRONOUS',\n 'ST_WRITE',\n 'TMP_MAX',\n 'WCONTINUED',\n 'WCOREDUMP',\n 'WEXITED',\n 'WEXITSTATUS',\n 'WIFCONTINUED',\n 'WIFEXITED',\n 'WIFSIGNALED',\n 'WIFSTOPPED',\n 'WNOHANG',\n 'WNOWAIT',\n 'WSTOPPED',\n 'WSTOPSIG',\n 'WTERMSIG',\n 'WUNTRACED',\n 'W_OK',\n 'XATTR_CREATE',\n 'XATTR_REPLACE',\n 'XATTR_SIZE_MAX',\n 'X_OK',\n '_Environ',\n '__all__',\n '__builtins__',\n '__cached__',\n '__doc__',\n '__file__',\n '__loader__',\n '__name__',\n '__package__',\n '__spec__',\n '_execvpe',\n '_exists',\n '_exit',\n '_fspath',\n '_fwalk',\n '_get_exports_list',\n '_putenv',\n '_spawnvef',\n '_unsetenv',\n '_wrap_close',\n 'abc',\n 'abort',\n 'access',\n 'altsep',\n 'chdir',\n 'chmod',\n 'chown',\n 'chroot',\n 'close',\n 'closerange',\n 'confstr',\n 'confstr_names',\n 'cpu_count',\n 'ctermid',\n 'curdir',\n 'defpath',\n 'device_encoding',\n 'devnull',\n 'dup',\n 'dup2',\n 'environ',\n 'environb',\n 'errno',\n 'error',\n 'execl',\n 'execle',\n 'execlp',\n 'execlpe',\n 'execv',\n 'execve',\n 'execvp',\n 'execvpe',\n 'extsep',\n 'fchdir',\n 'fchmod',\n 'fchown',\n 'fdatasync',\n 'fdopen',\n 'fork',\n 'forkpty',\n 'fpathconf',\n 'fsdecode',\n 'fsencode',\n 'fspath',\n 'fstat',\n 'fstatvfs',\n 'fsync',\n 'ftruncate',\n 'fwalk',\n 'get_blocking',\n 'get_exec_path',\n 'get_inheritable',\n 'get_terminal_size',\n 'getcwd',\n 'getcwdb',\n 'getegid',\n 'getenv',\n 'getenvb',\n 'geteuid',\n 'getgid',\n 'getgrouplist',\n 'getgroups',\n 'getloadavg',\n 'getlogin',\n 'getpgid',\n 'getpgrp',\n 'getpid',\n 'getppid',\n 'getpriority',\n 'getrandom',\n 'getresgid',\n 'getresuid',\n 'getsid',\n 'getuid',\n 'getxattr',\n 'initgroups',\n 'isatty',\n 'kill',\n 'killpg',\n 'lchown',\n 'linesep',\n 'link',\n 'listdir',\n 'listxattr',\n 'lockf',\n 'lseek',\n 'lstat',\n 'major',\n 'makedev',\n 'makedirs',\n 'minor',\n 'mkdir',\n 'mkfifo',\n 'mknod',\n 'name',\n 'nice',\n 'open',\n 'openpty',\n 'pardir',\n 'path',\n 'pathconf',\n 'pathconf_names',\n 'pathsep',\n 'pipe',\n 'pipe2',\n 'popen',\n 'posix_fadvise',\n 'posix_fallocate',\n 'pread',\n 'putenv',\n 'pwrite',\n 'read',\n 'readlink',\n 'readv',\n 'remove',\n 'removedirs',\n 'removexattr',\n 'rename',\n 'renames',\n 'replace',\n 'rmdir',\n 'scandir',\n 'sched_get_priority_max',\n 'sched_get_priority_min',\n 'sched_getaffinity',\n 'sched_getparam',\n 'sched_getscheduler',\n 'sched_param',\n 'sched_rr_get_interval',\n 'sched_setaffinity',\n 'sched_setparam',\n 'sched_setscheduler',\n 'sched_yield',\n 'sendfile',\n 'sep',\n 'set_blocking',\n 'set_inheritable',\n 'setegid',\n 'seteuid',\n 'setgid',\n 'setgroups',\n 'setpgid',\n 'setpgrp',\n 'setpriority',\n 'setregid',\n 'setresgid',\n 'setresuid',\n 'setreuid',\n 'setsid',\n 'setuid',\n 'setxattr',\n 'spawnl',\n 'spawnle',\n 'spawnlp',\n 'spawnlpe',\n 'spawnv',\n 'spawnve',\n 'spawnvp',\n 'spawnvpe',\n 'st',\n 'stat',\n 'stat_float_times',\n 'stat_result',\n 'statvfs',\n 'statvfs_result',\n 'strerror',\n 'supports_bytes_environ',\n 'supports_dir_fd',\n 'supports_effective_ids',\n 'supports_fd',\n 'supports_follow_symlinks',\n 'symlink',\n 'sync',\n 'sys',\n 'sysconf',\n 'sysconf_names',\n 'system',\n 'tcgetpgrp',\n 'tcsetpgrp',\n 'terminal_size',\n 'times',\n 'times_result',\n 'truncate',\n 'ttyname',\n 'umask',\n 'uname',\n 'uname_result',\n 'unlink',\n 'unsetenv',\n 'urandom',\n 'utime',\n 'wait',\n 'wait3',\n 'wait4',\n 'waitid',\n 'waitid_result',\n 'waitpid',\n 'walk',\n 'write',\n 'writev']"
}
]
}
]
I can take the raw output and massage it in a text editor or other tool. This is time consuming, so I'm looking for an approach that is built into VSCode.

This is a bug that has been reported to the VS Code team and has a fix in already. The fix is in VS Code - Insiders (you can install that side by side with stable) to check. The fix should be in stable VS Code in the next full release:
https://github.com/microsoft/vscode/issues/130512

Installing the JSON formatter (or other similar extension) should work to prettify it in VSCode itself.

Segmentation Violation when running MATLAB r2017a with caffe (python script)

I get the following crash when running Matlab r2017a with caffe from python script (selective_search.py from https://github.com/sergeyk/selective_search_ijcv_with_python). I run it with user root on ubuntu 17.10 and CPU mode only!. I also tried to run *.m file with matlab and it works fine. you can see below more details. also you can see below the matlab script parameters (['matlab', '-softwareopengl' atc)
root#desktop-home:/home/USERNAME/deepLearning/hed/python# python detect.py /home.USERNAME/Desktop/input.txt /home/USERNAME/Desktop/output.csv
CPU mode !!!!!!
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0102 22:59:10.099709 3052 net.cpp:50] Initializing net from parameters:
name: "FCN"
input: "data"
input_dim: 1
input_dim: 3
...
...
...
I0102 22:59:10.636869 3052 upgrade_proto.cpp:620] Attempting to upgrade input file specified using deprecated V1LayerParameter: /home/USERNAME/deepLearning/hed/examples/hed/5stage-vgg.caffemodel
I0102 22:59:10.847916 3052 upgrade_proto.cpp:628] Successfully upgraded file specified using deprecated V1LayerParameter
Loading input...
selective_search_rcnn({'/home/USERNAME/Desktop/camera_500_X_500.jpg'}, '/tmp/tmphzvr5w.mat')
['matlab', '-softwareopengl', '-nojvm', '-r', "try; selective_search_rcnn({'/home/USERNAME/Desktop/camera_500_X_500.jpg'}, '/tmp/tmphzvr5w.mat'); catch; exit; end; exit"]
['matlab', '-softwareopengl', '-nojvm', '-r', "try; selective_search_rcnn({'/home/USERNAME/Desktop/camera_500_X_500.jpg'}, '/tmp/tmphzvr5w.mat'); catch; exit; end; exit"]
/home/USERNAME/deepLearning/hed/python/selective_search_ijcv_with_python
No protocol specified
------------------------------------------------------------------------
Segmentation violation detected at Tue Jan 2 22:59:13 2018
------------------------------------------------------------------------
Configuration:
Crash Decoding : Disabled - No sandbox or build area path
Crash Mode : continue (default)
Current Graphics Driver: Unknown software
Current Visual : None
Default Encoding : UTF-8
Deployed : false
GNU C Library : 2.26 stable
Host Name : USERNAME-home
MATLAB Architecture : glnxa64
MATLAB Entitlement ID: 3324953
MATLAB Root : /usr/local/MATLAB/R2017a
MATLAB Version : 9.2.0.556344 (R2017a)
OpenGL : software
Operating System : Linux 4.13.0-21-generic #24-Ubuntu SMP Mon Dec 18 17:29:16 UTC 2017 x86_64
Processor ID : x86 Family 6 Model 158 Stepping 9, GenuineIntel
Window System : No active display
Fault Count: 1
Abnormal termination:
Segmentation violation
Register State (from fault):
RAX = 00007f80b7df83d0 RBX = 0000000000000000
RCX = 0000000000000000 RDX = 00007f80b7df8600
RSP = 00007f80b7df82a0 RBP = 00007f80b7df8790
RSI = 00007f80bc3b6833 RDI = 00007f80b7df8390
R8 = 0000000000000000 R9 = 0000000000000000
R10 = 00000000000003ad R11 = 00007f80dbe55fe0
R12 = 00007f80b7df8390 R13 = 00007f80b7df8600
R14 = 00007f80b7df88a0 R15 = 00007f80dc10a7a0
RIP = 00007f80bc37c628 EFL = 0000000000010202
CS = 0033 FS = 0000 GS = 0000
Stack Trace (from fault):
[ 0] 0x00007f80bc37c628 bin/glnxa64/libmwddux_impl.so+00661032
[ 1] 0x00007f80bc356be9 bin/glnxa64/libmwddux_impl.so+00506857
[ 2] 0x00007f80bc3466b6 bin/glnxa64/libmwddux_impl.so+00439990
[ 3] 0x00007f80bc34c85a bin/glnxa64/libmwddux_impl.so+00464986
[ 4] 0x00007f80bc348370 bin/glnxa64/libmwddux_impl.so+00447344
[ 5] 0x00007f80d7cbe7d2 bin/glnxa64/libmwms.so+03168210
[ 6] 0x00007f80bc34808a bin/glnxa64/libmwddux_impl.so+00446602
[ 7] 0x00007f80da64bf65 /usr/local/MATLAB/R2017a/bin/glnxa64/libboost_thread.so.1.56.0+00073573
[ 8] 0x00007f80db6757fc /lib/x86_64-linux-gnu/libpthread.so.0+00030716
[ 9] 0x00007f80db3a2b0f /lib/x86_64-linux-gnu/libc.so.6+01133327 clone+00000063
[ 10] 0x0000000000000000 <unknown-module>+00000000
If this problem is reproducible, please submit a Service Request via:
http://www.mathworks.com/support/contact_us/
A technical support engineer might contact you with further information.
Thank you for your help.** This crash report has been saved to disk as /root/matlab_crash_dump.3072-1 **
MATLAB is exiting because of fatal error
Traceback (most recent call last):
File "detect.py", line 178, in <module>
main(sys.argv)
File "detect.py", line 149, in main
detections = detector.detect_selective_search(inputs)
File "/home/USERNAME/deepLearning/hed/python/caffe/detector.py", line 120, in detect_selective_search
cmd='selective_search_rcnn'
File "/home/USERNAME/deepLearning/hed/python/selective_search_ijcv_with_python/selective_search.py", line 51, in get_windows
raise Exception("Matlab script did not exit successfully!")
Exception: Matlab script did not exit successfully!

Tensorflow Object Detection Jupyter Notebook no detection

I tried to run the Jupyter Notebook example for the object detection of tensorflow (tutorial) but there are no detections. I printed the scores and it's seems to work but the results are very bad. Does anyone have an idea what I might have done wrong.
print(scores[0]):
[ 0.03587309 0.02224856 0.01864638 0.01096715 0.0100315
0.0065446
0.00633551 0.00534311 0.00495995 0.00410238 0.00362363 0.00339175
0.00308251 0.0030337 0.00293387 0.00277085 0.00269581 0.00266825
0.00263924 0.00263331 0.00258721 0.00240822 0.00225823 0.00186966
0.00184308 0.00180467 0.00177474 0.00173643 0.0017281 0.00171935
0.00171891 0.00170284 0.00163754 0.00162967 0.00160267 0.00156545
0.00153614 0.00140936 0.00132406 0.00131524 0.00131041 0.00129431
0.00125819 0.0012553 0.00122365 0.00119179 0.00115673 0.00115186
0.00112368 0.00107096 0.00105803 0.00104337 0.00102719 0.00102337
0.00100349 0.00097767 0.0009685 0.00092741 0.00088506 0.00087696
0.0008734 0.00084825 0.00084135 0.00083512 0.00083396 0.00082068
0.00080583 0.00078979 0.00078059 0.00077475 0.00075449 0.00074426
0.00074421 0.00070195 0.00068741 0.00068138 0.00067261 0.00067125
0.00067032 0.00066041 0.0006473 0.00064205 0.00061964 0.00061793
0.00060834 0.00060468 0.00059547 0.00059478 0.00059461 0.00059436
0.00059426 0.00059411 0.00059406 0.00059392 0.00059365 0.00059351
0.00059191 0.00058798 0.00058682 0.00058148]
[ 0.01044157 0.00982138 0.00942336 0.00846517 0.00613665 0.00398568
0.00357755 0.00300539 0.00255862 0.00236576 0.00232631 0.00220291
0.00185227 0.00163544 0.00159791 0.00145071 0.0014366 0.0014137
0.00122685 0.00118978 0.00108457 0.00104252 0.00099215 0.00096401
0.0008708 0.00084774 0.00080484 0.00078507 0.00078379 0.00076875
0.00072774 0.00071732 0.00071343 0.00070812 0.00069253 0.0006762
0.00067269 0.00059905 0.00059367 0.000588 0.00056114 0.0005504
0.00051472 0.00051055 0.00050973 0.00048484 0.00047297 0.00046204
0.00044787 0.00043259 0.00042987 0.00042673 0.00041978 0.00040494
0.00040087 0.00039576 0.00039059 0.00037274 0.00036828 0.00036417
0.0003612 0.00034645 0.00034479 0.00034078 0.00033771 0.00033605
0.0003333 0.0003304 0.0003294 0.00032325 0.00031787 0.00031773
0.00031748 0.00031741 0.00031732 0.00031729 0.00031724 0.00031722
0.00031717 0.00031708 0.00031702 0.00031579 0.00030416 0.00030222
0.00029739 0.00029726 0.00028289 0.00026527 0.00026325 0.00024584
0.00024221 0.00024156 0.0002391 0.00023335 0.00021617 0.0002001
0.00019127 0.00018342 0.00017271 0.00015507]
I'm running the example with tensorflow 1.4, python 3.5 and I tested the installation as suggested.

I had the same issue. I found in a post that you have to change:
MODEL_NAME = 'ssd_mobilenet_v1_coco_2017_11_08'
To:
MODEL_NAME = 'ssd_mobilenet_v1_coco_11_06_2017'
and it worked fine.
Original answer: https://stackoverflow.com/a/47332228/8954260

Categories

Resources