I have a project that uses Google Dataflow. I have been successfully using the following command (and commands like it) for months to deploy templates.
.venv/bin/python -m dataflow.registry_files.delimited_file --runner=DataflowRunner --region=us-central1 --project=myproject --staging_location=gs://mybucket-staging/staging/gr265 --template_location=gs://mybucket-code/templates/gr265 --temp_location=gs://mybucket-staging/temp/gr265 --no_use_public_ips --save_main_session --setup_file=dataflow/setup.py --projectId=myproject --datasetId=padl_staging --tableId=gr265 --configFile=gs://mybucket-code/registry/registry_files.yaml --fileType=gr265
This command continues to work on windows 10 and Debian machines in my team.
Since I upgraded to catalina (10.15.1, with python 3.7.5 and apache-beam==2.16.0 ) I get the following error:
[libprotobuf ERROR google/protobuf/descriptor_database.cc:58] File already exists in database:
[libprotobuf FATAL google/protobuf/descriptor.cc:1370] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
Abort trap: 6
I have done all of the following, with many reboots:
Run xcode-select --install
Run brew update-reset, brew update, brew upgrade, and brew reinstall python all to no effect (except after brew update-reset, brew doctor works again)
Run brew uninstall protobuf and brew install protobuf
Run pip3 uninstall protobuf outside of the virtual environments
Deleted and re-created my virtual environments from their requirements files.
Several bits of voodoo involving /usr/local/include that I located elsewhere on stack overflow that didn't help.
I wondered if this was just my machine, and unfortunately was able to reproduce it on the other macOS Catalina laptop in my team, but not the laptop still running macOS Mojave.
According with the Apache Beam Issue 8368 this issue is related with a pyarrow version, you have to test with pyarrow 0.15.1 beam since is the one which works on MacOS 10.15 as is mentioned in this link.
Please let us know how it works.
I just leave it here, because it is a common problem has not resolved by maintainers and I didn't found the closest convenient solution in the time.
https://github.com/protocolbuffers/protobuf/issues/1941
https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=721791;filename=protobuf-2.4.1-3.1.debdiff;msg=5
The patch actually quite does not help, because in the next step the protobuf does hang in the Run function and call does not return.
The closest solution for myself was to completely avoid of any double linkage with the protobuf at any cost.
I am having the following issue with psycopg2. I am connect to a remote postgres server that is running 9.4.5.
I get the following error when I use SSLmode:required
OperationalError: (psycopg2.OperationalError) sslmode value "require" >invalid when SSL support is not compiled in
I am running Anaconda fully updated with psycopg2. I have deleted and reinstall Anaconda, removed and re-installed pip install psycopg2. Followed the brew install / brew uninstall suggested.
I have looked into the following threads from stack ==>
Psycopg2 Python SSL Support is not compiled in
as well as many other resources on the internet without success.
If anyone can help me out that would be fantastic - I feel like I have maxed out on my own abiliti to resolve this - hence I am posting up on Stack! Thanks
#cel resolved this for me in the comment thread above.
After working through all the different options it looks like using the
conda install -c conda-forge psycopg2=2.6.2
solution is the workaround that indeed worked for me.
If anyone has other resolutions let me know and I will add them here. I hope noone has the same issues.
I'm having some issues running the certbot-auto application on a CentOS 6 server, which has both Python 2.6 (/usr/bin/python) and 2.7 (/usr/bin/python2.7) installed.
A copy of the output from running ./certbot-auto with no arguments can be found in this paste: http://pastebin.com/g7WaZUra
The error code output is similiar to this question but my issue is almost certainly not memory related as this is a dedicated server with 32GB RAM and just a few low-traffic sites on it.
Managed to solve the issue, the development dependencies for Python 2.7 were missing. Fix by running yum install python27-devel.
Firstly I'm a first year computer science major so inexpeirence maybe my problem here but I can't find a solid solution after about 4 hours of googling and stumbling about.
I'm trying to install a library from github to use in python3.4 to play around with and learn about using api's. I'm using a Chromebook (Asus C300m) with ubuntu dual booted for coding.
I've tried various methods of using pip to install the library. It seems using the default pip command is broken for my version of ubuntu and throws an "incomplete read" error.
I tried using the get-pip.py file instead and that seems to work. But that installs the library only to the 2.7 version of python that came with the OS install not the 3.4 version I have to use for class and am currently learning in.
I also tried installing and using pip3 but that throws an "incomplete read" error just like the default pip command did.
I'm very willing to link my failed console commands but there have been so many I'm not sure which would be most helpful.
Thanks for any help anyone can provide.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to use/install gcc on Mac OS X 10.8 / Xcode 4.4
I cannot install any Python packages using easy_insall or pip, because of the following error. I've looked everywhere, and seen several variations of this error, but have not found a solution that is easy to understand/follow. Any help is much appreciated!
I'm running on Mac OS 10.8.1
Python version 2.7.3
Xcode version 4.5.2 (with Command Line Tools installed)
...if you need any more information in order to figure out the problem, please ask!
$ easy_install pil
Searching for pil
Reading http://pypi.python.org/simple/pil/
Reading http://www.pythonware.com/products/pil
Reading http://effbot.org/zone/pil-changes-115.htm
Reading http://effbot.org/downloads/#Imaging
Best match: PIL 1.1.7
Downloading http://effbot.org/media/downloads/PIL-1.1.7.tar.gz
Processing PIL-1.1.7.tar.gz
Writing /var/folders/9q/bvqtzkbx1hg1934b36zgk0y40000gn/T/easy_install-wfZs_Y/PIL-1.1.7/setup.cfg
Running PIL-1.1.7/setup.py -q bdist_egg --dist-dir /var/folders/9q/bvqtzkbx1hg1934b36zgk0y40000gn/T/easy_install-wfZs_Y/PIL-1.1.7/egg-dist-tmp-DXWOmC
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
--- using frameworks at /System/Library/Frameworks
unable to execute gcc-4.0: No such file or directory
error: Setup script exited with error: command 'gcc-4.0' failed with exit status 1
Not sure if this helps, but when I run sudo port select --list gcc, I get the following:
Available versions for gcc:
llvm-gcc42
mp-gcc45
none (active)
This is just a guess, based on the fact that you must have MacPorts installed (or sudo port select --list gcc would just give you a port: command not found error), but I suspect you've installed a MacPorts Python 2.7.3, and you're trying to install PIL for that.
If you want to add packages to a MacPorts Python installation, you should always first check to see if there's a port for it, and, if so, use it:
port search pil
In this case, you'll find there is a py27-pil. Install that.
Meanwhile, when this doesn't apply, you should almost always use pip instead of easy_install (there are a few exceptions, notably readline), and you need to make sure your path is set up properly so you're running the one you think you are, or you may end up running scripts set up for MacPorts python with Apple python.
However, I also suspect you have an out-of-date MacPorts installation that you installed for an earlier version of OS X, and didn't follow the upgrade instructions when upgrading to Mountain Lion. If this is true, the only decent fix is to get a list of installed ports, sudo rm -rf /opt/local, run the latest MacPorts installer, then sudo port install all of the previously-installed ports.
On top of that, last time I used MacPorts Python (which is quite a long time ago, so it may not be current information), it didn't install things into /Library (IIRC, it set up /opt/local/Library as an extra library directory instead), which implies that you actually have at least three Python 2.7 installations: Apple's (in /System/Library and /usr), MacPorts (in /opt/local), and some other version (in /Library and probably /usr/local), and you're using the second one's easy_install when you meant to use the third.
If you don't know how to deal with this kind of multiple-copies-of-the-same-thing insanity, and you don't have a very, very good reason to invite it, the right answer is to not install these other Pythons (and uninstall any you already have). Do you actually need MacPorts python at all? Can you just use the built-in Python? If not, why not? And can you at least use the python.org Python?
If you don't want to fix any of the base problems, and you really want to install things into a MacPorts Python that expects you to have gcc-4.0 around, you might be able to install the macports gcc4.0 port, although you may have to trick things into believing that macports-gcc-4.0 is really gcc-4.0. (There might be a way to do that properly with MacPorts; if not, a symlink might work.) But this is probably a bad idea, and I wouldn't be surprised if you came back with "That seemed to work but then I got this other error 348 minutes into rebuilding stuff", and nobody would be able to help you.