Develop Reference

High availability HDFS client python

In HDFSCLI docs it says that it can be configured to connect to multiple hosts by adding urls separated with semicolon ; (https://hdfscli.readthedocs.io/en/latest/quickstart.html#configuration).
I use kerberos client, and this is my code -
from hdfs.ext.kerberos import KerberosClient hdfs_client = KerberosClient('http://host01:50070;http://host02:50070')
And when I try to makedir for example, I get the following error - requests.exceptions.InvalidURL: Failed to parse: http://host01:50070;http://host02:50070/webhdfs/v1/path/to/create

Apparently the version of hdfs I installed was old, the code didn't work with version 2.0.8, and it did work with version 2.5.7

Error while uploading instructions using PyFirmata package

So I am trying to learn the PyFirmata module of python that allows us to write codes in Python and upload it to an Arduino Board.
I am learning it from this website.
Code that causes the inbuilt LED to blink:
import pyfirmata
import time
board = pyfirmata.Arduino('/dev/ttyACM0') #This Line gives error
while True:
board.digital[13].write(1)
time.sleep(1)
board.digital[13].write(0)
time.sleep(1)
But, When I run it( in Sublime Text 3) , I get this error:
raise SerialException("could not open port {!r}: {!r}".format(self.portstr, ctypes.WinError()))
serial.serialutil.SerialException: could not open port '/dev/ttyACM0': FileNotFoundError(2, 'The
system cannot find the path specified.', None, 3)
[Finished in 7.0s]
Can Anyone help me to fix this error?

The port you are trying to access doesn't exist or your program doesn't have permission to access it.
First checkout whether the port exist or not. If you have Arduino software installed, you can check from it. In case it's missing, another port should appear, smth like /dev/ttyACM*, where * is any number. Choose it and try running.
In case it exists, but the program still doesn't work visit the official arduino site.
This problem is solved in two command. Open terminal and type:
ls -l /dev/ttyACM*
Then you'll get something like this:
crw-rw---- 1 root dialout 188, 0 5 apr 23.01 ttyACM0
Here what we need is dialout. After the above command, type the below command, and the restart your device.
sudo usermod -a -G your_username
This should work, but there are cases, when it doesn't. Then you need to add your port to root access:
=============================================================================
cd ~/etc/udev/rules.d
In this directory, create a new .rules file and add the following to it:
KERNEL=="ttyACM0", MODE="0666"
This should work.

gaierror: [Errno 8] nodename nor servname provided, or not known (with macOS Sierra)

socket.gethostbyname(socket.gethostname()) worked well on OS X El Capitan. However, it's not working now after the Mac updated to macOS Sierra.
Thanks!
import socket
socket.gethostbyname(socket.gethostname())
Traceback (most recent call last):
File "<pyshell#26>", line 1, in <module>
socket.gethostbyname(socket.gethostname())
gaierror: [Errno 8] nodename nor servname provided, or not known

There is an answer in OP's comments based on another question,
which requires to edit /etc/hosts, i.e echo 127.0.0.1 $HOSTNAME >> /etc/hosts.
Alternatively, you can use socket.gethostbyname('localhost'), assuming that localhost is what you need and it is defined in your /etc/hosts.

Same problem tome.
I change the code to:
import socket
socket.gethostbyname("")
And it works now.

I had the same problem today (with MacOS Mojave) and was able to fix it using the link that #FBL provided in the comments.
Specifically, I just had to go to System Preferences -> Sharing and enable sharing. (I enabled printer sharing). Then I verified it was working via ping $HOST. Afterwards, I was able to turn sharing back off and everything (both ping $HOST and python -c 'import socket; print socket.gethostbysocketname(socket.gethostname())) still worked.

Rather than using the localhost, use the computer name as specified in /etc/hosts.
For example, /etc/host would have something like this:
127.0.0.1 ET02282-MAC.local
127.0.0.1 localhost
And in your connection use "ET02282-MAC.local"

It seems like a python bug? You need to set the hostname & ipv4 , after this, you can fix it.
import socket
hostname = socket.gethostname()
print(hostname) >> samzong
# make sure do it
!echo "127.0.0.1 samzong" >> /private/etc/hosts

Another simple solution is
Go terminal paste sudo nano /etc/hosts
find your ip adress that is needed , IPv4 etc.
add a new line like
190.123.123.3 Yourterminalname.local
it worked for me

I was having the same issue. I was not able to ping localhost.
ping localhost
>> error ping: cannot resolve localhost: Unknown host
I changed my hosts file.
sudo nano /etc/hosts
Add this to the file (use Tab/4 spaces to separate>> 127.0.0.1{tab/4spaces}localhost) Save it and restart the terminal.
127.0.0.1 localhost
To confirm if everything works fine, try pinging your localhost
ping localhost

If you are using Jupyter notebook or not try this
IP4=str(ipaddress.ip_address(8888))
IPAddr = socket.gethostbyname(IP4)
don't forget to import ipaddress & socket

I was having the same issue.
macOS Catalina
Version: 10.15.7
The solution was::
import socket
if env == "PRODUCTION":
ip_address = socket.gethostbyname(socket.gethostname())
else:
ip_address = ""

shutil.copy2 on a cifs share only works as root

-when writing the file to the cifs share from linux as root using python 2.7 shutil, no issue
-when writing the file manually using cp as a normal user, no issues
-when using python shutil.copy or copy2 as a normal user:
File "/usr/lib64/python2.7/shutil.py", line 120, in copy
copymode(src, dst)
File "/usr/lib64/python2.7/shutil.py", line 91, in copymode
os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/mnt/RW_H-drive/file.csv

I was wrong about Debian vs CentOS in the comment above, the Debian system drive died and I'm running my jobs on an enterprise box where the only possibilities are CentOS or SUSE so I can't compare /etc/fstab files. I was trying to mount the share using dir_mode=0777,file_mode=0777 without the uid=xxxx. I don't understand why but when you specify dir_mode and file_mode you get the permissions you expect and can write files to the share manually but using shutil fails with the error above. If you remove dir_mode and file_mode and use user uid=xxxx where xxxx is the user ID of any locally recognozised user (Local,LDAP or NIS) then permissions work as expected manually (cp or touch command etc) and also via python with shutil. I may have had the same issue with Debian when I first setup the fstab file that's why I had this little bell telling me to try uid, nothing else. Bizare thing is that the UID of the user in fstab and the cifs user are unrelated as far as I recall, they don't even have to have the same name. Also both mount lines work when you change files on the share as root user.
Example:
//nasx/sharex/ /mnt/RW_H-drive/ cifs credentials=/root/smbcredentials,_netdev,uid=4321 0 0
Rather than:
//nasx/sharex/ /mnt/RW_H-drive/ cifs credentials=/root/smbcredentials,_netdev,file_mode=0777,dir_mode=0777 0 0

what is this error when deploying google python-appengine application?

this is the error message .. any help will be appreciated .
2011-02-23 23:09:11 Running command: "['C:\\Python25\\pythonw.exe', '-u', 'C:\\Program Files\\Google\\google_appengine\\appcfg.py', '--no_cookies', u'--email=adham587#gmail.com', '--passin', 'update', 'C:\\Users\\adham\\Desktop\\images']"
Application: refacingme; version: 1.
Server: appengine.google.com.
Scanning files on local disk.
Initiating update.
2011-02-23 23:09:42,223 WARNING appengine_rpc.py:405 ssl module not found.
Without the ssl module, the identity of the remote host cannot be verified, and
connections may NOT be secure. To fix this, please install the ssl module from
http://pypi.python.org/pypi/ssl .
To learn more, see http://code.google.com/appengine/kb/general.html#rpcssl .
Password for XXXXX#gmail.com: Error 409: --- begin server output ---
Another transaction by user XXXXXXXX is already in progress for this app and major version. That user can undo the transaction with appcfg.py's "rollback" command.
--- end server output ---
2011-02-23 23:09:46 (Process exited with code 1)
You can close this window now.

This problem can occur if an update is started and does not finish for whatever reason. As the error message notes, the correct thing to do is to give appcfg.py the rollback command. That will undo the failed changes and reset your app so it is ready for an update.

Try
easy_install ssl
The message says that module is missing.
#Adam: duly noted. I apologize for missing the "Warning text".
I believe his error will go away in a while since another operation might be going on at the same time. He should install ssl module nevertheless. If after a while and it doesn't help, he should perform a rollback.
Google App Engine: appcfg.py rollback