Develop Reference

Failed to create multiple containers using docker python SDk

I am using python docker SDK to create multiple chrome containers. Below is my script
Here first I have pull docker image then try to create 2 containers out it. But it is failing with an error message that ports are already in use but I am incrementing the ports value as per container count.
import docker, sys
class CreateContainer:
def __init__(self):
self.client = CreateContainer.create_client()
#staticmethod
def create_client():
client = docker.from_env()
return client
def pull_image(self, image_name):
image = client.images.pull(image_name)
print(image.name)
def create_containers(self, image, container_name, expose_port, countainer_count=1):
container = self.client.containers.run(
image,
name=container_name,
hostname=container_name,
ports=expose_port,
detach=True
)
for line in container.logs():
print(line)
return container
if __name__ == '__main__':
threads = int(sys.argv[1])
c_obj = CreateContainer()
for i in range(1, threads+1):
c_obj.create_containers("selenium/standalone-chrome", "Chrome_{0}".format(i), expose_port={5550+i:4444})
------Run----------
python test.py 2
------error-----
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\site-packages\docker\api\client.py", line 268, in _raise_for_status
response.raise_for_status()
File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localnpipe/v1.40/containers/260894dbaec6946e5f31fdbfb5307182d2f621c12a38f328f6efac58df58854d/start
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Desktop\FreeLance\Utility\src\docker_engine\create_container.py", line 53, in <module>
c_obj.create_containers("selenium/standalone-chrome", "Chrome_{0}".format(i), expose_port={5550+i:4444})
File "C:\Users\Desktop\FreeLance\Utility\src\docker_engine\create_container.py", line 20, in create_containers
container = self.client.containers.run(
File "C:\Program Files\Python39\lib\site-packages\docker\models\containers.py", line 818, in run
container.start()
File "C:\Program Files\Python39\lib\site-packages\docker\models\containers.py", line 404, in start
return self.client.api.start(self.id, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\docker\utils\decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "C:\Program Files\Python39\lib\site-packages\docker\api\container.py", line 1111, in start
self._raise_for_status(res)
File "C:\Program Files\Python39\lib\site-packages\docker\api\client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "C:\Program Files\Python39\lib\site-packages\docker\errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localnpipe/v1.40/containers/260894dbaec6946e5f31fdbfb5307182d2f621c12a38f328f6efac58df58854d/start: Internal Server Error ("driver failed programming external connectivity on endpoint Chrome_2 (c0c77528743a1e3153201565b2cb520243b66adbb903bb69e91a00c4399aca62): Bind for 0.0.0.0:4444 failed: port is already allocated")

While proving port the syntax is container_port:host_port
c_obj.create_containers("selenium/standalone-chrome", "Chrome_{0}".format(i), expose_port={4444:5550+i})
documentation : https://docker-py.readthedocs.io/en/stable/containers.html

Python paho mqtt throw Type Error: "an integer is required"

I have five raspberry pi 3 to collect IOT data. Only one client throw this error,
Here is the traceback message:
Traceback (most recent call last):
File "./uploadRecordDegree.py", line 57, in <module>
auth=auth, client_id=client_id)
File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/publish.py", line 232, in single
protocol, transport, proxy_args)
File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/publish.py", line 167, in multiple
client.connect(hostname, port, keepalive)
File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 941, in connect
return self.reconnect()
File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 1119, in reconnect
return self._send_connect(self._keepalive)
File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 2647, in _send_connect
self._pack_str16(packet, self._client_id)
File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 2496, in _pack_str16
packet.extend(data)
TypeError: an integer is required
Here is my test code:
BOX = {"1234": "balabala"}
host = "192.168.10.11"
port = 1883
topic = "iot/device"
auth = {'username': 'xxxx','password': 'xxxxx'}
payload = json.dumps(BOX)
client_id = "testNode"
publish.single(topic, payload, qos=1, hostname=host, port=port,
auth=auth, client_id=client_id)
My Python version is 3.5.2
paho-mqtt version is 1.5.1
Each client is in the same paho-mqtt version, same code.
I cant figure out what's the problem going.

SFTP connection failed socket.timeout?

I am attempting to connect to one of our SFTP servers in order to automate a report. We pull a file from this server once a week and I am not able to get this connections to work. I am not finding any useful references to the below traceback error. Anyone have an idea of what could be causing this problem?
I get a socket.timeout error after 2 or 3 seconds of executing the code.
import paramiko
cli = paramiko.SSHClient()
cli.set_missing_host_key_policy(paramiko.AutoAddPolicy())
cli.connect(hostname='AHOST', port=21, username="USER", password="PASS")
stdin_, stdout_, stderr_ = cli.exec_command("ls -l ~")
print(stdout_.readlines())
cli.close()
Traceback:
Exception: Error reading SSH protocol banner
Traceback (most recent call last):
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 2138, in _check_banner
buf = self.packetizer.readline(timeout)
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\packet.py", line 367, in readline
buf += self._read_timeout(timeout)
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\packet.py", line 576, in _read_timeout
raise socket.timeout()
socket.timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 1966, in run
self._check_banner()
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 2143, in _check_banner
"Error reading SSH protocol banner" + str(e)
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Traceback (most recent call last):
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 2138, in _check_banner
buf = self.packetizer.readline(timeout)
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\packet.py", line 367, in readline
buf += self._read_timeout(timeout)
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\packet.py", line 576, in _read_timeout
raise socket.timeout()
socket.timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/user_name/PycharmProjects/PDD_Report/MAIN/MAIN.py", line 12, in <module>
cli.connect(hostname='A40T', port=21, username="user_name", password="gate001")
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\client.py", line 397, in connect
t.start_client(timeout=timeout)
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 587, in start_client
raise e
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 1966, in run
self._check_banner()
File "C:\Users\user_name\Desktop\Python 3.6.2\lib\site-packages\paramiko\transport.py", line 2143, in _check_banner
"Error reading SSH protocol banner" + str(e)
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

You are connecting to FTP port 21.
SSH/SFTP uses port 22 (what is the default value of the port parameter, which you override).
Side note 1: Your question is about SFTP. Yet, instead of using SFTPClient, you use SSHClient and execute shell commands – That has nothing to do with SFTP.
Side note 2: Do not use AutoAddPolicy like this. You
lose security by doing so.
See Paramiko "Unknown Server".

Surviving icinga2 restart in a python requests stream

I have been working on a chatbot interface to icinga2, and have not found a persistent way to survive the restart/reload of the icinga2 server. After a week of moving try/except blocks, using requests sessions, et al, it's time to reach out to the community.
Here is the current iteration of the request function:
def i2api_request(url, headers={}, data={}, stream=False, *, auth=api_auth, ca=api_ca):
''' Do not call this function directly; it's a helper for the i2* command functions '''
# Adapted from http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/icinga2-api
# Section 11.10.3.1
try:
r = requests.post(url,
headers=headers,
auth=auth,
data=json.dumps(data),
verify=ca,
stream=stream
)
except (requests.exceptions.ChunkedEncodingError,requests.packages.urllib3.exceptions.ProtocolError, http.client.IncompleteRead,ValueError) as drop:
return("No connection to Icinga API")
if r.status_code == 200:
for line in r.iter_lines():
try:
if stream == True:
yield(json.loads(line.decode('utf-8')))
else:
return(json.loads(line.decode('utf-8')))
except:
debug("Could not produce JSON from "+line)
continue
else:
#r.raise_for_status()
debug('Received a bad response from Icinga API: '+str(r.status_code))
print('Icinga2 API connection lost.')
(The debug function just flags and prints the indicated error to the console.)
This code works fine handling events from the API and sending them to the chatbot, but if the icinga server is reloaded, as would be needed after adding a new server definition in /etc/icinga2..., the listener crashes.
Here is the error response I get when the server is restarted:
Exception in thread Thread-11:
Traceback (most recent call last):
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 447, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 228, in _error_catcher
yield
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 498, in read_chunked
self._update_chunk_length()
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 451, in _update_chunk_length
raise httplib.IncompleteRead(line)
http.client.IncompleteRead: IncompleteRead(0 bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/errbot/err3/lib/python3.4/site-packages/requests/models.py", line 664, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 349, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 526, in read_chunked
self._original_response.close()
File "/usr/lib64/python3.4/contextlib.py", line 77, in __exit__
self.gen.throw(type, value, traceback)
File "/home/errbot/err3/lib/python3.4/site-packages/requests/packages/urllib3/response.py", line 246, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/home/errbot/plugins/icinga2bot.py", line 186, in report_events
for line in queue:
File "/home/errbot/plugins/icinga2bot.py", line 158, in i2events
for line in queue:
File "/home/errbot/plugins/icinga2bot.py", line 98, in i2api_request
for line in r.iter_lines():
File "/home/errbot/err3/lib/python3.4/site-packages/requests/models.py", line 706, in iter_lines
for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
File "/home/errbot/err3/lib/python3.4/site-packages/requests/models.py", line 667, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
With Icinga2.4, this crash happened every time the server was restarted. I thought the problem had gone away after we upgraded to 2.5, but it now appears to have turned into a heisenbug.

I wound up getting advice on IRC to reorder the try/except blocks and make sure they were in the right places. Here's the working result.
def i2api_request(url, headers={}, data={}, stream=False, *, auth=api_auth, ca=api_ca):
''' Do not call this function directly; it's a helper for the i2* command functions '''
# Adapted from http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/icinga2-api
# Section 11.10.3.1
debug(url)
debug(headers)
debug(data)
try:
r = requests.post(url,
headers=headers,
auth=auth,
data=json.dumps(data),
verify=ca,
stream=stream
)
debug("Connecting to Icinga server")
debug(r)
if r.status_code == 200:
try:
for line in r.iter_lines():
debug('in i2api_request: '+str(line))
try:
if stream == True:
yield(json.loads(line.decode('utf-8')))
else:
return(json.loads(line.decode('utf-8')))
except:
debug("Could not produce JSON from "+line)
return("Could not produce JSON from "+line)
except (requests.exceptions.ChunkedEncodingError,ConnectionRefusedError):
return("Connection to Icinga lost.")
else:
debug('Received a bad response from Icinga API: '+str(r.status_code))
print('Icinga2 API connection lost.')
except (requests.exceptions.ConnectionError,
requests.packages.urllib3.exceptions.NewConnectionError) as drop:
debug("No connection to Icinga API. Error received: "+str(drop))
sleep(5)
return("No connection to Icinga API.")

Can't connect to JIRA Python through REST api https url

I try to connect to a jira dev sandbox through https but it comes up with an SSL23_GET_SERVER_HELLO:unknown protocol error
This is the error log/stack trace. I try both ports 8080 and 443 but no joy.
>>> from jira.client import JIRA
>>> options = {'server':'localhost:8080'}
>>> auth = ('username', 'password')
>>> jira = JIRA(options, auth)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ve/lib/python2.6/site-packages/jira/client.py", line 88, in __init__
self._create_http_basic_session(*basic_auth)
File "/home/ve/lib/python2.6/site-packages/jira/client.py", line 1369, in _create_http_basic_session
r = self._session.post(url, data=json.dumps(payload))
File "/home/ve/lib/python2.6/site-packages/requests/sessions.py", line 284, in post
return self.request('post', url, data=data, **kwargs)
File "/home/ve/lib/python2.6/site-packages/requests/sessions.py", line 241, in request
r.send(prefetch=prefetch)
File "/home/ve/lib/python2.6/site-packages/requests/models.py", line 638, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
>>> options = {'server':'localhost:443'}
>>> auth = ('username', 'password')
>>> jira = JIRA(options, auth)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ve/lib/python2.6/site-packages/jira/client.py", line 88, in __init__
self._create_http_basic_session(*basic_auth)
File "/home/ve/lib/python2.6/site-packages/jira/client.py", line 1369, in _create_http_basic_session
r = self._session.post(url, data=json.dumps(payload))
File "/home/ve/lib/python2.6/site-packages/requests/sessions.py", line 284, in post
return self.request('post', url, data=data, **kwargs)
File "/home/ve/lib/python2.6/site-packages/requests/sessions.py", line 241, in request
r.send(prefetch=prefetch)
File "/home/ve/lib/python2.6/site-packages/requests/models.py", line 631, in send
raise ConnectionError(sockerr)
requests.exceptions.ConnectionError: [Errno 110] Connection timed out
>>>

Try this:
from jira.client import JIRA
options = {'server':'localhost:8080'}
jira = JIRA(options)
jira = JIRA(basic_auth=('username', 'password'))
If it doesn't help, any chance the have OpenSSL conflicts?
When curl is linked against OpenSSL 0.9.8 and tries to access a server running OpenSSL 1.0.0, the SSL handshake fails with:
curl: (35) error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)

Got it now. Port 8080 was set up and was not a defauld for http connection whether as for https I used validation manipulation basically setting validate = False in the jira-python method body