Develop Reference

Django request timeout for some static files when running manage.py test package.to.unit.test

Context:
We run Cypress tests which use instances of our application started using manage.py test package.test.suite. The test fixtures and environment are all set up using a class extended from django.contrib.staticfiles.testing.StaticLiveServerTestCase. A unit test/method is added to the extended class which invokes Cypress as a subprocess to run the tests and asserts against the exit code of the subprocess.
Versions:
Python: 3.6.8
Django: 2.2.3
macOS: Mojave 10.14.6
The problem:
This worked well until yesterday when I updated Cypress via npm. Now when I start a server using manage.py test some.test.suite the server will fail to serve all of the static resources requested by the browser. Specifically, it almost always fails to serve some .woff fonts and a random javascript/css file or two. The rest of the files are served but those 5 or 6 which the browser never receives a response for. Eventually I'll get a ConnectionResetError: [Errno 54] Connection reset by peer error (stack trace below) in the terminal. Additionally, if I enable the cache in my browser and attempt a few refreshes things will work fine, almost as if theres a limit to the number of files that can be served at once and once some files are cached in the browser the number of requested files falls below that limit.
When I do python manage.py runserver, however, I don't seem to have this problem at all with or without caching enabled.
Stack Trace:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 52655)
Traceback (most recent call last):
File "/Users/myuser/.pyenv/versions/3.6.8/lib/python3.6/socketserver.py", line 320, in _handle_request_noblock
self.process_request(request, client_address)
File "/Users/myuser/.pyenv/versions/3.6.8/lib/python3.6/socketserver.py", line 351, in process_request
self.finish_request(request, client_address)
File "/Users/myuser/.pyenv/versions/3.6.8/lib/python3.6/socketserver.py", line 364, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/Users/myuser/.pyenv/versions/3.6.8/lib/python3.6/socketserver.py", line 724, in __init__
self.handle()
File "/Users/myuser/Path/To/Project/venv/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 171, in handle
self.handle_one_request()
File "/Users/myuser/Path/To/Project/venv/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 179, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "/Users/myuser/.pyenv/versions/3.6.8/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer
----------------------------------------
Additional Notes/Observations
The problem also occurs when I run the tests headless.
Installing Cypress did run into a hitch that required me to reinstall Electron and Cypress, clear the npm cache and delete the Cypress cache folder.
It's also strange that the problem seems to be with the python environment and not Cypress itself since accessing the web server via my browser and not Cypress produces the same behaviour; so perhaps Cypress is just a red herring and some sort of shared dependency was changed/updated in the process.
If I attempt to directly access one of the files that doesn't get served during a request, which is displayed as 'pending' in Chrome Developer's tools network tab, directly via a second tab it will succeed (but sometimes be lagged by several seconds).
After closing the test server and attempting to run it again I will get an [OSError: [Errno 48] Address already in use] error for up to a minute or two. Previously the server would relinquish the address/port immediately upon closing (or I'm assuming as such since I've never seen this before and had rapidly closed and re-ran tests to test fixture changes in the past).
Things I've Tried:
Rebuilding my virtualenv from scratch
Copying my old venv folder from a Time Machine backup from a time when things worked
Reverting the version of Cypress back to what it was prior to the problem
Looking into timeouts and connection limits of using manage.py test vs manage.py runserver (Didn't find anything).
Toggling DEBUG mode and profiling on/off.
Switching between Chrome and Chromium in Cypress
Building new environments with different python versions (3.6.5, 3.6.7 and 3.6.8)
Switching Django back to 2.2.2 from 2.2.3
Any help would be appreciated!
Update
Looking into this further it looks like the requests for the files that don't get a response never make it to the WSGIHandler form the Socket layer (or even if they make it to the socket layer, I'd assume they do though).
Update 2
I see the same behaviour with manage.py runserver if I include the --nothreading switch. I had a co-worker give it a test and he indeed saw the same behaviour with manage.py runserver --nothreading but manage.py test test.to.run still functioned fine for him.
Also removing the font references from the css/templates just results in a different set of 5 files that aren't served.

Google Cloud IoT core exercise

I am working on Google Cloud IoT core and there I came across a problem in one of the samples (end-to-end example) provided in the online docs: google cloud iot exercise
There are two scripts, for Server and Device, and while running the device script I am facing this problem:

From the commandline args, it looks like you're passing in rsa_cert.pem, which is your SSL private key. As #class said, you need to wget the Google root certificate (wget https://pki.google.com/roots.pem) and then pass the path to downloaded roots.pem for the --ca_certs argument.

error image
Traceback (most recent call last):
File "cloudiot_pubsub_example_mqtt_device.py", line 249, in <module>
main()
File "cloudiot_pubsub_example_mqtt_device.py", line 213, in main
client.connect(args.mqtt_bridge_hostname, args.mqtt_bridge_port)
File "/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py", line
768, in connect
return self.reconnect()
File "/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py", line
927, in reconnect
sock.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 788, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
(_ssl.c:581)

It's possible that your firewall (e.g. the Cloud Shell Machine) is blocking Python from connecting via port 8883. Can you try calling the cloudiot_pubsub_example_mqtt_device.py script with the port set to 443, e.g.
python <your_existing_parameters> --mqtt_bridge_port=443
You may also want to try using the HTTP device sample to publish messages as it also doesn't use port 8883, which may be blocked on your network.
In my tests, I was only able to run the exercise from the Google Cloud Shell after setting my port to 443, this hopefully will resolve the issue for you.
Note If you're encountering issues with verifying the server certificate, you need to download the Google root certificate by calling:
wget https://pki.google.com/roots.pem
Update You may also want to try to set the Python version in your virtual environment to Python 2 by setting up the virtual environment as:
virtualenv env --python=python2

Cannot connect to Sqlserver from python script in windows XP

I am using a windows XP machine that is using Microsoft SQl Server Enterprise Edition version 8.00.194, SQL-DMO version: 8.00.02 and ODBC version : 03.52.0000. I have another database of MySql on some other machine in the same network and I want to retrieve the data from that database to my XP machine's Sqlserver database. For this purpose I am trying to check whether I can connect to the local machine with my following python script (I use the pymssql library for this purpose):
import pymssql
con1=pymssql.connect(server="localhost:1433",user="XXXX",password="XXXX",database="XXXX")
print con1
con1.close()
I have purposefully removed the original username, password and db name. After running the above script I get the error:
Traceback (most recent call last):
File "D:\py_script_data_1st\1.py", line 9, in <module>
con1=pymssql.connect(server="localhost:1433",user="xxxx",password="xxxx",database="xxxx")
File "pymssql.pyx", line 636, in pymssql.connect (pymssql.c:10178)
OperationalError: (20009, 'DB-Lib error message 20009, severity 9:\nUnable to connect: Adaptive Server is unavailable or does not exist\nNet-Lib error during Unknown error (10061)\n')
I have searched extensively but did not get any working solution for that error. What I got was to open the port 1433. I have opened the sqlserver.exe instance in my firewall by following the instructions given https://support.microsoft.com/en-us/kb/841251
When I do telnet on port 1433 by using:
telnet localhost 1433
I get error that says could not connect to port 1433.
I would also like to add that the database on which I am working is of a library system. There are two machines on which issues/returns trasactions of the book takes place. So the other machine (which is also a windows XP machine) connects to the server machine using a software that uses netbios-ssn for the communication. I found this fact by using netstat command. Here is the output of that command:
TCP issue:netbios-ssn 192.168.48.45:2211 ESTABLISHED
TCP issue:netbios-ssn 192.168.48.48:3456 ESTABLISHED
I am extremely confused about these facts. I don't understand how can other machine access server's database using netbios-ssn port?
So, I have effectively two questions and solution to any one of them can help me out.
What is the solution for the problem of the python script ?
If the first problem cannot be solved, how can I achieve the connection to the database using netbios-ssn port?
Please help and feel free to ask any further information if you want.

pika.exceptions.ProbableAuthenticationError when trying to send message to remote queue

I'm trying to run RabbitMQ Python tutorial but with sender on virtualbox host machine and receiver and queue on virtualbox guest machine. So I modified mentioned send.py code by only replacing localhost with 192.168.1.5. When I run it, i receive following error:
...
File "/home/damian/.virtualenvs/kivy_1.9/local/lib/python2.7/site-packages/pika/adapters/base_connection.py", line 153, in _check_state_on_disconnect
raise exceptions.ProbableAuthenticationError
pika.exceptions.ProbableAuthenticationError
rabbitmq-server seems to be running, because when I stop it send.py gives me:
...
File "/home/damian/.virtualenvs/kivy_1.9/local/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 301, in _adapter_connect
raise exceptions.AMQPConnectionError(error)
pika.exceptions.AMQPConnectionError: Connection to 192.168.1.5:5672 failed: [Errno 111] Connection refused
which makes perfect sense.
How to fix that ProbableAuthenticationError?
Host machine is Debian 7 with Python 2.7.3 and pika 0.9.14, guest is Ubuntu 15.04 with rabbitmq-server 3.4.3-2

This is because you are trying to authenticate using the username and password guest remotely. Starting with RabbitMQ 3.3 you need to create a new account to use remotely, and guest/guest can only be used locally.
This is taken from the change log here.
25603 prevent access using the default guest/guest credentials except via localhost since (1.0.0)
It's possible to modify the RabbitMQ configuration to allow remote access using the guest account, by removing guest from loopback_users, but it's recommended to create a new user to follow best practices.
[{rabbit, [{loopback_users, []}]}].

Python mod_wsgi database connection not working

I've installed Postgres, mod_wsgi and python3.5 on a centos system. I've tested the connection and it works fine both in the interactive Python shell as well as the development application (it's a Pyramid app so it's the dev server from that).
I'm getting the following error:
postgresql.exceptions.ClientCannotConnectError: could not establish connection to server
CODE: 08001
LOCATION: CLIENT
CONNECTION: [failed]
failures[0]:
socket('127.0.0.1', 5432)
Traceback (most recent call last):
File "/var/www/wsgi/lib/python3.3/site-packages/postgresql/protocol/client3.py", line 136, in connect
self.socket = self.socket_factory(timeout = timeout)
File "/var/www/wsgi/lib/python3.3/site-packages/postgresql/python/socket.py", line 64, in __call__
s.connect(self.socket_connect)
PermissionError: [Errno 13] Permission denied
The above exception was the direct cause of the following exception:
postgresql.exceptions.ConnectionRejectionError: Permission denied
CODE: 08004
LOCATION: CLIENT
CONNECTOR: [Host] pq://username:***#localhost:5432/db_name
category: None
DRIVER: postgresql.driver.pq3.Driver
I spent some time looking at the mod_wsgi documentation and found there is a possible issue with sockets. However, I've already implemented that change (and the corresponding error from apache vanished).
My assumption is that it's a socket error still as the connection sting works fine elsewhere in the same Python install and even within the dev setup.
Can anybody point me in the right direction from here please?

Even if you set the database connect string to an IP address, if it is 127.0.0.1, I have encountered PostgreSQL database connectors which will try and optimise things and connect via the local UNIX socket instead. If the user that your code runs under with mod_wsgi cannot access that UNIX socket for accessing PostgreSQL it will fail with a permissions error.
I can't remember whether it was py-postgresql module that I saw this with, but did hit it a couple of weeks ago with on of the PostgreSQL database modules.
Try replacing 127.0.0.1 with the actual host name or proper IP address.