Develop Reference

How to properly connect to SQL Server from a python script when python packages are based on github?

Suppose that due to an HTTP 403 Error it's not possible to download the packages from the PyPi repo (nor pip install <package> commands) which causes me to install the pyodbc by cloning the repo from Github (https://github.com/mkleehammer/pyodbc) and by running the next .cmd windows file:
cd "root_folder"
git activate
git clone https://github.com/mkleehammer/pyodbc.git --depth 1
Note that this package is downloaded to the same root folder where my python script is, after this I try to set a connection to Microsoft SQL Server:
import pyodbc as pyodbc
# set connection settings
server="servername"
database="DB1"
user="user1"
password="123"
# establishing connection to db
conn = pyodbc.connect("DRIVER={SQL Server};SERVER="+server+";DATABASE="+database+";UID="+user+";PWD="+password)
cursor=conn.cursor()
print("Succesful connection to sql server")
However, when I run the above code the next traceback error arises:
Traceback (most recent call last):
File "/dcleaner.py", line 47, in
conn = pyodbc.connect("DRIVER={SQL Server};SERVER="+server+";DATABASE="+database+";UID="+user+";PWD="+password)
AttributeError: module 'pyodbc' has no attribute 'connect'
Do you know how can I properly connect from a py script to a sql-server based database?

After you have cloned PYODBC
cd "root_folder"
git activate
git clone https://github.com/mkleehammer/pyodbc.git --depth 1
On your Local Machine, Go into the cloned directory and open terminal and run below command
python setup.py build
if it errors then try to install appropriate C++ compiler (the error might reveal this detail/ on VSCode it gave the URL to open and download, which I have shared below) install it from here link - choose this one
reboot machine and run this again
python setup.py build #if success then continue with below one
python setup.py install
after that you should be able to import and run the below from your local machine
import pyodbc as pyodbc

Run Python script from browser using virtual environment on EC2 instance

TLDR;
I created a virtual environment on my EC2 instance. How can I access this from the browser?
Hey everyone,
I created a virtual environment, following this tutorial, on my EC2 instance to run a simple Python script. Within the terminal, it works without errors. However, I have made a web application and I would like to activate this script from the browser using the virtual environment. When I try this, I get a "Permission denied" error.
PHP
$output=shell_exec('bash /var/app/current/scripts/script.sh');
echo "<pre>$output</pre>";
script.sh
#!/bin/bash
source /home/ec2-user/venv/python3/bin/activate
python3 /var/app/current/scripts/test.py
test.py
from datetime import datetime
from bs4 import BeautifulSoup
import requests
print('hello')
print(datetime.now())
url = "https://www.stackoverflow.com/"
website = requests.get(url).text
soup = BeautifulSoup(website, "html.parser")
print(soup.title)
error
/var/app/current/scripts/script.sh: line 2: /home/ec2-user/venv/python3/bin/activate: Permission denied
Traceback (most recent call last):
File "/var/app/current/scripts/test.py", line 2, in <module>
from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'
What I have tried:
I tried to change the permissions on the virtual environment using the following:
chmod a+x /home/ec2-user/venv
This should give all users access to the virtual environment folder: /home/ec2-user/venv
However, I am still getting the error:
/home/ec2-user/venv/python3/bin/activate: Permission denied
I have also tried to give all users the possibility of executing the activation script (/home/ec2-user/venv/python3/bin/activate):
chmod 665 /home/ec2-user/venv/python3/bin/activate
Which results in:
-rw-rw-r-x 1 ec2-user ec2-user /home/ec2-user/venv/python3/bin/activate
However, I still get the same error:
/home/ec2-user/venv/python3/bin/activate: Permission denied
Note:
Note that if I only import datetime and I comment out bs4 and requests (along with everything else regarding BeautifulSoup), then the script works great as it does not have to access the virtual environment to pull in the packages.
*Virtual environment tutorial

You get this error because you have not added libraries that are used in the python script to the virtual env.
In the tutorial you mentioned only boto library is installed.
You need to install libraries you use.
Run this from the command line:
source /home/ec2-user/venv/python3/bin/activate
pip install beautifulsoup4
pip install requests
Alternatively you can create a file and name it for example /home/ec2-user/requirements.txt for example and list all requirements your script use:
beautifulsoup4
requests
Then you can use this file to install all requirements into virtual env:
source /home/ec2-user/venv/python3/bin/activate
pip install -r /home/ec2-user/requirements.txt

Solved!
I got some help from this post, however, needed to modify a few things.
Let's dive into what his answer was:
sudo chown -R your_username:your_username path/to/virtuaelenv/
Okay, this is great, but I needed a bit of information.
For me, the web application's username is webapp.
Then, one thing that isn't very clear above is the path. So, my path is:
/home/ec2-user/venv/python3/bin/activate
as mentioned above. Here, you need to change permissions to the /home/ec2-user and NOT to /home/ec2-user/venv
So, to give my application permission to my virtual environment, I needed ran:
sudo chown -R webapp:webapp /home/ec2-user
That worked in the browser! However, this took away my ability to work with it on the server. To do so, I would have to switch it back to:
sudo chown -R ec2-user:ec2-user /home/ec2-user
Being far from ideal to switch back and forth, I tried to change the permissions with chmod instead.
sudo chmod 711 /home/ec2-user
Now I have read, write, and execution permissions whereas everyone else, including the web app, can only execute.
Now it all works 🤓

install z3 in a remote linx server control without being root

I am trying to install z3 on a remote server that I am not the root of. I followed the steps to the point where I have this message:
Z3 was successfully built.
Z3Py scripts can already be executed in the 'build/python'
Use the following command to install Z3 at prefix /usr.
sudo make install
Since it says that Z3py scripts can already be executed, is the next command necessary? if so, how can I execute it without being root. Is there an alternative?
I have changed the prefix to a directory that I have write access for. Again, it installed z3 and z3py successfully but then it says:
Use the following command to install Z3 at prefix /z3/z3-master.
sudo make install
when I use make install this is what I get:
mkdir: cannot create directory ‘/z3’: Permission denied
Makefile:4462: recipe for target 'install' failed
make: *** [install] Error 1

Configure it like this: python scripts/mk_make.py --prefix=/a/place/with/write/access

python socket protocol not support

I useed python-can-isotp on RaspberryPi3, and test it with example code, but I got an error.
My simple code:
import isotp
s = isotp.socket()
s2 = isotp.socket()
# Configuring the sockets.
s.set_fc_opts(stmin=5, bs=10)
#s.set_general_opts(...)
#s.set_ll_opts(...)
s.bind("vcan0" rxid=0x123 txid=0x456) # We love named parameters!
s2.bind("vcan0", rxid=0x456, txid=0x123)
s2.send(b"Hello, this is a long payload sent in small chunks of 8 bytes.")
print(s.recv())
ERROR:
File "/usr/local/opt/python-3.7.0/lib/python3.7/socket.py", line 151, in __init__
_socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 93] Protocol not supported
Can someone please help me find the solution to the problem?

I followed these instructions (with slight modifications) to make isotp work on a Raspberry Pi 3B+ with the latest Raspbian Stretch.
Here is the exact sequence of commands I carried out:
sudo apt update
sudo apt upgrade
git clone https://github.com/hartkopp/can-isotp.git
cd can-isotp
sudo apt install build-essential raspberrypi-kernel-headers
make install
sudo make modules_install
modprobe can
sudo insmod ./net/can/can-isotp.ko
However, I don't use it in Python. I simply tested the tools provided for use with isotp, such as e.g. isotpsend and isotprecv.

stratum-mining-proxy error - Can't decode message

I'm attempting to run stratum-mining-proxy with minerd. Proxy starts and runs with the following command:
python ./mining_proxy.py -o ltc-stratum.kattare.com -p 3333 -pa scrypt
Proxy starts fine. Run Minerd (U/P removed):
minerd -a scrypt -r 1 -s 6 -o http://127.0.0.1:3333 -O USERNAME.1:PASSWORD
Following errors are received. This one from the proxy:
2013-07-18 01:33:59,981 ERROR protocol protocol.dataReceived # Processing of message failed
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/stratum-0.2.12-py2.7.egg/stratum/protocol.py", line 185, in dataReceived
self.lineReceived(line, request_counter)
File "/usr/local/lib/python2.7/dist-packages/stratum-0.2.12-py2.7.egg/stratum/protocol.py", line 216, in lineReceived
raise custom_exceptions.ProtocolException("Cannot decode message '%s'" % line)
'rotocolException: Cannot decode message 'POST / HTTP/1.1
And this from minerd. What am I doing wrong? Any help is appreciated!
[2013-07-18 01:33:59] HTTP request failed: Empty reply from server
[2013-07-18 01:33:59] json_rpc_call failed, retry after 30 seconds

I am a little curious, I don't know as a fact but I was under the impression that the mining proxy was for BTC not LTC.
But anyways I believe I got a similar message when I first installed it as well. To fix, or rather to actually get it running I had to use the Git installation method instead of installing manually.
Installation on Linux using Git
This is advanced option for experienced users, but give you the easiest way for updating the proxy.
1.git clone git://github.com/slush0/stratum-mining-proxy.git
2.cd stratum-mining-proxy
3.sudo apt-get install python-dev # Development package of Python are necessary
4.sudo python distribute_setup.py # This will upgrade setuptools package
5.sudo python setup.py develop # This will install required dependencies (namely Twisted and Stratum libraries), but don't install the package into the system.
6.You can start the proxy by typing "./mining_proxy.py" in the terminal window. Using default settings, proxy connects to Slush's pool interface.
7.If you want to connect to another pool or change other proxy settings, type "./mining_proxy.py --help".
8.If you want to update the proxy, type "git pull" in the package directory.