I have written a simple python script to check if squid works normally
/scripts/proxychecker
#!/usr/bin/python
import urllib2
import sys, os, time
import socket
socket.setdefaulttimeout(5)
proxy_support = urllib2.ProxyHandler({'http': 'http://127.0.0.1:3128/'})
opener = urllib2.build_opener(proxy_support)
urllib2.install_opener(opener)
try:
response = urllib2.urlopen('http://python.org/')
html = response.read()
except:
#os.system("/etc/init.d/squid3 stop");
#os.system("/etc/init.d/squid3 start");
os.system("/etc/init.d/squid3 restart");
#os.system("service squid3 restart");
I shutdown squid, and manually executed this script "/scripts/proxychecker ",
it did bring up squid
but if I add this script to cron job:
*/1 * * * * /scripts/proxychecker >/root/debug.txt
it did not work
and from /root/debug.txt
it said
"Since the script you are attempting to invoke has been converted to an
Upstart job, you may also use the start(8) utility, e.g. start squid3"
I do not think this is the reason why squid not be brought up
because I have changed command as bellows , and still did not work
#os.system("/etc/init.d/squid3 stop");
#os.system("/etc/init.d/squid3 start");
os.system("/etc/init.d/squid3 restart");
#os.system("service squid3 restart");
it is a strange problem
also
cron is created by root
and this script also executed by root, permission should not be the reason
Use restart squid instead of service squid restart.
Related
I am following the Google Cloud Functions python testing example here:
https://cloud.google.com/functions/docs/testing/test-http
import os
import subprocess
import uuid
import requests
from requests.packages.urllib3.util.retry import Retry
def test_args():
name = str(uuid.uuid4())
port = os.getenv('PORT', 8080) # Each functions framework instance needs a unique port
process = subprocess.Popen(
[
'functions-framework',
'--target', 'hello_http',
'--port', str(port)
],
cwd=os.path.dirname(__file__),
stdout=subprocess.PIPE
)
# Send HTTP request simulating Pub/Sub message
# (GCF translates Pub/Sub messages to HTTP requests internally)
BASE_URL = f'http://localhost:{port}'
retry_policy = Retry(total=6, backoff_factor=1)
retry_adapter = requests.adapters.HTTPAdapter(
max_retries=retry_policy)
session = requests.Session()
session.mount(BASE_URL, retry_adapter)
name = str(uuid.uuid4())
res = session.post(
BASE_URL,
json={'name': name}
)
assert res.text == 'Hello {}!'.format(name)
# Stop the functions framework process
process.kill()
process.wait()
After running pytest against this test file, the test passes and the code exits nearly immediately, but it doesn't appear to have killed the process:
± ps -ef | grep functions
thoraxe 11661 1985 49 20:30 pts/0 00:00:02 /home/thoraxe/.pyenv/versions/3.9.13/envs/avogadro-trainer-3-9-13/bin/python3.9 /home/thoraxe/.pyenv/versions/3.9.13/envs/avogadro-trainer-3-9-13/bin/functions-framework --target train --port 8080
thoraxe 11684 11661 0 20:30 pts/0 00:00:00 /home/thoraxe/.pyenv/versions/3.9.13/envs/avogadro-trainer-3-9-13/bin/python3.9 /home/thoraxe/.pyenv/versions/3.9.13/envs/avogadro-trainer-3-9-13/bin/functions-framework --target train --port 8080
I'm using Python 3.9.13 in a virtual environment on Fedora.
As this is sample code from Google, I'd expect it to work, but something is definitely not working here. Can someone suggest what I might be doing wrong?
When a Python assertion fails, the program exits immediately and does not continue. The kill/wait are never actually executed unless the test is successful. This is a major bummer because the function framework will continue to run in the background and new code changes aren't apparently picked up on subsequent pytest runs.
Using a different wrapper framework like https://github.com/okken/pytest-check solves the problem because all steps will be performed, even if there are failures.
However, note that legitimate Python failures/errors/explosions will still result in the functions framework not properly exiting.
I have tested several proposal from anwers from other post but I can't get my server con run on the background from powershell using >pythonw server.py as I do with my other scrips.
my code is:
#!/usr/bin/env python3
from http.server import HTTPServer, SimpleHTTPRequestHandler, test
import socketserver
from urllib.parse import urlparse
from urllib.parse import parse_qs
import requests
import asyncio
import sys, os, signal, threading
class MyHttpRequestHandler(SimpleHTTPRequestHandler):
def end_headers (self):
self.send_header('Access-Control-Allow-Origin', '*')
SimpleHTTPRequestHandler.end_headers(self)
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
html = f"31"
self.wfile.write(bytes(html, "utf8"))
def create_server():
port = 8000
handler_object = MyHttpRequestHandler
my_server = socketserver.TCPServer(("", port), handler_object)
print("serving at port:" + str(port))
my_server.serve_forever()
threading.Thread(target=create_server).start()
the server runs just ok on foreground (calling it from PW >python server.py) but in background it does not reply.
I’m not sure it could easily be done using PowerShell as that is meant to be an interactive shell rather than a background system.
A couple of ideas. One is harder than the other.
Compile the python code into an EXE using something like PyInstaller and then run the EXE as a Windows Service (which is difficult to do at best)
Set up a scheduled task that runs the python script using pythonw when you log in to the system. This option would probably be the easier of the two and could be done through PowerShell using New-ScheduledTask
I suppose a third way would be through IIS but I’m not familiar with that unfortunately.
Edit: one other idea! Install Docker for Windows and run the server on a container rather than on your Windows host itself.
I have a python script that uses the http.server module that I would like to run without a terminal window being shown. Unfortunately, due to how I'm doing this, running the script in pythonw.exe does not work.
Here is the script:
import os
from http.server import CGIHTTPRequestHandler, HTTPServer
handler = CGIHTTPRequestHandler
handler.cgi_directories = ['/scripts']
server = HTTPServer(('localhost', 1271), handler)
server.serve_forever()
Unfortunately, I don't know any way get any error logs because, y'know, pythonw doesn't show the console. If anyone can tell me how to get the error logs, I'll be happy to add them to the bottom of this post.
I'm running 64-bit Windows 10 and python 3.6.6, if that makes a difference.
I'm sorry if this is a stupid question, but I—for the life of me—cannot find a solution anywhere.
You can just store output to a text file like this:
from http.server import CGIHTTPRequestHandler, HTTPServer
import sys
handler = CGIHTTPRequestHandler
handler.cgi_directories = ['/scripts']
server = HTTPServer(('localhost', 1271), handler)
sys.stderr = open('log.txt', 'w', 1)
server.serve_forever()
Flow of the program is:
Connect to OpenSSH server on Linux machine using Paramiko library
Open X11 session
Run xterm executable
Run some other program (e.g. Firefox) by typing executable name in the terminal and running it.
I would be grateful if someone can explain how to cause some executable to run in a terminal which was open by using the following code and provide sample source code (source):
import select
import sys
import paramiko
import Xlib.support.connect as xlib_connect
import os
import socket
import subprocess
# run xming
XmingProc = subprocess.Popen("C:/Program Files (x86)/Xming/Xming.exe :0 -clipboard -multiwindow")
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(SSHServerIP, SSHServerPort, username=user, password=pwd)
transport = ssh_client.get_transport()
channelOppositeEdges = {}
local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])
inputSockets = []
def x11_handler(channel, (src_addr, src_port)):
local_x11_socket = xlib_connect.get_socket(*local_x11_display[:3])
inputSockets.append(local_x11_socket)
inputSockets.append(channel)
channelOppositeEdges[local_x11_socket.fileno()] = channel
channelOppositeEdges[channel.fileno()] = local_x11_socket
transport._queue_incoming_channel(channel)
session = transport.open_session()
inputSockets.append(session)
session.request_x11(handler = x11_handler)
session.exec_command('xterm')
transport.accept()
while not session.exit_status_ready():
readable, writable, exceptional = select.select(inputSockets,[],[])
if len(transport.server_accepts) > 0:
transport.accept()
for sock in readable:
if sock is session:
while session.recv_ready():
sys.stdout.write(session.recv(4096))
while session.recv_stderr_ready():
sys.stderr.write(session.recv_stderr(4096))
else:
try:
data = sock.recv(4096)
counterPartSocket = channelOppositeEdges[sock.fileno()]
counterPartSocket.sendall(data)
except socket.error:
inputSockets.remove(sock)
inputSockets.remove(counterPartSocket)
del channelOppositeEdges[sock.fileno()]
del channelOppositeEdges[counterPartSocket.fileno()]
sock.close()
counterPartSocket.close()
print 'Exit status:', session.recv_exit_status()
while session.recv_ready():
sys.stdout.write(session.recv(4096))
while session.recv_stderr_ready():
sys.stdout.write(session.recv_stderr(4096))
session.close()
XmingProc.terminate()
XmingProc.wait()
I was thinking about running the program in child thread, while the thread running the xterm is waiting for the child to terminate.
Well, this is a bit of a hack, but hey.
What you can do on the remote end is the following: Inside the xterm, you run netcat, listen to any data coming in on some port, and pipe whatever you get into bash. It's not quite the same as typing it into xterm direclty, but it's almost as good as typing it into bash directly, so I hope it'll get you a bit closer to your goal. If you really want to interact with xterm directly, you might want to read this.
For example:
terminal 1:
% nc -l 3333 | bash
terminal 2 (type echo hi here):
% nc localhost 3333
echo hi
Now you should see hi pop out of the first terminal. Now try it with xterm&. It worked for me.
Here's how you can automate this in Python. You may want to add some code that enables the server to tell the client when it's ready, rather than using the silly time.sleeps.
import select
import sys
import paramiko
import Xlib.support.connect as xlib_connect
import os
import socket
import subprocess
# for connecting to netcat running remotely
from multiprocessing import Process
import time
# data
import getpass
SSHServerPort=22
SSHServerIP = "localhost"
# get username/password interactively, or use some other method..
user = getpass.getuser()
pwd = getpass.getpass("enter pw for '" + user + "': ")
NETCAT_PORT = 3333
FIREFOX_CMD="/path/to/firefox &"
#FIREFOX_CMD="xclock&"#or this :)
def run_stuff_in_xterm():
time.sleep(5)
s = socket.socket(socket.AF_INET6 if ":" in SSHServerIP else socket.AF_INET, socket.SOCK_STREAM)
s.connect((SSHServerIP, NETCAT_PORT))
s.send("echo \"Hello there! Are you watching?\"\n")
s.send(FIREFOX_CMD + "\n")
time.sleep(30)
s.send("echo bye bye\n")
time.sleep(2)
s.close()
# run xming
XmingProc = subprocess.Popen("C:/Program Files (x86)/Xming/Xming.exe :0 -clipboard -multiwindow")
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(SSHServerIP, SSHServerPort, username=user, password=pwd)
transport = ssh_client.get_transport()
channelOppositeEdges = {}
local_x11_display = xlib_connect.get_display(os.environ['DISPLAY'])
inputSockets = []
def x11_handler(channel, (src_addr, src_port)):
local_x11_socket = xlib_connect.get_socket(*local_x11_display[:3])
inputSockets.append(local_x11_socket)
inputSockets.append(channel)
channelOppositeEdges[local_x11_socket.fileno()] = channel
channelOppositeEdges[channel.fileno()] = local_x11_socket
transport._queue_incoming_channel(channel)
session = transport.open_session()
inputSockets.append(session)
session.request_x11(handler = x11_handler)
session.exec_command("xterm -e \"nc -l 0.0.0.0 %d | /bin/bash\"" % NETCAT_PORT)
p = Process(target=run_stuff_in_xterm)
transport.accept()
p.start()
while not session.exit_status_ready():
readable, writable, exceptional = select.select(inputSockets,[],[])
if len(transport.server_accepts) > 0:
transport.accept()
for sock in readable:
if sock is session:
while session.recv_ready():
sys.stdout.write(session.recv(4096))
while session.recv_stderr_ready():
sys.stderr.write(session.recv_stderr(4096))
else:
try:
data = sock.recv(4096)
counterPartSocket = channelOppositeEdges[sock.fileno()]
counterPartSocket.sendall(data)
except socket.error:
inputSockets.remove(sock)
inputSockets.remove(counterPartSocket)
del channelOppositeEdges[sock.fileno()]
del channelOppositeEdges[counterPartSocket.fileno()]
sock.close()
counterPartSocket.close()
p.join()
print 'Exit status:', session.recv_exit_status()
while session.recv_ready():
sys.stdout.write(session.recv(4096))
while session.recv_stderr_ready():
sys.stdout.write(session.recv_stderr(4096))
session.close()
XmingProc.terminate()
XmingProc.wait()
I tested this on a Mac, so I commented out the XmingProc bits and used /Applications/Firefox.app/Contents/MacOS/firefox as FIREFOX_CMD (and xclock).
The above isn't exactly a secure setup, as anyone connecting to the port at the right time could run arbitrary code on your remote server, but it sounds like you're planning to use this for testing purposes anyway. If you want to improve the security, you could make netcat bind to 127.0.0.1 rather than 0.0.0.0, setup an ssh tunnel (run ssh -L3333:localhost:3333 username#remote-host.com to tunnel all traffic received locally on port 3333 to remote-host.com:3333), and let Python connect to ("localhost", 3333).
Now you can combine this with selenium for browser automation:
Follow the instructions from this page, i.e. download the selenium standalone server jar file, put it into /path/to/some/place (on the server), and pip install -U selenium (again, on the server).
Next, put the following code into selenium-example.py in /path/to/some/place:
#!/usr/bin/env python
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox() # Get local session of firefox
browser.get("http://www.yahoo.com") # Load page
assert "Yahoo" in browser.title
elem = browser.find_element_by_name("p") # Find the query box
elem.send_keys("seleniumhq" + Keys.RETURN)
time.sleep(0.2) # Let the page load, will be added to the API
try:
browser.find_element_by_xpath("//a[contains(#href,'http://docs.seleniumhq.org')]")
except NoSuchElementException:
assert 0, "can't find seleniumhq"
browser.close()
and change the firefox command:
FIREFOX_CMD="cd /path/to/some/place && python selenium-example.py"
And watch firefox do a Yahoo search. You might also want to increase the time.sleep.
If you want to run more programs, you can do things like this before or after running firefox:
# start up xclock, wait for some time to pass, kill it.
s.send("xclock&\n")
time.sleep(1)
s.send("XCLOCK_PID=$!\n") # stash away the process id (into a bash variable)
time.sleep(30)
s.send("echo \"killing $XCLOCK_PID\"\n")
s.send("kill $XCLOCK_PID\n\n")
time.sleep(5)
If you want to do perform general X11 application control, I think you might need to write similar "driver applications", albeit using different libraries. You might want search for "x11 send {mouse|keyboard} events" to find more general approaches. That brings up these questions, but I'm sure there's lots more.
If the remote end isn't responding instantaneously, you might want to sniff your network traffic in Wireshark, and check whether or not TCP is batching up the data, rather than sending it line by line (the \n seems to help here, but I guess there's no guarantee). If this is the case, you might be out of luck, but nothing is impossible. I hope you don't need to go that far though ;-)
One more note: if you need to communicate with CLI programs' STDIN/STDOUT, you might want to look at expect scripting (e.g. using pexpect, or for simple cases you might be able to use subprocess.Popen.communicate](http://docs.python.org/2/library/subprocess.html#subprocess.Popen.communicate)).
Summary: Python cgi script runs as expected when called from a simple python debug server, but fails with 500 Premature end of script headers error when run from ~/public_html/cgi-bin/
Problems
My CGI script works fine when run through a simple python webserver, and I see the right output when navigating to nameofmyhost.com:8080/...
However, when running the same script from my public_html/cgi-bin directory it gives me a 500 premature end of script headers error. What can I do to fix this?
The permissions on the file seem ok:
drwxrwxrwx cgi-bin
-rwxr-xr-x cgi-bin/generate_list.py
This is simple_httpd.py, the simple python webserver
from http.server import HTTPServer, CGIHTTPRequestHandler
port = 8080
httpd = HTTPServer(('', port), CGIHTTPRequestHandler)
print("Starting simple_httpd on port: " + str(httpd.server_port))
httpd.serve_forever()
CGI script generate_list.py:
#! /usr/local/bin/python3
import athletemodel
import yate
import glob
data_files = glob.glob("data/*.txt")
athletes = athletemodel.put_to_store(data_files)
print(yate.start_response())
print(yate.include_header("Coach Kelly's List of Athletes"))
print(yate.start_form("generate_timing_data.py"))
print(yate.para("Select an athlete from the list to work with:"))
for each_athlete in athletes:
print(yate.radio_button("which_athlete", athletes[each_athlete].name))
print (yate.end_form("Select"))
print(yate.include_footer({"Home": "/index.html"}))
I'm guessing I need to maybe explicitly state my directory somewhere, maybe?
PS: I am going through the Head First Python book by Oreilly
Debugging steps
Check that the server can locate my files
Server finds simple html file in public_home [OK]
Check that the CGI script can execute without error.
Content-type: text/html
<html>
<head>
<title>Coach Kelly's List of Athletes</title>
[...]
CGI script ran from command line - outputs as expected [OK]
Check that the server can execute a simple CGI script in the same location
Try a simple CGI script to see if the server is able to execute any CGI scripts at all:
#!/usr/local/bin/python3
print("Content-Type; text/html")
print("")
print("<html><body><h1>hello</h1></body></html>")
Server fails to execute simple CGI script, giving the same error [FAIL]
Fixes
Fix 1: Change the data path to be absolute instead of relative:
- data_files = glob.glob("data/*.txt")
+ data_files = glob.glob("/home/delliott/public_html/webapp/data/*.txt")
The problem is that the base of your school webserver is not the same as the base of the simple_httpd.py server. This means that you will have to provide absolute paths to your data directory instead of relative paths.
Change the following lines:
- data_files = glob.glob("data/*.txt")
+ data_files = glob.glob("/home/<username>/public_html/data/*.txt")
This should now behave as expected (if I understand your problem correctly.)
[Edit]: A quick way to check if your cgi-bin scripts work is to run them from the command line. Log into your school server and try the cgi script ion its own:
$ cd /home/<username>/public_html/
$ python3 my_cgi_script.py
This should print out the html that you expect, or a stack trace.