I am playing around with web scraping and Tor.
I managed to make it work with both requests and Selenium + PhantomJS. However, I need that the Tor browser is opened for the script to work.
This is why I am trying now to automatise the complete process; that is: open Tor browser automatically, run some script and at the end close the browser automatically. But I am struggling with it.
#open Tor browser
os.system('open /Applications/TorBrowser.app')
#code to scrape
#close Tor browser
???
Open
To open the browser, some other options I found out there are not working.
import subprocess
subprocess.Popen('/Applications/TorBrowser.app') #permission denied
or
os.system('start /Applications/TorBrowser.app') #sh: start: command not found
However, the following line worked:
os.system('open /Applications/TorBrowser.app')
Close
The main problem is to close the browser afterwards, as none of the commands found in other posts worked.
Those include:
os.system("taskkill /im /Applications/TorBrowser.app /f") #sh: taskkill: command not found
or
os.system("kill /Applications/TorBrowser.app") #sh: line 0: kill: /Applications/TorBrowser.app: arguments must be process or job IDs
or
os.close('/Applications/TorBrowser.app') #TypeError: an integer is required (got type str)
Any suggestions of how to close it?
And is there a better way to open it?
Edit: I'm on Mac with Python 3.
This worked for me:
from selenium import webdriver
import os
import subprocess
#start Tor
sproc=subprocess.Popen('"C:\\Users\\My name\\Desktop\\Tor Browser\\Browser\\firefox.exe"' )
#start PhantomJS
service_args = [ '--proxy=localhost:9150', '--proxy-type=socks5', ]
driver = webdriver.PhantomJS(service_args=service_args)
#get page
driver.get("https://stackoverflow.com/questions/40161921/how-to-open-and-close-tor-browser-automatically-with-python")
print(driver.page_source)
driver.close()
#kill process
sproc.kill()
I think you should add some time pauses between commands:
import time
time.sleep(20)# wait 20 seconds
Another way to open Tor:
os.system('"C:\\Users\\My Name\\Desktop\\Tor Browser\\Browser\\firefox.exe"' )
But this time your command will wait until the called process stops himself (may be user will close it). According to your question it is not what you want. To control executing process let it runs and use special variable to kill it whenever you want.
Also pay attention to string path: double quotes inside single quotes. There are other ways to pass strings with spaces to system commands, for example: running an outside program (executable) in python?.
Try this in jupyter:
import webbrowser
urL='https://YOUR WEBSITE ADDRESS HERE'
mozilla_path="C:\\Users\\T14s\\Desktop\\Tor Browser\\Browser\\firefox.exe"
webbrowser.register('firefox', None,webbrowser.BackgroundBrowser(mozilla_path))
webbrowser.get('firefox').open_new_tab(urL)
import os
import time
time.sleep(10)
os.system("taskkill /im firefox.exe /f")
TOR is based on firefox - hence firefox comes up a lot.
Related
I need to login to IBM i System using Python without entering the username and password manually.
I used py3270 library but it is not able to detect the Emulator wc3270. The emulator I use has .hod extension and opens with IBM i Launcher.
Can anyone help me with this? what could be the possible solution for this?
os.system() is a blocking statement. That is, it blocks, or stops further Python code from being executed until whatever os.system() is doing has completed. This problem needs us to spawn a separate thread, so that the Windows process executing the ACS software runs at the same time the rest of the Python code runs. subprocess is one Python library that can handle this.
Here is some code that opens an ACS 5250 terminal window and pushes the user and password onto that window. There's no error checking, and there are some setup details that my system assumes about ACS which your system may not.
# the various print() statements are for looking behind the scenes
import sys
import time
import subprocess
from pywinauto.application import Application
import pywinauto.keyboard as keyboard
userid = sys.argv[1]
password = sys.argv[2]
print("Starting ACS")
cmd = r"C:\Users\Public\IBM\ClientSolutions\Start_Programs\Windows_x86-64\acslaunch_win-64.exe"
system = r'/system="your system name or IP goes here"'
# Popen requires the command to be separate from each of the parameters, so an array
result = subprocess.Popen([cmd, r"/plugin=5250",system], shell=True)
print(result)
# wait at least long enough for Windows to get past the splash screen
print("ACS starting - pausing")
time.sleep(5)
print("connecting to Windows process")
ACS = Application().connect(path=cmd)
print(ACS)
# debugging
windows = ACS.windows()
print(windows)
dialog = ACS['Signon to IBM i']
print(dialog)
print("sending keystrokes")
keyboard.send_keys(userid)
keyboard.send_keys("{TAB}")
keyboard.send_keys(password)
keyboard.send_keys("{ENTER}")
print('Done.')
Currently, I am facing the same issue. I was able to run the IBMi (ACS), however, once it run, my python script stop functioning as if the app is preventing the python from being running. In generally speaking, the app seems to not detecting the script.But once I closed the app, my python script continue to work.. I put some indication e.g timesleep, however as i mentioned earlier, it only continue to that line of code once IBM is closed. There will be few lines to be added to move the selection to 5250 and inject the credential.
*I tried with pyautogui, still facing the same issue. so now i tried pywinauto import keyboard .
#Variables
dir = sys.argv[1]
username = sys.argv[2]
password = sys.argv[3]
x = dir.split("\\")
print(x[-1])
command = "cd \ && cd Users/Public/Desktop && " + '"' + x[-1] + '"'
print(command)
os.system(command)
------ FROM THIS LINE OF CODE ONWARDS, IT STOPPED RUNNING ONCE IBM IS LAUNCHED ---
print('TIME START')
time.sleep(5)
print('TIME END')
keyboard.send_keys(username)
keyboard.send_keys(password)
keyboard.send_keys("{ENTER}")
print('Done.')
Appreciate your help to look into this matter. Thanks
I’d like to open a few apps using a very simple python script:
from subprocess import call
call("/Applications/Google Chrome.app/Contents/MacOS/Google Chrome")
call("/Applications/MongoDB Compass.app/Contents/MacOS/MongoDB Compass")
The problem is that opening them this way seems to open a terminal window along with the app itself - for chrome, it outputs this in the terminal for example:
Last login: Sun Oct 23 00:20:38 on ttys000
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome ; exit;
nick#Nicks-MBP ~ % /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome ; exit;
objc[3817]: Class WebSwapCGLLayer is implemented in both /System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebCore.framework/Versions/A/Frameworks/libANGLE-shared.dylib (0x7ffb45565ec8) and /Applications/Google Chrome.app/Contents/Frameworks/Google Chrome Framework.framework/Versions/106.0.5249.119/Libraries/libGLESv2.dylib (0x116ba9668). One of the two will be used. Which one is undefined.
So it hijacks the terminal and does not proceed to this next line:
call("/Applications/MongoDB Compass.app/Contents/MacOS/MongoDB Compass")
If I try to call these:
call(("/Applications/Google Chrome.app"))
call(("/Applications/MongoDB Compass.app"))
I get this error, with other posts stating that it may be a dir and not an app:
OSError: [Errno 13] Permission denied
How can this be fixed? Note that I do not want to do this despite it working:
os.system("open /Applications/" + app + ".app")
Because I need to be able to wait for the apps to finish opening before running another command, hence the use of Subprocess.call. Thank you.
UPDATE:
I now have this:
print("start")
call(
[("/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome")],
shell=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.STDOUT,
)
print("end")
But the print("end") line only executes when I exit out of chrome. How can I get it to wait for Chrome to load and then print 'end' after? Also it requires Shell=True for some reason, otherwise it complains with:
FileNotFoundError: [Errno 2] No such file or directory: '/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome
Updated Answer
This also appears to work and doesn't involve the shell:
import subprocess as sp
print("start")
sp.run(["open", "-a", "Google Chrome"])
print("end")
Original Answer
This appears to do what you want, though I have no explanation as to why:
import subprocess as sp
print("start")
sp.Popen(
["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"],
stdin =sp.DEVNULL,
stdout=sp.DEVNULL,
stderr=sp.DEVNULL)
print("end")
I'm trying to write a python script that can launch DaVinci Resolve in headless mode, then send it some commands via its API, then close it.
What I'm looking for would look something like
Open resolve.exe with argument --nogui
Do stuff with the API here
Terminate this instance of Resolve
I've managed to launch an instance of Resolve in headless. But it always ends up being a subprocess of something else. While it's running as a subprocess, I can't get the API to communicate with it.
Here's the code of tried
import subprocess
args = ["C:\Program Files\Blackmagic Design\DaVinci Resolve\Resolve.exe", '--nogui']
resolve_headles = subprocess.Popen(args)
from python_get_resolve import GetResolve
resolve = GetResolve()
This should return an object of Resolve, but it always fails.
I believe this is because its running as a subprocess of my IDE
I've also tried this
from subprocess import call
dir = "C:\Program Files\Blackmagic Design\DaVinci Resolve"
cmdline = "Resolve.exe --nogui"
rc = call("start cmd /K " + cmdline, cwd=dir, shell=True)
This just has the same problem of Resolve running as a subprocess of Windows Command Processor.
I have a batch script which eventually runs two python file (one after another), but I am unable to handle the exit code from one workflow to another. Due to which my batch script is failing
batch file snippet:
#echo off
echo "Starting the automation Script"
cd "C:\Desktop\AutoImpement\"
echo "running the loging"
start python login.py
start python OrderTicket.py
pause
login script:
import time
from selenium import webdriver
browser = webdriver.Chrome(executable_path="C:\Desktop\AutoImpement\ChromeDriver")
browser.get('https://localhost:8080/login/#')
browser.find_element_by_id(“Login”).send_keys(“<userName>”)
browser.find_element_by_id (“Password”).send_keys(“password”)
browser.find_element_by_id(“submit”).click()
time.sleep(5)
browser.find_element_by_id(“ItemName”).send_keys(“test”)
browser.find_element_by_id (“Quantity”).send_keys(“5”)
browser.find_element_by_id(“Address”).send_keys(“Test”)
browser.find_element_by_id(“submitOrder”).click()
time.sleep(3)
browser.quit()
Verify the Order Script
import time
from selenium import webdriver
browser = webdriver.Chrome(executable_path="C:\Desktop\AutoImpement\ChromeDriver")
browser.get('https://localhost:8080/OrderDetails')
browser.find_element_by_id(“SreachOrder”).send_keys(“test”)
browser.find_element_by_id(“findOrder”).click()
time.sleep(3)
browser.quit()
When I run the batch file, only the login script is running successfully but the control is not shifting to the next script which verifies the order from the first file. I tried with sending the exit code from the python by changing the following but didn't work.
import time
from selenium import webdriver
try:
browser = webdriver.Chrome(executable_path="C:\Desktop\AutoImpement\ChromeDriver")
browser.get('https://localhost:8080/login/#')
browser.find_element_by_id(“Login”).send_keys(“<userName>”)
browser.find_element_by_id (“Password”).send_keys(“password”)
browser.find_element_by_id(“submit”).click()
time.sleep(5)
browser.find_element_by_id(“ItemName”).send_keys(“test”)
browser.find_element_by_id (“Quantity”).send_keys(“5”)
browser.find_element_by_id(“Address”).send_keys(“Test”)
browser.find_element_by_id(“submitOrder”).click()
time.sleep(3)
exit(0)
except:
print("Error Occured")
exit(1)
finally:
browser.quit()
In the above case in your batch file. Both the Scripts will run simultaneously.
Remove Start
python login.py
python OrderTicket.py
The Second will run only after first is complete.
I want to programmatically initialize a Firefox profile so that all files that are normally created on first run of Firefox are created.
I have tried using firefox -CreateProfile but it only creates the folder and a times.json file. I need the full profile. What I probably need to do is start and close Firefox using Python, but I cannot get Firefox to close.
I am using subprocess.Popen so I can retrieve the pid to later close Firefox with proc.terminate(), but it is not working. The pid's process is no longer found. Perhaps it has something to do with Firefox multiprocess?
import os
import subprocess
import time
profile_name = 'new_profile'
profile_path = f"C:\\Users\\user\\profiles\\{profile_name}"
os.system(f'firefox -CreateProfile "{profile_name} {profile_path}" -no-remote')
proc = subprocess.Popen(['firefox', '-profile', profile_path, '-no-remote'])
print(f"Pid: proc.pid")
time.sleep(3)
proc.terminate() # Not working
time.sleep(1)
os.system(f"taskkill /F /pid {proc.pid}") # Process not found
Running the script and confirming with taskkill the process on the original pid is no longer found:
Pid: 16472
ERROR: The process "16472" not found.
How can I close Firefox using this approach?