When using this library https://github.com/andersinno/python-database-sanitizer, I get an error that it cannot find mysqldump even though it is in my PATH.
edwin$ database-sanitizer mysql://root#localhost/test
Traceback (most recent call last):
File "/Users/edwin/code/test/venv/bin/database-sanitizer", line 11, in <module>
sys.exit(main())
File "/Users/edwin/code/test/venv/lib/python3.6/site-packages/database_sanitizer/__main__.py", line 60, in main
config=config,
File "/Users/edwin/code/test/venv/lib/python3.6/site-packages/database_sanitizer/dump/__init__.py", line 46, in run
for line in db_module.sanitize(url=parsed_url, config=config):
File "/Users/edwin/code/test/venv/lib/python3.6/site-packages/database_sanitizer/dump/mysql.py", line 68, in sanitize
stdout=subprocess.PIPE,
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mysqldump': 'mysqldump'
I also tried in the python interpreter and didn't have the error.
>>> import subprocess
>>> subprocess.Popen('mysqldump')
<subprocess.Popen object at 0x1081e2b38>
>>> Usage: mysqldump [OPTIONS] database [tables]
OR mysqldump [OPTIONS] --databases [OPTIONS] DB1 [DB2 DB3...]
OR mysqldump [OPTIONS] --all-databases [OPTIONS]
For more options, use mysqldump --help
I am using OSX.
Related
I have to run HIVE queries using python script eg,
For this $ bdscc -e "show databases"
I want to do the same in python, but I am getting error-
>>> import subprocess
>>> subprocess.call(['bdscc', '-e', 'show databases'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Is there any way to do this in python?
PS- The command $ bdscc -e "show databases" from terminal is running fine.
UPDATE-- bdscc is an alias set by my company for HIVE shell
I am able to execute this command successfully from command line and it copies the desired csv to the s3 bucket aws s3 cp /Users/kaswani/tips.csv s3://dplearn/
But, when I try to run the same from within python using subprocess it throws error:
subprocess.check_output(['aws','s3','cp','/Users/kaswani/tips.csv','s3://dplearn/'])
Error:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Users/kaswani/anaconda/envs/aws/lib/python3.5/subprocess.py", line 626, in check_output
**kwargs).stdout
File "/Users/kaswani/anaconda/envs/aws/lib/python3.5/subprocess.py", line 693, in run
with Popen(*popenargs, **kwargs) as process:
File "/Users/kaswani/anaconda/envs/aws/lib/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "/Users/kaswani/anaconda/envs/aws/lib/python3.5/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'aws'
I have not been able to debug it. Subprocess works just fine for basic commands like ls etc
Using, shell=True makes is work, i.e. subprocess.check_output(['aws','s3','cp','/Users/kaswani/tips.csv','s3://dplearn/'], shell=True)
I'm getting errors when trying to install abpcrawler (Repository below).
https://github.com/adblockplus/abpcrawler
When running the command
./run.py -b /usr/bin/firefox urls.txt outputdir
I am receiving the following error:
bash-3.2# ./run.py -b /usr/bin/firefox urls.txt outputdir
Communicating with client on port 29922
['/usr/bin/firefox', '--crawler-port', '29922', '-foreground', '-profile', '/tmp/tmpgZYg1r.mozrunner']
Traceback (most recent call last):
File "./run.py", line 195, in <module>
run()
File "./run.py", line 178, in run
runner.start()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mozrunner/base/browser.py", line 67, in start
BaseRunner.start(self, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mozrunner/base/runner.py", line 102, in start
self.process_handler.run(self.timeout, self.output_timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mozprocess/processhandler.py", line 700, in run
self.proc = self.Process([self.cmd] + self.args, **args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mozprocess/processhandler.py", line 103, in __init__
universal_newlines, startupinfo, creationflags)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Am I missing something? I have Mercurial installed and am able to clone the build tools repo.
Judging from the error message, /usr/bin/firefox couldn't be started because:
OSError: [Errno 2] No such file or directory
ABP Crawler requires Firefox to be installed, it will automate Firefox in order to collect data. If Firefox is actually installed, maybe the path is /usr/local/bin/firefox or something similar. You can run which firefox from the command line if you aren't sure.
I have the following command I run in terminal:
mongoexport --db database_name --collection agents --type=csv --fieldFile agFieldsTest.txt --out file/path/agTestInfo.csv
I tried to run it using:
>>> import subprocess
>>> subprocess.call(["mongoexport --db database_name --collection agents --type=csv --fieldFile agFieldsTest.txt --out file/path/agTestInfo.csv"])
I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
The best way is to break up the command into individual "words":
>>> subprocess.call(["mongoexport", "--db", "database_name", "--collection", "agents", "--type=csv", "--fieldFile", "agFieldsTest.txt", "--out", "file/path/agTestInfo.csv"])
Alternatively, you can use shell=True to have the shell do that for you:
>>> subprocess.call(["mongoexport --db database_name --collection agents --type=csv --fieldFile agFieldsTest.txt --out file/path/agTestInfo.csv"], shell=True)
Attempting to call a shell script with options from a python script.
The line ./dropbox_uploader.sh -s download /test/pictures pictures/ runs fine via SSH but errors when called from a python script:
import subprocess
subprocess.call(['./dropbox_uploader.sh -s download /test/pictures pictures/'])
Here is the error message:
Traceback (most recent call last):
File "sync.py", line 2, in <module>
subprocess.call(['./dropbox_uploader.sh -s download /test/pictures pictures/'])
File "/usr/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
In case if the first argument of subprocess.call is a list, it must contain the executable and arguments as separate items:
subprocess.call(['./dropbox_uploader.sh', '-s',
'download', '/test/pictures', 'pictures/'])
or, maybe, more convenient:
import shlex
cmd = './dropbox_uploader.sh -s download /test/pictures pictures/'
subprocess.call(shlex.split(cmd))
There is also an option to delegate parsing and execution to the shell:
cmd = './dropbox_uploader.sh -s download /test/pictures pictures/'
subprocess.call(cmd, shell=True)
(But please note the security warning)