Right up front to be clear, I am not fluent in programming or python, but generally can accomplish what I need to with some research. Please excuse any bad formatting structure, as this is my first post to a board like this
I recently updated my laptop from Ubuntu 18.04 to 20.04. I created a full system backup with Dejadup, which due to a missing file, could not be restored. Research brought me to post on here from 2019 for manually restoring these files. The process called for 2 scripts, 1 to unpack and the second to reconstruct the files, both created by Hamish Downer.
The first,
"for f in duplicity-full.*.difftar.gz; do echo "$f"; tar xf "$f"; done"
seemed to work well and did unpack the files.
The second,
#!/usr/bin/env python3
import argparse
from pathlib import Path
import shutil
import sys"
is the start of a re-constructor script. Using terminal from within the directory I am trying to rebuild I enter the first line and return.
When I enter the second line of code the terminal just "hangs" with no activity, and will only come back to the prompt if I double click the cursor. I receive no errors or warnings. When I enter the third line of code
"from pathlib import Path"
and return I then get an error
from: can't read /var/mail/pathlib
The problem seems to originate with the "import argparse" command and I assume is due to a symlink.
argparse is located in /usr/local/lib/python3.8/dist-packages (1.4.0)
python3 is located in /usr/bin/
Python came with the Ubuntu 20.04 distribution package.
Any help with reconstructing these files would be greatly appreciated, especially in a batch as this script is meant to do versus trying to do them one file at a time.
Update: I have tried adding the "re-constructor" part of this script without success. This is a link to the script I want to use:
https://askubuntu.com/questions/1123058/extract-unencrypted-duplicity-backup-when-all-sigtar-and-most-manifest-files-are
Re-constructor script:
class FileReconstructor():
def __init__(self, unpacked_dir, restore_dir):
self.unpacked_path = Path(unpacked_dir).resolve()
self.restore_path = Path(restore_dir).resolve()
def reconstruct_files(self):
for leaf_dir in self.walk_unpacked_leaf_dirs():
target_path = self.target_path(leaf_dir)
target_path.parent.mkdir(parents=True, exist_ok=True)
with target_path.open('wb') as target_file:
self.copy_file_parts_to(target_file, leaf_dir)
def copy_file_parts_to(self, target_file, leaf_dir):
file_parts = sorted(leaf_dir.iterdir(), key=lambda x: int(x.name))
for file_part in file_parts:
with file_part.open('rb') as source_file:
shutil.copyfileobj(source_file, target_file)
def walk_unpacked_leaf_dirs(self):
"""
based on the assumption that all leaf files are named as numbers
"""
seen_dirs = set()
for path in self.unpacked_path.rglob('*'):
if path.is_file():
if path.parent not in seen_dirs:
seen_dirs.add(path.parent)
yield path.parent
def target_path(self, leaf_dir_path):
return self.restore_path / leaf_dir_path.relative_to(self.unpacked_path)
def parse_args(argv):
parser = argparse.ArgumentParser()
parser.add_argument(
'unpacked_dir',
help='The directory with the unpacked tar files',
)
parser.add_argument(
'restore_dir',
help='The directory to restore files into',
)
return parser.parse_args(argv)
def main(argv):
args = parse_args(argv)
reconstuctor = FileReconstructor(args.media/jerry/ubuntu, args.media/jerry/Restored)
return reconstuctor.reconstruct_files()
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))
I think you are typing the commands to the shell instead of python interpreter. Please check your prompt, python (started with python3) has >>>.
Linux has an import command (part of the ImageMagick) and understands import argparse, but it does something completely different.
import - saves any visible window on an X server and outputs it as an
image file. You can capture a single window, the entire screen, or any
rectangular portion of the screen.
This matches the described behaviour. import waits for a mouse click and then creates a large output file. Check if there is a new file named argparse.
An executable script contains instruction to be processed by an interpreter and there are many possible interpreters, several shells (bash and alternatives), languages like Perl, Python, etc. and also some very specialized like nft for firewall rules.
If you execute a script from the command line, the shell reads its first line. If it starts with #! characters (called "shebang"), it uses the program listed on that line. (note: /usr/bin/env there is just a helper to find the exact location of a program).
But if you want to use an interpreter interactively, you need to start it explicitly. The shebang line has no special meaning in this situation, only as the very first line of a script. Otherwise it is just a comment and is ignored.
Related
I am making a body tracking application where I want to run Open Pose if the user chooses to track their body movements. The OpenPose binary file can be run like so:
bin\OpenPoseDemo.exe --write_json 'path\to\dump\output'
So, in my Python script, I want to have a line of code that would run Open Pose, instead of having to ask the user to manually run OpenPose by opening a separate command line window. For that, I have tried:
import os
os.popen(r"C:\path\to\bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'")
But this gives the following error:
Error:
Could not create directory: 'C:\Users\Admin\Documents\Openpose\. Status error = -1. Does the parent folder exist and/or do you have writing access to that path?
Which I guess means that OpenPose can be opened only by going inside the openpose directory where the bin subdirectory resides. So, I wrote a shell script containing this line:
bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'
and saved it as run_openpose_binary.sh in the openpose directory (i.e., the same directory where bin is located).
I then tried to run this shell script from within my Python script like so:
import subprocess
subprocess.call(['sh', r'C:\path\to\openpose\run_openpose_binary.sh'])
and this gives the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I also tried the following:
os.popen(r"C:\path\to\openpose\run_openpose_binary.sh")
and
os.system(r"C:\path\to\openpose\run_openpose_binary.sh")
These do not produce any error, but instead just pop up a blank window and closes.
So, my question is, how do I run the OpenPoseDemo.exe from within my Python script?
For your last method, you're missing the return value from os.popen, which is a pipe. So, what you need is something like:
# untested as I don't have access to a Windows system
import os
with os.popen(r"/full/path/to/sh C:/path/to/openpose/run_openpose_binary.sh") as p:
# pipes work like files
output_of_command = p.read().strip() # this is a string
or, if you want to future-proof yourself, the alternative is:
# untested as I don't have access to a Windows system
popen = subprocess.Popen([r'/full/path/to/sh.exe', r'/full/path/to/run_openpose_binary.sh')], stdin=subprocess.PIPE, stdout=subprocess.PIPE,encoding='utf-8')
stdout, stderr = popen.communicate(input='')
Leave a comment if you have further difficulty.
I've had to fight this battle several times and I've found a solution. It's likely not the most elegant solution but it does work, and I'll explain it using an example of how to run OpenPose on a video.
You've got your path to the openpose download and your path to the video, and from there it's a 3-line solution. First, change the current working directory to that openpose folder, and then create your command, then call subprocess.run (I tried using subprocess.call and that did not work. I did not try shell=False but I have heard it's a safer way to do so. I'll leave that up to you.)
import os
import subprocess
openpose_path = "C:\\Users\\me\\Desktop\\openpose-1.7.0-binaries-win64-gpu-python3.7-flir-3d_recommended\\openpose\\"
video_path = "C:\\Users\\me\\Desktop\\myvideo.mp4"
os.chdir(openpose_path)
command = "".join(["bin\\OpenPoseDemo.exe", " -video ", video_path])
subprocess.run(command, shell=True)
I'm still relatively new to Python.
I'm supposed to write a Python script that prints out all files and directories under certain conditions:
You can find the explanation of the exercise below.
The solution to the exercise of my course Scripting-languages doesn't seem to print the actual results and I've searched and tried but I don't see what's wrong with it. After all it's supposed to be the solution.
The supposed solution:
from pathlib import Path
import argparse
from pathlib import Path
import os
import re
def minimum_size(size):
def check(filename):
return os.path.getsize(filename) >= size
parser = argparse.ArgumentParser(prog='find')
parser.add_argument('path', default='.')
parser.add_argument('--minimum-size', dest='minimum_size', default=0, type=int)
parser.add_argument('--maximum-size', dest='maximum_size', default=float('inf'), type=int)
parser.add_argument('--no-directories', dest='no_directories', default=False, action='store_true')
parser.add_argument('--no-files', dest='no_files', default=False, action='store_true')
parser.add_argument('--extension', dest='extension')
parser.add_argument('--contains', dest='contains')
args = parser.parse_args()
path = Path(args.path)
for entry in path.glob('**/*'):
if not os.path.getsize(entry) >= args.minimum_size:
continue
if not os.path.getsize(entry) <= args.maximum_size:
continue
if args.no_directories and os.path.isdir(entry):
continue
if args.no_files and os.path.isfile(entry):
continue
if args.extension and not entry.suffix == args.extension:
continue
if args.contains:
if not os.path.isfile(entry):
continue
with open(entry, 'r') as file:
contents = file.read()
if not re.search(args.contains, contents):
continue
print(entry)
The explanation:
My own comments about this: I think the examples are supposed to include the extension when calling the file and so I do do that when trying to execute the code.
I even include the whole path to the file so that I won't get an 'ObjectNotFound' error.
When I run the solution in Powershell in Python, the command prompt opens and closes immediately.
When I run it directly in Powershell, the command prompt opens, lists all the output and closes immediately when it's done. If you could give me a way to just stop the command prompt from closing so that I can see the result. I'd already be very happy.
Write a script `find.py` that prints out all files and directories
that satisfy certain conditions.
The following functionality must be supported:
* One positional parameter that specifies the directory in which to start looking recursively.
* `--minimum-size=N`: only files whose size is at least `N` must be listed.
* `--maximum-size=N`: only files whose size is at most `N` must be listed.
* `--no-directories`: don't list directories.
* `--no-files`: don't list files.
* `--extension=EXT`: only list entries with extension `EXT`
* `--contains=REGEX`: only list files whose contents match `REGEX`.
To get a better feel for how your script should work, feel free
to experiment with the solution. For example, try
`bash
# List everything starting from two directories up
$ python find ../..
# List all python files in current directory
$ python find --extension=.py .
`
Instead of running the script in Powershell which uses Command prompt to execute the script. Run the script directly in Command prompt so you can see the output.
To change terminal in Visual Studio Code: Click on the current terminal to see a dropdown menu and click "Select Default Shell". Choose "Command Prompt" from the options.
I had this script working for me, before I decided I'm gonna rewrite everything and make it portable.
Without delving too much into the details, there's a central Bash script, which calls 5 other Bash scripts in their own respective folders. I have no intention of porting to Windows anytime soon, as of current this is just for Linux.
The execution path of the central Bash script is:
dos.1/1-init.sh dos.1/
dos.2/1-trace-to-file.sh dos.2/ dos.1/
dos.3/1-recognize-categories.sh dos.3/
dos.4/1-ping-in-groups.sh dos.4/ dos.3/
dos.5/init.sh dos.5/ dos.4/
I run with ./init.sh
Before the script was 'portable' I was using explicit file paths inside each respective script. All was well and good. The program itself is a combination of Bash and Python, and writes to files in one directory, so that they can be manipulated in various ways, before being read back into different parts of the program.
I understand that the fastest way to do this would be to write a monolithic Python script, using subprocess calls for the Bash side of things... However, I am doing it this way to ease maintenance, and (before I started making it 'portable') it was lightning fast.
My issue now is this: each time I have to read text into Python (either from SQL or from file) there's always this added garbage. Up until this point, I have been using sed, awk and Python's .rstrip() function to manage this... Which is all well and good, but this one damn function will not play nice... And I feel there must be a better way.
In bash I call it with:
$prog_dir=$1
$data_dir=$2
$prog_dir/2fast-ping.py $data_dir/group0.txt > $prog_dir/group0_averages.txt
$prog_dir/2fast-ping.py $data_dir/group1.txt > $prog_dir/group1_averages.txt
...
Now I know that I could write to file from within Python, but in this instance I have other reasons not to.
The issue, is that when the 2fast-ping.py script is ran, it reads the text file in with commas and a newline char. I have vigorously checked and I can confirm that the group#.txt files 100% do not contain commas. Here's the Python:
import sys
import subprocess
import select
from concurrent.futures import ThreadPoolExecutor
filename = sys.argv[1]
f = open(filename, "r")
ips = [elem.rstrip('\n') for elem in f]
print(ips)
f.close()
The script goes on to do some work on the IPs afterwards, but this is the painful part. If I call the script direct from CLI: ./2fast-ping.py ../dos.3/group0.txt, the text is processed PROPERLY and the superseding instructions actually function. But, when called from the first init script, the program basically sh*ts itself because each line is read in with commas. It works until the point where it starts to use the processed info, then:
<actual IP would be here>
ping: ('##.###.###.###',): Name or service not known
Of course, the issue is the ('',) But, Python is adding that in, and I don't know how to stop it :(
Any ideas?
Python code was okay, just passing an additional / with the argument :(
I'm developing a plugin for Inkscape. Some versions:
Inkscape v0.92.3
Windows 10, version 1803 (build 17134.165)
Python 3.7 explicitly installed
MonoDevelop Version 7.7 Preview (7.7) Extra versions below
Installation Locations:
Inkscape: C:\Program Files\Inkscape
Extension: C:\Program Files\Inkscape\share\extensions
Contains: myplugin.inx, myplugin.py, MyPlugin.exe
I've made a plugin which, for development reasons, works as currently intended.
Most important of all, it runs when I run it either from MonoDevelop, or the built exe itself (both with the generated .dll's etc in the same location, or with only the exe copied to a different location).
I use (a slightly edited version of) SugarPillStudio's python script to run the .exe file. However, when I run that python script by invoking the extension, the .exe is not launched. Inkscape blinks a message that says 'MyPlugin is launching...' and closes that as fast as it opens.
I know that the python script works, because I have it print debugging lines to a .log file on my desktop. I know that the .exe doesn't launch because I have it also writing lines to the same .log file, first thing when the main() is invoked. When I (successfully) run the .exe it does print to the file, when I run the extension it doesn't.
This leads me to believe there's a problem with the python script in invoking the .exe. Any help?
Python Script:
#!/usr/bin/env python
'''
sugarpillstudios.com/wp/?p=142
'''
import os, sys, subprocess, datetime
f=open("C:\Users\Diamundo\Documents\plugin.log", "a+")
f.write("[PYT] %s Python script called at: %s.\n" % (datetime.datetime.now().isoformat(), os.getcwd() ) )
argv = []
for arg in sys.argv[:]:
if arg.startswith("--executable="):
executable = arg.split("=")[1]
else:
argv.append(arg)
argv[0] = executable
f.write("[PYT] %s %s\n" % ( datetime.datetime.now().isoformat(), executable ) )
process = subprocess.Popen(argv,shell=False,stdout=subprocess.PIPE)
print process.communicate()[0]
Plugin.inx:
<inkscape-extension>
<name>MyPlugin</name>
<id>name.space.plugin.main</id>
<param name="executable" type="string" gui-hidden="true">MyPlugin.exe</param>
<effect>
<object-type>all</object-type>
<effects-menu>
<submenu _name="MyPlugin"/>
</effects-menu>
</effect>
<script>
<command reldir="extensions" interpreter="python">myplugin.py</command>
</script>
</inkscape-extension>
Extra Monodevelop versions:
Runtime:
Microsoft .NET 4.0.30319.42000
GTK+ 2.24.26 (Light theme)
GTK# 2.12.45
NuGet
Version: 4.3.1.4445
.NET Core
Runtime: C:\Program Files\dotnet\dotnet.exe
Runtime Versions:
2.0.9
2.0.5
SDK: C:\Program Files\dotnet\sdk\2.1.202\Sdks
SDK Versions:
2.1.202
2.1.4
MSBuild SDKs: Not installed
Inkscape uses Python 2.7, which it brings with it, unless you set that differently in the settings file (edit manually).
If you want to write an Inkscape extension, you can learn how to do this by:
reading https://inkscape.org/develop/extensions/
following the examples in other extensions that work (e.g. for running additional Inkscape instances, you could follow this one: https://gitlab.com/su-v/inx-pathops/blob/master/src/pathops.py)
Loosely based on the pathops.py file, linked by Moini in her answer, I've come up with the following file.
About
It uses the inkex.py (source on GitLab) library to declare an Inkscape Effect. The Effect class uses the OptionParser library to parse the default given parameters (e.g. --id=$$ for selected nodes where $$ is the XML node's 'id' tag's value). By adding the custom executable option, we can also parse this.
Parsing arguments
After the OptionParser is done parsing, the values will be visible in self.options, i.e. our executable now lives in self.options.executable (because of the action="store" and dest="executable" parameters).
Furthermore, the temporary SVG-file as created by Inkscape, can be found in self.svg_file.
Saving edits
As previously said, Inkscape makes a temporary file with the contents of the SVG in its then current state. Any edits you(r plugin) make(s) should not be saved back to this file, but returned to Inkscape itself - this is the premise of the Effect class: it edits an SVG and returns the edit to Inkscape. Further reading here.
Instead, in your plugin you should (readonly) open the file, read its contents, and then edit it. When you're done editing, write the entire SVG to your commandline.
Then, the line out, err = process.communicate(None) will grab your plugin's output and error-output. These are used to return information to Inkscape.
Notes
The structure of the cmd array is of no importance, except the fact that the executable should come as the very first element. All other array-elements can be anything in any order, I just added '--id=$$' to every ID because that's the way Inkscape uses, and this way it looks the same as if there's no Python middleware present. The same goes for the self.svg_file which I placed last, Inkscape does the same in its arguments - you could also make '--file='+self.svg_file from it for clarity.
Source
#!/usr/bin/env python
import os
from subprocess import Popen, PIPE
import time
try:
import inkex_local as inkex
except ImportError:
import inkex
#import simplestyle
class MyPlugin(inkex.Effect):
def __init__(self):
inkex.Effect.__init__(self)
self.OptionParser.add_option("--executable", action="store", type="string", dest="executable", default="MyPlugin.exe")
def effect(self):
out = err = None
cmd = []
cmd.append(self.options.executable)
for id in self.options.ids:
cmd.append("--id=" + id)
cmd.append(self.svg_file)
#inkex.debug(cmd);
process = Popen(cmd, shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)
out, err = process.communicate(None)
if process.returncode == 0:
print out
elif err is not None:
inkex.errormsg(err)
if __name__ == '__main__':
myplugin = MyPlugin()
myplugin.affect()
Idea
Basically, what my script does is checking C:/SOURCE for .txt files and add a timestamp to it. To replicate it you can basically make that folder and put some txt files in there. Then, it's supposed to run a .vbs file, which then runs a .bat files with some rclone commands which don't matter here. I did it like this because there wont be a CMD window opening when running the rclone command through the .vbs file.
Python code
import time, os, subprocess
while True:
print("Beginning checkup")
print("=================")
timestamp = time.strftime('%d_%m_%H_%M') # only underscores: no naming issues
the_dir = "C:/SOURCE"
for fname in os.listdir(the_dir):
if fname.lower().endswith(".txt"):
print("found " + fname)
time.sleep(0.1)
new_name = "{}-{}.txt".format(os.path.splitext(fname)[0], timestamp)
os.rename(os.path.join(the_dir, fname), os.path.join(the_dir, new_name))
time.sleep(0.5)
else:
subprocess.call(['cscript.exe', "copy.vbs"])
time.sleep(60)
VBScript code
Set WshShell = CreateObject("WScript.Shell" )
WshShell.Run Chr(34) & "copy.bat" & Chr(34), 0
Set WshShell = Nothing
The only important part for the Python script is below the very last else, where the subprocess.call() is supposed to run the .vbs file. What happens when running the script is it shows the first two lines that always come up when running CMD, but then nothing.
How could I fix that? I tried:
subprocess.call("cscript copy.vbs")
subprocess.call("cmd /c copy.vbs")
both with the same outcome, it doesn't do anything.
Anyone have an idea?
Why are you invoking a VBScript to invoke a batch script from Python? You should be able to simple run whatever the batch script is doing directly from your Python code. But even if you wanted to keep the batch script, something like this should do just fine without VBScript as an intermediary.
subprocess.call(['cmd', '/c', 'copy.bat'])
You may want to give the full path of the batch file, though, to avoid issues like the working directory not being what you think it is.
If your batch script resides in the same directory as the Python script, you can build the path with something like this:
import os
import subprocess
scriptdir = os.path.dirname(__file__)
batchfile = os.path.join(scriptdir, 'copy.bat')
subprocess.call(['cmd', '/c', os.path.realpath(batchfile)])
It seems there is no such an operation that could not be done using plain Python. Scan a directory, copy a file -- Python has it all in the standard library. See os.path and shutil modules.
Adding VB scripts and launching subprocesses make your code complex and difficult to debug.