Develop Reference

Problem with broken backup and python script

Right up front to be clear, I am not fluent in programming or python, but generally can accomplish what I need to with some research. Please excuse any bad formatting structure, as this is my first post to a board like this
I recently updated my laptop from Ubuntu 18.04 to 20.04. I created a full system backup with Dejadup, which due to a missing file, could not be restored. Research brought me to post on here from 2019 for manually restoring these files. The process called for 2 scripts, 1 to unpack and the second to reconstruct the files, both created by Hamish Downer.
The first,
"for f in duplicity-full.*.difftar.gz; do echo "$f"; tar xf "$f"; done"
seemed to work well and did unpack the files.
The second,
#!/usr/bin/env python3
import argparse
from pathlib import Path
import shutil
import sys"
is the start of a re-constructor script. Using terminal from within the directory I am trying to rebuild I enter the first line and return.
When I enter the second line of code the terminal just "hangs" with no activity, and will only come back to the prompt if I double click the cursor. I receive no errors or warnings. When I enter the third line of code
"from pathlib import Path"
and return I then get an error
from: can't read /var/mail/pathlib
The problem seems to originate with the "import argparse" command and I assume is due to a symlink.
argparse is located in /usr/local/lib/python3.8/dist-packages (1.4.0)
python3 is located in /usr/bin/
Python came with the Ubuntu 20.04 distribution package.
Any help with reconstructing these files would be greatly appreciated, especially in a batch as this script is meant to do versus trying to do them one file at a time.
Update: I have tried adding the "re-constructor" part of this script without success. This is a link to the script I want to use:
https://askubuntu.com/questions/1123058/extract-unencrypted-duplicity-backup-when-all-sigtar-and-most-manifest-files-are
Re-constructor script:
class FileReconstructor():
def __init__(self, unpacked_dir, restore_dir):
self.unpacked_path = Path(unpacked_dir).resolve()
self.restore_path = Path(restore_dir).resolve()
def reconstruct_files(self):
for leaf_dir in self.walk_unpacked_leaf_dirs():
target_path = self.target_path(leaf_dir)
target_path.parent.mkdir(parents=True, exist_ok=True)
with target_path.open('wb') as target_file:
self.copy_file_parts_to(target_file, leaf_dir)
def copy_file_parts_to(self, target_file, leaf_dir):
file_parts = sorted(leaf_dir.iterdir(), key=lambda x: int(x.name))
for file_part in file_parts:
with file_part.open('rb') as source_file:
shutil.copyfileobj(source_file, target_file)
def walk_unpacked_leaf_dirs(self):
"""
based on the assumption that all leaf files are named as numbers
"""
seen_dirs = set()
for path in self.unpacked_path.rglob('*'):
if path.is_file():
if path.parent not in seen_dirs:
seen_dirs.add(path.parent)
yield path.parent
def target_path(self, leaf_dir_path):
return self.restore_path / leaf_dir_path.relative_to(self.unpacked_path)
def parse_args(argv):
parser = argparse.ArgumentParser()
parser.add_argument(
'unpacked_dir',
help='The directory with the unpacked tar files',
)
parser.add_argument(
'restore_dir',
help='The directory to restore files into',
)
return parser.parse_args(argv)
def main(argv):
args = parse_args(argv)
reconstuctor = FileReconstructor(args.media/jerry/ubuntu, args.media/jerry/Restored)
return reconstuctor.reconstruct_files()
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))

I think you are typing the commands to the shell instead of python interpreter. Please check your prompt, python (started with python3) has >>>.
Linux has an import command (part of the ImageMagick) and understands import argparse, but it does something completely different.
import - saves any visible window on an X server and outputs it as an
image file. You can capture a single window, the entire screen, or any
rectangular portion of the screen.
This matches the described behaviour. import waits for a mouse click and then creates a large output file. Check if there is a new file named argparse.
An executable script contains instruction to be processed by an interpreter and there are many possible interpreters, several shells (bash and alternatives), languages like Perl, Python, etc. and also some very specialized like nft for firewall rules.
If you execute a script from the command line, the shell reads its first line. If it starts with #! characters (called "shebang"), it uses the program listed on that line. (note: /usr/bin/env there is just a helper to find the exact location of a program).
But if you want to use an interpreter interactively, you need to start it explicitly. The shebang line has no special meaning in this situation, only as the very first line of a script. Otherwise it is just a comment and is ignored.

How to run Open Pose binary (.exe) from within a Python script?

I am making a body tracking application where I want to run Open Pose if the user chooses to track their body movements. The OpenPose binary file can be run like so:
bin\OpenPoseDemo.exe --write_json 'path\to\dump\output'
So, in my Python script, I want to have a line of code that would run Open Pose, instead of having to ask the user to manually run OpenPose by opening a separate command line window. For that, I have tried:
import os
os.popen(r"C:\path\to\bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'")
But this gives the following error:
Error:
Could not create directory: 'C:\Users\Admin\Documents\Openpose\. Status error = -1. Does the parent folder exist and/or do you have writing access to that path?
Which I guess means that OpenPose can be opened only by going inside the openpose directory where the bin subdirectory resides. So, I wrote a shell script containing this line:
bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'
and saved it as run_openpose_binary.sh in the openpose directory (i.e., the same directory where bin is located).
I then tried to run this shell script from within my Python script like so:
import subprocess
subprocess.call(['sh', r'C:\path\to\openpose\run_openpose_binary.sh'])
and this gives the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I also tried the following:
os.popen(r"C:\path\to\openpose\run_openpose_binary.sh")
and
os.system(r"C:\path\to\openpose\run_openpose_binary.sh")
These do not produce any error, but instead just pop up a blank window and closes.
So, my question is, how do I run the OpenPoseDemo.exe from within my Python script?

For your last method, you're missing the return value from os.popen, which is a pipe. So, what you need is something like:
# untested as I don't have access to a Windows system
import os
with os.popen(r"/full/path/to/sh C:/path/to/openpose/run_openpose_binary.sh") as p:
# pipes work like files
output_of_command = p.read().strip() # this is a string
or, if you want to future-proof yourself, the alternative is:
# untested as I don't have access to a Windows system
popen = subprocess.Popen([r'/full/path/to/sh.exe', r'/full/path/to/run_openpose_binary.sh')], stdin=subprocess.PIPE, stdout=subprocess.PIPE,encoding='utf-8')
stdout, stderr = popen.communicate(input='')
Leave a comment if you have further difficulty.

I've had to fight this battle several times and I've found a solution. It's likely not the most elegant solution but it does work, and I'll explain it using an example of how to run OpenPose on a video.
You've got your path to the openpose download and your path to the video, and from there it's a 3-line solution. First, change the current working directory to that openpose folder, and then create your command, then call subprocess.run (I tried using subprocess.call and that did not work. I did not try shell=False but I have heard it's a safer way to do so. I'll leave that up to you.)
import os
import subprocess
openpose_path = "C:\\Users\\me\\Desktop\\openpose-1.7.0-binaries-win64-gpu-python3.7-flir-3d_recommended\\openpose\\"
video_path = "C:\\Users\\me\\Desktop\\myvideo.mp4"
os.chdir(openpose_path)
command = "".join(["bin\\OpenPoseDemo.exe", " -video ", video_path])
subprocess.run(command, shell=True)

How to run python from C# without .exe file

I am a Python beginner and wrote some Python code that I want to run from my C# code.
In all the answers I have seen already, the way was to make a .exe file from the .py one and run it by system call.
However, I want it so that I do not need to make the .exe file and can write the commands with arguments as I could in the command line.
C:\Users\ntuser> python C:\Users\ntuser\Documents\run_python.py 3
Is there a way to do it?
I found a way to pass just one command—but I need to make two: 1. Go to "C:\Users\ntuser" 2. Run the Python code.
Thanks!

OK, so I found a solution.
Thanks to UnholySheep for the help.
What you need to do is:
Make the python as system variable
Go by code to your home directory:
Directory.SetCurrentDirectory(Environment.GetFolderPath(Environment.SpecialFolder.UserProfile));
Add this code for calling the python code:
System.Diagnostics.Process process = new System.Diagnostics.Process();
System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
startInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;//if you want to hide the window
startInfo.FileName = "cmd.exe";
startInfo.Arguments = "/C python pytonPath\\python_code.py";
process.StartInfo = startInfo;
process.Start();
process.WaitForExit();//if you want to wait

Running a Python model

I've been given a piece of code which is a physical model (filename 'agnsim.py') and some instructions to run it which I'm confused by.
The instructions say that I should import the code using
import agnsim as agn
and then to run the model with
ed = agn.Wilp(dens=3., incr=0.2, drac=2.0)
The argument above in Wilp will configure the run.
My question is, how do I actually run this? Do I create a separate .py file that contains these two lines of code?
I've only ever run simple python programs before using e.g. >>>python file.py

You should literally just open up the python console and type those two lines in.
$python
import agnsim as agn
ed = agn.Wilp(dens=3., incr=0.2, drac=2.0)
Make sure that agnsim.py is located at the same folder level from which you start python. E.g. If you're in "My Documents" and "agnsim.py" is in my documents, you should cd to "My Documents" and then start python there (from the command line).

get the list of metadata associated to a file using python in Ubuntu

I'm trying to get the list of meta-data associated to a file, using python in Ubuntu.
Without using python, the command "extract" works very well but I don't know how to use it with python, I always get a message saying that "extract" is not defined.

I assume you're asking about the metadata that appears in the Windows "Properties" dialogue under the "Summary" tab. (If not, just disregard this.) Here's how I managed it.
Download and install Python win32 extensions. This will put win32, win32com, etc. into your Python[ver]/Lib/site-packages folder. These bring the win32api, win32com, etc. For some reason, I couldn't get the version for Python 2.6 (in build 216) to work. I updated my system to Python 2.7 and used the 216 build for Python 2.7, and it worked. (To download & install, follow the link above, click the link reading 'pywin32', click the link for the latest build (currently 216), click the link for the .exe file that matches your system and Python installation (for me, it was pywin32-216.win32-py2.7.exe). Run the .exe file.)
Copy and paste the code from the "Get document summary information" page on Tim Golden's tutorial into a .py file on your own computer.
Tweak the code. You don't really have to tweak the code, but if you run this Tim's script as your main module, and if you don't supply a pathname as your first sys.argv, then you'll get an error. To make the tweak, scroll down to the bottom of the code, and omit the final block, which starts with if __name__ == '__main__':.
Save your file as something like property_reader.py, and call its property_sets(filepath) method. This method returns a generator object. You can iterate through the generator to see all the properties and their values. You could implement it like this:
# Assuming 'property_reader.py' is the name of the module/file in which you saved Tim Golden's code...
import property_reader
propgenerator = property_reader.property_sets('[your file path]')
for name, properties in propgenerator:
print name
for k, v in properties.items ():
print " ", k, "=>", v
The output of the above code will be something like the following:
DocSummaryInformation
PIDDSI_CATEGORY => qux
SummaryInformation
PIDSI_TITLE => foo
PIDSI_COMMENTS => flam
PIDSI_AUTHOR => baz
PIDSI_KEYWORDS => flim
PIDSI_SUBJECT => bar

extract is based on the libextractor library. You can access the library from Python by installing the python-extractor package on Ubuntu.

In case you're using Windows your question has been addressed on SO already.