I am checking out a source for a given url using a python script and I want to go to the downloadedFoler/src directory and perform a mvn clean install. I want to do it in the same script. Thank in advance.
You can do the following:
import os
import subprocess
# Context Manager to change current directory.
# I looked at this implementation on stackoverflow but unfortunately do not have the link
# to credit the user who wrote this part of the code.
class changeDir:
def __init__(self, newPath):
self.newPath = os.path.expanduser(newPath)
# Change directory with the new path
def __enter__(self):
self.savedPath = os.getcwd()
os.chdir(self.newPath)
# Return back to previous directory
def __exit__(self, etype, value, traceback):
os.chdir(self.savedPath)
# folderPath = path of the folder you want to run mvn clean install on
with changeDir(folderPath):
# ****** NOTE ******: using shell=True is strongly discouraged since it possesses security risks
subprocess.call(["mvn", "clean", "install"], shell=True)
Related
I wrote a test for a function that renames files from e.g. /videos/vid_youtube.mp4 to /videos/youtube/vid.mp4. The test patches the fs with Pyfakefs.
When the code actually renames the file, I get this error.
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/code/project/test/DLV/videos/vid_youtube.mp4' -> '/home/user/code/project/test/DLV/videos/youtube/vid.mp4'
This is how I set up fakefs
def setUp(self) -> None:
self.setUpPyfakefs()
self.fs.create_dir(Path(Dirs.VIDEOS)) # /home/user/code/project/test/DLV/videos
self.fs.create_file(Path(Dirs.VIDEOS / "vid_youtube.mp4"))
The code under test.
class Files:
#staticmethod
def rename_video_files():
all_files = Collect.video_files()
for files_for_type in all_files:
for file in all_files[files_for_type]:
path = Path(file)
platform = Files.detect_platform(path)
platform_dir = Path(Dirs.VIDEOS, platform)
platform_dir.mkdir(exist_ok=True)
new_name = path.stem.replace(f'_{platform}', '')
new_path = Dirs.VIDEOS / platform / f'{new_name}{path.suffix}'
old_path = Dirs.VIDEOS / path
old_path.rename(new_path) # throws FileNotFoundError
I debugged the test and the method under test and even passed the fake fs to rename_video_files(fakefs) to inspect the files and directories. All files and directories look correct.
What is going wrong here?
The problem here is most likely the static initialization of Dirs.VIDEOS. This is initialized at load time as a pathlib.Path, and won't be patched later at the time you setup pyfakefs (the same problem would happen if you where to use unittest.patch for patching).
There are two ways to fix this:
adapt the code to not initialize the path statically
This could be done by statically defining the str path, and converting it to a Path at run time, or by using a method to get the path instead of an attribute (e.g. Dirs.VIDEO() instead of Dirs.VIDEO`).
adapt the test to reload the tested code
If reloading the tested code after pyfakefs has been initialized, it will be correctly patched. pyfakefs provides an argument in setUpPyfakefs that does that:
from pyfakefs.fake_filesystem_unittest import TestCase
from my_module import video_files
from my_module.video_files import Dirs, Files
class MyTest(TestCase):
def setUp(self) -> None:
self.setUpPyfakefs(modules_to_reload=[video_files])
self.fs.create_dir(
Path(Dirs.VIDEOS)) # /home/user/code/project/test/DLV/videos
self.fs.create_file(Path(Dirs.VIDEOS / "vid_youtube.mp4"))
(under the assumption, that your code under test is located in my_module.video_files.py)
Disclaimer:
I'm a contributor to pyfakefs.
I been writing markdown files lately, and have been using the awesome table of content generator (github-markdown-toc) tool/script on a daily basis, but I'd like it to be regenerated automatically each time I press Ctrl+s, right before saving the md file in my sublime3 environment.
What I have done till now was to generate it from the shell manually, using:
gh-md-toc --insert my_file.md
So I wrote a simple plugin, but for some reason I can't see the result I wanted.
I see my print but the toc is not generated.
Does anybody has any suggestions? what's wrong?
import sublime, sublime_plugin
import subprocess
class AutoRunTOCOnSave(sublime_plugin.EventListener):
""" A class to listen for events triggered by ST. """
def on_post_save_async(self, view):
"""
This is called after a view has been saved. It runs in a separate thread
and does not block the application.
"""
file_path = view.file_name()
if not file_path:
return
NOT_FOUND = -1
pos_dot = file_path.rfind(".")
if pos_dot == NOT_FOUND:
return
file_extension = file_path[pos_dot:]
if file_extension.lower() == ".md": #
print("Markdown TOC was invoked: handling with *.md file")
subprocess.Popen(["gh-md-toc", "--insert ", file_path])
Here's a slightly modified version of your plugin:
import sublime
import sublime_plugin
import subprocess
class AutoRunTOCOnSaveListener(sublime_plugin.EventListener):
""" A class to listen for events triggered by ST. """
def on_post_save_async(self, view):
"""
This is called after a view has been saved. It runs in a separate thread
and does not block the application.
"""
file_path = view.file_name()
if not file_path:
return
if file_path.split(".")[-1].lower() == "md":
print("Markdown TOC was invoked: handling with *.md file")
subprocess.Popen(["/full/path/to/gh-md-toc", "--insert ", file_path])
I changed a couple things, along with the name of the class. First, I simplified your test for determining if the current file is a Markdown document (fewer operations means less room for error). Second, you should include the full path to the gh-md-toc command, as it's possible subprocess.Popen can't find it on the default path.
I figured out, since gh-md-toc is a bash script, I replaced the following line:
subprocess.Popen(["gh-md-toc", "--insert ", file_path])
with:
subprocess.check_call("gh-md-toc --insert %s" % file_path, shell=True)
So now it works well, on each save.
I have a similar question to this one:Similar Question.
I have a GUI and where the user can input information and the other scripts use some of that information to run.I have 4 different scripts for each button. I run them as a subprocess so that the main gui doesn’t act up or say that it’s not responding. This is an example of what I have since the code is really long since I used PAGE to generate the gui.
###Main.py#####
import subprocess
def resource_path(relative_path):
#I got this from another post to include images but I'm also using it to include the scripts"
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = sys._MEIPASS
except Exception:
base_path = os.path.abspath(".")
return os.path.join(base_path, relative_path)
Class aclass:
def get_info(self):
global ModelNumber, Serial,SpecFile,dateprint,Oper,outputfolder
ModelNumber=self.Model.get()
Serial=self.SerialNumber.get()
outputfolder=self.TEntry2.get()
SpecFile= self.Spec_File.get()
return ModelNumber,Serial,SpecFile,outputfolder
def First(self):
aclass.get_info(self) #Where I use the resource path function
First_proc = subprocess.Popen([sys.executable, resource_path('first.py'),str(ModelNumber),str(Serial),str(path),str(outputfolder)])
First_proc.wait()
#####First.py#####
import numpy as np
import scipy
from main import aclass
ModelNumber = sys.argv[1]
Serial = sys.argv[2]
path = sys.argv[3]
path_save = sys.argv[4]
and this goes on for my second,third, and fourth scripts.
In my spec file, I added:
a.datas +=[('first.py','C\\path\\to\\script\\first.py','DATA')]
a.datas +=[('main.py','C\\path\\to\\script\\main.py','DATA')]
this compiles and it works, but when I try to convert it to an .exe, it crashes because it can't import first.py properly and its own libraries (numpy,scipy....etc). I've tried adding it to the a.datas, and runtime_hooks=['first.py'] in the spec file...and I can't get it to work. Any ideas? I'm not sure if it's giving me this error because it is a subprocess.
Assuming you can't restructure your app so this isn't necessary (e.g., by using multiprocessing instead of subprocess), there are three solutions:
Ensure that the .exe contains the scripts as an (executable) zipfile—or just use pkg_resources—and copy the script out to a temporary directory so you can run it from there.
Write a multi-entrypoint wrapper script that can be run as your main program, and also run as each script—because, while you can't run a script out of the packed exe, you can import a module out of it.
Using pkg_resources again, write a wrapper that runs the script by loading it as a string and running it with exec instead.
The second one is probably the cleanest, but it is a bit of work. And, while we could rely on setuptools entrypoints to some of the work, trying to explain how to do this is much harder than explaining how to do it manually,1 so I'm going to do the latter.
Let's say your code looked like this:
# main.py
import subprocess
import sys
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, 'vikings.py', spam])
subprocess.run([sys.executable, 'waitress.py', spam, eggs])
# vikings.py
import sys
print(' '.join(['spam'] * int(sys.argv[1])))
# waitress.py
import sys
import time
spam, eggs = int(sys.argv[1]), int(sys.argv[2]))
if eggs > spam:
print("You can't have more eggs than spam!")
sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")
So, you run it like this:
$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!
We want to reorganize it so there's a script that looks at the command-line arguments to decide what to import. Here's the smallest change to do that:
# main.py
import subprocess
import sys
if sys.argv[1][:2] == '--':
script = sys.argv[1][2:]
if script == 'vikings':
import vikings
vikings.run(*sys.argv[2:])
elif script == 'waitress':
import waitress
waitress.run(*sys.argv[2:])
else:
raise Exception(f'Unknown script {script}')
else:
spam, eggs = sys.argv[1], sys.argv[2]
subprocess.run([sys.executable, __file__, '--vikings', spam])
subprocess.run([sys.executable, __file__, '--waitress', spam, eggs])
# vikings.py
def run(spam):
print(' '.join(['spam'] * int(spam)))
# waitress.py
import sys
import time
def run(spam, eggs):
spam, eggs = int(spam), int(eggs)
if eggs > spam:
print("You can't have more eggs than spam!")
sys.exit(2)
print("Frying...")
time.sleep(2)
raise Exception("This sketch is getting too silly!")
And now:
$ python3 main.py 3 4
spam spam spam
You can't have more eggs than spam!
A few changes you might want to consider in real life:
DRY: We have the same three lines of code copied and pasted for each script, and we have to type each script name three times. You can just use something like __import__(sys.argv[1][2:]).run(sys.argv[2:]) with appropriate error handling.
Use argparse instead of this hacky special casing for the first argument. If you're already sending non-trivial arguments to the scripts, you're probably already using argparse or an alternative anyway.
Add an if __name__ == '__main__': block to each script that just calls run(sys.argv[1:]), so that during development you can still run the scripts directly to test them.
I didn't do any of these because they'd obscure the idea for this trivial example.
1 The documentation is great as a refresher if you've already done it, but as a tutorial and explanatory rationale, not so much. And trying to write the tutorial that the brilliant PyPA guys haven't been able to come up with for years… that's probably beyond the scope of an SO answer.
I need to add the win_unicode_console module to my usercustomize.py file, as described by the documentation.
I've discovered my user site packages directory with:
>>> import site
>>> site.getusersitepackages()
'C:\\Users\\my name\\AppData\\Roaming\\Python\\Python35\\site-packages'
I haven't been able to get to this directory using any method. I've tried using pushd instead of cd to emulate a network drive, and I've also tried getting there using run. No matter what I do in python, or in cmd terminal. I get the response The network path was not found.
Here is an example of one I've tried in cmd:
C:\>pushd \\Users\\my name\\AppData\\Roaming\\Python\\Python35\\site-packages
The network path was not found.
What am I doing wrong, or what could be wrong with the path?
DOS style backslashes don't need to be escaped within the Windows console (else they may have used forward slashes way back when!).
Follow these steps to manually create usercustomize.py:
Start->Run:cmd
Make sure you're on the C: drive
c:
Create the directory. mkdir creates the missing parents. Obviously, change "my name" as appropriate.
mkdir C:\Users\my name\AppData\Roaming\Python\Python35\site-packages
Create usercustomize.py:
notepad C:\Users\my name\AppData\Roaming\Python\Python35\site-packages\usercustomize.py
Click "yes" to create your file.
Edit as appropriate
Or use the following script to have Python do it for you:
import site
import os
import os.path
import io
user_site_dir = site.getusersitepackages()
user_customize_filename = os.path.join(user_site_dir, 'usercustomize.py')
win_unicode_console_text = u"""
# win_unicode_console
import win_unicode_console
win_unicode_console.enable()
"""
if os.path.exists(user_site_dir):
print("User site dir already exists")
else:
print("Creating site dir")
os.makedirs(user_site_dir)
if not os.path.exists(user_customize_filename):
print("Creating {filename}".format(filename=user_customize_filename))
file_mode = 'w+t'
else:
print("{filename} already exists".format(filename=user_customize_filename))
file_mode = 'r+t'
with io.open(user_customize_filename, file_mode) as user_customize_file:
existing_text = user_customize_file.read()
if not win_unicode_console_text in existing_text:
# file pointer should already be at the end of the file after read()
user_customize_file.write(win_unicode_console_text)
print("win_unicode_console added to {filename}".format(filename=user_customize_filename))
else:
print("win_unicode_console already enabled")
I'm using fabric to connect to remote host, when i'm there, I try to call a script that I made (It parses the file I give in argument). But when I call the script from inside my Fabfile.py, it assumes the path I gave is from the machine I launch the fabfile from (so not my remote host)
In my fabfile.py I have:
Import import servclasse
env.host='host1'
def listconf():
#here I browes to the correct folder
s=servclasse.Server("my.file") #this is where I want it to open the host1:my.file file and instanciate a classe from what it parsed
If i do this, it tries to open the file from the folder where servclass.py is. Is there a way to give a "remote path" in argument? I would rather not downloading the file.
Should I upload the script servclasse.py with the operation.put before calling it?
Edit: more info
In my servclasse I have this:
def __init__(self, path):
self.config = ConfigParser.ConfigParser(allow_no_value=True)
self.config.readfp(open(path))
The function open() was the problem.
I figured out how to do it so i'll drop it here in case someone read this topic one day :
def listconf():
#first I browes to the correct folder then
contents = StringIO.StringIO()
get("MyFile",contents)
contents.seek(0)
s=Server(contents)
and in the servclass.py
def __init__(self, objfile):
self.config = ConfigParser.ConfigParser(allow_no_value=True)
self.config.readfp(objfile)
#and i do my stuffs