I'm getting an error when I try to load one of the configuration files. Here is the folder structure:
script.rb
root_folder
command_line_interface.py
extras
utils.py
AZ
config1.txt
config2.txt
BZ
config1.txt
config2.txt
The query folows: script.rb > command_line_interface.py > utils.py > config2.txt
In script.rb:
IO.popen("./root_folder/command_line_interface.py -arg1 5")
In command_line_interface.py:
if __name__ == '__main__':
sys.path.append(os.path.dirname(sys.argv[0]))
parser = argparse.ArgumentParser(description='Test')
parser.add_argument('-arg1', '--arg1', action='store', type=int, help='Test', dest='arg1')
args = parser.parse_args()
if args.arg1:
utils.run(action)
In utils.py dinamically load config.txt according to action parameter using base_path:
def run(action):
base_path = 'extras/BZ/'
.
.
.
filename = base_path + 'config2.txt'
with open(filename) as f:
result = json.load(f)
IOError: [Errno 2] No such file or directory: '\extras\BZ\config2.txt'
It works when called directly in windows prompt inside root_folder, but it does't when called from script.rb. It needs to run on both cases.
Does anyone knows what is causing the error?
your ruby script isn't started inside the root_folder, so your relative base path isn't correct. You can use os.getcwd() to fetch your current working directory. A relative path always starts from there.
In general it is better to always use absolute paths, when possible. For creating absolute pathes, it is good practice to use os.path.join() instead of just concatenating strings.
Remember that if your run script.rb from its folder then ALL OF THE CODE gets run as if it were in that folder.
And so I think you should try setting base_path to './root_folder/extras/BZ/' just to check if that's the problem.
The solution was:
def run(action):
base_path = os.path.join(os.path.dirname(sys.argv[0]), 'extras/BZ/')
Related
So this is a question about how to handle settings files and relative paths in python (probably also something about best practice).
So I have coded a smaller project that i want to deploy to a docker image and everything is set up now except when I try to run the python task (Through cron) I get the error: settings/settings.yml not found.
tree .
├───settings
│ └───settings/settings.yml
└───main.py
And am referencing the yml file as
open('settings/settings.yml', 'r') as f:
config = yaml.load(f, Loader=yaml.FullLoader)
I can see this is what is causing the problem but am unsure about how to fix it. I wish to reference the main file basically by using the entry_points from setuptools in the future so my quick fix with cd'ing before python main.py will not be a lasting solution.
Instead of hardcoding a path as a string, you can find the directories and build the file path with os.path. For example:
import os
import yaml
current_dir = os.path.dirname(os.path.abspath(__file__))
settings_dir = os.path.join(current_dir, "settings")
filename = "settings.yml"
settings_path = os.path.join(settings_dir, filename)
with open(settings_path, "r") as infile:
settings_data = yaml.load(infile)
This way it can be run in any file system and the python file can be called from any directory.
My file structure is as follows:
kkg/
builder/test.py
builder/data
api/api.py
__init__py
'kkg' is my package name, and in init.py some function are defined, and implementations of these function are written api.py.
In test.py, I have:
import kkg
kkg.load('builder/data/')
Inside the 'load' of the api.py, I have code:
abspath = os.path.abspath(os.path.dirname(__file__))
...
for file in files:
file_path = os.path.join(abspath, data_path)
The data_path is the parameter 'builder/data/' passed from test.py. The path.join reports an error:
Caused by: java.io.FileNotFoundException: /Users/comin/kkg/kkg/api/data/people.csv
The correct data path, if parsed properly, should be:
/Users/comin/kkg/kkg/data/people.csv
I run the 'test.py' inside the builder/ directory. I think the reason there is an unnecessary 'api' in the path generated is because the code piece where error occurs in the api.py.
Perhaps I shouldn't use the join(abspath, data_path) to get the absolute directory. How to get the path correctly parsed?
EDIT:
I changed the path parameter:
kkg.load('../builder/data/')
but then this code failed:
if not os.path.isdir(data_path):
raise ValueError('data_path must be a directory, not file name!')
Why does it raise an error when I added '..' to the path? It is not considered as a directory due to the '..'?
you want the parent directory I think
abspath = os.path.abspath(os.path.join(os.path.dirname(__file__),".."))
you seem to be struggling with this more than you should...it might be easier to just do
path_to_data_file_folder = os.path.abspath(sys.argv[1])
then call it with python main.py /path/to/data/folder
It is much easier to use pathlib over os library:
# api/api.py
from pathlib import Path
KKG_FOLDER = Path(__file__).parent.parent
DATA_FOLDER = KKG_FOLDER / 'builder/data'
print(DATA_FOLDER)
This is more verbose and easier to understand.
I am working on a project on path D:/code/project.
project.py
import os
import argparse
def create(path):
if os.path.exists(path):
os.chdir(path)
app = open("app.py", "w+")
app.write("print(\"Hello world!\")")
else:
path_list = path.split("/")
for i in path_list:
try:
os.mkdir(i)
except FileExistsError:
pass
except FileNotFoundError:
print("Invalid path")
exit()
os.chdir(i)
app = open("app.py", "w+")
app.write("print(\"Hello world!\")")
if __name__ == "__main__":
parser = argparse.ArgumentParser()
create_parser = parser.add_subparsers().add_parser("create")
create_parser.add_argument("path", nargs="?", default=".", metavar="path", type=str)
args = parser.parse_args()
create(vars(args)["path"])
Basically, it has a custom command called create which takes path as an argument. When it detects that path already exists, it will create a app.py at the directory specified, and if it does not, it will try and create the path and app.py.
However, when I run
D:/code/project> python project.py create D:/newpath
Instead of creating a new directory newpath under D:, it creates newpath under my current directory (D:/code/project).
How do I change it such that changing directory to a root directory will actually switch correctly?
Your issue is with this line:
path_list = path.split("/")
On windows, that doesn't correctly split the path. It leaves the drive letter 'D:' as a fragment all by itself, but when you try to change directory to that path, it does nothing (assuming the current directory was on D: somewhere). It is the same behavior you get with the cd command on a command prompt. Drive letters in paths are pretty weird, they don't work like normal path prefixes.
To correctly split the path, use os.path.split(path). It will correctly use 'D:/' as the first element of the path, which when you change to it will put you on the root folder of your drive.
If you're using a recent version of Python (3.4+), you could also try using the pathlib module for a better, more object oriented way of manipulating paths:
from pathlib import Path
def create(path_str):
path = Path(path_str)
path.mkdir(parents=True, exist_ok=True) # creates parent folders as needed
with open(path / 'app.py', 'w') as app: # / operator joins path fragments
app.write('print("Hello world!")')
couple of days back an infected computer infected my USB drive with a shortcut virus and. I had lots of software in that USB drive i plugged it in my Linux machine and cleaned a lot of files but what the virus did is it created an .exe file in each folder with that folder's name. Each .exe file is capable of infecting another PC. So me with a lot of time in my hand was trying to make a python script that goes to each folder check if a .exe file with that folder name exists delete it and run the function for every sub-folder recursively. but it is not working here what i made so far
#!/usr/bin/python
from __future__ import print_function
import os
import argparse
def deldir(fol):
# changing to the directory
os.chdir(fol)
#Trivial case delte the virus in the root folder
cwd = os.getcwd()
#getting just the name of the folder rather than full path by splitting from "/"
cwd = str(cwd).split('/')[-1]
#if any .exe file exists with that name
if any(str(cwd + '.exe') in s for s in os.listdir('.')):
# delete that file
os.remove(str(cwd + '.exe'))
print('Deleting virus file')
#making a list with all the contents in the directory
dirlist = os.listdir('.')
print(dirlist)
#Looping through the directory that we just made a list of its contents
for i in dirlist:
#checking if it is a directory or not
if os.path.isdir(i):
#changing dir to that directory
os.chdir(i)
#getting the current working directory
cwd = os.getcwd()
#getting just the name of the folder rather than full path by splitting from "/"
cwd = str(cwd).split('/')[-1]
#if any .exe file exists with that name
if any(str(cwd + '.exe') in s for s in os.listdir('.')):
# delete that file
os.remove(str(cwd + '.exe'))
print('Deleting virus file', str(cwd + '.exe'))
#listing filders in current directory
for j in os.listdir('.'):
#passing that subdirectory to the function itself if its a folder
if os.path.isdir(j):
deldir(j)
else:
print('Not a folder so skipping')
def main():
#parsing the command line arguments
parser = argparse.ArgumentParser()
parser.add_argument("fol", help="Directory to enter")
args = parser.parse_args()
#passing the command line argument with the folder path to "deldir" function
deldir(args.fol)
if __name__ == "__main__":
main()
Please point me to right direction i have no idea what i am doing wrong. Thanks for reading
Here what I see mistake in your code.
1
os.getcwd -> os.getcwd()
It's a function
2.
You using os.chdir(i) inside the for loop. this will work only first directory in that directory.
3.
The algorithm is wrong, It's not checking the directories level 1,3,5,... for virus.
As Hiro and MYGz's comments. os.walk/os.scandir is easier way to go for recursive directory. but your way is better for practice and learn programming.
I got it done the way i wanted with os.walk() it's really easy that way and that takes away the fun ! :'( . I was hoping to fix my own code but somethings not right. well anyways here's the working code if anyone encounters this type of virus or something.
#!/usr/bin/python
from __future__ import print_function
import os
import argparse
def deldir(fol):
for root, dirs, files in os.walk(fol):
try:
os.remove(str(root + '/' + str(root).split('/')[-1 ] + '.exe'))
except:
print('Does not exists' ,str(root + '/' + str(root).split('/')[-1 ] + '.exe'))
def main():
#parsing the command line arguments
parser = argparse.ArgumentParser()
parser.add_argument("fol", help="Directory to enter")
args = parser.parse_args()
#passing the command line argument with the folder path to "deldir" function
deldir(args.fol)
if __name__ == "__main__":
main()
If you are running a frozen python script (frozen using py2exe) from a directory and drive different from where the script is present, what is the best way to determine the path of the executing script?
Few solutions I have tried
inspect.getfile(inspect.currentframe())
Problem: Does not return the full path. It only returns the script name.
os.path.abspath( __file__ )
Problem: Doesn't work on Windows
os.path.dirname(sys.argv[0])
Problem: Returns empty string.
os.path.abspath(inspect.getsourcefile(way3))
Will not work if the drive is different from the pwd
os.path.dirname(os.path.realpath(sys.argv[0]))
Will not work if the drive is different from the pwd
Here is a minimal not-working example
D:\>path
PATH=c:\Python27\;c:\Users\abhibhat\Desktop\ToBeRemoved\spam\dist\;c:\gnuwin32\bin
D:\>cat c:\Users\abhibhat\Desktop\ToBeRemoved\spam\eggs.py
import os, inspect, sys
def way1():
return os.path.dirname(sys.argv[0])
def way2():
return inspect.getfile(inspect.currentframe())
def way3():
return os.path.dirname(os.path.realpath(sys.argv[0]))
def way4():
try:
return os.path.abspath( __file__ )
except NameError:
return "Not Found"
def way5():
return os.path.abspath(inspect.getsourcefile(way3))
if __name__ == '__main__':
print "Path to this script is",way1()
print "Path to this script is",way2()
print "Path to this script is",way3()
print "Path to this script is",way4()
print "Path to this script is",way5()
D:\>eggs
Path to this script is
Path to this script is eggs.py
Path to this script is D:\
Path to this script is Not Found
Related Questions:
How to know the path of the running script in Python?
How do I get the path and name of the file that is currently executing?
python, path of script [closed]
Note
#Fenikso's solution will work if the script resides on the same drive where you are executing but when its on a different drive, it will not work
Another approach which works with cxFreeze when running from another drive even using PATH:
import sys
if hasattr(sys, 'frozen'):
print(sys.executable)
else:
print(sys.argv[0])
From Python:
H:\Python\Examples\cxfreeze\pwdme.py
From command line:
D:\>h:\Python\Examples\cxfreeze\dist\pwdme.exe
h:\Python\Examples\cxfreeze\dist\pwdme.exe
Using PATH:
D:\>pwdme.exe
h:\Python\Examples\cxfreeze\dist\pwdme.exe
IMHO, code that acts differently depending from absolute paths is not a good solution.
It will be probably better a relative path solution. Use dirname to know the relative directory and os.sep for cross platform compatibility.
if hasattr(sys, "frozen"):
main_dir = os.path.dirname(sys.executable)
full_real_path = os.path.realpath(sys.executable)
else:
script_dir = os.path.dirname(__file__)
main_dir = os.path.dirname(os.path.realpath(sys.argv[0]))
full_real_path = os.path.realpath(sys.argv[0])
the frozen attribute is python standard.
Take a look also at Esky :
http://pypi.python.org/pypi/esky
Try this:
WD = os.path.dirname(os.path.realpath(sys.argv[0]))
That is what I use with cx_Freeze to get the directory from where the .exe is really running.