Develop Reference

python yaml path after deployment

So this is a question about how to handle settings files and relative paths in python (probably also something about best practice).
So I have coded a smaller project that i want to deploy to a docker image and everything is set up now except when I try to run the python task (Through cron) I get the error: settings/settings.yml not found.
tree .
├───settings
│ └───settings/settings.yml
└───main.py
And am referencing the yml file as
open('settings/settings.yml', 'r') as f:
config = yaml.load(f, Loader=yaml.FullLoader)
I can see this is what is causing the problem but am unsure about how to fix it. I wish to reference the main file basically by using the entry_points from setuptools in the future so my quick fix with cd'ing before python main.py will not be a lasting solution.

Instead of hardcoding a path as a string, you can find the directories and build the file path with os.path. For example:
import os
import yaml
current_dir = os.path.dirname(os.path.abspath(__file__))
settings_dir = os.path.join(current_dir, "settings")
filename = "settings.yml"
settings_path = os.path.join(settings_dir, filename)
with open(settings_path, "r") as infile:
settings_data = yaml.load(infile)
This way it can be run in any file system and the python file can be called from any directory.

Compiled exe needs to be run as admin but uncompiled py file doesn't?

I recently compiled a python project I have been working on for some time to an exe, only to find you'd need to run it as an administrator for it to work, despite it not requiring any elevation while it was simply a .py file. It returns an internal error "cannot create temporary directory!" if you don't run it with elevation. I presume this has to do something with the program reading/writing to the users appdata/locallow folder (it stores some of it's files in there). Is there a simple fix to this? I'd much prefer it if the apps files could remain in one of the appdata folders whether it be local, locallow or roaming.
This can be reproduced by following these steps:
create a main.py file with the following code inside:
import os, json
LOG_DIR = os.path.expanduser('~\AppData\Locallow\AppName\logs')
LOG_PATH = os.path.expanduser('~\AppData\Locallow\AppName\logs\log.txt')
PREFS_DIR = os.path.expanduser('~\AppData\Locallow\AppName\prefs')
PREFS_PATH = os.path.expanduser('~\AppData\Locallow\AppName\prefs\prefs.json')
user_prefs = None
def log_init():
if os.path.isfile(LOG_PATH):
with open(LOG_PATH, 'a') as f:
f.close()
else:
if not os.path.isdir(LOG_DIR):
os.mkdir(LOG_DIR)
with open(LOG_PATH, 'w') as f:
f.close()
def prefs_init():
global user_prefs
if os.path.isfile(PREFS_PATH):
user_prefs = json.load(open(PREFS_PATH, 'r'))
else:
if not os.path.isdir(PREFS_DIR):
os.mkdir(PREFS_DIR)
with open(PREFS_PATH, 'w') as f:
f.write('{"welc_msg": "DirNav - The all in one file browser and terminal!", "file_color": {}}')
f.close()
user_prefs = json.load(open(PREFS_PATH, 'r'))
log_init()
prefs_init()
Make the following folder: C:/Users/YOUR_USER/AppData/Locallow/AppName
Running main.py will generate 'prefs' and 'logs' with respective files inside.
Delete 'prefs' and 'logs'
Compile main.py using this command: pyinstaller -F main.py
Running the new .exe that is in the 'dist' folder from cmd will result in an 'INTERNAL ERROR: cannot create temporary directory!' message.
Run the .exe again but this time run as an administrator and it will work perfectly fine
The main premise here is why does the executable need to be run as an admin yet the .py file doesn't? And how do I make it so the executable doesn't need to be run as an admin?

How to open a .txt file in Flask? [duplicate]

This question already has an answer here:
Refering to a directory in a Flask app doesn't work unless the path is absolute
(1 answer)
Closed 5 years ago.
I'm trying to build a website using the Flask framework for Python.
I'm on a Linux Ubuntu server, with Apache2.
On my website, whenever someone enters the URL "/Elv_1.html", I want to open a .txt file, get some values and create a graph using pygal. Here is my code:
#app.route('/river_1.html')
def riv_1():
try:
document = open('temp.txt','r')
temp_list = []
for n in document:
n = n.rstrip('\n')
n = int(n)
temp_list.append(n)
document.close()
graf = pygal.Line(title=u'Tempt last 24h')
graf.x_labels = (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24)
graf.add('Temp', temp_list)
graf = graf.render_data_uri()
return render_template('river_1.html', graf=graf)
except Exception, e:
return str(e)
if __name__ == '__main__':
app.run(debug=True)
The file 'temp.txt' is located in the same directory as the __init__.py file. __init__.py is the Flask app that the code comes from.
When I do this on my computer using localhost to run the server, it works just fine. However, when I upload this to my Linux server and try to enter that specific URL, it shows the following error:
[Error 2] No such file or directory: 'temp.txt'
Any suggestions as to why it doesn't appear to find the file?

Try using the os module when specifying the path to your file. I am asuming you are using a windows pc when runing on localhost?
import os
document_path = os.getcwd()+'temp.txt'
document = open(documnet_path, 'r')

Make sure you are running the server from it's directory. So if you have this structure as shown below, you can't simply open terminal and type server/__init__.py, because you are in your home directory (/home/username/). You need to cd to server and there run ./__init__.py
/home/
username/
server/
__init__.py
temp.txt
Or, if you want to run it from somewhere else, run open the file from os.path.abspath(os.path.dirname(__file__)) + '/temp.txt') (Tested with python 3.5.2)
See python docs for os.path.

Python unable to find file in folder tree

I'm getting an error when I try to load one of the configuration files. Here is the folder structure:
script.rb
root_folder
command_line_interface.py
extras
utils.py
AZ
config1.txt
config2.txt
BZ
config1.txt
config2.txt
The query folows: script.rb > command_line_interface.py > utils.py > config2.txt
In script.rb:
IO.popen("./root_folder/command_line_interface.py -arg1 5")
In command_line_interface.py:
if __name__ == '__main__':
sys.path.append(os.path.dirname(sys.argv[0]))
parser = argparse.ArgumentParser(description='Test')
parser.add_argument('-arg1', '--arg1', action='store', type=int, help='Test', dest='arg1')
args = parser.parse_args()
if args.arg1:
utils.run(action)
In utils.py dinamically load config.txt according to action parameter using base_path:
def run(action):
base_path = 'extras/BZ/'
.
.
.
filename = base_path + 'config2.txt'
with open(filename) as f:
result = json.load(f)
IOError: [Errno 2] No such file or directory: '\extras\BZ\config2.txt'
It works when called directly in windows prompt inside root_folder, but it does't when called from script.rb. It needs to run on both cases.
Does anyone knows what is causing the error?

your ruby script isn't started inside the root_folder, so your relative base path isn't correct. You can use os.getcwd() to fetch your current working directory. A relative path always starts from there.
In general it is better to always use absolute paths, when possible. For creating absolute pathes, it is good practice to use os.path.join() instead of just concatenating strings.

Remember that if your run script.rb from its folder then ALL OF THE CODE gets run as if it were in that folder.
And so I think you should try setting base_path to './root_folder/extras/BZ/' just to check if that's the problem.

The solution was:
def run(action):
base_path = os.path.join(os.path.dirname(sys.argv[0]), 'extras/BZ/')

Python's docx module giving AsserionError when .exe is created

I've a Python file titled my_python_file.py that makes, among other things, a .doc file using the python-docx module. The .doc is created perfectly and gives no problem. The problem comes when I build a .exe of my script and I try to make the .doc. An AssertionError problem appears.
This is my exe maker code (exe_maker.py):
from distutils.core import setup
import py2exe, sys, os
sys.argv.append('py2exe')
setup(
options = {'py2exe': {'bundle_files': 3, 'compressed': True, 'includes': ['lxml.etree', 'lxml._elementpath', 'gzip', 'docx']}},
windows = [{'script': "my_python_file.py"}],
zipfile = None,
)
It seems that moving the python script to a different location produces the error.
File "docx.pyc", line 1063, in savedocx
AssertionError
This is the savedocx line:
document = newdocument()
[...]
coreprops = coreproperties(title=title, subject=subject, creator=creator, keywords=keywords)
approps = appproperties()
contenttypes2 = contenttypes()
websettings2 = websettings()
wordrelationships2 = wordrelationships(relationships)
path_save = "C:\output"
savedocx(document, coreprops, approps, contenttypes2, websettings2, wordrelationships2, path_save)
The savedox is well writen as it works when it's not an .exe file.
How can I make the docx module work correctly? Do I've to add any other path/variable more when I make the exe?
Thanks in advance

I solved the problem by edditing the api.py file of docx egg folder which is located in the Python folder of the system.
Changing this:
_thisdir = os.path.split(__file__)[0]
_default_docx_path = os.path.join(_thisdir, 'templates', 'default.docx')
To this:
thisdir = os.getcwd()
_default_docx_path = os.path.join(thisdir, 'templates', 'default.docx')
The first one was taking the actual running program and adding it to the path to locate the templates folder.
C:\myfiles\myprogram.exe\templates\default.docx
The solution takes only the path, not the running program.
C:\myfiles\templates\default.docx

Instead of changing some library file, I find it easier and cleaner to tell python-docx explicitly where to look for the template, i.e.:
document = Document('whatever/path/you/choose/to/some.docx')
This effectively solves the py2exe and docx path problem.