Python erronously adding file extension to path string - python

I'm currently struggling with the following problem:
My folder structer looks like:
master
- resources
customFile.fmu
fileCallingFMU.py
While executing the fileCallingFMU.py I pass a path string like
path = "./resources/customFile.fmu"
My script contains a super function where I pass the path variable. But everytime I execute the script it trips with an exception:
Exception has occurred: FileNotFoundError
[Errno 2] No such file or directory: b'2021-11-16_./resources/customFile.fmu.txt'
File "[projectFolder]\fileCallingFMU.py", line 219, in __init__
super().__init__(path, config, log_level)
File "[projectFolder]\fileCallingFMU.py", line 86, in <module>
env = gym.make(env_name)
My urging question now is the following:
Why and how does python manipulate the path variable with a date-prefix and .txt as file extension?!
Hope anyone can enlighten me on this one...
EDIT
I'm trying to get the example of ModelicaGym running.
My fileCallingFMU.py contains the following code:
path = "./resources/customFile.fmu"
env_entry_point = 'cart_pole_env:JModelicaCSCartPoleEnv'
config = {
'path': path,
'm_cart': m_cart,
'm_pole': m_pole,
'theta_0': theta_0,
'theta_dot_0': theta_dot_0,
'time_step': time_step,
'positive_reward': positive_reward,
'negative_reward': negative_reward,
'force': force,
'log_level': log_level
}
from gym.envs.registration import register
env_name = env_name
register(
id=env_name,
entry_point=env_entry_point,
kwargs=config
)
env = gym.make(env_name)
The full code for the entryPoint can be found here.

As jjramsey pointed out the problem was burried within the ModelicaGym library.
The logger could not create a propper log file, because the model name was not properly stored in the self.model variable.
Source of this error lied in the line
self.model_name = model_path.split(os.path.sep)[-1]
due to the fact that the os library was not able to separate my path string
"./resources/customFile.fmu"
After changing it to
".\\resources\\customFile.fmu"
everythig works as expected.
Thanks again!

Related

Problems connecting to Brownie/Rinkeby

I'm pretty new to this programming and maybe need more practice but I've been having problems trying to connect to the Rinkeby testnet and can't seem to see the problem (Windows 10, Powershell) not sure if it's the .env or how I set them up but I'll send the code or the respective files created with screenshots to see I someone could illuminate the problem. Copy pasted the code and some screenshots of the environment variables. Your help is very much appreciated. For some reason the export commands in the .env are seems as they are not executing(Third attachment).
Thank you
Error:
Brownie v1.18.1 - Python development framework for Ethereum
BrownieSimpleStorageProject is the active project.
Running 'scripts\deploy.py::main'...
File "C:\Users\jorge\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\brownie\_cli\run.py", line 51, in main
return_value, frame = run(
File "C:\Users\jorge\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\brownie\project\scripts.py", line 103, in run
return_value = f_locals[method_name](*args, **kwargs)
File ".\scripts\deploy.py", line 25, in main
deploy_simple_storage()
File ".\scripts\deploy.py", line 5, in deploy_simple_storage
account = get_account()
File ".\scripts\deploy.py", line 21, in get_account
return accounts.add(config["wallets"]["from_key"])
KeyError: 'wallets'
deploy.py:
from brownie import accounts, config, SimpleStorage, network
def deploy_simple_storage():
account = get_account()
simple_storage = SimpleStorage.deploy({"from": account})
# Transaction
# Call
stored_value = simple_storage.retrieve()
print(stored_value)
transaction = simple_storage.store(15, {"from": account})
transaction.wait(1)
updated_value = simple_storage.retrieve()
print(updated_value)
def get_account():
if network.show_active() == "development":
return accounts[0]
else:
return accounts.add(config["wallets"]["from_key"])
def main():
deploy_simple_storage()
Here's the .env file:
export WEB3_INFURA_PROJECT_ID=xxx
export PRIVATE_KEY=xxxx
brownie-config.yaml file:
dotenv: .env
wallets:
from_key: ${PRIVATE_KEY}
Enviroment variable
Environment variable
.env file export
config reads from brownie-config.yaml file which is in the root directory. make sure the location of the file is in root and its name is correct in the project.

Cannot load azure.ml workspace from within a webservice entry_script! Where is the `/var/azureml-app/` located?

I am creating an azure-ml webservice. The following script shows the code for creating the webservice and deploying it locally.
from azureml.core.model import InferenceConfig
from azureml.core.environment import Environment
from azureml.core import Workspace
from azureml.core.model import Model
ws = Workspace.from_config()
model = Model(ws,'textDNN-20News')
ws.write_config(file_name='config.json')
env = Environment(name="init-env")
python_packages = ['numpy', 'pandas']
for package in python_packages:
env.python.conda_dependencies.add_pip_package(package)
dummy_inference_config = InferenceConfig(
environment=env,
source_directory="./source_dir",
entry_script="./init_score.py",
)
from azureml.core.webservice import LocalWebservice
deployment_config = LocalWebservice.deploy_configuration(port=6789)
service = Model.deploy(
ws,
"myservice",
[model],
dummy_inference_config,
deployment_config,
overwrite=True,
)
service.wait_for_deployment(show_output=True)
As it can be seen, the above code deploys "entry_script = init_score.py" to my local machine. Within the entry_script, I need to load the workspace again to connect to azure SQL database. I do it like the following :
from azureml.core import Dataset, Datastore
from azureml.data.datapath import DataPath
from azureml.core import Workspace
def init():
pass
def run(data):
try:
ws = Workspace.from_config()
# create tabular dataset from a SQL database in datastore
datastore = Datastore.get(ws, 'sql_db_name')
query = DataPath(datastore, 'SELECT * FROM my_table')
tabular = Dataset.Tabular.from_sql_query(query, query_timeout=10)
df = tabular.to_pandas_dataframe()
return len(df)
except Exception as e:
output0 = "{}:".format(type(e).__name__)
output1 = "{} ".format(e)
output2 = f"{type(e).__name__} occured at line {e.__traceback__.tb_lineno} of {__file__}"
return output0 + output1 + output2
The try-catch block is for catching the potential exception thrown and return it as an output.
The exception that I keep getting is:
UserErrorException: The workspace configuration file config.json, could not be found in /var/azureml-app or its
parent directories. Please check whether the workspace configuration file exists, or provide the full path
to the configuration file as an argument. You can download a configuration file for your workspace,
via http://ml.azure.com and clicking on the name of your workspace in the right top.
I have actually tried to save the config file by passing an absolute path to the path argument of both ws.write_config(path='my_absolute_path'), and also when loading it to the Workspace.from_config(path='my_absolute_path'), but I got pretty much the same error:
UserErrorException: The workspace configuration file config.json, could not be found in /var/azureml-app/my_absolute_path or its
parent directories. Please check whether the workspace configuration file exists, or provide the full path
to the configuration file as an argument. You can download a configuration file for your workspace,
via http://ml.azure.com and clicking on the name of your workspace in the right top.
Looks like even providing the path does not change the root directory that the entry script starts locating from.
I also tried to directly saving the file to /var/azureml-app/, but this path is not recognized when I passed it to the ws.write_config(path='/var/azureml-app/').
Do you have any idea where exactly is the /var/azureml-app/?
Any idea on how to fix this?

Python in ArcGIS

I wrote the following code, which results in an error and I don't know how to fix it to work.
The code is:
# Name: ClipGDBtoNewGDB.py
# Description: Take an input GDB, create a list, iterate through each
feature class, clipping it and writing it to a new GDB.
# Author: tuilbox
# Import system modules
import arcpy, os
from arcpy import env
# Set workspace
env.workspace = arcpy.GetParameterAsText(0)
arcpy.env.overwriteOutput=True
# Set local variables
fclist = arcpy.ListFeatureClasses()
clip_features = arcpy.GetParameterAsText(1)
output_directory=arcpy.GetParameterAsText(2)
xy_tolerance = ""
outgdb=os.path.join(output_directory, arcpy.GetParameterAsText(3))
if not arcpy.Exists(outgdb):
arcpy.CreateFileGDB_management(output_directory,
arcpy.GetParameterAsText(3))
# Execute Clip within for loop
for fc in fclist:
arcpy.Clip_analysis(fc, clip_features, os.path.join(outgdb, fc))
The error is: Traceback (most recent call last):
File "F:/GIS_Joseph/Lab10_Joseph/ClipGDBtoNewGDB.py", line 17, in <module>
arcpy.CreateFileGDB_management(output_directory, arcpy.GetParameterAsText(3))
File "C:\Program Files (x86)\ArcGIS\Desktop10.5\ArcPy\arcpy\management.py", line 18878, in CreateFileGDB
raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000735: File GDB Location: Value is required
ERROR 000735: File GDB Name: Value is required
Failed to execute (CreateFileGDB).
Any help would be appreciated. Thank you.
With this type of question it would be helpful to let us know what parameters you are passing into your script. Have you passed a valid parameter in position 3? Use arcpy.AddMessage to double check what value you are attempting to pass to arcpy.CreateFileGDB_management.

Python fabric calling script "remote path"

I'm using fabric to connect to remote host, when i'm there, I try to call a script that I made (It parses the file I give in argument). But when I call the script from inside my Fabfile.py, it assumes the path I gave is from the machine I launch the fabfile from (so not my remote host)
In my fabfile.py I have:
Import import servclasse
env.host='host1'
def listconf():
#here I browes to the correct folder
s=servclasse.Server("my.file") #this is where I want it to open the host1:my.file file and instanciate a classe from what it parsed
If i do this, it tries to open the file from the folder where servclass.py is. Is there a way to give a "remote path" in argument? I would rather not downloading the file.
Should I upload the script servclasse.py with the operation.put before calling it?
Edit: more info
In my servclasse I have this:
def __init__(self, path):
self.config = ConfigParser.ConfigParser(allow_no_value=True)
self.config.readfp(open(path))
The function open() was the problem.
I figured out how to do it so i'll drop it here in case someone read this topic one day :
def listconf():
#first I browes to the correct folder then
contents = StringIO.StringIO()
get("MyFile",contents)
contents.seek(0)
s=Server(contents)
and in the servclass.py
def __init__(self, objfile):
self.config = ConfigParser.ConfigParser(allow_no_value=True)
self.config.readfp(objfile)
#and i do my stuffs

py2exe/py2app and docx don't work together

Installed docx on Windows 7 here:
D:\Program Files (x86)\Python27\Lib\site-packages as shown below:
Installed docx on OS X at /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/docx-0.0.2-py2.7.egg-info as shown below:
Following is the sample script (named as docx_example.py), which runs absolutely fine on the python interpreter:
#!/usr/bin/env python
'''
This file makes an docx (Office 2007) file from scratch, showing off most of python-docx's features.
If you need to make documents from scratch, use this file as a basis for your work.
Part of Python's docx module - http://github.com/mikemaccana/python-docx
See LICENSE for licensing information.
'''
from docx import *
if __name__ == '__main__':
# Default set of relationshipships - these are the minimum components of a document
relationships = relationshiplist()
# Make a new document tree - this is the main part of a Word document
document = newdocument()
# This xpath location is where most interesting content lives
docbody = document.xpath('/w:document/w:body', namespaces=nsprefixes)[0]
# Append two headings and a paragraph
docbody.append(heading('''Welcome to Python's docx module''',1) )
docbody.append(heading('Make and edit docx in 200 lines of pure Python',2))
docbody.append(paragraph('The module was created when I was looking for a Python support for MS Word .doc files on PyPI and Stackoverflow. Unfortunately, the only solutions I could find used:'))
# Add a numbered list
for point in ['''COM automation''','''.net or Java''','''Automating OpenOffice or MS Office''']:
docbody.append(paragraph(point,style='ListNumber'))
docbody.append(paragraph('''For those of us who prefer something simpler, I made docx.'''))
docbody.append(heading('Making documents',2))
docbody.append(paragraph('''The docx module has the following features:'''))
# Add some bullets
for point in ['Paragraphs','Bullets','Numbered lists','Multiple levels of headings','Tables','Document Properties']:
docbody.append(paragraph(point,style='ListBullet'))
docbody.append(paragraph('Tables are just lists of lists, like this:'))
# Append a table
docbody.append(table([['A1','A2','A3'],['B1','B2','B3'],['C1','C2','C3']]))
docbody.append(heading('Editing documents',2))
docbody.append(paragraph('Thanks to the awesomeness of the lxml module, we can:'))
for point in ['Search and replace','Extract plain text of document','Add and delete items anywhere within the document']:
docbody.append(paragraph(point,style='ListBullet'))
# Search and replace
print 'Searching for something in a paragraph ...',
if search(docbody, 'the awesomeness'): print 'found it!'
else: print 'nope.'
print 'Searching for something in a heading ...',
if search(docbody, '200 lines'): print 'found it!'
else: print 'nope.'
print 'Replacing ...',
docbody = replace(docbody,'the awesomeness','the goshdarned awesomeness')
print 'done.'
# Add a pagebreak
docbody.append(pagebreak(type='page', orient='portrait'))
docbody.append(heading('Ideas? Questions? Want to contribute?',2))
docbody.append(paragraph('''Email <python.docx#librelist.com>'''))
# Create our properties, contenttypes, and other support files
coreprops = coreproperties(title='Python docx demo',subject='A practical example of making docx from Python',creator='Mike MacCana',keywords=['python','Office Open XML','Word'])
appprops = appproperties()
contenttypes = contenttypes()
websettings = websettings()
wordrelationships = wordrelationships(relationships)
# Save our document
savedocx(document,coreprops,appprops,contenttypes,websettings,wordrelationships,'docx_example.docx')
Following is the setup script (named as docx_setup.py) to create the standalone (.app in Mac OSX and .exe in Windows 7):
import sys,os
# Globals: START
main_script='docx_example'
dist_dir_main_path=os.path.abspath('./docx-bin')
compression_level=2
optimization_level=2
bundle_parameter=1
skip_archive_parameter=False
emulation_parameter=False
module_cross_reference_parameter=False
ascii_parameter=False
includes_list=['lxml.etree','lxml._elementpath','gzip']
# Globals: STOP
# Global Functions: START
def isDarwin():
return sys.platform=='darwin'
def isLinux():
return sys.platform=='linux2'
def isWindows():
return os.name=='nt'
# Global Functions: STOP
if isDarwin():
from setuptools import setup
# Setup distribution directory: START
dist_dir=os.path.abspath('%s/osx' %(dist_dir_main_path))
if os.path.exists(dist_dir):
os.system('rm -rf %s' %(dist_dir))
os.system('mkdir -p %s' %(dist_dir))
# Setup distribution directory: STOP
APP = ['%s.py' %(main_script)]
OPTIONS={'argv_emulation': False,
'dist_dir': dist_dir,
'includes': includes_list
}
print 'Creating standalone now...'
setup(app=APP,options={'py2app': OPTIONS},setup_requires=['py2app'])
os.system('rm -rf build')
os.system('tar -C %s -czf %s/%s.tgz %s.app' %(dist_dir,dist_dir,main_script,main_script))
os.system('rm -rf %s/%s.app' %(dist_dir,main_script))
print 'Re-distributable Standalone file(s) created at %s/%s.zip. Unzip and start using!!!' %(dist_dir,main_script)
elif isWindows():
from distutils.core import setup
import py2exe
# Setup distribution directory: START
dist_dir=os.path.abspath('%s/win' %(dist_dir_main_path))
if os.path.exists(dist_dir):
os.system('rmdir /S /Q %s' %(dist_dir))
os.system('mkdir %s' %(dist_dir))
# Setup distribution directory: STOP
OPTIONS={'compressed': compression_level,
'optimize': optimization_level,
'bundle_files': bundle_parameter,
'dist_dir': dist_dir,
'xref': module_cross_reference_parameter,
'skip_archive': skip_archive_parameter,
'ascii': ascii_parameter,
'custom_boot_script': '',
'includes': includes_list
}
print 'Creating standalone now...'
setup(options = {'py2exe': OPTIONS},zipfile = None,windows=[{'script': '%s.py' %(main_script)}])
print 'Re-distributable Standalone file(s) created in the following location: %s' %(dist_dir)
os.system('rmdir /S /Q build')
Now comes the real problem.
Following is the error posted on Mac OS X console after trying to use the docx_example.app, created using the command python docx_setup.py py2app:
docx_example: Searching for something in a paragraph ... found it!
docx_example: Searching for something in a heading ... found it!
docx_example: Replacing ... done.
docx_example: Traceback (most recent call last):
docx_example: File "/Users/admin/docx-bin/osx/docx_example.app/Contents/Resources/__boot__.py", line 64, in <module>
docx_example: _run('docx_example.py')
docx_example: File "/Users/admin/docx-bin/osx/docx_example.app/Contents/Resources/__boot__.py", line 36, in _run
docx_example: execfile(path, globals(), globals())
docx_example: File "/Users/admin/docx-bin/osx/docx_example.app/Contents/Resources/docx_example.py", line 75, in <module>
docx_example: savedocx(document,coreprops,appprops,contenttypes,websettings,wordrelationships,'docx_example.docx')
docx_example: File "docx.pyc", line 849, in savedocx
docx_example: AssertionError
docx_example: docx_example Error
docx_example Exited with code: 255
Following is the error posted in docx_example.exe.log file in Windows 7 after trying to use the docx_example.exe, created using the command python docx_setup.py py2exe:
Traceback (most recent call last):
File "docx_example.py", line 75, in <module>
File "docx.pyo", line 854, in savedocx
WindowsError: [Error 3] The system cannot find the path specified: 'D:\\docx_example\\docx_example.exe\\template'
As you can see, both OS X and Windows 7 are referring to something similar here. Please help.
i have found a solution
in api.py
From
_thisdir = os.path.split(__file__)[0]
To
_thisdir = 'C:\Python27\Lib\site-packages\docx'
Or whatever your docx file is
What's going on (at least for py2exe) is something similar to this question.
The documentation on data_files is here.
What you basically have to do is change
setup(options = {'py2exe': OPTIONS},zipfile = None,windows=[{'script': '%s.py' %(main_script)}])
to
data_files = [
('template', 'D:/Program Files (x86)/Python27/Lib/site-packages/docx-template/*'),
]
setup(
options={'py2exe': OPTIONS},
zipfile=None,
windows=[{'script': '%s.py' %(main_script)}],
data_files=data_files
)
The exact place where the template files are may be wrong above, so you might need to adjust it.
But there may be several other sets of data_files you need to include. You may want to go about retrieving them programatically with an os.listdir or os.walk type of command.
As mentioned in the other post, you will also have to change
bundle_parameter=1
to
bundle_parameter=2
at the top of the file.
You can solve the entire problem by using this API which is based in python-docx. The advantage of the API is that this one doesnt have the savedoc function so you will not have any other AssertionError.
For the WindowsError: [Error 3] The system cannot find the path specified: 'D:\\docx_example\\docx_example.exe\\template' error you need to edit the api.py file of docx egg folder which is located in the Python folder of the system (in my computer: C:\Python27\Lib\site-packages\python_docx-0.3.0a5-py2.7.egg\docx)
Changing this:
_thisdir = os.path.split(__file__)[0]
_default_docx_path = os.path.join(_thisdir, 'templates', 'default.docx')
To this:
thisdir = os.getcwd()
_default_docx_path = os.path.join(thisdir, 'templates', 'default.docx')
The first one was taking the actual running program and adding it to the path to locate the templates folder.
C:\myfiles\myprogram.exe\templates\default.docx
The solution takes only the path, not the running program.
C:\myfiles\templates\default.docx
Hope it helps!
Instead of changing some library file, I find it easier and cleaner to tell python-docx explicitly where to look for the template, i.e.:
document = Document('whatever/path/you/choose/to/some.docx')
This effectively solves the py2exe and docx path problem.

Categories

Resources