Develop Reference

Python erronously adding file extension to path string

I'm currently struggling with the following problem:
My folder structer looks like:
master
- resources
customFile.fmu
fileCallingFMU.py
While executing the fileCallingFMU.py I pass a path string like
path = "./resources/customFile.fmu"
My script contains a super function where I pass the path variable. But everytime I execute the script it trips with an exception:
Exception has occurred: FileNotFoundError
[Errno 2] No such file or directory: b'2021-11-16_./resources/customFile.fmu.txt'
File "[projectFolder]\fileCallingFMU.py", line 219, in __init__
super().__init__(path, config, log_level)
File "[projectFolder]\fileCallingFMU.py", line 86, in <module>
env = gym.make(env_name)
My urging question now is the following:
Why and how does python manipulate the path variable with a date-prefix and .txt as file extension?!
Hope anyone can enlighten me on this one...
EDIT
I'm trying to get the example of ModelicaGym running.
My fileCallingFMU.py contains the following code:
path = "./resources/customFile.fmu"
env_entry_point = 'cart_pole_env:JModelicaCSCartPoleEnv'
config = {
'path': path,
'm_cart': m_cart,
'm_pole': m_pole,
'theta_0': theta_0,
'theta_dot_0': theta_dot_0,
'time_step': time_step,
'positive_reward': positive_reward,
'negative_reward': negative_reward,
'force': force,
'log_level': log_level
}
from gym.envs.registration import register
env_name = env_name
register(
id=env_name,
entry_point=env_entry_point,
kwargs=config
)
env = gym.make(env_name)
The full code for the entryPoint can be found here.

As jjramsey pointed out the problem was burried within the ModelicaGym library.
The logger could not create a propper log file, because the model name was not properly stored in the self.model variable.
Source of this error lied in the line
self.model_name = model_path.split(os.path.sep)[-1]
due to the fact that the os library was not able to separate my path string
"./resources/customFile.fmu"
After changing it to
".\\resources\\customFile.fmu"
everythig works as expected.
Thanks again!

How do I specify the path to the library to run a script from another script?

I need to check the change of one parameter, if it has changed - I need to restart the script.
code:
import subprocess, os.path, time
from engine.db_manager import DbManager
DB = DbManager(os.path.abspath(os.path.join('../webserver/db.sqlite3')))
tmbot = None
telegram_config = DB.get_config('telegram')
old_telegram_token = ''
vkbot = None
vk_config = DB.get_config('vk')
old_vk_token = ''
while True:
telegram_config = DB.get_config('telegram')
if old_telegram_token != telegram_config['token']:
if vkbot != None:
tmbot.terminate()
tmbot = subprocess.Popen(['python', 'tm/tm_bot.py'])
old_telegram_token = telegram_config['token']
print('telegram token was updated')
vk_config = DB.get_config('vk')
if old_vk_token != vk_config['token']:
if vkbot != None:
vkbot.terminate()
vkbot = subprocess.Popen(['python', 'vk/vk_bot.py'])
old_vk_token = vk_config['token']
print('vk token was updated')
time.sleep(30)
I get errors:
enter image description here

While there might be subtle differences between unix and windows, the straight-up answer is: you can use PYTHONPATH environment variable to let python know where to look for libraries.
However, if you use venv, I'd recommend activating it first, or calling the relevant binary instead of setting the environment variable.
Consider this scenario: you have a venv at /tmp/so_demo/venv, and you try to run this file:
$ cat /tmp/so_demo/myscript.py
import requests
print("great success!")
Running the system python interpreter will not find the requests module, and will yield the following error:
$ python3 /tmp/so_demo/myscript.py
Traceback (most recent call last):
File "/tmp/so_demo/myscript.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
As I have installed requests in the venv, if I provide the path to python, it will know where to look:
$ PYTHONPATH='/tmp/so_demo/venv/lib/python3.8/site-packages/' python3 /tmp/so_demo/myscript.py
great success!
but using the binary inside the venv is better, as you don't need to be familiar with the internal venv directory paths, and is much more portable (notice that the path I provided earlier depicts the minor version):
$ /tmp/so_demo/venv/bin/python /tmp/so_demo/myscript.py
great success!

Python script doesnt work in differents virtual machines

I have a script that clean logs in determinate path.
This script works in one virtual machine but when i export to another one, this script doesn't work. The versions of python are the same:
[ec2-user#host1 h]$ python3 --version
Python 3.5.1
The error:
Traceback (most recent call last):
File "jenk.py", line 54, in <module>
dicta = dict(path)
File "jenk.py", line 51, in dict
dicto[elem] = { (dirs['Subdir'][elem]['Path']) : (dirs['Subdir'][elem]['Num_Subdir']) }
KeyError: 'Path'
This is the code:
def PathToDict(path):
st = os.lstat(path)
result = {}
if S_ISDIR(st.st_mode):
result['Path'] = path
result['Subdir'] = {
name: PathToDict(path+'/'+name)
for name in os.listdir(path)}
result['Num_Subdir'] = int(len([i for i, j, k in os.walk(path)]))
# result['RealSize'] = subprocess.check_output(['du','-sh', path]).split()[0].decode('utf-8')
else:
result['type'] = 'file'
return result
dirs = PathToDict(path)
#Getting new dict with clean info with name of dir and number of dirs
def dict(path):
dicto = {}
for elem in dirs['Subdir']:
dicto[elem] = { (dirs['Subdir'][elem]['Path']) : (dirs['Subdir'][elem]['Num_Subdir']) }
return dicto
Any idea? COuld be problem of modules?

there is not 'Path' in elem.
Try to debug
print(dirs['Subdir'][elem])
or use
dirs['Subdir'][elem].get('Path')

One thing that might make stuff os dependent is that you manually join a file path in
name: PathToDict(path+'/'+name)
However, that '/' might not be valid on all devices. Instead it is better to use the os.path module in python to join paths. This will automatically select the correct separator.
name: PathToDict(os.path.join(path,name))
This would at least make that part of the code platform independent.
Additionally, passing your input through os.path.normpath will make sure that the input gets correctly parsed to something that is readable on your platform.

Python3.6 |- Import a module from a list with __import__ -|

I'm creating a really basic program that simulates a terminal with Python3.6, its name is Prosser(The origin is that "Prosser" sounds like "Processer" and Prosser is a command processer).
A problem that I'm having is with command import, this is, all the commands are stored in a folder called "lib" in the root folder of Prosser and inside it can have folders and files, if a folder is in the "lib" dir it can't be named as folder anymore, its name now is package(But this doesn't care for now).
The interface of the program is just a input writed:
Prosser:/home/pedro/documents/projects/prosser/-!
and the user can type a command before the text, like a normal terminal:
Prosser:/home/pedro/documents/projects/prosser/-! console.write Hello World
let's say that inside the "lib" folder exists one folder called "console" and inside it has a file called "write.py" that has the code:
class exec:
def main(args):
print(args)
As you can see the first 2 lines is like a important structure for command execution: The class "exec" is the main class for the command execution and the def "main" is the main and first function that the terminal will read and execute also pass the arguments that the user defined, after that the command will be responsible to catch any error and do what it will be created to do.
At this moment, everything is ok, but now comes the true help that I need of U guys, the command import.
Like I writed the user can type any command, and in the example above I typed a "console.write Hello World" and exists one folder called "console" and one file "write.py". The point is that the packages can be defined by a "dot", this is:
-! write Hello World
Above I only typed "write" and this says that the file is only inside the "lib" folder, it doesn't has a package to storage and separate it, so it is a Freedom command(A command that doesn't has packages or nodes).
-! console.write Hello World
Now I typed above "console.write" and this says that the file has a package or node to storage and separate it, this means that it is a Tied command(A command that has packages or nodes).
With that, a file is separated from the package(s) with a dot, the more dots you put, more folders will be navigated to find the file and proceed to the next execution.
Code
Finnaly the code. With the import statement I tryied this:
import os
import form
curDir = os.path.dirname(os.path.abspath(__file__)) # Returns the current direrctory open
while __name__ == "__main__":
c = input('Prosser:%s-! ' % (os.getcwd())).split() # Think the user typed "task.kill"
path = c[0].split('.') # Returns a list like: ["task", "kill"]
try:
args = c[1:] # Try get the command arguments
format = form.formater(args) # Just a text formatation like create a new line with "$n"
except:
args = None # If no arguments the args will just turn the None var type
pass
if os.path.exists(curDir + "/lib/" + "/".join(path) + ".py"): # If curDir+/lib/task/kill.py exists
module = __import__("lib." + '.'.join(path)) # Returns "lib.task.kill"
module.exec.main(args) # Execute the "exec" class and the def "**main**"
else:
pathlast = path[-1] # Get the last item, in this case "kill"
path.remove(path[-1]) # Remove the last item, "kill"
print('Error: Unknow command: ' + '.'.join(path) + '. >>' + pathlast + '<<') # Returns an error message like: "Error: Unknow command: task. >>kill<<"
# Reset the input interface
The problem is that when the line "module = __import__("lib." + '.'.join(path))" is executed the console prints the error:
Traceback (most recent call last):
File "/home/pedro/documents/projects/prosser/main.py", line 18, in <module>
module.exec.main(path) # Execute the "exec" class and the def "**main**"
AttributeError: module 'lib' has no attribute 'exec'
I also tried to use:
module = \_\_import\_\_(curDir + "lib." + '.'.join(path))
But it gets the same error. I think it's lighter for now. I'd like if someone help me or find some replacement of the code. :)

I think you have error here:
You have diffirent path here:
if os.path.exists(curDir + "/lib/" + "/".join(path) + ".py")
And another here, you dont have curDir:
module = __import__("lib." + '.'.join(path)) # Returns "lib.task.kill"
You should use os.path.join to build paths like this:
module = __import__(os.path.join(curdir, 'lib', path + '.py'))

removing py files and retaining pyc files breaks inspection code

The function below works just fine. But if I remove all py files (and leave the pycs intact) then I get an error:
To explain what I mean by 'intact' here is more or less what I did:
1. write a bunch of py files and stick them in a friendly directory structure
2. test code code. It works
3. compile all py files to get pyc files
4. delete py files
5. test code. It fails
The function:
def get_module_name_and_line():
"""
return the name of the module from which the method calling this method was called.
"""
import inspect
lStack = inspect.stack()
oStk = lStack[2]
oMod = inspect.getmodule(oStk[0])
oInfo = inspect.getframeinfo(oStk[0])
sName = oMod.__name__ #<<<<<<<<<<<<<<<<<< ERROR HERE
iLine = oInfo.lineno
return sName,iLine
The error:
AttributeError: 'NoneType' object has no attribute '__name__'
So oMod is None in this error. If the py files are around then oMod is never None.
The question:
Why does inspect only return a module if py files are intact? How can I make this function work without py files.
Full Traceback:
Original exception was:
Traceback (most recent call last):
File "/home/criticalid/programs/damn.py", line 630, in <module>
File "/home/criticalid/programs/golly/class_foo.py", line 121, in moo
File "/home/criticalid/programs/golly/class_foo.py", line 151, in get_module_name_and_line
AttributeError: 'NoneType' object has no attribute '__name__'

This works for me. It assumes that all modules are in packages within the current working directory. And it doesn't return the __main__ module, rather its file name.
I'm sure there is a better solution but this solves my problems.
def get_module_name_and_line():
"""
return the name of the module from which the method calling this method was called.
"""
def get_name_from_path(sPath):
import os
sCWD = os.getcwd()
lCWD = list(os.path.split(sCWD))
lPath = list(os.path.split(sPath))
lPath[-1] = '.'.join(lPath[-1].split('.')[:-1]) #remove file extension
lRet = [s for s in lPath[len(lCWD)-1:]]
return '.'.join(lRet)
import inspect
lStack = inspect.stack()
oStk = lStack[2]
iLine = inspect.getlineno(oStk[0])
sName = get_name_from_path(inspect.getfile(oStk[0]))
return sName,iLine