Develop Reference

Why does python3.7 pass argv elements alongside 'garbage'

I had this script working for me, before I decided I'm gonna rewrite everything and make it portable.
Without delving too much into the details, there's a central Bash script, which calls 5 other Bash scripts in their own respective folders. I have no intention of porting to Windows anytime soon, as of current this is just for Linux.
The execution path of the central Bash script is:
dos.1/1-init.sh dos.1/
dos.2/1-trace-to-file.sh dos.2/ dos.1/
dos.3/1-recognize-categories.sh dos.3/
dos.4/1-ping-in-groups.sh dos.4/ dos.3/
dos.5/init.sh dos.5/ dos.4/
I run with ./init.sh
Before the script was 'portable' I was using explicit file paths inside each respective script. All was well and good. The program itself is a combination of Bash and Python, and writes to files in one directory, so that they can be manipulated in various ways, before being read back into different parts of the program.
I understand that the fastest way to do this would be to write a monolithic Python script, using subprocess calls for the Bash side of things... However, I am doing it this way to ease maintenance, and (before I started making it 'portable') it was lightning fast.
My issue now is this: each time I have to read text into Python (either from SQL or from file) there's always this added garbage. Up until this point, I have been using sed, awk and Python's .rstrip() function to manage this... Which is all well and good, but this one damn function will not play nice... And I feel there must be a better way.
In bash I call it with:
$prog_dir=$1
$data_dir=$2
$prog_dir/2fast-ping.py $data_dir/group0.txt > $prog_dir/group0_averages.txt
$prog_dir/2fast-ping.py $data_dir/group1.txt > $prog_dir/group1_averages.txt
...
Now I know that I could write to file from within Python, but in this instance I have other reasons not to.
The issue, is that when the 2fast-ping.py script is ran, it reads the text file in with commas and a newline char. I have vigorously checked and I can confirm that the group#.txt files 100% do not contain commas. Here's the Python:
import sys
import subprocess
import select
from concurrent.futures import ThreadPoolExecutor
filename = sys.argv[1]
f = open(filename, "r")
ips = [elem.rstrip('\n') for elem in f]
print(ips)
f.close()
The script goes on to do some work on the IPs afterwards, but this is the painful part. If I call the script direct from CLI: ./2fast-ping.py ../dos.3/group0.txt, the text is processed PROPERLY and the superseding instructions actually function. But, when called from the first init script, the program basically sh*ts itself because each line is read in with commas. It works until the point where it starts to use the processed info, then:
<actual IP would be here>
ping: ('##.###.###.###',): Name or service not known
Of course, the issue is the ('',) But, Python is adding that in, and I don't know how to stop it :(
Any ideas?

Python code was okay, just passing an additional / with the argument :(

Some functions from library works in python shell but not in a script

I'm having trouble getting certain functions from a library called art (https://github.com/sepandhaghighi/art) to run in a script, though they work fine in a shell. The script/commands entered sequentially look like this:
from art import *
randart() <(function fails in script, but succeeds in shell)
tart("test") <(different function, same library, succeeds in both shell and script)
import sys
print(sys.version)
The python version is 3.7.5 for both the shell and the script. The first function does not throw an error when run in a script, but does not give any output. Its desired output is a random ascii_art from a collection. I feel like I'm missing something really simple. Any ideas? The documentation on github reports "Some environments don't support all 1-Line arts", however they are the same python version on the same machine. Are there other portions of the environment that could be the cause?

You need to print randart() while writing in script. Make a habit of using print() for everything while printing. Shell is the place which returns the value by default whereas you need to tell the script window what to do with any name or function.
so use this:
from art import *
print(randart())

in the shell it is implicitly printed ... in a script you must explicitly
print(randart())

%USERPROFILE% env variable for python

I am writing a script in Python 2.7.
It needs to be able to go whoever the current users profile in Windows.
This is the variable and function I currently have:
import os
desired_paths = os.path.expanduser('HOME'\"My Documents")
I do have doubts that this expanduser will work though. I tried looking for Windows Env Variables to in Python to hopefully find a list and know what to convert it to. Either such tool doesn't exist or I am just not using the right search terms since I am still pretty new and learning.

You can access environment variables via the os.environ mapping:
import os
print(os.environ['USERPROFILE'])
This will work in Windows. For another OS, you'd need the appropriate environment variable.
Also, the way to concatenate strings in Python is with + signs, so this:
os.path.expanduser('HOME'\"My Documents")
^^^^^^^^^^^^^^^^^^^^^
should probably be something else. But to concatenate paths you should be more careful, and probably want to use something like:
os.sep.join(<your path parts>)
# or
os.path.join(<your path parts>)
(There is a slight distinction between the two)
If you want the My Documents directory of the current user, you might try something like:
docs = os.path.join(os.environ['USERPROFILE'], "My Documents")
Alternatively, using expanduser:
docs = os.path.expanduser(os.sep.join(["~","My Documents"]))
Lastly, to see what environment variables are set, you can do something like:
print(os.environ.keys())
(In reference to finding a list of what environment vars are set)

Going by os.path.expanduser , using a ~ would seem more reliable than using 'HOME'.

How can I call an OpenModelica model in Python with OMPython?

I have an OpenModelica model made with OMEdit. In order to get a concrete example I designed the following:
Now I would like to run the model in Python. I can do this by using OMPython. After importing OMPython and loading the files I use the following command to run the simulation:
result = OMPython.execute("simulate(myGain, numberOfIntervals=2, outputFormat=\"mat\")")
The simulation now runs and the results are written to a file.
Now I would like to run the same model but with an different parameter for the constant block.
How can I do this?
Since the parameter is compiled into the model it should not be possible to change it. So what I need is a model like that:
Is it possible to call the model from Python and set the variable "a" to a specific value?
With the command OMPython.execute("simulate(...)") I can specify some environment variables like "numberOfIntervals" or "outputFormat" but not more.

You can send more flags to the simulate command. For example simflags to override parameters. See https://openmodelica.org/index.php/forum/topic?id=1011 for some details.
You can also use the buildModel(...) command followed by system("./ModelName -overrideFile ...") to avoid re-translation and re-compilation or with some minor scripting parallel parameter sweeps. If you use Linux or OSX it should be easy to call OMPython to create the executable and then call it yourself. On Windows you need to setup some environment variables for it to work as expected.

I believe you are looking for the setParameterValue command. You can read about it here: https://openmodelica.org/download/OMC_API-HowTo.pdf
Basically you would add a line similar to OMPython.execute("setParameterValue(myGain, a, 20)") to your python script before the line where you run the simulation, so long as a is a parameter in your model.

Create one new folder in windows
In this folder put/create 2 new files file1.py and file2.bat
The file1.py content is:
import os
import sys
sys.path.insert(0, "C:\OpenModelica1.11.0-32bit\share\omc\scripts\PythonInterface")
from OMPython import OMCSession
sys.path.insert(0, "C:\OpenModelica1.11.0-32bit\lib\python")
os.environ['USER'] = 'stefanache'
omc = OMCSession()
omc.sendExpression("loadModel(Modelica)")
omc.sendExpression("loadFile(getInstallationDirectoryPath() + \"/share/doc/omc/testmodels/BouncingBall.mo\")")
omc.sendExpression("instantiateModel(BouncingBall)")
omc.sendExpression("simulate(BouncingBall)")
omc.sendExpression("plot(h)")`
the file2.bat content is:
#echo off
python file1.py
pause
then click on file2.bat... and please be patient!
The plotted result window will appear.

How can I add a command to the Python interactive shell?

I'm trying to save myself just a few keystrokes for a command I type fairly regularly in Python.
In my python startup script, I define a function called load which is similar to import, but adds some functionality. It takes a single string:
def load(s):
# Do some stuff
return something
In order to call this function I have to type
>>> load('something')
I would rather be able to simply type:
>>> load something
I am running Python with readline support, so I know there exists some programmability there, but I don't know if this sort of thing is possible using it.
I attempted to get around this by using the InteractivConsole and creating an instance of it in my startup file, like so:
import code, re, traceback
class LoadingInteractiveConsole(code.InteractiveConsole):
def raw_input(self, prompt = ""):
s = raw_input(prompt)
match = re.match('^load\s+(.+)', s)
if match:
module = match.group(1)
try:
load(module)
print "Loaded " + module
except ImportError:
traceback.print_exc()
return ''
else:
return s
console = LoadingInteractiveConsole()
console.interact("")
This works with the caveat that I have to hit Ctrl-D twice to exit the python interpreter: once to get out of my custom console, once to get out of the real one.
Is there a way to do this without writing a custom C program and embedding the interpreter into it?
Edit
Out of channel, I had the suggestion of appending this to the end of my startup file:
import sys
sys.exit()
It works well enough, but I'm still interested in alternative solutions.

You could try ipython - which gives a python shell which does allow many things including automatic parentheses which gives you the function call as you requested.

I think you want the cmd module.
See a tutorial here:
http://wiki.python.org/moin/CmdModule

Hate to answer my own question, but there hasn't been an answer that works for all the versions of Python I use. Aside from the solution I posted in my question edit (which is what I'm now using), here's another:
Edit .bashrc to contain the following lines:
alias python3='python3 ~/py/shellreplace.py'
alias python='python ~/py/shellreplace.py'
alias python27='python27 ~/py/shellreplace.py'
Then simply move all of the LoadingInteractiveConsole code into the file ~/py/shellreplace.py Once the script finishes executing, python will cease executing, and the improved interactive session will be seamless.