passing bash array to python list - python

I'm trying to pass an array from bash to python using the old getenv method however I keep getting this error:
./crcFiles.sh: line 7: export: `0021': not a valid identifier
Traceback (most recent call last):
File "/shares/web/vm3618/optiload/prog/legalLitres.py", line 30, in <module>
for i in mdcArray.split(' '):
AttributeError: 'NoneType' object has no attribute 'split'
could someone please explain why the $mdcNo isn't passing from bash to python successfully?
Code .sh:
#!/bin/bash
mdcNo=('0021' '0022' '0036' '0055' '0057' '0059' '0061' '0062' '0063' '0065' '0066' '0086' '0095' '0098' '0106' '0110' '0113' '0114' '0115' '0121' '0126' '0128' '0135' '0141' '0143' '0153' '0155' '0158')
localDIR=/shares/web/vm3618/optiload/prog
export mdcNo
$localDIR/legalLitres.py
for i in "${mdcNo[#]}"
do
echo $i
cp $localDIR/MDC$i/*/QqTrkRec.txt $localDIR/crccalc/.
cd $localDIR/crccalc
./crccalc.py QqTrkRec.txt
cp $localDIR/crccalc/QqTrkRec.txt $localDIR/MDC$i/.
done
code .py:
#!/usr/bin/python
import glob
import os
mdcArray = os.getenv('mdcNo')
#Legal Litres that hex and decimal
legalLitresHex = "47E0"
legalLitresTxt = '18,400'
# file name and Legal Litres header
legalHeader = ":00F0:"
hexFile = "QqTrkRec.txt"
# insert comment to explain change
comment = "#\n# 2015 Nov 20: Legal Litres changed to 18,400\n#\n"
commentFlag0 = "# SetDATA"
commentFlag1 = "# SetDATA"
try:
for i in mdcArray.split(' '):
line = ""
Qqfile = glob.glob("/shares/web/vm3618/optiload/prog/MDC"+i+"/*/"+hexFile)
outFile = Qqfile[0]+".new"
print i

If you want to pass a shell array to the Python script, your best bet is to do so as command line arguments. If you run the Python script like this:
python code.py "${mdcNo[#]}"
... then the Python code can just loop over sys.argv, which is always a list. (Specifically, the passed-in array will be the slice sys.argv[1:], since sys.argv[0] is always set to the name of the script itself.)
You were attempting to pass it in through the environment; the problem there is that the environment is a one-dimensional array of strings, with no support for arrays. If the command line is not an option and you have no choice but to use the environment, you'll have to set the environment variable to a string with some delimiter between elements, and split it inside the Python code. The bash for that case would look something like this:
export mdcList='0021,0022,0036,0055,0057,0059,0061,0062,0063,0065,0066,0086,0095,0098,0106,0110,0113,0114,0115,0121,0126,0128,0135,0141,0143,0153,0155,0158'
Or if you already have a bash array, you can build the strong from it:
export mdcList=${mdcNo[0]}
for i in "${mdcNo[#]:1}"; do
mdcList+=,$i
done
Either way, the Python script can recover the array as a list like this:
mdc_no = os.getenv('mdcList').split(',')
If your array elements aren't just numbers, you can replace the comma with something less likely to show up inside an element value; the traditional choice would be the ASCII Unit Separator (U+001F, $'\x1f' in Bash, '\x1f' in Python).

I think Mark Reed already gave you a very good explanation and solution.
Nevertheless, have you considered using python's argparse?
#!/usr/bin/env python
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument('stuff', nargs='+')
args = parser.parse_args()
print args.stuff
if __name__ == '__main__':
main()
Use:
$ mdcNo=('0021' '0022' '0036' '0055' '0057' '0059' '0061' '0062' '0063' '0065' '0066' '0086' '0095' '0098' '0106' '0110' '0113' '0114' '0115' '0121' '0126' '0128' '0135' '0141' '0143' '0153' '0155' '0158')
$ python argp.py "${mdcNo[#]}"
['0021', '0022', '0036', '0055', '0057', '0059', '0061', '0062', '0063', '0065', '0066', '0086', '0095', '0098', '0106', '0110', '0113', '0114', '0115', '0121', '0126', '0128', '0135', '0141', '0143', '0153', '0155', '0158']

Related

Index error while working with sys argv in PyCharm

I'm using 'sys' module to get the filename as an argument in the command line while I'm running the script in cmd it is working as I want, but if I run this in PyCharm it raises an error Index Error: list index out of range. How to get rid of this error in PyCharm?
Here is the Code I'm trying to run:
import sys
def read_lines(file):
list_of_numbers = []
with open(file, mode='r') as read_file:
for number in read_file:
number = number.strip()
list_of_numbers.append(number)
return list_of_numbers
if __name__ == '__main__':
fun = read_lines(sys.argv[1])
print(fun)
While running the script directly from pycharm it raises following error:
Traceback (most recent call last):
File "D:\pythonProjects\test.py", line 10, in <module>
fun = read_lines(sys.argv[1])
IndexError: list index out of range
Presumably that's because you don't provide any arguments when running the script in PyCharm.
You can run the following script to print all the arguments:
import sys
if __name__ == '__main__':
print(sys.argv)
If you run it in the command line something like this
python3 test.py filename.txt
You should the output
['test.py', 'filename.txt']
(the first argument is the name of your script).
In PyCharm, you also have to specify filename.txt as a parameter.
Otherwise, you only get
['test.py']
which means that there is no element 1, hence the IndexError.
You can fix it by adding the filename.txt parameter to your run configuration in PyCharm.

How do I specify the path to the library to run a script from another script?

I need to check the change of one parameter, if it has changed - I need to restart the script.
code:
import subprocess, os.path, time
from engine.db_manager import DbManager
DB = DbManager(os.path.abspath(os.path.join('../webserver/db.sqlite3')))
tmbot = None
telegram_config = DB.get_config('telegram')
old_telegram_token = ''
vkbot = None
vk_config = DB.get_config('vk')
old_vk_token = ''
while True:
telegram_config = DB.get_config('telegram')
if old_telegram_token != telegram_config['token']:
if vkbot != None:
tmbot.terminate()
tmbot = subprocess.Popen(['python', 'tm/tm_bot.py'])
old_telegram_token = telegram_config['token']
print('telegram token was updated')
vk_config = DB.get_config('vk')
if old_vk_token != vk_config['token']:
if vkbot != None:
vkbot.terminate()
vkbot = subprocess.Popen(['python', 'vk/vk_bot.py'])
old_vk_token = vk_config['token']
print('vk token was updated')
time.sleep(30)
I get errors:
enter image description here
While there might be subtle differences between unix and windows, the straight-up answer is: you can use PYTHONPATH environment variable to let python know where to look for libraries.
However, if you use venv, I'd recommend activating it first, or calling the relevant binary instead of setting the environment variable.
Consider this scenario: you have a venv at /tmp/so_demo/venv, and you try to run this file:
$ cat /tmp/so_demo/myscript.py
import requests
print("great success!")
Running the system python interpreter will not find the requests module, and will yield the following error:
$ python3 /tmp/so_demo/myscript.py
Traceback (most recent call last):
File "/tmp/so_demo/myscript.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
As I have installed requests in the venv, if I provide the path to python, it will know where to look:
$ PYTHONPATH='/tmp/so_demo/venv/lib/python3.8/site-packages/' python3 /tmp/so_demo/myscript.py
great success!
but using the binary inside the venv is better, as you don't need to be familiar with the internal venv directory paths, and is much more portable (notice that the path I provided earlier depicts the minor version):
$ /tmp/so_demo/venv/bin/python /tmp/so_demo/myscript.py
great success!

Neatly displaying an output from the command prompt in PHP

I wish to neatly display the output of a python file after it has run some commands in the command prompt.
Currently, a part of my PHP code looks like this:
$output2 = shell_exec('python amassprocess.py 2>&1' . $asnum);
echo $output2
It basically sends the user input to the python file amassprocess.py and runs the file with the user input.
The amassprocess.py file looks like this currently:
import os, sys
#asnum = sys.argv[1]
asnum = "46489"
print("Here is a list of top level seed domain related to the AS Number provided:")
topdomain = os.system('cmd /c "amass.exe intel -asn {}"'.format(asnum))
for i in topdomain:
print(i)
print("<br>")
PS: ignore the 2 asnum, one of them is for testing
The resulting output from running the python script is this:
Here is a list of top level seed domain related to the AS Number provided:
ttvnw.net
justin.tv
twitch.tv
socialcam.com
Traceback (most recent call last):
File "z:/xampp/htdocs/majorproject/amassprocess.py", line 8, in <module>
for i in topdomain:
TypeError: 'int' object is not iterable
However, what i hope the output to be when i run the python script is as follows:
Here is a list of top level seed domain related to the AS Number provided:
ttvnw.net
<br>
justin.tv
<br>
twitch.tv
<br>
socialcam.com
This so that when it is displayed in PHP, it looks like this:
Here is a list of top level seed domain related to the AS Number provided:
ttvnw.net
justin.tv
twitch.tv
socialcam.com
Currently in PHP, the result are all just displayed in a single line like so:
ttvnw.net justin.tv twitch.tv twitchcon.com socialcam.com Traceback (most recent call last): File "amassprocess.py", line 7, in for i in topdomain: TypeError: 'int' object is not iterable
I would be grateful for any help or ideas. Thank you.
Using your code, you can replace newlines with <br />:
$output2 = shell_exec('python amassprocess.py 2>&1' . $asnum);
echo nl2br($output2);
Or you can get an array of lines and join on <br />::
exec('python amassprocess.py 2>&1' . $asnum, $output2);
echo implode('<br />', $output2);
Alternately, with your code, you can display in <pre> or maybe <code> tags, which browsers will render pre-formatted with newlines, tabs etc:
$output2 = shell_exec('python amassprocess.py 2>&1' . $asnum);
echo "<pre>$output2</pre>";
you can do the following in your python code.
define one variable to hold like x="<br>"
and then use for output print(i+x) this will concatenate your string.

Python3.6 |- Import a module from a list with __import__ -|

I'm creating a really basic program that simulates a terminal with Python3.6, its name is Prosser(The origin is that "Prosser" sounds like "Processer" and Prosser is a command processer).
A problem that I'm having is with command import, this is, all the commands are stored in a folder called "lib" in the root folder of Prosser and inside it can have folders and files, if a folder is in the "lib" dir it can't be named as folder anymore, its name now is package(But this doesn't care for now).
The interface of the program is just a input writed:
Prosser:/home/pedro/documents/projects/prosser/-!
and the user can type a command before the text, like a normal terminal:
Prosser:/home/pedro/documents/projects/prosser/-! console.write Hello World
let's say that inside the "lib" folder exists one folder called "console" and inside it has a file called "write.py" that has the code:
class exec:
def main(args):
print(args)
As you can see the first 2 lines is like a important structure for command execution: The class "exec" is the main class for the command execution and the def "main" is the main and first function that the terminal will read and execute also pass the arguments that the user defined, after that the command will be responsible to catch any error and do what it will be created to do.
At this moment, everything is ok, but now comes the true help that I need of U guys, the command import.
Like I writed the user can type any command, and in the example above I typed a "console.write Hello World" and exists one folder called "console" and one file "write.py". The point is that the packages can be defined by a "dot", this is:
-! write Hello World
Above I only typed "write" and this says that the file is only inside the "lib" folder, it doesn't has a package to storage and separate it, so it is a Freedom command(A command that doesn't has packages or nodes).
-! console.write Hello World
Now I typed above "console.write" and this says that the file has a package or node to storage and separate it, this means that it is a Tied command(A command that has packages or nodes).
With that, a file is separated from the package(s) with a dot, the more dots you put, more folders will be navigated to find the file and proceed to the next execution.
Code
Finnaly the code. With the import statement I tryied this:
import os
import form
curDir = os.path.dirname(os.path.abspath(__file__)) # Returns the current direrctory open
while __name__ == "__main__":
c = input('Prosser:%s-! ' % (os.getcwd())).split() # Think the user typed "task.kill"
path = c[0].split('.') # Returns a list like: ["task", "kill"]
try:
args = c[1:] # Try get the command arguments
format = form.formater(args) # Just a text formatation like create a new line with "$n"
except:
args = None # If no arguments the args will just turn the None var type
pass
if os.path.exists(curDir + "/lib/" + "/".join(path) + ".py"): # If curDir+/lib/task/kill.py exists
module = __import__("lib." + '.'.join(path)) # Returns "lib.task.kill"
module.exec.main(args) # Execute the "exec" class and the def "**main**"
else:
pathlast = path[-1] # Get the last item, in this case "kill"
path.remove(path[-1]) # Remove the last item, "kill"
print('Error: Unknow command: ' + '.'.join(path) + '. >>' + pathlast + '<<') # Returns an error message like: "Error: Unknow command: task. >>kill<<"
# Reset the input interface
The problem is that when the line "module = __import__("lib." + '.'.join(path))" is executed the console prints the error:
Traceback (most recent call last):
File "/home/pedro/documents/projects/prosser/main.py", line 18, in <module>
module.exec.main(path) # Execute the "exec" class and the def "**main**"
AttributeError: module 'lib' has no attribute 'exec'
I also tried to use:
module = \_\_import\_\_(curDir + "lib." + '.'.join(path))
But it gets the same error. I think it's lighter for now. I'd like if someone help me or find some replacement of the code. :)
I think you have error here:
You have diffirent path here:
if os.path.exists(curDir + "/lib/" + "/".join(path) + ".py")
And another here, you dont have curDir:
module = __import__("lib." + '.'.join(path)) # Returns "lib.task.kill"
You should use os.path.join to build paths like this:
module = __import__(os.path.join(curdir, 'lib', path + '.py'))

WLST execute stored variable "connect()" statement

So, I am passing a environment variable from bash to python;
#!/usr/bin/env python2
import os
#connect("weblogic", "weblogic", url=xxx.xxx.xxx.xxx:xxxx)
os.environ['bash_variable']
via wlst.sh I can print exported bash_variable, but how do I execute stored variable? Basically, I am trying to remove the original connect statement and pass a variable that has said information. Thanks
Question though, why wouldn't you called the script with the variable as an argument and use sys.argv[] ?
By example something like this.
import os
import sys
import traceback
from java.io import *
from java.lang import *
wlDomain = sys.argv[1]
wlDomPath = sys.argv[2]
wlNMHost = sys.argv[3]
wlNMPort = sys.argv[4]
wlDPath="%s/%s" %(wlDomPath,wlDomain)
wlNMprop="/apps/bea/wls/scripts/.shadow/NM.prop"
try:
print "Connection to Node Manager"
print ""
loadProperties(wlNMprop)
nmConnect(username=NMuser,password=NMpass,host=wlNMHost,port=wlNMPort,domainName=wlDomain,domainDir=wlDPath,mType='ssl',verbose='true')
except:
print "Fatal Error : No Connection to Node Manager"
exit()
print "Connected to Node Manager"
The NM.prop file is a 600 file with the username/password for the NM.
EDIT :
So from what I understand you want to do something like this :
URLS = ['t3s://Host1:Port1','t3s://Host2:Port2','t3s://Host3:Port3']
for urls in URLS:
connect('somebody','password',urls)
{bunch of commands}
disconnect()
And the values of the list URLS would be define by the environment.
The way I see it you have 3 choices :
Have 1 script per environment, more or less identical save for the URLS list
Have 1 script but with a conditionnal branching on sys.argv[1] (the environment as a parameter) and create the list there.
Have 1 script which use a parameter file for each environment according to the environment. Each parameter file containing the list in question.
Something like that :
propENV = sys.argv[1]
propPath = "/path1/path2"
propFile = "%s/%s" %(propPath,propENV)
loadProperties(propFile)
I would probably use the properties file option myself as it is more flexible from an operational standpoint...at least IMHO.

Categories

Resources