Develop Reference

Find and replace variable value in a .py file using another python program

Here is my challenge. I have a radioConfig.py file which contains variable values which need to be changed if and when the user changes location or scan times. This would be used with students so i'm programming a GUI, pysimplegui, to change the values of those variable.
Here is what i have so far but its not working. Its replacing the variable name, not the value.
I'm using a Rpi and python3. I studied electronics and my program skills are with C. I'm not sure if this is the best way to solve my challenge nor do i know of the python options which exist that could be useful. Any help would be amazing.
#File: GuiTest.py before code is executed
freqCenter = 21000000
freqBandwidth = 8000000
upconvFreqHz = 125000000
fftWindow = "BlacHarr"
notes = "Test"
rtlSampleRateHz = 2000000
#-----------------------------------------------------------------------
#Program which will be a gui asking user for input values
freqCenterGUI = 20800280
with open('GuiTest.py', 'r') as file :
filedata = file.read()
filedata = filedata.replace('freqCenter', str(freqCenterGUI).strip('()'))
with open('GuiTest.py', 'w') as file:
file.write(filedata)
#File: GuiTest.py after code is executed
20800280 = 21000000
freqBandwidth = 8000000
upconvFreqHz = 125000000
notes = "Test"
rtlSampleRateHz = 2000000
#-----------------------------------------------------------------------

I'd say : use a config file.
Modifying a script is really not good practice !
In your cfg.ini :
[_]
freqCenter = 21000000
freqBandwidth = 8000000
upconvFreqHz = 125000000
fftWindow = BlacHarr
notes = Test
rtlSampleRateHz = 2000000
Then use configparser :
import configparser
cfg = configparser.ConfigParser()
with open('cfg.ini', encoding='utf-8') as f:
cfg.read_file(f)
cfg['_']['freqCenter'] = '20800280'
with open('cfg.ini', 'w', encoding='utf-8') as f:
cfg.write(f)
EDIT :
Or, as suggested by #juanpa.arrivillaga, using a json is also a great solution !
Matter of taste... :)

Quick and dirty:
File 1:
freqCenter = 21000000
freqBandwidth = 8000000
upconvFreqHz = 125000000
fftWindow = "BlacHarr"
notes = "Test"
rtlSampleRateHz = 2000000
File 2
#Program which will be a gui asking user for input values
freqCenterGUI = "20800280"
my_dict={}
file_data=open("GuiTest.py", "r")
in_lines=file_data.readlines()
for line in in_lines:
var, val = line.split('=')
if var.strip() == "freqCenter":
val = freqCenterGUI + "\n"
my_dict[var]=val
f_out=open("GuiTest.py", "w")
for key,val in my_dict.items():
f_out.write(str(key)+"="+str(val))
f_out.close()

How to store a html file in a variable as a string in python

I have html file and I want to store the contents of that file in a variable as a string value. I have tried this but it returns a blank string. How can I do this?
def myfunc():
str = ""
with open("/home/suman/Alert_logo_report.html") as report_file:
raw_html = report_file.readlines()
str = ''.join(raw_html)
return str

Strange, I don't see that behavior:
def get_content(path):
with open(path) as f:
raw_lines = f.readlines()
content = ''.join(raw_lines)
return content
gives me the HTML content.
On the other hand:
-you're overwriting str that is a basic python function transforming something in a string. I cannot see this ending well (meaning: I've done it in the past, it wasn't pretty)
-there is not need to declare a variable, this is python
-I don't understand why reading lines, then joining them without doing anything.
The way I see it, this would be better:
def get_content(path):
with open(path) as f:
raw = f.read()
return raw

Use """string_literal""" instead of "string_literal" or 'string_literal'. """string_literal""" is for multi line strings. This will work.
def myfunc():
str = """"""
with open("employees.html") as report_file:
raw_html = report_file.readlines()
str = """""".join(raw_html)
return str

Try this
f = open("0101.html", "r")
str1 = f.readlines()
print(str1)
f.close()

Editing a downloaded CSV in memory before writing

Forewarning: I am very new to Python and programming in general. I am trying to use Python 3 to get some CSV data and making some changes to it before writing it to a file. My problem lies in accessing the CSV data from a variable, like so:
import csv
import requests
csvfile = session.get(url)
reader = csv.reader(csvfile.content)
for row in reader:
do(something)
This returns:
_csv.Error: iterator should return strings, not int (did you open the file in text mode?)
Googling revealed that I should be feeding the reader text instead of bytes, so I also attempted:
reader = csv.reader(csvfile.text)
This also does not work as the loop works through it letter by letter instead of line by line. I also experimented with TextIOWrapper and similar options with no success. The only way I have managed to get this to work is by writing the data to a file, reading it, and then making changes, like so:
csvfile = session.get(url)
with open("temp.txt", 'wb') as f:
f.write(csvfile.content)
with open("temp.txt", 'rU', encoding="utf8") as data:
reader = csv.reader(data)
for row in reader:
do(something)
I feel like this is far from the most optimal way of doing this, even if it works. What is the proper way to read and edit the CSV data directly from memory, without having to save it to a temporary file?

you don't have to write to a temp file, here is what I would do, using the "csv" and "requests" modules:
import csv
import requests
__csvfilepathname__ = r'c:\test\test.csv'
__url__ = 'https://server.domain.com/test.csv'
def csv_reader(filename, enc = 'utf_8'):
with open(filename, 'r', encoding = enc) as openfileobject:
reader = csv.reader(openfileobject)
for row in reader:
#do something
print(row)
return
def csv_from_url(url):
line = ''
datalist = []
s = requests.Session()
r = s.get(url)
for x in r.text.replace('\r',''):
if not x[0] == '\n':
line = line + str(x[0])
else:
datalist.append(line)
line = ''
datalist.append(line)
# at this point you already have a data list 'datalist'
# no need really to use the csv.reader object, but here goes:
reader = csv.reader(datalist)
for row in reader:
#do something
print(row)
return
def main():
csv_reader(__csvfilepathname__)
csv_from_url(__url__)
return
if __name__ == '__main__':
main ()
not very pretty, and probably not very good in regards to memory/performance, depending on how "big" your csv/data is
HTH, Edwin.

Parsing specific contents in a file

I have a file that looks like this
!--------------------------------------------------------------------------DISK
[DISK]
DIRECTION = 'OK'
TYPE = 'normal'
!------------------------------------------------------------------------CAPACITY
[CAPACITY]
code = 0
ID = 110
I want to read sections [DISK] and [CAPACITY].. there will be more sections like these. I want to read the parameters defined under those sections.
I wrote a following code:
file_open = open(myFile,"r")
all_lines = file_open.readlines()
count = len(all_lines)
file_open.close()
my_data = {}
section = None
data = ""
for line in all_lines:
line = line.strip() #remove whitespace
line = line.replace(" ", "")
if len(line) != 0: # remove white spaces between data
if line[0] == "[":
section = line.strip()[1:]
data = ""
if line[0] !="[":
data += line + ","
my_data[section] = [bit for bit in data.split(",") if bit != ""]
print my_data
key = my_data.keys()
print key
Unfortunately I am unable to get those sections and the data under that. Any ideas on this would be helpful.

As others already pointed out, you should be able to use the ConfigParser module.
Nonetheless, if you want to implement the reading/parsing yourself, you should split it up into two parts.
Part 1 would be the parsing at file level: splitting the file up into blocks (in your example you have two blocks: DISK and CAPACITY).
Part 2 would be parsing the blocks itself to get the values.
You know you can ignore the lines starting with !, so let's skip those:
with open('myfile.txt', 'r') as f:
content = [l for l in f.readlines() if not l.startswith('!')]
Next, read the lines into blocks:
def partition_by(l, f):
t = []
for e in l:
if f(e):
if t: yield t
t = []
t.append(e)
yield t
blocks = partition_by(content, lambda l: l.startswith('['))
and finally read in the values for each block:
def parse_block(block):
gen = iter(block)
block_name = next(gen).strip()[1:-1]
splitted = [e.split('=') for e in gen]
values = {t[0].strip(): t[1].strip() for t in splitted if len(t) == 2}
return block_name, values
result = [parse_block(b) for b in blocks]
That's it. Let's have a look at the result:
for section, values in result:
print section, ':'
for k, v in values.items():
print '\t', k, '=', v
output:
DISK :
DIRECTION = 'OK'
TYPE = 'normal'
CAPACITY :
code = 0
ID = 110

Are you able to make a small change to the text file? If you can make it look like this (only changed the comment character):
#--------------------------------------------------------------------------DISK
[DISK]
DIRECTION = 'OK'
TYPE = 'normal'
#------------------------------------------------------------------------CAPACITY
[CAPACITY]
code = 0
ID = 110
Then parsing it is trivial:
from ConfigParser import SafeConfigParser
parser = SafeConfigParser()
parser.read('filename')
And getting data looks like this:
(Pdb) parser
<ConfigParser.SafeConfigParser instance at 0x100468dd0>
(Pdb) parser.get('DISK', 'DIRECTION')
"'OK'"
Edit based on comments:
If you're using <= 2.7, then you're a little SOL.. The only way really would be to subclass ConfigParser and implement a custom _read method. Really, you'd just have to copy/paste everything in Lib/ConfigParser.py and edit the values in line 477 (2.7.3):
if line.strip() == '' or line[0] in '#;': # add new comment characters in the string
However, if you're running 3'ish (not sure what version it was introduced in offhand, I'm running 3.4(dev)), you may be in luck: ConfigParser added the comment_prefixes __init__ param to allow you to customize your prefix:
parser = ConfigParser(comment_prefixes=('#', ';', '!'))

If the file is not big, you can load it and use Regexes to find parts that are of interest to you.

Python script how can it do short

I have written a script on a python "icecast server", and I changed some strings in "/etc/icecast2/icecast.xml" like this:
import os,sys,re
def ices2():
changedir=open(pathh + "icecast3.xml", "w")
data=open("/etc/icecast2/icecast.xml").read()
changedir.write(re.sub("<source-password>hackme</source-password>","<source-password>123</source-password>" % x,data))
changedir.close()
ices2()
def ices1():
changedir1=open(pathh + "icecast2.xml", "w")
data=open(pathh + "icecast3.xml").read()
changedir1.write(re.sub("<relay-password>hackme</relay-password>", "<relay-password>123</relay-password>" % x,data))
changedir1.close()
os.remove(pathh + "icecast3.xml")
ices1()
def ices():
changedir2=open("/etc/icecast2/icecast.xml", "w")
data=open(pathh + "icecast2.xml").read()
changedir2.write(re.sub("<admin-password>hackme</admin-password>","<admin-password>123</admin-password>" % x,data))
changedir2.close()
os.remove(pathh + "icecast2.xml")
ices()
...but it's too long for the script. How can I shorten it? I need to do some changes in one file, open it to make changes and close it without any lost data. I know that it can be done in one function, but how to do it I don't know.
I need three changes in one function like this:
def ices():
changedir=open(pathh + "icecast3.xml", "w")
data=open("/etc/icecast2/icecast.xml").read()
changedir.write(re.sub("<source-password>hackme</source-password>","<source-password>123</source-password>",data))
changedir1.write(re.sub("<relay-password>hackme</relay-password>", "<relay-password>123</relay-password>",data))
changedir2.write(re.sub("<admin-password>hackme</admin-password>","<admin-password>123</admin-password>",data))
changedir.close()
i did it in one function and my script short than upper one. But it's wrong i need do it correctly
changedir=open(pathh + "icecast3.xml", "w")
data=open("/etc/icecast2/icecast.xml").read()
Here I create a new file "pathh + "icecast3.xml" (pathh-/home/user/Downloads), but I need to open file:
"/etc/icecast2/icecast.xml"
...read it and write changes to the same file.

All three functions do the same so you can join them into one. This is not complete solution but I think that you could go on from here on your own:
import os,sys,re
def ices(in_path, out_path, remove=False):
changedir = open(out_path, "w")
data = open(in_path, 'r')
changedir.write(re.sub("<source-password>hackme</source-password>","<source-password>123</source-password>" % x,data.read())) # this is wrong as well but I take it as an example
changedir.close()
data.close()
if remove:
os.remove(in_path)
You can call this function with:
ices(base_path + 'icecast2.xml', base_path + 'icecast3.xml', True)
Hints:
it's better to use os.path.join for creating the full paths (as opposed to string concatenation)
look at with statement and cosider using it for increased readability
EDIT (respecting the clarification in comment):
Sorry I missed the different strings in write. You can do it simply like this:
f = open(filename, 'r')
data = f.read()
f.close()
for tag in ['source', 'relay', 'admin']
sub_str = "<{tag_name}>%s</{tag_name}>".format(tag_name=tag+'-password')
data = re.sub(sub_str % 'hackme', sub_str % '123', data)
f = open(filename+'.new', 'w')
f.write(data)
f.close()