how can I write a script that it changes by execution?
For example, two a have script exist from two row:
list = [1, 2, 3, 4, 5]
sliced_list = list[0:1]
Executing it, 2nd row should be:
sliced_list = list[1:2]
and then,
sliced_list = list[2:3]
I want to modify variable "sliced_list" everytime I run this file.
Generally this is not something you should ever want to do, since it is likely to result in non-deterministic behavior, and in the event of a bug its possible to completely overwrite your script and lose data.
If you want to change the date your script is operating on you should store it persistently in some fashion. This could be in a separate file somewhere or in an environment variable.
But to do what your asking you would need to open the script, copy the contents, and modify the content as you desire like this:
with open("/path/to/script.py", 'r+') as script:
contents = script.read()
# ... some string logic here
# Point cursor to the beginning of the file
# If the original contents were longing than the new contents
# you'll have unwanted data at the end of the file.
script.seek(0)
script.write(contents)
You could save the start index into a file when you run the script. Then increment it and save it. Something like what is shown below.
import os
List=[1,2,3,4,5]
file=open('file.txt','r+')
start_index=int(file.read())
print(List[start_index:start_index+1])
file.close()
os.remove('file.txt')
file=open('file.txt','w')
file.write(str(start_index+1))
file.close()
Related
I'm working on a program that, in short, creates a CSV file and compares it line by line to a previously generated CSV file (created on previous script execution), and logs differences between the two.
I'm having a weird issue where csv.dictreader does not appear to be reading all rows of the NEW log - but it DOES read all rows of the OLD log. What makes this issue even more puzzling is that if I run the program again, it creates a new log, and will now read the previous log all the way to the end.
Here's what I mean if that made no sense:
Run program to generate LOG A
Run program again to generate LOG B
csv.dictreader does not read all the way through LOG B, but it DOES read all the way through LOG A
Run program again to generate LOG C
csv.dictreader does NOT read all the way through LOG C, but it DOES read all the way through LOG B (which it previously didn't although no information in the csv file has changed)
Here's the relevant function:
def compare_data(newLog, oldLog):
# this function compares the new scan Log to the old Log to determine if, and how much, memory space has changed
# within each directory
# both arguments should be filenames of the logs
# use the .csv library to create dictionary objects out of files and read thru them
newReader = csv.DictReader(newLog)
oldReader = csv.DictReader(oldLog)
oldDirs, newDirs = [], []
# write data from new log into dictionary list
for row in newReader:
newLogData = {}
newLogData['ScanDate'] = row['ScanDate']
newLogData['Directory'] = row['Directory']
newLogData['Size'] = int(row['Size'])
newDirs.append(newLogData)
# write data from old log into another dictionary list
for row in oldReader:
oldLogData = {}
oldLogData['ScanDate'] = row['ScanDate']
oldLogData['Directory'] = row['Directory']
oldLogData['Size'] = int(row['Size'])
oldDirs.append(oldLogData)
# now compare data between the two dict lists to determine what's changed
for newDir in newDirs:
for oldDir in oldDirs:
dirMatch = False
if newDir['Directory'] == oldDir['Directory']:
# we have linked dirs. now check to see if their size has changed
dirMatch = True
if newDir['Size'] != oldDir['Size']:
print(newDir['Directory'], 'has changed in size! It used to be', oldDir['Size'],
'bytes, now it\'s', newDir['Size'], 'bytes! Hype')
# now that we have determined changes, now we should check for unique dirs
# unique meaning either a dir that has been deleted since last scan, or a new dir added since last scan
find_unique_dirs(oldDirs, newDirs)
Based on the fact that the old log gets read in its entirety, I don't think it would be an issue of open quotes in filenames or anything like that.
So if I have the same piece of code inside of 10 separate .ipynb files with different names and lets say that the code is as follows.
x = 1+1
so pretty simple stuff, but I want to change the variable x to y. Is their anyway using python to loop through each .ipynb file and do some sort of find and replace anywhere it sees x to change it or replace it with y? Or will I have to open each file up in Jupiter notebook and make the change manually?
I never tried this before, but the .ipynb files are simply JSONs. These pretty much function like nested dictionaries. Each cell is contained within the key 'cells', and then the 'cell_type' tells you if the cell is code. You then access the contents of the code cell (the code part) with the 'source' key.
In a notebook I am writing I can look for a particular piece of code like this:
import json
with open('UW_Demographics.ipynb') as f:
ff = json.load(f)
for cell in ff['cells']:
if cell['cell_type'] == 'code':
for elem in cell['source']:
if "pd.read_csv('UWdemographics.csv')" in elem:
print("OK")
You can iterate over your ipynb files, identify the code you want to change using the above, change it and save using json.dump in the normal way.
I have a Python code which is logging some data into a .csv file.
logging_file = 'test.csv'
dt = datetime.datetime.now()
f = open(logging_file, 'a')
f.write('\n "{:%H:%M:%S}",{},{}'.format(dt,x,y,))
The above code is the core part and this produces continuous data in .csv file as
"00:34:09" ,23.05,23.05
"00:36:09" ,24.05,24.05
"00:38:09" ,26.05,26.05
... etc.,
Now I wish to add the following lines in first row of this data. time, data1,data2.I expect output as
time, data1, data2
"00:34:09" ,23.05,23.05
"00:36:09" ,24.05,24.05
"00:38:09" ,26.05,26.05
... etc.,
I tried many ways. Those ways not produced me the result as preferred format.But I am unable to get my expected result.
Please help me to solve the problem.
I would recommend writing a class specifically for creating and managing logs.Have it initialize a file, on creation, with the expected first line (don't forget a \n character!), and keep track of any necessary information about that log(the name of the log it created, where it is, etc). You can then have the class 'write' to the log (append the log, really), you can create new logs as necessary, and, you can have it check for existing logs, and make decisions about either updating what is existing, or scrapping it and starting over.
I've got the following code, but I can't quite get it to work:
import glob
import os
import processing # for QGIS functionality
os.chdir("/home/mydir/vector")
columnid = ['SAMP300','SAMP100','SAMP30', 'SAMP10']
for x in glob.glob("prefix*.shp"):
for y in columnid:
for z in [1, 2, 3]:
output_alg0='"/home/mydir/vector/samples/{x}_{y}_{z}.shp"'.format(x=x.rstrip('.shp'), y=y, z=z)
output_0='processing.runalg("qgis:randompointsinsidepolygonsvariable","/home/mydir/vector/{x}",0,"{y}",1000,output_alg0)'.format(x=x, y=y)
The script reads through a directory of .shp files, and uses variables to name the outputs. output_alg0 creates the output file name used in the next step. The file name is based on the original file, and two variables within the loop. output_0 is the actual QGIS algorithm that is run, which references each .shp file in the loop, passes the columnid variable and some fixed parameters, and references output_alg0 for naming the output .shp file.
If I append print in front of the two commands within the loop, I get the output that I'm expecting (i.e., the {x},{y}, and {z} variables are correctly populated. Furthermore, the script gives no error when executed, but no output is produced.
Here's an example of the output by appending print and parentheses to the two lines within the loop:
output_alg0="/home/mydir/vector/samples/prefix_SAMP10_3.shp"
output_0=processing.runalg("qgis:randompointsinsidepolygonsvariable","/home/mydir/vector/prefix.shp",0,"SAMP10",1000,output_alg0)
I can copy and paste both lines exactly as they appear above into the QGIS Python Console and the commands are executed as expected (i.e., random point file is generated based on the input shapefile, and an output shapefile is created as specified).
I think it has something to do with how I'm using .format and/or perhaps how I'm using single and/or double quotations, and/or some sort of Python/QGIS interaction that I don't quite understand.
EDIT: Instead of using the two output_ prefixes, I also tried out this one-liner version, and the script executes (without error), but no output is created.
import glob
import os
import processing # for QGIS functionality
os.chdir("/home/mydir/vector")
columnid = ['SAMP300','SAMP100','SAMP30', 'SAMP10']
for x in glob.glob("prefix*.shp"):
for y in columnid:
for z in [1, 2, 3]:
'processing.runalg("qgis:randompointsinsidepolygonsvariable","/home/mydir/vector/{w}",0,"{y}",1000,"/home/mydir/vectorsamples/{x}_{y}_{z}.shp")'.format(w=x, x=x.rstrip('.shp'), y=y, z=z)
Currently your loop is generating the variables how you want them to and assigning them to output_alg0 and output_0 the way you want, but you don't ever DO anything with them after that.
I've asked for your expected output in a comment. If you add that we can probably figure out how to get you there. My GUESS is that you're trying to execute output_0 (which currently is just a string that looks like a python command). In which case you'd have to use exec(output_0) but be wary of using exec in general. It's a scary command.
EDIT:
If I were you, I'd do something more like:
# inside your loops::
output_alg0='"/home/mydir/vector/samples/{x}_{y}_{z}.shp"'.format(x=x.rstrip('.shp'), y=y, z=z)
output_0 = processing.runalg("qgis:randompointsinsidepolygonsvariable", "/home/mydir/vector/{x}".format(x=x), 0, y, 1000, output_alg0)
That way you're running the script directly rather than writing a bit of code then executing it.
Changed my EDIT-ed line written as:
'processing.runalg("qgis:randompointsinsidepolygonsvariable","/home/mydir/vector/{w}",0,"{y}",1000,"/home/mydir/vectorsamples/{x}_{y}_{z}.shp")'.format(w=x, x=x.rstrip('.shp'), y=y, z=z)
to this, which wraps exec with quotations marks around the line above:
exec('processing.runalg("qgis:randompointsinsidepolygonsvariable","/home/mydir/vector/{w}",0,"{y}",1000,"/home/mydir/vectorsamples/{x}_{y}_{z}.shp")'.format(w=x, x=x.rstrip('.shp'), y=y, z=z))
The above modification to the script is running and iterates through the list of .shp files within the specified directory; however, I don't think this is the most kosher answer possible. I'll hold off accepting my own answer if something better and/or more QGIS-friendly comes along.
PS-thanks to #Adam Smith for pointing out exec
I have the following program running
collector.py
data=0
while True:
#collects data
data=data+1
I have another program cool.py which wants to access the current data value. How can I do this?
Ultimately, something like:
cool.py
getData()
*An Idea would be to use a global variable for data?
You can use memory mapping.
http://docs.python.org/2/library/mmap.html
For example you open a file in tmp directore, next u mapping this file to memory in both program and write u data to this file.