Noob question here. I'm scheduling a cron job for a Python script for every 2 hours, but I want the script to stop running after 48 hours, which is not a feature of cron. To work around this, I'm recording the number of executions at the end of the script in a text file using a tally mark x and opening the text file at the beginning of the script to only run if the count is less than n.
However, my script seems to always run regardless of the conditions. Here's an example of what I've tried:
with open("curl-output.txt", "a+") as myfile:
data = myfile.read()
finalrun = "xxxxx"
if data != finalrun:
[CURL CODE]
with open("curl-output.txt", "a") as text_file:
text_file.write("x")
text_file.close()
I think I'm missing something simple here. Please advise if there is a better way of achieving this. Thanks in advance.
The problem with your original code is that you're opening the file in a+ mode, which seems to set the seek position to the end of the file (try print(data) right after you read the file). If you use r instead, it works. (I'm not sure that's how it's supposed to be. This answer states it should write at the end, but read from the beginning. The documentation isn't terribly clear).
Some suggestions: Instead of comparing against the "xxxxx" string, you could just check the length of the data (if len(data) < 5). Or alternatively, as was suggested, use pickle to store a number, which might look like this:
import pickle
try:
with open("curl-output.txt", "rb") as myfile:
num = pickle.load(myfile)
except FileNotFoundError:
num = 0
if num < 5:
do_curl_stuff()
num += 1
with open("curl-output.txt", "wb") as myfile:
pickle.dump(num, myfile)
Two more things concerning your original code: You're making the first with block bigger than it needs to be. Once you've read the string into data, you don't need the file object anymore, so you can remove one level of indentation from everything except data = myfile.read().
Also, you don't need to close text_file manually. with will do that for you (that's the point).
Sounds more for a job scheduling with at command?
See http://www.ibm.com/developerworks/library/l-job-scheduling/ for different job scheduling mechanisms.
The first bug that is immediately obvious to me is that you are appending to the file even if data == finalrun. So when data == finalrun, you don't run curl but you do append another 'x' to the file. On the next run, data will be not equal to finalrun again so it will continue to execute the curl code.
The solution is of course to nest the code that appends to the file under the if statement.
Well there probably is an end of line jump \n character which makes that your file will contain something like xx\n and not simply xx. Probably this is why your condition does not work :)
EDIT
What happens if through the python command line you type
open('filename.txt', 'r').read() # where filename is the name of your file
you will be able to see whether there is an \n or not
Try using this condition along with if clause instead.
if data.count('x')==24
data string may contain extraneous data line new line characters. Check repr(data) to see if it actually a 24 x's.
Related
Is there a way to precurse a write function in python (I'm working with fasta files but any write function that works with text files should work)?
The only way I could think is to read the whole file in as an array and count the number of lines I want to start at and just re-write that array, at that value, to a text file.
I was just thinking there might be a write an option or something somewhere.
I would add some code, but I'm writing it right now, and everyone on here seems to be pretty well versed, and probably know what I'm talking about. I'm an EE in the CS domain and just calling on the StackOverflow community to enlighten me.
From what I understand you want to truncate a file from the start - i.e remove the first n lines.
Then no - there is no way you can do without reading in the lines and ignoring the lines - this is what I would do :
import shutil
remove_to = 5 # Remove lines 0 to 5
try:
with open('precurse_me.txt') as inp, open('temp.txt') as out:
for index, line in enumerate(inp):
if index <= remove_to:
continue
out.write(line)
# If you don't want to replace the original file - delete this
shutil.move('temp.txt', 'precurse_me.txt')
except Exception as e:
raise e
Here I open a file for the output and then use shutil.move() to replace the input file only after the processing (the for loop) is complete. I do this so that I don't break the 'precurse_me.txt' file in case the processing fails. I wrap the whole thing in a try/except so that if anything fails it doesn't try to move the file by accident.
The key is the for loop - read the input file line by line; using the enumerate() function to count the lines as they come in.
Ignore those lines (by using continue) until the index says to not ignore the line - after that simply write each line to the out file.
I've run to a error. I've been trying to append a text file to itself like so:
file_obj = open("text.txt", "a+")
number = 6
def appender(obj, num):
count = 0
while count<=num:
read = file_obj.read()
file_obj.seek(0,2)
file_obj.write(read)
count+=1
appender(file_obj, number)
However, the text.txt file is then filled with strange ASCII symbols. At first, the file contains only a simple "hello", but after the code, it contains this:
hellohello䀀 猀· d娀 Ť搀Ŭ娀ͤ攀ɪ昀Ѥ萀 夀ɚ搀ť樀Ŧ搀茀 婙ݤ攀Ѫ昀ࡤ萀 夀њ搀
ɥ攀ժ昀
茀 婙攀ť樀ɦ搀茀 婙萀 ݚ搀࡚攀४攀ƃ娀搀⡓ 癳 祐桴湯䌠慨慲瑣牥䴠灡楰杮
䌠摯捥挠ㅰ㔲‰敧敮慲整牦浯✠䅍偐义升嘯久佄卒䴯䍉䙓⽔䥗䑎坏⽓偃㈱〵吮员‧楷桴朠湥潣敤祰
മഊ椊 and so on.
Any help will be appreciated
I think I can fix your problem, even though I can't reproduce it. There's a logic error: after you write, you fail to return to the start of the file for reading. In terms of analysis, you failed to do anything to diagnose the problem. At the very least, use a print statement to see what you're reading: that highlights the problem quite well. Here's the loop I used:
count = 0
while count<=num:
file_obj.seek(0) # Read from the beginning of the file.
read = file_obj.read()
print(count, read) # Trace what we're reading.
file_obj.seek(0, 2)
file_obj.write(read)
count+=1
This gives the expected output of 128 (2^(6+1)) repetitions of "hello".
EXTENSIONS
I recommend that you learn to use both the for loop and the with open ... as idiom. These will greatly shorten your program and improve the readability.
I am using this code and everything is working as expected:
with open("file.txt") as f:
for line in f:
f.write(line)
You just have the wrong mode - use 'r+' rather than 'a+'. See this link for a list of modes and an explanation of reading files.
Alternatively, I tried this, but it doesnt seem to get rid of the rows that have blank spaces (blank rows included in the number of rows I'd like to delete). Meanwhile, the code above appears to get rid of those blank spaces, but there is line termination.
next(filecsv) for i in range(10)
Use fileinput.input() with the inplace update file option:
from __future__ import print_function
import fileinput
skip_rows = int(input('How many rows to skip? '))
f = fileinput.input('input.csv', inplace=True)
for i in range(skip_rows):
f.readline()
for row in f:
print(row, end='')
This will skip the first skip_rows rows of the input file and overwrite it without you having to manage writing and moving a temporary file.
(You can omit importing print_function if you are using Python 3)
There are quite a few ways to grab input from a command line tool (which is what I am inferring you wrote). Here are a couple:
Option 1: created in a file called out.py
use sys.argv
import sys
arg1 = sys.argv[1]
print("passed in value: %s" % arg1)
Then run it by passing in an argument (note index 1, script is index 0)
python out.py cell1
passed in value: cell1
Option 2:
A potentially better way is to use a commandline tool framework like click: http://click.pocoo.org/5/. This has almost everything you could ever want to do, and they handle much of the hard logic for you.
You can prompt the user with a simple while loop and listen into standard input or using the input() function.
As to your question on how to delete lines in a file, you can read in the file as a list of lines.
lines=[]
with open('input.txt') as f:
lines=f.readlines()
You can then write back into the file everything past the lines you want to skip by using list slicing.
Also I am pretty sure similar questions have been asked before, try to Google or search Stack Overflow for your question or a subset of your question next time.
P.S.
I also want to add that if you are reading a very large file, it would be better if you read a line at a time, and outputted to a separate file. For a large enough file, you might run out of RAM to hold the file in memory.
Firstly I have opened up the csv file [...]
Did you consider to use pandas to process your data?
If so, pandas.read_csv, allows to skip lines using the skiprows parameter.
You will typically use an iterator to read files. You could do something like this:
numToSkip = 3
with open('somefile.txt') as f:
for i, line in enumerate(f):
if i < numToSkip : continue
# Do 'whatnot' processing here
I have a long string that I want to save in a text file with the code:
taxtfile.write(a)
but because the string is too long, the saved file prints as:
"something something ..... something something"
how do I make sure it will save the entire string without truncating it ?
it should work regardless of the string length
this is the code I made to show it:
import random
a = ''
number_of_characters = 1000000
for i in range(number_of_characters):
a += chr(random.randint(97, 122))
print(len(a)) # a is now 1000000 characters long string
textfile = open('textfile.txt', 'w')
textfile.write(a)
textfile.close()
you can put number_of_characters to whatever number you like but than you must wait for string to be randomized
and this is screenshot of textfile.txt: http://prntscr.com/bkyvs9
probably your problem is in string a.
I think this is just a representation in your IDE or terminal environment. Try something like the following, then open the file and see for yourself if its writing in its entirety:
x = 'abcd'*10000
with open('test.txt', 'w+') as fh:
fh.write(x)
Note the the above will write a file to whatever your current working directory is. You may first want to navigate to your ~/Desktop before calling Python.
Also, how are you building the string a? How is textfile being written? If the call to textfile.write(a) is occurring within a loop, there may be a bug in the loop. (Showing more of your code would help)
The Problem - Update:
I could get the script to print out but had a hard time trying to figure out a way to put the stdout into a file instead of on a screen. the below script worked on printing results to the screen. I posted the solution right after this code, scroll to the [ solution ] at the bottom.
First post:
I'm using Python 2.7.3. I am trying to extract the last words of a text file after the colon (:) and write them into another txt file. So far I am able to print the results on the screen and it works perfectly, but when I try to write the results to a new file it gives me str has no attribute write/writeline. Here it the code snippet:
# the txt file I'm trying to extract last words from and write strings into a file
#Hello:there:buddy
#How:areyou:doing
#I:amFine:thanks
#thats:good:I:guess
x = raw_input("Enter the full path + file name + file extension you wish to use: ")
def ripple(x):
with open(x) as file:
for line in file:
for word in line.split():
if ':' in word:
try:
print word.split(':')[-1]
except (IndexError):
pass
ripple(x)
The code above works perfectly when printing to the screen. However I have spent hours reading Python's documentation and can't seem to find a way to have the results written to a file. I know how to open a file and write to it with writeline, readline, etc, but it doesn't seem to work with strings.
Any suggestions on how to achieve this?
PS: I didn't add the code that caused the write error, because I figured this would be easier to look at.
End of First Post
The Solution - Update:
Managed to get python to extract and save it into another file with the code below.
The Code:
inputFile = open ('c:/folder/Thefile.txt', 'r')
outputFile = open ('c:/folder/ExtractedFile.txt', 'w')
tempStore = outputFile
for line in inputFile:
for word in line.split():
if ':' in word:
splitting = word.split(':')[-1]
tempStore.writelines(splitting +'\n')
print splitting
inputFile.close()
outputFile.close()
Update:
checkout droogans code over mine, it was more efficient.
Try this:
with open('workfile', 'w') as f:
f.write(word.split(':')[-1] + '\n')
If you really want to use the print method, you can:
from __future__ import print_function
print("hi there", file=f)
according to Correct way to write line to file in Python. You should add the __future__ import if you are using python 2, if you are using python 3 it's already there.
I think your question is good, and when you're done, you should head over to code review and get your code looked at for other things I've noticed:
# the txt file I'm trying to extract last words from and write strings into a file
#Hello:there:buddy
#How:areyou:doing
#I:amFine:thanks
#thats:good:I:guess
First off, thanks for putting example file contents at the top of your question.
x = raw_input("Enter the full path + file name + file extension you wish to use: ")
I don't think this part is neccessary. You can just create a better parameter for ripple than x. I think file_loc is a pretty standard one.
def ripple(x):
with open(x) as file:
With open, you are able to mark the operation happening to the file. I also like to name my file object according to its job. In other words, with open(file_loc, 'r') as r: reminds me that r.foo is going to be my file that is being read from.
for line in file:
for word in line.split():
if ':' in word:
First off, your for word in line.split() statement does nothing but put the "Hello:there:buddy" string into a list: ["Hello:there:buddy"]. A better idea would be to pass split an argument, which does more or less what you're trying to do here. For example, "Hello:there:buddy".split(":") would output ['Hello', 'there', 'buddy'], making your search for colons an accomplished task.
try:
print word.split(':')[-1]
except (IndexError):
pass
Another advantage is that you won't need to check for an IndexError, since you'll have, at least, an empty string, which when split, comes back as an empty string. In other words, it'll write nothing for that line.
ripple(x)
For ripple(x), you would instead call ripple('/home/user/sometext.txt').
So, try looking over this, and explore code review. There's a guy named Winston who does really awesome work with Python and self-described newbies. I always pick up new tricks from that guy.
Here is my take on it, re-written out:
import os #for renaming the output file
def ripple(file_loc='/typical/location/while/developing.txt'):
outfile = "output.".join(os.path.basename(file_loc).split('.'))
with open(outfile, 'w') as w:
lines = open(file_loc, 'r').readlines() #everything is one giant list
w.write('\n'.join([line.split(':')[-1] for line in lines]))
ripple()
Try breaking this down, line by line, and changing things around. It's pretty condensed, but once you pick up comprehensions and using lists, it'll be more natural to read code this way.
You are trying to call .write() on a string object.
You either got your arguments mixed up (you'll need to call fileobject.write(yourdata), not yourdata.write(fileobject)) or you accidentally re-used the same variable for both your open destination file object and storing a string.