Develop Reference

Python delete line and write over it

i have proplem here with my code
i am using for loop to save results in txt file
every time i use it it's saving the last line and delete the old line
here is code:
for i in domains:
requests.get(i)
if req.status_code == 200:
print('[+]:', i + '/')
file = open(save,'w')
file.write(domain)
file.close()
so after run the code it's save the last domain or last results and delete the old results
so i want solution
thanks.

Open the file in append mode:
open(filename,'a')
Your code should be:
for i in domains:
requests.get(i)
if req.status_code == 200:
print('[+]:', i + '/')
file = open(save,'a')
file.write(i)
file.close()

Try open it with a instead of w, so:
file = open(save, 'a')
it should appends to the file instead of writing to it

You sould use 'a' instead of 'w' in "file = open(save,'w')"

The problem here is arising because you are opening the file for writing on every iteration of the loop. Because you are opening the file with mode 'w', the file is being overwritten every time. (See documentation.)
It would be better to open the file once only, outside the loop.
For example:
file = open(save, 'w')
for i in domains:
# do some stuff...
file.write(some_data)
file.close()
You can also use with in conjuction with the open function:
with open(save, 'w') as file:
for i in domains:
# do some stuff...
file.write(some_data)
In this case, there is no need for an explicit call to file.close(). This will be done automatically when the loop ends, or even if something happens inside the with block like an exception being raised (i.e. error occurring) or a return from the function (if applicable).
Be aware that because of output buffering, data written to the file during one iteration might not actually get written to the file until some later iteration or after the file is closed. If this would be a problem, then you can insert a flush call to force a low-level write so that other processes can see the data:
with open(save, 'w') as file:
for i in domains:
# do some stuff...
file.write(some_data)
file.flush()
If there is some reason why you need to open the file inside the loop, then you could open it in append mode using 'a', and the data will be written after the end of the existing file. (Again, see the documentation linked earlier.) For example:
for i in domains:
# do some stuff...
file = open(save, 'a')
file.write(some_data)
file.close()
or again, better like this:
for i in domains:
with open(save, 'a') as file:
# do some stuff...
file.write(some_data)
Be aware that if the output file already exists before the program is started, then the contents of the existing file will remain because you are always opening it in append mode. Depending on your application, this might or might not be desirable.

Pytsk - Sending files to a server from a disk image

I am trying to send each file from a disk image to a remote server using paramiko.
class Server:
def __init__(self):
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.ssh.connect('xxx', username='xxx', password='xxx')
def send_file(self, i_node, name):
sftp = self.ssh.open_sftp()
serverpath = '/home/paul/Testing/'
try:
sftp.chdir(serverpath)
except IOError:
sftp.mkdir(serverpath)
sftp.chdir(serverpath)
serverpath = '/home/Testing/' + name
sftp.putfo(fs.open_meta(inode = i_node), serverpath)
However when I run this I get an error saying that "pytsk.File has no attribute read".
Is there any other way of sending this file to the server?

After a quick investigation I think I found what your problem is. Paramiko's sftp.putfo expects a Python file object as the first parameter. The file object of Pytsk3 is a completely different thing. Your sftp object tries to perform "read" on this, but Pytsk3 file object does not have a method "read", hence the error.
You could in theory try expanding Pytsk3.File class and adding this method but I would not hold my breath that it actually works.
I would just read the file to a temporary one and send that. Something like this (you would need to make temp file name handling more clever and delete the file afterwards but you will get the idea):
serverpath = '/home/Testing/' + name
tmp_path = "/tmp/xyzzy"
file_obj = fs.open_meta(inode = i_node)
# Add here tests to confirm this is actually a file, not a directory
tha = open(tmp_path, "wb")
tha.write(file_obj.read_random(0, file_obj.info.meta.size))
tha.close()
rha = open(tmp_path, "rb")
sftp.putfo(rha, serverpath)
rha.close()
# Delete temp file here
Hope this helps. This will read the whole file in memory from fs image to be written to temp file, so if the file is massive you would run out of memory.
To work around that, you should read the file in chunks looping through it with read_random in suitable chunks (the parameters are start offset and amount of data to read), allowing you to construct the temp file in a chunks of for example a couple of megabytes.
This is just a simple example to illustrate your problem.
Hannu

Delete a file after reading

In my code, user uploads file which is saved on server and read using the server path. I'm trying to delete the file from that path after I'm done reading it. But it gives me following error instead:
An error occurred while reading file. [WinError 32] The process cannot access the file because it is being used by another process
I'm reading file using with, and I've tried f.close() and also f.closed but its the same error every time.
This is my code:
f = open(filePath)
with f:
line = f.readline().strip()
tempLst = line.split(fileSeparator)
if(len(lstHeader) != len(tempLst)):
headerErrorMsg = "invalid headers"
hjsonObj["Line No."] = 1
hjsonObj["Error Detail"] = headerErrorMsg
data['lstErrorData'].append(hjsonObj)
data["status"] = True
f.closed
return data
f.closed
after this code I call the remove function:
os.remove(filePath)
Edit: using with open(filePath) as f: and then trying to remove the file gives the same error.

Instead of:
f.closed
You need to say:
f.close()
closed is just a boolean property on the file object to indicate if the file is actually closed.
close() is method on the file object that actually closes the file.
Side note: attempting a file delete after closing a file handle is not 100% reliable. The file might still be getting scanned by the virus scanner or indexer. Or some other system hook is holding on to the file reference, etc... If the delete fails, wait a second and try again.

Use below code:
import os
os.startfile('your_file.py')
To delete after completion:
os.remove('your_file.py')

This
import os
path = 'path/to/file'
with open(path) as f:
for l in f:
print l,
os.remove(path)
should work, with statement will automatically close the file after the nested block of code
if it fails, File could be in use by some external factor. you can use Redo pattern.
while True:
try:
os.remove(path)
break
except:
time.sleep(1)

There is probably an application that is opening the file; check and close the application before executing your code:
os.remove(file_path)
Delete files that are not used by another application.

how to get (txt) file content from FileField?

I have a problem with opening a text file in django that is stored in my database. I want to access it via FileField of my model... the model looks something like this
class MyModel(models.Model):
saved_file = FileField()
I upload a test file via admin interface, which works ok. In my view I want to access this file. If I open it with standard python open() it works ok...
f = open(path, 'r')
a = f.readlines()
return render_to_response('base.html', {'content': a}, context_instance=RequestContext(request))
this displays lines of the file ok...
according to https://docs.djangoproject.com/en/dev/ref/models/fields/#filefield one gets a FieldFile proxy when FileField from a model is called, so
f = MyModel.objects.all().get(id=0).saved_file
should store FieldFile in f, furthermore documentation states that one opens a file from model by casting .open(mode='rb') on FieldFile, so
file = f.open(mode='rb')
should work like python .open() as stated in the documentation. So to get lines I do
file.readlines()
should return me list of lines. What happens is that I get an error saying that .readlines() atribute does not exist. I do not need the file to display it, this is just a way to test if opening a file works, but I need the file content in a variable in my view to further use it in my business logic.
Could anyone suggest a way to get the file content out of a FileField frem a model?

The documentation states that files are opened in 'rb' mode by default, but you would want to open in 'r' to treat the file as a text file:
my_object = MyModel.objects.get(pk=1)
try:
my_object.saved_file.open('r')
lines = my_object.saved_file.readlines()
finally:
my_object.saved_file.close()
Even better, you can use a context manager in Django v2.0+
my_object = MyModel.objects.get(pk=1)
with my_object.saved_file.open('r') as f:
lines = f.readlines()

Since Django 2.0 File.open does return you a file, so suggested way to work is as with context manager:
saved_file = MyModel.objects.all().get(id=0).saved_file
with saved_file.open() as f:
data = f.readlines()
Applies to old versions of Django < 2
FieldFile.open opens the file, but doesn't return anything. So in your example file is None.
You should call readlines on FieldFile. In your example it would be:
f = MyModel.objects.all().get(id=0).saved_file
try:
f.open(mode='rb')
lines = f.readlines()
finally:
f.close()
UPD: I added try/finally block, as the good practice is to always close the resource, even if exception happened.

File management

I am working on python and biopython right now. I have a file upload form and whatever file is uploaded suppose(abc.fasta) then i want to pass same name in execute (abc.fasta) function parameter and display function parameter (abc.aln). Right now i am changing file name manually, but i want to have it automatically.
Workflow goes like this.
----If submit is not true then display only header and form part
--- if submit is true then call execute() and get file name from form input
--- Then displaying result file name is same as executed file name but only change in extension
My raw code is here -- http://pastebin.com/FPUgZSSe
Any suggestions, changes and algorithm is appreciated
Thanks

You need to read the uploaded file out of the cgi.FieldStorage() and save it onto the server. Ususally a temp directory (/tmp on Linux) is used for this. You should remove these files after processing or on some schedule to clean up the drive.
def main():
import cgi
import cgitb; cgitb.enable()
f1 = cgi.FieldStorage()
if "dfile" in f1:
fileitem = f1["dfile"]
pathtoTmpFile = os.path.join("path/to/temp/directory", fileitem.filename)
fout = file(pathtoTmpFile, 'wb')
while 1:
chunk = fileitem.file.read(100000)
if not chunk: break
fout.write (chunk)
fout.close()
execute(pathtoTmpFile)
os.remove(pathtoTmpFile)
else:
header()
form()
This modified the execute to take the path to the newly saved file.
cline = ClustalwCommandline("clustalw", infile=pathToFile)
For the result file, you could also stream it back so the user gets a "Save as..." dialog. That might be a little more usable than displaying it in HTML.