I'm a chemistry student and want to write a script to extract some data (like coupling constants and interproton distance) from gaussian output files.
I found a script which extracts chemical shifts from gaussian output files. However, I don't understand what does if file.find('freq-') !=-1 mean in the script.
Here's part of the script (since the script also does other things as well so I've just sown the bit relevant to my question):
def read_gaussian_freq_outfiles(list_of_files):
list_of_freq_outfiles = []
for file in list_of_files:
if file.find('freq-') !=-1:
list_of_freq_outfiles.append([file,int(get_conf_number(file)),open(file,"r").readlines()])
return list_of_freq_outfiles
def read_gaussian_outputfiles():
list_of_files = []
for file in glob.glob('*.out'):
list_of_files.append(file)
return list_of_files
I think in the def read_gaussian_outputfiles() bit, we create a list of file and simply add all file with extension '.out' to the list.
The read_gaussian_freq_outfiles(list_of_files) bit has probably list files which has "freq-" in the file name. But what does the file.find('freq-')!=-1 mean?
Does it mean if whatever we find in the file name doesn't equal to -1, or something else?
Some other additional information: the format of the gaussian output filename is: xxxx-opt_freq-conf-yyyy.out where xxxx is the name of your molecule and yyyy is a number.
When s.find(foo) fails to find foo in s, it returns -1. Therefore, when s.find(foo) does not return -1, we know it didn't fail.
read_gaussian_freq_outfiles looks for the term "freq-" in each of the names of files in list_of_files. If it succeeds in finding this phrase in the name of a file, it appends a list containing this file, a "conf number" (not sure what this is), and the contents of the file, to a list called list_of_freq_outfiles.
I created three files, goodbye.txt, hello.txt, and helloworld.txt to demonstrate usage.
In this example, I'll print all files that end with .txt, create a list of files, then print all files that have the phrase "goodbye" in the filename. This should only print goodbye.txt.
09:53 $ ls
goodbye.txt hello.txt helloworld.txt
(venv) ✔ ~/Desktop/ex
09:53 $ python
Python 2.7.11 (default, Dec 5 2015, 14:44:47)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import glob
>>> for file in glob.glob('*.txt'):
... print(file)
...
goodbye.txt
hello.txt
helloworld.txt
>>> list_of_files = [ file for file in glob.glob('*.txt') ]
>>> print(list_of_files)
['goodbye.txt', 'hello.txt', 'helloworld.txt']
>>> for file in list_of_files:
... if file.find('goodbye') != -1:
... print(file)
...
goodbye.txt
Indeed, goodbye.txt is the only file printed.
As the other answers also show: if .find() retrieves -1, it cannot find what you're looking for. This has to do with the fact that .find will return the first index at which it can find your query. So in the following sentence
The cat is on the mat
and sentence.find('cat'), it will return 4 (since 'cat' starts at index 4 (it starts at 0!)).
However, sentence.find('dog') will return the only thing it can return if it cannot find it: -1. If it returned 0 as the 'cannot find', you might think your query starts at index 0. With -1, you know it could not find it.
String find method in python looks at the occurrence of a sub-string in a given string (ref http://www.tutorialspoint.com/python/string_find.htm)
Here it is looking for all the filenames with 'freq-' sub-string in them.
Related
I was working on a small script in Python where I had to traverse through the directories which have multiple types of files, but I want to open only text files. So how can I do that? Below is my code.
import os,re
pat=re.compile(input("Enter the text you want to search for : "))
fpath=r'C:\Users\Python\Python_my_Scripts\'
for i in os.walk(fpath):
for fname in i[-1]:
fpath=os.path.join(i[0],fname)
try:
IN=open(fpath,"r")
except Exception as e:
print(e)
else:
line_num=0
for line in IN:
line_num+=1
if not re.search(r'^\s+#',line):
if re.search(pat, line):
print("{1:>2d} : {0}".format(fpath,line_num))
The code basically breaks in the try segment if a directory contains any non-text file.
Use glob to get a list of filenames by pattern:
import glob
glob.glob('*.txt')
Using python-magic you can check the file type, in the same way you would using the file command. You can then check the output from magic.from_file to see whether or not the file is a text file.
>>> import magic
>>> magic.from_file("/bin/bash")
'ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=75a0ba19d5276d9eb81d6f8e9e2cb285da333296, stripped'
>>> magic.from_file("/etc/fstab")
'ASCII text'
>>> if 'text' in magic.from_file("/etc/fstab").lower():
... print("a text file...")
...
a text file...
>>>
Iterate over files with os.walk or get files with glob packages and check whether you file is binary or text for that this might be helpful to you, How can I detect if a file is binary (non-text) in python?.
I've looked at a few posts regarding this topic but haven't seen anything specifically fitting my usage. This one looks to be close to the same issue. They talk about escaping the escape characters.
My issue is that I want this script to run on both MAC and PC systems - and to be able to process files in a sub-folder that has spaces in the folder name.
Right now I use this code gleaned partially from several different SO posts:
directory = raw_input("Enter the location of the files: ")
path = r"%s" % directory
for file in os.listdir(path):
I don't quite understand what the second line is doing - so perhaps that's the obvious line that needs tweaking. It works fine with regular folder names but not with spaces in the name.
I have tried using "\ " instead of just " ", - that didn't work - but in any case I'm looking for a code solution. I don't want the user to have to specify an escape character
On Windoze using sub folder name "LAS Data" in response to raw_input prompt (without quotation marks), I get an message that:
the system can't find the path specified "LAS Data\*.*"
Your problem is most likely not with the code you present, but with the input you gave. The message
the system can't find the path specified "LAS Data\*.*"
suggest that you (or the user) entered the directory together with a wildcard for files. A directory with the name "LAS Data\*.*" indeed does not exist (unless you did something special to your file system :-).
Try entering just "LAS Data" instead.
The raw string should not be needed either
> python
Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> directory = raw_input("Enter the location of the files: ")
Enter the location of the files: Directory with Spaces
>>> for file in os.listdir(directory):
... print(file)
...
hello-1.txt
hello-2.txt
>>>
I don't quite understand what the [path = r"%s" % directory] line is doing
It creates a copy path of directory:
>>> directory = raw_input()
LAS Data
>>> path = r"%s" % directory
>>> path == directory
True
>>> type(path), type(directory)
(<type 'str'>, <type 'str'>)
Really wondering where you got that from. Seems pretty pointless.
The below code shows that three files which are on a UNC share hosted on another machine have the same hash. It also shows that local files have different hashes. Why would this be? I feel that there is some UNC consideration that I don't know about.
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> fn_a = '\\\\some.host.com\\Shares\\folder1\\file_a'
>>> fn_b = '\\\\some.host.com\\Shares\\folder1\\file_b'
>>> fn_c = '\\\\some.host.com\\Shares\\folder2\\file_c'
>>> fn_d = 'E:\\file_d'
>>> fn_e = 'E:\\file_e'
>>> fn_f = 'E:\\folder3\\file_f'
>>> f_a = open(fn_a, 'r')
>>> f_b = open(fn_b, 'r')
>>> f_c = open(fn_c, 'r')
>>> f_d = open(fn_d, 'r')
>>> f_e = open(fn_e, 'r')
>>> f_f = open(fn_f, 'r')
>>> hashlib.md5(f_a.read()).hexdigest()
'54637fdcade4b7fd7cabd45d51ab8311'
>>> hashlib.md5(f_b.read()).hexdigest()
'54637fdcade4b7fd7cabd45d51ab8311'
>>> hashlib.md5(f_c.read()).hexdigest()
'54637fdcade4b7fd7cabd45d51ab8311'
>>> hashlib.md5(f_d.read()).hexdigest()
'd2bf541b1a9d2fc1a985f65590476856'
>>> hashlib.md5(f_e.read()).hexdigest()
'e84be3c598a098f1af9f2a9d6f806ed5'
>>> hashlib.md5(f_f.read()).hexdigest()
'e11f04ed3534cc4784df3875defa0236'
EDIT: To further investigate the problem, I also tested using a file from another host. It appears that changing the host will change the result.
>>> fn_h = '\\\\host\\share\\file'
>>> f_h = open(fn_h, 'r')
>>> hashlib.md5(f_h.read()).hexdigest()
'f23ee2dbbb0040bf2586cfab29a03634'
...but then I tried a different file on the new host, and got a new result!
>>> fn_i = '\\\\host\\share\\different_file'
>>> f_i = open(fn_i, 'r')
>>> hashlib.md5(f_i.read()).hexdigest()
'a8ad771db7af8c96f635bcda8fdce961'
So, now I'm really confused. Could it have something to do with the fact that the original host is a \\host.com format and the new host is a \\host format?
I did some additional research based on the comments and answers everyone provided. I decided I needed to study permutations of these two features of the code:
A raw string literal is used for the path name, i.e. whether or not:
A. The file path string is raw with single backslashes in the path, vs.
B. The file path string is not raw with double backslashes in the path
(FYI to those who don't know, a raw string is one which is proceeded by an "r" like this: r'This is a raw string')
The open function mode is r or rb.
(FYI again to those who don't know, the b in rb mode indicates to read the file as binary.)
The results demonstrated:
The string literal / backslashes make no difference in whether or not the hashes of different files are different
My error was not opening the file in binary mode. When using rb mode in open, I got different results.
Yay! And thanks for the help.
Use f1.seek(0) if you intend to use it again, otherwise it would be a file completely read and calling read() again would just return a empty string.
I don't reproduce your problem. I'm using Python 3.4 on Windows 7 here with the following test script which accesses files on a network hard disk:
import sys, hashlib
def main():
fn0 = r'\\NAS\Public\Software\Backup\Test\Vagrantfile'
fn1 = r'\\NAS\Public\Software\Backup\Test\z.xml'
with open(fn0, 'rb') as f:
h0 = hashlib.md5(f.read())
print(h0.hexdigest())
with open(fn1, 'rb') as f:
h1 = hashlib.md5(f.read())
print(h1.hexdigest())
if __name__ == '__main__':
sys.exit(main())
Running this results in two different hash values (as expected):
c:\src\python>python hashtest.py
8af202dffb88739c2dbe188c12291e3d
2ff3db61ff37ca5ceac6a59fd7c1018b
If reading the file contents returns different data for the remote files then passing that data into md5 has to result in different hash values. You might want to print out the first 80 bytes of each file as a check that you are getting what you expect.
I'm trying to solve exercise 15's extra credit questions of Zed Shaw's Learn Python the Hard Way but I've ran into a problem. The code is as follows:
from sys import argv
script, filename = argv
txt = open(filename)
print "Here's your file %r:" % filename
print txt.read()
print "I'll also ask you to type it again:"
file_again = raw_input("> ")
txt_again = open(file_again)
print txt_again.read()
print txt_again.read()
I understand all the code that has been used, but extra credit question 7 asks:
Startup python again and use open from the prompt. Notice how you can open files and run read on them right there?
I've tried inputting everything I could think of in terminal (on a mac) after first starting up python with the 'python' command, but I can't get the code to run. What should I be doing to get this piece of code to run from the prompt?
Zed doesn't say to run this particular piece of code from within Python. Obviously, that code is getting the filename value from the parameters you used to invoke the script, and if you're just starting up the Python shell, you haven't used any parameters.
If you did:
filename = 'myfilename.txt'
txt = open(filename)
then it would work.
I just started with open(xyz.txt)
Well, yes, of course that isn't going to work, because you don't have a variable xyz, and even if you did, it wouldn't have an attribute txt. Since it's a file name, you want a string "xyz.txt", which you create by putting it in quotes: 'xyz.txt'. Notice that Python treats single and double quotes more or less the same; unlike in languages like C++ and Java, there is not a separate data type for individual characters - they're just length-1 strings.
Basically, just like in this transcript (I've added blank lines to aid readability):
pax:~$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> xyz = open ("minimal_main.c")
>>> print xyz.read()
int main (void) {
return 0;
}
>>> xyz.close()
>>> <CTRL-D>
pax:~$ _
All it's showing you is that you don't need a script in order to run Python commands, the command line interface can be used in much the same way.
print open('ex15_sample.txt').read()
After running python in terminal, we'll use open('filename.txt') to open the file and using the dot operator we can apply the read() function directly on it.
After running Python in terminal,
abc = open ("ex15_sample.txt")
print abc.read()
That should do.
I'm trying to implement a simple method to read new lines from a log file each time the method is called.
I've looked at the various suggestions both on stackoverflow (e.g. here) and elsewhere for simulating "tail" functionality; most involve using readline() to read in new lines as they're appended to the file. It should be simple enough, but can't get it to work properly on OS X 10.6.4 with the included Python 2.6.1.
To get to the heart of the problem, I tried the following:
Open two terminal windows.
In one, create a text file "test.log" with three lines:
one
two
three
In the other, start python and execute the following code:
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.stat('test.log')
posix.stat_result(st_mode=33188, st_ino=23465217, st_dev=234881025L, st_nlink=1, st_uid=666, st_gid=20, st_size=14, st_atime=1281782739, st_mtime=1281782738, st_ctime=1281782738)
>>> log = open('test.log')
>>> log.tell()
0
>>> log.seek(0,2)
>>> log.tell()
14
>>>
So we see with the tell() that seek(0,2) brought us to the end of the file as reported by os.stat(), byte 14.
In the first shell, add another two lines to "test.log" so that it looks like this:
one
two
three
four
five
Go back to the second shell, and execute the following code:
>>> os.stat('test.log')
posix.stat_result(st_mode=33188, st_ino=23465260, st_dev=234881025L, st_nlink=1, st_uid=666, st_gid=20, st_size=24, st_atime=1281783089, st_mtime=1281783088, st_ctime=1281783088)
>>> log.seek(0,2)
>>> log.tell()
14
>>>
Here we see from os.stat() that the file's size is now 24 bytes, but seeking to the end of the file somehow still points to byte 14?? I've tried the same on Ubuntu with Python 2.5 and it works as I expect. I tried with 2.5 on my Mac, but got the same results as with 2.6.
I must be missing something fundamental here. Any ideas?
How are you adding two more lines to the file?
Most text editors will go through operations a lot like this:
fd = open(filename, read)
file_data = read(fd)
close(fd)
/* you edit your file, and save it */
unlink(filename)
fd = open(filename, write, create)
write(fd, file_data)
The file is different. (Check it with ls -li; the inode number will change for almost every text editor.)
If you append to the log file using your shell's >> redirection, it'll work exactly as it should:
$ echo one >> test.log
$ echo two >> test.log
$ echo three >> test.log
$ ls -li test.log
671147 -rw-r--r-- 1 sarnold sarnold 14 2010-08-14 04:15 test.log
$ echo four >> test.log
$ ls -li test.log
671147 -rw-r--r-- 1 sarnold sarnold 19 2010-08-14 04:15 test.log
>>> log=open('test.log')
>>> log.tell()
0
>>> log.seek(0,2)
>>> log.tell()
19
$ echo five >> test.log
$ echo six >> test.log
>>> log.seek(0,2)
>>> log.tell()
28
Note that the tail(1) command has an -F command line option to handle the case where the file is changed, but a file by the same name exists. (Great for watching log files that might be periodically rotated.)
Short answer: no, your assumptions are.
Your text editor is creating a new file with the same name, not modifying the old file in place. You can see in your stat result that the st_ino is different. If you were to do os.fstat(log.fileno()), you'd get the old size and old st_ino.
If you want to check for this in your implementation of tail, periodically compare the st_ino of the stat and fstat results. If they differ, there's a new file with the same name.