Keyerror when searching through file with same strings. - python

Hello I am having a KeyError when searching through a huge file with multiple serial numbers. The issue occurs when ever I am searching through the file and input a serial number that appears twice within the file. I want to search through this text file and print out the serial numbers I need or the user inputs.
This is the error:
if item[param] != correct:
KeyError: 'APN'
Here is a text example of the file I am searching through:
500,SHOWALL
APN=" "
APU=" "
APP=" "
IPD="127.0.0.1"
DSP=1710
IPU="127.0.0.1"
VWD="2"
600,SHOWALL
APN=""
APU=" "
APP=" "
IPD="127.0.0.1"
DSP=1710
IPU="127.0.0.1"
VWD="2"
700,SHOWALL
APN=" "
APU=" "
APP=" "
IPD="127.0.0.1"
DSP=1710
IPU="127.0.0.1"
VWD="2"
500,SHOWALL
APN=""
APU=" "
APP=" "
IPD="127.0.0.1"
DSP=1710
IPU="127.0.0.1"
VWD="2"
since 500 appears twice in the file it will run into a KeyError.
Here is an example of my code:
def process(infile, outfile, keywords):
keys = [[k[0], k[1], 0] for k in keywords ]
endk = None
with open(infile, "rb") as fdin:
with open(outfile, "ab") as fdout:
fdout.write("<" + words + ">" + "\n")
for line in fdin:
if endk is not None:
fdout.write(line)
if line.find(endk) >= 0:
fdout.write("\n")
endk = None
else:
for k in keys:
index = line.find(k[0])
if index >= 0:
fdout.write(line[index + len(k[0]):].lstrip())
endk = k[1]
k[2] += 1
if endk is not None:
raise Exception(endk + " not found before end of file")
return keys
This is the definition process I use for the input file, output file, and the keywords to search through the file. This is the output portion of the code:
while (count < (len(setNames))):
for number, item in enumerate(lst, 0):
print setNames[count]
for param, correct in correct_parameters.items():
if item[param] != correct:
print('{} = {} which is incorrect'.format(param, item[param]))
with open(compareResults, "ab") as fdout:
fdout.write('{}'.format(setNames[count]) + " " + '{} = {} which is incorrect'.format(param, item[param])+ '\n')
count += 1
My goal would be to some how allow the program to output the result twice if the serial number appears twice or more. So even if 500 appears two or more times within my text file. I still want it to print correct all the 500 results.
Here is a link to my full code. I did not post the full thing because it is very convoluted and needs some cleaning up to do before anything.
http://pastebin.com/aCe9N8vW
If any more information is need I will post below.

It sounds like you were not expecting a KeyError to occur. If that is the case, then using item.get() may only hide the error. Consider using try/except:
try:
if item[param] != correct:
report_incorrect()
except KeyError:
print('Expected key {}'.format(param))
Really you should go back through the code and determine why the key doesn't exist.
I attempted to look at the link you provided, but the code is not yet organized enough for me to easily read. It would help to provide some comments at the top about what it is supposed to do. For example,
"""This code reads an input log file and a configuration file.
For each serial number in the configuration file, it outputs how
many times it occurs in the log file.
"""
Also it would help to organize more things into functions and perhaps classes, again with comments about the goal of each. Perhaps when the code is cleaned up, the problem will go away.

You can try like this,
if item.get(param) != correct:
KeyError:
Raised when a mapping (dictionary) key is not found in the set of existing keys.
If you don't want to have an exception, you can use the get() method

Related

Can two identical strings not be equal to each other in Python?

Below is the code in question:
def search():
name = enter.get()
print(name)
name = name.lower()
data = open("moredata.txt")
found = False
results = str()
for line in data:
record = line.split("|")
record[0] = str(record[0])
print("'" + name + "'" "---" + "'" + record[0] + "'")
if record[0] == name:
found = True
line = str(line)
results += line
results = str(results).lstrip(' ')
continue
else:
found = False
continue
if not found:
print("No results found")
else:
print("These items match your search query:\n")
print(results)
# print(results)
print('\n---------------------------------\n')
In the text file I am using, pieces of information are separated by '|' and the function splits each piece into an array (I think?) and compares just the first value to what I put in the search bar.
I have used other text files with the same exact function, it works just fine: it verifies that the two strings are equal and then displays the entire line of the text file corresponding to what I wanted. However, when I search "a" which should register as equal to "a" from the line a|a|a|a|a|a in my text file, it doesn't. I know this is going to bite me later if I don't figure it out and move on because it works in some cases.
The line
print("'" + name + "'" "---" + "'" + record[0] + "'")
results in
'a'---'a'
'a'---'a'
'a'---'b'
when compared to lines
a|a|a|a|a|a
a|a|a|a|a|a
b|b|b|b|b|b
There are no empty lines between results, and both variable types are str().
The continue in your if statement in the loop is what's causing the problem as it should be a break.
Your program is finding it, but it's then being overwritten by the last iteration. Once your program has found, you should either never set found back to false, or preferably just cease the iteration altogether.
I would wager the other files you've tested with all end with the name that you were looking for, and this one is causing a problem because the name you're looking for isn't at the end.
Additionally, though technically not a problem, the other continue under the else in the for loop isn't necessary.
Your found variable is being overwritten on each iteration of the loop.
Therefore it will only be True if the last result matches.
You don't actually need your found variable at all. If there are results, your results variable will have data in it and you can test for that.

ValueError: not enough values to unpack (expected 11, got 1)

I wrote a script for system automation, but I'm getting the error described in the title. My code below is the relevant portion of the script. What is the problem?
import csv
import os
DIR = "C:/Users/Administrator/Desktop/key_list.csv"
def Customer_List(csv):
customer = open(DIR)
for line in customer:
row = []
(row['MEM_ID'],
row['MEM_SQ'],
row['X_AUTH_USER'],
row['X_AUTH_KEY'],
row['X_STORAGE_URL'],
row['ACCESSKEY'],
row['ACCESSKEYID'],
row['ACCESSKEY1'],
row['ACCESSKEYID1'],
row['ACCESSKEY2'],
row['ACCESSKEYID2'])=line.split()
if csv == row['MEM_ID']:
customer.close()
return(row)
else:
print ("Not search for ID")
return([])
id_input = input("Please input the Customer ID(Email): ")
result = Customer_List(id_input)
if result:
print ("iD: " + id['MEM_ID']
For the line
line.split()
What are you splitting on? Looks like a CSV, so try
line.split(',')
Example:
"one,two,three".split() # returns one element ["one,two,three"]
"one,two,three".split(',') # returns three elements ["one", "two", "three"]
As #TigerhawkT3 mentions, it would be better to use the CSV module. Incredibly quick and easy method available here.
The error message is fairly self-explanatory
(a,b,c,d,e) = line.split()
expects line.split() to yield 5 elements, but in your case, it is only yielding 1 element. This could be because the data is not in the format you expect, a rogue malformed line, or maybe an empty line - there's no way to know.
To see what line is causing the issue, you could add some debug statements like this:
if len(line.split()) != 11:
print line
As Martin suggests, you might also be splitting on the wrong delimiter.
Looks like something is wrong with your data, it isn't in the format you are expecting. It could be a new line character or a blank space in the data that is tinkering with your code.

Brain going to explode. Need to search external file and return muliple results

I'm not going to lie. I'm trying to do an assignment and I'm being beaten by it.
I need to have python prompt the user to enter a room number, then lookup that room number in a supplied .txt file which has csv [comma-separated values], and then show multiple results if there are any.
I was able to get python to return the first result ok, but then it stops. I got around the csv thing by using a hash command and .split (I would rather read it as a csv although I couldn't get it to work.) I had to edit the external file so instad of the data being seperated by commas it was seperated by semicolons, which is not ideal as I am not supposed to be messing with the supplied file.
Anyhow...
My external file looks like this:
roombookings.txt
6-3-07;L1;MSW001;1
6-3-07;L2;MSP201;1
6-3-07;L3;WEB201;1
6-3-07;L4;WEB101;1
6-3-07;L5;WEB101;1
7-3-07;L1;MSW001;2
7-3-07;L2;MSP201;2
7-3-07;L3;WEB201;2
7-3-07;L4;WEB101;2
7-3-07;L5;WEB101;2
8-3-07;L1;WEB101;1
8-3-07;L2;MSP201;3
Here's what my code looks like:
roomNumber = (input("Enter the room number: "))
def find_details(id2find):
rb_text = open('roombookings.txt', 'r')
each_line = rb_text.readline()
while each_line != '':
s = {}
(s['Date'], s['Room'], s['Course'], s['Stage']) = each_line.split(";")
if id2find == (s['Room']):
rb_text.close()
return(s)
each_line = rb_text.readline()
rb_text.close()
room = find_details(roomNumber)
if room:
print("Date: " + room['Date'])
print("Room: " + room['Room'])
print("Course: " + room['Course'])
print("Stage: " + room['Stage'])
If i run the program, I get prompted for a room number. If I enter, say, "L1"
I get:
Date: 6-3-07
Room: L1
Course: MSW001
Stage: 1
I should get 3 positive matches. I guess my loop is broken? Please help me save my sanity!
Edit. I've tried the solutions here but keeps either crashing the program (I guess I'm not closing the file properly?) or giving me errors. I've seriously been trying to sort this for 2 days and keep in mind I'm at a VERY basic level. I've read multiple textbooks and done many Google searches but it's all still beyond me, I'm afraid. I appreciate the assistance though.
Your code does "return(s)" the first time the "id2find" argument is exactly equal to the room.
If you want multiple matches, you could create an empty list before entering the loop, append every match to the list WITHOUT returning, return the list, and then use a for-loop to print out each match.
First. For iterating over lines in the file use next:
for line in rb_text:
# do something
Second. Your function returns after first match. How can it match more then one record? Maybe you need something like:
def find_details(id2find):
rb_text = open('roombookings.txt', 'r')
for line in rb_text:
s = {}
(s['Date'], s['Room'], s['Course'], s['Stage']) = line.split(";")
if id2find == (s['Room']):
yield s
rb_text.close()
And then:
for room in find_details(roomNumber):
print("Date: " + room['Date'])
print("Room: " + room['Room'])
print("Course: " + room['Course'])
print("Stage: " + room['Stage'])
And yes, you better use some CSV parser.
Your problem is the return(s) in find_details(). As soon as you have found an entry, you are leaving the function. You do not even close the file then. One solution is to use an empty list at the beginning, e.g results = [], and then append all entries which matches your requirements (results.append(s)).

Open file and read to find first instance of number. Python

Im stuck on a problem for an assignment, I need to write a program that opens a file on my computer, and scans that file for the first instance of a number. Once it is found it will return
The first number in , filenm is x
otherwise it will say there is no number in filenm.
My code so far is below:
When i run it no matter what it always says theres no number :(
filenm = raw_input("Enter a file name: ")
datain=open(filenm,"r")
try:
c=datain.read(1)
result = []
for line in datain:
c=datain.read(1)
while int(c) >= 0:
c = datain.read(1)
result.append(c)
except:
pass
if len(result) > 0:
print "The first number is",(" ".join(result))+" . "
else:
print "There is no number in" , filenm + "."
That's all you need:
import re
with open("filename") as f:
for line in f:
s=re.search(r'\d+',line)
if s:
print(s.group())
break
open the file;
read it in a loop char-by-char;
check if the char is digit, print whatever you want;
it means there are no numbers in the file, if end-of-file is reached, print "no numbers"
Use <string>.isdigit() method to check if the given string (a single character in your case) is a digit.
I don't recommend mixing iterating through a file
for line in datain:
with using the read method (or any similar one)
c=datain.read(1)
Just stick with one or the other. Personally, I would go with iterating here.
readlines() method returns a list of all the lines in the file. You can then iterate trough the list of characters in each line:
filenm = raw_input("Enter a file name: ")
datain=open(filenm,"r")
try:
result = []
for line in datain.readlines():
print 'line: ' + line
for each in line:
try:
# attempt casting a number to int
int(each)
# if it worked it add it to the result list
result.append(each)
except:
pass
except:
pass
print result
if len(result) > 0:
print "The first number is",(" ".join(result[0]))+". "
else:
print "There is no number in" , filenm + "."
This will only work with the first number character it finds, not sure if you actually need to extract multi digit numbers.
My thoughts:
1) As others noted, don't mask the exception. It would be better to let it be thrown - at least that way you find out what went wrong, if something went wrong.
2) You do want to read the file a line at a time, using for line in file:. The reason for this is that the numbers you want to read are basically "words" (things separated by spaces), and there isn't a built-in way to read the file a word at a time. file.read(1) reads a single byte (i.e. character, for an ASCII file), and then you have to put them together into words yourself, which is tedious. It's much easier to tell Python to split a line into words. At least, I'm assuming here that you're not supposed to pick out the "20" from "spam ham20eggs 10 lobster thermidor".
.readlines() is somewhat redundant; it's a convenience for making a list of the lines in the file - but we don't need that list; we just need the lines one at a time. There is a function defined called .xreadlines() which does that, but it's deprecated - because we can just use for line in file:. Seriously - just keep it simple.
3) int in Python will not return a negative value if the input is non-numeric. It will throw an exception. Your code does not handle that exception properly, because it would break out of the loop. There is no way to tell Python "keep going from where you threw the exception" - and there shouldn't be, because how is the rest of the code supposed to account for what happened?
Actually your code isn't too far off. There are a number of problems. One big one is that the try/except hides errors from you which might have help you figure things out yourself. Another was that you're reading the file with a mixture of a line at a time (and ignoring its contents entirely) as well as a character at a time.
There also seems to be a misunderstand on your part about what the int() function does when given a non-numeric character string, what it does is raise an exception rather than returning something less than 0. While you could enclose a call to it it in a try/except with the except being specifically for ValueError, in this case however it would be easier to just check ahead of time to see if the character is a digit since all you want to do is continue doing that until one that isn't is seen.
So here's one way your code could be revised that would address the above issues:
import string
filenm = raw_input("Enter a file name: ")
datain = open(filenm,"r")
# find first sequence of one or more digits in file
result = []
while True:
c = datain.read(1)
while c in string.digits: # digit?
result.append(c)
c = datain.read(1)
if c == "" or len(result) > 0: # end-of-file or a number has been found
break # get out of loop
if len(result) > 0:
print "The first number is'", "".join(result) + "'."
else:
print "There is no number in'", filenm + "'."
close(datain)

Retrieving Values with p.expect

I have a program that logs into a server and issues commands. The results are printed out at the end of the script. The below code shows the script I have created to pass commands through ssh.
import pexpect
ssh_newkey = 'Are you sure you want to continue connecting'
# my ssh command line
p=pexpect.spawn('ssh user#00.00.00.00')
i=p.expect([ssh_newkey,'password:',pexpect.EOF])
if i==0:
print "I say yes"
p.sendline('yes')
i=p.expect([ssh_newkey,'password:',pexpect.EOF])
if i==1:
print "I have entered the password. I will now flip camera through ",
p.sendline("user")
i=p.expect('user#hol-NA:')
p.sendline("cd /opt/ad/bin")
i=p.expect('user#hol-NA:')
p.sendline("./ptzflip")
i=p.expect('user#hol-NA:')
elif i==2:
print "I either got key or connection timeout"
pass
results = p.before # print out the result
print results
The results that the program prints out is:
Value = 1800
Min = 0
Max = 3600
Step = 1
I want to capture the values that are printed out.
In reponse to the questions below. I want to capture eg. 'Value' as a variable and '1800' as its value. I have tried to separate it in a dictionary as mentioned below but I get an error. When I enter:
results_dict = {}
for line in results:
name, val = line.split(' = ')
results_dict[name] = val
I get an error:
Traceback (most recent call last):
File "ptest.py", line 30, in <module>
name, val = line.split(' = ')
ValueError: need more than 1 value to unpack
When I check this code in Python it stores these values as a string. It stores it as:
'/opt/ad/bin$ ./ptzflip\r\nValue = 1800\r\nMin = 0\r\nMax = 3600\r\nStep = 1\r\n'
Can anyone help in this problem. Thanks
Are Value = 1800 etc. the contents of results? And you want to "capture" that?
Do you mean you want to parse those results? Or execute them as python?
If the former you could do something like (untested, unclean, doesn't deal carefully with whitespace):
results_dict = {}
for line in results.splitlines():
try:
name, val = line.split(' = ')
except ValueError:
continue
results_dict[name] = val
This gives you a python dictionary that you can use. If you know that the values are always numbers, you could convert them with int(val) or float(val)... (The try...except ignores lines of the incorrect form; there may be more robust ways to do this, such as if " = " in line)
If you actually want to end up with a variable named Value with the value 1800, you could use eval(results) [or a safer alternative], although this would need to remove lines without the right format first.
do you mean save it to a file?? then try this
open("output.txt","w").write(results)
or when you run the script on the command line:
$ python script.py > output.txt
otherwise, define what you mean by "capture"
After each call to p.expect returns, whatever the child process emitted just before the matched part (any number of lines), and the matched part itself, are accessible as the before and after properties of p -- that's your "capture" for you!
I.e., as the docs put it:
After each call to expect() the before
and after properties will be set to
the text printed by child application.
The before property will contain all
text up to the expected string
pattern. The after string will contain
the text that was matched by the
expected pattern. The match property
is set to the re MatchObject.
The example code in the docs right after this part can be helpful to understand this -- after establishing an FTP session with the interactive ftp client,
child.sendline('ls /pub/OpenBSD/')
child.expect('ftp> ')
print child.before # Print the result of the ls command.
Whenever this happens to me, it's because the string I'm splitting does not have the value I expected. Why not look at it directly?
for line in results:
print line
name, val = line.split(' = ')
results_dict[name] = val
If it were what you think it is, this would work (now in the interpreter):
>>> 'Value = 1800'.split(' = ')
['Value', '1800']

Categories

Resources