Develop Reference

Endless loop when importing a variable from another file in Python

I have a file called main.py and based on the user input I want to run the code either from file export.py, either from file import.py. This is how my code from main.py looks:
print("Hi, " + os.getlogin() + "!")
print("Extracting filenames...")
path = input('Tell me the path!\n')
filenames = [x for x in os.listdir(path)]
ID = [f for f in filenames if re.search("(?<=ID)(\d+)", f)]
filestxt = "Files.txt"
idtxt = "ID.txt"
PathFiles = os.path.join(path, filestxt)
PathID = os.path.join(path, idtxt)
file = open(PathFiles, "w+")
file.write('\n'.join(ID))
file.close()
with open(PathID, 'w') as f:
for item in ID:
list = re.findall("(?<=ID)(\d+)", item)
string = ('\n'.join(list))
f.write("%s\n" % string)
key = input("Export or Import(e/i)?:")
if key == 'e':
os.system('python export.py')
When I am hitting the 'e' button Python is running the code from export.py, but when it gets to the line
from main import PathID
instead of importing the variable which I need for the following function
with open(PathID) as f:
for line in f:
...
the code from main.py is running again and again from the beginning and I get the following lines in the console:
"Hi, " + os.getlogin() + "!"
"Extracting filenames..."
'Tell me the path!\n'
"Export or Import(e/i)?:"
All I want in export.py is to tell Python to read the ID.txt file from the path I have specified in the main.py file.
How can I call the function from main.py in export.py without getting this endless loop?

Try to use
if __name__ == '__main__':
before
key = input ("Export or Import (e / i) ?:")
if key == 'e':
os.system ('python export.py')
or before any code that should always be executed at startup.

To expand on Jossnix's answer:
When you execute os.system('python export.py'), you're launching a separate Python interpreter process.
When you execute from main import PathID within export.py, all of the code in main.py is run, and then, once it's finished running, the control flow is returned to export.py and it has access to PathID. The problem is that, as it stands, your main.py asks for user input. So, your main.py is stuck waiting for user input - you have to provide the input again to this new Python interpreter session! Hence, export.py is stuck while trying to import main.
Jossnix's solution works because it ensures that the user input component of main.py does not get run if main.py is being imported from another module, but will be run if main.py is executed as the main script.
I think you should get rid of the os.system('python export.py') line entirely. It's wasteful: you're launching a completely separate Python interpreter session and the print messages in main.py get run again (this is pretty confusing for the end-user!). I'd say you're better off having whatever code you want to run if the user enters the key 'e' wrapped in a function, and then run this function directly from main.py (if the user has indeed entered 'e'). You could do this: Create such a function (f) in export.py, taking a PathID argument. Then, within main.py, from export import f. Finally, if the user entered 'e', run f(PathID, ...).

Script writes in reverse

Sorry if I asked this wrong or formatted it wrong, this is my first time here.
Basically, this script is a very, very, simple text editor. The problem is, when it writes to a file, I want it to write:
Hi, my name
is bob.
But, it writes:
is bob.
Hi, my name
How can I fix this?
The code is here:
import time
import os
userdir = os.path.expanduser("~\\Desktop")
usrtxtdir = os.path.expanduser("~\\Desktop\\PythonEdit Output.txt")
def editor():
words = input("\n")
f = open(usrtxtdir,"a")
f.write(words + '\n')
nlq = input('Line saved. "/n" for new line. "/quit" to quit.\n$ ')
if(nlq == '/quit'):
print('Quitting. Your file was saved on your desktop.')
time.sleep(2)
return
elif(nlq == '/n'):
editor()
else:
print("Invalid command.\nBecause Brendan didn't expect for this to happen,\nthe program will quit in six seconds.\nSorry.")
time.sleep(6)
return
def lowlevelinput():
cmd = input("\n$ ")
if(cmd == "/edit"):
editor()
elif(cmd == "/citenote"):
print("Well, also some help from internet tutorials.\nBut Brendan did all the scripting!")
lowlevelinput()
print("Welcome to the PythonEdit Basic Text Editor!\nDeveloped completley by Brendan*!")
print("Type \"/citenote\" to read the citenote on the word Brendan.\nType \"/edit\" to begin editing.")
lowlevelinput()

Nice puzzle. Why are the lines coming out in reverse? Because of output buffering:
When you write to a file, the system doesn't immediately commit your data to disk. This happens periodically (when the buffer is full), or when the file is closed. You never close f, so it is closed for you when f goes out of scope... which happens when the function editor() returns. But editor() calls itself recursively! So the first call to editor() is the last one to exit, and its output is the last to be committed to disk. Neat, eh?
To fix the problem, it is enough to close f as soon as you are done writing:
f = open(usrtxtdir,"a")
f.write(words + '\n')
f.close() # don't forget the parentheses
Or the equivalent:
with open(usrtxtdir, "a") as f:
f.write(words + '\n')
But it's better to fix the organization of your program:
Use a loop to run editor(), not recursive calls.
An editor should be writing out the file at the end of the session, not with every line input. Consider collecting the user input in a list of lines, and writing everything out in one go at the end.
If you do want to write as you go, you should open the file only once, write repeatedly, then close it when done.

You need to close your file after writing, before you try to open it again. Otherwise your writes will not be finalized until the program is closed.
def editor():
words = input("\n")
f = open(usrtxtdir,"a")
f.write(words + '\n')
nlq = input('Line saved. "/n" for new line. "/quit" to quit.\n$ ')
f.close() # your missing line!
if(nlq == '/quit'):
print('Quitting. Your file was saved on your desktop.')
time.sleep(2)
return
elif(nlq == '/n'):
editor()
else:
print("Invalid command.\nBecause Brendan didn't expect for this to happen,\nthe program will quit in six seconds.\nSorry.")
time.sleep(6)
return

If you replace:
f = open(usrtxtdir,"a")
f.write(words + '\n')
with:
with open(usrtxtdir,"a") as f:
f.write(words + '\n')
It comes out in order. Pretty much always use with open() for file access. It handles the closing of the files for you automatically, even in the event of a crash. Although you might consider taking text in memory and writing it only upon quit. But that's not really part of the problem at hand.

Python's file.write() documentation states: "Due to buffering, the string may not actually show up in the file until the flush() or close() method is called"
Since you're recursively reopening the file and writing to it before closing it (or flushing the buffer), the outer value ('Hi, my name') isn't yet written when the inner frame (where you write 'is bob.') completes, which appears to automatically flush the write buffer.
You should be able to add file.flush() to correct it like this:
import time
import os
userdir = os.path.expanduser("~\\Desktop")
usrtxtdir = os.path.expanduser("~\\Desktop\\PythonEdit Output.txt")
def editor():
words = input("\n")
f = open(usrtxtdir,"a")
f.write(words + '\n')
f.flush() # <----- ADD THIS LINE HERE -----< #
nlq = input('Line saved. "/n" for new line. "/quit" to quit.\n$ ')
if(nlq == '/quit'):
print('Quitting. Your file was saved on your desktop.')
time.sleep(2)
return
elif(nlq == '/n'):
editor()
else:
print("Invalid command.\nBecause Brendan didn't expect for this to happen,\nthe program will quit in six seconds.\nSorry.")
time.sleep(6)
return
def lowlevelinput():
cmd = input("\n$ ")
if(cmd == "/edit"):
editor()
elif(cmd == "/citenote"):
print("Well, also some help from internet tutorials.\nBut Brendan did all the scripting!")
lowlevelinput()
print("Welcome to the PythonEdit Basic Text Editor!\nDeveloped completley by Brendan*!")
print("Type \"/citenote\" to read the citenote on the word Brendan.\nType \"/edit\" to begin editing.")
lowlevelinput()
Also, don't forget to close your file after you're done with it!

How do i output to console while providing input as well?

Essentially, i want what is in this thread: Output to console while preserving user input in ruby, but in Python. I have googled for quite a while, and found an ALMOST working solution, except that it blocked the main thread, as long as i wasn't typing anything in and pressing enter.
Some output of what i don't want to happen is here:
/raw:jtv!jtv#jtv.tmi.twitch.tv PRIVMSG #cobaltstreak :USERCOLOR ullr_son_of_sif #DAA520
Some example input of what i want is:
:jtv!jtv#jtv.tmi.twitch.tv PRIVMSG #cobaltstreak :USERCOLOR ullr_son_of_sif #DAA520
/raw
PRIV:jtv!jtv#jtv.tmi.twitch.tv PRIVMSG #cobaltstreak :SPECIALUSER nightbot subscriber
MSG #cobaltstreak :This shouldn't be here, but on the same line with /raw
This meaning, i want the bottom line of the console to preserve input, while outputting everything happening in the main thread without affecting input.
My current code is:
def console(q, m, lock):
while 1:
raw_input() # After pressing Enter you'll be in "input mode"
with lock:
i = raw_input('> ')
cmd = i.split(' ')[0]
msg = i.strip(cmd + ' ')
q.put(cmd)
m.put(msg)
if cmd == 'quit':
break
as well has:
cmd = cmd_queue.get()
msg = msg_queue.get()
action = cmd_actions.get(cmd)
if action is not None:
action(stdout_lock, msg)
Note the code above is the very first couple of lines in my while loop.
I am on Windows and using python 2.7.6

Python / Pexpect before output out of sync

I'm using Python/Pexpect to spawn an SSH session to multiple routers. The code will work for one router but then the output of session.before will get out of sync with some routers so that it will return the output from a previous sendline. This seems particularly the case when sending a blank line (sendline()). Anyone got any ideas? Any insight would be really appreciated.
Below is a sample of what I'm seeing:
ssh_session.sendline('sh version')
while (iresult==2):
iresult = ssh_session.expect(['>','#','--More--'],timeout=SESSION_TIMEOUT)
debug_print("execute_1 " + str(iresult))
debug_print("execute_bef " + ssh_session.before)
debug_print("execute_af " + ssh_session.after)
thisoutput = ssh_session.before
output += thisoutput
if(iresult==2):
debug_print("exec MORE")
ssh_session.send(" ")
else:
debug_print("exec: end loop")
for cmd in config_commands:
debug_print("------------------------------------------------\n")
debug_print ("running command " + cmd.strip() + "\n")
iresult=2
ssh_session.sendline(cmd.strip())
while (iresult==2):
iresult = ssh_session.expect([prompt+">",prompt+"#"," --More-- "],timeout=SESSION_TIMEOUT)
thisoutput = ssh_session.before
debug_print("execute_1 " + str(iresult))
debug_print("execute_af " + ssh_session.after)
debug_print("execute_bef " + thisoutput)
thisoutput = ssh_session.before
output += thisoutput
if(iresult==2):
debug_print("exec MORE")
ssh_session.send(" ")
else:
debug_print("exec: end loop")
I get this:
logged in
exec: sh version
execute_1 1
execute_bef
R9
execute_af #
exec: end loop
------------------------------------------------
running command config t
execute_1 1
execute_af #
execute_bef sh version
Cisco IOS Software, 1841 Software (C1841-IPBASEK9-M), Version 15.1(4)M4, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport...

I've run into this before with pexpect (and I'm trying to remember how I worked around it).
You can re-synchronize with the terminal session by sending a return and then expecting for the prompt in a loop. When the expect times out then you know that you are synchronized.
The root cause is probably that you are either:
Calling send without a match expect (because you don't care about the output)
Running a command that produces output but expecting for a pattern in the middle of that output and then not to next prompt that is at end of the output. One way to deal with this is to change your expect pattern to "(.+)PROMPT" - this will expect until the next prompt and capture all the output of the command sent (which you can parse in the next step).

I faced a similar problem. I tried waiting for the command to be printed on the screen and the sending enter.
I you want to execute say command 'cmd', then you do:
session.send(cmd)
index = session.expect([cmd, pexpect.TIMEOUT], 1)
session.send('\n')
index = session.expect([whatever you expect])
Worked for me.

I'm not sure this is the root of your problem, but it may be worth a try.
Something I've run into is that when you spawn a session that starts with or lands you in a shell, you have to deal with quirks of the TERM type (vt220, color-xterm, etc.). You will see characters used to move the cursor or change colors. The problem is almost guaranteed to show up with the prompt; the string you are looking for to identify the prompt appears twice because of how color changes are handled (the prompt is sent, then codes to backspace, change the color, then the prompt is sent again... but expect sees both instances of the prompt).
Here's something that handles this, guaranteed to be ugly, hacky, not very Pythonic, and functional:
import pexpect
# wait_for_prompt: handle terminal prompt craziness
# returns either the pexpect.before contents that occurred before the
# first sighting of the prompt, or returns False if we had a timeout
#
def wait_for_prompt(session, wait_for_this, wait_timeout=30):
status = session.expect([wait_for_this, pexpect.TIMEOUT, pexpect.EOF], timeout=wait_timeout)
if status != 0:
print 'ERROR : timeout waiting for "' + wait_for_this + '"'
return False
before = session.before # this is what we will want to return
# now look for and handle any additional sightings of the prompt
while True:
try:
session.expect(wait_for_this, timeout=0.1)
except:
# we expect a timeout here. All is normal. Move along, Citizen.
break # get out of the while loop
return before
s = pexpect.spawn('ssh me#myserver.local')
s.expect('password') # yes, we assume that the SSH key is already there
# and that we will successfully connect. I'm bad.
s.sendline('mypasswordisverysecure') # Also assuming the right password
prompt = 'me$'
wait_for_prompt(s, prompt)
s.sendline('df -h') # how full are my disks?
results = wait_for_prompt(s, prompt)
if results:
print results
sys.exit(0)
else:
print 'Misery. You lose.'
sys.exit(1)

I know this is an old thread, but I didn't find much about this online and I just got through making my own quick-and-dirty workaround for this. I'm also using pexpect to run through a list of network devices and record statistics and so forth, and my pexpect.spawn.before will also get out of sync sometimes. This happens very often on the faster, more modern devices for some reason.
My solution was to write an empty carriage return between each command, and check the len() of the .before variable. If it's too small, it means it only captured the prompt, which means it must be at least one command behind the actual ssh session. If that's the case, the program sends another empty line to move the actual data that I want into the .before variable:
def new_line(this, iteration):
if iteration > 4:
return data
else:
iteration+=1
this.expect(":")
this.sendline(" \r")
data = this.before
if len(data) < 50:
# The numer 50 was chosen because it should be longer than just the hostname and prompt of the device, but shorter than any actual output
data = new_line(this, iteration)
return data
def login(hostname):
this = pexpect.spawn("ssh %s" % hostname)
stop = this.expect([pexpect.TIMEOUT,pexpect.EOF,":"], timeout=20)
if stop == 2:
try:
this.sendline("\r")
this.expect(":")
this.sendline("show version\r")
version = new_line(this,0)
this.expect(":")
this.sendline("quit\r")
return version
except:
print 'failed to execute commands'
this.kill(0)
else:
print 'failed to login'
this.kill(0)
I accomplish this by a recursive command that will call itself until the .before variable finally captures the command's output, or until it calls itself 5 times, in which case it simply gives up.

What is an output file in python

I'm starting to work on problems for google's Code Jam. However I there seams to be a problem with my submission. Whenever I submit I am told "Your output should start with 'Case #1: '". My output a print statement starts with ""Case #%s: %s"%(y + 1, p)" which says Case #1: ext... when I run my code.
I looked into it and it said "Your output should start with 'Case #1: ': If you get this message, make sure you did not upload the source file in place of the output file, and that you're outputting case numbers properly. The first line of the output file should always start with "Case #1:", followed by a space or the end of the line."
So what is an output file and how would I incorporate it into my code?
Extra info: This is my code I'm saving it as GoogleCode1.py and submitting that file. I wrote it in the IDLE.
import string
firstimput = raw_input ("cases ")
for y in range(int(first)):
nextimput = raw_input ("imput ")
firstlist = string.split(nextimput)
firstlist.reverse()
p = ""
for x in range(len(firstlist)):
p = p +firstlist[x] + " "
p = p [:-1]
print "Case #%s: %s"%(y + 1, p)

Run the script in a shell, and redirect the output.
python GoogleCode1.py > GoogleCode1.out

I/O redirection aside, the other way to do this would be to read from and write to various files. Lookup file handling in python
input_file = open('/path/to/input_file')
output_file = open('/path/to/output_file', 'w')
for line in input_file:
answer = myFunction(line)
output_file.write("Case #x: "+str(answer))
input_file.close()
output_file.close()
Cheers

Make sure you're submitting a file containing what your code outputs -- don't submit the code itself during a practice round.