I am trying to open a file in read mode using Python. The error I am receiving suggest I am using the won filename or read mode. When I type the file path into my computer, it works. I tried to assign the input filename to a variable and then opening the variable in read mode. I also tried typing in the full path and opening the path in read mode. Both game me an error.
Code:
workingDirec = raw_input("What is the working directory?")
original_file = raw_input("The input filename is?")
def calculateZscore():
"Z score calc"
full_original = os.path.join(workingDirec,original_file)
print full_original
f = open ('C:\Users\tpmorris\ProgramingAndScripting\Trial 2 Data\Trial 2 Data\NCSIDS_ObsExp.txt','r')
print f
My results:
Using full path output:
What is the working directory?C:\Users\tpmorris\ProgramingAndScripting\Trial 2 Data\Trial 2 Data
The input filename is?NCSIDS_ObsExp.txt
C:\Users\tpmorris\ProgramingAndScripting\Trial 2 Data\Trial 2 Data\NCSIDS_ObsExp.txt
IOError: [Errno 22] invalid mode ('r') or filename: 'C:\Users\tpmorris\ProgramingAndScripting\Trial 2 Data\Trial 2 Data\NCSIDS_ObsExp.txt'
Using variable output:
IOError: [Errno 2] No such file or directory: 'full_original'
On Windows your paths must be escaped because Windows uses backslashes \ to denote path separators.
Backslashes however are typically used as escape sequences and are used in Python as such as well! So you have to "escape" them like this:
f = open ('C:\\Users\\tpmorris\\ProgramingAndScripting\\Trial 2 Data\\Trial 2 Data\\NCSIDS_ObsExp.txt','r')
See:
Windows path in Python
Using Python on Windows
Firstly , on Windows, you must escape backslashes (double backslash) if you are going to use Windows path syntax, for the reasons pointed out by #James Mills answer.
Another option is to use forward slashes; Python will interpret these correctly in os.path.
You could use as your command line path input:
C:/Users/tpmorris/ProgramingAndScripting/Trial 2 Data/Trial 2 Data
Or add
/NCSIDS_ObsExp.txt
to the above if you were going to use a hardcoded path.
Also you should make some small changes to your code if you want to print the contents of your text file:
First, your file open should be done using a with statement. This will ensure the file object's built in __enter__ and __exit__ methods get called, in particular, if you forget to close the file when you are done after you've opened it.
See Understanding Python's with statement for more.
Second, if you want to print each line in your text file, don't try to print the file object. Rather loop through the lines and print them.
So your code for accepting command line input should be:
import os
workingDirec = raw_input("What is the working directory?")
original_file = raw_input("The input filename is?")
full_original = os.path.join(workingDirec,original_file)
print full_original
with open(full_original,'r') as f:
for line in f:
print line
f.close()
I removed the def of a function to do something else in the midst of your file read code. That def should go elsewhere.
Related
I trying to open and read a file with open() in python, with the global variable $USER in Linux, but the program stops in the 2 line. I want to believe that the problem is in the open() function because I use $USER in the 1 line and all is ok:
os.system("/usr/bin/nmap {target} -oN /home/$USER/.nmap_diff/scan{today}.txt")
scantxt = open("/home/$USER/.nmap_diff/scan{today}.txt","rb")
The output is:
File "diffscanner.py", line 92, in scanner
scantxt = open("/home/$USER/.nmap_diff/scan{}.txt".format(today),"rb")
FileNotFoundError: [Errno 2] No such file or directory: '/home/$USER/.nmap_diff/scan2021-07-10.txt'
The output said the scan2021-07-10.txt has not found, but it really exist:
scan2021-07-10.txt
os.system executes the command(passed as string) in a subshell. That means, the command will have access to the Linux's environment variables, USER in your case.
On the other hand, open expects a path-like object such as a path string. The string is read as it is and not evaluated to replace USER(or any other environment variable) with actual values. If you want to use env var, use os.environ
import os
USER = os.environ['USER']
scantxt = open(f"/home/{USER}/.nmap_diff/scan{today}.txt","rb")
The problem is that $USER is being interpreted as a literal string by open, not as an environment variable. To expand environment variables in the string, use os.path.expandvars.
os.system(f"/usr/bin/nmap {target} -oN /home/$USER/.nmap_diff/scan{today}.txt")
result_path = os.path.expandvars(f"/home/$USER/.nmap_diff/scan{today}.txt")
with open(result_path, "r", encoding="utf-8") as f:
scantxt = f.read()
By the way, it also looks like the strings in your question were meant to be f-strings, but were missing the f prefixes. I have added them in my answer.
Also, I am assuming that you want the scan results as a string, so I have added the code for that as well. (It seems that nmap doesn't usually include any non-ascii characters in its output for the -oN option, but I am specifying the encoding as UTF-8 in case support for UTF-8 characters is added in a future version.)
How can i create a .pck file in a folder in Python?
The name of the folder is example
I tried:
file = open("\example\file.pck","wb")
But it says:
OSError: [Errno 22] Invalid argument: '\example\x0cile.pck'
EDIT:
Solved! The right command is:
file = open("example/file.pck", "wb")
Use forward slashes
open("/example/file.pck", "wb")
Your problem is likely that backslashes were being interpreted as escape sequences.
You need to use forward slashes and also it seems that the example folder you trying to access doesn't exist. That's probably because you wanted to enter relative address but you entered an absolute one. So it should be:
open("example/file.pck", "wb")
Paths can be tricky, especially if you want to run your code on Unix-based and Windows systems. You can avoid many problems by using os.path which generates paths that will work on any os.
To open a new file use the 'w+' option instead of 'rw'.
In your case:
import os
file_path = os.path.join(os.path.curdir, 'file.pck')
file = open(file_path,'w+')
I am trying to write code that takes 2 numbers in a text file and then divides them, showing the answer as a top heavy fraction. I have gotten the fractions part to work when I am inputting my own values in the program, but i cannot get the program to recognise the text file. I have tried putting them in the same directory and putting the full system path of the file, but nothing so far has worked. right now I am just trying to get the contents of the file to print.
with open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w') as f:
for line in f:
for word in line.split():
print(word)
I will then assign the 2 values to x and y, but I get this error:
Traceback (most recent call last):
File "C:\Python34\divider.py", line 2, in <module>
open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w')
OSError: [Errno 22] Invalid argument:'C:\\ProgramData\\Microsoft\\Windows\\Startmenu\\Programs\\Python 3.4\topheavy.txt'
open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w')
OSError: [Errno 22] Invalid argument:'C:\\ProgramData\\Microsoft\\Windows\\Startmenu\\Programs\\Python 3.4\topheavy.txt'
Two things:
When working with paths that contain backslashes, you either need to use two backslashes, or use the r'' form to prevent interpreting of escape sequences. For example, 'C:\\Program Files\\...' or r'C:\Program Files\...'.
Your error shows this: \\Startmenu\\. It appears that a space is missing between "Start" and "menu", despite the fact that the open line seems to have the right path.
Note: that the \topheavy.txt in your path is probably getting converted to <tab>opheavy.txt too. That's why there aren't two backslashes in front of it in the traceback.
You are using a "\" separator which is probably getting escaped somewhere (like that \t near the end. That's the Windows path separator, but also used as a string escape.
You can double up the "\" as "\". Easiest however is to prepend an r at the beginning to ignore .
r"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt"
Skip the recommendation to use / instead, you are not on Unix and there is no reason Python can't accommodate Windows, as long as you remember to take care about "\" also being an escape. Using r' at start also allows you to copy/paste from the string into another program or vice-versa.
also, it wouldn't hurt to test in c:\temp or similar to avoid issues where you may have mistyped your path.
Last, but not least, you need to open in "r" read mode, as previously mentioned.
You should add one more "/" in the last "/" of path for example:
open('C:\Python34\book.csv') to open('C:\Python34\\\book.csv')
Reference
I had this same error appear when trying to read a large file in Python 3.5.4. To solve it, instead of reading the whole file into memory with .read(), I read each line one by one:
with open('big.txt') as f:
for i in f:
print(i)
Just as is written on the Python Documentation, the IOError Exception occurs:
Raised when an I/O operation (such as a print statement, the built-in
open() function or a method of a file object) fails for an I/O-related
reason, e.g., “file not found” or “disk full”.
Open with "r" (read) instead of "w" (write)
And startmenu in these two lines are different?? Try using a forward instead of a back slash. Python will convert the forward slash to the appropriate delimiter for the OS it is running on
open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w')
OSError: [Errno 22] Invalid argument:'C:\ProgramData\Microsoft\Windows\Startmenu\Programs\Python 3.4\topheavy.txt'
Replace every \ with \\ in file path
My issue, rather arbitrary, was I was writing a file using open(filename, "w").write(...), where filename is an invalid pathname or includes unexpected slashes.
For example, converting a datetime.datetime.today() into a datestring with slashes or colons (Windows) and writing to non-existing directories will result in this error.
Change:
open("../backup/2021/08/03 15:02:61.json", "w").write(data)
To:
open("../backup/2021-08-03 15-02-61.json", "w").write(backup)
As an example.
I tried this code to open a file in Python:
f = open("/Desktop/temp/myfile.txt","file1")
It didn't work. I think this is because I didn't specify the right path. How can I fix the problem?
That doesn't work as you've got the wrong syntax for open.
At the interpreter prompt try this:
>>> help(open)
Help on built-in function open in module __builtin__:
open(...)
open(name[, mode[, buffering]]) -> file object
Open a file using the file() type, returns a file object.
So the second argument is the open mode. A quick check of the documentation and we try this instead:
f = open("/Desktop/temp/myfile.txt","r")
Edit: Oh and yes, your second argument is wrong. Didn't even notice that :)
Python looks where you tell it to for file opening. If you open up the interpreter in /home/malcmcmul then that will be the active directory.
If you specify a path, that's where it looks. Are you sure /Desktop/temp is a valid path? I don't know of many setups where /Desktop is a root folder like that.
Some examples:
If I have a file: /home/bartek/file1.txt
And I type python to get my interpreter within the directory /home/bartek/
This will work and fetch file1.txt ok: f = open("file1.txt", "r")
This will not work: f = open("some_other_file.txt", "r") as that file is in another directory of some sort.
This will work as long as I specify the correct path: f = open("/home/media/a_real_file.txt", "r")
To begin with, the second argument is the permissions bit: "r" for read, "w" for write, "a" for append. "file1" shouldn't be there.
Try:
f = open('Desktop/temp/myfile.txt', 'r')
This will open file relatively to current directory. You can use '/Desktop/temp/myfile.txt' if you want to open file using absolute path. Second parameter to open function is mode (don't know what file1 should mean in your example).
And regarding the question - Python follows OS scheme - looks in current directory, and if looking for modules, looks in sys.path afterwards. And if you want to open file from some subdirectory use os.path.join, like:
import os
f = open(os.path.join('Desktop', 'temp', 'myfile.txt'), 'r')
Then you're safe from the mess with '/' and '\'.
And see docs for built-in open function for more information about the way to use open function.
Just enter your file name and there is your data.....what it does?---->If path exists checks it is a file or not and then opens and read
import os
fn=input("enter a filename: ")
if os.path.exists(fn):
if os.path.isfile(fn):
with open(fn,"r") as x:
data=x.read()
print(data)
else:
print(fn,"is not a file: ")
else:
print(fn,"file doesnt exist ")
This:
import os
os.path
should tell you where python looks first. Of course, if you specify absolute paths (as you have), then this should not matter.
Also, as everyone else has said, your second argument in open is wrong. To find the proper way of doing it, try this code:
help(open)
A minor potential issue that the original post does not have but, also make sure the file name argument uses '/' and not '\'. This tripped me up as the file inspector used the incorrect '/' in its location.
'C:\Users\20\Documents\Projects\Python\test.csv' = Does not work
'C:/Users/20/Documents/Projects/Python/test.csv' = Works just fine
from pathlib import Path
import os
desired_directory = Path('C:/')
desired_directory = desired_directory / 'subfolder'
os.chdir(desired_directory)
That will force Python to look at the directory in the path you specify. It works well when using `subprocess.Popen' to use binary files, as in this snippet:
from subprocess import Popen
from shutil import which
instance_of_Popen = Popen(which('name_of_executable'))
print(instance_of_Popen.args)
I'm trying to automate downloading of some text files from a z/os PDS, using Python and ftplib.
Since the host files are EBCDIC, I can't simply use FTP.retrbinary().
FTP.retrlines(), when used with open(file,w).writelines as its callback, doesn't, of course, provide EOLs.
So, for starters, I've come up with this piece of code which "looks OK to me", but as I'm a relative Python noob, can anyone suggest a better approach? Obviously, to keep this question simple, this isn't the final, bells-and-whistles thing.
Many thanks.
#!python.exe
from ftplib import FTP
class xfile (file):
def writelineswitheol(self, sequence):
for s in sequence:
self.write(s+"\r\n")
sess = FTP("zos.server.to.be", "myid", "mypassword")
sess.sendcmd("site sbd=(IBM-1047,ISO8859-1)")
sess.cwd("'FOO.BAR.PDS'")
a = sess.nlst("RTB*")
for i in a:
sess.retrlines("RETR "+i, xfile(i, 'w').writelineswitheol)
sess.quit()
Update: Python 3.0, platform is MingW under Windows XP.
z/os PDSs have a fixed record structure, rather than relying on line endings as record separators. However, the z/os FTP server, when transmitting in text mode, provides the record endings, which retrlines() strips off.
Closing update:
Here's my revised solution, which will be the basis for ongoing development (removing built-in passwords, for example):
import ftplib
import os
from sys import exc_info
sess = ftplib.FTP("undisclosed.server.com", "userid", "password")
sess.sendcmd("site sbd=(IBM-1047,ISO8859-1)")
for dir in ["ASM", "ASML", "ASMM", "C", "CPP", "DLLA", "DLLC", "DLMC", "GEN", "HDR", "MAC"]:
sess.cwd("'ZLTALM.PREP.%s'" % dir)
try:
filelist = sess.nlst()
except ftplib.error_perm as x:
if (x.args[0][:3] != '550'):
raise
else:
try:
os.mkdir(dir)
except:
continue
for hostfile in filelist:
lines = []
sess.retrlines("RETR "+hostfile, lines.append)
pcfile = open("%s/%s"% (dir,hostfile), 'w')
for line in lines:
pcfile.write(line+"\n")
pcfile.close()
print ("Done: " + dir)
sess.quit()
My thanks to both John and Vinay
Just came across this question as I was trying to figure out how to recursively download datasets from z/OS. I've been using a simple python script for years now to download ebcdic files from the mainframe. It effectively just does this:
def writeline(line):
file.write(line + "\n")
file = open(filename, "w")
ftp.retrlines("retr " + filename, writeline)
You should be able to download the file as a binary (using retrbinary) and use the codecs module to convert from EBCDIC to whatever output encoding you want. You should know the specific EBCDIC code page being used on the z/OS system (e.g. cp500). If the files are small, you could even do something like (for a conversion to UTF-8):
file = open(ebcdic_filename, "rb")
data = file.read()
converted = data.decode("cp500").encode("utf8")
file = open(utf8_filename, "wb")
file.write(converted)
file.close()
Update: If you need to use retrlines to get the lines and your lines are coming back in the correct encoding, your approach will not work, because the callback is called once for each line. So in the callback, sequence will be the line, and your for loop will write individual characters in the line to the output, each on its own line. So you probably want to do self.write(sequence + "\r\n") rather than the for loop. It still doesn' feel especially right to subclass file just to add this utility method, though - it probably needs to be in a different class in your bells-and-whistles version.
Your writelineswitheol method appends '\r\n' instead of '\n' and then writes the result to a file opened in text mode. The effect, no matter what platform you are running on, will be an unwanted '\r'. Just append '\n' and you will get the appropriate line ending.
Proper error handling should not be relegated to a "bells and whistles" version. You should set up your callback so that your file open() is in a try/except and retains a reference to the output file handle, your write call is in a try/except, and you have a callback_obj.close() method which you use when retrlines() returns to explicitly file_handle.close() (in a try/except) -- that way you get explict error handling e.g. messages "can't (open|write to|close) file X because Y" AND you save having to think about when your files are going to be implicitly closed and whether you risk running out of file handles.
Python 3.x ftplib.FTP.retrlines() should give you str objects which are in effect Unicode strings, and you will need to encode them before you write them -- unless the default encoding is latin1 which would be rather unusual for a Windows box. You should have test files with (1) all possible 256 bytes (2) all bytes that are valid in the expected EBCDIC codepage.
[a few "sanitation" remarks]
You should consider upgrading your Python from 3.0 (a "proof of concept" release) to 3.1.
To facilitate better understanding of your code, use "i" as an identifier only as a sequence index and only if you irredeemably acquired the habit from FORTRAN 3 or more decades ago :-)
Two of the problems discovered so far (appending line terminator to each character, wrong line terminator) would have shown up the first time you tested it.
Use retrlines of ftplib to download file from z/os, each line has no '\n'.
It's different from windows ftp command 'get xxx'.
We can rewrite the function 'retrlines' to 'retrlines_zos' in ftplib.py.
Just copy the whole code of retrlines, and chane the 'callback' line to:
...
callback(line + "\n")
...
I tested and it worked.
you want a lambda function and a callback. Like so:
def writeLineCallback(line, file):
file.write(line + "\n")
ftpcommand = "RETR {}{}{}".format("'",zOsFile,"'")
filename = "newfilename"
with open( filename, 'w' ) as file :
callback_lambda = lambda x: writeLineCallback(x,file)
ftp.retrlines(ftpcommand, callback_lambda)
This will download file 'zOsFile' and write it to 'newfilename'