I'm uploading a file via SFTP using Paramiko with sftp.put(localFile, remoteFile). I make the necessary directory first if needed with
makeCommand = 'mkdir -p "' + remotePath + '"'
ssh.exec_command(makeCommand)
this was works sometimes but I'm occasionally getting the following error:
sftp.put(localFile, remoteFile)
File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 565, in put
File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 245, in open
File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 635, in _request
File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 682, in _read_response
File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 708, in _convert_status
IOError: [Errno 2] No such file
despite the fact that the local file definitely exists (and localFile is the correct path to it) and the remote path is made.
There is discussion here and here on a similar problem but none of the points raised there have helped me. My server supports the df -hi command.
Has anyone any advice on this or a possible solution?
EDIT
After suggestions below I tried changing the working directory with sftp.chdir(remoteDirectory) but this call produced the exact same error as above. So it seems this isn't just an upload issue. Any ideas?
It seems to be a remote folder permission problem. Although the remote folder was made before the file was uploaded, it appears the permissions on the folder were preventing an upload.
The problem is linked to this issue - if I set open permissions on the folder I'll be uploading to before I upload, the program can upload fine. Although for a permission issue I should be getting IOError: [Errno 13] Permission denied, since I made the changes I haven't encountered any errors.
I'm not sure if it's the response the server is giving Paramiko which is the issue, or a bug in Paramiko itself which is causing IOError: [Errno 2] No such file instead of a Errno 13, but this appears to have solved the problem.
The put method has a confirm parameter which is enabled by default, which will do a stat on the file after transfer.
In my case, the remote server i was transferring the file to, immediately moved any transferred files to another location to get processed which was causing the stat to fail. Setting the confirm parameter to False resolved this.
def put(self, localpath, remotepath, callback=None, confirm=True):
From the paramiko source sftp_client.py:
:param bool confirm:
whether to do a stat() on the file afterwards to confirm the file
size (since 1.7.7)
The IOError is local, so (for whatever reason) it seems that your local python cannot find localFile. Safety checking this before the call might help tracking down the problem:
if os.path.isfile(localFile):
sftp.put(localFile, remoteFile)
else:
raise IOError('Could not find localFile %s !!' % localFile)
If you're positive that localFile does exist, then this could just be a path problem - is localFile on an absolute or relative path? Either way, the if statement above will catch it.
EDIT
Tracing through the paramiko files shows that line 245 of sftp_client.py (the one throwing the exception) is actually
fr = self.file(remotepath, 'wb')
which is quite misleading as paramiko throws an IOError for a remote file! My best guess now is that remoteFile is either a missing directory or a directory you don't have access to.
Out of interest, can you list the remote dir
sftp.listdir(path=os.path.dirname(remoteFile))
to check that it's there (or maybe it's there and you can write to it)?
Are you sure the directory has been created and it is your remote working directory?
Paramiko has its own methods for creating new directories and navigating the remote file system. Consider using something like:
sftp.mkdir(remotedirectory)
sftp.chdir(remotedirectory)
sftp.put(localfile, remotefile)
I faced the same issue. it was a silly mistake.
Just use sftp.stat(your remote directory) to check if it's there.
then use sftp.put(localfileabsolutepath, remotedir+filename)
It will work for sure.
Had the same issue. In my case it was a timing problem:
self.mkdir(remote_dir)
sftp.put(local_file, remote_file)
The mkdir() function, which had
ssh.exec_command(f"mkdir -p {remote_dir}")
in it, didn't finish fast enough.
Changing the original code to
self.mkdir(remote_dir)
sleep(0.01)
sftp.put(local_file, remote_file)
fixed it.
Related
I am very new on this platform and I need help reading a file in python. First, it was a docx file because I am using Mac, I converted it into a txt file ,but still have file not found error.
Here is some code:
opdoc = open('PRACT.txt')
print(opdoc.readline())
for each in opdoc :
print(each)
and this the error output: FileNotFoundError: [Errno 2] No such file or directory: 'PRACT.txt'
Unless your file is in the exact same directory as your python file, you'll get this error. Even then, it's best practice to include some sort of relative filepath, like "./PRACT.txt". In general, your issue stems from the fact that Python doesn't know where to look for your file, and where it does know to look, the file isn't there. You need to provide the full path to the file, like:
with open("path/to/file/PRACT.txt", "rb") as f:
# read file etc
Make a new folder and put the txt file in it and also the programme itself. Or just put them in the same directory.
As you have this error ( No such file or directory: 'PRACT.txt') your code couldn't find the file because they were not in the same directory. When you use open("README.txt") the programme is assuming that the text is in the same directory as your code. If you don't want them in the same directory you could also try open(f"[file_path]") and that should work.
I always used the same way to export Dataframes to excel:
df.to_excel('file_name.xlsx', index=False, sheet_name='sheet_name')
To import files I define the path first and then, used a command pd.read_excel and it stil works well.
os.chdir("C:\\Users\\folder_1\\folder_2")
file="file_name.xlsx"
df=pd.read_excel(file)
But the export method stopped working
The errors returned where:
[Errno 2] No such file or directory:
[Errno 9] Bad file descriptor
I have already uninstalled and installed again Anaconda.
Anybody has faced this problem before? How to fix it?
I do not know if it's relevant. In a workbook of python there is the following:
if not self.fileclosed:
try:
self._store_workbook()
except IOError as e:
raise FileCreateError(e)
except LargeZipFile as e:
raise FileSizeError("Filesize would require ZIP64 extensions. "
"Use workbook.use_zip64().")
In your code, the path is a litteral word "path" (as a string). Normally, it should be a variable path that content is the string of the path
Not sure if this will work but try changing your path to this?
os.chdir(r"C:\\Users\\folder_1\\folder_2")
or manually test it:
df.to_excel(r'your_path\.xlsx')
and see whether it works?
This problem puzzled me.
Maybe the problem is in the code, I hope you take a look
with open(training_images_labels_path,'r') as file:
lines = file.readlines()
He says that the file does not exist
FileNotFoundError: [Errno 2] No such file or directory: '\\Desktop\\project\\data\\generated\\training_images_labels.txt'
Though the file exists
I need solutions
If it says that the file does not exist though the file exists, it means the path has been not given properly. Try giving the path correctly.
Method 1:
Giving correct path 'C:\\Users\\Public\\Desktop\\project\\data\\generated\\training_images_labels.txt' or
'C:\\Users\\<insert your username>\\Desktop\\project\\data\\generated\\training_images_labels.txt' is your path if I guess correctly
Method 2:
Using os module ( Recommended )
mydir = 'C:/Users/Public/Desktop/project/data/generated'
myfile = 'training_images_labels.txt'
training_images_labels_path = os.path.join(mydir, myfile)
with open(training_images_labels_path,'r') as file:
lines = file.readlines()
Method 3:
You can also try changing the working directory to the location where your data is present. ie Desktop>project>data>generated here and open the file with file name. ie
with open('training_images_labels.txt','r') as file:
lines = file.readlines()
I had the same problem with importing an excel file, which of course exists in the same directory with my .py file. The chosen solution above did not help me, and actually I didn't understand those three methods, as I am working on mac OS.
This method worked for me: in Spyder, I usually run the file by pressing the keys "Shift + Enter", which in this case made the problem. So, my solution was, to press on the "Run file" button (or the fn+F5 key) instead.
Maybe someone wants to explain the difference.
Looks like its a windows path you are working and i believe path really thrown in the error is wrong when compared to the actual where the txt file resides.. just cross check once, if that's the case try to pass the correct path in to the variable "training_images_labels_path"
Can you tell how you created this path.Some advise.
use path separator library to generate path to avoid this error.
training_images_labels_path
further try to navigate parent directory using python and print pth.may be some new line or linux/windwos convereted path or other special character in path. navigating parent directory and listing will solve
if still not solve try to navigatep parent-parent dir and print path
try hard
Watch your path if its correct or not. I had the same problem and it turned out i didnt had the good path set
quick question about Python on windows.
I have a script that compiles a program (using an install rule), and then moves the build products to a remote destination over the network.
However, I keep getting WindowsError 5 Access Denied.
All files are created from the script context, I have ownership and full control on all of them. The copy to the remote destination succeeds, but the failure is during the deletion process.
If I try to delete or rename the file manually within windows, I get no errors. Just the shutil.move fails.
I'm thinking maybe the API is trying to delete the file when the network operation is not yet complete?
Any input is very appreciated.
try:
shutil.move(directory, destination)
except OSError:
print "Failed to move %s to %s." %(directory, destination)
raise
...
Traceback (most recent call last):
File "C:\WIP\BuildMachine\build_machine.py", line 176, in <module>
main()
File "C:\WIP\BuildMachine.hg\BuilderInstance.py", line 496, in deployVersion
shutil.move(directory, destination)
File "C:\Python27\lib\shutil.py", line 300, in move
rmtree(src)
File "C:\Python27\lib\shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "C:\Python27\lib\shutil.py", line 250, in rmtree
os.remove(fullname)
WindowsError: [Error 5] Access is denied: '3_54_7__B_1\\Application_Release_Note.doc'
the problem with shutil.move on windows is that it doesn't handle the case where:
the source & destination aren't on the same drive AND
some files in the source directory are write-protected.
If both conditions are met, shutil.move cannot perform a os.rename, it has to:
copy the files (which isn't an issue)
delete the source files (which is an issue because of a limitation of shutil)
To fix that, I made myself a copy of the shutil module (under a different name) and added that line (for you it'll be right before line 250):
os.chmod(fullname,0o777) # <-- add that line
os.remove(fullname) # some versions have "unlink" instead
The rmtree function has the same issue on Windows.
On Linux this doesn't happen because file delete permissions aren't handled at the file level but at the directory level. On windows, it doesn't work that way. Adding os.chmod does the trick (even if it's a hack), and os.remove succeeds (unless the file is open in Word or whatever)
Note that shutil authors encourage you to copy & improve the functions. Also a note from the documentation of shutil.move:
A lot more could be done here... A look at a mv.c shows a lot of
the issues this implementation glosses over.
If you don't want to modify shutil, you can run a recursive chmod on the source files to make sure that shutil.move will work, for instance like this:
for root, dirs, files in os.walk(path):
for f in dirs+files:
os.chmod(os.path.join(root, f), 0o777)
You could also use shutil.copytree then a modified version of shutil.rmtree (since you know that source & dest aren't on the same filesystem)
I am encountering an odd behaviour from the file() builtin. I am using the unittest-xml-reporting Python package to generate results for my unit tests. Here are the lines that open a file for writing, a file which (obviously does not exist):
report_file = file('%s%sTEST-%s.xml' % \
(test_runner.output, os.sep, suite), 'w')
(code is taken from the package's Github page)
However, I am given the following error:
...
File "/home/[...]/django-cms/.tox/pytest/local/lib/python2.7/site-packages/xmlrunner/__init__.py", line 240, in generate_reports
(test_runner.output, os.sep, suite), 'w')
IOError: [Errno 2] No such file or directory: './TEST-cms.tests.page.NoAdminPageTests.xml'
I found this weird because, as the Python docs state, if the w mode is used, the file should be created if it doesn't exist. Why is this happening and how can I fix this?
from man 2 read
ENOENT O_CREAT is not set and the named file does not exist. Or, a
directory component in pathname does not exist or is a dangling
symbolic link.
take your pick :)
in human terms:
your current working directory, ./ is removed by the time this command is ran,
./TEST-cms.tests.page.NoAdminPageTests.xml exists but is a symlink pointing to nowhere
"w" in your open/file call is somehow messed up, e.g. if you redefined file builtin
file will create a file, but not a directory. You have to create it first, as seen here
It seems like the file which needed to be created was attempted to be created in a directory that has already been deleted (since the path was given as . and most probably the test directory has been deleted by that point).
I managed to fix this by supplying an absolute path to test_runner.output and the result files are successfully created now.