Develop Reference

FileNotFoundError: [Errno 2] No such file or directory when put file in hdfs

I use subprocess.popen in python to put files in hdfs. it runs accurately using python on the Windows cmd. but as I use vscode to run the code, I get "FileNotFoundError: [Errno 2] No such file or directory: Error.
hdfs_path = os.path.join(os.sep,'mongo_import_export')
#put csv into hdfs
put = Popen(['hadoop','dfs','-put','mongo-hadoop/import_export.csv','/mongo_import_export'], stdin=PIPE,bufsize=-1)
put.communicate()
Knowing that my file import_export.csv is in the file in witch the code is located and mong-hadoop folder is in my local files

VSCode is running the code in a different working directory than your local CMD. Use the absolute path to the files you want to put rather than relative paths.

read_pdf FileNotFoundError: [Errno 2] No such file or directory: in Python

I am trying to scrape tables from pdf with read_pdf in python. I am using read_pdf but it doesn't do the job. Also, to mention, I do this in MAC with Jupiter notebook.
This is what I do:
from tabula import read_pdf
file = read_pdf(r'C:\Users\myname\Rprojects\Reports_scraping\data_scraped\icnarc_29052020\icnarc_200529.pdf')
I get this error:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\myname\\Rprojects\\Reports_scraping\\data_scraped\\icnarc_29052020\\icnarc_200529.pdf'
How I can solve this issue?

just to check that the file exist, do you get True when running this:
import os
file_path = r'C:\Users\myname\Rprojects\Reports_scraping\data_scraped\icnarc_29052020\icnarc_200529.pdf'
print( os.path.isfile(file_path))
Edit file_path with wherever is the file(using Python 3).
And did you change "myname" in the path with your actual username... (just in case)
It is preferable to build your paths using os.path.join to make things compatible, on windows it will need to create a root "config.py" file, see
how to get the root folder on windows
#
having discussed with GaB, it seemed that he is using Jupyter notebook on Mac, which explains issues, I saw this link, but can't help more.
Jupyter - import pdf
os.path.join doc

There can be only one possibility, the file is not there, but you have already checked that I assume, if not, Once again check whether the spelling of file is correct. If this doesn't work, then do below trick
Execute the py code in same folder as file, and then use
from tabula import read_pdf
file = read_pdf(r'icnarc_200529.pdf')
Sometimes, this simple method does the trick.

Pathlib can't find a file on Ubuntu

Trying to access a CSV file in the following directory with a systemd service: /path/to/cwd/data/x.csv
Currently, hard-coding the path in the python file in my CWD with path = '/path/to/cwd/data/x.csv' allows the service to find the file without issues.
However, if I try to use pathlib like so:
from pathlib import Path
path = Path.cwd() / 'data' / 'x.csv'
The service gives me an error:
FileNotFoundError: [Errno 2] No such file or directory: '/data/x.csv'
I get the same error when I try to use the OS library to do path = os.path.join(os.getcwd(), 'data/x.csv')
I have no idea what's going on, when I compare the output of the paths generated by pathlib and os, they're exactly the same as what I type out, but they don't work and the hardcoded path does.

You can try to specify the cwd inside the service.
[Service]
WorkingDirectory=/PathToCwd
Then run it again.

How to create a file in c:/windows/system32 folder using python

When I run the following code I get an error named permission denied:
f=open('C:/Windows/System32/azm.txt','w')
f.write('+989193667998')
f.close()

On Windows, use backslashes instead of slashes to specify file paths:
f = open(r'c:\windows\system32\azm.txt', 'w')
And since you're trying to write to a system directory, make sure you run your code as Administrator.

Permission denied error while writing to a file in Python

I want to create a file and write some integer data to it in python. For example, I have a variable abc = 3 and I am trying to write it to a file (which doesn't exist and I assume python will create it on its own):
fout = open("newfile.dat", "w")
fout.write(abc)
First, will python create a newfile.dat on its own? Secondly, it's giving me this error:
IOError: [Errno 13] Permission denied: 'newfile.dat'
What's wrong here?

Please close the file if its still open on your computer, then try running the python code.
I hope it works

This also happens when you attempt to create a file with the same name as a directory:
import os
conflict = 'conflict'
# Create a directory with a given name
try:
os.makedirs(conflict)
except OSError:
if not os.path.isdir(conflict):
raise
# Attempt to create a file with the same name
file = open(conflict, 'w+')
Result:
IOError: [Errno 13] Permission denied: 'conflict'

I've had the same issue using the cmd (windows command line) like this
C:\Windows\System32> "G:\my folder\myProgram.py"
Where inside the python file something like this
myfile = open('myOutput.txt', 'w')
The error was that when you don't use a full path, python would use your current directory, and because the default directory on cmd is
C:\Windows\System32
that won't work, as it seems to be write-protected and needs permission & confirmation form an administrator
Instead, you should use full paths, for example:
myfile = open('G:\my folder\myOutput.txt', 'w')

Permission denied simply means the system is not having permission to write the file to that folder. Give permissions to the folder using "sudo chmod 777 " from terminal and try to run it. It worked for me.

I write python script with IDLE3.8(python 3.8.0)
I have solved this question:
if the path is
shelve.open('C:\\database.dat')
it will be
PermissionError: [Errno 13] Permission denied: 'C:\\database.dat.dat'.
But when I test to set the path as
shelve.open('E:\\database.dat')
That is OK!!!
Then I test all the drive(such as C,D,F...) on my computer,Only when the Path set in Disk
C:\\
will get the permission denied error.
So I think this is a protect path in windows to avoid python script to change or read files in system Disk(Disk C)

In order to write on a file by using a Python script, you would have to create a text file first.
Example A file such as C:/logs/logs.txt should exist.
Only then the following code works:
logfile=open(r"C:/logs/logs.txt",'w')
So summary.
A text file should exist on the specified location
Make sure you close the file before running the Python script.

To answer your first question: yes, if the file is not there Python will create it.
Secondly, the user (yourself) running the python script doesn't have write privileges to create a file in the directory.

If you are executing the python script via terminal pass --user to provide admin permissions.
Worked for me!
If you are using windows run the file as admin.
If you are executing via cmd, run cmd as admin and execute the python script.

Make sure that you have write permissions for that directory you were trying to create file by properties