I am working together with a colleague and he has Ubuntu while I have windows. We have a dataset of json files which have in them a "path" written. His paths look like this:
'C:/Users/krock/Desktop/FIIT/BP/Ubuntu/luadb/etc/luarocks_test/modules/30log/share/lua/5.3/30log.lua'
But this doesn't work on Windows, I was trying to do
some_string.replace('/', '\\')
But this results in strings written in json that look like this:
'C:\\Users\\krock\\Desktop\\FIIT\\BP\\Ubuntu\\luadb\\etc\\luarocks_test\\data_all'
On my windows machine, I can't read (the program) these paths as it give an error:
No such file or directory
Is there a solution to this?
EDIT: I tried using Path from pathlib, but I got another error saying:
TypeError: Object of type WindowsPath is not JSON serializable
I found the solution to this is to do str(Path(path_string)), but the result is again the path in double quotes.
Yes, the solution is to use Python's built in pathlib. Also, using string literals might help the clarity of your program.
https://docs.python.org/3/library/pathlib.html
This question is missing code samples, so can't be more specific, but generally speaking, doing this manually is error-prone. Consider using a library, such as pathlib. E.G:
>>> from pathlib import Path
>>> Path('luarocks_test/modules/30log/share/lua/5.3/30log.lua')
PosixPath('luarocks_test/modules/30log/share/lua/5.3/30log.lua')
On Windows, instantiating a Path would give you a WindowsPath. You'll also want to use relative, rather than absolute references, as the paths will be different on your workstations.
So I have a path to a server that looks like
\\foo\bar\baz
I am trying to convert this to a Path object with Python's pathlib library with something like
Path('\\foo').joinpath('bar').joinpath('baz')
Trying to see if this path exists with "path.exists()" however signals that the path doesn't exist.
I have also replace the starting path of the path with the following the same result:
Path(r'\\foo'), Path('\\\\foo'), Path('\foo)
Printing out the path after being parsed by Path appears to show the path being interpreted as, no matter how the foo root is entered
'C:/foo/bar/baz'
However when I simply plop the path in os.path.exists as a raw string, the path is successfully found (using a normal string results in a unicode error)
os.path.exists(r'\\foo\bar\baz')
Is there anyway to use Pathlib in such a way that the server path is successfully parsed / interpreted, or am I forced to just use a raw string with os.path operations?
r'\\foo' is just not a valid windows UNC path, even if foo exists. You need at least a sharename after that.
From Microsoft documentation:
A valid UNC path MUST contain two or more path components.
This creates a valid path object on which exists() will work properly:
>>> pathlib.Path(r'\\foo\bar').joinpath('baz')
WindowsPath('//foo/bar/baz')
I am having an issue with pathlib when I try to construct a file path that has a "." in its name, the pathlib module ignores it.
Here are example lines (I tried multiple versions, all resulted the same issue)
The issue is that the original file name will be coming from another application, so it is not like I can edit the name myself. I also do not want to do string replacement work arounds, if possible.
path=r"c:\temp"
1
p=Path(path).joinpath("myfile.001").with_suffix(".bat")
2
p=Path(path, "myfile.001").with_suffix(".bat")
3
p=Path(path).with_name("myfile.001").with_suffix(".bat")
All these lines will yield to
WindowsPath('C:/temp/myfile.bat')
So how do I make pathlib.Path to construct this full path properly. The final path has to be
WindowsPath('C:/temp/myfile.001.bat')
Not
WindowsPath('C:/temp/myfile.bat')
Naturally I am looking for a way to do it through pathlib itself, otherwise I can just use os.
thanks
You are telling pathlib to replace the suffix .001 with the suffix .bat. pathlib complies.
Tell pathlib to add .bat to the existing suffix.
p = Path(path, 'myfile.001')
p = p.with_suffix(p.suffix+'.001')
I'm trying to load a .csv file using the pd.read_csv() function when I get an error despite the file path being correct and using raw strings.
import pandas as pd
df = pd.read_csv('C:\\Users\\user\\Desktop\\datafile.csv')
df = pd.read_csv(r'C:\Users\user\Desktop\datafile.csv')
df = pd.read_csv('C:/Users/user/Desktop/datafile.csv')
all gives the error below:
FileNotFoundError: File b'\xe2\x80\xaaC:/Users/user/Desktop/tutorial.csv' (or the relevant path) does not exist.
Only when i copy the file into the working directory will it load correct.
Is anyone aware of what might be causing the error?
I had previously loaded other datasets with full filepaths without any problems and I'm currently only encountering issues since I've re-installed my python (via Anaconda package installer).
Edit:
I've found the issue that was causing the problem.
When I was copying the filepath over from the file properties window, I unwittingly copied another character that seems invisible.
Assigning that copied string also gives an unicode error.
Deleting that invisible character made any of above code work.
Try this and see if it works. This is independent of the path you provide.
pd.read_csv(r'C:\Users\aiLab\Desktop\example.csv')
Here r is a special character and means raw string. So prefix it to your string literal.
https://www.journaldev.com/23598/python-raw-string:
Python raw string is created by prefixing a string literal with ‘r’
or ‘R’. Python raw string treats backslash () as a literal character.
This is useful when we want to have a string that contains backslash
and don’t want it to be treated as an escape character.
$10 says your file path is correct with respect to the location of the .py file, but incorrect with respect to the location from which you call python
For example, let's say script.py is located in ~/script/, and file.csv is located in ~/. Let's say script.py contains
import pandas
df = pandas.read_csv('../file.csv') # correct path from ~/script/ where script.py resides
If from ~/ you run python script/script.py, you will get the FileNotFound error. However, if from ~/script/ you run python script.py, it will work.
I know following is a silly mistake but it could be the problem with your file.
I've renamed the file manually from adfa123 to abc.csv. The extension of the file was hidden, after renaming, Actual File name became abc.csv.csv. I've then removed the extra .csv from the name and everything was fine.
Hope it could help anyone else.
import pandas as pd
path1 = 'C:\\Users\\Dell\\Desktop\\Data\\Train_SU63ISt.csv'
path2 = 'C:\\Users\\Dell\\Desktop\\Data\\Test_0qrQsBZ.csv'
df1 = pd.read_csv(path1)
df2 = pd.read_csv(path2)
print(df1)
print(df2)
On Windows systems you should try with os.path.normcase.
It normalize the case of a pathname. On Unix and Mac OS X, this returns the path unchanged; on case-insensitive filesystems, it converts the path to lowercase. On Windows, it also converts forward slashes to backward slashes. Raise a TypeError if the type of path is not str or bytes (directly or indirectly through the os.PathLike interface).
import os
import pandas as pd
script_dir = os.getcwd()
file = 'example_file.csv'
data = pd.read_csv(os.path.normcase(os.path.join(script_dir, file)))
If you are using windows machine. Try checking the file extension.
There is a high possibility of file being saved as fileName.csv.txt instead of fileName.csv
You can check this by selecting File name extension checkbox under folder options (Please find screenshot)
below code worked for me:
import pandas as pd
df = pd.read_csv(r"C:\Users\vj_sr\Desktop\VJS\PyLearn\DataFiles\weather_data.csv");
If fileName.csv.txt, rename/correct it to fileName.csv
windows 10 screen shot
Hope it works,
Good Luck
Try using os.path.join to create the filepath:
import os
f_path = os.path.join(*['C:', 'Users', 'user', 'Desktop', 'datafile.csv'])
df = pd.read_csv(f_path)
I was trying to read the csv file from the folder that was in my 'c:\'drive but, it raises the error of escape,type error, unicode......as such but this code works
just take an variable then add r to read it.
rank = pd.read_csv (r'C:\Users\DELL\Desktop\datasets\iris.csv')
df=pd.DataFrame(rank)
There is an another problem on how to delete the characters that seem invisible.
My solution is copying the filepath from the file windows instead of the property windows.
That is no problem except that you should fulfill the filepath.
Experienced the same issue. Path was correct.
Changing the file name seems to solve the problem.
Old file name: Season 2017/2018 Premier League.csv
New file name: test.csv
Possibly the whitespaces or "/"
I had the same problem when running the file with the interactive functionality provided by Visual studio. Switched to running on the native command line and it worked for me.
For my particular issue, the failure to load the file correctly was due to an "invisible" character that was introduced when I copied the filepath from the security tab of the file properties in windows.
This character is e2 80 aa, the UTF-8 encoding of U+202A, the left-to-right embedding symbol. It can be easily removed by erasing (hitting backspace or delete) when you've located the character (leftmost char in the string).
Note: I chose to answer because the answers here do not answer my question and I believe a few folks (as seen in the comments) might meet the same situation as I did. There also seems to be new answers every now and then since I did not mark this question as resolved.
I had similar problem when I was using JupyterLab + Anaconda, and used my browser to type stuff on IDE.
My problem was simpler - there was a very subtle typo error. I didn't have to use raw text - either escaping or using {{r}} string worked for me :). If you use Anaconda with Jupyter Lab, I believe the Kernel starts with where you open the notebook from i.e. the working directory is that top level folder.
data = pd.read_csv('C:\\Users\username\Python\mydata.csv')
This worked for me. Note the double "\\" in "C:\\" where the rest of the folders use only a single "\".
I'm trying to use the TMP environment variable in a program. When I ask for
tmp = os.path.expandvars("$TMP")
I get
C:\Users\STEVE~1.COO\AppData\Local\Temp
Which contains the old-school, tilde form. A function I have no control over returns paths like
C:\Users\steve.cooper\AppData\Local\Temp\file.txt
My problem is this; I'd like to check if the file is in my temp drive, but I can't find a way to compare them. How do you tell if these two Windows directories;
C:\Users\STEVE~1.COO\AppData\Local\Temp
C:\Users\steve.cooper\AppData\Local\Temp
are the same?
Here is alternative solution using only ctypes from Standard Python Library.
tmp = unicode(os.path.expandvars("$TMP"))
import ctypes
GetLongPathName = ctypes.windll.kernel32.GetLongPathNameW
buffer = ctypes.create_unicode_buffer(GetLongPathName(tmp, 0, 0))
GetLongPathName(tmp, buffer, len(buffer))
print buffer.value
You will need the python win32 extensions from http://sourceforge.net/projects/pywin32/ or I use python packaged by ActiveState
They include the function win32file.GetLongPathName which will transform the 8.3 version into the full path.