So I tried to run this code.
import pandas
i = input("hi input a csv file..")
df = pandas.read_csv(i)
and I got an error saying
FileNotFoundError: File b'"C:\\Users\\thomas.swenson\\Downloads\\hi.csv"' does not exist
but then if I hard code that path that 'doesn't exist' into my program it works fine.
import pandas
df = pandas.read_csv("C:\\Users\\thomas.swenson\\Downloads\\hi.csv")
it works just fine.
Anyone know why this may be happening?
I'm running python 3.6 and using a virtualenv
looks like the input function was placing another set of quotes around the input.
so ill just have to remove them and it works fine.
Related
I made a simple code that loads in data called 'original.dat' from a folder named 'data' in my computer. The code was working great and i was able to view my spectra graph. This morning I ran the code again, but it completely crashed giving the error " OSError: data/original.dat not found." Even though nothing changed. The data is infact still in the folder named 'data' and there isn't any spelling mistakes. Can anyone please help understand why its now giving me this error? The code was working perfectly the day before.
here is the code I used :
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
OPUSimaginary = np.loadtxt('data/original.dat', delimiter = ',')
Data file position, Error:cant find the file,Error: suggested code to find file
Few things that you can do to avoid file not found type of issues:
Add a check if the file exists before trying to load it.
Example:
import os
os.path.isfile(fname) # Returns True is file found
Make sure the file permissions are correct and the file can be read.
sudo chmod -R a+rwx <file-name>
I'm new to the Databricks, need help in writing a pandas dataframe into databricks local file system.
I did search in google but could not find any case similar to this, also tried the help guid provided by databricks (attached) but that did not work either. Attempted the below changes to find my luck, the commands goes just fine, but the file is not getting written in the directory (expected wrtdftodbfs.txt file gets created)
df.to_csv("/dbfs/FileStore/NJ/wrtdftodbfs.txt")
Result: throws the below error
FileNotFoundError: [Errno 2] No such file or directory:
'/dbfs/FileStore/NJ/wrtdftodbfs.txt'
df.to_csv("\\dbfs\\FileStore\\NJ\\wrtdftodbfs.txt")
Result: No errors, but nothing written either
df.to_csv("dbfs\\FileStore\\NJ\\wrtdftodbfs.txt")
Result: No errors, but nothing written either
df.to_csv(path ="\\dbfs\\FileStore\\NJ\\",file="wrtdftodbfs.txt")
Result: TypeError: to_csv() got an unexpected keyword argument 'path'
df.to_csv("dbfs:\\FileStore\\NJ\\wrtdftodbfs.txt")
Result: No errors, but nothing written either
df.to_csv("dbfs:\\dbfs\\FileStore\\NJ\\wrtdftodbfs.txt")
Result: No errors, but nothing written either
The directory exists and the files created manually shows up but pandas to_csv never writes nor error out.
dbutils.fs.put("/dbfs/FileStore/NJ/tst.txt","Testing file creation and existence")
dbutils.fs.ls("dbfs/FileStore/NJ")
Out[186]: [FileInfo(path='dbfs:/dbfs/FileStore/NJ/tst.txt',
name='tst.txt', size=35)]
Appreciate your time and pardon me if the enclosed details are not clear enough.
Try with this in your notebook databricks:
import pandas as pd
from io import StringIO
data = """
CODE,L,PS
5d8A,N,P60490
5d8b,H,P80377
5d8C,O,P60491
"""
df = pd.read_csv(StringIO(data), sep=',')
#print(df)
df.to_csv('/dbfs/FileStore/NJ/file1.txt')
pandas_df = pd.read_csv("/dbfs/FileStore/NJ/file1.txt", header='infer')
print(pandas_df)
This worked out for me:
outname = 'pre-processed.csv'
outdir = '/dbfs/FileStore/'
dfPandas.to_csv(outdir+outname, index=False, encoding="utf-8")
To download the file, add files/filename to your notebook url (before the interrogation mark ?):
https://community.cloud.databricks.com/files/pre-processed.csv?o=189989883924552#
(you need to edit your home url, for me is :
https://community.cloud.databricks.com/?o=189989883924552#)
dbfs file explorer
I was trying to read a dataset but I'm getting this weird error:
FileNotFoundError: [Errno 2] File b'Position_Salaries.csv' does not exist: b'Position_Salaries.csv'
Even though the file does exist and its on the same directory of my code (As you can see in this pic)
The weird thing is, I've run that code many times without getting any kind of error, but suddenly (without changing anything)is not working. I literally just opened Spyder, try to run the code, but nope, that error appeared. Any ideas why is this happening and how can I solve it?
Part of the code I'm trying to run:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:2].values
y = dataset.iloc[:, 2].values
There is a tool tip that is obscuring your picture but from the information provided I would guess that your working directory is not set to your project directory. Try running the code from the terminal.
Sometimes it happens due to the changes in the format of your excel file. Make sure that the excel file is saved as CVS.
I am reading a csv file using pandas. It works fine if I run script as root user. But when I try to run it with different user it does not read data and gives:
error : KeyError: 'no item named 0'
it appears at:
dt = pd.read_csv('rt.csv', header=None).fillna('').set_index(0).to_dict()[1]
Btw, I am working on Ubuntu 12.02 and using anaconda, which is installed in root user and other user as well (which is giving error)
Please help.
You like have different pandas versions installed as user and root.
I get the same error with version 0.16.2 when I use the wrong delimiter.
Have a look at your data in rt.csv.
For example, this would work for a whitespace-delimited file:
dt = pd.read_csv('rt.csv', header=None,
delim_whitespace=True).fillna('').set_index(0).to_dict()[1]
Check the file and adapt the delimiter accordingly.
I'm trying to read a file using python and I keep getting this error
ERROR: Line magic function `%user_vars` not found.
My code is very basic just
names = read_csv('Combined data.csv')
names.head()
I get this for anytime I try to read or open a file. I tried using this thread for help.
ERROR: Line magic function `%matplotlib` not found
I'm using enthought canopy and I have IPython version 2.4.1. I made sure to update using the IPython installation page for help. I'm not sure what's wrong because it should be very simple to open/read files. I even get this error for opening text files.
EDIT:
I imported traceback and used
print(traceback.format_exc())
But all I get is none printed. I'm not sure what that means.
Looks like you are using Pandas. Try the following (assuming your csv file is in the same path as the your script lib) and insert it one line at a time if you are using the IPython Shell:
import pandas as pd
names = pd.read_csv('Combined data.csv')
names.head()