Develop Reference

More efficient way of reading a CSV file in Python with different delimeter

I am a high school student, so please don't mind if this question is stupid, but Is there a more efficient way of reading a csv file in python than using the csv module and file reader. Also, is there a way to specify a different delimiter or can it only read with "," as a delimiter.
import csv
with open(filename, 'r') as csvfile:
csvreader = csv.reader(csvfile)
fields = next(csvreader)
for row in csvreader:
rows.append(row)
Thank you so much!

You may use the "pandas" library to read a csv file (along with sql tables etc) and store the data as a pandas "dataframe", you may use these dataframes directly in plotly and prophet. You can also convert these dataframes into numpy arrays or lists based on the necessity.
You can install pandas using the following command:
pip install pandas
or you may install it in a conda environment using:
conda activate name_of_your_environment
conda install pandas
After that you can read a csv file using the following code to read a csv file and specify any delimeters:
import pandas as pd
df = pd.read_csv("Path to File/filename.csv",delimiter = "\t")
To change it into an numpy.ndarray you may use the "to_numpy()" method of the dataframe.
Similarly, to change it into a list you may use the "tolist()" method of the dataframe.
I hope this helps.

You can use the delimiter parameter of csv.reader to set the delimiter:
csvreader = csv.reader(csvfile, delimiter='\n')
The documentation for the CSV Python module should help you: https://docs.python.org/3/library/csv.html

How to append from one csv file to another?

Tried different ways. The closest way that may fit my need is the following code:
with open('list.csv', 'r') as reader, open('list-history.csv', 'a') as writer:
for row in reader:
writer.writerow(row)
I'm using 'a' and tried 'w' as well but no luck.
The result is no output at all.
Any suggestion, please? Thanks.

I think there should be a error with a stacktrace.
Here: writer.writerow(row)
Normally open() returns file object which doesn't have .writerow() method, normally, you should use .write(buffer) method.
Example
with open('list.csv', 'r') as reader, open('list-history.csv', 'a') as writer:
for row in reader:
writer.write(row)
For me it works well with test csv files. But it doesn't merge them, just appends content of one file to another one.

If both the csv columns have same name. Python's pandas module can help.
Example Code snippet.
import pandas as pd
df1 = pd.read_csv("csv1.csv")
df2 = pd.read_csv("csv2.csv")
df1.append(df2, ignore_index=True)
df1.to_csv("new.csv",index=False)

Output of terminal to a csv with separate columns in python

my code goes as follows:
import csv
with open('Remarks_Drug.csv', newline='', encoding ='utf-8') as myFile:
reader = csv.reader(myFile)
for row in reader:
product = row[0].lower()
filename = row[1]
product_patterns = ', '.join([i.split("+")[0].strip() for i in product.split(",")])
print(product_patterns, filename)
which outputs as below: (where film-coated tab should be one column and the filename should be another column)
film-coated tablet RECD outcome AUBAGIO IAIN-21 AoR.txt
solution for injection 093 Acceptance NO Safety profil.txt
I want to output this to a csv file with one column as product_patterns and another as filename. I wrote the below code but only the last row gets appended. Can anyone please help me with the looping here. The code I wrote is:
with open ('drug_output.csv', 'a') as csvfile:
fieldnames = ['product_patterns', 'filename']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writerow({'product_patterns':product_patterns, 'filename':filename})
enter image description here

Depending on the environment that you can use, it might be more practical to use more dedicated programs to solve your problem.
Especially the pandas package seems useful in your case.
Then you can load the csv using:
import pandas as pd
df=pd.read_csv(file_path)
After doing the necessary manipulations, you can save it again with
df.to_csv(file_path)
This will save you a lot of issues that typically occur when parsing line by line, and it should also increase performance a bit. Pandas is a pretty good package to learn anyway if you need to do some data manipulation.

python reading file returns 0x035022B0

i just started using python
and i've been trying to read a file
the code doesn't return any errors but it shows this message on the
debugging screen
<_csv.reader object at 0x035022B0>
i searched but i couldn't find any answers
i even tried other sample codes to see if the problem is in my writing
but it returned the same message
*Note = the file is in the same directory of the project
this is my code
import csv
with open('nora.csv', mode='r') as CSVFile:
read = csv.reader(CSVFile, delimiter =",")
print(read)
CSVFile.close()
thank you for your help in advance

Try using pandas to read csv files - it will place the csv information in a dataframe for you.
import pandas as pd
#Place the csv data into a DataFrame
file = pd.read_csv("/path/to/file.csv")
print file

i still don't know what that message meant
but i added a for loop to read instead of just adding print
and it worked
import csv
with open('players.csv', mode='r') as CSVFile:
read = csv.reader(CSVFile, delimiter =",")
for row in read :
print(row)
CSVFile.close()
thank you guys for helping <3 <3

Opening and reading an excel .xlsx file in python

I'm trying to open an excel .xlsx file with python but am unable to find a way to do it, I've tried using pandas but it's wanting to use a library called NumPy I've tried to install numpy but it still can't find numpy.
I've also tried using the xlrd library but I get the following traceback:
Traceback (most recent call last):
File "C:\test.py", line 3, in <module>
book = open_workbook('test.xlsx')
File "C:\Python27\lib\site-packages\xlrd\__init__.py", line 370, in open_workbook
biff_version = bk.getbof(XL_WORKBOOK_GLOBALS)
File "C:\Python27\lib\site-packages\xlrd\__init__.py", line 1323, in getbof
raise XLRDError('Expected BOF record; found 0x%04x' % opcode)
XLRDError: Expected BOF record; found 0x4b50
Which I assume is because XLRD can't read .xlsx files?
Anyone got any ideas?
EDIT:
import csv
with open('test.csv', 'rb') as csvfile:
data = csv.reader(csvfile, delimiter=',')
for row in data:
print "------------------"
print row
print "------------------"
for cell in row:
print cell

Maybe you could export your .xlsx to a .csv file?
Then you could try:
import csv
with open('file.csv','rb') as file:
contents = csv.reader(file)
[x for x in contents]
This may be useful:
http://docs.python.org/2/library/csv.html#csv.reader
Hope that helps!
EDIT:
If you want to locate a spectific cell, such as F13, you could make a nested list like a matrix and them refer to each element:
import csv
with open('file.csv','rb') as file:
contents = csv.reader(file)
matrix = list()
for row in contents:
matrix.append(row)
And then access F13 with matrix[5][12].
P.S.: I did not test this. If "row" is a list with each cell as an element, you keep appending all lines to the matrix, so the first index is row number and the second is the column number.

it seems that you are on a Linux Distro. I had the same problem too and this does not happen with "xlwt" library but only with "xlrd". what I did is not the right way to solve this problem but it makes things work for the time being to hopefully have an answer to that question soon ;I have installed "xlrd" on Windows and took the folder and pasted it on Linux in the directory where my python code is and it worked.

Since I know other people will also be reading this -
You can install the following module (it's not there automatically)
https://pypi.python.org/pypi/openpyxl
You can read the following to get a nice breakdown on how to use it
https://automatetheboringstuff.com/chapter12/