Develop Reference

Using relative paths in Python with AWS Cloud9 [duplicate]

This question already has answers here:
Reading a file using a relative path in a Python project
(6 answers)
Closed 2 years ago.
Using AWS Cloud9 for a Python 3.x application. I am trying to open a file (using with open) in the same directory as the python file, however, it only works if I define the absolute path.
Relative Path
import csv
with open("test.csv", newline='') as f:
reader = csv.reader(f, delimiter=' ', quotechar='|')
for row in reader:
print(', '.join(row))
Error in Terminal
Traceback (most recent call last):
File "/home/ec2-user/environment/test/test.py", line 3, in <module>
with open("test.csv", newline='') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test.csv'
Absolute Path
import csv
with open("/home/ec2-user/environment/test/test.csv", newline='') as f:
reader = csv.reader(f, delimiter=' ', quotechar='|')
for row in reader:
print(', '.join(row))
No Errors

Found a similar question, posted an answer below that works. Reading file using relative path in python project
import csv
from pathlib import Path
path = Path(__file__).parent / "test.csv"
with path.open() as f:
reader = list(csv.reader(f, delimiter=' ', quotechar='|'))
for row in reader:
print(', '.join(row))

I can't comment so I am gonna answer here and hope it is right for you.
AWS uses Linux OSes, if you want to use a file in Linux in the same folder the script you are running, you have to prepend ./ in the file's name, e.g. in your case:
import csv
with open("./test.csv", newline='') as f:
reader = csv.reader(f, delimiter=' ', quotechar='|')
for row in reader:
print(', '.join(row))

CSV NamedTemporaryFile not saving properly

I'm trying to make a csv file using Python's csv and tempfile tools. I've been declaring it as follows:
csvattachment = tempfile.NamedTemporaryFile(suffix='.csv', prefix=('student_' + studentID), delete=False)
with open(csvattachment.name, 'w+') as csvfile:
filewriter = csv.writer(csvfile, delimiter=',')
filewriter.writerow([ #WRITE CONTENT HERE])
What I am then doing after is attaching this file and sending it out. The problem with that is that instead of being called 'student_1736823.csv' the attachment name is something uglier like <tempfile._TemporaryFileWrapper object at 0x10cbf5e48>

The NamedTemporaryFile() class already returns an open file, you don't have to reopen it
with tempfile.NamedTemporaryFile(suffix='.csv', prefix=('student_' + studentID),
delete=False, mode='w+') as csvfile:
filewriter = csv.writer(csvfile, delimiter=',')
filewriter.writerow([ #WRITE CONTENT HERE])

Open file has data but reports back length 0 in python

I must be missing something very simple here, but I've been hitting my head against the wall for a while and don't understand where the error is. I am trying to open a csv file and read the data. I am detecting the delimiter, then reading in the data with this code:
with open(filepath, 'r') as csvfile:
dialect = csv.Sniffer().sniff(csvfile.read())
delimiter = repr(dialect.delimiter)[1:-1]
csvdata = [line.split(delimiter) for line in csvfile.readlines()]
However, my csvfile is being read as having no length. If I run:
print(sum(1 for line in csvfile))
The result is zero. If I run:
print(sum(1 for line in open(filepath, 'r')))
Then I get five lines, as expected. I've checked for name clashes by changing csvfile to other random names, but this does not change the result. Am I missing a step somewhere?

You need to move the file pointer back to the start of the file after sniffing it. You don't need to read the whole file in to do that, just enough to include a few rows:
import csv
with open(filepath, 'r') as f_input:
dialect = csv.Sniffer().sniff(f_input.read(2048))
f_input.seek(0)
csv_input = csv.reader(f_input, dialect)
csv_data = list(csv_input)
Also, the csv.reader() will do the splitting for you.

python code for Exporting scraped data to CSV

import csv
in_txt = csv.reader(open(post.text, "rb"), delimiter = '\t')
out_csv = csv.writer("C:\Users\sptechsoft\Documents\source3.csv", 'wb')
out_csv.writerows(in_txt)
when executing above code i am getting IO error and i need to save in CSV in seperate folder

You dont need to open file before passing it to csvreader.
You can directly pass the file to csvreader and it would work
import csv
in_txt = csv.reader("post.text", "rb", delimiter = '\t')
out_csv = csv.writer("C:\Users\sptechsoft\Documents\source3.csv", 'wb')
out_csv.writerows(in_txt)

Try the following:
import csv
with open(post.text, "rb") as f_input, open(r"C:\Users\sptechsoft\Documents\source3.csv", "wb") as f_output:
in_csv = csv.reader(f_input, delimiter='\t')
out_csv = csv.writer(f_output)
out_csv.writerows(in_csv)
The csv.reader() and csv.writer() needs either a list or a file object. It cannot open the file for you. By using with it ensures the files are correctly closed automatically afterwards.
Also do not forget to prefix your path string with r to disable any string escaping due to the backslashes.

erroneous line added while adding new columns python

I am trying to add extra columns in a csv file after processing an input csv file. But, I am getting extra new line added after each line in the output.
What's missing or wrong in my below code -
import csv
with open('test.csv', 'r') as infile:
with open('test_out.csv', 'w') as outfile:
reader = csv.reader(infile, delimiter=',')
writer = csv.writer(outfile, delimiter=',')
for row in reader:
colad = row[5].rstrip('0123456789./ ')
if colad == row[5]:
col2ad = row[11]
else:
col2ad = row[5].split(' ')[-1]
writer.writerow([row[0],colad,col2ad] +row[1:])
I am processing huge a csv file so would like to get rid of those extra lines.

I had the same problem on Windows (your OS as well, I presume?). CSV and Windows as combination make a \r\r\n at the end of each line (so: double newline).
You need to open the output file in binary mode:
with open('test_out.csv', 'wb') as outfile:
For other answers:
Python's CSV writer produces wrong line terminator
CSV in Python adding an extra carriage return