Develop Reference

Trying to open a csv file that lives in the same directory as my Python script, but getting error2 file doesn't exist?

I am trying to open a CSV file that I recently saved using Python. Here is global directory:
So the folder is called Parsing Data Using Python, and there are 3 files inside, we only concern ourselves with the codealong.py file, which is some Python code that I want to run to open the 'patrons.csv' file.
Here is the code in the codealong.py file:
import csv
html_output = ''
with open('patrons.csv','r') as data_file:
csv_data = csv.reader(data_file)
print(list(csv_data))
When I run this, I get the Error2: No such file or directory: 'patrons.csv'
Any ideas why I am getting this? Because I am saving patrons.csv in the same directory as codealong.py I thought the file would be detected!

One approach would be to set the working directory to be the same location as where your script is. To do this for any script, add this to the top:
import os
os.chdir(os.path.dirname(os.path.abspath(__file__)))
This takes the full name of where your script is located, takes just the path element and sets the current working directory to that.
You then would not need to specify the full path to your file.

Another approach using pathlib.
import csv
from pathlib import Path
file = Path(__file__).parent / "patrons.csv"
with file.open("r", encoding="utf-8") as data_file:
csv_data = csv.reader(data_file)
print(list(csv_data))

How to load a file in Pandas from a parent directory

I've not used Pandas in a while but wanted to load a JSON file.
I've traditionally had an overarching directory (on Mac) called DataAnalyis and store all the data I've collected in folders describing what they contain.
I then created a folder called IPythonnotebooks in which I kept my scripts.
Loading a file - let's call it 'dummy.json' was trivial. It's in a folder called dummy.
The code was simple:
import pandas as pd
df = pd.read_json('../dummy/dummy.json')
That doesn't work any more. What have I got wrong?
Update:
DataAnalysis
---dummy
----dummy.json
---IPythonnotebooks
----dummy.pynb
Apologies if this is not the correct way to present file structure. I start up the notebook file in the folder IPythonnotebooks

Sometimes JupyterLab (and maybe other 'notebooks') start with diff cwd than you might think.
import os
os.getcwd()
check if it matches your '../dummy/dummy.json'
or check if this works:
import os
fullpath = os.path.realpath("dummy.json")
df = pd.read_json(fullpath)

Import file of semi-known name to pandas and return filename

I'm writing a script to import a csv using pandas, then upload it to a SQL server. It all works when I have a test file with one name, but that won't be the case in production. I need to figure out how to import with only a semi-known filename (the filename will always follow the same convention, but there will be differences in the filenames such as dates/times). I should, however, note that there will only ever be one csv file in the folder it's checking. I have tried the wildcard in the import path, but that didn't work.
After it's imported, I also then need to return the filename.
Thanks!

Look into the OS module:
import os
files = os.listdir("./")
csv_files = [filename for filename in files if filename.endswith(".csv")]
csv_files is a list with all the files that ends with .csv

how to open excel file in python using pandas?

How do you open an xls or csv file in python without having to connect the whole path?
ex: instead of using c:/user/...filename how do you connect it with just filename?
is it possible using pandas? This is in order to transfer the code from on console to another and the code being able to open with ease. From my understanding, if I use the path and send the code to another computer the excel page won't open there. btw the code will be sent with the original excel sheet

In this case, I believe you would have to set your working directory to the absolute path of your .py file.
Note, for the code below, your .csv file should be in the same directory as your .py file.
import os.path
import pandas as pd
base_dir = os.path.dirname(os.path.abspath(__file__)) # set directory to location of .py file
os.chdir(base_dir) # change directory
csv_file = pd.read_csv('file.csv',sep=',') # read .csv

Similar to solution of #Ira H., but instead of changing working directory you can generate full path:
import os.path
import pandas as pd
base_dir = os.path.dirname(
os.path.abspath(__file__)
) # set directory to location of .py file
csv_file = pd.read_csv(f"{base_dir}\\full_paths.csv", sep=",") # read .csv

Is there a way to be able to use a variable path using os

The goal is to run through a half stable and half variable path.
I am trying to run through a path (go to lowest folder which is called Archive) and fill a list with files that have a certain ending. This works quite well for a stable path such as this.
fileInPath='\\server123456789\provider\COUNTRY\CATEGORY\Archive
My code runs through the path (recursive) and lists all files that have a certain ending. This works well. For simplicity I will just print the file name in the following code.
import csv
import os
fileInPath='\\\\server123456789\\provider\\COUNTRY\\CATEGORY\\Archive
fileOutPath=some path
csvSeparator=';'
fileList = []
for subdir, dirs, files in os.walk(fileInPath):
for file in files:
if file[-3:].upper()=='PAR':
print (file)
The problem is that I can manage to have country and category to be variable e.g. by using *

The standard library module pathlib provides a simple way to do this.
Your file list can be obtained with
from pathlib import Path
list(Path("//server123456789/provider/".glob("*/*/Archive/*.PAR"))
Note I'm using / instead of \\ pathlib handles the conversion for you on windows.