Develop Reference

Issue running python script in separate folder

I Have a directory structured as follows:
application
├── app
│ └── folder
│ └── file_1.py
│ └── Model_data
│ └──data.csv
└── app2
└── some_folder
└── file_2.py
I want to import a function from file_1 inside of file_2. I use:
from application.app.folder.file_1 import load_data
t = load_data()
the problem is that this returns an error. Within the function load_data I call pandas and import csv data from a sub-folder.
df = pd.read_csv('Model_data/data.csv')
this returns a "file doesn't exist error".
how do I resolve this?
file_1 runs fine from within the directory.

You can try changing 'Model_data/data.csv' to its absolute path. example C:/application/app/folder/Model_data/data.csv

You can use a relative path from file_1.py:
from pathlib import Path
def load_data():
file_1_path = Path(__file__)
filename = file_1_path.parent / "Model_data" / "data.csv"
df = pd.read_csv(filename)

Export specific CSV filename (eg: abc*.csv) from list of folders within a folder using python through looping method

How to iterate through multiple folders within a folder and export specific CSV filename that begins for example- "abc*.csv" and export them into a new folder directory.
Trying to search similar example in Stack overflow but most examples were reading multiple CSV files within a folder and combine them to one data frame. Thanks.

import os
import shutil
files = []
source_dir = ['./files/']
dest_dir = './export/'
# List all files from all directories within the path
while len(source_dir) > 0:
for (dirpath, dirnames, filenames) in os.walk(source_dir.pop()):
source_dir.extend(dirnames)
files.extend(map(lambda n: os.path.join(
*n), zip([dirpath] * len(filenames), filenames)))
# Loop thru files to copy/move the matching CSVs
for file in files:
if file.startswith('abc') and file.endswith('.csv'):
shutil.copy(file, dest_dir)
# shutil.move(file, dest_dir) # Use .move to move the file instead
If you have this file structure:
files
├── dir1
│   ├── abc789.csv
│   └── abc789.txt
├── dir2
│   ├── abc098.csv
│   └── abc098.txt
└── dir3
├── abc456.csv
└── subdir3
├── abc123.csv
└── abc123.txt
Only the matching files will be exported:
$ ls export/
abc098.csv abc123.csv abc456.csv abc789.csv

How to get filepath directory and use it to read my excel file? (Mac)

I'm creating a basketball data visualization app, and I've already completed the GUI, now just trying to import my database which is an excel file. I'm using pandas, and when I run this code, I get the "No such file or directory" error. I understand I must get the filepath, but how do I do this (Mac OS X) and implement it to direct my code to my file?
I tried directly copying and pasting the filepath with path = r'C:(insert path here)'
#Basketball DataVis (Data Visualization)
#pylint:disable = W0614
#By Robert Smith
#Import
import tkinter
import os
import pandas as pd
from tkinter import *
from PIL import Image, ImageTk
from pandas import *
#Import the excel file to use as a database
data = pd.read_excel("nbadata.xlsx", sheetname= "Sheet1")

Easiest way is to open an instance of the terminal and then drag the file into the terminal screen - this will print the path which you can then use in your script.
Note that mac filepaths don't begin with C:

I will suggest you to use recursive approach to solve your problem if you don't know where is your xlsx file (so you can't provide relative or absolute path) but you know the exact name of it and you also know the root directory under which this file exists.
For this kind of scenario, just pass the root path and filename to the recursive function and it will give a list of absolute paths of all matched file names.
Finally you can choose the 1st one from that list if you are sure there're no more files with the same name or you can print the list on console and retry.
I found this method best in my case and I have presented a simple example for that as follows.
Directory structure:
H:\RishikeshAgrawani\Projects\GenWork\Python3\try\test>tree . /f
Folder PATH listing for volume New Volume
Volume serial number is C867-828E
H:\RISHIKESHAGRAWANI\PROJECTS\GENWORK\PYTHON3\TRY\TEST
│ tree
│
├───c
│ docs.txt
│
├───cpp
│ docs.md
│
├───data
│ nbadata.xlsx
│
├───js
│ docs.js
│
├───matlab
│ docs.txt
│
├───py
│ │ docs.py
│ │
│ └───docs
│ docs.txt
│
└───r
docs.md
Here is the recursive implementation, please have a look and try.
import os
def search_file_and_get_abspaths(path, filename):
"""
Description
===========
- Gives list of absolute path of matched file names by performing recursive search
- [] will be returned in there is no such file under the given path
"""
matched_paths = []
if os.path.isdir(path):
files = os.listdir(path)
for file in files:
fullpath = os.path.join(path, file)
if os.path.isdir(fullpath):
# Recusive search in child directories
matched_paths += search_file_and_get_abspaths(fullpath, filename)
elif os.path.isfile(fullpath):
if fullpath.endswith(filename):
if not path in matched_paths:
matched_paths.append(fullpath)
return matched_paths
if __name__ == "__main__":
# Test case 1 (Multiple files exmample)
matched_paths = search_file_and_get_abspaths(r'H:\RishikeshAgrawani\Projects\GenWork\Python3\try\test', 'docs.txt');
print(matched_paths)
# Test case 2 (Single file example)
matched_paths2 = search_file_and_get_abspaths(r'H:\RishikeshAgrawani\Projects\GenWork\Python3\try\test', 'nbadata.xlsx');
print(matched_paths2)
# ['H:\\RishikeshAgrawani\\Projects\\GenWork\\Python3\\try\\test\\c\\docs.txt', 'H:\\RishikeshAgrawani\\Projects\\GenWork\\Python3\\try\\test\\matlab\\docs.txt', 'H:\\RishikeshAgrawani\\Projects\\GenWork\\Python3\\try\\test\\py\\docs\\docs.txt']
if matched_paths2:
xlsx_path = matched_paths2[0] # If your file name is unique then it will only be 1
print(xlsx_path) # H:\RishikeshAgrawani\Projects\GenWork\Python3\try\test\data\nbadata.xlsx
data = pd.read_excel(xlsx_path, sheetname= "Sheet1")
else:
print("Path does not exist")

File not found error while using pandas in django

I have written a naive bayes classifier for text messages and it's script is as follows:
tester.py
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score
df = pd.read_table('a.txt', sep='\t', header=None, names=['label', 'text'])
...
On testing, it worked correctly. Now I have created a django project where this script and the a.txt file are placed alongside views.py and urls.py. When user enters a comment, it is processed in views file as:
views.py
from .tester import predictor
def result(request):
content = request.POST['content']
res = predictor(content)
status = ''
if res == 0:
status = 'not spam'
else:
status = 'spam'
return render(request, 'spammer/result.html', {'status':status,})
Where predictor is a function I have added to tester.py:
def predictor(comment):
tester = [comment]
contest = count_vector.transform(tester) #count_vector=CountVectorizer()
a = naive_bayes.predict(contest) #naive_bayes=MultinomialNB()
return a[0]
However on running the server, there is an error:
File "pandas/_libs/parsers.pyx", line 384, in pandas._libs.parsers.TextReader.__cinit__
File "pandas/_libs/parsers.pyx", line 695, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: File b'a.txt' does not exist
which is not the case. Where am I going wrong? I have installed pandas,scipy,sklearn in virtual environment using pip and tester.py as well as a.txt are in the same directory as views.py,urls.py

Because your filesystem looks like:
yourproject
├── yourapp
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ ├── views.py
│ └── a.txt
└── manage.py
And you run it like
python manage.py runserver
So your working directory is yourproject/, while your file is relatively located at yourapp/a.txt.
What counts it's your working directory, aka the directory from where you run the python command. Not the current Python file location.

How to copy files from a directory based on items in a list of filenames in Python

I'm new to Python and I have created following function to copy files from the directory(livepromptDir) based on the item in the list(promptList). So far, it only copy the first item in the list to the destination directory. Please help! Thanks in advance.
def copyItemToPrompt():
#This function will copy all of the appropriate Voice Prompt files from LivePrompts directory to promptDir based on promptList
os.chdir(livepromptDir)
try:
for i in range(0,len(promptList)):
for filename in fnmatch.filter(os.listdir(livepromptDir), promptList[i]):
shutil.copy(filename, promptDir)
return
except Exception as error:
log(logFile, 'An error has occurred in the copyLiveToPrompt function: ' + str(error))
raise

You want to move the return outside of the for loop, otherwise your function returns after the first iteration. Actually, you don't even need the return:
def copyItemToPrompt():
"""This function will copy all of the appropriate Voice Prompt files from LivePrompts directory to promptDir based on promptList"""
os.chdir(livepromptDir)
try:
for i in range(0,len(promptList)):
for filename in fnmatch.filter(os.listdir(livepromptDir), promptList[i]):
shutil.copy(filename, promptDir)
except Exception as error:
log(logFile, 'An error has occurred in the copyLiveToPrompt function: ' + str(error))
raise

As #rcriii mentioned, the return is what's short-circuiting your function. I'm not certain what you're trying to accomplish, but I think you just want to copy a list of files from one dir to another given a list of glob-patterns.
If that's the case, and given you have a dir like this:
.
├── a
│   ├── file1
│   ├── file2
│   └── tmp3
└── b
This function should give you a bit cleaner way to do this (things like for i in range... generally aren't used like you have here.) Also, changing dirs can sometimes give you problems in the future if you fail to change back.
import shutil
from itertools import chain
from os import path
from glob import glob
def copy_with_patterns(src, dest, patterns):
# add src dir to given patterns
patterns = (path.join(src, x) for x in patterns)
# get filtered list of files
files = set(chain.from_iterable(glob(x) for x in patterns))
# copy files
for filename in files:
shutil.copy(filename, filename.replace(src, dest))
Invoking this function like this:
copy_with_patterns('a', 'b', ['file*'])
Will make your dir now look like this:
.
├── a
│   ├── file1
│   ├── file2
│   └── tmp3
└── b
├── file1
└── file2