I have create a python project that update the data excel for my work, the problem is i need to create a config file so that others people can use my python project without changing the code.
My code:
import pandas as pd
import datetime
timestr = datetime.date.today().strftime('%d%m%Y')
Buyerpath = 'https://asd.cvs'
Sellerpath = 'https://dsa.csv'
Onlinepath = 'https://sda.csv'
Totalpath = pd.DataFrame({'BuyValue': Buyerpath.Buytotal,
'SellerValue': Sellerpath.Sellertotal,
'OnlineValue':Onlinepath.onlinetotal})
Totalpath.to_excel(index=False, excel_writer=r'C:\Users\Mike\Desktop\Result\ResultTotal'+timestr+'.xlsx')
I need the config file to allow others people use my python code and save the excel update in the output folder.
I think what you're asking is: how do I hide strings/variables that are specific to my local machine? correct?
The easiest way to do this is with another python file
my-project
├── __init__.py
├── main.py
└── secrets.py
put your the strings into you're secrets file, then import the values into your main.py file
from .secrets import secret_val
Related
As per the title i am having trouble accessing certain excel files in another folder based upon their filenames.
I have a folder containing a bunch of excel files which share a common name,but all have an individual datestamp appended on as a suffix in the format /files/filename_%d%m%Y.xlsx giving me a directory like this:
├── files
│ ├── filename_10102021.xlsx
│ ├── filename_07102021.xlsx
│ ├── filename_11102021.xlsx
│ └── filename_14102021.xlsx
├── notebooks
│ └── my_notebook.ipynb
From the my_notebook.ipynb file, I would like to navigate to the files directory, and get the 2 most recent excel files according to the suffixed date and open them in the notebook as pandas dfs so I can compare the columns for any differences. In the directory I provided above, the 2 files I would get are filename_14102021.xlsx and filename_11102021.xlsx but would like this solution to work dynamically as the files folder gets new files with new dates as time goes on. (so hardcoding those two names would not work)
My first thought is to do something like:
import os
import sys
import pandas as pd
sys.path.append('../files')
files = sorted(os.listdir(), reverse=True)
most_recent_df = pd.read_excel(files[0], engine='openpyxl', index_col=0)
second_most_recent_df = pd.read_excel(files[1], engine='openpyxl', index_col=0)
and then do my comparison between the dataframes.
However this code fails to do what I want as even with using sys.path.append, the os.listdir function returns a list of the notebooks directory which tells me the problem lies in this 2 lines:
sys.path.append('../files')
files = sorted(os.listdir(), reverse=True)
How do I fix my code to move into the files directory so that a list of all the excel files is returned?
thank you!
It should work directly using
files = sorted(os.listdir(r'path\to\folder'), reverse=True)
IMO you don't need to use sys.path.append
After using the example from the docs, for dcc.Upload, I am able to save the csv file in my project directory by using to_csv(location2), location2 is a file string to my project directory ('E:/dash_app/components/new.csv')
it important to note that the goal is to have a multipage project where I can toggle the displayed screen from a dropdown in a navbar. so the project should run correctly until the csv file is uploaded and there after it will display all the graphs and charts and information that is returned from the csv.file .
My project structure looks as follows:
Dash_app
__pycache__
.vscode
Components
Import_csv.py
csv_clean.py
app.py
Index.py
requirements
Under components directory there are many more components that get imported into index.py.
I import import_csv into index.py, the dcc.Upload component is in csv_upload.py, therefore by running index.py it runs import_csv.py where the csv file can be uploaded into my directory.
The problem is that I use functions from csv_clean.py to take the uploaded csv file as an input in order to return values so that I can use those values in other scrips in the 'components' directory (for graph and card modules).
The input to all the functions in csv_clean.py will be location2.
I receive the error: in " csv_clean.py data = pd.read(location)" no such file or directory exist : location2
The csv_clean.py script looks like as follows:
`` Import pandas as pd from components.import_csv import import location2
def daily_roi_income(file):
csv_df = pd.read_csv(file)
roi = csv_df.loc[csv_df.Remark=='Roi Income', ['Date', 'Credit']]
return roi
Roi = daily_roi_income(location2)
``
So this function cannot work unless the csv file exists. And I cannot run Index.py to upload the csv unless this works.
What is an approach to take if code is dependent on other code to run first.
Thanks in advance.
I have the following file structure:
app/
data/
data.csv
module/
__init__.py
script.py
__init__.py
config.py
My config class contains a set of Pandas dataframes read in from csv files. These dataframes are used across my code base:
import pandas as pd
class Config:
data = pd.read_csv('data/data.csv')
data2 = pd.read_csv('data/data2.csv')
My script contains:
from config import Config
Which returns:
FileNotFoundError: File b'data/data.csv' does not exist
if the run.py script is stored anywhere but the top level.
Is there any way to have class that has some stored file paths without causing a file not found error if importing the class in different scripts across the code base?
Try to specify the file in path resolved by the current file. For example, if your app/config.py is this:
import pandas as pd
class Config:
data = pd.read_csv('data/data.csv')
data2 = pd.read_csv('data/data2.csv')
You want to write the path like this:
mydir = os.path.dirname(os.path.abspath(__file__))
data = pd.read_csv(os.path.join(mydir, 'data/data.csv'))
data2 = pd.read_csv(os.path.join(mydir, 'data/data2.csv'))
which __file__ is the magic variable in Python to tell you which file you're in.
Basically, whenever your script try to open a file, it is relative to your current directory. Therefore, you need the above trick to find the file relative to the script's directory instead of current dir.
Let's say I have a folder containing the main-file main.py and a configuration-file config.json.
In the JSON-file there are multiple entries containing different filenames and the name of a method within each file.
Depending on user input from main.py I get the corresponding filename and the method from config.json and now I want to execute that method.
I tried the following
file = reports [id]["filename"]
method = reports [id]["method"]
# which would give me: file = 'test.py' and method = 'execute'
import file
file.method()
This didn't work obviously. The problem is that I can't know which files there are during compilation.
Other developers would add scripts to a specific folder and add their entries to the configuration file (kind of like a plugin). Does anybody have a solution here?
Thanks in advance
Thanks for your input... importlib was the way to go. I used this answer here:
How to import a module given the full path?
i am trying to import specified modules from test_file hierarchy
something like :
test_case1.py
test_subsuite_2
test_sub_2.1.1.py
test_suite2
is it possible to do a run import on this hierarchy
/project/main.py
/project/test_files
test_files folder hierarchy is like this :
test_files
test_suite1
test_case1.py
test_subsuite_1
test_sub1_1.py
test_sub1_2.py
test_subsuite_2
test_subsuite_2_1
test_sub_2.1.1.py
test_sub2_1.py
test_sub3_2.py
test_suite2
test_case2.py
test_subsuite2_1
test_sub21_1.py
test_sub21_2.py
test_subsuite2_2
test_sub22_1.py
test_sub23_2.py
Create an empty file called __init__.py in all you folders. Then you can import using . as a folder separator. Documentation here.
The key is to create a blank file __init__.py on all subfolders containing the files you want to import.
In your case, you will have to create the __init__.py files on all of the following folders-
test_files
test_files\test_suite1
test_files\test_suite1\test_subsuite_2
test_files\test_suite1\test_subsuite_2\test_subsuite_2_1
Also, when you import files be careful about specifying the import path correctly, with the entire path from the topmost level specified, and with different folder levels separated by a .
For example, you must import test_case1 by specifying :
from test_files.test_suite1 import test_case1
Similarly, test_subsuite_2 can be imported by specifying :
from test_files.test_suite1 import test_subsuite_2