I have a dash app with multiple pages.
.
├── adduser.py
├── app.py
├── apps/
│ ├── __init__.py
│ ├── exome.py
│ ├── familial.py
│ ├── oncomine.py
│ ├── prenatal.py
│ ├── rapidexome.py
│ └── targeted.py
├── auth.py
├── data/
├── dataprocessor.py
├── index.py
├── layouts.py
└── static
└── fileiconn.png
I just want to prompt the use for an admin password when they try to access the /apps/exome route. is there a way to do that?
I am trying to work from csv files located inside of a django app. I am trying to load the file using pandas like: pd.read_csv("...") without success, I keep getting an error.
Here is what the directory tree looks like:
├── __pycache__
│ ├── forms.cpython-36.pyc
│ ├── models.cpython-36.pyc
│ ├── views.cpython-36.pyc
│ └── urls.cpython-36.pyc
├── apps.py
├── files
│ ├── t1.csv
│ ├── t2.csv
│ ├── t3.csv
│ ├── t4.csv
│ └── parametre.csv
├── finished_apps.py
├── forms.py
├── migrations
│ ├── 0001_initial.py
│ ├── __init__.py
│ └── __pycache__
│ ├── 0001_initial.cpython-36.pyc
│ ├── 0002_remove_carriers_carriersheet.cpython-36.pyc
│ ├── 0003_auto_20211021_1200.cpython-36.pyc
│ ├── 0004_auto_20211021_1203.cpython-36.pyc
│ └── __init__.cpython-36.pyc
├── models.py
├── views.py
├── templates
│ ├── add_carrier.html
│ ├── base.html
│ ├── delete_carrier.html
│ ├── delete_carrier_confirmation.html
│ ├── _carrierdetails.html
│ ├── _carrierlist.html
│ ├── simulation.html
│ └── update_carrier.html
└── urls.py
I have tried the following inside of views.py
df = pd.read_csv("/files/t1.csv") #not working
df = pd.read_csv("./files/t1.csv") #not working
df = pd.read_csv("t1.csv") #not working
df = pd.read_csv("../files/t1.csv") #not working
I have also tried doing that:
from files import t1
the errors that I am getting are as such:
No such file or directory (/file/t1.csv) #for example
cannot import name 't1'
that's not working either.
I am now wondering whether is it possible to import a file this way or I am missing something obvious here!
Get the path to views.py from the __file__ variable and use that to find the path to your CSV:
import os
import pandas as pd
path = os.path.join(os.path.dirname(__file__), 'files/t1.csv')
df = pd.read_csv(path)
I'm using PyCharm and often rely on the alt+enter shortcut to automatically import classes and functions.
However, it doesn't use the absolute import path. It works fine locally but when I push to GitHub my tests fail in TravisCI.
Does anyone know a way to force PyCharm to import with the absolute path?
I need to import like this drone_squadron.api.drone_api if I use something like this api.drone_api the remote tests can't find the import. This is for all local imports.
I'd prefer for all imports to be absolute, all the time. Relative imports have caused me problems in packaging up projects. I think it's just easier to use absolute imports all the time.
Git Repo
https://github.com/sarcoma/drone_squadron_api_prototype
Tree Structure
.
├── coverage.xml
├── LICENSE.md
├── pytest.ini
├── README.md
├── requirements.txt
├── drone_squadron
│ ├── app.py
│ ├── endpoints.http
│ ├── flask.cfg
│ ├── __init__.py
│ ├── load_fixtures.py
│ ├── main.py
│ ├── router.py
│ ├── schema.py
│ ├── test_flask.cfg
│ ├── api
│ │ ├── base_api.py
│ │ ├── drone_api.py
│ │ ├── gimbal_api.py
│ │ ├── __init__.py
│ │ ├── price_api.py
│ │ ├── round_type_api.py
│ │ ├── scanner_api.py
│ │ ├── squadron_api.py
│ │ ├── steering_api.py
│ │ ├── thruster_api.py
│ │ ├── user_api.py
│ │ └── weapon_api.py
│ ├── authentication
│ │ ├── __init__.py
│ │ └── login.py
│ ├── crud
│ │ ├── base_crud.py
│ │ ├── drone_crud.py
│ │ ├── gimbal_crud.py
│ │ ├── __init__.py
│ │ ├── item_crud.py
│ │ ├── price_crud.py
│ │ ├── round_type_crud.py
│ │ ├── scanner_crud.py
│ │ ├── squadron_crud.py
│ │ ├── status_crud.py
│ │ ├── steering_crud.py
│ │ ├── thruster_crud.py
│ │ ├── user_crud.py
│ │ └── weapon_crud.py
│ ├── database
│ │ ├── database.py
│ │ ├── drones.db
│ │ ├── drones_test.db
│ │ └── __init__.py
│ ├── enums
│ │ ├── __init__.py
│ │ ├── round_type.py
│ │ └── status.py
│ ├── error
│ │ ├── error.py
│ │ └── __init__.py
│ ├── fixtures
│ │ ├── gimbal_fixtures.py
│ │ ├── __init__.py
│ │ ├── round_type_fixtures.py
│ │ ├── scanner_fixtures.py
│ │ ├── status_fixtures.py
│ │ ├── steering_fixtures.py
│ │ ├── thruster_fixtures.py
│ │ ├── user_fixtures.py
│ │ └── weapon_fixtures.py
│ ├── model
│ │ ├── base_model.py
│ │ ├── drone_model.py
│ │ ├── __init__.py
│ │ └── squadron_model.py
│ ├── request
│ │ ├── __init__.py
│ │ └── json_request_handler.py
│ ├── response
│ │ ├── __init__.py
│ │ └── json_response.py
│ ├── service
│ │ └── calculate_cost.py
│ ├── transformer
│ │ ├── __init__.py
│ │ ├── json_transformer.py
│ │ └── transformer.py
│ └── validation
│ ├── abstract
│ │ ├── __init__.py
│ │ └── validation_abstract.py
│ ├── drone_validation.py
│ ├── field.py
│ ├── __init__.py
│ ├── validation_link.py
│ └── validations.py
└── tests
├── drones_test.db
├── __init__.py
├── test_api
│ ├── conftest.py
│ ├── __init__.py
│ ├── test_auth.py
│ ├── test_drone.py
│ ├── test_gimbal.py
│ ├── test_price_list.py
│ ├── test_round_type.py
│ ├── test_scanner.py
│ ├── test_squadron.py
│ ├── test_steering.py
│ ├── test_thruster.py
│ └── test_weapon.py
└── test_crud
├── conftest.py
├── __init__.py
├── test_drone_crud.py
├── test_gimbal_crud.py
├── test_scanner_crud.py
├── test_squadron_crud.py
├── test_status_crud.py
├── test_steering_crud.py
├── test_thruster_crud.py
├── test_user_crud.py
└── test_weapon_crud.py
In Python, imports can be relative or absolute.
Absolute imports are resolved from your project's root directory:
import drone_squadron.api.drone_api
Relative imports are resolved from the current python package.
import ..api.drone
In your case, the issue is NOT a relative/absolute confusion, PyCharm always add absolute imports.
The problem is that PyCharm probably consider the folder drone_squadron in your project as a "root directory". That's wrong! The root directory is the top-level folder corresponding to the whole git project (the folder containing LICENSE.md, README.md, etc.)
In PyCharm, right click on the folder drone_squadron, then open sub-menu Mark directory as (in the bottom) then select Unmark as Source Root.
After that action, your import will be added the way you want.
i am trying to write a function in python for comparing two folders (with exactly same subdirectory structure and file lists).
The folder is bound to contain .c .h .txt .cat .sys .pdb, but the major concentration is on C and header files.
The output of this diff(folder1, folder2) should return the following
print new added driver files (c and .h alone) in folder2
print deleted driver files in folder 2 as compared to folder 1
(this can be done by two lever for loop storing os.walk results in two lists and subsequently subtracting them )
my challenge :diff() function should start comparing files in all subdirectories and if it finds even one (c or h) files modified in folder2, it should end the diff routine and return a FLAG = 1
eg.
folder1\foo\bar\1.c folder2\foo\bar\1.c --> same
folder1\foo\car\2.c folder2\foo\car\2.c --> same
folder1\foo\dar\13.c folder2\foo\dar\13.c --> different -> return flag=1
folder1\foo\far\211.c folder2\foo\far\211.c --> not compared
I have tried to use os.walk(path) function to do this and store all the files in two seperate list. but i find it extremely long and complicated for multiple files in this location.
Also, if there is a method to ignore perforce headers, comments, extra spacing in comparison it would enhance my script
Any advice greatly appreciated
You should be able to accomplish this using Python's filecmp library.
EDITED ANSWER
Addressing additional comments by #DennisNinj
Thanks, Is there anyway to include .c and .h for file comparison? I
have more than 20 types of files in each folder and over 1000 files in
each folder? – Dennis Ninj
#DennisNinj Yes, it's possible, just a bit more tricky. The currently published version of filecmp.dircmp does not support wildcard or regex matching for its "ignore" and "hide" filters. (There's been a patch submitted to support wildcards in dircmp.) So it means you have to do the filtering manually.
Here is an updated example that gets you closer to what you're looking to accomplish.
ATTENTION: Do note that due to the requirement to stop method execution once a differing C or header file is found, there's a possibility you won't get the file listings of every "added / deleted driver" available in the compared directories since it may not have had a chance to traverse all sub directories.
ccodedircomparison.py
import re
from filecmp import dircmp
def main():
dcmp = dircmp("/Users/joeyoung/web/stackoverflow/dircomparison/test1", "/Users/joeyoung/web/stackoverflow/dircomparison/test2")
if diffs_found(dcmp):
print "FLAG = 1"
def diffs_found(dcmp):
c_files_regex = re.compile(r".*\.[ch]$")
deleted_drivers = []
if len(dcmp.left_only) > 0:
for left_only_file in dcmp.left_only:
c_files_match = c_files_regex.match(left_only_file)
if c_files_match:
deleted_drivers.append(left_only_file)
if len(deleted_drivers) > 0:
print "Drivers deleted from {dirname}: [{deleted_drivers_list}]".format(dirname=dcmp.right, deleted_drivers_list=', '.join(deleted_drivers))
added_drivers = []
if len(dcmp.right_only) > 0:
for right_only_file in dcmp.right_only:
c_files_match = c_files_regex.match(right_only_file)
if c_files_match:
added_drivers.append(left_only_file)
if len(added_drivers) > 0:
print "Drivers added to {dirname}: [{added_drivers_list}]".format(dirname=dcmp.right, added_drivers_list=', '.join(dcmp.right_only))
if len(dcmp.diff_files) > 0:
differing_c_files = []
for diff_file in dcmp.diff_files:
c_files_match = c_files_regex.match(diff_file)
if c_files_match:
differing_c_files.append(diff_file)
if len(differing_c_files) > 0:
print "C files whose content differs ({dirname}): [{differing_c_files}]".format(dirname=dcmp.right, differing_c_files=', '.join(differing_c_files))
return True
for sub_dcmp in dcmp.subdirs.values():
return diffs_found(sub_dcmp)
return False
if __name__ == '__main__':
main()
Example output
(.virtualenvs)macbook:dircomparison joeyoung$ python ccodedircomparison.py
Drivers deleted from /Users/joeyoung/web/stackoverflow/dircomparison/test2/support: [thisismissingfromtest2.c]
Drivers added to /Users/joeyoung/web/stackoverflow/dircomparison/test2/support: [addedfile1.h]
C files whose content differs (/Users/joeyoung/web/stackoverflow/dircomparison/test2/support): [samefilenamedifftext1.h, samefilename1.c]
FLAG = 1
Test environment directory structure
(.virtualenvs)macbook:dircomparison joeyoung$ tree test1 test2
test1
├── affected.test.js
├── blob.test.js
├── cache.test.js
├── constants.test.js
├── database_fail.test.js
├── each.test.js
├── exec.test.js
├── extension.test.js
├── fts-content.test.js
├── issue-108.test.js
├── map.test.js
├── named_columns.test.js
├── named_params.test.js
├── null_error.test.js
├── nw
│ ├── Makefile
│ ├── index.html
│ ├── package.json
│ ├── thisismissingfromtest2.c
│ └── thisismissingfromtest2.txt
├── open_close.test.js
├── other_objects.test.js
├── parallel_insert.test.js
├── prepare.test.js
├── profile.test.js
├── rerun.test.js
├── scheduling.test.js
├── serialization.test.js
├── support
│ ├── createdb.js
│ ├── elmo.png
│ ├── helper.js
│ ├── onlyintest1.txt
│ ├── prepare.db
│ ├── samefilename1.c
│ ├── samefilename1.txt
│ ├── samefilenamedifftext1.h
│ ├── samefilenamedsametext1.h
│ ├── script.sql
│ ├── thisismissingfromtest2.c
│ └── thisismissingfromtest2.txt
├── trace.test.js
└── unicode.test.js
test2
├── affected.test.js
├── blob.test.js
├── cache.test.js
├── constants.test.js
├── database_fail.test.js
├── each.test.js
├── exec.test.js
├── extension.test.js
├── fts-content.test.js
├── issue-108.test.js
├── map.test.js
├── named_columns.test.js
├── named_params.test.js
├── null_error.test.js
├── nw
│ ├── Makefile
│ ├── index.html
│ └── package.json
├── open_close.test.js
├── other_objects.test.js
├── parallel_insert.test.js
├── prepare.test.js
├── profile.test.js
├── rerun.test.js
├── scheduling.test.js
├── serialization.test.js
├── support
│ ├── addedfile1.h
│ ├── createdb.js
│ ├── elmo.png
│ ├── helper.js
│ ├── prepare.db
│ ├── samefilename1.c
│ ├── samefilename1.txt
│ ├── samefilenamedifftext1.h
│ ├── samefilenamedsametext1.h
│ └── script.sql
├── trace.test.js
└── unicode.test.js
ORIGINAL ANSWER BEFORE THE EDIT IS BELOW
My example doesn't do exactly what you describe, but there should be enough between this example and the filecmp.dircmp() documentation to get you started.
dircomparison.py
from filecmp import dircmp
def main():
dcmp = dircmp("/Users/joeyoung/web/stackoverflow/dircomparison/test1", "/Users/joeyoung/web/stackoverflow/dircomparison/test2")
if diffs_found(dcmp):
print "DIFFS FOUND!"
else:
print "NO DIFFS FOUND"
def diffs_found(dcmp):
if len(dcmp.left_only) > 0:
print dcmp.report_full_closure()
return True
elif len(dcmp.right_only) > 0:
print dcmp.report_full_closure()
return True
else:
for sub_dcmp in dcmp.subdirs.values():
if diffs_found(sub_dcmp):
return True
return False
if __name__ == '__main__':
main()
Example output
(.virtualenvs)macbook:dircomparison joeyoung$ python dircomparison.py
diff /Users/joeyoung/web/stackoverflow/dircomparison/test1/support /Users/joeyoung/web/stackoverflow/dircomparison/test2/support
Only in /Users/joeyoung/web/stackoverflow/dircomparison/test1/support : ['onlyintest1.txt']
Identical files : ['createdb.js', 'elmo.png', 'helper.js', 'prepare.db', 'script.sql']
None
DIFFS FOUND!
The actual directory structures
So you can see what my test environment looked like.
(.virtualenvs)macbook:dircomparison joeyoung$ tree test1
test1
├── affected.test.js
├── blob.test.js
├── cache.test.js
├── constants.test.js
├── database_fail.test.js
├── each.test.js
├── exec.test.js
├── extension.test.js
├── fts-content.test.js
├── issue-108.test.js
├── map.test.js
├── named_columns.test.js
├── named_params.test.js
├── null_error.test.js
├── nw
│ ├── Makefile
│ ├── index.html
│ └── package.json
├── open_close.test.js
├── other_objects.test.js
├── parallel_insert.test.js
├── prepare.test.js
├── profile.test.js
├── rerun.test.js
├── scheduling.test.js
├── serialization.test.js
├── support
│ ├── createdb.js
│ ├── elmo.png
│ ├── helper.js
│ ├── onlyintest1.txt
│ ├── prepare.db
│ └── script.sql
├── trace.test.js
└── unicode.test.js
2 directories, 33 files
(.virtualenvs)macbook:dircomparison joeyoung$ tree test2
test2
├── affected.test.js
├── blob.test.js
├── cache.test.js
├── constants.test.js
├── database_fail.test.js
├── each.test.js
├── exec.test.js
├── extension.test.js
├── fts-content.test.js
├── issue-108.test.js
├── map.test.js
├── named_columns.test.js
├── named_params.test.js
├── null_error.test.js
├── nw
│ ├── Makefile
│ ├── index.html
│ └── package.json
├── open_close.test.js
├── other_objects.test.js
├── parallel_insert.test.js
├── prepare.test.js
├── profile.test.js
├── rerun.test.js
├── scheduling.test.js
├── serialization.test.js
├── support
│ ├── createdb.js
│ ├── elmo.png
│ ├── helper.js
│ ├── prepare.db
│ └── script.sql
├── trace.test.js
└── unicode.test.js
2 directories, 32 files
Here's a rough outline of what I would do:
Create a set() of folder1 file names folder1_set
os.walk() is your friend, load up folder1 and start to walk the files.
for each file, open it. Modifying the path to point to the second folder, open the 2nd file and check for equality
Add the file name to folder1_set
Finally, we have to keep a track of files which could be missing in folder2. Using os.walk() on folder2, you could keep another set() of filenames folder2_set
folder1_set - folder2_set would give you any items in folder1, not in folder2 and vice versa would give you the opposite.
Edit 1
To prepare for comments you can read the file in line by line, using .strip() on the line to remove whitespace at either end, then check for the presence of /* at the start of the stripped line
For single line comments you must also check if */ is present at the end of the line and if so, exclude it.
For multi line comments you can ignore all lines until you come across */ at the end of the line.
This also has the benefit of clearing out empty lines (removes all spaces).
Check out os.path.splitext to easily find the file extensions whilst walking.
My project layout is as follows:
├ ...
├── pve
│ ├── blahblah
│ │ ├── TestDefinition.py
│ │ ├── TestDefinition.pyc
│ │ ├── __init__.py
│ │ └── __init__.pyc
│ └── pve.py
├── src
│ └── definitions
│ └── THISFILE.yml
└── ...
I need to be able to fetch files (THISFILE.yml for example) from src/definitions by the pve/blahblah/TestDefinition.py class.
How do I properly access the project root? Once I have that, I can access the .yml files relative.
TIA.
I like to create some sort of configuration file in the project root that has an aboslute path to the root. I do this only because the frameworks i usually use (django, scrapy) have some sort of convention like this
├ ...
├── pve
│ ├── blahblah
│ │ ├── TestDefinition.py
│ │ ├── TestDefinition.pyc
│ │ ├── __init__.py
│ │ └── __init__.pyc
│ └── pve.py
├── src
│ └── definitions
│ └── THISFILE.yml
└── settings.py
# settings.py
import os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
DEFINITIONS_ROOT = os.path.join(PROJECT_ROOT, 'src', 'definitions')
from myproject import settings
settings.DEFINITIONS_ROOT