I was trying to process some data in a GitHub action.
However, due to a Japanese file name, I can not read the file successfully by:
pd.read_csv('C:\\202204_10エリア計.csv')
And I was trying to rename it before read it by:
for filename in os.listdir(download_path):
if filename.startswith('202204'):
filename = filename.encode('utf-8').decode(locale.getpreferredencoding(False))
print(filename) # this print '202204_10エリア計.csv' on github action
os.rename(os.path.join(download_path, filename), os.path.join(download_path, '202204.csv')
But it gets error:
[WinError 2] The system cannot find the file specified: 'C:\\202204_10エリア計.csv' -> 'C:\\202204.csv'
You should be able to sidestep the encoding/decoding entirely with pathlib:
from pathlib import Path
fn = next(p for p in Path(download_path).glob("*.csv") if p.name.startswith("202204"))
fn.rename(fn.with_stem("202204"))
This is a bit of a workaround to whatever the real issue is, however.
That said I have never needed to meddle with the encoding when using os.path, and a quick search of the docs doesn't turn up anything, so you may be fine if you simple remove your encoding/decoding step. I would expect the os.path api to use the same internal representation throughout.
Related
I am working on the listener portion of a backdoor program (for an ETHICAL hacking course) and I would like to be able to read files from any part of my linux system and not just from within the directory where my listener python script is located - however, this has not proven to be as simple as specifying a typical absolute path such as "~/Desktop/test.txt"
So far my code is able to read files and upload them to the virtual machine where my reverse backdoor script is actively running. But this is only when I read and upload files that are in the same directory as my listener script (aptly named listener.py). Code shown below.
def read_file(self, path):
with open(path, "rb") as file:
return base64.b64encode(file.read())
As I've mentioned previously, the above function only works if I try to open and read a file that is in the same directory as the script that the above code belongs to, meaning that path in the above content is a simple file name such as "picture.jpg"
I would like to be able to read a file from any part of my filesystem while maintaining the same functionality.
For example, I would love to be able to specify "~/Desktop/another_picture.jpg" as the path so that the contents of "another_picture.jpg" from my "~/Desktop" directory are base64 encoded for further processing and eventual upload.
Any and all help is much appreciated.
Edit 1:
My script where all the code is contained, "listener.py", is located in /root/PycharmProjects/virus_related/reverse_backdoor/. within this directory is a file that for simplicity's sake we can call "picture.jpg" The same file, "picture.jpg" is also located on my desktop, absolute path = "/root/Desktop/picture.jpg"
When I try read_file("picture.jpg"), there are no problems, the file is read.
When I try read_file("/root/Desktop/picture.jpg"), the file is not read and my terminal becomes stuck.
Edit 2:
I forgot to note that I am using the latest version of Kali Linux and Pycharm.
I have run "realpath picture.jpg" and it has yielded the path "/root/Desktop/picture.jpg"
Upon running read_file("/root/Desktop/picture.jpg"), I encounter the same problem where my terminal becomes stuck.
[FINAL EDIT aka Problem solved]:
Based on the answer suggesting trying to read a file like "../file", I realized that the code was fully functional because read_file("../file") worked without any flaws, indicating that my python script had no trouble locating the given path. Once the file was read, it was uploaded to the machine running my backdoor where, curiously, it uploaded the file to my target machine but in the parent directory of the script. It was then that I realized that problem lied in the handling of paths in the backdoor script rather than my listener.py
Credit is also due to the commentator who pointed out that "~" does not count as a valid path element. Once I reached the conclusion mentioned just above, I attempted read_file("~/Desktop/picture.jpg") which failed. But with a quick modification, read_file("/root/Desktop/picture.jpg") was successfully executed and the file was uploaded in the same directory as my backdoor script on my target machine once I implemented some quick-fix code.
My apologies for not being so specific; efforts to aid were certainly confounded by the unmentioned complexity of my situation and I would like to personally thank everyone who chipped in.
This was my first whole-hearted attempt to reach out to the stackoverflow community for help and I have not been disappointed. Cheers!
A solution I found is putting "../" before the filename if the path is right outside of the dictionary.
test.py (in some dictionary right inside dictionary "Desktop" (i.e. /Desktop/test):
with open("../test.txt", "r") as test:
print(test.readlines())
test.txt (in dictionary "/Desktop")
Hi!
Hello!
Result:
["Hi!", "Hello!"]
This is likely the simplest solution. I found this solution because I always use "cd ../" on the terminal.
This not only allows you to modify the current file, but all other files in the same directory as the one you are reading/writing to.
path = os.path.dirname(os.path.abspath(__file__))
dir_ = os.listdir(path)
for filename in dir_:
f = open(dir_ + '/' + filename)
content = f.read()
print filename, len(content)
try:
im = Image.open(filename)
im.show()
except IOError:
print('The following file is not an image type:', filename)
I was wondering if it was possible to get the file path of a temporary file made using the tempfile library. Basically, I'm trying to make a function that intakes some data, and generates a temporary csv file based off of said data. I was wondering if there was a way to get the path of this temporary file?
Use tempfile.NamedTemporaryFile to create a temporary file with a name, and then use the .name attribute of the object.
Note that there are platform-specific limitations on how this name can be used. The documentation says:
Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
tempfile.NamedTemporaryFile has a .dir property which will give you want you want.
EDIT: No, it is not .name, #Barmar, but looking through the source code for tempfile, I don't see a .dir property either. However, you can use the .name property in conjunction with os.path's dirname method as follows:
with tempfile.NamedTemporaryFile(suffix='.csv', prefix=os.path.basename(__file__)) as tf:
tf_directory = os.path.dirname(tf.name)
This works just fine to get the full path with the filename
file = tempfile.NamedTemporaryFile()
filename = file.name
The output:
/tmp/tmp_sl4v9ps
Anyway, if you need the path of the tempfile directory you can use tempfile.gettempdir()
I'm getting the name of a file in Django after an Image save :
path-> 'companies/92_dsa/log/Hydrangeas.jpg' as it is in database
I do a clone of the file, an resize (is an image) and want to save the new file with a different name.
I get the directory of the original file:
folder = os.path.dirname(path)
the filename and extension:
filename, extension = os.path.splitext(os.path.basename(media_path))
then create a
new_filename = filename + '_sz' + extension
and the I want to recreate the path:
new_path = os.path.join(folder, new_filename)
and the problem(slash-backslash before the filename):
'companies/94_sda/logos\Hydrangeas_sz.jpg'
I'm working in Windows, bur the final deploy probably will be on Linux, so I want a fix indifferent of the OS.
so I want a fix indifferent of the OS.
Unfortunately, you can't really have your cake and eat it.
You say that
I'm working in Windows, bur the final deploy probably will be on Linux
This implies you are running the program on Windows, but dealing with *nix file names (be it Linux, Unix, or mac OS).
To do this completely os-independent ... you would need to split the original path on "/" to get all the sub components and then re-join them with os.path.join.
But then you need to deal with the fact that directory structures for absolute paths are very different between the two OS's - not to mention the leading drive specifier on Windows. This is less of an issue if you are only dealing with relative paths.
In short, the root of your problem is that the database contains Linux-style paths and you are processing them on Windows. You would have a similar problem if it was the other way around.
You need to choose your deployment platform and code for it.
Alternatively, write your code to simply remove the extension from the full path and replace it with "_sz."+extension
Since you don't actually care about the path in relation to the host OS (because you've chosen to store paths POSIX style in your DB), you can just use string joining: new_path = '/'.join([folder, new_filename]), or you could import the posixpath module directly import posixpath; new_path = posixpath.join(folder, new_filename).
You could also investigate PathLib, though that may be overkill for you.
I'm doing a program in which Chimera needs to be opened, I'm opening it with:
def generate_files_bat(filename):
f = open(filename, 'w')
text = """echo off SET PATH=%PATH%;"C:\\Program Files (x86)\\Chimera 1.6.1\\bin" chimera colpeps.cmd"""
print >>f, text
f.close()
But I need to find Chimera apart from the computer the python program is running. Is there any way the path can be searched by the python program in any computer?
Generally speaking, I don't think it is such a good idea to search the path for a program. Imagine, for example that two different versions were installed on the machine. Are-you sure to find the right one? Maybe a configuraition file parsed with the standard module ConfigParser would be a better option?
Anyway, to go back to your question, in order to find a file or directory, you could try to use os.walk which recursively walks trough a directory tree.
Here is an example invoking os.walk from a generator, allowing you to collect either the first or all matching file names. Please note that the generator result is only based on file name. If you require more advanced filtering (say, to only keep executable files), you will probably use something like os.stat() to extend the test.
import os
def fileInPath(name, root):
for base, dirs, files in os.walk(root):
if name in files:
yield os.path.join(base, name)
print("Search for only one result:")
print(next(fileInPath("python", "/home/sylvain")))
print("Display all matching files:")
print([i for i in fileInPath("python", "/home/sylvain")])
There is which for Linux and where for Windows. They both give you the path to the executable, provided it lies in a directory that is 'searched' by the console (so it has to be in %PATH% in case of Windows)
There is a package called Unipath, that does elegant, clean path calculations.
Have look here for the AbstractPath constructor
Example:
from unipath import Path
prom_dir = Path(__file__)
Long time reader, first time poster. I am very new to python and I will try to ask my question properly.
I have posted a snippet of the .py code I am using below. I am attempting to get the latest modified file in the current directory to be listed and then pass it along later in the code.
This is the error I get in my log file when I attempt to run the file:
WindowsError: [Error 2] The system cannot find the file specified: '05-30-2012_1500.wav'
So it appears that it is in fact pulling a file from the directory, but that's about it. And actually, the file that it pulls up is not the most recently modified file in that directory.
latest_page = max(os.listdir("/"), key=os.path.getmtime)
cause = channel.FilePlayer.play(latest_page)
os.listdir returns the names of files, not full paths to those files. Generally, when you use os.listdir(SOME_DIR), you then need os.path.join(SOME_DIR, fname) to get a path you can use to work with the file.
This might work for you:
files = [os.path.join("/", fname) for fname in os.listdir("/")]
latest = max(files, key=os.path.getmtime)
cause = channel.FilePlayer.play(latest)