I'm trying to code a simple application that must read all currently open files within a certain directory.
More specificly, I want to get a list of files open anywhere inside my Documents folder,
but I don't want only the processes' IDs or process name, I want the full path of the open file.
The thing is I haven't quite found anything to do that.
I couldn't do it neither in linux shell (using ps and lsof commands) nor using python's psutil library. None of these is giving me the information I need, which is only the path of currently open files in a dir.
Any advice?
P.S: I'm tagging this as python question (besides os related tags) because it would be a plus if it could be done using some python library.
This seems to work (on Linux):
import subprocess
import shlex
cmd = shlex.split('lsof -F n +d .')
try:
output = subprocess.check_output(cmd).splitlines()
except subprocess.CalledProcessError as err:
output = err.output.splitlines()
output = [line[3:] for line in output if line.startswith('n./')]
# Out[3]: ['file.tmp']
it reads open files from current directory, non-recursively.
For recursive search, use +D option. Keep in mind, that it is vulnerable to race condition - when you get your ouput, situation might have changed already. It is always best to try to do something (open file), and check for failure, e.g. open file and catch exception or check for null FILE value in C.
Related
I am working on the listener portion of a backdoor program (for an ETHICAL hacking course) and I would like to be able to read files from any part of my linux system and not just from within the directory where my listener python script is located - however, this has not proven to be as simple as specifying a typical absolute path such as "~/Desktop/test.txt"
So far my code is able to read files and upload them to the virtual machine where my reverse backdoor script is actively running. But this is only when I read and upload files that are in the same directory as my listener script (aptly named listener.py). Code shown below.
def read_file(self, path):
with open(path, "rb") as file:
return base64.b64encode(file.read())
As I've mentioned previously, the above function only works if I try to open and read a file that is in the same directory as the script that the above code belongs to, meaning that path in the above content is a simple file name such as "picture.jpg"
I would like to be able to read a file from any part of my filesystem while maintaining the same functionality.
For example, I would love to be able to specify "~/Desktop/another_picture.jpg" as the path so that the contents of "another_picture.jpg" from my "~/Desktop" directory are base64 encoded for further processing and eventual upload.
Any and all help is much appreciated.
Edit 1:
My script where all the code is contained, "listener.py", is located in /root/PycharmProjects/virus_related/reverse_backdoor/. within this directory is a file that for simplicity's sake we can call "picture.jpg" The same file, "picture.jpg" is also located on my desktop, absolute path = "/root/Desktop/picture.jpg"
When I try read_file("picture.jpg"), there are no problems, the file is read.
When I try read_file("/root/Desktop/picture.jpg"), the file is not read and my terminal becomes stuck.
Edit 2:
I forgot to note that I am using the latest version of Kali Linux and Pycharm.
I have run "realpath picture.jpg" and it has yielded the path "/root/Desktop/picture.jpg"
Upon running read_file("/root/Desktop/picture.jpg"), I encounter the same problem where my terminal becomes stuck.
[FINAL EDIT aka Problem solved]:
Based on the answer suggesting trying to read a file like "../file", I realized that the code was fully functional because read_file("../file") worked without any flaws, indicating that my python script had no trouble locating the given path. Once the file was read, it was uploaded to the machine running my backdoor where, curiously, it uploaded the file to my target machine but in the parent directory of the script. It was then that I realized that problem lied in the handling of paths in the backdoor script rather than my listener.py
Credit is also due to the commentator who pointed out that "~" does not count as a valid path element. Once I reached the conclusion mentioned just above, I attempted read_file("~/Desktop/picture.jpg") which failed. But with a quick modification, read_file("/root/Desktop/picture.jpg") was successfully executed and the file was uploaded in the same directory as my backdoor script on my target machine once I implemented some quick-fix code.
My apologies for not being so specific; efforts to aid were certainly confounded by the unmentioned complexity of my situation and I would like to personally thank everyone who chipped in.
This was my first whole-hearted attempt to reach out to the stackoverflow community for help and I have not been disappointed. Cheers!
A solution I found is putting "../" before the filename if the path is right outside of the dictionary.
test.py (in some dictionary right inside dictionary "Desktop" (i.e. /Desktop/test):
with open("../test.txt", "r") as test:
print(test.readlines())
test.txt (in dictionary "/Desktop")
Hi!
Hello!
Result:
["Hi!", "Hello!"]
This is likely the simplest solution. I found this solution because I always use "cd ../" on the terminal.
This not only allows you to modify the current file, but all other files in the same directory as the one you are reading/writing to.
path = os.path.dirname(os.path.abspath(__file__))
dir_ = os.listdir(path)
for filename in dir_:
f = open(dir_ + '/' + filename)
content = f.read()
print filename, len(content)
try:
im = Image.open(filename)
im.show()
except IOError:
print('The following file is not an image type:', filename)
Well I searched a lot and found different ways to open program in python,
For example:-
import os
os.startfile(path) # I have to give a whole path that is not possible to give a full path for every program/software in my case.
The second one that I'm currently using
import os
os.system(fileName+'.exe')
In second example problem is:-
If I want to open calculator so its .exe file name is calc.exe and this happen for any other programs too (And i dont know about all the .exe file names of every program).
And assume If I wrote every program name hard coded so, what if user installed any new program. (my program wont able to open that program?)
If there is no other way to open programs in python so Is that possible to get the list of all install program in user's computer.
and there .exe file names (like:- calculator is calc.exe you got the point).
If you want to take a look at code
Note: I want generic solution.
There's always:
from subprocess import call
call(["calc.exe"])
This should allow you to use a dict or list or set to hold your program names and call them at will. This is covered also in this answer by David Cournapeau and chobok.
You can try with os.walk :
import os
exe_list=[]
for root, dirs, files in os.walk("."):
#print (dirs)
for j in dirs:
for i in files:
if i.endswith('.exe'):
#p=os.getcwd()+'/'+j+'/'+i
p=root+'/'+j+'/'+i
#print(p)
exe_list.append(p)
for i in exe_list :
print('index : {} file :{}'.format(exe_list.index(i),i.split('/')[-1]))
ip=int(input('Enter index of file :'))
print('executing {}...'.format(exe_list[ip]))
os.system(exe_list[ip])
os.getcwd()+'/'+i prepends the path of file to the exe file starting from root.
exe_list.index(i),i.split('/')[-1] fetches just the filename.exe
exe_list stores the whole path of an exe file at each index
Can be done with winapps
First install winapps by typing:
pip install winapps
After that use the library:
# This will give you list of installed applications along with some information
import winapps
for app in winapps.list_installed():
print(app)
If you want to search for an app you can simple do:
application = 'chrome'
for app in winapps.search_installed(application):
print(app)
I am running Spark 1.6.1 with Python 2.7 on Windows 7.
The root scratch dir: /tmp/hive on HDFS is writable and my current permissions are: rwxrwxrwx (using winutils tools).
I want to stream files from a directory. According to the doc, the function textFileStream(directory):
Create an input stream that monitors a Hadoop-compatible file system
for new files and reads them as text files. Files must be wrriten to
the monitored directory by “moving” them from another location within
the same file system. File names starting with . are ignored.
When I launch Spark Streaming command:
lines = ssc.textFileStream(r"C:/tmp/hive/")
counts = lines.flatMap(lambda line: line.split(" "))\
.map(lambda x: (x, 1))\
.reduceByKey(lambda a, b: a+b)
counts.pprint()
ssc.start()
and then create the files to stream in my directory, nothing happens.
I also tried this:
lines = ssc.textFileStream("/tmp/hive/")
and
lines = ssc.textFileStream("hdfs://tmp/hive/")
which is HDFS path related, but nothing happens again.
Do I do something wrong?
Try using "file:///C:/tmp/hive" as a directory on Windows, worked for me on Windows 8 with Spark 1.6.3 but I had to fiddle a bit with file name and content before I made it work. I also tried with other paths so can confirm that it works the same way with paths which are not related in any way to winutits, e.g. you can use "file:///D:/someotherpath" if you have your data there
It is not straightforward though, I had a file in the monitored directory and did few content and file name changes before it got picked up, and then at some point it stopped reacting to my changes and getting picked up so results are not consistent. Guess it's a Windows thing.
I know it works so every time I try I know have to be patient and try few name changes before it gets picked up but that's only good to prove it works, obviously not good for anything else.
One other thing i was doing is using Unix eof instead of Windows eof in files but cannot assert it is required
ive read through the path.exists() and path.isdir() questions on here, but none that ive found so far, deal with check if a particular file type exists in a directory or not... maybe im not searching the correct terms for this.
basically, i want to poll a set of folders to see if there are txt files there... if they are there, i will want to run a string of pexpect commands (each with different usernames/passwords) to put *.txt them to a remote server location. I have my put and get pexpect script working already.
I tried using a wildcard such as this, but of course no such luck.
>>> print(os.path.exists("/mnt/path/to/shared/folder/*.txt"))
False
instead of having 15 cron jobs, doing a blind put *.txt every 5 minutes, i'd like to run just one script that checks all folder locations. if txt files exists = true (then do pexpect job) if false, go to next folder path and check if txt files exist = true (then do pexpect job) ect....
glob seems to be the ticket. I tested it with this:
import glob
if next(glob.iglob("/path/to/files/*.txt"), None):
print "there are txt files" # there are text files
else:
print "there are no text files"
$ python check.py
there are txt files
You want to use glob.
import glob
if glob.glob("/mnt/path/to/shared/folder/*.txt"):
# there are text files
else:
# No text files
Glob will return a list of files matching the wildcard-accessible path. If there are no files, it will return an empty list. This is really just os.listdir and fnmatch.filter together.
If memory is an issue, use glob.iglob as 200OK suggests in the comments:
import glob
if next(glob.iglob("/mnt/path/to/shared/folder/*.txt"), None):
# there are text files
else:
# No text files
iglob builds an iterator instead of a list, which is massively more memory-saving.
If your problem specifically is to find files with particular criteria, consider opening/reading a pipe to the program find.
As in:
find dir1 dir2 dir3 -name "*.txt"
That program has dozens of options for filtering based on the type of file (symlink, etc) and should give you a lot of flexibility that might be easier than writing it yourself with various python libraries.
Is it possible to loop through a set of selected files, process each, and save the output as new files using Apple Automator?
I have a collection of .xls files, and I've gotten Automator to
- Ask for Finder Items
- Open Finder Items
- Convert Format of Excel Files #save each .xls file to a .csv
I've written a python script that accepts a filename as an argument, processes it, and saves it as p_filename in the directory the script's being run from. I'm trying to use Run Shell Script with the /usr/bin/python shell and my python script pasted in.
Some things don't translate too well, though, especially since I'm not sure how it deals with python's open('filename','w') command. It probably doesn't have permissions to create new files, or I'm entering the command incorrectly. I had the idea to instead output the processed file as text, capture it with Automator, and then save it to a new file.
To do so, I tried to use New Text File, but I can't get it to create a new text file for each file selected back in the beginning. Is it possible to loop through all the selected Finder Items?
Why do you want this done in the folder of the script? Or do you mean the folder of the files you are getting from the Finder items? In that case just get the path for each file passed into Python.
When you run open('filename','w') you should thus pass in a full pathname. Probably what's happening is you are actually writing to the root directory rather than where you think you are.
Assuming you are passing your files to the shell command in Automator as arguments then you might have the following:
import sys, os
args = sys.argv[1:]
for a in args:
p = os.path.dirname(a)
mypath = p + "/" + name
f = open(mypath, "w")