I have multiple windows servers in an internal network(meaning, not connected to the internet).
At the moment, I am pulling files from these servers using Windows Remote Desktop, for each server I have an IP address, login & password.
My goal is to use Python to automate this process, I want to be able to run a script that will access these internal servers and get files from them.
Is there any Python module that handles such tasks, or how should I approach this problem?
Windows' file sharing protocol is called SMB. There are some python libraries for this; the first google result for me was this one: https://pysmb.readthedocs.io/en/latest/
Related
I never used a server before (only my local machine), and recently I was given a server with Python files on it. I was also given SSH key and a password.
I'm trying to connect PyCharm to the server from my local machine so that I can write code on my local machine, upload the code to the server, and use code from existing files on the server.
The only website I found is this, and I followed the instructions to create an SSH interpreter, but I'm not sure what to do next (in order to see the files, write files to the server, etc,).
I'm trying to understand all the methods available to execute remote commands on Windows through the impacket scripts:
https://www.coresecurity.com/corelabs-research/open-source-tools/impacket
https://github.com/CoreSecurity/impacket
I understand the high level explanation of psexec.py and smbexec.py, how they create a service on the remote end and run commands through cmd.exe -c but I can't understand how can you create a service on a remote windows host through SMB. Wasn't smb supposed to be mainly for file transfers and printer sharing? Reading the source code I see in the notes that they use DCERPC to create this services, is this part of the smb protocol? All the resources on DCERPC i've found were kind of confusing, and not focused on its service creating capabilities. Looking at the sourcecode of atexec.py, it says that it interacts with the task scheduler service of the windows host, also through DCERPC. Can it be used to interact with all services running on the remote box?
Thanks!
DCERPC (https://en.wikipedia.org/wiki/DCE/RPC) : the initial protocol, which was used as a template for MSRPC (https://en.wikipedia.org/wiki/Microsoft_RPC).
MSRPC is a way to execute functions on the remote end and to transfer data (parameters to these functions). It is not a way to directly execute remote OS commands on the remote side.
SMB (https://en.wikipedia.org/wiki/Server_Message_Block ) is the file sharing protocol mainly used to access files on Windows file servers. In addition, it provides Named Pipes (https://msdn.microsoft.com/en-us/library/cc239733.aspx), a way to transfer data between a local process and a remote process.
One common way for MSRPC is to use it via Named Pipes over SMB, which has the advantage that the security layer provided by SMB is directly approached for MSRPC.
In fact, MSRPC is one of the most important, yet very less known protocols in the Windows world.
Neither MSRPC, nor SMB has something to do with remote execution of shell commands.
One common way to execute remote commands is:
Copy files (via SMB) to the remote side (Windows service EXE)
Create registry entries on the remote side (so that the copied Windows Service is installed and startable)
Start the Windows service.
The started Windows service can use any network protocol (e.g. MSRPC) to receive commands and to execute them.
After the work is done, the Windows service can be uninstalled (remove registry entries and delete the files).
In fact, this is what PSEXEC does.
All the resources on DCERPC i've found were kind of confusing, and not
focused on its service creating capabilities.
Yes, It’s just a remote procedure call protocol. But it can be used to start a procedure on the remote side, which can just do anything, e.g. creating a service.
Looking at the sourcecode of atexec.py, it says that it interacts with
the task scheduler service of the windows host, also through DCERPC.
Can it be used to interact with all services running on the remote
box?
There are some MSRPC commands which handle Task Scheduler, and others which handle generic service start and stop commands.
A few final words at the end:
SMB / CIFS and the protocols around are really complex and hard to understand. It seems ok trying to understand how to deal with e.g. remote service control, but this can be a very long journey.
Perhaps this page (which uses Java for trying to control Windows service) may also help understanding.
https://dev.c-ware.de/confluence/pages/viewpage.action?pageId=15007754
I normally use a bash script to grab all the files onto local machine and use glob to process all the files. Just wondering what would be the best way to use python (instead of another bash script) to ssh into each server and process those files?
My current program runs as
for filename in glob.glob('*-err.txt'):
input_open = open (filename, 'rb')
for line in input_open:
do something
My files all have the ending -err.txt and the directories where they reside in the remote server have the same name /documents/err/. I am not able to install third party libraries as I don't have the permission.
UPDATE
I am trying to not to scp the files from the server but to read it on the remote server instead..
I want to use a local python script LOCALLY to read in files on remote server.
The simplest way to do it is to use paramico_scp to use ssh copy from the remote server (How to scp in python?)
If you are not allowed to download any libraries, you can create SSH key pair so that connecting to server does not require a password (https://www.debian.org/devel/passwordlessssh). You then can for each file do
import os
os.system('scp user#host:/path/to/file/on/remote/machine /path/to/local/file')
Note that using system is usually considered less portable than using libraries. If you give the script that use system('scp ...') to copy the files and they do not have SSH key pair set up, they will experience problems
Looks like you want to use a local Python script remotely. This has been answered here.
I'm connected to a VM on a private network at address 'abc.def.com' using ssh, and on that VM there's an application that hosts a Python web app (IPython Notebook) that I can access by pointing my local browser to 'abc.def.com:7777'.
From that web app I can call shell commands by preceding them with '!', for example !ls -lt will list the files in the VM current working directory. But since I'm using my own laptop's browser, I think I should be able to run shell commands on my local files as well. How would I do that?
If that's not possible, what Python/shell command can I run from within the web app to automatically get my laptop's IP address to use things like scp? I know how to get my IP address, but I'd like to create a program that will automatically enable scp for whoever uses it.
You have ssh access so you could possibly write a python function that would let you transfer files via scp the secure copy command which uses ssh to communicate. If you exchange keys with the server you wouldn't have to put in a password so I see no problem from that standpoint. The issue is if you have an address for your local machine to be accessed from the server.
I work on various remotes from my laptop all day and from my laptop to the sever I could have this function:
def scp_to_server(address, local_file, remote_file):
subprocess.call(['scp',local_file,"myusername#{}:{}".format(address, remote_file)])
that would copy a file from my local machine to the remote provided the paths were correct, I have permissions to copy the files, and my local machine's id_rsa.pub key is in the ~/.ssh/authorized_keys file on the remote.
I have no way to initiate a secure copy from the remote to my local machine however because I don't have an address to access the local machine from that I can "see" on the remote.
If I open the terminal on my laptop and run hostname I see mylaptop.local and on the remote I see remoteserver#where.i.work.edu but the first is a local address I can see it from other machines on my LAN at home, (because I have configured that) but I can't see mylaptop.local from the remote. I know there is a way to configure that so I could find my laptop at home from anywhere, but I never had the need to do that (since I bring the laptop with me) so I can't help you there. I think there are a few more hurdels to go-over than you would like.
You could implement the function above on your local machine and transfer the files that way though.
I have been interested in finding an alternative to the UI in SAS for quite some time now. We license SAS on our server instead of our desktops, so furthermore we have to launch a remote desktop application to execute code.
I was able to use a Telnet connection instead to remotely connect to the server, and batch execute SAS programs. Then I was interested in whether a python script could be made to connect remotely, and batch execute code, and this script could be executed in jEdit as a BeanShell script.
So far, I have Python code which successfully opens and closes the Telnet connection. It can do basic shell functions like call "dir". However, when I pass the exact same line that I use to execute SAS from command prompt on the remote server with a telnet connection in Python, nothing happens.
Is it possible the server is preventing me from executing code from a script? I use a "read_until" statement for the prompt before running any code.
Here's a few ideas...
The issue you are having above may be related to Local Security Policy settings in Windows (if it is running on a windows server). I'm far from an expert on that stuff but I remember older SAS/Intranet installations required some rumaging around in there to get them working.
As an alternative to the approach you are trying above you could also setup a SAS session on the server that listens for incoming socket requests as per this article:
http://analytics.ncsu.edu/sesug/2000/p-1003.pdf
And finally... Not sure if this helps or not by I remotely execute SAS jobs using PSEXEC. A description of how I set it all up can be found here:
http://www.runsubmit.com/questions/260/hide-sas-batch-jobs-winxp
Good luck
This paper outlines how you can use a Python script to connect to a Unix server using SSH, copy the SAS program written locally onto the server, batch submit it, and download the results back to your local machine, all using a BeanShell macro script for jEdit.