I am working on a script to transfer some files to a Cisco IOS device using netmiko FileTransfer. Below is the code that I found to accomplish this. However I cant seem to find where the source_file should be or how to specify where on the host that files lives. How do I specify where to copy that file from?
dest_file_system = 'disk0:/'
source_file = 'test1.txt' # where should this file be live?
dest_file = 'test1.txt'
with FileTransfer(ssh_conn, source_file=source_file, dest_file=dest_file,
file_system=dest_file_system) as scp_transfer:
if not scp_transfer.check_file_exists():
if not scp_transfer.verify_space_available():
raise ValueError("Insufficient space available on remote device")
print("\nTransferring file\n")
scp_transfer.transfer_file()
The source file from the scripts perspective if you simple want to call it by name should be in the same directory as the script itself. If you want to move the file to a new directory the search path in the script is relative to the directory the script is run from. Example 1 - The file_name.txt is in the same directory as your script In your script simply call the file source = "file_name.txt". Example 2 create test_folder in current directory that holds your script and call it test_folder move the file_name.txt into test_folder. In your script your source variable would no need to look like this source = "test_folder/file_name.txt"
Hopefully, till now you found the solution already. If not then You can try the netmiko file transfer feature also. That will be a more secure and efficient way to do it.
try below sample script.
from getpass import getpass
from netmiko import ConnectHandler, file_transfer
password = getpass()
cisco = {
"device_type": "cisco_ios",
"host": "cisco1.twb-tech.com",
"username": "pyclass",
"password": password,
}
source_file = "test1.txt"
dest_file = "test1.txt"
direction = "put"
file_system = "flash:"
ssh_conn = ConnectHandler(**cisco)
transfer_dict = file_transfer(
ssh_conn,
source_file=source_file,
dest_file=dest_file,
file_system=file_system,
direction=direction,
overwrite_file=True,
)
print(transfer_dict)
Related
I am creating an azure-ml webservice. The following script shows the code for creating the webservice and deploying it locally.
from azureml.core.model import InferenceConfig
from azureml.core.environment import Environment
from azureml.core import Workspace
from azureml.core.model import Model
ws = Workspace.from_config()
model = Model(ws,'textDNN-20News')
ws.write_config(file_name='config.json')
env = Environment(name="init-env")
python_packages = ['numpy', 'pandas']
for package in python_packages:
env.python.conda_dependencies.add_pip_package(package)
dummy_inference_config = InferenceConfig(
environment=env,
source_directory="./source_dir",
entry_script="./init_score.py",
)
from azureml.core.webservice import LocalWebservice
deployment_config = LocalWebservice.deploy_configuration(port=6789)
service = Model.deploy(
ws,
"myservice",
[model],
dummy_inference_config,
deployment_config,
overwrite=True,
)
service.wait_for_deployment(show_output=True)
As it can be seen, the above code deploys "entry_script = init_score.py" to my local machine. Within the entry_script, I need to load the workspace again to connect to azure SQL database. I do it like the following :
from azureml.core import Dataset, Datastore
from azureml.data.datapath import DataPath
from azureml.core import Workspace
def init():
pass
def run(data):
try:
ws = Workspace.from_config()
# create tabular dataset from a SQL database in datastore
datastore = Datastore.get(ws, 'sql_db_name')
query = DataPath(datastore, 'SELECT * FROM my_table')
tabular = Dataset.Tabular.from_sql_query(query, query_timeout=10)
df = tabular.to_pandas_dataframe()
return len(df)
except Exception as e:
output0 = "{}:".format(type(e).__name__)
output1 = "{} ".format(e)
output2 = f"{type(e).__name__} occured at line {e.__traceback__.tb_lineno} of {__file__}"
return output0 + output1 + output2
The try-catch block is for catching the potential exception thrown and return it as an output.
The exception that I keep getting is:
UserErrorException: The workspace configuration file config.json, could not be found in /var/azureml-app or its
parent directories. Please check whether the workspace configuration file exists, or provide the full path
to the configuration file as an argument. You can download a configuration file for your workspace,
via http://ml.azure.com and clicking on the name of your workspace in the right top.
I have actually tried to save the config file by passing an absolute path to the path argument of both ws.write_config(path='my_absolute_path'), and also when loading it to the Workspace.from_config(path='my_absolute_path'), but I got pretty much the same error:
UserErrorException: The workspace configuration file config.json, could not be found in /var/azureml-app/my_absolute_path or its
parent directories. Please check whether the workspace configuration file exists, or provide the full path
to the configuration file as an argument. You can download a configuration file for your workspace,
via http://ml.azure.com and clicking on the name of your workspace in the right top.
Looks like even providing the path does not change the root directory that the entry script starts locating from.
I also tried to directly saving the file to /var/azureml-app/, but this path is not recognized when I passed it to the ws.write_config(path='/var/azureml-app/').
Do you have any idea where exactly is the /var/azureml-app/?
Any idea on how to fix this?
Hi I am pretty new in this AWS world, what I am trying to do is connect a python client to the AWS IoT service and publish a message, I am using the SDK python and its example, but I have problems whit the certification process, I already have created the thing, the policies and the certification and I downloaded the files, but in the python program I have no idea if I am writing the path to this files in a correct way,
First I tried writing the whole path of each file and nothing then I tried just putting "certificados\thefile" and nothing .
The error that pops up says the error is the path which precesily I do not how to write it.
Thanks for taking the time and sotty if this question is too basic I am just jumping into this.
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
import time as t
import json
import AWSIoTPythonSDK.MQTTLib as AWSIoTPyMQTT
# Define ENDPOINT, CLIENT_ID, PATH_TO_CERT, PATH_TO_KEY, PATH_TO_ROOT, MESSAGE, TOPIC, and RANGE
ENDPOINT = "MYENDPOINT"
CLIENT_ID = "testDevice"
PATH_TO_CERT = "certificados/5a7e19a0269abe740ac8b38a1bfdab115d14074eb212167a3ba359c0d237a8c3-certificate.pem.crt"
PATH_TO_KEY = "certificados/5a7e19a0269abe740ac8b38a1bfdab115d14074eb212167a3ba359c0d237a8c3-private.pem.key"
PATH_TO_ROOT = "certificados/AmazonRootCA1.pem"
MESSAGE = "Hello World"
TOPIC = "Prueba/A"
RANGE = 20
myAWSIoTMQTTClient = AWSIoTPyMQTT.AWSIoTMQTTClient(CLIENT_ID)
myAWSIoTMQTTClient.configureEndpoint(ENDPOINT, 8883)
myAWSIoTMQTTClient.configureCredentials(PATH_TO_ROOT, PATH_TO_KEY, PATH_TO_CERT)
myAWSIoTMQTTClient.connect()
print('Begin Publish')
for i in range (RANGE):
data = "{} [{}]".format(MESSAGE, i+1)
message = {"message" : data}
myAWSIoTMQTTClient.publish(TOPIC, json.dumps(message), 1)
print("Published: '" + json.dumps(message) + "' to the topic: " + "'test/testing'")
t.sleep(0.1)
print('Publish End')
myAWSIoTMQTTClient.disconnect()
I have created a directory on my deskopt to store this files, its name is "certificados" and from there I am taking the path but it doesn't work.
OSError: certificados/AmazonRootCA1.pem: No such file or directory
Also I am using VS code to run this application.
The error is pretty clear, it can't find the CA cert file at the path you've given it. The path you've given will be interpreted relative to where the files are executed, which is most likely going to be relative to the python file it's self. If that's not the Desktop then you need to provide the fully qualified path:
So assuming Linux, change the paths to:
PATH_TO_CERT = "/home/user/Desktop/certificados/5a7e19a0269abe740ac8b38a1bfdab115d14074eb212167a3ba359c0d237a8c3-certificate.pem.crt"
PATH_TO_KEY = "/home/user/Desktop/certificados/5a7e19a0269abe740ac8b38a1bfdab115d14074eb212167a3ba359c0d237a8c3-private.pem.key"
PATH_TO_ROOT = "/home/user/Desktop/certificados/AmazonRootCA1.pem"
I have been struggling with installing pyrebase on my second PC, which has the same python version 3.8.2 as my main PC, my main PC has this pyrebase script working properly
from pyrebase import pyrebase
import os
import time
project_root = os.path.dirname(os.path.dirname(__file__))
keys_path = os.path.join(project_root, 'keys')
hylKeyPath = os.path.join(keys_path, 'ServiceAccountKey.json')
firebase = pyrebase.initialize_app({
"apiKey": "aksdjalksjdlkajsdlkjalkdja",
"authDomain": "lkjsakdjlkjsad.firebaseapp.com",
"databaseURL": "https://asdlkasjldkjaslkd.firebaseio.com",
"storageBucket": "asdasdadjaslkjhd.appspot.com",
"serviceAccount": asdKeyPath
})
storage = firebase.storage()
def sleepCountDown(t):
while t > 0:
print(f"sleeping for {t} seconds...")
t -= 1
time.sleep(1)
while True:
print('fetching data')
files = storage.child('/').list_files()
for file in files:
if 'records/' in file.name:
# get the file url path
# print(storage.child(file.name).get_url(None))
# downloads file
storage.child(file.name).download(os.path.basename(file.name))
# deletes file? kinda deletes the entire folder
storage.delete(file.name)
sleepCountDown(10)
But somehow I am not able to install pyrebase on my second PC so I had to install pyrebase4.
But it seems this one has some bugs and keeps highlighting storage.child(file.name).download(os.path.basename(file.name)) saying that storage: No value for argument 'filename' in method call.
then when I run, it says "Crypto has no method or something"
anyone knows what is going on?
Just noticed the syntax was different on pyrebase4, it's .download(path, filename) the documentation was outdated.
Instead of:
storage.child(file.name).download(os.path.basename(file.name))
Use:
storage.child("filename").download(filename="filename" , path="E:/ddf/")
Where E:/ddf/ is path you want to save your file.
I'm using fabric to connect to remote host, when i'm there, I try to call a script that I made (It parses the file I give in argument). But when I call the script from inside my Fabfile.py, it assumes the path I gave is from the machine I launch the fabfile from (so not my remote host)
In my fabfile.py I have:
Import import servclasse
env.host='host1'
def listconf():
#here I browes to the correct folder
s=servclasse.Server("my.file") #this is where I want it to open the host1:my.file file and instanciate a classe from what it parsed
If i do this, it tries to open the file from the folder where servclass.py is. Is there a way to give a "remote path" in argument? I would rather not downloading the file.
Should I upload the script servclasse.py with the operation.put before calling it?
Edit: more info
In my servclasse I have this:
def __init__(self, path):
self.config = ConfigParser.ConfigParser(allow_no_value=True)
self.config.readfp(open(path))
The function open() was the problem.
I figured out how to do it so i'll drop it here in case someone read this topic one day :
def listconf():
#first I browes to the correct folder then
contents = StringIO.StringIO()
get("MyFile",contents)
contents.seek(0)
s=Server(contents)
and in the servclass.py
def __init__(self, objfile):
self.config = ConfigParser.ConfigParser(allow_no_value=True)
self.config.readfp(objfile)
#and i do my stuffs
What's the simplest way to store the application secrets (passwords, access tokens) for a Python script? I thought it'd be a *.yml file like in Ruby but surprisingly I found that it wasn't the case. So what is it then? What are the most simplest solutions?
I want to put them in a separate file because that way I'll be able not to push that file to a github repository.
I think storing credentials inside another *py file is your safest bet. Then just import it. Example would look like this
config.py
username = "xy"
password = "abcd"
main.py
import config
login(config.username, config.password)
I was dealing exactly the same question and actually ended up with the same solution as kecer suggested. Since I need to use it in dozens of scripts, I've created own library. Let me share this solution with you.
credlib.py -- universal library to handle credentials
class credential:
def __init__(self, hostname, username, password):
self.hostname = hostname
self.username = username
self.password = password
mycredentials.py -- my local file to store all credentials
from credlib import credential
sys_prod = credential("srv01", "user", "pass")
sys_stg = credential("srv02", "user", "pass")
sys_db = credential("db01", "userdb", "passdb")
mysystemlib.py -- this is a general library to access my system (both new credential system and legacy is supported)
from credlib import credential
def system_login(*args): # this is new function definition
#def system_login(hostname, username, password): # this was previous function definition
if len(args) == 1 and isinstance(args[0], credential):
hostname = args[0].hostname
username = args[0].username
password = args[0].password
elif len(args) == 3:
hostname = args[0]
username = args[1]
password = args[2]
else:
raise ValueError('Invalid arguments')
do_login(hostname, username, password) # this is original system login call
main.py -- main script that combines credentials and system libs
from mycredentials import sys_stg, sys_db
import mysystemlib
...
mysystemlib.system_login(sys_stg)
Please note that the legacy hostname/username/password way still works so it does not affect old scripts:
mysystemlib.system_login("srv02", "user", "pass")
This has a lot benefits:
same credential system across all our python scripts
files with passwords are separated (files can have more strict permissions)
files are not stored in our git repositories (excluded via .gitignore) so that our python scripts/libs can be shared with others without exposing credentials (everyone defines their own credentials in their local files)
if a password needs to be changed, we do it at a single place only
Personally I prefer to use yaml files, with the pyyaml library.
Documentation here: https://pyyaml.org/wiki/PyYAMLDocumentation
Creating a .gitignore rule is very quick and painless and there is zero chances of making a mistake. You can added the rule with echo on Linux / UNIX like system with:
echo -e '*.yaml\n*.yml' >> .gitignore
Below is an example of retrieving the settings from a settings .yaml file in the same folder / location of the reader.
Code Snippets:
#!/usr/bin/env python3
import yaml
from pathlib import Path
def get_settings():
full_file_path = Path(__file__).parent.joinpath('settings.yaml')
with open(full_file_path) as settings:
settings_data = yaml.load(settings, Loader=yaml.Loader)
return settings_data