I am reading data from influxdb into pandas with the below code:-
import pandas as pd
from influxdb import InfluxDBClient
client = InfluxDBClient(host='10.0.0.0', port=7076, username='abcdefghi',
password='dsdsd', ssl=False, verify_ssl=False)
read_client.switch_database('test')
q = 'SELECT * FROM "abc"
df = pd.DataFrame(read_client.query(
q, chunked=False, chunk_size=10000).get_points())
df
Then I am doing some processing.
Now, I have 50 domains and for some domains the influxdbclient is different. Now I want to do something so that it can take input from argpase and I can pass the read client on command line.
I want to keep one common connecting string which will be default and if we pass --influx_host on command line then it should consider that. I am new to this kind of programming in python. I was able to achieve passing variables but not able to Create a connection string. I will appreciate any help.
Basically if I can do something like below or anything better.:-
python test.py -influx_host 10.0.0.10 --port 7076
You should get arguments
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--influx_host', default='10.0.0.10')
parser.add_argument('--port', default=7076)
args = parser.parse_args() # it will use values from `sys.argv`
and use args.influx_host and args.port in code
client = InfluxDBClient(host=args.influx_host, port=args.port, ...)
And now you can run it with arguments
python test.py --influx_host 10.0.0.10 --port 7076
or without arguments
python test.py
or with only some of arguments
python test.py --influx_host 10.0.0.10
python test.py --port 7076
If you run without --influx_host and/or --port then it uses default values from .add_argument()
Not tested because I don't have influxdb but args shows expected values
import argparse
import pandas as pd
from influxdb import InfluxDBClient
# --- args ---
parser = argparse.ArgumentParser()
parser.add_argument('--influx_host', default='10.0.0.10')
parser.add_argument('--port', default=7076)
args = parser.parse_args() # it will use values from `sys.argv`
#args = parser.parse_args(['--influx_host', 'localhost', ]) # for tests
print(args)
# --- rest ---
client = InfluxDBClient(host=args.influx_host, port=args.port, username='abcdefghi',
password='dsdsd', ssl=False, verify_ssl=False)
read_client.switch_database('test')
q = 'SELECT * FROM "abc"'
df = pd.DataFrame(read_client.query(q, chunked=False, chunk_size=10000).get_points())
print(df)
Related
I am trying to write data to a disk on a remote machine via ssh using a Python script. However it is giving an error of dd: /dev/xbd2d: Device not configured.
import argparse
import os
import time
parser = argparse.ArgumentParser(description='basis')
parser.add_argument("-g", default=1, help="")
args = parser.parse_args()
volume = args.g
instance_ip=10.1.12.3
cmd_ssh='ssh -tt -i basis.pem root#'+instance_ip+ ''' "date | dd of=/dev/xbd2d"'''
os.system(cmd_ssh)
What is quite unusual is that if use the command:
ssh -tt -i basis.pem root#10.1.12.3 "date | dd of=/dev/xbd2d"
in a terminal, it executes correctly without any problem and writes the data to the disk. I wrote the same script in C++ and it worked fine but for some reason Python gives me dd: /dev/xbd2d: Device not configured.
Checking the quotes around the variable and also using \" instead of the triple quotes solved my problem as recommended by Eran
import argparse
import os
import time
parser = argparse.ArgumentParser(description='basis')
parser.add_argument("-g", default=1, help="")
args = parser.parse_args()
volume = args.g
instance_ip=10.1.12.3
cmd_ssh="ssh -tt -i basis.pem root#"+str(instance_ip)+" \"date | dd of=/dev/xbd2d\""
os.system(cmd_ssh)
I am in the bit of a weird situation where I need a Python function to run from within a script, with the script then called from my main code.
I wanted to use the subprocess module, and know how to use it to pass arguments to a pure script, but the thing is, I need to pass the arguments to the nested Python function within, most of which are optional and have default values.
I thought arparse would help me do this somehow.
Here is an example of what I am trying:
## Some Argparse, which will hopefully help
import argparse
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('file_name', help='Name of resulting csv file')
parser.add_argument('sub_name', help='Sub-name of resulting csv file')
parser.add_argument('follow', help='Account(s) to follow', required=True)
parser.add_argument('locations', help='Locations')
parser.add_argument('languages', help='Languages')
parser.add_argument('time_limit', help='How long to keep stream open')
args = parser.parse_args()
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth = api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
... function code ...
... more function code ...
...
...
## End of script
If you are passing arguments to functions all you need to do is feed them into the function when you're executing them:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output_file_name", help="Name of resulting csv file")
parser.add_argument("-s", "--sub_name", default="stream_", help="Sub-name of resulting csv file")
parser.add_argument("-f", "--follow", help="Account(s) to follow", required=True)
parser.add_argument("-loc", "--locations", default=None, help="Locations")
parser.add_argument("-lan", "--languages", default=None, help="Languages")
parser.add_argument("-t", "--time_limit", default=20, help="How long to keep stream open")
options = parser.parse_args()
# then just pass in the arguments when you run the function
twitter_stream_listener(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit)
# or, pass the arguments into the functions when defining your function
def twitter_stream_listener_with_args(file_name=options.output_file_name,
sub_name=options.sub_name,
auth=api.auth,
filter_track=None,
follow=options.follow,
locations=options.locations,
languages=options.languages,
time_limit=options.time_limit):
# does something
pass
# then run with default parameters
twitter_stream_listener_with_args()
You can specify defaults in the argparse section (if that is what you are trying to achieve):
#!/usr/bin/python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--argument', default = 'something', type = str, help = 'not helpful')
parser.add_argument('--arg2', default = None, type = str, help = 'not helpful')
args = parser.parse_args()
def foo(arg , arg2 ):
print(arg)
if not arg2 is None:
print(arg2)
foo(args.argument, args.arg2)
Then calling:
$ ./test.py
something
$ ./test.py --argument='somethingelse'
somethingelse
$ ./test.py --arg2=123
something
123
$ ./test.py --arg2='ipsum' --argument='lorem'
lorem
ipsum
Is this helpful?
You can do it like that:
import argparse
## Actual Function
def twitter_stream_listener(file_name=None,
sub_name='stream_',
auth=api.auth,
filter_track=None,
follow=None,
locations=None,
languages=None,
time_limit=20):
# Your content here
if __name__ == '__main__':
parser = argparse.ArgumentParser()
## All arguments, with only "follow" being required
parser.add_argument('follow', help='Account(s) to follow')
parser.add_argument('--file_name', help='Name of resulting csv file')
parser.add_argument('--sub_name', help='Sub-name of resulting csv file')
parser.add_argument('--locations', help='Locations')
parser.add_argument('--languages', help='Languages')
parser.add_argument('--time_limit', help='How long to keep stream open')
args = parser.parse_args()
twitter_stream_listener(file_name=args.file_name, sub_name=args.sub_name, follow=args.follow,
locations=args.locations, languages=args.languages, time_limit=args.time_limit)
follow will be the only required argument and the rest optional. Optional ones have to be provided with -- at the beginning. You can easily use the module with subprocess if you need it.
Example call using command line:
python -m your_module_name follow_val --file_name sth1 --locations sth2
Here is the current code.
import time
import collections
from modules import outputs
from modules import scrub
from modules import lookups
parser = argparse.ArgumentParser(description='AppMap Converter to Generate Asset Files from AppMapp Data')
parser.add_argument("operation", nargs='?', default="empty", help='The operation to perform')
parser.add_argument("input", nargs='?', default="empty", help='The input AppMapp File Path')
parser.add_argument("output", nargs='?', default="empty", help='The output Asset File Path')
args = parser.parse_args()
start = time.time()
if(args.operation == "Convert"):
input_file_path = args.input
output_file_path = args.output
#DO LOTS OF STUFF
else:
exit()
The script is called sacsproc, so I run it from the command line as follows:
./sacsproc Convert input.csv output.csv
This all works nicely, the problem is that I need more sacsproc commands which may have a totally different set of secondary parameters. i.e. one command might be:
./sacsproc Clean -rts input.csv output.csv err.csv
Thus, I am trying to determine how one defines arguments that are conditional on the first argument? In my mind, I'm thinking about the zfs command line utilities that do what I am trying to do (e.g. zpool create mirror sdb sdc vs. zpool remove sda).
use subparsers
subparsers = parser.add_subparsers(help="sub-command help")
group1 = subparsers.add_parser("something",help="do something")
group1.set_defaults(which="g1") # some default value (so you know which group was activated)
group1.add_argument("ARG",help='do something on ARG')
group2 = subparsers.add_parser("other",help="do something else")
group2.set_defaults(which="g2") # give some default value
group2.add_argument("ARG",help='do something else on ARG')
ok ...
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help="sub-command help")
g1 = subparsers.add_parser("thing1",help="bind to a port and just echo back anything it gets ... with a prompt")
g1.set_defaults(which="g1")
g1.add_argument("input",help='the input file')
g1.add_argument("output",help='the output file')
g2 = subparsers.add_parser("thing2",help="create a bridge between two ports, this is useful for generating a logfile")
g2.set_defaults(which="g2")
g2.add_argument("input",help='thie input file')
g2.add_argument("output",help='the output file')
g2.add_argument("error",help="the err file")
def print_help(args):
print "ARGS:",args
try:
parser.parse_args(args)
except:
print "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n"
print_help(["-h"])
print_help(["thing1","-h"])
print_help(["thing2","-h"])
I am writing a program which basically logs into the remote server(Linux) and execute few commands. For that I need to get the inputs like "Server IP ", "Username", and "password" from the command line. For which am using the argparse module. My code looks something like this:
import sys
import argparse
import getpass
import re
from time import gmtime, strftime
import paramiko
try:
import interactive
except ImportError:
from . import interactive
def do_connect(esxi):
"""Connect to host in the esxi host"""
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect()
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='DEV Installer App')
parser.add_argument("--esxiIP", help="Mention the Esxi Host IP", required=True)
parser.add_argument("--userName", help="Enter the UserName of the Host", required=True)
parser.add_argument("--password", help="Enter the Password", required=True)
args = parser.parse_args()
What I am trying to do here is , I have a function for connecting to the remote server and am getting the inputs using argparse in the main() function. I need to pass these input arguments to the do_connect function. How can I accomplish this?
args is a Namespace instance object with attributes for each command-line argument. The arguments are based on the option switch, unless you give it a different name explicitly.
In this case, you'll have args.esxiIP, args.userName and args.password attributes, which you can then pass on to another function. If your function takes just the esxiIP argument, pass that on:
do_connect(args.esxiIP)
If it needs more arguments, explicitly pass these on too.
Demo:
>>> args = parser.parse_args('--esxiIP 10.20.30.40 --userName foo --password bar'.split())
>>> args
Namespace(esxiIP='10.20.30.40', password='bar', userName='foo')
>>> args.esxiIP
'10.20.30.40'
>>> args.userName
'foo'
>>> args.password
'bar'
i have a script which take hh:mm:ss inputs
for example
vid_cut.py -t 00:00:30 00:10:00
but when i do
from sys import argv
from argparse import ArgumentParser as AP
ap = AP()
ap.add_argument('-f',type=str)
ap.add_argument('-tp',nargs='+',type=str)
arg = ap.parse_args(argv)
print arg['-tp']
i got
vid_cut.py: error: unrecognized arguments: vid_cut.py
how can I make argparse understand my inputs?
updates
I have now solved the problem, with the following code
# imports
from os import system as sc
from sys import argv
from argparse import ArgumentParser as AP
from itertools import tee , izip
# cmd synthesis
ap = AP()
ap.add_argument('-f',type=str)
#ap.add_argument('-tp',nargs='+',type=str)
ap.add_argument('-tp',nargs='+',type=str)
arg = vars(ap.parse_args())
print argv
print arg
f_name = arg['f']
tp = map(HMS_S, arg['tp'])
ffmpeg_cmd = "ffmpeg -sameq -ss %s -t %s -i %s %s"
# system call
for t,dt in diff(tp):
print ffmpeg_cmd % (t,dt,f_name,f_name + str(t))
the only question is I don't know why when do we need to do
arg = vars(ap.parse_args(something))
it seems argv has been magically processed.
answer: argv[1:] will be automatically processed, unless you have other stuff to parse.
Since you haven't given very much code I'll show an example that will work:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-t', nargs='+', type=str)
# If no argument is given to parse_args() then the argument strings of
# the program are taken from sys.argv. And be sure not to pass in
# sys.argv[0] if you specify them instead supply sys.argv[1:].
args = parser.parse_args()
print(args.t)
If run with "-t 00:00:30 00:10:00" as arguments then I get this value back:
['00:00:30', '00:10:00']
Without more code it's hard to know what's wrong, but you may be passing all of sys.argv to parse_args(). You should pass sys.argv[1:], i.e. only the arguments, not the program name.