Develop Reference

Openpyxl module: return weird value(not error) + hope to calculate

I wrote some codes trying to let the user be able to check the percentage of the money they spent(compared to the money they earned). Almost every step perform normally, until the final part.
a_c[('L'+row_t)].value return:
=<Cell 'Sheet1'.B5>/<Cell 'Sheet1'.J5>
yet I hope it should be some value.
Code:
st_column = st_column_r.capitalize()
row_s = str(a_c.max_row)
row_t = str(a_c.max_row + 1)
row = int(row_t)
a_c[('J'+row_t)] = ('=SUM(I2,J'+row_s+')') #總收入
errorprevention = a_c[('J'+row_t)].value
a_c[(st_column+row_t)] = ('=SUM('+(st_column+'2')+','+(st_column+row_s)+')')
a_c['L'+row_t].number_format = FORMAT_PERCENTAGE_00
if errorprevention != 0:
a_c[('L'+row_t)] = ('='+str(a_c[(st_column+row_t)])+'/'+str(a_c[('J'+row_t)]))
print('過往支出中，'+inputtype[st_column]+'類別佔總收入的比率為:'+a_c[('L'+row_t)].value)

Try changing the formula creation to;
a_c[('L' + row_t)].value = '=' + a_c[(st_column + row_t)].coordinate + '/' + a_c[('J' + row_t)].coordinate
or use an f string
a_c[('L' + row_t)].value = f"={a_c[(st_column + row_t)].coordinate}/{a_c[('J' + row_t)].coordinate}"

How to create a JKS or P12 keystore with Python

I'm working on a Python 3 script that among other things, at some point it needs to create a .JKS or .P12 keystore. I use to have a bash script that used keytool for this:
keytool -genkey -keyalg RSA -alias certAlias \
-keystore keystore.jks -storepass $keyPass \
-validity 360 -keysize 2048 \
-noprompt -dname "CN=com.myCompany, OU=ID, O=AwesomeSoft, L=SF, S=CA, C=US" \
-keypass $keyPass
mv ./keystore.jks src/main/resources/
Now i'm moving the same functionality from that bash script to python and I having some issues to figure it out and any pointer will ne more than welcome.. you may noticed that the example above is for jks, not p12... the newer version have to be able to, depending on a variable before called certType with create one or the other... or create a jks and later convert it to p12... i'm open to options..
Thanks in advance!!

Found my answer:
import os
certAlias = 'cert'
certAlg = 'RSA'
certSigAlg = 'SHA1withRSA'
certExp = '365'
certKeySize = '2048'
certKeyType = 'PKCS12' # Select PKCS12 or JKS
certKeyPass = 'password123'
fileName = 'keystore'
dname = 'CN=mySite.com'
#
if certKeyType == "PKCS12":
fileExt = 'p12'
elif certKeyType == "JKS":
fileExt = 'jks'
certFile = fileName + '.' + fileExt
keytool = 'keytool -genkey -noprompt \
-alias ' + certAlias + ' \
-keypass ' + certKeyPass + ' \
-keyalg ' + certAlg + ' \
-sigalg ' + certSigAlg + '\
-validity ' + certExp + ' \
-dname ' + dname + ' \
-keysize ' + certKeySize + ' \
-keystore ' + certFile + ' \
-storepass '+ certKeyPass +' \
-storetype ' + certKeyType
os.system(keytool)
I did this and works but I will be playing to add more logic... hope it helps anyone.

"Extended" template-based INI file parsing in Python

I need to parse (and feed to a database) configuration files like this one (actually it is Asterisk's sip.conf):
[client-template](!,natted-template)
foo=foovalue
moo=moovalue
[client](client-template)
bar=barvalue
This syntax means that client-template is a template itself (because of ! in parentheses) based on natted-template (defined elsewhere). And client is an object definition, based on client-template.
I could use ConfigParser, but it looks like I need something more powerful or customized?

Well, now I've tried pyparsing:
import pyparsing as pp
filename = 'client.conf'
nametag = pp.Word(pp.alphanums + "-_")
variable = pp.Word(pp.alphanums)
value = pp.Word(pp.alphanums + "=")
vardef = pp.Group(variable('variable') + pp.Literal("=").suppress() + value('value'))
vardefs = pp.Group(pp.ZeroOrMore(vardef))('vardefs')
section = pp.Group(pp.Literal("[").suppress() \
+ nametag('objectname') \
+ pp.Literal("]").suppress() \
+ pp.Optional(
pp.Literal("(").suppress() \
+ pp.Optional("!")('istemplate')
+ pp.ZeroOrMore(pp.Optional(",").suppress() + nametag)('parenttemplates') \
+ pp.Literal(")").suppress()
) \
+ vardefs)
section = section + pp.Optional(pp.SkipTo(section)).suppress()
section_group = pp.Group(section + pp.ZeroOrMore(section))('sections')
config = (pp.SkipTo(section_group).suppress() \
+ section_group)
# res = config.parseString(open(filename).read())
This is a part of a solution (and this is comments unaware so far), but I can proceed with it.
Please, if there is more elegant solution, let me know.

Perfomance improvement - Looping with Get Method

I've built a program to fill up a databank and, by the time, it's working. Basically, the program makes a request to the app I'm using (via REST API) returns the data I want and then manipulate to a acceptable form for the databank.
The problem is: the GET method makes the algorithm too slow, because I'm acessing the details of particular entries, so for each entry I have to make 1 request. I have something close to 15000 requests to do and each row in the bank is taking 1 second to be made.
Is there any possible way to make this requests faster? How can I improve the perfomance of this method? And by the way, any tips to measure the perfomance of the code?
Thanks in advance!!
here's the code:
# Retrieving all the IDs I want to get the detailed info
abc_ids = serializers.serialize('json', modelExample.objects.all(), fields=('id'))
abc_ids = json.loads(abc_ids)
abc_ids_size = len(abc_ids)
# Had to declare this guys right here because in the end of the code I use them in the functions to create and uptade the back
# And python was complaining that I stated before assign. Picked random values for them.
age = 0
time_to_won = 0
data = '2016-01-01 00:00:00'
# First Loop -> Request to the detailed info of ABC
for x in range(0, abc_ids_size):
id = abc_ids[x]['fields']['id']
url = requests.get(
'https://api.example.com/v3/abc/' + str(
id) + '?api_token=123123123')
info = info.json()
dealx = dict(info)
# Second Loop -> Picking the info I want to uptade and create in the bank
for key, result in dealx['data'].items():
# Relevant only for ModelExample -> UPTADE
if key == 'age':
result = dict(result)
age = result['total_seconds']
# Relevant only For ModelExample -> UPTADE
elif key == 'average_time_to_won':
result = dict(result)
time_to_won = result['total_seconds']
# Relevant For Model_Example2 -> CREATE
# Storing a date here to use up foward in a datetime manipulation
if key == 'add_time':
data = str(result)
elif key == 'time_stage':
# Each stage has a total of seconds that the user stayed in.
y = result['times_in_stages']
# The user can be in any stage he want, there's no rule about the order.
# But there's a record of the order he chose.
z = result['order_of_stages']
# Creating a list to fill up with all stages info and use in the bulk_create.
data_set = []
index = 0
# Setting the number of repititions base on the number of the stages in the list.
for elemento in range(0, len(z)):
data_set_i = {}
# The index is to define the order of the stages.
index = index + 1
for key_1, result_1 in y.items():
if int(key_1) == z[elemento]:
data_set_i['stage_id'] = int(z[elemento])
data_set_i['index'] = int(index)
data_set_i['abc_id'] = id
# Datetime manipulation
if result_1 == 0 and index == 1:
data_set_i['add_date'] = data
# I know that I totally repeated the code here, I was trying to get this part shorter
# But I could not get it right.
elif result_1 > 0 and index == 1:
data_t = datetime.strptime(data, "%Y-%m-%d %H:%M:%S")
data_sum = data_t + timedelta(seconds=result_1)
data_sum += timedelta(seconds=3)
data_nova = str(data_sum.year) + '-' + str(formaters.DateNine(
data_sum.month)) + '-' + str(formaters.DateNine(data_sum.day)) + ' ' + str(
data_sum.hour) + ':' + str(formaters.DateNine(data_sum.minute)) + ':' + str(
formaters.DateNine(data_sum.second))
data_set_i['add_date'] = str(data_nova)
else:
data_t = datetime.strptime(data_set[elemento - 1]['add_date'], "%Y-%m-%d %H:%M:%S")
data_sum = data_t + timedelta(seconds=result_1)
data_sum += timedelta(seconds=3)
data_nova = str(data_sum.year) + '-' + str(formaters.DateNine(
data_sum.month)) + '-' + str(formaters.DateNine(data_sum.day)) + ' ' + str(
data_sum.hour) + ':' + str(formaters.DateNine(data_sum.minute)) + ':' + str(
formaters.DateNine(data_sum.second))
data_set_i['add_date'] = str(data_nova)
data_set.append(data_set_i)
Model_Example2_List = [Model_Example2(**vals) for vals in data_set]
Model_Example2.objects.bulk_create(Model_Example2_List)
ModelExample.objects.filter(abc_id=id).update(age=age, time_to_won=time_to_won)

if the bottleneck is in your network request, there isn't much you can do except to perhaps use gzip or deflate but with requests ..
The gzip and deflate transfer-encodings are automatically decoded for
you.
If you want to be doubly sure, you can add the following headers to the get request.
{ 'Accept-Encoding': 'gzip,deflate'}
The other alternative is to use threading and have many requests operate in parrallel, a good option if you have lot's of bandwidth and multiple cores.
Lastly, there are lots of different ways to profile python including with cprofile + kcachegrind combo.

Scripting Python for Linux commands

I have a question. I have been really trying to learn Python. For a project, I want to make an ncurses GUI for my backup server. My backup server runs rdiff-backup, and I want to have the ncurses take in variable names and plug them into my script. I have been trying to do a lot of reading so I don't ask dumb questions.
Here is my function for running the script:
def runScript():
# Cannot concatenate 'str' and 'list' objects
#script = rdiff + rdiffArgs
script = rdiff + ' ' + rdiffVerbosity + ' ' + rdiffStatistics \
+ ' ' + clientName + '#' + clientHost + '::' + clientDir \
+ ' ' + serverDir
os.system(script)
What I originally thought would be neat was to add all the variables into a list, so I could just run say
script = rdiff + rdiffArgs
Is there a better way to do this without all the space concatenation?
Thanks for your assistance
EDIT: Let me post the whole script so far. I wasn't very clear and I really appreciate your help and patience
#!/usr/bin/env python
import os
import smtplib
# Global variables
rdiff = '/usr/bin/rdiff-backup'
rdiffVerbosity = '-v5'
rdiffStatistics = '--print-statistics'
emailSmtp = 'smtp.gmail.com'
smtpPort = '465'
emailUsername = 'reports'
emailPassword = '3kc9dl'
emailTo = 'user#domain.com'
emailFrom = 'internal#domain.com'
serverName = 'root'
serverHost = 'SV-Datasafe'
serverDir = '/srv/backup/SV-Samba01'
clientName = 'root'
clientHost = 'SV-Samba01'
clientDir = '/srv'
rdiffArgs = rdiffArgs = [rdiffVerbosity, rdiffStatistics, \
clientName + '#' + clientHost + '::' \
+clientDir + ' ' + serverDir]
time = ''
dateStamp = datetime.now()
def sendEmail():
subject = dateStamp + clientName
body = clientDir + ' on ' + clientHost + ' backed up to ' + serverName + \
' in the directory ' + serverDir + ' on ' + dateStamp
message = """\
From: %s
To: %s
Subject: %s
%s
""" % (emailFrom, emailTo, subject, body)
deliverEmail = smtplib.SMTP(emailSmtp, port=smtpPort)
deliverEmail.login(emailUsername, emailPassword)
def runScript():
# Cannot concatenate 'str' and 'list' objects
#script = rdiff + rdiffArgs
script = rdiff + ' ' + rdiffVerbosity + ' ' + rdiffStatistics \
+ ' ' + clientName + '#' + clientHost + '::' + clientDir \
+ ' ' + serverDir
os.system(script)
# TODO:: Logging

you can use format specifiers
def runScript():
script = "%s %s %s#%s %s::%s %s" %(rdiff,rdiffVerbosity,rdiffStatistics,clientName,clientHost,clientDir,serverDir)
os.system(script)
or say your rdiffArgs is already in a list
rdiffArgs = [rdiffVerbosity,rdiffStatistics,clientName,clientHost,clientDir,serverDir]
you can join them with a space
rdiffArgs = ' '.join(rdiffArgs)
lastly, just so you might want to know, you can import rdiff in your script , since rdiff-backup is written in Python
from rdiff_backup.Main import Main as backup
task=['/etc', '/tmp/backup']
backup(task)
the above backs up /etc/ to /tmp/backup. That way, you don't have to make system call to rdiff-backup. Of course, this is up to you. making system call is sometimes easier

try to use subprocess module and pass arguments as list e.g.
client = clientName + '#' + clientHost + '::' + clientDir
cmd = [rdiff, rdiffVerbosity, rdiffStatistics, client , serverDir]
p = Popen(cmd ", shell=True)
print os.waitpid(p.pid, 0)[1]
or if have args already as list use something like this
cmd = [rdiff] + args

You join paths using os.path.join
You concatenate strings like so: "".join(['a', 'b']) or ", ".join(['c', 'd'])
Which part is difficult? I am not sure I understand the question 100%
Is this it?
script = rdiff + " ".join(rdiffArgs)