Develop Reference

Checking for Duplicates twice over in a File - Python

config.yml example,
DBtables:
CurrentMinuteLoad:
CSV_File: trend.csv
Table_Name: currentminuteload
GUI image,
This may not be the cleanest route to take.
I'm making a GUI that creates a config.yml file for another python script I'm working with.
Using pysimplegui, My button isn't functioning the way I'd expect it to. It currently and accurately checks for the Reference name (example here would be CurrentMinuteLoad) and will kick it back if it exists, but will skip the check for the table (so the ELIF statement gets skipped). Adding the table still works, I'm just not getting the double-check that I want. Also, I have to hit the Okay button twice in the GUI for it to work?? A weird quirk that doesn't quite make sense to me.
def add_table():
window2.read()
with open ("config.yml","r") as h:
if values['new_ref'] in h.read():
sg.popup('Reference name already exists')
elif values['new_db'] in h.read():
sg.popup('Table name already exists')
else:
with open("config.yml", "a+") as f:
f.write("\n " + values['new_ref'] +":")
f.write("\n CSV_File:" + values['new_csv'])
f.write("\n Table_Name:" + values['new_db'])
f.close()
sg.popup('The reference "' + values['new_ref'] + '" has been included and will add the table "' + values['new_db'] + '" to PG Admin during the next scheduled upload')

When you use h.read(), you should save the value since it will read it like a stream, and subsequent calls for this method will result in an empty string.
Try editing the code like this:
with open ("config.yml","r") as h:
content = h.read()
if values['new_ref'] in content:
sg.popup('Reference name already exists')
elif values['new_db'] in content:
sg.popup('Table name already exists')
else:
# ...

You should update the YAML file using a real YAML parser, that will allow you
to check on duplicate values, without using in, which will give you false
positives when a new value is a substring of an existing value (or key).
In the following I add values twice, and show the resulting YAML. The
first time around the check on new_ref and new_db does not find
a match although it is a substring of existing values. The second time
using the same values there is of course a match on the previously added
values.
import sys
import ruamel.yaml
from pathlib import Path
def add_table(filename, values, verbose=False):
error = False
yaml = ruamel.yaml.YAML()
data = yaml.load(filename)
dbtables = data['DBtables']
if values['new_ref'] in dbtables:
print(f'Reference name "{values["new_ref"]}" already exists') # use sg.popup in your code
error = True
for k, v in dbtables.items():
if values['new_db'] in v.values():
print(f'Table name "{values["new_db"]}" already exists')
error = True
if error:
return
dbtables[values['new_ref']] = d = {}
for x in ['new_cv', 'new_db']:
d[x] = values[x]
yaml.dump(data, filename)
if verbose:
sys.stdout.write(filename.read_text())
values = dict(new_ref='CurrentMinuteL', new_cv='trend_csv', new_db='currentminutel')
add_table(Path('config.yaml'), values, verbose=True)
print('========')
add_table(Path('config.yaml'), values, verbose=True)
which gives:
DBtables:
CurrentMinuteLoad:
CSV_File: trend.csv
Table_Name: currentminuteload
CurrentMinuteL:
new_cv: trend_csv
new_db: currentminutel
========
Reference name "CurrentMinuteL" already exists
Table name "currentminutel" already exists

Can I use abstract methods to import file-specific formatting of (Python) pandas data?

I have a class FileSet with a method _process_series, which contains a bunch of if-elif blocks doing filetag-specific formatting of different pandas.Series:
elif filetag == "EntityA":
ps[filetag+"_Id"] = str(ps[filetag+"_Id"]).strip()
ps[filetag+"_DateOfBirth"] = str(pd.to_datetime(ps[filetag+"_DateOfBirth"]).strftime('%Y-%m-%d')).strip()
ps[filetag+"_FirstName"] = str(ps[filetag+"_FirstName"]).strip().capitalize()
ps[filetag+"_LastName"] = str(ps[filetag+"_LastName"]).strip().capitalize()
ps[filetag+"_Age"] = relativedelta(datetime.today(), datetime.strptime(ps[filetag+"_DateOfBirth"], "%Y-%m-%d")).years
return ps
I'd like to define an abstract format method in the class and keep these blocks of formatting in separate modules that are imported when _process_series is called for a given filetag. Forgive the pseudo-code, but something like:
for tag in filetag:
from my_formatters import tag+'_formatter' as fmt
ps = self.format(pandas_series, fmt)
return ps
And the module would contain the formatting block:
# my_formatters.EntityA_formatter
ps[filetag+"_Id"] = str(ps[filetag+"_Id"]).strip()
ps[filetag+"_DateOfBirth"] = str(pd.to_datetime(ps[filetag+"_DateOfBirth"]).strftime('%Y-%m-%d')).strip()
ps[filetag+"_FirstName"] = str(ps[filetag+"_FirstName"]).strip().capitalize()
ps[filetag+"_LastName"] = str(ps[filetag+"_LastName"]).strip().capitalize()
ps[filetag+"_Age"] = relativedelta(datetime.today(), datetime.strptime(ps[filetag+"_DateOfBirth"], "%Y-%m-%d")).years
return ps

You can create a function in it's own .py file and import it. If you create the same function in each file you can then call it.
here is f1.py:
def gimme():
return 'format 1'
here is f2.py:
def gimme():
return 'format 2'
Then you main file:
module_names = ['f1','f2']
for module_name in module_names:
import_test = __import__(module_name)
result = import_test.gimme()
result = import_test.gimme()
print(result)
Which gives the output:
format 1
format 2

Why not use globals with asterisk:
from my_formatters import *
for tag in filetag:
fmt = globals()[tag + '_formatter']
ps = self.format(pandas_series, fmt)
return ps
I converted your pseudocode to real code.
globals documentation:
Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

Your psuedocode could be made into real code like so:
import my_formatters
for tag in filetag:
fmt = getattr(my_formatters, tag + '_formatter')
ps = self.format(pandas_series, fmt)
return ps

Python Flask show multiple results in a list

I am messing around with a script in Flask I have this portion here
def get_interfaces_list2(device):
output_interfaces = device.send_command('show interfaces switchport')
current_dir = os.getcwd()
template_file = open(current_dir + "/scripts/textfsm/show_interface_switchport.template", "r")
template = TextFSM(template_file)
parsed_interfaces = template.ParseText(output_interfaces)
interface_list = []
for interface_data in parsed_interfaces:
resultDict = {}
resultDict["interface"] = interface_data[0]
resultDict["admin_mode"] = interface_data[5]
resultDict["access_vlan"] = interface_data[6]
resultDict["voice_vlan"] = interface_data[8]
resultDict["trunking_vlans"] = interface_data[9]
interface_list.append(resultDict)
Return interface_list
I would like to add another command to add more info from the switch
output_interfaces1 = device.send_command('show interfaces description')
current_dir = os.getcwd()
template_file = open(current_dir + "/scripts/textfsm/show_interface_description.template", "r")
template = TextFSM(template_file)
parsed_interfaces1 = template.ParseText(output_interfaces1)
interface_list1 = []
for interface_data1 in parsed_interfaces1:
resultDict["descrip"] = interface_data1
interface_list.append(interface_list1)
return interface_list
I would like to combine this into a single list and return that info in an HTML

If I understood correctly, you are currently saving information about an interface in a dictionary and storing that dict in a list. You then want to add more information about the interface. I think there are two approaches you can take here:
Run a single for loop on both parsed_interfaces and parsed_interfaces1 and store all of the info in one shot.
Store the info from your first loop in another dictionary instead of a list where the key is the interface name. Then in the second loop use that key to access the nested dict and store the new info.

How to instantiate an ontology using rdflib?

I have an ontology where I have defined series of classes, subclasses and properties. Now I want to automatically instantiate the ontology with Python code and save it in RDF/XML again and load it in Protege. I have written the following code:
def instantiating_ontology(rdf_address):
from rdflib import *
g = Graph()
input_RDF = g.parse(rdf_address)
#input_RDF = g.open(rdf_address, create=False)
myNamespace="http://www.semanticweb.org/.../ontologies/2015/3/RNO_V5042_RDF"
rno = Namespace(myNamespace+"#")
nodeClass = URIRef(rno+"Node")
arcClass = URIRef(rno+"Arc")
#owlNamespace = 'http://www.w3.org/2002/07/owl#NamedIndividual'
namedIndividual = URIRef('http://www.w3.org/2002/07/owl#NamedIndividual')
rdftype = URIRef("http://www.w3.org/1999/02/22-rdf-syntax-ns#type")
for i in range(0,100):
individualName = rno + "arc_"+str(arcID)
#arc_individual= BNode(individualName)
arc_individual = BNode()
#g.add()
#g.add((arc_individual,rdftype, namedIndividual))
g.add((arc_individual,rdftype, arcClass))
g.add((arc_individual,rdftype, arcClass))
#g.commit()
output_address ="RNO_V5042_RDF.owl"
g.serialize(destination = output_address)
The file contains the added triples to the rdf/xml:
<rdf:Description rdf:nodeID="N0009844208f0490887a02160fbbf8b98">
<rdf:type rdf:resource="http://www.semanticweb.org/ehsan.abdolmajidi/ontologies/2015/3/RNO_V5042#Arc"/>
but when I open the file in Protege there are no instances for the classes.
Can someone tell me if the way I defined instances is wrong or I should use different tags?

After playing around with the code and the results, I realized that the notion rdf:nodeID should be replaced with rdf:about. to do so I only needed to change
for i in range(0,100):
individualName = rno + "arc_"+str(arcID)
#arc_individual= BNode(individualName)
arc_individual = BNode() #---> remove this one
arc_individual = URIRef(individualName) #----> add this one
g.add((arc_individual,rdftype, arcClass))
g.add((arc_individual,rdftype, arcClass))
arc_individual = URIRef(individualName)
that might seem easy but took me sometime to understand. I hope this can help others. :D

LibreOffice - How to create a file dialog via python macro?

I'd like to know if it's possible to create a standard file dialog to save a pdf via a python macro. I've tried to write some code based on this outdated documentation: wiki.openoffice.org but LibreOffice crashes after execution:
import os
import uno
import sys
import traceback
from com.sun.star.ui.dialogs.TemplateDescription import FILESAVE_SIMPLE
def file_dialog():
try:
oCtx = uno.getComponentContext()
oServiceManager = oCtx.getServiceManager()
oFilePicker = oServiceManager.createInstanceWithArgumentsAndContext(
'com.sun.star.ui.dialogs.FilePicker',
(FILESAVE_SIMPLE,),
oCtx
)
oFilePicker.Title = 'Export as'
#oDisp = oFilePicker.Text
oFilePicker.execute()
except:
pass
#oDisp = traceback.format_exc(sys.exc_info()[2])
At the end I need to pass the selected path to write the document, but oDisp = oFilePicker.Text returns: (<type 'exceptions.AttributeError'>. Moreover is there a way to set the file type?
Does anyone have experience with it?

I used Xray on the oFilePicker object. There are a couple of interesting methods called setCurrentFilter and appendFilterGroup. Just based on the names, they might be used to filter what file types are visible. Unfortunately I'm not sure how to use them.
Also with Xray, I determined that Text is not a method or property of the oFilePicker object. I'm not sure what the code snippet is trying to do there? If retrieve the filepath, 1) that needs to be done after the .execute and 2) the selected filepath is stored as an array of strings, so the path has to be pulled out of the array. Most of my work in OpenOffice is in StarBasic; below is a working example in Basic of printing the filepath selected by the user:
Sub TestFilePicker
oFilePickerDlg = createUnoService( "com.sun.star.ui.dialogs.FilePicker" )
oFilePickerDlg.setTitle("My test title")
If oFilePickerDlg.execute() > 0 Then
Print ConvertFromURL(oFilePickerDlg.Files(0))
End If
End Sub

Answer given and accepted (because the question was cross posted!) here:
import uno
from com.sun.star.beans import PropertyValue
#shortcut:
createUnoService = (
XSCRIPTCONTEXT
.getComponentContext()
.getServiceManager()
.createInstance
)
def pypdf_test():
desktop = XSCRIPTCONTEXT.getDesktop()
doc = desktop.getCurrentComponent()
# filter data
fdata = []
fdata1 = PropertyValue()
fdata1.Name = "SelectPdfVersion"
fdata1.Value = 1
fdata2 = PropertyValue()
fdata2.Name = "Quality"
fdata2.Value = 100
fdata.append(fdata1)
fdata.append(fdata2)
args = []
arg1 = PropertyValue()
arg1.Name = "FilterName"
arg1.Value = "writer_web_pdf_Export"
arg2 = PropertyValue()
arg2.Name = "FilterData"
arg2.Value = uno.Any("[]com.sun.star.beans.PropertyValue", tuple(fdata) )
args.append(arg1)
args.append(arg2)
fileurl = FilePicker()
if fileurl:
doc.storeToURL( fileurl, tuple(args) )
def FilePicker(path=None, mode=1):
"""
Datei öffnen: `mode in (0, 6, 7, 8, 9)`
Datei Schreiben `mode in (1, 2, 3, 4, 5, 10)`
see: ('''http://api.libreoffice.org/docs/idl/ref/
namespacecom_1_1sun_1_1star_1_1ui_1_1
dialogs_1_1TemplateDescription.html''' )
"""
filepicker = createUnoService( "com.sun.star.ui.dialogs.OfficeFilePicker" )
if path:
filepicker.setDisplayDirectory(path )
filepicker.initialize( ( mode,) )
if filepicker.execute():
return filepicker.getFiles()[0]