I'm developing too to compare database schema of Test and Prod database.
I can succesfully compare schema and print to command line.
However I don't know how to store results to JSON, CSV file or any file. Please advice!
from pprint import pprint
from sqlalchemydiff import compare
from sqlalchemy.engine import URL
import pyodbc
import time
# Pass through Pyodbc string
conn_string_dw_test = "DRIVER=..."
conn_string_dw_prod = "DRIVER=..."
connection_url_dw_test = URL.create("mssql+pyodbc", query={"odbc_connect": conn_string_dw_test})
connection_url_dw_prod = URL.create("mssql+pyodbc", query={"odbc_connect": conn_string_dw_prod})
print('')
print('-----SCHEMA COMPARE FOR TEST AND PROD DW-----')
result_dw = compare(connection_url_dw_test, connection_url_dw_prod)
if result_dw.is_match:
print('')
print('DW Schemas are identical')
print('')
else:
print('')
print('We detected following differences')
print('DW Test is on Left. DW Prod is on Right')
print('')
pprint(result_dw.errors)
# Export CSV
filename = "SchemaCompareReports\SchemaCompareReport_" + time.strftime("%Y%m%d-%H%M%S") + ".csv"
result_dw.to_csv(filename) # NOT WORKING
print("Report exported: " + filename)
ERROR in first try:
traceback (most recent call last):
File ".\SchemaComparePOC.py", line 74, in
result_dw.to_csv(filename)
AttributeError: 'CompareResult' object has no attribute 'to_csv'
I also tried in second try to save results to json file, but got error:
filename = "SchemaCompareReport_DW_" + time.strftime("%Y%m%d-%H%M%S") + ".json"
a_file = open(filename, "w")
json.dump(result_dw.dump_errors, a_file)
a_file.close()
Error of second try:
Traceback (most recent call last):
File "./SchemaComparePOC.py", line 106, in <module>
json.dump(result_dw.dump_errors, a_file)
File "C:\Python\Python3.8.9\lib\json\__init__.py", line 179, in dump
for chunk in iterable:
File "C:\Python\Python3.8.9\lib\json\encoder.py", line 438, in _iterencode
o = _default(o)
File "C:\Python\Python3.8.9\lib\json\encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type method is not JSON serializable
In third try I got no error, but file was empty:
filename = "SchemaCompareReport" + time.strftime("%Y%m%d-%H%M%S") + ".json"
a_file = open(filename, "w")
json.dump(result_dw.dump_errors.__dict__, a_file)
a_file.close()
Related
I am working on AZURE Cognitive API Search. While getting the result from API, I want to write it into a new JSON File. I tried to access the analyse_result variable with the line but it does not work. It shows that the object is not JSON Serializable. My code is-
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
from azure.cognitiveservices.vision.computervision.models import VisualFeatureTypes
from msrest.authentication import CognitiveServicesCredentials
from array import array
import os
from PIL import Image
import sys
import time
import json
import csv
subscription_key = ""
endpoint = ""
computervision_client = ComputerVisionClient(endpoint, CognitiveServicesCredentials(subscription_key))
def azure_ocr_api(): #image_url
local_image_url = r"E:\Bank of Baroda\BOB IMAGE\Cheque309086.jpeg"
# read_response = computervision_client.read_in_stream(open("./Images/" + image_url,'rb'), raw=True)
read_response = computervision_client.read_in_stream(open(local_image_url,'rb'), raw=True)
# Get the operation location (URL with an ID at the end) from the response
read_operation_location = read_response.headers["Operation-Location"]
# Grab the ID from the URL
operation_id = read_operation_location.split("/")[-1]
# Call the "GET" API and wait for it to retrieve the results
while True:
read_result = computervision_client.get_read_result(operation_id)
if read_result.status not in ['notStarted', 'running']:
break
time.sleep(1)
list = []
if read_result.status == OperationStatusCodes.succeeded:
for text_result in read_result.analyze_result.read_results:
for line in text_result.lines:
with open('data.json', 'w', encoding='utf-8') as f:
json.dump(line, f, ensure_ascii=False, indent=4)
# print(list)
# pass
# return list
azure_ocr_api()
print("End of Computer Vision quickstart.")
The code shows a error like this -
Traceback (most recent call last):
File "e:\Bank of Baroda\m.py", line 44, in <module>
azure_ocr_api()
File "e:\Bank of Baroda\m.py", line 40, in azure_ocr_api
json.dump(line, f, ensure_ascii=False, indent=4)
File "C:\Users\Clasher\anaconda3\lib\json\__init__.py", line 179, in dump
for chunk in iterable:
File "C:\Users\Clasher\anaconda3\lib\json\encoder.py", line 438, in _iterencode
o = _default(o)
File "C:\Users\Clasher\anaconda3\lib\json\encoder.py", line 179, in default
TypeError: Object of type Line is not JSON serializable
Please help.
i have this problem: i'm tryna list all filenames from a directory and then print them with numbers on the left to let users select the file to use. Numbers are there because they match with the index of the position of filenames in the list. When i select a specific string filename in the list and pass it to
`f = open (filename, "r")
data = json.loads(f.read())`
i get:
Traceback (most recent call last):
File "test.py", line 170, in
data = json.loads(f.read())
AttributeError: 'str' object has no attribute 'loads'
full code:
jsons=glob.glob("*.json")
print(type(jsons))
print(type(jsons[0]))
n=0
if(jsons):
for json in jsons:
print(str(n) + ' - ' + json)
n=n+1
jsonselection=int(input('select your settings file: '))
filename=(" ".join(jsons[jsonselection].split()))
print()
print('your selection: ' + filename)
else:
sys.exit(colored('no json files available.','red'))
f = open (filename, "r")
data = json.loads(f.read())
actually if i pass to the method a random variable defined by me like name='file' it works.. i just can't understand why. Thanks in advance for the help
That would most likely be because of this line of code.
jsons=glob.glob("*.json")
and then you have
for json in jsons:
Im having this error while opening mutiples files at once with tkinter on Python3.
My Code while opening a file(s):
def OpenFile():
Tk().withdraw()
filename = askopenfilenames(title='Choose Combolist', filetypes=[("Text Files", "*.txt")])
if filename:
TxtTimeName(filename)
pass
else:
NoFileSelected()
return filename
def TxtTimeName(filename):
CurrentTime = strftime(" %H-%M-%S")
TxtName = os.path.basename(filename)
TxtName = TxtName.replace(".txt", " ")
FullName = TxtName + CurrentTime + ".txt"
return FullName
My code while using theese file(s):
def MailToUser():
filename = OpenFile()
FullName = TxtTimeName(filename)
ctypes.windll.kernel32.SetConsoleTitleW("TextTool | Made by VRX | Mode: Mail To User")
sys.stdout.flush()
try:
os.mkdir('Mail to User')
except Exception as E:
pass
f = open(str("./Mail to User/" + "Combined.txt"),"w+")
StartTime()
with open(filename, "r+", encoding="utf-8", errors="ignore") as file:
for line in file:
word = line.strip()
firstDelPos = line.find("#")
secondDelPos = line.find(":")
stringAfterReplace = line.replace(line[firstDelPos + 0:secondDelPos], "")
try:
f.write(stringAfterReplace)
except Exception as E:
pass
EndTime()
ERROR:
Traceback (most recent call last):
File "C:\Users\VRX\OneDrive\Desktop\Desktop\Python Projects\combotool\combotool.py", line 494, in <module>
Main()
File "C:\Users\VRX\OneDrive\Desktop\Desktop\Python Projects\combotool\combotool.py", line 420, in Main
TextSorter()
File "C:\Users\VRX\OneDrive\Desktop\Desktop\Python Projects\combotool\combotool.py", line 82, in TextSorter
filename = OpenFile()
File "C:\Users\VRX\OneDrive\Desktop\Desktop\Python Projects\combotool\combotool.py", line 27, in OpenFile
TxtTimeName(filename)
File "C:\Users\VRX\OneDrive\Desktop\Desktop\Python Projects\combotool\combotool.py", line 38, in TxtTimeName
TxtName = os.path.basename(filename)
File "C:\Users\VRX\AppData\Local\Programs\Python\Python38\lib\ntpath.py", line 208, in basename
return split(p)[1]
File "C:\Users\VRX\AppData\Local\Programs\Python\Python38\lib\ntpath.py", line 177, in split
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not tuple
It gives me this error everytime even when im selecting one file, it works when I use askopenfilename instead of askopefilenames but I can only select 1 file.
The error is because of the following:
askopenfilenames returns a tuple of filenames, even if there's only one
you are passing the result of askopenfilenames to os.path.basename, but that function expects a single filename
That is why you get the error expected str, bytes, or os.PathLike object, not tuple: you are passing a tuple and the error message says you can't do that.
if using windows, find ntpath.py file in your environment
[miniconda3\envs\your environment\Lib\ntpath.py]
Go to line 38
def _get_bothseps(path):
if isinstance(path, bytes):
return b'\/'
else:
return '\/'
replace return '\/'
with return'/'
save ntpath.py
hope this helps. It worked for me.
I am currently working on a project for which I need to download a few thousand citations from PubMed. I am currently using BioPython and have written this code:
from Bio import Entrez
from Bio import Medline
from pandas import *
from sys import argv
import os
Entrez.email = "my_email"
df = read_csv("my_file_path")
i=0
for index, row in df.iterrows():
print (row.id)
handle = Entrez.efetch(db="pubmed",rettype="medline",retmode="text", id=row.id)
records = Medline.parse(handle)
for record in records:
try:
abstract = str(record["AB"])
except:
abstract = "none"
try:
title = str(record["TI"])
except:
title = "none"
try:
mesh = str(record["MH"])
except:
mesh = "none"
path = 'my_file_path'
filename= str(row.id) + '.txt'
filename = os.path.join(path, filename)
file = open(filename, "w")
output = "title: "+str(title) + "\n\n" + "abstract: "+str(abstract) + "\n\n" + "mesh: "+str(mesh) + "\n\n"
file.write(output)
file.close()
print (i)
i=i+1
However, I receive the following error when this code is run:
Traceback (most recent call last):
File "my_file_path", line 13, in <module>
handle = Entrez.efetch(db="pubmed",rettype="medline",retmode="text", id=row.id)
File "/.../anaconda/lib/python3.5/site-packages/biopython-1.68-py3.5-macosx-10.6-x86_64.egg/Bio/Entrez/__init__.py", line 176, in efetch
if ids.count(",") >= 200:
AttributeError: 'numpy.int64' object has no attribute 'count'
Here are the first few columns of the CSV file:
id
10029645
10073846
10078088
10080457
10088066
...
Your error is at
handle = Entrez.efetch(db="pubmed",rettype="medline",retmode="text", id=row.id)
From the documentation
id
UID list. Either a single UID or a comma-delimited list of UIDs
From the examples I see, id is a string, not a numpy.int64 out of a pandas dataframe. You should convert that row.id to a string
td = 'date of transaction.txt'
tf = 'current balance.txt'
tr = 'transaction record.txt'
for line in open(tf):pass
for line2 in open(td):pass
for line3 in open(tr):pass
print line2,line,line3
"""
so call recall back last record
"""
rd=raw_input('\ndate of transaction: ')
print "Choose a type of transaction to proceed... \n\tA.Withdrawal \n\tB.Deposit \n\tC.Cancel & exit"
slc=raw_input('type of transaction: ')
i=1
while (i>0):
if slc=="A" or slc=="B" or slc=="C":
i=0
else:
i=i+1
slc=raw_input('invalid selection, please choose again...(A/B/C): ')
if slc=="A":
rr=input('\namount of transaction: ')
Line_len = 10 # or however long a line is, since in my example they all looked the same
SEEK_END = 2
file = open(tf, "r")
file.seek(-Line_len, SEEK_END)
a = int(str(file.read(Line_len)).split(" ")[0].strip())
rf=a-rr
f1=open(tf, 'a+')
f1.write('\n'+rf)
f1.close()
d1=open(td, 'a+')
d1.write('\n'+rd)
d1.close
r1=open(tr, 'a+')
r1.write('\n-'+rr)
r1.close
else:
print 'later'
above is my code, the function is to get data(last line) from txt file and read it, get new data and write it to the txt file again by creating new line.
my txt file(current balance.txt) should look like this:
2894.00
2694.00
but when i try to use the last line which is 2694.00 to do calculation(rf=a-rr), it failed returning this error:
Traceback (most recent call last):
File "C:\Python27\acc.py", line 27, in <module>
file.seek(-Line_len, SEEK_END)
IOError: [Errno 22] Invalid argument
else if i use this code:
for line in open(tf):
pass
a = line
rf=a-rr
it return this error:
Traceback (most recent call last):
File "C:\Python27\acc.py", line 27, in <module>
rf=a-rr
TypeError: unsupported operand type(s) for -: 'str' and 'int'
I seriously have no idea why...please help me...
To obtain last line of the file, you can simple do
with open('my_file.txt') as file:
last_line = file.readlines()[-1]
#last_line is a string value pointing to last line, to convert it into float, you can do
number = float(last_line.strip('\n').strip(' '))
The function input is giving you a string. Try doing:
rf=a-float(rr)