get SQLSERVER json and using in python - python
Guys can anyone help? I have a sqlserver json:
SELECT CEQ.CODCARGOEQ AS CODCARGO, INITCAP(ISNULL((CAR.DESCRCARGO), \'SEM DESCRICAO\')) AS DESCRICAO
FROM TFPCAR CAR
INNER JOIN AD_TBCARGOEQUIVALENTE CEQ
ON CEQ.CODCARGOEQ = CAR.CODCARGO
WHERE CAR.ATIVO = \'S\' AND CAR.CODCARGO > 0
FOR JSON PATH, ROOT(\'LISTACARGOS\')
That sql returns to me a follow json
{"LISTACARGOS":[{"CODCARGO":1,"DESCRICAO":"Advogado"},
{"CODCARGO":2,"DESCRICAO":"Agente De Limpeza"},
{"CODCARGO":3,"DESCRICAO":"Agente De Transporte"},
{"CODCARGO":4,"DESCRICAO":"Ajud De Manutenção"},{"CODCARGO":5,"DESCRICAO":"Ajud Farmacia"},{"CODCARGO":6,"DESCRICAO":"Ajudante De Fossa"},{"CODCARGO":7,"DESCRICAO":"Almoxarife"},{"CODCARGO":8,"DESCRICAO":"Analista Comercial"},{"CODCARGO":9,"DESCRICAO":"Analista De Complice"},{"CODCARGO":10,"DESCRICAO":"Analista De Dho"},{"CODCARGO":11,"DESCRICAO":"Analista De Pessoal"},{"CODCARGO":12,"DESCRICAO":"Analista De Projetos"},{"CODCARGO":13,"DESCRICAO":"Analista De Recursos Humanos"},{"CODCARGO":14,"DESCRICAO":"Analista De Ti"},{"CODCARGO":15,"DESCRICAO":"Analista Financeiro"},{"CODCARGO":16,"DESCRICAO":"Apoio Tecnico"},{"CODCARGO":17,"DESCRICAO":"Artificie Eletrônica"},{"CODCARGO":18,"DESCRICAO":"Asses.Tec.Juridico"},{"CODCARGO":19,"DESCRICAO":"Assist De Comunic E Marketing"},{"CODCARGO":20,"DESCRICAO":"Assist. Administrativo Senior"},{"CODCARGO":21,"DESCRICAO":"Assistente Administrativo"},{"CODCARGO":22,"DESCRICAO":"Assistente De Adm De Pessoas"},{"CODCARGO":23,"DESCRICAO":"Assistente Social"},
{"CODCARGO":487,"DESCRICAO":"Tecnico Social (Psic)"},{"CODCARGO":488,"DESCRICAO":"Tecnico Social Advogado"},{"CODCARGO":489,"DESCRICAO":"Tecnico Social I"},{"CODCARGO":490,"DESCRICAO":"Tecnico Social I Assistente Social"},{"CODCARGO":491,"DESCRICAO":"Tecnico Social I Psicologo"},{"CODCARGO":492,"DESCRICAO":"Tecnico Social Ii"},{"CODCARGO":493,"DESCRICAO":"Tecnico Social Ii Assistente Social"},{"CODCARGO":494,"DESCRICAO":"Tecnico Social Ii Psicologo"},{"CODCARGO":495,"DESCRICAO":"Tecnico Social Iii"},{"CODCARGO":496,"DESCRICAO":"Tecnico Social Iv"},{"CODCARGO":497,"DESCRICAO":"Tecnico Social Ix"},{"CODCARGO":498,"DESCRICAO":"Tecnico Social Psicologo"},{"CODCARGO":499,"DESCRICAO":"Tecnico Social V"},{"CODCARGO":500,"DESCRICAO":"Tecnico Social Vi"},{"CODCARGO":501,"DESCRICAO":"Tecnico Social Vii"},{"CODCARGO":502,"DESCRICAO":"Tecnico Social Viii"},{"CODCARGO":503,"DESCRICAO":"Tecnico Social X"},{"CODCARGO":504,"DESCRICAO":"Tecnico Social Xi"},{"CODCARGO":505,"DESCRICAO":"Tecnico Social Xii"},{"CODCARGO":506,"DESCRICAO":"Tecnico Social Xiii"},{"CODCARGO":507,"DESCRICAO":"Tecnico Social Xiv"},{"CODCARGO":508,"DESCRICAO":"Tecnico Social Xv"},{"CODCARGO":509,"DESCRICAO":"Técnico Terapeuta Ocupacional"},{"CODCARGO":510,"DESCRICAO":"Teledigifonista"},{"CODCARGO":511,"DESCRICAO":"Telefonista E Recepcionista"},{"CODCARGO":512,"DESCRICAO":"Tratorista"},{"CODCARGO":513,"DESCRICAO":"Varredor (A)"},{"CODCARGO":514,"DESCRICAO":"Viveirista Florestal"},{"CODCARGO":515,"DESCRICAO":"Motorista"},{"CODCARGO":516,"DESCRICAO":"Gestor Financeiro"},{"CODCARGO":517,"DESCRICAO":"Fonoaudiólogo"},{"CODCARGO":518,"DESCRICAO":"Pintor"},
{"CODCARGO":519,"DESCRICAO":"Op Sus Facil"},{"CODCARGO":520,"DESCRICAO":"Téc De Radiologia"},{"CODCARGO":521,"DESCRICAO":"Gerente De Compliance"},{"CODCARGO":522,"DESCRICAO":"Eletricista"},{"CODCARGO":523,"DESCRICAO":"Técnico Em Mecânica"},{"CODCARGO":524,"DESCRICAO":"Auxiliar De Conservação De Vias"},{"CODCARGO":525,"DESCRICAO":"Jornalista"},{"CODCARGO":526,"DESCRICAO":"Assessor Pedagogico"},{"CODCARGO":527,"DESCRICAO":"Oficial De Manutenção Civil"},{"CODCARGO":528,"DESCRICAO":"Zelador"},{"CODCARGO":529,"DESCRICAO":"Supervisor Geral"},{"CODCARGO":530,"DESCRICAO":"Coordenação De Contratos E Medição"},{"CODCARGO":531,"DESCRICAO":"Supervisor Administrativo"},{"CODCARGO":532,"DESCRICAO":"Nutricionista"},{"CODCARGO":533,"DESCRICAO":"Especialista Em Saúde"},{"CODCARGO":534,"DESCRICAO":"Analista Fiscal"},
{"CODCARGO":535,"DESCRICAO":"Assistente De Projetos Socioassistenciais"},
{"CODCARGO":536,"DESCRICAO":"Auxiliar Geral De Conservação De Vias Permanentes"}]}
I need do catch this full json in my python API, and for that i use the follow function:
def get_listcargo(myquery):
cursor = conn()
cursor.execute(myquery)
for row in cursor:
cargolist = [elem for elem in row]
return (cargolist)
But python returns to me a truncate data like this
"},{"CODCARGO":507,"DESCRICAO":"Tecnico Social Xiv"},{"CODCARGO":508,"DESCRICAO":"Tecnico Social Xv"},{"CODCARGO":509,"DESCRICAO":"Técnico Terapeuta Ocupacional"},{"CODCARGO":510,"DESCRICAO":"Teledigifonista"},{"CODCARGO":511,"DESCRICAO":"Telefonista E Recepcionista"},{"CODCARGO":512,"DESCRICAO":"Tratorista"},{"CODCARGO":513,"DESCRICAO":"Varredor (A)"},{"CODCARGO":514,"DESCRICAO":"Viveirista Florestal"},{"CODCARGO":515,"DESCRICAO":"Motorista"},{"CODCARGO":516,"DESCRICAO":"Gestor Financeiro"},{"CODCARGO":517,"DESCRICAO":"Fonoaudiólogo"},{"CODCARGO":518,"DESCRICAO":"Pintor"},{"CODCARGO":519,"DESCRICAO":"Op Sus Facil"},{"CODCARGO":520,"DESCRICAO":"Téc De Radiologia"},{"CODCARGO":521,"DESCRICAO":"Gerente De Compliance"},{"CODCARGO":522,"DESCRICAO":"Eletricista"},{"CODCARGO":523,"DESCRICAO":"Técnico Em Mecânica"},{"CODCARGO":524,"DESCRICAO":"Auxiliar De Conservação De Vias"},{"CODCARGO":525,"DESCRICAO":"Jornalista"},{"CODCARGO":526,"DESCRICAO":"Assessor Pedagogico"},{"CODCARGO":527,"DESCRICAO":"Oficial De Manutenção Civil"},{"CODCARGO":528,"DESCRICAO":"Zelador"},{"CODCARGO":529,"DESCRICAO":"Supervisor Geral"},{"CODCARGO":530,"DESCRICAO":"Coordenação De Contratos E Medição"},{"CODCARGO":531,"DESCRICAO":"Supervisor Administrativo"},{"CODCARGO":532,"DESCRICAO":"Nutricionista"},{"CODCARGO":533,"DESCRICAO":"Especialista Em Saúde"},{"CODCARGO":534,"DESCRICAO":"Analista Fiscal"},{"CODCARGO":535,"DESCRICAO":"Assistente De Projetos Socioassistenciais"},{"CODCARGO":536,"DESCRICAO":"Auxiliar Geral De Conservação De Vias Permanentes"}]}
what i have done wrong?
FOR JSON results are streamed to the client using a single-column multi-row resultset. Long JSON results will be broken over multiple rows. So you need to iterate the cursor, concatenating the values in the first column to reconstruct the whole JSON document.
eg
def get_listcargo(myquery):
cursor = conn()
cursor.execute(myquery)
json = ''
for row in cursor:
json = json + row[0]
return (json)
Related
the phone number input in sqlite using fastapi and toetoise-orm
I' m creating a fastapi API usinging tortoise-orm and sqlite and the bloblem that i have is that in one of my models witch is "Member", i wanna have a possibility for the member to put their phone number with their country code bu i dont'n kwow how to handle that; I'm using "phon_num= fields.CharField(max_length=13)", I wonder how am I gonna allow only numbers to be put in that fields. tanks #app.post("/members") async def create_member(name:str, email:EmailStr, phon_num:str,adress:str): member = await Member.create(name=name, email=email,phon_num=phon_num, adress=adress) if member.phon_num.isnumeric: member.save() return member else: return {"status": "vous avez un probleme dans le remplissage de votre formulaire", "detail": "votre numero ne doit contenir que des chiffre ansi que l'indicatif pay '+code pays', avant d'entrer le numero"} I expected that the if statement could control either the input is a string witch contains only the number and verify if in that string there is a "+" character for the country code.
Python add when entries have the same name
I have a model with the registered sales of a publishing house.with the following code, I can get the name of the book and the quantity the client bought: a=Sale.objects.all() for b in a: print(str(b.book) + ' ' + str(b.quantity)) So I get something like this: Del mismo modo en el sentido contrario 15 Sobre el horror 1 Del mismo modo en el sentido contrario 5 Del mismo modo en el sentido contrario 2 Un lápiz labial para una momia 1 La cólera en los tiempos del amor 3 La cólera en los tiempos del amor 1 La cólera en los tiempos del amor 1 El tambor encantado 1 What I need now is a loop so every entry that has the same book name, sum or add that quantity number. Any ideas on how to accomplish this?
For Django, to do this entirely in the database, it sounds like you want an aggregation using .values(): for book, total_sales in Sale.objects.values('book').annotate(total_sales=Sum('quantity')).values_list('book', 'total_sales'): print(book, total_sales) The general case, if this wasn't only about Django, is to use a collections.Counter: books_sold = collections.Counter() for sale in Sale.objects.all(): books_sold[str(sale.book)] += b.quantity for book, quantity in books_sold.items(): print(book, quantity)
If book is a ForeignKey to a Book model, it makes more sense to .annotate(…) [Django-doc] the Books instead: from django.db.models import Sum books = Book.objects.annotate( total_quantity=Sum('sale__quantity') ) for book in books: print(f'{book} {book.total_quantity}')
Django sort list
I have this queryset: <QuerySet [<ProductAttributeValue: Utilizare: Lazurán 3 în 1 Lazur Protector se utilizează pentru lăcuirea decorativă.>, <ProductAttributeValue: Randament: 16 m2/lit., într-un strat>, <ProductAttributeValue: Straturi recomandate: 2>, <ProductAttributeValue: Timp de uscare la 23 °C: 2 ore>, <ProductAttributeValue: Timp de reaplicare la 23 °C: 2 ore>, <ProductAttributeValue: Mod de aplicare: după o amestecare prealabilă se aplică cu pensula>, <ProductAttributeValue: Ambalare: 0.75L, 2.5L>]> I would like to sort this queryset by fields name. For example when I display it I want that order: (Utilizare, Mod de aplicare, Randament, Ambalare, Straturi recomandate, Timp de uscare la 23 °C) Actual code: self.attribute_values.exclude( attribute__name__in=['Buttons', 'Ambalare']).exclude( attribute__type='file').order_by('pk') There is any solution to sort this list in a given order?
Try this: querySet.objects.order_by('field_name') From the django-documentation-site: https://docs.djangoproject.com/en/3.0/ref/models/querysets/#order-by
self.attribute_values.exclude( attribute__name__in=['Buttons', 'Ambalare']).exclude( attribute__type='file').order_by('pk').order_by('field_name') That should work, and also take a quick read on the documentation here.
Delete a string phrase from a data frame column and replace it python
So, I have two dataframes. the first dataframe is dataset conatians several columns, what i will use in this dataframe is the dataset['text_msg'], this columns contains text data. The second Dataframe sentences_to_exclude contains the data which type is text type. The column that i will use in this dataframe is sentences_to_exclude['sentences']. What i need to do is to verify if there are sentences from sentences_to_exclude['sentences'] in the first dataframe and remove the whole sentence. I have tried a function but it didn't work for me: Here is the function i've used ==> def remove_words(data): words_to_remove = sentences_to_exclude['sentences'].lower().split(" ") text_body = dataset['text_msg'] for word in words_to_remove: text_body = text_body.replace(word,'' ) return text_body Here's an exemple of sentences_to_exclude['sentences'] pour un traitement optimal de votre demande, veuillez indiquer les informations ci-dessous and for the fisrt data frame here's an example of the dataset['text_msg']: pour un traitement optimal de votre incident, nous vous prions de renseigner les informations ci-dessous : - code transaction : - numero de facture / commande client : - criteres dexecution et message derreur (a attacher en pj) description detaillee de votre demande Hope that my request is clear Thank you for help in advance Example Data sentences = ['code transaction', 'Pour un traitement efficace'] text = [ ' i should delete code transaction ', ' i am trying to delete Pour un traitement efficace only from this sentence ' ] df1 = pd.DataFrame({'Sentences ': sentences }) df2 = pd.DataFrame({'Text': text})
Still don't understand your question correctly, I will try to help you, but please next time you have to include example data. To answer your question I will give example dataset and explain how to remove words or sentences from other text: # This is our example data sentences = ['code transaction', 'Pour un traitement efficace'] text = [ ' i should delete code transaction ', ' i am trying to delete Pour un traitement efficace only from this sentence ' ] df1 = pd.DataFrame({'Sentences': sentences}) df2 = pd.DataFrame({'Text': text}) # df1 Sentences 0 code transaction 1 Pour un traitement efficace # df2 Text 0 i should delete code transaction 1 i am trying to delete Pour un traitement effi... Next we want to harmonize our data so we wont have mismatches, so we convert to uppercase: df1['Sentences'] = df1.Sentences.str.upper() df2['Text'] = df2.Text.str.upper() Sentences 0 CODE TRANSACTION 1 POUR UN TRAITEMENT EFFICACE Text 0 I SHOULD DELETE CODE TRANSACTION 1 I AM TRYING TO DELETE POUR UN TRAITEMENT EFFI... Now our data is in the right format, we can remove the text from one dataset to another df2['Text_cleaned'] = df2.Text.str.replace('|'.join(df1.Words), '') Text Text_cleaned 0 I SHOULD DELETE CODE TRANSACTION I SHOULD DELETE 1 I AM TRYING TO DELETE POUR UN TRAITEMENT EFFI... I AM TRYING TO DELETE ONLY FROM THIS SENTENCE What does '|'.join(df1.Sentences) do? It returns a string delimited by | '|'.join(df1.Words) 'CODE TRANSACTION|POUR UN TRAITEMENT EFFICACE' Hope this helps you and answers your question. You can now apply this logic to your own data.
Remove line pattern script in Python not fully functional
For logs parsing, I need to remove a pattern in a log file: Pattern: Acces : Lecture donnees (ou liste de repertoire) Privileges : - Nombre de SID restreint : 0 Masque d acces : 0x1 " Log sample: # This line has to be kept 2014-03-03 09:50:20,2014-03-03 09:50:20,560,Success Audit event,Accès aux objets,Security,LA\USER1,LA-SERVER1,"Objet ouvert : Serveur de l’objet : Security Type d’objet : File Objet : \vol\vol01\PROD\prod.conf Identificateur du handle : 554 Identificateur de l’opération : - ID du processus : 2050 Nom du fichier image : Server Soft Utilisateur principal : user1 Domaine principal : LA ID d’ouv. de session principale : (0x0, 0x9596) Utilisateur client : 1.2.3.4 Domaine client : - ID d’ouv. de session client : - Acces : Lecture données (ou liste de répertoire) Écriture données (ou ajout fichier) Ajout données (ou ajout sous-répertoire ou créer instance de canal) WRITE_DAC Privilèges : - Nombre de SID restreint : 0 Masque d’accès : 0x40007 " # This line has to be removed 2014-03-03 09:52:20,2014-03-03 09:50:20,560,Success Audit event,Accès aux objets,Security,LA\USER2,LA-SERVER1,"Objet ouvert : Serveur de l’objet : Security Type d’objet : File Objet : \vol\vol01\PROD\prod.conf Identificateur du handle : 554 Identificateur de l’opération : - ID du processus : 2050 Nom du fichier image : Server Soft Utilisateur principal : user1 Domaine principal : LA ID d’ouv. de session principale : (0x0, 0x9597) Utilisateur client : 1.2.3.5 Domaine client : - ID d’ouv. de session client : - **Acces : Lecture donnees (ou liste de repertoire) Privileges : - Nombre de SID restreint : 0 Masque d acces : 0x1 "** # This line has to be removed 2014-03-03 09:53:20,2014-03-03 09:50:20,560,Success Audit event,Accès aux objets,Security,LA\USER3,LA-SERVER1,"Objet ouvert : Serveur de l’objet : Security Type d’objet : File Objet : \vol\vol01\PROD\prod.conf Identificateur du handle : 554 Identificateur de l’opération : - ID du processus : 2050 Nom du fichier image : Server Soft Utilisateur principal : user1 Domaine principal : LA ID d’ouv. de session principale : (0x0, 0x9597) Utilisateur client : 1.2.3.6 Domaine client : - ID d’ouv. de session client : - **Acces : Lecture donnees (ou liste de repertoire) Privileges : - Nombre de SID restreint : 0 Masque d acces : 0x1 "** I found a script here (thx to ATOzTOA) to do the job (I just added sys module): import sys fname = sys.argv[1] def delete_line(dello): data = open(fname).readlines() i = 0 for line in data: if dello in line: data.pop(i) i += 1 open(fname, "w").write("".join(data)) delete_line("Acces : Lecture donnees (ou liste de repertoire) Privileges : - Nombre de SID restreint : 0 Masque d acces : 0x1\"") When I run the script, it didn't remove the 2 lines of logs which contain the pattern : only one of the two. I have to run once or twice again the script to delete all the lines I don't know why.
You shouldn't increment i when calling pop(). Also, you shouldn't use pop() when iterating over a list. This is because if you remove an element from the middle of a list, then you skip some items while iterating. You should rewrite delete_line() as follows: def delete_line(dello): with open(fname) as f: data = [line for line in f if dello not in line] with open(fname, 'w') as f: f.write(''.join(data)) (Note that there is room for improvement.) An another problem is that you are giving delete_line() the wrong argument: there's a space between 0x1 and ". This is the correct line: delete_line('Acces : Lecture donnees (ou liste de repertoire) Privileges : - Nombre de SID restreint : 0 Masque d acces : 0x1 "') By the way, there's already an excellent program that filters out lines from a file: it's grep (with the -v option).
I can't replicate your error locally. However, I will suggest that you can try the python replace method instead. http://www.tutorialspoint.com/python/string_replace.htm str.replace(old, new[, max]) which will search for the replaced it with what you want in your string. Therefore, the one possible solution to your problem will be : import sys fname = sys.argv[1] def delete_line(dello): cleaned_data = [] data = open(fname).readlines() for line in data: line = line.replace(dello, "") cleaned_data.append(line) open(fname, "w").write("".join(data)) delete_line("Acces : Lecture donnees (ou liste de repertoire) Privileges : - Nombre de SID restreint : 0 Masque d acces : 0x1\"") You may replace your for loop with a list comprehension clean_data = [line.replace(dello,"") for line in data]