Develop Reference

Attempting to Traverse a search API Collecting results in Python 3.9

I am attempting to yield all results from a search api.
There are 3 key cases:
Case 1: There are 0 results yielded
Case 2: There are 1-9 results yielded
Case 3: There are 10 results yielded
In both cases 1 and 2, we can conclude that there are no results deeper. i.e if we searched for "iiiiia" and it yields 0 then that means there are no results for "iiiiia" and no results deeper than "iiiiia". If "iiiiia" yields 1-9 results, then we can concluded "iiiiia" yields 109 results and no results deeper.
In case 3, if we search for "iiiiia" and it yields 10, we can conclude that "iiiiia" has 10 results, and there may or may not be results deeper as 10 means 10+ results possible. In this case we would have to move one layer down the chain and traverse through those results. i.e "iiiiia0"-"iiiiiaz" and if any of those yield 10 we would also have to go one layer deeper.
an example of how this may look:
a
a0
a00
a000
a0000
a0001
...
a000z
a001
a002
a003
a0030
...
a0034
a00340
...
a0034z
a0035...
Here is my attempted code:
import json
from time import sleep
import urllib3
import requests
import re
import random
import sys
import numpy
# Saving the reference of the standard output
original_stdout = sys.stdout
header = {}
link = '/search?query='
http = requests.session()
fileName = '/output.txt'
hasCompleted = 0
number = '11'
def search(target):
global targetList
global link
global header
global http
resp = http.request('GET',link+target,headers=header)
match = re.findall('"key":"(.+?)","',resp.text)
if len(match) == 10:
return False
if len(match) == 0:
return " "
elif len(match) < 10:
return resp.text
def treeSearch():
global fileName
global number
global hasCompleted
if hasCompleted == 1:
new = (int(int(number, 36) / 36) + 1)
new = numpy.base_repr(new, 36)
number = new
hasCompleted = 0
if hasCompleted == 0:
x = 0
new = int(int(number, 36)*36)
new = numpy.base_repr(new, 36)
number = new
while x < 37:
new = int(int(number, 36) + 1)
new = numpy.base_repr(new, 36)
number = new
result = search(number)
print(number)
if result:
with open(fileName, 'a+') as f:
sys.stdout = f
print(result)
sys.stdout = original_stdout
x = x + 1
else:
treeSearch()
temp = number
number = (int(int(number, 36) / 36) + 1) #maybe not + 1
print(number)
number = numpy.base_repr(number, 36)
print(number)
result = search(number)
if not result == " ":
new = int(int(number, 36)*36)
new = numpy.base_repr(new, 36)
number = new
hasCompleted = 1
treeSearch()
Here is my output:
111
1111
11111
111111
1111111
11111111
111111111
1111111111
11111111111
111111111111
1111111111111
11111111111111
111111111111111
1111111111111111
1111111111111112
1111111111111113
1111111111111114
1111111111111115
1111111111111116
1111111111111117
1111111111111118
1111111111111119
111111111111111A
111111111111111B
111111111111111C
111111111111111D
111111111111111E
111111111111111F
111111111111111G
111111111111111H
111111111111111I
111111111111111J
111111111111111K
111111111111111L
111111111111111M
111111111111111N
111111111111111O
111111111111111P
111111111111111Q
111111111111111R
111111111111111S
111111111111111T
111111111111111U
111111111111111V
111111111111111W
111111111111111X
111111111111111Y
111111111111111Z
1111111111111120
1111111111111121
6316397706306666889217
11111111110QR5T
11111111110QR5U
11111111110QR5V
11111111110QR5W
11111111110QR5X
11111111110QR5Y
11111111110QR5Z
11111111110QR60
11111111110QR61
11111111110QR62
11111111110QR63
11111111110QR64
11111111110QR65
11111111110QR66
11111111110QR67
11111111110QR68
11111111110QR69
11111111110QR6A
11111111110QR6B
11111111110QR6C
11111111110QR6D
11111111110QR6E
11111111110QR6F
11111111110QR6G
11111111110QR6H
11111111110QR6I
11111111110QR6J
11111111110QR6K
11111111110QR6L
11111111110QR6M
11111111110QR6N
11111111110QR6O
11111111110QR6P
11111111110QR6Q
11111111110QR6R
11111111110QR6S
11111111110QR6T
11111111110QR6U
175455491841851850753
11111111110L4X
11111111110L4Y
11111111110L4Z
11111111110L50
11111111110L51
11111111110L52
11111111110L53
11111111110L54
11111111110L55
11111111110L56
11111111110L57
11111111110L58
11111111110L59
11111111110L5A
11111111110L5B
11111111110L5C
11111111110L5D
11111111110L5E
11111111110L5F
11111111110L5G
11111111110L5H
11111111110L5I
11111111110L5J
11111111110L5K
11111111110L5L
11111111110L5M
11111111110L5N
11111111110L5O
11111111110L5P
11111111110L5Q
11111111110L5R
11111111110L5S
11111111110L5T
11111111110L5U
11111111110L5V
11111111110L5W
11111111110L5X
11111111110L5Y
4873763662273662977
11111111110XT
11111111110XU
11111111110XV
11111111110XW
11111111110XX
11111111110XY
11111111110XZ
11111111110Y0
11111111110Y1
11111111110Y2
11111111110Y3
11111111110Y4
11111111110Y5
11111111110Y6
11111111110Y7
11111111110Y8
11111111110Y9
11111111110YA
11111111110YB
11111111110YC
11111111110YD
11111111110YE
11111111110YF
11111111110YG
11111111110YH
11111111110YI
11111111110YJ
11111111110YK
11111111110YL
11111111110YM
11111111110YN
11111111110YO
11111111110YP
11111111110YQ
11111111110YR
11111111110YS
11111111110YT
11111111110YU
135382323952046193
11111111110X
11111111110Y
11111111110Z
111111111110
111111111111
111111111121
111111111122
111111111123
111111111124
111111111125
111111111126
111111111127
111111111128
111111111129
11111111112A
11111111112B
11111111112C
11111111112D
11111111112E
11111111112F
11111111112G
11111111112H
11111111112I
11111111112J
11111111112K
11111111112L
11111111112M
11111111112N
11111111112O
11111111112P
11111111112Q
11111111112R
11111111112S
11111111112T
11111111112U
11111111112V
11111111112W
11111111112X
11111111112Y
11111111112Z
111111111130
111111111131
3760620109779064
11111111114
111111111141
111111111142
111111111143
111111111144
111111111145
111111111146
111111111147
111111111148
111111111149
11111111114A
11111111114B
11111111114C
11111111114D
11111111114E
11111111114F
11111111114G
11111111114H
11111111114I
11111111114J
11111111114K
11111111114L
11111111114M
11111111114N
11111111114O
11111111114P
11111111114Q
11111111114R
11111111114S
11111111114T
11111111114U
11111111114V
11111111114W
11111111114X
11111111114Y
3760620109779066
11111111116
11111111117
11111111118
11111111119
1111111111A
1111111111B
1111111111C
1111111111D
1111111111E
1111111111F
1111111111G
1111111111H
1111111111I
1111111111J
1111111111K
1111111111L
1111111111M
1111111111N
1111111111O
1111111111P
1111111111Q
1111111111R
1111111111S
1111111111T
1111111111U
1111111111V
1111111111W
1111111111X
1111111111Y
1111111111Z
11111111120
11111111121
11111111122
11111111123
11111111124
11111111125
11111111126
11111111127
104461669716087
1111111113
1111111114
1111111115
1111111116
1111111117
1111111118
1111111119
111111111A
111111111B
111111111C
111111111D
111111111E
111111111F
111111111G
111111111H
111111111I
111111111J
111111111K
111111111L
111111111M
111111111N
111111111O
111111111P
111111111Q
111111111R
111111111S
111111111T
111111111U
111111111V
111111111W
111111111X
111111111Y
111111111Z
1111111120
1111111121
1111111122
1111111123
1111111124
2901713047671
111111113
111111114
111111115
111111116
111111117
111111118
111111119
11111111A
11111111B
11111111C
11111111D
11111111E
11111111F
11111111G
11111111H
11111111I
11111111J
11111111K
11111111L
11111111M
11111111N
11111111O
11111111P
11111111Q
11111111R
11111111S
11111111T
11111111U
11111111V
11111111W
11111111X
11111111Y
11111111Z
111111120
111111121
111111122
111111123
111111124
80603140215
11111113
11111114
11111115
11111116
11111117
11111118
11111119
1111111A
1111111B
1111111C
1111111D
1111111E
1111111F
1111111G
1111111H
1111111I
1111111J
1111111K
1111111L
1111111M
1111111N
1111111O
1111111P
1111111Q
1111111R
1111111S
1111111T
1111111U
1111111V
1111111W
1111111X
1111111Y
1111111Z
11111120
11111121
11111122
11111123
11111124
2238976119
1111113
11111131
11111132
11111133
11111134
11111135
11111136
11111137
11111138
11111139
1111113A
1111113B
1111113C
1111113D
1111113E
1111113F
1111113G
1111113H
1111113I
1111113J
1111113K
1111113L
1111113M
1111113N
1111113O
1111113P
1111113Q
1111113R
1111113S
1111113T
1111113U
1111113V
1111113W
1111113X
1111113Y
1111113Z
11111140
11111141
2238976121
1111115
11111151
11111152
11111153
11111154
11111155
11111156
11111157
11111158
11111159
1111115A
1111115B
1111115C
1111115D
1111115E
1111115F
1111115G
1111115H
1111115I
1111115J
1111115K
1111115L
1111115M
1111115N
1111115O
1111115P
1111115Q
1111115R
1111115S
1111115T
1111115U
1111115V
1111115W
1111115X
1111115Y
1111115Z
11111160
11111161
2238976123
1111117
11111171
11111172
11111173
11111174
11111175
11111176
11111177
11111178
11111179
1111117A
1111117B
1111117C
1111117D
1111117E
1111117F
1111117G
1111117H
1111117I
1111117J
1111117K
1111117L
1111117M
1111117N
1111117O
1111117P
1111117Q
1111117R
1111117S
1111117T
1111117U
1111117V
1111117W
1111117X
1111117Y
1111117Z
11111180
11111181
2238976125
1111119
111111A
111111B
111111C
111111D
111111E
111111F
111111G
111111H
111111I
111111J
111111K
111111L
111111M
111111N
111111O
111111P
111111Q
111111R
111111S
111111T
111111U
111111V
111111W
111111X
111111Y
111111Z
1111120
1111121
1111122
1111123
1111124
1111125
1111126
1111127
1111128
1111129
111112A
62193783
111113
1111131
1111132
1111133
1111134
1111135
1111136
1111137
1111138
1111139
111113A
111113B
111113C
111113D
111113E
111113F
111113G
111113H
111113I
111113J
111113K
111113L
111113M
111113N
111113O
111113P
111113Q
111113R
111113S
111113T
111113U
111113V
111113W
111113X
111113Y
111113Z
1111140
1111141
62193785
111115
My code traverses in only deeper once then comes out, I will keep working on my code however I am hoping to find an easier solution or possibly a library that can perform this style of search. Thanks!

how to solv: gPRC with python,"the JSON object must be str, bytes or bytearray, not RepeatedScalarContainer"

This is my first question on this site, and my English is not so good.So,if there is any misunderstanding, please tell me. Thank you.
I am trying use gPRC to send a message ,which is defined as :
message PrepareMsg{
message Data{
string node_id = 1;
string vote = 2;
}
Data data = 1;
repeated string signature = 2;
}
My code for Cilent just like this:
1 def pre_prepare(self, block):
2 request = grpc_pb2.PrePrepareMsg()
3 request.data.node_id = self.node_id
4 request.data.block.CopyFrom(block)
5 a = request.data.SerializeToString()
6 temp_sign = signing(self.signer, a)
7 for i in temp_sign:
8 request.signature.append(hex(i))
9 self_node = set()
10 self_node.add(p2p.SELF_IP_PORT)
11 print(self_node)
12 nodes = set(p2p.Node.get_nodes_list()) - self_node
13 print("print nodes in broadcast:")
14 print(nodes)
15 for i in nodes:
16 channel = grpc.insecure_channel(i)
17 stub = grpc_pb2_grpc.ConsensusStub(channel)
18 try:
19 response = stub.PrePrepare(request)
20 print(response.Result)
21 except Exception as e:
22 print("get except: %s" % str(e))
I find there is a error :"Exception calling application: the JSON object must be str, bytes or bytearray, not RepeatedScalarContainer"
I don't know why it happened.

Using text file data, classification and make other text file in python

using python, i want to seperated some data file.
file form is text file and there are no tabs only one space between inside data.
here is example file,
//test.txt
Class name age room fund.
13 A 25 B101 300
12 B 21 B102 200
9 C 22 B103 200
13 D 25 B102 100
20 E 23 B105 100
13 F 25 B103 300
11 G 25 B104 100
13 H 22 B101 300
I want to take only line containing specific data,
class : 13 , fund 300
,and save another text file.
if this code was worked, making text file is that
//new_test.txt
Class name age room fund.
13 A 25 B101 300
13 F 25 B103 300
13 H 22 B101 300
thanks.
Hk

This should do.
with open('new_test.txt','w') as new_file:
with open('test.txt') as file:
print(file.readline(),end='',file=new_file)
for line in file:
arr=line.strip().split()
if arr[0]=='13' and arr[-1]=='300':
print(line,end='',file=new_file)
However, you should include your code when asking a question. It ensures that the purpose of this site is served.

If you want to filter your data:
def filter_data(src_file, dest_file, filters):
data = []
with open(src_file) as read_file:
header = [h.lower().strip('.') for h in read_file.readline().split()]
for line in read_file:
values = line.split()
row = dict(zip(header, values))
data.append(row)
for k, v in filters.items():
if data and row.get(k, None) != v:
data.pop()
break
with open(dest_file, 'w') as write_file:
write_file.write(' '.join(header) + '\n')
for row in data:
write_file.write(' '.join(row.values()) + '\n')
my_filters = {
"class": "13",
"fund": "300"
}
filter_data(src_file='test.txt', dest_file='new_test.txt', filters=my_filters)

write from a specific column

In my piece of code, I have,
out.write(datGroup)
out.write(' '*20+": Space Group\n")
out.write(datLatx +" ")
out.write(datLaty +" ")
out.write(datLatz +" ")
out.write('/'),
out.write(' '*20+": a,b,c,alpha,beta,gamma\n")
which gives the output as:
189 : Space Group
10 11 12 / : a,b,c,alpha,beta,gamma
which is correct for what I have done, but not what I have want.
I want the ": Spacegroup" and ": a,b,c.." to start from the same column, irresepective of where the data of its row ended, like:
189 : Space Group
10 11 12 / : a,b,c,alpha,beta,gamma
Can I do that in python3?
EDIT after jonrsharpe's example
So I have tried:
def write_row(out, data, header, col=20):
out.write("{0}: {1}".format(" ".join(map(str, data)).ljust(col), header))
def print_data(self, BSub):
.....
with open("Init", mode='w') as out:
write_row(out, (datGroup,), "Space Group")
which is giving error:
$ python3 geninit.py
Traceback (most recent call last):
File "geninit.py", line 109, in print_data
write_row(out, (datGroup,), "Space Group")
NameError: global name 'write_row' is not defined

I would create a function to do this, for example:
def print_row(data, header, col=20):
print("{0}: {1}".format(" ".join(map(str, data)).ljust(col), header))
Then call it for each case:
print_row((datGroup,), "Space Group")
print_row((datLatx, datLaty, datLatz), "a,b,c,alpha,beta,gamma")
This gives, for example
>>> datGroup = 189
>>> datLatx, datLaty, datLatz = 11, 12, 13
>>> print_row((datGroup,), "Space Group")
189 : Space Group
>>> print_row((datLatx, datLaty, datLatz, "/"), "a,b,c,alpha,beta,gamma")
11 12 13 / : a,b,c,alpha,beta,gamma
This uses:
map(str, ...) to convert all of the data into strings;
" ".join(...) to put spaces between the data items;
str.ljust to pad the left "column" with spaces, keeping it the same width; and
str.format to put the padded left column and the header around the ": ".
You can then adapt this to write to your output file (which you could pass as an argument) rather than print the data, e.g.:
def write_row(out, data, header, col=20):
out.write(...)
and call
write_row(out, (datGroup,), "Space Group")

Getting error: AttributeError: class <CLASS NAME> has no attribute '<METHOD NAME>'

Here is the code:
1 #!/usr/bin/env python
2
3 import re, os, sys, jira, subprocess
4
5 class Check_jira:
6
7 def verify_commit_text(self, tags):
8 for line in tags:
9 if re.match('^NO-TIK',line):
10 return True
11 elif re.match('^NO-REVIEW', line):
12 return True
13 elif re.match(r'[a-zA-Z]+-\d+', line):
14 # Validate the JIRA ID
15 m = re.search("([a-zA-Z]+-\d+)",line)
16 if m:
17 my_args = m.group(1)
18 result = Check_jira.CheckForJiraIssueRecord(my_args)
19 if result == False:
20 util.warn("%s does not exist"%my_args)
21 else:
22 return True
23 return True
24 else:
25 return False
26 if __name__ == '__main__':
27 p = Check_jira()
28 commit_text_verified = p.verify_commit_text(os.popen('hg tip --template "{desc}"'))
29
30 if (commit_text_verified):
31 sys.exit(0)
32 else:
33 print >> sys.stderr, ('[obey the rules!]')
34 sys.exit(1);
35 def CheckForJiraIssueRecord(object):
36
37 sys.stdout = os.devnull
38 sys.stderr = os.devnull
39
40
41 try:
42 com = jira.Commands()
43 logger = jira.setupLogging()
44 jira_env = {'home':os.environ['HOME']}
45 command_cat= "cat"
46 command_logout= "logout"
47 #my_args = ["QA-656"]
48 server = "http://jira.myserver.com:8080/rpc/soap/jirasoapservice-v2?wsdl"
49 except Exception, e:
50 sys.exit('config error')
51
52 class Options:
53 pass
54 options = Options()
55
56 options.user = 'user'
57 options.password = 'password'
58
59 try:
60
61 jira.soap = jira.Client(server)
62 jira.start_login(options, jira_env, command_cat, com, logger)
63 issue = com.run(command_cat, logger, jira_env, my_args)
64 except Exception, e:
65 print sys.exit('data error')
so maybe:
1. if name == 'main': shoudl be at the bottom ?
2. So, i have 2 classes (Check_jira) and (Options)
3. Check_jira has 2 functions verify_commit_text() and CheckForJiraIssueRecord()
4. I pass object as an argument to CheckForJiraIssueRecord since i am passing my_args to it , on its usage.
5. Not sure how to call one function from another function in the same class
6. Error i am getting is :
Traceback (most recent call last):
File "/home/qa/hook-test/.hg/check_jira.py", line 31, in
commit_text_verified = p.verify_commit_text(os.popen('hg tip --template "{desc}"'))
File "/home/qa/hook-test/.hg/check_jira.py", line 21, in verify_commit_text
result = Check_jira.CheckForJiraIssueRecord(my_args)
AttributeError: class Check_jira has no attribute 'CheckForJiraIssueRecord'
transaction abort!
rollback completed
abort: pretxncommit.jira hook exited with status 1

class Check_jira ends on line 25 and has only one method. Then you have an if block, and CheckForJiraIssueRecord is just a function defined in this block (that is, the function is defined if __name__ == '__main__'.
Just put the if block outside after the whole class definition.