Develop Reference

How to extract required information using GPT-3 API

I tried the steps mentioned in this article.
https://matthewbilyeu.com/blog/2022-09-01/responding-to-recruiter-emails-with-gpt-3
There is a screenshot that says: Here's an example from the OpenAI Playground.
I typed all the text in "playground" but do not get similar response as shown in that image. I expected similar text like {"name":"William", "company":"BillCheese"} I am not sure how to configure the parameters in openAI web interface.
Update:
I used this code:
import json
import re, textwrap
import openai
openai.api_key = 'xxx'
prompt = f"""
Hi Matt! This is Steve Jobs with Inforation Edge Limited ! I'm interested in having you join our team here.
"""
completion = openai.Completion.create(
model="text-davinci-002",
prompt=textwrap.dedent(prompt),
max_tokens=20,
temperature=0,
)
try:
json_str_response = completion.choices[0].text
json_str_response_clean = re.search(r".*(\{.*\})", json_str_response).groups()[0]
print (json.loads(json_str_response_clean))
except (AttributeError, json.decoder.JSONDecodeError) as exception:
print("Could not decode completion response from OpenAI:")
print(completion)
raise exception
and got this error:
Could not decode completion response from OpenAI:
AttributeError: 'NoneType' object has no attribute 'groups'

You're running into this problem: Regex: AttributeError: 'NoneType' object has no attribute 'groups'
Take a look at this line:
json_str_response_clean = re.search(r".*(\{.*\})", json_str_response).groups()[0]
The regex can't find anything matching the pattern, so it returns None. None does not have .groups() so you get an error. I don't have enough details to go much further, but the link above might get you there.

I don't know why both the questioner as well as one reply above me are using RegEx. According to the OpenAI documentation, a Completion will return a JSON object.
No need to catch specific things complexly - just load the return into a dictionary and access the fields you need:
import json
# ...
# Instead of the try ... except block, just load it into a dictionary.
response = json.loads(completion.choices[0].text)
# Access whatever field you need
response["..."]

this worked for me:
question = "Write a python function to detect anomlies in a given time series"
response = openai.Completion.create(
model="text-davinci-003",
prompt=question,
temperature=0.9,
max_tokens=150,
top_p=1,
frequency_penalty=0.0,
presence_penalty=0.6,
stop=[" Human:", " AI:"]
)
print(response)
print("==========Python Code=========")
print(response["choices"][0]["text"])

Cannot (always) fetch an attribute in an object even though it exists

I'm currently developing locally an Azure function that communicates with Microsoft Sentinel, in order to fetch the alert rules from it, and more specifically their respective querys :
credentials = AzureCliCredential()
alert_rules_operations = SecurityInsights(credentials, SUBSCRIPTION_ID).alert_rules
list_alert_rules = alert_rules_operations.list(resource_group_name=os.getenv('RESOURCE_GROUP_NAME'), workspace_name=os.getenv('WORKSPACE_NAME'))
The issue is that when I'm looping over list_alert_rules, and try to see each rule's query, I get an error:
Exception: AttributeError: 'FusionAlertRule' object has no attribute 'query'.
Yet, when I check their type via the type() function:
list_alert_rules = alert_rules_operations.list(resource_group_name=os.getenv(
'RESOURCE_GROUP_NAME'), workspace_name=os.getenv('WORKSPACE_NAME'))
for rule in list_alert_rules:
print(type(rule))
##console: <class 'azure.mgmt.securityinsight.models._models_py3.ScheduledAlertRule'>
The weirder issue is that this error appears only when you don't print the attribute. Let me show you:
Print:
for rule in list_alert_rules:
query = rule.query
print('query', query)
##console: query YAY I GET WHAT I WANT
No print:
for rule in list_alert_rules:
query = rule.query
...
##console: Exception: AttributeError: 'FusionAlertRule' object has no attribute 'query'.
I posted the issue on the GitHub repo, but I'm not sure whether it's a package bug or a runtime issue. Has anyone ran into this kind of problems?
BTW I'm running Python 3.10.8
TIA!
EDIT:
I've tried using a map function, same issue:
def format_list(rule):
query = rule.query
# print('query', query)
# query = query.split('\n')
# query = list(filter(lambda line: "//" not in line, query))
# query = '\n'.join(query)
return rule
def main(mytimer: func.TimerRequest) -> None:
# results = fetch_missing_data()
credentials = AzureCliCredential()
alert_rules_operations = SecurityInsights(
credentials, SUBSCRIPTION_ID).alert_rules
list_alert_rules = alert_rules_operations.list(resource_group_name=os.getenv(
'RESOURCE_GROUP_NAME'), workspace_name=os.getenv('WORKSPACE_NAME'))
list_alert_rules = list(map(format_list, list_alert_rules))

I have tried with same as you used After I changed like below; I get the valid response.
# Management Plane - Alert Rules
alertRules = mgmt_client.alert_rules.list_by_resource_group('<ResourceGroup>')
for rule in alertRules:
# Try this
test.query = rule.query //Get the result
#print(rule)
if mytimer.past_due:
logging.info('The timer is past due!')
Instead of this
for rule in list_alert_rules:
query = rule.query
Try below
for rule in list_alert_rules:
# Try this
test.query = rule.query

Sorry for the late answer as I've been under tons of work these last few days.
Python has an excellent method called hasattr that checks if the object contains a specific key.
I've used it in the following way:
for rule in rules:
if hasattr(rule, 'query'):
...
The reason behind using this is because the method returns object of different classes, however inherited from the one same mother class.
Hope this helps.

Listing IBM Cloud Resources using ResourceControllerV2 and pagination issues

I'm using the Python ibm-cloud-sdk in an attempt to iterate all resources in a particular IBM Cloud account. My trouble has been that pagination doesn't appear to "work for me". When I pass in the "next_url" I still get the same list coming back from the call.
Here is my test code. I successfully print many of my COS instances, but I only seem to be able to print the first page....maybe I've been looking at this too long and just missed something obvious...anyone have any clue why I can't retrieve the next page?
try:
####### authenticate and set the service url
auth = IAMAuthenticator(RESOURCE_CONTROLLER_APIKEY)
service = ResourceControllerV2(authenticator=auth)
service.set_service_url(RESOURCE_CONTROLLER_URL)
####### Retrieve the resource instance listing
r = service.list_resource_instances().get_result()
####### get the row count and resources list
rows_count = r['rows_count']
resources = r['resources']
while rows_count > 0:
print('Number of rows_count {}'.format(rows_count))
next_url = r['next_url']
for i, resource in enumerate(resources):
type = resource['id'].split(':')[4]
if type == 'cloud-object-storage':
instance_name = resource['name']
instance_id = resource['guid']
crn = resource['crn']
print('Found instance id : name - {} : {}'.format(instance_id, instance_name))
############### this is SUPPOSED to get the next page
r = service.list_resource_instances(start=next_url).get_result()
rows_count = r['rows_count']
resources = r['resources']
except Exception as e:
Error = 'Error : {}'.format(e)
print(Error)
exit(1)

From looking at the API documentation for listing resource instances, the value of next_url includes the URL path and the start parameter including its token for start.
To retrieve the next page, you would only need to pass in the parameter start with the token as value. IMHO this is not ideal.
I typically do not use the SDK, but a simply Python request. Then, I can use the endpoint (base) URI + next_url as full URI.
If you stick with the SDK, use urllib.parse to extract the query parameter. Not tested, but something like:
from urllib.parse import urlparse,parse_qs
o=urlparse(next_url)
q=parse_qs(o.query)
r = service.list_resource_instances(start=q['start'][0]).get_result()

Could you use the Search API for listing the resources in your account rather than the resource controller? The search index is set up for exactly that operation, whereas paginating results from the resource controller seems much more brute force.
https://cloud.ibm.com/apidocs/search#search

Why do I get an error from my API but only when I use some requests?

I am using an API from this site https://dev.whatismymmr.com, and I want to specifically request for the closestRank but I just get a KeyError: 'ranked.closestRank'. but I can get the entire ['ranked'] object (which contains the closestRank) but I just end up with a lot of information I don't need.
How can I end up with just the Closest rank?
My code
import requests
LeagueName = input ("Summoner name")
base = ("https://eune.whatismymmr.com/api/v1/summoner?name=")
Thething = base + LeagueName
print (Thething)
response = requests.get(Thething)
print(response.status_code)
MMR = response.json()
print (MMR['ranked.closestRank'])
The API command
<queue>.closestRank (the queue is the game mode, it can be normal or ranked)
you can use the summoner name babada27 for testing.

Hope this is what you are looking for -
change The last line to
print (MMR["ranked"]["closestRank"])

How do I handle different errors when working with an API?

I am working a project in Python were I am gathering rain data and in this case temperature data from the Netatmo weather stations (Its basically just a private weather station you can set up in you garden and it will collect rain data, temperature, wind etc.)
When using the patatmo API you need a user with credential, a client. This client then has 500 requests pr. Hour which can be used on different requests, among these are the client.GetPublicdata request and the client.Getmeassure request. The Getmeassure request requires a station id and a module id, which I get from the the Getpuclicdat request. If I run out of requests I will catch that error using the except ApiResponseError: and I will then change the client credentials since I am not alone in this project and have credentials from two other people. My issues are:
If a station or module ID is not found using the Getmeassure request it will return a different type of ApiResponseError, which in my current code also is caught in the previously mentioned except, and this results in an endless loop where the code just changes credential all the time.
The code looks like this:
from patatmo.api.errors import ApiResponseError
...
...
...
a_test1 = 0
while a_test1 == 0:
try:
Outdoor_data = client.Getmeasure(
device_id = stations_id ,
module_id = modul_ID ,
type = typ ,
real_time = True ,
date_begin = Start ,
date_end = End
)
time.sleep(p)
a_test1 = 1
except ApiResponseError:
credentials = cred_dict[next(ns)]
client = patatmo.api.client.NetatmoClient()
client.authentication.credentials = credentials
client.authentication.tmpfile = 'temp_auth.json'
print('Changeing credentials')
credError = credError + 1
if credError > 4:
time.sleep(600)
credError = 0
pass
except:
print('Request Error')
time.sleep(p)
The documentation for the error.py script, that was made by someone else, looks like this:
class ApiResponseError(BaseException):
pass
class InvalidCredentialsError(ApiResponseError):
pass
class InvalidApiInputError(BaseException):
pass
class InvalidRegionError(InvalidApiInputError):
def __init__(self):
message = \
("'region' required keys: "
"lat_ne [-85;85], lat_sw [-85;85], "
"lon_ne [-180;180] and lon_sw [-180;180] "
"with lat_ne > lat_sw and lon_ne > lon_sw")
super().__init__(message)
class InvalidRequiredDataError(InvalidApiInputError):
def __init__(self):
message = "'required_data' must be None or in {}".format(
GETPUBLICDATA_ALLOWED_REQUIRED_DATA)
super().__init__(message)
class InvalidApiRequestInputError(BaseException):
pass
class InvalidPayloadError(InvalidApiRequestInputError):
pass
API_ERRORS = {
"invalid_client": InvalidCredentialsError("wrong credentials"),
"invalid_request": ApiResponseError("invalid request"),
"invalid_grant": ApiResponseError("invalid grant"),
"Device not found": ApiResponseError("Device not found - Check Device ID "
"and permissions")
}
What I want to do is catch the error depending what type of error I get, and I just doesn't seem to have any luck doing so

Which of those exceptions subclasses do you get on request overrun? Use only that one to drive the auth swap. If it’s not a particular one but is shared with other bad events, you will need to examine the exception’s variables. Also you will need to figure out which exception to work around - bad station id might mean someone is offline so ignore and try later. Vs logic flaws in your program, abend on those, fix, retry.
Exception handling is on a first-catch, first-handled basis. Your most specific classes have to do the “except” first to match first, else a generic one would grab it and handle it. Watch your APIs exception hierarchy carefully!
maxexc = 1000
countexc = 1
while ...
# slow your loop
time.sleep(p)
try:
... what you normally do
# dont use the too generic ApiResponseError here yet
except requestoverrunexception as e:
... swap credendentials
countexc+=1
except regionexception as e:
... ignore this region for a while
countexc += 1
# whoops abend and fix
except ApiResponseError as e:
print vars(e)
raise
except Exception as e:
print(e)
raise
I would bail after too many exceptions, thats what countexc is about.
Also 500 requests an hour seems generous. Don’t try to fudge that unless you have a real need. Some providers may even have watchdogs and get rid of you if you abuse them.