Develop Reference

Pass text to a Python script and return the result using R JSON

I have a string in R that I would like to pass to python in order to compute something and return the result back into R.
I have the following which "works" but not as I would like.
The below passes a string from R, to a Python file, uses openAI to collect the text data and then load it back into R.
library(reticulate)
computePythonFunction <- "
def print_openai_response():
import openai
openai.api_key = 'ej-powurjf___OpenAI_API_KEY___HGAJjswe' # you will need an API key
prompt = 'write me a poem about the sea'
response = openai.Completion.create(engine = 'text-davinci-003', prompt = prompt, max_tokens=1000)
#response['choices'][0]['text']
print(response)
"
py_run_string(computePythonFunction)
py$print_openai_response()
library("rjson")
fromJSON(as.character(py$print_openai_response()))
I would like to store the results in R objects - i.e. Here is one output from the python script.
{
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"text": "\n\nThe sea glitters like stars in the night \nWith hues, vibrant and bright\nThe waves flow gentle, serene, and divine \nLike the sun's most gentle shine\n\nAs the sea reaches, so wide, so vast \nAn adventure awaits, and a pleasure, not passed\nWhite sands, with seaweed green \nForms a kingdom of the sea\n\nConnecting different land and tide \nThe sea churns, dancing with the sun's pride\nAs a tempest forms, raging and wild \nThe sea turns, its colors so mild\n\nA raging storm, so wild and deep \nProtecting the creatures that no one can see \nThe sea is a living breathing soul \nA true and untouchable goal \n\nThe sea is a beauty that no one can describe \nAnd it's power, no one can deny \nAn ever-lasting bond, timeless and free \nThe love of the sea, is a love, to keep"
}
],
"created": 1670525403,
"id": "cmpl-6LGG3hDNzeTZ5VFbkyjwfkHH7rDkE",
"model": "text-davinci-003",
"object": "text_completion",
"usage": {
"completion_tokens": 210,
"prompt_tokens": 7,
"total_tokens": 217
}
}
I am interested in the text generated but I am also interested in the completion_tokens, promt_tokens and total_tokens.
I thought about save the Python code as a script, then pass the argument to it such as:
myPythin.py arg1.
How can I return the JSON output from the model to an R object? The only input which changes/varies in the python code is the prompt variable.

To get a dictionary as a table in markdown

So I have written the below code
paa_required_data = {
"Case age": paa_pending_since,
"PAA Case ID": all_paa_cases,
"PAA Owner": data[data["Status"] == "Pending Amazon Action"].Owner.to_list(),
}
mac_required_data = {
"Case age": mac_pending_since,
"Mac Case ID": all_mac_cases,
"MAC Owner": data[data["Status"] == "Merchant Action Completed"].Owner.to_list(),
}
paa_new_data = pandas.DataFrame(paa_required_data).sort_values(by=['Case age'], ascending=False).to_string(index=False)
mac_new_data = pandas.DataFrame(mac_required_data).sort_values(by=['Case age'], ascending=False).to_string(index=False)
chime_msg = f"/md Hello #Present \n" \
f"Good morning. I request you to kindly work on these PAA and MAC cases on priority and get them " \
f"resolved at the earliest.\n" \
f"### PAA Cases as of {datetime.date.today()}\n" \
f"{paa_new_data}\n" \
f"### MAC Cases as of {datetime.date.today()}\n" \
f"{mac_new_data}\n" \
All the values in the dictionary are lists. The chime_msg is the message sent to chime through webhooks. I want to use tables in markdown to get something like this
Case age
PAA Case ID
PAA Owner
value 1
value 2
Value 3
But when I try to use the .to_markdown() after sorting the dataframe, I get an error saying that 'str' object has no attribute 'to_markdown' . Also when I use .to_markdown without sorting, the output in chime is |---😐:--- (gets converted to an emoji)
Can anyone help me getting the table output shown above?

How can I navigate to the next page through Selenium in Python?

I am writing a script that scrapes information from a large travel agency. My code closely follows the tutorial at https://python.gotrained.com/selenium-scraping-booking-com/.
However, I would like to be able to navigate to the next page as I'm now limited to n_results = 25. Where do I add this in the code? I know that I can target the pagination button with driver.find_element_by_class_name('paging-next').click(), but I don't know where to incorporate it.
I have tried to put it in the for loop within the scrape_results function, which I have copied below. However, it doesn't seem to work.
def scrape_results(driver, n_results):
'''Returns the data from n_results amount of results.'''
accommodations_urls = list()
accommodations_data = list()
for accomodation_title in driver.find_elements_by_class_name('sr-hotel__title'):
accommodations_urls.append(accomodation_title.find_element_by_class_name(
'hotel_name_link').get_attribute('href'))
for url in range(0, n_results):
if url == n_results:
break
url_data = scrape_accommodation_data(driver, accommodations_urls[url])
accommodations_data.append(url_data)
return accommodations_data
EDIT
I have added some more code to clarify my input and output. Again, I mostly just used code from the GoTrained tutorial and added some code of my own. How I understand it: the scraper first collects all URLs and then scrapes the info of the individual pages one by one. I need to add the pagination loop in that first part – I think.
if __name__ == '__main__':
try:
driver = prepare_driver(domain)
fill_form(driver, 'Waterberg, South Africa') # my search argument
accommodations_data = scrape_results(driver, 25) # 25 is the maximum of results, higher makes the scraper crash due to the pagination problem
accommodations_data = json.dumps(accommodations_data, indent=4)
with open('booking_data.json', 'w') as f:
f.write(accommodations_data)
finally:
driver.quit()
Below is the JSON output for one search result.
[
{
"name": "Lodge Entabeni Safari Conservancy",
"score": "8.4",
"no_reviews": "41",
"location": "Vosdal Plaas, R520 Marken Road, 0510 Golders Green, South Africa",
"room_types": [
"Tented Chalet - Wildside Safari Camp with 1 game drive",
"Double or Twin Room - Hanglip Mountain Lodge with 1 game drive",
"Tented Family Room - Wildside Safari Camp with 1 game drive"
],
"room_prices": [
"\u20ac 480",
"\u20ac 214",
"\u20ac 650",
"\u20ac 290",
"\u20ac 693"
],
"popular_facilities": [
"1 swimming pool",
"Bar",
"Very Good Breakfast"
]
},
...
]

How to get fortnite stats in python

So i was trying to find something to code, and i decided to use python to get fortnite stats, i came across the fortnite_python library and it works, but it displays item codes for items in the shop when i want it to display the names. Anyone know how to convert them or just disply the name in the first place? This is my code.
​
fortnite = Fortnite('c954ed23-756d-4843-8f99-cfe850d2ed0c')
store = fortnite.store()
fortnite.store()
It outputs something like this
[<StoreItem 12511>,

To print out the attributes of a Python object you can use __dict__ e.g.
from fortnite_python import Fortnite
from json import dumps
fortnite = Fortnite('Your API Key')
# ninjas_account_id = fortnite.player('ninja')
# print(f'ninjas_account: {ninjas_account_id}') # ninjas_account: 4735ce91-3292-4caf-8a5b-17789b40f79c
store = fortnite.store()
example_store_item = store[0]
print(dumps(example_store_item.__dict__, indent=2))
Output:
{
"_data": {
"imageUrl": "https://trackercdn.com/legacycdn/fortnite/237112511_large.png",
"manifestId": 12511,
"name": "Dragacorn",
"rarity": "marvel",
"storeCategory": "BRSpecialFeatured",
"vBucks": 0
},
"id": 12511,
"image_url": "https://trackercdn.com/legacycdn/fortnite/237112511_large.png",
"name": "Dragacorn",
"rarity": "marvel",
"store_category": "BRSpecialFeatured",
"v_bucks": 0
}
So it looks like you want to use name attribute of StoreItem:
for store_item in store:
print(store_item.name)
Output:
Dragacorn
Hulk Smashers
Domino
Unstoppable Force
Scootin'
Captain America
Cable
Probability Dagger
Chimichanga!
Daywalker's Kata
Psi-blade
Snap
Psylocke
Psi-Rider
The Devil's Wings
Daredevil
Meaty Mallets
Silver Surfer
Dayflier
Silver Surfer's Surfboard
Ravenpool
Silver Surfer Pickaxe
Grand Salute
Cuddlepool
Blade
Daredevil's Billy Clubs
Mecha Team
Tricera Ops
Combo Cleaver
Mecha Team Leader
Dino
Triassic
Rex
Cap Kick
Skully
Gold Digger
Windmill Floss
Bold Stance
Jungle Scout

It seems that the library doesn't contain a function to get the names. Also this is what the class of a item from the store looks like:
class StoreItem(Domain):
"""Object containing store items attributes"""
and thats it.

Python 3.5: How to read a db of JSON objects

so I'm new to working with JSON and I'm trying to work with the openrecipe database from here. The db dump you get looks like this...
{ "_id" : { "$oid" : "5160756d96cc62079cc2db16" }, "name" : "Hot Roast Beef Sandwiches", "ingredients" : "12 whole Dinner Rolls Or Small Sandwich Buns (I Used Whole Wheat)\n1 pound Thinly Shaved Roast Beef Or Ham (or Both!)\n1 pound Cheese (Provolone, Swiss, Mozzarella, Even Cheez Whiz!)\n1/4 cup Mayonnaise\n3 Tablespoons Grated Onion (or 1 Tbsp Dried Onion Flakes))\n1 Tablespoon Poppy Seeds\n1 Tablespoon Spicy Mustard\n1 Tablespoon Horseradish Mayo Or Straight Prepared Horseradish\n Dash Of Worcestershire\n Optional Dressing Ingredients: Sriracha, Hot Sauce, Dried Onion Flakes Instead Of Fresh, Garlic Powder, Pepper, Etc.)", "url" : "http://thepioneerwoman.com/cooking/2013/03/hot-roast-beef-sandwiches/", "image" : "http://static.thepioneerwoman.com/cooking/files/2013/03/sandwiches.jpg", "ts" : { "$date" : 1365276013902 }, "cookTime" : "PT20M", "source" : "thepioneerwoman", "recipeYield" : "12", "datePublished" : "2013-03-13", "prepTime" : "PT20M", "description" : "When I was growing up, I participated in my Episcopal church's youth group, and I have lots of memories of weekly meetings wh..." }
{ "_id" : { "$oid" : "5160756f96cc6207a37ff777" }, "name" : "Morrocan Carrot and Chickpea Salad", "ingredients" : "Dressing:\n1 tablespoon cumin seeds\n1/3 cup / 80 ml extra virgin olive oil\n2 tablespoons fresh lemon juice\n1 tablespoon honey\n1/2 teaspoon fine sea salt, plus more to taste\n1/8 teaspoon cayenne pepper\n10 ounces carrots, shredded on a box grater or sliced whisper thin on a mandolin\n2 cups cooked chickpeas (or one 15- ounce can, drained and rinsed)\n2/3 cup / 100 g dried pluots, plums, or dates cut into chickpea-sized pieces\n1/3 cup / 30 g fresh mint, torn\nFor serving: lots of toasted almond slices, dried or fresh rose petals - all optional (but great additions!)", "url" : "http://www.101cookbooks.com/archives/moroccan-carrot-and-chickpea-salad-recipe.html", "image" : "http://www.101cookbooks.com/mt-static/images/food/moroccan_carrot_salad_recipe.jpg", "ts" : { "$date" : 1365276015332 }, "datePublished" : "2013-01-07", "source" : "101cookbooks", "prepTime" : "PT15M", "description" : "A beauty of a carrot salad - tricked out with chickpeas, chunks of dried pluots, sliced almonds, and a toasted cumin dressing. Thank you Diane Morgan." }
{ "_id" : { "$oid" : "5160757096cc62079cc2db17" }, "name" : "Mixed Berry Shortcake", "ingredients" : "Biscuits\n3 cups All-purpose Flour\n2 Tablespoons Baking Powder\n3 Tablespoons Sugar\n1/2 teaspoon Salt\n1-1/2 stick (3/4 Cup) Cold Butter, Cut Into Pieces\n1-1/4 cup Buttermilk\n1/2 teaspoon Almond Extract (optional)\n Berries\n2 pints Mixed Berries And/or Sliced Strawberries\n1/3 cup Sugar\n Zest And Juice Of 1 Small Orange\n SWEET YOGURT CREAM\n1 package (7 Ounces) Plain Greek Yogurt\n1 cup Cold Heavy Cream\n1/2 cup Sugar\n2 Tablespoons Brown Sugar", "url" : "http://thepioneerwoman.com/cooking/2013/03/mixed-berry-shortcake/", "image" : "http://static.thepioneerwoman.com/cooking/files/2013/03/shortcake.jpg", "ts" : { "$date" : 1365276016700 }, "cookTime" : "PT15M", "source" : "thepioneerwoman", "recipeYield" : "8", "datePublished" : "2013-03-18", "prepTime" : "PT15M", "description" : "It's Monday! It's a brand new week! The birds are chirping! The coffee's brewing! Everything has such hope and promise! A..." }
I tried the following code to read in the database
import json
f = r'<file_path>\recipeitems-latest.json'
with open(f) as dfile:
data = json.load(dfile)
print(data)
With this I received the following Traceback
Traceback (most recent call last):
File "C:/Users/<redacted>/Documents/<redacted>/project/test_json.py", line 7, in <module>
data = json.load(dfile)
File "C:\Users\<redacted>\AppData\Local\Continuum\Anaconda3\Lib\json\__init__.py", line 265, in load
return loads(fp.read(),
File "C:\Users\<redacted>\AppData\Local\Continuum\Anaconda3\Lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 101915: character maps to <undefined>
The only way I could find around this error was to only have one entry in the json file. Is the db formatted incorrectly or am I reading in the data wrong?
Thanks for any help!

The file is not a json array. Each line of the file is a json document, but the whole file is not in json format.
Read the file by lines, and use json.loads:
with open('some_file') as f:
for line in f:
doc = json.loads(line)
You may also need to pass the encoding parameter to open(). See here.