Develop Reference

How do I merge and sort JSON objects using its counts?

I got two json objects that I need to combine together based on ID and do count and sort operations on it.
Here is the first object comments:
[
{
"userId": 1,
"id": 1,
"title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
"body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
},
{
"userId": 1,
"id": 2,
"title": "qui est esse",
"body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla"
},
{
"userId": 1,
"id": 3,
"title": "ea molestias quasi exercitationem repellat qui ipsa sit aut",
"body": "et iusto sed quo iure\nvoluptatem occaecati omnis eligendi aut ad\nvoluptatem doloribus vel accusantium quis pariatur\nmolestiae porro eius odio et labore et velit aut"
},
{
"userId": 1,
"id": 4,
"title": "eum et est occaecati",
"body": "ullam et saepe reiciendis voluptatem adipisci\nsit amet autem assumenda provident rerum culpa\nquis hic commodi nesciunt rem tenetur doloremque ipsam iure\nquis sunt voluptatem rerum illo velit"
},
]
This is second json object:
[
{
"postId": 1,
"id": 1,
"name": "id labore ex et quam laborum",
"email": "Eliseo#gardner.biz",
"body": "laudantium enim quasi est quidem magnam voluptate ipsam eos\ntempora quo necessitatibus\ndolor quam autem quasi\nreiciendis et nam sapiente accusantium"
},
{
"postId": 1,
"id": 2,
"name": "quo vero reiciendis velit similique earum",
"email": "Jayne_Kuhic#sydney.com",
"body": "est natus enim nihil est dolore omnis voluptatem numquam\net omnis occaecati quod ullam at\nvoluptatem error expedita pariatur\nnihil sint nostrum voluptatem reiciendis et"
},
{
"postId": 1,
"id": 3,
"name": "odio adipisci rerum aut animi",
"email": "Nikita#garfield.biz",
"body": "quia molestiae reprehenderit quasi aspernatur\naut expedita occaecati aliquam eveniet laudantium\nomnis quibusdam delectus saepe quia accusamus maiores nam est\ncum et ducimus et vero voluptates excepturi deleniti ratione"
},
{
"postId": 1,
"id": 4,
"name": "alias odio sit",
"email": "Lew#alysha.tv",
"body": "non et atque\noccaecati deserunt quas accusantium unde odit nobis qui voluptatem\nquia voluptas consequuntur itaque dolor\net qui rerum deleniti ut occaecati"
},
{
"postId": 2,
"id": 5,
"name": "et fugit eligendi deleniti quidem qui sint nihil autem",
"email": "Presley.Mueller#myrl.com",
"body": "doloribus at sed quis culpa deserunt consectetur qui praesentium\naccusamus fugiat dicta\nvoluptatem rerum ut voluptate autem\nvoluptatem repellendus aspernatur dolorem in"
},
{
"postId": 2,
"id": 6,
"name": "repellat consequatur praesentium vel minus molestias voluptatum",
"email": "Dallas#ole.me",
"body": "maiores sed dolores similique labore et inventore et\nquasi temporibus esse sunt id et\neos voluptatem aliquam\naliquid ratione corporis molestiae mollitia quia et magnam dolor"
},
]
Object one is basically posts with poster details and object two is comments with commenter details.
So expected that object one has one to many relationships with second object. For example one post has many comments. This relationship is based on id in object one is postId in object two. The ultimate objective is to count and sort post by number of comments.
I attempt the problem with simple for loops and creating new json object, I managed to combine them together, but I dont know how to count and sort them properly.
in the views:
for i in posts:
if (id==postId):
newobj.append(objtwo[i])
count+=1
else:
newobj.append(count)
count=0
Normally I use django ORM to sort this but I dont have access to the database and model of the table. How to count and sort the new object so it can return list of posts with most comments counts and descend to lower comments counts?

Assuming your posts and comments data structures are lists, you can use python's defaultdict to count the comments. Then, use posts.sort(key=...) to sort your posts based on the collected counts using the key parameter. Altogether, it could like like this:
import json
from collections import defaultdict
posts = [ ... ]
comments = [ ... ]
# data structure to count the to comments
# automatically initializes to 0
comments_per_post = defaultdict(int)
# iterate through the comments to increase the count for the posts
for comment in comments:
comments_per_post[comment['postId']] += 1
# add comment count to post
for post in posts:
post['number_of_comments'] = comments_per_post[post['id']]
# sort the posts based on the counts collected
posts.sort(key=lambda post: post['number_of_comments'], reverse=True)
# print them to verify
# number of comments per Post will be in the `number_of_comments` key on the post dict.
print(json.dumps(posts, indent=2))
Note: this sorts the posts array in-place. If you don't want this, you can use sorted_posts = sorted(posts, key=... instead.

My answer is very similar to Byted's answer.
I would use Counter from the built-in collections to count the number of postIds in the second object.
Then sort the first object by using these counts from the previous step as a sorting key. Counter object returns 0 if a key is not present in it, so just use it as a lookup as a sorting key. The negative sign ensures a descending order (because sorted() sorts in ascending order by default).
import json
from collections import Counter
# count the comments
counts = Counter([d['postId'] for d in objtwo])
# add the counts to each post
for d in objone:
d["number of comments"] = counts[d['id']]
# sort posts by number of comments in descending order
objone.sort(key=lambda x: -x['number of comments'])
# convert to json
json.dumps(objone, indent=4)
Intermediate output for this input:
print(counts)
# Counter({1: 4, 2: 2})

Convert nested JSON to HTML Unordered List with Python

I did a fair amount of google searching but didn't find a suitable answer for someone who would like take arbitrarily nested JSON, for example from an API response, and display it as an unordered list using HTML using python.
Ultimately, the issue was solved with a fairly short recursive function.
Here's an example of the type of input I was dealing with:
{
"_id": "61dc4e9130473a8465a11cd0",
"index": 0,
"guid": "a2a7e550-8bf4-4be5-b0e0-2b124a2ca7e1",
"isActive": false,
"balance": "$1,011.28",
"picture": "http://placehold.it/32x32",
"age": 25,
"eyeColor": "green",
"name": "Monique Dickerson",
"gender": "female",
"company": "AQUASURE",
"about": "Dolore dolor excepteur tempor excepteur nulla occaecat Lorem dolor cillum sint velit. Minim labore irure ea anim duis in enim laboris. Aute amet ut sunt ea. Do irure sint commodo ea id. Amet dolore culpa anim irure ipsum est labore nostrud irure.\r\n",
"registered": "2015-12-15T11:10:14 +05:00",
"latitude": -63.913924,
"longitude": -21.554531,
"tags": [
"eiusmod",
"dolore",
"pariatur",
"in",
"ipsum",
"Lorem",
"adipisicing"
]
}
Here was the desired output:
_id: 61dc4e9130473a8465a11cd0
index: 0
guid: a2a7e550-8bf4-4be5-b0e0-2b124a2ca7e1
isActive: False
balance: $1,011.28
picture: http://placehold.it/32x32
age: 25
eyeColor: green
name: Monique Dickerson
gender: female
company: AQUASURE
about: Dolore dolor excepteur tempor excepteur nulla occaecat Lorem dolor cillum sint velit. Minim labore irure ea anim duis in enim laboris. Aute amet ut sunt ea. Do irure sint commodo ea id. Amet dolore culpa anim irure ipsum est labore nostrud irure.\r\n
registered: 2015-12-15T11:10:14 +05:00
latitude: -63.913924
longitude: -21.554531
tags:
eiusmod
dolore
pariatur
in
ipsum
Lorem
adipisicing

I was able to solve the problem successfully with this bit of code:
items = []
def render(json_data, v=""):
if isinstance(json_data, dict):
items.append(f"<ul>")
for k2, v2 in json_data.items():
render(k2 ,v2) # <---If we have a dict, apply function again
items.append(f"</ul>")
elif isinstance(v, dict):
items.append(f"<li>{json_data}: <ul>")
for k2, v2 in v.items():
render(k2 ,v2) # <---If we have a dict, apply function again
items.append("</ul></li>")
elif isinstance(v, list):
items.append(f"<li>{json_data}:<ul>")
for i in v:
if isinstance(i, str):
items.append(f"<li>{i}</li>")
elif isinstance(i, dict):
render(i, v)
items.append("</ul></li>")
else:
items.append(f"<li>{json_data}: {v}</li>")
render(data)
html_str = "".join(items)
html_str

Python user paragraph input

How can I copy and paste this as user input in Python?
"Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet,
consectetur, adipisci velit, sed quia non numquam eius modi tempora
incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut
enim ad minima veniam, quis nostrum exercitationem ullam corporis
suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur?
Quis autem vel eum iure reprehenderit qui in ea voluptate velit
esse quam nihil molestiae consequatur, vel illum qui dolorem eum
fugiat quo voluptas nulla pariatur?"
Edit:
input() doesn't work, I am getting this in console:
https://pastebin.com/raw/Pc55u0KX

After inserting multiline text, click enter with empty input and it will insert and split all lines into list.
def multiline_input(sentinel=''):
for inp in iter(input, sentinel):
yield inp.split()
lis = list(multiline_input())
print(lis)
Source: Python: Multiline input converted into a list

Counting how many different words have exactly the letter 'o' twice

I'm new at ApacheSpark. Currently, I'm performing some statistical analysis for a text. I start reading the text and storing it in a variable as follows:
loremRDD = sc.textFile(fileName, 8).map(removePunctuation).filter(lambda x: len(x)>0)
#First 10 lines
loremRDD.take(10)
The result is a PythonRDD[66] at RDD at PythonRDD.scala:53 variable. Below there are the first 10 lines of the text:
['aut minima deleniti et autem minus illo esse dolores eligendi corrupti dolore minima nostrum eos nobis nam nihil aspernatur nam ut quae sint laborum ut dolores error possimus aperiam consequatur',
'pariatur sed quo non itaque qui pariatur saepe ad quis consequatur nihil iste molestias et eos ut expedita vel reiciendis dolorem enim doloribus quam architecto aperiam',
'sed repudiandae pariatur similique est aut sequi animi in aperiam enim ipsa enim dolorem inventore aut quo odio in consequatur et',
'aspernatur ad esse et aliquid itaque dolores rerum quia commodi explicabo non magnam nostrum consectetur non sint eum nulla et aut quis doloribus itaque nulla molestiae quis est est quo facilis incidunt a ipsa in itaque sed aut nobis facere dignissimos atque unde cum ea vero',
'tenetur vel quod voluptatum laudantium dolores neque aut est modi qui aperiam itaque aperiam quae ratione doloremque aut delectus quas qui',
'qui placeat vel ipsam praesentium sint recusandae dicta minus praesentium omnis sequi a sed veritatis porro ab et officia esse commodi pariatur sequi cumque',
'mollitia facilis amet deleniti quia laborum commodi et molestias maxime quia dignissimos inventore neque libero deleniti ad quo corrupti numquam quis accusantium',
'architecto harum sunt et enim nisi commodi et id reprehenderit illum molestias illo facilis fuga eum illum quasi fugit qui',
'modi voluptatem quia et saepe inventore sed quo ea debitis explicabo vel perferendis commodi exercitationem sequi eum dolor cupiditate ab molestiae nemo ullam neque hic ipsa cupiditate dolor molestiae neque nam nobis nihil mollitia unde',
'voluptates quod in ipsum dicta fuga voluptatibus sint consequatur quod optio molestias nostrum repellendus consequatur aliquam fugiat provident omnis minus est quisquam exercitationem eum voluptas fugit quae eveniet perspiciatis assumenda maxime']
I need to know how many different words have twice the letter 'o'. For example, the word dolorem has twice the letter 'o'.
At the moment, I´ve created distintWordsRDD which stores all differents words contained in the text as follows:
loremWordsRDD = loremRDD.flatMap(lambda x: x.split(' '))
distintWordsMapRDD = loremWordsRDD.map(lambda word: (word,1)).reduceByKey(lambda a,b:a+b)
distintWordsRDD=distintWordsMapRDD.keys().distinct()
# Showing 8 first words
print(distintWordsRDD.take(8))
The result of the 8 first words is:
['tempora', 'sapiente', 'vitae', 'nisi', 'quidem', 'consectetur', 'perferendis', 'debitis']
My problem is that I don´t know how to retrieve from distintWordsRDD a list with the words that have two 'o'.

The following should work:
your_text=''.join(your_original_list_of_texts)
result=[i for i in your_text.split() if i.count('o')==2]
print(result)
['dolores', 'dolore', 'dolores', 'dolorem', 'doloribus', 'dolorem', 'odio']
However the text that you provided is split to many subtexts ('sometext1', 'sometext2', 'sometext3, etc) and it needs some additional work so that it will come to a simple text format ('alltext')
If you provide exact details of your input text, i will adjust the code so that it will work properly with the input without additional manual work

If you only have one string sentence:
results = set(word for word in sentence.split() if word.count('o') == 2)
If you have a list sentences of strings (which is what you show in your question):
results = set(
word
for sentence in sentences
for word in sentence.split()
if word.count('o') == 2
)
I'm using set to unify the results.
Output for the list of sentences in your example:
{'odio', 'dolorem', 'dolore', 'doloremque', 'dolor', 'doloribus', 'optio', 'commodi', 'porro', 'dolores'}
If you need a list then just convert the set in a list: results = list(results)).

I managed to solve this problem doing the following:
results = distintWordsRDD.filter(lambda word: word.count('o')==2)
print (results.collect())
print(results.count())
Result:
['porro', 'odio', 'laboriosam', 'doloremque', 'doloribus', 'dolores', 'dolor', 'corporis', 'commodi', 'optio', 'dolorum', 'dolore', 'dolorem']
13

How to print very long string without wrapping in Jupyter notebook?

Example:
print("Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo."
Outputs:
Sed ut perspiciatis unde omnis iste natus error sit voluptatem
accusantium doloremque laudantium, totam rem aperiam, eaque
ipsa quae ab illo inventore veritatis et quasi architecto
beatae vitae dicta sunt explicabo.
Desired output:
Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo.

Directly go through this page
https://github.com/jupyter/notebook/issues/106

Type in your cmd prompt or equivalent: jupyter --config-dir to get your jupyter settings location.
Create a folder nbconfig inside the settings location /.jupyter. Inside, create a file notebook.json with the following
{
"MarkdownCell": {
"cm_config": {
"lineWrapping": false
}
},
"CodeCell": {
"cm_config": {
"lineWrapping": false
}
}
}
Restart jupyter and reload, then try

You need HTML output that has some CSS applied. Here is the code that you may try:
import IPython.display as dp
long_txt = "Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo."
outp = dp.HTML("<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>" +long_txt+ "</span>")
Now you get outp as an HTML object. You can render it and get the long one-line text.
outp
The output will be:
Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo.
Hope it helps.