Develop Reference

Send json between Python and PHP

I have this code in Python (is interesterd poart of the code)
elements = [{'id': 1, 'name': 'Alex'}, {'id': 2, 'name': 'Jessica'}]
elements_json = json.dumps(elements)
requests.post('http://localhost:8000/api/add-element', data = {'json': elements_json})
Elements sometimes contain more elements.
In PHP (Laravel) I have this code:
public function Store(Request $request) {
$json = json_decode($request->json );
echo $json;
}
I want to store all elements but for test I want to print that but it not display me empty result. I test that on Postman. Where problem is?

How to return a specific fields using Pymongo and MongoDB?

I am trying to return just two fields from my MongoDB database and this query correct sends back my username and posts from the client:
db.users.find({'email_address': /^johnny/}, {'username': 1, 'user_posts': 1, '_id': 0});
I'm trying to switch this to PyMongo so I can query in Python and my query looks like this:
regx = Regex('^{0}'.format('johnny'))
query = {'postcode': regx}, {'username': 1, 'user_posts': 1, '_id': 0}
user_list = mycol.find(query).limit(5)
The bit where it's failing is here:
{'username': 1, 'user_posts': 1, '_id': 0}
As without this filter the documents are sent in full fine. With that I get this error message from PyMongo:
TypeError: filter must be an instance of dict, bson.son.SON, or any other type that inherits from collections.Mapping
I assume that my filter is malformed and it's no longer a dictionary so have tried various alternatives of wrapping in quotes and then .format() etc but cannot seem to hit the magical combination.

The problem is that the following is not a dictionary but a tuple of two dictionaries:
query = {'postcode': regx}, {'username': 1, 'user_posts': 1, '_id': 0}
if you wanted to use it directly you could do it with
user_list = mycol.find(query[0], query[1]).limit(5)
but probably best to actually assign it to two separate variables like so
query, filter = {'postcode': regx}, {'username': 1, 'user_posts': 1, '_id': 0}
and then
user_list = mycol.find(query, filter).limit(5)

Turned out it was quite easy.. I just needed to have two dictionaries. 1 for the query and the other for the filter.
regx = Regex('^{0}'.format('johnny'))
filter_stuff = {'username': 1, 'user_posts': 1, '_id': 0}
Then just build the full query as follows:
user_list = mycol.find(query, filter_stuff).limit(5)

(Python) Extract value in a dict in list

Hello,
I do a new project with a Discord bot for a game (iRacing) to keep information during a race. I use pyirsdk and i look all source code to try to understand how it works.
However, I have a dict in a list and i'm blocked.
DriverInfo:
{
'DriverHeadPosX': -0.579,
'DriverHeadPosY': 0.398,
'DriverHeadPosZ': 0.571,
'Drivers':
[
{'CarIdx': 0, 'UserName': 'Enzo Foucaud', 'AbbrevName': '', 'Initials': '', 'UserID': 409133, 'TeamID': 0, 'TeamName': 'Enzo Foucaud', 'CarNumber': '64', 'CarNumberRaw': 64, 'CarPath': 'mercedesamggt3', 'CarClassID': 0, 'CarID': 72, 'CarIsPaceCar': 0, 'CarIsAI': 0, 'CarScreenName': 'Mercedes AMG GT3', 'CarScreenNameShort': 'Mercedes AMG GT3', 'CarClassShortName': None, 'CarClassRelSpeed': 0, 'CarClassLicenseLevel': 0, 'CarClassMaxFuelPct': '1.000 %', 'CarClassWeightPenalty': '0.000 kg', 'CarClassPowerAdjust': '0.000 %', 'CarClassColor': 16777215, 'IRating': 1, 'LicLevel': 1, 'LicSubLevel': 1, 'LicString': 'R 0.01', 'LicColor': '0xundefined', 'IsSpectator': 0, 'CarDesignStr': '1,dff000,1a4b9b,ffffff', 'HelmetDesignStr': '1,dff000,1a4b9b,ffffff', 'SuitDesignStr': '1,dff000,1a4b9b,ffffff', 'CarNumberDesignStr': '0,0,FFFFFF,777777,000000', 'CarSponsor_1': 0, 'CarSponsor_2': 0, 'CurDriverIncidentCount': 0, 'TeamIncidentCount': 0}
]
}
I do this to know if 'UserName' exist or not
DriversInfo_list = ir['DriverInfo']
print('DriverInfo:', DriversInfo_list)
for i in DriversInfo_list:
if i == 'Drivers':
print("Drivers Exists")
SoloDriver = DriversInfo_list['Drivers']
print(SoloDriver)
if [d['UserName'] for d in SoloDriver if 'UserName' in d]:
print('Username Exists')
Console :
DriverInfo: {...}
Drivers Exists
[{'CarIdx': 0, ...}]
Username Exists
Now, i want to extract many info of 'Drivers' (dict) in DriverInfo who is a list.
For example i want the value of UserName but i don't know how i can do this. I search during all the day .. ^^ i do many lesson to try this ^^
Thank's for your answer

Assume each driver dict in the Drivers list has exactly the same keys. Otherwise will throw KeyError. To avoid this error check if the key exists in lambda.
drivers = ir['DriverInfo']['Drivers']
result = list(map((lambda x: x['UserName']), drivers))

It depends if all DriversInfo items contain only a list of one dictionary in the driver key. But you can do like:
DriversInfo_list = ir['DriverInfo']
for soloDriverInfo in DriversInfo_list['Drivers']:
#Iterate in list of driver key
print(f"username: {soloDriverInfo['UserName']}, car: {soloDriverInfo['CarScreenName']}")

Printing the response of a RethinkDB query in a reasonable way

I am participating in the Yelp Dataset Challenge and I'm using RethinkDB to store the JSON documents for each of the different datasets.
I have the following script:
import rethinkdb as r
import json, os
RDB_HOST = os.environ.get('RDB_HOST') or 'localhost'
RDB_PORT = os.environ.get('RDB_PORT') or 28015
DB = 'test'
connection = r.connect(host=RDB_HOST, port=RDB_PORT, db=DB)
query = r.table('yelp_user').filter({"name":"Arthur"}).run(connection)
print(query)
But when I run it at the terminal in a virtualenv I get this as an example response:
<rethinkdb.net.DefaultCursor object at 0x102c22250> (streaming):
[{'yelping_since': '2014-03', 'votes': {'cool': 1, 'useful': 2, 'funny': 1}, 'review_count': 5, 'id': '08eb0b0d-2633-4ec4-93fe-817a496d4b52', 'user_id': 'ZuDUSyT4bE6sx-1MzYd2Kg', 'compliments': {}, 'friends': [], 'average_stars': 5, 'type': 'user', 'elite': [], 'name': 'Arthur', 'fans': 0}, ...]
I know I can use pprint to pretty print outputs but a bigger issue that I don't understand how to resolve is just printing them in an intelligent manner, like not just showing "..." as the end of the output.
Any suggestions?

run returns an iterable cursor. Iterate over it to get all the rows:
query = r.table('yelp_user').filter({"name":"Arthur"})
for row in query.run(connection):
print(row)

Can't access data from a table in a web page

I am trying to access content from the table from this page http://www.afl.com.au/afl/stats/player-ratings/overall-standings# but however when I do so using beautiful soup in python, I am getting the data but from the 'All' filter selection not from a certain club filter. How can I achieve my goal?
I need to access all data from the table corresponding to a club in the filter. Please help me.
Please see the image below.
and also data from all pages:
I have used the following code
from bs4 import BeautifulSoup
import urllib2
import lxml.html
import xlwt
import unicodedata
infoList = []
lLink = "http://www.afl.com.au/afl/stats/player-ratings/overall-standings#club/CD_T140"
header = {'User-Agent': 'Mozilla/5.0'}
req_for_players = urllib2.Request(lLink,headers=header)
page_for_players = urllib2.urlopen(req_for_players)
soup_for_players = BeautifulSoup(page_for_players)
table = soup_for_players.select('table.player-ratings')[0]
for group_header in table.select('tbody tr span'):
player = group_header.string
infoList.append(player)
print infoList
The list infoList thus generated contains data corresponding to the "All" filter. But I want data according to the filter I choose.

you don't need to parse the table - use Firebug or any similar tool to watch the response while clicking on some page in paginator and you'll see that it serves you the data in JSON! Pure win!!!
There you can see the URL format for JSON data:
http://www.afl.com.au/api/cfs/afl/playerRatings?roundId=CD_R201401411&pageSize=40&pageNum=2
and this way you can fetch even the first page of data without parsing HTML and maybe you can fetch all data at once by setting some high value to pageSize variable

When the page loads the first time, all the players are there in the table regardless of what filter you've chosen. The filter is only invoked a short while later. That is why you are getting data on all the players.
Underneath the page is calling the following when the filter is invoked:
http://www.afl.com.au/api/cfs/afl/playerRatings?roundId=CD_R201401411&teamId=CD_T140&pageSize=40&pageNum=1
to get one team's players. CD_T140 are the Western Bulldogs in this case. You can see the different possible values in the selLadderTeam select element. You cannot simply call this url however as you will get a 401 error. Looking at the headers that are sent across, there is one that stands out. A token seems to be required. So using the requests library (it's more user-friendly than urllib2) you can do the following:
>>> import requests
>>> url = "http://www.afl.com.au/api/cfs/afl/playerRatings?roundId=CD_R201401411&teamId=CD_T40&pageSize=100"
>>> h = {'x-media-mis-token':'e61767b39a7680235476bb33aa946c0e'}
>>> r = requests.get(url, headers=h)
>>> r
<Response [200]>
>>> j = r.json()
>>> len(j['playerRatings'])
45
>>> j['playerRatings'][0]
{u'roundId': u'CD_R201401411', u'player': {u'playerId': u'CD_I260257', u'playerName': {u'givenName': u'Scott', u'surname': u'Pendlebury'}, u'captain': False, u'playerJumperNumber': None}, u'draftYear': u'2005', u'detailedRatings': [{u'trend': u'NO_CHANGE', u'ranking': 2, u'ratingType': u'OVERALL', u'ratingPoints': 674}, {u'trend': u'NO_CHANGE', u'ranking': 1, u'ratingType': u'TEAM', u'ratingPoints': 674}, {u'trend': u'NO_CHANGE', u'ranking': 2, u'ratingType': u'POSITION', u'ratingPoints': 674}], u'team': {u'teamId': u'CD_T40', u'teamName': u'Collingwood', u'teamAbbr': u'COLL', u'teamNickname': u'Magpies'}, u'position': u'MIDFIELDER'}
>>> j['playerRatings'][44]
{u'roundId': u'CD_R201401411', u'player': {u'playerId': u'CD_I295012', u'playerName': {u'givenName': u'Matt', u'surname': u'Scharenberg'}, u'captain': False, u'playerJumperNumber': None}, u'draftYear': u'2013', u'detailedRatings': [{u'trend': u'NO_CHANGE', u'ranking': 0, u'ratingType': u'OVERALL', u'ratingPoints': 0},{u'trend': u'NO_CHANGE', u'ranking': 0, u'ratingType': u'TEAM', u'ratingPoints': 0}, {u'trend': u'NO_CHANGE', u'ranking': 0, u'ratingType': u'POSITION', u'ratingPoints': 0}], u'team': {u'teamId': u'CD_T40', u'teamName': u'Collingwood', u'teamAbbr': u'COLL', u'teamNickname': u'Magpies'}, u'position': u'MEDIUM_DEFENDER'}
>>>
Notes: I don't know exactly what roundID is. I increased pageSize to something that would likely return all of a team's players and removed pageNum. They could change the token at any time.