test_data = {
"transactions": [{
"id": 5494,
"specific_wallet_id": 2079,
"evaluation_process_transaction_id": 5229,
"transaction_type": "Earn",
"points": 9,
"created_at": "2022-09-20T16:14:19.000+05:30",
"updated_at": "2022-09-20T16:14:19.000+05:30",
"custom_wallet_transaction_id": "null",
"order_meta_data": {},
"points_validity": "2023-09-20T00:00:00.000+05:30",
"points_status": "Active",
"transaction_amount": 450.0,
"transaction_at": "2022-09-20T16:14:18.000+05:30",
"tnx_status": "Successful",
"current_points": 9,
"programme_id": 4,
"customer_id": 1,
"reason_code": "Earn",
"channel": "app",
"order_id": "1",
"transaction_references": "null",
"points_value": 9.0,
"issuer": {
"id": 1,
"name": "oo",
"code": "oo",
"image_url": "null"
},
"partner": null
},
{
"id": 5486,
"specific_wallet_id": 2079,
"evaluation_process_transaction_id": 5221,
"transaction_type": "Earn",
"points": 9,
"created_at": "2022-09-20T15:46:15.000+05:30",
"updated_at": "2022-09-20T15:46:15.000+05:30",
"custom_wallet_transaction_id": "null",
"order_meta_data": {},
"points_validity": "2023-09-20T00:00:00.000+05:30",
"points_status": "Active",
"transaction_amount": 450.0,
"transaction_at": "2022-09-20T15:46:14.000+05:30",
"tnx_status": "Successful",
"current_points": 9,
"programme_id": 4,
"customer_id": 2189,
"reason_code": "Earn",
"channel": "app",
"order_id": "uuu",
"transaction_references": "null",
"points_value": 9.0,
"issuer": {
"id": 1,
"name": "oo",
"code": "ooo",
"image_url": "null"
},
"partner": {
"id": 1,
"name": "qq",
"code": "qq",
"image_url": "https://d157777v0iph40..net///offers/avis.jpg"
}
}
],
"success": "true",
"error": "null"
}
results=[]
results.extend(
[
AllTransaction(
issuer_id=Config.evaluation_issuer_id,
issuer_name="",
issuer_logo="",
wallet_id=o["specific_wallet_id"],
partner=o["partner"],
points=o["points"],
points_value_in_currency=(
o["points"] * 1
),
transaction_at=o["transaction_at"],
transaction_id=o["id"],
transaction_type=o["transaction_type"],
transaction_meta={
"order_id": o["order_id"],
"channel": o["channel"],
"order_meta_data": o["order_meta_data"],
},
transaction_name=o["transaction_type"],
)
for o in test_data["transactions"]
]
)
return TransactionResponse(results=results)
model code
class TransactionDetailsPartner(BaseModel):
partner: Optional[dict] =None
class TransactionDetails(BaseModel):
points: int
points_value_in_currency: float
transaction_id: int
transaction_at: datetime
transaction_type: str
transaction_meta: Optional[dict]
transaction_name: Optional[TransactionType]
class TransactionDetailsIssuer(BaseModel):
issuer_id: int
issuer_name: str
issuer_logo: str
wallet_id: int
class AllTransaction(TransactionDetails, TransactionDetailsIssuer, TransactionDetailsPartner):
pass
**this is my main method and here my partnerdata is null in one entry at test_data
and in pydantic model i made as optional but still it throwing error if i pass null data how to resolve this **
how to access this partner key even it is null and at model or api level is ok.
thanks in advance
The code that you gave is not valid Python, null is a valid JSON value but in Python it has to be None.
Try to load the string as JSON and then parse it to a Pydantic model.
Related
I have been stuck on this for the last 2 days, first time in a long time I had to use stackoverflow. But I love this community
What I am trying to do?
I am trying to upload a JSON object to update a table in a cloud SQL instance.
Error I am getting
I have tried using a service account and 02Auth credentials but I still get these errors.
Code below
from pprint import pprint
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials
from google.oauth2 import service_account
import google.auth.credentials
credentials = GoogleCredentials.get_application_default()
quota_project_id = "alert-groove-360914"
scopes = ["https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/sqlservice.admin"]
#credentials, _ = google.auth.load_credentials_from_file(
# "alert-groove-360914-0d5821a7921c.json", scopes=scopes, quota_project_id=quota_project_id
# )
#credentials = service_account.Credentials.from_service_account_file("alert-groove-360914-0d5821a7921c.json")
service = discovery.build('sqladmin', 'v1beta4', credentials=credentials)
# Project ID of the project that contains the instance.
project = 'XXXXXXXX' # TODO: Update placeholder value.
# Cloud SQL instance ID. This does not include the project ID.
instance = 'employment-types' # TODO: Update placeholder value.
database_instance_body = {
"id": 3,
"first_name": "XXXXXXXX",
"last_name": "XXXXXXX",
"email": "XXXXXX28#gmail.com",
"mobile_number": "XXXXXXXXX",
"country_id": 224,
"new_user": False,
"status": 1,
"reason": 'null',
"duration": 'null',
"linkedin_id": 'null',
"sync_contacts": False,
"linkedin_sync": False,
"available": True,
"radar_visibility": 'null',
"details": {
"organization": "Foras",
"isSelfEmployed": False,
"gender_id": 2,
"location_id": 457,
"gender": 'null',
"about": "The best around and nothing will ever get me down",
"image": "https:\/\/foras.tedmob.com\/storage\/users\/image-8596.jpg",
"lat": "25.20082000253009",
"long": "55.280527271443894",
"Gender_Rel": "Male",
"job_title": "Engineer",
"Experience": "10+ years",
"Location": "Dubai"
},
"interests": [
{
"id": 106,
"title": "Nightlife",
"icon": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-5908.png",
"image": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-7960.png",
"category": {
"category": "Culture"
}
},
{
"id": 102,
"title": "Music",
"icon": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-5908.png",
"image": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-7960.png",
"category": {
"category": "Culture"
}
},
{
"id": 98,
"title": "Personal finance",
"icon": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-5908.png",
"image": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-7960.png",
"category": {
"category": "Finance"
}
},
{
"id": 101,
"title": "Crypto currency ",
"icon": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-5908.png",
"image": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-7960.png",
"category": {
"category": "Finance"
}
},
{
"id": 104,
"title": "Breaking",
"icon": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-5908.png",
"image": "https:\/\/foras.tedmob.com\/storage\/interests\/basket-ball-7960.png",
"category": {
"category": "Culture"
}
}
],
"languages": [],
"goals": [
{
"id": 2,
"title": "Find a new job",
"icon": "https:\/\/foras.tedmob.com\/storage\/goals\/find-a-new-job-6804.png",
"pivot": {
"user_id": 3,
"goal_id": 2
},
"category": {
"category": "Career"
}
},
{
"id": 11,
"title": "Explore a career change",
"icon": "https:\/\/foras.tedmob.com\/storage\/goals\/explore-a-career-change-5885.png",
"pivot": {
"user_id": 3,
"goal_id": 11
},
"category": {
"category": "Career"
}
},
{
"id": 12,
"title": "Find co-founders",
"icon": "https:\/\/foras.tedmob.com\/storage\/goals\/find-co-founders-2170.png",
"pivot": {
"user_id": 3,
"goal_id": 12
},
"category": {
"category": "Networking"
}
}
],
"country": {
"id": 224,
"name": "United Arab Emirates",
"iso": "AE",
"phone_code": "+971"
},
"academic_levels": [
{
"id": 4,
"title": "Master's degree",
"pivot": {
"user_id": 3,
"academic_level_id": 4
}
}
],
"experiences": [
{
"id": 1,
"title": "test1",
"company_name": "test",
"isCurrentlyWorking": True,
"start_date": "2021-06-15 00:00:00",
"end_date": "2022-09-13 00:00:00",
"end_position": True,
"headline": "test",
"industry": "test",
"description": "description",
"employment_type": 1
}
]
}
request = service.instances().update(project=project, instance=instance, body=database_instance_body)
request.execute()
I am trying to parse this json file and I am having trouble.
The json looks like this:
<ListObject list at 0x2161945a860> JSON: {
"data": [
{
"amount": 100,
"available_on": 1621382400,
"created": 1621264875,
"currency": "usd",
"description": "0123456",
"exchange_rate": null,
"fee": 266,
"fee_details": [
{
"amount": 266,
"application": null,
"currency": "usd",
"description": "processing fees",
"type": "fee"
}
],
"id": "txn_abvgd1234",
"net": 9999,
"object": "balance_transaction",
"reporting_category": "charge",
"source": "cust1",
"sourced_transfers": {
"data": [],
"has_more": false,
"object": "list",
"total_count": 0,
"url": "/v1/source"
},
"status": "pending",
"type": "charge"
},
{
"amount": 25984,
"available_on": 1621382400,
"created": 1621264866,
"currency": "usd",
"description": "0326489",
"exchange_rate": null,
"fee": 93,
"fee_details": [
{
"amount": 93,
"application": null,
"currency": "usd",
"description": "processing fees",
"type": "fee"
}
],
"id": "txn_65987jihgf4984oihydgrd",
"net": 9874,
"object": "balance_transaction",
"reporting_category": "charge",
"source": "cust2",
"sourced_transfers": {
"data": [],
"has_more": false,
"object": "list",
"total_count": 0,
"url": "/v1/source"
},
"status": "pending",
"type": "charge"
},
],
"has_more": true,
"object": "list",
"url": "/v1/balance_"
}
I am trying to parse it in python with this script:
import pandas as pd
df = pd.json_normalize(json)
df.head()
but what I am getting is:
What i need is to parse each of these data points in its own column. So i will have 2 row of data with columns for each data points.
Something like this:
How do i do this now?
All but one of your fields are direct copies from the JSON, so you can just make a list of the fields you can copy, and then do the extra processing for the fee_details.
import json
import pandas as pd
inp = """{
"data": [
{
"amount": 100,
"available_on": 1621382400,
"created": 1621264875,
"currency": "usd",
"description": "0123456",
"exchange_rate": null,
"fee": 266,
"fee_details": [
{
"amount": 266,
"application": null,
"currency": "usd",
"description": "processing fees",
"type": "fee"
}
],
"id": "txn_abvgd1234",
"net": 9999,
"object": "balance_transaction",
"reporting_category": "charge",
"source": "cust1",
"sourced_transfers": {
"data": [],
"has_more": false,
"object": "list",
"total_count": 0,
"url": "/v1/source"
},
"status": "pending",
"type": "charge"
},
{
"amount": 25984,
"available_on": 1621382400,
"created": 1621264866,
"currency": "usd",
"description": "0326489",
"exchange_rate": null,
"fee": 93,
"fee_details": [
{
"amount": 93,
"application": null,
"currency": "usd",
"description": "processing fees",
"type": "fee"
}
],
"id": "txn_65987jihgf4984oihydgrd",
"net": 9874,
"object": "balance_transaction",
"reporting_category": "charge",
"source": "cust2",
"sourced_transfers": {
"data": [],
"has_more": false,
"object": "list",
"total_count": 0,
"url": "/v1/source"
},
"status": "pending",
"type": "charge"
}
],
"has_more": true,
"object": "list",
"url": "/v1/balance_"
}"""
copies = [
'id',
'net',
'object',
'reporting_category',
'source',
'amount',
'available_on',
'created',
'currency',
'description',
'exchange_rate',
'fee'
]
data = json.loads(inp)
rows = []
for inrow in data['data']:
outrow = {}
for copy in copies:
outrow[copy] = inrow[copy]
outrow['fee_details'] = inrow['fee_details'][0]['description']
rows.append(outrow)
df = pd.DataFrame(rows)
print(df)
Output:
timr#tims-gram:~/src$ python x.py
id net object reporting_category source amount ... created currency description exchange_rate fee fee_details
0 txn_abvgd1234 9999 balance_transaction charge cust1 100 ... 1621264875 usd 0123456 None 266 processing fees
1 txn_65987jihgf4984oihydgrd 9874 balance_transaction charge cust2 25984 ... 1621264866 usd 0326489 None 93 processing fees
[2 rows x 13 columns]
timr#tims-gram:~/src$
How can I convert the stripe model object into JSON to receive complete hierarchical data at client end?
stripeCustomer = stripe.Customer.retrieve(<stripe customer id>)
sendResponseToClient(stripeCustomer)
I am receiving only 1st level of data as json at client end, second level data from Stripe JSON object is not formatted.
JSON Data Example: (2nd level of data is not received at client end,)
Customer customer id=cus_DTWEPsfrHx3ikZ at 0x00000a> JSON: {
"id": "cus_DTWEPsfrHx3ikZ",
"object": "customer",
"account_balance": 0,
"created": 1535093686,
"currency": "usd",
"default_source": null,
"delinquent": false,
"description": null,
"discount": null,
"email": "rakesh16+test9#gmail.com",
"invoice_prefix": "E91FF30",
"livemode": false,
"metadata": {
},
"shipping": null,
"sources": {
"object": "list",
"data": [
],
"has_more": false,
"total_count": 0,
"url": "/v1/customers/cus_DTWEPsfrHx3ikZ/sources"
},
"subscriptions": {
"object": "list",
"data": [
{
"id": "sub_DTWEALN3urFael",
"object": "subscription",
"application_fee_percent": null,
"billing": "charge_automatically",
"billing_cycle_anchor": 1535093688,
"cancel_at_period_end": false,
"canceled_at": null,
"created": 1535093688,
"current_period_end": 1537772088,
"current_period_start": 1535093688,
"customer": "cus_DTWEPsfrHx3ikZ",
"days_until_due": null,
"discount": null,
"ended_at": null,
"items": {
"object": "list",
"data": [
{
"id": "si_DTWEuZaU4pw9Cv",
"object": "subscription_item",
"created": 1535093688,
"metadata": {
},
"plan": {
"id": "plan_free",
"object": "plan",
"active": true,
"aggregate_usage": null,
"amount": 0,
"billing_scheme": "per_unit",
"created": 1535008667,
"currency": "usd",
"interval": "month",
"interval_count": 1,
"livemode": false,
"metadata": {
},
"nickname": "free",
"product": "prod_DT8B8auk3CRNdw",
"tiers": null,
"tiers_mode": null,
"transform_usage": null,
"trial_period_days": null,
"usage_type": "licensed"
},
"quantity": 1,
"subscription": "sub_DTWEALN3urFael"
}
],
"has_more": false,
"total_count": 1,
"url": "/v1/subscription_items?subscription=sub_DTWEALN3urFael"
},
"livemode": false,
"metadata": {
},
"plan": {
"id": "plan_free",
"object": "plan",
"active": true,
"aggregate_usage": null,
"amount": 0,
"billing_scheme": "per_unit",
"created": 1535008667,
"currency": "usd",
"interval": "month",
"interval_count": 1,
"livemode": false,
"metadata": {
},
"nickname": "free",
"product": "prod_DT8B8auk3CRNdw",
"tiers": null,
"tiers_mode": null,
"transform_usage": null,
"trial_period_days": null,
"usage_type": "licensed"
},
"quantity": 1,
"start": 1535093688,
"status": "active",
"tax_percent": null,
"trial_end": null,
"trial_start": null
}
],
"has_more": false,
"total_count": 1,
"url": "/v1/customers/cus_DTWEPsfrHx3ikZ/subscriptions"
},
"tax_info": null,
"tax_info_verification": null
}
You can convert the returned Stripe object model to JSON using the following technique to ignore non-serializable fields:
stripeObject = stripe.SomeAPICall(...)
jsonEncoded = json.dumps(stripeObject, default=lambda o: '<not serializable>')
pythonDict = json.loads(jsonEncoded)
you can to_dict() api which will convert the stripe model object into dictionary format and then can be eventually converted into JSON string.
I have a variable in Robot Frame Work having Json response body in the following format
{
"jsonrpc": "2.0",
"id": "787878",
"result":
{
"content":
[
{
"id": 30,
"userID": "user",
"eventType":
{
"name": "test"
},
"timestamp": "2013-03-13T11:00:28.537Z",
"source": "Service",
"reason": null,
"comment": null,
"dataCache":
{
"lastLogin":
{
"id": 103,
"fieldName": "value1",
"newValue": "Wed Mar 13 07:00:28 EDT 2013",
"oldValue": null,
"type": "java.util.Date",
"auditEvent": null
}
},
"authority": "authenticate",
"auditedObject": null
}
],
"pageNumber": 0,
"pageSize": 99,
"numberOfElements": 1,
"totalElements": 1,
"lastPage": true,
"totalPages": 1
}
}
SO from this How can I get the content of only datacache as shown in the below
{
"lastLogin":
{
"id": 103,
"fieldName": "value1",
"newValue": "Wed Mar 13 07:00:28 EDT 2013",
"oldValue": null,
"type": "java.util.Date",
"auditEvent": null
}
},
If I use a variable like ${variable['result']['content']} , I will get entire body in the content But I just want a body in "DataCache":
Please help me to solve this issue..
${variable['result']['content'][0]['dataCache']}
Is this what you're looking for? Maybe you need some more practice with JSON then...
${variable['result']['content'][0]['dataCache']}
worked in my case.
In my controller I'm getting a json string (called c.order_history which looks like:
[
{
"status": [
{
"status": "created",
"timestamp": "2012-04-06 00:14:10"
},
{
"status": "authed",
"timestamp": "2012-04-06 00:14:17"
}
],
"product_info": [
{
"id": 3,
"quantity": 1,
"created": "2012-04-06 00:14:10",
"image_id": 13341
},
{
"id": 2,
"quantity": 1,
"created": "2012-04-06 00:14:10",
"image_id": 13323
},
{
"id": 1,
"quantity": 1,
"created": "2012-04-06 00:14:10",
"image_id": 13322
}
],
"shipping_charge": "0.00",
"order_number": "0723094747433",
"shipping_address": {
"country_code": null,
"extended_address": "Unit Z",
"locality": "Las Vagas",
"company": null,
"phone": null,
"postal_code": "31415",
"full_name": "Boris Karloff",
"nickname": null,
"region": "NV",
"street_address": "123 Random Way"
},
"subtotal": "59.00"
}
]
I'm passing it through json.loads(order_history) to turn it into a dict, and then trying to extract each key so I can then get the subsequent key/values within them like:
c.product_info = [{'product_info' : product_info} for product_info in c.order_history]
Which outputs the entire json string, but it's just named product_info now. Can someone steer me in the right direction on how I can access say, the timestamp value, product_info[0]['image_id'] and shipping_address values, etc.?
It looks like c.order_history will be a list of dictionaries, to just grab the product_info key from each of these dictionaries in a list comprehension you would do the following:
[{'product_info': order['product_info']} for order in c.order_history]