Parse JSON within Python - python

I just wanted to ask what can I do to solve this issue I have.
Essentially I am making a stock checker for sneakers from Adidas, I know the endpoint to obtain the stock but the JSON data given back to me whilst readable and contains what I need also contains a bunch of other information that is unnecessary to what I am trying to do.
Example of a link to an endpoint:
http://production.store.adidasgroup.demandware.net/s/adidas-GB/dw/shop/v16_9/products/(BZ0221)?client_id=c1f3632f-6d3a-43f4-9987-9de920731dcb&expand=availability,variations,prices
This is a link to the JSON containing the stock of the shoe, price and availability. However, if you try to open it you'll see that it responds a bunch of useless info such as the description of the shoe and the price which I do not need.
A github repository that I was using to try and get to grips with the requests I am trying to make is:
https://github.com/yzyio/adidas-stock-checker/blob/master/assets/index.js
I can get it to give me the JSON response I am just trying to strip what I don't need and keep what I do need which I am finding very difficult especially in python.
Many Thanks!

Since you've said you can get a JSON response from the server than the first think you need to do is tell python to load it as JSON.
import json
data = json.loads(response_from_server)
After doing this you can now access the values in your JSON object the way you would access them via a Python dict.
data["artist"]["id"]

Related

Python Product Tag Update using Shopify API - 400 Error - Product :Required Parameter Missing Or Invalid

Just built my first Shopify store and wanted to use python and API to bulk-update product tags on all our products.
However, I've run into a snag on my PUT call. I keep getting a 400 Response with the message '{"errors":{"product":"Required parameter missing or invalid"}}'.
I've seen other people with similar issues on here but none of their solutions seem to be working for me.
Has anyone else run into this problem that can help me figure it out?
Here are some screenshots of my code, a printout of the payload, and the error in the response.
Code, Payload and Response:
I can successfully get the product info using the API and was originally sending the entire JSON payload that was returned just updated it with my product tags and sent it back through the API.
To narrow potential pain points, I'm keeping it simple now and just including "id" and "tags" in the payload but still get the same error.
We figured it out! It turns out that when we initially created the store we used a domain store name that was misspelled so we created a new domain store name with the correct spelling. However, the original domain store name was not deleted (not even sure it can be), and even though the new domain store name forwards to the original one and allows for GET requests, for PUT requests we had to use the original misspelled domain store name and the PUTs work fine.
I figured this out by using fiddler to capture a manual product update via the Shopify website to see what the payload looked like and that's why I noticed the store URL was different than the one we were using.
Hope this helps someone with a similar issue!

Python use API to get data

I am kind of stucked in trying to solve following issue: I try to access a web-page in order to get some data for a supplier (need to do it for work) in an automated way, using an api
The API is called https://wl-api.mf.gov.pl and shall provide information stored in json for supplier which can be found over their tax ID.
I use the request package and I am able to manage to get positive response:
import requests
nip=7393033097
response=requests.get("https://wl-api.mf.gov.pl")
print(response) --> Response [200]
If I click on the link and scroll until I find the specific part for the tax information, I find the following line
GET /api/search/nip/{nip}
So what I did is to add this line into my response variable, since this is how I understood it - and there is the point where I think I am wrong
response=requests.get("https://wl-api.mf.gov.pl/search/7393033097/{7393033097}")
However, I cannot access it.
Am I doing something wrong - I do believe yes - and can anyone give me a little help :)
Update: If I check the requirements / documentation I find following information where I need a bit support to implement it
GET /api/search/nip/{nip}
(nip?date)
Single entity search by nip
**Path parameters**
nip (required)
*Path Parameter — Nip*
**Query parameters**
date (required)
*Query Parameter — format: date*
**Return type**
EntityResponse
Example data
Content-Type: application/json
I think this line:https://wl-api.mf.gov.pl/search/7393033097/{7393033097} should be like this:
https://wl-api.mf.gov.pl/search/nip/7393033097

Using Python - convert string into JSON only instead of - dict of dict?

I am creating a watcher for my Videos section in youtube which will keep monitoring the stats of latest video uploaded. I am not using Selenium because it will keep the browser engaged and interrupt. But using requests_html to load the /videos page and return me the stats like how many view are now and send me a message on Telegram app.
So when I did requests_html call i retrieve a small json which has all the videos & stats but the command json.load() actually coverts it into dict of dict. The intention is to use the result of json.load() as JSON so that I can perform json parsing.
Attached the snapshot of JSON structure and highlighted the desired keys
I couldnt find any better example that tell me how to convert it into just JSON instead of dict of dict. because there are multiple levels approx 20+ to retrieve the desired key value. I have seen very simple example that do the following, but it is not possible to write those many nodes in the below statement.
aKeyValue = dictName['parentnode']['childNode']
Secondly if it is just JSON then I believe jsonpath_ng and its parse methods can be used to retrieve a desired key with having to provide the complete path from root. If this is tnot the right way, please suggest any other Py Module.
the recursive functions didnt work properly. I tried may be 10+ different functions, none worked on the JSON. Finally I found jsonpath-ng to work after I read thru its entire documentation
It was pretty simple, I dont know why I didnt figure this out earlier.
json.loads(jsonString)
parse($..gridRenderer.items[0].gridVideoRenderer.viewCountText.simpleText)
But at the same time the statement below doesnt work. Although statement below wasnt required, but i just tried. It worked on jsonpath evaluator online but not in the python code.
$..gridRenderer.items[0].gridVideoRenderer.viewCountText[?(simpleText="5927 views")]
If someone could point out why the above statement wouldnt work in python code ?

How to get the *widget-token* for downloading csv-files from Google Trends?

Im currently trying to retrieve data from the google trends site directly into python. By inspecting the "download csv"-button I was able to extract the underlying url of the file, which looks like this:
https://trends.google.com/trends/api/widgetdata/multiline/csv?req=%7B%22time%22%3A%222018-08-30%202019-08-30%22%2C%22resolution%22%3A%22WEEK%22%2C%22locale%22%3A%22de%22%2C%22comparisonItem%22%3A%5B%7B%22geo%22%3A%7B%7D%2C%22complexKeywordsRestriction%22%3A%7B%22keyword%22%3A%5B%7B%22type%22%3A%22BROAD%22%2C%22value%22%3A%22trump%22%7D%5D%7D%7D%5D%2C%22requestOptions%22%3A%7B%22property%22%3A%22%22%2C%22backend%22%3A%22IZG%22%2C%22category%22%3A0%7D%7D&token=APP6_UEAAAAAXWrEbGVLW-ssfeQvOJgr9938DRgYO1sm&tz=-120
Unquoted :
https://trends.google.com/trends/api/widgetdata/multiline/csv?req={"time":"2018-08-30 2019-08-30","resolution":"WEEK","locale":"de","comparisonItem":[{"geo":{},"complexKeywordsRestriction":{"keyword":[{"type":"BROAD","value":"trump"}]}}],"requestOptions":{"property":"","backend":"IZG","category":0}}&token=APP6_UEAAAAAXWrEbGVLW-ssfeQvOJgr9938DRgYO1sm&tz=-120
I can now easily get this csv into a pandas dataframe. Ideally, I would now just manipulate the url in order to make custom requests and load new data. The problem I have is that I cannot use the same token parameter, since it is somehow generated newly for each individual csv-request. I think that the answer from shaochuancs in Origin of tokens in Google trends API call describes the problem as I am facing it. Can anyone explain how I can request this token which I could then use for the second request ( the actual csv download )? :/

Accessing Data from a graph on a website using python

I have generally wondering if it is possible to access data displayed on graphs such as these: https://coinmarketcap.com/currencies/bitcoin/
I have looked at the .htm file and other links within this, yet none seem to be containing this. Would anyone know how to retrieve and print information from the graph using python?
You can retrieve the information from coinmarketcap using their API.
https://coinmarketcap.com/api/
This will allow you to request data from them, sometimes with given parameters (e.g. the currency) and you'll retrieve the data in a JSON format, which you can then again parse and display with python.
There are plenty of API tutorials out there for python on the web. Good luck!

Categories

Resources