I would like to extract some data from the following line of text. The data of interest are:'exchange_symbol under both market_pair_base and 'market_pair_quote, all the data under exchange_reported.
Any help would be great.
{'status': {'timestamp': '2019-10-04T02:57:39.238Z', 'error_code': 0, 'error_message': None, 'elapsed': 14, 'credit_count': 1, 'notice': None}, 'data': {'id': 112, 'name': 'Liquid', 'slug': 'liquid', 'num_market_pairs': 190, 'market_pairs': [{'market_pair_base': {'exchange_symbol': 'ETH', 'currency_id': 1027, 'currency_symbol': 'ETH', 'currency_type': 'cryptocurrency'}, 'market_pair_quote': {'exchange_symbol': 'JPY', 'currency_id': 2797, 'currency_symbol': 'JPY', 'currency_type': 'fiat'}, 'quote': {'exchange_reported': {'price': 18522.8757, 'volume_24h_base': 12901.4143123, 'volume_24h_quote': 238971293.660934, 'last_updated': '2019-10-04T02:57:04.000Z'}, 'USD': {'price': 173.503768779353, 'volume_24h': 2238444.00576794, 'last_updated': '2019-10-04T02:57:04.000Z'}}, 'market_id': 4431, 'market_pair': 'ETH/JPY', 'category': 'spot', 'fee_type': 'percentage'}, {'market_pair_base': {'exchange_symbol': 'XRP', 'currency_id': 52, 'currency_symbol': 'XRP', 'currency_type': 'cryptocurrency'}, 'market_pair_quote': {'exchange_symbol': 'JPY', 'currency_id': 2797, 'currency_symbol': 'JPY', 'currency_type': 'fiat'}, 'quote': {'exchange_reported': {'price': 26.55199, 'volume_24h_base': 8223150.63965144, 'volume_24h_quote': 218341013.552519, 'last_updated': '2019-10-04T02:56:04.000Z'}, 'USD': {'price': 0.248712479001935, 'volume_24h': 2045200.18079406, 'last_updated': '2019-10-04T02:56:04.000Z'}}, 'market_id': 16254, 'market_pair': 'XRP/JPY', 'category': 'spot', 'fee_type': 'percentage'}]}}
I tried the following code but this send an error :
File "proc_manual_comp.py", line 112, in <module>
for item in data['data'][currency]['market_pairs']
TypeError: 'int' object is not subscriptable
the code:
c_price = []
url = 'https://pro-api.coinmarketcap.com/v1/exchange/market-pairs/latest'
parameters = {
'id':'112',
'start':'1',
'limit':'91'
}
headers = {
'Accepts': 'application/json',
'X-CMC_PRO_API_KEY': 'XXXXXXXXXXXXXXXXXXXXXXX',
}
session = Session()
session.headers.update(headers)
response = session.get(url, params=parameters)
data = json.loads(response.text)
for currency in data['data']:
used_list = [
item['market_pairs']['market_pair_base']
for item in data['data'][currency]['market_pairs']
]
price = pd.DataFrame.from_records(used_list)
print(price)
price['timestamp'] = pd.to_datetime(price['timestamp'])
price['timestamp'] = price['timestamp'].astype(str).str[:-6]
price_c = price.set_index('timestamp').close
c_price.append(price_c.rename(currency))
print(c_price)
c_price = pd.concat(c_price, axis=1)
Expected output:
market_pair_base market_pair_quoted last_updated price
ETH JPY 2019-10-04T02:57:04.000Z 18522.8757
Assumed JSON as given in d variable
Try below snippet:
target_df=pd.DataFrame(columns=['market_pair_base','market_pair_quote','price','last_updated'])
target=dict()
usedlist=d['data']['market_pairs']
for i in range(len(usedlist)):
target['market_pair_base']=[usedlist[i]['market_pair_base']['exchange_symbol']]
target['market_pair_quote'] = [usedlist[i]['market_pair_quote']['exchange_symbol']]
target['price'] = [usedlist[i]['quote']['exchange_reported']['price']]
target['last_updated'] = [usedlist[i]['quote']['exchange_reported']['last_updated']]
target_df=pd.concat([target_df, pd.DataFrame(target)], ignore_index=True)
This print(target_df) will give desired output as below:
market_pair_base market_pair_quote price last_updated
0 ETH JPY 18522.87570 2019-10-04T02:57:04.000Z
1 XRP JPY 26.55199 2019-10-04T02:56:04.000Z
it seems like a problem with your for loop.
for currency in data['data']:
here since data['data'] is a dict hence values of currency is going to be keys of dict, an alternate way can be something like this
for market_pairs in data['data']['market_pairs']:
used_list = [market_pair['market_pair_base'] for market_pair in market_pairs]
price = pd.DataFrame.from_records(used_list)
this is based on the fact that you wanted to put all market_pair_base in a Dataframe.
Related
I found an old thread ( how can I close position on OKX V5 based on CCXT Python? ) but is not working.
I'm having problem in closing my current margin-spot position. Here's my code:
params = {'tdMode': 'cross', 'ccy': 'USDT'}
order = await self.okx.create_order(
symbol='XRP/USDT', type='market', side='sell',
amount=99.43, price=0.39024, params=params
)
Result:
{'amount': 99.431119,
'average': 0.39005,
'clientOrderId': 'e847386590ce4dBCce71dedc37e12254',
'cost': 38.78310796595,
'datetime': '2022-12-02T17:29:58.048Z',
'fee': {'cost': 0.03878310796595, 'currency': 'USDT'},
'fees': [{'cost': 0.03878310796595, 'currency': 'USDT'}],
'filled': 99.431119,
'id': '518962064594153472',
Places the first order correctly. Now to close this order I use barrely the same code, changing sides:
params = {'tdMode': 'cross', 'ccy': 'USDT'}
order = await self.okx.create_order(
symbol='XRP/USDT', type='market', side='buy',
amount=99.43, price=0.39017, params=params
)
Problem is that the amount in the second order seeems to be 'contracts'/'price' instead:
result:
{'amount': 99.43111,
'average': 0.39015,
'clientOrderId': 'e847386590ce4dBC328bc6dfc937795f',
'cost': 99.43110980145,
'datetime': '2022-12-02T17:36:47.913Z',
'fee': {'cost': 0.254853543, 'currency': 'XRP'},
'fees': [{'cost': 0.254853543, 'currency': 'XRP'}],
'filled': 254.853543,
'id': '518963783692562432',
What am I doing wrong ?
Are you changing the default tgtCcy value?
Can you try to place the market order without the price (market orders execute at the market price so you don't need to specify it)?
I'm making a call to an api which is returning a JSON response, whcih i am then trying to retrieve certain data from within the response.
{'data': {'9674': {'category': 'token',
'contract_address': [{'contract_address': '0x2a3bff78b79a009976eea096a51a948a3dc00e34',
'platform': {'coin': {'id': '1027',
'name': 'Ethereum',
'slug': 'ethereum',
'symbol': 'ETH'},
'name': 'Ethereum'}}],
'date_added': '2021-05-10T00:00:00.000Z',
'date_launched': '2021-05-10T00:00:00.000Z',
'description': 'Wilder World (WILD) is a cryptocurrency '
'launched in 2021and operates on the '
'Ethereum platform. Wilder World has a '
'current supply of 500,000,000 with '
'83,683,300.17 in circulation. The last '
'known price of Wilder World is 2.28165159 '
'USD and is down -6.79 over the last 24 '
'hours. It is currently trading on 21 active '
'market(s) with $2,851,332.76 traded over '
'the last 24 hours. More information can be '
'found at https://www.wilderworld.com/.',
'id': 9674,
'is_hidden': 0,
'logo': 'https://s2.coinmarketcap.com/static/img/coins/64x64/9674.png',
'name': 'Wilder World',
'notice': '',
'platform': {'id': 1027,
'name': 'Ethereum',
'slug': 'ethereum',
'symbol': 'ETH',
'token_address': '0x2a3bff78b79a009976eea096a51a948a3dc00e34'},
'self_reported_circulating_supply': 19000000,
'self_reported_tags': None,
'slug': 'wilder-world',
'subreddit': '',
'symbol': 'WILD',
'tag-groups': ['INDUSTRY',
'CATEGORY',
'INDUSTRY',
'CATEGORY',
'CATEGORY',
'CATEGORY',
'CATEGORY'],
'tag-names': ['VR/AR',
'Collectibles & NFTs',
'Gaming',
'Metaverse',
'Polkastarter',
'Animoca Brands Portfolio',
'SkyVision Capital Portfolio'],
'tags': ['vr-ar',
'collectibles-nfts',
'gaming',
'metaverse',
'polkastarter',
'animoca-brands-portfolio',
'skyvision-capital-portfolio'],
'twitter_username': 'WilderWorld',
'urls': {'announcement': [],
'chat': [],
'explorer': ['https://etherscan.io/token/0x2a3bff78b79a009976eea096a51a948a3dc00e34'],
'facebook': [],
'message_board': ['https://medium.com/#WilderWorld'],
'reddit': [],
'source_code': [],
'technical_doc': [],
'twitter': ['https://twitter.com/WilderWorld'],
'website': ['https://www.wilderworld.com/']}}},
'status': {'credit_count': 1,
'elapsed': 7,
'error_code': 0,
'error_message': None,
'notice': None,
'timestamp': '2022-01-20T21:33:04.832Z'}}
The data i am trying to get is 'logo': 'https://s2.coinmarketcap.com/static/img/coins/64x64/9674.png', but this sits within [data][9674][logo]
But as this script to running in the background for other objects, i won't know what the number [9674] is for other requests.
So is there a way to get that number automatically?
[data] will always be consistent.
Im using this to get the data back
session = Session()
session.headers.update(headers)
response = session.get(url, params=parameters)
pprint.pprint(json.loads(response.text)['data']['9674']['logo'])
You can try this:
session = Session()
session.headers.update(headers)
response = session.get(url, params=parameters)
resp = json.loads(response.text)
pprint.pprint(resp['data'][next(iter(resp['data']))]['logo'])
where next(iter(resp['data'])) - returns first key in resp['data'] dict. In your example it '9674'
With .keys() you get a List of all Keys in a Dictionary.
So you can use keys = json.loads(response.text)['data'].keys() to get the keys in the data-dict.
If you know there is always only one entry in 'data' you could use json.loads(response.text)['data'][keys[0]]['logo']. Otherwise you would need to iterate over all keys in the list and check which one you need.
I have below list which stored in data
{'id': 255719,
'display_name': 'Off Broadway Apartments',
'access_right': {'status': 'OWNED', 'api_enabled': True},
'creation_time': '2021-04-26T15:53:29+00:00',
'status': {'value': 'OFFLINE', 'last_change': '2021-07-10T17:26:50+00:00'},
'licence': {'id': 74213,
'code': '23AF-15A8-0514-2E4B-04DE-5C19-A574-B20B',
'bound_mac_address': '00:11:32:C2:FE:6A',
'activation_time': '2021-04-26T15:53:29+00:00',
'type': 'SUBSCRIPTION'},
'timezone': 'America/Chicago',
'version': {'agent': '3.7.0-b001', 'package': '2.5.1-0022'},
'location': {'latitude': '41.4126', 'longitude': '-99.6345'}}
I would like to convert into data frame.can anyone advise?
I tried below code
df = pd.DataFrame(data)
but it's not coming properly as many nested lists. can anyone advise?
from pandas.io.json import json_normalize
# load json
json = {'id': 255719,
'display_name': 'Off Broadway Apartments',
'access_right': {'status': 'OWNED', 'api_enabled': True},
'creation_time': '2021-04-26T15:53:29+00:00',
'status': {'value': 'OFFLINE', 'last_change': '2021-07-10T17:26:50+00:00'},
'licence': {'id': 74213,
'code': '23AF-15A8-0514-2E4B-04DE-5C19-A574-B20B',
'bound_mac_address': '00:11:32:C2:FE:6A',
'activation_time': '2021-04-26T15:53:29+00:00',
'type': 'SUBSCRIPTION'},
'timezone': 'America/Chicago',
'version': {'agent': '3.7.0-b001', 'package': '2.5.1-0022'},
'location': {'latitude': '41.4126', 'longitude': '-99.6345'}}
Create a fuction to flat nested jsons:
def flatten_json(y):
out = {}
def flatten(x, name=''):
if type(x) is dict:
for a in x:
flatten(x[a], name + a + '_')
elif type(x) is list:
i = 0
for a in x:
flatten(a, name + str(i) + '_')
i += 1
else:
out[name[:-1]] = x
flatten(y)
return out
You can now use that function on your original json file:
flat = flatten_json(json)
df = json_normalize(flat)
Results:
id display_name ... location_latitude location_longitude
0 255719 Off Broadway Apartments ... 41.4126 -99.6345
I have a complex situation which I hope to solve and which might profit us all. I collected data from my API, added a pagination and inserted the complete data package in a tuple named q1 and finally I have made a dictionary named dict_1of that tuple which looks like this:
dict_1 = {100: {'ID': 100, 'DKSTGFase': None, 'DK': False, 'KM': None,
'Country: {'Name': GE', 'City': {'Name': 'Berlin'}},
'Type': {'Name': '219'}, 'DKObject': {'Name': '8555', 'Object': {'Name': 'Car'}},
'Order': {'OrderId': 101, 'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': Audi, 'Client': {‘1’ }}, 'DKComponent': {'Name': ‘John’}},
{200: {'ID': 200, 'DKSTGFase': None, 'DK': False, ' KM ': None,
'Country: {'Name': ES', 'City': {'Name': 'Madrid'}}, 'Type': {'Name': '220'},
'DKObject': {'Name': '8556', 'Object': {'Name': 'Car'}},
'Order': {'OrderId': 102, 'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': Mercedes, 'Client': {‘2’ }}, 'DKComponent': {'Name': ‘Sergio’}},
Please note that in the above dictionary I have just stated 2 records. The actual dictionary has 1400 records till it reaches ID 1500.
Now I want to 2 things:
I want to change some keys for all the records. key DK has to become DK1. Key Name in Country has to become Name1 and Name in Object has to become 'Name2'
The second thing I want is to make a dataFrame of the whole bunch of data. My expected outcome is:
This is my code:
q1 = response_2.json()
next_link = q1['#odata.nextLink']
q1 = [tuple(q1.values())]
while next_link:
new_response = requests.get(next_link, headers=headers, proxies=proxies)
new_data = new_response.json()
q1.append(tuple(new_data.values()))
next_link = new_data.get('#odata.nextLink', None)
dict_1 = {
record['ID']: record
for tup in q1
for record in tup[2]
}
#print(dict_1)
for x in dict_1.values():
x['DK1'] = x['DK']
x['Country']['Name1'] = x['Country']['Name']
x['Object']['Name2'] = x['Object']['Name']
df = pd.DataFrame(dict_1)
When i run this I receive the following Error:
Traceback (most recent call last):
File "c:\data\FF\Desktop\Python\PythongMySQL\Talky.py", line 57, in <module>
x['Country']['Name1'] = x['Country']['Name']
TypeError: 'NoneType' object is not subscriptable
working code
lists=[]
alldict=[{100: {'ID': 100, 'DKSTGFase': None, 'DK': False, 'KM': None,
'Country': {'Name': 'GE', 'City': {'Name': 'Berlin'}},
'Type': {'Name': '219'}, 'DKObject': {'Name': '8555', 'Object': {'Name': 'Car'}},
'Order': {'OrderId': 101, 'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': 'Audi', 'Client': {'1' }}, 'DKComponent': {'Name': 'John'}}}]
for eachdict in alldict:
key=list(eachdict.keys())[0]
eachdict[key]['DK1']=eachdict[key]['DK']
del eachdict[key]['DK']
eachdict[key]['Country']['Name1']=eachdict[key]['Country']['Name']
del eachdict[key]['Country']['Name']
eachdict[key]['DKObject']['Object']['Name2']=eachdict[key]['DKObject']['Object']['Name']
del eachdict[key]['DKObject']['Object']['Name']
lists.append([key, eachdict[key]['DK1'], eachdict[key]['KM'], eachdict[key]['Country']['Name1'],
eachdict[key]['Country']['City']['Name'], eachdict[key]['DKObject']['Object']['Name2'], eachdict[key]['Order']['Client']])
pd.DataFrame(lists, columns=[<columnNamesHere>])
Output:
{100: {'ID': 100,
'DKSTGFase': None,
'KM': None,
'Country': {'City': {'Name': 'Berlin'}, 'Name1': 'GE'},
'Type': {'Name': '219'},
'DKObject': {'Name': '8555', 'Object': {'Name2': 'Car'}},
'Order': {'OrderId': 101,
'CreatedOn': '2018-07-06T16:54:36.783+02:00',
'ModifiedOn': '2018-07-06T16:54:36.783+02:00',
'Name': 'Audi',
'Client': {'1'}},
'DKComponent': {'Name': 'John'},
'DK1': False}}
I am trying to ge around with APIs in general. To test this I coded this little snippet of code to get a list of all the channels on the Swedish national public service radio, and I want to print the ID and NAME of the channels:
import requests as rq
import json
from pprint import pprint
resp = rq.get('http://api.sr.se/api/v2/channels?
format=json&indent=TRUE')
respjson = json.loads(resp.text)
pprint (respjson['id'])
And I get the error
File "sr-api.py", line 9, in <module>
pprint (respjson['id']['name'])
KeyError: 'id'
The (abbreviated) 'respjson' looks like this
{'channels': [{'channeltype': 'Rikskanal',
'color': '31a1bd',
'id': 132,
'image': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg',
'liveaudio': {'id': 132,
'statkey': '/app/direkt/p1[k(132)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/132.mp3'},
'name': 'P1',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=132',
'siteurl': 'http://sverigesradio.se/p1',
'xmltvid': 'p1.sr.se'},
{'channeltype': 'Lokal kanal',
'color': 'c31eaa',
'id': 200,
'image': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg',
'liveaudio': {'id': 200,
'statkey': '/app/direkt/p4 jämtland[k(200)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/200.mp3'},
'name': 'P4 Jämtland',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=200',
'siteurl': 'http://sverigesradio.se/jamtland/',
'xmltvid': 'p4jmtl.sr.se'}],
'copyright': 'Copyright Sveriges Radio 2017. All rights reserved.',
'pagination': {'nextpage': 'http://api.sr.se/v2/channelsformat=json&indent=true&page=2',
'page': 1,
'size': 10,
'totalhits': 55,
'totalpages': 6}}
Channels is a list. You have to iterate on it to get all channels and print their ids.
# starting from respjson
respjson = {
'channels': [
{
'channeltype': 'Rikskanal',
'color': '31a1bd',
'id': 132,
'image': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg',
'liveaudio': {'id': 132,
'statkey': '/app/direkt/p1[k(132)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/132.mp3'},
'name': 'P1',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=132',
'siteurl': 'http://sverigesradio.se/p1',
'xmltvid': 'p1.sr.se'},
{
'channeltype': 'Lokal kanal',
'color': 'c31eaa',
'id': 200,
'image': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg',
'liveaudio': {'id': 200,
'statkey': '/app/direkt/p4 jämtland[k(200)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/200.mp3'},
'name': 'P4 Jämtland',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=200',
'siteurl': 'http://sverigesradio.se/jamtland/',
'xmltvid': 'p4jmtl.sr.se'
}
],
'copyright': 'Copyright Sveriges Radio 2017. All rights reserved.',
'pagination': {
'nextpage': 'http://api.sr.se/v2/channelsformat=json&indent=true&page=2',
'page': 1,
'size': 10,
'totalhits': 55,
'totalpages': 6
}
}
for channel in respjson['channels']:
print(channel['id'])
What you want to do is look thru the dictionaries presented inside the channels, you can do that with the following...
for dic in respjson['channels']:
pprint(dic['id'])