Extracting nested items in JSON file with Python - python

I'm using IBM Watson and python3 to translate an audio file into a text file. It returns the results in a JSON-format. The resulting JSON is like this...
{
"results": [
{
"alternatives": [
{
"transcript": "Marcus white is the managing director of quartz power group an energy company ",
"confidence": 0.85,
"word_confidence": [
[
"Marcus",
0.678
],
[
"white",
0.99
],
[
"is",
0.988
],
[
"the",
1.0
],
[
"managing",
1.0
],
[
"director",
1.0
],
[
"of",
1.0
],
[
"quartz",
0.394
],
[
"power",
0.737
],
[
"group",
0.968
],
[
"an",
0.783
],
[
"energy",
0.989
],
[
"company",
0.844
]
],
"timestamps": [
[
"Marcus",
12.28,
12.78
],
[
"white",
12.78,
13.17
],
[
"is",
13.17,
13.33
],
[
"the",
13.33,
13.42
],
[
"managing",
13.42,
13.83
],
[
"director",
13.83,
14.39
],
[
"of",
14.39,
14.52
],
[
"quartz",
14.52,
15.0
],
[
"power",
15.0,
15.36
],
[
"group",
15.36,
15.79
],
[
"an",
15.93,
16.08
],
[
"energy",
16.08,
16.45
],
[
"company",
16.45,
16.95
]
]
}
],
"final": true
},
{
"alternatives": [
{
"transcript": "every month the departmental manages meet to discuss high level issues in the company ",
"confidence": 0.925,
"word_confidence": [
[
"every",
1.0
],
[
"month",
0.993
],
[
"the",
0.728
],
[
"departmental",
1.0
],
[
"manages",
0.7
],
[
"meet",
0.77
],
[
"to",
1.0
],
[
"discuss",
1.0
],
[
"high",
0.835
],
[
"level",
0.984
],
[
"issues",
1.0
],
[
"in",
0.67
],
[
"the",
0.927
],
[
"company",
0.994
]
],
"timestamps": [
[
"every",
18.1,
18.39
],
[
"month",
18.39,
18.93
],
[
"the",
18.96,
19.07
],
[
"departmental",
19.07,
19.73
],
[
"manages",
19.73,
20.29
],
[
"meet",
20.29,
20.56
],
[
"to",
20.56,
20.66
],
[
"discuss",
20.66,
21.12
],
[
"high",
21.12,
21.33
],
[
"level",
21.33,
21.64
],
[
"issues",
21.64,
22.08
],
[
"in",
22.08,
22.18
],
[
"the",
22.18,
22.27
],
[
"company",
22.27,
22.75
]
]
}
],
"final": true
},
This format repeats itself for each translated segment. I'm trying to extract all transcriptions, that is the value for transcript. I tried this:
index = 0
for [index]["transcript"] in ["results"][0]["alternatives"]:
print (["results"][0]["alternatives"][index]["transcript"])
index += 1
But this fails due to each transcription-object being embedded in a list. The full path for the first transcription value is:
d["results"][0]["alternatives"][0]["transcript"]
How can I iterate through this list to extract all transcription values and add them to a string variable?
Cheers :)


for result in d['results']:
for alternative in result['alternatives']:
# add to your string here
print alternative['transcript']
In general, python lets you iterate over objects in its lists instead of having to iterate using indices, which is more common in languages like C++.

Related

Mongo geoJSON corrupted polygon

Im searching in the MongoDB by using a GEOJSON polygon without holes. I get an error when accesing the cursor. The error is:
OperationFailure: Secondary loops not contained by first exterior loop - secondary loops must be holes.
Code:
import pymongo
import json
def loadjson(filename):
with open(filename) as f:
mydict = json.load(f)
return mydict
client = pymongo.MongoClient('mongodb://' \
+ 'username' \
+ ':' \
+ "password" + \
'#localhost:num')
gjOr=loadjson('polygon.json')
database = client.database
collection = database["name"]
filParam = {"field": {"$geoWithin": {"$geometry": gjOr["geometry"]}}}
exParam = {"parameter": 1}
cursorItineraries = collection.find(filParam, exParam).batch_size(2)
for i in cursorItineraries:
print(i)
Error :
OperationFailure: Secondary loops not contained by first exterior loop - secondary loops must be holes: [ [ 4.423298831017474, 45.92246510136637 ], [ 4.423272328729544, 45.9224124914541 ], [ 4.423296014293201, 45.92245885344253 ], [ 4.423298831017474, 45.92246510136637 ] ] first loop: [ [ 45.80122758057101, 4.376352681950241 ], [ 45.8044838524037, 4.376220111724892 ], [ 45.80502003601278, 4.384299007716068 ], [ 45.80303745929081, 4.388437890276315 ], [ 45.81747689026152, 4.380885175476993 ], [ 45.82029916312589, 4.38306756294014 ], [ 45.82485210019716, 4.37663747923372 ], [ 45.82534211272147, 4.381172591846549 ], [ 45.83402693020424, 4.384295932265717 ], [ 45.83750180635732, 4.391204268525621 ], [ 45.83984258047744, 4.387611887543129 ], [ 45.84600085935993, 4.383682442715087 ], [ 45.85031045846961, 4.391211993814656 ], [ 45.85985548506667, 4.396744466054351 ], [ 45.8651960481546, 4.395075656820409 ], [ 45.86464844812487, 4.390221929905438 ], [ 45.86619589736199, 4.361702458243602 ], [ 45.87693527428829, 4.343076054047009 ], [ 45.88006074780913, 4.340914771281566 ], [ 45.8901897554254, 4.337545959836689 ], [ 45.89739332872952, 4.326829637196063 ], [ 45.90391638987086, 4.32382549599472 ], [ 45.905667965693, 4.327990843251258 ], [ 45.91579311238129, 4.346817863136088 ], [ 45.91668891008069, 4.342257953265195 ], [ 45.9216251460626, 4.338982596932619 ], [ 45.92997139286699, 4.346628121156797 ], [ 45.93185900371628, 4.337155914633498 ], [ 45.94115045303277, 4.322661854063225 ], [ 45.9423274157955, 4.313001397538913 ], [ 45.95343445337902, 4.294525894838753 ], [ 45.95835570732176, 4.301188120177739 ], [ 45.9732026997347, 4.289431726797376 ], [ 45.97782302017173, 4.276356808225782 ], [ 45.97999822296752, 4.273060858987787 ], [ 45.98120074142516, 4.259590688670001 ], [ 45.98675497867428, 4.248658637365085 ], [ 45.99308572400004, 4.247947359178051 ], [ 45.99614759409149, 4.246662259181909 ], [ 45.99563177014321, 4.251765125404147 ], [ 45.99854856923263, 4.265285854935993 ], [ 45.99274541001228, 4.270233183990548 ], [ 45.99239729542708, 4.275045816032846 ], [ 45.99568744605374, 4.289609476470726 ], [ 45.98892048061472, 4.300698529315564 ], [ 45.99439301256719, 4.304534538221632 ], [ 46.0051569133273, 4.31261301303475 ], [ 46.00804632488756, 4.310942716001039 ], [ 46.01128401825687, 4.308859856852684 ], [ 46.02162453052691, 4.306458712603317 ], [ 46.0215031603917, 4.305677962950914 ], [ 46.02080232740203, 4.301223173083229 ], [ 46.01839338755868, 4.288344635543249 ], [ 46.0218025518577, 4.28062738017141 ], [ 46.02393104466132, 4.276598681060639 ], [ 46.03355588764234, 4.270436612377799 ], [ 46.03608569278646, 4.261172108339184 ], [ 46.03914954925174, 4.261574439856753 ], [ 46.04921372290215, 4.251694718008523 ], [ 46.05182172309517, 4.254034942879366 ], [ 46.05394712145236, 4.25770790411945 ], [ 46.05200674707666, 4.266837389513737 ], [ 46.05371908926419, 4.285713698620556 ], [ 46.06348629342862, 4.297946350027559 ], [ 46.06607892681861, 4.300652169094047 ], [ 46.07849083960166, 4.302039193299913 ], [ 46.07922531473847, 4.305599848572079 ], [ 46.08175515713837, 4.311015193331216 ], [ 46.08505691781212, 4.310620920782156 ], [ 46.09488700704484, 4.311938135622067 ], [ 46.10036561585792, 4.306608118283756 ], [ 46.11003344270973, 4.30990269743367 ], [ 46.11227779249233, 4.313417891784874 ], [ 46.11685288921163, 4.320328061893218 ], [ 46.13001463483668, 4.322725351722958 ], [ 46.13326362593953, 4.330293102924566 ], [ 46.13554160597837, 4.333348440257132 ], [ 46.13830957901551, 4.357233936672898 ], [ 46.13849351079732, 4.362195204890961 ], [ 46.1498366233474, 4.377795652104825 ], [ 46.14969360157965, 4.382423398963222 ], [ 46.14767758414561, 4.386051143732711 ], [ 46.13709181954217, 4.408851926159123 ], [ 46.13594976925808, 4.417961239075054 ], [ 46.15142016044436, 4.424267792344496 ], [ 46.15560425283834, 4.431770589521594 ], [ 46.16791483778646, 4.439193639737003 ], [ 46.16998904853596, 4.435772060175736 ], [ 46.18010788550735, 4.414915433192237 ], [ 46.18156481929947, 4.411150008217347 ], [ 46.18227866102438, 4.406966209179808 ], [ 46.18456671088968, 4.403509174410789 ], [ 46.18925424643544, 4.396783755671023 ], [ 46.19570191957066, 4.39747939068123 ], [ 46.20316836045577, 4.423571836902974 ], [ 46.20706171749928, 4.422359640985007 ], [ 46.21010077361071, 4.420471659265854 ], [ 46.21965844746768, 4.407356367902624 ], [ 46.21982737938801, 4.388683312501862 ], [ 46.22571105646608, 4.386829907900418 ], [ 46.23755075963317, 4.39209447339758 ], [ 46.24354546160409, 4.388785758384324 ], [ 46.24619830411654, 4.386173183144986 ], [ 46.24794900440396, 4.38889827660288 ], [ 46.2643415894465, 4.393582057210332 ], [ 46.26655827183746, 4.397449686073675 ], [ 46.27187809126278, 4.397064201352286 ], [ 46.27457798360896, 4.396974174839016 ], [ 46.28435539290889, 4.399371429518632 ], [ 46.29277972056357, 4.407707294944299 ], [ 46.29609636649197, 4.406422030172297 ], [ 46.29537513021594, 4.411216732267508 ], [ 46.29633635905063, 4.424994132856644 ], [ 46.30286120593534, 4.427650449880751 ], [ 46.29631496657498, 4.438330372797787 ], [ 46.29309133554136, 4.440062626728649 ], [ 46.29647617285935, 4.453855067913997 ], [ 46.29405157990985, 4.46279605745037 ], [ 46.28443417433566, 4.47602647504644 ], [ 46.28709961733546, 4.484382424854218 ], [ 46.28803138168911, 4.489069361733742 ], [ 46.27010728719618, 4.50209226866315 ], [ 46.26716933623024, 4.504596974106646 ], [ 46.2719961499778, 4.542944499553459 ], [ 46.27395480446275, 4.547049893978835 ], [ 46.2832619806662, 4.548685286189365 ], [ 46.29129615541792, 4.556044471210963 ], [ 46.29402942721709, 4.55844617134213 ], [ 46.29306430926767, 4.570871194910193 ], [ 46.29001185910924, 4.572758405594865 ], [ 46.27707566137231, 4.573323169325609 ], [ 46.27025349602457, 4.58326888257721 ], [ 46.26839674289919, 4.587119484978682 ], [ 46.26643179980633, 4.615050419767011 ], [ 46.2654086056003, 4.617672483965549 ], [ 45.80592709186426, 4.865072663474678 ], [ 45.80434184846744, 4.864499321786514 ], [ 45.80433950695036, 4.864498347340356 ], [ 45.80545594864911, 4.848595888972264 ], [ 45.76435319429793, 4.801430760920382 ], [ 45.71917418711324, 4.781774751613478 ], [ 45.68363470756887, 4.80870542554455 ], [ 45.80122758057101, 4.376352681950241 ] ], full error: {'ok': 0.0, 'errmsg': 'Secondary loops not contained by first exterior loop - secondary loops must be holes: [ [ 4.423298831017474, 45.92246510136637 ], [ 4.423272328729544, 45.9224124914541 ], [ 4.423296014293201, 45.92245885344253 ], [ 4.423298831017474, 45.92246510136637 ] ] first loop: [ [ 45.80122758057101, 4.376352681950241 ], [ 45.8044838524037, 4.376220111724892 ], [ 45.80502003601278, 4.384299007716068 ], [ 45.80303745929081, 4.388437890276315 ], [ 45.81747689026152, 4.380885175476993 ], [ 45.82029916312589, 4.38306756294014 ], [ 45.82485210019716, 4.37663747923372 ], [ 45.82534211272147, 4.381172591846549 ], [ 45.83402693020424, 4.384295932265717 ], [ 45.83750180635732, 4.391204268525621 ], [ 45.83984258047744, 4.387611887543129 ], [ 45.84600085935993, 4.383682442715087 ], [ 45.85031045846961, 4.391211993814656 ], [ 45.85985548506667, 4.396744466054351 ], [ 45.8651960481546, 4.395075656820409 ], [ 45.86464844812487, 4.390221929905438 ], [ 45.86619589736199, 4.361702458243602 ], [ 45.87693527428829, 4.343076054047009 ], [ 45.88006074780913, 4.340914771281566 ], [ 45.8901897554254, 4.337545959836689 ], [ 45.89739332872952, 4.326829637196063 ], [ 45.90391638987086, 4.32382549599472 ], [ 45.905667965693, 4.327990843251258 ], [ 45.91579311238129, 4.346817863136088 ], [ 45.91668891008069, 4.342257953265195 ], [ 45.9216251460626, 4.338982596932619 ], [ 45.92997139286699, 4.346628121156797 ], [ 45.93185900371628, 4.337155914633498 ], [ 45.94115045303277, 4.322661854063225 ], [ 45.9423274157955, 4.313001397538913 ], [ 45.95343445337902, 4.294525894838753 ], [ 45.95835570732176, 4.301188120177739 ], [ 45.9732026997347, 4.289431726797376 ], [ 45.97782302017173, 4.276356808225782 ], [ 45.97999822296752, 4.273060858987787 ], [ 45.98120074142516, 4.259590688670001 ], [ 45.98675497867428, 4.248658637365085 ], [ 45.99308572400004, 4.247947359178051 ], [ 45.99614759409149, 4.246662259181909 ], [ 45.99563177014321, 4.251765125404147 ], [ 45.99854856923263, 4.265285854935993 ], [ 45.99274541001228, 4.270233183990548 ], [ 45.99239729542708, 4.275045816032846 ], [ 45.99568744605374, 4.289609476470726 ], [ 45.98892048061472, 4.300698529315564 ], [ 45.99439301256719, 4.304534538221632 ], [ 46.0051569133273, 4.31261301303475 ], [ 46.00804632488756, 4.310942716001039 ], [ 46.01128401825687, 4.308859856852684 ], [ 46.02162453052691, 4.306458712603317 ], [ 46.0215031603917, 4.305677962950914 ], [ 46.02080232740203, 4.301223173083229 ], [ 46.01839338755868, 4.288344635543249 ], [ 46.0218025518577, 4.28062738017141 ], [ 46.02393104466132, 4.276598681060639 ], [ 46.03355588764234, 4.270436612377799 ], [ 46.03608569278646, 4.261172108339184 ], [ 46.03914954925174, 4.261574439856753 ], [ 46.04921372290215, 4.251694718008523 ], [ 46.05182172309517, 4.254034942879366 ], [ 46.05394712145236, 4.25770790411945 ], [ 46.05200674707666, 4.266837389513737 ], [ 46.05371908926419, 4.285713698620556 ], [ 46.06348629342862, 4.297946350027559 ], [ 46.06607892681861, 4.300652169094047 ], [ 46.07849083960166, 4.302039193299913 ], [ 46.07922531473847, 4.305599848572079 ], [ 46.08175515713837, 4.311015193331216 ], [ 46.08505691781212, 4.310620920782156 ], [ 46.09488700704484, 4.311938135622067 ], [ 46.10036561585792, 4.306608118283756 ], [ 46.11003344270973, 4.30990269743367 ], [ 46.11227779249233, 4.313417891784874 ], [ 46.11685288921163, 4.320328061893218 ], [ 46.13001463483668, 4.322725351722958 ], [ 46.13326362593953, 4.330293102924566 ], [ 46.13554160597837, 4.333348440257132 ], [ 46.13830957901551, 4.357233936672898 ], [ 46.13849351079732, 4.362195204890961 ], [ 46.1498366233474, 4.377795652104825 ], [ 46.14969360157965, 4.382423398963222 ], [ 46.14767758414561, 4.386051143732711 ], [ 46.13709181954217, 4.408851926159123 ], [ 46.13594976925808, 4.417961239075054 ], [ 46.15142016044436, 4.424267792344496 ], [ 46.15560425283834, 4.431770589521594 ], [ 46.16791483778646, 4.439193639737003 ], [ 46.16998904853596, 4.435772060175736 ], [ 46.18010788550735, 4.414915433192237 ], [ 46.18156481929947, 4.411150008217347 ], [ 46.18227866102438, 4.406966209179808 ], [ 46.18456671088968, 4.403509174410789 ], [ 46.18925424643544, 4.396783755671023 ], [ 46.19570191957066, 4.39747939068123 ], [ 46.20316836045577, 4.423571836902974 ], [ 46.20706171749928, 4.422359640985007 ], [ 46.21010077361071, 4.420471659265854 ], [ 46.21965844746768, 4.407356367902624 ], [ 46.21982737938801, 4.388683312501862 ], [ 46.22571105646608, 4.386829907900418 ], [ 46.23755075963317, 4.39209447339758 ], [ 46.24354546160409, 4.388785758384324 ], [ 46.24619830411654, 4.386173183144986 ], [ 46.24794900440396, 4.38889827660288 ], [ 46.2643415894465, 4.393582057210332 ], [ 46.26655827183746, 4.397449686073675 ], [ 46.27187809126278, 4.397064201352286 ], [ 46.27457798360896, 4.396974174839016 ], [ 46.28435539290889, 4.399371429518632 ], [ 46.29277972056357, 4.407707294944299 ], [ 46.29609636649197, 4.406422030172297 ], [ 46.29537513021594, 4.411216732267508 ], [ 46.29633635905063, 4.424994132856644 ], [ 46.30286120593534, 4.427650449880751 ], [ 46.29631496657498, 4.438330372797787 ], [ 46.29309133554136, 4.440062626728649 ], [ 46.29647617285935, 4.453855067913997 ], [ 46.29405157990985, 4.46279605745037 ], [ 46.28443417433566, 4.47602647504644 ], [ 46.28709961733546, 4.484382424854218 ], [ 46.28803138168911, 4.489069361733742 ], [ 46.27010728719618, 4.50209226866315 ], [ 46.26716933623024, 4.504596974106646 ], [ 46.2719961499778, 4.542944499553459 ], [ 46.27395480446275, 4.547049893978835 ], [ 46.2832619806662, 4.548685286189365 ], [ 46.29129615541792, 4.556044471210963 ], [ 46.29402942721709, 4.55844617134213 ], [ 46.29306430926767, 4.570871194910193 ], [ 46.29001185910924, 4.572758405594865 ], [ 46.27707566137231, 4.573323169325609 ], [ 46.27025349602457, 4.58326888257721 ], [ 46.26839674289919, 4.587119484978682 ], [ 46.26643179980633, 4.615050419767011 ], [ 46.2654086056003, 4.617672483965549 ], [ 45.80592709186426, 4.865072663474678 ], [ 45.80434184846744, 4.864499321786514 ], [ 45.80433950695036, 4.864498347340356 ], [ 45.80545594864911, 4.848595888972264 ], [ 45.76435319429793, 4.801430760920382 ], [ 45.71917418711324, 4.781774751613478 ], [ 45.68363470756887, 4.80870542554455 ], [ 45.80122758057101, 4.376352681950241 ] ]', 'code': 2, 'codeName': 'BadValue', 'operationTime': Timestamp(1611827959, 1), '$clusterTime': {'clusterTime': Timestamp(1611827959, 1), 'signature': {'hash': b'\x8d\xb6\x9b7\x0b\xe7!\x11\xe6\xc1\x13\x06\xf4\xe0\xca\xfe\xcbN\x0cy', 'keyId': 6872375198939611144}}}

Dimension of a cuboid from coordinates in a NumPy array

I have an array of 3D coordinates representing a cuboid and its projection in 2D:
"cuboid": [
[ 16.793300628662109, 12.848400115966797, 96.942703247070313 ],
[ 26.579500198364258, 6.4942998886108398, 97.793800354003906 ],
[ 24.501899719238281, 4.3057999610900879, 105.34329986572266 ],
[ 14.715700149536133, 10.659799575805664, 104.49220275878906 ],
[ 12.280900001525879, 5.444699764251709, 93.554603576660156 ],
[ 22.067100524902344, -0.90930002927780151, 94.405799865722656 ],
[ 19.989500045776367, -3.0978999137878418, 101.95529937744141 ],
[ 10.203300476074219, 3.2562000751495361, 101.10420227050781 ]
],
"projected_cuboid": [
[ 613.068115234375, 371.80960083007813 ],
[ 688.7794189453125, 321.01361083984375 ],
[ 658.66741943359375, 301.3988037109375 ],
[ 588.180908203125, 348.36520385742188 ],
[ 580.83660888671875, 314.70660400390625 ],
[ 659.55560302734375, 262.602294921875 ],
[ 630.6068115234375, 246.66099548339844 ],
[ 557.5220947265625, 294.74038696289063 ]
]
Is there a way to get the dimension (height, width, and length) of the cuboid by just having those values?

Single value decomposition(SVD) issue on 3D data (Python)

I have a trajectory which contains several frames of some 3D data, which looks like the following (I am posting the whole frame for the sake of reproducibilty of my problem):
data1= [[ 89.29, 57.35 , 107.81999 ],
[ 91.37 , 60.39 , 109.119995],
[ 89.130005 , 61.289997 , 112.12999 ],
[ 89.45 , 57.729996 ,113.509995],
[ 93.149994 , 58.199997 ,114.20999 ],
[ 92.03999 , 61.21 ,116.44 ],
[ 89.48999 , 58.71 ,117.979996],
[ 92.42 , 56.399998 ,118.84 ],
[ 94.25 , 59.14 ,120.73999 ],
[ 91.44 , 59.62 ,123.28999 ],
[ 91.25 , 55.739998 ,124. ],
[ 94.95 , 55.829998 ,124.67 ],
[ 94.36 , 58.699997 ,127.079994],
[ 91.5 , 57.05 ,129.05 ],
[ 93.16 , 53.57 ,129.28 ],
[ 96.36 , 55.35 ,130.5 ],
[ 94.15 , 57.53 ,133. ],
[ 92.24 , 54.42 ,134.18 ],
[ 95.479996 , 52.35 ,134.88 ],
[ 96.81 , 55.429996 ,136.77 ],
[ 93.509995 , 55.73 ,138.76 ],
[ 94.06999 , 51.98 ,139.5 ],
[ 97.63 , 52.929996 ,140.72 ],
[ 96.08 , 55.72 ,142.92 ],
[ 93.63999 , 53.269997 ,144.65 ],
[ 96.149994 , 50.45 ,144.79999 ],
[ 99.10999 , 52.1 ,146.4 ],
[ 97.369995 , 54.16 ,149. ],
[ 94.2 , 55.65 ,150.56 ]]
I want to fit the line to my data. After checking answers of previous question (1), I borrowed/wrote the following code :
def Range(data):
x=[]
y=[]
for i in range(0, len(data.T)):
x.append(np.mean(data.T[i])- np.std(data.T[i]))
y.append(np.mean(data.T[i]) + np.std(data.T[i]))
normx,normy=np.linalg.norm(x),np.linalg.norm(y)
minimum=normx-(normx+normy)/2
maximum=normy-(normx+normy)/2
return(minimum,maximum)
def axis(data):
minimum,maximum=Range(data)
datamean = data.mean(axis=0)
uu, dd, vv = np.linalg.svd(data - datamean)
linepts = vv[0] * np.mgrid[minimum:maximum:2j][:, np.newaxis]
linepts += datamean
return(linepts)
The value of vv[0] turns out to be [ 0.15970461,-0.17264067,0.9719515 ], which looks perfect for my further usage. However, for following the data :
data2= [ 90.993996 , 62.075 ,108.487 ],
[ 88.036995 , 59.8525 ,109.3855 ],
[ 90.5945 , 57.614998 ,111.061005],
[ 92.17 , 60.6205 ,112.681496],
[ 88.934006 , 61.622 ,114.4255 ],
[ 88.077995 , 57.929 ,115.34 ],
[ 91.642 , 57.3 ,116.81049 ],
[ 91.431496 , 60.4655 ,118.813 ],
[ 88.269 , 59.22 ,120.685 ],
[ 89.883995 , 55.7975 ,121.2585 ],
[ 93.115 , 57.497 ,122.68849 ],
[ 91.090004 , 59.724 ,125.11 ],
[ 89.355 , 56.712498 ,126.7305 ],
[ 92.6985 , 54.758 ,126.52 ],
[ 94.4685 , 57.383 ,128.4515 ],
[ 91.9065 , 57.297997 ,131.2145 ],
[ 91.99349 , 53.578995 ,131.1695 ],
[ 95.752495 , 53.376 , 132.0325 ],
[ 95.24799 , 55.990997, 134.80699 ],
[ 92.29199 , 54.0455 , 136.277 ],
[ 94.5055 , 50.9205 , 136.68399 ],
[ 97.5085 , 52.947 , 137.85399 ],
[ 95.353 , 54.6695 , 140.651 ],
[ 94.194496 , 51.2645 , 141.4345 ],
[ 97.6015 , 49.722 , 141.7245 ],
[ 99.26149 , 52.813496 ,143.35449 ],
[ 96.79849 , 53.233498, 146.2645 ],
[ 96.237 , 49.554 , 146.97299 ]]
The value of vv[0] turns out to be [-0.18894662 , 0.24432637,-0.9511066 ], which is in opposite sign from my expected result. I am unsure why it would produce such a result. Could anyone help me sort out this issue?
1 Fitting a line in 3D

The first principal component is a direction. Think of it as a line of infinite length. It doesn't matter if you flip this line, the direction remains the same.
In other words: the algorithm does not consider the order of the points that the object has travelled to. So for the algorithm, the object flying backwards through the points is just as valid as flying forwards.

How do I write an accumulator to squash an array around a midpoint?

Say I've got an array of arrays as follows:
array([[ 7108.4, -600. ],
[ 7101.3, -500. ],
[ 7094.3, -400. ],
[ 7087.2, -300. ],
[ 7080.1, -200. ],
[ 7073.1, -100. ],
[ 7058.3, 100. ],
[ 7051.3, 200. ],
[ 7044.2, 300. ],
[ 7037.1, 400. ],
[ 7030.1, 500. ],
[ 7023. , 600. ]])
This is a list of prices and quantities, with the positive numbers being buys and the negative numbers being sells.
If the current market price is 7050, I want to 'squash' my prices, as follows:
array([[ 7108.4, -600. ],
[ 7101.3, -500. ],
[ 7094.3, -400. ],
[ 7087.2, -300. ],
[ 7080.1, -200. ],
[ 7073.1, -100. ],
[ 7050. , 300. ], # Price & quantity changed here
[ 7044.2, 300. ],
[ 7037.1, 400. ],
[ 7030.1, 500. ],
[ 7023. , 600. ]])
and and do the same in thing in the other direction, so if the market price were 7085:
array([[ 7108.4, -600. ],
[ 7101.3, -500. ],
[ 7094.3, -400. ],
[ 7087.2, -300. ],
[ 7085. , -300. ], # Price & quantity change here
[ 7058.3, 100. ],
[ 7051.3, 200. ],
[ 7044.2, 300. ],
[ 7037.1, 400. ],
[ 7030.1, 500. ],
[ 7023. , 600. ]])
What's the cleanest way to write such a thing? Ideally, I'd like to write the squashing and the quantity grouping as two separate steps.

This is the kind of operations that are made easier by pandas.
>>> import numpy as np
>>> import pandas as pd
>>>
>>> t = np.array([[ 7108.4, -600. ],
... [ 7101.3, -500. ],
... [ 7094.3, -400. ],
... [ 7087.2, -300. ],
... [ 7080.1, -200. ],
... [ 7073.1, -100. ],
... [ 7058.3, 100. ],
... [ 7051.3, 200. ],
... [ 7044.2, 300. ],
... [ 7037.1, 400. ],
... [ 7030.1, 500. ],
... [ 7023. , 600. ]])
>>>
>>> df = pd.DataFrame(t, columns=['price', 'quantity'])
>>> (df.price > 7050) & (df.quantity > 0)
0 False
1 False
2 False
3 False
4 False
5 False
6 True
7 True
8 False
9 False
10 False
11 False
dtype: bool
>>> df[(df.price > 7050) & (df.quantity > 0)]
price quantity
6 7058.3 100.0
7 7051.3 200.0
>>> df[(df.price > 7050) & (df.quantity > 0)].quantity.sum()
300.0
>>> df[(df.price < 7085) & (df.quantity < 0)]
price quantity
4 7080.1 -200.0
5 7073.1 -100.0
>>> df[(df.price < 7085) & (df.quantity < 0)].quantity.sum()
-300.0
You could still stick with numpy if you want:
>>> import numpy as np
>>> import pandas as pd
>>>
>>> t = np.array([[ 7108.4, -600. ],
... [ 7101.3, -500. ],
... [ 7094.3, -400. ],
... [ 7087.2, -300. ],
... [ 7080.1, -200. ],
... [ 7073.1, -100. ],
... [ 7058.3, 100. ],
... [ 7051.3, 200. ],
... [ 7044.2, 300. ],
... [ 7037.1, 400. ],
... [ 7030.1, 500. ],
... [ 7023. , 600. ]])
>>>
>>>
>>> t[:,0]
array([ 7108.4, 7101.3, 7094.3, 7087.2, 7080.1, 7073.1, 7058.3,
7051.3, 7044.2, 7037.1, 7030.1, 7023. ])
>>> t[:,0] > 7050
array([ True, True, True, True, True, True, True, True, False,
False, False, False], dtype=bool)
>>> (t[:,0] > 7050) & (t[:,1] > 0)
array([False, False, False, False, False, False, True, True, False,
False, False, False], dtype=bool)
>>> t[(t[:,0] > 7050) & (t[:,1] > 0)]
array([[ 7058.3, 100. ],
[ 7051.3, 200. ]])
>>> t[(t[:,0] < 7085) & (t[:,1] < 0)]
array([[ 7080.1, -200. ],
[ 7073.1, -100. ]])
>>> t[(t[:,0] < 7085) & (t[:,1] < 0)][:,1]
array([-200., -100.])
>>> t[(t[:,0] < 7085) & (t[:,1] < 0)][:,1].sum()
-300.0

since Eric answer is great but as per your comment :
Is possible to modify the original array, so that the output is the
full list of orders with changes inside?
Here is my approach without numpy :
array=[[ 7108.4, -600. ],
[ 7101.3, -500. ],
[ 7094.3, -400. ],
[ 7087.2, -300. ],
[ 7080.1, -200. ],
[ 7073.1, -100. ],
[ 7058.3, 100. ],
[ 7051.3, 200. ],
[ 7044.2, 300. ],
[ 7037.1, 400. ],
[ 7030.1, 500. ],
[ 7023. , 600. ]]
price=0
positive_value=7050
updated_list=[]
track=[]
for index,item in enumerate(array):
if item[0]>positive_value:
if item[1]>0:
price+=item[1]
track.append(index)
else:
updated_list.append(item)
else:
updated_list.append(item)
updated_list.insert(track[0],[positive_value,price])
print(updated_list)
output:
[[7108.4, -600.0], [7101.3, -500.0], [7094.3, -400.0], [7087.2, -300.0], [7080.1, -200.0], [7073.1, -100.0], [7050, 300.0], [7044.2, 300.0], [7037.1, 400.0], [7030.1, 500.0], [7023.0, 600.0]]
for second one:
array=[[ 7108.4, -600. ],
[ 7101.3, -500. ],
[ 7094.3, -400. ],
[ 7087.2, -300. ],
[ 7080.1, -200. ],
[ 7073.1, -100. ],
[ 7058.3, 100. ],
[ 7051.3, 200. ],
[ 7044.2, 300. ],
[ 7037.1, 400. ],
[ 7030.1, 500. ],
[ 7023. , 600. ]]
price=0
positive_value=7085
updated_list=[]
track=[]
for index,item in enumerate(array):
if item[0] < positive_value:
if item[1] < 0:
price+=item[1]
track.append(index)
else:
updated_list.append(item)
else:
updated_list.append(item)
updated_list.insert(track[0],[positive_value,price])
print(updated_list)
output:
[[7108.4, -600.0], [7101.3, -500.0], [7094.3, -400.0], [7087.2, -300.0], [7085, -300.0], [7058.3, 100.0], [7051.3, 200.0], [7044.2, 300.0], [7037.1, 400.0], [7030.1, 500.0], [7023.0, 600.0]]

Issue with merging multiple JSON files in Python

I am trying to combine multiple JSON files in a Ubuntu platform. As an example, the data from two files are as follows:
File_1
{
"artist":"Gob",
"timestamp":"2011-08-09 01:59:41.352247",
"similars":[
[
"TRTOVWD128F92F4227",
1
],
[
"TRUXNUD128F92F41D0",
0.97294099999999994
],
[
"TRNNOJO128F42992E9",
0.073926900000000004
],
[
"TRGZHTT128F423B2A4",
0.068387699999999996
],
[
"TRGYKYD128F42625F6",
0.065579700000000005
],
[
"TRGIWHY128F42625F5",
0.064063700000000001
],
[
"TRJCJTX128F930CACE",
0.063140100000000005
],
[
"TRMYNWT128F426254B",
0.0613825
],
[
"TRRQOJI128F428C865",
0.061121599999999998
],
[
"TRBNYHM128F428A569",
0.061121599999999998
],
[
"TRDLOYE128F4241E72",
0.060951900000000003
],
[
"TRNRVEW12903CBA24F",
0.060332700000000003
],
[
"TRKKIPG12903CBA083",
0.060155
],
[
"TRZHTGP128F428A63B",
0.059873599999999999
],
[
"TRKQSGZ128F428A851",
0.059873599999999999
],
[
"TRTOPDF128F42AD88A",
0.059687799999999999
],
[
"TRIWOPM128F4241E53",
0.058958900000000002
],
[
"TRCCJUW128F14652DB",
0.057935
],
[
"TRERDDF128F428ECC4",
0.057566600000000002
],
[
"TROKWNN128F421A3D8",
0.057379800000000002
],
[
"TRWGOOK128F42AE765",
0.057125000000000002
],
[
"TRFMNKP128F428ADC0",
0.056875099999999998
],
[
"TRDMLZT128F42A01A8",
0.055808900000000002
],
[
"TRGCJVM128E0780E48",
0.0547389
],
[
"TRRXGAY128F14652D7",
0.0538065
],
[
"TRIPEHH128F1462DFF",
0.052843000000000001
],
[
"TRDUOIP128F147D5A7",
0.051851500000000002
],
[
"TRZCHHD12903CC80A1",
0.051251699999999997
],
[
"TRFDDQS128F426243F",
0.051018300000000003
],
[
"TRZDKAR128F42591B8",
0.050740899999999999
],
[
"TRDVXUG128F1456CBF",
0.050486299999999998
],
[
"TRULRYN128F145FC1C",
0.050219800000000002
],
[
"TRMOWIA128F425CE0F",
0.049977500000000001
],
[
"TRUVPMZ128F42B6DF3",
0.049762000000000001
],
[
"TRSBDWW128F4262666",
0.049643699999999999
],
[
"TRKPHWQ128F4264F8C",
0.0495173
],
[
"TRBBLXU128F42623A1",
0.049416700000000001
],
[
"TRJKLLM128F1456C57",
0.049001599999999999
],
[
"TRSAAEI128F4216C24",
0.048813500000000003
],
[
"TRFXICT128F4264F8A",
0.048776199999999999
],
[
"TRINVLH12903CBE5A1",
0.048334500000000002
],
[
"TRMUUJR128F4262475",
0.048306500000000002
],
[
"TRTORTD128F1456AFA",
0.0468265
],
[
"TRECUJO12903CA7120",
0.046065599999999998
],
[
"TRXIRBQ128F93431BB",
0.0456938
],
[
"TRFDDVK128F42B6DF0",
0.045623799999999999
],
[
"TRSRGPM128F421A30B",
0.043976800000000003
],
[
"TRVUPPR128F429507D",
0.042872500000000001
],
[
"TRMHCZC128F428A4CD",
0.040675200000000002
],
[
"TRUFDRV128F4262352",
0.040675200000000002
],
[
"TRUZZHT128F93229AF",
0.039422199999999998
],
[
"TRLSIHL128F429AF18",
0.039002099999999998
],
[
"TRGETCK128F1460DB1",
0.038499499999999999
],
[
"TRSXXNU128F428AEF2",
0.038303799999999999
],
[
"TRFZXSY128F9330D9F",
0.037855199999999999
],
[
"TRPHFYF128F92F27FA",
0.037772100000000003
],
[
"TRNRHSL128F9337B55",
0.036998000000000003
],
[
"TRPTGNZ128F421A56B",
0.036713099999999999
],
[
"TRPAASI128F9337B6E",
0.036410499999999998
],
[
"TRGCROO128F93431C4",
0.035754300000000003
],
[
"TRCUHZL128F4235446",
0.034968699999999998
],
[
"TRDPOTJ128F429AF0C",
0.034860500000000003
],
[
"TROZUXM128F42790A2",
0.0346483
],
[
"TRJVLOQ128F9345A82",
0.034547799999999997
],
[
"TRQTFRP128F145FC1E",
0.033934600000000002
],
[
"TRQEWHR128F421A3F5",
0.032314599999999999
],
[
"TRNTPJA128F4265039",
0.030702900000000002
],
[
"TRDGXWY12903CF52BD",
0.030292300000000001
],
[
"TRBLEMZ128F93102D0",
0.029224300000000002
],
[
"TRBUUYO128F421A405",
0.028448500000000002
],
[
"TREVBDI12903CED7E6",
0.0279674
],
[
"TRKREBF128F429B317",
0.0258321
],
[
"TRZBYPR128F4233A8D",
0.025655000000000001
],
[
"TRTAZUQ12903CFEA78",
0.024545399999999998
],
[
"TRAIPRO128F429AE69",
0.024304699999999999
],
[
"TRTTVUZ128F92FADD3",
0.023320899999999999
],
[
"TRUYEJI128F4265041",
0.022173700000000001
],
[
"TRAXVGT128F9344507",
0.0213992
],
[
"TRJJBLH128F4260DA1",
0.0175365
],
[
"TRAMCWR128F4233F7F",
0.0161158
],
[
"TRXBLME128F424330F",
0.015760900000000001
],
[
"TRMUQXM128F4260D99",
0.015696000000000002
],
[
"TRHRZBJ128EF345514",
0.0156951
],
[
"TRJXIBT128F42454DB",
0.014519199999999999
],
[
"TRTHPOY128F9345AA5",
0.0137264
],
[
"TRRFGJU128F933B2E6",
0.0012336199999999999
],
[
"TRMYJUA128F428A590",
0.00123149
],
[
"TRNMVTE128F933B2EC",
0.00122703
],
[
"TRYALZM128F1483C7D",
0.0012245299999999999
],
[
"TRZVEJU128F4234F4E",
0.00121805
],
[
"TRQAZDO128F145639F",
0.0012166600000000001
],
[
"TRJXNJM12903CF57ED",
0.0012155
],
[
"TRVAOGO128F427C9D6",
0.00120951
],
[
"TRZMZDS128F422843B",
0.0012065000000000001
],
[
"TRXIEOF12903CE8212",
0.0012058699999999999
],
[
"TRPVVUG128F42A36AA",
0.0012057599999999999
],
[
"TRXGVXS128F428AA5C",
0.0012019400000000001
],
[
"TRUBOGF128E078A5B9",
0.0012017900000000001
],
[
"TRITZSB128F4277CC2",
0.0012014
],
[
"TRGHPHX128F9343544",
0.0011975600000000001
],
[
"TRUKWPE128F428114F",
0.00119666
],
[
"TROBGRB128F93229AB",
0.0011964199999999999
],
[
"TRGKTMW12903CFAE65",
0.00119637
]
],
"tags":[
[
"punk rock",
"100"
],
[
"punk",
"60"
]
],
"track_id":"TRAAAFD128F92F423A",
"title":"Face the Ashes"
}
File_2
{
"artist":"CLP",
"timestamp":"2011-08-02 06:36:59.879759",
"similars":[
],
"tags":[
],
"track_id":"TRAAAVG12903CFA543",
"title":"Insatiable (Instrumental Version)"
}
I wrote a Python script to combine them. I added a new line and a comma after each record.
import glob
read_files = glob.glob("*.json")
with open("merged_file.json", "wb") as outfile:
for f in read_files:
with open(f, "rb") as infile:
outfile.write(infile.read())
outfile.write(',\n')
The output from the merge file is:
{
"artist":"Gob",
"timestamp":"2011-08-09 01:59:41.352247",
"similars":[
[
"TRTOVWD128F92F4227",
1
],
[
"TRUXNUD128F92F41D0",
0.97294099999999994
],
[
"TRNNOJO128F42992E9",
0.073926900000000004
],
[
"TRGZHTT128F423B2A4",
0.068387699999999996
],
[
"TRGYKYD128F42625F6",
0.065579700000000005
],
[
"TRGIWHY128F42625F5",
0.064063700000000001
],
[
"TRJCJTX128F930CACE",
0.063140100000000005
],
[
"TRMYNWT128F426254B",
0.0613825
],
[
"TRRQOJI128F428C865",
0.061121599999999998
],
[
"TRBNYHM128F428A569",
0.061121599999999998
],
[
"TRDLOYE128F4241E72",
0.060951900000000003
],
[
"TRNRVEW12903CBA24F",
0.060332700000000003
],
[
"TRKKIPG12903CBA083",
0.060155
],
[
"TRZHTGP128F428A63B",
0.059873599999999999
],
[
"TRKQSGZ128F428A851",
0.059873599999999999
],
[
"TRTOPDF128F42AD88A",
0.059687799999999999
],
[
"TRIWOPM128F4241E53",
0.058958900000000002
],
[
"TRCCJUW128F14652DB",
0.057935
],
[
"TRERDDF128F428ECC4",
0.057566600000000002
],
[
"TROKWNN128F421A3D8",
0.057379800000000002
],
[
"TRWGOOK128F42AE765",
0.057125000000000002
],
[
"TRFMNKP128F428ADC0",
0.056875099999999998
],
[
"TRDMLZT128F42A01A8",
0.055808900000000002
],
[
"TRGCJVM128E0780E48",
0.0547389
],
[
"TRRXGAY128F14652D7",
0.0538065
],
[
"TRIPEHH128F1462DFF",
0.052843000000000001
],
[
"TRDUOIP128F147D5A7",
0.051851500000000002
],
[
"TRZCHHD12903CC80A1",
0.051251699999999997
],
[
"TRFDDQS128F426243F",
0.051018300000000003
],
[
"TRZDKAR128F42591B8",
0.050740899999999999
],
[
"TRDVXUG128F1456CBF",
0.050486299999999998
],
[
"TRULRYN128F145FC1C",
0.050219800000000002
],
[
"TRMOWIA128F425CE0F",
0.049977500000000001
],
[
"TRUVPMZ128F42B6DF3",
0.049762000000000001
],
[
"TRSBDWW128F4262666",
0.049643699999999999
],
[
"TRKPHWQ128F4264F8C",
0.0495173
],
[
"TRBBLXU128F42623A1",
0.049416700000000001
],
[
"TRJKLLM128F1456C57",
0.049001599999999999
],
[
"TRSAAEI128F4216C24",
0.048813500000000003
],
[
"TRFXICT128F4264F8A",
0.048776199999999999
],
[
"TRINVLH12903CBE5A1",
0.048334500000000002
],
[
"TRMUUJR128F4262475",
0.048306500000000002
],
[
"TRTORTD128F1456AFA",
0.0468265
],
[
"TRECUJO12903CA7120",
0.046065599999999998
],
[
"TRXIRBQ128F93431BB",
0.0456938
],
[
"TRFDDVK128F42B6DF0",
0.045623799999999999
],
[
"TRSRGPM128F421A30B",
0.043976800000000003
],
[
"TRVUPPR128F429507D",
0.042872500000000001
],
[
"TRMHCZC128F428A4CD",
0.040675200000000002
],
[
"TRUFDRV128F4262352",
0.040675200000000002
],
[
"TRUZZHT128F93229AF",
0.039422199999999998
],
[
"TRLSIHL128F429AF18",
0.039002099999999998
],
[
"TRGETCK128F1460DB1",
0.038499499999999999
],
[
"TRSXXNU128F428AEF2",
0.038303799999999999
],
[
"TRFZXSY128F9330D9F",
0.037855199999999999
],
[
"TRPHFYF128F92F27FA",
0.037772100000000003
],
[
"TRNRHSL128F9337B55",
0.036998000000000003
],
[
"TRPTGNZ128F421A56B",
0.036713099999999999
],
[
"TRPAASI128F9337B6E",
0.036410499999999998
],
[
"TRGCROO128F93431C4",
0.035754300000000003
],
[
"TRCUHZL128F4235446",
0.034968699999999998
],
[
"TRDPOTJ128F429AF0C",
0.034860500000000003
],
[
"TROZUXM128F42790A2",
0.0346483
],
[
"TRJVLOQ128F9345A82",
0.034547799999999997
],
[
"TRQTFRP128F145FC1E",
0.033934600000000002
],
[
"TRQEWHR128F421A3F5",
0.032314599999999999
],
[
"TRNTPJA128F4265039",
0.030702900000000002
],
[
"TRDGXWY12903CF52BD",
0.030292300000000001
],
[
"TRBLEMZ128F93102D0",
0.029224300000000002
],
[
"TRBUUYO128F421A405",
0.028448500000000002
],
[
"TREVBDI12903CED7E6",
0.0279674
],
[
"TRKREBF128F429B317",
0.0258321
],
[
"TRZBYPR128F4233A8D",
0.025655000000000001
],
[
"TRTAZUQ12903CFEA78",
0.024545399999999998
],
[
"TRAIPRO128F429AE69",
0.024304699999999999
],
[
"TRTTVUZ128F92FADD3",
0.023320899999999999
],
[
"TRUYEJI128F4265041",
0.022173700000000001
],
[
"TRAXVGT128F9344507",
0.0213992
],
[
"TRJJBLH128F4260DA1",
0.0175365
],
[
"TRAMCWR128F4233F7F",
0.0161158
],
[
"TRXBLME128F424330F",
0.015760900000000001
],
[
"TRMUQXM128F4260D99",
0.015696000000000002
],
[
"TRHRZBJ128EF345514",
0.0156951
],
[
"TRJXIBT128F42454DB",
0.014519199999999999
],
[
"TRTHPOY128F9345AA5",
0.0137264
],
[
"TRRFGJU128F933B2E6",
0.0012336199999999999
],
[
"TRMYJUA128F428A590",
0.00123149
],
[
"TRNMVTE128F933B2EC",
0.00122703
],
[
"TRYALZM128F1483C7D",
0.0012245299999999999
],
[
"TRZVEJU128F4234F4E",
0.00121805
],
[
"TRQAZDO128F145639F",
0.0012166600000000001
],
[
"TRJXNJM12903CF57ED",
0.0012155
],
[
"TRVAOGO128F427C9D6",
0.00120951
],
[
"TRZMZDS128F422843B",
0.0012065000000000001
],
[
"TRXIEOF12903CE8212",
0.0012058699999999999
],
[
"TRPVVUG128F42A36AA",
0.0012057599999999999
],
[
"TRXGVXS128F428AA5C",
0.0012019400000000001
],
[
"TRUBOGF128E078A5B9",
0.0012017900000000001
],
[
"TRITZSB128F4277CC2",
0.0012014
],
[
"TRGHPHX128F9343544",
0.0011975600000000001
],
[
"TRUKWPE128F428114F",
0.00119666
],
[
"TROBGRB128F93229AB",
0.0011964199999999999
],
[
"TRGKTMW12903CFAE65",
0.00119637
]
],
"tags":[
[
"punk rock",
"100"
],
[
"punk",
"60"
]
],
"track_id":"TRAAAFD128F92F423A",
"title":"Face the Ashes"
},
{
"artist":"CLP",
"timestamp":"2011-08-02 06:36:59.879759",
"similars":[
],
"tags":[
],
"track_id":"TRAAAVG12903CFA543",
"title":"Insatiable (Instrumental Version)"
}
When I validate these records using JSON Lint (http://jsonlint.com/), it tells me that the file is broken and not a valid JSON. even after spending quite sometime, I am not able to figure out what is going wrong with the merging. It would be helpful is anyone has any thoughts on this.

You can't just concatenate two JSON strings to make valid JSON (or combine them by tacking ',\n' to the end of each).
Instead, you could combine the two (as Python objects) into a Python list, then use json.dump to write it to a file as JSON:
import json
import glob
result = []
for f in glob.glob("*.json"):
with open(f, "rb") as infile:
result.append(json.load(infile))
with open("merged_file.json", "wb") as outfile:
json.dump(result, outfile)
If you wanted to do it without the (unnecesssary) intermediate step of parsing each JSON file, you could merge them into a list like this:
import glob
read_files = glob.glob("*.json")
with open("merged_file.json", "wb") as outfile:
outfile.write('[{}]'.format(
','.join([open(f, "rb").read() for f in read_files])))

There is a module called jsonmerge which merges dictionaries. It can be used very simple by just providing two dictionaries or you can define schema's that described how to merge, like instead of overwriting same key's, automatically create a list and append to it.
base = {
"foo": 1,
"bar": [ "one" ],
}
head = {
"bar": [ "two" ],
"baz": "Hello, world!"
}
from jsonmerge import merge
result = merge(base, head)
print(result)
>>> {'foo': 1, 'bar': ['two'], 'baz': 'Hello, world!'}
More examples with complex rules: https://pypi.org/project/jsonmerge/#description

If you wanted to produce a JSON list of those objects, you are missing opening [ and closing ] brackets here, and will have written one comma too many.
It'll be easier to have Python decode the objects, then write the output as a new JSON list:
import json
import glob
read_files = glob.glob("*.json")
output_list = []
for f in read_files:
with open(f, "rb") as infile:
output_list.append(json.load(infile))
with open("merged_file.json", "wb") as outfile:
json.dump(output_list, outfile)

Python solution:
You have file1.json and file2.json files.
Each file has structure:
[{"key1": "value1"}] - (in file1)
[{"key2": "value2"}] - (in file2)
And your goal to merge them and get next view:
[{"key1": "value1"},
{"key2": "value2"}]
Code:
import glob
glob_data = []
for file in glob.glob('../../file*.json'):
with open(file) as json_file:
data = json.load(json_file)
i = 0
while i < len(data):
glob_data.append(data[i])
i += 1
with open('../../finalFile.json', 'w') as f:
json.dump(glob_data, f, indent=4)

Your generated JSON isn't valid. You end up with something like:
{...}, {...}
which, of course, is not valid.
If you want to go with your approach, you should remove the last } and the first { from the new file, and then merge them (you still have to be sure that the JSON files you want to merge are in a valid format).
I suggest I use the json module for what you need to achieve, it is cleaner.

Categories

Resources