Related
I am having some trouble when trying to load geojson data to Athena. I am trying to load world data boundaries (from this link)
The thing is, for some reason AWS Athena does not let me load MultiPolygon countries data (loading done based on this), only Polygons. As a result of this, only the first Polygon of a MultiPolygon is loaded.
Because of this, I am trying to find another solution to load the data. I have thought of the following:
Try to build a DataFrame with the name of the country, the list of coordinates and the corresponding type (Polygon or MultiPolygon). Depending on the type, generate a new column in wkb hex format accepted by Athena. Generate a partitioned parquet file and try to load it into Athena.
Any thoughts on how to do this?
Coordinates are coming in this format:
Polygon:
[[[-17.102499999999907, 32.823330000000055], [-17.05305999999996, 32.80944000000005], [-17.029999999999916, 32.810560000000066], [-17.013199999999927, 32.81346000000008], [-16.959999999999923, 32.830820000000074], [-16.913329999999917, 32.83916000000005], [-16.902219999999943, 32.83791000000008], [-16.715559999999925, 32.758890000000065], [-16.720839999999953, 32.74528000000004], [-16.819579999999917, 32.64611000000008], [-16.82188999999994, 32.64435000000009], [-16.83916999999991, 32.638610000000085], [-16.94360999999992, 32.637500000000045], [-16.988329999999905, 32.65527000000009], [-17.067779999999914, 32.67694000000006], [-17.102499999999907, 32.68333000000007], [-17.15805999999992, 32.709160000000054], [-17.19596999999993, 32.72888000000006], [-17.206389999999942, 32.73750000000007], [-17.232779999999934, 32.77000000000004], [-17.239169999999945, 32.77833000000004], [-17.254519999999957, 32.81284000000005], [-17.190699999999936, 32.86861000000005], [-17.169719999999927, 32.87028000000004], [-17.15888999999993, 32.865270000000066], [-17.151949999999943, 32.85750000000007], [-17.13333999999992, 32.838880000000074], [-17.12527999999992, 32.83194000000009], [-17.102499999999907, 32.823330000000055]]]
MultiPolygon:
[[[[111.41152000000005, 2.376390000000071], [111.36804000000006, 2.359580000000051], [111.35081000000008, 2.371940000000052], [111.30393000000004, 2.450000000000045], [111.30554000000006, 2.466390000000046], [111.31184000000007, 2.497220000000084], [111.30497000000003, 2.594440000000077], [111.29525000000007, 2.6808300000000713], [111.29332000000005, 2.73333000000008], [111.29540000000003, 2.748470000000054], [111.30359000000004, 2.767640000000085], [111.31248000000005, 2.775830000000041], [111.32895000000008, 2.780350000000055], [111.34539000000007, 2.771810000000073], [111.35443000000004, 2.760000000000047], [111.37776000000008, 2.708050000000071], [111.38165000000004, 2.698330000000055], [111.38388000000003, 2.680000000000062], [111.38388000000003, 2.652500000000088], [111.38026000000008, 2.615000000000066], [111.37693000000007, 2.536110000000064], [111.37692000000004, 2.495280000000093], [111.37831000000006, 2.482780000000048], [111.38109000000003, 2.471940000000074], [111.39943000000005, 2.408050000000059], [111.40583000000004, 2.393890000000056], [111.41152000000005, 2.376390000000071]]], [[[104.21191000000005, 2.7113900000000513], [104.16775000000007, 2.705280000000073], [104.15554000000003, 2.710550000000069], [104.14165000000003, 2.730830000000082], [104.12941000000006, 2.753890000000069], [104.12497000000008, 2.770000000000038], [104.12329000000005, 2.781670000000076], [104.16948000000008, 2.892500000000041], [104.18385000000006, 2.879440000000045], [104.18794000000003, 2.86779000000007], [104.18941000000007, 2.866390000000081], [104.19912000000005, 2.841940000000079], [104.20415000000003, 2.826670000000092], [104.21719000000007, 2.785560000000089], [104.21970000000005, 2.774440000000083], [104.22073000000006, 2.721870000000081], [104.21191000000005, 2.7113900000000513]]], [[[117.6869200000001, 4.168340000000057], [117.64373, 4.212990000000047], [117.63890000000004, 4.22861000000006], [117.64972, 4.238050000000044], [117.68581000000006, 4.259880000000066], [117.72388000000001, 4.260000000000048], [117.7460900000001, 4.258330000000058], [117.7602700000001, 4.255280000000084], [117.76971000000003, 4.25111000000004], [117.78554000000008, 4.239440000000059], [117.90356000000008, 4.174040000000048], [117.87944000000005, 4.173050000000046], [117.83971000000008, 4.171940000000063], [117.72664000000009, 4.169720000000041], [117.6869200000001, 4.168340000000057]]], [[[118.59604000000002, 4.638330000000053], [118.5681800000001, 4.599440000000072], [118.52970000000005, 4.600280000000055], [118.34987000000001, 4.67205000000007], [118.39462000000003, 4.676050000000089], [118.47887000000003, 4.689170000000047], [118.57640000000004, 4.650830000000042], [118.59604000000002, 4.638330000000053]]], [[[100.31762000000003, 5.335420000000056], [100.28704000000005, 5.254440000000045], [100.20386000000008, 5.271110000000078], [100.19609000000003, 5.293330000000083], [100.17968000000008, 5.427500000000066], [100.18580000000003, 5.462290000000053], [100.24774000000008, 5.466670000000079], [100.26110000000006, 5.466940000000079], [100.27275000000003, 5.464720000000057], [100.29413000000005, 5.458050000000071], [100.30525000000006, 5.451940000000093], [100.30827000000005, 5.4461200000000645], [100.32025000000004, 5.427780000000041], [100.32469000000003, 5.38083000000006], [100.31762000000003, 5.335420000000056]]], [[[99.86954000000003, 6.419300000000078], [99.89386000000007, 6.4019400000000815], [99.91545000000008, 6.387080000000083], [99.92302000000007, 6.333610000000078], [99.87162000000006, 6.288190000000043], [99.73956000000004, 6.248890000000074], [99.65694000000008, 6.360830000000078], [99.65221000000008, 6.369720000000086], [99.64695000000006, 6.385000000000048], [99.64193000000006, 6.407220000000052], [99.64222000000007, 6.422080000000051], [99.70332000000008, 6.426110000000051], [99.71666000000005, 6.425000000000068], [99.72747000000004, 6.4222200000000385], [99.73220000000003, 6.413050000000055], [99.74942000000004, 6.407780000000059], [99.79413000000005, 6.411670000000072], [99.85295000000008, 6.464150000000075], [99.85913000000005, 6.436390000000074], [99.86954000000003, 6.419300000000078]]], [[[102.09523000000007, 6.236140000000091], [102.12302000000005, 6.218050000000062], [102.16666000000004, 6.193610000000092], [102.18588000000005, 6.205730000000074], [102.22165000000007, 6.217500000000086], [102.31303000000003, 6.189440000000047], [102.33388000000008, 6.175550000000044], [102.35860000000008, 6.151670000000081], [102.38540000000006, 6.116530000000068], [102.41193000000004, 6.070830000000058], [102.43274000000008, 6.020000000000039], [102.48803000000004, 5.902640000000076], [102.50166000000007, 5.882080000000087], [102.53972000000005, 5.853170000000091], [102.58136000000007, 5.828610000000083], [102.60524000000004, 5.812220000000082], [102.62345000000005, 5.795690000000093], [102.64609000000007, 5.763470000000041], [102.66573000000005, 5.729860000000087], [102.84220000000005, 5.589300000000037], [102.87690000000003, 5.56833000000006], [102.92135000000007, 5.547430000000077], [102.96087000000006, 5.537010000000066], [103.03775000000007, 5.47694000000007], [103.06442000000004, 5.44805000000008], [103.09607000000005, 5.410280000000057], [103.12191000000007, 5.377360000000067], [103.18192000000005, 5.282780000000059], [103.20832000000007, 5.240550000000042], [103.22803000000005, 5.205550000000073], [103.24025000000006, 5.176940000000059], [103.24802000000005, 5.157500000000084], [103.26554000000004, 5.113330000000076], [103.27913000000007, 5.086110000000076], [103.35608000000008, 4.949170000000038], [103.40998000000008, 4.858050000000048], [103.44107000000008, 4.765280000000075], [103.45497000000006, 4.626940000000047], [103.45380000000006, 4.484790000000089], [103.46498000000003, 4.411670000000072], [103.47635000000008, 4.374720000000082], [103.48553000000004, 4.349860000000092], [103.49371000000008, 4.308750000000089], [103.48650000000004, 4.280550000000062], [103.46637000000004, 4.234440000000063], [103.44656000000003, 4.16644000000008], [103.41414000000003, 4.150760000000048], [103.39595000000003, 4.111390000000085], [103.39413000000008, 4.085000000000093], [103.40107000000006, 4.057780000000093], [103.40912000000003, 4.032220000000052], [103.41385000000008, 3.958890000000053], [103.37636000000003, 3.863060000000075], [103.33636000000007, 3.74410000000006], [103.37108000000006, 3.642780000000073], [103.38498000000004, 3.622220000000084], [103.42497000000003, 3.57278000000008], [103.44830000000007, 3.548060000000077], [103.46303000000006, 3.531810000000064], [103.47643000000005, 3.498680000000092], [103.45775000000003, 3.472780000000057], [103.44163000000003, 3.440140000000042], [103.42636000000005, 3.392500000000041], [103.42580000000004, 3.371670000000051], [103.42719000000005, 3.329720000000065], [103.43065000000007, 3.309720000000084], [103.43941000000007, 3.280560000000093], [103.44442000000004, 3.258330000000057], [103.45137000000005, 3.218330000000037], [103.45442000000003, 3.180970000000059], [103.45137000000005, 3.148610000000076], [103.44551000000007, 3.126530000000059], [103.43469000000005, 3.052780000000041], [103.43247000000008, 2.962220000000059], [103.43802000000005, 2.925830000000076], [103.45135000000005, 2.888190000000065], [103.46942000000007, 2.857780000000048], [103.48246000000006, 2.8366700000000833], [103.50165000000004, 2.808050000000094], [103.52246000000008, 2.780830000000094], [103.55469000000005, 2.7443100000000413], [103.59220000000005, 2.706110000000079], [103.61885000000007, 2.680560000000071], [103.63702000000006, 2.6640900000000443], [103.65331000000003, 2.660830000000089], [103.72775000000007, 2.640000000000043], [103.76526000000007, 2.625830000000064], [103.82025000000004, 2.575900000000047], [103.83386000000007, 2.517360000000053], [103.82706000000007, 2.476390000000037], [103.83637000000004, 2.455140000000085], [103.89832000000007, 2.385830000000055], [103.97746000000006, 2.243050000000039], [104.01969000000003, 2.136670000000038], [104.05830000000003, 2.059170000000051], [104.11580000000004, 1.966110000000071], [104.18830000000003, 1.8050000000000632], [104.22331000000003, 1.7179200000000492], [104.25359000000003, 1.633330000000057], [104.29329000000007, 1.437780000000088], ...]], [[[117.4333200000001, 6.628330000000062], [117.41235000000006, 6.6258300000000645], [117.35068000000001, 6.6409700000000385], [117.33859000000007, 6.649440000000084], [117.33527000000004, 6.658890000000042], [117.34274000000005, 6.672530000000052], [117.3986000000001, 6.676940000000059], [117.41278, 6.680000000000064], [117.42720000000008, 6.686110000000042], [117.43776000000003, 6.69264000000004], [117.44456000000002, 6.706110000000081], [117.43859000000009, 6.718330000000037], [117.42302000000007, 6.721670000000074], [117.40332000000001, 6.723120000000051], [117.40012000000002, 6.736800000000073], [117.4104000000001, 6.7470800000000395], [117.4679000000001, 6.761390000000063], [117.49192000000005, 6.744300000000067], [117.51486, 6.705830000000049], [117.51555000000008, 6.689440000000047], [117.51277000000005, 6.678610000000049], [117.50074000000006, 6.663330000000087], [117.46693000000005, 6.642500000000041], [117.45388000000003, 6.635000000000048], [117.44414000000006, 6.631110000000092], [117.4333200000001, 6.628330000000062]]], [[[117.59206000000006, 4.1698200000000725], [117.5288700000001, 4.175280000000043], [117.49775, 4.178890000000081], [117.47554000000002, 4.183610000000044], [117.45276000000001, 4.18861000000004], [117.43943000000002, 4.195830000000058], [117.42442000000005, 4.214720000000057], [117.4202600000001, 4.224720000000047], [117.41081000000008, 4.242500000000064], [117.40179, 4.254580000000089], [117.25179000000003, 4.353890000000092], [117.23944000000006, 4.35833000000008], [117.22360000000003, 4.358750000000043], [117.21111000000008, 4.354440000000068], [117.2027700000001, 4.345970000000079], [117.19179000000008, 4.336250000000064], [117.17665000000011, 4.335550000000069], [117.04387000000008, 4.336940000000084], [116.9369200000001, 4.350830000000087], [116.92442000000005, 4.352220000000045], [116.91053000000011, 4.352220000000045], [116.88916000000006, 4.349300000000085], [116.87693000000002, 4.345280000000059], [116.83985000000007, 4.33042000000006], [116.68877000000009, 4.33075000000008], [116.64194000000009, 4.334720000000061], [116.53360000000009, 4.330830000000049], [116.52762000000007, 4.319300000000055], [116.46582000000001, 4.294030000000077], [116.45416, 4.294580000000053], [116.4416500000001, 4.299440000000061], [116.42581000000007, 4.310000000000059], [116.31470000000002, 4.357780000000048], [116.15332000000001, 4.355000000000075], [116.1402700000001, 4.337220000000059], [116.07416, 4.283610000000067], [116.06110000000001, 4.2779200000000515], [116.04762000000005, 4.281110000000069], [116.03088000000002, 4.300760000000082], [116.0041500000001, 4.330000000000041], [115.98665000000005, 4.339720000000057], [115.97609, 4.343050000000062], [115.88804000000005, 4.36812000000009], [115.87248, 4.361110000000053], [115.85755000000006, 4.344100000000083], [115.85054000000002, 4.32167000000004], [115.7702700000001, 4.244720000000086], [115.76139, 4.239720000000091], [115.69081000000006, 4.180280000000039], [115.68331, 4.167360000000087], [115.6597200000001, 4.108600000000081], [115.66110000000003, 4.097780000000057], [115.66249000000005, 4.078330000000051], [115.61638000000005, 3.854170000000067], [115.58528000000001, 3.741670000000056], [115.57693000000006, 3.708610000000078], [115.57139000000006, 3.66611000000006], [115.57138000000009, 3.612780000000043], [115.57416, 3.594720000000052], [115.57887000000005, 3.585550000000069], [115.60193000000004, 3.539170000000069], [115.62608, 3.45778000000007], [115.62769000000003, 3.434300000000064], [115.61387000000002, 3.420420000000092], [115.59693000000004, 3.424720000000093], [115.58297000000005, 3.427990000000079], [115.57083, 3.4177800000000502], [115.55887000000007, 3.389170000000092], [115.55331000000001, 3.373890000000074], [115.53888000000006, 3.333610000000078], [115.53415000000007, 3.31833000000006], [115.52998000000002, 3.301670000000058], [115.51193, 3.2100000000000932], [115.50998000000004, 3.198330000000055], [115.51305000000002, 3.184030000000064], [115.52859000000001, 3.176110000000051], [115.4988800000001, 3.050280000000043], [115.49553000000003, 3.040000000000077], [115.48720000000003, 3.027500000000088], [115.37804000000006, 2.991800000000069], [115.31832000000009, 2.987780000000043], [115.31026000000008, 2.997780000000091], [115.30554000000006, 3.006670000000042], [115.30138000000011, 3.016390000000058], [115.24971000000005, 3.010830000000055], [115.15208000000007, 2.922080000000051], [115.13971000000004, 2.9061100000000692], [115.11832000000004, 2.851110000000062], [115.08236000000011, 2.613610000000051], [115.09206000000006, 2.600550000000055], [115.10762, 2.59722000000005], [115.12318000000005, 2.605550000000049], [115.1336, 2.611530000000073], [115.16944000000001, 2.605140000000062], [115.18525, 2.596940000000074], [115.19914000000006, 2.583050000000071], [115.2360900000001, 2.52965000000006], [115.23082000000011, 2.5080600000000572], [115.21568000000002, 2.492780000000039], [115.20387000000005, 2.487220000000093], ...]], [[[117.28298000000007, 7.319440000000043], [117.27832000000001, 7.250000000000057], [117.2763900000001, 7.238610000000051], [117.27304000000004, 7.228330000000085], [117.25054, 7.179170000000056], [117.19274000000007, 7.173330000000078], [117.17221000000006, 7.173330000000078], [117.1599900000001, 7.171940000000063], [117.14943000000005, 7.168610000000058], [117.1208200000001, 7.155550000000062], [117.09818000000007, 7.13722000000007], [117.0894300000001, 7.118890000000078], [117.07929000000001, 7.105000000000075], [117.06645000000003, 7.105070000000069], [117.05193000000008, 7.17083000000008], [117.06248000000005, 7.271670000000086], [117.07068000000004, 7.284030000000087], [117.08805000000007, 7.294170000000065], [117.0932600000001, 7.293450000000064], [117.14512000000002, 7.334030000000041], [117.22221000000002, 7.3527800000000525], [117.26818000000003, 7.343890000000044], [117.27832000000001, 7.333750000000066], [117.28298000000007, 7.319440000000043]]], [[[116.87471000000005, 7.221800000000087], [116.85694000000001, 7.183890000000076], [116.85234000000003, 7.188330000000064], [116.8787400000001, 7.274170000000083], [116.88666, 7.280280000000062], [117.00088000000005, 7.35292000000004], [117.00943000000007, 7.342220000000054], [117.01749000000007, 7.308330000000069], [117.01804000000004, 7.266740000000084], [116.9580400000001, 7.241670000000056], [116.90818000000002, 7.233470000000068], [116.87471000000005, 7.221800000000087]]]]
Thank you in advance,
Best regards.
The VSCode Interactive window for Jupyter truncates long output:
import os
dir(os)
['CLD_CONTINUED',
'CLD_DUMPED',
'CLD_EXITED',
'CLD_TRAPPED',
'DirEntry',
'EX_CANTCREAT',
'EX_CONFIG',
'EX_DATAERR',
'EX_IOERR',
'EX_NOHOST',
'EX_NOINPUT',
'EX_NOPERM',
'EX_NOUSER',
'EX_OK',
'EX_OSERR',
'EX_OSFILE',
'EX_PROTOCOL',
'EX_SOFTWARE',
'EX_TEMPFAIL',
'EX_UNAVAILABLE',
'EX_USAGE',
'F_LOCK',
'F_OK',
'F_TEST',
'F_TLOCK',
show more (open the raw output data in a text editor) ...
'waitid',
'waitid_result',
'waitpid',
'walk',
'write',
'writev']
This is nice functionality as it stops commands that generate a lot of output from overwhelming the Interactive window.
How can I see the entire output?
I can click on the "show more" link, but the output is in a JSON format which is difficult to read.
[
{
"metadata": {
"outputType": "execute_result",
"executionCount": 8,
"metadata": {}
},
"outputItems": [
{
"mimeType": "text/plain",
"data": "['CLD_CONTINUED',\n 'CLD_DUMPED',\n 'CLD_EXITED',\n 'CLD_TRAPPED',\n 'DirEntry',\n 'EX_CANTCREAT',\n 'EX_CONFIG',\n 'EX_DATAERR',\n 'EX_IOERR',\n 'EX_NOHOST',\n 'EX_NOINPUT',\n 'EX_NOPERM',\n 'EX_NOUSER',\n 'EX_OK',\n 'EX_OSERR',\n 'EX_OSFILE',\n 'EX_PROTOCOL',\n 'EX_SOFTWARE',\n 'EX_TEMPFAIL',\n 'EX_UNAVAILABLE',\n 'EX_USAGE',\n 'F_LOCK',\n 'F_OK',\n 'F_TEST',\n 'F_TLOCK',\n 'F_ULOCK',\n 'GRND_NONBLOCK',\n 'GRND_RANDOM',\n 'MutableMapping',\n 'NGROUPS_MAX',\n 'O_ACCMODE',\n 'O_APPEND',\n 'O_ASYNC',\n 'O_CLOEXEC',\n 'O_CREAT',\n 'O_DIRECT',\n 'O_DIRECTORY',\n 'O_DSYNC',\n 'O_EXCL',\n 'O_LARGEFILE',\n 'O_NDELAY',\n 'O_NOATIME',\n 'O_NOCTTY',\n 'O_NOFOLLOW',\n 'O_NONBLOCK',\n 'O_PATH',\n 'O_RDONLY',\n 'O_RDWR',\n 'O_RSYNC',\n 'O_SYNC',\n 'O_TMPFILE',\n 'O_TRUNC',\n 'O_WRONLY',\n 'POSIX_FADV_DONTNEED',\n 'POSIX_FADV_NOREUSE',\n 'POSIX_FADV_NORMAL',\n 'POSIX_FADV_RANDOM',\n 'POSIX_FADV_SEQUENTIAL',\n 'POSIX_FADV_WILLNEED',\n 'PRIO_PGRP',\n 'PRIO_PROCESS',\n 'PRIO_USER',\n 'P_ALL',\n 'P_NOWAIT',\n 'P_NOWAITO',\n 'P_PGID',\n 'P_PID',\n 'P_WAIT',\n 'PathLike',\n 'RTLD_DEEPBIND',\n 'RTLD_GLOBAL',\n 'RTLD_LAZY',\n 'RTLD_LOCAL',\n 'RTLD_NODELETE',\n 'RTLD_NOLOAD',\n 'RTLD_NOW',\n 'R_OK',\n 'SCHED_BATCH',\n 'SCHED_FIFO',\n 'SCHED_IDLE',\n 'SCHED_OTHER',\n 'SCHED_RESET_ON_FORK',\n 'SCHED_RR',\n 'SEEK_CUR',\n 'SEEK_DATA',\n 'SEEK_END',\n 'SEEK_HOLE',\n 'SEEK_SET',\n 'ST_APPEND',\n 'ST_MANDLOCK',\n 'ST_NOATIME',\n 'ST_NODEV',\n 'ST_NODIRATIME',\n 'ST_NOEXEC',\n 'ST_NOSUID',\n 'ST_RDONLY',\n 'ST_RELATIME',\n 'ST_SYNCHRONOUS',\n 'ST_WRITE',\n 'TMP_MAX',\n 'WCONTINUED',\n 'WCOREDUMP',\n 'WEXITED',\n 'WEXITSTATUS',\n 'WIFCONTINUED',\n 'WIFEXITED',\n 'WIFSIGNALED',\n 'WIFSTOPPED',\n 'WNOHANG',\n 'WNOWAIT',\n 'WSTOPPED',\n 'WSTOPSIG',\n 'WTERMSIG',\n 'WUNTRACED',\n 'W_OK',\n 'XATTR_CREATE',\n 'XATTR_REPLACE',\n 'XATTR_SIZE_MAX',\n 'X_OK',\n '_Environ',\n '__all__',\n '__builtins__',\n '__cached__',\n '__doc__',\n '__file__',\n '__loader__',\n '__name__',\n '__package__',\n '__spec__',\n '_execvpe',\n '_exists',\n '_exit',\n '_fspath',\n '_fwalk',\n '_get_exports_list',\n '_putenv',\n '_spawnvef',\n '_unsetenv',\n '_wrap_close',\n 'abc',\n 'abort',\n 'access',\n 'altsep',\n 'chdir',\n 'chmod',\n 'chown',\n 'chroot',\n 'close',\n 'closerange',\n 'confstr',\n 'confstr_names',\n 'cpu_count',\n 'ctermid',\n 'curdir',\n 'defpath',\n 'device_encoding',\n 'devnull',\n 'dup',\n 'dup2',\n 'environ',\n 'environb',\n 'errno',\n 'error',\n 'execl',\n 'execle',\n 'execlp',\n 'execlpe',\n 'execv',\n 'execve',\n 'execvp',\n 'execvpe',\n 'extsep',\n 'fchdir',\n 'fchmod',\n 'fchown',\n 'fdatasync',\n 'fdopen',\n 'fork',\n 'forkpty',\n 'fpathconf',\n 'fsdecode',\n 'fsencode',\n 'fspath',\n 'fstat',\n 'fstatvfs',\n 'fsync',\n 'ftruncate',\n 'fwalk',\n 'get_blocking',\n 'get_exec_path',\n 'get_inheritable',\n 'get_terminal_size',\n 'getcwd',\n 'getcwdb',\n 'getegid',\n 'getenv',\n 'getenvb',\n 'geteuid',\n 'getgid',\n 'getgrouplist',\n 'getgroups',\n 'getloadavg',\n 'getlogin',\n 'getpgid',\n 'getpgrp',\n 'getpid',\n 'getppid',\n 'getpriority',\n 'getrandom',\n 'getresgid',\n 'getresuid',\n 'getsid',\n 'getuid',\n 'getxattr',\n 'initgroups',\n 'isatty',\n 'kill',\n 'killpg',\n 'lchown',\n 'linesep',\n 'link',\n 'listdir',\n 'listxattr',\n 'lockf',\n 'lseek',\n 'lstat',\n 'major',\n 'makedev',\n 'makedirs',\n 'minor',\n 'mkdir',\n 'mkfifo',\n 'mknod',\n 'name',\n 'nice',\n 'open',\n 'openpty',\n 'pardir',\n 'path',\n 'pathconf',\n 'pathconf_names',\n 'pathsep',\n 'pipe',\n 'pipe2',\n 'popen',\n 'posix_fadvise',\n 'posix_fallocate',\n 'pread',\n 'putenv',\n 'pwrite',\n 'read',\n 'readlink',\n 'readv',\n 'remove',\n 'removedirs',\n 'removexattr',\n 'rename',\n 'renames',\n 'replace',\n 'rmdir',\n 'scandir',\n 'sched_get_priority_max',\n 'sched_get_priority_min',\n 'sched_getaffinity',\n 'sched_getparam',\n 'sched_getscheduler',\n 'sched_param',\n 'sched_rr_get_interval',\n 'sched_setaffinity',\n 'sched_setparam',\n 'sched_setscheduler',\n 'sched_yield',\n 'sendfile',\n 'sep',\n 'set_blocking',\n 'set_inheritable',\n 'setegid',\n 'seteuid',\n 'setgid',\n 'setgroups',\n 'setpgid',\n 'setpgrp',\n 'setpriority',\n 'setregid',\n 'setresgid',\n 'setresuid',\n 'setreuid',\n 'setsid',\n 'setuid',\n 'setxattr',\n 'spawnl',\n 'spawnle',\n 'spawnlp',\n 'spawnlpe',\n 'spawnv',\n 'spawnve',\n 'spawnvp',\n 'spawnvpe',\n 'st',\n 'stat',\n 'stat_float_times',\n 'stat_result',\n 'statvfs',\n 'statvfs_result',\n 'strerror',\n 'supports_bytes_environ',\n 'supports_dir_fd',\n 'supports_effective_ids',\n 'supports_fd',\n 'supports_follow_symlinks',\n 'symlink',\n 'sync',\n 'sys',\n 'sysconf',\n 'sysconf_names',\n 'system',\n 'tcgetpgrp',\n 'tcsetpgrp',\n 'terminal_size',\n 'times',\n 'times_result',\n 'truncate',\n 'ttyname',\n 'umask',\n 'uname',\n 'uname_result',\n 'unlink',\n 'unsetenv',\n 'urandom',\n 'utime',\n 'wait',\n 'wait3',\n 'wait4',\n 'waitid',\n 'waitid_result',\n 'waitpid',\n 'walk',\n 'write',\n 'writev']"
}
]
}
]
I can take the raw output and massage it in a text editor or other tool. This is time consuming, so I'm looking for an approach that is built into VSCode.
This is a bug that has been reported to the VS Code team and has a fix in already. The fix is in VS Code - Insiders (you can install that side by side with stable) to check. The fix should be in stable VS Code in the next full release:
https://github.com/microsoft/vscode/issues/130512
Installing the JSON formatter (or other similar extension) should work to prettify it in VSCode itself.
from csv import reader
def func(sku_list):
values = []
with open(sku_list, 'r', encoding = 'utf-8') as pr:
rows = reader(pr)
for sku in rows:
values.append(sku[1])
return(values)
if __name__ == '__main__':
dir_path = "C:/Users/XXXX/Downloads/"
vendors = dir_path + 'file.csv'
new_prices = func(vendors)
print(new_prices)
sku_list is a csv file filled with pairs of brand names and their skus that I have downloaded from my db, for some reason as it iters through the rows and grabs just the sku value, hence sku[1], it stops well short of the actual length I expect the list to be
sku_list is 85,892 tuples long but when I print out the values appended to the list values it simply returns this:
['SKU', 'MWGB4896', 'MWGB4872', 'MWGB4848', 'MWGB3648', 'WGB4896', 'WGB4872', 'WGB4848', 'WGB3648', 'WGB2436', 'WGB1824', 'BKGB4896NT', 'BKGB4872NT', 'BKGB4848NT', 'BKGB3648NT', 'BKGB2436NT', 'BKGB1824NT', 'WFC2418G', 'WFC2418', 'WFC3624', 'WFC2418LB', 'WFC3648LB', 'WFC3624LB', 'WFC3624G', 'WFC3648G', 'WFC3648', 'LOWFC3624LB', 'LOWFC3624G', 'LOWFC3624', 'LOWFC2418LB', 'LOWFC2418G', 'LOWFC2418', 'LOWFC3648LB', 'LOWFC3648', 'LOWFC3648G', 'WM-7-B', 'WM-7-G', 'WM-7-BK', 'WMC-7', 'WM-7-R', 'APS-50', 'APS-70', 'APS-60', 'APS-84', 'SS15W', 'SC15W', 'SB15W', 'MFL-2W', 'WP-48', 'WP-40', 'WP-36', 'MP-48', 'MP-40', 'MP-36', 'OP-40', 'OP-36', 'OP-48', 'FFVSU96-2', 'FFVSU144-2', 'FFVSU192-2', '1-WA-1B', '1-WA-1BP', 'WCS-12', 'WCS-144', 'OPLD3416LSPP-2', 'OPLD3416LSPP-4', 'OPLD3416LSPP-5', 'OPLD3416LSPP-7', 'OPLD3416LSPP-8', 'OPLD1818LSPP-2', 'OPLD1818LSPP-4', 'OPLD1818LSPP-5', 'OPLD1818LSPP-7', 'OPLD1818LSPP-8', 'OPLD1818L-2', 'OPLD1818L-5', 'OPLD1818L-4', 'OPLD3416L-2', 'OPLD3416L-4', 'OPLD3416L-5', 'OPLD3416L-7', 'OPLD3416L-8', 'OPLD1818L-7', 'OPLD1818L-8', 'OPLD3416SPP-8-892', 'OPLD3416SPP-8-897', 'OPLD3416SPP-8-878', 'OPLD3416SPP-8-885', 'OPLD3416SPP-8-887', 'OPLD3416SPP-8-890', 'OPLD3416SPP-8-845', 'OPLD3416SPP-8-854', 'OPLD3416SPP-8-856', 'OPLD3416SPP-8-876', 'OPLD3416SPP-8-802', 'OPLD3416SPP-8-706', 'OPLD3416SPP-8-705', 'OPLD3416SPP-8-704', 'OPLD3416SPP-8-837', 'OPLD3416SPP-8-831', 'OPLD3416SPP-8-819', 'OPLD3416SPP-8-812', 'OPLD3416SPP-8-685', 'OPLD3416SPP-8-683', 'OPLD3416SPP-8-679', 'OPLD3416SPP-8-531', 'OPLD3416SPP-8-703', 'OPLD3416SPP-8-702', 'OPLD3416SPP-8-701', 'OPLD3416SPP-8-700', 'OPLD3416SPP-7-892', 'OPLD3416SPP-7-897', 'OPLD3416SPP-7-887', 'OPLD3416SPP-7-890', 'OPLD3416SPP-8-530', 'OPLD3416SPP-7-845', 'OPLD3416SPP-7-854', 'OPLD3416SPP-7-831', 'OPLD3416SPP-7-837', 'OPLD3416SPP-7-878', 'OPLD3416SPP-7-885', 'OPLD3416SPP-7-856', 'OPLD3416SPP-7-876', 'OPLD3416SPP-7-703', 'OPLD3416SPP-7-702', 'OPLD3416SPP-7-705', 'OPLD3416SPP-7-704', 'OPLD3416SPP-7-802', 'OPLD3416SPP-7-706', 'OPLD3416SPP-7-819', 'OPLD3416SPP-7-812', 'OPLD3416SPP-7-530', 'OPLD3416SPP-7-679', 'OPLD3416SPP-7-531', 'OPLD3416SPP-7-685', 'OPLD3416SPP-7-683', 'OPLD3416SPP-7-701', 'OPLD3416SPP-7-700', 'OPLD3416SPP-5-878', 'OPLD3416SPP-5-885', 'OPLD3416SPP-5-887', 'OPLD3416SPP-5-890', 'OPLD3416SPP-5-892', 'OPLD3416SPP-5-897', 'OPLD3416SPP-5-812', 'OPLD3416SPP-5-819', 'OPLD3416SPP-5-831', 'OPLD3416SPP-5-837', 'OPLD3416SPP-5-845', 'OPLD3416SPP-5-854', 'OPLD3416SPP-5-856', 'OPLD3416SPP-5-876', 'OPLD1818SPP-8-819', 'OPLD1818SPP-8-831', 'OPLD1818SPP-8-802', 'OPLD1818SPP-8-812', 'OPLD1818SPP-8-854', 'OPLD1818SPP-8-856', 'OPLD1818SPP-8-837', 'OPLD1818SPP-8-845', 'OPLD1818SPP-8-701', 'OPLD1818SPP-8-702', 'OPLD1818SPP-8-685', 'OPLD1818SPP-8-700', 'OPLD1818SPP-8-705', 'OPLD1818SPP-8-706', 'OPLD1818SPP-8-703', 'OPLD1818SPP-8-704', 'OPLD1818SPP-8-887', 'OPLD1818SPP-8-885', 'OPLD1818SPP-8-878', 'OPLD1818SPP-8-876', 'OPLD1818SPP-8-897', 'OPLD1818SPP-8-892', 'OPLD1818SPP-8-890', 'OPLD3416SPP-4-837', 'OPLD3416SPP-4-831', 'OPLD3416SPP-4-854', 'OPLD3416SPP-4-845', 'OPLD3416SPP-4-802', 'OPLD3416SPP-4-706',
'OPLD3416SPP-4-819', 'OPLD3416SPP-4-812', 'OPLD3416SPP-4-890', 'OPLD3416SPP-4-887', 'OPLD3416SPP-4-897', 'OPLD3416SPP-4-892', 'OPLD3416SPP-4-876', 'OPLD3416SPP-4-856', 'OPLD3416SPP-4-885', 'OPLD3416SPP-4-878', 'OPLD3416SPP-5-531', 'OPLD3416SPP-5-679', 'OPLD3416SPP-5-683', 'OPLD3416SPP-5-685', 'OPLD3416SPP-5-530', 'OPLD3416SPP-5-704', 'OPLD3416SPP-5-705', 'OPLD3416SPP-5-706', 'OPLD3416SPP-5-802', 'OPLD3416SPP-5-700', 'OPLD3416SPP-5-701', 'OPLD3416SPP-5-702', 'OPLD3416SPP-5-703', 'OPLD3416SPP-2-837', 'OPLD3416SPP-2-831', 'OPLD3416SPP-2-819', 'OPLD3416SPP-2-812', 'OPLD3416SPP-2-802', 'OPLD3416SPP-2-706', 'OPLD3416SPP-2-705', 'OPLD3416SPP-2-704', 'OPLD3416SPP-2-890', 'OPLD3416SPP-2-887', 'OPLD3416SPP-2-885', 'OPLD3416SPP-2-878', 'OPLD3416SPP-2-876', 'OPLD3416SPP-2-856', 'OPLD3416SPP-2-854', 'OPLD3416SPP-2-845', 'OPLD3416SPP-4-531', 'OPLD3416SPP-4-679', 'OPLD3416SPP-4-530', 'OPLD3416SPP-2-892', 'OPLD3416SPP-2-897', 'OPLD3416SPP-4-704', 'OPLD3416SPP-4-705', 'OPLD3416SPP-4-702', 'OPLD3416SPP-4-703', 'OPLD3416SPP-4-700', 'OPLD3416SPP-4-701', 'OPLD3416SPP-4-683', 'OPLD3416SPP-4-685', 'OPLD3416SPP-2-530', 'OPLD3416SPP-2-531', 'OPLD3416SPP-2-679', 'OPLD3416SPP-2-683', 'OPLD3416SPP-2-685', 'OPLD3416SPP-2-700', 'OPLD3416SPP-2-701', 'OPLD3416SPP-2-702', 'OPLD3416SPP-2-703', 'OPLD1818SPP-7-819', 'OPLD1818SPP-7-831', 'OPLD1818SPP-7-837', 'OPLD1818SPP-7-845', 'OPLD1818SPP-7-705', 'OPLD1818SPP-7-706', 'OPLD1818SPP-7-802', 'OPLD1818SPP-7-812', 'OPLD1818SPP-7-701', 'OPLD1818SPP-7-702', 'OPLD1818SPP-7-703', 'OPLD1818SPP-7-704', 'OPLD1818SPP-7-679', 'OPLD1818SPP-7-683', 'OPLD1818SPP-7-685', 'OPLD1818SPP-7-700', 'OPLD1818SPP-8-531', 'OPLD1818SPP-8-530', 'OPLD1818SPP-8-683', 'OPLD1818SPP-8-679', 'OPLD1818SPP-7-897', 'OPLD1818SPP-7-887', 'OPLD1818SPP-7-885', 'OPLD1818SPP-7-892', 'OPLD1818SPP-7-890', 'OPLD1818SPP-7-856', 'OPLD1818SPP-7-854', 'OPLD1818SPP-7-878', 'OPLD1818SPP-7-876', 'OPLD1818SPP-5-819', 'OPLD1818SPP-5-831', 'OPLD1818SPP-5-802', 'OPLD1818SPP-5-812', 'OPLD1818SPP-5-705', 'OPLD1818SPP-5-706', 'OPLD1818SPP-5-703', 'OPLD1818SPP-5-704', 'OPLD1818SPP-5-701', 'OPLD1818SPP-5-702', 'OPLD1818SPP-5-685', 'OPLD1818SPP-5-700', 'OPLD1818SPP-5-679', 'OPLD1818SPP-5-683', 'OPLD1818SPP-5-530', 'OPLD1818SPP-5-531', 'OPLD1818SPP-7-531', 'OPLD1818SPP-7-530', 'OPLD1818SPP-5-897', 'OPLD1818SPP-5-892', 'OPLD1818SPP-5-890', 'OPLD1818SPP-5-887', 'OPLD1818SPP-5-885', 'OPLD1818SPP-5-878', 'OPLD1818SPP-5-876', 'OPLD1818SPP-5-856', 'OPLD1818SPP-5-854', 'OPLD1818SPP-5-845', 'OPLD1818SPP-5-837', 'OPLD1818SPP-4-701', 'OPLD1818SPP-4-702', 'OPLD1818SPP-4-703', 'OPLD1818SPP-4-704', 'OPLD1818SPP-4-705', 'OPLD1818SPP-4-706', 'OPLD1818SPP-4-802', 'OPLD1818SPP-4-812', 'OPLD1818SPP-4-530', 'OPLD1818SPP-4-531', 'OPLD1818SPP-4-679', 'OPLD1818SPP-4-683', 'OPLD1818SPP-4-685', 'OPLD1818SPP-4-700', 'OPLD1818SPP-4-887', 'OPLD1818SPP-2-837', 'OPLD1818SPP-2-845', 'OPLD1818SPP-2-854', 'OPLD1818SPP-2-856', 'OPLD1818SPP-2-802', 'OPLD1818SPP-2-812', 'OPLD1818SPP-2-819', 'OPLD1818SPP-2-831', 'OPLD1818SPP-2-890', 'OPLD1818SPP-2-892', 'OPLD1818SPP-2-897', 'OPLD1818SPP-2-876', 'OPLD1818SPP-2-878', 'OPLD1818SPP-2-885', 'OPLD1818SPP-2-887', 'OPLD1818SPP-2-531', 'OPLD1818SPP-2-530', 'OPLD1818SPP-2-683', 'OPLD1818SPP-2-679', 'OPLD1818SPP-2-704', 'OPLD1818SPP-2-703',
'OPLD1818SPP-2-706', 'OPLD1818SPP-2-705', 'OPLD1818SPP-2-700', 'OPLD1818SPP-2-685', 'OPLD1818SPP-2-702', 'OPLD1818SPP-2-701', 'OPLD1818SPP-4-876', 'OPLD1818SPP-4-878', 'OPLD1818SPP-4-854', 'OPLD1818SPP-4-856', 'OPLD1818SPP-4-837', 'OPLD1818SPP-4-845', 'OPLD1818SPP-4-819', 'OPLD1818SPP-4-831', 'OPLD1818SPP-4-897', 'OPLD1818SPP-4-890', 'OPLD1818SPP-4-892', 'OPLD1818SPP-4-885', 'PLD4832DPP-2-845', 'PLD4832DPP-2-837', 'PLD4832DPP-2-856', 'PLD4832DPP-2-854', 'PLD4832DPP-2-878', 'PLD4832DPP-2-876', 'PLD4832DPP-2-887', 'PLD4832DPP-2-885', 'PLD4832DPP-2-892', 'PLD4832DPP-2-890', 'PLD4832DPP-2-897', 'PLD4832DPP-4-531', 'PLD4832DPP-4-530', 'PLD4832DPP-4-679', 'PLD4832DPP-4-683', 'PLD4832DPP-4-685', 'PLD4832DPP-4-700', 'PLD4832DPP-4-701', 'PLD4832DPP-4-702', 'PLD4832DPP-4-703', 'PLD4832DPP-4-704', 'PLD4832DPP-4-705', 'PLD4832DPP-4-706', 'PLD4832DPP-4-802', 'PLD4832DPP-4-812', 'PLD4832DPP-4-819', 'PLD4832DPP-4-831', 'PLD4832DPP-4-837', 'PLD4832DPP-4-845', 'PLD4832DPP-4-878', 'PLD4832DPP-4-876', 'PLD4832DPP-4-856', 'PLD4832DPP-4-854', 'PLD4832DPP-4-892', 'PLD4832DPP-4-890', 'PLD4832DPP-4-887', 'PLD4832DPP-4-885', 'PLD4832DPP-4-897', 'PLD4832DPP-5-683', 'PLD4832DPP-5-679', 'PLD4832DPP-5-531', 'PLD4832DPP-5-530', 'PLD4832DPP-5-701', 'PLD4832DPP-5-702', 'PLD4832DPP-5-685', 'PLD4832DPP-5-700', 'PLD4832DPP-5-705', 'PLD4832DPP-5-706', 'PLD4832DPP-5-703', 'PLD4832DPP-5-704', 'PLD4832DPP-5-819', 'PLD4832DPP-5-831', 'PLD4832DPP-5-802', 'PLD4832DPP-5-812', 'PLD4832DPP-5-854', 'PLD4832DPP-5-856', 'PLD4832DPP-5-837', 'PLD4832DPP-5-845', 'PLD4832DPP-2-701', 'PLD4832DPP-2-702', 'PLD4832DPP-2-685', 'PLD4832DPP-2-700', 'PLD4832DPP-2-679', 'PLD4832DPP-2-683', 'PLD4832DPP-2-530', 'PLD4832DPP-2-531', 'PLD4832DPP-2-819', 'PLD4832DPP-2-831', 'PLD4832DPP-2-802', 'PLD4832DPP-2-812', 'PLD4832DPP-2-705', 'PLD4832DPP-2-706', 'PLD4832DPP-2-703', 'PLD4832DPP-2-704', 'PLD4226DPP-8-887', 'PLD4226DPP-8-890', 'PLD4226DPP-8-892', 'PLD4226DPP-8-897', 'PLD4226DPP-8-856', 'PLD4226DPP-8-876', 'PLD4226DPP-8-878', 'PLD4226DPP-8-885', 'PLD4226DPP-8-831', 'PLD4226DPP-8-837', 'PLD4226DPP-8-845', 'PLD4226DPP-8-854', 'PLD4226DPP-8-706', 'PLD4226DPP-8-802', 'PLD4226DPP-8-812', 'PLD4226DPP-8-819', 'PLD4226DPP-5-892', 'PLD4226DPP-5-897', 'PLD4226DPP-5-887', 'PLD4226DPP-5-890', 'PLD4226DPP-7-530', 'PLD4226DPP-7-683', 'PLD4226DPP-7-685', 'PLD4226DPP-7-531', 'PLD4226DPP-7-679', 'PLD4226DPP-7-702', 'PLD4226DPP-7-703', 'PLD4226DPP-7-700', 'PLD4226DPP-7-701', 'PLD4226DPP-5-705', 'PLD4226DPP-5-704', 'PLD4226DPP-5-703', 'PLD4226DPP-5-702', 'PLD4226DPP-5-819', 'PLD4226DPP-5-812', 'PLD4226DPP-5-802', 'PLD4226DPP-5-706', 'PLD4226DPP-5-854', 'PLD4226DPP-5-845', 'PLD4226DPP-5-837', 'PLD4226DPP-5-831', 'PLD4226DPP-5-885', 'PLD4226DPP-5-878', 'PLD4226DPP-5-876', 'PLD4226DPP-5-856', 'PLD4226DPP-7-892', 'PLD4226DPP-7-897', 'PLD4226DPP-8-530', 'PLD4226DPP-8-531', 'PLD4226DPP-8-679', 'PLD4226DPP-8-683', 'PLD4226DPP-8-685', 'PLD4226DPP-8-700', 'PLD4226DPP-8-701', 'PLD4226DPP-8-702', 'PLD4226DPP-8-703', 'PLD4226DPP-8-704', 'PLD4226DPP-8-705', 'PLD4226DPP-7-705', 'PLD4226DPP-7-704', 'PLD4226DPP-7-802', 'PLD4226DPP-7-706', 'PLD4226DPP-7-819', 'PLD4226DPP-7-812', 'PLD4226DPP-7-837', 'PLD4226DPP-7-831', 'PLD4226DPP-7-854', 'PLD4226DPP-7-845', 'PLD4226DPP-7-876', 'PLD4226DPP-7-856', 'PLD4226DPP-7-885', 'PLD4226DPP-7-878', 'PLD4226DPP-7-890', 'PLD4226DPP-7-887', 'PLD4226DPP-2-892', 'PLD4226DPP-2-897', 'PLD4226DPP-2-887', 'PLD4226DPP-2-890', 'PLD4226DPP-2-878', 'PLD4226DPP-2-885', 'PLD4226DPP-2-856', 'PLD4226DPP-2-876', 'PLD4226DPP-4-683', 'PLD4226DPP-4-685', 'PLD4226DPP-4-531', 'PLD4226DPP-4-679', 'PLD4226DPP-4-530', 'PLD4226DPP-2-705', 'PLD4226DPP-2-704', 'PLD4226DPP-2-703', 'PLD4226DPP-2-702', 'PLD4226DPP-2-701', 'PLD4226DPP-2-700', 'PLD4226DPP-2-685', 'PLD4226DPP-2-683', 'PLD4226DPP-2-854', 'PLD4226DPP-2-845', 'PLD4226DPP-2-837', 'PLD4226DPP-2-831', 'PLD4226DPP-2-819', 'PLD4226DPP-2-812', 'PLD4226DPP-2-802', 'PLD4226DPP-2-706', 'PLD4226DPP-4-892', 'PLD4226DPP-4-897', 'PLD4226DPP-4-878', 'PLD4226DPP-4-885', 'PLD4226DPP-4-887', 'PLD4226DPP-4-890', 'PLD4226DPP-5-683', 'PLD4226DPP-5-685', 'PLD4226DPP-5-700', 'PLD4226DPP-5-701', 'PLD4226DPP-5-530', 'PLD4226DPP-5-531', 'PLD4226DPP-5-679', 'PLD4226DPP-4-705', 'PLD4226DPP-4-704', 'PLD4226DPP-4-802', 'PLD4226DPP-4-706', 'PLD4226DPP-4-701', 'PLD4226DPP-4-700', 'PLD4226DPP-4-703', 'PLD4226DPP-4-702', 'PLD4226DPP-4-854', 'PLD4226DPP-4-845', 'PLD4226DPP-4-876', 'PLD4226DPP-4-856', 'PLD4226DPP-4-819', 'PLD4226DPP-4-812', 'PLD4226DPP-4-837', 'PLD4226DPP-4-831', 'PLD4226DPP-2-530', 'PLD4226DPP-2-679', 'PLD4226DPP-2-531', 'PLD5438DPP-5-683', 'PLD5438DPP-5-685', 'PLD5438DPP-5-531', 'PLD5438DPP-5-679', 'PLD5438DPP-5-702', 'PLD5438DPP-5-703', 'PLD5438DPP-5-700', 'PLD5438DPP-5-701', 'PLD5438DPP-5-706', 'PLD5438DPP-5-802', 'PLD5438DPP-5-704', 'PLD5438DPP-5-705', 'PLD5438DPP-5-831', 'PLD5438DPP-5-837', 'PLD5438DPP-5-812', 'PLD5438DPP-5-819', 'PLD5438DPP-5-876', 'PLD5438DPP-5-856', 'PLD5438DPP-5-854', 'PLD5438DPP-5-845', 'PLD5438DPP-5-890', 'PLD5438DPP-5-887', 'PLD5438DPP-5-885', 'PLD5438DPP-5-878', 'PLD5438DPP-5-897', 'PLD5438DPP-5-892', 'PLD5438DPP-7-679', 'PLD5438DPP-7-531', 'PLD5438DPP-7-530', 'PLD5438DPP-4-530', 'PLD5438DPP-4-531', 'PLD5438DPP-4-679', 'PLD5438DPP-4-683', 'PLD5438DPP-4-685', 'PLD5438DPP-4-700', 'PLD5438DPP-4-701', 'PLD5438DPP-4-702', 'PLD5438DPP-4-703', 'PLD5438DPP-4-704', 'PLD5438DPP-4-705', 'PLD5438DPP-4-706', 'PLD5438DPP-4-802', 'PLD5438DPP-4-812', 'PLD5438DPP-4-819', 'PLD5438DPP-4-837', 'PLD5438DPP-4-831', 'PLD5438DPP-4-854', 'PLD5438DPP-4-845', 'PLD5438DPP-4-876', 'PLD5438DPP-4-856', 'PLD5438DPP-4-885', 'PLD5438DPP-4-878', 'PLD5438DPP-4-890', 'PLD5438DPP-4-887', 'PLD5438DPP-4-897', 'PLD5438DPP-4-892', 'PLD5438DPP-5-530',
'PLD5438DPP
None
the final sku in there PLD5438DPP should be PLD5438DPP-5-683 and for some reason the list cuts off there, which is only element 564/85,892, and the program terminates without an error code
I cannot attach the file of skus, this is for my job, just hoping someone can shed light on what I am doing to cause the list to cut short like that
this may or may not also be relevant but when I call .append(sku) as opposed to sku[1] and grab the whole tuple the same issue occurs but at element 292, exactly half of the amount of element appended when only doing half the tuple
The issue seems to be one on my local machine, where it was unable to print such a long list, this one was of size 85,892
for those with similar issues see if any of our specs are overlapping and that may determine the cause of this issue:
VSCode:
Version: 1.52.0 (user setup)
Commit: 940b5f4bb5fa47866a54529ed759d95d09ee80be
Date: 2020-12-10T22:45:11.850Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Windows_NT x64 10.0.18363
Python:
3.9.0
see discussion in comments for other details
I tried to run the Jupyter Notebook example for the object detection of tensorflow (tutorial) but there are no detections. I printed the scores and it's seems to work but the results are very bad. Does anyone have an idea what I might have done wrong.
print(scores[0]):
[ 0.03587309 0.02224856 0.01864638 0.01096715 0.0100315
0.0065446
0.00633551 0.00534311 0.00495995 0.00410238 0.00362363 0.00339175
0.00308251 0.0030337 0.00293387 0.00277085 0.00269581 0.00266825
0.00263924 0.00263331 0.00258721 0.00240822 0.00225823 0.00186966
0.00184308 0.00180467 0.00177474 0.00173643 0.0017281 0.00171935
0.00171891 0.00170284 0.00163754 0.00162967 0.00160267 0.00156545
0.00153614 0.00140936 0.00132406 0.00131524 0.00131041 0.00129431
0.00125819 0.0012553 0.00122365 0.00119179 0.00115673 0.00115186
0.00112368 0.00107096 0.00105803 0.00104337 0.00102719 0.00102337
0.00100349 0.00097767 0.0009685 0.00092741 0.00088506 0.00087696
0.0008734 0.00084825 0.00084135 0.00083512 0.00083396 0.00082068
0.00080583 0.00078979 0.00078059 0.00077475 0.00075449 0.00074426
0.00074421 0.00070195 0.00068741 0.00068138 0.00067261 0.00067125
0.00067032 0.00066041 0.0006473 0.00064205 0.00061964 0.00061793
0.00060834 0.00060468 0.00059547 0.00059478 0.00059461 0.00059436
0.00059426 0.00059411 0.00059406 0.00059392 0.00059365 0.00059351
0.00059191 0.00058798 0.00058682 0.00058148]
[ 0.01044157 0.00982138 0.00942336 0.00846517 0.00613665 0.00398568
0.00357755 0.00300539 0.00255862 0.00236576 0.00232631 0.00220291
0.00185227 0.00163544 0.00159791 0.00145071 0.0014366 0.0014137
0.00122685 0.00118978 0.00108457 0.00104252 0.00099215 0.00096401
0.0008708 0.00084774 0.00080484 0.00078507 0.00078379 0.00076875
0.00072774 0.00071732 0.00071343 0.00070812 0.00069253 0.0006762
0.00067269 0.00059905 0.00059367 0.000588 0.00056114 0.0005504
0.00051472 0.00051055 0.00050973 0.00048484 0.00047297 0.00046204
0.00044787 0.00043259 0.00042987 0.00042673 0.00041978 0.00040494
0.00040087 0.00039576 0.00039059 0.00037274 0.00036828 0.00036417
0.0003612 0.00034645 0.00034479 0.00034078 0.00033771 0.00033605
0.0003333 0.0003304 0.0003294 0.00032325 0.00031787 0.00031773
0.00031748 0.00031741 0.00031732 0.00031729 0.00031724 0.00031722
0.00031717 0.00031708 0.00031702 0.00031579 0.00030416 0.00030222
0.00029739 0.00029726 0.00028289 0.00026527 0.00026325 0.00024584
0.00024221 0.00024156 0.0002391 0.00023335 0.00021617 0.0002001
0.00019127 0.00018342 0.00017271 0.00015507]
I'm running the example with tensorflow 1.4, python 3.5 and I tested the installation as suggested.
I had the same issue. I found in a post that you have to change:
MODEL_NAME = 'ssd_mobilenet_v1_coco_2017_11_08'
To:
MODEL_NAME = 'ssd_mobilenet_v1_coco_11_06_2017'
and it worked fine.
Original answer: https://stackoverflow.com/a/47332228/8954260
I'm trying to install libvips via Brew, but when i install i get this error about Python not being in the right location.
Error message
martins-mbp:~ martinnielsen$ brew install vips
==> Downloading http://www.vips.ecs.soton.ac.uk/supported/7.40/vips-7.40.10.tar.Already downloaded: /Library/Caches/Homebrew/vips-7.40.10.tar.gz
==> ./configure --prefix=/usr/local/Cellar/vips/7.40.10
==> make check
/usr/local/Cellar/gobject-introspection/1.42.0/bin/g-ir-scanner --add-include-path=. --namespace=Vips --nsversion=8.0 --libtool="/bin/sh ../libtool" --include=GObject-2.0 --library=libvips.la --warn-all --verbose --identifier-prefix=Vips --identifier-prefix=vips --symbol-prefix=vips --symbol-prefix=im --symbol-prefix=im_ --cflags-begin -I../libvips/include --cflags-end arithmetic/abs.c arithmetic/add.c arithmetic/arithmetic.c arithmetic/avg.c arithmetic/binary.c arithmetic/boolean.c arithmetic/complex.c arithmetic/deviate.c arithmetic/divide.c arithmetic/getpoint.c arithmetic/hist_find.c arithmetic/hist_find_indexed.c arithmetic/hist_find_ndim.c arithmetic/hough.c arithmetic/hough_circle.c arithmetic/hough_line.c arithmetic/invert.c arithmetic/linear.c arithmetic/math.c arithmetic/math2.c arithmetic/max.c arithmetic/measure.c arithmetic/min.c arithmetic/multiply.c arithmetic/nary.c arithmetic/profile.c arithmetic/project.c arithmetic/relational.c arithmetic/remainder.c arithmetic/round.c arithmetic/sign.c arithmetic/statistic.c arithmetic/stats.c arithmetic/subtract.c arithmetic/sum.c arithmetic/unary.c arithmetic/unaryconst.c cimg/cimg_dispatch.c colour/colour.c colour/colourspace.c colour/dE00.c colour/dE76.c colour/dECMC.c colour/float2rad.c colour/icc_transform.c colour/Lab2LabQ.c colour/Lab2LabS.c colour/Lab2LCh.c colour/Lab2XYZ.c colour/LabQ2Lab.c colour/LabQ2LabS.c colour/LabQ2sRGB.c colour/LabS2Lab.c colour/LabS2LabQ.c colour/LCh2Lab.c colour/LCh2UCS.c colour/rad2float.c colour/scRGB2sRGB.c colour/scRGB2XYZ.c colour/sRGB2scRGB.c colour/UCS2LCh.c colour/XYZ2Lab.c colour/XYZ2scRGB.c colour/XYZ2Yxy.c colour/Yxy2XYZ.c conversion/bandary.c conversion/bandbool.c conversion/bandjoin.c conversion/bandmean.c conversion/bandrank.c conversion/cache.c conversion/cast.c conversion/conversion.c conversion/copy.c conversion/embed.c conversion/extract.c conversion/falsecolour.c conversion/flatten.c conversion/flip.c conversion/gamma.c conversion/grid.c conversion/ifthenelse.c conversion/insert.c conversion/join.c conversion/msb.c conversion/recomb.c conversion/replicate.c conversion/rot.c conversion/rot45.c conversion/scale.c conversion/sequential.c conversion/subsample.c conversion/tilecache.c conversion/wrap.c conversion/zoom.c convolution/compass.c convolution/conv.c convolution/convolution.c convolution/convsep.c convolution/correlation.c convolution/fastcor.c convolution/gaussblur.c convolution/im_aconv.c convolution/im_aconvsep.c convolution/im_conv.c convolution/im_conv_f.c convolution/sharpen.c convolution/spcor.c create/black.c create/buildlut.c create/create.c create/eye.c create/fractsurf.c create/gaussmat.c create/gaussnoise.c create/grey.c create/identity.c create/im_benchmark.c create/invertlut.c create/logmat.c create/mask.c create/mask_butterworth.c create/mask_butterworth_band.c create/mask_butterworth_ring.c create/mask_fractal.c create/mask_gaussian.c create/mask_gaussian_band.c create/mask_gaussian_ring.c create/mask_ideal.c create/mask_ideal_band.c create/mask_ideal_ring.c create/other_dispatch.c create/point.c create/sines.c create/text.c create/tonelut.c create/xyz.c create/zone.c draw/draw.c draw/draw_circle.c draw/draw_flood.c draw/draw_image.c draw/draw_line.c draw/draw_mask.c draw/draw_rect.c draw/draw_smudge.c draw/drawink.c foreign/analyze2vips.c foreign/analyzeload.c foreign/csv.c foreign/csvload.c foreign/csvsave.c foreign/dzsave.c foreign/fits.c foreign/fitsload.c foreign/fitssave.c foreign/foreign.c foreign/jpeg2vips.c foreign/jpegload.c foreign/jpegsave.c foreign/magick2vips.c foreign/magickload.c foreign/matlab.c foreign/matload.c foreign/matrixload.c foreign/matrixsave.c foreign/openexr2vips.c foreign/openexrload.c foreign/openslide2vips.c foreign/openslideload.c foreign/pngload.c foreign/pngsave.c foreign/ppm.c foreign/ppmload.c foreign/ppmsave.c foreign/radiance.c foreign/radload.c foreign/radsave.c foreign/rawload.c foreign/rawsave.c foreign/tiff2vips.c foreign/tiffload.c foreign/tiffsave.c foreign/vips2jpeg.c foreign/vips2tiff.c foreign/vips2webp.c foreign/vipsload.c foreign/vipspng.c foreign/vipssave.c foreign/webp2vips.c foreign/webpload.c foreign/webpsave.c freqfilt/freqfilt.c freqfilt/freqmult.c freqfilt/fwfft.c freqfilt/invfft.c freqfilt/phasecor.c freqfilt/spectrum.c histogram/hist_cum.c histogram/hist_equal.c histogram/hist_ismonotonic.c histogram/hist_local.c histogram/hist_match.c histogram/hist_norm.c histogram/hist_plot.c histogram/hist_unary.c histogram/histogram.c histogram/maplut.c histogram/percent.c histogram/stdif.c iofuncs/base64.c iofuncs/buf.c iofuncs/buffer.c iofuncs/cache.c iofuncs/enumtypes.c iofuncs/error.c iofuncs/gate.c iofuncs/generate.c iofuncs/header.c iofuncs/image.c iofuncs/init.c iofuncs/mapfile.c iofuncs/memory.c iofuncs/object.c iofuncs/operation.c iofuncs/rect.c iofuncs/region.c iofuncs/semaphore.c iofuncs/sink.c iofuncs/sinkdisc.c iofuncs/sinkmemory.c iofuncs/sinkscreen.c iofuncs/system.c iofuncs/threadpool.c iofuncs/type.c iofuncs/util.c iofuncs/vector.c iofuncs/vips.c iofuncs/vipsmarshal.c iofuncs/window.c morphology/countlines.c morphology/hitmiss.c morphology/labelregions.c morphology/morph.c morphology/morphology.c morphology/rank.c mosaicing/global_balance.c mosaicing/im_avgdxdy.c mosaicing/im_chkpair.c mosaicing/im_clinear.c mosaicing/im_improve.c mosaicing/im_initialize.c mosaicing/im_lrcalcon.c mosaicing/im_lrmerge.c mosaicing/im_lrmosaic.c mosaicing/im_remosaic.c mosaicing/im_tbcalcon.c mosaicing/im_tbmerge.c mosaicing/im_tbmosaic.c mosaicing/match.c mosaicing/merge.c mosaicing/mosaic.c mosaicing/mosaic1.c mosaicing/mosaicing.c resample/affine.c resample/interpolate.c resample/quadratic.c resample/resample.c resample/shrink.c resample/similarity.c resample/transform.c video/im_video_test.c video/video_dispatch.c include/vips/basic.h include/vips/vips.h include/vips/object.h include/vips/image.h include/vips/error.h include/vips/foreign.h include/vips/interpolate.h include/vips/header.h include/vips/operation.h include/vips/enumtypes.h include/vips/arithmetic.h include/vips/conversion.h include/vips/type.h libvips.la --output Vips-8.0.gir
/bin/sh: /usr/local/Cellar/gobject-introspection/1.42.0/bin/g-ir-scanner:
/usr/local/opt/python/bin/python: bad interpreter: No such file or directory
make[2]: *** [Vips-8.0.gir] Error 126
make[1]: *** [check-recursive] Error 1
make: *** [check-recursive] Error 1
READ THIS: https://github.com/Homebrew/homebrew/wiki/troubleshooting
If reporting this issue please do so at (not Homebrew/homebrew): https://github.com/homebrew/homebrew-science/issues
I looked for the folder /usr/local/opt/python/bin/python but i don't have it.
If i do which Python in the Terminal i get /usr/bin/Python which i guess should be where the script should look for Python.
Can i somehow alias the correct folder to allow libvips to build?
I fixed the error by installing python via homebrew aswell.
So i did homebrew install python first and then brew install homebrew/science/vips --with-webp --with-graphicsmagick afterwards.