Develop Reference

Strange Plotly behaviour with Choropleth Mapbox

I want to create a choropleth map out of a GeoJSON file that looks like this:
{"type": "FeatureCollection", "features": [
{'type': 'Feature', 'geometry': {'type': 'MultiPolygon', 'coordinates': [[[[... , ...] ... [..., ...]]]]}, 'properties': {'id': 'A'},
{'type': 'Feature', 'geometry': {'type': 'MultiPolygon', 'coordinates': [[[[... , ...] ... [..., ...]]]]}, 'properties': {'id': 'B'},
...
]}
with each id property being different for each feature.
I mapped each feature (by the id property) to it's particular region as it follows:
regions = {
'A': 'MOUNTAINS',
'B': 'BEACH',
...
}
and then created a DataFrame to store each id and each region:
ids = []
for feature in geojson['features']:
ids.append(feature['properties']['id'])
df = pd.DataFrame (ids, columns = ['id'])
df['region'] = df['id'].map(regions)
That returns a DataFrame like this:
id region
0 A MOUNTAIN
1 B BEACH
2 C PLAIN
3 D FOREST
...
I then tried to create a choropleth map with that info:
fig = px.choropleth_mapbox(df, geojson=geojson, color="region",
locations="id", featureidkey="properties.id",
center={"lat": -9.893, "lon": -50.423},
mapbox_style="white-bg", zoom=9)
fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0})
However, this results in an excessively long running time, which crashes about a minute or so later, with no error.
I wanted to check if there was something wrong with the GeoJSON file and/or with the mapping, so I assigned random numeric data to each id in df, by:
df['random_number'] = np.random.randint(0,100,size=len(df))
and re-tried the map with the following code:
fig = px.choropleth_mapbox(df, geojson=geojson, color="random_number",
locations="id", featureidkey="properties.id",
center={"lat": -9.893, "lon": -50.423},
mapbox_style="white-bg", zoom=9)
fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0})
and it worked, so I am guessing there is some kind of trouble with the non-numeric values in the region column of df, which are not being properly passed to the choropleth map.
Any advice, help or solution will be much appreciated!

How to make multiples filters in one describe_[resource] with boto3?

is possible make something like the example to get one sg from an specific vpc?:
sgs = client.describe_security_groups(
Filters=[
{'Name': 'vpc-id', 'Values': ['vpc-1111111']},
{'Name': 'GroupName': 'Values': ['sg_name']}
]
)

I'm not sure I understand your question but if you want to filter the results based on both a vpc-id and the name of a group, you would do this:
sgs = client.describe_security_groups(
Filters=[{'Name': 'vpc-id', 'Values': ['vpc-11111111']},
{'Name': 'group-name', 'Values': ['sg-name']}])
Does that help?

Using Python and xlsxwriter to concatenate cell references with cell count

Hi I have a working script to generate a line chart using Xlsxwriter, however I am looking for a way to concatenate an earlier hit count with the cell range for my generated chart as the script is used to iterate over several similar files in the directory so the overall 'hit count' varies for each file.
The script first looks through a text file for a string and collects some stats using line spitting drops the collected figures into Excel and and generates a hit count each time the particular string is found (total)
Then charts are generated using thee collected stats..
Here's my chart generating section...
chart1 = workbook.add_chart({'type': 'line'})
chart1.add_series({
'name': 'My Chart',
'categories': '=Sheet1!$A$2:$A$2200',
'values': '=Sheet1!$B$2:$B$2200',
'line': {'color': 'purple'},
})
I am hoping to generate the chart by referencing the 'total' count in the row count. So I am looking for something along the lines of
'categories': '=Sheet1!$A$2:$A$'+total,
'values': '=Sheet1!$B$2:$B$'+total,
I hope this makes sense? Basically I am looking to have a varying cell row range dependent on the count of hits, is this possible? Or alternatively is there a 'last row' reference in xlsxwriter for this type of circumstance?
Thanks,
MikG

The chart.add_series() method also accepts a list of values so you can do something like this:
chart1.add_series({
'name': 'My Chart',
'categories': ['Sheet1', 1, 0, total -1, 0],
'values': ['Sheet1', 1, 1, total -1, 1],
'line': {'color': 'purple'},
})

Python jsonpath Filter Expression

Background:
I have the following example data structure in JSON:
{'sensor' : [
{'assertions_enabled': 'ucr+',
'deassertions_enabled': 'ucr+',
'entity_id': '7.0',
'lower_critical': 'na',
'lower_non_critical': 'na',
'lower_non_recoverable': 'na',
'reading_type': 'analog',
'sensor_id': 'SR5680 TEMP (0x5d)',
'sensor_reading': {'confidence_interval': '0.500',
'units': 'degrees C',
'value': '42'},
'sensor_type': 'Temperature',
'status': 'ok',
'upper_critical': '59.000',
'upper_non_critical': 'na',
'upper_non_recoverable': 'na'}
]}
The sensor list will actually contain many of these dicts containing sensor info.
Problem:
I'm trying to query the list using jsonpath to return me a subset of sensor dicts that have sensor_type=='Temperature' but I'm getting 'False' returned (no match). Here's my jsonpath expression:
results = jsonpath.jsonpath(ipmi_node, "$.sensor[?(#.['sensor_type']=='Temperature')]")
When I remove the filter expression and just use "$.sensor.*" I get a list of all sensors, so I'm sure the problem is in the filter expression.
I've scanned multiple sites/posts for examples and I can't seem to find anything specific to Python (Javascript and PHP seem to be more prominent). Could anyone offer some guidance please?

The following expression does what you need (notice how the attribute is specified):
jsonpath.jsonpath(impi_node, "$.sensor[?(#.sensor_type=='Temperature')]")

I am using jsonpath-ng which seems to be active (as of 23.11.20) and I provide solution based on to Pedro's jsonpath expression:
data = {
'sensor' : [
{'sensor_type': 'Temperature', 'id': '1'},
{'sensor_type': 'Humidity' , 'id': '2'},
{'sensor_type': 'Temperature', 'id': '3'},
{'sensor_type': 'Density' , 'id': '4'}
]}
from jsonpath_ng.ext import parser
for match in parser.parse("$.sensor[?(#.sensor_type=='Temperature')]").find(data):
print(match.value)
Output:
{'sensor_type': 'Temperature', 'id': '1'}
{'sensor_type': 'Temperature', 'id': '3'}
NOTE: besides basic documentation provided on project's homepage I found additional information in tests.

Numeric sort of list of dictionary objects

I am very new to python programming and have yet to buy a textbook on the matter (I am buying one from the store or Amazon today). In the meantime, can you help me with the following problem I have encountered?
I have an list of dictionary objects like this:
stock = [
{ 'date': '2012', 'amount': '1.45', 'type': 'one'},
{ 'date': '2012', 'amount': '1.4', 'type': 'two'},
{ 'date': '2011', 'amount': '1.35', 'type': 'three'},
{ 'date': '2012', 'amount': '1.35', 'type': 'four'}
]
I would like to sort the list by the amount date column and then by the amount column so that the sorted list looks like this:
stock = [
{ 'date': '2011', 'amount': '1.35', 'type': 'three'},
{ 'date': '2012', 'amount': '1.35', 'type': 'four'},
{ 'date': '2012', 'amount': '1.4', 'type': 'two'},
{ 'date': '2012', 'amount': '1.45', 'type': 'one'}
]
I now think I need to use sorted() but as a beginner I am having difficulties understanding to concepts I see.
I tried this:
from operator import itemgetter
all_amounts = itemgetter("amount")
stock.sort(key = all_amounts)
but this resulted in an list that was sorted alphanumerically rather than numerically.
Can someone please tell me how to achieve this seemingly simple sort? Thank-you!

Your sorting condition is too complicated for an operator.itemgetter. You will have to use a lambda function:
stock.sort(key=lambda x: (int(x['date']), float(x['amount'])))
or
all_amounts = lambda x: (int(x['date']), float(x['amount']))
stock.sort(key=all_amounts)

Start by converting your data into a proper format:
stock = [
{ 'date': int(x['date']), 'amount': float(x['amount']), 'type': x['type']}
for x in stock
]
Now stock.sort(key=all_amounts) will return correct results.
As you appear to be new in programming, here's a word of general advice if I may:
Proper data structure is 90 percent of success. Do not try to work around broken data by writing more code. Create a structure adequate to your task and write as less code as possible.

You can also use the fact that python's sort is stable:
stock.sort(key=lambda x: int(x["amount"]))
stock.sort(key=lambda x: int(x["date"]))
Since the items with the same key keep their relative positions when sorting (they're never swapped), you can build up a complicated sort by sorting multiple times.