I have some campaigns that I want to modify but I've to call the server and export all the campaign settings modify or add the information I want and send it back to the server.
My modifications are stored in an SQLite db, so first, I query the db looking for the account IDs (I can have many campaigns that are in different accounts), then I look for campaigns per account.
theCursor.execute('SELECT * FROM SandboxPubTB')
rows = theCursor.fetchall()
for row in rows:
accounts.append(row[7])
for account in set(accounts):
theCursor.execute('SELECT CAMPAIGNNAME, CAMPAIGNID FROM SandboxPubTB WHERE ACCOUNTID =?', (account,) )
campaignRows = theCursor.fetchall()
for campaign in campaignRows:
campId= campaign[0] + "," + campaign[1] + "," + account
campaigns.append(campId)
My campId list contains the campaign name, campaign ID and the campaigns account ID in each entry.
After that, I iterate through the list and assign the values to a variable at the campaign level:
for campaignNameId in set(campaigns):
campaignNameId = campaignNameId.split(',')
campaignName = campaignNameId[0]
campaignId = campaignNameId[1]
campaignAccount = campaignNameId[2]
Now it's time to call the server to export the campaign settings:
targetUrl = "https://sites.com/api/1.0/" + campaignAccount + "/campaigns/"+ campaignId
resp = requests.get(url=targetUrl, headers=header)
r = resp.json()
temp = r['publisher_bid_modifier']['values']
If I print temp this is what I get:
[{
'target': 'msn-can',
'cpc_modification': 0.5
}, {
'target': 'msn-can-home',
'cpc_modification': 0.5
}, {
'target': 'smartify-journalistatecom',
'cpc_modification': 0.5
}, {
'target': 'foxnews-iosapp',
'cpc_modification': 1.22
}]
And finally, here is where I'm stuck. What I'm trying to do is iterate through the dictionary list above and if "target" value exists and "cpc_modification" is different than what I've in the db I'll change the cpc value. If "target" doesn't exist I want to append "target" and "cpc_modification" to the dictionary list.
I succeeded with the first part, but the append part is different. In the elif, even if I use an else, the temp.append triggers an infinite loop and I have no idea why.
for dt in r['publisher_bid_modifier']['values']:
#print(dt['target'])
#if dt['target']:
theCursor.execute('SELECT ADJUST, SITE FROM SandboxPubTB WHERE CAMPAIGNNAME =?', (campaignName,) )
campaignRows = theCursor.fetchall()
for t in campaignRows:
if t[1] == dt['target'] :
dt['cpc_modification'] = "{:.2f}".format((int(t[0]) / 100) + 1)
elif dt['target'] not in temp:
temp.append("{'target': "'" + t[1] + "'", 'cpc_modification': "'" + str(t[0]) + "'"}")
That's weird because I've tried to emulate the same behavior with a local variable and it seems to work without problem.
data = [{
"target": "publisher1",
"cpc_modification": 1.5
},
{
"target": "publisher2",
"cpc_modification": 0.9
}
]
for t in data:
if t['target'] == "publisher10":
t['cpc_modification'] = 1.9
else:
data.append({'target': 'publisher10', 'cpc_modification': 12})
print(t['target'], t['cpc_modification'])
print(data)
I've tried many things, and I can't figure out what is wrong.
I believe your problem is in here. You are iterating over 'data' but you are also appending stuff to 'data' as you are iterating over it. You should create a new variable to append to.
temp = data
for t in data:
if t['target'] == "publisher10":
t['cpc_modification'] = 1.9
else:
temp.append({'target': 'publisher10', 'cpc_modification': 12})
print(t['target'], t['cpc_modification'])
print(temp)
I made an example here of what I believe is happending
Related
*New to Programming
Question: I need to use the below "Data" (two rows as arrays) queried from sql and use it to create the message structure below.
data from sql using fetchall()
Data = [[100,1,4,5],[101,1,4,6]]
##expected message structure
message = {
"name":"Tom",
"Job":"IT",
"info": [
{
"id_1":"100",
"id_2":"1",
"id_3":"4",
"id_4":"5"
},
{
"id_1":"101",
"id_2":"1",
"id_3":"4",
"id_4":"6"
},
]
}
I tried to create below method to iterate over the rows and then input the values, this is was just a starting, but this was also not working
def create_message(data)
for row in data:
{
"id_1":str(data[0][0],
"id_2":str(data[0][1],
"id_3":str(data[0][2],
"id_4":str(data[0][3],
}
Latest Code
def create_info(data):
info = []
for row in data:
temp_dict = {"id_1_tom":"","id_2_hell":"","id_3_trip":"","id_4_clap":""}
for i in range(0,1):
temp_dict["id_1_tom"] = str(row[i])
temp_dict["id_2_hell"] = str(row[i+1])
temp_dict["id_3_trip"] = str(row[i+2])
temp_dict["id_4_clap"] = str(row[i+3])
info.append(temp_dict)
return info
Edit: Updated answer based on updates to the question and comment by original poster.
This function might work for the example you've given to get the desired output, based on the attempt you've provided:
def create_info(data):
info = []
for row in data:
temp_dict = {}
temp_dict['id_1_tom'] = str(row[0])
temp_dict['id_2_hell'] = str(row[1])
temp_dict['id_3_trip'] = str(row[2])
temp_dict['id_4_clap'] = str(row[3])
info.append(temp_dict)
return info
For the input:
[[100, 1, 4, 5],[101,1,4,6]]
This function will return a list of dictionaries:
[{"id_1_tom":"100","id_2_hell":"1","id_3_trip":"4","id_4_clap":"5"},
{"id_1_tom":"101","id_2_hell":"1","id_3_trip":"4","id_4_clap":"6"}]
This can serve as the value for the key info in your dictionary message. Note that you would still have to construct the message dictionary.
I want to update the value of Entry1 using upsert. I have a sensor that returns the value of Entry1. If sensor is blocked, the value is true. If sensor is not blocked then the value is False.
machineOne = None
oneIn = 1
while True:
global machineOneId
global userId
try:
if Entry1.get_value() and oneIn < 2:
machineOne = Entry1.get_value()
print('entered looopp ONeeeE', machineOne)
machine1 = {
'Entry1': Entry1.get_value(),
'Exit1': Exit1.get_value(),
'id': 'test'
}
result = Machine1.insert_one(machine1)
myquery = {"Entry1": 'true'}
newvalues = {"$set": {"id": result.inserted_id}}
#result = Machine1.insert_one(machine1)
Machine1.update_one(myquery, newvalues)
userId = result.inserted_id
oneIn += 1
print('added', result.inserted_id, oneIn)
elif machineOne:
print('entered looopp', userId)
myquery = {"id": userId}
newvalues = {"$set": {"id": Entry1.get_value()}}
upsert = True
#result = Machine1.insert_one(machine1)
Machine1.update_one(myquery, newvalues)
if Exit1.get_value():
print('added',)
finally:
print('nothings happened', machineOne)
what is expected: i should be able to update the Entry1 from true to false in the same log, displayed in robo3t
Good Afternoon #digs10 ,
I read your post and I think the error is how you locate the document that you want to update.
I remember that MongoDB document primary key is "_id" instead of "id". You could take a look here MongoDB Documents
For what I see in the code (I don't know Python but it is readable), you are referring to the document "Entry1" using the field "id" instead of "_id.
Try modifing the line myquery = {"id": userId} for myquery = {"_id": userId}.
I hope this answer can help you.
Best Regards,
JB
P.S: I saw this question in my email and I took a quick read of it, If I misunderstood it, please let me know.
I have this JSON file.
{
"reviewers":[
{
"user":{
"name":"keyname",
"emailAddress":"John#email",
"id":3821,
"displayName":"John Doe",
"active":true,
"slug":"jslug",
"type":"NORMAL",
"link":{
"url":"/users/John",
"rel":"self"
},
},
"role":"REVIEWER",
"approved":true
},
{
"user":{
"name":"keyname2",
"emailAddress":"Harry#email",
"id":6306,
"displayName":"Harry Smith",
"active":true,
"slug":"slug2",
"link":{
"type":"NORMAL",
"url":"/users/Harry",
"rel":"self"
},
},
"role":"REVIEWER",
"approved":false
}
],
}
Initially, I was using a snippet of code that would go through and grab the full names of the reviewers.
def get_reviewers(json):
reviewers = ""
for key in json["reviewers"]:
reviewers += key["user"]["displayName"] + ", "
reviewers = reviewers[:-2]
return reviewers
which would return "John Doe, Harry Smith". However, now I'm trying to get it so that the script will return a (A) next to the name of the user if their tag equals true "approved"=true.
So for example the code above would get the names, then see that John's approved tag is true and Harry's is false, then return "John Doe(A), Harry Smith". I'm just not sure where to even begin to do this. Can anyone point me in the right direction?
This is what I've been trying so far but obviously it isn't working as I'd like it to.
def get_reviewers(stash_json):
reviewers = ""
for key in stash_json["reviewers"]:
if stash_json["reviewers"][0]["approved"] == true:
reviewers += key["user"]["displayName"] + "(A)" + ", "
else:
reviewers += key["user"]["displayName"] + ", "
reviewers = reviewers[:-2]
return reviewers
which outputs Jason Healy(A), Joan Reyes(A)
This is what my stash_json outputs when put through pprint.
You probably want something along the lines of this:
def get_reviewers(stash_json):
reviewers = ""
for item in stash_json["reviewers"]:
if item["approved"]:
reviewers += item["user"]["displayName"] + "(A)" + ", "
else:
reviewers += item["user"]["displayName"] + ", "
reviewers = reviewers[:-2]
return reviewers
I think part of your confusion comes from the fact that "reviewers" is a list of dict elements, and each dict element has a key-value approved, but also a key "user" which value itself is another dict.
Read the JSON file carefully, and for debugging purposes, use plenty of
print(...)
print(type(...)) # whether something is a dict, list, str, bool etc
or
from pprint import pprint # pretty printing
pprint(...)
This looks like a good place to use join and list comprehension:
def get_reviewers(stash_json):
return ", ".join([item['user']['displayName'] + ('(A)' if item['approved'] else '') for item in stash_json['reviewers']])
When I query the AdWords API to get search volume data and trends through their TargetingIdeaSelector using the Python client library the returned data looks like this:
(TargetingIdeaPage){
totalNumEntries = 1
entries[] =
(TargetingIdea){
data[] =
(Type_AttributeMapEntry){
key = "KEYWORD_TEXT"
value =
(StringAttribute){
Attribute.Type = "StringAttribute"
value = "keyword phrase"
}
},
(Type_AttributeMapEntry){
key = "TARGETED_MONTHLY_SEARCHES"
value =
(MonthlySearchVolumeAttribute){
Attribute.Type = "MonthlySearchVolumeAttribute"
value[] =
(MonthlySearchVolume){
year = 2016
month = 2
count = 2900
},
...
(MonthlySearchVolume){
year = 2015
month = 3
count = 2900
},
}
},
},
}
This isn't JSON and appears to just be a messy Python list. What's the easiest way to flatten the monthly data into a Pandas dataframe with a structure like this?
Keyword | Year | Month | Count
keyword phrase 2016 2 10
The output is a sudsobject. I found that this code does the trick:
import suds.sudsobject as sudsobject
import pandas as pd
a = [sudsobject.asdict(x) for x in output]
df = pd.DataFrame(a)
Addendum: This was once correct but new versions of the API (I tested
201802) now return a zeep.objects. However, zeep.helpers.serialize_object should do the same trick.
link
Here's the complete code that I used to query the TargetingIdeaSelector, with requestType STATS, and the method I used to parse the data to a useable dataframe; note the section starting "Parse results to pandas dataframe" as this takes the output given in the question above and converts it to a dataframe. Probably not the fastest or best, but it works! Tested with Python 2.7.
"""This code pulls trends for a set of keywords, and parses into a dataframe.
The LoadFromStorage method is pulling credentials and properties from a
"googleads.yaml" file. By default, it looks for this file in your home
directory. For more information, see the "Caching authentication information"
section of our README.
"""
from googleads import adwords
import pandas as pd
adwords_client = adwords.AdWordsClient.LoadFromStorage()
PAGE_SIZE = 10
# Initialize appropriate service.
targeting_idea_service = adwords_client.GetService(
'TargetingIdeaService', version='v201601')
# Construct selector object and retrieve related keywords.
offset = 0
stats_selector = {
'searchParameters': [
{
'xsi_type': 'RelatedToQuerySearchParameter',
'queries': ['donald trump', 'bernie sanders']
},
{
# Language setting (optional).
# The ID can be found in the documentation:
# https://developers.google.com/adwords/api/docs/appendix/languagecodes
'xsi_type': 'LanguageSearchParameter',
'languages': [{'id': '1000'}],
},
{
# Location setting
'xsi_type': 'LocationSearchParameter',
'locations': [{'id': '1027363'}] # Burlington,Vermont
}
],
'ideaType': 'KEYWORD',
'requestType': 'STATS',
'requestedAttributeTypes': ['KEYWORD_TEXT', 'TARGETED_MONTHLY_SEARCHES'],
'paging': {
'startIndex': str(offset),
'numberResults': str(PAGE_SIZE)
}
}
stats_page = targeting_idea_service.get(stats_selector)
##########################################################################
# Parse results to pandas dataframe
stats_pd = pd.DataFrame()
if 'entries' in stats_page:
for stats_result in stats_page['entries']:
stats_attributes = {}
for stats_attribute in stats_result['data']:
#print (stats_attribute)
if stats_attribute['key'] == 'KEYWORD_TEXT':
kt = stats_attribute['value']['value']
else:
for i, val in enumerate(stats_attribute['value'][1]):
data = {'keyword': kt,
'year': val['year'],
'month': val['month'],
'count': val['count']}
data = pd.DataFrame(data, index = [i])
stats_pd = stats_pd.append(data, ignore_index=True)
print(stats_pd)
I am querying a dataset that works like this: each time I query it produces a different value of Bytes_Written and Bytes_Read. What I cannot seem to accomplish is to subtract the current value from the previous value and keep doing this every second.
Here is what the data looks like:
{
"Number of Devices": 2,
"Block Devices": {
"bdev0": {
"Backend_Device_Path": "/dev/disk/by-path/ip-192.168.26.1:3260-iscsi-iqn.2010-10.org.openstack:volume-d1c8e7c6-8c77-444c-9a93-8b56fa1e37f2-lun-010.0.0.142",
"Capacity": "2147483648",
"Guest_Device_Name": "vdb",
"IO_Operations": "97069",
"Bytes_Written": "34410496",
"Bytes_Read": "363172864"
},
"bdev1": {
"Backend_Device_Path": "/dev/disk/by-path/ip-192.168.26.1:3260-iscsi-iqn.2010-10.org.openstack:volume-b27110f9-41ba-4bc6-b97c-b5dde23af1f9-lun-010.0.0.146",
"Capacity": "2147483648",
"Guest_Device_Name": "vdb",
"IO_Operations": "93",
"Bytes_Written": "0",
"Bytes_Read": "380928"
}
}
}
The code that queries the data:
def counterVolume_one():
#Get data
url = 'http://url:8080/vrio/blk'
r = requests.get(url)
data = r.json()
wanted = {'Bytes_Written', 'Bytes_Written', 'IO_Operation'}
for d in data['Block Devices'].itervalues():
values = {k: v for k, v in d.iteritems() if k in wanted}
print json.dumps(values)
counterVolume_one()
The way I want to get the output is:
{"IO_Operations": "97069", "Bytes_Read": "363172864", "Bytes_Written": "34410496"}
{"IO_Operations": "93", "Bytes_Read": "380928", "Bytes_Written": "0"}
Here is what I want to accomplish:
first time query = first set of values
after 1 sec
second time query = first set of values-second set of values
after 1 sec
third time query = second set of values-third set of values
Expected output would be a json object as below
{'bytes-read': newvalue, 'bytes-written': newvalue, 'io_operations': newvalue}
The simplest fix may be to modify the counterVolume_one() function so that it accepts parameters that define the current state, and that you'd update as you collect data. For example, the following code collects and sums the the fields your interested in from the JSON documents:
FIELDS = ('Bytes_Written', 'Bytes_Read', 'IO_Operation')
def counterVolume_one(state):
url = 'http://url:8080/vrio/blk'
r = requests.get(url)
data = r.json()
for field in FIELDS:
state[field] += data[field]
return state
state = {"Bytes_Written": 0, "Bytes_Read": 0, "IO_Operation": 0}
while True:
counterVolume_one(state)
time.sleep(1)
for field in FIELDS:
print("{field:s}: {count:d}".format(field=field,
count=state[field]))
A more correct fix might be to use a class to hold the state, and that has a method that updates the state. But, I think following the idea above will get you the solution quickest.