I am loop through json and writing to file and passing that to another funtion to upload using a curl command.
Here is the error message i am getting:
Traceback (most recent call last):
File "D:\python_ws\test.py", line 33, in <module>
main((sys.argv[1:]))
File "D:\python_ws\test.py", line 30, in main
upload_file_bams(out_file)
File "D:\python_ws\test.py", line 7, in upload_file
file = open(rel_file).read()
TypeError: coercing to Unicode: need string or buffer, file found
I tried different ways but seems like i am missing some basics here. any help is appreciated.
Here is the my code:
#!/usr/bin/env python
import urllib2,json,sys,requests,subprocess,os
def upload_file(rel_file):
url = "https://localhost:8080/artifactory/list/default.generic.local/ct/releases/"
user = "john.smith"
file = open(rel_file).read()
cmd = 'curl --verbose --user %s --upload-file %s %s' %(user,file,url)
print 'trying to execute %s' % cmd
x = subprocess.Popen('cmd', shell=True)
#subprocess.call('cmd', shell=True)
retval = x.wait()
def main(argv):
#environment
env = sys.argv[1]
rel_name=sys.argv[2]
consul_url=("http://localhost:9090") % (env)
list_services = "/v1/catalog/services"
services_url = consul_url+list_services
list_of_services = urllib2.urlopen(services_url).read()
each_service = json.loads(list_of_services)
#to remove keys with blank values
newdict = dict([(vkey,vdata) for vkey, vdata in each_service.iteritems() if(vdata) ])
try:
out_file = open(rel_name,"wb")
json.dump(newdict,out_file, indent=4)
finally:
out_file.close()
#uploading release json file to BAMS
upload_file(out_file)
if __name__ == "__main__":
main((sys.argv[1:]))
When you call upload_file(), you pass it out_file which is of type file instead of string (file name). The function open() takes for the first argument the file name that you want to open.
Related
I am opening and reading in a json file called, say, my_file.json located at ..\config\my_file.json in a python file called run_test.py. Ultimately, I would like to turn the JSON object into a python dictionary and do other stuff with it in python. I am running the below code from my terminal to execute the class:
python -c "from run_test import Update; x = Update('my_file.json'); print(x.why())"
This returns:
Traceback (most recent call last): File "<string>", line 1, in <module> TypeError: why() missing 1 required positional argument: 'file_details'
I can't figure out why I'm getting this error. When I run (with return f in the body of read_file):
python -c "from run_test import Update; x = Update('my_file.json'); print(x.read_file())
This prints the dictionary I need. However, I would like to pass it to why so I can do other stuff with that dictionary. Yet, when I pass f to why, I get the above error. Why?
Here is the class I'm using:
import json
class Update:
def __init__(self, config_file_name):
self.config_file_name = config_file_name
def why(self, file_details):
return file_details
def read_file(self):
f = json.load(open('..\config\\' + self.config_file_name))
self.why(f)
I have a fairly simple use-case but i'm not understanding the error message i'm receiving.
I'm using the requests and pyral modules, pyral (http://pyral.readthedocs.io/en/latest/interface.html#) is really just a wrapper for Rally's Restful api. My goal is to get a file (attachment) from a Rally (a CA product) UserStory and store it to a local file system.
For context, here is my environment setup (authenticate to Rally and create an object). I've obviously removed authentication information.
from pyral import Rally, rallyWorkset
options = [arg for arg in sys.argv[1:] if arg.startswith('--')]
args = [arg for arg in sys.argv[1:] if arg not in options]
server, user, password, apikey, workspace, project = rallyWorkset(options)
rally = Rally(server='rally1.rallydev.com',
user='**********', password='***********',
apikey="**************",
workspace='**************', project='**************',
server_ping=False)
After that I get a response object for just one user story (see the query for US845), i do this just to simplify the problem.
r = rally.get('UserStory', fetch = True, projectScopeDown=True, query = 'FormattedID = US845')
and then I use the built-in iterator to get the user story from the RallyRESTResponse object.
us = r.next()
from there it feels like I should be able to easily use the getAttachment() method that accepts a artifact (us) and filename (name of an attachment). I'm able to use getAttachmentNames(us) to return a list of attachment names. The issue arrises when i try something like
attachment_names = rally.getAttachmentNames(us) #get attachments for this UserStory
attachment_file = rally.getAttachment(us, attachment_names[0]) #Try to get the first attachment
returns an error like this
Traceback (most recent call last):
File "<ipython-input-81-a4a342a59c5a>", line 1, in <module>
attachment_file = rally.getAttachment(us, attachment_names[0])
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1700, in getAttachment
att.Content = base64.decodebytes(att_content.Content) # maybe further txfm to Unicode ?
File "C:\Miniconda3\lib\base64.py", line 552, in decodebytes
_input_type_check(s)
File "C:\Miniconda3\lib\base64.py", line 520, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
I receive a similar error if i try to use
test_obj = rally.getAttachments(us)
Which returns an error like this:
Traceback (most recent call last):
File "<ipython-input-82-06a8cd525177>", line 1, in <module>
rally.getAttachments(us)
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1721, in getAttachments
attachments = [self.getAttachment(artifact, attachment_name) for attachment_name in attachment_names]
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1721, in <listcomp>
attachments = [self.getAttachment(artifact, attachment_name) for attachment_name in attachment_names]
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1700, in getAttachment
att.Content = base64.decodebytes(att_content.Content) # maybe further txfm to Unicode ?
File "C:\Miniconda3\lib\base64.py", line 552, in decodebytes
_input_type_check(s)
File "C:\Miniconda3\lib\base64.py", line 520, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
It seems that i'm fundamentally misunderstanding the parameters that this method requires? Has anyone been able to do this successfully before? For what it's worth i have no issues using the addAttachment() method with a workflow similar to the above. I've tried converting the filename (string) with the bytes() method to utf-8 but that didn't help.
I've also looked at this example in the pyral source, but i receive exactly the same error when trying to execute that.
https://github.com/klehman-rally/pyral/blob/master/examples/get_attachments.py
It looks like the issue in restapi.py script - there is no decodebytes method in base64 library:
att.Content = base64.decodebytes(att_content.Content)
All available methods are described at:
RFC 3548: Base16, Base32, Base64 Data Encodings
So, workaround is to replace decodebytes by base64.b64decode in restapi.py. At least, it works me.
E.g. location at Mac OS X:
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyral/restapi.py
I have used the below code to get all attachment since getAttachments is not working as expected. it will create a file in the current dir with the same name.
import sys
import string
import base64
from pyral import rallyWorkset, Rally,RallyRESTResponse
rally = Rally(server, user=USER_NAME, password=PASSWORD, workspace=workspace, project=project)
criterion = 'FormattedID = US57844'
response = rally.get('HierarchicalRequirement', query=criterion, order="FormattedID",pagesize=200, limit=400, projectScopeDown=True)
artifact = response.next()
context, augments = rally.contextHelper.identifyContext()
for att in artifact.Attachments:
resp = rally._getResourceByOID(context, 'AttachmentContent', att.Content.oid, project=None)
if resp.status_code not in [200, 201, 202]:
break
res = RallyRESTResponse(rally.session, context, "AttachmentContent.x", resp, "full", 1)
if res.errors or res.resultCount != 1:
print("breaking the for loop")
att_content = res.next()
cont = att_content.Content
x = base64.b64decode(cont)
output = open(att.Name, 'wb')
output.write(x)
Hi I tried to generate the genesis file but get this error:
C:\Python34>python mk_genesis_block.py --extradata 0x11bbe8db4e347b4e8c937c1c837
0e4b5ed33adb3db69cbdb7a38e1e50b1b82fa > genesis_block.json
File "mk_genesis_block.py", line 293
print json.dumps(evaluate(), indent=4)
^
SyntaxError: invalid syntax
Edit:
Here is the surrounding lines:
if __name__ == '__main__':
print json.dumps(evaluate(), indent=4)
Then it's EOF. The whole file can be viewed here
Since the offending line seems to be only output, I commented it and got another error:
C:\Python34>python -tt mk_genesis_block.py --extradata 0x11bbe8db4e347b4e8c937c1
c8370e4b5ed33adb3db69cbdb7a38e1e50b1b82fa > genesis_block.json
Traceback (most recent call last):
File "mk_genesis_block.py", line 124, in <module>
EXTRADATA = (d[2:] if d[:2] == '0x' else d).decode('hex')
AttributeError: 'str' object has no attribute 'decode'
which in conjunction with the other error makes me wonder whether a string instead of a json object is being operated on? Here is the whole arg parsing part:
# Grab the extra data command line argument
if '--extradata' in sys.argv:
d = (sys.argv+[None])[sys.argv.index('--extradata') + 1]
EXTRADATA = (d[2:] if d[:2] == '0x' else d).decode('hex')
else:
EXTRADATA = ''
I also made a test file importing the json package, dumps and decode methods work.
print in python3 is a method not a statement print( "text" ) ... also I believe str.decode was remove in python3 ... instead use codecs.decode(my_str,encoding)
i am unable to generate pdf report using ExportReport function of Arcgis 10.2. getting error of IOError: Could not open report template
tried different template files but still getting this error.
Template directory is correct. checked all the directory positions.
Code
import arcpy
mxd = arcpy.mapping.MapDocument(r"C:\Users\Abrar ahmad\Documents\ArcGIS\New_Rwp_Cencus(12-17-2014).mxd")
df = arcpy.mapping.ListDataFrames(mxd)[0]
for lyr in arcpy.mapping.ListLayers(mxd, "",df):
if lyr.name == "New_Districts_5_without_limit":
arcpy.mapping.ExportReport(lyr,r"C:\Users\Abrar ahmad\Documents\ArcGIS\test.rlf",r"C:\Users\Abrar ahmad\Documents\ArcGIS\ProjectReport2.pdf","USE_RLF")
del mxd
Complete Error Data:
Runtime error
Traceback (most recent call last):
File "", line 7, in
File "c:\program files (x86)\arcgis\desktop10.2\arcpy\arcpy\utils.py", line 181, in fn_
return fn(*args, **kw)
File "c:\program files (x86)\arcgis\desktop10.2\arcpy\arcpy\mapping.py", line 515, in ExportReport
return report_source._arc_object.ExportReport(*gp_fixargs((report_layout_file, output_file, dataset_option, report_title, starting_page_number, page_range, report_definition_query, extent, field_map), True))
IOError: Could not open report template
Execute your code inside try/except/finally and see if this gives some hint:
import arcpy
import sys
import traceback
mxd = arcpy.mapping.MapDocument(r"C:\Users\Abrar ahmad\Documents\ArcGIS\New_Rwp_Cencus(12-17-2014).mxd")
try:
df = arcpy.mapping.ListDataFrames(mxd)[0]
for lyr in arcpy.mapping.ListLayers(mxd, "",df):
if lyr.name == "New_Districts_5_without_limit":
arcpy.mapping.ExportReport(lyr,r"C:\Users\Abrar ahmad\Documents\ArcGIS\test.rlf",r"C:\Users\Abrar ahmad\Documents\ArcGIS\ProjectReport2.pdf","USE_RLF")
except arcpy.ExecuteError:
# Get the tool error messages
msgs = arcpy.GetMessages(2)
# Return tool error messages for use with a script tool
arcpy.AddError(msgs)
# Print tool error messages for use in Python/PythonWin
print msgs
except:
# Get the traceback object
tb = sys.exc_info()[2]
tbinfo = traceback.format_tb(tb)[0]
# Concatenate information together concerning the error into a message string
pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info()[1])
msgs = "ArcPy ERRORS:\n" + arcpy.GetMessages(2) + "\n"
# Return python error messages for use in script tool or Python Window
arcpy.AddError(pymsg)
arcpy.AddError(msgs)
# Print Python error messages for use in Python / Python Window
print pymsg + "\n"
print msgs
finally:
del mxd
I have two python files. My test.py import td.py file witch i found internet. Td.py file looking signals from TelldusCenter program.
Now if i run test.py file it shows me signals what i get from TelldusCenter app and output is something like: "Door - ON"
Now i like to print that "Door - ON" text to file but i dont know how.
Here is my test.py file
#!/usr/bin/env python
import td
import time
def myDeviceEvent(deviceId, method, data, callbackId):
print '%s' %( td.getName(deviceId) )+' - %s' %(td.methodsReadable.get(method, 'Unknown' ))
td.registerDeviceEvent(myDeviceEvent)
try:
while(1):
time.sleep(1)
except KeyboardInterrupt:
print 'KeyboardInterrupt received, exiting'
"td.registerDeviceEvent(myDeviceEvent)" print output to terminal now. I try to print that to file but it just give me error.
a = open("output.txt", "w")
a.write(td.registerDeviceEvent(myDeviceEvent))
Traceback (most recent call last): File "testi.py", line 11, in
a.write(td.registerDeviceEvent(myDeviceEvent)) TypeError: expected a character buffer object
From my interpretation of the code, td.registerDeviceEvent(myDeviceEvent) registers a callback. It does not produce a string itself. This is why you cannot output the 'result' of the registration.
Instead try this:
#!/usr/bin/env python
import td
import time
a = open("output.txt", "w")
def myDeviceEvent(deviceId, method, data, callbackId):
a.write('%s' %( td.getName(deviceId) ) + ' - %s' %(td.methodsReadable.get(method, 'Unknown')
td.registerDeviceEvent(myDeviceEvent)
Change
def myDeviceEvent(deviceId, method, data, callbackId):
print '%s' %( td.getName(deviceId) )+' - %s' %(td.methodsReadable.get(method, 'Unknown' ))
to
def myDeviceEvent(deviceId, method, data, callbackId):
with open("Output.txt", "w") as outputFile:
outputFile.write('%s' %( td.getName(deviceId) )+' - %s' %(td.methodsReadable.get(method, 'Unknown' )))
You can use with statement to handle files and its scope. You dont have to worry about the closing the file properly, when you use with. That takes care of it.
Edit: You can use modern string formatting like this. Read more about it here http://docs.python.org/2/library/string.html#string-formatting
def myDeviceEvent(deviceId, method, data, callbackId):
with open("Output.txt", "w") as outputFile:
outputFile.write('{} - {}'.format(td.getName(deviceId), td.methodsReadable.get(method, 'Unknown')))
You should consider the logging module with a basic configuration :
import logging
FORMAT = '%(asctime)s - %(message)s'
logging.basicConfig(format=FORMAT, filename='Output.txt', level=logging.INFO)
logging.info('My message')
File Output.txt :
2013-10-17 09:26:08,496 - My message