I'm using this prometheus json exporter https://github.com/catawiki/json_exporter and
I can't figure out how to use the strftime parameter. It is required to use the format from https://docs.python.org/3/library/time.html#time.strftime. As I understand a string should be defined. If I set:
strftime: '%Y-%m-%d'
and use the variable as
params:
start_date: $strftime
I have this error
Traceback (most recent call last):
File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/local/lib/python2.7/site-packages/json_exporter/main.py", line 257, in run
self.scrape()
File "/usr/local/lib/python2.7/site-packages/json_exporter/main.py", line 279, in scrape
data = render(self.body, variables)
File "/usr/local/lib/python2.7/site-packages/json_exporter/main.py", line 125, in render
return Template(tmpl).safe_substitute(variables)
File "/usr/local/lib/python2.7/string.py", line 207, in safe_substitute
return self.pattern.sub(convert, self.template)
TypeError: expected string or buffer
What is the correct way to use this ?
Thank you.
Everything seems to match with documentation, even if the documentation is not really clear about wether datetime.strftime or time.strftime fields are to be used (but interestingly, both know about %Y, event the latter).
Some things to try:
remove the quotes around your format template.
specify your template directly in the params, instead of using a link
Also, from the error stacktrace, you are running Python 2.7, which means:
the documentation you need is on this page. Not sure there would be a difference though.
you're 2 years out of active support period, and should seriously consider updating your system.
Related
Using cassandra-driver version 3.24.0, I'm able to connect to a cluster and run queries (using protocol 4) without any issues.
However when I call cluster.shutdown(), or let the script completes which calls it automatically I think, I get the following exception :
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "cassandra\cluster.py", line 221, in cassandra.cluster._shutdown_clusters
File "cassandra\cluster.py", line 1759, in cassandra.cluster.Cluster.shutdown
File "cassandra\cluster.py", line 3147, in cassandra.cluster.Session.shutdown
File "cassandra\pool.py", line 493, in cassandra.pool.HostConnection.shutdown
File "C:\Users\myuser\AppData\Local\Programs\Python\Python37\lib\site-packages\cassandra\io\asyncorereactor.py", line 386, in close
ConnectionShutdown("Connection to %s was closed" % self.endpoint))
File "cassandra\connection.py", line 207, in cassandra.connection.DefaultEndPoint.__str__
TypeError: %d format: a number is required, not str
I couldn't find anything about this, it looks like an issue in the library itself but I assume I must be passing something it doesn't like somewhere.
Any ideas what could be causing this ? Weather I use execution profiles or the legacy API, same result.
Thanks
Have the same issue.
Found that in cassandra/connection.py in row 207 there is str method, which uses port as %d. All you need is during cluster creation send int, not str
cluster = Cluster(contact_points=[hostname], port=int(port), execution_profiles=profiles, auth_provider=auth)
I faced the same issue. I type cast the port no to int value.
int(port) and issue is resolved.
On a pipeline defined using the latest Apache Beam SDK for Python 2.2.0, I get this error when running a simple pipeline that reads and writes a BigQuery table.
Since a few rows have timestamps with year < 1900, the read operation fails. How can I patch this dataflow_worker package?
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
(4d31192aa4aec063): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 582, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 167, in execute
op.start()
File "dataflow_worker/native_operations.py", line 38, in dataflow_worker.native_operations.NativeReadOperation.start
def start(self):
File "dataflow_worker/native_operations.py", line 39, in dataflow_worker.native_operations.NativeReadOperation.start
with self.scoped_start_state:
File "dataflow_worker/native_operations.py", line 44, in dataflow_worker.native_operations.NativeReadOperation.start
with self.spec.source.reader() as reader:
File "dataflow_worker/native_operations.py", line 48, in dataflow_worker.native_operations.NativeReadOperation.start
for value in reader:
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativefileio.py", line 198, in __iter__
for record in self.read_next_block():
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativeavroio.py", line 95, in read_next_block
yield self.decode_record(record)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativebigqueryavroio.py", line 110, in decode_record
record, self.source.table_schema)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativebigqueryavroio.py", line 104, in _fix_field_values
record[field.name], field)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativebigqueryavroio.py", line 83, in _fix_field_value
return dt.strftime('%Y-%m-%d %H:%M:%S.%f UTC')
ValueError: year=200 is before 1900; the datetime strftime() methods require year >= 1900
Unfortunately, you cannot patch it to work with timestamps because that is the internal implementation of Google's Apache Beam runner: Dataflow. So you will have to wait until this is fixed by Google (should this be identified as a bug). Please, report it as soon as possible because this is more a limitation of Python's version used rather than a bug.
The problem comes from strftime as you can see in the error. The documentation explicitly mentions it won't work with any year prior to 1900.
A workaround, on your end though, is to convert the timestamp to a string (you can do this in BigQuery as specified in the documentation). And then in your Beam pipeline you can reconvert it again to some timestamp or whatever suits you best.
You also have an example on how to convert a datetimeobject to a string as the template of your error in answer. In the same question there is another answer that explains what has happened with this bug and how has it been solved (in Python) and what you can do. Unfortunately, the solution seems to avoid using strftime at all, and use some alternative instead.
I have a few twitterbots that I run on my raspberryPi. I have most functions wrapped in a try / except to ensure that if something errors it doesn't break the program and continues to execute.
I'm also using Python's Streaming library as my source of monitoring for the tags that I want the bot to retweet.
Here is an issue that happens that kills the program although I have the main function wrapped in a try/except:
Unhandled exception in thread started by <function startBot5 at 0x762fbed0>
Traceback (most recent call last):
File "TwitButter.py", line 151, in startBot5
'<botnamehere>'
File "/home/pi/twitter/bots/TwitBot.py", line 49, in __init__
self.startFiltering(trackList)
File "/home/pi/twitter/bots/TwitBot.py", line 54, in startFiltering
self.myStream.filter(track=tList)
File "/usr/local/lib/python3.4/dist-packages/tweepy/streaming.py", line 445, in filter
self._start(async)
File "/usr/local/lib/python3.4/dist-packages/tweepy/streaming.py", line 361, in _start
self._run()
File "/usr/local/lib/python3.4/dist-packages/tweepy/streaming.py", line 294, in _run
raise exception
File "/usr/local/lib/python3.4/dist-packages/tweepy/streaming.py", line 263, in _run
self._read_loop(resp)
File "/usr/local/lib/python3.4/dist-packages/tweepy/streaming.py", line 313, in _read_loop
line = buf.read_line().strip()
AttributeError: 'NoneType' object has no attribute 'strip'
My setup:
I have a parent class TwitButter.py, that creates an object from the TwitBot.py. These objects are the bots, and they are started on their own thread so they can run independently.
I have a function in the TwitBot that runs the startFiltering() function. It is wrapped in a try/except, but my except code is never triggered.
My guess is that the error is occurring within the Streaming library. Maybe that library is poorly coded and breaks on the line that is specified at the bottom of the traceback.
Any help would be awesome, and I wonder if others have experienced this issue?
I can provide extra details if needed.
Thanks!!!
This actually is problem in tweepy that was fixed by github #870 in 2017-04. So, should be resolved by updating your local copy to latest master.
What I did to discover that:
Did a web search to find the tweepy source repo.
Looked at streaming.py for context on the last traceback lines.
Noticed the most recent change to the file was the same problem.
I'll also note that most of the time you get a traceback from deep inside a Python library, the problem comes from the code calling it incorrectly, rather than a bug in the library. But not always. :)
I have the following code in Python 3.3:
ftpHost="myhost"
link=ftplib.FTP(ftpHost)
link.login(user="me",passwd="alsome")
t=link.nlst()
link.quit()
print(t)
The output I get is
['.','..','dir1','dir2']
In the Python 3 document, it is written that FTP.nlst has been "deprecated since version 3.3: use mlsd() instead". When I tried
ftpHost="myhost"
link=ftplib.FTP(ftpHost)
link.login(user="me",passwd="alsome")
t=link.mlsd()
link.quit()
print(t)
The output I get is
<generator object mlsd at 0x0000000002A0F120>
which is not the desired answer. I can't figure out where within t can I get the directory listing. How can I use mlsd to get the directory listing?
If I try #fourtheye 's suggestion and change to
t=list(link.mlsd)
the error I get is
Traceback (most recent call last):
File "C:/xyz/python-test/python-test.py", line 17, in <module>
main()
File "C:/xyz/python-test/python-test.py", line 12, in main
t=list(link.mlsd())
File "C:\WinPython-64bit-3.3.2.1\python-3.3.2.amd64\lib\ftplib.py", line 556, in mlsd
self.retrlines(cmd, lines.append)
File "C:\WinPython-64bit-3.3.2.1\python-3.3.2.amd64\lib\ftplib.py", line 446, in retrlines
with self.transfercmd(cmd) as conn, \
File "C:\WinPython-64bit-3.3.2.1\python-3.3.2.amd64\lib\ftplib.py", line 386, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "C:\WinPython-64bit-3.3.2.1\python-3.3.2.amd64\lib\ftplib.py", line 352, in ntransfercmd
resp = self.sendcmd(cmd)
File "C:\WinPython-64bit-3.3.2.1\python-3.3.2.amd64\lib\ftplib.py", line 259, in sendcmd
return self.getresp()
File "C:\WinPython-64bit-3.3.2.1\python-3.3.2.amd64\lib\ftplib.py", line 233, in getresp
raise error_perm(resp)
ftplib.error_perm: 500 MLSD not understood
Generators dont keep entire results in the memory. The results will be generated when asked for one, with next function. In order to get the values as a list, use the list function with link.mlsd(), which will internally call next on the generator object.
Just change
t=link.mlsd()
to
t=list(link.mlsd())
Who runs the FTP host? The 500 MLSD not understood means they don't support the MLSD command. That's a later extension to the FTP command set, so they're just out of date. The specification is here:
https://www.rfc-editor.org/rfc/rfc3659
Unless and until they upgrade to a "modern" version of FTP, just continue to use nlst().
Appreciate your helping first, I am new for the python 3.x.
When I try to use Python 3.x to parse the testlink xmlprc server. I got below error, but I can run the code under Python 2.x, any idea?
import xmlrpc.client
server = xmlrpc.client.Server("http://172.16.29.132/SITM/lib/api/xmlrpc.php") //here is my testlink server
print (server.system.listMethods()) //I can print the methods list here
print (server.tl.ping()) // Got error.
Here is the error:
['system.multicall', 'system.listMethods', 'system.getCapabilities', 'tl.repeat', 'tl.sayHello', 'tl.ping', 'tl.setTestMode', 'tl.about', 'tl.checkDevKey', 'tl.doesUserExist', 'tl.deleteExecution', 'tl.getTestSuiteByID', 'tl.getFullPath', 'tl.getTestCase', 'tl.getTestCaseAttachments', 'tl.getFirstLevelTestSuitesForTestProject', 'tl.getTestCaseCustomFieldDesignValue', 'tl.getTestCaseIDByName', 'tl.getTestCasesForTestPlan', 'tl.getTestCasesForTestSuite', 'tl.getTestSuitesForTestSuite', 'tl.getTestSuitesForTestPlan', 'tl.getLastExecutionResult', 'tl.getLatestBuildForTestPlan', 'tl.getBuildsForTestPlan', 'tl.getTotalsForTestPlan', 'tl.getTestPlanPlatforms', 'tl.getProjectTestPlans', 'tl.getTestPlanByName', 'tl.getTestProjectByName', 'tl.getProjects', 'tl.addTestCaseToTestPlan', 'tl.assignRequirements', 'tl.uploadAttachment', 'tl.uploadTestCaseAttachment', 'tl.uploadTestSuiteAttachment', 'tl.uploadTestProjectAttachment', 'tl.uploadRequirementAttachment', 'tl.uploadRequirementSpecificationAttachment', 'tl.uploadExecutionAttachment', 'tl.createTestSuite', 'tl.createTestProject', 'tl.createTestPlan', 'tl.createTestCase', 'tl.createBuild', 'tl.setTestCaseExecutionResult', 'tl.reportTCResult']
Traceback (most recent call last):
File "F:\SQA\Python\Testlink\Test.py", line 5, in <module>
print (server.tl.ping())
File "C:\Python31\lib\xmlrpc\client.py", line 1029, in __call__
return self.__send(self.__name, args)
File "C:\Python31\lib\xmlrpc\client.py", line 1271, in __request
verbose=self.__verbose
File "C:\Python31\lib\xmlrpc\client.py", line 1070, in request
return self.parse_response(resp)
File "C:\Python31\lib\xmlrpc\client.py", line 1164, in parse_response
p.feed(response)
File "C:\Python31\lib\xmlrpc\client.py", line 454, in feed
self._parser.Parse(data, 0)
xml.parsers.expat.ExpatError: junk after document element: line 2, column 0
When I've seen this message before, it happened because the contents of the transported data wasn't escaped for XML transport. The solution was to wrap the data in an XMLRPC Binary object.
In your case, you don't control the server side, so the above isn't a solution for you but it may suggest what the actual problem is.
Also, the Python 2 versus Python 3 difference suggests that there is a text/bytes issue at work.
To help diagnose the issue, set verbose=True so you can see the actual HTTP request/response headers and the XML request/response. That may show you what is at line 2: column 0. You may find that the issue may be with the PHP script not wrapping up binary data in base64 encoding as required by the XMLRPC spec.
Thank you , I find out all the methods list, only 'tl.sayHello', 'tl.ping','tl.about' has this problem, and all of them are pass a string with a PHP automatic loader empty file *.class.php to the parser, other methods are pass a xml file. So I give up to use those methods and the script works fine.