I have been using AIOKafka for some time, I had no problems with it, until today.
Strange TypeError shows up when I try to send a message using AIOKafkaProducer.send_and_wait. I also posted this question as issue on AIOKafka's github repository, but it looks like they are kind of inactive. Maybe here someone can help me.
here is the code:
import asyncio
from aiokafka import AIOKafkaConsumer, AIOKafkaProducer
loop = asyncio.get_event_loop()
producer = AIOKafkaProducer(loop=loop, bootstrap_servers="localhost:9092")
async def _initialize(prod, future):
await prod.start()
await prod.send_and_wait("main_topic", str.encode("hello!!"))
future = asyncio.Future()
task = asyncio.ensure_future(_initialize(producer, future))
loop.run_until_complete(task)
print("loop ended!")
loop.close()
here is the error message I get:
yilmazali#yilmazali:~$ python3 aiokafkatest.py
Unexpected error in sender routine
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/aiokafka/producer/producer.py", line 374, in _sender_routine
task.result()
File "/usr/local/lib/python3.6/dist-packages/aiokafka/producer/producer.py", line 418, in _send_produce_req
response = yield from self.client.send(node_id, request)
File "/usr/local/lib/python3.6/dist-packages/aiokafka/client.py", line 415, in send
request, expect_response=expect_response)
File "/usr/local/lib/python3.6/dist-packages/aiokafka/conn.py", line 165, in send
message = header.encode() + request.encode()
File "/usr/local/lib/python3.6/dist-packages/kafka/util.py", line 159, in __call__
return self.method()(self.target(), *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/struct.py", line 42, in _encode_self
[self.__dict__[name] for name in self.SCHEMA.names]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in encode
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in <listcomp>
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in encode
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in <listcomp>
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in encode
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in <listcomp>
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in encode
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in <listcomp>
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in encode
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in <listcomp>
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 93, in encode
return Int32.encode(len(value)) + value
TypeError: object of type '_io.BytesIO' has no len()
I did no changes in my kafka structure or libraries. My kafka broker looks fine. I can produce/consume messages with shell scripts.
I had no problems with AIOKafka for the last 2-3 months, the code above worked fine. Out of nowhere, this error showed up and I'm wondering what the problem is.
Any help will be appreciated.
With kindest regards,
Ali
--
update: We ran this code segment on a friend's computer, it worked fine. I advertised my kafka to outside and she successfully wrote to my local kafka topic with the code above. AIOKafka library versions are 0.4.0 on both machines. Also asyncio versions are 3.4.3 on both machines. In short, the problem is not about my kafka or libraries. Something is wrong with my machine but god knows what specifically causes this.
Finally worked on my machine. I simply uninstalled and installed aiokafka module.
Although I am not satisfied with this solution and I would like to venture deeper to the heart of problem, I'm glad I can continue doing my work now.
Hope this helps to fellow strangers with same problem.
May be you've updated kafka-python to version higher than 1.3.5. I did so and aiokafka started failing. I returned at kafka-python 1.3.5 and it seemed OK
Related
So I found this project
https://codereview.stackexchange.com/questions/192281/download-song-or-video-from-youtube-using-pytube
online and it allows you to dowload youtube videos from online but when I enter the link and click download it gives me this error and I have no idea what it means and I have tried to search google for the answer but I can't find any solutions so I hope that someone could help me.
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\bob\AppData\Local\Programs\Python\Python37\lib\tkinter\
__init__.py", line 1705, in __call__
return self.func(*args)
File "C:\Users\bob\Desktop\youtube.py", line 104, in checkYoutubeLink
self.downloadWindow()
File "C:\Users\bob\Desktop\youtube.py", line 112, in downloadWindow
self.app = SecondPage(self.newWindow, self.youtubeEntryVar.get(), self.Folde
rName, self.ChoicesVar.get())
File "C:\Users\bob\Desktop\youtube.py", line 134, in __init__
self.yt = YouTube(self.youtubeEntry)
File "C:\Users\bob\AppData\Local\Programs\Python\Python37\lib\site-pac
kages\pytube\__main__.py", line 88, in __init__
self.prefetch_init()
File "C:\Users\bob\AppData\Local\Programs\Python\Python37\lib\site-pac
kages\pytube\__main__.py", line 96, in prefetch_init
self.prefetch()
File "C:\Users\bob\AppData\Local\Programs\Python\Python37\lib\site-pac
kages\pytube\__main__.py", line 170, in prefetch
age_restricted=self.age_restricted,
File "C:\Users\bob\AppData\Local\Programs\Python\Python37\lib\site-pac
kages\pytube\extract.py", line 121, in video_info_url
group=0,
File "C:\Users\bob\AppData\Local\Programs\Python\Python37\lib\site-pac
kages\pytube\helpers.py", line 65, in regex_search
.format(pattern=pattern),
pytube.exceptions.RegexMatchError: regex pattern (\W[\'"]?t[\'"]?: ?[\'"](.+?)[\
'"]) had zero matches
Since this error was coming from pytube, I did a quick search of their issues in github. This is the issue for the problem:
https://github.com/nficano/pytube/issues/381
It says it is fixed in version 9.5.0, pip install -U pytube should install the fixed version.
traceback is the sequence of unreturned-from calls made up until the point where the actual error occurred. It helps you identify not only the specific error, but the context in which it occurred.
I am trying to interface with an Ocean optics spectrometer using seabreeze (seabreeze) and pyseabreeze (pyseabreeze).
If I enter python from the command prompt (cmd) in windows and enter the following code line by line, it works. However, if I put it in a script (spec_test.py) and try running it from cmd.exe by using 'python spec_test.py', it will not work.
I have searched around a lot for similar problems but none seem to cover the issue I'm having. I am running Windows 7 64 bit, python 3.5.2 installed using anaconda. I also had to install pyusb and libusb to use pyseabreeze.
The code:
import seabreeze
seabreeze.use('pyseabreeze')
import seabreeze.spectrometers as sb
devs = sb.list_devices()
print(devs)
spec = sb.Spectrometer(devs[0])
print(spec.model)
and the error message I get:
[<SeaBreezeDevice USB2000PLUS:FLMS02379>]
Traceback (most recent call last):
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\common.py", line 14, in decorated_func
return func(*args, **kwargs)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\spectrometer.py", line 46, in open
self.open_device(device.handle)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\communication.py", line 37, in open_device
device.set_configuration()
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\core.py", line 869, in set_configuration
self._ctx.managed_set_configuration(self, configuration)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\core.py", line 148, in managed_set_configuration
self.backend.set_configuration(self.handle, cfg.bConfigurationValue)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\backend\libusb0.py", line 493, in set_configuration
_check(_lib.usb_set_configuration(dev_handle, config_value))
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\usb\backend\libusb0.py", line 431, in _check
raise USBError(errmsg, ret)
usb.core.USBError: [Errno None] b'libusb0-dll:err [set_configuration] could not set config 1: win error: The parameter is incorrect.\r\n'
Traceback (most recent call last):
File "<ipython-input-9-ead886eb3666>", line 1, in <module>
runfile('C:/Users/Raman Lab/Python code/Spectrometers/spec_testing.py', wdir='C:/Users/Raman Lab/Python code/Spectrometers')
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 866, in runfile
execfile(filename, namespace)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Raman Lab/Python code/Spectrometers/spec_testing.py", line 7, in <module>
spec = sb.Spectrometer(devs[0])
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\spectrometers.py", line 62, in __init__
self._open_device(device)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\spectrometers.py", line 90, in _open_device
lib.device_open(self._dev)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\wrapper.py", line 81, in device_open
return device.interface.open(device)
File "C:\Users\Raman Lab\AppData\Local\Continuum\Anaconda3\lib\site-packages\seabreeze\pyseabreeze\interfaces\common.py", line 23, in decorated_func
raise SeaBreezeError(msg)
SeaBreezeError: An error occured during opening.
Thanks for any help!
Edit:
For some reason I figured this out, thought it might help describe the issue. If I enter interactive python from cmd.exe and paste the code instead of typing it in manually, I get the same error. This makes me think (most likely I'm wrong) that it is somehow tied up with speed of imports. I tried adding a sleep for a few seconds in between lines 3 and 5, to simulate what happens when I am typing in the prompt, but that didn't help. I hope this was descriptive enough.
Your initial guess was not far off. I had exactly the same problem, however, on Windows 7 32bit with Python 2.7. The example code gchaks linked to was running fine.
When I tried executing the script several times in quick succession, the error message changed and pointed at a empty devices list. I added a delay of 2 seconds after
devs = sb.list_devices()
and that solved the problem. Your code was running just fine on a Linux machine btw.
Another hint: If your code should run once, but then you run into another error message, make sure you close the device connection or dis- and reconnect the spectrometer.
I've been migrating some Python 2.7.11 code to 3.5.1 after running into trouble with unicode. This was the last straw - since I started using the venv module there's no reason to be on 2.7 just because someone doesn't like 3!
The problem occurs while trying to run a one-way sync (ie. downloading changes only).
Here is the full error message, paths shortened:
Traceback (most recent call last):
File "%SCRIPT%.py", line 209, in <module>
updated_schedules = dbx_sync.One_Way_Sync(config['Dropbox Parameters']['Directory'], config['Dropbox Parameters']['Base Path'])
File "%COMMON_PATH%\modules\dropbox_sync_schedules.py", line 62, in One_Way_Sync
result = client.delta(cursor, base_path)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\client.py", line 569, in delta
return self.rest_client.POST(url, params, headers)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\rest.py", line 322, in POST
return cls.IMPL.POST(*n, **kw)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\rest.py", line 260, in POST
is_json_request=is_json_request)
File "%COMMON_PATH%\env-home\lib\site-packages\dropbox\rest.py", line 235, in request
raise ErrorResponse(r, r.read())
dropbox.rest.ErrorResponse: [400] 'Invalid "cursor" parameter: u"b\'\'"'
Searching for "invalid cursor parameter" wasn't any help, so I thought I'd come here.
u"b\'\'" is the key here. I just couldn't understand how that representation had ended up being sent as a string.
The issue was in reading the old cursor from a file (which for this example is empty): in Python 2 I had opened the file in mode rb - in Python 3 just r is all that's required, and everything works.
Hurrah!
I am calling an R script file from python using pyrserve. I have rserve running. At arbitrary points in the R script, pyrserve gives an error and quits:
Traceback (most recent call last):
File "scriptV2.py", line 272, in <module>
rConn.eval("source(file.PropensityFlow)")
File "/Users/dipayanmaiti/Py3.3venv/lib/python3.3/site-packages/pyRserve/rconn.py", line 47, in decoCheckIfClosed
return func(self, *args, **kw)
File "/Users/dipayanmaiti/Py3.3venv/lib/python3.3/site-packages/pyRserve/rconn.py", line 119, in eval
return rparse(src, atomicArray=atomicArray)
File "/Users/dipayanmaiti/Py3.3venv/lib/python3.3/site-packages/pyRserve/rparser.py", line 539, in rparse
return rparser.parse()
File "/Users/dipayanmaiti/Py3.3venv/lib/python3.3/site-packages/pyRserve/rparser.py", line 349, in parse
self.lexer.readHeader()
File "/Users/dipayanmaiti/Py3.3venv/lib/python3.3/site-packages/pyRserve/rparser.py", line 94, in readHeader
self.responseCode = struct.unpack(b'<i', self.read(3) + b'\x00')[0]
File "/Users/dipayanmaiti/Py3.3venv/lib/python3.3/site-packages/pyRserve/rparser.py", line 149, in read
raise EndOfDataError()
pyRserve.rparser.EndOfDataError
I have set rserv.conf to the following:
maxinbuf 20000000
maxsendbuf 0
Does anybody know why this happens? This looks like some buffer problem, because the R script runs by itself.
It is a late answer, but in such situations it is useful to run Rserve in debug mode so its output can be monitored in a separate shell.
R CMD Rserve.dbg
In some rare cases I've seen Rserve printing warnings to the console, and when this happened the command sent thru pyRserve didn't return any value from Rserve - which led to the 'EndOfDataError' above.
I recently started writing a simple client using the Blogger API to do some basic posting I implemented the client in Python and used the example code verbatim from the Blogger Developer's Guide to login, get the blog id, and make a new post. I ran the script and everything went fine until I got to this line:
return blogger_service.Post(entry, '/feeds/%s/posts/default' % blog_id)
I got the error message:
Traceback (most recent call last):
File "cs1121post.py", line 38, in <module>
cs1121post()
File "cs1121post.py", line 33, in cs1121post
return blogger_service.Post(entry, '/feeds/%s/posts/default' % blog_id)
File "/usr/local/lib/python2.7/dist-packages/gdata/service.py", line 1236, in Post
media_source=media_source, converter=converter)
File "/usr/local/lib/python2.7/dist-packages/gdata/service.py", line 1322, in PostOrPut
headers=extra_headers, url_params=url_params)
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 93, in optional_warn_function
return f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/atom/service.py", line 176, in request
content_length = CalculateDataLength(data)
File "/usr/local/lib/python2.7/dist-packages/atom/service.py", line 736, in CalculateDataLength
return len(str(data))
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 377, in __str__
return self.ToString()
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 374, in ToString
return ElementTree.tostring(self._ToElementTree(), encoding=string_encoding)
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 369, in _ToElementTree
self._AddMembersToElementTree(new_tree)
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 331, in _AddMembersToElementTree
member._BecomeChildElement(tree)
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 357, in _BecomeChildElement
self._AddMembersToElementTree(new_child)
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 342, in _AddMembersToElementTree
ExtensionContainer._AddMembersToElementTree(self, tree)
File "/usr/local/lib/python2.7/dist-packages/atom/__init__.py", line 224, in _AddMembersToElementTree
tree.text = self.text.decode(MEMBER_STRING_ENCODING)
AttributeError: 'list' object has no attribute 'decode'
By which I'm taking it that ElementTree is at fault here. I installed ElementTree via
sudo python setup.py install
in case it matters. Is there some known incompatibility between ElementTree and Python v2.7.1? Has this happened to anybody else and how did you get it working? If you need any additional information, please reply to the thread. All the source code that is relevant is basically just the example code from the Developers Guide mentioned above. I haven't modified that at all (not even the variable names). Any input is greatly appreciated.
The stacktrace is actually pretty clear about this: You're calling decode() on a list instead of a tree element. Try getting the first element from the list and calling decode() on that:
firsttext = self.text[0].decode(MEMBER_STRING_ENCODING)