I'm trying to create an async subprocess and pass a variable to stdin but trio.run_process and trio.open_process both give an attribute error. It doesn't specify what is causing the error.
Traceback (most recent call last):
File "./update.py", line 122, in pkgUpdates
trio.run(self.asetup, password)
File "/home/user/.local/lib/python3.7/site-packages/trio/_core/_run.py", line 1444, in run
raise runner.main_task_outcome.error
File "./update.py", line 65, in asetup
nursery.start_soon(self.upProc, password, 'update')
File "/home/user/.local/lib/python3.7/site-packages/trio/_core/_run.py", line 506, in __aexit__
raise combined_error_from_nursery
File "./update.py", line 75, in upProc
await trio.open_process(['sudo', '-S', 'apt-get', cmd, '-y'],
File "/home/user/.local/lib/python3.7/site-packages/trio/_deprecate.py", line 125, in __getattr__
raise AttributeError(name)
AttributeError: open_process
I've also tried using trio.Process with stdin=io.BytesIO(password) but this gives an io.UnsupportedOperation error. Passing only bytes also gives an error.
The function is:
async def upProc(self, password, cmd):
await trio.open_process(['sudo', '-S', 'apt-get', cmd, '-y'],
stdin=password.encode(), capture_stdout=True)
if (cmd == 'update'):
await self.upProc(password, 'upgrade')
return
The reason you're getting an AttributeError on open_process and run_process is that they were only added in Trio v0.12.0, and you're probably using v0.11.0. This isn't your fault – I only released v0.12.0 about 20 minutes ago :-). But I guess you were looking at the docs for the development version, and that threw you off.
Assuming you can upgrade to v0.12.0, I think you want:
await trio.run_process(['sudo', '-S', 'apt-get', cmd, '-y'], stdin=password_bytes)
This starts the process, waits for it to complete, and then returns some information about the process, all in one shot.
Related
I'm trying to write a bot using py-cord 2.3.2 under Python 3.9. When the /pin command is used, the bot should pin the user's message.
I have this code:
#bot.slash_command(name="pin", description="Make a pinned message.")
#default_permissions(manage_messages=True)
async def pin(ctp, arg):
message = arg
await message.pin()
print ('Used used the command ""/pin"')
I get an error which says:
Ignoring exception in command pin:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/discord/commands/core.py", line 124, in wrapped
ret = await coro(arg)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/discord/commands/core.py", line 980, in _invoke
await self.callback(ctx, **kwargs)
File "/Users/fk/Downloads/amiebot (no config)/tweescord.py", line 46, in pin
await message.pin()
AttributeError: 'str' object has no attribute 'pin'
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/discord/bot.py", line 1114, in invoke_application_command
await ctx.command.invoke(ctx)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/discord/commands/core.py", line 375, in invoke
await injected(ctx)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/discord/commands/core.py", line 132, in wrapped
raise ApplicationCommandInvokeError(exc) from exc
discord.errors.ApplicationCommandInvokeError: Application Command raised an exception:
AttributeError: 'str' object has no attribute 'pin'
Why does this happen, and how can I fix it?
You can't get a message directly as a slash command argument. But you can get a message-id as a slash command argument.
The only thing you need to change in the function declaration is changing arg to message_id. After that, you can verify if the message with the specified ID exists.
try:
message = await ctp.channel.fetch_message(int(message_id))
except discord.Forbidden:
print("bot is missing permissions to get message")
except discord.NotFound:
print("message was not found, it doesn't exist in this channel")
else:
print("message found")
await message.pin()
You can read these two little sections for more detailed documentation:
channel.fetch_message: await fetch_message(id, /)
How can you create slash commands? Slash Commands
I want to extract the user stats from unofficial tiktokapi (https://github.com/davidteather/TikTok-Api)
the code i used is below
from TikTokApi.tiktok import TikTokApi
with TikTokApi() as api: # .get_instance no longer exists
for trending_video in api.trending.videos():
user_stats = trending_video.author.info_full['stats']
if user_stats['followerCount'] >= 10000:
print(user_stats)
but i keep on getting this error -
RuntimeError: This event loop is already running
Task exception was never retrieved
future: <Task finished name='Task-5' coro=<Connection.run() done, defined at C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_connection.py:240> exception=NotImplementedError()>
Traceback (most recent call last):
File "C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_connection.py", line 247, in run
await self._transport.connect()
File "C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_transport.py", line 132, in connect
raise exc
File "C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_transport.py", line 120, in connect
self._proc = await asyncio.create_subprocess_exec(
File "C:\Users\siddh\anaconda3\lib\asyncio\subprocess.py", line 236, in create_subprocess_exec
transport, protocol = await loop.subprocess_exec(
File "C:\Users\siddh\anaconda3\lib\asyncio\base_events.py", line 1676, in subprocess_exec
transport = await self._make_subprocess_transport(
File "C:\Users\siddh\anaconda3\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
i tried using asyncio but the error keeps on coming any fixes to this
please note that if you're running the code in Jupyter Notebook - it won't work. As the documentation of this library says:
Note: Jupyter (ipynb) only works on linux
Paste your code into .py file and try running it as a Python script instead with:
python tiktok_script.py
Caught ExecuteBatchError exception on writing a message to the call of the following type
call: grpc.experimental.aio.StreamStreamCall.
ExecuteBatchError: Failed "execute_batch": (<grpc._cython.cygrpc.SendMessageOperation object at 0x7f3d1a7679b0>,)
...
await call.write(msg)
File "grpc/experimental/aio/_call.py", line 451, in write
await self._write(request)
File "grpc/experimental/aio/_call.py", line 429, in _write
await self._cython_call.send_serialized_message(serialized_request)
File "src/python/grpcio/grpc/_cython/_cygrpc/aio/call.pyx.pxi", line 368, in send_serialized_message
File "src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pyx.pxi", line 145, in _send_message
File "src/python/grpcio/grpc/_cython/_cygrpc/aio/callback_common.pyx.pxi", line 98, in execute_batch
It seems this is that place. Also, I see that sometimes such exception type is handled. Should it be handled in _send_message too?
I'm not aware too much in Cython and what happens under these calls and would be grateful for any info or suggestions.
python 3.8.6
grpcio 1.31.0
I'm trying to test some asyncio functionality on Windows. I'm using Python 3.7.
This complains bitterly about some NotImplementedError
import asyncio
import os
import time
import sys
#this works
async def say_after(delay, what):
await asyncio.sleep(delay)
print(what)
#this doesn't
async def test_async(num):
print('Task #{0} start'.format(num))
proc = await asyncio.create_subprocess_shell(
'C:/Python37/python test_repl.py',
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
stdin=asyncio.subprocess.PIPE )
stdout, stderr = await proc.communicate("sleep 10")
cmd = 'python'
print(f'[{cmd!r} exited with {proc.returncode}]')
if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')
async def test_loop():
task1 = asyncio.create_task(
test_async(1))
task2 = asyncio.create_task(
test_async(2))
print(f"started at {time.strftime('%X')}")
await task1
await task2
print(f"finished at {time.strftime('%X')}")
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
asyncio.run(test_loop())
I've tried several loop types (Protractor and Default). I've tried different functions through the shell and directly invoking the program (_shell and _exec in the subprocess lingo). Nothing seems to work. For a much hype I've read about asyncio, it can't possibly be a Linux only thing. I must be doing something wrong. Can you please point me in the right direction.
The specific error I get is the following:
--------------Async--------------------
started at 22:39:55
Task #1 start
Task #2 start
Traceback (most recent call last):
File "multirun.py", line 45, in <module>
asyncio.run(test_loop())
File "C:\Python37\lib\asyncio\runners.py", line 43, in run
return loop.run_until_complete(main)
File "C:\Python37\lib\asyncio\base_events.py", line 568, in run_until_complete
return future.result()
File "multirun.py", line 38, in test_loop
await task1
File "multirun.py", line 20, in test_async
stdin=asyncio.subprocess.PIPE)
File "C:\Python37\lib\asyncio\subprocess.py", line 202, in create_subprocess_shell
stderr=stderr, **kwds)
File "C:\Python37\lib\asyncio\base_events.py", line 1486, in subprocess_shell
protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
File "C:\Python37\lib\asyncio\base_events.py", line 444, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
Task exception was never retrieved
future: <Task finished coro=<test_async() done, defined at multirun.py:12> exception=NotImplementedError()>
Traceback (most recent call last):
File "multirun.py", line 20, in test_async
stdin=asyncio.subprocess.PIPE)
File "C:\Python37\lib\asyncio\subprocess.py", line 202, in create_subprocess_shell
stderr=stderr, **kwds)
File "C:\Python37\lib\asyncio\base_events.py", line 1486, in subprocess_shell
protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
File "C:\Python37\lib\asyncio\base_events.py", line 444, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
Also, I got the same error when trying to run the notepad.
As of Python 3.8, this issue should no longer exist, as the proactor event loop is now the default event loop on Windows. Original answer follows below.
The problem is that, despite appearances, you're not actually using the ProactorEventLoop. asyncio.run() creates a new event loop based on the current loop creation policy, which you've never changed. Creating a new loop for each run is normally a feature because it guarantees cleanup of the resources associated with the loop - but in this case it's incompatible with the example from the documentation. (Edit: the example has since been removed because ProactorEventLoop was made the default.)
A quick fix is to change asyncio.run(test_loop()) to loop.run_until_complete(test_loop()). A better fix is to set the event loop policy to one that creates the proactor loop.
I'm trying to make a worker run only one task at a time, then shutdown. I've got the shutdown part working correctly (some background here: celery trying shutdown worker by raising SystemExit in task_postrun signal but always hangs and the main process never exits), but when it shuts down, I'm getting an error:
[2013-02-13 12:19:05,689: CRITICAL/MainProcess] Couldn't ack 1, reason:AttributeError("'NoneType' object has no attribute 'method_writer'",)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/kombu/transport/base.py", line 104, in ack_log_error
self.ack()
File "/usr/local/lib/python2.7/site-packages/kombu/transport/base.py", line 99, in ack
self.channel.basic_ack(self.delivery_tag)
File "/usr/local/lib/python2.7/site-packages/amqplib/client_0_8/channel.py", line 1742, in basic_ack
self._send_method((60, 80), args)
File "/usr/local/lib/python2.7/site-packages/amqplib/client_0_8/abstract_channel.py", line 75, in _send_method
self.connection.method_writer.write_method(self.channel_id,
AttributeError: 'NoneType' object has no attribute 'method_writer'
Why is this happening? Not only does it not ack, but it also purges all of the other tasks that are left in the queue (big problem).
How do I fix this?
UPDATE
Below is the stack trace with everything updated (pip install -U kombu amqp amqplib celery):
[2013-02-13 11:58:05,357: CRITICAL/MainProcess] Internal error: AttributeError("'NoneType' object has no attribute 'method_writer'",)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/celery/worker/__init__.py", line 372, in process_task
req.execute_using_pool(self.pool)
File "/usr/local/lib/python2.7/dist-packages/celery/worker/job.py", line 219, in execute_using_pool
timeout=task.time_limit)
File "/usr/local/lib/python2.7/dist-packages/celery/concurrency/base.py", line 137, in apply_async
**options)
File "/usr/local/lib/python2.7/dist-packages/celery/concurrency/base.py", line 27, in apply_target
callback(target(*args, **kwargs))
File "/usr/local/lib/python2.7/dist-packages/celery/worker/job.py", line 333, in on_success
self.acknowledge()
File "/usr/local/lib/python2.7/dist-packages/celery/worker/job.py", line 439, in acknowledge
self.on_ack(logger, self.connection_errors)
File "/usr/local/lib/python2.7/dist-packages/kombu/transport/base.py", line 98, in ack_log_error
self.ack()
File "/usr/local/lib/python2.7/dist-packages/kombu/transport/base.py", line 93, in ack
self.channel.basic_ack(self.delivery_tag)
File "/usr/local/lib/python2.7/dist-packages/amqp/channel.py", line 1562, in basic_ack
self._send_method((60, 80), args)
File "/usr/local/lib/python2.7/dist-packages/amqp/abstract_channel.py", line 57, in _send_method
self.connection.method_writer.write_method(
AttributeError: 'NoneType' object has no attribute 'method_writer'
Exiting in task_postrun is not recommended as task_postrun is executed outside of the "task body" error handling.
Exactly what happens when a task calls sys.exit is not well defined,
and actually it depends on the pool being used.
With multiprocessing the child process will simply be replaced by a new one.
In other pools the worker will shutdown, but this is something that is likely to change
so that it's consistent with multiprocessing behavior.
Calling exit outside of the task body is regarded as an internal error (crash).
The "task body" is whatever executes at task.__call__()
I think maybe a better solution for this would be to use a custom execution
strategy:
from celery.worker import strategy
from functools import wraps
#staticmethod
def shutdown_after_strategy(task, app, consumer):
default_handler = strategy.default(task, app, consumer)
def _shutdown_to_exit_after(fun):
#wraps(fun)
def _inner(*args, **kwargs):
try:
return fun(*args, **kwargs)
finally:
raise SystemExit()
return _inner
return _decorate_to_exit_after(default_handler)
#celery.task(Strategy=shutdown_after_strategy)
def shutdown_after():
print('will shutdown after this')
This isn't exactly beautiful, but the execution strategy is there to optimize
task execution and not to be easily extendable (the worker "precompiles" the execution
path for each task type by caching Task.Strategy)
In Celery 3.1 you can extend the worker and consumer using "bootsteps", so likely
there will be a pretty solution then.