I am attempting to try using Deferrable opertators however, the example seem to be lacking info on where BaseTrigger and TriggerEvent are imported from. I cant seem to find info on the docs too
https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html?highlight=triggerevent
def __init__(self, moment):
super().__init__()
self.moment = moment
def serialize(self):
return ("airflow.triggers.temporal.DateTimeTrigger", {"moment": self.moment})
async def run(self):
while self.moment > timezone.utcnow():
await asyncio.sleep(1)
yield TriggerEvent(self.moment)
Is BaseTrigger the same as airflow.models.trigger.Trigger?
BaseTrigger is located in airflow.triggers.base.BaseTrigger and TriggerEvent is located in airflow.triggers.base.TriggerEvent.
An IDE such as PyCharm is very helpful in finding such things: press double shift and search BaseTrigger will get you to module directly.
Related
How do I go about retrieving a list of commands (Commands) that require the user to have the 'administrator' permission (to use them)? I am making a 'help' command; I would like to be able to retrieve all commands that require a certain permission in a list.
For example:
class Moderate(commands.Cog):
#commands.command()
#has_permissions(administrator=True)
async def kick(ctx, member):
#kick member
cog = Moderate
print(cog.commands.checks) -> {"kick" : administrator=True} #How to get this or the commands my bot has with this permission requirement?
My solution to this problem is to read the file and get the commands' names from it by checking if there's a "has_permissions(" string before it:
import os
file_path = os.path.abspath(__file__) #Get the file path of this file
self = open(file_path, "r", encoding="utf-8") #Specify the encoding as it's a Python file
code = self.readlines()
commands = [] #This is the list of all the commands your bot has
for line_number, line in enumerate(code):
if "has_permissions(administrator" in line: #Check if the decorator ("#has_permissions(administrator)") is in the line
command = code[line_number + 1] #Get the line after the decorator's
command_name = command.replace("async def ", "").strip()[:-1] #Removing "async def ", colon and extra spaces
if "command = code[line_number + " not in command_name: #Adding all commands' names except this code we're using
commands.append(command_name)
print(commands) #See the 100 commands your bot has
This will not work if you check if the user has the 'administrator' permission in a way other than the #has_permissions(administrator=True) decorator. For example, if you have a 'ban' command like this, it won't be added to the list:
bot.command()
async def ban(message, member:discord.Member):
if message.author.guild_permissions.administrator: #The method won't work if you check using this
#Ban the member
This also won't work if you have extra blank lines between the decorator and the command's definition etc.
However, you can modify this to meet your requirements.
Here's a less hacky way of doing it that I personally prefer:
We can use a decorator to set a special attribute in the Command object, which we can then look at later. (Note: it is important that the #special decorator is placed before the #client.command. It needs to work off the Command object, not the actual callable function itself.)
def special(command):
command.__special_flag__ = None # the value is irrelevant, you just need to set it
return command
#special
#client.command()
async def special_command(ctx):
await ctx.send('hi')
#special
#client.command()
async def list_special_commands(ctx):
for command in client.commands:
if hasattr(command, '__special_flag__'): # this checks if "__special_flag__" is set
await ctx.send(f'special command! {command}')
await ctx.send('ok')
You can then iterate through every command in the client and check if that flag is set. You can then add this extra decorator to any command you want to check permissions for. Then, in your help command, only display commands with this flag set.
Output looks like this:
As #The Amateur Coder mentioned, this might take a bit of time to add them all, so you can make this combined decorator that does all of it:
def combine_stuff(function):
function = commands.has_permissions(...)(function)
command = client.command(...)(function)
command = special(command)
Explanation
There exists a very simple way to see which commands require administrator permissions. All command checks' predicates are in closures, and for has_permissions, one of the nonlocal variables of these functions will be the specified permissions.
This code must be run after all the commands are defined. For example, it can be run in on_ready, or before bot.run. You do not need to modify any other functions to run this code.
Code
#bot.event
async def on_ready():
command_list = []
for command in bot.walk_commands():
for check in command.checks:
closure = check.__closure__
if closure is None:
continue
# loop through variables used by the check
for cell in closure:
# if administrator=True was passed
if cell.cell_contents.get("administrator"):
command_list.append(command)
break
print(command_list)
Reference
How do I access/modify variables from a function's closure?
Definition of has_permissions
I'm trying to change the help command to use a pagination version of help.
I understand that the following line of code removes the help command entirely:
bot.remove_command('help')
The docs/dicord.py server offer the following example as a way to change the default help command:
class MyHelpCommand(commands.MinimalHelpCommand):
def get_command_signature(self, command):
return '{0.clean_prefix}{1.qualified_name} {1.signature}'.format(self, command)
class MyCog(commands.Cog):
def __init__(self, bot):
self._original_help_command = bot.help_command
bot.help_command = MyHelpCommand()
bot.help_command.cog = self
def cog_unload(self):
self.bot.help_command = self._original_help_command
I'm still a newbie in python, and I've only been learning rewrite for about 3 days - I'm struggling to find any working examples or an explanation that doesn't lead me back to the above code. I can't work out how to implement this into my own code - so my question is, could anyone provide further explaination into how this would be implemented using cogs?
You can use help_command=None. It delete default help command and you can create your help command. Example:
bot = commands.Bot(command_prefix='!', help_command=None)
#bot.command()
async def help(context):
await context.send("Custom help command")
If you don't set help_command=None and try to create your help command, you get this error: discord.errors.ClientException: Command help is already registered.
You don't really need to remove the command... It isn't good, using the (prefix)help commandname <- It wont appear then... If you want it embed you can do.
class NewHelpName(commands.MinimalHelpCommand):
async def send_pages(self):
destination = self.get_destination()
for page in self.paginator.pages:
emby = discord.Embed(description=page)
await destination.send(embed=emby)
client.help_command = NewHelpName()
The built in help command is of great use
i have this code in a python file:
from dec import my_decorator
import asyncio
#my_decorator
async def simple_method(bar): # , x, plc_name, var_name):
print("Henlo from simple_method\npartent:{}".format(parent))
return
#my_decorator
async def other_simple_meth(bar, value):
print("Henlo from other_simple_meth:\t Val:{}".format(value))
return
async def main():
print("Start Module-Export")
open('module_functions.py', 'a').close()
# Write all decorated functions to modue_functions.py
print("Functions in module_functions.py exported")
while True:
asyncio.sleep(2)
print("z...z...Z...")
My goal is to write all decorated functions (inc. the import dependencies) into a second module file (here "module_functions.py"). My 'module_functions.py' file should look like this:
from dec import my_decorator
import asyncio
#my_decorator
async def simple_method(bar): # , x, plc_name, var_name):
print("Henlo from simple_method\npartent:{}".format(parent))
return
#my_decorator
async def other_simple_meth(bar, value):
print("Henlo from other_simple_meth:\t Val:{}".format(value))
return
I know how to get references and names of a function, but not how to "copy/paste" the functioncode (incl. decorator and all dependencies) into a seperated file. Is this even possible?
EDIT: I know that pickle and dill exist, but this may not fullfill the goal. The problem is, that someone else may not know the order of the dumped file and loading them back may/will cause problem. As well it seems to be not possible to edit such loaded functions again.
I found a (not ideal, but ok) solution for my problems.
I) Find and write functions, coroutines etc. into a file (works):
Like #MisterMiyagi suspected, is the inspect module a good way to go. For the common stuff, it is possible with inspect.getsource() to get the code and write them into a file:
# List of wanted stuff
func_list = [simple_method, meth_with_input, meth_with_input_and_output, func_myself]
with open('module_functions.py', 'a') as module_file:
for func in func_list:
try:
module_file.write(inspect.getsource(func))
module_file.write("\n")
except:
print("Error :( ")
II) But what about decorated stuff(seems to work)?
I) will not work for decorated stuff, it is just ignored without throwing an exception. What seems to be used is from functools import wraps.
In many examples the #wraps decorator is added into the decorator class. This was not possible for me, but there is a good workaround:
#wraps(lambda: simple_method) #<---add wraps-decorator here
#my_decorator
async def simple_method(parent): # , x, plc_name, var_name):
print("Henlo from simple_method\npartent:{}".format(parent))
return
Wraps can be placed above the original decorated method/class/function and it seems to behave like I want. Now we can add simple_methodinto the func_listof I).
III) What about the imports?
Well it seems to be quite tricky/impossible to actually read the dependencies of a function. My workaround is to drop all wanted imports into a class (sigh). This class can be throw into the func_listof I) and is written into the file.
EDIT:
There is a cleaner way, which may works, after some modification, with I) and II) as well. The magic module is ast.
I have overwritten following:
class ImportVisitor(ast.NodeVisitor):
def __init__(self, target):
super().__init__()
self.file_target = target
"pick these special nodes via overwriting: visit_classname." \
"classnames are listed in https://docs.python.org/3.6/library/ast.html#abstract-grammar"
def visit_Import(self, node):
"Overwrite func!"
"Write all statements just with import like - import ast into file_target"
str = 'import '+', '.join(alias.name for alias in node.names)
self.file_target.write(str+"\n")
def visit_ImportFrom(self, node):
"Overwrite func!"
"Write all statements with from ... import (like - from os.path import basename) into file_tagrget"
str = 'from '+ node.module+ ' import '+', '.join(alias.name for alias in node.names)
self.file_target.write(str+"\n")
Now I can parse my own script name and fill the module_file with the imports and from...imports it will find while visiting all nodes in this tree:
with open('module_functions.py', 'a') as module_file:
with open(basename(__file__), "rb") as f:
tree = ast.parse(f.read(), basename(__file__))
visitor = ImportVisitor(module_file)
visitor.visit(tree)
module_file.write("\n\n")
Say I have a variable that helps with a network connection (eg: API access token) that I want to use in a module that I'm importing (for use in expanding on existing functions), is there a way to pass that variable to the imported script without adding a new class or method to initialize it?
eg:
Script one (performer.py):
import telepot
import botActions # Possibly adding an argument here?
botActions.sendMessage("locationID", "message")
accessToken = "SAMPLE"
Script two (botActions.py):
import botApiWrapper
def sendMessage(locationID, text):
bot.sendMessage(locationID, text)
print("Message sent to %g: %t".format(g=locationID, t=text))
bot = botApiWrapper.Bot(sys.argv[0])
So is there a way to pass the variable to the second script from the first one, or would I have to define an initializing function that I call after importing the file?
The canonical way to do something like this is to define an initialization function from the other file. Instead of bot = botApiWrapper.Bot(sys.argv[0]), try something like
def newbot(arg):
bot = botApiWrapper.Bot(arg)
return bot
and then call that function from your other module.
If you need state to operate, define a class.
class Bot(object):
def __init__(self, arg):
self.arg = arg
def sendMessage(self, locationId, text):
print "Using arg %s to send message to location %s with text %s" % \
(self.arg, locationId, text)
...thereafter letting you initialize as many bots as you like:
import botApiWrapper
botA = botApiWrapper.Bot("A")
botA.sendMessage("foo", "bar")
botB = botApiWrapper.Bot("B")
botB.sendMessage("baz", "qux")
When I use a standard Queue to send samples to a process, everything works fine. However, since my needs are simple, I tried to use a SimpleQueue and for some reason the 'empty' method doesn't work. Here's the details:
Error comes from the consumer process (when sample_queue is Queue, everything works, when sample_queue is SimpleQueue, things break):
def frame_update(i):
while not self.sample_queue.empty():
sample = self.sample_queue.get()
for line in lines:
While executing sample_queue.empty() -- SimpleQueue.empty(), from Python 3.6 on windows (queues.py) we get:
def empty(self):
return not self._poll()
Where self._poll() has been set in init by:
def __init__(self, *, ctx):
self._reader, self._writer = connection.Pipe(duplex=False)
self._rlock = ctx.Lock()
self._poll = self._reader.poll
if sys.platform == 'win32':
self._wlock = None
else:
self._wlock = ctx.Lock()
So I follow the self._reader which is set from connection.Pipe (connection.py):
...
c1 = PipeConnection(h1, writable=duplex)
c2 = PipeConnection(h2, readable=duplex)
Ok, great. The _reader is going to be a PipeConnection and pipe connection has this method:
def _poll(self, timeout):
if (self._got_empty_message or
_winapi.PeekNamedPipe(self._handle)[0] != 0):
return True
return bool(wait([self], timeout))
Alright -- So a couple of questions:
1) Shouldn't the init of SimpleQueue be assigning self.poll to self._reader._poll instead of self._reader.poll? Or am I missing something in the inheritance hierarchy?
2) The PipeConnection _poll routine takes a timeout parameter, so #1 shouldn't work...
*) -- Is there some other binding of PipeConnection _poll that I'm missing?
Am I missing something? I am using Python3.6, Windows, debugging in PyCharm and I follow all the paths and they're in the standard multiprocessing paths. I'd appreciate any help or advice. Thanks!
EDIT: After further review, I can see that PipeConnection is a subclass of _ConnectionBase which does indeed have a 'poll' method and it is bound with a default timeout parameter.
So the question is: When SimpleQueue is initializing and sets
self._poll = self._reader.poll
Why doesn't it go up the class hierarchy to grab that from _ConnectionBase?
After looking at why the Queue type works and why the SimpleQueue doesn't, I found that Queue sets the _poll method 'after_fork' as well as before. SimpleQueue doesn't. By changing the setstate method to add self._poll = self._reader.poll as follows (queues.py, line 338), SimpleQueue works
def __setstate__(self, state):
(self._reader, self._writer, self._rlock, self._wlock) = state
self._poll = self._reader.poll
Seems like a bug to me unless I'm really misunderstanding something. I'll submit a bug report and reference this post. Hope this helps someone!
http://bugs.python.org/issue30301