I'm currently designing a chat bot for sake of brevity here's a simplified version of what I have.
A main module
import bot
<import Command>
bot.connect()
def msg_handler(msg):
bot.handle_message(Command.handler)
bot.start()
A command module
def handler(msg):
if msg.startswith("!"):
prefix,rest = get_command_prefix(msg)
if prefix in <handlers>:
<handlers>[prefix](rest)
Multiple specific commands for example
<register>
def echo(msg):
send(msg)
So my question is how to connect these together
specifically how do i register the various commands to a central module which has shared code
Some options I have considered
giving each module an init function which registers it
def init(registry):
registry.register(name,handler)
some disadvantages are that most init functions will be almost the same and it's very explicit
decorator and import
import Registry
#Registry.register
def echo(...):
...
Here the problem would be if the registry moves or changes slightly it could mean changing each command
Some final notes
one does not know how many modules there are at runtime
i'm trying to minimize code duplication
it would be nice to be able to load additional modules at runtime
a similar problem arises between command.py and main.py so a hierarchical or multi level solution would be nice
is there a general/know approach to this sort of problem
EDIT
A small after thought is that the specific "command.py" might differ for example a normal version and a testing version which might make my second proposal not as applicable
Related
I have a program that, for data security reasons, should never persist anything to local storage if deployed in the cloud. Instead, any input / output needs to be written to the connected (encrypted) storage instead.
To allow deployment locally as well as to multiple clouds, I am using the very useful fsspec. However, other developers are working on the project as well, and I need a way to make sure that they aren't accidentally using local File I/O methods - which may pass unit tests, but fail when deployed to the cloud.
For this, my idea is to basically mock/replace any I/O methods in pytest with ones that don't work and make the test fail. However, this is probably not straightforward to implement. I am wondering whether anyone else has had this problem as well, and maybe best practices / a library exists for this already?
During my research, I found pyfakefs, which looks like it is very close what I am trying to do - except I don't want to simulate another file system, I want there to be no local file system at all.
Any input appreciated.
You can not use any pytest addons to make it secure. There will always be ways to overcome it. Even if you patch everything in the standard python library, the code always can use third-party C libraries which can't be patched from the Python side.
Even if you, by some way, restrict every way the python process can write the file, it will still be able to call the OS or other process to write something.
The only ways are to run only the trusted code or to use some sandbox to run the process.
In Unix-like operating systems, the workable solution may be to create a chroot and run the program inside it.
If you're ok with just preventing opening files using open function, you can patch this function in builtins module.
_original_open = builtins.open
class FileSystemUsageError(Exception):
pass
def patched_open(*args, **kwargs):
raise FileSystemUsageError()
#pytest.fixture
def disable_fs():
builtins.open = patched_open
yield
builtins.open = _original_open
I've done this example of code on the basis of the pytest plugin which is written by the company in which I work now to prevent using network in pytests. You can see a full example here: https://github.com/best-doctor/pytest_network/blob/4e98d816fb93bcbdac4593710ff9b2d38d16134d/pytest_network.py
Description
I am writing a FastAPI app for making multiplayer chess games. But unfortunately I am stuck on a circular import problem (or so I am guessing) and I'm not able to move forward. Basically I have a socket manager variable in the api/main.py which needs to be imported in api/endpoints/game.py, but in api/main.py I also include the API routers. So whenever I include the API routers it would call that module, which would then call api/main.py which would again call game.py to include the router, therefore creating a circular imports problem. This is the complete python traceback: here.
Code
The code can be seen on GitHub, here it the repository link https://github.com/p0lygun/astounding-arapaimas/tree/feature/api/games/api.
My Tries
I tried all the "hacks" of injecting the module using sys and os but nope they too caused the same problem. I tried moving the socket_manager into a completely different file too but that would too cause the same problem.
If any other information is required let me know, posting a question for the first time so not so sure what all is needed. Thanks!
Edits
Edit 1
codemation#0324 on discord suggested a better way to access the socket manager like this:
Interestingly I get the same error afterwards too.
Edit 2
The problem is happening along the lines where I add JWT Auth depend into the API Router and then use Depend(get_db) in the endpoint definition function. An alternative would be not Depend the get_db() and just call it normally and pass it on to the CRUD functions for the timing.
To be clear, when I remove , dependencies=[Depends(auth.JWTBearer())] it works perfectly.
I am using vs code for python scripts. These scripts run on the server only.
The server however, passes certain variables to the script while executing the script.
e.g mbo is always passed in it. mbo is special keyword which corresponds to a some class.
Sample mbo.py
class Mbo:
def getString(column: str)-> str:
return 'ABC'
def setString(columnName: str)-> None:
# do something with columnName.
Goal:
In my project in any python file whenever the user types mbo followed by a . vs code should show autoscomplete for .getString() and .setString() without importing this class as it is passed to the script by server.
I can try to write an extension for vs code to add this feature.
Here, I am stuck that what kind of extension is needed here. A LSP? I don't want to loose the feature of the existing python LSP for python.
Can any one proficient with vs code extension API guide me in right direction.
Note: I cannot import this Mbo class just for autocompletion in vscode because I import it. Then I run the same script on server. The server throws errors about the file.
You could try making the imports conditional:
try:
mbo
except NameError:
import mbo
That might be enough to make one of the two IntelliSense engines in the Python extension work.
Otherwise you are looking at your own extension. LSP is obviously the best option but there's also the classic style of registering classes. The VS Code docs have the details. But you are still probably going to clash with the Python extension as both it and your extension will be registered to work with Python files.
This problem involves the collision of several problems, all of which I understand only somewhat well, but I include them together because they could all be the entry point for a solution. Here is the best description I can give.
I have an app, in python. (I imagine I could theoretically solve all of these problems by learning Cocoa and ObjectiveC, but that seems like QUITE a lift, for this problem -- AND, as noted below, this problem may not actually be related to python, really, at all. I just don't know.) A CORE feature of this app is to trigger a minigame, with a hotkey -- meaning, the hotkey itself is fundamental to the desired functionality. And furthermore, I would really like to package this app, to let other people use it. (Locally, it works great! Hey!)
The problem starts with the fact that adding the hotkey -- which I am doing with
import keyboard
keyboard.add_hotkey('windows+shift+y', trigger_minigame)
-- requires root access. Due to DIRE WARNINGS in another SO post Forcing a GUI application to run as root (which, honestly, I only vaguely understand), I would like to grant that access to ONLY this part of the program. I IMAGINE, such an approach would look something like this:
# needs_root.py
import keyboard
from shouldnt_have_root import trigger_minigame
keyboard.add_hotkey('windows+shift+y', trigger_minigame)
# shouldnt_have_root.py
def minigame():
buncha pygame, GUI stuff (which is dangerous???)
def trigger_minigame():
adds event to minigame's event queue
# bash script
sudo python needs_root.py
HOWEVER -- there are several major challenges!
The biggest is that I don't even know if THAT is safe, since I don't know how security and permissions (especially with imports) works at all! And more generally, how dangerous are the imports? It appears that I may in fact have to import substantially more, to make it clear what event queue the trigger is adding an event TO -- and I don't know how to have that communication happen, while still isolating the GUI parts (or generally dangerous ones) from unnecessary and hazardous access.
There's another layer too though; packaging it through pyinstaller means that I can't target the scripts directly, because they'll have been turned into binaries, but according to THIS answer Packaging multiple scripts in PyInstaller it appears I can just target the binaries instead, i.e. have the first binary call
osascript -e 'do shell script "python needs_root_binary" with admin.'
to get the user to bless only the necessary part, but I don't know if that will put OTHER obstacles, or vulnerabilities (or inter-file communication difficulties), in the way.
LAST, I could try STARTING as root, and then switching away from it, as soon as the hotkey is set (and before anything else happens) -- but would that be safe? I'm still worried about the fact that it involves running sudo on the whole app.
In any event --
is this as big a mess as it feels?
How do I give root access to only a piece of a packaged .app, that I've written in python?
I'd advice You to:
enable the root access,
write the script,
disable the root access
as it's closer described in here.
The Pyinstaller is another chapter. When I was making software requiring usage of hotkeys, I was forced to use another than keyboard, because it wasn't working properly on PC without Python, therefore I made a hotkey with tkinter built-in function canvas.bind() (more info here).
Hopefully I helped.
You can not run a specific Python function as root, only the Python process executing your script can be run with elevated permissions.
So my answer is: your problem as described is unsolvable.
My script depends on loading lots of variables in a minute and uses them globally in many functions. Every time I call that script in iPython, it loads them again, taking time.
I tried to take these calls to load and populate functions out of that script, but then these global variables are not available to the functions in the script.
It gives NameError: name 'clf' is not defined error message.
Is there a best way to refactor this code to keep these globals in memory and make the script use them? The script loads many variables like these, and uses them in other functions as globals.
vectorizer_title, vectorizer_desc, clf,
df_instance, vocab, all_tokens, df_dist_all,
df_soc2class_proba, dict_p2s,
dict_f2m, token_pattern, cleanup_pattern,
excluded_words = load_data_and_model(lang)
dict_token2idx_all, dict_token2idx_instance,
dist_array, token_dist_to_instance_min,
dict_bigram_by_instance, denominate,
similar_threshold = populate_data(1)
I had asked this question after trying
from depended_library import *
it had not worked in iPython.
But used with python and used in a Flask Web API it works.
Importing library using the "from" statement executes also the codes out of functions in the depended_library in addition to defining functions.
(If someone explains the problem with iPython and suggest a solution, I shall select it as answer.)