I have this very nasty python script which is, to say the least not very well managed and I'd like to improve it. The way I thought about doing that is breaking up the code into what they do and then importing them into the main function that runs everything. But some of my defs have a global (like the key word based global) in them, how do I scope those out in separate files exactly?
for example main,py wi have:
import function
message = {}
while (true):
function.function(message)
print(message)
and function,py has :
def function(some variables) :
global message
if (somevariable = something):
message = "xyz"
message in the second file is giving me an error
EDIT: So I see that I need to give a bit more context, there are infact 3 processes (functions) that are running and the global message is what I'm using to pass information between all these threads. So Ideally Id like to separate all the processes as different files and then keep adding them as a new thread. How do I go about this?
Related
I am relatively new to coding and to python. I am trying to make a telegram bot using telebot.
I have a flow in which I have to go. eg: after /start, I should get a text. So I created a running_work_list in which my workflow is arranged in order and after completion, I delete the index 0 elements. 'start' is the first element in the array.
So I tried to implement something like this so that I can comfortably code each step rather than looking at input and deciding.
But despite putting inside an if statement, '#bot.message_handler()' is running even if the condition fails.
if running_work_list[0]=='start':
print('inside if')
#bot.message_handler(commands=['start','new test'])
def start(message):
print('user sent start')
bot.send_message(message.chat.id,welcome_message)
running_work_list.pop(0)
print(f'work flow deciding list {running_work_list}')
#bot.message_handler(content_types=['text','photo','poll'])
def bot_intro(message):
print('here')
print(f'user sent {message.text}')
bot.send_message(message.chat.id,BOT_INTRO_MSG)
Below are the outputs I got. The user had sent /start first and then some other random text.
inside if
user sent start
work flow deciding list ['test_name', 'test_description', 'test_time', 'test_image', 'test_poll']
here
user sent fdf
Is it like the '#bot.message_handler()' will run even if we put it inside an if statement?
Executing #bot.message_handler decorator marks the function to be called when the message matches the filters that you've passed as arguments (content_types or commands). They are called by the telebot library directly and do not execute the if statement after they have been registered.
The proper way to do this is to invert the logic: define the message handler functions at the top level of the program and decide what you should do in the if within the function.
I am currently working on a system that sends webhooks to multiple Discord using dhooks.
This is the function used to call the function which sends the webhooks. The length of webhooks is 2 and it's a list of lists.
for hook in webhooks:
thread.start_new_thread(send_hook,(hook, embed, tag))
This is the send_hook function:
def send_hook(hook, embed, tag):
embed.set_footer(text="{} x Will".format(hook[0])) <-- This is the part where the error happens
webhook = Webhook(hook[1])
webhook.send(embed=embed)
The error I am getting is, when I set the footer in line 2 of send_hook, the variable inserted into the embed variable is sometimes sent to the wrong webhooks - almost as if its being over ridden.
As an example:
This is my list webhooks: [["Webhook 1 text", "discordwebhook1"], ["Webhook 2 text", "discordwebhook1"]].
What will happen is that in the channel of discordwebhook1 the footer will say "Webhook 1 text", but in the channel of discordwebhoo2 the footer will say "Webhook 1 text" as-well.
I have tried creating a new variable of the embed in the send_hook function - however this also didn't work (code below).
def send_hook(hook, embed, tag):
new_embed = embed
new_embed.set_footer(text="{} x Will".format(hook[0])) <-- This is the part where the error happens
webhook = Webhook(hook[1])
webhook.send(embed=new_embed)
I appreciate all help!
Thanks
How do you get 'embed'? Notice that 'embed' and 'tag' are always passed to every newly created thread, you need to do a deepcopy of each if necessary
from copy import deepcopy
for hook in webhooks:
thread.start_new_thread(send_hook,(hook, deepcopy(embed), tag))
You are experiencing a race condition. Two threads have access to the same variable, and they are both modifying the variable. The outcome of your program depends on which one reaches the code that changes the variable first.
There are two possible solutions, depending on how you want the problem to be resolved:
If you don't need all threads to share the same value, or if the object is small and cheap to copy, make a copy of the variable you are passing before you pass it, by passing deepcopy(embed) instead of embed - see the solution posted by #user1438644 for code.
If you want all threads to share the same value, or if it is expensive to make copies of the object, you will need to create a lock with my_lock = threading.Lock(), and when you get to the race condition part of your program (i.e., the part that modifies the shared variable), surround it with a context manager with my_lock:, which acquires the lock at the start and returns the lock when finished. For example:
import threading
my_lock = threading.Lock()
# your code here
for hook in webhooks:
threading.Thread(send_hook, args=(hook, embed, tag))
# more code here
def send_hook(hook, embed, tag):
# Ensure embed is only changed by one thread at a time
with my_lock:
print("Lock acquired")
embed.set_footer(text="hello world")
webhook = Webhook(hook[1])
webhook.send(embed=embed)
I'm currently working on a project where I need to send data via Serial persistently but need to occasionally change that data based in new inputs. My issue is that my current loop only functions exactly when a new input is offered by raw_input(). Nothing runs again until another raw_input() is received.
My current (very slimmed down) loop looks like this:
while True:
foo = raw_input()
print(foo)
I would like for the latest values to be printed (or passed to another function) constantly regardless of how often changes occur.
Any help is appreciated.
The select (or in Python 3.4+, selectors) module can allow you to solve this without threading, while still performing periodic updates.
Basically, you just write the normal loop but use select to determine if new input is available, and if so, grab it:
import select
while True:
# Polls for availability of data on stdin without blocking
if select.select((sys.stdin,), (), (), 0)[0]:
foo = raw_input()
print(foo)
As written, this would print far more than you probably want; you could either time.sleep after each print, or change the timeout argument to select.select to something other than 0; if you make it 1 for instance, then you'll update immediately when new data is available, otherwise, you'll wait a second before giving up and printing the old data again.
How will you type in your data at the same time while data is being printed?
However, you can use multithreading if you make sure your source of data doesn't interfere with your output of data.
import thread
def give_output():
while True:
pass # output stuff here
def get_input():
while True:
pass # get input here
thread.start_new_thread(give_output, ())
thread.start_new_thread(get_input, ())
Your source of data could be another program. You could connect them using a file or a socket.
I'm trying to connect to a TeamSpeak server using the QueryServer to make a bot. I've taken advice from this thread, however I still need help.
This is The TeamSpeak API that I'm using.
Before the edits, this was the summary of what actually happened in my script (1 connection):
It connects.
It checks for channel ID (and it's own client ID)
It joins the channel and starts reading everything
If someone says an specific command, it executes the command and then it disconnects.
How can I make it so it doesn't disconnect? How can I make the script stay in a "waiting" state so it can keep reading after the command is executed?
I am using Python 3.4.1.
I tried learning Threading but either I'm dumb or it doesn't work the way I thought it would. There's another "bug", once waiting for events, if I don't trigger anything with a command, it disconnects after 60 seconds.
#Librerias
import ts3
import threading
import datetime
from random import choice, sample
# Data needed #
USER = "thisisafakename"
PASS = "something"
HOST = "111.111.111.111"
PORT = 10011
SID = 1
class BotPrincipal:
def __init__(self, manejador=False):
self.ts3conn = ts3.query.TS3Connection(HOST, PORT)
self.ts3conn.login(client_login_name=USER, client_login_password=PASS)
self.ts3conn.use(sid=SID)
channelToJoin = Bot.GettingChannelID("TestingBot")
try: #Login with a client that is ok
self.ts3conn.clientupdate(client_nickname="The Reader Bot")
self.MyData = self.GettingMyData()
self.MoveUserToChannel(ChannelToJoin, Bot.MyData["client_id"])
self.suscribirEvento("textchannel", ChannelToJoin)
self.ts3conn.on_event = self.manejadorDeEventos
self.ts3conn.recv_in_thread()
except ts3.query.TS3QueryError: #Name already exists, 2nd client connect with this info
self.ts3conn.clientupdate(client_nickname="The Writer Bot")
self.MyData = self.GettingMyData()
self.MoveUserToChannel(ChannelToJoin, Bot.MyData["client_id"])
def __del__(self):
self.ts3conn.close()
def GettingMyData(self):
respuesta = self.ts3conn.whoami()
return respuesta.parsed[0]
def GettingChannelID(self, nombre):
respuesta = self.ts3conn.channelfind(pattern=ts3.escape.TS3Escape.unescape(nombre))
return respuesta.parsed[0]["cid"]
def MoveUserToChannel(self, idCanal, idUsuario, passCanal=None):
self.ts3conn.clientmove(cid=idCanal, clid=idUsuario, cpw=passCanal)
def suscribirEvento(self, tipoEvento, idCanal):
self.ts3conn.servernotifyregister(event=tipoEvento, id_=idCanal)
def SendTextToChannel(self, idCanal, mensajito="Error"):
self.ts3conn.sendtextmessage(targetmode=2, target=idCanal, msg=mensajito) #This works
print("test") #PROBLEM HERE This doesn't work. Why? the line above did work
def manejadorDeEventos(sender, event):
message = event.parsed[0]['msg']
if "test" in message: #This works
Bot.SendTextToChannel(ChannelToJoin, "This is a test") #This works
if __name__ == "__main__":
Bot = BotPrincipal()
threadprincipal = threading.Thread(target=Bot.__init__)
threadprincipal.start()
Prior to using 2 bots, I tested to launch the SendTextToChannel when it connects and it works perfectly, allowing me to do anything that I want after it sends the text to the channel. The bug that made entire python code stop only happens if it's triggered by the manejadorDeEventos
Edit 1 - Experimenting with threading.
I messed it up big time with threading, getting to the result where 2 clients connect at same time. Somehow i think 1 of them is reading the events and the other one is answering. The script doesn't close itself anymore and that's a win, but having a clone connection doesn't looks good.
Edit 2 - Updated code and actual state of the problem.
I managed to make the double connection works more or less "fine", but it disconnects if nothing happens in the room for 60 seconds. Tried using Threading.timer but I'm unable to make it works. The entire question code has been updated for it.
I would like an answer that helps me to do both reading from the channel and answering to it without the need of connect a second bot for it (like it's actually doing...) And I would give extra points if the answer also helps me to understand an easy way to make a query to the server each 50 seconds so it doesn't disconnects.
From looking at the source, recv_in_thread doesn't create a thread that loops around receiving messages until quit time, it creates a thread that receives a single message and then exits:
def recv_in_thread(self):
"""
Calls :meth:`recv` in a thread. This is useful,
if you used ``servernotifyregister`` and you expect to receive events.
"""
thread = threading.Thread(target=self.recv, args=(True,))
thread.start()
return None
That implies that you have to repeatedly call recv_in_thread, not just call it once.
I'm not sure exactly where to do so from reading the docs, but presumably it's at the end of whatever callback gets triggered by a received event; I think that's your manejadorDeEventos method? (Or maybe it's something related to the servernotifyregister method? I'm not sure what servernotifyregister is for and what on_event is for…)
That manejadorDeEventos brings up two side points:
You've declared manejadorDeEventos wrong. Every method has to take self as its first parameter. When you pass a bound method, like self.manejadorDeEventos, that bound self object is going to be passed as the first argument, before any arguments that the caller passes. (There are exceptions to this for classmethods and staticmethods, but those don't apply here.) Also, within that method, you should almost certainly be accessing self, not a global variable Bot that happens to be the same object as self.
If manejadorDeEventos is actually the callback for recv_in_thread, you've got a race condition here: if the first message comes in before your main threads finishes the on_event assignment, the recv_on_thread won't be able to call your event handler. (This is exactly the kind of bug that often shows up one time in a million, making it a huge pain to debug when you discover it months after deploying or publishing your code.) So, reverse those two lines.
One last thing: a brief glimpse at this library's code is a bit worrisome. It doesn't look like it's written by someone who really knows what they're doing. The method I copied above only has 3 lines of code, but it includes a useless return None and a leaked Thread that can never be joined, not to mention that the whole design of making you call this method (and spawn a new thread) after each event received is weird, and even more so given that it's not really explained. If this is the standard client library for a service you have to use, then you really don't have much choice in the matter, but if it's not, I'd consider looking for a different library.
At the time I'm just doing a python myprogra.py & and let this program do its thing:
import urllib2
import threading
import json
url = 'https://something.com'
a = []
def refresh():
# refresh in 5 minutes
threading.Timer(300.0, refresh).start()
# open url
try:
data = urllib2.urlopen(url).read(1000)
except:
return 0
# decode json
q = data.decode('utf-8')
q = json.loads(q)
# store in a
a.append(q['ticker'])
if len(a) > 288:
a.pop()
truc = json.dumps(a)
f = open('ticker.json', 'w')
f.write(truc)
f.close()
refresh()
I have two questions:
how comes it work since I didn't write global a at the start of the function
should I use a cron for this kind of thing instead of what I'm doing? (I'm using a debian server)
There is no issue with accessing the variable a the way you do, because you never assign to it within the refresh function. It is accessed the very same way as the url variable or even the json import is accessed. If you were to assign to a (rather than calling a method such as append on it), then you would create a local variable shadowing the global a. The global keyword avoids the creation of a local variable for assignments.
It is up to you whether you use a program that sleeps or cron, but here are some things to keep in mind:
Your program keeps state across requests in the variable a. If you were to use cron and invoke your program multiple times, you would need to store this state somewhere else.
If your program crashes (e.g. invalid data is returned and json decoding fails with an exception), cron would start it again, so it would eventually recover. This may or may not be desired.
When run via cron, you lower the memory footprint of the system at the expense of more computation (Python interpreter being initialized every five minutes).