Develop Reference

Users banned by my Telegram bot do not stay banned?

TL;DR: Methods listed in "Attempts" are not working as they should. Whenever I run those Python methods, they end up only kicking the user (or maybe banning them for at maximum 2-3 minutes), regardless if I include until_date or not. All group permissions are enabled and in BotFather both group privacy mode and "allow groups?" are disabled.
I am new to Stack Overflow, so I'm not sure if the formatting of my question is fine and if a lengthy description is preferable.
Context: I have a (private) Telegram group with a little bit over 700 users. There are two other admins and five bots (Rose, Group Help, Combot and two that I coded). In the past couple of weeks I have been facing issues with banning users through my own bot, while Rose and Group Help bots seem to be able to.
Issue: My bot limits the amount of entries (per user) to the group to avoid people lurking, or just raiding the group when I share an invitation link. Group help bot and Rose kick people out if they don't solve the captcha and present themselves in the first 5 minutes upon entering. I keep Combot to avoid letting known spammers in my group. The problem I'm facing is that whenever the ban is issued by my own bot, the user will simply be kicked out. I tried various methods to ban a user in my Python script, but nothing worked. I tried switching from python-telegram-bot to AIOgram, but it still doesn't work. I am not sure if it's caused by a conflict between bots (which wouldn't make much sense), wrong methods, or if it's a group/account-related problem. I tried my bot in a "test group" and it worked fine.
Screenshot of an example of the anomaly from the group logs channel: Since I can't embed pictures yet, here is the imgur link.
Part of the current (pseudo)code using AIOgram:
#dp.message_handler(content_types=["new_chat_members"])
async def newUser(message: types.Message):
for user in message.new_chat_members:
con = psycopg2.connect(...)
#[...SQL query fetching number of entries of a user...]
if "DB is empty":
#..Initialize..
else:
cur.execute(sql_script) #cur is the cursor
#Returns a list with at most one tuple, where the first instance is the number of entries
entries_list = cur.fetchall()
if len(entries_list) == 0 or None:
#Adds user to a table in Heroku DB and initializes entry = 1
insertUser(tablename, userid, 1)
await bot.send_message(channel_id, log_message)
con.close()
elif entries_list[0][0] < 2:
updateUserEntries(tablename, entries_list[0][0] + 1, userid)
await bot.send_message(channel_id, log_message)
con.close()
else: #Ban on third entry
await chat.kick(userid, until_date = datetime.now() + timedelta(hours = 6))
await bot.send_message(chat.id, ban_message)
await bot.send_message(channel_id, log_message)
#Removes user from the table (i.e. resets the entry count)
removeUser(tablename, userid)
con.close()
Attempts: The methods I tried using to ban users with are:
update.effective_chat.ban_member(chat_id, user_id) (python-telegram-bot),
bot.kick_chat_member(chat_id, user_id) (python-telegram-bot),
bot.ban_chat_member(chat_id, user_id) (python-telegram-bot – which suggested me to use kick_chat_member instead) and
message.chat.kick(user_id) (aiogram – where message is passed whenever a new user joins the group through Dispatcher.message_handler(content_types='new_chat_members')).
I originally only wanted to temporarily ban the users, so there should be an extra until_date parameter too. But I also tried permanently banning them, which according to the documentation any time less than 30 seconds or more than 366 days counts as a permanent ban. Nothing worked. (By the way, the script in general works and runs great. I am not sharing the full script because I am not comfortable with it being public, and the problem also does not pertain to the whole script anyways.)
Further details: My bot has all group permissions enabled except for adding admins and "remain anonymous". In BotFather it has both group privacy mode and "allow groups?" disabled (because I only use it for my group). At the moment the library I'm using for the bot (in Python) is AIOgram. Previously, it was all written using python-telegram-bot. I am using it in conjunction with psycopg2 to store user data in Heroku's database. At one point I even tried creating a "ban table" in my database, but it still failed to keep the users out since the ban didn't actually ban people. The ban actually only acts as a "kick".
I was thinking about setting up a webhook, but I am not familiar with Flask or Django, and webhooks in general. So that might take me some time. However, I'm not sure it would make a difference since it should not differ much, except maybe speed and memory wise.
I am not sure what I'm doing wrong. I tried to solve the problem in every way that I am capable of (considering that I am pretty new to Telegram API and Telegram python libraries, especially AIOgram).
If anyone has any ideas on how I can go about solving this issue, I would really appreciate it!
(Sidenote: I contacted Telegram support more than a week ago, but I haven't received any response.)

It's not obvious to me what goes wrong here, but I have a few hints/remarks:
which library you use to make the api request should be irrelevant. you can even make the request manually via the browser/curl/whatever if you like to
webhooks have nothing to do with this - how you fetch updates is indpendent of making requests to the api
python-telegram-bot interprets timezone-naive datetimes as UTC - that may have an effect how you pass the until_date
Dislaimer: I'm currently the maintainer of python-telegram-bot.

Thank you, #CallMeStag for your remarks! I ended up "solving" the problem as I describe below. Although, I am still not sure why this worked but using any of the methods I listed in the question didn't.
Disclaimer: This is not an efficient way to approach the issue, and I do not guarantee it will work. I also don't have a high traffic of people joining the group, so these solutions might not work for you.
Partial (non-)solutions: I think there is a "phantom issue" between my bot and Telegram. Somehow Telegram (or Python, I'm not sure) doesn't understand the "kick" methods as "ban", whether I include until_date or not. (Also, Telegram support is non-existent). So, I thought of three (or four) alternative ways to ban someone without having to actually ban them:
Solution 1: Create a table in your database that will contain all the users that you will eventually ban. For your "ban" function, include a message handler that listens for new chat members. Whenever a new user joins (use new_chat_members to catch the event), make it run through all the entries in the table you created (e.g. you could use SELECT COUNT(*) FROM {BANTABLENAME} WHERE USERID = {USERID}) so that it will tell you if the user is banned. If the query returns a value greater than 0, you can just make an IF statement and kick the user out. (You can obviously make it a bit more complicated by manually adding an until_date). The downside is that it can fail to keep a user out since there is a bit of latency/delay when you run SQL queries.
Solution 2: This solution still requires that you create a table of banned users in your database. But instead of letting people join your group directly, direct them to your bot's chat first. Add a command handler for "/start". Whenever a user starts the bot, make it run through the table entries and depending on the result, the bot will either reply with the group link or a message informing them that they can't join (or nothing).
Solution 3: Set up a webhook. I don't know how to do it yet, so you should look it up on the internet. It will require you to learn at least Flask, or Django. This is definitely a nicer solution compared to the other ones. Usually, this "ban problem" does not pertain to whether you're using a webhook or not (like CallMeStag said). But compared to my next proposal (which works somehow), this is definitely a better practice.
Solution 4 (Verified): I was curious and wanted a more direct solution, so I decided I would just try manually making requests through www.api.telegram.org/bot<TOKEN>/METHOD?PARAMETERS using GET and POST HTTPS methods in Python. This is definitely not an ideal solution if you have a high traffic of users. Because you might get errors due to too many requests. Also, depending on what you're trying to do, it might be too slow. So definitely use webhooks instead. In my case, I only need to POST the ban using the banChatMember method, which also happens with a pretty low frequency. So I should not encounter problems. (I will update if I do).
Further doubts: I am not sure why solution 4 works, but directly banning with python-telegram-bot or AIOgram doesn't. I did not try Telethon or Pyrogram. They should be similar, but they also allow you to log-in as a user (i.e. it will start a session of your account through the bot), which could potentially work.
Update: It doesn't work. Banned users still don't get banned. It only works when I manually do it for some reason. The whole thing really doesn't make any sense. It's like it doesn't work while polling, but it works if I manually send out www.api.telegram.org/bot<token>/banChatMember?chat_id=<chatid>&user_id=<userid>.

Django and CFR 21 Part 11

I have to transform my Django application so that it is compliant with "21 CFR Part 11", that is make electronic records have the same validity as signed paper records. Is there any project or application I should look at?
Some issues:
audit trail: every change in selected models must be traced (who, when, what)
detect unauthorized record editing: if a record has been changed/added/deleted outside normal procedure, the application should detect it
for particular operations, user has to enter the password again
passwords must be changed periodically and must satisfy certain criteria
etc...
I've found no ready solution on the net...

I work in an environment requiring CFR 21 Part 11 and similar. I have not yet gotten our apps fully compliant, but I have gone through a number of trial and errors so helpfully I can get you started in a few places.
1) I would also suggest Django reversion; however, you will require a little more than what it offers to achieves a variable level audit trail with the action that was taken (in addition to by whom and when). For this I used one of the reversion signals to turn the comment field into a dict that could be evaluated and then called for any variable in the row and the action that was taken on it etc. This is below:
https://github.com/etianen/django-reversion
#receiver(reversion.pre_revision_commit)
def it_worked(sender, **kwargs):
currentVersion = kwargs.pop('versions')[0].field_dict
fieldList = currentVersion.keys()
fieldList.remove('id')
commentDict = {}
try:
pastVersion = reversion.get_for_object(kwargs.pop('instances')[0])[0].field_dict
except IndexError:
for field in fieldList:
commentDict[field] = "Created"
except TypeError:
for field in fieldList:
commentDict[field] = "Deleted"
else:
for field in fieldList:
try:
pastTest = pastVersion[field]
except KeyError:
commentDict[field] = "Created"
else:
if currentVersion[field] != pastTest:
commentDict[field] = "Changed"
else:
commentDict[field] = "Unchanged"
comment = commentDict
revision = kwargs.pop('revision')
revision.comment = comment
revision.save()
kwargs['revision'] = revision
sender.save_revision
2/3) You are going to need to use an object-level permission system for this. I have implemented django-guardian. Pretty much the only limit on the complexity you can implement is how many things you can keep straight yourself. The base set of permissions you will need to implement are view, edit, delete, and some sort of data controller/manager role; however, you will probably want to go more complicated. I would highly suggest using class-based-views and mixins for permission checking, but function based can work as well. This can also be used to prompt for password for certain actions because you can control what happens to a field in any way you like.
https://github.com/lukaszb/django-guardian
4) Expiring passwords can be implemented with even just the Django auth system if you want or any user account management app. You will just need to add an extra field to record whatever datetime you want to begin your expiry countdown. Then on login just check for time from countdown and see if they have gone beyond the window, and if so require them to create a new password by directing them through the built-in view for password change or which mechanism is appropriate to your app.
I will tell you the most difficult part of implementing CFR 21 Part 11 will be getting the appropriate people to tell you exactly what your project should do to meet requirements, and getting inspected for compliance can be time consuming and costly.
Hope this helps you get started.

Django Reversion might give you a start on an audit trail, although you probably don't need all of its facilities.
For 2, 3 and 4 on your list, those are things you'll most likely end up coding yourself.

Is there a function to check whether an ID you want to use for an entity is available?

I think I read something about a function appengine has that can tell whether an ID / key you want to use for an entity is available, or if there was a function to get an available ID to choose. App engine team said also that we should set the ID when the entity is created and not change it. But in practice we can just copy everything to a new entity with the new ID?
Thanks!
Update
I think the function I'm looking for is allocateIDs from the docs:
http://code.google.com/appengine/docs/python/datastore/functions.html

To reserve one or more IDs, use allocate_ids(). To check whether an ID is already taken, just construct a Key for it using Key.from_path(kind, id) and try to db.get() it. Also note that IDs for keys with a parent are taken from separate pools and are only unique among keys with the same parent.

On the page describing transactions, a use case is presented where the entity in question, a SalesAccount is updated, or if the account doesn't exist, it is created instead. The technique is to just try to load the entity with the given key; and if it returns nothing, create it. It's important to do this inside a transaction to avoid the situation where two users are both racing for the same key, and both see that it doesn't exist (and both try to create it).

Warning while saving opportunity in OpenERP

After creating an opportunity, when I click on save, it gives me this warning for some users:
"Operation prohibited by access rules, or performed on an already deleted document (Operation: read, Document type: User Modification)."
What's causing this problem?
Update: I have created a new user of admin type and I added these groups:
Sales/User,Sales/User All Leads,Survey/User,Tools/User. It gives me a warning while creating an opportunity.
I added the following groups to the new user and it's working fine: Employee, PartnerManager, Marketing/User, Accounting/Accountant, Accounting/Invoice, Accounting/Manager, Administration/Access Rights, Human Resource/Manager, Human Resource/User, Knowledge/User, Marketing/Manager, Project/Manager, Sales/Manager, Tools/Manager, trimax/AdminMeeting, Trimax/‌​SalesExecutive, Trimax/Vertical, Trimax/SalesHead, Trimax/SalesManager, Useability/An‌alyticAccounting, Useability/Extended, View, Useability/MultiCompanies, Useability/No One, Useability/Product Uos View, Useability/Product Variant, Warehouse/Manager, Warehouse/User, Tools/User, Administration/Configuration
But in already created user if I remove the above groups which give warning ,then also it shows same warning.
The solution I mentioned above was for new users,but for some existing users it was still giving problem. So I removed some unwanted groups, which I didn't needed for those users and it worked, now it does not show warning. Was the warning due to some access right overlap or something else?

Those users probably don't have access rights for the opportunity object, or some child object. Read the access rights documentation for more details.
Update: You said that you're having trouble configuring a new user. As an experiment, try adding permissions to an existing user instead of starting a brand new user. Also, check that you've configured the roles as well as the permissions, that trips me up sometimes.
Another Update: You said that removing some groups stopped the warning. It may be that you have removed all access rules from the object. If no groups are explicitly granted access to something in OpenERP, then everyone is granted access. If you really want to figure out what's going on, I suggest you read the documentation I linked to above. You can also search for the error message in the source code and see exactly what it is complaining about. I find it really helpful to run the OpenERP server in debug mode, and step through the code when I'm trying to understand some weird behaviour like this. You can also try and figure out exactly which change triggers this error by adding and removing groups until you find a single change that makes the problem happen.

Hi these is due to the access right problem .So you give the proper access right to your user(for creating the oppurtunity)

same problem here.I'm using Multi-company, I provid all permistion to the user

SQLAlchemy many-to-many orphan deletion

I'm trying to use SQLAlchemy to implement a basic users-groups model where users can have multiple groups and groups can have multiple users.
When a group becomes empty, I want the group to be deleted, (along with other things associated with the group. Fortunately, SQLAlchemy's cascade works fine with these more simple situations).
The problem is that cascade='all, delete-orphan' doesn't do exactly what I want; instead of deleting the group when the group becomes empty, it deletes the group when any member leaves the group.
Adding triggers to the database works fine for deleting a group when it becomes empty, except that triggers seem to bypass SQLAlchemy's cascade processing so things associated with the group don't get deleted.
What is the best way to delete a group when all of its members leave and have this deletion cascade to related entities.
I understand that I could do this manually by finding every place in my code where a user can leave a group and then doing the same thing as the trigger however, I'm afraid that I would miss places in the code (and I'm lazy).

The way I've generally handled this is to have a function on your user or group called leave_group. When you want a user to leave a group, you call that function, and you can add any side effects you want into there. In the long term, this makes it easier to add more and more side effects. (For example when you want to check that someone is allowed to leave a group).

I think you want cascade='save, update, merge, expunge, refresh, delete-orphan'. This will prevent the "delete" cascade (which you get from "all") but maintain the "delete-orphan", which is what you're looking for, I think (delete when there are no more parents).

I had the same problem about 3 months ago, i have a Post/Tags relation and wanted to delete unused Tags. I asked on irc and SA's author told me that cascades on many-to-many relations are not supported, which kind of makes sense since there is no "parent" in many-to-many.
But extending SA is easy, you can probably use a AttributeExtension to check if the group became empty when is removed from a User and delete it from there.

Could you post a sample of your table and mapper set up? It might be easier to spot what is going on.
Without seeing the code it is hard to tell, but perhaps there is something wrong with the direction of the relationship?