Develop Reference

error retrieving the user data from tiktok

I want to extract the user stats from unofficial tiktokapi (https://github.com/davidteather/TikTok-Api)
the code i used is below
from TikTokApi.tiktok import TikTokApi
with TikTokApi() as api: # .get_instance no longer exists
for trending_video in api.trending.videos():
user_stats = trending_video.author.info_full['stats']
if user_stats['followerCount'] >= 10000:
print(user_stats)
but i keep on getting this error -
RuntimeError: This event loop is already running
Task exception was never retrieved
future: <Task finished name='Task-5' coro=<Connection.run() done, defined at C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_connection.py:240> exception=NotImplementedError()>
Traceback (most recent call last):
File "C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_connection.py", line 247, in run
await self._transport.connect()
File "C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_transport.py", line 132, in connect
raise exc
File "C:\Users\siddh\anaconda3\lib\site-packages\playwright\_impl\_transport.py", line 120, in connect
self._proc = await asyncio.create_subprocess_exec(
File "C:\Users\siddh\anaconda3\lib\asyncio\subprocess.py", line 236, in create_subprocess_exec
transport, protocol = await loop.subprocess_exec(
File "C:\Users\siddh\anaconda3\lib\asyncio\base_events.py", line 1676, in subprocess_exec
transport = await self._make_subprocess_transport(
File "C:\Users\siddh\anaconda3\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
i tried using asyncio but the error keeps on coming any fixes to this

please note that if you're running the code in Jupyter Notebook - it won't work. As the documentation of this library says:
Note: Jupyter (ipynb) only works on linux
Paste your code into .py file and try running it as a Python script instead with:
python tiktok_script.py

uvicorn error on AWS EC2 with uvicorn + fastapi

I have a server running locally. When I run it on AWS EC2 and send a request from outside on port 8000, I get the following error:
$ uvicorn sql_app.main:app --host="0.0.0.0" --port=8000
INFO: Started server process [9806]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 170, in handle_events
event = self.conn.next_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 425, in next_event
event = self._extract_next_receive_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
event = self._reader(self._receive_buffer)
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_readers.py", line 73, in maybe_read_from_IDLE_client
request_line_re, lines[0], "illegal request line: {!r}", lines[0]
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 88, in validate
raise LocalProtocolError(msg)
h11._util.RemoteProtocolError: illegal request line: bytearray(b'\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03\x91\xa5\xe2Y\xf0\xa1\xdd\x1d+\x08\x1c\r\x15X\x1d#\x1e/\xb1N\x00\xb5\xe5\xec\xf3F\x1fm\x03\xa1{> \xa80\xb4\x14\x1aUs\xaa\xcd\xc3<s\xcd\xd1\x17\xdf3\x0e\xdbh\xd1c\x88}\x8c\x1f\xa5\x15\x9aa\x14I\x00 ')
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 170, in handle_events
event = self.conn.next_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 425, in next_event
event = self._extract_next_receive_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
event = self._reader(self._receive_buffer)
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
raise LocalProtocolError("illegal request line")
h11._util.RemoteProtocolError: illegal request line
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 170, in handle_events
event = self.conn.next_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 425, in next_event
event = self._extract_next_receive_event()
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
event = self._reader(self._receive_buffer)
File "/home/ec2-user/.local/lib/python3.7/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
raise LocalProtocolError("illegal request line")
h11._util.RemoteProtocolError: illegal request line
WARNING: Invalid HTTP request received.
It would be very nice if you could tell me how to do it on port 80.

I was getting the same arcane WARNING: Invalid HTTP request received. error with an unhelpful stack trace. I tried all of the environment variable tweaks recommended and none worked (see FastAPI issue #680, uvicorn issue #441).
My issue was that when I was calling my FastAPI microservice I was using https when my microservice did not have HTTPS support. I changed the url from https to http and it started working as expected.
Note that if your service requires HTTPS support you can add HTTPS support as Ilgizar Murzakov suggests.

Had the same issue. Disabling http2 on application load balancer helped me.

I had the same issue and solved it by adding HTTPS support. I configured NGINX with LetsEncrypt certificate. Make sure that the ports that you use are open at your security group settings.

this is because uvicorn updated to display detailed error. uvicorn #886
then, use uvicorn 0.13.1 which is before the update. this version won't display the error.
P.S: this update is to fix this error. and uvicorn haven't fixed the error for now. uvicorn #1296

SynchronousOnlyOperation error when upgrading from python 3.6 to python 3.7 using django channels

I've been attempting to upgrade python from 3.6 to 3.7 for our Django & Django channels application. With that change, Django throws a SynchronousOnlyOperation anytime any HTTP request is made (even if it has nothing to do with WebSockets). My guess is that somehow the python upgrade has made the Django check more strict.
I believe that Django channels are serving both the HTTP requests and WebSocket requests so it expects all code to be async compliant.
How do I get Django channels runserver to run the wsgi app synchronously, while the channel consumers asynchronously?
# project/asgi.py
application = ProtocolTypeRouter({"http": get_asgi_application(), "websocket": routing})
# project/wsgi.py
application = get_wsgi_application()
Stacktrace:
It's clear to me that the auth middleware that is running for a normal wsgi view is not async compliant, how can I get it to run in a sync environment?
ERROR Internal Server Error: /a/api
Traceback (most recent call last):
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 233, in _get_session
return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/___/___/___/___/apps/core/middleware.py", line 60, in __call__
request.is_user_verified = request.user.is_verified()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 240, in inner
self._setup()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 376, in _setup
self._wrapped = self._setupfunc()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django_otp/middleware.py", line 38, in _verify_user
user.otp_device = None
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 270, in __setattr__
self._setup()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 376, in _setup
self._wrapped = self._setupfunc()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/middleware.py", line 24, in <lambda>
request.user = SimpleLazyObject(lambda: get_user(request))
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/middleware.py", line 12, in get_user
request._cached_user = auth.get_user(request)
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/__init__.py", line 174, in get_user
user_id = _get_user_session_key(request)
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/__init__.py", line 58, in _get_user_session_key
return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 65, in __getitem__
return self._session[key]
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 238, in _get_session
self._session_cache = self.load()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/db.py", line 43, in load
s = self._get_session_from_db()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/db.py", line 34, in _get_session_from_db
expire_date__gt=timezone.now()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 425, in get
num = len(clone)
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 269, in __len__
self._fetch_all()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 1303, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1154, in execute_sql
cursor = self.connection.cursor()
File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/asyncio.py", line 24, in inner
raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.
Versions
Django==3.1.1
channels==3.0.2
channels-redis==3.2.0

So a summary of what I've tried:
First, I have a channels app that had no database at all, no contrib apps except staticfiles.
I added database, DRF and one app with a single model and list/detail views
Enabled the sync http consumer:
# dwtools.routing
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application
import romanize.routing
application = ProtocolTypeRouter(
{
"websocket": URLRouter(romanize.routing.websocket_urlpatterns),
"http": get_asgi_application(),
},
)
settings:
INSTALLED_APPS = [
"channels",
"romanize.apps.RomanizeConfig",
"django.contrib.staticfiles",
"rest_framework",
"agencies.apps.AgenciesConfig",
]
WSGI_APPLICATION = "dwtools.wsgi.application"
ASGI_APPLICATION = "dwtools.routing.application"
I can pull up DRF's standard views, create entries etc.
Then added a simple middleware that accesses the database (gets a count of the number of Agencies and puts it in request). Not even using MiddlewareMixin, just barebones init can call protocol.
System check identified no issues (0 silenced).
December 07, 2020 - 14:38:56
Django version 3.1.4, using settings 'dwtools.settings'
Starting ASGI/Channels version 3.0.2 development server at http://127.0.0.1:3401/
# ^^^^^^^^ ^^^^^^^^^^^ ( Channels runserver)
Quit the server with CONTROL-C.
HTTP GET /agencies/ 200 [0.05, 127.0.0.1:60460]
The stack uses local nginx to route to port 3401 for /ws (websocket) and /agencies (http). Perhaps you spot something...

Finally got to the bottom of this. For us, we were using gevent and had this line in our manage.py
# manage.py
import gevent
gevent.patch_all()
For whatever reason, that does not play nicely with python 3.7+, Django channels, and Django (where python 3.6, Django channels, and Django worked fine).
We were lucky and were able to remove this from our codebase; we did not figure out how to fix the gevent patch.

Running telethon on pythonanywhere

try to run a basic scripts that adds members to a telegram group over pythoneverywhere server. I installed the main package which is telethon in the server. but i am getting the error in the image below. what am i not doing right?
error message is given below
21:02 ~/Villians TV/Villians5/TeleGram-Scraper-master $ python3 villians5.py
Traceback (most recent call last):
File "villians5.py", line 17, in <module>
client.connect()
File "/home/somti13/.local/lib/python3.8/site-packages/telethon/sync.py", line 39, in syncified
return loop.run_until_complete(coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
return future.result()
File "/home/somti13/.local/lib/python3.8/site-packages/telethon/client/telegrambaseclient.py", line 472, in connect
if not await self._sender.connect(self._connection(
File "/home/somti13/.local/lib/python3.8/site-packages/telethon/network/mtprotosender.py", line 125, in connect
await self._connect()
File "/home/somti13/.local/lib/python3.8/site-packages/telethon/network/mtprotosender.py", line 250, in _connect
raise ConnectionError('Connection to Telegram failed {} time(s)'.format(self._retries))
ConnectionError: Connection to Telegram failed 5 time(s)

The mtproto connection type for telegram does not work for free accounts on PythonAnywhere. Only http(s) connections out of PythonAnywhere will work for free accounts.

Redis example giving HTTP 400: Bad request error

I am trying to cache MySQL queries in my Cherrypy server.
I could not figure out how to solve the error when I was installing pylibmc, so I decided to use Redis-py.
Here I am trying a very simple example.
import redis
cache = redis.StrictRedis(host='localhost', port=8080, db=0)
...
...
cache.set('0', '1') # I also tested with other string keys, but failed with same error
and it's throwing the following error!
[05/May/2014:13:11:13] HTTP Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/Library/Python/2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/Library/Python/2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "server.py", line 92, in submit_data
cache.set(str(idx), '1')#res)
File "/Library/Python/2.7/site-packages/redis/client.py", line 897, in set
return self.execute_command('SET', *pieces)
File "/Library/Python/2.7/site-packages/redis/client.py", line 461, in execute_command
return self.parse_response(connection, command_name, **options)
File "/Library/Python/2.7/site-packages/redis/client.py", line 471, in parse_response
response = connection.read_response()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 339, in read_response
response = self._parser.read_response()
File "/Library/Python/2.7/site-packages/redis/connection.py", line 118, in read_response
(str(byte), str(response)))
InvalidResponse: Protocol Error: H, TTP/1.1 400 Bad Request
I could not figure out what was wrong, and my website runs without a problem on localhost at port 8080 when I am not using Redis.

Your webserver is running on port 8080, not your Redis server. Your Redis server is most likely running on port 6379, unless you changed your config for some reason. Right now you are trying to run Redis queries against your webserver, and that is not going to work. Make sure you are connecting to the correct Redis server address and port and then try again.