I am facing an error of "ImportError: cannot import name 'fetch' from 'js' (unknown location)" whenever I try to install any module in pyodide.
Initially, I tried the following lines of code :
await pyodide.loadPackage('micropip')
await pyodide.runPythonAsync(`
import micropip
await micropip.install('textblob')
from textblob import TextBlob
// ... something
`)
but it still gave the above error.
then i tried to run the code from the official documentation (https://pyodide.org/en/stable/usage/loading-packages.html)
await pyodide.loadPackage("micropip");
await pyodide.runPythonAsync(`
import micropip
micropip.install('snowballstemmer')
import snowballstemmer
stemmer = snowballstemmer.stemmer('english')
print(stemmer.stemWords('go goes going gone'.split()))
`);
but it still gave me the same error.
could someone guide me how to solve this?
i also tried to loadpackage('js') and importing it inside the runPythonAsync but it still failed.
here's the full version of the error :-
PythonError: Traceback (most recent call last):
File "/lib/python3.9/asyncio/futures.py", line 201, in result
raise self._exception
File "/lib/python3.9/asyncio/tasks.py", line 258, in __step
result = coro.throw(exc)
File "/lib/python3.9/site-packages/micropip/_micropip.py", line 185, in install
transaction = await self.gather_requirements(requirements, ctx, keep_going)
File "/lib/python3.9/site-packages/micropip/_micropip.py", line 175, in gather_requirements
await gather(*requirement_promises)
File "/lib/python3.9/asyncio/futures.py", line 284, in __await__
yield self # This tells Task to wait for completion.
File "/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
future.result()
File "/lib/python3.9/asyncio/futures.py", line 201, in result
raise self._exception
File "/lib/python3.9/asyncio/tasks.py", line 256, in __step
result = coro.send(None)
File "/lib/python3.9/site-packages/micropip/_micropip.py", line 284, in add_requirement
metadata = await _get_pypi_json(req.name)
File "/lib/python3.9/site-packages/micropip/_micropip.py", line 88, in _get_pypi_json
return json.loads(await fetch_string(url))
File "/lib/python3.9/site-packages/micropip/_micropip.py", line 60, in fetch_string
return await (await pyfetch(url, **kwargs)).string()
File "/lib/python3.9/site-packages/pyodide/http.py", line 229, in pyfetch
from js import fetch as _jsfetch, Object
ImportError: cannot import name 'fetch' from 'js' (unknown location)
guidance needed.
If you are trying to achieve this in node.js you need to install node-fetch (or any other library that provides fetch function) and make it available as global/globalThis element so that pyodide's js can also access it.
For your example from the documentation regarding snowballstemmer, I was able to run it with the following code inside a file named example.mjs and executing it as node example.mjs:
import fetch from "node-fetch";
globalThis.fetch = fetch;
let pyodide_pkg = await import("./pyodide/pyodide.js");
let pyodide = await pyodide_pkg.loadPyodide({
indexURL: "pyodide",
});
await pyodide.loadPackage("micropip");
await pyodide.runPythonAsync(`
import micropip
await micropip.install('snowballstemmer')
import snowballstemmer
stemmer = snowballstemmer.stemmer('english')
print(stemmer.stemWords('go goes going gone'.split()))
`);
This gives the expected output and the stemmed results as:
['go', 'goe', 'go', 'gone']
PS: While it is a bad practice to use global/globalThis in general but since fetch in browsers is available as a member of window (i.e. window.fetch), I guess it is okay to use it this way.
Related
https://nats-io.github.io/nats.py/modules.html
Traceback (most recent call last):
File "c:\Users\lb\nats.py", line 22, in <module>
asyncio.run(main())
File "D:\anaconda\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "D:\anaconda\lib\asyncio\base_events.py", line 616, in run_until_complete
return future.result()
File "c:\Users\lb\nats.py", line 6, in main
nc = await nats.connect(servers=["nats://216.48.189.5:4222"])
AttributeError: module 'nats' has no attribute 'connect'
An Attribute error: I am not able to figure out what is the issue with 'connect'.
import asyncio
import nats
async def main():
# Connect to NATS!
nc = await nats.connect(servers=["nats://216.48.189.5:4222"])
# Receive messages on 'foo'
sub = await nc.subscribe("foo")
# Publish a message to 'foo'
await nc.publish("foo", b'Hello from Python!')
# Process a message
msg = await sub.next_msg()
print("Received:", msg)
# Close NATS connection
await nc.close()
if __name__ == '__main__':
asyncio.run(main())
Help me with this connect issue, please.
The solution is to rename your file to anything else.
When you type import nats python recursively tries to find a file or folder named nats. In this case the first file if will find is nats.py.
You never define a function named connect in your nats.py so that fails. You want to instead let the recursive import continue upto site_packages where the actual nats folder is that contains the connect function.
For example, if you name this file hello.py:
import hello
print(hello.hello)
print(hello)
print(hello.hello.hello)
You will see that all 3 print statements print the same thing, since hello will be the file itself.
Simply renaming your file anything else will prevent python from finding the module too soon and will keep searching till it find the correct module.
I just ran your code and got no error.
Try uninstalling and reinstalling nats:
pip install nats-py
I have tracked my error to this code, which is in a cog:
#cog_ext.cog_slash(name="help")
async def help_slash(self, ctx, Command_Name=None):
await helpMethod(ctx, Command_Name)
Here is the full error:
Traceback (most recent call last):
File "main.py", line 102, in <module>
class bot(commands.Cog):
File "main.py", line 110, in bot
async def help_slash(self, ctx, Command_Name=None):
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord_slash/cog_ext.py", line 63, in wrapper
opts = manage_commands.generate_options(cmd, desc, connector)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord_slash/utils/manage_commands.py", line 313, in generate_options
SlashCommandOptionType.from_type(param.annotation) or SlashCommandOptionType.STRING
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord_slash/model.py", line 488, in from_type
and isinstance(t, typing._UnionGenericAlias) # noqa
AttributeError: module 'typing' has no attribute '_UnionGenericAlias'
My research suggests this may be an error with Python itself. If so, is there a way to circumvent the problem? I have tested helpMethod: the error comes from the declaration of the slash command.
Thanks so much for reading.
Look at Python 3.8 typing library code, you can see _UnionGenericAlias is not defined.
However, in Python 3.9, it is defined.
So the solution is you have to upgrade your Python version.
I'm looking for a solution to a probably quite simple problem and really would appreciate some help or a hint. I have basic knowledge of python and webscraping.
I want to explore a certain hashtag and the community behind it on twitter. Using twint I've downloaded all tweets mentioning the hashtag into a .csv file. After that I cleaned up the .csv so that there aren't multiple entries of the same user (from multiple tweets with the same hashtag) and saved it as a .txt. I now would like to get some more information about those approximately 1.500 users in said list – mainly the date they joined twitter, number of tweets would be a bonus.
What I've tried:
Twint should be able to do this, but it didn't work (I'm using the docker image provided on their github). I tried to get the user info with:
twint --userlist /bin/userlist.txt --user-full -o userlistfull.csv --csv
Twint puts out a long error message, which if I understand it correctly is related to an open bug in twint:
CRITICAL:root:twint.get:User:'url'
ERROR:root:twint.run:Twint:Lookup:Unexpected exception occurred.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/twint/run.py", line 307, in Lookup
await get.User(self.config.Username, self.config, db.Conn(self.config.Database))
File "/usr/local/lib/python3.6/site-packages/twint/get.py", line 228, in User
await Users(j_r, config, conn)
File "/usr/local/lib/python3.6/site-packages/twint/output.py", line 177, in Users
user = User(u)
File "/usr/local/lib/python3.6/site-packages/twint/user.py", line 31, in User
_usr.url = ur['data']['user']['legacy']['url']
KeyError: 'url'
Traceback (most recent call last):
File "/usr/local/bin/twint", line 8, in <module>
sys.exit(run_as_command())
File "/usr/local/lib/python3.6/site-packages/twint/cli.py", line 339, in run_as_command
main()
File "/usr/local/lib/python3.6/site-packages/twint/cli.py", line 324, in main
run.Lookup(c)
File "/usr/local/lib/python3.6/site-packages/twint/run.py", line 386, in Lookup
run(config)
File "/usr/local/lib/python3.6/site-packages/twint/run.py", line 329, in run
get_event_loop().run_until_complete(Twint(config).main(callback))
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/site-packages/twint/run.py", line 235, in main
await task
File "/usr/local/lib/python3.6/site-packages/twint/run.py", line 270, in run
await self.Lookup()
File "/usr/local/lib/python3.6/site-packages/twint/run.py", line 307, in Lookup
await get.User(self.config.Username, self.config, db.Conn(self.config.Database))
File "/usr/local/lib/python3.6/site-packages/twint/get.py", line 228, in User
await Users(j_r, config, conn)
File "/usr/local/lib/python3.6/site-packages/twint/output.py", line 177, in Users
user = User(u)
File "/usr/local/lib/python3.6/site-packages/twint/user.py", line 31, in User
_usr.url = ur['data']['user']['legacy']['url']
KeyError: 'url'
I've tried to loop over the list and let twint lookup each username individually but it doesnt work either:
import twint
import os
import sys
import nest_asyncio
nest_asyncio.apply()
c = twint.Config()
with open("userlist.txt", "r") as a_file:
for line in a_file:
stripped_line = line.strip()
stripped_line = c.Username
twint.run.Search(c)
Running it with Google Colab I gives me
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 8.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 27.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 64.0 secs
What I'm looking for
What is the easiest solution to get the join dates of those users in the list? Should I use a different library? Could I loop over the list with something like beautifulsoup and scrape the join dates? How would I do this?
Help would be very much aprreciated, thanks in advance!
Try to install it using
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git#origin/master#egg=twint
and make sure your python version higher than 3.6 source
Just replace this line in twint/user.py:
_usr.url = ur['data']['user']['legacy']['url']
to this:
try:
_usr.url = ur['data']['user']['legacy']['url']
except:
_usr.url = ''
I've been trying to make my discord bot translate texts using a module called googletrans. It seems fairly simple and it should have worked without any hassle, or so I thought.
So after my import statements, I have translator = Translator().
My following cog code is:
#commands.command(aliases=["tl", "Tl", "Translate"])
async def translate(self, ctx, *, message):
language = translator.detect(message)
translation = translator.translate(message)
embed = discord.Embed(color=discord.Color.dark_theme())
embed.add_field(name=f"Language: {language} ", value=f'{translation}')
await ctx.send(embed=embed)
But it shows this error: discord.ext.commands.errors.CommandInvokeError: Command raised an exception: AttributeError: 'NoneType' object has no attribute 'group'.
Where am I going wrong? Any help would be much appreciated!
Edit: the full traceback:
Ignoring exception in command translate:
Traceback (most recent call last):
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\discord\ext\commands\core.py", line 85, in wrapped
ret = await coro(*args, **kwargs)
File "C:\Users\wave computer\PycharmProjects\pythonProject\cogs\translate.py", line 13, in translate
language = translator.detect(message)
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\googletrans\client.py", line 255, in detect
data = self._translate(text, 'en', 'auto', kwargs)
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\googletrans\client.py", line 78, in _translate
token = self.token_acquirer.do(text)
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\googletrans\gtoken.py", line 194, in do
self._update()
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\googletrans\gtoken.py", line 62, in _update
code = self.RE_TKK.search(r.text).group(1).replace('var ', '')
AttributeError: 'NoneType' object has no attribute 'group'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\discord\ext\commands\bot.py", line 902, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\discord\ext\commands\core.py", line 864, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "C:\Users\wave computer\PycharmProjects\pythonProject\venv\lib\site-packages\discord\ext\commands\core.py", line 94, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: AttributeError: 'NoneType' object has no attribute 'group'
I had a same problem. Installing alpha version of google trans helped, so try doing this:
pip install googletrans==3.1.0a0
And:
from googletrans import Translator
#client.command()
async def translate(ctx, lang, *, thing):
translator = Translator()
translation = translator.translate(thing, dest=lang)
await ctx.send(translation.text)
I cannot add comments since I do not have enough reputation, so I will edit this post once I get more information about your code.
Could you please upload the full traceback? Also what happens if before calling the method group on the object, you ensures that it is not None ? (I don't know if this is in your code or in the library)
Have you tried debugging and checking at each step what the variables are and if they are not what you think they should be
Finally, I also think that maybe your variable message is always empty, since you catch other parameters with * right before message so this makes the message parameter a keyword-only parameter, which, to the best of my knowledge cannot be passed when using a bot command
Have you tried changing your function's signature to this :
async def translate(self, ctx, message):
?
UPDATE : From what it seems, the problem happens inside the library, however this can happen if the parameter given to the library function is ill-formed.
If you simulate the bot command (await the method in a main file with ctx=None and message="Hello my friend, how are you today ? I hope you are fine"), what happens?
As can be seen on the pypi page about googletrans on section "Language detection", most of the time the function is called with a sentence. Maybe if the sentence is short enough, the detection does not work ?
Also, I think you should indicate in your question what library you are using because there is also Google's official translate API that can be used
I have been using AIOKafka for some time, I had no problems with it, until today.
Strange TypeError shows up when I try to send a message using AIOKafkaProducer.send_and_wait. I also posted this question as issue on AIOKafka's github repository, but it looks like they are kind of inactive. Maybe here someone can help me.
here is the code:
import asyncio
from aiokafka import AIOKafkaConsumer, AIOKafkaProducer
loop = asyncio.get_event_loop()
producer = AIOKafkaProducer(loop=loop, bootstrap_servers="localhost:9092")
async def _initialize(prod, future):
await prod.start()
await prod.send_and_wait("main_topic", str.encode("hello!!"))
future = asyncio.Future()
task = asyncio.ensure_future(_initialize(producer, future))
loop.run_until_complete(task)
print("loop ended!")
loop.close()
here is the error message I get:
yilmazali#yilmazali:~$ python3 aiokafkatest.py
Unexpected error in sender routine
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/aiokafka/producer/producer.py", line 374, in _sender_routine
task.result()
File "/usr/local/lib/python3.6/dist-packages/aiokafka/producer/producer.py", line 418, in _send_produce_req
response = yield from self.client.send(node_id, request)
File "/usr/local/lib/python3.6/dist-packages/aiokafka/client.py", line 415, in send
request, expect_response=expect_response)
File "/usr/local/lib/python3.6/dist-packages/aiokafka/conn.py", line 165, in send
message = header.encode() + request.encode()
File "/usr/local/lib/python3.6/dist-packages/kafka/util.py", line 159, in __call__
return self.method()(self.target(), *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/struct.py", line 42, in _encode_self
[self.__dict__[name] for name in self.SCHEMA.names]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in encode
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in <listcomp>
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in encode
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in <listcomp>
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in encode
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in <listcomp>
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in encode
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 170, in <listcomp>
[self.array_of.encode(item) for item in items]
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in encode
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 132, in <listcomp>
for i, field in enumerate(self.fields)
File "/usr/local/lib/python3.6/dist-packages/kafka/protocol/types.py", line 93, in encode
return Int32.encode(len(value)) + value
TypeError: object of type '_io.BytesIO' has no len()
I did no changes in my kafka structure or libraries. My kafka broker looks fine. I can produce/consume messages with shell scripts.
I had no problems with AIOKafka for the last 2-3 months, the code above worked fine. Out of nowhere, this error showed up and I'm wondering what the problem is.
Any help will be appreciated.
With kindest regards,
Ali
--
update: We ran this code segment on a friend's computer, it worked fine. I advertised my kafka to outside and she successfully wrote to my local kafka topic with the code above. AIOKafka library versions are 0.4.0 on both machines. Also asyncio versions are 3.4.3 on both machines. In short, the problem is not about my kafka or libraries. Something is wrong with my machine but god knows what specifically causes this.
Finally worked on my machine. I simply uninstalled and installed aiokafka module.
Although I am not satisfied with this solution and I would like to venture deeper to the heart of problem, I'm glad I can continue doing my work now.
Hope this helps to fellow strangers with same problem.
May be you've updated kafka-python to version higher than 1.3.5. I did so and aiokafka started failing. I returned at kafka-python 1.3.5 and it seemed OK