I'm trying to run the deit colab notebook found here:
https://colab.research.google.com/github/facebookresearch/deit/blob/colab/notebooks/deit_inference.ipynb
but I'm running into an issue in the second cell, specifically the import timm line, which returns this:
ImportError: cannot import name 'container_abcs' from 'torch._six'
Issue related to this error here:
Try a specific version of timm library:
!pip install timm==0.3.2
when I install torch==1.9.0 and torch-geometric,
the old code has the errors.
here is my solution:
TORCH_MAJOR = int(torch.__version__.split('.')[0])
TORCH_MINOR = int(torch.__version__.split('.')[1])
if TORCH_MAJOR == 0:
import collections.abc as container_abcs
else:
from torch._six import container_abcs
change to：
TORCH_MAJOR = int(torch.__version__.split('.')[0])
TORCH_MINOR = int(torch.__version__.split('.')[1])
if TORCH_MAJOR == 1 and TORCH_MINOR < 8:
from torch._six import container_abcs,int_classes
else:
import collections.abc as container_abcs
int_classes = int
In my case it worked with
pip install timm==0.4.12
Related
Is the H2OModelSelectionEstimator deprecated? When I run the code
from h2o.estimators import H2OModelSelectionEstimator
I get the message: ImportError: cannot import name 'H2OModelSelectionEstimator' from 'h2o.estimators'
Try this instead:
from h2o.estimators.model_selection import H2OModelSelectionEstimator
If you can't import it, then you probably don't have the latest version of H2O, so you should download it. ModelSelection was just released in 3.36.0.1.
I have written a function that will install the Module required to run a script. My problem is that the function runs every time the script is running. I need to run the function only the first time the script is running so that after installing the module the function does not run every time the script is running.
My code is
import importlib
import subprocess
import pkg_resources
import os, time, json, datetime, sys
def import_and_install(package):
try:
importlib.import_module(package)
except (ModuleNotFoundError, pkg_resources.DistributionNotFound) as e:
print("{0} module is not installed.\n Don't worry. will take care\n".format(package))
package = [package]
subprocess.check_call([sys.executable, '-m', 'pip', 'install'] + package)
packages = ['pandas', 'numpy', 'threading', 'xlwings']
for package in packages:
import_and_install(package)
import pandas as pd
import threading
import xlwings as xw
import numpy as np
I am not sure what you mean by running the script only the first time the script is running?.
does that mean you want to run the script only once per computer/VM?.
by the looks of it, it seems that the script that you want to run is for handling dependencies and packaging management, if that is the case I would recommend you use packages manager instead such as Poetry(https://python-poetry.org/).
if you still consist on doing everything on your own.
the simplest solution I could think of Is creating a file that stores a flag that tells you if the script has already ran or not.
but there might be better solutions
We can compare the list of packages required with the list of packages installed.
To get the list of packages installed we can use:
import pkg_resources
pkg_resources.working_set
This will return an iterator from where we can get the modules installed names (key) and versions (version).
import pkg_resources
modules_installed = {pkg.key:pkg.version for pkg in pkg_resources.working_set}
# this will return a dict like this one, for example:
# {'setuptools': '65.3.0',
# 'pip': '22.2.2',
# 'xlwings': '0.28.5'
# ...}
Now we can compare them with a function that will help to verify if a package is install and also verify the version in case that the package has '==' in the value:
# package: for example 'pandas==1.5.0'
# modules_installed: {pkg.key:pkg.version for pkg in pkg_resources.working_set}
def is_pkg_installed(package, modules_installed) -> bool:
pkg_name = package.split('==')[0] if '==' in package else package
pkg_ver = package.split('==')[1] if '==' in package else None
if pkg_name not in modules_installed:
return False
elif pkg_ver:
installed_version = modules_installed[pkg_name]
if pkg_ver != installed_version:
return False
return True
Using list comprehensions we can get the list of packages_to_install:
packages_to_install = [package for package in packages_list
if not is_pkg_installed(package, modules_installed)]
# values example: packages_to_install = ['pandas==1.5.0', 'numpy']
To install the packages_to_install, we can use the next function that installs all the packages that are in the list:
import subprocess
import sys
def pip_install(packages: list) -> None:
if not packages:
return
# this will do a call like this one:
# ../python.exe -m pip install pandas=1.5.0, numpy
subprocess.check_call([sys.executable, '-m', 'pip', 'install'] + packages)
finally the complete example, where pandas has a specific version selected pandas==1.5.0:
import subprocess
import pkg_resources
import sys
def pip_install(packages: list) -> None:
if not packages:
return
subprocess.check_call([sys.executable, '-m', 'pip', 'install'] + packages)
def is_pkg_installed(package, modules_installed) -> bool:
pkg_name = package.split('==')[0] if '==' in package else package
pkg_ver = package.split('==')[1] if '==' in package else None
if pkg_name not in modules_installed:
return False
elif pkg_ver:
installed_version = modules_installed[pkg_name]
if pkg_ver != installed_version:
return False
return True
def install_packages(packages_list: list) -> None:
modules_installed = {pkg.key:pkg.version for pkg in pkg_resources.working_set}
packages_to_install = [package for package in packages_list
if not is_pkg_installed(package, modules_installed)]
if packages_to_install:
print(f"{packages_to_install} modules are not installed.\nDon't worry. will take care\n")
pip_install(packages_to_install)
packages = ['pandas==1.5.0', 'numpy', 'threading', 'xlwings']
install_packages(packages)
import pandas as pd
import threading
import xlwings as xw
import numpy as np
install_packages() will get the list of packages that are not installed or have different version, and if the list is not empty, run pip_install()
I am running my ML code and getting this error-
Enter website name=> www.google.com
Traceback (most recent call last):
File "Dphishing.py", line 12, in <module>
p2.category2(website)
File "C:\xampp\htdocs\Detect_Phishing_Website\p2.py", line 8, in category2
page = whois.whois(website)
AttributeError: module 'whois' has no attribute 'whois'
My code is:
# -*- coding: utf-8 -*-
import p1
import p2
import p3
import p4
import pandas as pd
#import numpy as np
website = str(input("Enter website name=> "))
p1.category1(website)
p2.category2(website)
p3.category3(website)
p4.category4(website)
read = pd.read_csv(r'C:\Users\Anushree\Desktop\college\4th year project\Detect_Phishing_Website\phishing5.txt',header = None,sep = ',')
read = read.iloc[:,:-1].values
dataset = pd.read_csv(r'C:\Users\Anushree\Desktop\college\4th year project\Detect_Phishing_Website\Training Dataset1.csv')
X = dataset.iloc[:,:-1].values
y = dataset.iloc[:,-1].values
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.2,random_state = 1001)
from sklearn.ensemble import RandomForestRegressor
regressor = RandomForestRegressor(n_estimators = 10,criterion = "mse",random_state = 2)
regressor.fit(X_train,y_train)
y_pred = regressor.predict(X_test)
from sklearn.model_selection import cross_val_score
accuracy = cross_val_score(estimator = regressor,X=X_train,y=y_train,cv = 5)
accuracy.mean()
accuracy.std()
Detect_phishing_website = regressor.predict(read)
if Detect_phishing_website == 1:
print("legitimate website")
elif Detect_phishing_website == 0:
print ('suspicious website')
else:
print('phishing website')
The code of file p2.py is:
import re
import whois
def category2(website):
file_obj = open(r'C:\Users\Anushree\Desktop\college\4th year project\Detect_Phishing_Website\phishing5.txt','a')
#8 Domain Registration Length
page = whois.whois(website)
if type(page.expiration_date) == list:
domain_reg_len = (page.expiration_date[0] - page.creation_date[0]).days
else:
domain_reg_len = (page.expiration_date - page.creation_date).days
#print domain_reg_len
if domain_reg_len <= 365:
file_obj.write('-1,')
else:
file_obj.write('1,')
#9 Using Non-Standard Port
match_port = re.search(':[//]+[a-z]+.[a-z0-9A-Z]+.[a-zA-Z]+:([0-9#]*)',website)
if match_port:
print (match_port.group())
if match_port.group(1) == '#':#represent multiple ports are active on url
file_obj.write('-1,')
else:
file_obj.write('1,')
else:
file_obj.write('1,')
file_obj.close()
I have already tried uninstalling whois and then reinstalling python-whois using the command pip install python-whois. But that hasn't helped with the error.
How can I understand what is going wrong, and how I can correct it?
Reason for your error:
You have not installed the whois command on your system.
Ubuntu: Use sudo apt install whois
Windows: Download and install from here
First uninstall any whois module with pip uninstall whois and pip uninstall python-whois
Solution 1: Use python-whois
Install python-whois with pip install python-whois
Then make sure you already installed the whois command on your machine.
Then your code should work.
Solution 2: Use whois
Install whois command on your machine. If you are on ubuntu sudo apt install whois will do.
Install whois module with pip install whois,
Then use whois.query() instead of whois.whois() in your code.
Source
I can't import pytagcloud in jupyter notebook. How do I solve this problem? I searched some tutorials and also installed other packages required but still doesn't work?
Do you have any suggestion?
Here is my code. Thanks.
import pytagcloud as pytagcloud
import codecs
from bs4 import BeautifulSoup
from konlpy.tag import Twitter
# utf-16 인코딩으로 파일을 열고 글자를 출력하기 --- (※1)
samsung = codecs.open("samsung.txt", encoding="utf-8")
line = samsung.readlines()
twitter = Twitter()
word_dic = {}
for line in line:
malist = twitter.pos(line)
for word in malist:
if word[1] == "Noun": # 명사 확인하기 --- (※3)
if not (word[0] in word_dic):
word_dic[word[0]] = 0
word_dic[word[0]] += 1 # 카운트하기
# 많이 사용된 명사 출력하기 --- (※4)
keys = sorted(word_dic.items(), key=lambda x:x[1], reverse=True)
for word, count in keys[:40]:
print("{0}({1}) ".format(word, count), end="")
print()
keys
import pytagcloud
taglist = pytagcloud.make_tags(keys, maxsize = 80)
taglist
pytagcloud.create_tag_image(taglist, 'wordcolud.jpg', size = (900,600), fontname = 'Nobile', rectangular = False)
%matplotlib inline
import matplotlib.pyplot as plt
from wordcloud import WordCloud as wordcloud
wordcloud = WordCloud(stopwords = stopwords)
wordcloud = wordcloud.generate_from_keys(keys)
wordcloud = WordCloud().generate(keys)
draw_wordcloud_from_rss(keys)
cmd:pytagcloud
cmd:pygame
cmd:simplejson
jupyter notebook: pytagcloud
I asked one of my colleagues and she gave the answer!
So the problem was simple. I installed all those libraries in 'cmd'
but I had to install them in "Anaconda prompt".
So, if you have same problem as mine, try this.
# Anaconda prompt
pip install pygame
pip install -U pytagcloud
pip install simplejson
jupyter notebook.
# Import.
I succeeded to import pygame, pytagcloud and simplejson libraries.
Yet, I have still errors in my code of the post above and there is more libraries to install(konply..etc). error is everywhere!
Anyway, I hope this helps someone.
Using rpy2, I want to check if a given package is installed. If it is, I import it. If not, I install it first.
How do I check if it's installed?
from rpy2 import *
if not *my package is installed*:
rpy2.interactive as r
r.importr("utils")
package_name = "my_package"
r.packages.utils.install_packages(package_name)
myPackage = importr("my_package")
Here is a function that'd do it on the Python side
(note the contriburl, that should be set to a CRAN mirror, and that the case where installing the library is failing is not handled).
from rpy2.rinterface import RRuntimeError
from rpy2.robjects.packages import importr
utils = importr('utils')
def importr_tryhard(packname, contriburl):
try:
rpack = importr(packname)
except RRuntimeError:
utils.install_packages(packname, contriburl = contriburl)
rpack = importr(packname)
return rpack
You can use the following function I got from #SaschaEpskamp's answer to another SO post:
pkgTest <- function(x)
{
if (!require(x,character.only = TRUE))
{
install.packages(x,dep=TRUE)
if(!require(x,character.only = TRUE)) stop("Package not found")
}
}
And use this instead to load your packages:
r.source("file_with_pkgTest.r")
r.pkgTest("utils")
In general, I would recommend not try to write much R code inside Python. Just create a few high-level R functions which do what you need, and use those as a minimal interface between R and Python.
import sys,subprocess
your_package = 'nltk'
package_names = subprocess.Popen([pip freeze],
stdout=subprocess.PIPE).communicate()[0]
pakage = package_names.split('\n')
for package in packages:
if package ==your_package:
print 'true'