I have installed Google BigQuery and I'm trying to query from BigQuery inside of Python.
from google.cloud import bigquery
client = bigquery.Client(project='aaa18')
But I'm getting this error and I don't know what this means :
C:\Users\udgtlvr\AppData\Local\Continuum\anaconda3\python.exe C:/Users/udgtlvr/untitled4/bigquery
Traceback (most recent call last):
File "C:/Users/udgtlvr/untitled4/bigquery", line 2, in <module>
from google.cloud import bigquery
File "C:\Users\udgtlvr\AppData\Local\Continuum\anaconda3\lib\site-packages\google\cloud\bigquery\__init__.py", line 35, in <module>
from google.cloud.bigquery.client import Client
File "C:\Users\udgtlvr\AppData\Local\Continuum\anaconda3\lib\site-packages\google\cloud\bigquery\client.py", line 43, in <module>
from google import resumable_media
ImportError: cannot import name 'resumable_media' from 'google' (unknown location)
What are the steps I need to do in order to get this thing work?
Besides what #Hitobat mentions in his comment (reinstalling the google-cloud-bigquery module), you could also try only installing the module that is failing, the module google-resumable-media.
To verify which module is missing or might be outdated, you can run the following command:
pip freeze | grep google
This will list all the google's modules you have installed.
Just to make sure you did install the required official packages, you should have used this source first https://googleapis.dev/python/bigquery/latest/index.html
They specify the install considering you use a virtual environment
Windows
pip install virtualenv
virtualenv <your-env>
<your-env>\Scripts\activate
<your-env>\Scripts\pip.exe install google-cloud-bigquery
If you did right, installing the Google Cloud Python library usually installs the required dependencies (in your case the google-resumable-media)
pip install google-cloud
As I see your problem is more than an Anaconda/Local Python paths problem... So you may first verify where the Python Client for Google BigQuery is being installed.
If you definitely want to work with Anaconda environments, then try to install the packages from the Anaconda Navigator environments manager... That should solve the dependencies problem.
Create a service account with desired BigQuery roles and download JSON key file (example: data-lab.json). Use below code:
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = "data-lab.json"
from google.cloud import bigquery
client = bigquery.Client()
This is what I do and works fine . I use Colab/Jupyter Notebooks to access Bigquery.
First Option is to authenticate using browser
from google.colab import auth
auth.authenticate_user()
The second option is to use service account for authentication which is explained above
Once you are done with authentication then the below code should work
from google.cloud import bigquery
client = bigquery.Client(project='project_id')
%load_ext google.cloud.bigquery
Run Bigquery commands using magic link
%%bigquery --project project_id
select * from table limit 1
Related
I am trying to launch a script I wrote that is supposed to read data from a firebase db but it throws the following error:
Traceback (most recent call last):
File "myScript.py", line 8, in <module>
from firebase import Firebase
File "/Users/georgeoprea/Library/Python/3.8/lib/python/site-packages/firebase/__init__.py", line 20, in <module>
from Crypto.PublicKey import RSA
ModuleNotFoundError: No module named 'Crypto'
I have tried to install Crypto in the following ways:
pip3 install Crypto
pip3 install pycryptodome
When I run pip3 show Crypto I get the following output:
Name: crypto
Version: 1.4.1
Summary: Simple symmetric GPG file encryption and decryption
Home-page: https://github.com/chrissimpkins/crypto
Author: Christopher Simpkins
Author-email: git.simpkins#gmail.com
License: MIT license
Location: /Users/georgeoprea/Library/Python/3.8/lib/python/site-packages
Requires: Naked, shellescape
Required-by:
This is the list of imports I have in myScript.py:
from bs4 import BeautifulSoup
import time
import smtplib
from datetime import datetime
import json
import random
from email.message import EmailMessage
from firebase import Firebase
OS info: macOS 12.1 running on M1 Pro.
What could I do to have my script recognise the Crypto module?
This might be somewhat related to this question.
/e:
Ok, since you are using this firebase package, I can hopefully help you out.
First of all, it's the package's fault that it isn't running. While it depends on many external packages, it has none of them defined.
This is what I had to do in a clean virtual environment just to be able to do from firebase import Firebase:
pip install sseclient python_jwt gcloud pycryptodome requests-toolbelt
Here is the requirements.txt I ended up with in the clean environment. Notice, that this was only for importing a single class from the package. There still might be other dependencies hidden somewhere, waiting to throw an exception.
I encourage you to give feedback to the developer about this issue. Even better, fix this yourself and open a Pull Request. You might help others with the same issue.
I found a workaround for this. I simply used another module to read from the firebase db. Instead of using firebase I used firebase_admin as mentioned in the firebase documentation. firebase_admin doesn't use Crypto so there's no more problem from this point of view. However I had to change a little bit how I retrieve and write data.
I'm having a problem with some python code that connects to an azure storage container.
Code:
from azure.storage.blob import BlockBlobService
import logging
import os, sys
block_blob_service = BlockBlobService(account_name = accountName, account_key = accountKey, connection_string=connectionString)
block_blob_service.create_blob_from_path(containerName,blob_name=blobName,file_path=fileNameToUpload)
Ok, so this code works when executed using a command prompt.
When its executed using a SQL agent job:
line 1, in from azure.storage.blob import BlockBlobService ModuleNotFoundError: No module named 'azure'. Process Exit Code 1. The step failed.
pip list:
azure-common 1.1.27
azure-core 1.19.0
azure-storage-blob 1.5.0
azure-storage-common 1.4.2
Using python 3.7.4
The credential that I use to run the SQL agent job is mapped to my userid which has admin privileges on the server.
I used the quickstart to get me started.
Can anyone help please?
You are referencing an old version of azure.storage.version v1.5.0, in the latest version, v12.x.x, you need to use BlobServiceClient instead.
######################################
pip install azure-storage-blob==12.9.0
######################################
blob_service_client = BlobServiceClient(account_url=url, credential=self._account_key)
The link you mentioned it's already pointing to the latest version
I installed flask-upload module in windows 10:
pip install flask flask-wtf flask-uploads
The results were:
Successfully installed Jinja2-2.11.2 MarkupSafe-1.1.1 WTForms-2.3.1 Werkzeug-1.0.1 click-7.1.2 flask-1.1.2 flask-uploads-0.2.1 flask-wtf-0.14.3 itsdangerous-1.1.0
Then in the text editor there is an error when I import the module as shown in the screenshot. unable to import flask-uploads
After running the app.py the following are the errors in cmd:
Error: While importing "app", an ImportError was raised:
Traceback (most recent call last):
File "c:\users\seanv\onedrive\documents\web dev\##pprojects\flask\flask_uploads\myenv\lib\site-packages\flask\cli.py", line 240, in locate_app
__import__(module_name)
File "C:\Users\seanv\OneDrive\Documents\web dev\##pprojects\flask\flask_uploads\app.py", line 4, in <module>
from flask_uploads import configure_uploads, IMAGES, UploadSet
File "c:\users\seanv\onedrive\documents\web dev\##pprojects\flask\flask_uploads\myenv\lib\site-packages\flask_uploads.py", line 26, in <module>
from werkzeug import secure_filename, FileStorage
ImportError: cannot import name 'secure_filename' from 'werkzeug' (c:\users\seanv\onedrive\documents\web dev\##pprojects\flask\flask_uploads\myenv\lib\site-packages\werkzeug\__init__.py)
May someone who understands the problem help me with possible solutions or suggestions. Thank you in advance.
Your app is using Flask-Uploads.
Back in February 2020, there was an update for Werkzeug, a library which Flask and many libraries, including Flask-Uploads, is based on.
This update introduced a breaking change, as Werkzeug changed its API, ie. the import of secure_filename.
I provided a pull request to Flask-Uploads, which the maintainer accepted. But very sadly and unfortunately the maintainer did not want to provide a new package for PyPi.
So, while you could install the updated Flask-Uploads via a commit id from its GitHub repository, you cannot any longer install it from PyPi.
I asked the maintainer for a new release, I also offered my help, but no chance.
So, finally, I decided to fork the library.
Here is the new package on PyPi
https://pypi.org/project/Flask-Reuploaded/
Here is the repository
https://github.com/jugmac00/flask-reuploaded
It is a drop-in replacement. So you just have to install the new package and it just works. No need to change any imports or code in your application.
I recently tried to view Google Calendar using python . To do that I referred Python Quickstart. I did the steps that were mentioned in the website.
I completed the second step and downloaded the secretclient.json file.
In the terminal I typed in :
pip install --upgrade google-api-python-client
as I was instructed in the website.
I ran the python that was on the website and when I compiled I got the error:
Blockquote
Traceback (most recent call last):
File "quickstart.py", line 2, in
from apiclient import discovery
ModuleNotFoundError: No module named 'apiclient'
The Lines which correspond to the error are :
from apiclient import discovery
Why is the apiclient module unavailable ?
Could it be that you're using a different python version than what the pip installed? For example, if you use python3 to execute the problematic import line, but pip is for python2. Or if you use conda or another python distribution that uses a different path to import the packages from.
You can verify it if you just open from the command line:
python
then
from apiclient import discovery
and check if you still get the error.
you can resolve this by going to Script folder of your Python installation directory and running from there
e.g.
cd D:\Python27\Scripts\
python
from apiclient import discovery
Mainly this issue arises when u have more than one python installation , as noob have suggested
I installed the google module by Mario Vilas in my virtual environment in ubuntu 14.04 with python2.7
https://pypi.python.org/pypi/google
I have done this before in both windows and Ubuntu and it worked fine.
However, now when I do the following
>>> from google import search
Traceback (most recent call last):
File "<input>", line 1, in <module>
ImportError: cannot import name search
I am using pycharm and I can view the package and its modules and I can auto insert using ctrl+space
I tried giving total privileges to the virtual venv package using sudo chmod -R ugo+rX but to no avail
The shortest work around for this will be:
from googlesearch import search
Your installation of Python came with a built-in module named google which is taking precedence over the one you installed. You have two options:
Remove the built-in module.
Use importlib to import the desired module by its filesystem path:
google = importlib.import_module('/usr/local/lib/python2.7/site-packages/google/__init__.py')
"from google import search" is giving error as there is no module with the name "google".After "pip install google" i checked the path to find out out the module in lib, but i was not able to find. I checked and found a module with "googlesearch". By doing the below change in my code i was able to solve the issue
OLD : "from google import search"
NEW : "from googlesearch import search"
simply install both google and google-search
pip install google
pip install google-search
It works for me
Just import google and you will be all set :)
import google
It is tested and verified.
I have gone through the same problem and i solve it by importing googlesearch API like this:
from googlesearch import *