I'm new to Python and trying to try manipulate data using 'elasticsearch'. Initially I am just trying to connect using their standard example.
I succesfully installed the package using
pip install elasticsearch
When running the code:
from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()
doc = {
'author': 'kimchy',
'text': 'Elasticsearch: cool. bonsai cool.',
'timestamp': datetime.now(),
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)
print(res['result'])
res = es.get(index="test-index", doc_type='tweet', id=1)
print(res['_source'])
es.indices.refresh(index="test-index")
res = es.search(index="test-index", body={"query": {"match_all": {}}})
print("Got %d Hits:" % res['hits']['total']['value'])
for hit in res['hits']['hits']:
print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])
I get the following error:
[Running] python -u "c:\Users\USERNAME\Dropbox\Stuff\Python\Elasticsearch.py"
Traceback (most recent call last):
File "c:\Users\USERNAME\Dropbox\Stuff\Python\Elasticsearch.py", line 10, in <module>
from elasticsearch import Elasticsearch
ModuleNotFoundError: No module named 'elasticsearch
I looked around and heard something about a .bash_profile but I do not understand what that means?
My environment variable for PYTHONPATH contains
C:\Users\USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Python 3.7
Your Python path probably only contains an alias to the Python executable, but it's trying to import stuff from there. You should open C:\Users\USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Python 3.7
and get the Target from right click > properties on the Python executable.
Set the PYTHONPATH to whatever that path is. It will probably look like C:\Users\USERNAME\AppData\Local\Programs\Python\Python37.
Related
I need to check the change of one parameter, if it has changed - I need to restart the script.
code:
import subprocess, os.path, time
from engine.db_manager import DbManager
DB = DbManager(os.path.abspath(os.path.join('../webserver/db.sqlite3')))
tmbot = None
telegram_config = DB.get_config('telegram')
old_telegram_token = ''
vkbot = None
vk_config = DB.get_config('vk')
old_vk_token = ''
while True:
telegram_config = DB.get_config('telegram')
if old_telegram_token != telegram_config['token']:
if vkbot != None:
tmbot.terminate()
tmbot = subprocess.Popen(['python', 'tm/tm_bot.py'])
old_telegram_token = telegram_config['token']
print('telegram token was updated')
vk_config = DB.get_config('vk')
if old_vk_token != vk_config['token']:
if vkbot != None:
vkbot.terminate()
vkbot = subprocess.Popen(['python', 'vk/vk_bot.py'])
old_vk_token = vk_config['token']
print('vk token was updated')
time.sleep(30)
I get errors:
enter image description here
While there might be subtle differences between unix and windows, the straight-up answer is: you can use PYTHONPATH environment variable to let python know where to look for libraries.
However, if you use venv, I'd recommend activating it first, or calling the relevant binary instead of setting the environment variable.
Consider this scenario: you have a venv at /tmp/so_demo/venv, and you try to run this file:
$ cat /tmp/so_demo/myscript.py
import requests
print("great success!")
Running the system python interpreter will not find the requests module, and will yield the following error:
$ python3 /tmp/so_demo/myscript.py
Traceback (most recent call last):
File "/tmp/so_demo/myscript.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
As I have installed requests in the venv, if I provide the path to python, it will know where to look:
$ PYTHONPATH='/tmp/so_demo/venv/lib/python3.8/site-packages/' python3 /tmp/so_demo/myscript.py
great success!
but using the binary inside the venv is better, as you don't need to be familiar with the internal venv directory paths, and is much more portable (notice that the path I provided earlier depicts the minor version):
$ /tmp/so_demo/venv/bin/python /tmp/so_demo/myscript.py
great success!
I need to get a list of all albums of my YouTube Music uploads library with ytmusicapi and I never work with Python before. I created the headers_auth.json and a test.py file with the following code from the example:
from ytmusicapi import YTMusic
ytmusic = YTMusic('headers_auth.json')
playlistId = ytmusic.create_playlist("test", "test description")
search_results = ytmusic.search("Oasis Wonderwall")
ytmusic.add_playlist_items(playlistId, [search_results[0]['videoId']])
I ran from the Ubuntu terminal: python /home/do/Desktop/ytmusicapi/ytmusicapi-master/test.py
Result error:
Traceback (most recent call last): File
"/home/do/Desktop/ytmusicapi/ytmusicapi-master/test.py", line 1, in
from ytmusicapi import YTMusic File "/home/do/Desktop/ytmusicapi/ytmusicapi-master/ytmusicapi/init.py",
line 2, in
from ytmusicapi.ytmusic import YTMusic File "/home/do/Desktop/ytmusicapi/ytmusicapi-master/ytmusicapi/ytmusic.py",
line 28
auth: str = None,
^ SyntaxError: invalid syntax
How to fix that?
This is because you are using a different version of Python.
The library you are using requires Python>=3.5 wheraas you seem to be using Python2.
If you are curious this code
def f(x:int):
return
is valid in Python3.5+ versions but not in Python2.
You will have to either switch to Python3 to use that library or else you can clone the repo and convert the whole repo to Python2 using something like this(though there might be other ways too).
I'd recommend you simply switch to Python3.5 or higher.
I'm attempting to get a simple scraper up and running to gather data and would like to use Python Scrapy. The rest of the app will be through Nodejs/Express, so I would like to call this script on demand when I need fresh/new data.
The python code runs fine locally through piecharm, but I am seeing issues when it is run as a script.
Through node when I run the server locally and hit /name, it fails with "no module named 'scrapy'
When I run the server through the Anaconda prompt this works fine and scrapy is imported with no error.
I have installed scrapy via conda at the location the express server is being run for both 1 and 2.
From what I've read this may have to do with scrapys need of the Twisted reactor, but as I'm new to Python it's not clear to me what the anaconda terminal is doing differently, and what I would need to do properly from the node side in order to use scrapy.
Nodejs:
app.get('/name', callName);
function callName(req, res) {
console.log("test");
var spawn = require('child_process').spawn;
const pyProg = spawn('python', ['pythonscript.py']);
pyProg.stdout.on('data', function(data) {
console.log(data.toString());
res.write(data);
res.end('end');
});
}
//Print URL for accessing server
console.log('Server running at http://127.0.0.1:8000/')
app.listen(process.env.PORT || 8000, () => console.log("Listening on " + (process.env.PORT || 8000)));
Python script:
try:
import sys
import scrapy
data = "python starting"
print(data)
sys.stdout.flush()
except Exception as exception:
print(exception, False)
print(exception.__class__.__name__ + ": " + exception.message)
Update:
When running import scrapy from the Anaconda interpreter (the other from the comments resulted in "no module found")
Traceback (most recent call last):
File "", line 1, in
File "\Anaconda3\lib\site-packages\scrapy__init__.py", line 34, in
from scrapy.spiders import Spider
File "\Anaconda3\lib\site-packages\scrapy\spiders__init__.py", line 10, in
from scrapy.http import Request
File "\Anaconda3\lib\site-packages\scrapy\http__init__.py", line 11, in
from scrapy.http.request.form import FormRequest
File "\Anaconda3\lib\site-packages\scrapy\http\request\form.py", line 11, in
import lxml.html
File "\Anaconda3\lib\site-packages\lxml\html__init__.py", line 54, in
from .. import etree
ImportError: DLL load failed: The specified module could not be found.
So this looks to be not just interpreter related, but perhaps something additional with Anacondas variables it uses for the terminal?
I am using simple Python script to fetch example data from Solr using Pysolr. First I created my core using the following
[user#user solr-7.1.0]$ ./bin/solr create -c json_db
WARNING: Using _default configset. Data driven schema functionality is enabled by default, which is
NOT RECOMMENDED for production use.
To turn it off:
curl http://localhost:8983/solr/json_db/config -d '{"set-user-property": {"update.autoCreateFields":"false"}}'
Created new core 'json_db'
[user#user solr-7.1.0]$ ./bin/post -c json_db example/exampledocs/*.json
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/json_db/update...
Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file books.json (application/json) to [base]/json/docs
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/json_db/update...
Time spent: 0:00:00.398
After creating the core I ran simple python script to fetch data
from pysolr import Solr
conn = Solr('http://localhost:8983/solr/json_db/')
results = conn.search('*:*')
I am getting this error
Traceback (most recent call last):
File "/home/user/PycharmProjects/APP/application/solr_test.py", line 4, in <module>
results = conn.search({'*:*'})
File "/home/user/PycharmProjects/APP/venv/lib/python3.5/site-packages/pysolr.py", line 723, in search
response = self._select(params, handler=search_handler)
File "/home/user/PycharmProjects/APP/venv/lib/python3.5/site-packages/pysolr.py", line 421, in _select
return self._send_request('get', path)
File "/home/user/PycharmProjects/APP/venv/lib/python3.5/site-packages/pysolr.py", line 396, in _send_request
raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 404): [Reason: Error 404 Not Found]
But when I try to run the query directly from solr I got results like the following
Can somebody guide me what I am doing wrong here ? Thanks
You can just run the script below to fetch the result without using pysolr library.
#! /usr/bin/python
import urllib
import json as simplejson
import pprint
import sys
url = 'give the url here'
wt = "wt=json"
connection = urllib.urlopen(url)
if wt == "wt=json":
response = simplejson.load(connection)
else:
response = eval(connection.read())
print "Number of hits: " + str(response['response']['numFound'])
pprint.pprint(response['response']['docs'])
I'm coding in python, using version 2.6, working with the Uber API, and when I try to import the library uber_rides.auth it throws this error:
Traceback (most recent call last):
File "C:\Inetpub\vhosts\underdevelopment.biz\httpdocs\web\webtemp3\uber\socket.py", line 4, in <module>
from uber_rides.auth import AuthorizationCodeGrant
File "C:\Inetpub\vhosts\underdevelopment.biz\httpdocs\web\webtemp3\uber\uber_rides\auth.py", line 133
query_params = [qp: query_params[qp][0] for qp in query_params]
^
SyntaxError: invalid syntax
The original code of my script is this:
print('Content-Type: text/plain')
print('')
from uber_rides.auth import AuthorizationCodeGrant
def main():
auth_flow = AuthorizationCodeGrant(
'xxxxxx-xxxxxxx',
'xxxxx-xxxxx',
'xxx-xxxxxxxxx',
'',
)
auth_url = auth_flow.get_authorization_url()
if __name__ == "__main__":
main()
It seems the error is from the library but I can't find it yet.
Yes, that's invalid Python syntax. However, it is not clear how you ended up with that file.
Someone or something altered that file. That's not the original source code as distributed by Uber, where that line uses the correct syntax for a dictionary comprehension:
query_params = {qp: query_params[qp][0] for qp in query_params}
Re-install the project, the error is not there upstream.
Note that the above syntax is only available in Python 2.7 and up. You could try to replace it with a dict() call with a generator expression, see Alternative to dict comprehension prior to Python 2.7:
query_params = dict((qp, query_params[qp][0]) for qp in query_params)
Take into account that there may be other issues with the code, upgrading to Python 2.7 is probably the better option.