I'm new to Apache Solr. I get an error when I add the documents using pysolr package in python.
I have tried following:
from __future__ import print_function
import pysolr
solr = pysolr.Solr('http://localhost:8983', timeout=10)
solr.add([
{
"id": "doc_1",
"title": "A very small test document about elmo",
}
])
I get the following error:
Traceback (most recent call last):
File "ex1.py", line 16, in <module>
{ "id": "child_doc_2", "title": "seed"},
File "/home/system/anaconda3/lib/python3.6/site-packages/pysolr.py", line 918, in add
overwrite=overwrite, handler=handler)
File "/home/system/anaconda3/lib/python3.6/site-packages/pysolr.py", line 500, in _update
return self._send_request('post', path, message, {'Content-type': 'text/xml; charset=utf-8'})
File "/home/system/anaconda3/lib/python3.6/site-packages/pysolr.py", line 412, in _send_request
raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 404): [Reason: Error 404 Not Found]
The name of your core or collection should be part of the URL, as well as the /solr path prefix:
solr = pysolr.Solr('http://localhost:8983/solr/<collectionname>', timeout=10)
Related
I wrote some code that logs in with 2fa and a proxy and then gets some info and returns it. It is not finished yet but the login section will not work. Here is the code, if someone could tell me what is wrong I would greatly appreciate it. As a side note I am using a socks5 ipv6 proxy, not sure if it must be ipv4 but I couldn't find an answer. Obviously I have removed my credentials from the code but they are put where it is indicated.
Here is my code:
from instagrapi import Client
import uuid
client_session_id = str(uuid.uuid4())
settings = {
"uuids": {
"phone_id": "be60e0a9-7201-5492-da04-4f2b85b3b592",
"uuid": "be60e0a9-7201-5492-da04-4f2b85b3b592",
"client_session_id": client_session_id,
"advertising_id": "1c413050-795d-f321-eeb4-4962ca0a731c",
"device_id": "android-f4f80570fe070b5f"
},
"cookies": {},
"last_login": 1674240519.7862692,
"device_settings": {
"cpu": "h1",
"dpi": "640dpi",
"model": "C6503",
"device": "C6503",
"resolution": "1794x1080",
"app_version": "117.0.0.28.123",
"manufacturer": "Sony",
"version_code": "164321834",
"android_release": "4.2.2",
"android_version": 23
},
"user_agent": "Opera/9.80 (Android 4.2.2; Linux; Opera Mobi/ADR-1210241554; U; en-us) Presto/2.11.355 Version/12.10"
}
cl = Client(settings)
proxy = "socks5://username:password#127.0.0.1:30235"
cl.set_proxy(proxy)
seed = 'PUT SEED HERE'
code = cl.totp_generate_code(seed)
cl.login('USERNAME', 'PASSWORD', verification_code=code)
user_id = cl.user_id_from_username("instagram")
followers = cl.user_followers(user_id, 0)
print(followers)
cl.logout()
When I run it I get this error:
Traceback (most recent call last):
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\adapters.py", line 456, in send
conn = self.get_connection(request.url, proxies)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\adapters.py", line 345, in get_connection
proxy = prepend_scheme_if_needed(proxy, "http")
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\utils.py", line 988, in prepend_scheme_if_needed
parsed = parse_url(url)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\urllib3\util\url.py", line 397, in parse_url
return six.raise_from(LocationParseError(source_url), None)
File "<string>", line 3, in raise_from
urllib3.exceptions.LocationParseError: Failed to parse: socks5://socks5://username:password#127.0.0.1:30235
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\NAME\Desktop\learning\app.py", line 41, in <module>
cl.login('USERNAME', 'PASSWORD', verification_code=code)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\instagrapi\mixins\auth.py", line 371, in login
self.pre_login_flow()
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\instagrapi\mixins\auth.py", line 47,
in pre_login_flow
self.set_contact_point_prefill("prefill")
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\instagrapi\mixins\auth.py", line 147, in set_contact_point_prefill
return self.private_request("accounts/contact_point_prefill/", data, login=True)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\instagrapi\mixins\private.py", line 451, in private_request
raise e
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\instagrapi\mixins\private.py", line 438, in private_request
self._send_private_request(endpoint, **kwargs)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\instagrapi\mixins\private.py", line 288, in _send_private_request
response = self.private.post(
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 635, in post
return self.request("POST", url, data=data, json=json, **kwargs)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Users\NAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\adapters.py", line 458, in send
raise InvalidURL(e, request=request)
requests.exceptions.InvalidURL: Failed to parse: socks5://username:password#127.0.0.1:30235
This is source code of set_proxy function:
def set_proxy(self, dsn: str):
if dsn:
assert isinstance(
dsn, str
), f'Proxy must been string (URL), but now "{dsn}" ({type(dsn)})'
self.proxy = dsn
proxy_href = "{scheme}{href}".format(
scheme="http://" if not urlparse(self.proxy).scheme else "",
href=self.proxy,
)
self.public.proxies = self.private.proxies = {
"http": proxy_href,
"https": proxy_href,
}
return True
self.public.proxies = self.private.proxies = {}
return False
as you can see, it seems your private proxy schema string has a problem. Check your proxy string, it should be same with: How to make python Requests work via SOCKS proxy
I am trying to learn firebase to build apps more easily, but I've been having slight issues with adding databases to the server.
Here is the code:
import pyrebase as pb
fbconfig = {"apiKey": "AIzaSyCy5R9rbQWt4AzBo5sZeGBKcoyfHZ1fD-s",
"authDomain": "fir-fb8e9.firebaseapp.com",
"databaseURL": "https://fir-fb8e9.firebaseio.com",
"projectId": "fir-fb8e9",
"storageBucket": "fir-fb8e9.appspot.com",
"messagingSenderId": "816508440796",
"appId": "1:816508440796:web:e03ef3c4810606f8faa419",
"measurementId": "G-0MWJ5YES8M"}
fb = pb.initialize_app(fbconfig)
db = fb.database()
d1={"age":16,"Name":"Ebraheem Alrawi","Group":"For Fun"}
db.push(d1)
I tried changing the databaseURL links over and over again but it would give me the same error
Traceback (most recent call last):
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\pyrebase\pyrebase.py", line 503, in raise_detailed_error
request_object.raise_for_status()
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\requests\models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://fir-fb8e9.firebaseio.com/.json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/Test/PycharmProjects/speedplate/firebasetest/main.py", line 34, in <module>
db.push(d1)
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\pyrebase\pyrebase.py", line 310, in push
raise_detailed_error(request_object)
File "C:\Users\Test\PycharmProjects\speedplate\venv\lib\site-packages\pyrebase\pyrebase.py", line 507, in raise_detailed_error
raise HTTPError(e, request_object.text)
requests.exceptions.HTTPError: [Errno 404 Client Error: Not Found for url: https://fir-fb8e9.firebaseio.com/.json] {
"error" : "404 Not Found"
}
Process finished with exit code 1
I tried searching in google but there were no results
According to the Pyrebase documentation found https://github.com/thisbejim/Pyrebase#database
Create a database reference location.
db = firebase.database()
db.child("users").child("Morty")
Save data with an auto-generated key.
data = {"name": "Mortimer 'Morty' Smith"}
db.child("users").push(data)
Define your own keys use the set() method.
data = {"name": "Mortimer 'Morty' Smith"}
db.child("users").child("Morty").set(data)
Update data for an existing entry.
db.child("users").child("Morty").update({"name": "Mortiest Morty"})
I'm trying to interface Firebase with the official Admin SDK for Python (https://firebase.google.com/docs/database/admin/start).
However, I'm doing something wrong, as I'm not authorized somehow
This is my code:
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
# Fetch the service account key JSON file contents
cred = credentials.Certificate('./ServiceAccountKey.json')
# Initialize the app with a None auth variable, limiting the server's access
firebase_admin.initialize_app(cred, {
'databaseURL': 'https://*[database name]*.firebaseio.com',
'databaseAuthVariableOverride': None
})
# The app only has access to public data as defined in the Security Rules
ref = db.reference('/public_resource')
print(ref.get())
This is the error I get:
python3 firebase_test.py
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/firebase_admin/db.py", line 943, in request
return super(_Client, self).request(method, url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/firebase_admin/_http_client.py", line 117, in request
resp.raise_for_status()
File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://*[database name]*.firebaseio.com/public_resource.json?auth_variable_override=null
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "firebase_test.py", line 16, in <module>
print(ref.get())
File "/usr/local/lib/python3.6/site-packages/firebase_admin/db.py", line 222, in get
return self._client.body('get', self._add_suffix(), params=params)
File "/usr/local/lib/python3.6/site-packages/firebase_admin/_http_client.py", line 129, in body
resp = self.request(method, url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/firebase_admin/db.py", line 945, in request
raise _Client.handle_rtdb_error(error)
firebase_admin.exceptions.UnauthenticatedError: Unauthorized request.
I'm using a Raspberry Pi 4 B with installed Python 3.6.
Can somebody point me into the right direction of why this happens and how to fix it?
Found it out!
Under database, go to rules and change it to:
{
/* Visit https://firebase.google.com/docs/database/security to learn more about security rules. */
"rules": {
".read": true,
".write": true
}
}
It was automatically set to false after I've waited for a too long period since my last access.
I am connected to remote ElasticSearch server and trying to get data from it but not able to solve SerializationError. it will be great if anyone can help me. any other approach using Python is appreciated
from elasticsearch import Elasticsearch, RequestsHttpConnection
import certifi
es = Elasticsearch( ['23t4t273545273547288fr8t83rtf8.us-east-1.aws.found.io'],
https_auth=('userid', 'pwd'),
connection_class= (RequestsHttpConnection),
port=9200,
use_ssl=True,
verify_certs=True,
ca_certs=certifi.where(),
)
if es.ping():
print('Yay Connect')
else:
print('Awww it could not connect!')
search_param = {
"query": {
"terms": {
"contentType": "criteria"
}
}
}
response = es.search(index="content-page-rwr-v1", body=search_param)
print ('response:', response)
I am getting following error in python
Yay Connect
Traceback (most recent call last):
File "Q:/MIS-DEV/QA/Pranit/PyCharm Projects/ALL Orig Scripts/ES.py", line 175, in <module>
response = es.search(index="content-page-rwr-v1", body=search_param)
File "C:\Python27\Env\lib\site-packages\elasticsearch\client\utils.py", line 76, in _wrapped
return func(*args, params=params, **kwargs)
File "C:\Python27\Env\lib\site-packages\elasticsearch\client\__init__.py", line 636, in search
doc_type, '_search'), params=params, body=body)
File "C:\Python27\Env\lib\site-packages\elasticsearch\transport.py", line 345, in perform_request
data = self.deserializer.loads(data, headers_response.get('content-type'))
File "C:\Python27\Env\lib\site-packages\elasticsearch\serializer.py", line 79, in loads
raise SerializationError('Unknown mimetype, unable to deserialize: %s' % mimetype)
elasticsearch.exceptions.SerializationError: Unknown mimetype, unable to deserialize: text/html
Thank you in advance
I believe what is happening is that ES < 6.0 expects the content-type to be explicitly set in the header (specifically, application/json). You can modify the connection class to do this as suggested in this github issue
I'm trying to build an app using the spotipy python library to access the spotify api.
My oauth code looks like this and it seems to work except for initialising the client with the right auth parameter.
self.sp_auth=spotipy.oauth2.SpotifyOAuth(secrets.sp_auth_id,
secrets.sp_auth_pw, secrets.sp_callback_url,
scope="playlist-modify-public user-library-read", state=state)
...
url = self.sp_auth.get_authorize_url()
send url to user.
after user said she/he has given permission:
auth code is fetched from webserver and used to generate a token.
self.auth_token=self.sp_auth.get_access_token(self.auth_code)
self.auth_token then looks like this:
{'access_token' : 'BQD ... qE7K3PBZKB6iZFU3_4p',
'token_type' : 'Bearer',
'expires_in' : 3600,
'refresh_token' : 'AQCOS2Xo ... MK09ry7-a-fl61OwhuO1Q',
'scope' : 'playlist-modify-public user-library-read',
'expires_at' : 1548247835}
then I initialize the spotipy client module like this:
self.sp = spotipy.Spotify(auth=self.auth_token)
then I try the following:
playlists = self.sp.current_user_playlists(limit=10)
which raises this Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 119, in _internal_call
r.raise_for_status()
File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.spotify.com/v1/me/playlists?limit=10&offset=0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/telegram/ext/dispatcher.py", line 279, in process_update
handler.handle_update(update, self)
File "/usr/local/lib/python3.6/dist-packages/telegram/ext/callbackqueryhandler.py", line 143, in handle_update
return self.callback(dispatcher.bot, update, **optional_args)
File "spotify_playlist_bot_v2.py", line 140, in button_auth_done
User.data[user_id].msg_start(bot, update)
File "spotify_playlist_bot_v2.py", line 84, in msg_start
self.msg_choose_playlist()
File "spotify_playlist_bot_v2.py", line 90, in msg_choose_playlist
playlists = self.sp.current_user_playlists(limit=10)
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 355, in current_user_playlists
return self._get("me/playlists", limit=limit, offset=offset)
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 146, in _get
return self._internal_call('GET', url, payload, kwargs)
File "/usr/local/lib/python3.6/dist-packages/spotipy/client.py", line 124, in _internal_call
headers=r.headers)
spotipy.client.SpotifyException: http status: 400, code:-1 - https://api.spotify.com/v1/me/playlists?limit=10&offset=0:
Only valid bearer authentication supported
It looks like I'm not passing the token to the spotipy client correctly. For example self.sp = spotipy.Spotify(auth="random_bullshit") gives me the same Exception. I also tried passing the token like this auth=self.auth_token['access_token'] with the same result. The documentation doesn't say anything about what the auth parameter should be exactly and I'm not really understanding the source code. But I'd say it suggests that auth=self.auth_token['access_token'] is the right thing to do.
Thanks!
As I already suggested in my last edit auth=self.auth_token['access_token'] was the right thing I just had a typo in it. Anyway since the Documentation doesn't say a lot about the auth parameter this might help some people.