I'm trying to integrate ElasticSearch with my Django project using the package django-elasticsearch-dsl, and I am getting this error:
>> $ curl -X GET http://localhost:9200
curl: (7) Failed to connect to localhost port 9200: Connection refused
I downloaded django-elasticsearch-dsl using the commands:
pip install https://github.com/sabricot/django-elasticsearch-dsl/archive/6.4.0.tar.gz
and
pip install django-elasticsearch-dsl, but they both caused the same results.
I don't believe this is a duplicate question because every other question I have read pertaining to this error has dealt with only the ElasticSearch library, not the django-elasticsearch-dsl library. The latter is built on top of the former, but I can't seem to find a elasticsearch.yml file as detailed in all other posts.
Here is what is installed in my virtual environment:
>> pip freeze
Django==2.2.2
django-elasticsearch-dsl==6.4.0
elasticsearch==7.0.2
elasticsearch-dsl==7.0.0
lazy-object-proxy==1.4.1
mccabe==0.6.1
pylint==2.3.1
python-dateutil==2.8.0
pytz==2019.1
requests==2.22.0
typed-ast==1.4.0
urllib3==1.25.3
According to this tutorial, the command http://127.0.0.1:9200 should return what looks like a JSON response, but instead I get the error:
curl: (7) Failed to connect to localhost port 9200: Connection refused
Have you made documents.py for each app? You need to make it to push the data to the elasticsearch database. But firstly you need to install elasticsearch properly(in your case its not installed properly).
Try this Tutorial for installing elasticsearch, i used it recently and it worked like a charm.
Link
And this is the path for elasticsearch.yml, (/etc/elasticsearch/elasticsearch.yml).
And dont forget to start it using - sudo systemctl start elasticsearch(check its status using - sudo systemctl status elasticsearch )
Related
I am trying to run a Python script to insert some data into an Oracle table, from a docker image.
I was given following connection string :
jdbc:oracle:thin:#(DESCRIPTION=(CONNECT_TIMEOUT=3)(RETRY_COUNT=2)(FAILOVER=ON)(LOAD_BALANCE=NO) (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=host_1)(PORT=1521))) (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=host_2)(PORT=1521))) (CONNECT_DATA=(SERVICE_NAME=service_name)))
I'm trying to connect using cx_Oracle package :
try:
# establish a new connection
with cx_Oracle.connect(self.oracle_user,
self.oracle_pwd,
self.oracle_dsn) as connection:
logger.info('ElasticsearchFinder.oracle_write : connexion established with DB')
with oracle_dsn being the connexion string (minus the jdbc:oracle:thin:# part)
I also tried things like
cx_Oracle.connect(self.oracle_user+'/'+self.oracle_pwd+'#'+self.oracle_dsn)
as seen in some examples, but I always get the following timeout error :
cx_Oracle.DatabaseError: ORA-12170: TNS:Connect timeout occurred
telnet host 1521 works fine, I also tried changing the CONNECT_TIMEOUT value.
I also tried
dsn_tns = cx_Oracle.makedsn(self.oracle_host, self.oracle_port, service_name = self.oracle_service_name)
cx_Oracle.connect(self.oracle_user,self.oracle_pwd,dsn_tns)
as suggested here but then I get
cx_Oracle.DatabaseError: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
I am running my script from a Docker image build with Dockerfile :
FROM oraclelinux:7.8
RUN yum -y install oracle-release-el7 && \
yum-config-manager --enable ol7_oracle_instantclient && \
yum -y install oracle-instantclient18.3-basic && \
rm -rf /var/cache/yum
COPY ./fetch_session_iptv.py /opt/
COPY ./conf/fetch_session_iptv.conf /opt/conf/
#COPY ./conf/certs/* /opt/conf/certs/
COPY ./logs /opt/logs
RUN yum install -y \
#https://yum.oracle.com/repo/OracleLinux/OL7/developer/x86_64/getPackage/oracle-instantclient18.3-basic-18.3.0.0.0-2.x86_64.rpm \
https://yum.oracle.com/repo/OracleLinux/OL7/developer/x86_64/getPackage/python-cx_Oracle-7.3-1.el7.x86_64.rpm \
https://yum.oracle.com/repo/OracleLinux/OL7/developer/x86_64/getPackage/python36-pytz-2016.10-2.0.1.el7.noarch.rpm
COPY ./python_requirements/elasticsearch-7.8.0-py2.py3-none-any.whl .
COPY ./python_requirements/certifi-2020.4.5.2-py2.py3-none-any.whl .
COPY ./python_requirements/urllib3-1.25.9-py2.py3-none-any.whl .
RUN pip3 install --user \
certifi-2020.4.5.2-py2.py3-none-any.whl \
urllib3-1.25.9-py2.py3-none-any.whl \
elasticsearch-7.8.0-py2.py3-none-any.whl \
cx_Oracle
RUN sh -c "echo /usr/lib/oracle/18.3/client64/lib > /etc/ld.so.conf.d/oracle-instantclient.conf"
RUN ldconfig
RUN export ORACLE_HOME=/usr/lib/oracle/18.3/client64/
RUN yum -y install telnet
#CMD ["/bin/bash"]
CMD [ "python3", "/opt/fetch_session_iptv.py" ]
I can't understand what's going wrong?
[UPDATE] I've set
ORACLE_HOME=/usr/lib/oracle/18.3/client64/bin
TNS_ADMIN=$ORACLE_HOME/admin
LD_LIBRARY_PATH=/usr/lib/oracle/18.3/client64/lib
PATH=$PATH:$ORACLE_HOME
I've added a tnsnames.ora file in TNS_ADMIN directory, with
CNX=(DESCRIPTION=(CONNECT_TIMEOUT=3)(RETRY_COUNT=2)(FAILOVER=ON)(LOAD_BALANCE=NO) (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=host_1)(PORT=1521))) (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=host_2)(PORT=1521))) (CONNECT_DATA=(SERVICE_NAME=service_name)))
Now, when I try sqlplus MY_USER#CNX I get the same :
ORA-12170: TNS:Connect timeout occured
but at least it seems to accept the connexion string and user. Prior to setting all the environment variables correctly, I only got different TNS error messages about TNS: listener does not currently know of service requested in connect descriptor or the net service name being incorrect.
[UPDATE 2]
I've checked with the person who gave me all connexion info: user, password and connection string are correct. They're running Oracle Database 12c (12.1.0.2.0), which according to this page is compatible with the 18c client I'm using.
I don't know what else could be a possible reason for this timeout error?
Start by using the equivalent connection string in cx_Oracle:
self.oracle_dsn = "(DESCRIPTION=(CONNECT_TIMEOUT=3)(RETRY_COUNT=2)(FAILOVER=ON)(LOAD_BALANCE=NO) (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=host_1)(PORT=1521))) (ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=host_2)(PORT=1521))) (CONNECT_DATA=(SERVICE_NAME=service_name)))"
try:
# establish a new connection
with cx_Oracle.connect(self.oracle_user,
self.oracle_pwd,
self.oracle_dsn) as connection:
logger.info('ElasticsearchFinder.oracle_write : connexion established with DB')
The cx_Oracle manual on connecting and connect strings is here.
Personally I'd use 19c Instant Client, which will connect to the same DB versions as 18c - and doesn't need to have ldconfig run. See Docker for Oracle Database Applications in Node.js and Python.
Update your question with information and I can update this answer likewise.
I am using elastic beanstalk to deploy my Django application. Today it suddenly stopped working without any breaking changes from the application side (I've changed some templates, nothing more).
The deployment time outs after 10 minutes of trying to deploy the app and nothing happens.
The only more or less useful hints I can see in the log is this:
[2020-02-20T15:00:20.437Z] INFO [19057] - [Application update .../postbuild_0_myproject/Command 01_migrate] : Activity execution failed, because: SystemCheckError: System check identified some issues:
ERRORS:
education.Author.photo: (fields.E210) Cannot use ImageField because Pillow is not installed.
HINT: Get Pillow at https://pypi.org/project/Pillow/ or run command "pip install Pillow".
education.Course.cover_image: (fields.E210) Cannot use ImageField because Pillow is not installed.
HINT: Get Pillow at https://pypi.org/project/Pillow/ or run command "pip install Pillow".
education.CourseCategory.icon_image: (fields.E210) Cannot use ImageField because Pillow is not installed.
HINT: Get Pillow at https://pypi.org/project/Pillow/ or run command "pip install Pillow".
Using staging settings
App receivers connected
(ElasticBeanstalk::ExternalInvocationError)
[2020-02-20T15:00:20.437Z] INFO [19057] - [Application update .../postbuild_0_myproject/Command 01_migrate] : Activity failed.
[2020-02-20T15:00:20.437Z] INFO [19057] - [Application update .../postbuild_0_myproject] : Activity failed.
[2020-02-20T15:00:20.437Z] INFO [19057] - [Application update ...] : Activity failed.
[2020-02-20T15:00:20.507Z] INFO [19057] - [Application update app-9a24-200220_145942-stage-200220_145942#142/AppDeployStage0/EbExtensionPostBuild] : Activity failed.
[2020-02-20T15:00:20.507Z] INFO [19057] - [Application update app-9a24-200220_145942-stage-200220_145942#142/AppDeployStage0] : Activity failed.
[2020-02-20T15:00:20.508Z] INFO [19057] - [Application update app-9a24-200220_145942-stage-200220_145942#142] : Completed activity. Result:
Application update - Command CMD-AppDeploy failed
But I already have Pillow in requirements.txt and the log above says:
Requirement already satisfied: Pillow==6.2.1 in /opt/python/run/venv/lib64/python3.6/site-packages (from -r /opt/python/ondeck/app/requirements.txt (line 51))
How can I troubleshoot and fix this? And how can I avoid similar issues in the future? I am really frightened that the same problem may randomly pop out on production environment.
Here's some more info about the configuration:
Here's what I have in .ebextensions:
01_packages.config:
packages:
yum:
git: []
postgresql93-devel: []
db-migrate.config
container_commands:
01_migrate:
command: "django-admin.py migrate"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: myproject.settings
django.config
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: myproject/wsgi.py
wsgi_custom.config
files:
"/etc/httpd/conf.d/wsgihacks.conf":
mode: "000644"
owner: root
group: root
content: |
WSGIPassAuthorization On
This one is a pain and a known issue with Django when using the ImageField model/form. Due to Pythons dynamic import system it will suddenly appear and annoyed the hell out of me when I first came across it.
The way I normally fix this is by using conda and its equivalent of a virtualenv to ensure the right interpreter (the one with my packages) is used.
If you are not using a virtualenv or equivalent, set one up now, if you are already using one then check you are installing pillow with pip3 install pillow - the pip3 being important here as on debian (and many other) systems normal pip will only install for python 2.x.
Using conda will ensure this doesnt happen in production, but I would also add it to your checklist of things to test when deploying - check correct version of pillow setup and working.
I had two Elastic Beanstalk environments with the same issue (one web tier env and a worker env).
On one of them the issue was resolved by restarting the environment.
The other one failed to restart and timed out every time on any operation. This one I managed to fix by going to configuration > capacity and changing the minimum and maximum number of instances to 0. I've applied the changes, waited for them to apply and then returned the previous values for min and max instance numbers.
That fixed the issue.
I still have no idea what caused the issue in the first place and would love to receive some comment on that.
I am trying to run an mlflow server locally.
To do so, I am using:
mlflow server --backend-store-uri="sqlite:///C:\\path\\to\\project_folder\\backend\\mlflow_data.db"
--default-artifact-root="file:///C:\\path\\to\\project_folder\\artifact_store\\"
Where
- backend-store-uri: URI to which to persist experiment and run data (sqlite database in our case).
- default-artifact-root: Local or S3 URI to store artifacts, for new experiments (local folder in our case).
I have already install the packages:
numpy==1.17.3
pandas==0.25.3
jupyterlab==1.0.10
scikit-learn==0.21.3
matplotlib==3.1.2
mlflow==1.4.0
torch==1.3.1+cpu
torchvision==0.4.2+cpu
xgboost==0.90
The problem is that I am getting this error:
Fatal error in launcher: Unable to create process using "d:\bld\mlflow_1572494804636\_h_env\python.exe" "C:\Users\user\AppData\Local\Continuum\miniconda3\envs\mlflow-tut orial\Scripts\mlflow.exe" server --backend-store-uri=sqlite:///C:\\projects\\user\\mlflow_tutorial\\backend\\mlflow_ui_data.db --default-artifact-root=file:///C:\\projects \\user\\mlflow_tutorial\\artifact_store\\
When I run the mlflow server command. Any ideas ?
It seems that there is a space in the path:
I'm trying to run the python heroku tutorial and it won't work in windows. This is from this repository.
I posted this previously but I was able to get a more descriptive error message. It should be said that I've installed postgres.
Furthermore, I can't run it locally using the method defined in the git respository. Both the createdb and foreman commands don't work. This is despite installing foreman.
django.core.exceptions.ImproperlyConfigured: 'django_postgrespool' isn't an available database backend.
Try using 'django.db.backends.XXX', where XXX is one of:
u'base', u'mysql', u'oracle', u'postgresql_psycopg2', u'sqlite3'
Error was: DLL load failed: The specified module could not be found.
Looks like python does not know what django-postgrespool is.
Perhaps it did not install properly. Check the output of pip install -r requirements.txt
DATABASES['default']['ENGINE'] = 'django_postgrespool'
in your settings.py is what this is referring to. For me i still dont know why this is causing a problem. I've install psycopg2 and I even tried installing 'pip install django-postgrespool'. It worked once i commented out:
DATABASES['default'] = dj_database_url.config()
DATABASES['default']['ENGINE'] = 'django_postgrespool'
this helps me run the app locally by using the
heroku local web -f Procfile.windows
I was following the same tutorial and what worked for me was changing the settings.py file, this line:
# Enable Connection Pooling (if desired)
DATABASES['default']['ENGINE'] = 'django_postgrespool'
To this:
# Enable Connection Pooling (if desired)
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
I'm attempting to run stratum-mining-proxy with minerd. Proxy starts and runs with the following command:
python ./mining_proxy.py -o ltc-stratum.kattare.com -p 3333 -pa scrypt
Proxy starts fine. Run Minerd (U/P removed):
minerd -a scrypt -r 1 -s 6 -o http://127.0.0.1:3333 -O USERNAME.1:PASSWORD
Following errors are received. This one from the proxy:
2013-07-18 01:33:59,981 ERROR protocol protocol.dataReceived # Processing of message failed
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/stratum-0.2.12-py2.7.egg/stratum/protocol.py", line 185, in dataReceived
self.lineReceived(line, request_counter)
File "/usr/local/lib/python2.7/dist-packages/stratum-0.2.12-py2.7.egg/stratum/protocol.py", line 216, in lineReceived
raise custom_exceptions.ProtocolException("Cannot decode message '%s'" % line)
'rotocolException: Cannot decode message 'POST / HTTP/1.1
And this from minerd. What am I doing wrong? Any help is appreciated!
[2013-07-18 01:33:59] HTTP request failed: Empty reply from server
[2013-07-18 01:33:59] json_rpc_call failed, retry after 30 seconds
I am a little curious, I don't know as a fact but I was under the impression that the mining proxy was for BTC not LTC.
But anyways I believe I got a similar message when I first installed it as well. To fix, or rather to actually get it running I had to use the Git installation method instead of installing manually.
Installation on Linux using Git
This is advanced option for experienced users, but give you the easiest way for updating the proxy.
1.git clone git://github.com/slush0/stratum-mining-proxy.git
2.cd stratum-mining-proxy
3.sudo apt-get install python-dev # Development package of Python are necessary
4.sudo python distribute_setup.py # This will upgrade setuptools package
5.sudo python setup.py develop # This will install required dependencies (namely Twisted and Stratum libraries), but don't install the package into the system.
6.You can start the proxy by typing "./mining_proxy.py" in the terminal window. Using default settings, proxy connects to Slush's pool interface.
7.If you want to connect to another pool or change other proxy settings, type "./mining_proxy.py --help".
8.If you want to update the proxy, type "git pull" in the package directory.