Executing sparql query from python to virtuoso server in linux? - python

I am having problem for running the following program (sparql_test.py). I am running it from Linux machine. I am installing Virtuoso server in the same Linux machine. In the Linux server, I don't have sudo permission nor browser access. But, I can execute SPARQL query from isql prompt (SQL>) successfully.
Program: sparql_test.py
from SPARQLWrapper import SPARQLWrapper, JSON
sparql = SPARQLWrapper("http://localhost:8890/sparql")
sparql.setQuery("select ?s where { ?s a <http://ehrofip.com/data/Admissions>.} limit 10")
sparql.setReturnFormat(JSON)
result = sparql.query().convert()
for res in result["results"]["bindings"]:
print(res)
I got the following error:
[suresh#deodar complex2vec]$ python sparql_test.py
Traceback (most recent call last):
File "sparql1.py", line 14, in "<module>"
result = sparql.query().convert()
File "/home/suresh/.local/lib/python2.7/site-packages/SPARQLWrapper/Wrapper.py", line 687, in query
return QueryResult(self._query())
File "/home/suresh/.local/lib/python2.7/site-packages/SPARQLWrapper/Wrapper.py", line 667, in _query
raise e
urllib2.HTTPError: HTTP Error 502: Bad Gateway
However, the above program run smoothly in my own laptop. What might be the problem? Is this issue of connection?
Thank you
Best,
Suresh

I do not believe this error is raised by Virtuoso. I believe it is raised by SPARQLWrapper.
It looks like there's something between the outside world (which includes the Linux machine itself) and the Virtuoso listener on port 8890. The "Bad Gateway" suggests there may be two things -- a reverse proxy, and a firewall.
Port 8890 (set as [HttpServer]:Listen in the INI file) must be open to communications, direct or proxied, for SPARQL access to work.
iSQL talks to port 1111 (set as [Parameters]:Listen in the INI file), which apparently doesn't have a similar block/proxy.

Related

Connect from a cisco device to http server on debian

I'm trying to communicate with a http server which is running on debian strech from a brand new out of the box cisco device. Now, the so called zero touch configuration is no problem:
The Switch gets an IP address and such via DHCP and a link to where to fetch it's initial configuration.
The switch gets its basic configuration such as user credentials etc.
The problem rises when I try to search through a database on the server from the switch. In this database some variables are stored. Depending the serialnumber of the switch, it should receive a specific hostname, Mgmt address etc.
On those new switches there is a python module integrated so I ran some tests. I tried to fetch the serial number and got them whithout any problems. The moment I tried to write the serial number on a txt file on the server I got this error
Traceback (most recent call last): File "", line 1, in
IOError: [Errno 2] No such file or directory:
'http://10.232.152.19:80/temp.txt'
Code so far:
from cli import cli
def get_serial():
serial = cli("show version | include System Serial\n")
serial = (serial.split()[-1])
f = open ("http://10.232.152.19:80/temp.txt", "a")
f.write(serial)
f.close
get_serial()
The problem you are facing is because you are trying to open a file from the network. You need to download the file first in you system and then open it. You should use urllib to fetch the file and then open it. then save it and again push it back.
import urllib
txt = urllib.urlopen(target_url).read()

Cannot acquire connection to Neo4j database

I am trying to connect to my Neo4j graph database server from a new machine. I can successfully connect from an older machine but do not wish to use the older one anymore.
I have reduced the problem to a simple script that returns an exception:
from neo4j.v1 import GraphDatabase, basic_auth
auth = basic_auth("username","password")
session = GraphDatabase.driver("bolt://remote.server:7687",auth=auth).session()
statement = """MATCH (a:Protein)
WHERE a.name={name}
RETURN a.Accession"""
tx = session.begin_transaction()
record = tx.run(statement,{'name':"ARCH_HUMAN"}).single()
print record['a.Accession']
session.close()
And the error message is:
File "Test.py", line 10, in <module>
tx = session.begin_transaction()
File "/home/username/anaconda2/lib/python2.7/site-packages/neo4j/v1/api.py", line 432, in begin_transaction
self._connect()
File "/home/username/anaconda2/lib/python2.7/site-packages/neo4j/v1/api.py", line 269, in _connect
self._connection = self._acquirer(access_mode)
File "/home/username/anaconda2/lib/python2.7/site-packages/neo4j/v1/direct.py", line 52, in acquire
raise ServiceUnavailable("Cannot acquire connection to {!r}".format(self.address))
neo4j.exceptions.ServiceUnavailable: Cannot acquire connection to Address(host='remote.server', port=7687)
Port 7687 is open (confirmed via netstat -tulpn and iptables -L), and neo4j is configured to listen to 0.0.0.0:7687. In addition, .neo4j/known_hosts contains an entry for host 0.0.0.0
What's strange is that I get a different error message (neo4j.exceptions.AuthError) if I break the authentication by using an incorrect password. So the connection is being made to check the password, but still I cannot connect with the correct auth.
What's going on?
I too had the same issue and turns out the driver was the issue.
I did some experiments and found out that the last driver that it works for is neo4j-driver==v1.1.0 but the next version neo4j-driver==v1.2.0 it stops working for some reason.
Try uncomment dbms.connectors.default_listen_address=0.0.0.0 And check this
# Bolt connector
dbms.connector.bolt.enabled=true
dbms.connector.bolt.tls_level=OPTIONAL
dbms.connector.bolt.listen_address=:7687
# HTTP Connector. There must be exactly one HTTP connector.
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=:7474
# HTTPS Connector. There can be zero or one HTTPS connectors.
dbms.connector.https.enabled=true
dbms.connector.https.listen_address=:7473

Unable to connect to ldaps on windows server 2016 from python-ldap

I have successfully used python-ldap to connect to a windows 2012 R2 server over ldaps in the past. The procedure I used for this was as follows:
python code:
import ldap
ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
ldap.set_option(ldap.OPT_DEBUG_LEVEL, 255)
ip = '<redacted>'
url = "%s://%s:%d" % ('ldaps', ip, 636)
ld = ldap.initialize(url)
ld.protocol_version = 3
ld.set_option(ldap.OPT_REFERRALS, ldap.OPT_OFF)
user = '<redacted>'
passwd = '<redacted>'
ld.simple_bind_s('<redacted>\%s' % user, passwd)
And on the windows server, I used the 'server manager' to add a 'AD CS' role, and created a root certificate. I do not care about verifying the certificate, just using some encryption. After creating the root certificate, LDAPS was enabled on the server, and this code runs without error.
Now, I have followed the exact same procedure on windows server 2016, and the results are not so nice. I have managed to get a few errors from the same script. Usually either 'A TLS packet with unexpected length was received.' or 'Error in the push function.'. I have searched for a few hours but I have not been able to find a solution. Does anyone know if extra steps are needed for configuration on the windows server, or if something about my script is incorrect?
The client I am testing with is using python 2.7 and ubuntu 14.04. pip2.7 has updated the python-ldap library to the latest version. Lere is an example of the failed script run:
ldap_create
ldap_url_parse_ext(ldaps://<redacted>:636)
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP <redacted>:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying <redacted>:636
ldap_pvt_connect: fd: 3 tm: -1 async: 0
TLS: can't connect: Error in the push function..
ldap_err2string
Traceback (most recent call last):
File "test_ldap.py", line 13, in <module>
ld.simple_bind_s('<redacted>\%s' % user, passwd)
File "/usr/local/lib/python2.7/dist-packages/ldap/ldapobject.py", line 228, in simple_bind_s
msgid = self.simple_bind(who,cred,serverctrls,clientctrls)
File "/usr/local/lib/python2.7/dist-packages/ldap/ldapobject.py", line 222, in simple_bind
return self._ldap_call(self._l.simple_bind,who,cred,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls))
File "/usr/local/lib/python2.7/dist-packages/ldap/ldapobject.py", line 108, in _ldap_call
result = func(*args,**kwargs)
ldap.SERVER_DOWN: {'info': 'Error in the push function.', 'errno': 104, 'desc': "Can't contact LDAP server"}
So it seems that this is one of those windows things which I just do not understand. After coming into work on the next day, the same code above just started working. It seems that windows server may require many hours in order to allow LDAPS to become available to connect to.

Python telnetlib client doesn't appear to be opening telnet connection to server

I am trying to open a telnet connection, write one string, then print everything from the telnet server in Python. I assume I am missing something obvious because the documentation seems pretty self-explanatory and doing what I think is the exact same thing in terminal works fine.
Here is the Python code:
import telnetlib
telnet = telnetlib.Telnet()
telnet.open('192.168.1.128', 9801, 10)
telnet.write("SYSTEM_CAL")
print(telnet.read_all())
This times out after 10 seconds / doesn't successfully connect to the server I assume. Here is the output:
Traceback (most recent call last):
File "/Volumes/Work/Scripting/Telnet Test/main.py", line 9, in <module>
print(telnet.read_all())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/telnetlib.py", line 385, in read_all
self.fill_rawq()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/telnetlib.py", line 576, in fill_rawq
buf = self.sock.recv(50)
socket.timeout: timed out
In the image below I connect to the same server in terminal and everything works fine. At the end I give the "DISCONNECT" command taken by the server which is why the connection closes.
Am I missing something? Why is this working in Terminal and not in Python?
I suppose you are sending your command too early, and the receiving end is ignoring it. Try to read the banner first:
print(telnet.read_until("Welcome."))
telnet.write("SYSTEM_CAL")
print(telnet.read_all())
EDIT:
I also note that you don't have a line-terminating sequence at the end of "SYSTEM_CAL". I presume that, in the interactive session, you press ↵ ENTER after you type "SYSTEM_CAL". If that is the case, you'll need to add \n at the end of the write() call:
telnet.write("SYSTEM_CAL\n")
Depending upon other factors outside of your question, it is possible that you may need one of the following instead:
telnet.write("SYSTEM_CAL\r")
telnet.write("SYSTEM_CAL\r\n")

How to run localtunnel v2 properly

I'm using localtunnel v1. But I found that v2 allows you to customize the subdomain, and I need this feature.
I followed the tutorial described in the README from the repository, but it confused me in several parts and, in the end, it did not work.
First step is to run some web-app: checked, on port no. 8000.
Then, it says something about hostnames:
Localtunnel does some stuff with the hostname, so you want to set up two
hostnames. One for localtunnel registration, one for your localtunnel.
Normally it expects a wildcard, but we'll just hardcode a hostname for
this example tunnel.
example.localtunnel.local -> 127.0.0.1
localtunnel.local -> 127.0.0.1
You can do this in /etc/hosts or use that fancy ghost utility.
I've got lost here, but still I edited my /etc/hosts:
127.0.0.1 localhost
127.0.1.1 my-pc-name
127.0.0.1 example.localtunnel.local
127.0.0.1 localtunnel.local
Next step...
Now you can start the server. It's based on a configuration file in the
config directory. You can make your own, but this one is configured to
run the server on port 9999 and expects the hostname localtunnel.local
ginkgo config/default.conf.py
Which one? Anyway... I created myconfig.conf.py based on the files in localtunnel repo's dir /deploy:
port = 9999
hostname = 'localtunnel.local'
service = 'localtunnel.server.TunnelBroker'
But, when I run:
lt --broker 127.0.0.1:9999 --name example 8000
I got:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 390, in run
result = self._run(*self.args, **self.kwargs)
File "/usr/local/lib/python2.7/dist-packages/localtunnel/client.py", line 53, in listen
msg = self.ws.receive(msg_obj=True)
TypeError: receive() got an unexpected keyword argument 'msg_obj'
<Greenlet at 0xb6e0db1cL: <bound method TunnelClient.listen of <localtunnel.client.TunnelClient object at 0xb6def52c>>> failed with TypeError
And in the ginkgo process:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent/pywsgi.py", line 438, in handle_one_response
self.run_application()
File "/usr/local/lib/python2.7/dist-packages/ws4py/server/geventserver.py", line 85, in run_application
self.result = self.application(self.environ, start_response_for_upgrade)
File "/usr/local/lib/python2.7/dist-packages/ws4py/server/wsgi/middleware.py", line 131, in __call__
environ.copy()))
TypeError: handle_websocket() takes exactly 3 arguments (2 given)
<BrokerFrontend fileno=6 address=0.0.0.0:9999>: Failed to handle request:
request = GET /t/example HTTP/1.1 from ('127.0.0.1', 35907)
application = <ws4py.server.wsgi.middleware.WebSocketUpgradeMiddleware object at 0x95bc2ac>
127.0.0.1 - - [2012-05-14 17:18:18] "GET /t/example HTTP/1.1" 101 162 0.000933
And, obviously, http://example.localtunnel.local:9999 does not work.
How to fix this? And where I have to modify to change the final subdomain?
Sorry about the creepy english.
Edit
I've followed the paul suggestion and did the downgrading. But although changes have happened, errors still occur. ginkgo process:
$ ginkgo eco.conf.py
Starting process with eco.conf.py...
127.0.0.1 - - [2012-05-22 20:21:11] "GET /t/example HTTP/1.1" 400 116 0.000190
localtunnel process:
$ lt --broker 127.0.0.1:9999 --name example 8000
Traceback (most recent call last):
File "/usr/local/bin/lt", line 9, in <module>
load_entry_point('localtunnel==0.4.0', 'console_scripts', 'lt')()
File "/usr/local/lib/python2.7/dist-packages/localtunnel/client.py", line 31, in main
client.serve_forever()
File "/usr/local/lib/python2.7/dist-packages/ginkgo/core.py", line 188, in serve_forever
self.start()
File "/usr/local/lib/python2.7/dist-packages/ginkgo/core.py", line 124, in start
ready = not self.do_start()
File "/usr/local/lib/python2.7/dist-packages/localtunnel/client.py", line 42, in do_start
self.ws.connect()
File "/usr/local/lib/python2.7/dist-packages/ws4py-0.1.5-py2.7.egg/ws4py/client/threadedclient.py", line 72, in connect
self.process_response_line(response_line)
File "/usr/local/lib/python2.7/dist-packages/ws4py-0.1.5-py2.7.egg/ws4py/client/__init__.py", line 61, in process_response_line
raise HandshakeError("Invalid response status: %s %s" % (code, status))
ws4py.exc.HandshakeError: Invalid response status: 400 Bad Handshake
Although ginkgo does not give any error now, localtunnel still raising errors different from previous errors. Apparently it tries to GET "/t/example" in the connecting process.
It looks like this software is expecting an older version of ws4py. The current version (0.2.1) of ws4py matches what it looks like you have, while the 0.1.5 version of ws4py matches what localtunnel is trying to use.
Downgrading to ws4py 0.1.5 may be sufficient to solve the problems you're having.
On the other hand, though, this doesn't seem like the best-supported software in the world. Are you sure it's the right solution for your problem? I've looked through the code and all the docs provided in that repo, and what I get is that it sets up this weird tcp-tunnel-over-json-over-websockets (yeah, websockets, for something with python on both the server and client side!) thing, without even providing any particular security or encryption or robustness capabilities of its own, and it appears to do nothing that other more common tools can't do better. But granted, I may be missing something important.
I think you must be following the instructions to set up the localtunnel.com server (ie. if you wanted to run your own localtunnel server on some other domain).
Installing localtunnel v2 for normal use should be as easy as running pip install localtunnel (possibly with sudo).
Once that's done, just run localtunnel-beta -n <subdomain> 8000
For more details, see Jeff's blog post.

Categories

Resources