I'm trying to make a query to get the predicate or the relation between 2 entities already exist in Arabic DBpedia...
I'm trying to do that in Python using the SPARQL Endpoint interface to Python (SPARQLWrapper), so I set the Data Set Name, with the query like that:
sparql = SPARQLWrapper("http://ar.dbpedia.org/sparql")
sparql.setReturnFormat(JSON)
property = []
query = "SELECT ?property WHERE {{ <{}> ?property <{}> }}".format('http://ar.dbpedia.org/resource/فرنسا', 'http://ar.dbpedia.org/resource/باريس')
sparql.setQuery(query)
string_s = sparql.query().convert()
if len(string_s['results']['bindings']) != 0:
bindings = string_s['results']['bindings']
for b in bindings:
property.append(b['property']['value'])
print(property)
The problem is when I specified the Data set name as (http://ar.dbpedia.org/sparql), it gave me an error in connection like that:
ConnectionResetError: [WinError 10054] An existing connection was
forcibly closed by the remote host
And when I change it to the Default one (http://dbpedia.org/sparql), it gives me no relation between the 2 entities (and I guess this is because I am making a request on the data set that's not having the two entities!), I test the previous code on English resources by changing the data set name to the default one, and changing the query to this:
query = "SELECT ?property WHERE {{ <{}> ?property <{}> }}".format('http://dbpedia.org/resource/France', 'http://dbpedia.org/resource/Paris')
it worked and gave me the (?property) like this:
['http://dbpedia.org/ontology/wikiPageWikiLink',
'http://dbpedia.org/ontology/capital']
So my question is, How could I do the same request and get the same answer on the Arabic DBpedia? How can I inquire about the property that links between the 2 Arabic resources (Arabic entities) that already exist in Arabic DBpedia?
Related
As a test I have been trying to load Ensembl gene-transcript-protein data through a python program into OrientDB (logged in as root). Nodes work fine,(transcript to protein) edges load tens of thousands of edges fine, but then give an error message like
Invalid authentication info for access to the database com.orientechnologies.orient.core.metadata.security.auth.OTokenAuthInfo#
hInfo#3b531bc6 etc
for all remaining edges
Terminal output:
.....
DB name="ensembl"
228190
com.orientechnologies.orient.core.exception.OSecurityAccessException - Invalid authentication info for access to the database com.orientechnologies.orient.core.metadata.security.auth.OTokenAuthInfo#48703c3d
DB name="ensembl"
228191
com.orientechnologies.orient.core.exception.OSecurityAccessException - Invalid authentication info for access to the database com.orientechnologies.orient.core.metadata.security.auth.OTokenAuthInfo#1c80d217
DB name="ensembl"
228192
Debug console (vscode) and inspectin of the input data show nothing unexpected
relevant section of the code:
enter code here
Create the tr2prot edge
tr2protTable=pandas.read_table(dataPath + "tr2prot.txt",sep=',')
(s1,s2)=tr2protTable.shape
for i1 in range(s1):
if i1%1000 == 0: print(i1)
row=tr2protTable.iloc[i1]
uIDfrom=(row['TranscriptStableID']).replace("'","\'")
uIDto = (row['ProteinStableID']).replace("'","\'")
commandStr=f"create edge Tr2prot from (select from Transcript where uID= '{uIDfrom}') to (select from Protein where uID = '{uIDto}')"
tr2protList.append(commandStr)
try:
tr2prot = client.command(commandStr)
except Exception as e:
print(e)
print(i1)
In case its relevant, I tried to set the session token as indicated in the manual
client.connect("root", root-password)
client.set_session_token(True)
import pyorient
client = pyorient.OrientDB("localhost", 2424) # host, port
# Reattach to Session
if sessionToken != '':
client.set_session_token(sessionToken)
else:
# Open Database
client.db_drop("ensembl")
client.db_open("ensembl", "root", root-password)
# Fetch Session Token
sessionToken = client.get_session_token()
o
If I generate a batch submission, I do not seem to have the problem, but errors, when, for some reason, the vertex is not there. Is there something like the neo4j apoc.load.csv switch failOnError:false in OrientDB?
Would also help.
Any suggestions would be great
Hans
I created a web service in Flask over uwsgi. I thought I would follow good practice and create a SessionPool with 20 connections to be safe. Each call to a web service endpoint, I acquire a connection from the pool, and at the end I release it.
When using Locust to swarm test the API, I was getting hundreds of failures, nearly 100% on some of the longer responses (30Mb JSON response). Smaller payloads were much better, but with intermittent failures.
The minute I switched back to bad practice and created a brand new connection and cursor within the method itself, all my problems vanished. 100% success on 1000s of stress test calls.
My errors were varied. TNS Bad Packet, incorrect number of connections from pool, request cancelled by user....you name it, it was there.
So I can't use Oracle connection pooling with flask it seems, or have a single connection at the Flask application level (this generated errors, not sure why, which is why I switched to connection pooling).
Any advice on creating scalable apps using cx_Oracle in flask.
My original code was:
pool = cx_Oracle.SessionPool("user", "password", "myserver.company.net:1521/myservice", min=10, max=10, increment=0, getmode=cx_Oracle.SPOOL_ATTRVAL_WAIT, encoding="UTF-8")
def read_products_search(search=None):
"""
This function responds to a request for /api/products
with the complete lists of people
:return: json string of list of people
"""
conn_ariel = pool.acquire()
cursor_ariel = conn_ariel.cursor()
search=search.lower()
print("product search term is: ", search)
# Create the list of products from our data
sql = """
SELECT DRUG_PRODUCT_ID, PREFERRED_TRADE_NAME, PRODUCT_LINE, PRODUCT_TYPE, FLAG_PASSIVE, PRODUCT_NUMBER
FROM DIM_DRUG_PRODUCT
WHERE lower(PREFERRED_TRADE_NAME) LIKE '%' || :search1 || '%' or lower(PRODUCT_LINE) LIKE '%' || :search2 || '%' or lower(PRODUCT_NUMBER) LIKE '%' || :search3 || '%'
ORDER BY PREFERRED_TRADE_NAME ASC
"""
cursor_ariel.execute(sql, {"search1":search,"search2":search, "search3":search })
products = []
for row in cursor_ariel.fetchall():
r = reg(cursor_ariel, row, False)
product = {
"drug_product_id" : r.DRUG_PRODUCT_ID,
"preferred_trade_name" : r.PREFERRED_TRADE_NAME,
"product_line" : r.PRODUCT_LINE,
"product_type" : r.PRODUCT_TYPE,
"flag_passive" : r.FLAG_PASSIVE,
"product_number" : r.PRODUCT_NUMBER
}
# logging.info("Adding Product: %r", product)
products.append(product)
if len(products) == 0:
products = None
pool.release(conn_ariel)
return products
When you create the pool, use threaded=True.
See How to use Python Flask with Oracle Database.
at the moment I am working with the Azure Cloud. I want to set up an AzureSQL database and use AlwaysEncrypted to ensure that the data is 'always encrypted' ;-). Furthermore I would like to set up AzureFunctions which are able to connect to the Database as well as write records in.
I already set up an AzureSQL Database but I do not know how to work with it. I started two attempts:
Set up table directly in SSMS, fill data in table, create keys and encrypt it with the wizard. This works totally fine and I am able to see the plain data only if I set the 'AlwaysEncrypted' Checkbox while Connecting to the database.
My second attempt was to include the 'always encrypt directly in the queries. I tried the following:
CREATE COLUMN MASTER KEY CMK_test_1
WITH (
KEY_STORE_PROVIDER_NAME = 'AZURE_KEY_VAULT',
KEY_PATH = '<PATH_TO_AZURE_KEY_VAULT>'
)
CREATE COLUMN ENCRYPTION KEY CEK_test_1
WITH VALUES
(
COLUMN_MASTER_KEY = CMK_test_1,
ALGORITHM = 'RSA_OAEP',
ENCRYPTED_VALUE = <VALUE>
)
Create Table dbo.AlwaysEncryptedTest
(
ID int identity(1,1) PRIMARY KEY
, FirstName varchar(25) COLLATE Latin1_General_BIN2 ENCRYPTED WITH (
ENCRYPTION_TYPE = RANDOMIZED,
ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256',
COLUMN_ENCRYPTION_KEY = CEK_test_1) not NULL
, LastName varchar(25) COLLATE Latin1_General_BIN2 ENCRYPTED WITH (
ENCRYPTION_TYPE = RANDOMIZED,
ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256',
COLUMN_ENCRYPTION_KEY = CEK_test_1) not NULL
, City varchar(25) COLLATE Latin1_General_BIN2 ENCRYPTED WITH (
ENCRYPTION_TYPE = RANDOMIZED,
ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256',
COLUMN_ENCRYPTION_KEY = CEK_test_1) not NULL
, StreetName varchar(25) COLLATE Latin1_General_BIN2 ENCRYPTED WITH (
ENCRYPTION_TYPE = RANDOMIZED,
ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256',
COLUMN_ENCRYPTION_KEY = CEK_test_1) not NULL
)
I know that I have to use an application to put records in the database but I could not find a tutorial or something else that helps me to do so. I found some C# explenation on the Microsoft website but this did not help me to do the job. In best case I would write the connection in python.
Any help is appreciated.
Best
P
If you want to connect Azure SQL server which enables always encrypt with Azure key vault in python application, we can use ODBC driver to implement it.
Regarding how to implement it, we need to add ColumnEncryption=Enabled into connection string to tell odbc application always encrypt has been enabled. Besides, since we use Azure key vault store, we also need to add KeyStoreAuthentication KeyStorePrincipalId and KeyStoreSecret to make ODBC application connect Azure key vault, get encryption key.
For more details, please refer to here and here
For example
Create a service principal to connect Azure key vault
az login
az ad sp create-for-rbac --skip-assignment --sdk-auth
az keyvault set-policy --name $vaultName --key-permissions get, list, sign, unwrapKey, verify, wrapKey --resource-group $resourceGroupName --spn <clientId-of-your-service-principal>
Code
server = '<>.database.windows.net'
database = ''
username = ''
password = ''
driver= '{ODBC Driver 17 for SQL Server}'
KeyStoreAuthentication='KeyVaultClientSecret'
KeyStorePrincipalId='<clientId-of-your-service-principal>'
KeyStoreSecret='<clientSecret-of-your-service-principal>'
conn_str=f'DRIVER={driver};SERVER={server};PORT=1443;DATABASE={database};UID={username};PWD={password};ColumnEncryption=Enabled;KeyStoreAuthentication={KeyStoreAuthentication};KeyStorePrincipalId={KeyStorePrincipalId};KeyStoreSecret={KeyStoreSecret}'
with pyodbc.connect(conn_str) as conn:
with conn.cursor() as cursor:
cursor.execute("SELECT * FROM [dbo].[Patients]")
row = cursor.fetchone()
while row:
print (row)
row = cursor.fetchone()
I have a Apache Ignite database running which I want to interact with using Python thin client (pyignite). I've already performed create, read and update operations, but I meet problems with the delete ones. For now, even if the submission of delete request does not raise any error, the entries that are supposed to be deleted are not.
I've tried deleting those same entries running the same delete query in terminal via jdbc:ignite:thin://127.0.0.1/ and this does succesfully remove the targeted entries.
Here is how I unsuccesfully tried to delete data :
self.client = Client()
self.client.connect('127.0.0.1', 10800)
patientID = 5
IS_DEFINED_QUERY = "SELECT * FROM Patients WHERE PatientID = ?"
result = self.client.sql(
IS_DEFINED_QUERY,
query_args=[patientID]
)
try:
next(result)
DELETE_QUERY = "DELETE FROM Patients WHERE PatientID = ?"
self.client.sql(
DELETE_QUERY,
query_args=[patientID])
except StopIteration:
raise KeyDoesNotExist()
Any help would be greatly appreciated, thanks !
EDIT: I've got some suggestions saying it might come from database settings that would prevent Thin Client from executing deletions, any thought about it ?
I parse a database into an RDFlib graph. I now want to INSERT the triples from this graph into the GraphDB triple store. The code works fine when I execute it on an older version of GraphDB-Lite hosted on Sesame. However, I get an error while executing the same query on the now standalone GraphDB 7.0.0. The graph is partially parsed before the error is raised and the inserted triples do show up in the triple store.
This is part of the code:
graphdb_url = 'http://my.ip.address.here:7200/repositories/Test3/statements'
##Insert into Sesame
for s,p,o in graph1:
pprint.pprint ((s,p,o))
queryStringUpload = 'INSERT DATA {%s %s %s}' %(s,p,o)
# queryStringUpload = 'DELETE WHERE {?s ?p ?o .}'
# print queryStringUpload
sparql = SPARQLWrapper(graphdb_url)
sparql.method = 'POST'
sparql.setQuery(queryStringUpload)
sparql.query()
Following is the error:
ARQLWrapper.SPARQLExceptions.QueryBadFormed: QueryBadFormed: a bad request has been sent to the endpoint, probably the sparql query is bad formed.
Response:
MALFORMED QUERY: Lexical error at line 1, column 93. Encountered: "/" (47), after : "purl.org"
What is causing the error and how do I resolve it?
It was a syntax error. I had URIs starting with http:/ instead of http:// in some places.