I've been trying out an API call that contains content with the character "=". When I check the API content inside the C# API, the parameter with the "=" character is missing.
Theres also an issue with the "+" character, which simply disappears and gets replaced with " ". I've tried different encodings but the issue persists.
Do you have any idea where the issue might be?
import requests
import json
qry = "select SUM(this + that) as a from asd WHERE a = 10"
URL = "http://127.0.0.1:42100"
PARAMS = {'request_type':'translate_model',
'content_string': qry}
r = requests.post(url = URL, params = PARAMS)
In the example above the content_string parameter is not present in the C# API parameters.
Related
I am working on a python program that turns on/off my mining rig with Nicehash API. I did everything that I need to but I am stuck with what to put in query string.
In header of post request must be X-auth which which is built from API Key and HMAC signature. On NiceHash website it says that HMAC signature must be build like this:
Input structure is the following:
-API Key
-X-Time request header value
-X-Nonce request header value
-Empty field
-X-Organization-Id request header value
-Empty field
-Request method (example: GET, POST, DELETE, PUT)
-Request path (example: /main/api/v2/hashpower/orderBook)
-Request query string (example: algorithm=X16R&page=0&size=100, The query string should be the same as
passed to the server - without the leading question mark)
Input is a byte array composed of ordered fields using zero byte (0x00) as a separator. There is no
separator before the first field or after the last field. Some fields are always empty in which case the
separators immediately follow one another. For request body you should use the raw bytes as they are sent
to the server. For JSON messages the character encoding should always be UTF-8.
So input should look like this (already hashed in UTF-8) and then be hashed again (in order from first to last:
API KEY,TIME,NONCE,ORGANISATION ID,REQUEST METHOD,REQUEST PATH,REQUEST QUERY STRING) :
4ebd366d-76f4-4400-a3b6-e51515d054d6 ⊠ 1543597115712 ⊠ 9675d0f8-1325-484b-9594-c9d6d3268890 ⊠ ⊠ da41b3bc-3d0b-4226-b7ea-aee73f94a518 ⊠ ⊠ GET ⊠ /main/api/v2/hashpower/orderBook ⊠ algorithm=X16R&page=0&size=100
(this sign: ⊠ is zero byte: 0x00)
My code:
import hashlib
import uuid
import requests
import json
import hmac
url = "https://api2.nicehash.com/main/api/v2/mining/rigs/status2"
path = "/main/api/v2/mining/rigs/status2"
ms = str(json.loads(requests.get('https://api2.nicehash.com/api/v2/time').text)['serverTime'])
req_id = str(uuid.uuid4())
nonce = str(uuid.uuid4())
org_id = organizationId
sec_key = secretKey
apiKey = apiKey
method = "POST"
query = ?
input = bytearray(f"{apiKey}\00{ms}\00{nonce}\00\00{org_id}\00\00{method}\00{path}\00{query}", "utf-8")
secret=hmac.new(bytearray(sec_key, "utf-8"), input, hashlib.sha256).hexdigest()
auth = apiKey + ":" + secret
#HEADER
header = {
"X-Time":ms,
"X-Nonce":nonce,
"X-Organization-Id":org_id,
"X-Auth":auth,
"X-Request-Id":req_id}
#BODY
body = {
"rigId": "SOMETHING",
"action": "STOP"
}
r=requests.post(url=url, headers=header, params=body)
print(r)
print(r.json())
But on website the example demostrates if you want to get a hashpower order book and I understand query string there. But I dont know what to put in query string in my case.
For now this is the main problem for me so the rest of the code is not finished yet.
try this:
query="action=STOP&rigId=yourigid" #stops certain rig
I tried to use https://www.nicehash.com/docs/rest and https://www.nicehash.com/docs/
picture of what worked for me
I'm making an api call, and trying to store its response in Postgres using Django models.
Here's what I have been doing:
response = requests.post(url='some.url.com', data=json.dumps(data), headers={'some': 'header'})
response_data = json.loads(response.content.decode('utf-8'))
#handler is a object of a model
handler.api_response = response_data
handler.save()
But this used to fail, when my json had fields like 'field_name': '\x00\x00\x00\x00\x00'. It used to give following error :
DataError at /api/booking/reprice/
unsupported Unicode escape sequence
LINE 1: ... NULL, "api_status" = 0, "api_response" = '{"errorRe...
^
DETAIL: \u0000 cannot be converted to text.
CONTEXT: JSON data, line 1: ..."hoursConfirmed": 0, "field_name":...
How i tried to resolve this is by using the following:
response = requests.post(url='some.url.com', data=json.dumps(data), headers={'some': 'header'})
response_data = json.loads(response.content.decode('utf-8'))
#handler is a object of a model
handler.api_response = json.loads(json.dumps(response_data).encode("unicode-escape").decode())
handler.save()
The initial issue was solved then. But recently, when i got a field with value 'field2_name': 'Hey "Whats up"'. This thing failed by giving error:
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 143 (char 142)
Probably because json.loads() got confused with " inside the value as an enclosing " and not an escaped ".
Now, If i print the initial response just after json.loads(response.content.decode('utf-8')) statement, it shows the field as \x00\x00\x00\x00\x00.
But output of following code:
response = requests.post(url='some.url.com', data=json.dumps(data), headers={'some': 'header'})
response_data = json.loads(response.content.decode('utf-8'))
print(json.dumps(response_data))
This shows the field as \\u0000\\u0000\\u0000\\u0000\\u0000.
How does \x00 change to \\u0000
And how do i save this field into postgres tables ?
This is what i could think of.
json.loads(json.dumps(response_data).replace('\\u0000',''))
To add this statement before saving to postgres.
Is there a better way ?
Is the code response_data = json.loads(response.content.decode('utf-8')) wrong ? Or causing not to escape that particular character ?
FWIW, I recently ran into this and your proposed solution also worked for me. I think it's probably the simplest way to deal with it. Apparently this is the only character that can't go into a Postgres JSON column.
json.loads(json.dumps(response_data).replace('\\u0000',''))
I tested using Python to translate a curl to get some data.
import requests
import json
username="abc"
password="123"
headers = {
'Content-Type': 'application/json',
}
params = (
('version', '2017-05-01'),
)
data = '{"text":["This is message one."], "id":"en-es"}'
response = requests.post('https://somegateway.service/api/abc', headers=headers, params=params, data=data, auth=(username, password))
print(response.text)
The above works fine. It returns json data.
It seems ["This is message one."] is a list. I want to use a variable that loads a file to replace this list.
I tried:
with open(f,"r",encoding='utf-8') as fp:
file_in_list=fp.read().splitlines()
toStr=str(file_in_list)
data = '{"text":'+toStr+', "id":"en-es"}'
response = requests.post('https://somegateway.service/api/abc', headers=headers, params=params, data=data, auth=(username, password))
print(response.text)
But it returned error below.
{
"code" : 400,
"error" : "Mapping error, invalid JSON"
}
Can you help? How can I have valid response.text?
Thanks.
update:
The content of f contains only five lines below:
This is message one.
this is 2.
this is three.
this is four.
this is five.
The reason your existing code fails is that str applied to a list of strings will only rarely give you valid JSON. They're not intended to do the same thing. JSON only allows double-quoted strings; Python allows both single- and double-quoted strings. And, unless your strings all happen to include ' characters, Python will render them with single quotes:
>>> print(["abc'def"]) # gives you valid JSON, but only by accident
["abc'def"]
>>> print(["abc"]) # does not give you valid JSON
['abc']
If you want to get the valid JSON encoding of a list of strings, don't try to trick str into giving you valid JSON by accident, just use the json module:
toStr = json.dumps(file_in_list)
But, even more simply, you shouldn't be trying to figure out how to construct JSON strings in the first place. Just create a dict and json.dumps the whole thing:
data = {"text": file_in_list, "id": "en-es"}
data_str = json.dumps(data)
Being able to do this is pretty much the whole point of JSON: it's a simple way to automatically serialize all of the types that are common to all the major scripting languages.
Or, even better, let requests do it for you by passing a json argument instead of a data argument:
data = {"text": file_in_list, "id": "en-es"}
response = requests.post('https://somegateway.service/api/abc', headers=headers, params=params, json=data, auth=(username, password))
This also automatically takes care of setting the Content-Type header to application/json for you. You weren't doing that—and, while many servers will accept your input without it, it's illegal, and some servers will not allow it.
For more details, see the section More complicated POST requests in the requests docs. But there really aren't many more details.
tldr;
toStr = json.dumps(file_in_list)
Explanation
Assuming your file contains something like
String_A
String_B
You need to ensure that toStr is:
Enclosed by [ and ]
Every String in the list is enclosed by quotation marks.
So your raw json (as a String) is equal to '{"text":["String_A", "String_B"], "id":"en-es"}'
I'm trying to do API-request and I need API-key to different view. I'm trying to use session variable, but the key seems to be in some other format than trying to use variable from Sqlite database. API requests work with the key from database, but not with session variable
How I get API-key from database and from session:
key_session = request.session['key']
key_db = APIkey.objects.values_list('key', flat=True).get(pk=2)
Both of these return same values, when I print them. Key example:
3h3asdh-asdasd:oisf87sdf87a5df76asdf83jhjhasgd8
I'm using base64.encodestring function when trying to do authentication to API-service with my key:
query = request.GET.get('query')
url = urllib2.Request('https://api.someapiwebsite.com',
None, headers={'Content-Type':'application/json'})
base64string = base64.encodestring('%s' % (key_session)).replace('\n', '')
If I print base64string with session variable (key_session), I get:
MmoihjsdasdoihhaG5tbjpuq9876eq9asd98a7Nmd3dWYzN2JmbWZ2aW1nMGVw==
If I print base64string with session variable (key_db), only difference is the two last characters == is now 'IC', and I think that's why the authentication to API service is failing:
MmoihjsdasdoihhaG5tbjpuq9876eq9asd98a7Nmd3dWYzN2JmbWZ2aW1nMGVwIC
What is making this difference in the base64 encoded string?
Edit:
I can see difference when using print repr():
print repr(key_db)
3h3asdh-asdasd:oisf87sdf87a5df76asdf83jhjhasgd8
print repr(key_session)
3h3asdh-asdasd:oisf87sdf87a5df76asdf83jhjhasgd8\x02\x02
One of the strings probably contains some trailing characters that print isn't showing. If you use repr then you should be able to see what the difference is.
print(repr(key_session))
print(repr(key_db))
You can then strip any characters as necessary before encoding the string, for example:
key_session = key_session.rstrip('\x02')
I am translating a large number of strings and using urllib2 to send requests to the API. My program will run fine, but I always get HTTP Error 400 when translating certain strings in particular. Each request I make is exactly the same, except for the text parameter, so I think it must be the text somehow causing the request to be malformed. Here are two strings that I know of that always cause this error:
#monaeltahawy hatla2eh fel jym bytmrn wla 3arf en fe 7aga bt7sl :d
and
#yaratambash ta3aly a3zemek 3l fetar
for example.
I know for certain that it isn't the "#" character causing the error, or the fact that "#" is at the front of the string. The API has processed strings with these attributes just fine before.
It also is not the nonsense words in these strings causing issues, because the API has processed nonsense words fine before as well. It just returns the same string that I sent to it.
Here is the code where the error seems to be coming from:
tweet = tweet.encode("utf-8")
to = "en"
translate_params = { 'text' : tweet, 'to' : to }
request = urllib2.Request('http://api.microsofttranslator.com/v2/Http.svc/Translate?' + urllib.urlencode(translate_params))
request.add_header('Authorization', 'Bearer '+ self.access_token)
response = urllib2.urlopen(request)
# Removes XML tags to return only the translated text
response_text = ET.fromstring(response.read())
response_text = ET.tostring(response_text, encoding = 'utf8', method = 'text')
return response_text
I am running Python 2.7 in Eclipse 4.3.2.
Any insight or suggestions would be very much appreciated.