I'm trying to use Curl to perform a call with the DUO API.
I tried reviewed their docs here: https://duo.com/docs/adminapi#authentication
The docs says to pass the creds as HMAC key for the request but now sure how to get that going.
This is what I got so far:
curl --request GET \
--header 'Authorization: Basic 'Integration key:Secret key'' \
--header "Content-Type: application/x-www-form-urlencoded" \
"https://api-12345678.duosecurity.com/auth/v2/check"
Which returns
{"code": 40101, "message": "Missing request credentials", "stat": "FAIL"}
Can one point me to the right direction for an example in Bash. If not in Python.
First, your request format does not seem correct, because Integration key:Secret key'' is outside the header (look at the way the syntax is highlighted in the question).
Try:
curl --request GET \
--header 'Authorization: Basic' \
--header 'Integration key: Secret key' \
--header 'Date: Tue, 21 Aug 2012 17:29:18 -0000' \
--header "Content-Type: application/x-www-form-urlencoded" \
"https://api-12345678.duosecurity.com/auth/v2/check"
It's somewhat uncommon to have header names with a space and a lowercase like Integration key, so you may need to experiment with variants, like Integration-Key.
Second, the 401xx series errors mean:
401 The “Authorization”, “Date”, and/or “Content-Type” headers were missing or invalid.
You'll need to add the missing the Date header, required by the authenticator.
In case anyone else stumbles on this, here's what I came up with:
#!/bin/bash -u
FORM="Content-Type: application/x-www-form-urlencoded"
NOW=$(date -R)
#get these from the Duo Admin interface
INT="<integration key>"
KEY="<secret passcode>"
API="<api host>.duosecurity.com"
URL="/auth/v2/check"
REQ="$NOW\nGET\n$API\n$URL\n"
#could also use awk here, or the --binary mode as suggested elsewhere
HMAC=$(echo -n "$REQ" | openssl sha1 -hmac "$KEY" | cut -d" " -f 2)
AUTH=$(echo -n "$INT:$HMAC" | base64 -w0)
curl -s -H "Date: $NOW" -H $FORM -H "Authorization: Basic $AUTH" https://$API$URL
Running this yields:
{"response": {"time": 1539726254}, "stat": "OK"}
Reference: Duo Api docs section on authentication
Related
I am trying to write data with the python code you can see in the GUI (IP:8086) for InfluxDB. My influxdb is version 2.5. I have the right influxdb-client for this version. I have tried creating new token, but the result is the same. I have followed all the steps in the tutorial.
The code is:
import influxdb_client, os, time
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS
token = os.environ.get(“INFLUXDB_TOKEN”)
org = “censured”
url = “https://us-west-2-1.aws.cloud2.influxdata.com/”
client = influxdb_client.InfluxDBClient(url=url, token=token, org=org)
bucket=“censured”
write_api = client.write_api(write_options=SYNCHRONOUS)
for value in range(5):
point = (*
Point(“measurement1”)*
.tag(“tagname1”, “tagvalue1”)*
.field(“field1”, value)*
)*
write_api.write(bucket=bucket, org=“censured”, record=point)*
time.sleep(1) # separate points by 1 second*
And the output is this one:
influxdb_client.rest.ApiException: (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({‘Date’: ‘Mon, 28 Nov 2022 12:30:02 GMT’, ‘Content-Type’: ‘application/json; charset=utf-8’, ‘Content-Length’: ‘55’, ‘Connection’: ‘keep-alive’, ‘trace-id’: ‘b834ca17b533f0e8’, ‘trace-sampled’: ‘false’, ‘x-platform-error-code’: ‘unauthorized’, ‘Strict-Transport-Security’: ‘max-age=15724800; includeSubDomains’, ‘X-Influxdb-Request-ID’: ‘2ae0c8dfaf88a4d59bc51e69332d3cdc’, ‘X-Influxdb-Build’: ‘Cloud’})
HTTP response body: {“code”:“unauthorized”,“message”:“unauthorized access”}
I am not able of writing or adding any data to the database.
I have tried changing url for localhost:8086, with same result.
I have tried communicating with the API with this code, but same result:
curl --request POST "http://localhost:8086/api/v2/write?org=YOUR_ORG&bucket=YOUR_BUCKET&precision=ns" --header "Authorization: Token YOUR_API_TOKEN" --header "Content-Type: text/plain; charset=utf-8" --header "Accept: application/json" --data-binary '
airSensors,sensor_id=TLM0201 temperature=73.97038159354763,humidity=35.23103248356096,co=0.48445310567793615 1630424257000000000
airSensors,sensor_id=TLM0202 temperature=75.30007505999716,humidity=35.651929918691714,co=0.5141876544505826 1630424257000000000
'
{"code":"unauthorized","message":"unauthorized access"}
Does StackOverflow really autodeletes Hey guys from beginning of text? :D Hello, i have a problem i cant seem to wrap my mind around.
from influxdb_client import InfluxDBClient
from influxdb_client.client.write_api import SYNCHRONOUS
# You can generate a Token from the "Tokens Tab" in the UI
org = "myorg"
bucket = "mybucket"
token = 'valid_token'
client = InfluxDBClient(url="http://localhost:8086", token=token)
write_api = client.write_api(write_options=SYNCHRONOUS)
d='airSensors,sensor_id=TLM0201 temperature=70.97038159354763,humidity=55.23103248356096,co=0.78445310567793615 1637124357000000000'
write_api.write(bucket, org, d)
This runs and returns no error, i tried making a mistake in eg. bucket and it raises, bad token raises unauthorized, etc..
BUT there is no data in database when i check. BUT when i run this exact line through curl:
curl --request POST \
"http://localhost:8086/api/v2/write?org=myorg&bucket=mybucket&precision=ns" \
--header "Authorization: Token valid_token" \
--header "Content-Type: text/plain; charset=utf-8" \
--header "Accept: application/json" \
--data-binary '
airSensors,sensor_id=TLM0201 temperature=73.97038159354763,humidity=35.23103248356096,co=0.48445310567793615 1637024357000000000
airSensors,sensor_id=TLM0202 temperature=75.30007505999716,humidity=35.651929918691714,co=0.5141876544505826 1637024357000000000
'
This runs also with no errors but this time it actually writes into db.
Am i crazy or what? I tried everything, writing through Points, series,... u name it but it refuses to commit or smthn? Anyone had similar problem?
I run influxdb-client=1.23.0 on python=3.8.10 and Influxdb=2.0.7
Thanks for ur time. Q.
I guess you should use write_api.close() in the end of your write or use with:
with client.write_api() as write_api:
write_api.write(bucket, org, d)
https://github.com/influxdata/influxdb-client-python#writes
I am trying to write a program that does chemical search on https://echa.europa.eu/ and gets the result. The "Search for Chemicals" field is on the middle of the main webpage. I want to get the resulting URLs from doing search for each chemicals by providing the cas number (ex. 67-56-1). It seems that the URL I get does not include the cas number provided.
https://echa.europa.eu/search-for-chemicals?p_p_id=disssimplesearch_WAR_disssearchportlet&p_p_lifecycle=0&_disssimplesearch_WAR_disssearchportlet_searchOccurred=true&_disssimplesearch_WAR_disssearchportlet_sessionCriteriaId=dissSimpleSearchSessionParam101401584308302720
I tried inserting different cas number (71-23-8) into "p_p_id" field, but it didn't give expected search result.
https://echa.europa.eu/search-for-chemicals?p_p_id=71-23-8
I also examined the headers of GET methods requested from Chrome which also did not include the cas number.
Is the website using variables to store the input query? Is there a way or a tool that can be used to get the resulting URL including searching cas number?
Once I figure this out, I'll be using Python to get the data and save it as excel file.
Thanks.
You need to get the JESSIONID cookie by requesting the main url once then send a POST on https://echa.europa.eu/search-for-chemicals. But this needs also some required URL parameters
Using curl and bash :
query="71-23-8"
millis=$(($(date +%s%N)/1000000))
curl -s -I -c cookie.txt 'https://echa.europa.eu/search-for-chemicals'
curl -s -L -b cookie.txt 'https://echa.europa.eu/search-for-chemicals' \
--data-urlencode "p_p_id=disssimplesearch_WAR_disssearchportlet" \
--data-urlencode "p_p_lifecycle=1" \
--data-urlencode "p_p_state=normal" \
--data-urlencode "p_p_col_id=column-1" \
--data-urlencode "p_p_col_count=2" \
--data-urlencode "_disssimplesearch_WAR_disssearchportlet_javax.portlet.action=doSearchAction" \
--data-urlencode "_disssimplesearch_WAR_disssearchportlet_backURL=https://echa.europa.eu/home?p_p_id=disssimplesearchhomepage_WAR_disssearchportlet&p_p_lifecycle=0&p_p_state=normal&p_p_mode=view&p_p_col_id=column-1&p_p_col_count=2" \
--data-urlencode "_disssimplesearchhomepage_WAR_disssearchportlet_sessionCriteriaId=" \
--data "_disssimplesearchhomepage_WAR_disssearchportlet_formDate=$millis" \
--data "_disssimplesearch_WAR_disssearchportlet_searchOccurred=true" \
--data "_disssimplesearch_WAR_disssearchportlet_sskeywordKey=$query" \
--data "_disssimplesearchhomepage_WAR_disssearchportlet_disclaimer=on" \
--data "_disssimplesearchhomepage_WAR_disssearchportlet_disclaimerCheckbox=on"
Using python and scraping with beautifulsoup
import requests
from bs4 import BeautifulSoup
import time
url = 'https://echa.europa.eu/search-for-chemicals'
query = '71-23-8'
s = requests.Session()
s.get(url)
r = s.post(url,
params = {
"p_p_id": "disssimplesearch_WAR_disssearchportlet",
"p_p_lifecycle": "1",
"p_p_state": "normal",
"p_p_col_id": "column-1",
"p_p_col_count": "2",
"_disssimplesearch_WAR_disssearchportlet_javax.portlet.action": "doSearchAction",
"_disssimplesearch_WAR_disssearchportlet_backURL": "https://echa.europa.eu/home?p_p_id=disssimplesearchhomepage_WAR_disssearchportlet&p_p_lifecycle=0&p_p_state=normal&p_p_mode=view&p_p_col_id=column-1&p_p_col_count=2",
"_disssimplesearchhomepage_WAR_disssearchportlet_sessionCriteriaId": ""
},
data = {
"_disssimplesearchhomepage_WAR_disssearchportlet_formDate": int(round(time.time() * 1000)),
"_disssimplesearch_WAR_disssearchportlet_searchOccurred": "true",
"_disssimplesearch_WAR_disssearchportlet_sskeywordKey": query,
"_disssimplesearchhomepage_WAR_disssearchportlet_disclaimer": "on",
"_disssimplesearchhomepage_WAR_disssearchportlet_disclaimerCheckbox": "on"
}
)
soup = BeautifulSoup(r.text, "html.parser")
table = soup.find("table")
data = [
(
t[0].find("a").text.strip(),
t[0].find("a")["href"],
t[0].find("div", {"class":"substanceRelevance"}).text.strip(),
t[1].text.strip(),
t[2].text.strip(),
t[3].find("a")["href"] if t[3].find("a") else "",
t[4].find("a")["href"] if t[4].find("a") else "",
)
for t in (t.find_all('td') for t in table.find_all("tr"))
if len(t) > 0 and t[0].find("a") is not None
]
print(data)
Note that I've set the timestamp parameter (formDate param) in case of it's actually checked on the server
I am following this link in order to make an HTTP POST to an Eventhub in Azure. the error I am getting is 401 40103: Invalid authorization token signature. According to Azure, the POST should have the following format:
POST https://your-namespace.servicebus.windows.net/your-event-hub/partitions/0/messages?timeout=60&api-version=2014-01 HTTP/1.1
Authorization: SharedAccessSignature sr=your-namespace.servicebus.windows.net&sig=your-sas-key&se=1403736877&skn=RootManageSharedAccessKey
Content-Type: application/atom+xml;type=entry;charset=utf-8
Host: your-namespace.servicebus.windows.net
Regarding the Authorization header, I have several questions:
My secret key (sig) has an equal sign, should I substitute it by
%3d?
I am currently doing also the POST operation with the
example scripts in Python and C from Azure. In these examples, is
only required to introduce the endpoint with all the credentials and
the post/get operations works fine. Is it possible to perform the
put/get operations introducing directly the endpoint in the api
rest, or obtaining the authorization header that is performed bellow
the python or c code?
Thanks.
I'm able to make it work by creating a SAS policy (nssend below) on the namespace, not on the Event Hub.
$ curl -X POST -i "https://breakingnews.servicebus.windows.net/sucheventsmuchwow/messages" \
-H "Authorization: SharedAccessSignature sr=https%3A%2F%2Fbreakingnews.servicebus.windows.net%2Fsucheventsmuchwow%2Fmessages&sig=SAS_SIG_GOES_HERE&se=1536733370&skn=nssend" \
-H "Content-Type: application/json" \
--data-ascii "{ \"message\": \"So many events, so little time.\" }"
HTTP/1.1 201 Created
Server: Microsoft-HTTPAPI/2.0
...
So that works.
However, i'm getting a HTTP 401, just like you are, when using an Event Hub level SAS policy to generate the signature (as opposed to a namespace level policy).
This is what i used to generate the SAS token —
// Make a SAS token
// See https://learn.microsoft.com/en-us/rest/api/eventhub/generate-sas-token
// Appologies for JavaScript
// npm install moment
const moment = require('moment');
const crypto = require('crypto');
function create_sas_token(uri, key_name, key)
{
// Token expires in one hour
var expiry = moment().add(7, 'days').unix();
var string_to_sign = encodeURIComponent(uri) + '\n' + expiry;
var hmac = crypto.createHmac('sha256', key);
hmac.update(string_to_sign);
var signature = hmac.digest('base64');
var token = 'SharedAccessSignature sr=' +
encodeURIComponent(uri) +
'&sig=' + encodeURIComponent(signature) +
'&se=' + expiry + '&skn=' + key_name;
return token;
}
let token = create_sas_token('https://breakingnews.servicebus.windows.net/sucheventsmuchwow/messages', 'MySendPolicy', 'MySendPolicyPrimaryKey=');
console.log(token);
UPDATE
Thanks to Clemens Vasters —
Try omitting "/messages"— Clemens Vasters 📨, Messenger (#clemensv) September 5, 2018
Your string to sign (resource URI) should omit /messages, e.g.
create_sas_token('https://breakingnews.servicebus.windows.net/sucheventsmuchwow',
'MyEventHubLevelPolicy', 'hUbPriMAry/KEy=');
then craft your request as follows —
$ curl -X POST -i "https://breakingnews.servicebus.windows.net/sucheventsmuchwow/messages" \
-H "Authorization: SharedAccessSignature sr=https%3A%2F%2Fbreakingnews.servicebus.windows.net%2Fsucheventsmuchwow&sig=DONT_INCLUDE_/MESSAGES_IN_STRING_TO_SIGN&se=1536757127&skn=MyEventHubLevelPolicy" \
-H "Content-Type: application/json" \
--data-ascii "{ \"message\": \"SAS signed with Event Hub level policy\" }"
HTTP/1.1 201 Created
Server: Microsoft-HTTPAPI/2.0
...
TL;DR:
Your POST URL should include the trailing /messages, but the string to sign (resource URI) should not. Always. Regardless if using Namespace or Hub-scoped policy.
Well first am sorry because i didn't know how to ask my question, last time in a security challenge i was trying to send some request with curl, after a few moment where they have a lot test to find out how the challenge is really working , i tried to write some python code for generating my request automatically and win some time
here are some of the request that i used to try :
The basic one
curl http://10.20.0.50:80/
then i have to specify the path example :
curl http://10.20.0.50:80/transfert/solde
curl http://10.20.0.50:80/account/creat
...
some time add authorization or cookie ...
curl http://10.20.0.50:80/transfert/solde -H "Authorization:Basic bXlhcGk6U3VwZXJTZWNyZXRQYXMkdzByZA==" -H "cookie: PHPSESSID=23c3jc3spuh27cru38kf9l2au5;"
or add some parameters :
curl http://10.20.0.50:80/transfert/solde -H "Authorization:Basic bXlhcGk6U3VwZXJTZWNyZXRQYXMkdzByZA==" -H "cookie: PHPSESSID=23c3jc3spuh27cru38kf9l2au5;" --data-raw '{"id":"521776"}' -v
So the thing is i have to test a lot of thing with and without authorization with and without cookie and changing cookie some times and add --data-raw ... i tried to write a script to do this for me but it's ugly :
url = "http://10.20.0.50:80/"
auth = ' -H "Authorization:Basic bXlhcGk6U3VwZXJTZWNyZXRQYXMkdzByZA=="'
def generate(path,c=None,h=True,plus = None):
#c cookie , h if we put authentification
#plus add more code at the end of the request
global auth # authentification
global url
if c:
cook = ' -H "cookie: PHPSESSID={};"'.format(c)
req = "curl "+url+path
if h:#h bool
req += auth
if c :
req += cook
if plus :
req += plus
req+=" -v "
return req
I removed one parameter the --data-row for readability, the idea is that i want to know if there is a better way for doing that ! and not just with this example but in general, if i want to create python code that generate a code source of class where i have to specify the name of class the attributes and type and the code generate a template ...
I hope that you can help me :D
PS : Sorry for my English if i mad some mistakes
Maybe, a one way to "improve" your code is doing something like this:
def generate(command = "", headers = [], raws = [], other = [], v = True):
if headers:
command += "".join(" -H " + k for k in h)
if raws:
command += "".join(" --data-raw " + k for k in raw)
if v:
command += " -v"
if other:
command += "".join(" " + k for k in other)
return command
h = ['"Authorization:Basic bXlhcGk6U3VwZXJTZWNyZXRQYXMkdzByZA=="', '"cookie: PHPSESSID=23c3jc3spuh27cru38kf9l2au5;"']
raw = ["'{\"id\":\"521776\"}'"]
cmd = "curl http://10.20.0.50:80/transfert/solde"
command1 = generate(command=cmd,headers=h,raws= raw)
command2 = generate(command=cmd,headers=h,raws=raw, v=False)
command3 = generate(command=cmd,v = False)
print("command1:",command1)
print("command2:", command2)
print("command3:", command3)
Output:
command1: curl http://10.20.0.50:80/transfert/solde -H "Authorization:Basic bXlhcGk6U3VwZXJTZWNyZXRQYXMkdzByZA==" -H "cookie: PHPSESSID=23c3jc3spuh27cru38kf9l2au5;" --data-raw '{"id":"521776"}' -v
command2: curl http://10.20.0.50:80/transfert/solde -H "Authorization:Basic bXlhcGk6U3VwZXJTZWNyZXRQYXMkdzByZA==" -H "cookie: PHPSESSID=23c3jc3spuh27cru38kf9l2au5;" --data-raw '{"id":"521776"}'
command3: curl http://10.20.0.50:80/transfert/solde