Develop Reference

Ruby net/http GET requests with empty body [duplicate]

This question already has answers here:
Ruby - net/http - following redirects
(6 answers)
Closed 17 days ago.
I'm currently simply trying to get a simple GET request working in Ruby, however, I'm seeing some strange behavior.
I have an Open Web Analytics application running with Docker and it is reachable at http://127.0.0.1:8080/.
I can reach the login site and everything works fine.
Now I want to do a GET request with Ruby to analyze the body of that request but I cannot get it to work, in other languages like Python or simple GET requests over the terminal it works fine. Why not with Ruby?
Here is my very basic Ruby code:
require 'net/http'
url = 'http://127.0.0.1:8080/'
uri = URI(url)
session = Net::HTTP.new(uri.host, uri.port)
response = session.get(uri.request_uri)
puts response.body
Which doesn't output anything. If I look into the NGINX logs from the container, I can see the request being made but there is no further redirection as with the other methods (see below).
172.23.0.1 - - [02/Feb/2023:20:02:59 +0000] "GET / HTTP/1.1" 302 5 "-" "Ruby" "-" 0.088 0.088 . -
If I do a simple GET over the terminal, it works:
GET http://127.0.0.1:8080/
will output the correct body, and in the NGINX logs I can see the following:
172.23.0.1 - - [02/Feb/2023:20:20:10 +0000] "GET / HTTP/1.1" 302 5 "-" "lwp-request/6.61 libwww-perl/6.61" "-" 0.086 0.088 . -
172.23.0.1 - - [02/Feb/2023:20:20:10 +0000] "GET /index.php?owa_do=base.loginForm&owa_go=http%3A%2F%2F127.0.0.1%3A8080%2F& HTTP/1.1" 200 3200 "-" "lwp-request/6.61 libwww-perl/6.61" "-" 0.086 0.088 . -
Doing it in Python with the following basic code also works and gives similar results as with the terminal GET version:
import requests
x = requests.get("http://127.0.0.1:8080/")
print(x.content)
What am I doing wrong?

Got it working with following redirects (see here):
begin
response = Net::HTTP.get_response(URI.parse(url))
url = response['location']
end while response.is_a?(Net::HTTPRedirection)

Python + Flask Facebook messenger bot doesn't work

I tried a bunch of tutorials, here is an example of one of them
https://nickmccullum.com/build-facebook-bot-python-flask/
Request is send:
127.0.0.1 - - [02/May/2022 10:18:15] "GET /?hub.mode=subscribe&hub.challenge=646416116&hub.verify_token=YOUR_VERIFY_TOKEN HTTP/1.1" 200 -
But when trying to validate, the Webhook throws this error:
Validation of the callback URL or confirmation token failed. Confirm the information provided or try again later.
what could be the problem

How to handle ah/start with python GAE app?

I had to change the scaling type on my GAE app to handle long-running requests.
But now the app has stopped responding, all I see are:
2021-04-09 01:22:59 default[20210409t011947] "GET /_ah/start HTTP/1.1" 200
2021-04-09 01:22:59 default[20210409t011947] "GET /_ah/start HTTP/1.1" 200
2021-04-09 01:23:00 default[20210409t011947] "GET /_ah/start HTTP/1.1" 200
2021-04-09 01:23:00 default[20210409t011947] "GET /_ah/start HTTP/1.1" 200
I understand I have to respond to this startup event with some type of 200 response which I am doing like this:
#app.route('/_ah/start')
def startup():
'''app starting'''
logging.info('handle _ah/start')
return {
'status': 'OK',
'msg': 'boot'
}
But I never see that endpoint actually get called. Instead i just get the logs like I'm not responding to it.
After some digging I found this message talking about how maybe traffic wasn't getting switched over between instances and use --promote to switch but doesn't work.
The main.py has some logging that should come out at startup but I don't see anything.

turns out there was a front-end proxy intercepting the events!

unable to confirm subscribing to HTTP\S endpoint from AWS SNS using python?

I am using python fastApi as a framework, I have the following post method:
async def post(request:Request):
logger.info(f"recieve a sns notification {request}")
return {"message": f"got message {await request.body()} "}
I am aware that in the above method there is no logic to confirm it is just a showcase for my real issue
I subscribed the endpoint through the console, and the status is Pending confirmation
every time press the "Request Confirmation" button I can see in cloudWatch logger:
"POST /api/customer-profiles HTTP/1.1" 307 Temporary Redirect
when I use the following code from my local machine:
import requests
import boto3
import json
url = "https://<my-domain>/api/customer-profiles"
payload = {"a": {"a":"v"}}
res = requests.post(url=url, data=json.dumps(payload))
the response:
<Response [200]>
with content
b'{"message":"got message b\'{\\"a\\": {\\"a\\": \\"v\\"}}\' "}'
in cloud watch is can also see:
INFO: <ip> - "POST /api/customer-profiles HTTP/1.1" 307 Temporary Redirect
#logger wirtes!!
recieve a sns notification <starlette.requests.Request object at 0x7ff459d50610>
INFO: <ip> - "POST /api/customer-profiles/ HTTP/1.1" 200 OK
I Am not sure what i am doing wrong
The endpoints is working (also all test are good) but only for the SNS confirmation its NOT,
what am I doing wrong? why the endpoint never get the request?

its all come to a "/"
see the following answer
fastapi make the redirect between www.example.com/api to www.exmaple.com/api/ there is a location redirect between the tow.
once I wrote the subscriber HTTP in the SNS with a \ all was solved.

requests.exceptions.HTTPError: 401 Client Error atlassian-python-api

I am trying to connect to a Confluence page using the python wrapper on the API (as I am not familiar with any of this) but I keep getting the following error:
requests.exceptions.HTTPError: 401 Client Error
I know that people talk about this being caused by the necessity of using an API token but the page runs on an old version of Confluence and I have been told that we cannot use access tokens.
So has anyone any other idea? Here's a small code:
from atlassian import Confluence
confluence = Confluence(
url='https://address',
username='name',
password='pwd'
)
confluence.create_page(
space='Test',
title='A title',
body='something')
I have tried to use an older version of atlassian-python-api just in case there was some conflict but it got me the same error.

Your code looks ok. Authenticating to Confluence using Basic Auth should work without generating an API token, afaik.
The 401 status definitely suggests a problem with the authentication though. The obvious reason for this would be of course wrong credentials, but I assume that you have double checked that the credentials work when interactively logging into confluence with a browser.
To get a better sense of the error, you can import logging to debug your requests and response:
from atlassian import Confluence
import logging
logging.basicConfig(filename='conf_connect.log', filemode='w', level=logging.DEBUG)
try:
c = Confluence(url='https://conf.yoursystem.com', username='name', password='pwd')
# atlassian API does not raise error on init if credentials are wrong, this only happens on the first request
c.get_user_details_by_username('name')
except Exception as e:
logging.error(e)
The Confluence module internally also uses logging, so the requests and responses will appear in your conf_connect.log logfile:
DEBUG:atlassian.rest_client:curl --silent -X GET -H 'Content-Type: application/json' -H 'Accept: application/json' 'https://conf.yoursystem.com/rest/api/user?username=name'
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conf.yoursystem.com:443
DEBUG:urllib3.connectionpool:https://conf.yoursystem.com:443 "GET /rest/api/user?username=name HTTP/1.1" 401 751
DEBUG:atlassian.rest_client:HTTP: GET rest/api/user -> 401
DEBUG:atlassian.rest_client:HTTP: Response text -> <!doctype html><html lang="en"><head><title>HTTP Status 401 – Unauthorized</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 401 – Unauthorized</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> Basic Authentication Failure - Reason : AUTHENTICATED_FAILED</p><p><b>Description</b> The request has not been applied because it lacks valid authentication credentials for the target resource.</p><hr class="line" /><h3>Apache Tomcat/9.0.33</h3></body></html>
ERROR:root:401 Client Error: for url: https://conf.yoursystem.com/rest/api/user?username=name
The response body may include some information on the reason:
HTTP Status 401 – UnauthorizedType Status ReportMessage Basic Authentication Failure - Reason : AUTHENTICATED_FAILEDDescription The request has not been applied because it lacks valid authentication credentials for the target resource.
The reason AUTHENTICATED_FAILED suggests something is likely wrong with your credentials. If you want to dig deeper into that, you can use this SO answer to also display the headers that are being sent with your request.
However, if your reason is AUTHENTICATION_DENIED the problem is likely the following: If you have too many failed authentication attempts in a row, a CAPTCHA challenge is triggered, and this error will occur until the Failed Login Count is reset. This can easily happen when you are developing a script and test it frequently. To remedy this, either open a browser and manually (re-)logon to Confluence, completing the CAPTCHA, or resolve it from the Confluence User Management.