Develop Reference

AWS Lambda to RDS PostgreSQL

Hello fellow AWS contributors, I’m currently working on a project to set up an example of connecting a Lambda function to our PostgreSQL database hosted on RDS. I tested my Python + SQL code locally (in VS code and DBeaver) and it works perfectly fine with including only basic credentials(host, dbname, username password). However, when I paste the code in Lambda function, it gave me all sorts of errors. I followed this template and modified my code to retrieve the credentials from secret manager instead.
I’m currently using boto3, psycopg2, and secret manager to get credentials and connect to the database.
List of errors I’m getting-
server closed the connection unexpectedly. This probably means the server terminated abnormally before or while processing the request
could not connect to server: Connection timed out. Is the server running on host “db endpoint” and accepting TCP/IP connections on port 5432?
FATAL: no pg_hba.conf entry for host “ip:xxx”, user "userXXX", database "dbXXX", SSL off
Things I tried -
RDS and Lambda are in the same VPC, same subnet, same security group.
IP address is included in the inbound rule
Lambda function is set to run up to 15 min, and it always stops before it even hits 15 min
I tried both database endpoint and database proxy endpoint, none of it works.
It doesn’t really make sense to me that when I run the code locally, I only need to provide the host, dbname, username, and password, that’s it, and I’m able to write all the queries and function I want. But when I throw the code in lambda function, it’s requiring all these secret manager, VPC security group, SSL, proxy, TCP/IP rules etc. Can someone explain why there is a requirement difference between running it locally and on lambda?
Finally, does anyone know what could be wrong in my setup? I'm happy to provide any information in related to this, any general direction to look into would be really helpful. Thanks!

Following the directions at the link below to build a specific psycopg2 package and also verifying the VPC subnets and security groups were configured correctly solved this issue for me.
I built a package for PostgreSQL 10.20 using psycopg2 v2.9.3 for Python 3.7.10 running on an Amazon Linux 2 AMI instance. The only change to the directions I had to make was to put the psycopg2 directory inside a python directory (i.e. "python/psycopg2/") before zipping it -- the import psycopg2 statement in the Lambda function failed until I did that.
https://kalyanv.com/2019/06/10/using-postgresql-with-python-on-aws-lambda.html
This the VPC scenario I'm using. The Lambda function is executing inside the Public Subnet and associated Security Group. Inbound rules for the Private Subnet Security Group only allow TCP connections to 5432 for the Public Subnet Security Group.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.Scenarios.html#USER_VPC.Scenario1

HTTPS on EC2 instance running python project

I'm having considerable difficulty getting HTTPS to resolve on my EC2 instance, which runs a python project. The request just times out (ERR_CONNECTION_TIMED_OUT). HTTP runs ok, however. The steps I've taken are as follows.
I've created a certificate in ACM for the following domains: *.mywebsite.com and mywebsite.com
I've setup Route 53 as follows:
Routing policy on the A records is Simple.
I've gone into the Listener for my Load Balancer for my EC2 instance and CHANGED the port from 80 (HTTP) TO 443 (HTTPS) and added my certificate.
Note: the "Forward To" is a Target Group running on port 80 (HTTP). I've read that this is correct.
I've then gone into the Inbound Rules for my Security group, and added HTTPS
At this point, I've got the following questions:
a) Given that this is a python/Django project, is enabling HTTPS for EC2 possible to do this through the aws website or do I need to add config files and deploy to my instance?
b) Do I need to create a target group running on HTTPS?
c) Do I need listeners on my load balance for port 80 and port 443 or just port 443?
d) On my security group, do I need port 80 to go to 0.0.0.0/0 and ::0/?
e) Should the A record by the DNS name of the load balancer or should it be the CNAME of my environment?
Thanks for your help! Once we get the answer here, I'm going to write a guide and post it on youtube.

Let me start by giving you a little bit of overview of how a request flows in this case.
As you have rightly guessed, the Load Balancer, Application Load Balancer to be specific can handle SSL traffic. This also means that from the Load Balancer to the origin server, the mentioned target group in this case, only http traffic will flow and not https. So You don't have to worry about handling certificates on the server. The response from the origin server is then again wrapped up in an SSL tunnel and send back to the client by the ALB.
This means that your end user should be able to connect to the Load Balancer port 443 atleast and also on port 80 (which can redirect to 443).
This means the security group of your load balancer should have port 443 (and optionally 80) open to the world, or to your users.
As between the origin server and the ALB, the traffic flows in the port that your app is running, that is what the security group of the server should allow the access to the ALB.
To rephrase, the server (EC2) security group should allow the ALB on whichever port the application is running.
Note: This doesn't have to be 80 or 443, it can also be 8080, as long as your target group knows about it and is forwarding the request on that port.
Now to answer your questions:
a) Given that this is a python/Django project, is enabling HTTPS for EC2 possible to do this through the aws website or do I need to add config files and deploy to my instance?
You don't have to do this. As I mentioned, the encryption/decryption can be offloaded to ALB. Read more about it int he docs here.
b) Do I need to create a target group running on HTTPS?
This builds up on the previous question, no you don't have to. The app server/EC2 instance should not be concerned with this.
c) Do I need listeners on my load balance for port 80 and port 443 or just port 443?
This depends on your use case. The base necessity is to have only 443. If you want to allow users to still land on the http site and then be redirected to a more secure https version, you can again make use of the ALB for this. More about it here.
d) On my security group, do I need port 80 to go to 0.0.0.0/0 and ::0/? For ALB, yes but not for the EC2 instances. Remember that Ec2 never communicates directly with users, only with the ALB. So you can control the traffic on EC2 more tightly.
e) Should the A record by the DNS name of the load balancer or should it be the CNAME of my environment?
Use Alias records. They are much easier to manage, and AWS will take care of the mapping. More about this here.

How to find out which IP and Port is JIRA() method accessing?

I am using the following script to get issues from Jira.
from jira import JIRA
options = {'server': 'https://it.company.com/'}
jira = JIRA(options, basic_auth=('user', 'password'), max_retries=1)
issues = jira.search_issues('project="Web"', startAt=0, maxResults=50)
I want to replace https://it.company.com/ with https://ip:port.
I usedping to get the IP.
I used nmap for checking ports, but no matter what https://ip:port input I use, I can't get a connection. I also tried these ports.
How can I find out which IP and Port is JIRA() accessing?

The https protocol uses port 443. Refer to wikipedia for details.
However accessing a server via https://server_name/ is different from accessing a server via https://server_ip_address/. This is because during TLS negotiation, server_name is passed to the server via TLS SNI (Server Name Indication). This way multiple virtual websites may be hosted at the same server_ip_address. See wikipedia for details.

If the script works and you just want to know how the connection looks, I recommend letting it run and in the background execute netstat -ano.
If the script doesn't work and you just want to know where it tries to connect, I recommend installing wireshark.
Edit: In any case you (most likely) won't be able to replace it with ip:port because servers treat HTTP requests to an IP different than how they treat requests to a name.

Ask the Jira admin to tell you. Configured in conf/server.xml like any Tomcat app, or there may be a remote proxy such as nginx configured in front of the Jira

deploying my django site to windows server without using IIS

I finished writing a Django application and now I want to deploy it,
I have a Windows server and have successfully installed Python and Django on it,
Now my app runs on localhost on my windows server,
Now I want to make the site public, meaning that anyone who goes to the IP address of my windows server can browse my site,
Is there a simple way to do this without using IIS?
thank you

Step One
Set a static ip for your server (It's possible without this, but easier)
Once set, log into your router as admin, and forward port 80 to your servers ip address.
There is a tutorial for this at https://portforward.com
Step Two
If you already have a domain name, ignore this bit
Purchase a domain name from an domain name from a domain name registrar such as
1and1 / Ionos (https://ionos.com)
(I would personally advise against https://GoDaddy.com, the prices tend to be odd there)
Step 2.1
Go into your domains settings, and forward traffic to the external ip address of your router.
Hope that this helped!

I've never done what your planing but if you plan to host just one domain then localhost(127.0.0.1) will work just fine but if you plane to host multiple domain you will need to find a way to resolve the right domain to the right site.
Ports to open on both router and Win Server
80 for HTTP
465 for HTTPS
Make sure that you have a Static IP provided by your ISP, you will also need to make sure that your ISP does not Block port 80 if they do ask them to unblock.
If you don't have a static IP or ISP don't allow you to open port 80 then you can use DynDNS to forward traffic to your server, but this option is not the best.
Your Server will also need a static IP to the server as mentiond by Legorooj

Troubleshooting Websockets with EC2 on AWS using Django

I am using Django-Channels to try to get real time features such as chat/messaging, notifications, etc. Right now, I have gotten everything to work fine on my laptop using the settings described in the docs here: http://channels.readthedocs.io/en/latest/. I use a local redis server for testing purposes.
However, when I deploy to my Amazon EC2 Elastic Beanstalk server (using an AWS ElastiCache Redis), the WebSocket functionality fails. I was reading and I think it is due to the fact that Amazon's HTTPS does not support WebSockets, so I need to switch to Secure TCP.
I tried doing that with:
https://blog.jverkamp.com/2015/07/20/configuring-websockets-behind-an-aws-elb/
and
https://medium.com/#Philmod/load-balancing-websockets-on-ec2-1da94584a5e9#.ak2jh5h0q
but to no avail.
Does anyone have any success implementing WebSockets with CentOS/Apache and Django on AWS EB? The Django-Channels package is fairly new so I was wondernig if anyone has experienced and/or overcome this hurdle.
Thanks in advance

AWS has launched new Application Load Balancer that supports web sockets. Change your ELB to Application Load Balancer and that will fix your issue.
https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/

As described here it's possible to run Django Channels on Elastic Beanstalk using an Application Load Balancer.
In a simplified form, it's basically:
Create an ALB
Add 2 target groups, one that points to port 80, and one that points to Daphne port, ie 8080.
Create 2 path rules. Let the default route point to target group 1 (port 80), and set the second to use a relative path, ie. /ws/ and point it to target group 2.
Add Daphne and workers to supervisord (or another init system)
DONE! Access Daphne/websockets through the relative url ws://example.com/ws/.

I suppose ALB is the only way. The reason is because with the SSL protocol listner in the classic LB, the session stickiness and X-Forwaded headers won't be forwarded and will result in the proxy server redirect loop. Doc is here,
http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-listener-config.html
I'll update the answer if I find out a way with the existing CLB.