Develop Reference

HTTPS on EC2 instance running python project

I'm having considerable difficulty getting HTTPS to resolve on my EC2 instance, which runs a python project. The request just times out (ERR_CONNECTION_TIMED_OUT). HTTP runs ok, however. The steps I've taken are as follows.
I've created a certificate in ACM for the following domains: *.mywebsite.com and mywebsite.com
I've setup Route 53 as follows:
Routing policy on the A records is Simple.
I've gone into the Listener for my Load Balancer for my EC2 instance and CHANGED the port from 80 (HTTP) TO 443 (HTTPS) and added my certificate.
Note: the "Forward To" is a Target Group running on port 80 (HTTP). I've read that this is correct.
I've then gone into the Inbound Rules for my Security group, and added HTTPS
At this point, I've got the following questions:
a) Given that this is a python/Django project, is enabling HTTPS for EC2 possible to do this through the aws website or do I need to add config files and deploy to my instance?
b) Do I need to create a target group running on HTTPS?
c) Do I need listeners on my load balance for port 80 and port 443 or just port 443?
d) On my security group, do I need port 80 to go to 0.0.0.0/0 and ::0/?
e) Should the A record by the DNS name of the load balancer or should it be the CNAME of my environment?
Thanks for your help! Once we get the answer here, I'm going to write a guide and post it on youtube.

Let me start by giving you a little bit of overview of how a request flows in this case.
As you have rightly guessed, the Load Balancer, Application Load Balancer to be specific can handle SSL traffic. This also means that from the Load Balancer to the origin server, the mentioned target group in this case, only http traffic will flow and not https. So You don't have to worry about handling certificates on the server. The response from the origin server is then again wrapped up in an SSL tunnel and send back to the client by the ALB.
This means that your end user should be able to connect to the Load Balancer port 443 atleast and also on port 80 (which can redirect to 443).
This means the security group of your load balancer should have port 443 (and optionally 80) open to the world, or to your users.
As between the origin server and the ALB, the traffic flows in the port that your app is running, that is what the security group of the server should allow the access to the ALB.
To rephrase, the server (EC2) security group should allow the ALB on whichever port the application is running.
Note: This doesn't have to be 80 or 443, it can also be 8080, as long as your target group knows about it and is forwarding the request on that port.
Now to answer your questions:
a) Given that this is a python/Django project, is enabling HTTPS for EC2 possible to do this through the aws website or do I need to add config files and deploy to my instance?
You don't have to do this. As I mentioned, the encryption/decryption can be offloaded to ALB. Read more about it int he docs here.
b) Do I need to create a target group running on HTTPS?
This builds up on the previous question, no you don't have to. The app server/EC2 instance should not be concerned with this.
c) Do I need listeners on my load balance for port 80 and port 443 or just port 443?
This depends on your use case. The base necessity is to have only 443. If you want to allow users to still land on the http site and then be redirected to a more secure https version, you can again make use of the ALB for this. More about it here.
d) On my security group, do I need port 80 to go to 0.0.0.0/0 and ::0/? For ALB, yes but not for the EC2 instances. Remember that Ec2 never communicates directly with users, only with the ALB. So you can control the traffic on EC2 more tightly.
e) Should the A record by the DNS name of the load balancer or should it be the CNAME of my environment?
Use Alias records. They are much easier to manage, and AWS will take care of the mapping. More about this here.

How to find out which IP and Port is JIRA() method accessing?

I am using the following script to get issues from Jira.
from jira import JIRA
options = {'server': 'https://it.company.com/'}
jira = JIRA(options, basic_auth=('user', 'password'), max_retries=1)
issues = jira.search_issues('project="Web"', startAt=0, maxResults=50)
I want to replace https://it.company.com/ with https://ip:port.
I usedping to get the IP.
I used nmap for checking ports, but no matter what https://ip:port input I use, I can't get a connection. I also tried these ports.
How can I find out which IP and Port is JIRA() accessing?

The https protocol uses port 443. Refer to wikipedia for details.
However accessing a server via https://server_name/ is different from accessing a server via https://server_ip_address/. This is because during TLS negotiation, server_name is passed to the server via TLS SNI (Server Name Indication). This way multiple virtual websites may be hosted at the same server_ip_address. See wikipedia for details.

If the script works and you just want to know how the connection looks, I recommend letting it run and in the background execute netstat -ano.
If the script doesn't work and you just want to know where it tries to connect, I recommend installing wireshark.
Edit: In any case you (most likely) won't be able to replace it with ip:port because servers treat HTTP requests to an IP different than how they treat requests to a name.

Ask the Jira admin to tell you. Configured in conf/server.xml like any Tomcat app, or there may be a remote proxy such as nginx configured in front of the Jira

Can VM / Machine IP be used instead of Proxy Server for Scrapy

I have a Scrapy crawler and I want to rotate the IP so my application will not be blocked. I am setting IP in scrapy using request.meta['proxy'] = 'http://51.161.82.60:80' but this is a VM's IP. My question is can VM or Machine's IP be used for scrapy or I need a proxy server?
Currently I am doing this. This does not throw any error but when I get response from http://checkip.dyndns.org it is my own IP not updated IP which I set in meta. That is why I want to know if I do need proxy server.

The reason you are getting your own IP is because your VM is 'transparent'. You will need to intercept your request at the VM, remove tracking headers such as X-Forwarded-For, and your server has to know who to respond to when it receives the response from the website you are crawling.
The simplest solution though, is to install a proxy service on your VM, for example Squid, then set forwarded_for off to make it an anonymous proxy server. There may be other request options to tweak to make it truly anonymous. Remember to secure the whitelisted IP addresses with http_access allow specialIP and acl specialIP src x.x.x.x in /etc/squid/squid.conf. The default port of Squid is 3128.

Definitely you need a proxy server. meta data is only a field in the http request. the server side still knows the public ip that really connecting from the tcp connection layer.

Redirect IP camera stream from a Raspberry to my website

So here's my setup:
IP camera -> Raspberry Pi (Raspbian) -> WiFi -> my server
I am currently using motion to retrieve the camera's stream on my RPi. I am able to view it on the local network (192.168.x.x:8080) through my browser (it's an Mjpeg stream).
I would now like to publish this online so I can view it from http://camera.example.com/ for example.
The difference here is that I would like to do so independently of the WiFi network used (so I cannot simply open a port on my router to accept a connection from the server).
I think this would be possible using WebSockets but I never used them before. Or is there some tool that already exists AND is easy to use ? There are many streaming tools out there, but they all seem to be Windows-GUI programs rather than command line tools.
The choice of language is Python, but if for some reason another language would be more suited that is fine too. Also, I do not need to use motion specifically, so if there is a better alternative that would work too. Thanks !

As a set of minimum steps you will need
A domain name that points to your public IP address
A way of keeping the DNS records for the domain up to date as your IP periodically changes (a free dynamic IP from noip.com will help with the first point, and they have a client you can install which will keep their DNS updated with your current IP)
A port forwarding rule on your router to forward port 8080 (and the stream port for the camera stream, probably 8081 but you can change that in the Motion config) to the internal (192.168.x.x) IP of your Pi
A DHCP reservation in your router to reserve the IP of the Pi (otherwise if the internal IP changes you will need to change the port forwarding rule)
You will now be able to access on the internet via the domain name e.g. http://camera.example.com:8080
BUT...
You have just allowed an insecure http (unencrypted) access into a device on your home network, which could then be exploited (someone could view your cameras, or perhaps gain further access to the Pi and other devices on your network...)
You can enable authentication for the web control gui in Motion config but it’s still being served over http and so easy to hack or to intercept.
So, I would also want to ensure it is all accessible only via https (secure,encrypted).
Items you will need:
an SSL certificate for your domain (available for free from letsencrypt.org)
a web server on the Pi (since Motion doesn’t use any installed webserver but instead has its own inbuilt one) - I’d recommend Nginx or Apache
certbot (to generate/install the certificate on the pi)
configure the web server to be a reverse proxy and serve the http motion website as https using your SSL certificate
secure the website (both apache and nginx support http basic authentication which, if the reverse proxy is configured correctly, will be served over https so encrypted, which is better than unencrypted, base64 encoded (and easily decoded) credential info transmitted in the clear for all to see/intercept).
Other authentication options are available with some extra work but as a bare minimum basic auth and full https are better than nothing.

How to connect gcm server using python?

I am using ubuntu 14.4 . i tried to send push notification to mobile phone.I referred following '''https://www.digitalocean.com/community/tutorials/how-to-create-a-server-to-send-push-notifications-with-gcm-to-android-devices-using-python ''' its working for my local pc .
same code I am trying in web server but i cant able to send push notification .i got error like "gcm.gcm.GCMAuthenticationException: There was an error authenticating the sender account" my webserver also ubuntu 14.4 . so please any one help me
gcm.py
from gcm import *
gcm= GCM("as........k")
data={"message from":"123","messageto":"1234","message":"Hi","time":"10.00AM","langid":"1"}
reg_id='AP...JBA'
gcm.plaintext_request(registration_id=reg_id,data=data)
i added my server ip in white list but still i am getting same error

You need to add your IP to the white-listed IP list.
The article you linked mentioned it...
gcm: add your API KEY from the Google API project; make sure your
server's IP address is in the allowed IPs
When you create your access key you specify which servers can be used there, so you will need to edit the allowed server list by adding your server's IP.
Make sure to update your Authorization key is defined in your request.
Ensure that outbound ports 5228, 5229, and 5230 are open.
For further errors, look at Google's page

I had the same problem and solved it cleaning the whitelist, saving it and re-inserting the ip of my server in the whitelist.
It seemed so but it's not true. It's just random: sometimes works, sometimes it returns the error mentioned.