In documentation we can read:
"The IP of the outgoing IP address to use for the performing the request."
That is not clear for me. Anyone can explain with more details what is the purpose of bindaddress ?
The computer where Scrapy is running might have multiple network connections, each with their own unique IP network address (or addresses, plural). For example, a laptop might have a WiFi connection and a wired Ethernet connection. A larger server-class system might have several Ethernet connections. Even a system that has a single network connection might have multiple IP addresses, some for IPv4 and others for IPv6.
The bindaddress option can be used to tell Scrapy which one of those local IP addresses should be used as the source address on its outgoing requests.
If you don't specify which local address you want Scrapy to use, then Scrapy will let the system choose the address. That choice is usually the local address that the system thinks is "closest" to the destination address of the request. This would be the usual situation. Unless you have a particular need to use a specific source address, there's no reason to use the bindaddress option.
Related
How do you lookup the local hostname from an IPv6 address in Python?
I'm trying to diagnose network bandwidth hogs, and I'm using Wireshark to find which IPv6 addresses are using the most bandwidth, but it doesn't include any hostsnames, so I'm left with a CSV of IPv6 addresses and total bytes transmitted.
Note, I'm not talking about converting a generic IP to domain, since that only works for DNS, not your local network's hostnames.
The only command line tool I know for doing this is:
sudo nmap -sn 192.168.1.0/24
but that doesn't list IPv6 addresses.
This is what I found looking online, I'm sorry but i think that what you are lookin at isn't possible.
From WireShark Q&A:
What you're looking at are probably neighbor discovery packets, which
are sent to a special multicast address called "solicited node
multicast".
It's basically the replacement mechanism for ARP, which is not used
for IPv6 anymore.
Usually, if you need to find out where something is coming from you
should try to determine the MAC address of the source, and then log in
to your switches to find the port where that MAC address is connected.
There should be a command that will show you the MAC address table of
the switch. If you can find a port where only the MAC address in
question is listed you need to follow the cable from that port to the
device.
If you have more than one MAC address listed for a port it is usually
a connection to another switch. In that case you need to log in to the
switch it connects to and repeat your search.
My goal is to have remote control of a device on a WLAN. This device has software that enables me to configure this wireless network (IP, mask, gateway, dns). I can successfully connect this device, and my computer to a common network. Since both machines share the same network, I made the assumption that I would be able to open up a socket between them. Knowing the IP and port of the device that I am attempting to control remotely I used the following code, only to receive a timeout:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('192.168.xxx.xxx', XXXX))
(I am using python 2.7 on mac OS 10.11.6)
The network that I am connected to is on a different subnet that the IP that I assigned to my device. I also tried this having an IP on the same subnet as my network. There could be a number of things keeping me from opening a socket. That's not really what I'm after. The heart of my question is whether or not I can use python's 'socket' module to connect to a device wirelessly.
Yes you can.
So you get a timeout when you try to connect to a wireless device. There are several steps you can take in order to troubleshoot this.
Make sure your device has a program running that is listening to the port you want to connect to. Identify if the device can answer ICMP packets in general and can be pinged in particular. Try to ping the device. If ping succeeds, it means that basic connectivity is established and the problem is somewhere higher in the OSI stack.
- I can ping the device - great, it means that the problem is somewhere in TCP or Application Layer of the TCP/IP stack. Make sure the computer, the device, and intermediate networking equipment allow for TCP connections to the particular host and port. Then proceed to your application and the device software. Add some code to the question, post the stack trace you get or ask another one on SO.
- I can't ping the device - great. There's no connectivity between the devices and you're to identify the reason.
I) Draw a network diagram. How many intermediate network devices are placed in between the computer and the device? What are they, routers, switches? (Just in case, home grade wifi modem is a router.) Get an idea of how IP datagrams should travel across the net.
II) You said that the device can be used to configure an IP network. At least for troubleshooting purposes I would ignore this option and rely on a static IP or your router's DHCP server. Using an existing DHCP will ensure there's no IP misconfigurations.
III) Review routing tables of all the devices you have. Do they have an appropriate default gateway? Does a router knows how to pass the packets to the device. You're probably in trouble if the computer and the device are in the same subnet but attached to different network interfaces. Split the network in two subnets if needed and set up static routes between them on the router.
You can also use wireshark to see if data you send leaves the computer or is dropped right there by some nasty firewall.
There's a lot of caveats in getting a LAN working. You may want to ask questions on networking.stackexchange if these simple steps doesn't help you or if you have major troubles following them. Or just leave a comment here, I'd be happy to help.
I am looking for an easy way to convert a MAC address to the corresponding IP address in a local network. In my case, there are only two devices: a very normal PC (192.168.0.1) and a scientific instrument which has an arbitrary IP address (192.168.0.xxx) hard coded in its ROM. The PC and the instrument are directly connected over a UDP socket with a CAT5 cable.
I know the MAC address of the instrument, but please assume that its IP address is unknown. I would like to write a C/C++ application which talks with the instrument using a socket connection. But I need to know the IP address before opening a socket (WinSock on Windows, sys/socket on OS X and Linux).
Currently I use a very dirty way as shown below.
Execute ping command ping 192.168.0.2 (NOTE: the instrument does not respond to ping)
Repeat this from 192.168.0.2 to 192.168.0.255
Execute arp -a to print a list of IP and MAC addresses
Find the known MAC address and the corresponding IP address from the list
I would like to know how to retrieve the IP address in a more sophisticated way. It will be very nice if I can use the same method on Mac, Linux and Windows machines.
As far as I know, I have to broadcast a ARP packet to the network in order to retrieve a MAC address from a known IP address. But I could not find a way to get an IP address from a MAC address.
There's no good, generic solution for this as it is the reverse of intended behavior. Lower level protocols are not supposed to need to be aware of higher layer ones, so operations at the MAC layer don't have any good way of finding out about IP addresses. And then you get into the situation you're in now. So you can either employ a hack like the code you already have, or you can tackle this from a different direction. Is there any non-code way to determine the IP address of the device before hand? Such as setting it explicitly or putting it in a configuration file for your app. Alternatively, can you have the device send out spurious ARP requests? The PC should update its ARP cache based off of incoming requests as well as responses to requests it made.
We had to do this a while back, but I don't think we got it working properly.
I don't have the API calls off-hand, but they're easy to find in the Windows API. That's what we used, so our solution wouldn't be portable to non-Windows systems.
In our case, we ran into the same hurdle--no easy translation. What we ended up having to do was get a list of all the NICs available, and then loop through each one trying to match our given MAC address against the MAC address obtained from the NIC structure.
Once we found a match, we looked up the IP address given to the NIC structure.
We kept on going to see if we found any other matches in order to log an error. It's a good thing we did, because I believe we did find it multiple times, and it wasn't due to a MAC address being cloned.
That's when we learned that this would be an even harder problem, and we decided to abandon the whole thing and stick to just IP addresses.
How about try the system command arp within c++
system("arp");
This gives you a IP-MAC translation table.
few days ago I was also facing this issue but after many struggle I got its solution below
How MAC to IP address converter tool works?
This MAC address converter can convert MAC address to IPv4 IP Address and convert MAC address to IPv6 IP Address, these internet protocol Addresses are very common to use. It takes MAC Address as input string and generates a query against given MAC to IP address and MAC conversion option like to MAC to IPV6 or MAC to IPV4 or both for MAC address conversion together. After this MAC conversion you can also revert MAC to IP conversion changes by using IPv6 to IPv4. Query generates an output response according to selected options.If you insert any invalid input produces an invalid input message response
I have machine with multiple NICs that can be connected to one net, to different nets or every other possible way. Python script that uses requests module to send POST/GET requests to the server is ran on that machine.
So, the question is next: how can I know in python script from which interface requests will be sent?
Here there are two different answers. The one is for the case where you want to specify the NIC to send the request and the other is for what you're asking: find the correct NIC.
For the second answer, I can only say that it depends. Are these NICs on the same network / subnet? Are they bonded?
If they are on a different network then you can know the host IP Address and use the computer's routing table to see which NIC the packet will go through. If they are bonded on the same network then the request will leave from the bond interface since it is (normally) the one with an IP Address. If they are on the same network and have a different IP on the same subnet then it depends. Can you provide some more information in that case?
To cut the story short: is there any way to get IP of untrusted client calling remote proxy object, using Pyro4?
So person A is calling my object from IP 123.123.123.123 and person B from IP 111.111.111.111 is there any way to distinguish them in Pyro4, assuming that they cannot be trusted enough to submit their own IP.
I ran across your question when looking for the same thing. If I understand, on the server side you want to know the IP address the client's socket is connected from, right? If so, this'll do it:
Pyro4.current_context.client.sock.getpeername()[0]
Found that in this section of the Pyro4 user's guide
Here is my solution to my problem: since I didn't really need to get specific addresses of clients using pyro, just to distinguish clients in specific subnet (in my classrom) from other ips (students working from home). I just started two pyro clients on two ports and then filtered traffic using a firewall.