Develop Reference

shodan - country code python

I want to list once my script in python search for specific strings , but I also want to add country code first two letters , but when I try then it says invalid KeyError: 'country_code', but the api says ocation.country_code how can I achieve that?
#!/usr/bin/python
import shodan
SHODAN_API_KEY="xxxxxxxxxxxxxxxxxxxx"
api = shodan.Shodan(SHODAN_API_KEY)
try:
# Search Shodan
results = api.search('ProFTPd-1.3.3c')
# Show the results
for result in results['matches']:
print '%s' % result['ip_str']
print '%s' % result['country_code']
except shodan.APIError, e:
print 'Error: %s' % e

I think this is the method You are using in Python
https://github.com/achillean/shodan-python/blob/master/shodan/client.py#L324
and it triggers:
return self._request('/shodan/host/search', args)
Shodan API documentation:
https://developer.shodan.io/api
check out /shodan/host/search API
I just saw that the answer is in Your question but You ate one letter from location (ocation).
Try this:
print '%s' % result['location']['country_code']
So field You are looking for is there but it is in another dictionary.
I would recommend to read API documentation well next time and as Nofal Daud said, Python error are self explanatory if You have KeyError on dict it means that field is not there. Next time listen to Python it will reveal the truth.

Python Urllib Urlopen won't return after new line?

I'm trying to get authenticated by an API I'm attempting to access. I'm using urllib.parse.urlencode to encode the parameters which go in my URL. I'm using urllib.request.urlopen to fetch the content.
This should return 3 values from the server, such as:
SID=AAAAAAAAAAA
LSID=BBBBBBBBBBB
AUTH=CCCCCCCCCCC
The problem is it only returns the first value, and the trailing new line character.
import urllib.request
import urllib.parse
Emailparamx = 'Email'
Emailparam = Emailparamx.encode('utf-8')
email = 'myemail#stackoverflow.com'
email = email.encode('utf-8')
Passwdparam = 'Passwd'
Passwdparam = Passwdparam.encode('utf-8')
password = 'hidden'
password = password.encode('utf-8')
Accounttypeparam = 'accountType'
Accounttypeparam = Accounttypeparam.encode('utf-8')
accounttype = 'GOOGLE'
accounttype = accounttype.encode('utf-8')
Serviceparam = 'service'
Serviceparam = Serviceparam.encode('utf-8')
service = 'adwords'
service = service.encode('utf-8')
url = 'https://accounts.google.com/ClientLogin?'
urlen = url.encode('utf-8')
data = [(Emailparamx, email), (Passwdparam, password),
(Accounttypeparam, accounttype), (Serviceparam, service)]
auth = ''
dataurl = urllib.parse.urlencode(data)
accessurl = (url + "%s" % dataurl)
fh = urllib.request.urlopen(accessurl)
equals = '='
eqenc = equals.encode('utf-8')
try:
msg = fh.readline().split(eqenc)
print (msg)
And then msg prints
[b'SID', b'AAAAAAAAAAAAAAAAA\n']
I know that's some seriously ugly code, I'm about a week old in Python. Any help would be greatly appreciated.

The problem is that you're only calling readline once, so it only reads one line. If you want to read the lines one by one, you have to keep calling readline in a loop until done:
while True:
msg = fh.readline()
if not msg:
break
msg = msg.split(eqenc)
print(msg)
However, there's really no good reason to call readline here, because any file-like object (including a urlopen object) is already an iterable full of lines, so you can just do this:
for msg in fh:
print(msg)
Meanwhile, your original code has a try without an except or a finally, which will just raise a SyntaxError. Presumably you wanted something like this:
try:
for msg in fh:
print(msg)
except Exception as e:
print('Exception: {}'.format(e))
While we're at it, we can simplify your code a bit.
If you look at the examples:
Here is an example session that uses the GET method to retrieve a URL containing parameters:
That's exactly what you want to do here (except for the last line). All the extra stuff you're doing with encoding the strings is not only unnecessary, but incorrect. UTF-8 is the wrong encoding is the wrong encoding to use for URLs (you get away with it because all of your strings are pure ASCII); urlopen requires a string rather than an encoded byte string (although, at least in CPython 3.0-3.3, it happens to work if you give it byte strings that happen to be encoded properly); urlencode can take byte strings but may not do the right thing (you want to give it the original Unicode so it can quote things properly); etc.
Also, you probably want to decode the result (which is sent as ASCII—for more complicated examples, you'll have to either parse the fh.getheader('Content-Type'), or read the documentation for the API), and strip the newlines.
You also may want to build a structure you can use in your code instead of just printing it out. For example, if you store the results in login_info, and you need the SID in a later request, it's just login_info['SID'].
So, let's wrap things up in a function, then call that function:
import urllib.request
import urllib.parse
def client_login(email, passwd, account_type, service):
params = {'Email': email,
'Passwd': passwd,
'accountType': account_type,
'service': service}
qs = urllib.parse.urlencode(params)
url = 'https://accounts.google.com/ClientLogin?'
with urllib.request.urlopen(url + qs) as fh:
return dict(line.strip().decode('ascii').split('=', 1) for line in fh)
email = 'myemail#stackoverflow.com'
password = 'hidden'
accounttype = 'GOOGLE'
service = 'adwords'
try:
results = client_login(email, password, accounttype, service)
for key, value in results.items():
print('key "{}" is "{}".format(key, value))
except Exception as e:
print('Exception: {}'.format(e))

expected string or buffer when processing text file with python

I'm processing a large (120mb) text file from my thunderbird imap directory and attempting to extract to/from info from the headers using mbox and regex. the process runs for a while until I eventually get an exception: "TypeError: expected string or buffer".
The exception references the fifth line of this code:
PAT_EMAIL = re.compile(r"[0-9A-Za-z._-]+\#[0-9A-Za-z._-]+")
temp_list = []
mymbox = mbox("data.txt")
for email in mymbox.values():
from_address = PAT_EMAIL.findall(email["from"])
to_address = PAT_EMAIL.findall(email["to"])
for item in from_address:
temp_list.append(item) #items are added to a temporary list where they are sorted then written to file
I've run the code on other (smaller) files, so I'm guessing the issue is my file. The file appears to be just a bunch of text. Can someone point me in the write direction for debugging this?

There can only be one from address (I think!):
In the following:
from_address = PAT_EMAIL.findall(email["from"])
I have a feeling you're trying to duplicate the work of email.message_from_file and email.utils.parseaddr
from email.utils import parseaddr
>>> s = "Jon Clements <jon#example.com>"
>>> from email.utils import parseaddr
>>> parseaddr(s)
('Jon Clements', 'jon#example.com')
So you can use parseaddr(email['from'])[1] to get the email address and use that.
Similarly, you may wish to look at email.utils.getaddresses to handle to and cc addresses...

Well, I didn't solve the issue but have worked around it for my own purposes. I inserted a try statement so that the iteration just continues past any TypeError. For every thousand email addresses I'm getting about 8 failures, which will suffice. Thanks for your input!
PAT_EMAIL = re.compile(r"[0-9A-Za-z._-]+\#[0-9A-Za-z._-]+")
temp_list = []
mymbox = mbox("data.txt")
for email in mymbox.values():
try:
from_address = PAT_EMAIL.findall(email["from"])
except(TypeError):
print "TypeError!"
try:
to_address = PAT_EMAIL.findall(email["to"])
except(TypeError):
print "TypeError!"
for item in from_address:
temp_list.append(item) #items are added to a temporary list where they are sorted then written to file

get dbus.Struct properties

I'm trying to write a notification script using python-dbus.
How can I get properties from a dbus.Struct object?
For example if I print it out as string, it is
dbus.Struct((dbus.String(u'msg_subject:Re: email subject'),), signature=None)
I need to get the inner string.

Looks like dbus.Struct inherits from tuple, so you should be able to do this:
>>> msg = dbus.Struct((dbus.String(u'msg_subject:Re: email subject'),), signature=None)
>>> msg[0]
dbus.String(u'msg_subject:Re: email subject')

String formatting: string specifier in a string constant

Is there a way to insert a string in a string constant/variable that contains a string specifier?
Example:
temp_body = 'Hello %s, please visit %s to confirm your registration.'
body = temp_body % (name, url)
But this raises a TypeError.

Usually it is the way strings are generated e.g. msg template will be loaded from db or some file and things inserted in between, what is url and name in your case?
This works on my machine
>>> temp_body = 'Hello %s, please visit %s to confirm your registration.'
>>> temp_body%("anurag", "stackoverflow")
'Hello anurag, please visit stackoverflow to confirm your registration.'
Also try if str(name), str(url) works , ost probably it won't and try to fix that problem instead.

Works on my machine(TM).
Are you sure that name and url really are strings? What do you get when you do
>>> type(name), type(url)