Develop Reference

Python: Using a variable in a filename output [duplicate]

Sorry for this very basic question. I am new to Python and trying to write a script which can print the URL links. The IP addresses are stored in a file named list.txt. How should I use the variable in the link? Could you please help?
# cat list.txt
192.168.0.1
192.168.0.2
192.168.0.9
script:
import sys
import os
file = open('/home/list.txt', 'r')
for line in file.readlines():
source = line.strip('\n')
print source
link = "https://(source)/result”
print link
output:
192.168.0.1
192.168.0.2
192.168.0.9
https://(source)/result
Expected output:
192.168.0.1
192.168.0.2
192.168.0.9
https://192.168.0.1/result
https://192.168.0.2/result
https://192.168.0.9/result

You need to pass the actual variable, you can iterate over the file object so you don't need to use readlines and use with to open your files as it will close them automatically. You also need the print inside the loop if you want to see each line and str.rstrip() will remove any newlines from the end of each line:
with open('/home/list.txt') as f:
for ip in f:
print "https://{0}/result".format(ip.rstrip())
If you want to store all the links use a list comprehension:
with open('/home/list.txt' as f:
links = ["https://{0}/result".format(ip.rstrip()) for line in f]
For python 2.6 you have to pass the numeric index of a positional argument, i.e {0} using str.format .
You can also use names to pass to str.format:
with open('/home/list.txt') as f:
for ip in f:
print "https://{ip}/result".format(ip=ip.rstrip())

Get the link inside the loop, you are not appending data to it, you are assigning to it every time. Use something like this:
file = open('/home/list.txt', 'r')
for line in file.readlines():
source = line.strip('\n')
print source
link = "https://%s/result" %(source)
print link

Try this:
lines = [line.strip('\n') for line in file]
for source in lines:
print source
for source in lines:
link = "https://{}/result".format(source)
print link
The feature you just described is often called string interpolation.
In Python, this is called string formatting.
There are two styles of string formatting in Python: the old style and the new style.
What I've shown in the example above is the new style, in which we format with a string method named format.
While the old style uses the % operator, eg. "https://%s/result" % source

Use format specifier for string and also put the link printing section in the for loop only
something like this:
import sys
import os
file = open('/home/list.txt', 'r')
for line in file.readlines():
source = line.strip('\n')
print source
link = "https://%s/result”%source
print link

import sys
import os
file = open('/home/list.txt', 'r')
for line in file.readlines():
source = line.strip('\n')
print source
link = "https://" + str(source) + "/result”
print link

Convert IP in text file to Hostname

I'm trying to get hostnames using IPs from text file, but I'm unable to read all IPs from text file and the output shows only one IP.
Below is my COde,
import os
import socket
with open('ips.txt', 'r') as f:
for line in f.read().strip('\n'):
ip = line.strip()
b = socket.getfqdn(ip)
print b
Thanks.

The problem is in:
for line in f.read().strip('\n'):
this iterates over the whole file content (f.read()) without the trailing \n. Strings are iterables in Python, so essentially you are just iterating over each character of the text file.
Instead, as file objects are iterable, you can do the iteration line by line and get the relevant FQDN:
with open('ips.txt', 'r') as f:
for line in f:
ip = line.strip()
fqdn = socket.getfqdn(ip)
# print(fqdn) # Python 3
print fqdn # python 2

How to check if a block of lines has a particular keyword using python?

I am checking a text file with blocks of commands as following -
File start -
!
interface Vlan100
description XYZ
ip vrf forwarding XYZ
ip address 10.208.56.62 255.255.255.192
!
interface Vlan101
description ABC
ip vrf forwarding ABC
ip address 10.208.55.126 255.255.255.192
no ip redirects
no ip unreachables
no ip proxy-arp
!
File End
and I want to create a txt file where if in source file I am getting a pattern vrf forwarding ABC output should be interface Vlan101
as of now what I have done following script but it showing only the line which contains the pattern.
import re
f = open("output_file.txt","w") #output file to be generated
shakes = open("input_file.txt","r") #input file to read
for lines in shakes:
if re.match("(.*)ABC(.*)",lines):
f.write(lines)
f.close()

Easiest: read the file, cut where ! is, then for each of those, if there's the desired text, get the first line:
with open("input_file.txt") as r, open("output_file.txt", "w") as w:
txt = r.read()
result = [block.strip().split("\n")[0]
for block in txt.split('!')
if 'vrf forwarding ABC' in block]
w.write("\n".join(result))

Just to be clear, I imagine that you want to replace any instances of "interface Vlan101" with "vrf forwarding ABC". In this case, I had test.txt as the input file and out.txt as the output file with all the replaced instances as was needed. I used a list comprehension--with a list string method-- to replace the substrings of "interface Vlan101" with "vrf forwarding ABC".
with open("test.txt") as f:
lines = f.readlines()
new_lines = [line.replace("interface Vlan101", "vrf forwarding ABC" for line in lines]
with open("out.txt", "w") as f1:
f1.writelines(new_lines)
Hope this helps.

If you are just interested in the interface, you can do following as well.
#Read File
with open('sample.txt', 'r') as f:
lines = f.readlines()
#Capture 'interfaces'
interfaces = [i for i in lines if i.strip().startswith('inter')]
#Write it to a file
with open('output.txt', 'w') as f:
f.writelines(interfaces)

With your code you are going through the document line by line.
If you want to parse blocks (between "!"-signs) you could split the blocks into lines first (though if it's a really large document, you may need to consider something else as this will read the entire document into memory)
import re
f = open("output_file.txt","w") #output file to be generated
source = open("input_file.txt","r") #input file to read
lines = "".join(source) #creates a string from the document
shakes = lines.replace("\n","").replace("! ","\n")
# remove all newlines and create new ones from "!"-block delimiter
# retrieve all text before "vrf forwarding ABC"
finds = re.findall("(.*)vrf forwarding ABC",shakes)
# return start of line
# if the part you want is the same length in all,
# then you could use find[:17] instead of
# find to get only the beginning. otherwise you need to modify your
# regex to only take the first 2 words of the line.
for find in finds:
f.write(find)
f.close()
Alternatively, if you want to use match per line, you can do the same as above, however instead of replacing "!" with new line, you can just split it, and then use the previous code and go line by line.
Hope this helps!

facing issue with "wget" in python

I am very novice to python. I am facing issue with "wget" as well as " urllib.urlretrieve(str(myurl),tail)"
when I run script it's downloading files but filename are ending with "?"
my complete code :
import os
import wget
import urllib
import subprocess
with open('/var/log/na/na.access.log') as infile, open('/tmp/reddy_log.txt', 'w') as outfile:
results = set()
for line in infile:
if ' 200 ' in line:
tokens = line.split()
results.add(tokens[6]) # 7th token
for result in sorted(results):
print >>outfile, result
with open ('/tmp/reddy_log.txt') as infile:
results = set()
for line in infile:
head, tail = os.path.split(line)
print tail
myurl = "http://data.xyz.com" + str(line)
print myurl
wget.download(str(myurl))
# urllib.urlretrieve(str(myurl),tail)
output :
# python last.py
0011400026_recap.xml
http://data.na.com/feeds/mobile/android/v2.0/video/games/high/0011400026_recap.xml
latest_1.xml
http://data.na.com/feeds/mobile/iphone/article/league/news/latest_1.xml
currenttime.js
Listing the files :
# ls
0011400026_recap.xml? currenttime.js? latest_1.xml? today.xml?

A possible explanation of the behaviour you experience is that you do
not sanitize your input line
with open ('/tmp/reddy_log.txt') as infile:
...
for line in infile:
...
myurl = "http://data.xyz.com" + str(line)
wget.download(str(myurl))
When you iterate on a file object, (for line in infile:) the string
you get is terminated by a newline ('\n') character — if you do not
remove the newline before using line, oh well, the newline character
is still there in what is produced by your use of line …
As an illustration of this concept, have a look at the transcript
of a test I've done
08:28 $ cat > a_file
a
b
c
08:29 $ cat > test.py
data = open('a_file')
for line in data:
new_file = open(line, 'w')
new_file.close()
08:31 $ ls
a_file test.py
08:31 $ python test.py
08:31 $ ls
a? a_file b? c? test.py
08:31 $ ls -b
a\n a_file b\n c\n test.py
08:31 $
As you can see, I read lines from a file and create some files using
line as the filename and guess what, the filenames as listed by ls
have a ? at the end — but we can do better, as it's explained in the
fine manual page of ls
-b, --escape
print C-style escapes for nongraphic characters
and, as you can see in the output of ls -b, the filenames are not
terminated by a question mark (it's just a placeholder used by default
by the ls program) but are terminated by a newline character.
While I'm at it, I have to say that you should avoid to use a
temporary file to store the intermediate results of your computation.
A nice feature of Python is the presence of generator expressions,
if you want you can write your code as follows
import wget
# you matched on a '200' on the whole line, I assume that what
# you really want is to match a specific column, the 'error_column'
# that I symbolically load from an external resource
from my_constants import error_column, payload_column
# here it is a sequence of generator expressions, each one relying
# on the previous one
# 1. the lines in the file, stripped from the white space
# on the right (the newline is considered white space)
# === not strictly necessary, just convenient because
# === below we want to test for non-empty lines
lines = (line.rstrip() for line in open('whatever.csv'))
# 2. the lines are converted to a list of 'tokens'
all_tokens = (line.split() for line in lines if line)
# 3. for each 'tokens' in the 'all_tokens' generator expression, we
# check for the code '200' and possibly generate a new target
targets = (tokens[payload_column] for tokens in all_tokens if tokens[error_column]=='200')
# eventually, use the 'targets' generator to proceed with the downloads
for target in targets: wget.download(target)
Don't be fooled by the amount of comments, w/o comments my code is just
import wget
from my_constants import error_column
lines = (line.rstrip() for line in open('whatever.csv'))
all_tokens = (line.split() for line in lines if line)
targets = (tokens[payload_column] for tokens in all_tokens if tokens[error_column]=='200')
for target in targets: wget.download(target)

Using fileinput (Python) for a search-and-replace while also sending messages to console

I have lines
for line in fileinput.input(file_full_path, inplace=True):
newline, count = re.subn(search_str, replace_str, line.rstrip())
# ... display some messages to console ...
print newline # this is sent to the file_full_path
which are supposed to replace all occurrences of search_str in the file file_full_path and replace them with replace_str. The fileinput maps stdout to the given file. So, print newline and things sent to sys.stdout are sent to the file and not to the console.
I would like to, in the middle of the process, display some messages to console, e.g. I could show the portion of the line in which the replacement is going to occur, or some other messages, and then continue with the print newline into the file. How to do this?

From Python docs:
Optional in-place filtering: if the keyword argument inplace=1 is
passed to fileinput.input() or to the FileInput constructor, the file
is moved to a backup file and standard output is directed to the input
file (if a file of the same name as the backup file already exists, it
will be replaced silently).
so you should write to stderr to display messages in the console, like this:
import sys
for line in fileinput.input(file_full_path, inplace=True):
newline, count = re.subn(search_str, replace_str, line.rstrip())
sys.stderr.write("your message here")
print newline