Develop Reference

Saving API text to textfile PYTHON

I have an api that allows you to download proxies. Every time I try to save this to a note file on python, it saves it with spaces. If I print it however, there are no spaces to be found. Why is this happening and how can I remove the spaces?
import requests
proxyrequest = requests.get("https://api.proxyscrape.com?request=getproxies&proxytype=http")
with open("proxies.txt", "w") as proxywrite:
proxywrite.write(proxyrequest.text)
What I get:
1.10.189.84:44452
1.0.160.41:4145
1.0.150.125:4145
1.10.188.93:37389
1.0.142.155:4145
1.0.155.32:4145
1.0.220.235:4145
1.0.161.67:4145
114.104.137.34:1080
What I need:
1.10.189.84:44452
1.0.160.41:4145
1.0.150.125:4145
1.10.188.93:37389
1.0.142.155:4145
1.0.155.32:4145
1.0.220.235:4145
1.0.161.67:4145
114.104.137.34:1080

Here is a solution that works fine :)
import requests
proxyrequest = requests.get("https://api.proxyscrape.com?request=getproxies&proxytype=http")
proxyrequest_format = proxyrequest.text.strip()
proxyrequest_format = proxyrequest_format.replace("\r","")
list_proxies = list(proxyrequest_format.split("\n"))
with open("proxies.txt", "w") as proxywrite:
for proxy in list_proxies:
proxywrite.write("%s\n" % proxy)
Output:
1.10.188.202:8080
1.0.210.16:8080
1.119.166.180:8080
1.10.188.85:8080
1.197.204.40:9999
01.10.188.202:8080
1.0.190.69:8080
1.2.254.185:8080

It is because of the '\n' in the text. You can fix it by removing '\n' from the text. Use the following code instead:
proxywrite.write(proxyrequest.text.replace('\n',''))

Python executable file closes without showing outputs

I created a code that use Google Books API on Jupiter in python language. I would like to create an .exe file in order to use it on other PCs. I did it with pyinstaller name_of_the_script.py, but when I execute it after I entered the second input, the command window disappears without showing outputs, also if I put an input line at the end in order to keep alive the script until I press a key.
Here the code:
import requests
quote = input('Inserisci la citazione: ')
lingua = input('\nInserisci lingua (sigla, ad esempio ''it'' per ''Italiano''): ')
key = 'xxxxxxxxx'
parms = {'q':quote, 'key':key, 'maxResults':5,'langRestrict':lingua}
r = requests.get(url='https://www.googleapis.com/books/v1/volumes', params = parms)
rj = r.json()
for i in range(0,3):
print('\n' + rj['items'][i]['volumeInfo']['title'] + '\n')
for authors in rj['items'][i]['volumeInfo']['authors']:
print(authors)
print('\n' + '\n')
input('press enter to quit')
What is wrong?

I solved!
It was an error in the conversion of the .ipynb file into .py file.
To convert it I used a code found here in SO and now it works.
Here the post that helped me:
Is it possible to generate an executable (.exe) in a jupyter-notebook?

Twitter Steam analysis issue

I am trying to take a twitter stream, save it to a file and then analyze the contents. However I am having an issue with files generated in the program as opposed to create by CLI
Twitter Analysis program:
import json
import pandas as pd
import matplotlib.pyplot as plt
tweets_data = []
tweets_file = open(“test.txt”, “r”)
for line in tweets_file:
try:
tweet = json.loads(line)
tweets_data.append(tweet)
except:
continue
tweets = pd.DataFrame()
tweets[‘text’] = map(lambda tweet: tweet[‘text’], tweets_data)
However with the last line I keep getting “KeyError: ‘text’ “ which I understand to be related to the fact it can’t find the key.
When I first run the twitter search program, if I output the results to a file from the CLI, it works fine with no issues. If I save the output to a file from inside the program, it gives me the error?
Twitter Search program:
class Output(StreamListener):
def on_data(self, data):
with open(“test.txt”, “a”) as tf:
tf.write(data)
def on_error(self, status):
print status
L = Output()
auth = OAuthHandler(consKey, consSecret)
auth.set_access_token(Tok1, Tok2)
stream = Stream(auth, L)
stream.filter(track=[´cyber´])
If I run the above as is, analyzing the test.txt will give me the error. But if I remove the line and instead run the program as:
python TwitterSearch.py > test.txt
then it will work with no problem when running text.txt through the analysis program
I have tried changing the file handling from append to write which was of no help.
I also added the line:
print tweet[‘text’]
tweets[‘text’] = map(lambda tweet: tweet[‘text’], tweets_data)
This worked and showing that the program can see a value for the text key. I also compared the output file from the program and the CLI and could not see any difference. Please help me to understand and resolve the problem?

File pointer - sending and reception in a function

Python 2.7
Download a test file from: www.py4inf.com/code/mbox.txt
Briefly, I need list all lines that start with 'From' and take only the mail address. Selecting line by line.
If the condition is true, write in other file the ( only) mail address (result). I could wrote the code and it is working. But It would be better if I use functions. I crashed when I tried to pass the parameters. I have a problem when I have a function that receive a parameter and send one or two.
The result is: copy line by line ALL input file in output file almost like a recursion. No search nothing and the file is very big.
At last, have you any page to read about funtions, paramt, passing paramt, pass reference, and other. Ask is easy and I prefer read and try to understand and if I have a problem, light a cande in the middle of the night!.
#Li is the input paramet. Line from fileRead(the pointer of the file).
#if the condition is true select all lines that start with From:
def funFormat(li):
if li.startswith('From:'):
li = li.rstrip('')
li = li.replace('From: ',' \n')
return li?
fileRead=open('mbox_small.txt','r')
for eachline in fileRead:
result=funFormat(eachline)
fileWrite =open('Resul73.txt', 'a')
fileWrite.write( result )
fileRead.close()
fileWrite.close()

You're opening a file every time you need to write, and closing it just at the end. Maybe that's what's messing it up? Try this and let me know if it works -
#Li is the input paramet. Line from fileRead(the pointer of the file).
#if the condition is true select all lines that start with From:
def funFormat(li):
if li.startswith('From:'):
li = li.rstrip('')
li = li.replace('From: ',' \n')
return li
else:
return None
fileRead = open('mbox_small.txt', 'r')
fileWrite = open('Resul73.txt', 'a')
for eachline in fileRead:
result=funFormat (eachline)
if result:
fileWrite.write (result)
fileRead.close()
fileWrite.close()
Also, I suggest you to read up on with blocks. This will help you work with files more efficiently. As for functions, there are enough resources online.

Trigger to automatically remove EOL whitespace?

Can one write a perforce trigger to automatically remove whitespace at submission time? Preferably in python? What would that look like? Or can you not modify files as they're being submitted?

To my knowledge this cannot be done, since you cannot put the modified file-content back to the server. The only two trigger types that allow you to see the file-content with p4 print are change-content and change-commit. For the latter, the files are already submitted on the server and for the former, while you can see the (unsubmitted) file content, there is no way to modify it and put it back on the server.
The only trigger that is possible is to reject files with EOL whitespace to be submitted, so that the submitters can fix the files on their own. Here is an excerpt of a similar one that checks for tabs in files, please read the docu on triggers and look at the Perforce site for examples:
def fail(sComment):
print sComment
sys.exit(1)
return
sCmd = "p4 -G files //sw/...#=%s" % sChangeNr
stream = os.popen(sCmd, 'rb')
dictResult = []
try:
while 1:
dictResult.append(marshal.load(stream))
except EOFError:
pass
stream.close()
failures = []
# check all files for tabs
for element in dictResult:
depotFile = element['depotFile']
sCmd = "p4 print -q %s#=%s" % (depotFile,sChangeNr)
content = os.popen(sCmd, 'rb').read()
if content.find('\t') != -1:
failures.append(depotFile)
if len(failures) != 0:
error = "Files contain tabulators (instead of spaces):\n"
for i in failures:
error = error + str(i) + "\n"
fail(error)