I've been experimenting with a Python CGI script to send an e-mail (hosted with a comercial web host - 123reg), and the problem is whenever I run the script from my web browser, it sends two identical e-mails.
The code to send the mail is definitely only being executed once, there are no loops which could cause it to happen twice, I am definitely not clicking the button twice. No exceptions are thrown and the "success" page is sent to the browser as normal.
The strangest thing is that when I comment out the code to print the result page (which is very simple and has no side effects, just 3 print statements in a row) and replace it with a dummy print statement (print "Content-type: text/plain\n\ntest"), it works properly and only sends one e-mail.
I have tried googling the problem to no avail.
I am at my wit's end because this problem doesn't make any sense to me. I'm pretty sure it must be my script since inexplicably it works when you comment out those print statements.
I'd appreciate any help, thanks.
EDIT:
Here's the code which, when commented out, fixes the problem:
print "Content-type: text/html"
print
print page
EDIT:
The code to send the e-mail:
#send_email function: sends message from from_addr, assumes valid input
def send_email(from_addr, message):
#form the email headers/text:
email = "From: " + from_addr + "\n"
email += "To: " + TO[0] + "\n"
email += "Subject: " + SUBJECT + "\n"
email += "\n"
email += message
#return true for success, false for failure:
try:
server = smtplib.SMTP(SERVER)
server.sendmail(from_addr, TO, email)
server.quit()
return True;
except smtplib.SMTPException:
return False;
#end of send_email function
I'd post the code to format the page variable, but all it does is read from a file, format a string and return the string. Nothing unusual going on.
EDIT
OK, I've commented out the file IO code in the create_page function and it solves the issue, but I don't understand why, and I don't know how to modify it so that it'll work properly.
The create_page function, and therefore the file IO, was still being executed when I found that commenting out the print statements solved the problem.
This is the file IO code from before I commented it out (it's at the very start of the create_page function and the rest of the function simply modifies the page string, then returns it):
#read the template from the file:
frame_f = open(FRAME)
page = frame_f.read()
frame_f.close()
EDIT:
I have just replaced the file IO by copying and pasting the file text directly into a string in my source file, so there is no longer any file IO. This still hasn't fixed the problem. At this point my only theory is that computers hate me...
EDIT:
I'll have to post this here since stackoverflow won't let me answer my own question since I'm a newbie here...
EDIT:
OK, I posted it as an actual answer now.
PROBLEM SOLVED!
It turns out that it was the browser's fault all along. The reason I didn't notice this sooner was because I tested it in both Firefox and Chrome ages ago to rule the browser out, however it turns out that both Chrome and Firefox share this same bug.
I realised what was happening when the server logs finally updated, I realised that often GET requests were immediately (1 second later) followed by another GET request. I did some googling and found this:
What causes Firefox to make a GET request after submitting a form via the POST method?
It turns out that if you have an img tag with an empty src attribute e.g.
<img src=""/>
(I had some javascript which modified that tag), Firefox will send a duplicate GET request in place of a request for the image. It also turns out that Chrome has the same problem. This also explains why the problem was only happening when I was trying to include my html template.
It would help if you posted more code, but does the "page" variable contain code that would execute the email server a second time, or cause a page refresh that would trigger the email a second time.
The same thing will happen if you have a Javascript call with an empty src or "#" as src:
<script type="text/javascript" src="#"></script>
Perhaps also with an empty href for a css link. I haven't experienced that, but I'd expect the same behavior.
Related
EDIT:
In a similar vein, when I now try to log into their account with a post request, what is returned is none of the errors they suggest on their site, but is in fact a "JSON exception". Is there any way to debug this, or is an error code 500 completely impossible to deal with?
I'm well aware this question has been asked before. Sadly, when trying the proposed answers, none worked. I have an extremely simple Python project with urllib, and I've never done web programming in Python before, nor am I even a regular Python user. My friend needs to get access to content from this site, but their user-friendly front-end is down and I learned that they have a public API to access their content. Not knowing what I'm doing, but glad to try to help and interested in the challenge, I have very slowly set out.
Note that it is necessary for me to only use standard Python libraries, so that any finished project could easily be emailed to their computer and just work.
The following works completely fine minus the "originalLanguage" query, but when using it, which the API has documented as an array value, no matter whether I comma-separate things, or write "originalLanguage[0]" or "originalLanguage0" or anything that I've seen online, this creates the error message from the server: "Array value expected but string detected" or something along those lines.
Is there any way for me to get this working? Because it clearly can work, otherwise the API wouldn't document it. Many thanks.
In case it helps, when using "[]" or "<>" or "{}" or any delimeter I could think of, my IDE didn't recognise it as part of the URL.
import urllib.request as request
import urllib.parse as parse
def make_query(url, params):
url += "?"
for i in range(len(params)):
url += list(params)[i]
url += '='
url += list(params.values())[i]
if i < len(params) - 1:
url += '&'
return url
base = "https://api.mangadex.org/manga"
params = {
"limit": "50",
"originalLanguage": "en"
}
url = make_query(base, params)
req = request.Request(url)
response = request.urlopen(req)
I am trying to send an email as below using conflictedblocks_string,string >>>>>>> gets printed fine but gets messedup when sending as email, can anyone explain why and how to fix it?
conflictedblocks_string = ''
conflictedblocks = {'README': '<<<<<<< HEAD\nTBD1\n=======\nTRP1\n>>>>>>> b9bde66...\n', 'DO_NOT_READ': 'Probably a new file'}
for key,value in conflictedblocks.items():
conflictedblocks_string += key + ":" + "\n" + value + "\n"
print conflictedblocks_string --> `>>>>>>>` prints fine
sendemail(conflictedblocks_string ) --> `>>>>>>>` messed up while sending email
sendemail api snippet:
body = '''%s''' % (data)
msg = MIMEText(body)
mail = smtplib.SMTP('company.apple.com', 25)
mail.sendmail(sender, receivers, msg.as_string())
CURRENT OUTPUT:-
EXPECTED OUTPUT:-
README:
<<<<<<< HEAD
TBD1
=======
TRP1
>>>>>>> b9bde66...
DO_NOT_READ:
Probably a new file
There's nothing at all wrong with your code. Or the mail servers. The email has >>>>>>> in it, just as you intended.
However, many mail programs and webmail systems translate > at the start of a line into an indent marker when formatting mail for viewing.
Traditionally, > at the start of a line is how you mark that you're quoting someone inline. So, to make email threads earlier to read, mail clients turn those quotes into something that looks more like quotes.
For example, this is a traditional plain-text email:
John shouted:
> My father said:
>> No! You will BE KILL BY DEMONS
> No! I must kill the demons
The radio said:
> No, John, You are the demons.
And then John was a zombie.
An email client might render it like this:
John shouted:
My father said:
No! You will BE KILL BY DEMONS
No! I must kill the demons
The radio said:
No, John, You are the demons.
And then John was a zombie.
There's no universal workaround that will work for every client, because every client has its own heuristic code that messes things up. But there a few things that often work.
First, if you send both HTML and plain-text versions of your mail, most of the fancy clients that would have treated > as formatting will display the HTML instead, which you can format however you want, while clients that refuse to display HTML will probably also not try to do anything with the >.
Another option is to put the diff you're trying to include as an attachment, instead of the body. You can try to mark it as an inline attachment, in hopes that some clients will show it without making the user click on the attachment and open it, but I don't think too many clients like inline plain-text attachments.
Prefixing the line with a space often works, like this:
<<<<<<< HEAD
TBD1
=======
TRP1
>>>>>>> b9bde66
But of course the person reading the mail will have to know about the extra space—and if they're copying and pasting, they'll have to remember to remove it.
If that works, prefixing every line in the email, or just every line in the diff, with a space, would also work. It doesn't look quite as ugly, but if anything it can cause more copy-paste headaches.
It is unrelated to Python and unrelated to mail sending...
Historically, when mails were just plain ASCII text, the > character was used (as a convention) in responses to mark citations from the original mail.
With HTML and richer character sets, the citations are now indicated with vertical bars | and different colors.
In order to give a nicer user experience, mail readers interpret > characters in plain text mails and format them as modern citation marks.
So:
you mail was correctly sent and contains the correct >
the culprit is your mail reader which (wrongly here) formats initial > in a body line as if it was a citation
# settings.py
EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
# view.py
from django.core.mail import send_mail
def send_letter(request):
the_text = 'this is a test of a really long line that has more words that could possibly fit in a single column of text.'
send_mail('some_subject', the_text, 'me#test.com', ['me#test.com'])
The Django view code above, results in a text file that contains a broken line:
this is a test of a really long line that has more words that could possibl=
y fit in a single column of text.
-------------------------------------------------------------------------------
Anyone know how to change it so the output file doesn't have linebreaks? Is there some setting in Django that controls this? Version 1.2 of Django.
Update - to back up a level and explain my original problem :) I'm
using the django-registration app, which sends an email with an
account activation link. This link is a long URL, with a random
token at the end (30+ characters), and as a result, the line is breaking in the middle of the token.
In case the problem was using the Django's filebased EmailBackend, I switched to the smtp backend and ran the built-in Python smtpd server, in debugging mode. This dumped my email to the console, where it was still broken.
I'm sure django-registration is working, with zillions of people using it :) So it must be something I've done wrong or mis-configured. I just have no clue what.
Update 2 - according to a post in a Django list, it's really the underlying Python email.MIMEText object, which, if correct, only pushes the problem back a little more. It still doesn't tell me how to fix it. Looking at the docs, I don't see anything that even mentions line-wrapping.
Update 3 (sigh) - I've ruled out it being a MIMEText object problem. I used a pure Python program and the smtplib/MIMEText to create and send a test email, and it worked fine. It also used a charset = "us-ascii", which someone suggested was the only charset to not wrap text in MIMEText objects. I don't know if that's correct or not, but I did look more closely at my Django email output, and it has a charset of "utf-8".
Could the wrong charset be the problem? And if so, how do I change it in Django?
Here's the entire output stream from Django's email:
---------- MESSAGE FOLLOWS ----------
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: some_subject
From: me#test.com
To: me#test.com
Date: Tue, 17 May 2011 19:58:16 -0000
this is a test of a really long line that has more words that could possibl=
y fit in a single column of text.
------------ END MESSAGE ------------
You might be able to get your email client to not break on the 78 character soft limit by creating an EmailMessage object and passing in headers={'format': 'flowed'} Like so:
from django.core.mail import EmailMessage
def send_letter(request):
the_text = 'this is a test of a really long line that has more words that could possibly fit in a single column of text.'
email = EmailMessage(
subject='some_subject',
body=the_text,
from_email='me#test.com',
to=['me#test.com'],
headers={'format': 'flowed'})
email.send()
If this doesn't work, try using a non-debug smtp setup to send the file to an actual email client that renders the email according to rules defined in the email header.
Try to define EMAIL_BACKEND in your settings.py. Maybe it doesn't solve your problem, but is the right place where to define it, otherwise it's likely not going to be used.
(Since I'm not sure I'm solving your problem here, I was trying to make a comment on your, but apparently I cannot.)
The email lines aren't "broken" per se -- they're just represented in the quoted-printable encoding. As such, at 76 characters, =\n is inserted. Any competent mail client ought to decode the message properly and remove the break.
If you want to represent the body of an email decoded, you can use this by passing decode=True to the the get_payload method:
body = email.get_payload(decode=True)
This tells the message to decode the quoted-printable encoding.
More to the point, if your main concern is getting the python console debugging server to print the message decoded, you could do something quick and dirty like this snippet rather than using the built-in DebuggingServer. More properly, you could parse the "data" string as an Email object, print out the headers you care about, then print the body with decode=True.
I've seen this is python2.5 and it's fixed in python2.7.
The relevant code in email/generator.py now has a comment saying
# Header's got lots of smarts, so use it. Note that this is
# fundamentally broken though because we lose idempotency when
# the header string is continued with tabs. It will now be
# continued with spaces. This was reversedly broken before we
# fixed bug 1974. Either way, we lose.
You can read about the bug here http://bugs.python.org/issue1974
Or you can just change the '\t' to ' ' in this line of email/generator.py
print >> self._fp, Header(
v, maxlinelen=self._maxheaderlen,
header_name=h, continuation_ws='\t').encode()
I have a CGI script for which I've successfully set a cookie (which I can see in Firefox/Chrome!) which has (say) the name uid and the content 1. I don't seem to understand how to access this cookie from another CGI script--and I'm working in Python 2.4 so a lot of the examples I've found may not apply.
This code prints "can't get uid" followed by the rest of the page:
c = Cookie.SimpleCookie(os.environ.get("HTTP_COOKIE"))
print("Content-Type: text/html")
print c.output()
print("\n\n")
uid = c.get("uid")
#uid = c["uid"].value # this would create an error and page would fail totally
if uid is None:
print("can't get uid")
uid = 1 # set manually to prevent the rest of the page from failing
I haven't done anything fishy with the domain the cookie applies to, so I don't understand why this doesn't grab the uid value. By the way, if I try to print c.output(), it's blank.
First thing is are you sure the webserver or the framework is setting the HTTP_COOKIE environment variable?
Otherwise, in one of your script you may want to store the cookies in the CookieJar file in the file system and access the set cookies from there.
import cookielib
COOKIEFILE = 'Cookies.lwp'
cookiejar = cookielib.LWPCookieJar()
cookiejar.load(COOKIEFILE)
cookiejar["uid"] = 1
cookiejar.save(COOKIEFILE)
Load the same cookiejar and do the get of uid in the other script.
Okay, I think I figured this out! I confirmed that os.environ.get("HTTP_COOKIE") was getting something, and then played with the order of the elements in my tiny test until it worked. Then I reproduced that order in my more complicated script. (Specifically: content type declaration, two newlines, get cookie, get value from cookie, everything else.)
The main thing I've learned about Python and CGI is that the order of elements (starting with the content type declaration) is very fussy. Thanks very much for the hints in the right direction.
I'm trying to learn to use Python to create dynamic web content. Problem I'm having right out the door, though, is that when I try to do a mySQL query, absolutely nothing happens. There's no error message... it looks like the script simply stops running when I import the module that enables connection to the database.
This does do exactly what I'd expect when I try to run it from the command line.
#!/usr/bin/python
print "Content-Type: text/xml"
print
#if I type, for example, print "<b>test</b>" here, it appears in the browser window
#msql contains the credentials for connecting to database
#it is NOT in public_html
import msql
#no print instructions after this point are followed
connex=msql.msqlConn()
db=msql.MySQLdb
cursor=connex.cursor(db.cursors.DictCursor)
cursor.execute("SELECT * FROM userActions")
#run the query
xmlOutput=""
rows=cursor.fetchall()
#output the results
for row in rows:
xmlOutput+="<action>"
xmlOutput+="<actionId>"+str(row["actionId"])+"</actionId>"
xmlOutput+="<userId>"+str(row["userId"])+"</userId>"
xmlOutput+="<actText>"+str(row["action"])+"</actText>"
xmlOutput+="<date>"+str(row["dateStamp"])+"</date>"
xmlOutput+="</action>"
xmlOutput="<list>"+xmlOutput+"</list>"
print xmlOutput
This would be my first stab at this, so it merely stands to reason that this should work. I've found nothing online that would suggest otherwise, though.
Please, take 24h to learn something like Django. Django has an ORM and a XML serializer that will make your life easier ensuring proper (and legal) xml, it really pays up.
You can enable cgi traceback to see what happens:
import cgitb
cgitb.enable()
(I don't think it causes your problem, and clients understand \n as well but it is better to use sys.write("...\r\n") instead of print to print HTTP headers)
Edited: try to add Content-length: XXXX\r\n to the header.