Develop Reference

Why is Django FieldFIle readline() returning the hex version of a text file?

Having an odd problem.
I have a Django app that opens a file (represented as a Django FieldFile) and reads each row using readline() as below:
with file.open(mode='r') as f:
row = f.readline()
# do something with row...
The file is text, utf-8 encoded and lines are terminated with \r\n.
The problem is each row is being read as the hex representation of the string, so instead of "Hello" I get "48656c6c6f".
A few stranger things:
It previously worked properly, but at some point an update has broken it (I've tried rolling back to previous commits and it is still wonky, so possibly a dependency has updated and not something from my requirements.txt). Missed it in my testing because it is in a very rarely used part of the app.
If I read the same file using readlines() instead of readline() I see the correct string representation of the file wrapped in [b'...']
The file reads normally if I do it using straight Python open() and readline() from an interpreter
Forcing text mode with mode='rt' doesn't change the behaviour, neither does mode='rb'
The file is stored in a Minio bucket, so the defaut storage is storages.backends.s3boto3.S3Boto3Storage from django-storages and not the default Django storage class. This means that boto3, botocore and s3fs are also in the mix, making it more confusing for me to debug.
Scratching my head at why this worked before and what I'm doing wrong.
Environment is Python 3.8, Django 2.2.8 and 3.0 (same result) running in Docker containers.
EDIT
Let me point out that the fix for this is simply using
row = f.readline().decode()
but I would still like to figure out what's happening.
EDIT 2
Further to this, FieldFile.open() is reading the file as a binary file, whereas plain Python open() is reading the file as a text file.

This seems very weird.
I think you will see the solution immediately after trying following (I will then update my answer or delete it if it really doesn't help to find it, but I'm quite confident)
A assume, that there is some code, that is monkeypatching file.open or the django view function.
What I suggest is:
Start your code with manage.py runserver
Ad following code to manage.py (as the very first lines)
import file
print("ID of file.open at manage startup is", id(file.open)
Then add code to your view directly one line above the file.open
print("ID of file.open before opening is", id(file.open)
If both ids are different, then something monkeypatched your open function.
If both are the same, then the problem must be somewhere else.
If you don not see the output of these two prints, something might have monkeypatched your view.
If this doesn't work, then try to use
open() instead of file.open()
Is there any particular reason you use file.open()
Addendum 1:
So what you sai is, that file is an object instance of a class is it a FileField?
In any case can you obtain the name of the file and open it with a normal open() to see whether it is only file.open() that does funny things or whether it is also open() reading it this stange way.
Did you just open the file from command line with cat filename (or if under windows with type filename?
If that doesn't work we could add traces to follow each line of the source code that is being executed.
Addendum 2:
Well if you can't try this in a manage.py runserver, what happens if you try to read the file with a manage.py shell?
Just open the shell and type something like:
from <your_application>.models import <YourModel>
entry = <YourModel>.objects.get(id=<idofentry>)
line1 = entry.<filefieldname>.open("r").read().split("\n")[0]
print("line1 = %r" % line1)
If this is still not conclusive, (but only if you can reproduce the issue with the management shell, then create a small file containing the lines.
from <your_application>.models import <YourModel>
entry = <YourModel>.objects.get(id=<idofentry>)
import pdb; pdb.set_trace()
line1 = entry.<filefieldname>.open("r").read().split("\n")[0]
print("line1 = %r" % line1)
And import it from the management shell.
The code should enter the debugger and now you can single step through the open function and see whether you end up on sime weird function in some monkeypatch.

Uploading JSON String as file to Azure File Storage - linebreaks disappear

I have a list of dicts in Python and want to upload it as a json-file to Azure File Storage. When I print the list locally the linebreaks exist. After uploading and manually checking the file on Azure File Storage I noticed that the linebreaks were non existent.
list_of_dicts = my_json_dicts
transformed_dict_str = '\n'.join([json.dumps(x) for x in list_of_dicts])
# print(transformed_dict_str) gives me the "dicts"/lines separated by linebreaks.
service.create_file_from_text(share_name, file_path, file_name.json, transformed_dict_str, encoding='utf-8')
Can anyone tell me why the uploaded file (when i open it in notepad after downloading manually via the browser interface of Azure) does not contain any linebreaks?
Edit:
When I write the string to a local path with the following code, the linebreaks still exist. So it must happen during the create_file_from_text function?
file = open("myjson.json", "w")
file.write(transformed_dict_str)
file.close()

Please use '\r\n' instead of '\n' in your code.
I can reproduce your issue when use '\n', but works fine using '\r\n' (in notepad, there is linebreaks).

How to copy a math output to clipboard in Python?

I'm super new to Python so I'm wondering if someone can help me out or link me to an appropriate post that explains this?
What I would like to do is
9999**9999
in Python Terminal, then copy the output directly to my clipboard or sent to a file.
I tried in Batch using
py 9999**9999 >>pythonoutput.txt
but only got an error of
python.exe: can't open file '9999**9999': [Errno 22] Invalid argument
and not sure how I could make that work either.
Any ideas? Cheers

Here's how to write (append) to a file:-
obj=open("yourfile.txt","a+") #open a reference to your file, in append mode. (Use 'w' for write, and 'r' for read if you ever need to)
obj.write("your chars, numbers or whatever here") #use this as many times as you want before closing
obj.close() #close your reference once you're done

Try using:
python -c print(9999*9999) > outfile.txt
You might want to use py instead of python there since you seem to have your executable renamed.

Sent to result to file is much easier than to clipboard.
In the python terminal,you can do this:
with open("/home/my/output","w") as file:#start a file object for writing
file.write(str(9999*9999))#write the content

Conversion of shell script to python script

Say I have the following HTML script:
<head>$name</head>
And I have the following shell script which replaces the variable in the HTML script with a name
#/bin/bash
report=$(cat ./a.html)
export name=$(echo aakash)
bash -c "echo \"$report\""
This works.
Now I have to implement the shell script in Python so that I am able to replace the variables in the HTML file and output the replaced contents in a new file. How do I do it?
An example would help. Thanks.

It looks like you're after a templating engine, but if you wanted a straight forward, no thrills, built into the standard library, here's an example using string.Template:
from string import Template
with open('a.html') as fin:
template = Template(fin.read())
print template.substitute(name='Bob')
# <head>Bob</head>
I thoroughly recommend you read the docs especially regarding escaping identifier names and using safe_substitute and such...

with open('a.html', 'r') as report:
data = report.read()
data = data.replace('$name', 'aakash')
with open('out.html', 'w') as newf:
newf.write(data)

Firstly you could save your html template like:
from string import Template
with open('a.html') as fin:
template = Template(fin.read())
Then if you want to substitute variables one at a time, you need to use safe_substitute and cast the result to a template every time. This wont return a key error even when a key value is not specified.
Something like:
new=Template(template.safe_substitute(name="Bob"))
After this , the new template is new , which needs to be modified again if you would want.

save text file and display its content on the browser

If the following script.py writes "some text here" to output.txt file, my URL will be http://my_name/script.py. My question is, how can I read the output.txt as soon as (right after) the following function creates it, so that my URL reads like http://my_name/output.txt.
Many thanks in advance.
#------ script.py -------
def write_txt(){
f=('./output.txt', 'w')
f.write("some text here")
}

try webbrowser lib.
import webbrowser
myurl = "file:///mydir/output.txt"
webbrowser.open(myurl)
However:
Note that on some platforms, trying to
open a filename using this function,
may work and start the operating
system’s associated program.
That is: your file will probably be open in your default text editor (p.e. notepad). A possible solution is to give a custom extension to your file (p.e. output.url) and to associate the extension to your browser (not tested)

Depends on various factors, like OS and webserver used.
Pipe the output to the browser specifying a correct content-type, or, given you script writes to an accessible location, issue a HTTP redirect code pointing to that location.