Error! blahfile is not UTF-8 encoded. Saving disabled - python

So, I'm trying to write a gzip file, actually from the net, but to simplify I wrote some very basic test.
import gzip
LINES = [b'I am a test line' for _ in range(100_000)]
f = gzip.open('./test.text.gz', 'wb')
for line in LINES:
f.write(line)
f.close()
It runs great, and I can see in Jupyter that it has created the test.txt.gz file in the directory listing. So I click on it expecting a whole host of garbage characters indicative of a binary file, like you would see in Notepad.
However, instead I get this ...
Error! test.text.gz is not UTF-8 encoded.
Saving disabled.
See console for more details
Which makes me think, oh my god, coding error, something is wrong with my encoding, my saving, can I save bytes ? Am I using the correct routines ?? And then spend 5 hours trying all combinations of code and modules.

The very simple answer to this is none of the above. This is a very misleading error message, especially when the code you've written was designed to save a binary file with a weird extension.
What this actually means is ...
I HAVE NO IDEA HOW TO DISPLAY THIS DATA ! - Yours Jupyter
So, go to your File Explorer, Finder navigate to the just saved file and open it. Voila !!
Everything worked exactly as planned, there is no error.
Hope this saves other people many hours of debugging, and please Jupyter, change your error message.

It is also possible to select the file and, instead of double-clicking to open, go to 'view' as it interprets it correctly (or well, mostly, depending on special characters, mine is in Spanish and apparently it doesn't support accents).
This way we can avoid looking for the directory where we got the file and not getting out of jupyter :)

Related

Can't write to file, even though I have permissions to do so In Python

As the tittle implies, i am having trouble writing to a certain file, even though I have permissions to do so. I'm not getting an error; It's like the lines of code are just being ignored. They're not doing anything.
f = open("C:/ProgramData/Microsoft/Windows/Start Menu/Programs/StartUp/startupfile.py", 'w')
f.write("Stuff I want to write")
f.close()
Can anyone help me? Again, there's no error message. The lines are just being flat-out ignored. And yes, I am running it with permissions through the console. Is the problem that it's in the windows directory? Or am I just missing something painfully obvious here?
EDIT:
Thanks for the help everyone, but it turns out that the problem was just Windows being incompetent. I found that the files appeared after a restart, which works anyway because I'm writing in the Startup folder.
I don't have enough reputation to make a simple comment, so don't interpret this as a solution to your problem, but did you try opening the file in the "wt" mode instead of only "w"? "w" mode opens the file in binary as seen here, so the issue could be that the program doesn't understand what you are trying to say because you want to edit it in the text mode. Again, I'm not sure if this is right and I'm a newbie myself, so don't interpret this as a solution, more like a commentary.

How to open a Unix Executable File using Python?

Sorry if this might be an easy question, but I'm trying to open a Unix Executable File using Python, but it doesn't have any file extensions attached to it. The file name looks something like 'filename_bib'. I typed this and it worked:
hdulist = open('filename_bib')
But next when I typed in hdulist.info() or hdulist.shape(), it doesn't give me anything, so I checked all its attributes and tried print(type()) and hdulist.attribute? for each attribute, but I didn't really understand any of the explanations, so I actually tried typing all of them to see what they would give me, but at some point it started giving me errors which said:
ValueError: I/O operation on closed file
so I think this may have happened when I tried using hdulist.close() or hdulist.closed(), but I don't know (1) if it was a mistake for me to try any of the attributes, (2) if it somehow changed anything from my original file, and (3) how to fix it.
I was told that this file contains bytes and that I should somehow be able to show a picture from it using Python, but this is my first time handling Unix Executable Files, and I have absolutely no idea how to start. I've handled fits and pl files before, but this is my first time trying to open something like this. I've tried looking up a bunch of things online already, but I can't find any instructions whatsoever. Please help me out if you know anything about this. I will be very grateful for any help that you could give me.
This is what it shows when I open it in Sublime:
enter image description here
As the default file access mode in python is "read only". Technically, since you have not mentioned any access mode in your command
hdulist = open('filename_bib')
file should only be for reading and nothing should have happend to the opened file.
Question:
Have you tried running it in UNIX by,
./filename_bib
What was the output?

Why truncate when we open a file in 'w' mode in python

I am going through Zed Shaw's Python Book. I am currently working on the opening and reading files chapters. I am wondering why we need to do a truncate, when we are already opening the file in a 'w' mode?
print "Opening the file..."
target = open(filename, 'w')
print "Truncating the file. Goodbye!"
target.truncate()
It's redundant since, as you noticed, opening in write mode will overwrite the file. More information at Input and Output section of Python documentation.
So Zed Shaw calls truncate() on a file that is already truncated. OK, that's pretty pointless. Why does he do that? Who knows!? Ask him!
Maybe he does it to show that the method exists? Could be, but that would be pretty daft, since I've never needed to truncate a file in my 15 years as a programmer so it has no place in a newbie book.
Maybe he does it because he thinks he has to truncate the file, and he simply isn't aware that it's pointless?
Maybe he does it intentionally to confuse newbies? That would fit with his general modus operandi, which seems to be to intentionally piss people off for absolutely no reason.
Update: The reason he does this is now clear. In later editions he lists this question as a "common question" in the chapter, and tells you to go read the docs. It's hence there to:
Teach you to read the documentation.
Understand every part of code you copy paste from somewhere before you copy-paste it.
You can debate if this is good teaching style or not, I wouldn't know.
The number of "Help I don't understand Zed Shaws book"-questions on SO had dwindled, so I can't say that it's any worse than any other book out there, which probably means it's better than many. :-)
If you would READ the questions before asking it, he answers it for you:
Extra Credit: " If you feel you do not understand this, go back
through and use the comment trick to get it squared away in your mind.
One simple English comment above each line will help you understand,
or at least let you know what you need to research more.
Write a script similar to the last exercise that uses read and argv to
read the file you just created.
There's too much repetition in this file. Use strings, formats, and
escapes to print out line1, line2, and line3 with just one
target.write() command instead of 6.
Find out why we had to pass a 'w' as an extra parameter to open. Hint:
open tries to be safe by making you explicitly say you want to write a
file.
If you open the file with 'w' mode, then do you really need the
target.truncate()?
Go read the docs for Python's open function and see if that's true." -
Zed Shaw.
He explicitly wants you to find these things out for yourself, this is why his extra credit is important.
He also EXPLICITLY states that he wants you to PAY ATTENTION TO DETAIL. Every little thing matters.
While it's not useful to truncate when opening in 'w' mode, it is useful in 'r+'. Though that's not the OP's question, I'm going to leave this here for anyone who gets lead here by Google as I did.
Let's say you open (with mode 'r+', remember there is no 'rw' mode) a 5 line indented json file and modify the json.load-ed object to be only 3 lines. If you target.seek(0) before writing the data back to the file, you will end up with 2 lines of trailing garbage. If you target.truncate() it you will not.
I know this seems obvious, but I'm here because I am fixing a bug that occurred after an object that stayed the exact same size for years... shrank because of a signing algorithm change. (What is not obvious is the unit tests I had to add to prevent this in the future. I wrote my longest docstring ever explaining why I'm testing signing with 2 ridiculously contrived algorithms.)
Hope this helps someone.
With truncate(), you can declare how much of the file you want to remove, based on where you're currently at in the file. Without parameters, truncate() acts like w, whereas w always just wipes the whole file clean. So, these two methods can act identically, but they don't necessarily.
That's just a reflection of the standard posix semantics. see man fopen(3). Python just wraps that.
When you open a file in write mode, you truncate the original (everything that was there before is deleted). Then whatever you write is added to the file. The problem is, write wants to add information from the beginning, and raises an IOError when the pointer is left at the end. For this type of writing you want to use append (open the file with the 'a+' argument).
Recently came across a scenario where I needed to create big files for test purposes. One quick way to do this is to use truncate:
with open('filename.bin', 'wb') as f:
f.truncate(1024 * 1024 * 1024) # 1GB
The file has no content, but reports to the OS the size you want and works in many testing scenarios.
Scenario:
I was making a ransomware and needed to encrypt the file, My aim is not to encrypt the complete file but that much only to corrupt it because I want it to be fast in what it does and so saving time in encrypting it all, so I decided to edit some text only.
Now
If I use write then my purpose is destroyed here because I would have to write the file a to z. Then what can I do?
well here truncate can be put in use.
Below is my code which just takes a token of last 16 digits in a file:
with open('saver.txt', 'rb+') as f:
text_len = len(f.read())
f.truncate(text_len-16)
f.close()
I open the file
Truncate only 16 characters from file which will be replaced by me later.
Notice I am using it in read only mode, If I use in write mode than File is truncated completely and it will throw error when our truncate command comes in.
Answering this question after 8.4 years. :)

python saving unicode into file

i'm having some trouble figuring out how to save unicode into a file in python. I have the following code, and if i run it in a script test.py, it should create a new file called priceinfo.txt, and write what's in price_info to the file. But i do not see the file, can anyone enlighten me on what could be the problem?
Thanks a lot!
price_info = u'it costs \u20ac 5'
f = codecs.open('priceinfo.txt','wb','utf-8')
f.write(price_info)
f.close()
I can think of several reasons:
the file gets created, but in a different directory. Be certain what the working
directory of the script is.
you don't have permission to create the file, in the directory where you want to create it.
you have some error in your Python script, and it does not get executed at all.
To find out which one it is, run the script in a command window, and check for any error output that you get.
Assuming no error messages from the program (which would be the result of forgetting to import the codecs module), are you sure you're looking in the right place? That code writes priceinfo.txt in the current working directory (IOW are you sure that you're looking inside the working directory?)

Python file read problem

file_read = open("/var/www/rajaneesh/file/_config.php", "r")
contents = file_read.read()
print contents
file_read.close()
The output is empty, but in that file all contents are there. Please help me how to do read and replace a string in __conifg.php.
Usually, when there is such kind of issues, it is very useful to start the interactive shell and analyze all commands.
For instance, it could be that the file does not exists (see comment from freiksenet) or you do not have privileges to it, or it is locked by another process.
If you execute the script in some system (like a web server, as the path could suggest), the exception could go to a log - or simply be swallowed by other components in the system.
On the contrary, if you execute it in the interactive shell, you can immediately see what the problem was, and eventually inspect the object (by using help(), dir() or the module inspect). By the way, this is also a good method for developing a script - just by tinkering around with the concept in the shell, then putting altogether.
While we are here, I strongly suggest you usage of IPython. It is an evolution of the standard shell, with powerful aids for introspection (just press tab, or a put a question mark after an object). Unfortunately in the latest weeks the site is not often not available, but there are good chances you already have it installed on your system.
I copied your code onto my own system, and changed the filename so that it works on my system. Also, I changed the indenting (putting everything at the same level) from what shows in your question. With those changes, the code worked fine.
Thus, I think it's something else specific to your system that we probably cannot solve here (easily).
Would it be possible that you don't have read access to the file you are trying to open?

Categories

Resources