Is it possible to modify the tags of a downloaded MP3, without writing it to the disk?
I am using
def downloadTrack(url)
track_data = requests.get(url)
audiofile = Mp3AudioInherited(track_data.content)
audiofile.initTag()
with the class Mp3AudioInherited inherited from core.AudioFile much like mp3.Mp3AudioFile. The only signficant difference:
class Mp3AudioInherited(core.AudioFile):
...
def _read(self):
with io.BytesIO(self.data) as file_obj:
self._tag = id3.Tag()
tag_found = self._tag.parse(file_obj, self._tag_version)
...
Unfortunately _tag.parse() throws a ValueError: Invalid type: <type '_io.BytesIO'>. Isn't BytesIO a file-like object?
Thanks and regards!
No, io.BytesIO objects are not file-like (i.e., they are not interchangeable with file objects) in Python 2. Try using StringIO.StringIO to get a memory-backed file-like object in Python 2.
Related
how to convert io.BytesIo type to werkzeug.datastructures FileStrorage type Flask
I've tried to do simple conversion with casting FileStorage(File) but the return is none
It would be nice if you provided an example of your use case.
To convert an object of type BytesIO into a FileStorage object I recommend the following method.
data = bytes([1,2,3,4,5,6])
file = FileStorage(
stream=io.BytesIO(data),
filename='filename.dat',
content_type='application/octet-stream',
content_length=len(data)
)
print(file, file.read())
I have a function that's reading a content object into a pandas dataframe.
import pandas as pd
from cStringIO import StringIO, InputType
def create_df(content):
assert content, "No content was provided, can't create dataframe"
if not isinstance(content, InputType):
content = StringIO(content)
content.seek(0)
return pd.read_csv(content)
However I keep getting the error TypeError: StringIO() argument 1 must be string or buffer, not cStringIO.StringIO
I checked the incoming type of the content prior to the StringIO() conversion inside the function and it's of type str. Without the conversion I get an error that the str object does not have a seek function. Any idea whats wrong here?
You only tested for InputType, which is a cStringIO.StringIO() instance that supports reading. You appear to have the other type, OutputType, the instance created for an instance that supports writing to:
>>> import cStringIO
>>> finput = cStringIO.StringIO('Hello world!') # the input type, it has data ready to read
>>> finput
<cStringIO.StringI object at 0x1034397a0>
>>> isinstance(finput, cStringIO.InputType)
True
>>> foutput = cStringIO.StringIO() # the output type, it is ready to receive data
>>> foutput
<cStringIO.StringO object at 0x102fb99d0>
>>> isinstance(foutput, cStringIO.OutputType)
True
You'd need to test for both types, just use a tuple of the two types as the second argument to isinstance():
from cStringIO import StringIO, InputType, OutputType
if not isinstance(content, (InputType, OutputType)):
content = StringIO(content)
or, and this is the better option, test for read and seek attributes, so you can also support regular files:
if not (hasattr(content, 'read') and hasattr(content, 'seek')):
# if not a file object, assume it is a string and wrap it in an in-memory file.
content = StringIO(content)
or you could just test for strings and [buffers](https://docs.python.org/2/library/functions.html#buffer(, since those are the only two types that StringIO() can support:
if isinstance(content, (str, buffer)):
# wrap strings into an in-memory file
content = StringIO(content)
This has the added bonus that any other file object in the Python library, including compressed files and tempfile.SpooledTemporaryFile() and io.BytesIO() will also be accepted and work.
This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed 25 days ago.
In Python I'm getting an error:
Exception: (<type 'exceptions.AttributeError'>,
AttributeError("'str' object has no attribute 'read'",), <traceback object at 0x1543ab8>)
Given python code:
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.load(jsonStr)['data']['children']
What does this error mean and what did I do to cause it?
The problem is that for json.load you should pass a file like object with a read function defined. So either you use json.load(response) or json.loads(response.read()).
Ok, this is an old thread but.
I had a same issue, my problem was I used json.load instead of json.loads
This way, json has no problem with loading any kind of dictionary.
Official documentation
json.load - Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.
json.loads - Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object using this conversion table.
You need to open the file first. This doesn't work:
json_file = json.load('test.json')
But this works:
f = open('test.json')
json_file = json.load(f)
If you get a python error like this:
AttributeError: 'str' object has no attribute 'some_method'
You probably poisoned your object accidentally by overwriting your object with a string.
How to reproduce this error in python with a few lines of code:
#!/usr/bin/env python
import json
def foobar(json):
msg = json.loads(json)
foobar('{"batman": "yes"}')
Run it, which prints:
AttributeError: 'str' object has no attribute 'loads'
But change the name of the variablename, and it works fine:
#!/usr/bin/env python
import json
def foobar(jsonstring):
msg = json.loads(jsonstring)
foobar('{"batman": "yes"}')
This error is caused when you tried to run a method within a string. String has a few methods, but not the one you are invoking. So stop trying to invoke a method which String does not define and start looking for where you poisoned your object.
AttributeError("'str' object has no attribute 'read'",)
This means exactly what it says: something tried to find a .read attribute on the object that you gave it, and you gave it an object of type str (i.e., you gave it a string).
The error occurred here:
json.load(jsonStr)['data']['children']
Well, you aren't looking for read anywhere, so it must happen in the json.load function that you called (as indicated by the full traceback). That is because json.load is trying to .read the thing that you gave it, but you gave it jsonStr, which currently names a string (which you created by calling .read on the response).
Solution: don't call .read yourself; the function will do this, and is expecting you to give it the response directly so that it can do so.
You could also have figured this out by reading the built-in Python documentation for the function (try help(json.load), or for the entire module (try help(json)), or by checking the documentation for those functions on http://docs.python.org .
Instead of json.load() use json.loads() and it would work:
ex:
import json
from json import dumps
strinjJson = '{"event_type": "affected_element_added"}'
data = json.loads(strinjJson)
print(data)
So, don't use json.load(data.read()) use json.loads(data.read()):
def findMailOfDev(fileName):
file=open(fileName,'r')
data=file.read();
data=json.loads(data)
return data['mail']
use json.loads() function , put the s after that ... just a mistake btw i just realized after i searched error
def getEntries (self, sub):
url = 'http://www.reddit.com/'
if (sub != ''):
url += 'r/' + sub
request = urllib2.Request (url +
'.json', None, {'User-Agent' : 'Reddit desktop client by /user/RobinJ1995/'})
response = urllib2.urlopen (request)
jsonStr = response.read()
return json.loads(jsonStr)['data']['children']
try this
Open the file as a text file first
json_data = open("data.json", "r")
Now load it to dict
dict_data = json.load(json_data)
If you need to convert string to json. Then use loads() method instead of load(). load() function uses to load data from a file so used loads() to convert string to json object.
j_obj = json.loads('["label" : "data"]')
I'd like to figure out how to embed binary content in a python script. For instance, I don't want to have any external files around (images, sound, ... ), I want all this content living inside of my python scripts.
Little example to clarify, let's say I got this small snippet:
from StringIO import StringIO
from PIL import Image, ImageFilter
embedded_resource = StringIO(open("Lenna.png", "rb").read())
im = Image.open(embedded_resource)
im.show()
im_sharp = im.filter(ImageFilter.SHARPEN)
im_sharp.show()
As you can see, the example is reading the external file 'Lenna.png'
Question
How to proceed to embed "Lenna.png" as a resource (variable) into my python script. What's the fastest way to achieve this simple task using python?
You might find the following class rather useful for embedding resources in your program. To use it, call the package method with paths to the files that you want to embed. The class will print out a DATA attribute that should be used to replace the one already found in the class. If you want to add files to your pre-built data, use the add method instead. To use the class in your program, make calls to the load method using context manager syntax. The returned value is a Path object that can be used as a filename argument to other functions or for the purpose of directly loading the reconstituted file. See this SMTP Client for example usage.
import base64
import contextlib
import pathlib
import pickle
import pickletools
import sys
import zlib
class Resource:
"""Manager for resources that would normally be held externally."""
WIDTH = 76
__CACHE = None
DATA = b''
#classmethod
def package(cls, *paths):
"""Creates a resource string to be copied into the class."""
cls.__generate_data(paths, {})
#classmethod
def add(cls, *paths):
"""Include paths in the pre-generated DATA block up above."""
cls.__preload()
cls.__generate_data(paths, cls.__CACHE.copy())
#classmethod
def __generate_data(cls, paths, buffer):
"""Load paths into buffer and output DATA code for the class."""
for path in map(pathlib.Path, paths):
if not path.is_file():
raise ValueError('{!r} is not a file'.format(path))
key = path.name
if key in buffer:
raise KeyError('{!r} has already been included'.format(key))
with path.open('rb') as file:
buffer[key] = file.read()
pickled = pickle.dumps(buffer, pickle.HIGHEST_PROTOCOL)
optimized = pickletools.optimize(pickled)
compressed = zlib.compress(optimized, zlib.Z_BEST_COMPRESSION)
encoded = base64.b85encode(compressed)
cls.__print(" DATA = b'''")
for offset in range(0, len(encoded), cls.WIDTH):
cls.__print("\\\n" + encoded[
slice(offset, offset + cls.WIDTH)].decode('ascii'))
cls.__print("'''")
#staticmethod
def __print(line):
"""Provides alternative printing interface for simplicity."""
sys.stdout.write(line)
sys.stdout.flush()
#classmethod
#contextlib.contextmanager
def load(cls, name, delete=True):
"""Dynamically loads resources and makes them usable while needed."""
cls.__preload()
if name not in cls.__CACHE:
raise KeyError('{!r} cannot be found'.format(name))
path = pathlib.Path(name)
with path.open('wb') as file:
file.write(cls.__CACHE[name])
yield path
if delete:
path.unlink()
#classmethod
def __preload(cls):
"""Warm up the cache if it does not exist in a ready state yet."""
if cls.__CACHE is None:
decoded = base64.b85decode(cls.DATA)
decompressed = zlib.decompress(decoded)
cls.__CACHE = pickle.loads(decompressed)
def __init__(self):
"""Creates an error explaining class was used improperly."""
raise NotImplementedError('class was not designed for instantiation')
The best way to go about this is converting your picture into a python string, and have it in a separate file called something like resources.py, then you simply parse it.
If you are looking to embed the whole thing inside a single binary, then you're looking at something like py2exe. Here is an example embedding external files
In the first scenario, you could even use base64 to (de)code the picture, something like this:
import base64
file = open('yourImage.png');
encoded = base64.b64encode(file.read())
data = base64.b64decode(encoded) # Don't forget to file.close() !
In my code, I have a load_dataset function that reads a text file and does some processing. Recently I thought about adding support to file-like objects, and I wondered over the best approach to this. Currently I have two implementations in mind:
First, type checking:
if isinstance(inputelement, basestring):
# open file, processing etc
# or
# elif hasattr(inputelement, "read"):
elif isinstance(inputelement, file):
# Do something else
Alternatively, two different arguments:
def load_dataset(filename=None, stream=None):
if filename is not None and stream is None:
# open file etc
elif stream is not None and filename is None:
# do something else
Both solutions however don't convince me too much, especially the second as I see way too many pitfalls.
What is the cleanest (and most Pythonic) way to accept a file-like object or string to a function that does text reading?
One way of having either a file name or a file-like object as argument is the implementation of a context manager that can handle both. An implementation can be found here, I quote for the sake of a self contained answer:
class open_filename(object):
"""Context manager that opens a filename and closes it on exit, but does
nothing for file-like objects.
"""
def __init__(self, filename, *args, **kwargs):
self.closing = kwargs.pop('closing', False)
if isinstance(filename, basestring):
self.fh = open(filename, *args, **kwargs)
self.closing = True
else:
self.fh = filename
def __enter__(self):
return self.fh
def __exit__(self, exc_type, exc_val, exc_tb):
if self.closing:
self.fh.close()
return False
Possible usage then:
def load_dataset(file_):
with open_filename(file_, "r") as f:
# process here, read only if the file_ is a string
Don't accept both files and strings. If you're going to accept file-like objects, then it means you won't check the type, just call the required methods on the actual parameter (read, write, etc.). If you're going to accept strings, then you're going to end up open-ing files, which means you won't be able to mock the parameters. So I'd say accept files, let the caller pass you a file-like object, and don't check the type.
I'm using a context manager wrapper. When it's a filename (str), close the file on exit.
#contextmanager
def fopen(filein, *args, **kwargs):
if isinstance(filein, str): # filename
with open(filein, *args, **kwargs) as f:
yield f
else: # file-like object
yield filein
Then you can use it like:
with fopen(filename_or_fileobj) as f:
# do sth. with f
Python follows duck typing, you may check that this is file object by function that you need from object. For example hasattr(obj, 'read') against isinstance(inputelement, file).
For converting string to file objects you may also use such construction:
if not hasattr(obj, 'read'):
obj = StringIO(str(obj))
After this code you will safely be able to use obj as a file.