Develop Reference

How to isolate titles from these image URLs?

I have a list of image urls contained in 'images'. I am trying to isolate the title from these image urls so that I can display, on the html, the image (using the whole url) and the corresponding title.
So far I have this:
titles = [image[149:199].strip() for image in images]
This gives me the stripped title in the following format (I provide two examples to show the pattern)
le_Art_Project.jpg/220px-
Rembrandt_van_Rijn_-Self-Portrait-_Google_Art_Project.jpg
and
cene_of_the_Prodigal_Son_-Google_Art_Project.jpg/220px-Rembrandt-Rembrandt_and_Saskia_in_the_Scene_of_the_Prodigal_Son-_Google_Art_Project.jpg
The bits in bold (above) are the bits I would like to remove. From the start I would like to remove everything before 220px and from the end: _-_Google_Art_Project.jpg
A newbie to python, I am struggling with syntax and furthermore as I am doing this while referring to the loop of images (list), the string manipulation is not straightforward and I am unsure of how to approach this.
The whole code for reference is below:
webscraper.py:
#app.route('/') #this is what we type into our browser to go to pages. we create these using routes
#app.route('/home')
def home():
images=imagescrape()
titles=[image[99:247].strip() for image in images]
images_titles=zip(images,titles)
return render_template('home.html',images=images,images_titles=images_titles)
What I've tried / am trying:
x = txt.strip("_-_Google_Art_Project.jpg")
Looking into strip - to get rid of the last part of the unwanted string.
I am unsure of how to combine this with getting rid of the leading string that I want to remove and also do so in the most elegant way given the structure/code I already have.
Visually, I am trying to remove the leading text as shown highlighted, as well as the last part of the string which is _-_Google_Art_Project.jpg.
Visual of HTML displayed:
UPDATE:
Based on an answer below - which is very helpful but doesn't quite perfectly solve it, I am trying this approach (without using the unquote import if possible and pure python string manipulation)
def titleextract(url):
#return unquote(url[58:url.rindex("/",58)-8].replace('_',''))
title=url[58:]
return title
The above, returns:
Rembrandt_van_Rijn_-_Self-Portrait_-_Google_Art_Project.jpg/220pxRembrandt_van_Rijn_-_Self-Portrait_-_Google_Art_Project.jpg
but I want:
Rembrandt_van_Rijn_-_Self-Portrait
or for the second title/image in the list:
Rembrandt_van_Rijn_-_Saskia_van_Uylenburgh%2C_the_Wife_of_the_Artist_-_Google_Art_Project.jpg/220px-Rembrandt_van_Rijn_-_Saskia_van_Uylenburgh%2C_the_Wife_of_the_Artist_-_Google_Art_Project.jpg
I want:
Rembrandt_van_Rijn_-_Saskia_van_Uylenburgh%2C_the_Wife_of_the_Artist

cene_of_the_Prodigal_Son_-_Google_Art_Project.jpg/220px-Rembrandt_-Rembrandt_and_Saskia_in_the_Scene_of_the_Prodigal_Son-_Google_Art_Project.jpg
You have this string and want to remove. Let's say I have this stored in x
y = x.lsplit("px-")[1]
z = x.rsplit("_Google_Art")[0]
This makes a list with 2 elements: stuff before "px-" in the string, and stuff after. We're just grabbing the stuff after, since you wanted to remove the stuff before. If "px-" isn't always in the string, then we need to find something else to split on. Then we split on something towards the end, and grab the stuff before it.
Edit: Addressing comment on how to split in that loop.. I think you are referring to this: titles=[image[149:199].strip() for image in images]
List comps are great but sometimes it's easier to just write it out. Haven't tested this but here's the idea:
titles = []
for image in images:
title = image[149:199].strip()
cleaned_left = title.lsplit("px-")[1]
cleaned_title = title.rsplit("_Google_Art")[0]
titles.append(cleaned_title)

import re # regular expressions used to match strings
from bs4 import BeautifulSoup # web scraping library
from urllib.request import urlopen # open a url connection
from urllib.parse import unquote # decode special url characters
#app.route('/')
#app.route('/home')
def home():
images=imagescrape()
# Iterate over all sources and extract the title from the URL
titles=(titleextract(src) for src in images)
# zip combines two lists into one.
# It goes through all elements and takes one element from the first
# and one element from the second list, combines them into a tuple
# and adds them to a sequence / generator.
images_titles = zip(images, titles)
return render_template('home.html', image_titles=images_titles)
def imagescrape():
result_images=[]
#html = urlopen('https://en.wikipedia.org/wiki/Prince_Harry,_Duke_of_Sussex')
html = urlopen('https://en.wikipedia.org/wiki/Rembrandt')
bs = BeautifulSoup(html, 'html.parser')
images = bs.find_all('img', {'src':re.compile('.jpg')})
for image in images:
result_images.append("https:"+image['src']+'\n') #concatenation!
return result_images
def titleextract(url):
# Extract the part of the string between the last two "/" characters
# Decode special URL characters and cut off the suffix
# Replace all "_" with spaces
return unquote(url[58:url.rindex("/", 58)-4]).replace('_', ' ')
{% for image, title in images_titles %}
<div class="card" style="width: 18rem;">
<img src="{{image}}" class="card-img-top" alt="...">
<div class="card-body">
<h5 class="card-title">{{title}}</h5>
<p class="card-text">Some quick example text to build on the card title and make up the bulk of the card's content.</p>
Go somewhere
</div>
</div>
{% endfor %}

How do I output plots from matplotlib to an html template in flask on a ubuntu server?

I am trying to set up a side project on DigitalOcean, and I am using the git framework from https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvii-deployment-on-linux to get started.
Within this framework, I have added code within one of the flask routes (/explore) in which I generate a plot with matplotlib, and I want to return this plot as an object when I render the template as the return function of this route. I don't need to save the plot if it can be sent to the template without doing so (e.g with io.BytesIO()), but I have been unable to get the syntax correct to use this approach and get the plot to render in the resulting template.
While my attempts with io.BytesIO() have been unsuccessful, if it would help to output the results with that approach, please let me know how to best utilize it, and I will attempt to run this code with the suggested changes and report the results.
Thank you in advance!
I have tried to save the file and send it to the template, as well as sending the file data via BytesIO(), but neither approach has worked for me.
Below is my attempt to save the file to the static directory and send the image to the template, but a solution that works in this environment with io.BytesIO() or similar without saving the file would be even better.
Here is the code that I added to the explore route in /app/main/routes.py to save the plot image to the static directory and return the path to the template:
new_graph_name = url_for('static', filename='tmp_png.png')
plt.savefig(new_graph_name)
return render_template('index.html', url=new_graph_name)
Here is the code that I added to the index.html template:
{% if url %}
<img src={{ url }} alt="Chart" height="42" width="42" />
{% endif %}

In terms of saving the plot and then displaying it, could you try something similar to the code the below? This has worked for me recently.
In routes.py:
#app.route("/")
def index():
new_graph_name = 'tmp_png'
plt.savefig('static/images/' + new_graph_name)
return render_template("index.html", new_graph_name=new_graph_name)
In index.html:
<img src="{{ url_for('static', filename='images/' + new_graph_name + '.png') }}"
With Bytes.IO I think I've tried something like this before:
In routes.py:
import io
from io import BytesIO
import base64
img = io.BytesIO()
fig.savefig(img)
img.seek(0)
buffer = b''.join(img)
b2 = base64.b64encode(buffer)
barplot=b2.decode('utf-8')
I cannot remember how I displayed it in the .html template but could it just be a matter of passing it as a variable?

python (flask) how to render html links from plain text input

I'm using python, flask, sqlalchemy/sqllite, jinja for my workflow.
I have a HEAP of text stored in the database and it's got paragraphs of plain text and in that text will be things like email#address.com, somedomain.com, www.somedomain.com, https://somedomain.com, https://www.somedomain.com, etc etc.
Is there a solution that exists for parsing the data as I load it and rendering it out as html links when I draw to the browser? Currently it just renders as text without links being active.
Thanks so much

A quick solution would be to use an existing library, python-textile, built for such a task.
For example:
import textile
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def index():
plain_text = 'This text contains a link to https://stackoverflow.com'
marked_up_text = textile.textile(plain_text)
return render_template('layout.html', body=marked_up_text)
layout.html
{{body | safe}}

How to display text with HTML-markup? (I use ckeditor)

I heard about the filter |safe, but if I understood correctly, that's unsafe and creates a backdoor for injections.
What are the alternatives to display full posts with formatted text?

I think when you not use the filter of |safe, then output should return as text only with html markup (not rendered as html output).
But, if you need to exclude some dangerous tags such as <script>location.reload()</script>, you need to handle it with custom templatetag filter..
I got good answer from: https://stackoverflow.com/a/699483/6396981, via BeautifulSoup.
from bs4 import BeautifulSoup
from django import template
from django.utils.html import escape
register = template.Library()
INVALID_TAGS = ['script',]
def clean_html(value):
soup = BeautifulSoup(value)
for tag in soup.findAll(True):
if tag.name in INVALID_TAGS:
# tag.hidden = True # you also can use this.
tag.replaceWith(escape(tag))
return soup.renderContents()
# clean_html('<h1>This is heading</h1> and this one is xss injection <script>location.reload()</script>')
# output:
# <html><body><h1>This is heading</h1> and this one is xss injection <script>location.reload()</script></body></html>
#register.filter
def safe_exclude(text):
# eg: {{ post.description|safe_exclude|safe }}
return clean_html(text)
Hope it usefull..

Is there a Django template filter that handles "...more" and when you click on it, it shows more of the text?

Suppose I have a huge paragraph.
I just want the top 15 words to be shown. After than, the person clicks "more" to see the rest of the stuff.

Just whipped this up, seems to do what you want, and there's no dependency on any external JS libs.
DISCLAIMER: I haven't tried this in IE, but chrome and firefox work fine.
from django import template
from django.utils.html import escape
from django.utils.safestring import mark_safe
register = template.Library()
import re
readmore_showscript = ''.join([
"this.parentNode.style.display='none';",
"this.parentNode.parentNode.getElementsByClassName('more')[0].style.display='inline';",
"return false;",
]);
#register.filter
def readmore(txt, showwords=15):
global readmore_showscript
words = re.split(r' ', escape(txt))
if len(words) <= showwords:
return txt
# wrap the more part
words.insert(showwords, '<span class="more" style="display:none;">')
words.append('</span>')
# insert the readmore part
words.insert(showwords, '<span class="readmore">... <a href="#" onclick="')
words.insert(showwords+1, readmore_showscript)
words.insert(showwords+2, '">read more</a>')
words.insert(showwords+3, '</span>')
# Wrap with <p>
words.insert(0, '<p>')
words.append('</p>')
return mark_safe(' '.join(words))
readmore.is_safe = True
To use it, just create a templatetags folder in your app, create the __init__.py file in there, and then drop this code into readmore.py.
Then at the top of any template where you want to use it, just add: {% load readmore %}
To use the filter itself:
{{ some_long_text_var|readmore:15 }}
The :15 tells how many words you want to show before the read more link.
If you want anything fancy like ajax loading of the full content, that's quite a bit different and would require a bit more infrastructure.

use truncatechars_html
refer to : https://docs.djangoproject.com/en/1.8/ref/templates/builtins/#truncatechars-html
truncatechars_html
Similar to truncatechars, except that it is aware of HTML tags. Any tags that are opened in the string and not closed before the truncation point are closed immediately after the truncation.
For example:
{{ value|truncatechars_html:9 }}
If value is "<p>Joel is a slug</p>", the output will be "<p>Joel i...</p>".
Newlines in the HTML content will be preserved.

There is truncatewords filter, although you still need a JavaScript helper to do what you described.

from django import template
from django.utils.html import escape
from django.utils.safestring import mark_safe
register = template.Library()
#register.filter
def readmore(text, cnt=250):
text, cnt = escape(text), int(cnt)
if len(text) > cnt:
first_part = text[:cnt]
link = u'%s' % _('read more')
second_part = u'%s<span class="hide">%s</span>' % (link, text[cnt:])
return mark_safe('... '.join([first_part, second_part]))
return text
readmore.is_safe = True

I rewrote an earlier answer to be cleaner and to handle string escaping properly:
#register.filter(needs_autoescape=True)
#stringfilter
def read_more(s, show_words, autoescape=True):
"""Split text after so many words, inserting a "more" link at the end.
Relies on JavaScript to react to the link being clicked and on classes
found in Bootstrap to hide elements.
"""
show_words = int(show_words)
if autoescape:
esc = conditional_escape
else:
esc = lambda x: x
words = esc(s).split()
if len(words) <= show_words:
return s
insertion = (
# The see more link...
'<span class="read-more">…'
' <a href="#">'
' <i class="fa fa-plus-square gray" title="Show All"></i>'
' </a>'
'</span>'
# The call to hide the rest...
'<span class="more hidden">'
)
# wrap the more part
words.insert(show_words, insertion)
words.append('</span>')
return mark_safe(' '.join(words))
The HTML in there assumes you're using Bootstrap and Fontawesome, but if that's not your flavor, it's easy to adapt.
For the JavaScript, assuming you're using jQuery (if you're using Bootstrap you probably are), you'll just need to add something like this:
$(".read-more").click(function(e) {
e.preventDefault();
var t = $(this);
t.parent().find('.more').removeClass('hidden');
t.addClass('hidden');
});