Develop Reference

Is there any option to check whether the URL is working or not? before passing it to the next function in scrapy , Python

I have 2 function blocks in my scraper
1.Parse
2.Parse_info
In the 1st block, I got the list of URLs.
Some of the URLs are working (they already have the 'https://www.example.com/' part)
Rest URLs are not working (they do not have the 'https://www.example.com/' part)
So before passing the URL to 2nd block i.e. parse_info; I want to validate the URL
and If it is not working I want to edit it and add the required part ('https://www.example.com/' part).

You could leverage the requests module and get the status code of the website - a guide to doing that is here.
Similarly, if you're just trying to validate whether the URL contains a specific portion i.e the 'https://www.example.com/', you can perform a regex query and do that.

My interpretation from your question is that you have a list of URLs, some of which have an absolute address like 'https://www.example.com/xyz' and some only have a relative reference like '/xyz' that belongs to the 'https://www.example.com' site.
If that is the case, you can use 'urljoin' to rationalise each of the URLs, for example:
>>> from urllib.parse import urljoin
>>> url = 'https://www.example.com/xyz'
>>> print(urljoin('https://www.example.com', url))
https://www.example.com/xyz
>>> url = '/xyz'
>>> print(urljoin('https://www.example.com', url))
https://www.example.com/xyz

QueryString in Django

I need one small help. I have one django application already with me and everything is working fine. Now requirement is without making change in existing functions we want to pass urls parameter as a query string
path('my-app/', include('myapp.urls', namespace='myapp')),
# I added this line because I want to use dcid as a querystring
url(r'^my-app/(?:(?P<dcid>\w+)/)',include('myapp.urls', namespace='myapp')),
and my function is already written like this
def orders_b2b_open_list_pageable(request):
We don't want to change in above functions but we want dcid in query string. How can we achieve that
I am doing this but when requesting like this
http://localhost:8002/my-app/1/orders/b2b/open/pageable/
I am getting following error
Thank You in advance

There's no need to parse it in your urls.py, that would change your function signiture.
You can handle it in your view, by pulling the querystring value from the request:
def orders_b2b_open_list_pageable(request):
dcid = request.GET.get('dcid', None)
...

How to get the absolute string from a name-spaced Django URL?

You know how in a Django app, you can redirect to a different view using a simple function in your views? Its usually something like this:
return redirect('myapp:my_url')
The above redirect would redirect me to an absolute URL which could be something like this:
https://example.com/my-url/
Now, my question is, how can I get the https://example.com/my-url/ as a string using a function in my view? I don't want to redirect, I just want to get it, and save it in my variable.
So, by doing something like this:
print my_function('myapp:my_url')
We would get an output in the terminal like so:
https://example.com/my-url/
Any help? Thanks!

You can get the "path" element of the URL (ie /my-url/) using the reverse function you have already mentioned.
The domain of the website can be added using request.build_absolute_uri().
print(request.build_absolute_uri(reverse('myapp:my_url'))
Note that I'm using the Python 3 print syntax for this example.

Django: how to pass an entire URL as a parameter inside the app URL?

I am trying to pass an entire, full length/structure URL within my app as an argument inside the app's URL. For example, I want to be able to do something like:
myapp.com/https://www.youtube.com/watch?v=dQw4w9WgXcQ so that I can take the URL entered after my app's home page and store it. However, I think the app is getting confused when the URL pasted has fragments and query arguments (ie: contains # and/or ?) My urls.py looks like this:
url(r'^(?P<url_param>[a-zA-Z0-9_.-/:?=#]*)/$', views.anywebsiteentered, name='anywebsiteentered')
When I try to write a view that looks like below to take the inputted URL and save it inside a model object, I always get the URL truncated before the query and fragment characters, what can I do so that my application picks up the entire URL string?
def anywebsiteentered(request, url_param = 'url_param'):
UrlBlob.objects.create(fullurl=url_param)
For example, the above object created if my app is at myapp.com/https://www.youtube.com/watch?v=dQw4w9WgXcQ only returns https://www.youtube.com/watch and not the query part of the URL. I suspect it is something I am doing with the passing of the URL because when I create this model object manually inside the python-django shell there is no problems at all.
Thanks for any help and hints. I really appreciate it.

If you needed to use some url in "path" area of another url you should escape it's special characters. For example use %3F instead "?". It's called url escaping.
For your purpose would be better pass url as argument like:
myapp.com/?url=http://www.youtube.com/watch?v=dQw4w9WgXcQ
— browser will do necessary escaping in this case.

You can get the query string from request.META:
def anywebsiteentered(request, url_param='url_param'):
full_url = url_param
query_string = request.META['QUERY_STRING']
if query_string:
full_url += u'?' + query_string
UrlBlob.objects.create(fullurl=full_url)

Catch full URL in python

I'm developing application using Bottle. How do I get full query string when I get a GET Request.
I dont want to catch using individual parameters like:
param_a = request.GET.get("a","")
as I dont want to fix number of parameters in the URL.
How to get full query string of requested url

You can use the attribute request.query_string to get the whole query string.

Use request.query or request.query.getall(key) if you have more than one value for a single key.
For eg., request.query.a will return you the param_a you wanted. request.query.b will return the parameter for b and so on.
If you only want the query string alone, you can use #halex's answer.