Develop Reference

How to create and or edit a page with pyWikiBot

The MediaWiki API has an edit function which is available within pywikibot.
According to https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.site.html
the function is called with a page parameter:
editpage(page, summary=None, minor=True, notminor=False, bot=True, recreate=True, createonly=False, nocreate=False, watch=None, **kwargs)[source]¶
A page needs a source to be constructed. I could not find an example for this.
E.g.: How to add something to edit summary when using Pywikibot?
has just one line as the accepted answer and does not use site or page. I am confused.
What is the proper approach do create and or edit a page with pyWikiBot directly using python code? (not script ...)
https://github.com/wikimedia/pywikibot/blob/master/pywikibot/page/init.py#L2328
has the constructor:
"""Page: A MediaWiki page."""
#deprecated_args(defaultNamespace='ns', insite=None)
def __init__(self, source, title='', ns=0):
"""Instantiate a Page object."""
if isinstance(source, pywikibot.site.BaseSite):
if not title:
raise ValueError('Title must be specified and not empty '
'if source is a Site.')
super(Page, self).__init__(source, title, ns)
Which is unfortunately not properly documented and part of the 6000 line init.py file having all the classes.
When trying
newPage=Page(site,pageTitle)
newPage.text=pageContent
I get
AttributeError: APISite instance has no attribute 'edit'
site.edit(newPage,'summary')

The following code works:
from pywikibot.page import Page
newPage=Page(site,pageTitle)
newPage.text=pageContent
newPage.save("summary")
see also https://www.mediawiki.org/wiki/Manual:Pywikibot/Create_your_own_script
It's just unfortunate that the documentation at https://pypi.org/project/pywikibot/ is not using standard python documentation tools.

With Python Kubernetes client, how to replicate `kubectl create -f` generally?

My Bash script using kubectl create/apply -f ... to deploy lots of Kubernetes resources has grown too large for Bash. I'm converting it to Python using the PyPI kubernetes package.
Is there a generic way to create resources given the YAML manifest? Otherwise, the only way I can see to do it would be to create and maintain a mapping from Kind to API method create_namespaced_<kind>. That seems tedious and error prone to me.
Update: I'm deploying many (10-20) resources to many (10+) GKE clusters.

Update in the year 2020, for anyone still interested in this (since the docs for the python library is mostly empty).
At the end of 2018 this pull request has been merged,
so it's now possible to do:
from kubernetes import client, config
from kubernetes import utils
config.load_kube_config()
api = client.ApiClient()
file_path = ... # A path to a deployment file
namespace = 'default'
utils.create_from_yaml(api, file_path, namespace=namespace)
EDIT: from a request in a comment, a snippet for skipping the python error if the deployment already exists
from kubernetes import client, config
from kubernetes import utils
config.load_kube_config()
api = client.ApiClient()
def skip_if_already_exists(e):
import json
# found in https://github.com/kubernetes-client/python/blob/master/kubernetes/utils/create_from_yaml.py#L165
info = json.loads(e.api_exceptions[0].body)
if info.get('reason').lower() == 'alreadyexists':
pass
else
raise e
file_path = ... # A path to a deployment file
namespace = 'default'
try:
utils.create_from_yaml(api, file_path, namespace=namespace)
except utils.FailToCreateError as e:
skip_if_already_exists(e)

I have written a following piece of code to achieve the functionality of creating k8s resources from its json/yaml file:
def create_from_yaml(yaml_file):
"""
:param yaml_file:
:return:
"""
yaml_object = yaml.loads(common.load_file(yaml_file))
group, _, version = yaml_object["apiVersion"].partition("/")
if version == "":
version = group
group = "core"
group = "".join(group.split(".k8s.io,1"))
func_to_call = "{0}{1}Api".format(group.capitalize(), version.capitalize())
k8s_api = getattr(client, func_to_call)()
kind = yaml_object["kind"]
kind = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', kind)
kind = re.sub('([a-z0-9])([A-Z])', r'\1_\2', kind).lower()
if "namespace" in yaml_object["metadata"]:
namespace = yaml_object["metadata"]["namespace"]
else:
namespace = "default"
try:
if hasattr(k8s_api, "create_namespaced_{0}".format(kind)):
resp = getattr(k8s_api, "create_namespaced_{0}".format(kind))(
body=yaml_object, namespace=namespace)
else:
resp = getattr(k8s_api, "create_{0}".format(kind))(
body=yaml_object)
except Exception as e:
raise e
print("{0} created. status='{1}'".format(kind, str(resp.status)))
return k8s_api
In above function, If you provide any object yaml/json file, it will automatically pick up the API type and object type and create the object like statefulset, deployment, service etc.
PS: The above code doesn't handler multiple kubernetes resources in one file, so you should have only one object per yaml file.

I see what you are looking for. This is possible with other k8s clients available in other languages. Here is an example in java. Unfortunately the python client library does not support that functionality yet. I opened a new feature request requesting the same and you can either choose to track it or contribute yourself :). Here is the link for the issue on GitHub.
The other way to still do what you are trying to do is to use java/golang client and put your code in a docker container.

Azure CosmosDB, Python - Replace a document: document_link?

I have a JSON document in my database that I want to modify frequently from my python program, once every 25 seconds. I know how to upload a document to the database and read a document from it, but I do not know how to modify/replace a document.
This link shows the functions offered in the python module. I see the ReplaceDocument function, but it takes in a document-link. Though how can I get the document link? Where am I suppose to look for this information?
Thanks.

It sounds like you had resolved it. Just as summary, the code below.
# Query a document
query = { 'query': 'SELECT * FROM <collection name> ....'}
docs = client.QueryDocuments(coll_link, query)
doc = list(docs)[0]
# Get the document link from attribute `_self`
doc_link = doc['_self']
# Modify the document
.....
# Replace the document via document link
client.ReplaceDocument(doc_link, doc)

April 2020
If you are reading MS Azure's Quickstart guide and following a supporting git repo, note that there might be some differences.
For example,
from azure.cosmos import exceptions, CosmosClient, PartitionKey
endpoint = 'endpoint'
key = 'key'
db_name = 'cosmos-db-name'
container_name = 'container-name'
client = CosmosClient(endpoint, key)
db = client.create_database_if_not_exists(id=db_name)
container = db.create_container_if_not_exists(id=container_name, partition_key=PartitionKey(path="/.."), offer_throughput=456
...
# Replace item
container.replace_item(doc_link, doc)
When it comes to doc_link and doc, in the above case, I encountered an error when I used doc['_self']. By using the primary key of the doc, the doc is updated.

How do I apply a monkeypatch to GAE?

Can you tell me how I apply this patch to google app engine as in where to put it? Thank you
def _user_init(self, email=None, _auth_domain=None,
_user_id=None, federated_identity=None, federated_provider=None):
if not _auth_domain:
_auth_domain = os.environ.get('AUTH_DOMAIN')
assert _auth_domain
if email is None and federated_identity is None:
email = os.environ.get('USER_EMAIL', email)
_user_id = os.environ.get('USER_ID', _user_id)
federated_identity = os.environ.get('FEDERATED_IDENTITY',
federated_identity)
federated_provider = os.environ.get('FEDERATED_PROVIDER',
federated_provider)
if not email and not federated_identity:
raise UserNotFoundError
self.__email = email
self.__federated_identity = federated_identity
self.__federated_provider = federated_provider
self.__auth_domain = _auth_domain
self.__user_id = _user_id or None
users.User.__init__ = _user_init

Just use it as-is: Put that code in a module that gets imported before you use the relevant User module or datastore functionality. I included the relevant line to patch the code (the last line) with the patch itself.

Overriding the constructor like this is not safe. If the internal implementation of the Users API changes in production, your application could break without warning.
What are you trying to accomplish here? I don't see any custom logic; it looks like you've just copied the constructor from the SDK verbatim. If you need to add custom logic, try subclassing UserProperty and/or wrapping the users API calls instead.

I think, this belongs to some application as a grep within the appengine sdk, for 'federated_identity' does not result any clues. BTW, you should be doing the same. Grep (or WinGrep) for terms like 'federated' to see if this partial patch can be applied to any source.
Thanks for the updated link. The patch can be added to the file google/appengine/api/users.py
You might just need to add the last line: users.User.__init__ = _user_init
I could figure this out after checking the latest code in the svn.

How Do I Suppress or Disable Warnings in reSTructuredText?

I'm working on a CMS in Python that uses reStructuredText (via docutils) to format content. Alot of my content is imported from other sources and usually comes in the form of unformatted text documents. reST works great for this because it makes everything look pretty sane by default.
One problem I am having, however, is that I get warnings dumped to stderr on my webserver and injected into my page content. For example, I get warnings like the following on my web page:
System Message: WARNING/2 (, line 296); backlink
My question is: How do I suppress, disable, or otherwise re-direct these warnings?
Ideally, I'd love to write these out to a log file, but if someone can just tell me how to turn off the warnings from being injected into my content then that would be perfect.
The code that's responsible for parsing the reST into HTML:
from docutils import core
import reSTpygments
def reST2HTML( str ):
parts = core.publish_parts(
source = str,
writer_name = 'html')
return parts['body_pre_docinfo'] + parts['fragment']

def reST2HTML( str ):
parts = core.publish_parts(
source = str,
writer_name = 'html',
settings_overrides={'report_level':'quiet'},
)
return parts['body_pre_docinfo'] + parts['fragment']

It seems the report_level accept string is an old version. Now, the below is work for me.
import docutils.core
import docutils.utils
from pathlib import Path
shut_up_level = docutils.utils.Reporter.SEVERE_LEVEL + 1
docutils.core.publish_file(
source_path=Path(...), destination_path=Path(...),
settings_overrides={'report_level': shut_up_level},
writer_name='html')
about level
# docutils.utils.__init__.py
class Reporter(object):
# system message level constants:
(DEBUG_LEVEL,
INFO_LEVEL,
WARNING_LEVEL,
ERROR_LEVEL,
SEVERE_LEVEL) = range(5)
...
def system_message(self, level, message, *children, **kwargs):
...
if self.stream and (level >= self.report_level # self.report_level was set by you. (for example, shut_up_level)
or self.debug_flag and level == self.DEBUG_LEVEL
or level >= self.halt_level):
self.stream.write(msg.astext() + '\n')
...
return msg
According to the above code, you know that you can assign the self.report_level (i.e. settings_overrides={'report_level': ...}) let the warning not show.
and I set it to SERVER_LEVEL+1, so it will not show any error. (you can set it according to your demand.)