Develop Reference

How do I access Online Archive mailbox using Python?

SOLVED ! :)
I used the following to move my mail from somewhere in my inbox into online archives with some important help mentioned below :
import win32com
import os
import sys
outlook = win32com.client.Dispatch('outlook.application')
mapi = outlook.GetNamespace("MAPI")
src = mapi.GetDefaultFolder(6).Folders["tobemoved"]
target = mapi.Folders["Online Archive - XXX"].Folders['Archive']
messages = src.Items
i = range(messages.count, 1, -1)
for x in i:
print(x)
messages(x).Move(target)
`
I have additional folder called
'Online-Archive-Same email address as "inbox" email '
that i currently can't locate it tried to use this link to figure out the enumeration of it . but no luck ..
as i must free up some disk space ill appreciate any help given.
P.S
tried the conventional way - with outlook struggling with connection issues and 22k email to be moved to be archived outlook just giving up on me :) feel free to advise anything that can resolve this issue.

You can access the Office 365 Online Archive folders like this:
Replace the example email with the exact email address you see in outlook.
import win32com.client
import win32com
app = win32com.client.gencache.EnsureDispatch("Outlook.Application")
outlook = app.GetNamespace("MAPI")
outlook_folder = outlook.Folders['Online Archive - Example#email.com'].Folders['Inbox']
item_count = outlook_folder.Items.Count
print(item_count)
180923

On the low (Extended MAPI) level (C++ or Delphi only), Online Archive is just another delegate Exchange mailbox. The only way to distinguish an archive mailbox from yet another delegate mailbox owned by some Exchange user is by reading PR_PROFILE_ALTERNATE_STORE_TYPE property in the archive store profile section - retrieve the store entry id (PR_ENTRYID), then find the matching row in the session stores table (IMAPISession::GetMsgStoresTable). For the matching row (use IMAPISession::CompareEntryIDs), retrieve PR_PROVIDER_UID property. Use its value to call IMAPISession.OpenProfileSection. Read PR_PROFILE_ALTERNATE_STORE_TYPE property from the IProfSect object and check if its value is "Archive" (unlike the store name, is not localized).
If Extended MAPI in C++ or Delphi is not an option, you can either
Try to find a matching store in the Namespace.Stores collection with the name starting with "Online Archive - " and the SMTP address of the user. Since that prefix is locale specific, that is not something I would use in production code.
Use Redemption (I am its author) - it exposes RDOExchangeMailboxStore.IsArchive property. If the archive store is not already opened in Outlook, you can also use RDOSession.GetArchiveMailbox. In VB script:
set rSession = CreateObject("Redemption.RDOSession")
rSession.MAPIOBJECT = Application.Session.MAPIOBJECT
userAddress = rSession.CurrentUser.SMTPAddress
set store = GetOpenArchiveMailboxForUser(userAddress)
if not store is Nothing Then
MsgBox "Found archive store for " & userAddress
Else
MsgBox "Could not find archive store for " & userAddress
End If
function GetOpenArchiveMailboxForUser(SmtpAddress)
set GetOpenArchiveMailboxForUser = Nothing
for each store in rSession.Stores
if TypeName(store) = "RDOExchangeMailboxStore" Then
Debug.Print store.Name & " - " & store.Owner.SMTPAddress & " - " & store.IsArchive
if store.IsArchive and LCase(store.Owner.SMTPAddress) = LCase(SmtpAddress) Then
set GetOpenArchiveMailboxForUser = store
exit for
End If
End If
next
end function

Jira Python - all issues from all projects

I have some code to connect to my company’s instance of the JIRA server via Jira’s Python Client. When i do a count on all the projects available in the server, I see close to 4300 projects, but when I loop through the projects to get a list of all the issues in all the projects, I cannot get more than 149 projects. There are about 100 issues in each project so I’d expect a total of 4300 projects and 430000 issues.
But I’m getting 149 projects and the loop exists with ~27000 issues. Has anyone faced this problem? Here’s the code I’m using:
import jira.client
from jira.client import JIRA
import re
import pandas as pd
options = {"server": "https://horizon.xxxxxx.com/issues",'verify':False}
jira = JIRA(options, basic_auth=('user','pwd'))
projects = jira.projects()
allissues = []
allprojects=[]
block_size=50
for projectA in projects:
block_num=0
while True:
start_idx=block_num*block_size
issues=jira.search_issues(f'project = {projectA.key}', start_idx, block_size)
if len(issues)== 0:
break
block_num+=1
for issue in issues:
allissues.append(issue)
allprojects.append(projectA.key)

I don't the setup to currently validate my code, but the code flow should be looking like this,
First import all your necessary libraries and get all the projects, then
projectsList=[]
issuesList=[]
for proj in projects:
while true:
startIndex=block_num*block_size
issues=jira.search_issues(f'project = {projectA.key}', start_idx
if len(issues)== 0:
break
block_num+=1
issueList.extend(issues)
projectsList.append(proj)
Few pointers:
1. Block size is not needed in search issues as argument because by default value is already set at 50.
2. You don't have to add project name every time you loop your issue
If you there are still issues with the results, then do provide any additional information such as any error info, debug logs (add some print statements) etc., that might help in helping you.

Adding quotes to the projects below solved:
import jira.client
from jira.client import JIRA
import re
import pandas as pd
options = {"server": "https://horizon.xxxxxxx.com/issues2",'verify':False}
jira = JIRA(options, basic_auth=('user','pwd'))
issues2=pd.DataFrame()
projects = jira.projects()
block_size=50
for projectA in projects:
block_num=0
while True:
start_idx=block_num*block_size
issues=jira.search_issues(f'project = "{projectA.key}"', start_idx, block_size)
if len(issues)== 0:
break
block_num+=1
allissues = []
allissues.extend(issues)
for allissue in allissues:
d={
'projectname': projectA.key,
'key':allissue .key,
'assignee':allissue .fields.assignee,
'creator':allissue .fields.creator,
'reporter':allissue .fields.reporter,
'created':allissue .fields.created,
'components':allissue .fields.components,
'description':allissue .fields.description,
'summary':allissue .fields.summary,
'fixVersion':allissue .fields.fixVersions,
'subtask':allissue .fields.issuetype.name,
'priority':allissue .fields.priority.name,
'resolution':allissue .fields.resolution,
'resolution.date':allissue .fields.resolutiondate,
'status.name':allissue .fields.status.name,
'statusdescription':allissue .fields.status.description,
'updated':allissue .fields.updated,
'versions':allissue .fields.versions,
'watches':allissue .fields.watches,
'storypoints':allissue .fields.customfield_10002
}
issues2=issues2.append(d, ignore_index=True)
issues2.head()

Does your user have Browse Permission on all the projects?
Are there Issue Security Schemes not allowing you to see some issues?

With Python Kubernetes client, how to replicate `kubectl create -f` generally?

My Bash script using kubectl create/apply -f ... to deploy lots of Kubernetes resources has grown too large for Bash. I'm converting it to Python using the PyPI kubernetes package.
Is there a generic way to create resources given the YAML manifest? Otherwise, the only way I can see to do it would be to create and maintain a mapping from Kind to API method create_namespaced_<kind>. That seems tedious and error prone to me.
Update: I'm deploying many (10-20) resources to many (10+) GKE clusters.

Update in the year 2020, for anyone still interested in this (since the docs for the python library is mostly empty).
At the end of 2018 this pull request has been merged,
so it's now possible to do:
from kubernetes import client, config
from kubernetes import utils
config.load_kube_config()
api = client.ApiClient()
file_path = ... # A path to a deployment file
namespace = 'default'
utils.create_from_yaml(api, file_path, namespace=namespace)
EDIT: from a request in a comment, a snippet for skipping the python error if the deployment already exists
from kubernetes import client, config
from kubernetes import utils
config.load_kube_config()
api = client.ApiClient()
def skip_if_already_exists(e):
import json
# found in https://github.com/kubernetes-client/python/blob/master/kubernetes/utils/create_from_yaml.py#L165
info = json.loads(e.api_exceptions[0].body)
if info.get('reason').lower() == 'alreadyexists':
pass
else
raise e
file_path = ... # A path to a deployment file
namespace = 'default'
try:
utils.create_from_yaml(api, file_path, namespace=namespace)
except utils.FailToCreateError as e:
skip_if_already_exists(e)

I have written a following piece of code to achieve the functionality of creating k8s resources from its json/yaml file:
def create_from_yaml(yaml_file):
"""
:param yaml_file:
:return:
"""
yaml_object = yaml.loads(common.load_file(yaml_file))
group, _, version = yaml_object["apiVersion"].partition("/")
if version == "":
version = group
group = "core"
group = "".join(group.split(".k8s.io,1"))
func_to_call = "{0}{1}Api".format(group.capitalize(), version.capitalize())
k8s_api = getattr(client, func_to_call)()
kind = yaml_object["kind"]
kind = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', kind)
kind = re.sub('([a-z0-9])([A-Z])', r'\1_\2', kind).lower()
if "namespace" in yaml_object["metadata"]:
namespace = yaml_object["metadata"]["namespace"]
else:
namespace = "default"
try:
if hasattr(k8s_api, "create_namespaced_{0}".format(kind)):
resp = getattr(k8s_api, "create_namespaced_{0}".format(kind))(
body=yaml_object, namespace=namespace)
else:
resp = getattr(k8s_api, "create_{0}".format(kind))(
body=yaml_object)
except Exception as e:
raise e
print("{0} created. status='{1}'".format(kind, str(resp.status)))
return k8s_api
In above function, If you provide any object yaml/json file, it will automatically pick up the API type and object type and create the object like statefulset, deployment, service etc.
PS: The above code doesn't handler multiple kubernetes resources in one file, so you should have only one object per yaml file.

I see what you are looking for. This is possible with other k8s clients available in other languages. Here is an example in java. Unfortunately the python client library does not support that functionality yet. I opened a new feature request requesting the same and you can either choose to track it or contribute yourself :). Here is the link for the issue on GitHub.
The other way to still do what you are trying to do is to use java/golang client and put your code in a docker container.

Pulling data from external Active Directory in Django

I currently have an app that uses a MySQL backend and I have a client that has profile information for its users stored on it but they also have Active Directory and want to know if I can pull information from that as well to retrieve information from there for a specific profile. I know that you can configure Django for multiple SQL database connections or substitute the authentication backend to Active Directory.
https://docs.djangoproject.com/en/1.9/topics/db/multi-db/
https://pythonhosted.org/django-auth-ldap/
But what I was wondering if I could do both MySQL and Active Directory at the same time or do I just need to connect to the Active Directory externally and retrieve the information that way?
Is this do-able and if so which would be the best way to approach it?

I have a similar situation with a Django site I administer. Here's the Django App I use:
https://github.com/etianen/django-python3-ldap
It allows me to use PostgreSQL for my database, and pull the user metadata I need out of Active Directory and into the user record by mapping fields. It is the best method I have found after several false starts.
If you're just looking to pull data from Active Directory and not into the Django user, here's the package and a code sample I've found to work:
Python 3 package: git+https://github.com/rbarrois/python-ldap.git#py3
Example, which you can modify to work with Django's ORM:
"""
This code provide an example of how to connect to LDAP (specifically, Active Directory)
using Python 3.
Requires python-ldap3, available via the following command:
pip install git+https://github.com/rbarrois/python-ldap.git#py3
"""
import ldap
LDAP_URI = 'ldap://ldap.server.com'
LDAP_DN = 'dc=server,dc=com'
LDAP_USERNAME = 'ldap_user#server.com'
LDAP_PASSWORD = ''
USER_NAME = 'username-to-test'
USER_IN_GROUP = 'CN=SomeGroup,DC=server,DC=com'
USER_NOT_IN_GROUP = 'CN=SomeGroupThatDoesNotExist,DC=server,DC=com'
try:
# Connect to LDAP / Active Directory
ldap_con = ldap.initialize(LDAP_URI)
ldap_con.protocol_version = 3
ldap_con.set_option(ldap.OPT_REFERRALS, 0)
ldap_con.simple_bind_s(LDAP_USERNAME, LDAP_PASSWORD)
# sAMAAccountName is Active Directory's 'username'
user_filter='(&(objectCategory=person)(objectClass=user)(sAMAccountName=' + USER_NAME + '))'
attrs = ['memberOf']
# Perform the search.
ldap_user = ldap_con.search_s(LDAP_DN, ldap.SCOPE_SUBTREE, user_filter, attrs)
# Active Directory returns a list of byte literals. Convert them to strings in a more sensibly named list.
ldap_groups = []
for value in ldap_user[0][1]['memberOf']:
ldap_groups.append(value.decode('utf-8'))
# Print the LDAP groups the user above is a member of, one per line.
for value in ldap_groups:
print(value)
# Perform check to see whether a user is in a group, or explicitly, a user it not in a group.
if USER_IN_GROUP in ldap_groups:
print(USER_NAME + " is a member of " + USER_IN_GROUP)
else:
print(USER_NAME + " is not a member of " + USER_IN_GROUP)
if USER_NOT_IN_GROUP in ldap_groups:
print(USER_NAME + " is a member of " + USER_NOT_IN_GROUP)
else:
print(USER_NAME + " is not a member of " + USER_NOT_IN_GROUP)
# Unbind from LDAP / Active Directory.
ldap_con.unbind()
except ldap.LDAPError:
print(ldap.LDAPError)
These two lines are essential when connecting to Active Directory using an LDAP package:
ldap_con.protocol_version = 3
ldap_con.set_option(ldap.OPT_REFERRALS, 0)

Creating a Google shared contact using the API - contact is created but not in the shared Directory

I'm currently using the shared_contacts_profiles.py script to load contacts from an external system into our Google Shared Domain contacts. I'd like to make the process more automated so I've tried to create a shared contact (with just a full name and email address) using a basic python script. The contact is created but it gets added to the administrator's contacts and not the Directory.
My code is
#!/usr/bin/python
import atom
import gdata.data
import gdata.contacts.client
import gdata.contacts.data
def main():
admin_email = 'admin#mydomain.com'
admin_password = 'P4ssw0rd'
domain_index = admin_email.find('#')
domain = admin_email[domain_index+1:]
contacts_client = gdata.contacts.client.ContactsClient(domain=domain)
contacts_client.client_login(email=admin_email,
password=admin_password,
source='shared_contacts_profiles',
account_type='HOSTED')
new_contact = gdata.contacts.data.ContactEntry()
new_contact.name = gdata.data.Name(
full_name=gdata.data.FullName(text='John Doe'))
new_contact.email.append(gdata.data.Email(address='john.doe#example.com',
primary='true',rel=gdata.data.WORK_REL))
contact_entry = contacts_client.CreateContact(new_contact)
print "Contact's ID: %s" % contact_entry.id.text
if __name__ == '__main__':
main()
I must be missing something fairly simple, but just can't see what it is.
EDIT * I think that shared_contacts_profiles.py sets the domain contact list when it sends batches to Google. I wasn't going to use batches as there are only ever a couple of contacts to add. I also suspect I should be using gdata.contacts.service.ContactsService and not gdata.contacts.client.ContactsClient
Thanks
Dave

In the end I used the original code as shown above with some additions. I needed to get the feed uri for the shared domain contact list and then supply that uri in the CreateContact.
feed_url = contacts_client.GetFeedUri(contact_list=domain, projection='full')
contact_entry = contacts_client.CreateContact(new_contact,insert_uri=feed_url)
Thanks
Dave