I'm using a SpreadsheetsListFeed to read data from a Google Spreadsheet.
The sheet contains a table of customer data, including firstname, lastname, email, phone, etc. The column headers are in the top row of the sheet, and the data is in the subsequent rows. I figured a SpreadsheetsListFeed would be the best option since the data is in a tabular format.
However, now I need to use the python app to generate a customerid for each customer, and write it to the spreadsheet (only if it doesn't already exist in the sheet). I'm wondering if there is a way to do this through the SpreadsheetsListFeed or if I need to use a SpreadsheetCellsFeed and calculate the row/column values based on the existing SpreadsheetsListFeed.
The documentation for the python gdata library is a little sparse, so any help would be appreciated. Thanks!
That's the reason why I wrote a small wrapper lib for the spreadsheet API. It lets you work with a spreadsheet like you would with a dict.
Related
This is part 1 of a series of questions I will make on this forum so bear with me on this one. I am a novice programmer who took on a large project because i like to torture myself, so please be kind.
I am writing a Python script to process an Excel document full of accounts (See example below), each one being the same format, extract specific type of data from it, and then export that data to a SQL table. This is the process flow I have in mind when illustrating the script on paper:
The input is a large Excel document containing bookkeeping accounts with this format below:
Account format example and the data to be extracted highlighted, I believe the software used to produce this is an antiquated accounting software named "Zeus"](https://i.stack.imgur.com/Htdze.png)
The data to be extracted is the account name and number (they're on the same cell so I find it easier to extract them altogether so that I can use them as a primary key in a SQL table; will talk about that on another post) and the whole table of details of the account as highlighted above. Mind you, there are thousands of bookkeeping accounts of this format on the document and multiple of these are used for the same account name and number, meaning they have the same header, but different details.
The data processing will go like this:
Use regular expressions to match, extract, and store in an array, each account name and number (so that I can keep record of every account number and use them as a primary key in a SQL table)
Extract and match the content of each account details table to their respective account name and number (haven't figured out how to do that yet, however, I will be using a relationship table to link them to their primary key once data is exported).
Export the extracted data into a database software (mySQL or MS Access... will most likely use MS Access).
After data is extracted and processed, a Excel report is to be created consisting on a table with the name and number of the account on the first column and then the details of the account on the following columns (will post about that later on).
Part 1: Excel data extraction/"scraping"
Quick note: I have tried multiple methods such as (MS Access, VBA and MS Power Automate) to do this and avoid having to manually code everything, ended up failing miserably, so I decided to bite the bullet and just do it.
So here's the question: after doing some research, I came across multiple methods to extract data from an excel, and several methods to use regex to do web scraping and PDF data extraction.
Is there a way to extract data from an Excel document through Python using regex match? If so, how could I do that?
PS: I will be documenting my journey through this forum on another post in order to help other fellow data entry workers.
Look into these python modules:
import xlwt
from xlwt.Workbook import *
import xlsxwriter
import numpy as np
import pandas as pd
from pandas import ExcelWriter
Then you can use pandas dataframe like:
data = pd.read_excel('testacct.xlsx')
This will put the entire spreadsheet into a dict with generic column names:
If there are multiple sheets, then the df object will be a list of dicts. Each column is a list or row data.
You can traverse the rows like:
cols = data.keys()
for row in range(len(data[cols[0]])):
for col in cols:
print(data[col][row])
print("--")
You can join the column data and strip out spaces.
Then you can use regex to any of the header values.
I have a google sheet connect to a google form. The form is compiled by team leaders with his members and some informations that are reported in a row by google.
I need to make another sheet with all data from members in a single column.
I won't copy and paste 'cause there are more than 50 leaders and thousand of members and the real problem is that some rows are full and some, considering that not all teams are made up of the same number of members, are half empty.
What is the fastest way to complite the sheet?
I need something like:
Rows from google sheet
[Team1; Bob, data; Rob, data]
[Team2; Rose, data; Mark, data; Jenny, data]
Result that I want:
[
[Bob, data],
[Rob, data],
[Rose, data],
[Mark, data],
[Jenny, data],
]
If there's no way to do it internally to Google Sheet can I use python and think to google sheet like a matrix?
You CAN do it internally using Google Apps Script. This allows users to make custom functions and code in JavaScript which can make changes to sheets in Google Sheets or get info based on what code you write.
Here is the link for the Apps Script documentation which is quite well written.
You will basically need to create 2 sheets, write a function to extract the cells you want from each row, and then input that data into the second sheet. You do not need to be very well versed in JavaScript to do this, I myself am not adept at Javascript however I am able to make functions as per my need.
Also some advice, please test it with a sample sheet first so that you do not delete data or make errors.
I am looking to write certain columns of data from an excel sheet to a HTML table. Not looking to write specific/fixed cells into the table always, need to do this based on conditions. For example, if I have a table with columns Name/Age/Occupation, I would like to make an HTML table using just columns Name and Occupation. Also, within Name, I would only like to write the names starting with 'N' onto the table and corresponding Occupation. The Excel sheet dynamically changes with new data everytime. Essentially, I would not want to write specific cells or range of cells into the table but only the data based on conditions I set. Any suggestions using python/html/jquery or other methods are welcome.
First you should edit the Excel file, export it as a .csv file and then work on the file using a program language of your preference. It would be much much more complicated if you try to work on the .xls or .xlsx files. I recommend using python with its library panda that works on csv files.
For parsing excel files, I've had good success using openpyxl
A Python library to read/write Excel 2010 xlsx/xlsm files
I am trying to write a script which takes the usernames from an excel sheet in a loop and then connect to an external API of a website and get the user ID's from it and give gave the response in the excel sheet. Please help me with an example code.
I need help on two things:-
1:- How to read a particular column elements from an excel sheet
2:- Write a code in the script which uses an API of a website to feed the excel sheet usernames in it in a loop and retrieve the user ID's
For reading the information from the excel sheet take a look at https://docs.python.org/2/library/csv.html
For retrieving the user IDs it depends on the API itself so you would need to provide more information.
Additionally, you might want to look at this Python library for the Instagram API.
This site contains pointers to the best information available about working with Excel files in the Python programming language.
This site will show you how to use APIs with python.
Sorry if the title is confusing. Basically what I am trying to do is create an excel sheet with data that is in a url that I have.
The url is a search API for twitter that retrieves the past 100 tweets with a given keyword of my choice. I am trying to to create an excel sheet that stores each tweet in it's own row. Essentially it will only be 1 column but will be 100 rows.
I have looked online but haven't really seen a way to do exactly what I need so if anyone knows a tutorial i should look at or could show me how to get started that would be great.
Thanks!
There will probably not be a tutorial on exactly how to do this. you need to put a couple different concepts together
Get the data from the url. This can be as simple as urllib.urlopen
Turn that data (string) into a usable format. Twitter will probably return json. Turn that into a python dict
Open a file for writing
Loop through twitter data and write to ouput file
You only need to create a .csv file. It will work great with excel. For one column file you just need to write the header then write each line of data. Python provides everything you need to create well formed csv files in the csv module in the standard library