Develop Reference

How to drop dataframe columns using both, a list and not from a list?

I am trying to drop pandas column in the following way. I have a list with columns to drop. This list will be used many times in my notebook. I have 2 columns which are only referenced once
drop_cols=['var1','var2']
df = df.drop(columns={'var0',drop_cols})
So basically, I want to drop all columns from list drop_cols in addition to a hard-coded "var0" column all in one swoop. This gives an error, How do I resolve?

df = df.drop(columns=drop_cols+['var0'])

From what I gather you have a set of columns you wish to drop from several different dataframes while at the same time adding another unique column to also be dropped a data frame. The command you have used is close but misses the point in that you can't create a concatenated list in the way you are trying to do it. This is how I would approach the problem.
Given a Dataframe of the form:
V0 V1 V2 V3
0 1 2 3 4
1 5 6 7 8
2 9 10 11 12
define a function to merge colnames
def mergeNames(spc_col, multi_cols):
rslt = [spc_col]
rslt.extend(multi+cols)
return rslt
Then with
drop_cols = ['V1', 'V2']
df.drop(columns=mergeNames('V0', drop_cols)
yields:
V3
0 4
1 8
2 12

Delete the rows that have a single zero in Pandas

I have imported an excel sheet using Pandas like this:
w = pd.read_excel(r"C:\Users\lvk\Downloads\Softwares\Prob.xls", header=None)
Once I imported the excel sheet, I need to delete the rows with even a single zero in any column.
Are there any functions in Python to do that?
Please let me know.
Input:
row1: 0 4 3 5
row2: 1 6 5 61
row3: 1 3 6 0
Expected output:
1 6 5 61

Pandas has very powerful interfaces for indexing and selecting data. Among them are the use of the loc keyword to access by rows, and square brackets to pass indexing logic to loc. Normally you might use the names of your columns to do logical operations on their values. Here I don't know the index or columns of your excel data, so we will just loop through all the columns that are there.
# We are going to look in each column
for col in w.columns:
# And select only the rows in w that don't have a 0 in that column
w = w.loc[w[column] != 0]

pandas dataframe with identical column names - is it valid process?

I was able to produce a pandas dataframe with identical column names.
Is it this normal fro a pandas dataframe?
How can I choose one of the two columns only?
Using the identical name, it has, as a result, to produce as output both columns of the dataframe?
Example given below:
# Producing a new empty pd dataset
dataset=pd.DataFrame()
# fill in a list with values to be added to the dataset later
cases=[1]*10
# Adding the list of values in the dataset, and naming the variable / column
dataset["id"]=cases
# making a list of columns as it is displayed below:
data_columns = ["id", "id"]
# Then, we call the pd dataframe using the defined column names:
dataset_new=dataset[data_columns]
# dataset_new
# It has as a result two columns with identical names.
# How can I process only one of the two dataset columns?
id id
0 1 1
1 1 1
2 1 1
3 1 1
4 1 1
5 1 1
6 1 1
7 1 1

You can use the .iloc to access either column.
dataset_new.iloc[:,0]
or
dataset_new.iloc[:,1]
and of course you can rename your columns just like you did when you set them both to 'id' using:
dataset_new.column = ['id_1', 'id_2']

df = pd.DataFrame()
lst = ['1', '2', '3']
df[0] = lst
df[1] = lst
df.rename(columns={0:'id'}, inplace=True)
df.rename(columns={1:'id'}, inplace=True)
print(df[[1]])

Pandas Read csv function not alleging column headers well

I have a csv file with no headers. The first column is ID and so on... Here is how I read that file in pandas.
rss_content=pd.read_csv("rss_content.csv",header=None,names=["id","feedId","url","imageUrl","title","desc","author","createTimestamp"])
However when the file gets imported I see the first two columns of the data become index and the Id column gets assigned to third column and so on. Basically the headers are shifted by two columns to right and first two columns have no header.
Why is that and how to fix it?

Assuming you have the following CSV file:
1,2,3,4,5
11,22,33,44,55
If you specify too less column names the rest columns will become index columns:
In [1]: fn = r'D:\temp\.data\41066716.csv'
In [2]: df = pd.read_csv(fn, header=None, names=['a','b','c'])
In [3]: df
Out[3]:
a b c
1 2 3 4 5
11 22 33 44 55

How to read all rows of a csv file using pandas in python?

I am using the pandas module for reading the data from a .csv file.
I can write out the following code to extract the data belonging to an individual column as follows:
import pandas as pd
df = pd.read_csv('somefile.tsv', sep='\t', header=0)
some_column = df.column_name
print some_column # Gives the values of all entries in the column
However, the file that I am trying to read now has more than 5000 columns and writing out the statement
some_column = df.column_name
is now not feasible. How can I get all the column values so that I can access them using indexing?
e.g to extract the value present at the 100th row and the 50th column, I should be able to write something like this:
df([100][50])

Use DataFrame.iloc or DataFrame.iat, but python counts from 0, so need 99 and 49 for select 100. row and 50. column:
df = df.iloc[99,49]
Sample - select 3. row and 4. column:
df = pd.DataFrame({'A':[1,2,3],
'B':[4,5,6],
'C':[7,8,9],
'D':[1,3,10],
'E':[5,3,6],
'F':[7,4,3]})
print (df)
A B C D E F
0 1 4 7 1 5 7
1 2 5 8 3 3 4
2 3 6 9 10 6 3
print (df.iloc[2,3])
10
print (df.iat[2,3])
10
Combination for selecting by column name and position of row is possible by Series.iloc or Series.iat:
print (df['D'].iloc[2])
10
print (df['D'].iat[2])
10

Pandas has indexing for dataframes, so you can use
df.iloc[[index]]["column header"]
the index is in a list as you can pass multiple indexes at one in this way.