I have the dataset shown below. I am trying to sort it so that the columns are in this order: Week End, Australia, Germany, France, etc...
I have tried using loc and assigning each of the data sets as variables but when I create a new DataFrame it causes an error. Any help would be appreciated.
This is the data before any changes:
Region
Week End
Value
Australia
2014-01-11
1.480510
Germany
2014-01-11
1.481258
France
2014-01-11
0.986507
United Kingdom
2014-01-11
1.973014
Italy
2014-01-11
0.740629
This is my desired output:
Week End
Australia
Germany
France
United Kingdom
Italy
2014-01-11
1.480510
1.481258
0.986507
1.973014
0.740629
What I've tried:
cols = (['Region','Week End','Value'])
df = GS.loc[GS['Brand'].isin(rows)]
df = df[cols]
AUS = df.loc[df['Region'] == 'Australia']
JPN = df.loc[df['Region'] == 'Japan']
US = df.loc[df['Region'] == 'United States of America']
I think that you could actually just do:
df.pivot(index="Week End", columns="Region", values="Value")
User 965311532's answer is much more concise, but an alternative approach using dictionaries would be:
new_df = {'Week End': df['Week End'][0]}
new_df.update({region: value for region, value in zip(df['Region'], df['Value'])})
new_df = pd.DataFrame(new_df, index = [0])
As user 965311532 pointed out, the above code will not work if there are more dates. In this case, we could use pandas groupby:
dates = []
for date, group in df.groupby('Week End'):
date_df = {'Week End': date}
date_df.update({region: value for region, value in zip(df['Region'], df['Value'])})
date_df = pd.DataFrame(date_df, index = [0])
dates.append(date_df)
new_df = pd.concat(dates)
Related
I have two dataframes,
df1 =
Countries description
Continents
values
C0001 also called America,
America
21tr
C0004 and C0003 are neighbhors
Europe
504 bn
on advancing C0005 with C0001.security
Europe
600bn
C0002, the smallest continent
Australi
1.7tr
df2 =
Countries
Id
US
C0001
Australia
C0002
Finland
C0003
Norway
C0004
Japan
C0005
df1 has columns Countries descriptions but instead of their actual names, codes are given.
df2 has countries with their codes.
I want to replace the countries Code(like C0001, C0002) with their Names in the df1, like this:
df1 =
Countries description
Continents
values
US also called America, some..
America
21tr
Norway and Finland are neighbhors
Europe
504 bn
on advancing Japan with US.security
Europe
600bn
Australia, the smallest continent
Austral
1.7tr
I tried with the Pandas merge method but that didnt work:
df3 = df1.merge(df2, on=['Countries'], how='left')
Thanks :)
Here is one way to approach it with replace :
d = dict(zip(df2["Id"], df2["Countries"]))
df1["Countries description"] = df1["Countries description"].replace(d, regex=True)
Output :
print(df1)
Countries description Continents values
0 US also called America, America 21tr
1 Norway and Finland are neighbhors Europe 504 bn
2 on advancing Japan with US.security Europe 600bn
3 Australia, the smallest continent Australi 1.7tr
Here's my inital dataset
sitename L1 L2 L3
America Continent NaN Others
Mexico NaN Spanish America
This is reference dataset
sitename L1 L2
America Country English
Mexico Country Spanish
This is expected output dataset
sitename L1 L2 L3
America Continent English Others
Mexico Country Spanish America
The reference data is only works on Null vales
For align by column sitemap convert it to index in both DataFrames and then replace missing values:
df = df1.set_index('sitemap').fillna(df2.set_index('sitemap')).reset_index()
Or:
df = df1.set_index('sitemap').combine_first(df2.set_index('sitemap')).reset_index()
Or:
df1 = df1.set_index('sitemap')
df2 = df2.set_index('sitemap')
df1.update(df2)
df = df1.reset_index()
You can use the:
df.fillna(df2)
If indexes are not the same, you can set them to be the same and use the same function:
df = df.set_index('sitname')
df.fillna(df2.set_index('sitename'))
Edit
How would I replace a specific number of values from a specified column in a DataFrame?
# babies born in countries
Date Country
1992-02-15 USA
1995-05-04 USA
1996-02-12 Canada
2003-12-17 France
2005-01-11 USA
Suppose I have the above data and it turns out that the birth country for the first two birth is wrong, instead of USA it should be Spain, France.
I tried the replace method but it changes the values altogether.
Desired result:
# babies born in countries
Date Country
1992-02-15 Spain
1995-05-04 France
1996-02-12 Canada
2003-12-17 France
2005-01-11 USA
Thank you!
To access the first two rows via indexing:
df.iloc[0:2, 1] = ['Spain', 'France']
Or:
df.loc[:2, 'Country'] = ['Spain', 'France']
Also if you need to access specific rows:
df.loc[[0,1], 'Country'] = ['Spain', 'France']
I started to learn about pandas and try to analyze a data
So in my data there is a column country which contain a few country,I only want to take the first value and change it to a new column.
An example First index have Colombia,Mexico,United Stated and I only wanna to take the first one Colombia [0] and delete the other contry[1:x],is this possible?
I try a few like loc,iloc or drop() but I hit a dead end so I asked in here
You can use Series.str.split:
df['country'] = df['country'].str.split(',').str[0]
Consider below df for example:
In [1520]: df = pd.DataFrame({'country':['Colombia, Mexico, US', 'Croatia, Slovenia, Serbia', 'Denmark', 'Denmark, Brazil']})
In [1521]: df
Out[1521]:
country
0 Colombia, Mexico, US
1 Croatia, Slovenia, Serbia
2 Denmark
3 Denmark, Brazil
In [1523]: df['country'] = df['country'].str.split(',').str[0]
In [1524]: df
Out[1524]:
country
0 Colombia
1 Croatia
2 Denmark
3 Denmark
Use .str.split():
df['country'] = df['country'].str.split(',',expand=True)[0]
I have the following data frame:
population GDP
country
United Kingdom 4.5m 10m
Spain 3m 8m
France 2m 6m
I also have the following information in a 2 column dataframe(happy for this to be made into another datastruct if that will be more beneficial as the plan is that it will be sorted in a VARS file.
county code
Spain es
France fr
United Kingdom uk
The 'mapping' datastruct will be sorted in a random order as countries will be added/removed at random times.
What is the best way to re-index the data frame to its country code from its country name?
Is there a smart solution that would also work on other columns so for example if a data frame was indexed on date but one column was df['county'] then you could change df['country'] to its country code? Finally is there a third option that would add an additional column that was either country/code which selected the right code based on a country name in another column?
I think you can use Series.map, but it works only with Series, so need Index.to_series. Last rename_axis (new in pandas 0.18.0):
df1.index = df1.index.to_series().map(df2.set_index('county').code)
df1 = df1.rename_axis('county')
#pandas bellow 0.18.0
#df1.index.name = 'county'
print (df1)
population GDP
county
uk 4.5m 10m
es 3m 8m
fr 2m 6m
It is same as mapping by dict:
d = df2.set_index('county').code.to_dict()
print (d)
{'France': 'fr', 'Spain': 'es', 'United Kingdom': 'uk'}
df1.index = df1.index.to_series().map(d)
df1 = df1.rename_axis('county')
#pandas bellow 0.18.0
#df1.index.name = 'county'
print (df1)
population GDP
county
uk 4.5m 10m
es 3m 8m
fr 2m 6m
EDIT:
Another solution with Index.map, so to_series is omitted:
d = df2.set_index('county').code.to_dict()
print (d)
{'France': 'fr', 'Spain': 'es', 'United Kingdom': 'uk'}
df1.index = df1.index.map(d.get)
df1 = df1.rename_axis('county')
#pandas bellow 0.18.0
#df1.index.name = 'county'
print (df1)
population GDP
county
uk 4.5m 10m
es 3m 8m
fr 2m 6m
Here are some brief ways to approach your 3 questions. More details below:
1) How to change index based on mapping in separate df
Use df_with_mapping.todict("split") to create a dictionary, then use a list comprehension to change it into {"old1":"new1",...,"oldn":"newn"} form then use df.index = df.base_column.map(dictionary) to get the changed index.
2) How to change index if the new column is in the same df:
df.index = df["column_you_want"]
3) Creating a new column by mapping on a old column:
df["new_column"] = df["old_column"].map({"old1":"new1",...,"oldn":"newn"})
1) Mapping for the current index exists in separate dataframe but you don't have the mapped column in the dataframe yet
This is essentially the same as question 2 with the additional step of creating a dictionary for the mapping you want.
#creating the mapping dictionary in the form of current index : future index
df2 = pd.DataFrame([["es"],["fr"]],index = ["spain","france"])
interm_dict = df2.to_dict("split") #Creates a dictionary split into column labels, data labels and data
mapping_dict = {country:data[0] for country,data in zip(interm_dict["index"],interm_dict['data'])}
#We only want the first column of the data and the index so we need to make a new dict with a list comprehension and zip
df["country"] = df.index #Create a new column if u want to save the index
df.index = pd.Series(df.index).map(mapping_dict) #change the index
df.index.name = "" #Blanks out index name
df = df.drop("county code",1) #Drops the county code column to avoid duplicate columns
Before:
county code language
spain es spanish
france fr french
After:
language country
es spanish spain
fr french france
2) Changing the current index to one of the columns already in the dataframe
df = pd.DataFrame([["es","spanish"],["fr","french"]], columns = ["county code","language"], index = ["spain", "french"])
df["country"] = df.index #if you want to save the original index
df.index = df["county code"] #The only step you actually need
df.index.name = "" #if you want a blank index name
df = df.drop("county code",1) #if you dont want the duplicate column
Before:
county code language
spain es spanish
french fr french
After:
language country
es spanish spain
fr french french
3) Creating an additional column based on another column
This is again essentially the same as step 2 except we create an additional column instead of assigning .index to the created series.
df = pd.DataFrame([["es","spanish"],["fr","french"]], columns = ["county code","language"], index = ["spain", "france"])
df["city"] = df["county code"].map({"es":"barcelona","fr":"paris"})
Before:
county code language
spain es spanish
france fr french
After:
county code language city
spain es spanish barcelona
france fr french paris