when I read the excel file to python.
import pandas as pd
data = pd.read_excel('copy.xlsx')
data
Some part of my time data was uploaded successfully but another part of the time data has some problems. Problem is on these columns (in_time, call_time, process_in_time, out_time).
Why is this happening?
And how to handle and normalize this time data?
The Excel data link is here >enter link description here
enter image description here
Related
when I read the excel file to python.
import pandas as pd
data = pd.read_excel('1.xlsx')
data
Some part of my time data uploaded successfully but the another part of time data has some problems. Problem is on these columns (in_time, call_time, process_in_time, out_time).
Why is this happened?
And how to handle and normalize this time data ?
I want to read some spectral data to my streamlit application. Each row in the file is a spectrum (different intensity values). So, I used st.fileuploader
spectra = st.file_uploader("upload file", type={"csv", "txt"})
spectra = pd.DataFrame(spectra_1_file)
st.write(spectra_1)
Now, I am getting only one row of data on each column I have each spectrum as comma-separated. I need to convert the input from the user to a similar fashion of pd.csv_read. Because, when I was using it, I was able to get a proper pandas data frame. I tried different methods to read. But, never got the proper way. this was the closest I could get.
Thank you in advance.
spectra = st.file_uploader("upload file", type={"csv", "txt"})
if spectra is not None:
spectra_df = pd.read_csv(spectra)
st.write(spectra_df)
By using Pandas and Streamlit, you can read and upload your CSV file into your localhost.
data = pd.read_csv("csv_practice.csv") #path folder of the data file
st.write(data) #displays the table of data
I'm trying to create a DB from an excel spreadsheet. I can fetch data from excel and display in the html page, but I am not able to store it in sqlite db.
Few ways you can try:
Save excel as csv. Read csv in python (link) and save in sqlite (link).
Read excel into a pandas dataframe (link), and then save dataframe to sqlite (link).
Read excel directly from python (link) and save data to sqlite.
I used below code which worked but it over rights file.
#import pandas software library
import pandas as pd
df = pd.read_excel(r'C:\Users\kmc487\PycharmProjects\myproject\Product List.xlsx')
#Print sheet1
print(df)
df.to_excel("output.xlsx", sheet_name="Sheet_1")
Below are the input file details:
My input file is in .xlsx format and file is stored as .xls(Need code to .xlsx format)
File has heading in second row(First row blank)
I have scenario where I am getting the file(stream:flowfile of NIFI) as of type csv file,
then creating the dataframe and dumping it thats it.
But after creating the dataframe the structure of the file got disturbed, if I open
the same flowfile on my disk i could see clear structure with columns separated with
tab, but with python dataframe I am not getting the same structure, if I get same structure i can person row manipulation.
Here What I am doing:
1: using ExecuteSQL processor, I am getting database record,
2: then passing this record to ConvertRecord processor to convert this avro record type csv file separated by tab.
[enter image description here][1]
3: Then the reading flowfile(step 2 folowfile) as python data using ExecuteStreamCommand, coz I am want to perform some action on the database record, to do this my record structure has been changed in data frame.
The flow file generated from ConvertRecord is ...
[enter image description here][2]
The out put of the dataframe is ..
[enter image description here][3]
[1]: https://i.stack.imgur.com/Z4P97.png
[2]: https://i.stack.imgur.com/AU3ke.png
[3]: https://i.stack.imgur.com/LcICS.png
sample python code...
import sys, csv;
import pandas as pd
class CustomPythonScript():
df = pd.read_csv(sys.stdin)
output = pd.DataFrame(df)
I am Trying to convert the YAML Data to Data frame through pandas with yamltodb package. but it is showing only the single row enclosed with header and only one data is showing. I tried to convert the Yaml file to JSON file and then tried normalize function. But it is not working out. Attached the screenshot for JSON function output. I need to categorize it under batman, bowler and runs etc. Code
Output Image and their code..
Just guessing, as I don’t know what your data actually looks like
import pandas as pd
import yaml
with open('fName.yaml', 'r') as f:
df = pd.io.json.json_normalize(yaml.load(f))
df.head()