I need to create a data pipeline using Python. I want to connect with MySql in Python and read the tables in dataframes, perform pre-processing and then load the data back to Mysql Db. I was able to connect to the MySql Db using mysql connector and then pre-process the dataframes. However, I'm not able to load these dataframes from Python back to Mysql. Error: ValueError: unknown type str96 python.
Please help me with methods to complete this task.
I'm new to programming. Any help will be greatly appreciated. Thanks!
It is a bug and has been fixed in version 1.1.3.
upgrade pandas package
pip3 install --upgrade pandas
Related
I am new to Python. Can someone tell me what software should I install (Pycharm??) so that I can write program to connect to Teradata. Any sample code in python to get a specific table extraction will be useful too for me.
I am using a python library to pull data from social media. I am trying to analyze it in elasticsearch/kibana. I have elasticsearch and kibana set up and they are working fine. The python library is pulling data fine also, but When I try to index my data I get the following error:
C:\Windows\system32>twint -es localhost:9200 -u twitterusername --limit 10
[+] Indexing to Elasticsearch # localhost:9200
CRITICAL:root:twint.get:User:
Is something wrong with my system?
I am using teradatasql package which native solution of Teradata as a connector between python and Teradata to load data from DB. However, I want to insert data frames I created in python back to DB. Is it possible to write data frames to the database by using teradatasql package?
Thanks
SQLAlchemy provides the linkage between pandas dataframes and a SQL database.
Typically, you would use the pandas dataframe to_sql method to insert the contents of a dataframe into a table in the database:
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html
We offer a SQLAlchemy dialect for the Teradata SQL Driver for Python:
https://pypi.org/project/teradatasqlalchemy/
You can install it with: pip install teradatasqlalchemy
I'm quite new to Python (using Python 3.6.5) and I need to create a connection to a database, and I do it in Access with the ODBC Driver (I guess?), but for some reasons I just can't install pyodbc here, so I need to do this connection without it. Is there a way to do it?
I'm sorry if there's not much details about it, but I can try to check anything if needed.
Edit - I don't think it's the point here to solve the fact that I can't get the use pip to install, since I know why it isn't working, but here is the print from the log of pip:
I have a standard django setup using postgres, but I also want to access data from a mssql database. I have installed pyodbc and was in the process of installing django-pyodbc but the instructions tell me how to make the mssql the default database which is not what I want.
I was hoping somebody could point me in the right direction. It doesnt bother me if the method bypasses django and just uses Python to retrieve the data
Cheers,
JJ
If you're using Django 1.2 or later, you can use Django's built-in multi-database support. You can follow the django-pyodbc directions, and give your database a name other than 'default'.