Develop Reference

Mapping fields inside other fields

Hello I would like to make an app that allows the user to import data from a source of his choice (Airtable, xls, csv, JSON) and export to a JSON which will be pushed to an Sqlite database using an API.
The "core" of the functionality of the app is that it allows the user to create a "template" and "map" of the source columns inside the destination columns. Which source column(s) go to which destination column is up to the user. I am attaching two photos here (used in airtable/zapier), so you can get a better idea of the end result:
adding fields inside fields - airtableadding fields inside fields - zapier
I would like to know if you can recommend a library or a way to come about this problem? I have tried to look for some python or nodejs libraries, I am lost between using ETL libraries, some recommended using mapping/zipping features, others recommend coding my own classes. Do you know any libraries that allow to do the same thing as airtable/zapier ? Any suggestions ?

Save file on databases is really a bad practice since it takes up a lot of database storage space and would add latency in the communication.
I hardly recommend saving it on disk and store the path on database.

Python Advice for Project: Taking input and writing to existing PDF?? WebApp?

I am somewhat of a beginner at Python and I am currently starting some brainstorming and planning for a project to simplify the tedious task of filling out a product order form for a friend's business. I am wanting to create a program with an interface that takes in input and writes to an already existing pdf form of the physical order form. I would also like to implement being able to then email that form to another coworker and having accessible information of previous orders from the current customer ordering.
I am most curious about how to write to an already existing pdf form and just fill in the blanks, potentially using a PDF Reader? Also, for product catalog data and customer order history, would using dictionaries suffice or would it be better to use some python database?
I know I could figure out how to individually do each of those tasks but I don't know how to tie it all together properly to distribute it, either making a WebApp or an executable file from the script and just use tkinter or another GUI library for the interface, or if there is a more obvious and convenient option?
If I were to do a WebApp, what would be my best option, I've seen options such as Anvil, Flask, Django, and I just don't know what would be best for what I'm trying to accomplish.
I know this is a longshot and vague but any advice or guidance in the right direction would be much appreciated!

I'd go with a web app here. Personally, I prefer Django, and I don't think that it would be too difficult to learn it well enough for your purposes!
Regarding saving product catalog data and customer order history: I would definitely recommend using a database for your backend, which Django helps you do! Database integration is almost ridiculously easy using Django.
Here's some resources for the specific things you asked about:
Django vs. Flask
Working with HTML forms (Django) (this will help with taking user input)
filling an existing PDF form in Python (this will help with using user input to fill out the pdfs)
How to send email attachments in Python? (this will help you email your coworker the filled out pdf)
If you need more help/guidance, definitely feel free to follow up!

How to populate my django database with json that I scraped from a website

I have scraped data from a website using their API on a Django application. The data is JSON (a Python dictionary when I retrieve it on my end). The data has many, many fields. I want to store them in a database, so that I can create endpoints that will allow for lookup and modifications (updates). I need to use their fields to create the structure of my database. Any help on this issue or on how to tackle it would be greatly appreciated. I apologize if my question is not concise enough, please let me know if there is anything I need to specify.
I have seen many, many people saying to just populate it, such as this example How to populate a Django sqlite3 database. The issue is, there are so many fields that I cannot go and actually create the django model fields myself. From what I have read, it seems like I may be able to use serializers.ModelSerializer, although that seems to just populate a pre-existing db with already defined model.

Tricky to answer without details, but I would consider doing this in two steps - first, convert your json data to a database schema, for example using a tool like sqlify: https://sqlify.io/convert/json/to/sqlite
Then, create a database from the generated schema file, and use inspectdb to generate your django models: https://docs.djangoproject.com/en/2.2/ref/django-admin/#inspectdb
You'll probably need to tweak the generated schema and/or models, but this should go a long way towards automating the process.

I would go for a document database, like Elasticsearch or MongoDB.
Those are made for this kind of situation, look it up.

Python and Django Insert Strings into a Database

I am using Django and Python for the "framework" of a website. What I am trying to do is create a database that stores user input. The main purpose here is to find out the most common inputs for analytics, etc.
I thought about just making a CSV file and inserting into it via Python. But having to open the file and write to it each time seems inefficient. Maybe I'm wrong.
Are there any best practices out there that could help achieve this?
Thanks

With Django you can create a model which gives you the abillity to use forms - which get displayed as a HTML form on your website.
All you now need to do is to save it. :)
To start: There's a good tutorial at the Django documentation especially this part (Django: Playing with the API) is interesting for you - but I recommend to start from the beginning of the tutorial first.

Is there a way to print out output in a pyramid view callable?

I am new to python and pyramid and I am trying to figure out a way to print out some object values that I am using in a view callable to get a better idea of how things are working. More specifically, I am wanting to see what is coming out of a sqlalchemy query.
DBSession.query(User).filter(User.name.like('%'+request.matchdict['search']+'%'))
I need to take that query and then look up what Office a user belongs to by the office_id attribute that is part of the User object. I was thinking of looping through the users that come up from that query and doing another query to look up the office information (in the offices table). I need to build a dictionary that includes some User information and some Office information then return it to the browser as json.
Is there a way that I can experiment with different attempts at this while viewing my output without having to rely on the browser. I am more of a front end developer so when I am writing javascript I just view my outputs using console.log(output).
console.log(output) is to JavaScript
as
????? is to Python (specifically pyramid view callable)
Hope the question is not dumb. Just trying to learn. Appreciate anyones help.

This is a good reason to experiment with pshell, Pyramid's interactive python interpreter. From within pshell you can tinker with things on the command-line and see what they will do before adding them to your application.
http://docs.pylonsproject.org/projects/pyramid/en/1.4-branch/narr/commandline.html#the-interactive-shell
Of course, you can always use "print" and things will show up in the console. SQLAlchemy also has the sqlalchemy.echo ini option that you can turn on to see all queries. And finally, it sounds like you just need to do a join but maybe aren't familiar with how to write complex database queries, so I'd suggest you look into that before resorting to writing separate queries. Likely a single query can return you what you need.