Data Engineer Terminologies

Transactional databases store all changes in a transaction log in order to recover the committed state of the database should the database crash for whatever reason. Log-based CDC takes advantage of this aspect of the transactional database to read the changes from the log. They can be replicated to downstream systems for other purposes. e.g. debezium is a CDC framework.

The write-ahead log or “wal” file is a roll-forward journal that records transactions that have been committed but not yet applied to the main database.

which deals with database schemas and descriptions, of how the data should reside in the database.

which deals with data manipulation, and includes most common SQL statements such SELECT, INSERT, UPDATE, DELETE etc, and it is used to store, modify, retrieve, delete and update data in database.

Related posts:

What is the purpose of a biopic? Different writers and directors have approached the subgenre with different purposes, and they often function as a means for actors to flex their creative muscles and…

Tesla quietly rolled out a new product for beach lovers and wave carvers this weekend– a limited edition surfboard. The tech company offered just 200 surfboards for $1,500 each, but unfortunately for…

At the end of the school year, we finally received an Individualized Education Plan (IEP) for my son. An IEP defines individualized educational goals for children determined to have a disability and…