Data Cleansing: Data cleansing is the process of identifying and correcting errors or inconsistencies in data to ensure it is accurate, complete, and ready for analysis. This can involve removing duplicates, filling in missing values, and correcting data that is incorrect or formatted improperly.
Structured vs. Unstructured Data: Structured data is organized and easy to search, like data in tables with rows and columns (e.g., databases). Unstructured data doesn’t have a specific format, making it harder to analyze, like emails, videos, or social media posts.
Schema: A schema is a blueprint or structure of a database that defines how data is organized, including tables, fields, and relationships between them. It tells you what kind of data can be stored and how it will be arranged.
Data Normalization: Data normalization is the process of organizing data in a database to reduce redundancy and improve efficiency. It involves dividing large tables into smaller, related ones and linking them using relationships, which helps minimize data duplication.
Denormalization: Denormalization is the opposite of normalization. It involves combining tables or adding redundant data to make queries faster. While this can speed up data retrieval, it may increase storage requirements and the complexity of updates.
DataPartitioning: Data partitioning is the process of dividing a large dataset into smaller, more manageable pieces, called partitions. This helps improve performance and makes it easier to maintain and process large volumes of data.
Data Aggregation: Data aggregation is the process of gathering and summarizing data, often from multiple sources, to provide a combined view. For example, summing up sales figures across different regions to get a total.
OLTP vs. OLAP: OLTP (Online Transaction Processing) is used for managing day-to-day operations, like handling sales transactions. OLAP (Online Analytical Processing) is used for analyzing data to support decision-making, like running reports to find trends in sales data.
PrimaryKey: A primary key is a unique identifier for a record in a database table. It ensures that each record is unique and can be used to reference that specific record.
ForeignKey: A foreign key is a field in one table that links to the primary key in another table. It helps maintain relationships between tables and ensures the integrity of the data.
DataRedundancy: Data redundancy occurs when the same piece of data is stored in multiple places. This can lead to inefficiencies, as changes in one place need to be reflected everywhere else, potentially leading to inconsistencies.
DataLineage: Data lineage tracks the journey of data as it moves from its source through various transformations to its final destination. It helps understand how data has been processed and where it came from.
DataReplication: Data replication involves copying data from one place to another to ensure that multiple locations have the same information. This is often done for backup, to improve access speed, or to distribute data across different systems.
DataSerialization: Data serialization is the process of converting data into a format that can be easily stored or transmitted and later reconstructed. For example, converting an object in a programming language into a format like JSON or XML.
DataCompression: Data compression reduces the size of data files by encoding information more efficiently. This makes it faster to transmit and takes up less storage space, though it may require decompression before use.
