Lesson 2: Why Databases? The Problem with Flat Files

Before databases were formalized, data was often stored in flat files (like simple text files or early spreadsheets). This approach quickly leads to chaos.

Challenges of Flat Files

Imagine running a large bookstore using only CSV files. Here are the issues you would face:

1. Data Redundancy (Duplication)

If the author 'Jane Doe' wrote 10 books, her name, address, and biography might be copied 10 times across your inventory file. If she moves, you must update 10 separate entries.

• The Database Solution: Store the author's details once in an 'Authors' table and link the books to that single entry.

2. Data Inconsistency

Due to redundancy, if you update Jane Doe's address in 9 places but forget the 10th, your data is inconsistent. You don't know which address is correct.

• The Database Solution: Enforce data integrity rules, ensuring that changes propagate correctly or that duplicates are eliminated.

3. Difficulty in Accessing Data (Querying)

Finding all books priced over $20 requires writing complex scripts to scan the entire file every time. As files grow, this becomes slow and error-prone.

• The Database Solution: Use Structured Query Language (SQL), a powerful language designed specifically to ask complex questions of the data quickly.

4. Security and Sharing Issues

It's hard to control who can view or modify specific parts of a single file. Also, if two people try to edit the file at the exact same moment, data corruption is likely.

• The Database Solution: Database Management Systems (DBMS) handle simultaneous access (concurrency control) and provide granular security permissions.