Reading 1: Why We Need Databases

Picking Up Where We Left Off

In Topic 5a, we traced the data investigation cycle and established that structured data — data that fits neatly into rows and columns — is the kind most amenable to organized storage and querying. We used a spreadsheet as our model: one row per student, one column per attribute, everything consistent and searchable.

For a single investigation with a bounded dataset, that works beautifully. But what happens when the data doesn't stay bounded? What happens when multiple people in different roles need access to the same data at the same time, but each of them needs to see a different slice of it? What happens when the data grows to hundreds of thousands of records, or when the same information is needed for purposes that weren't anticipated when it was first collected?

These are the conditions under which a spreadsheet stops being enough — and under which database systems become necessary.

The Problem with Separate Files

To understand why databases were invented, it helps to understand what came before them. In the early days of computing, organizations typically maintained separate data files for each department or function. A school district, for example, might have maintained:

• A student enrollment file managed by the registrar
• An attendance file managed by the attendance office
• A grades file managed by each school's academic office
• A transportation file managed by the bus coordinator
• A special services file managed by the special education department

Each file was maintained independently, often by different people using different software. This created serious problems.

The first problem was redundancy. Basic student information — name, date of birth, address, parent contacts — was duplicated across every file. When a family moved and updated their address, that update had to be made in five separate places. Miss one, and the data became inconsistent. The transportation office might be sending buses to an old address while the attendance office had the correct one.

The second problem was isolation. Because each department's data lived in its own silo, answering questions that crossed departmental lines was difficult or impossible. If a principal wanted to know whether students with high absenteeism were also receiving special services, someone had to manually cross-reference two separate files — a tedious, error-prone process.

The third problem was access control. With separate files, it was difficult to give different people different levels of access to the same underlying data. Either someone had access to the entire enrollment file, or they did not.

Database systems emerged as a solution to all three of these problems simultaneously.

What a Database Actually Is

The term database refers to a collection of data that is multidimensional — meaning that internal links between its entries make the information accessible from multiple perspectives. This is in contrast to a flat file (like a single spreadsheet), which presents information from only one point of view.

Consider a library. A traditional flat file of library books might list all books alphabetically by title. That works fine if you know the title you want. But what if you want all books by a particular author? Or all books in a particular genre published after 2015? Or all books currently checked out by students in fourth grade? Each of those questions requires a different "view" of the same data — and a flat file can only provide one.

A database, by contrast, stores the underlying data in a structured way that allows any of those questions to be answered without reorganizing the data itself. The same collection of information is accessible from any angle the user needs.

The Power of Integration

The defining characteristic of a database system — the thing that makes it fundamentally different from a collection of separate files — is integration. Rather than storing data in isolated silos, a database brings related information together into a single, coherent system that can serve many users and many purposes simultaneously.

Think about what this means for a school district. A well-designed student information system stores each piece of data once — a student's address, for example — and makes it available to every part of the system that needs it: enrollment, attendance, transportation, communications. When the address changes, it changes in one place and is immediately correct everywhere.

Moreover, because the data is integrated, questions that cross traditional departmental lines become answerable. The same database that handles enrollment can support attendance analysis, can inform transportation routing, can generate communications to families, and can feed state reporting requirements — all from the same underlying data, without duplication or manual cross-referencing.

This integration is not automatic or free. It requires careful design — deciding how information is structured, how different pieces relate to each other, and who can access what. We will explore those design decisions in Readings 2 and 3.

Metadata: Data About Data

One concept that becomes especially important in the context of databases is metadata — literally, data about data. Metadata does not describe the primary content of a record; it describes the context surrounding it.

A familiar example: when you take a photo with your phone, the image itself is the data. But embedded alongside it is metadata — the date and time the photo was taken, the GPS coordinates of your location, the type of camera used, the exposure settings. The photo and its metadata together are far more useful than the photo alone.

In a school context, consider a student's assessment score. The score itself is the data. The metadata might include: which version of the assessment was administered, on what date, in which testing room, by which proctor, under what accommodations (if any). Without that metadata, a score of 214 is just a number. With it, you can ask meaningful questions: Do scores differ between the morning and afternoon testing sessions? Do students with extended-time accommodations perform differently on this assessment than on others?

Metadata can also be collected automatically, without anyone consciously creating it. When a student logs into a learning management system, a record is created: which student, at what time, from what device, for how long. That log data is metadata about the student's interaction with the system — and it can be extraordinarily revealing, which is why it raises significant privacy considerations.

The Database Management System

A database does not run itself. The software layer that sits between the user (or the application) and the actual stored data is called the database management system, or DBMS. Understanding the DBMS concept matters because it clarifies something that often confuses people new to databases: the difference between the data and the software that manages it.

When a school counselor searches the student information system for all students in tenth grade with a GPA below 2.0, she is not directly interacting with the stored data. She is interacting with an application — a screen with fields and buttons — that sends her request to the DBMS. The DBMS translates that request into the actual operations needed to search the database, retrieves the matching records, and sends the results back to the application to display. The counselor never sees or touches the underlying data directly.

This separation has several important benefits. First, it allows the database to be stored and organized in whatever way is most efficient for the computer, while users interact with it in whatever way is most natural for them. Second, it allows multiple applications to share the same database without interfering with each other. Third — and critically — it allows the DBMS to enforce access controls, ensuring that each user sees only the data they are authorized to see.

That last benefit connects directly to a concept we will explore in Reading 2: schemas and subschemas, which are the mechanism by which a database grants different users access to different views of the same underlying data.

Now You Try

Read the following scenario, then answer the questions. A suggested response is in the answer key.

1. What problems would the district encounter trying to generate this report from their current separate spreadsheets?
2. How would a properly integrated database system make this report easier to produce and maintain?
3. The superintendent wants the transportation coordinator to be able to run the report, but does not want the coordinator to see individual student GPA data — only the route-level averages. What database concept would handle this restriction?
4. Name one piece of metadata that might be useful to include in the attendance records, and explain what additional question it would allow you to answer.