0.6.9 Clean Data Is Consistent Data

Small Differences Can Create Big Data Problems

People are good at recognizing values that “basically mean the same thing.”

Software often treats different representations as different values.

Imagine a Genre column containing:

Plain text
Rock
rock
ROCK
Rock 

A person may read all four as the same genre.

A spreadsheet summary may treat them as separate categories.

That can produce misleading counts and charts.

Clean data uses consistent representations for the same meaning.

Consistency Begins With Spelling

Suppose an artist appears as:

Plain text
Example Band
Exampel Band
ExampleBand

Those values may refer to the same artist.

A system does not automatically know that.

If the intended value is:

Plain text
Example Band

then the other spellings create inconsistent data.

Correcting inconsistent spelling makes matching and summarizing more reliable.

Capitalization Can Matter to Analysis

Consider:

Plain text
Jazz
jazz
JAZZ

A spreadsheet tool may group those values separately.

If the dataset intends one category, choose one representation and use it consistently.

The goal is not to say that one capitalization is universally correct.

The goal is for the dataset to use one intended form consistently.

Extra Spaces Are Still Characters

A value can look correct while containing hidden whitespace.

For example:

Plain text
Rock

and:

Plain text
Rock 

may appear nearly identical on screen.

The second value contains an extra trailing space.

That difference can interfere with matching, grouping, or lookup behavior.

Clean data avoids unnecessary leading or trailing spaces.

The Same Field Should Use the Same Kind of Value

Suppose a Year column contains:

Plain text
2024
2022
Unknown
2023

The field is mostly numeric, but one row contains text.

Whether that is acceptable depends on the dataset design.

The important point is that the field needs an intentional rule.

Possible approaches might include:

Do not mix formats accidentally.

Dates Need Consistency Too

Consider:

Plain text
8/5/2026
08-05-26
August 5, 2026

A person can interpret these dates.

A spreadsheet may interpret them differently depending on regional settings and data types.

A consistent dataset should use an agreed representation.

The exact format depends on the activity and system.

The key idea is consistency.

Categories Need Defined Vocabulary

Imagine a music dataset with Genre values:

Plain text
Hip Hop
Hip-Hop
Rap
Hip Hop/Rap

Are those intended to mean one category or several?

The dataset needs a clear decision.

Clean data is not only about correcting typing errors.

It is also about deciding which categories exist and using them predictably.

Missing Data Should Be Handled Intentionally

A blank cell can mean several things:

If the dataset uses blanks, make sure their meaning is understood.

Do not randomly mix:

Plain text
blank
N/A
Unknown
?
none

for the same missing-data meaning unless the dataset intentionally distinguishes them.

Duplicates Can Distort Results

Suppose the same song record appears twice.

A PivotTable could count two songs when only one unique record was intended.

Before removing a duplicate, confirm that it is truly a duplicate.

Two songs can share a title.

Two records can have similar values while representing different items.

This is one reason unique identifiers become important.

Cleaning Should Preserve Meaning

Data cleaning is not:

Change values until the table looks neat.

It is:

Make representations consistent without changing the underlying meaning.

If an artist's correct name is unusual, do not “fix” it into a more familiar name.

If two records represent different songs with the same title, do not merge them merely because the visible text matches.

Cleaning requires evidence about what the data is intended to represent.

Work One Field at a Time

A useful way to inspect a dataset is column by column.

For Genre, ask:

For Year, ask:

Focused review makes inconsistencies easier to notice.

Clean Data Supports Better Analysis

Sorting, filtering, PivotTables, and charts all depend on the underlying data.

If the dataset contains several accidental versions of the same category, the summary will reflect those accidental differences.

A polished chart cannot repair inconsistent source data.

Good analysis begins with data that represents the intended information consistently.