0.6.9 Clean Data Is Consistent Data

Small Differences Can Create Big Data Problems

People are good at recognizing values that “basically mean the same thing.”

Software often treats different representations as different values.

Imagine a Genre column containing:

Rock
rock
ROCK
Rock

A person may read all four as the same genre.

A spreadsheet summary may treat them as separate categories.

That can produce misleading counts and charts.

Clean data uses consistent representations for the same meaning.

Consistency Begins With Spelling

Suppose an artist appears as:

Example Band
Exampel Band
ExampleBand

Those values may refer to the same artist.

A system does not automatically know that.

If the intended value is:

Example Band

then the other spellings create inconsistent data.

Correcting inconsistent spelling makes matching and summarizing more reliable.

Capitalization Can Matter to Analysis

Consider:

Jazz
jazz
JAZZ

A spreadsheet tool may group those values separately.

If the dataset intends one category, choose one representation and use it consistently.

The goal is not to say that one capitalization is universally correct.

The goal is for the dataset to use one intended form consistently.

Extra Spaces Are Still Characters

A value can look correct while containing hidden whitespace.

For example:

Rock

and:

Rock

may appear nearly identical on screen.

The second value contains an extra trailing space.

That difference can interfere with matching, grouping, or lookup behavior.

Clean data avoids unnecessary leading or trailing spaces.

The Same Field Should Use the Same Kind of Value

Suppose a Year column contains:

2024
2022
Unknown
2023

The field is mostly numeric, but one row contains text.

Whether that is acceptable depends on the dataset design.

The important point is that the field needs an intentional rule.

Possible approaches might include:

  • require a valid year;
  • allow a defined missing-value representation;
  • place uncertain information in another field.

Do not mix formats accidentally.

Dates Need Consistency Too

Consider:

8/5/2026
08-05-26
August 5, 2026

A person can interpret these dates.

A spreadsheet may interpret them differently depending on regional settings and data types.

A consistent dataset should use an agreed representation.

The exact format depends on the activity and system.

The key idea is consistency.

Categories Need Defined Vocabulary

Imagine a music dataset with Genre values:

Hip Hop
Hip-Hop
Rap
Hip Hop/Rap

Are those intended to mean one category or several?

The dataset needs a clear decision.

Clean data is not only about correcting typing errors.

It is also about deciding which categories exist and using them predictably.

Missing Data Should Be Handled Intentionally

A blank cell can mean several things:

  • unknown;
  • not applicable;
  • not yet entered;
  • accidentally omitted.

If the dataset uses blanks, make sure their meaning is understood.

Do not randomly mix:

blank
N/A
Unknown
?
none

for the same missing-data meaning unless the dataset intentionally distinguishes them.

Duplicates Can Distort Results

Suppose the same song record appears twice.

A PivotTable could count two songs when only one unique record was intended.

Before removing a duplicate, confirm that it is truly a duplicate.

Two songs can share a title.

Two records can have similar values while representing different items.

This is one reason unique identifiers become important.

Cleaning Should Preserve Meaning

Data cleaning is not:

It is:

If an artist's correct name is unusual, do not “fix” it into a more familiar name.

If two records represent different songs with the same title, do not merge them merely because the visible text matches.

Cleaning requires evidence about what the data is intended to represent.

Work One Field at a Time

A useful way to inspect a dataset is column by column.

For Genre, ask:

  • Are spellings consistent?
  • Is capitalization consistent?
  • Are extra spaces present?
  • Do categories have clear meanings?

For Year, ask:

  • Are values represented consistently?
  • Are there impossible or malformed values?
  • How is missing information represented?

Focused review makes inconsistencies easier to notice.

Clean Data Supports Better Analysis

Sorting, filtering, PivotTables, and charts all depend on the underlying data.

If the dataset contains several accidental versions of the same category, the summary will reflect those accidental differences.

A polished chart cannot repair inconsistent source data.

Good analysis begins with data that represents the intended information consistently.