How do you design data quality requirements?

Question

Accepted Answer

**Data quality** — fitness of data for its intended use.

**Dimensions:**
- **Accuracy:** Data reflects real-world values.
- **Completeness:** No missing required data.
- **Consistency:** Same data across systems.
- **Timeliness:** Data available when needed.
- **Validity:** Data conforms to defined format/rules.
- **Uniqueness:** No unwanted duplicates.

**Requirements specification:**
- Define quality metrics per data domain.
- Set thresholds (e.g., 99.5% completeness).
- Define validation rules (format, range, referential).
- Establish monitoring and alerting.

**Implementation:**
- **Validation rules:** NOT NULL, CHECK constraints, FK.
- **Data profiling:** Analyze data patterns and anomalies.
- **Cleansing:** Deduplication, standardization, enrichment.
- **Monitoring:** Quality dashboards, anomaly alerts.

**Tools:**
- Great Expectations, Deequ, dbt tests.
- Monte Carlo, Soda.

**Governance:**
- Data stewards for quality ownership.
- Issue tracking and remediation process.

Systems Analyst: How do you design data quality requirements?