data-cleaning-annotation-workflow
by deyashmukh
DevOps & CloudBrief
Complete workflow for time series datasets (Energy, Manufacturing, Climate) on Kaggle to Data Annotation platform.
Description
What This Skill Does
Covers the end-to-end pipeline for preparing time series datasets from Kaggle and uploading them to the Data Annotation platform at data.smlcrm.com. Handles CSV cleaning with pandas, metadata configuration, column type and unit assignment, and group tagging until the dataset reaches CLEAN status.
Combines dataset discovery, cleaning, and platform-specific annotation configuration into one repeatable workflow instead of handling each step ad hoc.
When to Use It
- Preparing Kaggle energy datasets for ML annotation
- Configuring time/target/covariate columns in data.smlcrm.com
- Assigning group tags across all variables in a dataset
- Cleaning raw industrial CSVs before model training
- Uploading manufacturing or climate data with full metadata
Example Workflow
User asks: Prepare the steel industry energy consumption dataset from Kaggle for annotation
- 1
Downloads CSV from kaggle.com/datasets/csafrit2/steel-industry-energy-consumption
- 2
Runs scripts/clean_dataset.py to strip whitespace, remove duplicates, fill missing values, and parse timestamps
- 3
Uploads the raw CSV to data.smlcrm.com with name, domain, source URL, and description
- 4
Configures each column with its type (Time, Target, Covariate, Group) and units (kWh, kVarh, tCO2, ratio, seconds)
- 5
Selects all target and covariate variables, applies all group tags (WeekStatus, Day_of_week, Load_Type), then uploads the cleaned file
Dataset status changes from RAW to CLEAN with all columns typed, units set, and group assignments complete
Requirements
- Kaggle account (for dataset downloads via CLI or browser)
- Kaggle API key and kaggle.json configured (if using CLI download)
- Account on data.smlcrm.com
Installation
clawhub install deyashmukh/data-cleaning-annotation-workflowRun the CLI command, or paste the prompt into Claude Code, Cursor, Codex or whichever coding agent you use.