data-cleaning-annotation-workflow
deyashmukh tarafından
DevOps & CloudÖzet
Complete workflow for time series datasets (Energy, Manufacturing, Climate) on Kaggle to Data Annotation platform.
Açıklama
Bu Beceri Ne Yapar
Covers the end-to-end pipeline for preparing time series datasets from Kaggle and uploading them to the Data Annotation platform at data.smlcrm.com. Handles CSV cleaning with pandas, metadata configuration, column type and unit assignment, and group tagging until the dataset reaches CLEAN status.
Combines dataset discovery, cleaning, and platform-specific annotation configuration into one repeatable workflow instead of handling each step ad hoc.
Ne Zaman Kullanılır
- Preparing Kaggle energy datasets for ML annotation
- Configuring time/target/covariate columns in data.smlcrm.com
- Assigning group tags across all variables in a dataset
- Cleaning raw industrial CSVs before model training
- Uploading manufacturing or climate data with full metadata
Örnek İş Akışı
User asks: Prepare the steel industry energy consumption dataset from Kaggle for annotation
- 1
Downloads CSV from kaggle.com/datasets/csafrit2/steel-industry-energy-consumption
- 2
Runs scripts/clean_dataset.py to strip whitespace, remove duplicates, fill missing values, and parse timestamps
- 3
Uploads the raw CSV to data.smlcrm.com with name, domain, source URL, and description
- 4
Configures each column with its type (Time, Target, Covariate, Group) and units (kWh, kVarh, tCO2, ratio, seconds)
- 5
Selects all target and covariate variables, applies all group tags (WeekStatus, Day_of_week, Load_Type), then uploads the cleaned file
Dataset status changes from RAW to CLEAN with all columns typed, units set, and group assignments complete
Gereksinimler
- Kaggle account (for dataset downloads via CLI or browser)
- Kaggle API key and kaggle.json configured (if using CLI download)
- Account on data.smlcrm.com
Kurulum
clawhub install deyashmukh/data-cleaning-annotation-workflowCLI komutunu çalıştırın ya da istemi Claude Code, Cursor, Codex veya kullandığınız kodlama ajanına yapıştırın.