Quick reference
Mislabel 2D
Use it when you have a standard image classification dataset (class_a/, class_b/, …) and suspect some images were mislabeled.
Inputs. Folder‑structured image dataset (JPEG, PNG, WebP, TIFF).
Per‑image fields:
What to expect. A ranked list of suspected mislabels, each with a
proposed correct label and a confidence score. Triage from highest
gap to lowest.
Mislabel 3D (medical imaging)
Use it when you’re working with CT, MRI, or ultrasound data organized per patient or per volume. Inputs..nii.gzvolumes.- NIfTI‑style PNG stacks (
label_*/case001_0042.png). - Optional segmentation metadata to restrict analysis to an anatomical region of interest.
- Case‑level probability, whether the whole volume is suspect.
- Neighbor review, whether slices directly above and below disagree.
- Multi‑label probability, probability across the full label set for multi‑label cases.
Mislabel Broad (OOD + mislabel)
A superset of Mislabel 2D. Besides wrong labels, it finds out‑of‑distribution images, things that don’t belong in the dataset at all (logos, screenshots, unrelated photos, corrupted files). Use it when the dataset is from mixed sources, scraped from the web, or otherwise possibly contaminated. Issue types. Every flag is tagged with exactly one:mislabel, definitely the wrong class.ood, does not belong in the dataset.low_quality, the model is uncertain but no clear alternative.
- Mislabeled, high confidence wrong label.
- MislabeledCandidate, probable but less certain.
- MislabeledCandidateWeak, weak signal.
- Outlier, OOD, not a mislabel.
Poisoning
Use it when you’re ingesting data from untrusted sources (scraped web, user uploads, third‑party dumps) before training a production model. Catches. Backdoor triggers, adversarial patches, Glaze and Nightshade style‑transfer attacks, and any image tampered with to manipulate training. Per‑image fields:
Tunable sensitivity:
What to expect. Suspect images come with a visual explanation:
highlighted trigger regions, feature heatmaps, and a plain‑language
summary of which signals fired.
Text Analysis
Use it when you’re building fine‑tuning, RAG, or instruction datasets and need to sanitize them before training or publishing. Catches three problem classes in one pass:- Leaked secrets and PII, API keys, tokens, passwords, emails, IPs.
- Prompt injection attempts, system‑prompt overrides, jailbreaks.
- Topic outliers, paragraphs that don’t belong with the rest.
Built‑in patterns. GitHub PATs, AWS access keys, JWTs, generic
api_key=… / password=…, emails, IPv4, IPv6, plus a high‑entropy
token heuristic.
Built‑in injection phrases. A curated list of 12 phrases
(ignore previous instructions, reveal the system prompt,
disregard the earlier rules, …) plus jailbreak and role‑reassignment
heuristics.
What to expect. A searchable table of findings per document, with
the match highlighted in context. Triage feedback (confirm / dismiss)
is remembered across subsequent scans, so repeat patterns stop
nagging you.
Bias & Shortcut
Use it when you’re about to train a model and want to know whether it will actually learn the task or cheat on a spurious cue. Catches:- Background color correlates with label.
- Watermarks or logos leak the class.
- Resolution, aspect ratio, or file metadata is informative.
- A confounder column (site, scanner, date) determines the outcome.
Per‑image flags.
is_bias (subtle), is_shortcut (strong), plus
the underlying score.
Confounder columns. Drop an extras.csv alongside your images
with columns like site, scanner, date, patient_id. The engine
tests each one for label correlation and train/test shift.
What to expect. A dedicated bias results page
(/datasets/:id/bias) listing every hypothesis with its decision and
visualizations. Confirmed shortcuts come with specific mitigations.
Mask Quality
Use it when you’re preparing data for semantic or instance segmentation. Catches. Missing labels, inconsistent sizing, class imbalance, invalid pixel values. What to expect. Per‑mask flags with a textual description of what’s wrong and an overlay preview against the original image.Picking the right engine
I just got a dataset and want a quick health check
I just got a dataset and want a quick health check
Run Mislabel Broad. It catches mislabels, OOD, and low quality
in one pass and gives you the cleanest first impression.
I'm about to train and worry about shortcuts
I'm about to train and worry about shortcuts
Run Bias & Shortcut with your
extras.csv. Read the confirmed
hypotheses before training; train on the cured branch if any
confirmed shortcut would have leaked into the model.The dataset comes from the public web
The dataset comes from the public web
Run Poisoning with
aggressive=true for the first pass to see
the worst offenders, then drop sensitivity and run again to
settle on a production threshold.I'm building a fine‑tune corpus
I'm building a fine‑tune corpus
Run Text Analysis before anything else. Anything
CRITICAL
should be redacted with Healing before
training.
