◇ Predictive ML Flow React · Node · Flask · Django · Mongo Project Blueprint
Predictive Lead Scoring

A wizard that turns a CSV into a scored, explainable ICP.

Upload your CRM export, tell it which value counts as a win, and it trains a classifier that scores every record — then explains what drove each score and profiles the customers who actually convert.

Below, the full pipeline, what happens at each stage, and how it's built.
01

The Pipeline

Five stages, one direction. The first three are yours to set; the last two the system produces. Between them sits the data engineering most mental models leave out.

📄 01 Select data CSV → problem 🎯 02 Define the outcome one value = 1 🔧 03 Prepare the dataset validity · impute · balance 🧠 04 Train the model forest or boosting 📈 05 Score & explain probability + SHAP → ICP persona
you configure the system computes runs left to right, once per model run
📄
01
Select data
Upload the CSV, pick the business problem.
🎯
02
Define the outcome
One field, one value that counts as a win.
🔧
03
Prepare the dataset
Validity tests, features, imputation, balance.
🧠
04
Train the model
Random forest or XGBoost, tuned by grid search.
📈
05
Score & explain
A probability per record, plus why it got it.
outputs the ICP persona + driver list
02

Inside Each Stage

📄

01 Select data

Two choices hide inside this one: which rows, and which columns. The CSV you upload plus the problem card you pick decide the rows — each lands in a per-problem collection stamped with the org and object type, and the header row is sniffed to guess Lead, Opportunity, or Account. Which columns become features is decided later, at stage 3. Re-uploading for the same org replaces that org's rows rather than merging them.

Tab 0 · Node import layer · ml-problem-<problem_id>
🎯

02 Define the outcome

The model is a binary classifier, so the outcome has to collapse to 0 or 1. You pick a field, pick the single value that means success, and a new label column is written where that value becomes 1 and everything else becomes 0. The counts come back immediately so you can see the class balance before committing. It happens through the chat assistant, not a form — no code, no SQL.

Tab 0 chat · setProgressedToGroupByFieldValue → ProgressedTo_Group_*
🔧

03 Prepare the dataset

The stage most summaries skip. Mark ID and date columns so they're excluded, then run statistical validity — ANOVA across numeric features, Chi-squared across categorical ones — to get a straight yes/no on whether each column relates to the outcome at all. Then choose the final attributes, optionally enrich them (website, email domain, industry, revenue band), impute what's missing, and pick a class-balancing strategy. Synthetic rows are available when the positive class is too thin to learn from.

Tabs 2–3 · Django validity, enrichment and synthetic-data workers
🧠

04 Train the model

Everything passes through one column transformer first: numerics get median-imputed and scaled, semicolon-delimited picklists get split into one binary column per value, booleans expand into a true and a false column, ranked fields collapse to present-or-not. Then a train/test split, balancing by SMOTE oversampling or stratified undersampling, and a random forest or XGBoost fitted with grid search over depth and tree count. Model and preprocessor are written to S3; metrics go to Mongo.

Flask trainer · Randomforest_model.py / XGBoost_model.py
📈

05 Score and explain

The score is a probability — the model's estimated chance that a record is a positive. Anything above 0.5 becomes the high-intent set that gets explained record by record. Tree importance tells you which features the model leaned on overall; SHAP tells you how each feature pushed one specific record's score up or down. That difference is what makes "why is this lead hot?" answerable rather than just "this lead is hot."

Tabs 5–6 · SHAP payload · MLMostImportantRange · idealLeadProfile
The score, in one line
predictions_df['prediction_score_1'] = predictions_proba[:, 1].round(4)

Everything downstream — the high-intent cut, the SHAP explanations, the ideal value ranges, the persona card — is built on this single column.

03

The Eight-Stage Wizard

One page, eight panels. The five-stage pipeline above is what the user experiences; these are the tabs it's actually delivered through.

TAB 0
Prediction Event

Pick one of six business problems — lead scoring, ideal customer, churn, expansion, pipeline velocity, attribution — upload the CSV, let the object type be detected, then set the outcome in chat. Cards that don't match the detected object type grey out.

TAB 1
Data Pipeline

Readiness, schema, and quality checks on what was just imported.

TAB 2
Attribute Validity

Flag IDs and dates for exclusion, then test every remaining column against the outcome — ANOVA for numerics, Chi-squared for categoricals — and get back a related / not-related map.

TAB 3
ML Dataset

Choose the final attributes, enrich them, impute missing values, set the balance strategy, and optionally generate synthetic rows.

TAB 4
Model Setup

Algorithm, balancing method, per-attribute encoding and scaling, train/test split. The Train button fires the job from here.

TAB 5
Model Result

Accuracy, precision, recall and F1 tiles, the attribute importance chart, the feature range table, the ROC curve, and both confusion matrices.

TAB 6
SHAP

Per-record waterfall charts, positive and negative driver lists, and the Ideal Customer Profile persona card.

TAB 7
Production locked

Recurring scoring on new records as they arrive. Not open in the UI yet.

04

Architecture

Three tiers, one orchestrator. Worth knowing which channel is which, because they fail in different ways.

Frontend

React + TypeScript + Vite · PredictiveMlFlowPage.tsx

A single page switching panels on one tab variable. All eight stages, roughly three thousand lines, no routing between steps.

Orchestrator

Node + Express · mlModelData.js

The centre of gravity. Owns the multi-tenant Mongo database and the S3 buckets, and decides what gets dispatched where. Every query is scoped by organisation, so tenants never see each other's rows.

Synchronous HTTP

Flask trainer

train · score · validate

The actual training call, made and awaited in-process. If this is down, training fails loudly and immediately.

SQS FIFO + Lambda

Django workers

validity · enrichment · industry · revenue · synthetic

Heavy jobs queue up, a Lambda boots the box that hosts them, the queue drains, results report back. Failures here are quiet and delayed rather than immediate.

Shared stores

MongoDB + S3

Tenant_<customer> · ml-models-data1

The tiers don't push large payloads to each other — they hand over pointers. Node writes the rows and buckets; Python and Django read the same ones.

05

How It's Built

Six modular blocks. Algorithms and enrichment sources swap per client; the shape of the pipeline doesn't.

01

Problem store

ingest

Holds the uploaded rows, one collection per problem, scoped to the organisation.

02

Outcome labeller

binary target

Turns a chosen field and value into the 0/1 column the model learns against.

03

Validity & selection

feature gate

Statistical tests decide which columns reach the model, and which don't.

04

Preparation

dataset prep

Imputing, encoding, scaling, and balancing the training set.

05

Trainer

the model

Fits, tunes, evaluates, and stores the model and its metrics.

06

Explainer

the why

Turns a trained model into drivers, ideal ranges, and the ICP persona.

06

Stack

React TypeScript Node.js Flask Django MongoDB S3 SQS + Lambda scikit-learn XGBoost SHAP SDV