Learn OctOpus end to end.

The OctOpus loop

OctOpus runs one loop, end to end: connect data, it profiles and you ask questions, it trains and validates on a blind holdout, it delivers a proven model, you predict on new data; then it goes round again on the next batch. Each stop is a mascot; tap one to jump to that part of the guide.

↻

One loop

data in → model out

1 Connect dataupload or connect a source 2 Profile & askauto-profile, chat answers 3 TrainAI-written experiments 4 Validateblind holdout, vetoes 5 Delivermodel, report, OctoProof 6 Predictscore new data, repeat

Level 1

Get started

What OctOpus is, your first run in five steps, and which edition fits your team.

What is OctOpus

OctOpus automates recurring data-science work for enterprise teams: give it data and a business objective, and it builds, validates, and delivers the model and report without requiring a data scientist to operate it. It is an autonomous AI data scientist: an agent that understands your data, frames the problem, runs its own experiments, learns from what fails, validates winners on a held-out set it cannot see, and ships the best model with a signed proof of its reported quality.

Three things make it different from classic AutoML:

• It reasons, it doesn't enumerate. Each round is chosen from what the previous round actually showed, not the next cell of a fixed grid.
• It doesn't trust itself. Metrics are recomputed independently on an agent-blind holdout; leakage, degenerate predictions and metric substitutions are vetoed, not celebrated.
• You talk to it. The whole product is a conversation: describe the goal, review the plan, redirect mid-run, ask questions about your data, and get computed answers, not guesses.

Here's a complete run, start to finish (upload, plan, experiments, validated winner), in two minutes:

Everything you just watched is covered step by step in the sections below, same order as the Quickstart.

Quickstart

From zero to a validated model in five steps:

1. Open the app at /app (free tier, no credit card).
2. Load data: drop a CSV/Parquet/Excel file into the sidebar, pick a sample dataset, or connect a warehouse. OctOpus profiles it automatically and flags data-quality issues (ID columns, leaky features, heavy missingness) before you train.
3. Say what you want in chat, in plain language: predict customer churn forecast weekly revenue classify support tickets, only one experiment
4. Review the plan card: OctOpus proposes the target column, task, metric and experiment budget. Adjust any of them, then click ▶ Validate & train (or type go). Nothing trains until you validate.
5. Use the result: read the report, check the proof card, score new data in the Predict tab, hit the API, or download model.pkl.

Install OctOpus Desktop

The desktop edition runs the whole research loop on your own machine — profiling, training, dashboards and predictions all execute locally, and models, workspaces and the database stay on your disk. Pick your platform, then follow the six steps below.

1. Download the build for your platform above.
2. Install it. macOS: open the .dmg and drag OctOpus to Applications. Windows: run the installer. Linux: chmod +x the AppImage and run it, or sudo apt install ./OctOpus-linux-amd64.deb.
3. Launch OctOpus. The app starts its own local backend and opens the workspace — no server to configure.
4. Sign in and activate. Sign in to claim your seat, then paste your licence key under Settings → License & seats. One seat is one named person, on a bounded number of machines.
5. Connect your data. Drop a CSV, Excel or Parquet file into the sidebar, point at a local folder, or connect a warehouse.
6. Start working. Say what you want in plain language — the flow from here is identical to the Quickstart.

No licence yet? The app installs and opens without one; training, dashboards and chat activate when a key is applied. Contact us.

Web · Desktop · Enterprise

All three run the same engine, the same verification layer, and the same app. The rest of this guide applies to every edition.

Level 2

Using the app

Load data, talk to OctOpus, train, watch the run, verify the result, predict, build dashboards, transform data, compete on Kaggle.

Load your data

Three ways in, all landing in the same place. Once loaded, everything (questions, training, dashboards, transforms) works identically:

• Upload a file: CSV, TSV, Parquet, XLSX, JSON-Lines, or a ZIP of them. Drag it anywhere onto the app or use the sidebar.
• Connect a source: warehouses, databases, object storage, Google Sheets and more; see Data connectors. You can also ask in chat: find covid data online. OctOpus proposes real, importable datasets (Hugging Face, Kaggle, public sources) and you confirm before anything downloads.
• Pick a sample: say load the titanic sample or choose one from the sidebar to explore the product with known data.

On load, OctOpus profiles the dataset (types, distributions, missingness, target candidates) and posts a short data-quality review: things a reviewer would flag before training, like an all-unique ID column that a model would memorise. Address the high-severity ones before saying go.

Talk to OctOpus

Chat is the primary interface. With data loaded, OctOpus is a tool-using agent: when you ask a question about your data, it computes the answer against the actual rows (pandas and an embedded SQL engine run server-side) rather than guessing from a summary. Answers that came from real computation carry a provenance check mark.

• Ask anything about the data: how many customers churned last quarter? show me the top 20 rows by revenue which columns have missing values?
• Preview rows as real tables: show me 100 rows renders a scrollable table, not prose.
• Start real work: training, dashboards and transforms are all launched from chat; the sections below cover each.
• Attach a screenshot: paste or drop an image into the composer. A screenshot of a wrong chart or a confusing number is treated as a bug report: OctOpus identifies the defect and fixes it with the right tool instead of just describing the picture.

Charts never render in the chat column by design. Every visual lives on the Dashboard, where it's interactive and presentable. The chat answers with the numbers.

Speak, don't type: voice mode

Tap the mic and talk. OctOpus holds a real-time voice conversation about your data: it detects the language you're speaking automatically, answers out loud, and you can interrupt it mid-sentence. Major business languages are answered natively, and additional languages are supported on request. Pick your voice language from the mic menu; voice notes and call recordings can also be transcribed into datasets you can train on.

Call OctOpus on the phone

Teams on an enterprise plan can dial OctOpus and talk to it: a real phone call, answered by the same brain as the chat. Ask product questions, talk through a modeling problem, or get pointed to the right part of the app; the agent answers in short spoken sentences and hands anything screen-shaped back to the app. Operators wire it to their own number in minutes (Twilio webhook: see the self-hosting docs); calls are signature-verified and conversations aren't persisted.

Three working modes

The domain switcher (Data Science / Data Analysis / Data Engineering) pins how your messages are interpreted. Data Analysis is the default: questions, stats, dashboards. Pin Data Science and messages refine the training plan; pin Data Engineering and every message becomes a transform step. Unpinned, OctOpus routes each message by intent and asks before switching you across domains.

Train a model

1 · State the goal

Plain language, with as much or as little detail as you have:

predict churn forecast revenue per customer, monthly use TabPFN and only one experiment classify tickets by urgency, optimise F1

2 · Review the plan card

OctOpus analyses the data against your goal and proposes a plan: target column · task · metric · budget, plus any models you named. Everything is editable on the card: swap the target from a dropdown of your real columns, change the task or metric, set the number of experiments.

• Validate & train: the run starts. Typing go or yes does the same thing.
• Revise in chat: type a correction (the target should be lifetime_value, it's regression) and the plan re-draws around it.

Budgets

The budget is how many experiments the run may spend. Say it in numbers or words (3 experiments, only one experiment, just a single experiment all work) or set it on the card / the sidebar slider. What you said, what the card shows, and what actually runs are kept in lock-step.

Pinning models

Name a model and it becomes part of the agreement: only use TabPFN pins the entire run to TabPFN. The plan card shows Models you asked for (with a clear button to take it back), the repair loop is forbidden from swapping families, and if your pinned model genuinely cannot run, the run fails honestly rather than quietly training something else. Model names are typo-tolerant (tabfpn still pins TabPFN).

While a run is going

• Watch it think: the run narrates each stage: profiling, writing the research plan, preparing each experiment, training, validating, deciding what to try next. The Model tab shows the live leaderboard; the Logs tab has the raw feed; the Code tab shows the actual training code being written.
• Keep chatting: questions get real answers grounded in the live run state (what's going on? what's the best score so far?).
• Steer it: strategy messages become live hints that shape the next experiment: focus on the tenure features try a deeper model next optimise recall instead. If a hint changes the plan (target/task/metric), the card updates and the change takes effect next experiment.
• Stop anything by saying so: type stop / cancel / abort in chat to cancel whatever is in flight: a training run, a dashboard build, profiling, autopilot. The Esc key and the composer's stop button do the same. Best results so far are kept.
• Interrupted runs resume safely: if the server restarts mid-run, the run is marked resumable and the held-out data is hash-verified on resume, so metrics scored before and after the restart remain comparable. A tampered or lost holdout fails loudly rather than silently re-splitting.

Results & artifacts

When the run finishes you get, per run:

• The leaderboard: every experiment with its model, metrics, and status. Experiments the engine vetoed (leakage-suspect perfect scores, degenerate constant predictions, metric substitutions) are marked DISCARDED with the reason; they are never crowned, no matter how good the number looked.
• The report: a plain-language account of the plan, every experiment, the winner, validation, and recommendations. Downloadable; shareable via a link.
• Artifacts: the trained model (the full pipeline), the winning training code, the preprocessing pipeline, the research plan, metrics, feature importance / SHAP, and the signed certificate. All downloadable from the Artifacts tab as a bundle.
• Next steps in one click: deploy as an API, predict on a file, continue the run with more experiments, or re-run with changes.

Verify with OctoProof

Every completed run issues a signed, machine-verifiable certificate (proof_card.json) assembled from the run's own artifacts, never from the model's narration:

• the hash-pinned holdout manifest, re-checked at issue time (intact_at_issue);
• the champion's claimed metric vs the independently recomputed holdout metric, and the gap between them;
• every engine veto that fired (leakage, degenerate predictions, suspiciously perfect scores) with reasons;
• every metric substitution (planned metric vs what was actually reported);
• the baseline floor, dataset shape, and library versions.

The card is canonically serialized, hashed, and HMAC-signed. Change any number; verification fails. Missing evidence is stated ("not_carved"), never invented.

# fetch a run's certificate + live verification report
GET /api/run/{run_id}/proof

# verify ANY certificate: public, no auth; verification is the point
POST /api/proof/verify        (body: the certificate JSON)

# verify fully offline
python proof_card.py verify proof_card.json

When ground truth arrives later, the certificate is amended with realized production accuracy and re-signed. The previous signature is preserved in a chain, so the document's history is append-only. Hand the certificate to anyone; they don't have to trust you, or us.

Predict on new data

• In the app: open the Predict tab (or click Predict on data on the finished-run card) and drop a CSV with the same feature columns. Predictions render with per-row explanations and download as CSV.
• Via the API:

POST /api/run/{run_id}/predict-file
Content-Type: multipart/form-data
file: <your CSV with the same feature columns>

→ 200 OK
{ "predictions": [0.812, 0.034, ...], "total_rows": 1024 }

• Self-hosted: download model.pkl + prepare.py from Artifacts and serve them yourself with joblib.load.
• On a schedule: recurring scoring + drift checks via /api/schedules; see Monitoring.

Dashboards

Ask for one in chat: bi, dashboard, or a brief like build a dashboard about revenue by region. OctOpus composes a full analytical page from your data: KPI scorecards, charts chosen for the data's shape, and a written narrative whose figures are fact-checked against the real rows. Composing is a background job that can take a minute or two; progress streams in the Dashboard tab.

• Edit it in chat: make the churn chart a line chart, add average order value by month and remove the tenure histogram. Batch several changes in one message; the page recomposes as a whole.
• Filter: each categorical dimension gets a dropdown filter; picking a value cross-filters every chart (counts recompute from the matching rows, Power-BI-style). Clicking a bar / slice / treemap block does the same; the active filter shows as a clearable chip.
• Present: Present mode fullscreens the dashboard for meetings; theme toggles and a palette picker keep it on-brand.
• Persist: dashboards are cached per dataset/run and restore on reload; Download exports a standalone HTML file.

Clean & transform data

Two ways to reshape data before (or instead of) training, both producing an auditable, replayable pipeline (Power-Query-style applied steps, written as pandas by the model, gated by a safety layer, and previewed before anything is committed):

Data Engineering mode

Pin the Data Engineering domain and every chat message is a transform instruction: drop the website column fill missing ages with the median one-hot encode contract type. Each step shows the resulting preview; the applied-steps stack is visible, reorderable and undoable.

Slash commands (any mode)

/transform drop the website column      # apply one step
/transform list                          # show the applied-steps stack
/transform clear                         # revert to the original dataset
/transform save                          # export the pipeline to the notebook

Asking for a general clean-up (clean this dataset) proposes a reviewable plan with Apply/Cancel; it never mutates your data without confirmation. A derived training target (e.g. a churned flag that doesn't exist yet) is created by a transform first, then the training plan points at the new column.

Kaggle competitions

OctOpus can run a Kaggle competition end to end, with your own Kaggle account, spending your own quota, under your name:

1. Connect your account: add your Kaggle API token in Settings. Credentials are per-user, encrypted, and verified with a real round-trip. Competition actions never fall back to anyone else's account.
2. Open a competition: the train/test/sample files download into one dataset so the whole pipeline (training, conformance, submission) sees them together.
3. Train: as usual; the run knows it's competition mode and produces a submission.csv conformed to the sample (column order, header case, ID handling, class label decoding).
4. Submit: from the app. Before spending one of your daily submission slots (typically 5/day), the conformance report is checked; a file that would be rejected is blocked with the reasons listed. You can force past the gate (it's your quota), but you'll do it knowingly.

Level 3

Data

Every way data gets in (connectors, warehouses, public datasets) and how far it scales.

Data connectors

All connectors land data in the standard dataset cache, so discovery, training, chat and predict consume it exactly like an upload. Credentials are encrypted at rest with Fernet using a per-install random secret and never shown to LLM-authored code.

• File upload: CSV, TSV, Parquet, JSON-Lines, XLSX, ZIP archives.
• Cloud warehouses: BigQuery, Snowflake, Redshift, Databricks SQL, ClickHouse, Vertica. Queries push down to your warehouse; only the result set moves.
• Operational databases: Postgres, MySQL, SQL Server, MongoDB, Supabase (any SQLAlchemy URL + a read-only query).
• Object storage: Amazon S3, OneDrive, SharePoint, Google Drive.
• SaaS / collaboration: Google Sheets, Notion.
• Public data: Hugging Face Hub (full-text search, parquet import), Kaggle datasets (search by votes, single-file import), WMO WIS 2.0 weather/climate observations (global discovery catalogue, auto-pivoted to ML-ready tables).
• Marketing / messaging / BI: Google Ads, Meta Ads, Slack, Power BI.

Governed imports (column-level access policy, PII masking, materialization audit, end-to-end lineage) are available for enterprise workspaces.

Size limits & performance

• Files don't need to fit in RAM. Profiling and SQL over your dataset run on an embedded out-of-core columnar engine (DuckDB), multi-threaded, spilling to disk as needed. Chat's SQL answers on a multi-GB file are seconds, not minutes.
• Warehouse-scale SQL stays in your warehouse. Snowflake / BigQuery / Databricks queries execute there; OctOpus materializes only the result.
• Default caps: 2 GB per upload, 50 GB per dataset; both are policy knobs (OCTOPUS_MAX_UPLOAD_MB, OCTOPUS_MAX_DATASET_MB), not architectural limits.
• Heavy training offloads. GPU-hungry experiments (deep tabular, transformers, foundation models) can run on serverless GPU workers; the local install stays responsive.

Level 4

The engine

How the research loop thinks: experiments, model families, features, validation, tuning.

The research engine

The OctOpus engine is a closed-loop research agent. Where a traditional AutoML platform runs a fixed pipeline (pick from a model catalog, hyperparameter-search inside fixed bounds, ensemble the top-K), OctOpus operates the way a senior data scientist would: profile, plan, code, run, diagnose, iterate, validate, ship. Each round is a fresh decision informed by the prior round's results.

1. Profile: type inference, missingness, leakage detection, target imbalance, time-aware split eligibility, suggested features.
2. Discover: frame the task (target column, task type, validation strategy, headline metric), surfaced on the plan card for your confirmation.
3. Decide & write: the agent writes the research spec and the code for the next experiment.
4. Execute & reflect: the experiment runs in a sandbox; the agent reads the metrics and any errors, and decides what to try next.

Experiments are task-aware: the agent starts from a sound, domain-matched baseline and moves to stronger and more specialised model families as the evidence warrants. A rotation guard stops it settling on one family: if a family stops earning its place, the next experiment must try something else.

Supported models

Gradient boosting (the workhorses)

Modern tabular deep learning

Time-series neural networks (NeuralForecast)

Foundation models (zero-shot)

Classical ML (small-data fallback)

For datasets under ~500 rows, OctOpus prefers Ridge, ElasticNet, RandomForest, ExtraTrees, kNN, or SVM over deep nets: stable cross-validated estimates without overfitting.

NLP

• TF-IDF + Logistic Regression / SVM: strong baseline for short-text classification.
• SetFit: few-shot fine-tuning of a sentence-transformer.
• HuggingFace transformers: fine-tune of any encoder on the Hub.
• Sentence-transformers + classifier head: frozen encoder, lightweight head.

Image classification

• ResNet18 / ResNet50: torchvision pretrained backbones with a fine-tuned head.
• EfficientNet / EfficientNet-V2: better accuracy/parameter trade-off.
• ViT: via timm, for larger labelled datasets.

Supported tasks

• Binary classification: churn, fraud, conversion. Metrics: ROC-AUC, PR-AUC, F1, log-loss, recall@top-k.
• Multi-class classification: one of N classes. Metrics: accuracy, macro-F1, log-loss.
• Multi-label classification: any subset of N labels per row.
• Regression: revenue, price, score. Metrics: RMSE, MAE, R², MAPE.
• Time-series forecasting: single-series, panel/hierarchical, with exogenous variables. Metrics: WAPE, sMAPE, RMSE, MASE.
• NLP classification & regression: text inputs of any length.
• Anomaly detection: unsupervised (isolation forest, autoencoder, OneClassSVM) or rare-class supervised.
• Image classification: fine-tune of a pretrained backbone.

Recommendation, ranking, and survival analysis are on the roadmap.

Feature engineering

OctOpus generates a prepare.py per run that captures every preprocessing step:

• Type inference: categoricals, numerics, timestamps, IDs auto-detected.
• Missing-value imputation: median for numeric, most-frequent for categorical, learned indicator columns.
• Categorical encoding: target encoding, one-hot, ordinal, embeddings (deep models).
• Date features: day-of-week, week-of-year, hour, is-weekend, is-holiday, lag features (time series).
• Text features: TF-IDF, character n-grams, sentence-transformer embeddings.
• Numeric transforms: log, square-root, quantile-uniform, standardization, robust scaling.
• Interaction features: pairwise products / ratios when the data shape supports it.
• Leakage detection: drops columns with near-perfect target correlation or future-dated conditions.

The pipeline is reproducible: the same prepare.py applied to a new CSV produces the same features.

Validation & holdout governance

Every run uses an agent-blind holdout: a subset split off before any feature engineering or model code is shown to the LLM, stored outside the agent's workspace, and pinned by content hash in a manifest. The "winner" of a run is the model that generalizes to this holdout. Its metric is recomputed independently, never taken from the training script's own claims.

On top of the holdout, the engine enforces vetoes that no experiment can talk its way past:

• Suspiciously-perfect scores are leakage, not skill: vetoed, never crowned.
• Degenerate predictions (constant output, worse than the mean baseline): vetoed.
• Metric substitutions (planned AUC, reported accuracy): flagged, corrected next round, recorded on the proof card.
• A large CV-vs-holdout gap: the honest holdout number wins.

Validation strategies: stratified k-fold (classification), k-fold (regression), time-series split (no future bleeding into training; mandatory when a timestamp is detected), group k-fold (explicit group columns), walk-forward (forecasting).

Tuning & ensembles

Hyperparameters are tuned against a cross-validated objective, with the search budget scaled to the size of your dataset and unpromising trials cut short rather than run to completion.

Once there are several strong candidates, OctOpus will combine them into an ensemble, deliberately mixing different kinds of model, since agreement between similar models adds little. Where a single model is unstable, it is averaged over multiple fits instead.

Level 5

Developers

Drive everything from code: the REST API, the Python SDK, the CLI, MCP, and your own server.

REST API

Everything the app does is an HTTP call. The core surface:

Authenticate with your session token as a Bearer header. The full route list ships with the server's OpenAPI spec at /openapi.json.

Python SDK

The octopus_ml client wraps the API for notebook and pipeline use:

pip install octopus-ml-sdk

from octopus_ml import OctopusClient
import pandas as pd

client = OctopusClient(base_url="https://www.octoopus.dev", api_key="...")
df = pd.read_csv("customers.csv")

result = client.run(df, goal="predict churn, optimise AUC")
print(result.best_model, result.metrics)
preds = client.predict(result.run_id, df_new)

Command line

Every OctOpus surface is scriptable: drive a run, score a file, or verify a model certificate from a terminal, a CI job, or an agent. The SDK wraps the REST API; the platform ships as importable Python; certificates verify with zero dependencies.

# 1 · install the client and run a model from a CSV
$ pip install octopus-ml-sdk
$ octopus run customers.csv --goal "predict churn, optimise AUC" --budget 3
  ✓ profiled 7,043 rows × 21 cols · target=Churn · task=binary · metric=AUC
  ✓ exp1 AUC 0.842   exp2 AUC 0.857   exp3 AUC 0.861
  ★ winner · holdout AUC 0.858 · proof_card.json signed

# 2 · score new data against the validated winner
$ octopus predict --run run_a1b2c3 new_customers.csv > predictions.csv

# 3 · verify a model certificate: anywhere, offline, no account
$ python proof_card.py verify proof_card.json
  [PASS] payload_hash: content matches its recorded hash
  [PASS] signature:    issued by the holder of this key
  VERIFIED

Prefer Python? The same three steps in a notebook or pipeline:

from octopus_ml import OctopusClient
client = OctopusClient(base_url="https://www.octoopus.dev", api_key="...")
run = client.run(df, goal="predict churn, optimise AUC", budget=3)
preds = client.predict(run.run_id, df_new)
cert  = client.proof(run.run_id)          # the signed certificate

MCP server

OctOpus provides a Model Context Protocol server, so Claude Code, Cursor, or any MCP-aware agent can drive the platform as tools. It's included with the Desktop and Enterprise editions and connects to your OctOpus instance; setup instructions ship with your install.

Exposed tools include profile_csv, grade_dataset, estimate_run_cost, discover_direction, start_research_run, resume_run, get_run_status, stream_run_events, poll_until_done, compare_runs, list_run_artifacts, fetch_artifact, export_endpoint, predict, bootstrap_project, list_priors, get_realized_accuracy / submit_ground_truth, and the WMO dataset search / import pair, each mapping to the same governed REST API the app uses.

Private & self-hosted deployment

OctOpus can run entirely inside your own environment: fully on-device via the Desktop edition, or inside your own AWS / GCP / Azure account as a private / VPC install via OctOpus Enterprise. Both run the same engine, the same agent-blind holdout governance, and the same signed proof cards as the managed app, with bring-your-own model keys. Your data never leaves your environment.

Provisioning, configuration, SSO/SCIM, and managed updates are handled with your team during onboarding. Talk to us at ceo@octoopus.dev or see /enterprise.

The model you pick is yours. OctOpus is multi-provider by design (Claude, GPT, Gemini, DeepSeek, Kimi, NVIDIA NIM, or your own self-hosted endpoint), with an automatic fallback ladder when a provider is unavailable.

Level 6

Platform

Run it your way (desktop, enterprise VPC) and keep it safe: security, monitoring, drift.

Desktop app: Mac, Windows & Linux

The fully on-device edition runs entirely on your machine. Point it at a folder of CSV/Excel/Parquet files and it runs the whole pipeline right there. Your data never leaves the device; the only outbound traffic is to the LLM provider you configure. Same app, same engine, same signed proof cards. Ideal for HIPAA / PCI / AML and air-gapped-adjacent environments. Here it is on a real desktop:

Download OctOpus Desktop →

OctOpus Enterprise

Deploys inside your AWS, GCP, or Azure account as a private / VPC install: SSO/SCIM, audit logging, custom roles, dedicated workspaces, governed data integration (column-level policy, PII masking, lineage), and the same agent-blind holdout governance as the public app. Compliance documentation (SOC 2, GDPR, HIPAA, PCI) available under NDA: ceo@octoopus.dev. See /enterprise.

Security

Full posture at /security. Highlights:

• Encryption: TLS 1.2+ in transit; AES-256 / Fernet at rest.
• Key isolation: research-run subprocesses launch with a scrubbed environment; LLM-authored code cannot read provider API keys.
• Holdout governance: holdout data lives outside the agent's workspace, pinned by content hash.
• Secret redaction: all log streams pass through a redaction layer before reaching disk or UI.
• Signed certificates: every run's reported quality is independently verifiable (OctoProof).
• SSO/SCIM: SAML 2.0 + OIDC + SCIM 2.0 (Enterprise).
• Audit log: every run, every experiment, every error, queryable per workspace.

Monitoring, drift & explainability

Once a model is in use, OctOpus monitors input drift (KS test on numerics, chi-square on categoricals), prediction drift, schema drift (columns gained/lost/renamed), and, when ground truth arrives, realized performance, which is joined back to logged predictions, stamped onto the run's proof card, and used to re-rank what the engine tries first on similar data next time.

Every run also ships explainability artifacts: feature importance, SHAP values (bar + beeswarm), partial-dependence plots, and per-prediction reasoning in the Predict tab.

Level 7

Reference

You made it to the top. Every term the product uses, defined.

Glossary

Agent-blind holdout

A test set kept outside the agent's workspace so the LLM-authored experiment code cannot read it. The validated winner of a run is the model that holds up on this set.

AutoML

"Automated machine learning": historically a fixed pipeline (model catalog → grid search → ensemble). OctOpus uses the term loosely; the engine is autonomous, not just automated.

Autonomous AI data scientist

An AI agent that runs the full data-science loop (profile, frame, code, experiment, diagnose, validate, deploy) with minimal human input. Each round is a fresh decision, not a fixed step in a pipeline.

Baseline experiment

The first experiment of every run. Domain-matched single model + heavy feature engineering. Tabular → CatBoost / LightGBM. Time series → LightGBM with lag features. NLP → TF-IDF + Logistic Regression. Images → ResNet18.

Cross-validation (CV)

Estimating model performance by training on K-1 folds and validating on the K-th, rotated K times. Stratified k-fold by default for classification, k-fold for regression, time-series split for forecasting.

Discovery phase

The step where OctOpus picks the target column, task type, validation strategy, and headline metric from your plain-English goal and the data profile. Surfaced on the plan card for confirmation before any code runs.

Engine veto

A deterministic rule that overrides the agent's own KEEP decision: suspiciously-perfect scores (leakage), degenerate predictions, and holdout contradictions are discarded no matter how good the number looked. Vetoes are recorded on the proof card.

Experiment

One model-building attempt: a complete training script, executed, with its metrics parsed. A research run produces 1 to 15 experiments depending on the budget.

Foundation model

A model pretrained on a very large dataset, applied zero-shot to a new task. Used by OctOpus: TabPFN (tabular), Chronos / TiRex / TimesFM / Moirai (time series).

Headline metric

The single number that defines "best" for a run. Picked per task (ROC-AUC for binary classification, RMSE for regression, WAPE for forecasting), or set by you on the plan card.

Holdout manifest

The content-hash record of the holdout carve. A resumed run must match it exactly; a changed or missing holdout fails loudly instead of silently re-splitting.

Leakage

When a feature accidentally encodes the answer (a column populated AFTER the event you're predicting, an ID with target information, a time-leaking aggregate). Flagged at profiling; vetoed at scoring.

OctoProof / proof card

The signed, machine-verifiable certificate every run issues: holdout hashes, independently recomputed metrics, vetoes, substitutions, baseline floor. Verifiable by anyone at POST /api/proof/verify or offline.

Plan card

The reviewable card (target · task · metric · budget · requested models) that gates every training run. Nothing trains until you validate it.

program.md

The research spec written by OctOpus before any experiment runs. The next experiment reads it back.

Research run

A complete loop: profile → discover → experiments → validate → ship. Identified by a unique run ID.

Rotation guard

The engine policy preventing the agent from getting stuck on one model family: same family twice in the last eight experiments → the next must rotate.

SHAP

Per-row, per-feature contribution to the model's prediction, shipped with every run's artifact bundle.

Stacking

An ensemble where a meta-learner is trained on the out-of-fold predictions of multiple diverse base learners.

Validated winner

The single best model from a run, judged on the agent-blind holdout with an independently recomputed metric. The artifact downloaded as model.pkl.