{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "intro",
   "metadata": {},
   "source": [
    "# 04 - Full pipeline\n",
    "\n",
    "End-to-end workflow on the **MALDI-Kleb-AI** Amikacin task: optional batch-effect correction with MaldiBatchKit, on-the-fly augmentation with `SpectrumAugment`, stratified cross-validation with a deep classifier.\n",
    "\n",
    "Uses the same Zenodo dataset as the other notebooks (see notebook 01 for caching)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "load",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-04-28T09:53:43.321341Z",
     "iopub.status.busy": "2026-04-28T09:53:43.321241Z",
     "iopub.status.idle": "2026-04-28T09:54:00.530209Z",
     "shell.execute_reply": "2026-04-28T09:54:00.529463Z"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "055c247f355e4ea49b2752cdee59f81f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Processing spectra:   0%|          | 0/743 [00:00<?, ?spectrum/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X: (741, 6000) | prevalence(R): 49.80% | batches: ['Catania', 'Milan', 'Rome']\n"
     ]
    }
   ],
   "source": [
    "import sys, pathlib\n",
    "sys.path.insert(0, str(pathlib.Path.cwd().parent))\n",
    "from notebooks._demo import binary_labels, load_maldi_kleb_ai\n",
    "\n",
    "demo = load_maldi_kleb_ai(antibiotic='Amikacin', verbose=True)\n",
    "X, y = binary_labels(demo)\n",
    "batch = demo.batch.loc[X.index]\n",
    "print(f'X: {X.shape} | prevalence(R): {y.mean():.2%} | batches: {sorted(batch.unique())}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "augment-md",
   "metadata": {},
   "source": [
    "## Cross-validation with on-the-fly augmentation\n",
    "\n",
    "`SpectrumAugment` is a composable per-batch transform applied **only** during training. All m/z-axis parameters are in *bins*, so they stay meaningful across bin widths."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cv",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-04-28T09:54:00.531548Z",
     "iopub.status.busy": "2026-04-28T09:54:00.531393Z",
     "iopub.status.idle": "2026-04-28T09:54:08.171134Z",
     "shell.execute_reply": "2026-04-28T09:54:08.170363Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fold 0: AUROC = 0.623\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fold 1: AUROC = 0.729\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fold 2: AUROC = 0.603\n",
      "mean AUROC = 0.652 +/- 0.055\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from maldideepkit import MaldiCNNClassifier\n",
    "from maldideepkit.augment import SpectrumAugment\n",
    "\n",
    "augment = SpectrumAugment(\n",
    "    noise_std=0.01,\n",
    "    intensity_jitter=0.05,\n",
    "    mz_shift_max_bins=2,\n",
    "    blur_sigma=0.5,\n",
    ")\n",
    "\n",
    "scores = []\n",
    "cv = StratifiedKFold(n_splits=3, shuffle=True, random_state=0)\n",
    "for fold, (train_idx, test_idx) in enumerate(cv.split(X, y)):\n",
    "    clf = MaldiCNNClassifier.from_spectrum(\n",
    "        bin_width=3, input_dim=X.shape[1],\n",
    "        epochs=20, augment=augment, random_state=fold,\n",
    "    )\n",
    "    clf.fit(X.iloc[train_idx], y.iloc[train_idx])\n",
    "    proba = clf.predict_proba(X.iloc[test_idx])\n",
    "    auroc = roc_auc_score(y.iloc[test_idx], proba[:, 1])\n",
    "    scores.append(auroc)\n",
    "    print(f'fold {fold}: AUROC = {auroc:.3f}')\n",
    "\n",
    "print(f'mean AUROC = {np.mean(scores):.3f} +/- {np.std(scores):.3f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecosystem",
   "metadata": {},
   "source": [
    "## Position in the MaldiSuite ecosystem\n",
    "\n",
    "- **MaldiAMRKit** - I/O, binning, `MaldiSet` (used internally by `_demo.load_maldi_kleb_ai`).\n",
    "- **MaldiBatchKit** - batch-effect correctors that drop straight into a sklearn pipeline (`Harmony`, `ComBat`, `SpeciesAwareComBat`, `QualityWeightedComBat`).\n",
    "- **MaldiDeepKit** (this package) - sklearn-compatible deep classifiers and helpers (`SpectralEnsemble`, `SpectrumAugment`)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "maldideepkit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.17"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "055c247f355e4ea49b2752cdee59f81f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_2e38af2fd6154cc791286b5ea8892c42",
        "IPY_MODEL_3d5b24083ab640e99f1cab7d762bd28e",
        "IPY_MODEL_2903450db39c44df9d0a132c5ac0e43f"
       ],
       "layout": "IPY_MODEL_58b0b0019fbd47498aaa10164b2cfb79",
       "tabbable": null,
       "tooltip": null
      }
     },
     "064e016014a749ceaad5b2afc679df7a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "19d0e759bece407b99e389dad1f46058": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "2903450db39c44df9d0a132c5ac0e43f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_f6dc8e9900334932a6171ad78f3dec44",
       "placeholder": "",
       "style": "IPY_MODEL_c04a9b5cabf649bd9a17d73998254dd2",
       "tabbable": null,
       "tooltip": null,
       "value": " 743/743 [00:00&lt;00:00, 4303.25spectrum/s]"
      }
     },
     "2e38af2fd6154cc791286b5ea8892c42": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_bcd8c623d66b40d0a2e0f8c33da0688c",
       "placeholder": "",
       "style": "IPY_MODEL_cca0ee4610d347588e21157c0b6bbe7b",
       "tabbable": null,
       "tooltip": null,
       "value": "Processing spectra: 100%"
      }
     },
     "3d5b24083ab640e99f1cab7d762bd28e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_064e016014a749ceaad5b2afc679df7a",
       "max": 743,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_19d0e759bece407b99e389dad1f46058",
       "tabbable": null,
       "tooltip": null,
       "value": 743
      }
     },
     "58b0b0019fbd47498aaa10164b2cfb79": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "bcd8c623d66b40d0a2e0f8c33da0688c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c04a9b5cabf649bd9a17d73998254dd2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "cca0ee4610d347588e21157c0b6bbe7b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "f6dc8e9900334932a6171ad78f3dec44": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}