GitHub_collection_pykan/tutorials/API_8_regularization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "134e7f9d",
   "metadata": {},
   "source": [
    "# API 8: Regularization\n",
    "\n",
    "Regularization helps interpretability by making KANs sparser. This may require some hyperparamter tuning. Let's see how hyperparameters can affect training"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6459e11a",
   "metadata": {},
   "source": [
    "Load KAN and create_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c3faa4ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1000, 2]), torch.Size([1000, 1]))"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from kan import *\n",
    "import torch\n",
    "\n",
    "f = lambda x: torch.exp(torch.sin(torch.pi*x[:,[0]]) + x[:,[1]]**2)\n",
    "dataset = create_dataset(f, n_var=2)\n",
    "dataset['train_input'].shape, dataset['train_label'].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d51cb2fa",
   "metadata": {},
   "source": [
    "We apply L1 regularization to which tensor? Currently, we support five choices for reg_metric:\n",
    "* 'edge_forward_spline_n': the \"norm\" of edge, normalized (output std/input std), only consider the spline (ignorning symbolic)\n",
    "* 'edge_forward_sum': the \"norm\" of edge, normamlized (output std/input std), including both spline + symbolic\n",
    "* 'edge_forward_spline_u': the \"norm\" of edge, unnormalized (output std), only consider the spline (ignorning symbolic)\n",
    "* 'edge_backward': edge attribution score\n",
    "* 'node_backward': node attribution score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "97111d75",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checkpoint directory created: ./model\n",
      "saving model version 0.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "| train_loss: 3.20e-02 | test_loss: 3.11e-02 | reg: 6.97e+00 | : 100%|█| 20/20 [00:11<00:00,  1.79it\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "saving model version 0.1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAFICAYAAACcDrP3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAw30lEQVR4nO3deVxU9d4H8M8ZNtkURZRcE0TL7bqTgFsppFzTUNNMb4uglWV5y/SapWGpWT2K1s2nwZRs0UozEZfSqyiLoriVcd1wQ0UUGWIZZmDm9/xxL/NIuTBwZs4sn/frxetVzuJnvnD88DvnzBlJCCFAREQkI5XSAYiIyPGwXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikp2r0gGI7IEQAoWFhSgtLYWPjw/8/f0hSZLSsYhsFlcuRHeh0WiQkJCAkJAQBAQEoF27dggICEBISAgSEhKg0WiUjkhkkyR+EiXR7e3YsQOjR49GeXk5gP+sXqpVr1q8vLywYcMGREVFKZKRyFaxXIhuY8eOHYiOjoYQAkaj8Y73U6lUkCQJKSkpLBiiW7BciP5Ao9GgVatW0Gq1dy2WaiqVCp6ensjLy4Ofn5/lAxLZAR5zIfqDpKQklJeX16pYAMBoNKK8vBxffPGFhZMR2Q+uXIhuIYRASEgIcnNzYc6mIUkSgoKCcPr0aZ5FRgSWC1ENN27cQEBAQL0e7+/vL2MiIvvE3WJEtygtLa3X40tKSmRKQmTfWC5Et/Dx8anX4319fWVKQmTfWC5Et/D390dwcLDZx00kSUJwcDCaNGlioWRE9oXlQnQLSZLw8ssv1+mx06dP58F8ov/iAX2iP+D7XIjqjysXoj/w8/PDhg0bIEkSVKq7byLV79DfuHEji4XoFiwXotuIiopCSkoKPD09IUnSn3Z3Vf+Zp6cntm7disjISIWSEtkmlgvRHURFRSEvLw/Lli1DUFBQjduCgoKwbNkyXL58mcVCdBs85kJUC0II7N69G4888gh27dqFwYMH8+A90V1w5UJUC5IkmY6p+Pn5sViI7oHlQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCdA+VlZW4fPkycnJyAABnz57FzZs3YTQaFU5GZLv4McdEd6DRaLBhwwZ89dVXOHHiBEpKSqDX69GgQQMEBASgf//+mDx5MsLDw+Hq6qp0XCKbwnIhuo3MzEzMmDEDx48fR58+fRAdHY1u3brBx8cHGo0G2dnZSE5OxpkzZzBu3Di8++67CAgIUDo2kc1guRD9wU8//YRnnnkGPj4+WLRoEYYPHw69Xo9169ZBp9OhYcOGGD9+PCorK7Fu3TrMnz8fnTt3xtq1a9G8eXOl4xPZBJYL0S1OnTqFRx99FN7e3li3bh06deoESZKQm5uLnj17ori4GO3atUN2djYaN24MIQTS0tIwYcIEDBo0CImJifDw8FD6ZRApjgf0if7LYDBg4cKFKCoqwscff2wqlruRJAkRERFYsmQJfvzxR2zfvt1KaYlsG8uF6L/OnDmD5ORkxMTEICIi4p7FUk2SJIwaNQoPPfQQ1Go1qqqqLJyUyPbxFBei/8rIyEBpaSlGjx6N8+fPo6yszHRbXl4eDAYDAECv1+PEiRNo2LCh6fYWLVogJiYG8+fPR35+Plq1amX1/ES2hOVC9F///ve/4eXlhaCgIEydOhXp6emm24QQ0Ol0AIArV65g6NChptskScJHH32Erl27ory8HFeuXGG5kNNjuRD9l1arhaurKzw8PKDT6VBRUXHb+wkh/nRbVVUVPD09a5QQkTNjuRD9V7NmzaDVaqHRaBAaGgpvb2/TbVqtFhkZGaYSCQsLM71xUpIktGnTBgUFBVCpVGjcuLFSL4HIZrBciP6rV69e0Ov1yMrKwvvvv1/jttzcXPTp0wfFxcVo3rw51q9fDz8/P9PtkiRhzpw5CAwM5C4xIvBsMSIAwNGjR/HJJ5+gsrISq1evRllZGVxcXGp8VZMkCSqVyvTnKpUKV69exffff4/o6Gg0atRIwVdCZBtYLuTUjhw5gscffxw9e/bEyZMnMXHiRBw9ehTLly+v9SnFOp0OCxYsgFarxdSpU2t9CjORI+NuMXJK2dnZiI+PR3JyMtq3b481a9ZgwoQJ0Ol00Ov1eP/99+Hl5YUXXngBDRo0AAC4urrC1dXVtIoRQqCkpATvvfce1q1bh6VLl6Jjx45Kviwim8HLv5BTOXToEOLj47FlyxZ06NABb775Jp588skaVzW+fv06pk2bhi1btiAqKgozZszAgw8+iJMnT8JoNMLd3R3t27dHVlYWPvzwQxw9ehTx8fF44YUXauw+I3JmLBdyCllZWYiPj8fWrVvRsWNHzJ07F+PHj79jGZSVlUGtVmP58uW4du0agoKCEBISAl9fXxQVFeHkyZO4cuUKevXqhXnz5mHgwIFQqbiXmagay4Uc2oEDBxAfH49t27bhgQcewFtvvYUnnnii1iuM/Px87Nq1C6mpqcjNzUVFRQUaN26MLl26IDIyEqGhofDy8rLwqyCyPywXckiZmZmIj4/Hjh070KlTJ8ydOxdjx46t124rg8EAIQRUKhVXKUT3wC2EHEp6ejqioqIQHh6OvLw8rFu3DsePH7/rLrDacnFxgaurK4uFqBa4lZBDSEtLQ2RkJPr374+rV6/i22+/xbFjx/DEE0+wDIgUwK2O7NrevXsxZMgQDBgwAAUFBfjuu+9w9OhRjBkzhqVCpCBufWSX9uzZg4cffhiDBg3CzZs3sWHDBhw+fBijR49mqRDZAG6FZDeEENi9ezcGDx6Mhx9+GMXFxfjhhx+QnZ2Nxx9/nKVCZEO4NZLNE0Jg165dGDRoEB555BGUlJRg06ZNOHToEEaOHMnLrRDZIJYL2SwhBHbu3IkBAwZg6NCh0Gq12Lx5Mw4ePIjHHnuMpUJkw1guZHOEEPjpp5/Qv39/REZGQq/XY8uWLThw4AD++te/slSI7ADLhWyGEALbt29HeHg4Hn30URgMBqSkpGD//v0YPnw4S4XIjrBcSHFCCGzbtg1hYWGmEtm2bRsyMjIwbNgwlgqRHWK5kGKEEEhJScFDDz2E6OhouLi4YMeOHUhLS0NUVBRLhciOsVzI6oQQSE5ORmhoKEaMGAEPDw/89NNP2LdvH4YOHcpSIXIALBeyGiEENm/ejD59+mDkyJHw8vLCzp07kZqaiiFDhrBUiBwIy4UsTgiBTZs2oXfv3hg1ahR8fX3xr3/9y/Que5YKkeNhuZDFGI1GbNy4ET179kRMTAz8/Pywe/du7N69G4MGDVI6HhFZEMuFZGc0GrFhwwb07NkTY8aMQdOmTZGamopdu3Zh4MCBSscjIitguZBsjEYjvvvuO3Tv3h1jx45Fs2bNsHfvXvz888/o37+/0vGIyIpYLlRvBoMB69evx1/+8heMGzcOLVq0QFpaGn766SdEREQoHY+IFMByoTozGAxYt24dunXrhieffBKtW7dGeno6tm/fjrCwMKXjEZGCWC5kNoPBgK+//hpdu3bFhAkTcP/99yMzMxNbt25Fv379lI5HRDaA5UK1VlVVhS+//BJdunTBxIkTERwcjP379yMlJQWhoaFKxyMiG8JyoXuqqqrC2rVr0blzZ/ztb39DSEgIDhw4gOTkZPTt21fpeERkg1gudEdVVVVISkpCp06d8PTTT+PBBx/EwYMHTe+yJyK6E5YL/UllZSVWr16NBx98EM8++yy6dOmC7OxsbNq0Cb169VI6HhHZAVelA5DtqKysxNq1a7Fw4ULk5ubi8ccfN71vhYjIHCwXQmVlJZKSkrBo0SKcO3cOo0ePxsaNG9GtWzeloxGRnWK5ODG9Xo81a9Zg0aJFuHDhAsaOHYtNmzaha9euSkcjIjvHcnFCer0eq1evxqJFi3Dp0iWMHTsWycnJ6NKli9LRiMhB8IC+E1qyZAlefPFFhIWF4ZdffsG6detYLEQkK0kIIZQOQfW3YMECtG3bFr6+vhb7OwoLCxEbG2ux5ycix8FycRDjx4/HsWPHcPToUahUllmQPvXUU/j2228t8txE5Fh4zMVBSJKE4uJiJCUlYcqUKUrHISInx2MuDmTdunWYMWMGuBglIqWxXBxI//79odPpcP36daWjEJGTY7k4EEmSMH78eIwfP17pKETk5HjMxcGsXLkSjRo1gtFotNiBfSKie+G/Pg7Gx8cHgYGB+PTTT5WOQkROjOXigDZu3IjXX3+dB/aJSDEsFwdU/QFeGRkZCichImfFcnFAkiThk08+QUxMDFcvRKQIlouDevbZZ1FcXIz9+/crHYWInBDLxUFJkoRPP/0UI0aM4OqFiKyO5eLAnnnmGbi5uWHp0qVKRyEiJ8NycWCSJGHnzp2YNWsWLl68qHQcInIiLBcH16lTJ8yYMQMDBw5ERUWF0nGIyEmwXBycJElYvHgx2rZti5EjR6KyslLpSETkBFguTkClUmHr1q3Iz8/H+PHjuYIhIotjuTgJLy8vpKam4vLlyxg2bBiuXr2qdCQicmAsFyfi5+eHXbt2ITAwEGFhYdiyZQuMRqPSsYjIAbFcnIy3tze+/PJLzJo1C1OmTMHTTz+NnJwcvheGiGTFcnFCLi4umDp1Kvbu3QtJkjB06FBMmzYNBw8ehFarZdEQUb3x81yclCRJaN++PdasWYNDhw7hn//8J8aNG4fAwECEhoaia9eu6NixI0JCQhAQEABJkpSOTER2hOXi5FQqFfr27YvevXvj0qVLSEtLQ2ZmJr755htcvnwZRqMRAwYMwKuvvqp0VCKyIywXB1Kf3VmSJKFNmzaYMGECJkyYgMrKShQXF+O3337DmjVrMGrUKHTv3l2+sETk0FguDqJp06bYtGmTRZ57xIgR6Ny5M3Q6nUWen4gcjyR49NYhVFVVWfzvkCQJLi4uFv97iMj+sVyc0K3fch6oJyJL4KnITujIkSNwcXHBkSNHlI5CRA6K5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsnNVOkB9CCFQWFiI0tJS+Pj4wN/fH5IkKR3LpgkhUFRUBAAoKiqCEIIzqwX+rJmPM6sbR5mbXa5cNBoNEhISEBISgoCAALRr1w4BAQEICQlBQkICNBqN0hFtzq0zGzJkCIQQGDJkCGd2D/xZMx9nVjcONzdhZ7Zv3y68vb2FJElCkiQBwPRV/Wfe3t5i+/btSke1GZxZ3XBu5uPM6sYR52ZX5bJ9+3bh4uIiVCpVjeH/8UulUgkXFxe7+kZYCmdWN5yb+TizunHUuUlCCCH3asgSNBoNWrVqBa1WC6PReM/7q1QqeHp6Ii8vD35+fpYPaIM4s7rh3MzHmdWNI8/Nbo65JCUloby8vFbfAAAwGo0oLy/HF198YeFktoszqxvOzXycWd048tzsYuUihEBISAhyc3NhTlxJkhAUFITTp0/b5dkW9cGZ1Q3nZj7OrG4cfW52US43btxAQEBAvR7v7+8vYyLbx5nVDedmPs6sbhx9bnaxW6y0tLRejy8pKZEpif3gzOqGczMfZ1Y3jj43uygXHx+fej3e19dXpiT2gzOrG87NfJxZ3Tj63OyiXPz9/REcHGz2/kVJkhAcHIwmTZpYKJnt4szqhnMzH2dWN44+N7soF0mS8PLLL9fpsdOnT7fpg16WwpnVDedmPs6sbhx9bnZxQB9w7PPBLYUzqxvOzXycWd048tzsYuUCAH5+ftiwYQMkSYJKdffYKpUKkiRh48aNNv8NsCTOrG44N/NxZnXj0HOz9iUB6qu21+DZsWOH0lFtBmdWN5yb+TizunHEudlduQghRFFRkUhISBDBwcE1vgnBwcEiISFBaDQapSPaHM6sbjg383FmdeNoc7PLcqlmNBrFrl27BACxa9cuYTQalY5k8zizuuHczMeZ1Y2jzM1ujrncjiRJpn2Pfn5+Nn/2hC3gzOqGczMfZ1Y3jjI3uy4XIiKyTSwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZ2W25lJaW4tSpU/jll18AAPn5+dDr9Qqnsn2lpaW4cOECACAnJweXLl3i3O6hsrISly9fRk5ODgDg7NmzuHnzJoxGo8LJbBt/1sznSP+uSUIIoXQIc+Tm5iIxMRGbN2/GpUuXUFlZCZ1Oh4YNG6JHjx54+umnERMTA19fX6Wj2pRb53bhwgVotVq4u7vD29sbXbt25dxuQ6PRYMOGDfjqq69w4sQJlJSUQK/Xo0GDBggICED//v0xefJkhIeHw9XVVem4NoM/a+ZzxH/X7KZcDAYDvvnmG8yZMwdarRbDhg3D0KFD0aZNGxiNRpw5cwbbtm3D7t270bNnT6xYsQKdOnVSOrbiOLe6yczMxIwZM3D8+HH06dMH0dHR6NatG3x8fKDRaJCdnY3k5GScOXMG48aNw7vvvouAgAClYyuKP2vmc+iZCTtgMBjEJ598Iry9vcWwYcPEsWPHRFVVlcjIyBAJCQkiISFB5OTkCL1eL1JTU0Xv3r1Fx44dxS+//KJ0dEVxbnWzY8cOcd9994mQkBDx/fffi/LycqHRaMTKlStFQkKCWL16tdBqteL3338Xn332mWjRooUYOnSoyM/PVzq6YvizZj5Hn5ldlMvu3buFn5+fGDNmjLh586YwGo1CCCHmzp0rAAgAYu3atUIIIYxGo7hw4YIICwsTERERoqioSMHkyuLczHfy5EnRrl070aVLF/Hrr7+aZnb27FnRqFEjAUC0a9dO3Lx5Uwjxn7nt3btXtGrVSkycOFFUVFQoGV8x/Fkzn6PPzOYP6Gu1WsTHx6N58+ZYunQp/Pz8IEnSHe8vSRJat26NFStW4NSpU/jyyy+tmNZ2cG7mMxgMWLhwIYqKivDxxx+jU6dOd50Z8J+5RUREYMmSJfjxxx+xfft2K6W1HfxZM58zzMzmyyU7Oxv79+/Hiy++iJYtW95zYwf+843o3r07nnjiCaxZswbl5eVWSGpbODfznTlzBsnJyYiJiUFEREStZgb8Z26jRo3CQw89BLVajaqqKgsntS38WTOfM8zM5k9x2bNnDzw8PDBkyBDk5OTU2HCvXbtm+u+LFy/i+PHjpv/38/PDqFGj8OWXX+L8+fP2cxBMJpyb+TIyMlBaWorRo0fj/PnzKCsrM92Wl5cHg8EAANDr9Thx4gQaNmxour1FixaIiYnB/PnzkZ+fj1atWlk9v1L4s2Y+p5iZ0vvl7mXixImiQ4cO4tSpU6JNmzaiQYMGpi9XV1fTvkk3N7catz377LPi3LlzomnTpmLbtm1Kvwyr49zM98Ybbwg/Pz+Rk5MjHnnkkRpz8fDwMM1MkqQat3l6eop//vOfYt++fcLX11ccOHBA6ZdiVfxZM58zzMymVy5CCFRUVMDDwwMuLi6oqKhARUXFbe9bWVmJyspK0//r9Xq4u7ubHudMOLe60Wq1cHV1hYeHB3Q63R1ff/V8b1VVVQVPT08IIaDT6awR1ybwZ818zjIzmy4XSZLQtGlTZGVlwWAwYPDgwdBoNKbbT58+jdzcXABA165d0aJFC9Nt3bp1g0ajgU6nQ5MmTawdXVFyzE2r1db6mIOjaNasGbRaLTQaDUJDQ+Ht7W26TavVIiMjw1QiYWFhpjdOSpKENm3aoKCgACqVCo0bN1bqJVgdt1HzOc3MlFw21YZarRaenp5i7969oqqqqsbXnDlzTMvHpKSkGrcZDAaxZs0aERgYKPLy8pR+GVZX37m5ubmJTp06ibFjx4qEhARx8OBBUVlZqfTLsqitW7cKd3d3sXLlyj/N7NSpU6ZTke+//35x48aNP81t1qxZokOHDnZxmqicuI2azxlmZvNniz388MPw9fVFUlIShBBwcXExfalU/x9fpVLVuK2iogJffPEFIiIiEBgYqOArUMbgwYPh5uZW57kNHz4cS5YsQceOHfHjjz8iLi4OgwYNwmuvvYYNGzYgPz9fwVdnGX379kVQUBCSkpJQVlZWYy4uLi6m+0mSVGNuKpUKV69exffff4/o6Gg0atRIwVdhfdxGzecMM7P5crn//vvx1FNP4dtvv8WOHTsganG1GqPRiDVr1uDIkSN4+eWXa/zD4AwuX76MV155Bb///jvWr19fp7n9/e9/R3R0NBYsWICff/4ZX3/9NZ555hncvHkTCxcuxLBhwzBmzBj8z//8D7Kysuz24nq38vf3x0svvYTDhw9j+fLltT6lWKfTYcGCBdBqtZg6darT7U7kNmo+p5iZcoum2rt69aro06ePaN26tdi5c6cwGAxCCCHefvtt4erqKtzc3MSXX34pjEajqKysFGvXrhVNmzYVc+bMEVVVVQqntx6j0ShWr14tGjduLFq2bCmSkpIsMrfi4mKxY8cOMW/ePDFkyBDRvXt30a9fP/HKK6+Ib7/9Vly+fNlaL1l2paWl4oknnhA+Pj7io48+EuXl5cJoNIqzZ88Kf39/4erqKtq3b296R3VxcbF44403RKNGjcTnn3+udHzFcBs1n6PPzC7KRQghTpw4IXr27CmaNGki3nzzTXHmzBlx6tQpsWfPHrFnzx5x4cIFcfz4cfH888+LRo0aiWnTpomysjKlY1vNxYsXxbBhw4RKpRLPPPOM6fIklp6b0WgU//73v8Xnn38unnvuOdGzZ0/RvXt3MWrUKPHBBx+IjIwModPpLPWyLaKgoECMHTtWeHp6ilGjRonU1FRRUFAg9u3bJ1JTU0VmZqa4fv26SElJEYMHDxaNGzcWK1assIsN3pK4jZrPkWdmN1dFBv6zu2fBggVYv349XF1d0alTJ7Ru3RoGgwHnz5/HyZMn4e/vj1mzZmHSpEnw8PBQOrLFCSHw+eef4/XXX4evry9WrlyJ4cOH17iPNedWWlqKAwcOID09Henp6SgoKICHhwf69OmD8PBwhIeHo3Xr1vV92RZXVlYGtVqN5cuX49q1awgKCkJISAh8fX1RVFSEkydP4sqVK+jVqxfmzZuHgQMH1thX7qy4jZrPUWdmV+UC/Of6Tzk5OUhJSUFWVhYKCgrg5uaGdu3aYfDgwYiMjESzZs2UjmkVFy9eRFxcHHbu3Ilnn30WH374Ifz8/G57XyXmJoTA2bNnTUVz5MgRVFVVoXXr1ggPD0dERAR69eqFBg0ayPr3yik/Px+7du1CamoqcnNzUVFRgcaNG6NLly6IjIxEaGgovLy8lI5pU7iNms8RZ2Z35XIrIQQMBgMkSbL9g1syEkJArVZj5syZ8PPzw2effYaoqCizHq/E3MrKynDw4EGkp6cjLS0N+fn5cHd3R+/evREWFobw8HC0bdvWZg+IGwwGCCGgUqm4SqklZ91G68NRZmbX5eKMzp07h7i4OOzevRtxcXFYsmRJjWtc2QshBM6fP4+0tDRkZGQgOzsblZWVaNmypalo+vbtC09PT6WjElEdsFzshNFoxMqVKzF79mz4+/tDrVZjyJAhSseSjVarxcGDB5GRkYH09HTk5eXBzc0NPXr0QEREBMLDw9GuXTubXdUQUU0sFztw9uxZxMXFITU1Fc8//zwWL15sV5+lbS4hBC5evIiMjAykpaXh0KFD0Ov1CAwMNJ0U0Ldv3xqXZyEi28JysWFGoxGffPIJ5syZg+bNm0OtVmPw4MFKx7I6nU6H7OxspKWlIT09HRcvXoSLiwt69OhhKpv27dtzVUNkQ1guNur06dOIi4vDvn37MG3aNCxcuBA+Pj5Kx7IJeXl5pjPQsrKyoNPp0KxZM4SFhSEiIgKhoaGcFZHCWC42xmAwYMWKFZg7dy5atGgBtVqNgQMHKh3LZun1ehw+fNhUNufOnYNKpUL37t0RHh6OsLAwdOzYkasaIitjudiQkydPYvLkydi/fz9efvllvPvuuzyuYKYrV66YjtVkZWVBq9XC39/fVDT9+vWzy7PriOwNy8UGGAwGLF26FPPmzUPr1q2RmJiIiIgIpWPZvcrKShw5cgTp6enIyMjAmTNnoFKp0LVrV9OxmgceeIDvWSGyAJaLwnJycjB58mRkZWVhxowZeOedd/iObwvJz89HRkYGMjIysH//fpSVlaFx48am99X069fvjlc4ICLzsFwUUlVVhY8++gjvvPMO7r//fqxatQr9+vVTOpbTqKqqwrFjx0zHak6dOgVJktClSxdT2XTu3JmrGqI6Yrko4Ndff8XkyZNx+PBhvPbaa5g3bx7fia6w69evm97AmZmZidLSUjRq1Aj9+vVDREQE+vXrZ/sfK0tkQ1guVlRZWYkPPvgA8fHxCAkJQWJiIkJDQ5WORX9gMBjwyy+/mFY1OTk5AIAHH3zQdLWALl262PV1n4gsjeViJcePH8dzzz2H48ePY+bMmXjrrbds+mrA9P8KCwuRmZmJtLQ07N+/H8XFxWjYsCEeeugh01loTZs2VTomkU1huViYXq/H4sWLsXDhQnTs2BGrVq1C7969lY5FdWQ0GnHixAnT1QJ+++03CCHQsWNH0xlo3bp1g6urq9JRiRTFcrGgo0eP4rnnnsOvv/6Kf/zjH5gzZ47dfNAP1U5RUREyMzNNpztrNBp4e3vXWNU0b95c6ZhEVsdysQC9Xo/33nsPixcvRufOnbFq1Sr06NFD6VhkYUajETk5OaZjNb/++iuMRiNCQkJMZ6B1794dbm5uSkclsjiWi8yys7MxefJk5OTk4M0338Ts2bPh7u6udCxSQHFxMfbv329a1RQWFsLLywt9+/Y1fRJnYGCg0jGJLILlIhOdTocFCxZgyZIl6NatG1atWoW//OUvSsciG2E0GnHy5EnT6c7Hjh2D0WhEUFCQ6VhNjx49+IsIOQyWiwwOHjyI5557DqdPn8Zbb72FN954g7s+6K5KSkpw4MAB0y6069evo0GDBqZVTXh4OFq2bKl0TKI6Y7nUQ0VFBd555x18+OGH6NGjB1atWoWuXbsqHYvsjBACZ86cMZ2BdvToURgMBrRt29ZUNL169eLJIGRXWC51tH//fkyePBm5ubmYP38+XnvtNZ5+SrIoKyursaq5du0aPDw80Lt3b0RERCAsLAxt2rRROibRXbFczKTVavH2229j2bJl6N27N1atWoVOnTopHYsclBACubm5pqI5fPgwqqqq0KpVK1PR9OnTh2/IJZvDcjFDeno6YmNjceHCBcTHx+PVV1/laoWsqry8HAcPHjTtQrt69Src3d3Rq1cv0ydxtm3blh+ORopjudRCeXk55s6di+XLl+Ohhx5CYmIiHnjgAaVjkZMTQuDChQtIS0tDRkYGsrOzodfr0aJFC1PR9OnThx/hQIpgudzD3r17ERsbi8uXL+Pdd9/F9OnTecFCsklarRaHDh0yfRJnXl4e3Nzc0KNHD9OJAUFBQVzVkFWwXO6grKwMc+bMwccff4yIiAio1Wp06NBB6VhEtXbx4kVT0Rw8eBB6vR6BgYGmqwWEhobyY7TJYlgut7Fnzx7ExsYiPz8fixYtwrRp0/ihUWTXdDodsrOzTScGXLhwAS4uLujevbtpVRMSEsJVDcmG5XKLkpISzJ49GytXrsSAAQOgVqvRvn17pWMRyS4vL890tYCsrCxUVFQgICDAVDShoaHw9fVVOibZMZbLLeLi4rB+/XosXrwYzz//PFcr5BT0ej2OHDmC9PR0pKWl4dy5cxg9ejTmzp2rdDSyYw5fLua8vOr71mXXAHcnkC0zZzuorKwEgDpdwojbAVVz+DdpxMfHo1u3bn/686qqKlRVVcny5rOCggJMnTq13s9DZCn/+7//a9YJKRUVFZAkyaxLzhQWFmLs2LF1iUcOyOHLJScn50/L+ytXruCpp57C77//jjfffBMxMTH1+jvGjx/PciGbdu7cOcTFxd3zfgUFBfj8889x7NgxAECvXr0QGxsLPz+/ez521qxZLBcycfiDCpIkwcXFxfSlUqlMByzffvttTJw4Eb/99luN+5j7RWTr/rgd3O7r5MmTmDRpErRaLWbOnInXXnsN169fx8SJE3H9+nVuB2QWh1+5/NHnn3+OiooKLFmyBACwfPlyDB48GNevX+f+YnJa165dw+TJkzFjxgyMGTPGdDJLnz59sGLFCkyaNAkpKSn8vBmqNYdfudxKCIEXX3wR27dvhyRJkCQJU6ZMgV6vx9atW5WOR6QIo9GIv/3tb5g4cSLGjh1b4yxJlUqF6dOnIzg4GDNmzDDrxABybk5VLseOHYMkSTU+z16SJHzzzTd48sknueGQU/r444/h7u6OF1988bard0mSkJCQgAMHDuDEiRMKJCR75FTlMmrUKCxcuPBPG9Dw4cNRVlaGGzduKJSMSBlarRarV6/GqlWr7rpb2MPDA++88w5eeOEF/hJGteI05WI0GnHp0iW88sorf7pNkiQ89dRTeOKJJxRIRqScmTNnon///mjWrNk97zt8+HAYDAbs27fPCsnI3jnNAf3PPvsM99133x3Pavn000/RsGFDCCF4YJ+cgl6vR3p6OjIyMmp1f0mS8MEHH+D1119HRkYGtxO6K6dZufz973/H+vXr73i7t7c3XF1d8euvv1oxFZFy3nvvPXTt2hWenp61fkxYWBgqKytx5swZCyYjR+AU5VJeXg6dToewsLC73m/OnDl8Exg5BSEENm/ejISEBLMeJ0kSXn31Vbz00ksWSkaOwinKZdq0aQgLC7vnMv4f//gHTp8+zQOW5PAyMzPh6elZq3fe/9GECRNQUFAAvV4vfzByGE5RLmvXrsX3339/z/u5u7vDzc3NdOmLexFC4OjRo/VMR2R9s2fPxrx58+p03ESlUqFDhw5YtmyZ/MHIYThFuaxevbpWZ8MAwJtvvolx48bV6r67du3C0KFD6xONyOqqqqpQUlKCyMjIOj/H0qVLsX79eq7y6Y6colwmTZpU69/QZs+ejTNnztRqo5kwYQKWLl1a33hEVrVu3ToEBgbW62yv++67D0ajEb///ruMyciROEW5mMPNzQ0eHh7Iysq66/0qKytRWFiICRMmWCkZkTxWrFiBxYsX1+s5JElCREQE5s+fL08ocjgsl9tYtGjRPd9QGR8fj44dO/LTKsmuGI1G6PX6237Gkbni4+ORmpoqQypyRPyX8TZeeukl5OXlwWg03vZ2IQTef/99bNq0ybrBiOppz5498PHxkeUNkH5+fhBCQKfTyZCMHA3L5TZcXFzQqlWrO74H4OTJkwCAkJAQa8Yiqrd3330XM2fOlOW5JElCu3btkJiYKMvzkWNhudxBcnIyZs+efdsD+9HR0Zg/fz4vf0F2RQiBoqIiREdHy/ac8fHx+OKLL2R7PnIcLJc76Nq1K9zd3bFly5Yaf37t2jWcP38es2bNUigZUd2UlJQAgKyfGtm5c2fo9Xqekkx/wnK5A0mS8MMPP2DcuHGmdyILIfDII49g+vTp/FhXsjsrVqxAly5dZH1OSZKgUqlw9epVWZ+X7B/L5S4eeeQR9O3bF6NGjUJ+fj4++ugj5Ofn48MPP1Q6GpHZNm3ahLlz58r+vAMHDqz3qc3keFgudyFJErZt2waj0YgBAwYgMTERaWlpXLWQXfL29kaHDh1kf96ZM2fyMkj0Jw7/eS7u7u5ISUmp13O88MILyM3NxX333YezZ8/i7NmzNW5v27ZtvZ6fyNJcXV0xbdo0pKWlWeT5n3/+ee4aoxok4eBH4rRaba3vWz0Kc88Cc3Fxgbu7u1mPIbKmioqKWt9Xp9PBzc3N7DcIu7i4wM3Nzdxo5KAcvlzMYTQaIYSASqXiacbklIQQOHfuHFQqFVq3bs1dwFRnPOZyi+LiYoSFhaFZs2Y4cuSI0nGIrE6SJFRVVeGZZ57BpEmTUFxcrHQkslMsl1s0btwYO3bsQHBwMIYOHYrDhw8rHYnI6jp06AC1Wo2rV6/i+eefZ8FQnbBc/sDPzw87duxASEgIIiMjkZ2drXQkIqsLCQnBZ599hvz8fEydOpUFQ2ZjudxGo0aNsH37dnTo0AGRkZE4dOiQ0pGIrC4kJARqtRoFBQWYMmUKNBqN0pHIjrBc7qC6YB588EFERkbi4MGDSkcisrr27dtDrVbjxo0bLBgyC8vlLho2bIitW7eic+fOiIqKuucHiBE5ouDgYKjVahQWFmLKlCkoKipSOhLZAZbLPVQXTJcuXRAVFYUDBw4oHYnI6oKCgqBWq3Hz5k1MmTIFN2/eVDoS2TiWSy34+voiJSUF3bp1Q1RUFPbv3690JCKrCwoKQmJiIoqKilgwdE8sl1qqLpju3bvj0UcfRWZmptKRiKzu/vvvR2JiIjQaDeLi4lBYWKh0JLJRLBcz+Pj4YMuWLejRowceffRRpKenKx2JyOqqC+b3339nwdAdsVzMVF0wvXr1wvDhwy12IUAiW1ZdMKWlpYiNjWXB0J+wXOrA29sbycnJ6N27N4YPH459+/YpHYnI6tq2bYvExESUl5cjNjYWN27cUDoS2RCWSx1VF0zfvn0RHR2NvXv3Kh2JyOratGmDxMREaLVaxMbG4vr160pHIhvBcqkHLy8vbN68GaGhoYiOjkZqaqrSkYisrnXr1khMTIROp0NcXBwLhgCwXOqtumD69euHv/71rywYckqtWrWCWq2GTqdDbGwsCgoKlI5ECmO5yMDT0xM//vgjwsLCEB0djd27dysdicjqqgtGr9cjNjYW165dUzoSKYjlIhNPT09s2rQJ/fv3x4gRI/Cvf/1L6UhEVteqVSskJiaiqqoKsbGxyM/PVzoSKYTlIiNPT0/88MMPGDBgAEaMGIFdu3YpHYnI6lq2bInExEQYjUbExcWxYJwUy0VmDRo0wMaNGzFo0CA89thj2Llzp9KRiKyuRYsWpoKJjY3F1atXlY5EVsZysYDqgnn44Yfx2GOP4eeff1Y6EpHV3XfffVi1ahUAsGCcEMvFQjw8PPD9999jyJAhGDlyJH766SelIxFZXWBgIBITE6FSqRAbG4srV64oHYmshOViQR4eHvjuu+8wdOhQjBo1Ctu3b1c6EpHVBQYGQq1Wmwrm8uXLSkciK2C5WFh1wURFReHxxx/Htm3blI5EZHXVKxhXV1fExsYiLy9P6UhkYSwXK3B3d8f69esxbNgwxMTEYOvWrUpHIrK65s2bIzExEe7u7oiLi2PBODiWi5W4u7tj3bp1GD58OGJiYrBlyxalIxFZXbNmzaBWq+Hh4cEVjINjuVhRdcGMGDECY8aMQXJystKRiKyuumA8PT0xefJkXLp0SelIZAEsFytzc3PD119/jcceewxjx47F5s2blY5EZHUBAQFQq9Xw8vJCbGwsLl68qHQkkhnLRQFubm746quvMHLkSIwdOxabNm1SOhKR1TVt2hSJiYmmgrlw4YLSkUhGLBeFVBdMTEwMxo0bhx9++EHpSERW5+/vj8TERPj6+iI2Nhbnz59XOhLJhOWiIFdXV6xduxajR4/G+PHjsXHjRqUjEVmdv78/PvvsMzRs2BBxcXEsGAchCSGE0iGcXVVVFZ5++ml89913+PrrrzFmzJgatwshUFhYiNLSUvj4+MDf3x+SJCmUlsgybt68iSlTpkCj0UCtVqNdu3Y1bud2YF+4crEBrq6uSEpKwrhx4zBhwgR89913AACNRoOEhASEhIQgICAA7dq1Q0BAAEJCQpCQkACNRqNscCIZNWnSBGq1Gk2aNEFcXBxyc3MBcDuwV1y52BCDwYDnnnsO33zzDWbOnImEhASUl5cD+M9vbdWqf1vz8vLChg0bEBUVpUheIksoKirC1KlTUVhYiEmTJuGll17idmCHWC42xmAw4NFHH8XOnTshSRLu9u1RqVSQJAkpKSncsMihaDQajBw5Env37uV2YKdYLjZGo9GgZcuWpt/U7kWlUsHT0xN5eXnw8/OzbDgiK+F2YP94zMXGJCUlQavV1vr+RqMR5eXl+OKLLyyYisi6uB3YP65cbIgQAiEhIcjNzb3rboA/kiQJQUFBOH36NM+eIbvH7cAxsFxsyI0bNxAQEFCvx/v7+8uYiMj6uB04Bu4WsyGlpaX1enxJSYlMSYiUw+3AMbBcbIiPj0+9Hu/r6ytTEiLlcDtwDCwXG+Lv74/g4GCz9xdLkoTg4GA0adLEQsmIrIfbgWNgudgQSZLw8ssv1+mx06dP50FMcgjcDhwDD+jbGI1Gg1atWkGr1cJoNN7z/jy/nxwRtwP7x5WLjfHz88OGDRsgSRJUqrt/e6rfmbxx40ZuUORQuB3YP5aLDYqKikJKSgo8PT0hSdKflvnVf+bp6YmtW7ciMjJSoaRElsPtwL6xXGxUVFQU8vLysGzZMgQFBdW4LSgoCMuWLcPly5e5QZFD43Zgv3jMxQ4IIXDz5k2UlJTA19cXTZo04UFLcjrcDuwLy4WIiGTH3WJERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcnu/wA2BVzbUBSLnAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 500x400 with 22 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# train the model\n",
    "model = KAN(width=[2,5,1], grid=5, k=3, seed=2)\n",
    "model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_forward_spline_n'); # default\n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_forward_sum');\n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_forward_spline_u'); \n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_backward');\n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='node_backward');\n",
    "model.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3ae48d1",
   "metadata": {},
   "source": [
    "Note: To plot the KAN diagram, there are also three options\n",
    "* forward_u: same as edge_forward_spline_u\n",
    "* forward_n: same as edge_forward_spline_u\n",
    "* backward: same as edge_backward"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c8761496",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAFICAYAAACcDrP3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAw30lEQVR4nO3deVxU9d4H8M8ZNtkURZRcE0TL7bqTgFsppFzTUNNMb4uglWV5y/SapWGpWT2K1s2nwZRs0UozEZfSqyiLoriVcd1wQ0UUGWIZZmDm9/xxL/NIuTBwZs4sn/frxetVzuJnvnD88DvnzBlJCCFAREQkI5XSAYiIyPGwXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikp2r0gGI7IEQAoWFhSgtLYWPjw/8/f0hSZLSsYhsFlcuRHeh0WiQkJCAkJAQBAQEoF27dggICEBISAgSEhKg0WiUjkhkkyR+EiXR7e3YsQOjR49GeXk5gP+sXqpVr1q8vLywYcMGREVFKZKRyFaxXIhuY8eOHYiOjoYQAkaj8Y73U6lUkCQJKSkpLBiiW7BciP5Ao9GgVatW0Gq1dy2WaiqVCp6ensjLy4Ofn5/lAxLZAR5zIfqDpKQklJeX16pYAMBoNKK8vBxffPGFhZMR2Q+uXIhuIYRASEgIcnNzYc6mIUkSgoKCcPr0aZ5FRgSWC1ENN27cQEBAQL0e7+/vL2MiIvvE3WJEtygtLa3X40tKSmRKQmTfWC5Et/Dx8anX4319fWVKQmTfWC5Et/D390dwcLDZx00kSUJwcDCaNGlioWRE9oXlQnQLSZLw8ssv1+mx06dP58F8ov/iAX2iP+D7XIjqjysXoj/w8/PDhg0bIEkSVKq7byLV79DfuHEji4XoFiwXotuIiopCSkoKPD09IUnSn3Z3Vf+Zp6cntm7disjISIWSEtkmlgvRHURFRSEvLw/Lli1DUFBQjduCgoKwbNkyXL58mcVCdBs85kJUC0II7N69G4888gh27dqFwYMH8+A90V1w5UJUC5IkmY6p+Pn5sViI7oHlQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCdA+VlZW4fPkycnJyAABnz57FzZs3YTQaFU5GZLv4McdEd6DRaLBhwwZ89dVXOHHiBEpKSqDX69GgQQMEBASgf//+mDx5MsLDw+Hq6qp0XCKbwnIhuo3MzEzMmDEDx48fR58+fRAdHY1u3brBx8cHGo0G2dnZSE5OxpkzZzBu3Di8++67CAgIUDo2kc1guRD9wU8//YRnnnkGPj4+WLRoEYYPHw69Xo9169ZBp9OhYcOGGD9+PCorK7Fu3TrMnz8fnTt3xtq1a9G8eXOl4xPZBJYL0S1OnTqFRx99FN7e3li3bh06deoESZKQm5uLnj17ori4GO3atUN2djYaN24MIQTS0tIwYcIEDBo0CImJifDw8FD6ZRApjgf0if7LYDBg4cKFKCoqwscff2wqlruRJAkRERFYsmQJfvzxR2zfvt1KaYlsG8uF6L/OnDmD5ORkxMTEICIi4p7FUk2SJIwaNQoPPfQQ1Go1qqqqLJyUyPbxFBei/8rIyEBpaSlGjx6N8+fPo6yszHRbXl4eDAYDAECv1+PEiRNo2LCh6fYWLVogJiYG8+fPR35+Plq1amX1/ES2hOVC9F///ve/4eXlhaCgIEydOhXp6emm24QQ0Ol0AIArV65g6NChptskScJHH32Erl27ory8HFeuXGG5kNNjuRD9l1arhaurKzw8PKDT6VBRUXHb+wkh/nRbVVUVPD09a5QQkTNjuRD9V7NmzaDVaqHRaBAaGgpvb2/TbVqtFhkZGaYSCQsLM71xUpIktGnTBgUFBVCpVGjcuLFSL4HIZrBciP6rV69e0Ov1yMrKwvvvv1/jttzcXPTp0wfFxcVo3rw51q9fDz8/P9PtkiRhzpw5CAwM5C4xIvBsMSIAwNGjR/HJJ5+gsrISq1evRllZGVxcXGp8VZMkCSqVyvTnKpUKV69exffff4/o6Gg0atRIwVdCZBtYLuTUjhw5gscffxw9e/bEyZMnMXHiRBw9ehTLly+v9SnFOp0OCxYsgFarxdSpU2t9CjORI+NuMXJK2dnZiI+PR3JyMtq3b481a9ZgwoQJ0Ol00Ov1eP/99+Hl5YUXXngBDRo0AAC4urrC1dXVtIoRQqCkpATvvfce1q1bh6VLl6Jjx45Kviwim8HLv5BTOXToEOLj47FlyxZ06NABb775Jp588skaVzW+fv06pk2bhi1btiAqKgozZszAgw8+iJMnT8JoNMLd3R3t27dHVlYWPvzwQxw9ehTx8fF44YUXauw+I3JmLBdyCllZWYiPj8fWrVvRsWNHzJ07F+PHj79jGZSVlUGtVmP58uW4du0agoKCEBISAl9fXxQVFeHkyZO4cuUKevXqhXnz5mHgwIFQqbiXmagay4Uc2oEDBxAfH49t27bhgQcewFtvvYUnnnii1iuM/Px87Nq1C6mpqcjNzUVFRQUaN26MLl26IDIyEqGhofDy8rLwqyCyPywXckiZmZmIj4/Hjh070KlTJ8ydOxdjx46t124rg8EAIQRUKhVXKUT3wC2EHEp6ejqioqIQHh6OvLw8rFu3DsePH7/rLrDacnFxgaurK4uFqBa4lZBDSEtLQ2RkJPr374+rV6/i22+/xbFjx/DEE0+wDIgUwK2O7NrevXsxZMgQDBgwAAUFBfjuu+9w9OhRjBkzhqVCpCBufWSX9uzZg4cffhiDBg3CzZs3sWHDBhw+fBijR49mqRDZAG6FZDeEENi9ezcGDx6Mhx9+GMXFxfjhhx+QnZ2Nxx9/nKVCZEO4NZLNE0Jg165dGDRoEB555BGUlJRg06ZNOHToEEaOHMnLrRDZIJYL2SwhBHbu3IkBAwZg6NCh0Gq12Lx5Mw4ePIjHHnuMpUJkw1guZHOEEPjpp5/Qv39/REZGQq/XY8uWLThw4AD++te/slSI7ADLhWyGEALbt29HeHg4Hn30URgMBqSkpGD//v0YPnw4S4XIjrBcSHFCCGzbtg1hYWGmEtm2bRsyMjIwbNgwlgqRHWK5kGKEEEhJScFDDz2E6OhouLi4YMeOHUhLS0NUVBRLhciOsVzI6oQQSE5ORmhoKEaMGAEPDw/89NNP2LdvH4YOHcpSIXIALBeyGiEENm/ejD59+mDkyJHw8vLCzp07kZqaiiFDhrBUiBwIy4UsTgiBTZs2oXfv3hg1ahR8fX3xr3/9y/Que5YKkeNhuZDFGI1GbNy4ET179kRMTAz8/Pywe/du7N69G4MGDVI6HhFZEMuFZGc0GrFhwwb07NkTY8aMQdOmTZGamopdu3Zh4MCBSscjIitguZBsjEYjvvvuO3Tv3h1jx45Fs2bNsHfvXvz888/o37+/0vGIyIpYLlRvBoMB69evx1/+8heMGzcOLVq0QFpaGn766SdEREQoHY+IFMByoTozGAxYt24dunXrhieffBKtW7dGeno6tm/fjrCwMKXjEZGCWC5kNoPBgK+//hpdu3bFhAkTcP/99yMzMxNbt25Fv379lI5HRDaA5UK1VlVVhS+//BJdunTBxIkTERwcjP379yMlJQWhoaFKxyMiG8JyoXuqqqrC2rVr0blzZ/ztb39DSEgIDhw4gOTkZPTt21fpeERkg1gudEdVVVVISkpCp06d8PTTT+PBBx/EwYMHTe+yJyK6E5YL/UllZSVWr16NBx98EM8++yy6dOmC7OxsbNq0Cb169VI6HhHZAVelA5DtqKysxNq1a7Fw4ULk5ubi8ccfN71vhYjIHCwXQmVlJZKSkrBo0SKcO3cOo0ePxsaNG9GtWzeloxGRnWK5ODG9Xo81a9Zg0aJFuHDhAsaOHYtNmzaha9euSkcjIjvHcnFCer0eq1evxqJFi3Dp0iWMHTsWycnJ6NKli9LRiMhB8IC+E1qyZAlefPFFhIWF4ZdffsG6detYLEQkK0kIIZQOQfW3YMECtG3bFr6+vhb7OwoLCxEbG2ux5ycix8FycRDjx4/HsWPHcPToUahUllmQPvXUU/j2228t8txE5Fh4zMVBSJKE4uJiJCUlYcqUKUrHISInx2MuDmTdunWYMWMGuBglIqWxXBxI//79odPpcP36daWjEJGTY7k4EEmSMH78eIwfP17pKETk5HjMxcGsXLkSjRo1gtFotNiBfSKie+G/Pg7Gx8cHgYGB+PTTT5WOQkROjOXigDZu3IjXX3+dB/aJSDEsFwdU/QFeGRkZCichImfFcnFAkiThk08+QUxMDFcvRKQIlouDevbZZ1FcXIz9+/crHYWInBDLxUFJkoRPP/0UI0aM4OqFiKyO5eLAnnnmGbi5uWHp0qVKRyEiJ8NycWCSJGHnzp2YNWsWLl68qHQcInIiLBcH16lTJ8yYMQMDBw5ERUWF0nGIyEmwXBycJElYvHgx2rZti5EjR6KyslLpSETkBFguTkClUmHr1q3Iz8/H+PHjuYIhIotjuTgJLy8vpKam4vLlyxg2bBiuXr2qdCQicmAsFyfi5+eHXbt2ITAwEGFhYdiyZQuMRqPSsYjIAbFcnIy3tze+/PJLzJo1C1OmTMHTTz+NnJwcvheGiGTFcnFCLi4umDp1Kvbu3QtJkjB06FBMmzYNBw8ehFarZdEQUb3x81yclCRJaN++PdasWYNDhw7hn//8J8aNG4fAwECEhoaia9eu6NixI0JCQhAQEABJkpSOTER2hOXi5FQqFfr27YvevXvj0qVLSEtLQ2ZmJr755htcvnwZRqMRAwYMwKuvvqp0VCKyIywXB1Kf3VmSJKFNmzaYMGECJkyYgMrKShQXF+O3337DmjVrMGrUKHTv3l2+sETk0FguDqJp06bYtGmTRZ57xIgR6Ny5M3Q6nUWen4gcjyR49NYhVFVVWfzvkCQJLi4uFv97iMj+sVyc0K3fch6oJyJL4KnITujIkSNwcXHBkSNHlI5CRA6K5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsnNVOkB9CCFQWFiI0tJS+Pj4wN/fH5IkKR3LpgkhUFRUBAAoKiqCEIIzqwX+rJmPM6sbR5mbXa5cNBoNEhISEBISgoCAALRr1w4BAQEICQlBQkICNBqN0hFtzq0zGzJkCIQQGDJkCGd2D/xZMx9nVjcONzdhZ7Zv3y68vb2FJElCkiQBwPRV/Wfe3t5i+/btSke1GZxZ3XBu5uPM6sYR52ZX5bJ9+3bh4uIiVCpVjeH/8UulUgkXFxe7+kZYCmdWN5yb+TizunHUuUlCCCH3asgSNBoNWrVqBa1WC6PReM/7q1QqeHp6Ii8vD35+fpYPaIM4s7rh3MzHmdWNI8/Nbo65JCUloby8vFbfAAAwGo0oLy/HF198YeFktoszqxvOzXycWd048tzsYuUihEBISAhyc3NhTlxJkhAUFITTp0/b5dkW9cGZ1Q3nZj7OrG4cfW52US43btxAQEBAvR7v7+8vYyLbx5nVDedmPs6sbhx9bnaxW6y0tLRejy8pKZEpif3gzOqGczMfZ1Y3jj43uygXHx+fej3e19dXpiT2gzOrG87NfJxZ3Tj63OyiXPz9/REcHGz2/kVJkhAcHIwmTZpYKJnt4szqhnMzH2dWN44+N7soF0mS8PLLL9fpsdOnT7fpg16WwpnVDedmPs6sbhx9bnZxQB9w7PPBLYUzqxvOzXycWd048tzsYuUCAH5+ftiwYQMkSYJKdffYKpUKkiRh48aNNv8NsCTOrG44N/NxZnXj0HOz9iUB6qu21+DZsWOH0lFtBmdWN5yb+TizunHEudlduQghRFFRkUhISBDBwcE1vgnBwcEiISFBaDQapSPaHM6sbjg383FmdeNoc7PLcqlmNBrFrl27BACxa9cuYTQalY5k8zizuuHczMeZ1Y2jzM1ujrncjiRJpn2Pfn5+Nn/2hC3gzOqGczMfZ1Y3jjI3uy4XIiKyTSwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZ2W25lJaW4tSpU/jll18AAPn5+dDr9Qqnsn2lpaW4cOECACAnJweXLl3i3O6hsrISly9fRk5ODgDg7NmzuHnzJoxGo8LJbBt/1sznSP+uSUIIoXQIc+Tm5iIxMRGbN2/GpUuXUFlZCZ1Oh4YNG6JHjx54+umnERMTA19fX6Wj2pRb53bhwgVotVq4u7vD29sbXbt25dxuQ6PRYMOGDfjqq69w4sQJlJSUQK/Xo0GDBggICED//v0xefJkhIeHw9XVVem4NoM/a+ZzxH/X7KZcDAYDvvnmG8yZMwdarRbDhg3D0KFD0aZNGxiNRpw5cwbbtm3D7t270bNnT6xYsQKdOnVSOrbiOLe6yczMxIwZM3D8+HH06dMH0dHR6NatG3x8fKDRaJCdnY3k5GScOXMG48aNw7vvvouAgAClYyuKP2vmc+iZCTtgMBjEJ598Iry9vcWwYcPEsWPHRFVVlcjIyBAJCQkiISFB5OTkCL1eL1JTU0Xv3r1Fx44dxS+//KJ0dEVxbnWzY8cOcd9994mQkBDx/fffi/LycqHRaMTKlStFQkKCWL16tdBqteL3338Xn332mWjRooUYOnSoyM/PVzq6YvizZj5Hn5ldlMvu3buFn5+fGDNmjLh586YwGo1CCCHmzp0rAAgAYu3atUIIIYxGo7hw4YIICwsTERERoqioSMHkyuLczHfy5EnRrl070aVLF/Hrr7+aZnb27FnRqFEjAUC0a9dO3Lx5Uwjxn7nt3btXtGrVSkycOFFUVFQoGV8x/Fkzn6PPzOYP6Gu1WsTHx6N58+ZYunQp/Pz8IEnSHe8vSRJat26NFStW4NSpU/jyyy+tmNZ2cG7mMxgMWLhwIYqKivDxxx+jU6dOd50Z8J+5RUREYMmSJfjxxx+xfft2K6W1HfxZM58zzMzmyyU7Oxv79+/Hiy++iJYtW95zYwf+843o3r07nnjiCaxZswbl5eVWSGpbODfznTlzBsnJyYiJiUFEREStZgb8Z26jRo3CQw89BLVajaqqKgsntS38WTOfM8zM5k9x2bNnDzw8PDBkyBDk5OTU2HCvXbtm+u+LFy/i+PHjpv/38/PDqFGj8OWXX+L8+fP2cxBMJpyb+TIyMlBaWorRo0fj/PnzKCsrM92Wl5cHg8EAANDr9Thx4gQaNmxour1FixaIiYnB/PnzkZ+fj1atWlk9v1L4s2Y+p5iZ0vvl7mXixImiQ4cO4tSpU6JNmzaiQYMGpi9XV1fTvkk3N7catz377LPi3LlzomnTpmLbtm1Kvwyr49zM98Ybbwg/Pz+Rk5MjHnnkkRpz8fDwMM1MkqQat3l6eop//vOfYt++fcLX11ccOHBA6ZdiVfxZM58zzMymVy5CCFRUVMDDwwMuLi6oqKhARUXFbe9bWVmJyspK0//r9Xq4u7ubHudMOLe60Wq1cHV1hYeHB3Q63R1ff/V8b1VVVQVPT08IIaDT6awR1ybwZ818zjIzmy4XSZLQtGlTZGVlwWAwYPDgwdBoNKbbT58+jdzcXABA165d0aJFC9Nt3bp1g0ajgU6nQ5MmTawdXVFyzE2r1db6mIOjaNasGbRaLTQaDUJDQ+Ht7W26TavVIiMjw1QiYWFhpjdOSpKENm3aoKCgACqVCo0bN1bqJVgdt1HzOc3MlFw21YZarRaenp5i7969oqqqqsbXnDlzTMvHpKSkGrcZDAaxZs0aERgYKPLy8pR+GVZX37m5ubmJTp06ibFjx4qEhARx8OBBUVlZqfTLsqitW7cKd3d3sXLlyj/N7NSpU6ZTke+//35x48aNP81t1qxZokOHDnZxmqicuI2azxlmZvNniz388MPw9fVFUlIShBBwcXExfalU/x9fpVLVuK2iogJffPEFIiIiEBgYqOArUMbgwYPh5uZW57kNHz4cS5YsQceOHfHjjz8iLi4OgwYNwmuvvYYNGzYgPz9fwVdnGX379kVQUBCSkpJQVlZWYy4uLi6m+0mSVGNuKpUKV69exffff4/o6Gg0atRIwVdhfdxGzecMM7P5crn//vvx1FNP4dtvv8WOHTsganG1GqPRiDVr1uDIkSN4+eWXa/zD4AwuX76MV155Bb///jvWr19fp7n9/e9/R3R0NBYsWICff/4ZX3/9NZ555hncvHkTCxcuxLBhwzBmzBj8z//8D7Kysuz24nq38vf3x0svvYTDhw9j+fLltT6lWKfTYcGCBdBqtZg6darT7U7kNmo+p5iZcoum2rt69aro06ePaN26tdi5c6cwGAxCCCHefvtt4erqKtzc3MSXX34pjEajqKysFGvXrhVNmzYVc+bMEVVVVQqntx6j0ShWr14tGjduLFq2bCmSkpIsMrfi4mKxY8cOMW/ePDFkyBDRvXt30a9fP/HKK6+Ib7/9Vly+fNlaL1l2paWl4oknnhA+Pj7io48+EuXl5cJoNIqzZ88Kf39/4erqKtq3b296R3VxcbF44403RKNGjcTnn3+udHzFcBs1n6PPzC7KRQghTpw4IXr27CmaNGki3nzzTXHmzBlx6tQpsWfPHrFnzx5x4cIFcfz4cfH888+LRo0aiWnTpomysjKlY1vNxYsXxbBhw4RKpRLPPPOM6fIklp6b0WgU//73v8Xnn38unnvuOdGzZ0/RvXt3MWrUKPHBBx+IjIwModPpLPWyLaKgoECMHTtWeHp6ilGjRonU1FRRUFAg9u3bJ1JTU0VmZqa4fv26SElJEYMHDxaNGzcWK1assIsN3pK4jZrPkWdmN1dFBv6zu2fBggVYv349XF1d0alTJ7Ru3RoGgwHnz5/HyZMn4e/vj1mzZmHSpEnw8PBQOrLFCSHw+eef4/XXX4evry9WrlyJ4cOH17iPNedWWlqKAwcOID09Henp6SgoKICHhwf69OmD8PBwhIeHo3Xr1vV92RZXVlYGtVqN5cuX49q1awgKCkJISAh8fX1RVFSEkydP4sqVK+jVqxfmzZuHgQMH1thX7qy4jZrPUWdmV+UC/Of6Tzk5OUhJSUFWVhYKCgrg5uaGdu3aYfDgwYiMjESzZs2UjmkVFy9eRFxcHHbu3Ilnn30WH374Ifz8/G57XyXmJoTA2bNnTUVz5MgRVFVVoXXr1ggPD0dERAR69eqFBg0ayPr3yik/Px+7du1CamoqcnNzUVFRgcaNG6NLly6IjIxEaGgovLy8lI5pU7iNms8RZ2Z35XIrIQQMBgMkSbL9g1syEkJArVZj5syZ8PPzw2effYaoqCizHq/E3MrKynDw4EGkp6cjLS0N+fn5cHd3R+/evREWFobw8HC0bdvWZg+IGwwGCCGgUqm4SqklZ91G68NRZmbX5eKMzp07h7i4OOzevRtxcXFYsmRJjWtc2QshBM6fP4+0tDRkZGQgOzsblZWVaNmypalo+vbtC09PT6WjElEdsFzshNFoxMqVKzF79mz4+/tDrVZjyJAhSseSjVarxcGDB5GRkYH09HTk5eXBzc0NPXr0QEREBMLDw9GuXTubXdUQUU0sFztw9uxZxMXFITU1Fc8//zwWL15sV5+lbS4hBC5evIiMjAykpaXh0KFD0Ov1CAwMNJ0U0Ldv3xqXZyEi28JysWFGoxGffPIJ5syZg+bNm0OtVmPw4MFKx7I6nU6H7OxspKWlIT09HRcvXoSLiwt69OhhKpv27dtzVUNkQ1guNur06dOIi4vDvn37MG3aNCxcuBA+Pj5Kx7IJeXl5pjPQsrKyoNPp0KxZM4SFhSEiIgKhoaGcFZHCWC42xmAwYMWKFZg7dy5atGgBtVqNgQMHKh3LZun1ehw+fNhUNufOnYNKpUL37t0RHh6OsLAwdOzYkasaIitjudiQkydPYvLkydi/fz9efvllvPvuuzyuYKYrV66YjtVkZWVBq9XC39/fVDT9+vWzy7PriOwNy8UGGAwGLF26FPPmzUPr1q2RmJiIiIgIpWPZvcrKShw5cgTp6enIyMjAmTNnoFKp0LVrV9OxmgceeIDvWSGyAJaLwnJycjB58mRkZWVhxowZeOedd/iObwvJz89HRkYGMjIysH//fpSVlaFx48am99X069fvjlc4ICLzsFwUUlVVhY8++gjvvPMO7r//fqxatQr9+vVTOpbTqKqqwrFjx0zHak6dOgVJktClSxdT2XTu3JmrGqI6Yrko4Ndff8XkyZNx+PBhvPbaa5g3bx7fia6w69evm97AmZmZidLSUjRq1Aj9+vVDREQE+vXrZ/sfK0tkQ1guVlRZWYkPPvgA8fHxCAkJQWJiIkJDQ5WORX9gMBjwyy+/mFY1OTk5AIAHH3zQdLWALl262PV1n4gsjeViJcePH8dzzz2H48ePY+bMmXjrrbds+mrA9P8KCwuRmZmJtLQ07N+/H8XFxWjYsCEeeugh01loTZs2VTomkU1huViYXq/H4sWLsXDhQnTs2BGrVq1C7969lY5FdWQ0GnHixAnT1QJ+++03CCHQsWNH0xlo3bp1g6urq9JRiRTFcrGgo0eP4rnnnsOvv/6Kf/zjH5gzZ47dfNAP1U5RUREyMzNNpztrNBp4e3vXWNU0b95c6ZhEVsdysQC9Xo/33nsPixcvRufOnbFq1Sr06NFD6VhkYUajETk5OaZjNb/++iuMRiNCQkJMZ6B1794dbm5uSkclsjiWi8yys7MxefJk5OTk4M0338Ts2bPh7u6udCxSQHFxMfbv329a1RQWFsLLywt9+/Y1fRJnYGCg0jGJLILlIhOdTocFCxZgyZIl6NatG1atWoW//OUvSsciG2E0GnHy5EnT6c7Hjh2D0WhEUFCQ6VhNjx49+IsIOQyWiwwOHjyI5557DqdPn8Zbb72FN954g7s+6K5KSkpw4MAB0y6069evo0GDBqZVTXh4OFq2bKl0TKI6Y7nUQ0VFBd555x18+OGH6NGjB1atWoWuXbsqHYvsjBACZ86cMZ2BdvToURgMBrRt29ZUNL169eLJIGRXWC51tH//fkyePBm5ubmYP38+XnvtNZ5+SrIoKyursaq5du0aPDw80Lt3b0RERCAsLAxt2rRROibRXbFczKTVavH2229j2bJl6N27N1atWoVOnTopHYsclBACubm5pqI5fPgwqqqq0KpVK1PR9OnTh2/IJZvDcjFDeno6YmNjceHCBcTHx+PVV1/laoWsqry8HAcPHjTtQrt69Src3d3Rq1cv0ydxtm3blh+ORopjudRCeXk55s6di+XLl+Ohhx5CYmIiHnjgAaVjkZMTQuDChQtIS0tDRkYGsrOzodfr0aJFC1PR9OnThx/hQIpgudzD3r17ERsbi8uXL+Pdd9/F9OnTecFCsklarRaHDh0yfRJnXl4e3Nzc0KNHD9OJAUFBQVzVkFWwXO6grKwMc+bMwccff4yIiAio1Wp06NBB6VhEtXbx4kVT0Rw8eBB6vR6BgYGmqwWEhobyY7TJYlgut7Fnzx7ExsYiPz8fixYtwrRp0/ihUWTXdDodsrOzTScGXLhwAS4uLujevbtpVRMSEsJVDcmG5XKLkpISzJ49GytXrsSAAQOgVqvRvn17pWMRyS4vL890tYCsrCxUVFQgICDAVDShoaHw9fVVOibZMZbLLeLi4rB+/XosXrwYzz//PFcr5BT0ej2OHDmC9PR0pKWl4dy5cxg9ejTmzp2rdDSyYw5fLua8vOr71mXXAHcnkC0zZzuorKwEgDpdwojbAVVz+DdpxMfHo1u3bn/686qqKlRVVcny5rOCggJMnTq13s9DZCn/+7//a9YJKRUVFZAkyaxLzhQWFmLs2LF1iUcOyOHLJScn50/L+ytXruCpp57C77//jjfffBMxMTH1+jvGjx/PciGbdu7cOcTFxd3zfgUFBfj8889x7NgxAECvXr0QGxsLPz+/ez521qxZLBcycfiDCpIkwcXFxfSlUqlMByzffvttTJw4Eb/99luN+5j7RWTr/rgd3O7r5MmTmDRpErRaLWbOnInXXnsN169fx8SJE3H9+nVuB2QWh1+5/NHnn3+OiooKLFmyBACwfPlyDB48GNevX+f+YnJa165dw+TJkzFjxgyMGTPGdDJLnz59sGLFCkyaNAkpKSn8vBmqNYdfudxKCIEXX3wR27dvhyRJkCQJU6ZMgV6vx9atW5WOR6QIo9GIv/3tb5g4cSLGjh1b4yxJlUqF6dOnIzg4GDNmzDDrxABybk5VLseOHYMkSTU+z16SJHzzzTd48sknueGQU/r444/h7u6OF1988bard0mSkJCQgAMHDuDEiRMKJCR75FTlMmrUKCxcuPBPG9Dw4cNRVlaGGzduKJSMSBlarRarV6/GqlWr7rpb2MPDA++88w5eeOEF/hJGteI05WI0GnHp0iW88sorf7pNkiQ89dRTeOKJJxRIRqScmTNnon///mjWrNk97zt8+HAYDAbs27fPCsnI3jnNAf3PPvsM99133x3Pavn000/RsGFDCCF4YJ+cgl6vR3p6OjIyMmp1f0mS8MEHH+D1119HRkYGtxO6K6dZufz973/H+vXr73i7t7c3XF1d8euvv1oxFZFy3nvvPXTt2hWenp61fkxYWBgqKytx5swZCyYjR+AU5VJeXg6dToewsLC73m/OnDl8Exg5BSEENm/ejISEBLMeJ0kSXn31Vbz00ksWSkaOwinKZdq0aQgLC7vnMv4f//gHTp8+zQOW5PAyMzPh6elZq3fe/9GECRNQUFAAvV4vfzByGE5RLmvXrsX3339/z/u5u7vDzc3NdOmLexFC4OjRo/VMR2R9s2fPxrx58+p03ESlUqFDhw5YtmyZ/MHIYThFuaxevbpWZ8MAwJtvvolx48bV6r67du3C0KFD6xONyOqqqqpQUlKCyMjIOj/H0qVLsX79eq7y6Y6colwmTZpU69/QZs+ejTNnztRqo5kwYQKWLl1a33hEVrVu3ToEBgbW62yv++67D0ajEb///ruMyciROEW5mMPNzQ0eHh7Iysq66/0qKytRWFiICRMmWCkZkTxWrFiBxYsX1+s5JElCREQE5s+fL08ocjgsl9tYtGjRPd9QGR8fj44dO/LTKsmuGI1G6PX6237Gkbni4+ORmpoqQypyRPyX8TZeeukl5OXlwWg03vZ2IQTef/99bNq0ybrBiOppz5498PHxkeUNkH5+fhBCQKfTyZCMHA3L5TZcXFzQqlWrO74H4OTJkwCAkJAQa8Yiqrd3330XM2fOlOW5JElCu3btkJiYKMvzkWNhudxBcnIyZs+efdsD+9HR0Zg/fz4vf0F2RQiBoqIiREdHy/ac8fHx+OKLL2R7PnIcLJc76Nq1K9zd3bFly5Yaf37t2jWcP38es2bNUigZUd2UlJQAgKyfGtm5c2fo9Xqekkx/wnK5A0mS8MMPP2DcuHGmdyILIfDII49g+vTp/FhXsjsrVqxAly5dZH1OSZKgUqlw9epVWZ+X7B/L5S4eeeQR9O3bF6NGjUJ+fj4++ugj5Ofn48MPP1Q6GpHZNm3ahLlz58r+vAMHDqz3qc3keFgudyFJErZt2waj0YgBAwYgMTERaWlpXLWQXfL29kaHDh1kf96ZM2fyMkj0Jw7/eS7u7u5ISUmp13O88MILyM3NxX333YezZ8/i7NmzNW5v27ZtvZ6fyNJcXV0xbdo0pKWlWeT5n3/+ee4aoxok4eBH4rRaba3vWz0Kc88Cc3Fxgbu7u1mPIbKmioqKWt9Xp9PBzc3N7DcIu7i4wM3Nzdxo5KAcvlzMYTQaIYSASqXiacbklIQQOHfuHFQqFVq3bs1dwFRnPOZyi+LiYoSFhaFZs2Y4cuSI0nGIrE6SJFRVVeGZZ57BpEmTUFxcrHQkslMsl1s0btwYO3bsQHBwMIYOHYrDhw8rHYnI6jp06AC1Wo2rV6/i+eefZ8FQnbBc/sDPzw87duxASEgIIiMjkZ2drXQkIqsLCQnBZ599hvz8fEydOpUFQ2ZjudxGo0aNsH37dnTo0AGRkZE4dOiQ0pGIrC4kJARqtRoFBQWYMmUKNBqN0pHIjrBc7qC6YB588EFERkbi4MGDSkcisrr27dtDrVbjxo0bLBgyC8vlLho2bIitW7eic+fOiIqKuucHiBE5ouDgYKjVahQWFmLKlCkoKipSOhLZAZbLPVQXTJcuXRAVFYUDBw4oHYnI6oKCgqBWq3Hz5k1MmTIFN2/eVDoS2TiWSy34+voiJSUF3bp1Q1RUFPbv3690JCKrCwoKQmJiIoqKilgwdE8sl1qqLpju3bvj0UcfRWZmptKRiKzu/vvvR2JiIjQaDeLi4lBYWKh0JLJRLBcz+Pj4YMuWLejRowceffRRpKenKx2JyOqqC+b3339nwdAdsVzMVF0wvXr1wvDhwy12IUAiW1ZdMKWlpYiNjWXB0J+wXOrA29sbycnJ6N27N4YPH459+/YpHYnI6tq2bYvExESUl5cjNjYWN27cUDoS2RCWSx1VF0zfvn0RHR2NvXv3Kh2JyOratGmDxMREaLVaxMbG4vr160pHIhvBcqkHLy8vbN68GaGhoYiOjkZqaqrSkYisrnXr1khMTIROp0NcXBwLhgCwXOqtumD69euHv/71rywYckqtWrWCWq2GTqdDbGwsCgoKlI5ECmO5yMDT0xM//vgjwsLCEB0djd27dysdicjqqgtGr9cjNjYW165dUzoSKYjlIhNPT09s2rQJ/fv3x4gRI/Cvf/1L6UhEVteqVSskJiaiqqoKsbGxyM/PVzoSKYTlIiNPT0/88MMPGDBgAEaMGIFdu3YpHYnI6lq2bInExEQYjUbExcWxYJwUy0VmDRo0wMaNGzFo0CA89thj2Llzp9KRiKyuRYsWpoKJjY3F1atXlY5EVsZysYDqgnn44Yfx2GOP4eeff1Y6EpHV3XfffVi1ahUAsGCcEMvFQjw8PPD9999jyJAhGDlyJH766SelIxFZXWBgIBITE6FSqRAbG4srV64oHYmshOViQR4eHvjuu+8wdOhQjBo1Ctu3b1c6EpHVBQYGQq1Wmwrm8uXLSkciK2C5WFh1wURFReHxxx/Htm3blI5EZHXVKxhXV1fExsYiLy9P6UhkYSwXK3B3d8f69esxbNgwxMTEYOvWrUpHIrK65s2bIzExEe7u7oiLi2PBODiWi5W4u7tj3bp1GD58OGJiYrBlyxalIxFZXbNmzaBWq+Hh4cEVjINjuVhRdcGMGDECY8aMQXJystKRiKyuumA8PT0xefJkXLp0SelIZAEsFytzc3PD119/jcceewxjx47F5s2blY5EZHUBAQFQq9Xw8vJCbGwsLl68qHQkkhnLRQFubm746quvMHLkSIwdOxabNm1SOhKR1TVt2hSJiYmmgrlw4YLSkUhGLBeFVBdMTEwMxo0bhx9++EHpSERW5+/vj8TERPj6+iI2Nhbnz59XOhLJhOWiIFdXV6xduxajR4/G+PHjsXHjRqUjEVmdv78/PvvsMzRs2BBxcXEsGAchCSGE0iGcXVVVFZ5++ml89913+PrrrzFmzJgatwshUFhYiNLSUvj4+MDf3x+SJCmUlsgybt68iSlTpkCj0UCtVqNdu3Y1bud2YF+4crEBrq6uSEpKwrhx4zBhwgR89913AACNRoOEhASEhIQgICAA7dq1Q0BAAEJCQpCQkACNRqNscCIZNWnSBGq1Gk2aNEFcXBxyc3MBcDuwV1y52BCDwYDnnnsO33zzDWbOnImEhASUl5cD+M9vbdWqf1vz8vLChg0bEBUVpUheIksoKirC1KlTUVhYiEmTJuGll17idmCHWC42xmAw4NFHH8XOnTshSRLu9u1RqVSQJAkpKSncsMihaDQajBw5Env37uV2YKdYLjZGo9GgZcuWpt/U7kWlUsHT0xN5eXnw8/OzbDgiK+F2YP94zMXGJCUlQavV1vr+RqMR5eXl+OKLLyyYisi6uB3YP65cbIgQAiEhIcjNzb3rboA/kiQJQUFBOH36NM+eIbvH7cAxsFxsyI0bNxAQEFCvx/v7+8uYiMj6uB04Bu4WsyGlpaX1enxJSYlMSYiUw+3AMbBcbIiPj0+9Hu/r6ytTEiLlcDtwDCwXG+Lv74/g4GCz9xdLkoTg4GA0adLEQsmIrIfbgWNgudgQSZLw8ssv1+mx06dP50FMcgjcDhwDD+jbGI1Gg1atWkGr1cJoNN7z/jy/nxwRtwP7x5WLjfHz88OGDRsgSRJUqrt/e6rfmbxx40ZuUORQuB3YP5aLDYqKikJKSgo8PT0hSdKflvnVf+bp6YmtW7ciMjJSoaRElsPtwL6xXGxUVFQU8vLysGzZMgQFBdW4LSgoCMuWLcPly5e5QZFD43Zgv3jMxQ4IIXDz5k2UlJTA19cXTZo04UFLcjrcDuwLy4WIiGTH3WJERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcnu/wA2BVzbUBSLnAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 500x400 with 22 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.plot(metric='forward_u')\n",
    "#model.plot(metric='forward_n')\n",
    "#model.plot(metric='backward') # default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1692e33b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}