GitHub_collection_pykan/tutorials/.ipynb_checkpoints/API_8_regularization-checkpoint.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "134e7f9d",
   "metadata": {},
   "source": [
    "# API 8: Regularization\n",
    "\n",
    "Regularization helps interpretability by making KANs sparser. This may require some hyperparamter tuning. Let's see how hyperparameters can affect training"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6459e11a",
   "metadata": {},
   "source": [
    "Load KAN and create_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c3faa4ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(torch.Size([1000, 2]), torch.Size([1000, 1]))"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from kan import *\n",
    "import torch\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "print(device)\n",
    "\n",
    "f = lambda x: torch.exp(torch.sin(torch.pi*x[:,[0]]) + x[:,[1]]**2)\n",
    "dataset = create_dataset(f, n_var=2, device=device)\n",
    "dataset['train_input'].shape, dataset['train_label'].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d51cb2fa",
   "metadata": {},
   "source": [
    "We apply L1 regularization to which tensor? Currently, we support five choices for reg_metric:\n",
    "* 'edge_forward_spline_n': the \"norm\" of edge, normalized (output std/input std), only consider the spline (ignorning symbolic)\n",
    "* 'edge_forward_sum': the \"norm\" of edge, normamlized (output std/input std), including both spline + symbolic\n",
    "* 'edge_forward_spline_u': the \"norm\" of edge, unnormalized (output std), only consider the spline (ignorning symbolic)\n",
    "* 'edge_backward': edge attribution score\n",
    "* 'node_backward': node attribution score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "97111d75",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checkpoint directory created: ./model\n",
      "saving model version 0.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "| train_loss: 4.57e-02 | test_loss: 4.35e-02 | reg: 7.15e+00 | : 100%|█| 20/20 [00:04<00:00,  4.58it\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "saving model version 0.1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAFICAYAAACcDrP3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAxmUlEQVR4nO3deVxU9d4H8M8ZkEVAQcLKXAIkb9rqkpmUkgsY3ef2RNu1LLtkCgoqiguipKImLiwuqVip2W3V56muChWpWGo+ZqXXcEGSBBVFGWWZGWDm9/xRzA1XZjjMOTPzeb9evF7lLHz4MocPv3POHCQhhAAREZGMNEoHICIix8NyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdq5KByCyB0IIXLhwAVVVVfD29oa/vz8kSVI6FpFqceVCdANarRaZmZkICQlBQEAAAgMDERAQgJCQEGRmZkKr1SodkUiVJP4lSqJry83NRVRUFGpqagD8vnpp0LBqad26NTZt2oTw8HBFMhKpFcuF6Bpyc3MRGRkJIQRMJtN176fRaCBJErZs2cKCIfoTlgvRFbRaLTp27AidTnfDYmmg0Wjg6emJkpIS+Pr6tnxAIjvAYy5EV1i/fj1qamqaVCwAYDKZUFNTgw0bNrRwMiL7wZUL0Z8IIRASEoKioiJYsmlIkoSgoCAcP36cZ5ERgeVC1Eh5eTkCAgKa9Xh/f38ZExHZJ+4WI/qTqqqqZj2+srJSpiRE9o3lQvQn3t7ezXq8j4+PTEmI7BvLhehP/P39ERwcbPFxE0mSEBwcjHbt2rVQMiL7wnIh+hNJkhAXF2fVY+Pj43kwn+gPPKBPdAW+z4Wo+bhyIbqCr68vNm3aBEmSoNHceBNpeIf+5s2bWSxEf8JyIbqG8PBwbNmyBZ6enpAk6ardXQ3/5unpia1bt2Lo0KEKJSVSJ5YL0XWEh4ejpKQEGRkZCAoKanRbUFAQMjIyUFpaymIhugYecyFqAiEEtm/fjkGDBiEvLw9hYWE8eE90A1y5EDWBJEnmYyq+vr4sFqKbYLkQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCQ7lgsREcmO5UJERLJjuRDdRF1dHUpLS1FQUAAAOHHiBC5evAiTyaRwMiL14p85JroOrVaLTZs24f3338fhw4dRWVmJ2tpaeHh4ICAgAI8++iiio6PRv39/uLq6Kh2XSFVYLkTXsGfPHkycOBEHDx5Enz59EBkZifvuuw/e3t7QarX44Ycf8MUXX6CwsBDPP/88UlNTERAQoHRsItVguRBd4csvv8TIkSPh7e2NBQsW4IknnkBtbS0+/PBDGAwGtGnTBi+88ALq6urw4Ycf4o033kCPHj3w3nvv4dZbb1U6PpEqsFyI/uTYsWOIiIiAl5cXPvzwQ3Tv3h2SJKGoqAg9e/bEpUuXEBgYiB9++AF+fn4QQuDbb7/F8OHDMXDgQKxduxbu7u5KfxlEiuMBfaI/GI1GzJ8/HxUVFVi+fLm5WG5EkiSEhoYiLS0Nn332GXJycmyUlkjdWC5EfygsLMQXX3yBp59+GqGhoTctlgaSJOGpp57Cww8/jOzsbNTX17dwUiL14ykuRH/YvXs3qqqqEBUVhZMnT6K6utp8W0lJCYxGIwCgtrYWhw8fRps2bcy3d+jQAU8//TTeeOMNnD17Fh07drR5fiI1YbkQ/eHIkSNo3bo1goKCMHr0aHz33Xfm24QQMBgMAIDTp09jyJAh5tskScKSJUtw7733oqamBqdPn2a5kNNjuRD9QafTwdXVFe7u7jAYDNDr9de8nxDiqtvq6+vh6enZqISInBnLhegP7du3h06ng1arRd++feHl5WW+TafTYffu3eYSeeSRR8xvnJQkCZ07d8a5c+eg0Wjg5+en1JdApBosF6I/9OrVC3V1ddi3bx8WLlzY6LaioiL06dMHly5dwq233oqPPvoIvr6+5tslSUJSUhJuu+027hIjAs8WIzJ76KGHEBQUhPXr16O6uhouLi6NPhpIkgSNRmP+d41GgzNnzuDjjz9Ghw4d4ObmpuBXQaQOLBeiP/j7+2PcuHE4cOAAsrKymnxKscFgwJw5c1BWVoZdu3aha9euyMjIgE6na+HEROrFciH6k5EjR+Jvf/sbFi5ciKysLOh0OjRcxMLV1RWurq7mVYwQApcvX0ZKSgo++ugjLF++HEeOHMGwYcOQmJiIoKAgLF26FDU1NUp+SUSK4OVfiK5w/vx5jB07Fv/6178QHh6OiRMn4u6778bRo0dhMpng5uaGrl27Yt++fVi8eDF++uknzJkzBzExMebiOXHiBBYsWIANGzagXbt2SExMxJgxYxqdJEDkyFguRNdQXV2N7OxsZGVloaysDEFBQQgJCYGPjw8qKipw9OhRnD59Gr169UJKSgoGDBgAjebqHQG//vorFixYgHXr1sHPzw+TJ09GbGwsS4YcHsuF6AbOnj2LvLw87Ny5E0VFRdDr9fDz88M999yDoUOHom/fvmjduvVNn+fkyZN488038c4778DPzw+TJk1CbGwsvL29bfBVENkey4WoiYxGI0wmk/kMMWsUFxebS6Zt27ZISEjA2LFj4ePjI3NaImWxXIgsIIRo8gUtb+S3337DwoUL8fbbb8Pb2xsJCQkYN25co+uVEdkzlguRBeQqlwanTp3CwoULsXbtWnh7e2PixImIi4tjyZDdY7kQWUDucmlQUlKCtLQ0ZGdno3Xr1pgwYQLi4+PRtm1b2T8XkS2wXIgs0FLl0uD06dNIS0vD6tWr4enpifHjx2P8+PGNLjVDZA9YLkQWaOlyaXD69GksWrQIq1evhru7OyZMmMCSIbvCciGygK3KpcGZM2ewePFirFq1Cq1atTKvZNq1a2ezDETWYLkQWcDW5dKgrKwMixcvxsqVK+Hq6or4+HhMnDiRJUOqxXIhsoBS5dKgrKwMS5YswcqVK+Hi4oK4uDhMnDgR/v7+imUiuhaWC5EFlC6XBufOncPSpUuxYsUKSJKEcePGISEhAbfccovS0YgAsFyILKKWcmlQXl6OpUuXYvny5RBCYOzYsZg0aRICAgKUjkZOjuVCZAG1lUuD8vJypKenY9myZTCZTOaSad++vdLRyEmxXIgsoNZyaXDhwgVkZGQgKysLRqMRMTExmDx5Mm699Valo5GTYbkQWUDt5dLg4sWLyMzMRGZmJurq6jBmzBgkJibitttuUzoaOQmWC5EF7KVcGlRUVJhLxmAwmEvm9ttvVzoaOTiWC5EF7K1cGmi1WnPJ6PV6vP7665gyZQo6dOigdDRyUCwXIgvYa7k0uHTpErKyspCeng6dTodRo0Zh6tSpuOOOO5SORg6G5UJkAXsvlwaXLl3CsmXLkJ6ejurqanPJdOzYUelo5CBYLkQWcJRyaXD58mUsX74cS5cuRVVVFaKjozFt2jR06tRJ6Whk51guRBZwtHJpUFlZiRUrVmDJkiW4fPmyuWQ6d+6sdDSyUywXIgs4ark0qKqqMpfMpUuX8Oqrr2L69Ono0qWL0tHIzrBciCzg6OXSoKqqCm+99RYWL16MiooKjBw5EtOnT0dgYKDS0chOsFyILOAs5dKgurraXDIXL17EK6+8gunTpyMoKEjpaKRyLBciCzhbuTSoqanBqlWrsGjRIpSXl+Pll19GUlISgoODlY5GKsVyIbKAs5ZLg5qaGqxZswZpaWk4f/48XnrpJcyYMQNdu3ZVOhqpDMuFyALOXi4NdDqduWTKysrMJRMSEqJ0NFIJlguRBVgujen1emRnZ2PhwoU4e/Yshg8fjhkzZqBbt25KRyOFaZQOQET2y8PDA3FxcSgsLERmZia2b9+OHj164KWXXsKRI0eUjkcKYrkQUbN5eHhg7NixKCwsRFZWFvLz89GjRw+8+OKLKCgoUDoeKYC7xYgswN1iTWMwGPDuu+9iwYIFuHTpEkpLS+Hl5aV0LLIhlguRBZy9XDZv3gyj0QhXV9cW/TwmkwlRUVEt+jmoZbXsK4TIwThzsQDA3r17sX//fsyePRv9+vVrsc8zbdo0loudY7kQkUX69euHV199FYWFhUpHIRXjAX0issjMmTPx66+/wmQyKR2FVIzlQkQW8fDwgLe3Nz799FOlo5CKsVyIyGJvvfUWXn/9dfB8ILoelgsRWeyFF15AZWUlLl68qHQUUimWCxFZTKPRYNiwYfj73/+udBRSKZ4tRkRW2bhxI/z9/VFbWws3Nzel45DKcOVCRFbx9fVF9+7dER8fr3QUUiGWCxFZbevWrcjOzoZer1c6CqkMy4WIrNaxY0c89NBDePnll3nmGDXCYy5EZDVJkrBlyxa0b98eRUVF/LPHZMaVCxE1S7t27ZCYmIhBgwbxXftkxnIhomabN28eACAxMZG7xwgAd4sRkQw0Gg127tyJbt264bHHHsN//dd/Of0VpJ0dVy5EJIsuXbrgww8/xAsvvIC9e/dyBePkWC5EJJu//e1vWLx4McLDw5Gfn8+CcWLcLUZEspEkCbGxsdBoNPjrX/+KZcuWYcSIEdBo+Huss+F3nIhkJUkSxowZg3/+85+YNm0aoqOjcebMGa5inAzLhYhkJ0kSIiMjsXv3bly4cAGPPPIIVq1ahcuXL7NknATLhYhahCRJCAwMxObNm5GWlobs7GyEhobirbfewvnz51kyDo7lQkQtytXVFc8++yzy8/ORmJiI9957D/3790dSUhIOHDgAnU7HonFAPKBPRDbh7e2Nl156Cc888wx27NiBdevW4ZlnnsGtt96Khx56CN27d4efnx/q6uqUjkoyYLkQkUWaewVkSZIQFhaGgQMHoqSkBHv27MGePXuwadMmVFVVwd3dHb169ZIpLSlFElyPElETbd26FfX19S3y3EajEUajEZIkwcPDA5GRkS3yecg2WC5E1GS2/HHBy8fYNx7QJ6ImkyTJog+dTocff/wROp3O4seSfWO5EFmAC33LHDlyBL1798aRI0eUjkI2xnIhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItm5Kh2gOYQQuHDhAqqqquDt7Q1/f39IkqR0LFXjzKzTMLfKykr4+Phwbk0ghEBFRQUAoKKiAkIIzqwJHGUbtcuVi1arRWZmJkJCQhAQEIDAwEAEBAQgJCQEmZmZ0Gq1SkdUHc7MOlfOLSgoiHO7iT/PbPDgwRBCYPDgwZzZTTjcNirsTE5OjvDy8hKSJAlJkgQA80fDv3l5eYmcnBylo6oGZ2Ydzs1ynJl1HHFudlUuOTk5wsXFRWg0mkbDv/JDo9EIFxcXu/pGtBTOzDqcm+U4M+s46twkIYSQezXUErRaLTp27AidTgeTyXTT+2s0Gnh6eqKkpAS+vr4tH1CFODPrcG6W48ys48hzs5tjLuvXr0dNTU2TvgEAYDKZUFNTgw0bNrRwMvXizKzDuVmOM7OOI8/NLlYuQgiEhISgqKgIlsSVJAlBQUE4fvy4XZ5t0RycmXU4N8txZtZx9LnZRbmUl5cjICCgWY/39/eXMZH6cWbW4dwsx5lZx9HnZhe7xaqqqpr1+MrKSpmS2A/OzDqcm+U4M+s4+tzsoly8vb2b9XgfHx+ZktgPzsw6nJvlODPrOPrc7KJc/P39ERwcbPH+RUmSEBwcjHbt2rVQMvXizKzDuVmOM7OOo8/NLspFkiTExcVZ9dj4+HhVH/RqKZyZdTg3y3Fm1nH0udnFAX3Asc8HbymcmXU4N8txZtZx5LnZxcoFAHx9fbFp0yZIkgSN5saxNRoNJEnC5s2bVf8NaEmcmXU4N8txZtZx6LnZ+pIAzdXUa/Dk5uYqHVU1ODPrcG6W48ys44hzs7tyEUKIiooKkZmZKYKDgxt9E4KDg0VmZqbQarVKR1Qdzsw6nJvlODPrONrc7LJcGphMJpGXlycAiLy8PGEymZSOpHqcmXU4N8txZtZxlLnZzTGXa5Ekybzv0dfXV/VnT6gBZ2Ydzs1ynJl1HGVudl0uRESkTiwXIiKSHcuFiIhkx3IhIiLZsVyIiEh2LBciIpIdy4WIiGTHciEiItmxXIiISHYsFyIikh3LhYiIZMdyISIi2bFciIhIdiwXIiKSHcuFiIhkx3IhIiLZ2W25VFVV4dixYzh06BAA4OzZs6itrVU4lfpVVVWhuLgYAFBQUIBTp05xbjdRV1eH0tJSFBQUAABOnDiBixcvwmQyKZxM3fhas5wj/VyThBBC6RCWKCoqwtq1a/H555/j1KlTqKurg8FgQJs2bfDggw/ilVdewdNPPw0fHx+lo6rKn+dWXFwMnU4HNzc3eHl54d577+XcrkGr1WLTpk14//33cfjwYVRWVqK2thYeHh4ICAjAo48+iujoaPTv3x+urq5Kx1UNvtYs54g/1+ymXIxGIz744AMkJSVBp9Nh2LBhGDJkCDp37gyTyYTCwkJs27YN27dvR8+ePbFs2TJ0795d6diK49yss2fPHkycOBEHDx5Enz59EBkZifvuuw/e3t7QarX44Ycf8MUXX6CwsBDPP/88UlNTERAQoHRsRfG1ZjmHnpmwA0ajUaxYsUJ4eXmJYcOGiZ9//lnU19eL3bt3i8zMTJGZmSkKCgpEbW2t2Llzp+jdu7fo1q2bOHTokNLRFcW5WSc3N1fcfvvtIiQkRHz66aeipqZGaLVasWrVKpGZmSneffddodPpxOXLl8WaNWtEhw4dxJAhQ8TZs2eVjq4YvtYs5+gzs4ty2b59u/D19RXPPPOMuHjxojCZTEIIIZKTkwUAAUC89957QgghTCaTKC4uFo888ogIDQ0VFRUVCiZXFudmuaNHj4rAwEBxzz33iH//+9/mmZ04cUK0bdtWABCBgYHi4sWLQojf55afny86duwoXnrpJaHX65WMrxi+1izn6DNT/QF9nU6HOXPm4NZbb0V6ejp8fX0hSdJ17y9JEjp16oRly5bh2LFj2Lhxow3TqgfnZjmj0Yj58+ejoqICy5cvR/fu3W84M+D3uYWGhiItLQ2fffYZcnJybJRWPfhas5wzzEz15fLDDz9g7969iI2NxR133HHTjR34/RvxwAMP4LnnnsO6detQU1Njg6TqwrlZrrCwEF988QWefvpphIaGNmlmwO9ze+qpp/Dwww8jOzsb9fX1LZxUXfhas5wzzEz1p7js2LED7u7uGDx4MAoKChptuGVlZeb//u2333Dw4EHz//v6+uKpp57Cxo0bcfLkSfs5CCYTzs1yu3fvRlVVFaKionDy5ElUV1ebbyspKYHRaAQA1NbW4vDhw2jTpo359g4dOuDpp5/GG2+8gbNnz6Jjx442z68UvtYs5xQzU3q/3M289NJL4q677hLHjh0TnTt3Fh4eHuYPV1dX877JVq1aNbrt1VdfFb/++qu45ZZbxLZt25T+MmyOc7PclClThK+vrygoKBCDBg1qNBd3d3fzzCRJanSbp6enWLlypdi1a5fw8fER33//vdJfik3xtWY5Z5iZqlcuQgjo9Xq4u7vDxcUFer0eer3+mvetq6tDXV2d+f9ra2vh5uZmfpwz4dyso9Pp4OrqCnd3dxgMhut+/Q3z/bP6+np4enpCCAGDwWCLuKrA15rlnGVmqi4XSZJwyy23YN++fTAajQgLC4NWqzXffvz4cRQVFQEA7r33XnTo0MF823333QetVguDwYB27drZOrqiODfrtG/fHjqdDlqtFn379oWXl5f5Np1Oh927d5tL5JFHHjG/cVKSJHTu3Bnnzp2DRqOBn5+fUl+CTdXV1WHPnj04ceIEtFotX2tN5DTbp5LLpqbIzs4Wnp6eIj8/X9TX1zf6SEpKMi8f169f3+g2o9Eo1q1bJ2677TZRUlKi9Jdhc5yb5bZu3Src3NzEqlWrrprZsWPHzKci33nnnaK8vPyquU2dOlXcdddddnGaqLVOnTol1q5dK6KiooSvr6/QaDTCx8dHeHh48LVmAWfYPlV/ttjjjz8OHx8frF+/HkIIuLi4mD80mv/E12g0jW7T6/XYsGEDQkNDcdtttyn4FSiDc2s68fv7vdCnTx8EBQVh/fr1qK6ubjQXFxcX8/0lSWo0N41GgzNnzuDTTz9FZGQk2rZtq+BXI6/a2lps374dU6dOxf33348uXbpgzJgxKCsrw6RJk/D999/jwIEDaNOmDV9rFnCG7VP15XLnnXfixRdfxMcff4zc3FyIJlytxmQyYd26dfjxxx8RFxfX6AeDs+Dcbq6hVID/7KoYN24cDhw4gKysrCafUmwwGDB37lzodDqMHj26yacwq1VxcTFWr16N//7v/0ZAQAAGDx6M9957Dz179sQHH3yAsrIy7Nq1C8nJyejduzeCgoKsfq19++236NmzZ6MfqM7AKbZPZRZMljlz5ozo06eP6NSpk/j666+F0WgUQggxa9Ys4erqKlq1aiU2btwoTCaTqKurE++995645ZZbRFJSkqivr1c4vXI4t6uZTKZGH1eqqqoSzz33nPD29hZLliwRNTU1wmQyiRMnTgh/f3/h6uoqunbtan5H9aVLl8SUKVNE27ZtxTvvvKPAV9R8er1efPXVVyIhIUH06NFDaDQa0apVKzFgwAAxf/58ceDAAfNr53qsfa3169dPdO/eXcTExIiysjJbfLmq4ejbp12UixBCHD58WPTs2VO0a9dOzJgxQxQWFopjx46JHTt2iB07doji4mJx8OBBMWbMGNG2bVsxduxYUV1drXRsxXFuv7tRoVzp3Llz4tlnnxWenp7iqaeeEjt37hTnzp0Tu3btEjt37hR79uwR58+fF1u2bBFhYWHCz89PLFu2zC42+AYnTpwQK1asEE8++aTw9vYWGo1GdOzYUbz22mvi008/teq4kbWvte3bt4uBAweKvn37is2bNzfpe+QoHHn7tJurIgNAaWkp5s6di48++giurq7o3r07OnXqBKPRiJMnT+Lo0aPw9/fH1KlTMWLECLi7uysdWRWcdW5XvrQt2V1VXV2N7OxsZGVloaysDEFBQQgJCYGPjw8qKipw9OhRnD59Gr169UJKSgoGDBig6l07Op0O+fn52LZtG3JycnD8+HG4urqif//+GDZsGMLDw3Hvvfc2e5eeta+1y5cvIy0tDf/7v/+L/v37Y/bs2ao/piAXR90+7apcgN+v/1RQUIAtW7Zg3759OHfuHFq1aoXAwECEhYVh6NChaN++vdIxVcdZ5tacQrmWs2fPIi8vDzt37kRRURH0ej38/Pxwzz33YOjQoejbty9at27drM/REoQQKCwsRE5ODrZt24adO3dCr9ejU6dOiIiIwLBhwxAWFtboKgNyac5rbdeuXUhJSUF1dTUSExMRFRVl98ewmsIRt0+7K5c/E0LAaDRCkiT1H9xSEUecm9ylci1GoxFCCGg0GlWuUqqrq7Fz507z6qSoqAhubm4IDQ3FsGHDEBERgbvvvtumP6ytea1VVVUhLS0NmzdvRr9+/TB79uxG7/VwdI6yfdp1uZBzs0WhqJkQAkeOHEFOTg5ycnKwa9cuGAwG3HnnnebVycCBA+Ht7a10VKt89913SElJweXLlzF58mQ888wzqix1ujaWC9kdZy6VqqoqfPPNN+ZCKS4uhru7OwYMGICIiAhERETgrrvucpiZVFVVYcmSJfjkk0/w0EMPYe7cubjjjjuUjkVNwHIhu+CshSKEwOHDh81l8u2336Kurg5du3Y1l8mAAQNUedxHTnv27EFKSgoqKiqQkJCA559/nqsYlWO5kKo5Y6lcvnwZeXl52LZtG3Jzc1FSUgJPT08MHDjQXChdu3ZVOqbNVVdXIz09HR9++CF69+6NuXPnolOnTkrHoutguZDqOFuhCCFw8OBB8+qk4QKZ3bp1Mx87CQ0Nhaenp9JRVWHfvn2YOXMmysvLMXHiRAwfPpyrGBViuZBqOFOpaLVafP311+bVyZkzZ+Dl5YWwsDDz+04CAwOVjqlaNTU1yMzMxPvvv4+ePXti7ty56NKli9Kx6E9YLqQoZykUk8mEn376ybw62bt3L4xGI7p3725enfTv399u3iCnFvv378fMmTNx7tw5xMfHY8SIEVzFqATLhRThDKVy4cIFfPXVV8jJycGXX36JsrIyeHt7Y9CgQebVSefOnZWOaff0ej0yMzOxceNG3HfffUhNTeWqTwVYLmQzjl4oJpMJ+/fvR25uLnJycrBv3z6YTCbcd999CA8Px7Bhw9CvXz+4ubkpHdUh/fjjj0hOTsaZM2cQFxeHV155hasYBbFcqMU5cqmcP38eX375pXl1Ul5ejrZt22Lw4MGIiIhAeHg435dhQwaDAcuWLcOGDRvQo0cPpKamIjg4WOlYTonlQi3CUQvFaDRi3759yMnJQW5uLvbv3w8hBB588EGEh4cjIiICDz/8MFq1aqV0VKf2888/Izk5GSUlJYiNjcU//vEPu76Uij1iuZCsHLFUzp49iy+//BLbtm3DV199hYqKCvj5+WHIkCEYNmwYhgwZgttvv13pmHQFg8GAlStX4t1338Xdd9+N1NRUhISEKB3LabBcqNkcrVDq6+uxd+9e85ldP/74IwCgd+/e5jcx9unTB66urgonpaY4dOgQkpOTUVxcjJiYGERHR/N7ZwMsF7KaI5VKaWmpeXXy9ddf49KlS/D398fQoUPNqxN7u+Q5/UdtbS3eeustvP322+jWrRtSU1PRrVs3pWM5NJYLWcRRCqWurg67d+82r04OHjwISZLw0EMPmVcnvXr14n56B/PLL79gxowZKCoqwujRozFq1CgeH2shLBdqkj+/TOy1UE6dOmUuk7y8PFRWVqJ9+/aNVif+/v5Kx6QWVldXhzVr1mDNmjUIDg7GvHnzcPfddysdy+GwXOi67H2VYjAY8N1335kvsXL48GFoNBr069fPfGbXgw8+yPdCOKmCggIkJyfj+PHjGDVqFEaPHs33IMmI5UJXsedVysmTJ82rk2+++QbV1dW4/fbbzWUyePBg+Pn5KR2TVKKurg5r167FqlWrEBQUhNTUVPTo0UPpWA6B5UIA7HeVotfrsWvXLvPq5MiRI3BxcUH//v3Nb2K8//777ebrIWUcO3YMycnJOHLkCKKjoxETE8NVTDOxXJycPa5STpw4gZycHGzbtg07duyATqfDHXfcYb4A5OOPP462bdsqHZPsTH19Pd555x2sXLkSXbp0QWpqKu69916lY9ktlosTsrdVSk1NDfLz87Ft2zbk5OSgsLAQrVq1Qv/+/TFs2DBERESgR48eqv86yD4cP34cycnJKCgowMiRIzF27FherdoKLBcnYi+rFCEEjh07htzcXGzbtg35+fnQ6/Xo3LmzeXUSFhYGHx8fpaOSgzIajXj33XexYsUK3HHHHZg3bx7uv/9+pWPZFZaLg7OXVUp1dTW2b99uPhj/66+/ws3NDY8++qh5dfKXv/xFtfnJMZ04cQIzZ87EoUOH8PLLLyMuLg4eHh5Kx7ILLBcHpfZVihACBQUF5jLZtWsXamtrERgYaF6dDBw4EF5eXkpHJSdnMpmwYcMGZGVl4fbbb8fcuXPRs2dPpWOpHsvFgah9lVJZWYlvvvnGXCi//fYbPDw8MGDAAPO74kNCQlSXmwj4/TT35ORk/Pzzz3jxxRcxfvx4eHp6Kh1LtVguDkCtqxQhBP7973+by+S7775DXV0dQkJCzGXy2GOPoXXr1kpHJWoSk8mEjRs3IjMzE+3bt8fcuXPRu3dvpWOpEsvFTql1lXLp0iXk5eWZ33dSWloKT09PhIWFmf+0L/94E9m74uJizJw5EwcOHMDw4cMxYcIE/pJ0BZaLHVFjoQgh8PPPP5tXJ7t374bRaMRf/vIX87GT0NBQHgQlh2MymfDBBx8gPT0d/v7+mDNnDvr27at0LNVgudgJte76On36NDp16gQvLy88/vjj5tXJnXfeqXQ0Ips4deqUeRWzbds2/lnrP7Bc6CqWviQa7m9N6ampKIkaWLMN6PV6qw7wO+o2wD/HRlf5n//5n+u+4I1GI+rr62V5x7LJZEJUVFSzn4dIbnl5eRY/Rq/Xw83NzaKrbAshMGTIEIs/lz1gudBV9u7di/nz51/178eOHUNiYiLOnTuHmJgYvPzyy836PNOmTWO5kCr99NNPmDBhQpPvu3HjRpSUlMDPzw9RUVEYNGhQk1Yk6enpLBdyLlf+jfHz589jwIABGDFiBEaMGIGRI0eiW7du6N+/v0IJiVrWldvAlYQQ+Oijj5CZmYlnn30WUVFROHHiBObPn4+ff/4ZU6ZMcdhdXk3BcqGbEkIgLCwMkZGRWLJkCQCgqqoKTz75JC5evOjUGxA5r6+++gppaWlYtWoV+vTpA0mSEBoairCwMAwfPhx+fn4YNWqU024f/BN8dFOffPIJiouLsXbtWkiSBEmSEB0dDb1ej507dyodj8jmKioqMGXKFKSnp5uLBfj94HyXLl2wbt06LFu2DKdOnVI4qXJYLnRDQgiMHDkSH3/8caPdBJIkYdGiRXjuuecUTEdke0IIREdHm68wca2VSdeuXREdHY1XX33V4jPPHAXLhW5ozZo1aN26NSIiIq66LSYmBhcuXIDBYFAgGZEyDhw4gKKiIsydO/e6u7wkSUJ8fDwqKirw3Xff2TihOrBc6LqEEBg/fjw2b958zY3IxcUFgYGBmD17tgLpiGxPCIFx48ZhxowZaNWq1Q3vq9FoMGfOHCQkJDjl6oXlQtf12WefwdXVFY8++uh177Nx40YsXbrUKTcecj4//fQTdDodnnnmmSbdPzIyErW1tTh06FALJ1MflgtdkxACL7/8Mt55550bnu3St29f1NXVcdcYOTwhBOLj4zF58uQmnwHWsHssLi7O6X4BY7nQNe3fvx96vR7PPvvsDe8nSRI6deqEtLQ0GyUjUkZFRQUqKiowfPhwix73yiuv4MKFC6iurm6hZOrEcqFrGj16NFJTU5v0G1p2djbefPPNJj+3EIIrHbI7kydPxmOPPWbR5V2A349NPvDAA5g5c2YLJVMnvomSrunzzz9Hhw4dmnTfwYMHw2AwwGg0wsXF5ab337dvH6ZNm4ZevXo1NyaRTQghsG/fPqvP/Fq6dCkef/xxCCGc5k2VXLnQNXXs2LHJv6FJkoQ2bdrgo48+atL9X3vtNXTv3r058Yhs6ttvv4W7uzvatGlj1eMDAgLg6uqKw4cPy5xMvVguJIu0tDSMHz/+pvcTQuCXX3655oUxidQqKSkJkyZNsnrVIUkSRo4cicmTJ8ucTL1YLiSLV199FRcvXrzpGTGlpaXmlQ6RPTAajaioqGj21ShiY2NRUlLiNGeNsVxIFq6urnB3d8eOHTtueL+RI0fiySefdJr9zmT/tmzZAh8fn5teJflm3Nzc4OHhgV27dsmUTN1YLiSbpKQkjBw58rq3CyGwfft2ZGdn2y4UUTMtWLAA06dPl+W54uPjkZycLMtzqR3LhWQzZcoUlJSUwGQyXfP2ffv2wdXVFbfccouNkxFZx2QyobKyEpGRkbI839///vcm7T52BCwXko2bmxvatGmDf/7zn1fdJoTAs88+i1mzZnGXGNmNPXv2wN3dvUmn2DdFq1at4Obmhr1798ryfGrGciFZZWdnIzY29qrfzMrLy1FaWoqpU6cqlIzIcm+88Qb+8Y9/yPqco0aNcoo3VLJcSFZRUVHQ6XSNzucXQiAyMhIvvPBCsw+KEtmKEAJnzpxBdHS0rM8bHR2Ns2fPOvyuMW7pJCtJkjBr1ixERESguLgYGo0GW7duxcGDB5Gfn690PKImu3DhAgDA3d1d1ud1c3ODRqNBUVGRrM+rNly5kOxmzJiB1q1bY8SIEVizZg2ee+45vP/++/Dw8FA6GlGTLVy4EPfff3+LHCN84oknkJSUJPvzqglXLnRNer2+WY/Py8vDpEmT8P7772P16tWIjIxs9nMS2dKlS5cwffr0FrnIakJCArZs2YLy8nLZn1stJOHoO/7IYlu3bkV9fX2zn8dkMsFoNF73L/a5uLjIdoonkZzy8/NRW1tr0Vlier3e4tW5RqPBgAEDLI1nF1gudBVLXhImk8l8pVdLL0UOgKclkypZsg0IIXD+/HlcvnwZXbp0uemfP76So24DPOZiR2z1e4AkSU36WL9+Pdzd3TFu3LgmP+bKDyI1aurrVwiBlJQUDB48GEeOHIGbmxu3gT+wXMgq7777Ll577TW8/vrrWLlypVWrFiJ7ZjKZkJKSgs8++wwLFizAX//6V6UjqQp/IpDF3n77bbz22msYPXo0VqxYwWIhp2MymTBr1ixzsTz55JNKR1Id/lQgi2RnZ+P1119HbGwsli9f7tDLeqJrMZlMSE5Oxueff44333yTJ6VcB8uFmmzNmjUYM2YMxo0bh6ysLBYLOR2TyYQZM2bgX//6F9LS0vDEE08oHUm1WC7UJKtXr0ZMTAzi4uKQkZHBYiGnYzKZkJSUhC1btmDRokWIiIhQOpKqsVzopt566y3ExsYiPj4e6enpLBZyOkajEdOnT8fWrVuxaNEihIeHKx1J9fgOfbqhlStXIi4uDhMmTMDixYtZLOR0jEYjpk2bhi+//BJLlizBkCFDlI5kF7hyoetavnw54uLikJCQwGIhp2Q0GjF16lQWixVYLnRNWVlZGD9+PCZNmoS0tDQWCzmd+vp6TJkyBV999RWWLl2KwYMHKx3JrrBc6CqZmZmYOHEiEhMTsXDhQhYLOZ2GYsnLy0N6ejoGDRqkdCS7w3KhRjIyMpCQkICpU6diwYIFLBZyOvX19UhMTMQ333yD9PR0PP7440pHskssFzJbunQpJk2ahGnTpmHevHksFnI6dXV1mDRpErZv346MjAyEhYUpHclusVwIALB48WIkJiYiKSkJqampLBZyOnV1dZg8eTLy8/ORmZmJgQMHKh3JrrFcCIsWLcLUqVORnJyMOXPmsFjI6TSsWBqKxVH/xoot8X0uTm7hwoVISkrCzJkzkZKSwmIhp1NbW4uEhATs3r0bWVlZePTRR5WO5BBYLk5swYIFSE5ORkpKCmbNmqV0HCKbq62txcSJE7Fnzx5kZWUhNDRU6UgOg+XipObPn4+ZM2fijTfewMyZM5WOQ2RztbW1mDBhAvbu3Ytly5ahf//+SkdyKDzm4oRSU1Mxc+ZMzJ49m8VCTslgMGDChAn4/vvvsXz5chZLC+DKxcnMmTMHs2fPxty5c5GUlKR0HCKbMxgMGD9+PP7v//4Py5cvR79+/ZSO5JBYLk5k9uzZmDNnDubNm4dp06YpHYfI5gwGA+Lj47F//36sWLECDz/8sNKRHBbLxQkIIcyrlfnz52Pq1KlKRyKyOYPBgLi4OPzwww9YuXIl+vbtq3Qkh8ZycXBCCKSkpGDevHl48803kZiYqHQkIpszGAwYN24cfvzxR6xatQp9+vRROpLDY7k4MCEEZs2ahfnz5yMtLQ2TJk1SOhKRzen1esTFxZmLpXfv3kpHcgosFwclhEBycjLefPNNLFq0CAkJCUpHIrI5vV6PsWPH4uDBgywWG2O5OCAhBJKSkpCWloYlS5ZgwoQJSkcisjmdToexY8fi0KFDWLVqFXr16qV0JKfCcnEwQghMnz4dixYtwtKlSzF+/HilIxHZnE6nQ2xsLA4fPozVq1ejZ8+eSkdyOiwXByKEwNSpU7FkyRJkZGQgLi5O6UhENldTU4PY2Fj88ssvWL16NR588EGlIzkllouDEEIgMTER6enpyMrKwtixY5WORGRzNTU1iImJwZEjR7BmzRo88MADSkdyWiwXByCEwOTJk5GRkYFly5YhNjZW6UhENlddXY2YmBgcPXoUa9aswf333690JKfGcrEDQghcuHABlZWV8PHxgb+/v/nS+EIIJCQkICsrC8uXL0dMTIzCaYlaRsN2UFVVBW9v70bbQXV1NcaMGYPjx4+zWFSCF65UMa1Wi8zMTISEhCAgIABBQUEICAhASEgIMjMzUVFRgYkTJyIrKwsrV65ksZBDunI7CAwMbLQdlJSUYPTo0Th+/Diys7NZLCohCSGE0iHoarm5uYiKikJNTQ2A339rayBJEoQQcHV1hclkwqpVqzBq1CilohK1mKZsBy4uLggJCcEnn3yCe+65R6modAWuXFQoNzcXkZGR0Ol0EELgyv5v+P/6+noIIdC5c2clYhK1qKZuB0ajEceOHUNpaakSMek6uHJRGa1Wi44dO0Kn08FkMt30/hqNBp6enigpKYGvr2/LBySyAW4H9o8rF5VZv349ampqmrRBAYDJZEJNTQ02bNjQwsmIbIfbgf3jykVFhBAICQlBUVHRVbsAbkSSJAQFBeH48ePms2eI7BW3A8fAclGR8vJyBAQENOvx/v7+MiYisj1uB46Bu8VUpKqqqlmPr6yslCkJkXK4HTgGlouKeHt7N+vxPj4+MiUhUg63A8fAclERf39/BAcHW7y/WJIkBAcHo127di2UjMh2uB04BpaLikiSZPWVjOPj43kQkxwCtwPHwAP6KsPz+4m4HTgCrlxUxtfXF5s2bYIkSdBobvzt0Wg0kCQJmzdv5gZFDoXbgf1juahQeHg4tmzZAk9PT0iSdNUyv+HfPD09sXXrVgwdOlShpEQth9uBfWO5qFR4eDhKSkqQkZGBoKCgRrcFBQUhIyMDpaWl3KDIoXE7sF885mIHhBC4ePGi+e+5tGvXjgctyelwO7AvLBciIpIdd4sREZHsWC5ERCQ7lgsREcmO5UJERLJjuRARkexYLkREJDuWCxERyY7lQkREsmO5EBGR7FguREQkO5YLERHJjuVCRESyY7kQEZHsWC5ERCS7/wf8pPEzBcQuogAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 500x400 with 22 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# train the model\n",
    "model = KAN(width=[2,5,1], grid=3, k=3, seed=1, device=device)\n",
    "model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_forward_spline_n'); # default\n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_forward_sum');\n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_forward_spline_u'); \n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='edge_backward');\n",
    "#model.fit(dataset, opt=\"LBFGS\", steps=20, lamb=0.01, reg_metric='node_backward');\n",
    "model.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3ae48d1",
   "metadata": {},
   "source": [
    "Note: To plot the KAN diagram, there are also three options\n",
    "* forward_u: same as edge_forward_spline_u\n",
    "* forward_n: same as edge_forward_spline_u\n",
    "* backward: same as edge_backward"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c8761496",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAFICAYAAACcDrP3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABEUklEQVR4nO3dd3wUdf4/8NdsNmVTICQkJPQkBBBpCU0S0EMlUeFrAbuCeNypSBFEkOPniacnpzQJCIroCSgneoIVNPjlPJAmSIKhBAgpQJINqZu2m2z7/P6Ame8mpOzszu7Mbt7PxyMPSrZ89rPzmde85zOFY4wxEEIIIRJSyd0AQggh3ofChRBCiOQoXAghhEiOwoUQQojkKFwIIYRIjsKFEEKI5ChcCCGESI7ChRBCiOQoXAghhEiOwoUQQojkKFwIIYRIjsKFEEKI5ChcCCGESI7ChRBCiOQoXAghhEhOLXcDCPEEjDFUVFSgrq4OwcHBCA8PB8dxcjeLEMWiyoWQNuh0OqSlpSE+Ph4RERGIiYlBREQE4uPjkZaWBp1OJ3cTCVEkju5ESUjL0tPTMXXqVOj1egDXqhceX7UEBgZi586dSE1NlaWNhCgVhQshLUhPT8ekSZPAGIPVam31cSqVChzHYffu3RQwhNigcCGkGZ1Oh549e8JgMLQZLDyVSgWNRoPCwkKEhoa6voGEeACacyGkma1bt0Kv19sVLABgtVqh1+uxbds2F7eMEM9BlQshNhhjiI+PR15eHsQMDY7jEBsbi5ycHDqKjBBQuBDSRHl5OSIiIpx6fnh4uIQtIsQz0W4xQmzU1dU59fza2lqJWkKIZ6NwIcRGcHCwU88PCQmRqCWEeDYKF0JshIeHIy4uTvS8CcdxiIuLQ1hYmItaRohnoXAhxAbHcZg7d65Dz503bx5N5hNyHU3oE9IMnedCiPOociGkmdDQUOzcuRMcx0GlanuI8Gfo79q1i4KFEBsULoS0IDU1Fbt374ZGowHHcTfs7uL/T6PRYM+ePUhJSZGppYQoE4ULIa1ITU1FYWEh1q5di9jY2Ca/i42Nxdq1a1FUVETBQkgLaM6FEDswxvDzzz/jjjvuwL59+zBhwgSavCekDVS5EGIHjuOEOZXQ0FAKFkLaQeFCCCFEchQuhBBCJEfhQgghRHIULoQQQiRH4UIIIURyFC6EEEIkR+FCCCFEchQuhBBCJEfhQgghRHIULoQQQiRH4UIIIURyFC6EEEIkR+FCCCFEchQuhBBCJEfhQgghRHIULoQQQiRH4UJIO0wmE4qKipCdnQ0AyM3NRWVlJaxWq8wtI0S56DbHhLRCp9Nh586d2L59O86cOYPa2loYjUYEBAQgIiIC48ePx8yZM5GcnAy1Wi13cwlRFAoXQlpw5MgRLFiwAFlZWRg1ahQmTZqEoUOHIjg4GDqdDidOnMB3332Hixcv4pFHHsHf//53REREyN1sQhSDwoWQZvbu3YsZM2YgODgY//jHP3DPPffAaDRix44daGxsRKdOnfDoo4/CZDJhx44deO2113DzzTfjk08+Qbdu3eRuPiGKQOFCiI0LFy7grrvuQlBQEHbs2IFBgwaB4zjk5eUhMTER1dXViImJwYkTJ9ClSxcwxnDw4EE8/vjj+MMf/oAPP/wQ/v7+cn8MQmRHE/qEXGexWLB8+XJUVVXh3XffFYKlLRzHYdy4cVixYgW++eYb/Pjjj25qLSHKRuFCyHUXL17Ed999hylTpmDcuHHtBguP4zjcf//9uOWWW7B582aYzWYXt5QQ5aNDXAi57vDhw6irq8PUqVNRUFCA+vp64XeFhYWwWCwAAKPRiDNnzqBTp07C77t3744pU6bgtddeQ0lJCXr27On29hOiJBQuhFx37tw5BAYGIjY2Fs8++ywOHTok/I4xhsbGRgBAcXExJk6cKPyO4zisXr0aQ4YMgV6vR3FxMYUL6fAoXAi5zmAwQK1Ww9/fH42NjWhoaGjxcYyxG35nNpuh0WiahBAhHRmFCyHXRUZGwmAwQKfTYcyYMQgKChJ+ZzAYcPjwYSFEkpKShBMnOY5D7969UVpaCpVKhS5dusj1EQhRDAoXQq4bMWIETCYTjh07hrfffrvJ7/Ly8jBq1ChUV1ejW7du+PzzzxEaGir8nuM4LF26FFFRUbRLjBDQ0WKECEaPHo3Y2Fhs3boV9fX18PHxafLD4zgOKpVK+H+VSgWtVosvvvgC3bt3h5+fn4yfghBloHAh5Lrw8HDMmTMHGRkZWLdund2HFDc2NuL1119HSUkJDhw4gLi4OKxduxYGg8HFLSZEuShcCLExY8YM3HfffXj77bexbt06GAwG8BexUKvVUKvVQhXDGENNTQ2WLVuGzz//HBs2bMD58+dx991346WXXkJMTAzWrFkDvV4v50ciRBZ0+RdCmikrK8Ps2bPx/fffIzU1FQsWLMBNN92E8+fPw2q1ws/PD/369cOxY8ewatUqnDx5Eq+//jpmzZolBE9ubi6WL1+Obdu2ISwsDIsXL8Zzzz3X5CABQrwZhQshLaivr8fmzZuxbt06XL16FbGxsYiPj0dISAiqqqpw/vx5FBcXY8SIEVi2bBluu+02qFQ37gjIz8/H8uXLsWXLFnTp0gWLFi3C888/TyFDvB6FCyFtKCkpwb59+7B//37k5eWhoaEBXbp0weDBg5GSkoIxY8YgMDCw3dcpKCjAP/7xD/zzn/9Ely5d8NJLL+H5559HcHCwGz4FIe5H4UKInSwWC6xWq3CEmCMuXbokhEznzp2xcOFCzJ49GyEhIRK3lhB5UbgQIgJjzO4LWrbl8uXLeOutt/DRRx8hODgYCxcuxJw5c5pcr4wQT0bhQogIUoUL78qVK3jrrbfw4YcfIjg4GC+++CLmzp1LIUM8HoULISJIHS68wsJCvP3229i8eTMCAwOxYMECzJs3D507d5b8vQhxBwoXQkRwVbjwiouL8fbbb2PTpk3QaDSYP38+XnjhhSaXmiHEE1C4ECKCq8OFV1xcjBUrVmDTpk3w9/fH/PnzMX/+fAoZ4jEoXAgRwV3hwtNqtVi5ciXef/99+Pr64oUXXsD8+fMRFhbmtjYQ4ggKF0JEcHe48K5evYqVK1di48aNUKvVmDdvHl588UUKGaJYFC6EiCBXuPCuXr2KVatWYePGjfDx8cHcuXPx4osvIjw8XLY2EdISChdCRJA7XHilpaVYvXo1NmzYAI7jMGfOHCxcuBBdu3aVu2mEAKBwIUQUpYQLr7y8HKtXr8a7774Lxhhmz56Nl156CREREXI3jXRwFC6EiKC0cOGVl5djzZo1WL9+PaxWqxAykZGRcjeNdFAULoSIoNRw4VVUVOCdd97BunXrYLFYMGvWLCxatAjdunWTu2mkg6FwIUQEpYcLr7KyEmvXrkVaWhpMJhOee+45LF68GFFRUXI3jXQQFC6EiOAp4cKrqqoSQqaxsVEImejoaLmbRrwchQshInhauPB0Oh3S0tKwdu1aNDQ04JlnnsHLL7+M7t27y9004qUoXAgRwVPDhVddXY1169ZhzZo1MBgM+POf/4wlS5agR48ecjeNeBkKF0JE8PRw4VVXV2P9+vVYs2YN6uvrhZDp2bOn3E0jXoLChRARvCVceDU1NXj33XexevVq1NXVYebMmfjLX/6CXr16yd004uEoXAgRwdvChVdbW4sNGzZg1apVqKmpEUKmd+/ecjeNeCgKF0JE8NZw4dXV1QkhU11djaeffhpLly5Fnz595G4a8TAULoSI4O3hwqurq8N7772HlStXoqqqCjNmzMDSpUsRExMjd9OIh6BwIUSEjhIuvPr6eiFkKisr8dRTT2Hp0qWIjY2Vu2lE4ShcCBGho4ULT6/X4/3338eKFStQXl6O6dOn4//9v/+HuLg4uZtGFIrChRAROmq48PR6PT744AO8/fbbKCsrw5NPPolXXnkF/fr1k7tpRGEoXAgRoaOHC89gMAghc/XqVSFk4uPj5W4aUQiV3A0ghHgejUaDF154AXl5eVi7di3+93//FwMHDsS0adNw/vx5uZtHFIDChRDisICAAMydOxe5ublYt24dfv75ZwwaNAhPPPEEzp07J3fziIwoXAghTgsICMDs2bORm5uL9evX48CBAxg0aBAef/xxZGdny908IgOacyFEBJpzsU9jYyM+/vhjLF++HNXV1SguLkZQUJDczSJuROFCiAgdPVx27doFs9kMX19fl76P1WrF1KlTXfoexLXUcjeAEE/SkYMFAI4ePYrjx4/j9ddfx9ixY132PkuWLKFw8XAULoQQUZKSkjBjxgzk5ubK3RSiYDShTwgR5a9//Svy8/NhtVrlbgpRMAoXQogoAQEBCA4Oxr///W+5m0IUjMKFECLae++9h2eeeQZ0PBBpDYULIUS0xx57DLW1taisrJS7KUShKFwIIaKpVCrcfffdePTRR+VuClEoOlqMEOKQ7du3IywsDEajEX5+fnI3hygMVS6EEIeEhoZi0KBBmDdvntxNIQpE4UIIcdgPP/yADz74AA0NDXI3hSgMhQshxGE9e/bE6NGjMX36dDpyjDRBcy6EEIdxHIc9e/YgIiICeXl5dNtjIqDKhRDilLCwMCxevBi33347nbVPBBQuhBCnvfnmmwCARYsW0e4xAoB2ixFCJKBSqXDgwAH0798ft956K+69994OfwXpjo4qF0KIJPr06YPPP/8cjzzyCI4ePUoVTAdH4UIIkcx9992HVatWISUlBQcOHKCA6cBotxghRDIcx2H27NlQqVSYPHky1q9fj+nTp0Olou3Yjoa+cUKIpDiOw6xZs/DZZ59hyZIlmDlzJrRaLVUxHQyFCyFEchzHYdKkSThy5AjKy8sxduxYvPfee6ipqaGQ6SAoXAghLsFxHGJiYvDVV19h5cqV2Lx5M5KTk7Fx40aUlZVRyHg5ChdCiEup1Wo89NBD+OWXX7Bo0SJ88sknSEpKwtKlS5GRkQGDwUBB44VoQp8Q4hbBwcGYNm0aHnroIfz3v//Fli1bMHXqVHTr1g1jxozBTTfdhC5dusBkMsndVCIBChdCiCjOXgGZ4zhMmDABf/jDH1BYWIjDhw/j6NGj2LlzJ+rq6uDv74+RI0dK1FoiF45RPUoIsdOePXtgNptd8toWiwUWiwUcxyEgIACTJk1yyfsQ96BwIYTYzZ2rC7p8jGejCX1CiN04jhP1YzAYkJmZCYPBIPq5xLNRuBAiAhX64pw7dw4jRozAuXPn5G4KcTMKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghkqNwIYQQIjkKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghkqNwIYQQIjkKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghkqNwIYQQIjkKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghkqNwIYQQIjkKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghklPL3QBnMMZQUVGBuro6BAcHIzw8HBzHyd0sRaM+cwzfb7W1tQgJCaF+swNjDFVVVQCAqqoqMMaoz+zgLWPUIysXnU6HtLQ0xMfHIyIiAjExMYiIiEB8fDzS0tKg0+nkbqLiUJ85pnm/xcbGUr+1w7bP7rzzTgDAnXfeSX3WDq8bo8zD/PjjjywoKIhxHMc4jmMAhB/+/4KCgtiPP/4od1MVg/rMMdRv4lGfOcYb+82jwuXHH39kPj4+TKVSNen85j8qlYr5+Ph41BfhKtRnjqF+E4/6zDHe2m8cY4xJXQ25gk6nQ8+ePWEwGGC1Wtt9vEqlgkajQWFhIUJDQ13fQAWiPnMM9Zt41GeO8eZ+85g5l61bt0Kv19v1BQCA1WqFXq/Htm3bXNwy5aI+cwz1m3jUZ47x5n7ziMqFMYb4+Hjk5eVBTHM5jkNsbCxycnI88mgLZ1CfOYb6TTzqM8d4e795RLiUl5cjIiLCqeeHh4dL2CLloz5zDPWbeNRnjvH2fvOI3WJ1dXVOPb+2tlailngO6jPHUL+JR33mGG/vN48Il+DgYKeeHxISIlFLPAf1mWOo38SjPnOMt/ebR4RLeHg44uLiRO9f5DgOcXFxCAsLc1HLlIv6zDHUb+JRnznG2/vNI8KF4zjMnTvXoefOmzdP0ZNerkJ95hjqN/Gozxzj7f3mERP6gHcfD+4q1GeOoX4Tj/rMMd7cbx5RuQBAaGgodu7cCY7joFK13WyVSgWO47Br1y7FfwGuRH3mGOo38ajPHOPV/ebuSwI4y95r8KSnp8vdVMWgPnMM9Zt41GeO8cZ+87hwYYyxqqoqlpaWxuLi4pp8CXFxcSwtLY3pdDq5m6g41GeOoX4Tj/rMMd7Wbx4ZLjyr1cr27dvHALB9+/Yxq9Uqd5MUj/rMMdRv4lGfOcZb+s1j5lxawnGcsO8xNDRU8UdPKAH1mWOo38SjPnOMt/SbR4cLIYQQZaJwIYQQIjkKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghkqNwIYQQIjkKF0IIIZKjcCGEECI5ChdCCCGSo3AhhBAiOQoXQgghkqNwIYQQIjkKF0IIIZKjcCGEECI5jw2Xuro6XLhwAadOnQIAlJSUwGg0ytwq5aurq8OlS5cAANnZ2bhy5Qr1WztMJhOKioqQnZ0NAMjNzUVlZSWsVqvMLVM2WtbE86b1GscYY3I3Qoy8vDx8+OGH+Pbbb3HlyhWYTCY0NjaiU6dOSEhIwFNPPYUpU6YgJCRE7qYqim2/Xbp0CQaDAX5+fggKCsKQIUOo31qg0+mwc+dObN++HWfOnEFtbS2MRiMCAgIQERGB8ePHY+bMmUhOToZarZa7uYpBy5p43rhe85hwsVgs+Oyzz7B06VIYDAbcfffdmDhxInr37g2r1YqLFy/ihx9+wM8//4zExESsX78egwYNkrvZsqN+c8yRI0ewYMECZGVlYdSoUZg0aRKGDh2K4OBg6HQ6nDhxAt999x0uXryIRx55BH//+98REREhd7NlRcuaeF7dZ8wDWCwWtmHDBhYUFMTuvvtu9vvvvzOz2cwOHz7M0tLSWFpaGsvOzmZGo5Ht37+fjRw5kg0YMICdOnVK7qbLivrNMenp6Sw6OprFx8ezL7/8kun1eqbT6dj777/P0tLS2Mcff8wMBgOrqalhH3zwAevevTubOHEiKykpkbvpsqFlTTxv7zOPCJeff/6ZhYaGsgcffJBVVlYyq9XKGGPslVdeYQAYAPbJJ58wxhizWq3s0qVLLCkpiY0bN45VVVXJ2HJ5Ub+Jd/78eRYTE8MGDx7MTp8+LfRZbm4u69y5MwPAYmJiWGVlJWPsWr8dOHCA9ezZkz355JOsoaFBzubLhpY18by9zxQ/oW8wGPD666+jW7dueOeddxAaGgqO41p9PMdx6NWrF9avX48LFy7g008/dWNrlYP6TTyLxYLly5ejqqoK7777LgYNGtRmnwHX+m3cuHFYsWIFvvnmG/z4449uaq1y0LImXkfoM8WHy4kTJ3D06FE8//zz6NGjR7uDHbj2RQwfPhwPP/wwtmzZAr1e74aWKgv1m3gXL17Ed999hylTpmDcuHF29Rlwrd/uv/9+3HLLLdi8eTPMZrOLW6ostKyJ1xH6TPGHuPz3v/+Fv78/7rzzTmRnZzcZuFevXhX+fvnyZWRlZQn/Dg0Nxf33349PP/0UBQUFnjMJJhHqN/EOHz6Muro6TJ06FQUFBaivrxd+V1hYCIvFAgAwGo04c+YMOnXqJPy+e/fumDJlCl577TWUlJSgZ8+ebm+/XGhZE69D9Jnc++Xa8+STT7L+/fuzCxcusN69e7OAgADhR61WC/smfX19m/zu6aefZvn5+axr167shx9+kPtjuB31m3iLFy9moaGhLDs7m91xxx1N+sXf31/oM47jmvxOo9GwjRs3sl9++YWFhISwX3/9Ve6P4la0rInXEfpM0ZULYwwNDQ3w9/eHj48PGhoa0NDQ0OJjTSYTTCaT8G+j0Qg/Pz/heR0J9ZtjDAYD1Go1/P390djY2Orn5/vXltlshkajAWMMjY2N7miuItCyJl5H6TNFhwvHcejatSuOHTsGi8WCCRMmQKfTCb/PyclBXl4eAGDIkCHo3r278LuhQ4dCp9OhsbERoaGhYNdP57F3P7onk6rfOnfu7O6myyoyMhIGgwE6nQ5jxoxBUFCQ8DuDwYDDhw8LIZKUlCScOMlxHHr37o3S0lKoVCp06dJFro/gVmazGVlZWdBqtdDpdE4ta2FhYe5uvtuxa0fngjGGsLAwVFdXe3efyVUy2Wvz5s1Mo9GwAwcOMLPZ3ORn6dKlQvm4devWJr+zWCxsy5YtLCoqil25coVZLJYmP1arVTj0zxs5228RERHs0KFD7MqVK6yiooLp9Xqv7q/a2lq2cuVKplar2fvvv39Dn124cEE4FLlv376svLz8hn57+eWXWf/+/T3iMFFHlZSUsF27drGFCxeycePGsYSEBDZw4EAWEBDg1BgtLCyU+6NJil+/WCwWZjabmclkEn7MZjPbtGmT0+s1pfeZoisXALj99tsREhKCrVu3YuzYsU0us6FSqZr83cfHR/i3Xq/Htm3bMG7cOERHRwuPZTYXJGDXtyJ4fFXjDdWNs/1266234uabb4bRaERtbS10Oh1UKhU0Gg00Gg0CAwM9/pInjDHk5eUhIyMDFy5cQENDA6KiorB161Y89thjTSbsbfuI47gm/cYYQ3FxMb788kvce++9XlXxmUwmnDx5EocPH8ahQ4dw8eJFqFQqDBkyBNOnT0dycjL8/PyQnJzs1BiNiopy6+eSGrOpSlpap6hUKnAcJ/z7zjvvdHq9pvQ+U/zaoW/fvnjiiSfw4Ycf4oEHHsA999zT7srfarViy5YtyMzMxNdff33DiqH53/mFofmfLT3WU/Tp0wePP/44PvroI4f7zXYl2djYCIPBAL1ej/LycgCAn5+fEDQBAQEe00e1tbU4efIkMjMzUV1djcjISKSkpGDIkCGIiorCwoULsW7dOixZssSuAG1sbMQbb7wBg8GAZ5991mP6oTVarRaHDh3C4cOHcezYMej1eoSHhyMpKQl/+tOfMGbMmCbLhtVqdXiM/vrrr9i4cWOTMap0tuuJlsKkeZC0ROr1mhIpPlxUKhUWL16MgwcPYtasWfj4448xYcIEqFQqqFQqqNVq4YtkjMFisWDHjh1YtmwZZs2aheTk5Hbfo3nF0ry6sf1T6dUNv7BzHIfFixfj0KFDkvSbv78//P39ERoaCqvVKgRNXV0dqqurwXGcEDRKrGoYY7h48SIyMjKQk5MDtVqNm2++GYmJiejRo4fwuBkzZuDAgQN4++23ERgYiFmzZiEgIAAAoFaroVarm1QstbW1ePPNN7Fjxw688847GDBggCyfzxlGoxEZGRlCoOTn50OlUmHYsGF4+umnkZycjP79+zfZorblzBidPHkySktL8dVXX2HixIkIDg5286dvX1tVSfMfe7ljvSY3j7lw5dmzZzFt2jQUFBRg1qxZePrpp2G1WlFcXAwAiImJQXV1NTZu3IjPPvsMTz75JFasWIHAwEBJ3r95yPCUVN3YBgvfFnf0m9FohF6vh16vF45g8fX1FYJGzqqmpqZGqFJqamoQFRWFxMREDB48GP7+/i0+p6ysDLNnz8b333+P1NRULFiwADfddBPOnz8Pq9UKPz8/9OvXD8eOHcOqVatw8uRJvP7665g1a5bityZ5RUVFOHToEA4dOoTjx4+joaEBERERSE5ORlJSEsaMGSP6CryOLmslJSX46aefYDab8Yc//AE333yzKz6yXWwDpL0wkYLc6zVX8phwAa4NiNdffx1ffPEF1Go1Bg0ahF69esFisaCgoADnz59HeHg4Xn75ZUybNq3VlYcUWqpueO6ubmwHQUsLflFREd544w18/vnnLu8326rGYDDAbDY3qWo0Gg18fX2d+rz2tIGvUi5evAi1Wo0hQ4YgMTER0dHRdr1GfX09Nm/ejHXr1uHq1auIjY1FfHw8QkJCUFVVhfPnz6O4uBgjRozAsmXLcNttt7W6Za8EjY2NOHHihBAoly9fho+PDxISEoRA6devn9PLrKPLWkNDA/bv348zZ86gb9++mDhxolsuL99eVdL8765gb58tXrwY06dPd+l6TUoeFS6MMZhMJpw/fx579uzBsWPHUFpaCl9fX8TExGDChAlISUlBZGSkbO2z/ZPnyuqmpWqlJRaLBdnZ2di9e7db+42vagwGAxoaGsAYE6oa/uAAqfpEp9Ph5MmTOHnyJGpraxEdHS1UKX5+fg69ZklJCfbt24f9+/cjLy8PDQ0N6NKlCwYPHoyUlBSMGTNGkVuRjDFcuXJFCJPffvsNRqMR3bp1Q3JyMpKTkzF69Ogmh1tLxZllLT8/Hz/99BOMRiNuu+02DBkyRLJ2ubsqEaO9PrvzzjsRGRmp6A2Y5jwqXPg7/9ke+WWxWMBxnCJ3R7iyummvWmnvuXL0G1/V8JUNX9UEBAQIu9DEVjVWqxUXLlxARkYGcnNz4efnhyFDhiAhIcHuKsVeFosFjDFhv7jSGAwG/Pbbb0KgFBUVwdfXV6hOkpOTERMT49aVpyPLWmNjI/bv34/Tp0+jT58+mDhxYpMj98S8t9xViSNa6jPGGKxWq3CwgCfwmHDxxM5tiRTVjb3VitKZTKYmczWMMajVaiFo2qpqqqqqkJmZiZMnT6K+vh49evRAYmIiBg0a5HCV4mkYYygoKBAm4jMyMmA0GtG9e3chTEaOHKnIysoeBQUF+Omnn9DQ0CBUMa0tD0quSqTSfONa6TwmXDytY+0lprpxplpROsZYk7kak8nUpKrRaDTw8fHB+fPnkZGRgfz8fPj7+2Po0KFISEhAt27d5P4IbqHX63Hs2DHhvBOtVgs/Pz+MGDFCmDvp06eP1ywbRqMRBw4cQFZWFnr16oWUlBR07tzZriO4+H97C0/bwPaIcPG0TnVWS9WN1Wr1mi0we/BVjcFggFarxenTp3H+/HmYzWb07dsXY8aMweDBg11+cIDcGGPIzc0VqpPMzEyYzWb06tVLCJORI0cKh0t7I8YYLl26hL1798JgMCA5ORnDhg1z6lBgT+VJG9keES6e1KFSs906a2nweOMWGnBtfuPcuXNCleLj44P+/fsjPj4enTp1Eqoa/ig0b9oVVl9fj19//VUIlKtXr8Lf3x+jRo1CUlISkpOT0atXL7mb6TKtVSUmkwkHDx5EVlYWevbsidTUVISGhsrXUBl40oa24sPFkzpTanyotrRV1trcDf942z89SXl5OTIzM/H777/DYDCgd+/eSExMxE033SScmGkymZrsQuPnamwPd/akDRHGGHJycoSJ+N9//x0WiwV9+/YVwiQxMdFjDkEVo3mI2DNXcuXKFezduxf19fUYN24cEhISPHJZd5SnbGwrPlw8pSOlZDvYxHzu9g4WUOoANJvNyM7ORkZGBi5fvgyNRoNhw4YhISEBXbt2bfO5jF27fDl/YAB/eXLbuRolrpRra2tx9OhRoTopLy+HRqPBqFGjhN1dtlcO8Bb2nu3eHr6KyczMRI8ePZCSktJhrkbtKRvcig4XT+lEKbVVrYil9OqmrKwMGRkZyMrKQkNDA/r27YvExEQMHDjQ4UOkzWazEDR8VePj4yMETWBgoCwbKvwh03x1kpWVBavVitjYWOHIruHDh3vV7r3WggSQ7giuwsJC7N27F3V1dUKF1xHWFZ6w0a3ocOGP9VZyB0rF0WrFkfex/ZPnrrAxmUw4e/YsMjIyUFhYiMDAQKFKCQ8Pl/S9bKsag8EAo9EIAE3malxZ1VRXV+PIkSM4fPgwjhw5goqKCgQGBmLMmDFCdaL0K9uKIVVVIpbZbMbBgweRkZGB6OhopKamKv9eJ07yhA1vxYaLJ3SeVKSsVsRyV3Vz9epVZGZmIisrC42NjYiJiUFiYiIGDBjgthM5zWZzk7kaq9UKHx+fJnM1zrTFarXi7NmzwmHCZ86cgdVqRXx8vDB3MmzYMK84wq2tIOH/dPfyXFxcjPT0dNTU1CA5ORkjRozw6nWH1WoVKnMlUmy4dISqxV3VilhSVTcmkwlnzpxBRkYGioqKEBQUhOHDhyMhIUH2/eOMXbsdMb8Lja9q/P39hZM47alqqqqqhOrk8OHD0Ol0CA4Oxi233IKkpCQkJSXJdjkiqTQ/OVHJJyiazWYcPnwYJ06cQLdu3ZCamip5RawUSt8AV2S4KL3TpCBntSKW2MvYlJSUICMjA6dOnYLRaERcXBwSExPbvGy73CwWS5O5mtaqGqvVitOnTwsT8WfPngVjDAMHDhSqkyFDhijulgNiKLEqEUur1SI9PR3V1dUYO3YsRo4cqdhlzxlKrl4UGS7eXLUotVoRq3l1YzQacfr0aWRmZkKr1SI4OBiJiYkYPny4x52LYFvVGAwGISz5i2LW19ejc+fOGDt2LJKTkzF27Nh2j2pTKk+qSsQym804cuQIfvvtN0RGRiI1NdVjv6fWKHlDXHHhouTOcpYnVSv20mq1whFfZrMZ/fr1Q0JCAvr16yeEpxKOTBPDYrHg1KlTwpFd2dnZwtzJ8OHDhbmi4OBgYReaErccW9LepHvzv3uDkpISpKeno6qqCmPHjsWoUaM8esOuOaVWL4oLF2+sWrylWuE1Njbi9OnTyMjIQElJCTp16oThw4dj+PDhTW5/297cTfO/y6m0tBRHjhzBoUOHcPToUdTV1aFz587Crq5bbrlFOALJdq6msbERwLVbPtvO1Sjhc3lzVSKWxWLB0aNHcezYMURERCA1NRURERFyN0sSSt0gV1S4KLWTnOFN1UpRUREyMjJw5swZmM1mxMfHIzEx0e6bTImdu3Els9mMkydPCkd25eTkgOM43HzzzcJ5J4MGDWp3Y8BisTQ5As1isUClUjWZq3HX/EtHrErEunr1KtLT01FZWYkxY8Zg9OjRitvid4QSqxdFhYs3VS3eUq00NDTg1KlTyMjIQGlpKTp37oyEhAQMHz5ckjsFurO6uXr1qrCr69ixY6ivr0dYWFiTuRPbyssRtnM1/C2f+apGo9FIdsvn5pdM6chViVgWiwW//vorjh07hvDwcKSmpnrFEX1K2zBXTLgosXMc5Q3VypUrV5CRkYGzZ8/CYrFgwIABSExMRGxsrEs/k5TVjdFoxMmTJ4Uju3Jzc6FSqTB06FBhd9eAAQNcFv58VcNXNrZVDV/Z2FvVUFUivdLSUqSnp6O8vByjR4/GLbfcoqgtf7GUVr0oJly8oWrx9GrFYDAgKysLmZmZKCsrQ2hoKBITEzFs2DAEBwfL1i4x1U1xcbEQJseOHYPBYEDXrl2FMBkzZoxDdzWUQmNjoxA0tlUNHzR8VUNViftYLBYcP34cR48eRVhYGFJTUz323kBK20BXRLgorVMc4cnVyuXLl4UqhT9nIzExEX379lXkZ7FdZBsbG3HixAkcOnQIR44cQUFBAVQqlXBr36SkJPTv319xn4O/5bNer0d9fT3MZjMA3HDL5+aVCXGNsrIypKeno6ysDKNGjcLYsWMVUwGIoaTqRRHh4slVi6dWK3q9HllZWcjIyEBFRQXCwsKQkJCAYcOGISgoSO7mtamwsFCYOzl+/DgaGxsREREhhMno0aOFSktpR6Y137Vle54QvwuNv1qAr6+vEDRSzdWQ1lmtVhw/fhxHjhxBly5dkJqa6nHXflPShrrs4aKkzhDL06oVxq7d0S8jIwPnzp0DgCZVilI1NDQI1cmhQ4dw5coVqNVqJCQkCLu74uLibvgOpLqMjTPamytpqS22VY3BYIDZbAbHcU3marzh+mRKVV5ejvT0dJSWlmLkyJEYO3asR11xQSnVi+zh4olVi6dVK/X19fj999+RmZmJyspKhIeHIzExEUOHDkVgYKDczbsBH4L8YcInTpyA0WhEVFSUcJjwqFGjRFdYrYUNIE3gtFaV8K/r6FwJX9XwczWMMfj6+jY53NkTNm48idVqxW+//YYjR46gU6dOuOuuuxAdHS13s+yilA12WcNFKZ0ghqdUK4wx5OfnIyMjA+fPnwfHcRg0aBASExPRu3dvuZt3A4PBgOPHjwuBUlRUBF9fXyQmJgqB4oo5IGeqG3uqEqnba7Vam9wcja9qms/VEGlUVFRg79690Gq1GDFiBJKTkz2iilFC9SJruHhS1eIp1UpdXR1+//13ZGRkQKfTISIiAomJiRgyZAg0Go3czRPw4ccf2ZWRkQGTyYQePXoIYTJy5Ei3t7m1sGmv6pHrCC6TySQEDV/VqNXqJnM1Sl5ePQFjTNgt26lTJ6SkpCj+LqFK2HCXLVyU8OHtpfRqhTGGvLw8oUrx8fHBzTffjMTERPTs2VPu5gnq6+tx7NgxoTopKSmBn58fRo4cKcyd9O7dWxF9bFuR8N8/Y6xJiDS/dprcGGNN5mpMJlOTqkaj0XjVnS7draqqCunp6SguLhYqaiVXiXJXL7KFiydULUqvVmpra3Hy5ElkZmaiuroakZGRGDFiBAYPHoyAgAC5mwfGGHJzc4XqJDMzE2azGb179xaO7BoxYoTsbW1r9xb/p+2GhavnbqTCVzX8UWh8VWM7V6PE5VrJGGPIyMjAoUOHEBwcjJSUFEVtwNmSewNelnCR+0O3x3Ylo7RqhTGGixcvIiMjAzk5OVCr1UKVooRSva6uDr/++qsQKKWlpfD398fo0aOFQJFzMDY/OVGquRIlHJnWFsZYk7kavqqxvTkaVTX2q6qqwt69e1FUVIThw4dj/Pjxiqxi5KxeZAkXJVctSq1WqqurhfuJ1NTUICoqComJiRg8eLBL7wPfHsYYLly4IBwm/Pvvv8NqtaJv377C3EliYqJsKy6xVYmU72v7py0lBI7JZGqyC41fAfFBQ1VN+xhjOHnyJH755RcEBQVh4sSJijtYRs4NebeHC/9h5T4GuyX8Ckhp1QoAbNiwAXV1dRg8eDASExMVc1hkWVkZUlNTodFomlQn3bt3l7tpYIzBYrEAUMZlU1oKHCWswFuqajp16uR1N9ZyFZ1OJ1QxM2fOlO3yQq2Rq3pxOlzcmU2O7qpwB1e3ra6uTrjVrliubpvBYHD4DHIxz/Gm79MZrm4bH8juWNaUypHvs6amxqFg8aZlrclzlRgufPXQnDd9CdnZ2Tc8x2KxCCWsVFsZjDHcdNNNop6zb9++Vj+P1WqF2WyWZDcXYwx33HGHqMe3tmxIzZuWtfr6erf0G2NM1gucSom/v4+9GGMwm83w8fERVY0yxhAfHy+qbUoeB7acPhtI6g/ZWsc58j7uGlCA+C+hsLAQd9xxB/R6PfLz85Gfn4+KigqYTCb4+voiMjISAwcORJ8+fZzadbJv3z7R4XLq1CnMmTPnhv+/fPky1qxZg8rKSjz88MOYPHmyw+0CgHXr1ikyXNw1cN2loaEBDQ0NCA8Pd+n8XGVlpdeEi1arxbhx4+x6bHFxsXDEpkajweDBg+2+gd4vv/yi2HBx9j0kOdVUqg9p22nN52Uc3TJ05Rfg7NbqL7/8glOnTsHf3x8xMTGIjY1FQEAAGhoacOXKFXz//feIjIzEXXfdhS5dukjUavs0Pwu5qqoKM2fOxKRJkzB58mS8+uqr6Nu3L4YPH+7WdgFw+Zyds9+rlOOh+es62jYfHx9UVlYqYi7MU7S3UccYQ1ZWFg4ePIihQ4di8ODBqKysxM8//wytVovbbrvN5esfV87ZOTsOFHUdA6vVKky4urP0c5SznW82m5Gamoo+ffoIl1fn8bfb3b9/P/75z39i6tSp6NOnjyz9wRjDn/70J4wfPx4LFy4EcO2qynPnzsWBAwfc2iaVSiWc1CgFdy5jjla5UggLC0NhYaHb39eb5eTkYP/+/XjggQfQs2dPcByHvn37IjY2Fjt27EBgYCBGjRrlku+7rXHQ0lGRcpD/UJXrWjqvRKbzO93m9ttvR3x8PPz8/G5YCDiOQ6dOnTB58mTceuut+Oyzz5Cfny9LO/lrK7366qvC9/PAAw/AaDTixIkTsrSppWWDPxKx+WHHbb2GvY91lu17yLFc8+dgOPrefN9aLBavH5f2MBgM2LNnDyZPniwEC3Bt3Hbp0gUPPfQQDh06hOrqape2o/l3YXtVCX4s2PMa9j5WDMnDpfl5BfY2uPlWnTO7ANzJqaMp7Hgux3EYOXIk7rnnHuzYsQNXr151+P0cwRjDsmXLsHLlyia7yjiOw4IFC7Bo0SK3tqe1PrMdIPYETEuPVzpn2shvFBgMBtHPtVqtqK2thVarRXFxMUpLSz2iv1yFMYYvv/wSAwYMQExMTIvLZHh4OEaNGoV///vfbu0rfrnnz2tpb9l25ThwSeXCp6bYrUjb/YdK3h3mbhzHYejQobjtttuwbds24Ra57rBz504EBAQgKSnpht899NBDqK6uFm5u5U62uwT4gcGfmKtSqdpd5uQ+Udbe9gHSjYXQ0FCUlZXZ/XjGGOrr61FcXIzq6mqEhIQgMjISZrMZJSUlHTZgiouLUVFRgZSUlFa/G47jkJycDIPBgEuXLknehpb27jQPFjF7gFwxDiR9xdYGuT1bkaRtHMchKSkJffv2xSeffOK2XTkrVqzAmjVrWhxEPj4+6NGjBzZt2uTyttjiB4LtLgDgxhMlW9sSaz4IAUg6j9PS+zlKyo2s0NBQu3ZrMXbtpEqtVouKigqEhISgR48eCAkJgb+/P6Kjo2EwGITzYToSxhi+/vpr3H777e0eVMJxHCZOnIjvv//eJeO1+fqVX/favr/tY1r6LLbz3IC040DycAGaDnJ7Aqalic6OMu8iBsdxmDJlCmpra3HgwAGXv99///tf+Pj4ICEhodXHvPnmm24Lu+b46hjADZe3kHurrTVyVuQcx0GtVqOysrLF3zPGYDKZUFZWhpKSEvj7+6N79+7o1KlTk/5VqVQICgpy+y5aJdBqtTCZTBgyZIhdjx84cKBQ6blKSxtYLT2mNbbfq5Qke7WWtgaBG9OztQkoJVwGQwy5jvzx8fHB9OnTceDAgVZXElJgjOGVV17Ba6+91uZnHDJkCMxms1t3jfHLlO0GTEsHRLS01dbS1prt71zdbrlFR0ejurq6SdXB2LXL5FRVVaG4uBiMMXTv3h1hYWFQq9UttjsiIgKNjY0dauOPMYZvv/0Wt956q93fJb977Ntvv5W8r2yXcbHjAIDLx4Fb1ui2HxAQd+RMR1p47RUeHo6kpCRs27bNZf1z9uxZGI1GpKSktPk4juMQFRWFLVu2uKQdbb2v7U9bWto91lKVI0WbAPuX2fYqeVdQq9UIDg5GcXExjEYjTCYTampqUFRUhIaGBkRGRiIyMrLFIxht8b8zm80ua6vS8LcuaKuSb8mIESOg1+tdsgHWfEOrtcc0303c1t4iqUgWLq2loC3+AwJNjypr64qdFC434jgOt99+OywWC44cOeKS93jjjTcwe/Zsuxa4V199FR9//LHdr80Yk2Sg2bO8tTQ/Y08guZLc7x0REYGAgACUlJRAq9Wirq4OXbp0QXR0NDQajd1HMQYEBKCiosINrVaG3bt3t3p0WFtUKhWio6Oxd+9el7TLnuXZ9ve2VYwrl0VJwsWehjafaOV/7Nk6IjfiOA5PPvkk/vOf/6CxsVHy109LS8P06dPteuyYMWNgNBrtnuA9ffo0nn/+eWeaZzfbqtl2v3RLXLEhI9VkvpSByHEcunbtiujoaERFRSE6OhrBwcGiX79r167Q6/WStEnpGGMoLCzEXXfd5dDzJ0+ejIsXL8q2scyPA9sjeFubivCo3WK85vvI7SnnlEoJFVVkZCTi4uKwY8cOydvTrVs3u+fBOI5DUFAQ0tPT7Xr83/72N8TFxTnTPNGaL3PNcZxrz6tS2rLMcRx8fX3h5+fn8L0+nD0x05MUFBTAx8fH4WuzBQUFgeM42Q+C4Me0O8aB2yoXXvOAUcKgs91FZ+95OYD8KwyO4/Dggw/iypUrsi+08+fPx8qVK9t9HGMMeXl5LV4Y01Xamvi3fYwcPHnF3JGO6ExPTxc1kd8cx107GXr37t0St0xcG9qbp5RyHEgWLnKvaB3RUqi0dNCBkvn6+uL222/H9u3bZW3zfffdh+rq6nbbUFpaCo7jvObquY7yxPHSEo7jvH7XmNVqhcFgwNChQ516nVtuuQU1NTWyr1vctexJEi6uuoWmK7eMbMOkeaLbe16OUowdOxZmsxlZWVmytUGtVsPPzw+//fZbm4/761//ivHjx3vNyrUtjiwnSlu22hMUFASdTid3M1zq3Llzwu5DZ6jVaqjVahQUFEjTMIWTJFyUsnvLXi0dLdFSydjeLjKlfGaO4/DII4/g+++/d+mZ5u2ZOXMmXn311VZ/zxjDb7/9hmXLlrmxVfaRekPG9nUcWU6Usmy1p3PnzrJc/sedfv75Z0yYMEGS10pOTrZ7blIOUo4DzzpzUQLNg6WlYPS0sASAPn36oHPnzi473NEeM2bMwNWrV1sNuNOnT8PHxwehoaHubZgbObLceFq1YkuKO5IqGX/Y/MCBAyV5vWHDhsFgMHj0d26vDhUuLQVLa5rvImv+OkrDcRymTZuG48ePu/XClrZ8fX0RFBSEH3744YbfMcawaNEiPPvssx4X3K7iTf2gxDEhhcuXL4u+dXFbfHx84OPjg8uXL0vyekrmEeHi7ILbfA5FbGXS0vsrccXQqVMn9OvXD59//rlsg33ZsmVYvnz5De+v0+lQWlqKGTNmyNIuuTi7nCmdtx8x9tNPP2HkyJGSvubo0aPx008/SfqaSuQR4eKMli53YO+Abz7/YvunEvGHJl++fBnl5eWytOGOO+5AY2MjcnNzhf9jjGHOnDm46667brh9stLI9f06O0cjN2+cd2GMoba2FqNHj5b0dUeOHIna2lpFr0s6xJyLsyf1tHREmDOvY9suJfL19cWtt96KTz/9VJaFl+M4PPPMM3j++eeFy7sfPHgQOTk5ipzItyXVCWTOLmuextfXF7W1tXI3Q3L8IdbtXVpfLP5ioK688KwzpBoHig8XZ9kOckeP2ml+XR6lrzRuvfVWNDY24tSpU7K8/5/+9CcEBATglVdewc6dO7F48WL84x//cPjsZm+m9GXJHsHBwV55rsv+/fsRFRXlku9owIABij5qTAqS7KNw9WUzXHF4qLPtkKJNrryi7NSpU5GRkYGwsDCHnu/s9co++OADrF69Gnv27MErr7yCcePGSXINNCUva0pYTlvjykPUAwMDHbp9stI1NDRgwoQJLhmn48ePx7lz51BfX+/Q85U8DoTXYU6+iqMnirnj2H937hYS27acnByXn5PCGIOPjw/i4+NFPe/gwYOi7jJoMBgQEBBwQx/wFydtbZ5FpVJh/Pjxdr8PLWvXiG2bXq8X1T6TyST6CCn+QoiBgYGi2qZU+fn5sFgsdvc1Yww1NTXo3LmzqPfhOA6xsbGinqPkcdDkuXKEi6OXPXfHgFfqyshisUCv1yMwMFD0PmBXtq26uhparRbR0dGiBxbg2qOp+CvAOrLPvCMva1qtFiqVCpGRkS4fo0olts8uXLiAkpISjB49GgEBAaKe6+rv0/ZurWI58306PefS/Mx2e374D2l78xp7nueOtgH/N+jFPs+VbSsrK0NaWhrKysoU07bvvvsOd9xxB3bu3IlOnTo53N9Stgu4tlzxlZePj4/L2yWmz5r/tHZSr5xt69q1KxobG6HVakWPBW8h5vP+9NNP2L17NyIiIoT74ci9rAFoclsTd40DW7JM6HPctYCxDZmW7hYoB9s2kdZ98803+Nvf/oapU6di6dKlst+mmrFrt+q1DRV+QCkVPw74kFHKGPDz80N0dDTMZjO0Wi2NhVYwxrB3716cOXMGd999NwYNGiR3k4RxYBsqco1NWdcISg0ZCpi2ffXVV/jb3/6GBx98EH/5y19kDRZPDJXmlBgytgFTXFwsag6uI+CD5ezZs7j77rtx0003yd4epYQKTxGHIisxZChgWrZr1y688cYbePjhh7FkyRLZVuL8ri9PDpXmlBYyfMBYLBZotVoKmOsYY0hPTxeCRarrjjmCHwdKChWeMlpxndJChgKmqZ07d+Lvf/87Hn30Ubz88suyrMhtBxPgHaHSnJJChgKmKT5YsrOzcc8998gWLPw44OfElBQqPGW15jolhQwFzDVffvkl3nzzTTz22GNYtGiR21fmHSFUmlNKyFDAXMMYw48//ojs7GxMmjQJAwYMcHsbPCFUeMps1XVKCZmOHjBffPEFli9fjscffxwvvfSSW1foHTFUmlNCyHT0gLFarfjhhx9w7tw5TJo0Cf3793f7+3tKqPCU3brrlBAyHTVgvvjiC7z11lt44oknsHDhQret1K1WK8xmc4cOlebkDhk/Pz907969wwUMHywXLlzA5MmT3RosnhgqPM9o5XVyh0xHC5gdO3bgrbfewrRp0/Diiy+6ZcVuGyr8YOroodKcnCHj6+vboQKGD5acnBxMmjRJ9NUunHlfTw0Vnme19jo5Q6ajBMy//vUvrFixAtOnT8f8+fNdvnKnUBFPrpCxDRhvPkzZarViz549yMnJweTJk90SLN4QKjzPbPV1coUMP6C9NWD+9a9/YdWqVXjqqafwwgsvuHQFT6HiPDlChg8Yq9XqlQHDB8vFixfxP//zP+jXr5/L389bQoXn2a2/To6Q4S+P4G0Bs337dqxatQpPP/005s2b57KVPIWK9NwdMnzAMMa8KmCsVit2794tBEtcXJxL38vbQoXnHZ/iOneHjLcFzCeffILVq1fjj3/8I+bMmeOSFT2Fiuu5M2R8fX0RHR3tNQFjsVjw/fffIzc3F/fee6/LgsWbQ4XnXZ/mOneGjLcEzNatW/HOO+9g5syZmD17tuQrewoV93NXyHhLwFgsFuzevRv5+fm49957RV8K3x4dIVR43vmprnNXyNgGjNzXhHLE1q1bkZaWhj//+c94/vnnJV3h24aKSqWiUJGBO0KmecC48kZ4rsBXLK4KFttQ4ceBt4YKz7s/3XXuCBk+YGwvoe4JPv74Y6SlpeGZZ57Bc889J9lKv6VQ4VdwRB6uDhnbgNFqtR4TMBaLBd999x0KCgpw7733IiYmRrLXbilUOsoY6BDhwnN1yHhawHz00UdYv349nn32WcmChUJF+VwZMraT/J4QMHywXLp0Cffdd59kwdKRQ4XXocKF58qQ8ZSA+eijj7BhwwY899xzePbZZ51+PQoVz+OqkFGr1R4RMM2DpW/fvk6/JoXK/+mQ4cJrKWT4BcPZ11VywGzevBkbNmzArFmz8Mwzzzj1Ws1DRa1WU6h4GFeEjG3AKHEOxmw249tvv8Xly5dx//33Ox0sFCo36tDhwrMNGX5i3tmQUWrAbNq0Ce+99x5mz56NP//5zw6/TmuhQjyX1CHDBwwARQUMHyxXrlzB/fffjz59+jj8WhQqraO1gQ2pQ0ZpAbNp0yZs2rQJc+bMwcyZMx16DQoV7ydlyPABw3GcIgKGD5bCwkLcf//96N27t0OvQ6HSPlortEDKkFFCwDDG8P7772PTpk2YO3cu/vjHP4p+DQqVjkeqkFGr1YiOjpY9YMxmM7755hungoVCxX60dmiDVCEjZ8AwxvDee+/hgw8+wAsvvICnn35a1PMpVIgUIdM8YEwmk4ta2zI+WIqKivDAAw+IDhYKFfFoLWEHKUJGjoBhjGHjxo348MMPMX/+fDz11FN2P5dChTTXUsiICRrbgNFqtW4LGLPZjK+//hpFRUWYMmUKevXqZfdzKVQcR2sLEZwNGXcGDGMMGzZswEcffYQFCxZg+vTpdj2PQoW0xzZkAIiqZtwdMHywaLVaTJkyBT179rTreRQqzqO1hgOcCRl3BAxjDOvXr8c///lPLFy4ENOmTWv3ORQqRCxHQ8Z2kt+VAWMymfDVV1+JChYKFenQ2sMJjoaMKwOGMYZ169Zhy5YteOmll/DEE0+0+XgKFeIsR0LGx8fHpQFjMpnw9ddfo6SkBFOmTEGPHj3afDyFivRoLSIBR0LGFQHDGENaWhq2bt2KRYsW4fHHH2/1sRQqRGpiQ8Y2YKSc5OcrlpKSEkydOrXNYKFQcR1am0hIbMhIGTCMMbzzzjvYtm0bXn75ZTz22GMtPo5ChbiamJDhA8bHx0eSgDGZTNi1axdKS0sxdepU4SROW3x7KFRci9YqLsAPLn6BbStkpAgYxhjWrFmDTz/9FEuWLMEjjzxyw2MoVIi72RsyPj4+iI6OdjpgjEYjdu3ahbKyshaDpfkRbhQqrkVrFxezJ2TaCxjGGCoqKlBVVYWKioomj2GMYfXq1di+fTuWLFmChx9+uMlzKVSI3OwJGXsChjGG8vJyFBQUoLy8vMnzmwdLdHR0k+dRqLgfx5RwXZIOxHZQNb/AIx8ufNjodDps3boV69evR25urvC4uLg4zJ07F9OnT8eHH36Izz77DEuXLsWDDz7Y5H34u2PaXpyTELk134jil3fg2pWKtVotLBYLoqOj4efn1+44eOyxx/Cf//wHFRUVmDp1KqKiopq8T2vjjbgWhYtMWgsZfjDs3bsXDz74IPR6vfD/PL7K8fX1Re/evbFixQpMmTJFeF0KFeIJWgsZ24A5deoUHn300TbHgb+/P2bOnIm//vWviIqKolBRCFrzyKS13WUcx2Hv3r2YNGkS9Hp9i7vK+H+bTCbk5+cjKCiIdn8Rj2O7u8z2rH+VSoXo6Gj88ssvuO+++9odB0ajEe+//z5OnjxJu78UhCoXheAHhE6nQ+/evWEwGOw+KVOj0eDSpUsICwujQCEejQ8RnU6Hnj17oqGhQdQ4uHz5MsLCwihQFIDWRArBb2Vt27ZN2FKzB2MMer0e//rXvyhYiMfjq5lt27bZvYEF/N842L59OwWLQlDloiCMMcTHxyMvL0/0RTFjY2ORk5NDA4t4PBoH3oHCRUHKy8sRERHh1PPDw8MlbBEh7kfjwDvQfhQFqaurc+r5tbW1ErWEEPnQOPAOFC4KEhwc7NTzQ0JCJGoJIfKhceAdKFwUJDw8HHFxcaL3F3Mch7i4OISFhbmoZYS4D40D70DhoiAcx2Hu3LkOPXfevHk0iUm8Ao0D70AT+grDH99vMBiEM+3bolKpoNFoUFhYiNDQUNc3kBA3oHHg+ahyUZjQ0FDs3LlTON6/LfzZzbt27aIBRbwKjQPPR+GiQKmpqdi9ezc0Gk2Ti/rx+P/TaDTYs2cPUlJSZGopIa5D48CzUbgoVGpqKgoLC7F27VrExsY2+V1sbCzWrl2LoqIiGlDEq9E48Fw05+IBGGOorKxEbW0tQkJC6NpJpEOiceBZKFwIIYRIjnaLEUIIkRyFCyGEEMlRuBBCCJEchQshhBDJUbgQQgiRHIULIYQQyVG4EEIIkRyFCyGEEMlRuBBCCJEchQshhBDJUbgQQgiRHIULIYQQyVG4EEIIkRyFCyGEEMn9f8QwIdbXy+izAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x400 with 22 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.plot(metric='forward_u')\n",
    "#model.plot(metric='forward_n')\n",
    "#model.plot(metric='backward') # default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1692e33b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}