{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fetching live option chain from NSE in pandas dataframe\n",
    "\n",
    "![Option Chain](option_chain.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Option chain\n",
    "\n",
    "Option chain of a security is a single table which shows premium price, open interest, volume traded etc for both put and call options and for every strike price\n",
    "\n",
    "## Install required packages\n",
    "\n",
    "`pip install requests bs4 pandas`\n",
    "\n",
    "## Getting started\n",
    "\n",
    "We import the required modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "URL for option chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below headers are required for NSE to respond. Without these headers the request will just time out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {\n",
    "        \"Host\": \"www1.nseindia.com\",\n",
    "        \"Referer\": \"https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp\",\n",
    "        \"X-Requested-With\": \"XMLHttpRequest\",\n",
    "        \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36\",\n",
    "        \"Accept\": \"*/*\",\n",
    "        \"Accept-Encoding\": \"gzip, deflate, br\",\n",
    "        \"Accept-Language\": \"en-GB,en-US;q=0.9,en;q=0.8\",\n",
    "        \"Cache-Control\": \"no-cache\",\n",
    "        \"Connection\": \"keep-alive\",\n",
    "        }\n",
    "r = requests.get(url, headers=headers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open chrome and go to https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp . Press Ctrl+U to view the source. You can see that the table tag has an id - `octable`. We can extract the table using this id as below-"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs = BeautifulSoup(r.text)\n",
    "table = bs.find(\"table\", {\"id\": \"octable\"})\n",
    "# Get all rows\n",
    "table_rows = table.find_all('tr')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract the text in each cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = []\n",
    "for tr in table_rows:\n",
    "    td = tr.find_all('td')\n",
    "    if td:\n",
    "        row = [tr.text for tr in td]\n",
    "        l.append(row)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets write a function which tries to convert the cell to `numpy.float64` and if it fails because of `-` in many cells, it will return `numpy.nan`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def np_float(x):\n",
    "    try:\n",
    "        y = x.lstrip().rstrip().replace(',','')\n",
    "        return np.float64(y)\n",
    "    except:\n",
    "        return np.nan\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loop thru each cell to convert to a array of `numpy.float64`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = []\n",
    "for r in l:\n",
    "    row = [np_float(x) for x in r]\n",
    "    arr.append(row)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert the array to a dataframe and provide column headers manually"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(arr[:-1]) # Leave last row of summary\n",
    "df.columns = [\"CE Chart\", \"CE OI\", \"CE Change in OI\", \"CE Volume\", \"CE IV\", \"CE LTP\", \"CE Net Change\", \"CE Bid Qty\", \"CE Bid Price\", \"CE Ask Price\", \"CE Ask Quantity\",\n",
    "             \"Strike Price\",\n",
    "             \"PE Bid Qty\", \"PE Bid Price\", \"PE Ask Price\", \"PE Ask Qty\", \"PE Net Change\", \"PE LTP\", \"PE IV\", \"PE Volume\", \"PE Change in OI\", \"PE OI\", \"PE Chart\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's how the output looks like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   CE Chart  CE OI  CE Change in OI  CE Volume  CE IV  CE LTP  CE Net Change  \\\n",
      "0       NaN    NaN              NaN        NaN    NaN     NaN            NaN   \n",
      "1       NaN    NaN              NaN        NaN    NaN     NaN            NaN   \n",
      "2       NaN    NaN              NaN        NaN    NaN     NaN            NaN   \n",
      "3       NaN    NaN              NaN        NaN    NaN     NaN            NaN   \n",
      "4       NaN    NaN              NaN        NaN    NaN     NaN            NaN   \n",
      "\n",
      "   CE Bid Qty  CE Bid Price  CE Ask Price  ...  PE Bid Price  PE Ask Price  \\\n",
      "0      7350.0       3110.30       3844.05  ...          0.25           0.4   \n",
      "1      7350.0       3065.20       3786.65  ...          0.10           0.5   \n",
      "2      7350.0       3019.80       3732.70  ...          0.20           0.5   \n",
      "3      7350.0       2975.00       3674.90  ...          0.10           0.6   \n",
      "4      7350.0       2929.55       3618.65  ...          0.30           0.4   \n",
      "\n",
      "   PE Ask Qty  PE Net Change  PE LTP  PE IV  PE Volume  PE Change in OI  \\\n",
      "0      1725.0          -0.35    0.30  98.28     4032.0            300.0   \n",
      "1      3000.0            NaN     NaN    NaN        NaN              NaN   \n",
      "2      3000.0        -198.35    0.25  93.68     1680.0              NaN   \n",
      "3       600.0            NaN     NaN    NaN        NaN              NaN   \n",
      "4      1800.0          -0.15    0.35  93.10     1175.0           3225.0   \n",
      "\n",
      "    PE OI  PE Chart  \n",
      "0   450.0       NaN  \n",
      "1     NaN       NaN  \n",
      "2     NaN       NaN  \n",
      "3     NaN       NaN  \n",
      "4  5550.0       NaN  \n",
      "\n",
      "[5 rows x 23 columns]\n"
     ]
    }
   ],
   "source": [
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fetching option chain for any stock/index for any expiry date\n",
    "\n",
    "If you want to see the option chain for a security other than the default NIFTY, below is the URL format-\n",
    "[https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp?segmentLink=17&`instrument=OPTSTK`&`symbol=SBIN`&`date=30JUL2020`](https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp?segmentLink=17&instrument=OPTSTK&symbol=SBIN&date=30JUL2020)\n",
    "\n",
    "Observe the parameters at the end. Now let's start putting everything together in function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from urllib.parse import urlencode\n",
    "\n",
    "def np_float(x):\n",
    "    try:\n",
    "        y = x.lstrip().rstrip().replace(',','')\n",
    "        return np.float64(y)\n",
    "    except:\n",
    "        return np.nan\n",
    "\n",
    "def option_chain(symbol, instrument, date_=\"-\"):\n",
    "    base_url = \"https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp?\"\n",
    "    parameters = {\n",
    "        \"segmentLink\": 17,\n",
    "        \"instrument\": instrument,\n",
    "        \"symbol\": symbol,\n",
    "        \"date\": date_\n",
    "    }\n",
    "    url = base_url + urlencode(parameters)\n",
    "    headers = {\n",
    "        \"Host\": \"www1.nseindia.com\",\n",
    "        \"Referer\": \"https://www1.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp\",\n",
    "        \"X-Requested-With\": \"XMLHttpRequest\",\n",
    "        \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36\",\n",
    "        \"Accept\": \"*/*\",\n",
    "        \"Accept-Encoding\": \"gzip, deflate, br\",\n",
    "        \"Accept-Language\": \"en-GB,en-US;q=0.9,en;q=0.8\",\n",
    "        \"Cache-Control\": \"no-cache\",\n",
    "        \"Connection\": \"keep-alive\",\n",
    "        }\n",
    "    r = requests.get(url, headers=headers)\n",
    "    \n",
    "    bs = BeautifulSoup(r.text)\n",
    "    table = bs.find(\"table\", {\"id\": \"octable\"})\n",
    "    # Get all rows\n",
    "    table_rows = table.find_all('tr')\n",
    "    \n",
    "    l = []\n",
    "    for tr in table_rows:\n",
    "        td = tr.find_all('td')\n",
    "        if td:\n",
    "            row = [tr.text for tr in td]\n",
    "            l.append(row)\n",
    "    \n",
    "    arr = []\n",
    "    for r in l:\n",
    "        row = [np_float(x) for x in r]\n",
    "        arr.append(row)\n",
    "    \n",
    "    df = pd.DataFrame(arr[:-1])\n",
    "    df.columns = [\"CE Chart\", \"CE OI\", \"CE Change in OI\", \"CE Volume\", \"CE IV\", \"CE LTP\", \"CE Net Change\", \"CE Bid Qty\", \"CE Bid Price\", \"CE Ask Price\", \"CE Ask Quantity\",\n",
    "                 \"Strike Price\",\n",
    "                 \"PE Bid Qty\", \"PE Bid Price\", \"PE Ask Price\", \"PE Ask Qty\", \"PE Net Change\", \"PE LTP\", \"PE IV\", \"PE Volume\", \"PE Change in OI\", \"PE OI\", \"PE Chart\"]\n",
    "    return df\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now try to use this to get BANKNIFTY option chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     CE Chart    CE OI  CE Change in OI  CE Volume  CE IV   CE LTP  \\\n",
      "0         NaN     20.0             20.0        6.0    NaN  8071.25   \n",
      "1         NaN      NaN              NaN        NaN    NaN      NaN   \n",
      "2         NaN      NaN              NaN        NaN    NaN      NaN   \n",
      "3         NaN      NaN              NaN        NaN    NaN      NaN   \n",
      "4         NaN      NaN              NaN        NaN    NaN      NaN   \n",
      "..        ...      ...              ...        ...    ...      ...   \n",
      "112       NaN  78820.0          14980.0    18094.0  41.29     3.25   \n",
      "113       NaN  10100.0           5000.0     1787.0  41.39     2.65   \n",
      "114       NaN   3120.0           3120.0      333.0  42.27     2.55   \n",
      "115       NaN      NaN              NaN        NaN    NaN      NaN   \n",
      "116       NaN  36660.0          36660.0     3100.0  45.81     3.35   \n",
      "\n",
      "     CE Net Change  CE Bid Qty  CE Bid Price  CE Ask Price  ...  PE Bid Price  \\\n",
      "0          4390.85        40.0       8069.40       8207.15  ...          1.30   \n",
      "1              NaN      2400.0       7548.45       8812.25  ...          0.35   \n",
      "2              NaN        40.0       7794.35       8094.75  ...          1.30   \n",
      "3              NaN      2400.0       7369.30       8590.55  ...           NaN   \n",
      "4              NaN      2400.0       7275.05       8489.00  ...           NaN   \n",
      "..             ...         ...           ...           ...  ...           ...   \n",
      "112          -3.90       820.0          3.25          3.30  ...       2990.65   \n",
      "113          -3.10       160.0          2.65          3.20  ...       2973.75   \n",
      "114         -24.50        40.0          2.35          3.90  ...       3069.30   \n",
      "115            NaN      1200.0          2.55          4.00  ...       3176.25   \n",
      "116         -16.85       800.0          2.55          3.30  ...       3375.45   \n",
      "\n",
      "     PE Ask Price  PE Ask Qty  PE Net Change  PE LTP   PE IV  PE Volume  \\\n",
      "0            1.55       480.0          -1.00    1.30  119.58      330.0   \n",
      "1            2.95       200.0            NaN     NaN     NaN        NaN   \n",
      "2            1.65       140.0           0.25    1.65  118.69       32.0   \n",
      "3             NaN         NaN            NaN     NaN     NaN        NaN   \n",
      "4             NaN         NaN            NaN     NaN     NaN        NaN   \n",
      "..            ...         ...            ...     ...     ...        ...   \n",
      "112       3134.20       200.0            NaN     NaN     NaN        NaN   \n",
      "113       3463.30       500.0            NaN     NaN     NaN        NaN   \n",
      "114       3484.20       500.0            NaN     NaN     NaN        NaN   \n",
      "115       3597.65       500.0            NaN     NaN     NaN        NaN   \n",
      "116       3548.25        40.0            NaN     NaN     NaN        NaN   \n",
      "\n",
      "     PE Change in OI   PE OI  PE Chart  \n",
      "0             3900.0  4580.0       NaN  \n",
      "1                NaN     NaN       NaN  \n",
      "2              240.0   360.0       NaN  \n",
      "3                NaN     NaN       NaN  \n",
      "4                NaN     NaN       NaN  \n",
      "..               ...     ...       ...  \n",
      "112              NaN     NaN       NaN  \n",
      "113              NaN     NaN       NaN  \n",
      "114              NaN     NaN       NaN  \n",
      "115              NaN     NaN       NaN  \n",
      "116              NaN     NaN       NaN  \n",
      "\n",
      "[117 rows x 23 columns]\n"
     ]
    }
   ],
   "source": [
    "print(option_chain(\"BANKNIFTY\", \"OPTIDX\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SBIN option chain (Please change the expiry date to the latest future expiry dates, it does not seem to work on past expiry dates)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    CE Chart      CE OI  CE Change in OI  CE Volume  CE IV  CE LTP  \\\n",
      "52       NaN  5751000.0         732000.0     3355.0  63.13    0.65   \n",
      "53       NaN  1308000.0         540000.0     1138.0  64.01    0.45   \n",
      "54       NaN   405000.0          84000.0      159.0  69.34    0.35   \n",
      "55       NaN  1371000.0          66000.0      116.0  73.94    0.30   \n",
      "56       NaN   999000.0         180000.0      231.0  77.81    0.25   \n",
      "\n",
      "    CE Net Change  CE Bid Qty  CE Bid Price  CE Ask Price  ...  PE Bid Price  \\\n",
      "52          -0.15    195000.0          0.65          0.70  ...         54.25   \n",
      "53          -0.05     30000.0          0.40          0.45  ...         63.30   \n",
      "54          -0.05      9000.0          0.30          0.35  ...         73.20   \n",
      "55            NaN    111000.0          0.25          0.30  ...         82.60   \n",
      "56            NaN     33000.0          0.25          0.30  ...         92.80   \n",
      "\n",
      "    PE Ask Price  PE Ask Qty  PE Net Change  PE LTP  PE IV  PE Volume  \\\n",
      "52         55.20      3000.0           2.95    53.5  51.18       12.0   \n",
      "53         66.10     30000.0            NaN    62.3    NaN        NaN   \n",
      "54         75.80     30000.0            NaN    72.5    NaN        NaN   \n",
      "55         85.75     30000.0            NaN    92.5    NaN        NaN   \n",
      "56         95.80     33000.0            NaN    93.7    NaN        NaN   \n",
      "\n",
      "    PE Change in OI     PE OI  PE Chart  \n",
      "52          24000.0  273000.0       NaN  \n",
      "53              NaN   42000.0       NaN  \n",
      "54              NaN   18000.0       NaN  \n",
      "55              NaN   45000.0       NaN  \n",
      "56              NaN  210000.0       NaN  \n",
      "\n",
      "[5 rows x 23 columns]\n"
     ]
    }
   ],
   "source": [
    "print(option_chain(\"SBIN\", \"OPTSTK\", \"30JUL2020\").tail())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}