lstm_np.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-rkvSDzIUdLm"
   },
   "source": [
    "# Training an LSTM network on the Penn Tree Bank (PTB) dataset - Part I: Python/NumPy implementation  \n",
    "---\n",
    "\n",
    "[1]: http://people.idsia.ch/~juergen/\n",
    "[2]: http://colah.github.io/posts/2015-08-Understanding-LSTMs/\n",
    "[3]: https://www.coursera.org/specializations/deep-learning\n",
    "[4]: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture10.pdf\n",
    "[5]: https://arxiv.org/abs/1503.04069\n",
    "[6]: http://www.fit.vutbr.cz/~imikolov/rnnlm/simple-examples.tgz\n",
    "[7]: https://arxiv.org/abs/1409.2329\n",
    "[8]: https://www.python.org/\n",
    "[9]: http://www.numpy.org/\n",
    "[10]: https://www.tensorflow.org/\n",
    "[11]: https://www.tensorflow.org/guide/eager\n",
    "[12]: https://colab.research.google.com/notebooks/welcome.ipynb\n",
    "[13]: https://www.tensorflow.org/guide/graphs\n",
    "[14]: https://gist.github.com/karpathy/d4dee566867f8291f086\n",
    "[15]: https://github.com/tensorflow/models/tree/master/tutorials/rnn/ptb\n",
    "[16]: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/examples/rnn_ptb\n",
    "\n",
    "\n",
    "## An object-oriented implementation of an LSTM network in Python/NumPy\n",
    "\n",
    "This is a minimalist implementation of an LSTM network, partly inspired by Andrej Karpathy's [minimalist character-level language model][14]. There is no GPU acceleration built in. There are four main classes:\n",
    "\n",
    "1. Lstm\n",
    "2. Weight\n",
    "3. Network\n",
    "4. Embedding\n",
    "\n",
    "Instances of Lstm can be strung together to form a multi-layer recurrent neural network unrolled to any specified length. Each layer of the network has an associated instance of Weight for maintaining/updating weights and biases. All Lstm instances belonging to the same layer reference the same Weight instance. The Network class will handle this stringing together of Lstm instances, and an instance of Network represents a fully configured network. There is an Embedding class which encapsulates word embeddings, which are trainable dense limited-length vector representations of the vocabulary, used for input. Additionally, there are two derived classes:\n",
    "\n",
    "1. LstmWithOutput\n",
    "2. WeightWithOutput\n",
    "\n",
    "LstmWithOutput is a derived class, derived from Lstm. The only difference is that while the Lstm base class outputs the hidden state itself, LstmWithOutput adds an extra linear operator which acts on the hidden state to produce an output of a specified size. In the network implemented here, the deepest layer comprises of instances of this. Instances of LstmWithOutput must reference an instance of WeightWithOutput instead of Weight. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class Lstm(object):\n",
    "    \"\"\"An instance of this class forms a single node of an LSTM network. Nodes can be\n",
    "    strung together to form an LSTM neural network of any depth unrolled to any length.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self,wt):\n",
    "        \"\"\"Associate an instance of the Weight class with this Lstm instance.\n",
    "        All Lstm instances belonging to the same layer reference the same Weight instance.\n",
    "        \"\"\"\n",
    "        self.wt=wt\n",
    "\n",
    "    def fwd_prop(self,cprev,hprev,x):\n",
    "        \"\"\"Execute a forward prop.\n",
    "        \n",
    "        Arguments:\n",
    "        cprev -- previous cell state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        hprev -- previous hidden state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        x -- input, numpy array of shape (ni=input size, m=batch size)\n",
    "\n",
    "        Returns:\n",
    "        c -- current cell state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        h -- current hidden state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        y -- output, a reference to current hidden state, derived classes may over ride this\n",
    "        \"\"\"\n",
    "        ns=self.wt.W.shape[0]/4\n",
    "        ni=self.wt.W.shape[1]-ns#\n",
    "        m=x.shape[1]#batch size\n",
    "        if cprev.shape!=(ns,m):\n",
    "            raise Exception(\"cprev shape error: {}\".format(cprev.shape))\n",
    "        if hprev.shape!=(ns,m):\n",
    "            raise Exception(\"hprev shape error: {}\".format(hprev.shape))\n",
    "        if x.shape[0]!=ni:\n",
    "            raise Exception(\"x shape error: {}\".format(x.shape))\n",
    "        \n",
    "        self.cprev=cprev\n",
    "        self.hprev=hprev\n",
    "        self.x=x\n",
    "        fizo=np.dot(self.wt.W,np.concatenate((hprev,x),axis=0))+self.wt.b#hprev stacked on top of x\n",
    "        fizo=np.split(fizo,4,axis=0)#split into four\n",
    "        self.f=sigmoid(fizo[0])\n",
    "        self.i=sigmoid(fizo[1])\n",
    "        self.z=np.tanh(fizo[2])\n",
    "        self.o=sigmoid(fizo[3])\n",
    "        self.c=self.f*cprev+self.i*self.z\n",
    "        self.h=self.o*np.tanh(self.c)\n",
    "        return self.c,self.h,self.h#passes 'h' thru to output w/o any transformation\n",
    "        \n",
    "    def back_prop(self,dcnext,dhnext,dy):\n",
    "        \"\"\"Execute a back prop. A foward prop should be executed before calling this method.\n",
    "        \n",
    "        Arguments:\n",
    "        dcnext -- cell state gradient from next unit in the sequence, numpy array of shape \n",
    "                    (ns=hidden size, m=batch size)\n",
    "        dhnext -- hidden state gradient from next unit in the sequence, numpy array of shape \n",
    "                    (ns=hidden size, m=batch size)\n",
    "        dy -- hidden state gradient from output, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        \n",
    "        Returns:\n",
    "        dcprev -- previous cell state gradient for previous unit in the sequence, \n",
    "                    numpy array of shape (ns=hidden size, m=batch size)\n",
    "        dhprev -- previous hidden state gradient for previous unit in the sequence, \n",
    "                    numpy array of shape (ns=hidden size, m=batch size)\n",
    "        dx -- input gradient, numpy array of shape (ni=inpt size, m=batch size)\n",
    "        \"\"\"\n",
    "        if dcnext.shape!=self.c.shape:\n",
    "            raise Exception(\"dcnext shape error: {}\".format(dcnext.shape))\n",
    "        if dhnext.shape!=self.h.shape:\n",
    "            raise Exception(\"dhnext shape error: {}\".format(dhnext.shape))\n",
    "        if dy.shape!=self.h.shape:\n",
    "            raise Exception(\"dy shape error: {}\".format(dy.shape))\n",
    "\n",
    "        tanhc=np.tanh(self.c)\n",
    "        dh=dhnext+dy#backprop into h\n",
    "        dc=dh*self.o*(1-tanhc**2)+dcnext   #backprop into c\n",
    "        df=dc*self.cprev*self.f*(1-self.f) #backprop into f then thru sigmoid\n",
    "        di=dc*self.z*self.i*(1-self.i)     #backprop into i then thru sigmoid\n",
    "        dz=dc*self.i*(1-self.z**2)         #backprop into z then thru tanh\n",
    "        do=dh*tanhc*self.o*(1-self.o)      #backprop into o then thru sigmoid\n",
    "        \n",
    "        dfizo=np.concatenate((df,di,dz,do),axis=0)\n",
    "        dcprev=dc*self.f#backprop into cprev\n",
    "        dhprev,dx=np.split(np.dot(self.wt.W.T,dfizo),[self.hprev.shape[0]],axis=0)#backprop into hprev,x\n",
    "\n",
    "        self.wt.dW+=np.dot(dfizo,np.concatenate((self.hprev,self.x),axis=0).T)\n",
    "        #self.wt.db+=np.sum(dfizo,axis=1)[:,np.newaxis]#**WARNING** corrected 2018-05-19\n",
    "        self.wt.db+=np.sum(dfizo,axis=1,keepdims=True)#**WARNING** corrected 2018-05-19\n",
    "        return dcprev,dhprev,dx\n",
    "    \n",
    "    def savetxt(self, suffix):\n",
    "        np.savetxt(\"h\"+suffix,self.h)\n",
    "        np.savetxt(\"c\"+suffix,self.c)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class Weight(object):\n",
    "    \"\"\"Instances of this class are responsible for:\n",
    "    -- maintaining trainable weights and biases\n",
    "    -- collecting gradients during back prop\n",
    "    -- updating weights and biases using one of the update methods\n",
    "    Each layer of a recurrent neural network has a reference to an instance of this class or a class \n",
    "    derived from this.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self,ns,ni,eps=1e-2):\n",
    "        \"\"\"Initialize weights and biases.\n",
    "        \n",
    "        Arguments:\n",
    "        ns -- hidden size, cell size, output size\n",
    "        ni -- input size\n",
    "        \n",
    "        Keyword arguments:\n",
    "        eps -- multiplier to keep initial weights small\n",
    "        \"\"\"\n",
    "        init_scale=10\n",
    "        #trainable parameters\n",
    "        self.W=np.random.uniform(-init_scale,init_scale,(4*ns,ns+ni))*eps\n",
    "        #self.W=np.random.randn(4*ns,ns+ni)*eps\n",
    "        self.b=np.zeros((4*ns,1))\n",
    "        #self.b=np.random.randn(4*ns,1)\n",
    "\n",
    "        #gradients\n",
    "        self.dW=np.zeros_like(self.W)\n",
    "        self.db=np.zeros_like(self.b)\n",
    "\n",
    "        #Adam - momentum\n",
    "        self.vdW=np.zeros_like(self.W)\n",
    "        self.vdb=np.zeros_like(self.b)\n",
    "\n",
    "        #Adam - RMSprop\n",
    "        self.sdW=np.zeros_like(self.W)\n",
    "        self.sdb=np.zeros_like(self.b)\n",
    "\n",
    "        self.t=0\n",
    "        \n",
    "    def update_adam(self,alpha=2E-3,beta1=0.9,beta2=0.999,eps=1e-8):\n",
    "        \"\"\"Execute Adam update with specified hyper parameters.\n",
    "        \n",
    "        Keyword arguments:\n",
    "        alpha -- learning rate\n",
    "        beta1 -- Adam - adaptive moment estimation - momentum\n",
    "        beta2 -- Adam - adaptive moment estimation - RMSprop\n",
    "        eps -- small number to preclude division by zero\n",
    "        \"\"\"\n",
    "        self.t+=1\n",
    "        \n",
    "        self.vdW=beta1*self.vdW+(1-beta1)*self.dW\n",
    "        self.vdb=beta1*self.vdb+(1-beta1)*self.db\n",
    "\n",
    "        self.sdW=beta2*self.sdW+(1-beta2)*np.square(self.dW)\n",
    "        self.sdb=beta2*self.sdb+(1-beta2)*np.square(self.db)\n",
    "\n",
    "        denom=1-np.power(beta1,self.t)\n",
    "        vdW_corr=self.vdW/denom\n",
    "        vdb_corr=self.vdb/denom\n",
    "\n",
    "        denom=1-np.power(beta2,self.t)\n",
    "        sdW_corr=self.sdW/denom\n",
    "        sdb_corr=self.sdb/denom\n",
    "\n",
    "        self.W-=alpha*vdW_corr/(np.sqrt(sdW_corr)+eps)\n",
    "        self.b-=alpha*vdb_corr/(np.sqrt(sdb_corr)+eps)\n",
    "\n",
    "        self.dW.fill(0)\n",
    "        self.db.fill(0)\n",
    "\n",
    "    def update(self,alpha=2E-3):\n",
    "        \"\"\"Execute update with specified learning rate.\n",
    "        \n",
    "        Keyword arguments:        \n",
    "        alpha -- learning rate\n",
    "        \"\"\"        \n",
    "        self.W-=alpha*self.dW\n",
    "        self.b-=alpha*self.db\n",
    "\n",
    "        self.dW.fill(0)\n",
    "        self.db.fill(0)\n",
    "        \n",
    "    def get_gradient_norm(self,order=2):\n",
    "        return (np.linalg.norm(self.dW,order)**order+np.linalg.norm(self.db,order)**order)**(1/order)\n",
    "    \n",
    "    def shrink_gradient(self,factor):\n",
    "        self.dW*=factor\n",
    "        self.db*=factor\n",
    "\n",
    "    def savetxt(self, suffix):\n",
    "        np.savetxt(\"W\"+suffix,self.W)\n",
    "        np.savetxt(\"b\"+suffix,self.b)\n",
    "\n",
    "\n",
    "    def loadtxt(self, suffix):\n",
    "        self.W=np.loadtxt(\"W\"+suffix,ndmin=2)\n",
    "        self.b=np.loadtxt(\"b\"+suffix,ndmin=2)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class Network(object):\n",
    "    \"\"\"This is a composition class for configuring an Lstm network. An instance of this class  \n",
    "    represents a fully configured Lstm network of a specified depth, unrolled to a specified  \n",
    "    length. It encapsulates the portion of the network from after the embedding layer on the \n",
    "    input side to before the softmax layer on the outpt side. The method signatures are same as \n",
    "    those of the Lstm classes, with one notable difference. Every forward_prop and back_prop \n",
    "    method argument and return value is a list of numpy arrays instead of a single numpy array.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self,ns,nio,length):\n",
    "        \"\"\"Configure a recurrent neural network per specification. All but the deepest layer is made up \n",
    "        of Lstm's which ouput their hidden states. The deepest layer is made up of Lstm's which apply \n",
    "        a linear transformation to their hidden states to produce output. \n",
    "        \n",
    "        Arguments:\n",
    "        ns -- list of hidden sizes, cell sizes; len(ns) must be equal to the number of hidden layers\n",
    "        nio -- list of i/o sizes; len(nio) must be equal to one plus the number of hidden layers\n",
    "        length -- unrolled length of network\n",
    "        \"\"\"\n",
    "        if len(nio)!=len(ns)+1:\n",
    "            raise Exception(\"number of layers: {}; i/o's: {}\".format(len(ns),len(nio)))\n",
    "        \n",
    "        self.depth=len(ns)\n",
    "        for i in range(self.depth-1):\n",
    "            if ns[i]!=nio[i+1]:\n",
    "                raise Exception(\"layer# {}; hidden size: {}; output size: {}\".format(i,ns[i],nio[i+1]))\n",
    "        \n",
    "        self.length=length\n",
    "        self.wts=[Weight(ns[i],nio[i]) for i in range(self.depth-1)]+[WeightWithOutput(ns[-1],nio[-2],nio[-1])]\n",
    "        self.net=[[Lstm(self.wts[i]) for j in range(length)] for i in range(self.depth-1)]+[\n",
    "            [LstmWithOutput(self.wts[-1]) for j in range(length)]]\n",
    "    \n",
    "    def fwd_prop(self,cprev,hprev,x):\n",
    "        \"\"\"Execute a forward prop through the entire network.\n",
    "        \n",
    "        Arguments:\n",
    "        cprev -- list of previous cell states; len(cprev) must be equal to the number of layers\n",
    "        hprev -- list of previous hidden states; len(hprev) must be equal to the number of layers\n",
    "        x -- list of inputs; len(x) must be equal to the sequence length to which the network is \n",
    "               unrolled\n",
    "\n",
    "        Returns:\n",
    "        c -- list of current cell states; len(c) is equal to the number of layers\n",
    "        h -- list of current hidden states; len(h) is equal to the number of layers\n",
    "        y -- list of outputs; len(y) is equal to the sequence length to which the network is unrolled\n",
    "        \"\"\"\n",
    "        if len(cprev)!=self.depth:\n",
    "            raise Exception(\"length of cprev: {}; expected {}\".format(len(cprev),self.depth))\n",
    "        if len(hprev)!=self.depth:\n",
    "            raise Exception(\"length of hprev: {}; expected {}\".format(len(hprev),self.depth))\n",
    "        if len(x)!=self.length:\n",
    "            raise Exception(\"length of x: {}; expected {}\".format(len(x),self.length))\n",
    "        \n",
    "        c=cprev[:]#slicing a list returns a copy\n",
    "        h=hprev[:]\n",
    "        y=x[:]\n",
    "        for i in range(self.depth):\n",
    "            for j in range(self.length):\n",
    "                c[i],h[i],y[j]=self.net[i][j].fwd_prop(c[i],h[i],y[j])\n",
    "        return c,h,y\n",
    "\n",
    "    def back_prop(self,dcnext,dhnext,dy):\n",
    "        \"\"\"Execute a back prop through the entire network. A forward prop \n",
    "        through the network must be executed before calling this method.\n",
    "        \n",
    "        Arguments:\n",
    "        dcnext -- list of cell state gradients from next in sequence; \n",
    "                    len(dcnext) must be equal to the number of layers\n",
    "        dhnext -- list of hidden state gradients from next in sequence; \n",
    "                    len(dhnext) must be equal to the number of layers\n",
    "        dy -- list of output gradients; len(dy) must be equal to the \n",
    "                sequence length to which the network is unrolled\n",
    "        \n",
    "        Returns:\n",
    "        dcprev -- list of cell state gradients for previous in sequence; \n",
    "                    len(dcprev) is equal to the number of layers\n",
    "        dhprev -- list of hidden state gradients for previous in sequence; \n",
    "                    len(dhprev) is equal to the number of layers\n",
    "        dx -- list of input gradients; len(dx) is equal to the sequence \n",
    "                length to which the network is unrolled\n",
    "        \"\"\"\n",
    "        if len(dcnext)!=self.depth:\n",
    "            raise Exception(\"length of dcnext: {}; expected {}\".format(len(dcnext),self.depth))\n",
    "        if len(dhnext)!=self.depth:\n",
    "            raise Exception(\"length of dhnext: {}; expected {}\".format(len(dhnext),self.depth))\n",
    "        if len(dy)!=self.length:\n",
    "            raise Exception(\"length of dy: {}; expected {}\".format(len(dy),self.length))\n",
    "        \n",
    "        dc=dcnext[:]#slicing a list returns a copy\n",
    "        dh=dhnext[:]\n",
    "        dx=dy[:]\n",
    "        for i in reversed(range(self.depth)):\n",
    "            for j in reversed(range(self.length)):\n",
    "                dc[i],dh[i],dx[j]=self.net[i][j].back_prop(dc[i],dh[i],dx[j])\n",
    "        return dc,dh,dx\n",
    "    \n",
    "    def get_gradient_norm(self,order=2):\n",
    "        \"\"\"gradient norm\n",
    "        \"\"\"\n",
    "        s=0\n",
    "        for wt in self.wts:\n",
    "            s+=wt.get_gradient_norm(order)**order\n",
    "        return s**(1/order)\n",
    "\n",
    "    def shrink_gradient(self,factor):\n",
    "        \"\"\"shrink gradients to max norm\n",
    "        \"\"\"\n",
    "        for wt in self.wts:\n",
    "            wt.shrink_gradient(factor)\n",
    "    \n",
    "    def update_adam(self,alpha=2E-3,beta1=0.9,beta2=0.999,eps=1e-8):\n",
    "        \"\"\"Execute Adam update with specified hyper parameters.\n",
    "        \n",
    "        Keyword arguments:\n",
    "        alpha -- learning rate\n",
    "        beta1 -- Adam - adaptive moment estimation - momentum\n",
    "        beta2 -- Adam - adaptive moment estimation - RMSprop\n",
    "        eps -- small number to preclude division by zero\n",
    "        \"\"\"\n",
    "        for wt in self.wts:\n",
    "            wt.update_adam(alpha,beta1,beta2,eps)\n",
    "\n",
    "    def update(self,alpha=2E-3):\n",
    "        \"\"\"Execute update with specified learning rate.\n",
    "        \n",
    "        Keyword arguments:\n",
    "        alpha -- learning rate\n",
    "        \"\"\"\n",
    "        for wt in self.wts:\n",
    "            wt.update(alpha)\n",
    "            \n",
    "    def savetxt(self):\n",
    "        \"\"\"Save weights, cell state and hidden state for each layer in text files.\n",
    "        \"\"\"\n",
    "        for i in range(self.depth):\n",
    "            suffix=\"_{}.txt\".format(i)\n",
    "            self.net[i][-1].savetxt(suffix)\n",
    "            self.wts[i].savetxt(suffix)\n",
    "            \n",
    "    def loadtxt(self):\n",
    "        \"\"\"Load weights for each layer from text files.\n",
    "        \"\"\"\n",
    "        for i in range(self.depth):\n",
    "            suffix=\"_{}.txt\".format(i)\n",
    "            self.wts[i].loadtxt(suffix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class Embedding(object):\n",
    "    \"\"\"An instance of this class represents an embedding.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self,vocab_size,size,eps=1e-2):\n",
    "        \"\"\"\n",
    "        Initialize embedding array of shape (vocab_size, size).\n",
    "    \n",
    "        Arguments:\n",
    "        vocab_size -- size of vocabulary\n",
    "        size -- embedding size\n",
    "        \n",
    "        Keyword arguments:\n",
    "        eps -- multiplier to keep initial weights small\n",
    "        \"\"\"\n",
    "        init_scale=10\n",
    "        self.W=np.random.uniform(-init_scale,init_scale,(vocab_size,size))*eps\n",
    "        \n",
    "    def lookup(self,ids):\n",
    "        \"\"\"Look up an embedding and return a copy.\n",
    "\n",
    "        Arguments:\n",
    "        ids -- an array or list of word ids; len(ids) is batch size\n",
    "\n",
    "        Returns:\n",
    "        embedding -- numpy array of shape (m=batch size, embedding size)\n",
    "        \"\"\"\n",
    "        #Advanced indexing always returns a copy of the data \n",
    "        #(contrast with basic slicing that returns a view).\n",
    "        return self.W[ids]\n",
    "    \n",
    "    def update(self,ids,dx,alpha=2E-3):\n",
    "        \"\"\"Update embeddings of words indicated by word id's based on gradients and learning rate.\n",
    "\n",
    "        Arguments:\n",
    "        ids -- an array or list of word ids; len(ids) is batch size\n",
    "        dx -- gradients, numpy array of shape (embedding size, m=batch size)\n",
    "        alpha -- learning rate\n",
    "        \"\"\"\n",
    "        if dx.shape!=(self.W.shape[1],len(ids)):\n",
    "            raise Exception(\"dx shape error: {}\".format(dx.shape))\n",
    "\n",
    "        for i in range(len(ids)):\n",
    "            self.W[ids[i]]-=alpha*dx[:,i]\n",
    "\n",
    "    def savetxt(self,suffix):\n",
    "        np.savetxt(\"e\"+suffix,self.W)\n",
    "\n",
    "    def loadtxt(self,suffix):\n",
    "        self.W=np.loadtxt(\"e\"+suffix,ndmin=2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "H3NM-Ee6UdL4"
   },
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class LstmWithOutput(Lstm):\n",
    "    \"\"\"Extend the base class 'Lstm' by adding a linear transformation to the hidden state to \n",
    "    produce an output whose size can be different from that of the hidden state. Typically, \n",
    "    the deepest layer of a recurrent neural network is composed of instances of this derived class.\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self,wt):\n",
    "        \"\"\"Associate an instance of the 'WeightWithOutput' class with this Lstm instance.\n",
    "        All Lstm instances belonging to the same layer reference the same Weight instance.\n",
    "        \"\"\"\n",
    "        super().__init__(wt)\n",
    "        \n",
    "\n",
    "    def fwd_prop(self,cprev,hprev,x):\n",
    "        \"\"\"Execute a forward prop.\n",
    "        \n",
    "        Arguments:\n",
    "        cprev -- previous cell state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        hprev -- previous hidden state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        x -- input, numpy array of shape (ni=input size, m=batch size)\n",
    "\n",
    "        Returns:\n",
    "        c -- current cell state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        h -- current hidden state, numpy array of shape (ns=hidden size, m=batch size)\n",
    "        y -- output produced by applying a linear transformation to the hidden state, \n",
    "                numpy array of shape (no=output size, m=batch size)\n",
    "        \"\"\"\n",
    "        c,h,h_=super().fwd_prop(cprev,hprev,x)\n",
    "        return c,h,np.dot(self.wt.Why,h_)+self.wt.by\n",
    "\n",
    "        \n",
    "    def back_prop(self,dcnext,dhnext,dy):\n",
    "        \"\"\"Execute a back prop. A foward prop should be executed before calling this method.\n",
    "        \n",
    "        Arguments:\n",
    "        dcnext -- cell state gradient from next unit in the sequence, \n",
    "                    numpy array of shape (ns=hidden size, m=batch size)\n",
    "        dhnext -- hidden state gradient from next unit in the sequence, \n",
    "                    numpy array of shape (ns=hidden size, m=batch size)\n",
    "        dy -- output gradient, numpy array of shape (no=output size, m=batch size)\n",
    "        \n",
    "        Returns:\n",
    "        dcprev -- previous cell state gradient for previous unit in the sequence, \n",
    "                    numpy array of shape (ns=hidden size, m=batch size)\n",
    "        dhprev -- previous hidden state gradient for previous unit in the sequence, \n",
    "                    numpy array of shape (ns=hidden size, m=batch size)\n",
    "        dx -- input gradient, numpy array of shape (ni=input size, m=batch size)\n",
    "        \"\"\"\n",
    "        if dy.shape!=(self.wt.Why.shape[0],self.x.shape[1]):\n",
    "            raise Exception(\"dy shape error: {}\".format(dy.shape))\n",
    "\n",
    "        dh=np.dot(self.wt.Why.T,dy)#backprop into h\n",
    "        self.wt.dWhy+=np.dot(dy,self.h.T)\n",
    "        self.wt.dby+=np.sum(dy,axis=1,keepdims=True)#**WARNING** corrected 2018-05-19\n",
    "        return super().back_prop(dcnext,dhnext,dh)\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "KwEdaMgk6ikT"
   },
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class WeightWithOutput(Weight):\n",
    "    \"\"\"Extend the base class 'Weight' with additional trainable weight and bias arrays provisioning \n",
    "    for a linear transformation on the hidden state to produce output. Typically, the deepest \n",
    "    layer of a recurrent neural network has a reference to an instance of this derived class.\n",
    "    \n",
    "    Instances of this class are responsible for:\n",
    "    -- maintaining trainable weights and biases\n",
    "    -- collecting gradients during back prop\n",
    "    -- updating weights and biases using one of the update methods\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self,ns,ni,no,eps=1e-2):\n",
    "        \"\"\"Initialize weights and biases.\n",
    "        \n",
    "        Arguments:\n",
    "        ns -- hidden size, cell size\n",
    "        ni -- input size\n",
    "        no -- output size\n",
    "        \n",
    "        Keyword arguments:\n",
    "        eps -- multiplier to keep initial weights small\n",
    "        \"\"\"\n",
    "        \n",
    "        super().__init__(ns,ni,eps)\n",
    "        \n",
    "        init_scale=10\n",
    "        \n",
    "        #trainable parameters\n",
    "        self.Why=np.random.uniform(-init_scale,init_scale,(no,ns))*eps\n",
    "        #self.Why=np.random.randn(no,ns)*eps\n",
    "        self.by=np.zeros((no,1))\n",
    "        #self.by=np.random.randn(no,1)\n",
    "\n",
    "        #gradients\n",
    "        self.dWhy=np.zeros_like(self.Why)\n",
    "        self.dby=np.zeros_like(self.by)\n",
    "\n",
    "        #Adam - momentum\n",
    "        self.vdWhy=np.zeros_like(self.Why)\n",
    "        self.vdby=np.zeros_like(self.by)\n",
    "\n",
    "        #Adam - RMSprop\n",
    "        self.sdWhy=np.zeros_like(self.Why)\n",
    "        self.sdby=np.zeros_like(self.by)\n",
    "\n",
    "        \n",
    "    def update_adam(self,alpha=2E-3,beta1=0.9,beta2=0.999,eps=1e-8):\n",
    "        \"\"\"Execute Adam update with specified hyper parameters.\n",
    "        \n",
    "        Keyword arguments:\n",
    "        alpha -- learning rate\n",
    "        beta1 -- Adam - adaptive moment estimation - momentum\n",
    "        beta2 -- Adam - adaptive moment estimation - RMSprop\n",
    "        eps -- small number to preclude division by zero\n",
    "        \"\"\"\n",
    "        \n",
    "        #call base class method first so 't' gets incremented\n",
    "        super().update_adam(alpha,beta1,beta2,eps)\n",
    "        \n",
    "        self.vdWhy=beta1*self.vdWhy+(1-beta1)*self.dWhy\n",
    "        self.vdby=beta1*self.vdby+(1-beta1)*self.dby\n",
    "\n",
    "        self.sdWhy=beta2*self.sdWhy+(1-beta2)*np.square(self.dWhy)\n",
    "        self.sdby=beta2*self.sdby+(1-beta2)*np.square(self.dby)\n",
    "\n",
    "        denom=1-np.power(beta1,self.t)\n",
    "        vdWhy_corr=self.vdWhy/denom\n",
    "        vdby_corr=self.vdby/denom\n",
    "        \n",
    "        denom=1-np.power(beta2,self.t)\n",
    "        sdWhy_corr=self.sdWhy/denom\n",
    "        sdby_corr=self.sdby/denom\n",
    "\n",
    "        self.Why-=alpha*vdWhy_corr/(np.sqrt(sdWhy_corr)+eps)\n",
    "        self.by-=alpha*vdby_corr/(np.sqrt(sdby_corr)+eps)\n",
    "\n",
    "        self.dWhy.fill(0)\n",
    "        self.dby.fill(0)\n",
    "\n",
    "    def update(self,alpha=2E-3):\n",
    "        \"\"\"Execute update with specified learning rate.\n",
    "        \n",
    "        Keyword arguments:\n",
    "        alpha -- learning rate\n",
    "        \"\"\"\n",
    "        \n",
    "        #call base class method first so 't' gets incremented\n",
    "        super().update(alpha)\n",
    "        \n",
    "        self.Why-=alpha*self.dWhy\n",
    "        self.by-=alpha*self.dby\n",
    "\n",
    "        self.dWhy.fill(0)\n",
    "        self.dby.fill(0)\n",
    "        \n",
    "    def get_gradient_norm(self,order=2):\n",
    "        return (super().get_gradient_norm(order)**order+\n",
    "                np.linalg.norm(self.dWhy,order)**order+\n",
    "                np.linalg.norm(self.dby,order)**order)**(1/order)\n",
    "    \n",
    "    def shrink_gradient(self,factor):\n",
    "        \n",
    "        super().shrink_gradient(factor)\n",
    "        \n",
    "        self.dWhy*=factor\n",
    "        self.dby*=factor\n",
    "        \n",
    "    def savetxt(self, suffix):\n",
    "        \n",
    "        super().savetxt(suffix)\n",
    "        \n",
    "        np.savetxt(\"Why\"+suffix,self.Why)\n",
    "        np.savetxt(\"by\"+suffix,self.by)\n",
    "        \n",
    "    def loadtxt(self, suffix):\n",
    "        \n",
    "        super().loadtxt(suffix)\n",
    "        \n",
    "        self.Why=np.loadtxt(\"Why\"+suffix,ndmin=2)\n",
    "        self.by=np.loadtxt(\"by\"+suffix,ndmin=2)\n",
    "        \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GGQ35kvqUdMS"
   },
   "source": [
    "---\n",
    "## Test case - gradient check\n",
    "TestGradient is a subclass of Python unittest.TestCase. Its test methods implement gradient checking. The results of gradient checking on a 3 by 3 network yield relative errors less than 1e-7 on the inputs. On weights and biases, the relative error is less than 1e-7 when the magnitude of the gradient is greater than about 1e-4. When the magnitude is less than 1e-4, the maximum relative error is 1e-5. Though the latter exceeds the set threshold of 1e-7, this error is deemed to be negligible for practical purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "N4tu6dgEUdMU"
   },
   "outputs": [],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import unittest\n",
    "\n",
    "def sigmoid(x):\n",
    "    \"\"\"Return sigmoid of x.\n",
    "    \n",
    "    Arguments:\n",
    "    x -- numpy array\n",
    "    \"\"\"\n",
    "    return 1/(1+np.exp(-x))\n",
    "\n",
    "def softmax(x):\n",
    "    \"\"\"Return softmax of x. The subtraction of the max of the x array in the first line of code may \n",
    "     help avoid numerical overflow.\n",
    "    \n",
    "    Arguments:\n",
    "    x -- numpy array of shape (output size, batch size)    \n",
    "    \"\"\"\n",
    "    e_x = np.exp(x - np.max(x))\n",
    "    return e_x / e_x.sum(axis=0)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 1026
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 15433,
     "status": "ok",
     "timestamp": 1529264145724,
     "user": {
      "displayName": "Thomas Mathai",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "102416209494325007955"
     },
     "user_tz": 420
    },
    "id": "hrjTvd5rUdMb",
    "outputId": "21124030-1c9b-41a9-f2f9-69ba3b8c54c2"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEaCAYAAAA7YdFPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsvXuYFNW18P1bPUPPBbwBAiMQBgP6BhIvMd4SUVDkkiAGg8ZoJCZRI345Ceb1NTHnHJMXj9HXN/nMjWC8Bk4CmBBN0KPxkkCinohG8Yb5VIKIOAPIgJHrDDOzvj9213R1dVV39UzPTPewfs9Tz0zt3lW1anU/tWuttdfaoqoYhmEYRmdJ9LYAhmEYRnljA4lhGIbRJWwgMQzDMLqEDSSGYRhGl7CBxDAMw+gSNpAYhmEYXcIGEqOkEZENIjI59f+3ReTOEpBJRWRMkc/5XRH5ZTHP2ROIyMMi8oXelsPoXWwgMTqNiFwoIqtFZLeIbE39f5WISHdcT1W/p6qXdfU8IlKfGgwqiyFXKSMi9wQHvtT9PyQiO0Rks4j81K8LETlORJ4TkT2pv8dFnV9Vp6vqoh64j7IcaA8UbCAxOoWI/E/gR8D/BYYBQ4ErgU8AyYhjKnpMQAMROQ34YMhHPwO2AnXAccAZwFWpY5LA74FfAocBi4Dfp9oNIxQbSIyCEZFDgPnAVaq6XFV3qmONql6sqs2pfr8QkYWpt9/dwCQR+ZSIrBGR90XkbRH5buDcl4jIWyLSJCL/Gvgs461URE4Rkf8WkfdE5EURmej7bJWI3CAiT4nIThF5VEQGpz7+S+rveyKyS0RODbnHipQr7R+p458TkZG+LpNF5I3UW/0CvxUmIl8Skb+nPntEREb5PhsvIo+JyHYR2SIi3w65dj8RWSoiv+3sAzxlYfwE+GrIx6OBX6vqPlXdDPwBGJ/6bCJQCfxQVZtV9ceAAGdGXGeViFyW+v9SEXlSRL6fuvc3RWR6oO9NIvKMiPxTRH4vIgNTn00UkU2Bc28QkckiMg34NvDZ1Pf1ou9661Pfz5sicnFndGV0HRtIjM5wKlCFe3PNx0XAjcBBwJPAbmAOcCjwKWCuiHwaQETGAQuBS4AjgEHAiLCTishw4L+A/wAGAtcAvxWRwwPX/iIwBGclXZNqPz3191BVHaCqfw25xDeAzwGfBA4GvgTs8X0+AzgROBa4AJiakuvTuIfeecDhwBPA0tRnBwGP4x7cRwBjgD8G7qsG+B3QDFygqi1h9x+Dq4G/qOpLIZ/9CLhQRGpTepyekgncgPKSZtZOeon0QJOPk4HXgMHALcBdAVfnHJwujwBagR/nO6Gq/gH4HnBv6vs6VkT6p46drqoHAR8HXogpo1FkbCAxOsNgYJuqtnoNPstgr4ic7uv7e1V9SlXbU2/Aq1T15dT+S7iH7BmpvrOBB1X1Lymr5t+B9ggZPg88pKoPpc71GPA33IPf4x5VfV1V9wK/xrlx4nIZ8G+q+lrK2npRVZt8n9+squ+p6kZgpe/cXwFuUtW/p/TzPeC4lFUyA9isqj9I6WKnqq72nfNg3AP9H8AXVbWtAHk7SFlOXwGuj+jyZ9zA8D6wCae336U+GwD8M9D/n7gXgTi8pap3pGRfhHOfDfV9/p+q+oqq7sZ9vxd0weXZDnxYRGpUtVFV13byPEYXsYHE6AxNwGB/gFZVP66qh6Y+8/+u3vYfKCIni8hKEXlXRP6Ji6t4Lqcj/P1TDxv/w9vPKOD81OD1noi8B5yGe3B5bPb9vwf3kIzLSNwDPYqoc48CfuSTaTvONTQ8xjlPAY7BDVKh1VRF5AMp984uEdkVcZ4fAvNVNTggICIJ4BHgPqA/TveHAf8n1WUXbkDzczCwM4fcfjr0oqqeBefXu//38BbQj/T3H5vUb+OzuN9Po4j8l4j8j0LPYxQHG0iMzvBXnOvl3Bh9gw/EJcAKYKSqHgLchnvQAjTiHrYAiEgtzr0Vxtu4t9tDfVt/Vb25EzJFnT8sUB3nuK8E5KpR1f+Occ5HgZuAP4rI0LAOqrox5d4ZoKpRA+NZwP8VNyPLe7D/VUQuwrkBRwI/TcVAmoB7SFtya4FjAu6oY1LtxcAfZ/oAsB/YhnN51nofpKwUv5sy6ztT1UdU9Wzcy8P/B9xRJBmNArGBxCgYVX0P+N/Az0RktogMEJGEuGmi/fMcfhCwXVX3ichJuDiGx3Jghoiclgoyzyf6N/pL4BwRmZoKjFenArahMZUA7+LcIkfm6HMncIOIjBXHMSISNaj5uQ24TkTGg5uYICLnpz57EBgmIvNEpEpEDhKRk/0Hq+otuMH2j77JAYVyFC52cxxpl9s5wP2qug14ExebqhSRQ4EvAC+m+q0C2oCvpWT0gvV/6qQsQT4vIuNSLwnzgeUpN9jrQLW4yRj9gH/DxeE8tgD1KYsKERkqIjNTsZJmnCXVKVeg0XVsIDE6ReqB9w3gWtxU0i3Az4FvAv+d49CrgPkishPnw/+175xrgf8H9yBtBHbgfPhh138bZxF9GzcwvA38L2L8plMulxuBp1IuqFNCuv2/KdkexcUS7gJqYpz7fpybaJmIvA+8ggtmo6o7gbNxD/XNwBvApJBz3ICLWTzuzWoqBFXdqqqbvS3VvC0VKwI3EWAaTm/rcEHvq1PHtgCfxgXF38MFxj/dhaB/kP8EfoG7/2rga6nr/hP327gTeAdnofi/+9+k/jaJyPO47/l/Ag0492HHFGaj5xFb2MowjJ5ARFYBv1TVXq9OYBQXs0gMwzCMLmEDiWEYhtElzLVlGIZhdAmzSAzDMIwuYQOJYRiG0SX6dBltETkHOKe2tvbyESNG0N7eTiLR+bGz0OPj9M/VJ+qzsPY4bV29/0IwXfecrotxvUKON133TV2Htb/++uvbVPXwrI5BVLXPbyeccIKqqq5cuVK7QqHHx+mfq0/UZ2Htcdq6ev+FYLrOL08x6Ul9m667dr1S1XVYO/A3jfGM7dPBds8iGT58+OW//OUv2bVrFwMGFFJuKZNCj4/TP1efqM/C2uO0dfX+C8F03XO6Lsb1CjnedN03dR3WPmnSpOdU9WP55LQYiWEYhtE14pgt5b6Zayu+PMXCdJ1fnmLSV90tpuv8fc211c2Ya8tcW3H7lLOui3G9UnW3mK7Lw7XV69ZCT2x91SJpaFA9/XTVxsbSe3Pra7outM3ekqP7mK7jH18uFonFSMqYG26AJ5+E+fN7WxLDMA5kzLVVAKXibpkyZQL792evTppMtvHII09EHm+uLXO3FON403Xf1HVYu7m2+rBra/nyp/Sii1Rra1XB/Z08ebM2NuY+3lxb0X3M3RL/eNN1165XqroOa8eC7X3XItm1axd33HE8DzxwBP36tbN/f4Jp097i2ms35DzeLBJ7Sy7G8abrvqnrsHazSPqwRbJy5UqdNUv1qqtUX3jB/Z0wYWve480iie5jb8nxjzddd+16parrsHZiWiR9utZWX+a++9L/L1gAq1atBSb2ljiGYRzA2KwtwzAMo0tYjKQAzG8fH9O1+e1N110/3mIkJbT1tRjJ8uVPdSQi5uprMZL4fcxvH/9403XXrlequg5rxxIS+y6LF4+yRETDMEoGG0jKiJoaEIEVK4bT3g4LF7r9mprelswwjAMZG0jKiPXr4aKLoKqqDYDaWrj4YnjzzV4WzDCMAxoLthdAKQSAb711bEYi4jnnNHD11W+UXFCyL+g6qr3UdF2M65VqANh0bcH2ktn6UrB91izVc8/d1JGIOGtWdF8LtsfvYwHg+Mebrrt2vVLVdVg7lpDYN7nvPli16g2OPXY4Cxb0tjSGYRgWIzEMwzC6iA0khmEYRpcoS9eWiHwA+CmwDXhdVW/uZZEMwzAOWHrcIhGRu0Vkq4i8EmifJiKvicg6EflWntMcBfyXqn4JGNdtwpYpjY1wxhmwfXuyt0UxDOMAoDdcW78ApvkbRKQCWABMxw0MnxORcSLyERF5MLANAdYAF4rIn4CVPSx/yeMtwbto0ajIPt5gs3lzDwpmGEafpMddW6r6FxGpDzSfBKxT1fUAIrIMOFdVbwJmBM8hItcA30mdazlwT/dKXR7U1MC+fen9FSuGIwLV1bB3b2Zf/3rvP/tZz8ppGEbfolcSElMDyYOq+uHU/mxgmqpeltq/BDhZVb8acfyHge/iYiS7VPWakD5XAFcADB069IRly5Z1KpGoqSnJ/Pnj+M53XiWZ3F7SSXLNzQNZuPCDPPnkYJqbK6iqamXChCbmzv0HAwe2sGvXLj7zmem0tORf772rWEKiJcmZrrt+fLkkJJbKrC0JaYsc4VT1FVWdrapXhg0iqT63A/8beL6ysvOG1+LFo3j55UNyuolKhUGDWujfv5WWlgTJZBstLRXU1rYycGBLR58lS1Zz1llbOsqsVFW1MXnyFpYuXd1bYhuGUeaUykCyCRjp2x8BNPSSLABMnTqBSZMmsmLFcFSFFSuGc845M5g6dUJvipWXHTuSzJzZwIIFzzN9+lvs2JEZcM8ebBJZg00UTU1Jvv714woK4kcd05lzGYZRosRJfy/2BtQDr/j2K4H1wGggCbwIjC/W9TpTIqWhQfWii1Rra1XB/Z08eXPGGiD5KNVSEsH13r0yK/mYO1c1kXB/8+FdK+qYfOfKpZeGBo21Hksh57SyHfGPN1137XqlquuwdmKWSOnxGImILMUtLj4Y2IILmt8lIp8EfghUAHer6o1FuFaXijYGCyROm/YW1167IfbxfcWXPHXqhILjKlOmTGD//uxjnMcy25MZPFcuWb3vxStYma9/nHOWiq47S1/125uuyyNG0usFFXti62zRxuCb+4QJWws6vq+8uYVZZxdfrDmts+XLnwo95oUX4p0rTNbqandMcKuu7ju67ix99S3ZdJ2/7wFpkfQkfbGMfFR7d7+5RZWvj2LXrl3cccfxocfEOVeYrE1NycCstDYmTNjG3Ln/iDWjrlx03Rn66luy6doskpLZ+lIZ+aj2fG0NDarHHLOjoBiPn0LjKitXrow8Js65grKffrrrX1enKuKsEH+MpZR0HVeeYtJX35JN1/n7mkXSzZhFkm4Liyt0J8XUtSf7qFG72bChP/X1u/nXf/07Dz54BNu3J5k/f21J6TrftbqDvvqWbLo2i6RktgPZIskVV+hOiqHrKNn99+BZK7/97VNdksnekuMfXwq/60LlKSZ9Vddh7cS0SEolj8ToJrx13mtr3X45rfPuyV5Tk9leU5O+hzh1xQzD6F7MtVUA5era8lxDlZXttLbmD5QXg2Lp2pNdBNrbIZFQVCW1n38acVyZ4uq6qSnJd75zNPPnv5aRxGnulvh9zLVVmroOazfXlrm2Otq84PYddzxTUAJiVyiWrj3ZzzxTdfx49/eqq1SnT8+cRlxV1Zp3SnIxdD13rqpIe1Yipblb4vcx11b848vFtVWWC1sZhXHffe7vqlW7ueyy3pWlUDzZw5g711U7rq6G5uYEBx8Mw4aF921shK9//TgeeSS6j9fvwgvh3nsz+2VWVhYWLoSFC8MrKxvGgYbFSIyyZcsWuPJKePppmDmzIWNtleB6KzfcAC+/fAjz5+c+p7+8vp9yjjUZRndjMZICKNcYSSHyFIvu0LW/pH+wyGSwfzq2orS3Z78v+WMpTU1Jzj//VFSzYy79+rXx6KNPZJwzLNZkfvv4fcr5d12M65WqrsPaLUZiMZJe9SWvXLkytLhiIccHyVXo0eufb8pwWEkWL+4xZkx26Rb/tOJcsSbz28fvU86/62Jcr1R1HdaOxUiM3qZYqzAGV37MFZ9Yvx6uuQZ+9zvYs8e5oIYPh3XrnHWxb19FRywlGPdYty59nn374OCDybB8yjnWZBjdicVIjKJTXQ2TJk1k4UI3ZXfhQhDJzgeJSyHxibo6NwB4Qfh9+6C1FebMgVGjdjNnTjpuEjxvRQWMHQuPP+5iL51Zz96Lzdg6K8aBhA0kRtH57GcBlIpUFfmuBqbDBodcM7T8Qfgrr4TjjnMyrFt3EDU1mTPBVq50Vk0y2YYqTJ4MZ50FCxbknjEWRWcTJIOTAwyjnLBgewFYsD03UeuWgDJzZmFJkEFZr79+PAMHtjBjRkNGja2o/vlkSibbmDZtMytWHEF9/W6+8Y3n+OMfx8Q6r7/NmwDw6qsH0dqafZ1+/dr40Id2hk4Q8NPVWmh9NQBcCr/rIH1V12HtFmy3YHuPByWD65ZUVqp+8pOql15aeBJkV3TtD/KHraWSSIQH4sPqj+XSa0NDuhrxlCkNoQmSX/hC7pUgi1ULra8GgEvhd53v+t15fLkE2821ZRQNvwsqmWyjvR1GjYJ77umcm6iz+IP8QZn27YPPfz475jJ58paCXG81NXDEEc4lpQqPPlrHkiUuwO8lSP7qV7BoUe44keWnGH0BG0iMouLFJxYseL7TAevOUlPjHtbBIP/tt2fKtHNndsyltrY1b8a7F8OYOnVCxiwyP5de6mIzU6duZsQIOuJEFRXhA0Sh8R/DKEVsIDkA6MmZRPfd5wLVY8bs7nTAurNEvd2/8062TMGA/I4duXXjt3KWLFnNRRelBwmH8oUvOOvr2GNh5cohbNoEbW3u07Y2+NWvYPTozPM2NsJvfuNmlXmydGbwbWpKWrDe6DXKciARkXEi8msRWSgis3tbnlLnQCm1HvftvrERmprg3//dPfQXLCAjwO7vd+aZZ2RZObNnf5x773WDQ0WFs3rq63fz/vvpY5csWc2IEVCZytSqrIQRI7ItkhtugB07nDXlyRJn8A3O8lq8eFRoaRfD6Al6fCARkbtFZKuIvBJonyYir4nIOhH5Vp7TTAd+oqpzgTndJmyZM3XqhIyH4IoVw7uUz1GKBK2toKUR9oYeVU8rrB/AmDHZ8ZQpU+Cqq+C551zxyJEj92YMAIMGtTBjhtN7dbX7e8456UEtyg0X97vx7mHECHfcihXDi5KzYxidoTcskl8A0/wNIlIBLMANEOOAz6Wsjo+IyIOBbQjwn8CFIvJ/gUE9LH/Z4LlgvIdgVVVbnwvkBq0tz7Xmvd0vWJAZ24jz8PY/5FVdxvuePe4zL57y0EOZ1wmzaHINap0NsgcHIM91Bhp6HstPMXqCXskjEZF64EFV/XBq/1Tgu6o6NbV/HYCq3pTnPBXAfap6bshnVwBXAAwdOvSEZcuWHZB5JHfccTwPPHAE/fq1s39/zyxqFSVPMY/PlR/iX9jKn5/xmc+8yuLFx/Dkk4Npbq6gqqqNCRO2MXfuP0gmtzNgwACampIdOSvPPjuQ5uYKEgnliCP2Mm/eazzxxBC2bk3wve+9llPeOPfvyVbId9PUlGThwg9m3MPgwc28804NyWT2eXLlp/iLYHr3H4dS+F1bHkm8Pn06jwSoB17x7c8G7vTtXwL8NM/xtwO/Ak7Ld70DOY/EKzT4wguq5567qUcWtYqSp5jHB/NDggtbReVnVFS4vI7q6sz8Du96XmHIcePc32SyNSsPpFi5Df7vJteCY8Hil1demXkPo0e779Z/njj5Kf4imKWa22B5JPn7lkIeSalYJOcDU1X1stT+JcBJqvovXbyOZbb38Jub95Z7zTXPMnJkv06fJ5es3jWGDdvLY48NC32jD3tzP+WURvbtq2bYsOasDPkpUyawf3+2hVNZ2canPrW5o19vLLV7001H8+ijw5gyZTPXXfdaaJb/ZZe9wA9+cGJHBn1TU5If/nAsTz01GFXpsL4uvHAjl1/+McJK5udaptjPgfi7zidTdx1fLhZJqVT/3QSM9O2PABp6SRajCyxePIqXXz6EpUvHcu21G7rlGrffPpqXXjqEzZurmDmzgRkzGrj//sPZsaN/R59Bg1ro37+VlpYEyWQbLS0Jamtb+e5303M85s1Lu3ruvPNPoW6vOXNeyhgQFy8exauvDmTRolHMmfOWzzVU/PsMuu8efbSORx+ty3jge/dwyy1jefnlQ1i0aBRXX/0Ggwa1sGlTDaqQSLR33P+KFUcAMHz4HrZtq8oYZL/2tY0Z11+3rj/z5h3Pj360hg9+cHfxb9DoO8QxW4q9ke3aqgTWA6OBJPAiML5Y1zuQXVuFytNZilXqwyNM1lzXCOsfdB1NmLA15/X8LiMRV/7EW48k1zonYa6hXLqOs05LQ4OTIex6VVX5dVLIlkiozpy5KUuG8ePd5+PHZ+sqH33ldx2GubZKwLUlIkuBicBgYAvwHVW9S0Q+CfwQqADuVtUbi3Atc231kAsg25XUyoQJTcyd+4+cxQqjCJN1ypTT2b8/e6Jhv37t3HffQ13W9S23nNzhMrrxxg+xYUN/pk9/i2uv3dBxfytXHh664iJkuoZyXStOgUbPnVVT08revZWAAMrw4Xv58Y9f6NBpWu+DaG6upKqqjZNOakJVOiYLuLbtrFs3gMbGakByTiCYNOmM1PWCKCtX/vmA+l2HYa6tEOKMNuW+mUUSX56ukPlG3x5ZqDAOYbI2NKiOGZP5Nj12rHuzL5auO/OGX1urOnny5gwLo1CLKl8fbxs9OlsndXWq0J4xgeDKK51FE2XVeBMPwuR99FFn9fj71tervvhiZt+gZeXf70u/63zX787jzSIpAcwi6dk3N38Q+P77D2fnzv6h+RVxiJL1c587mc2bq+nXT9m/X6ir28eSJauLpusoy+rCCzdy3XUfYfv2JG1tCUQUVWcNtbYmmDbtrYyYUNi1gucOszCirK7KynY+9anGrDL3t946lhUrjmD48F3U1ipHHrmb995L8ve/H8TBB+/nnXdqGTFiD/v2VdDUVEV7u5BMtjFkSDPDh+/h5ptfyZLXO6ef+vrd3HPP3zL6BicB+C2tyy9f02d+10HMIgkhzmhT7ptZJPHlKRbdpeuoKbOF6jr4Nu3/LMqy8tqTydaO2EFU/CVKnoqK3FZJLqvLT67pzVHX8K6TSKjOmZN9/0ErxL/V1aWv7d1/vi0qPlbOv+tiXM8skjLDLJLee3MrdV0H4xT+z6IsK6/9Yx/byA9/+FHGjNkZ+kYfJU9TU5IvfvFE+vffz/btVbS0uGTHM8/cmhFL8qyuykqltVUYMmQf9967OuM8118/ngED9vPMM4MIj2f4Uaqr2zjjjHeZPHkLN9/8ISoq2nn33WrOOaeBOXPe4jvfOZrBg1v585+HkEgo7e0JEol22tuFww5r4c47n/NZTeFTpUEBN9X4+OO38P77/bnhhrVZMbJy/l0X43pmkZTpZhZJfHmKRanqOupNPplsjTwmaL3MnLkpb5JimDzBZMdgUqSHZ3Wdf76GxkXmzHHthxziv4f2WBZJVzbPwli+/Kksq2nAABeP8e6pvn5n5IJe5fy7Lsb1+qJFUpbVfw2js0TVuFq6dHXkMfkKJCYSuUv0B+tjvfqq+9veHl5Y8uGH4Wc/c+XlwdXNEklvixe79n/+03+Us0ra2twmKSOlIsxw6CT79rl7GTSohdbWzM/27HHFK7372rBhQEFFJBsb4etfP85qgpUp5toqAHNtxaeUdR1W4+ryy9fQ3DywI8Fw4MCWnGvQey6cwYObaWioyRlsD8uy92p8eVno/ut6/Z94YjAtLRUkk22cfvo2Vq06nNbWsHc/JZFoRzVBTU0bhxyyn/nzX+mYwpxMuoTE4cP38s47NUS7wrTj3urq9tLY6D39hYqKdiZNepe5c//BhReeHOHaAhHlE5/YxrPPHkZzcyXJZBu1tW18//svdiQ1hn1XfldjZqJndg0wc23F72OuLXNtlbULoJR1HRawX7lyZUbtKVUX+D7rrM0Z672PHZt2I+Vy/wTlCdbH8rt8gtf1+vvPO3du2qUV1w0VvM/6+uzpwOl9d09HHpm+1/Hj3eeeq6yuzrn3li9/Kue1x41zkxT8bsQoN2CuAH9UDTBzbcXvY64tw+gmgqXmH34YJk2amFVe/sgjoX//1oyFslpb4dxzG3jsMRg7Nu06yleiP6ykfNSaJCJw222Zxy9cmHZp5WPIEFi9Ovs+jz/euZ9m+5aCU4WjjoJ+/ZxnYv16175nD6xd69x2f/iDcwE2Njr33uzZH895/VdfBVXJWI7Yu7fq6nRbYyMcdFD0eRYudN+Lra1S+phrqwDMtRWfctJ1U1OSn/zkAzz9dF2W6+n73x/NkCHtGQUSr712daSLzMtUjyNPlMvrs5/dyLJlH8jIN6mpaWPv3gpyu6XSzJwZXTa+pqaVLVuq2bDB1Sarq9vHzTc/yeLFxxTkTouSJZls44QTtlJRUdmRXe/JN3XqZr761ef4zGemR7oNKyqUtjZXI62mZj8/+MHLGbW+zLUVv4+5tsy1Za6tIvfP1cebiRVVXj7sPLlK9C9f/lTeelqq0S6vODOvPLeU+xvubquqSs84mzs39/mC14yWIdq1520zZ27Kcs8VsvldbnPnZub9BHUbNrOuOzHXlrm2DCOUHTuSeZfoDeJ3Hc2b90bGUrv51lBvbIRTT4V774U5c7KvO2UKjB6dnn0FzpU2Y4b7P5Fwj9nx42HNGvemP2JE2tVWUeFmo82cCX/5i1vPfuHCcFncuvPqW23REdxPky9vxc1sC7rnOo4WZfr0sE+0w42VdpQICxfCEUe4+/jmN+ErXzmBJ57I1G1w+WRbGbJnKZUy8obRq8yfv5aJEycCbnDoLDU1pGIDwwH38F640MUG9u5N97vhBjd4eMd4cQyPhx5yg4Sqe9CrwuTJ7sF41VVwxRVw++3ugXnssbBy5RBafHl/bW3wq1/Fk7mtDaZM2czgwXX87ncuPlJdDS0tLnbj56CDYPdupb09ejBJJKC9XTnoIGHnzuzPVYWHHw47UjJ0FIaLE1UBad26GrBktHk6mz/fTaU2uhezSAyjiHh5KlVV7nU+uIa6P8DuEcy18Pq8+qrbb2tzD/Sf/zw7gO5ZQUuWrO7Ic4lLIgFVVe5tf+/eSg4+mI4A+b592YOICAwa5B7QiRxPDndc+CDizpM+cU1NOqenMyQScNpp72adw9NZ3DwWo2vYQGIYMfHcJbmSD+vq4OCDoaUl0THT6+CDYdgw9/n69c5l5aeiAs47Lz3YRCVNvvNOtGwXXXQymzb5XULh+F1j7e3Q3Aznnusssi1bogciEZg4EXaXLSDHAAAgAElEQVTudNbL88/D+edDv04sgqmafuzs3essoChEtGPQCpPtkktg4MCWjpl1Ik7fHsGBPBfmDus8NmurAGzWVnz6oq69WVrB5MNgv+uvH89BB+1m1qx3M5bzzZXgGJxl5Z8R1tKSYNCgFm6//bnItV02btzPNddM6KhOnJ7FJR3/Dxmyj6OP3sVTTw0OdU0lk20sWbKaH/5wLE8+OTjj2MMOa+HEE7fz2GPDmDbtLb785QZmzz6VqHVLwinAXAo5lz9JUgRGjdrNyJF72bOnnQ0bDuWf/+wXOsNMRPnNb/6akfQZRpx1YsBmbYUSJyJf7pvN2oovT7HoS7qOs45IHF03NKhedFHmOT7wAdWRIzVjxpdq5oywo45yfb/whdz35s0AyzUbqrExLYc3K6uiwlUvPuUU9/m4cZ2fbeVtcSsEFzobDFT79WvLqPw8c+YmFVEdMsS1+e8LnH7Dkj49Cl3d02Zt2awtwyiYoKsJlLFj47lL/Bx5JCxZktm2cSNs2pQdEL7vPrj7bjjuOHj9dde2aFFuf/+WLc7V85GPpOMsHpWVbn/+/LQc3qystjZ44onDefpp55rzYjOdYfhwuPRSOPnk7TQ0BPWWj9zWjecCXLbs6Y5EUq/2mSps3Qr33595XwBvv52d9OnXYZQrsdDv90DGBhLDyENdnZumm/blC2+84doLCeKuX+8ywytTcyUrK6G/ywkMnSasER6iqPb77nMPwbVrXezB36+11e0vXJgOqFd2Ys6mG5yi3eGJBNxzD3z9629w4YWwbFnuGEgmGhIHSTe0tbkpvv/2b+PZvNnpMyqmk0iE3583SDz9dDoe4sW1/BUM/HEtIz82kBhGDKZMcXkcXokPL0+jkLfWujoX7PZmQ7W2wu7d6Qd88E35zTdhzJjMc4wdCxs2uP/9weGpUydklFuJoqbGyT1nTrBKcHvHfeVi9mwYMGB/5Odvv+2u4eXR5JLFT1UVnHHG1o4SMX4qKuADH3CD7tat8Pe/H8xHP+omLUQNqp//PFkVisENakuXwvTpZOSihJWwMeJT8gOJiBwpIneJyHJfW38RWSQid4jIxb0pn3Fg8NBDcNZZLrcimWxDtXNvrd4D65xzMtvD3Cl1demHYTI1Uay1NX1NfxLekiWrY7mR9u51cu/cCePGubbDDoO2NkEkVxKi4ze/gV27ometgXuj90rtx6W5Gf7856G0t2cPDm1tzgW4u6NKitDY6I6JYvFiNwD5Ex9d4qUb3BobMwfwhx8On1YN+Wdz2Wyvbh5IRORuEdkqIq8E2qeJyGsisk5EvpXrHKq6XlW/HGg+D1iuqpcDM4sstmGE4g0CCxY83+m3Vm+tkQceyGzfsyd8YDr+eJeA+Mwz7u9xx4UXe5w9++Pcey8Z02D79898ux861MUvfv5zF0tYu9Y9THfsAJDIt3twLrkXXsjOkYnOW9GOPtlWTuEzRQtZV8WzFisryUh8bGsLt2ASidyWZTBrvtDPDwS6O7P9F8BPgY66pSJSASwAzgY2Ac+KyAqgArgpcPyXVHVryHlHAC+n/s/zDmUYxcF7S121ajeXXda5c6xfD9dcQ0cGeWWlc5sNGRI+MPnfjL3M98bGzHPU1sLHP76Ffv2GMnq0y3o/4QT/G7xjyxbn1jnpJGftPPJI/viFl2/S1uYGomCOzIgR8P77mYtsHXywa/P6VFc719V776Uf5N5545LPUvLjWYtvvunudfNmZ8lVVDjLbt++zAHlkkvCLct0lQKHlzWfTE6guTn682AVgwOBbs8jEZF64EFV/XBq/1Tgu6o6NbV/HYCqBgeR4HmWq+rs1P+XADtU9UERWaaqF4b0vwK4AmDo0KEnLFu2rE/mNhTSZnkkpaHrXFWD43LrrWNZseIIRNxDcfr0zNyWpqYkc+d+tCOvpKKinYEDW/joR3fw6KPDGDVqN2+91Z/KSieDt0570Fo444ytvPVWfzZs6M/MmQ3s2JHMypF5440BbN5cTb9+yv79gohy5pnvcOGFmzv6gEsc3LixhjVrDiNeTkkwF0Z87blKtCgnnridvXsrGDZsL48+OqxDT/X1uzuqHoOL9xx//D+ZP39t1nmiFhibM+clRo7sF1q9+aSTttPUlAxdq97D8kg6sQH1wCu+/dnAnb79S4Cf5jh+EHAb8A/gulRbf+AeYCFwcT4ZLI8kvjzFwnQdfa2whbUKZdas9MJT48ernnzytqzqt/7KwtH5HtmLXcXJEYm6n/HjXZ+ZMzeFyt3Q4Ba9KnZuibddfLHTgbcI2MiRaT3lOi4qZyRsgTH/dxus3jx+fHS+ikdfzCPpDYvkfGCqql6W2r8EOElV/6Ubrm2Z7WaRxOpTTrqOmyF//fXjGTiwhRkzGli+fATPP38Y77/fL2vNlVtuOZq3365i27YaWloqSCSUM8/cyoUXbmTp0g90vJGDMnz4Xm666SlGjuwXSyYRZfnyv3a8nXvWWG1tK7t3V+LPnvcdRT6rI3jfHs5llu+49LlFlNNOe5d589ZlWRBR99SvXxuPPvoEkNbxQw/VsX9/dsg5mWzjkUeeyGjrixZJb8za2gSM9O2PABp6QQ7DKEuWLFnNWWdt6Qh6pxFWrBjOpEkTmTp1AvPnr2XevDcYM2Y33/rWa5x6ahMtLW7BqJaWBLW1rQwc2MLNN7/MMcdsY/9+91l7O6xZcyiHHbafP/1piO9hKrzzTi1z5pzN1KkTOq7a1JRkzJhdnHbaux0yVVW1Mny4C8AsWjSKqVMnMGnSxFTyoLB7dz/SA4UENjcwJJNtJJNu2loiob5ij2EDj3TMyIoimWxj2DAvKKSAogqHHba/YxBpakry9a8fx/btySw9u8H3XY488p8d7jpPx0uXPp3Vd/LkLSxdujpaoL5EHLOlKxvZrq1KYD0wGkgCLwLju1MGc23Fl6dYHAi69i+m1NO69lwq3prnFRVtCm6tdc+9EySXS23ChK1Z7qnocivtGef3yo+MG5e7REsi4eTzypeMHq165pmqlZXe5+06dqzq44+nZYD2ENecc3f5F7+qrnYlZERUDz64MFed37UVLKUS1LMrV9PesX592HcSXKQsiLm2CkREluIWCxgMbAG+o6p3icgngR/iZmrdrao3dtP1zbVlrq1YfTqja3+Rv8svX9Ojuva7rW688UNs2NCfZLLzwftdu3blWP42iij3kzJ06B7efbeW9nbpcKMlEu089tgwKivbaW11cgIZEw8AVOO7tJJJV9QyurxKfjfX8OF72bq1OtQ1JeLchStWHBEql9915f9O/MU6g+T6baxb1595847nRz9awwc/uLtsXFvdbpGUwmYWSXx5ikVf1nVUQcJcRRy7k1mz3FK/XQner0wtVxssKplre/xx19+zMmprVceOdVZBff3OrLdzzyK6445nOuQMLlc8cmTaYvFbHxUVqjU1rq2qqlVHjFC94ALVj35UdeLEzTpiRKa1M3as6r33ur/+Io65ljD2W0yeZedZIp0J1OfSdRSeJTZ+fP6+cc7XJyyS3sYsErNI4vYpRNcbN+5n8eJjMqZ9nnJKI1/72sYOX3t36LqpKRlZCt27Xq4+ufBbWW5asabewP3B8Ow38kRCOx6pYZ/369fOJz/ZmPF2Hqab6AkEQVw9Lr81M23aW/Tr1y90SvVNNx3No48Oy7B4oqyXKVM289hjw6iocBbTGWdsZcaMRr71rWNIJJT9+ysydFFR0c6kSe8yd+4/QnUd9V0E77+pKZmzJP/KlX/OqRGzSMwiMYukSMf3tK6D/vDgdNfu0HWuUuje9XL1yYVnkQwapHrppaqzZ2sgDhG9VVSoPvZY5tt/VVVrZKwmTDdr1qgeemhzh9WRewufElxRkY7/TJ+uesopbnozqJ5/vovJjBzpPst1L0ce6f7v3z9toQSv78WC5s7NjJV5NDSo1tU5/QW/i+D9z50bLkt9veqddz6T0TfsWiVvkaSy0G9W1f+Vd0QqQcwiMYskbp9CdX3LLSdn+MO3bk3wve+91mn5cxH1tu73z0+ZMiH1xhzdJxe7du3ijjuO74j57NiRpKKinaefHkT//vtJJt1CUk8/PSh1RPYbedhiXP/n/7zEj388NuOtfOPG/fzgBydmtHmWkJPZi3vQca0giYQCLokykWjnzDMzLYOzzjo9lWAZJDquE3WtKBIJ5ROf2MZhh7VkLIiV7/vyfhu5rTClvn43P/nJqozfUdjiW2VhkQB/IpUBX66bWSTx5SkWpuv88sTFi10E/ff+t9Lly5/K2yeKfHEAz1+fToJs77A8/G/bXrzjscdUk8n9HbOcghbSzJmbOtpyJUuKqFZV7c9oq6x0s7uiYhW5zgcuplJXl54p5r/W7Nmqo0blPr6zWyKR/i6830ZDQ+ZCXN520klOb3V16b65Ft8qBYskTh7JGuD3InKJiJznbTGOMwyjCMRZL2PQoJZOr6nx2c8CaGRhxLVrXXHG+++Ho46CmTMbeO45V0TSXx/MW4zr7LOhpcWV8Xv99cwFpbyFqLy2fftcEmFwUanGRnfcSSftYPx4d1xFhauZNXny5ox1XUDp39/V1lq/HmbNCi/yeMghrvCkq3acbj//fFdWf+VKVxOsEGbNyqy4PHo0HWvM+2lvd5/5qatz9cuCdcROOAFeew0afNl1pb74Vt5gu4jcE9Ksqvql7hGpeJhry1xbcfuUuq69qaWnnfYuN9/8IcaM2cnNN6eLaoe526Kmn3rkypBP44Ltw4bt4z/+45WcU1Kj3GveOSdMeBdV4dlnD6O5uTLntGC/2ybMjbh69bDIdeenTdvc4SbL7apy7qORI/d2uKc0YtJA4a6vdH8RZfDgZm677XkGDmzJ0N/1149n3boBHH30+wC89trBjBmzK3RiQlSNtpJ3beHyPK6OY9qU8maurfjyFAvTdX55OkNUML0z1wu6zOK4t3Jdy6tvFeWmmjPHuWu8BMJc04L91woGmFeuXKnTp7vgvufyqahIu/JmzSq8fliubfRo1REj/G3pKcl1daoHHZTpwvInZYa5AHN9V2H36hGVUFryri1VbcPW+zCMXidsDZJc67fHwe8ySybbEHErMIaxdm30tTzZFi8O//yII2DuXLciYWOjq8DrX4nwvvtcifzDDtvPK69kr19/ww3u2I9+NO1Ky7XQ2H33wTvvhLuYwmSvr0+vfFlTk62D2lo48UQYPz7dlkg4l1Rjo1skzKO9PbNMS1ubc7PFXbsm19omnp7CFt/qbeK4tm4EDgHuBTpWOFDV57tXtK5jri1zbcXtU+q6DitZ7hVdDLpLCsFzmZ111jr++McxrF49kM2bqzPySPJdKygbKIcfvpcxY3azYYMrMZ8vKxzgllvqefjhUQwa1MLttz/H5z53cs6iiUHZ/a68+HkppO41oyXWcYmEMmhQM01NVZGFIhMJ5Te/+WtWHklz88CMHJM4BSKjKHnXVmqQWRmy/SmOuVMqm7m24stTLEzX+eUplFy1nIqlb8994uWReK6afNfKlVfjzU7yXD7BHJOoGUlVVblnaOWSp6Eh6I4KP0dcF1htbWZmvSd/sBZX1H0E3VVBF2VQR96su9/+9qnY310hn/X4rC1VnRSynZl3hDIMo6h4S/363ULFxnOftLW5WVnPP++utWFD7nXJg7Lt2JFe172uzs1Cam93LqHm5gSVlekZZevXh7uhwtZkr6yEyZO35J2tVFcHM2Zkt3tLA1dVObfYJZe42VD52LMnvephS0uiw422ZYs7x/jxcOSR2dcaMQIuvDDtrqqpgUmTJma5KI88Mq2jior0rLtCqhP0JnkHEhEZKiJ3icjDqf1xIhJcQ90wjG6mJ33kwWvV1+delzzY33MxefGTV191/Vz8QFi0KH1sXZ2bRkugPPx558Gxx27vmPLqTf+trW3NOa25sRFOOQXuvdetUT97tmtPJJyNMH48rF7tBrydO90DWyR7MPPiJh5jx8Ljj7vpz96Aet99LoayZo2Lyxx1VHqqsips2gSLFpEx3Rm0I95UW+uuu29fWkdtba7/z38efY+lRpw8kl8AjwDefLrXgXndJZBhGPFpbHSWgrc+RmdpakqGWhxhQf5JkybGDvJ7+Q9hiLiH9RlnwDPPZH/+2mswcuQ+9uxxD38vd8Vv7YTJfsMNbqDYscPJ77eurrrKPez9g/GWLW4ywHmp7LhEwm1HHun+Vle7v5MnuwD/vHlvcN99mbpRdRbU66+7vs89B1/4grNI/AOhx9696Xyfz38+O0fkvPPguOPgqquO7xbLs9jECbY/q6onisgaVT0+1faCqh7XIxJ2AQu2W7A9bp9y1bWXWzBtWuaa7YVyyy31/OEPo7JK0IcF+YMFKsMI5j8EC0F6Afw//nFIaCA+imSyjd/+9uEM3Xqyg6Ia/m4cp1RMsAz8k08O5rTTtmXl5fgLZF5wwamR+SyPPPKEr/RL+D26FRpdmRWnIzco+deW96966eEvCJlMbi+LYPsq3Lrpz6f2TwH+HCcAUyqbBdvjy1MsTNf55ekKuUpmFPs8+QpUhuG//+nTXb7FjBmayinJvQ67v+y7F2z3l3zxzp2vHIq3HXaY6osvOlmiCixGtb3wQniRxIYG1ZNPVh08OPNalZVpOb0imBdc4BbwCgvsf+EL7pxe+Zl8kwM8/MH6sgi2A98AVgAfFJGngMVA0ddXNwwjPsGSGVVVbZ0qmeGdx1siNqz0Rq5Auh/PzRZ0xdTXu3OsXetcS7ff/rcstw/QsZRuW5tz+3ixgmQyvORL2m2W26uyYwfcdpv7PyxPI1fbxRe7v9/8JqHus23bXMkVfwkXT84bbnDXHjQI/vxnNxwEWbQoXX5mbUQRgooK5+p6882uuxq7i8p8HVT1eRE5AzgaZ5+9pqr7u10ywzAiCdbfam5OxK6tFXaelpZEZI0uf1B/wQJYtWotbuHTTLyH7ze/CS+8cByvvZY58+rNN12yYTL5UZqbXVzi9ttdTKG9HU4//V02bhwKuAfriSfC/v1uwJk2zQ1UQdm9uIPnEopi4UK3Re1HtXkPdy/Zsq6OrHvfscP9raiAr3zFDVrBa3lUVGTW1qqpgenTXbA+/CVAaWsThg5138n69XDNNfC737mZZLW18PGPb+E//3No9M33AHEsElS1VVXXquorNogYRmngtxT8M4miLINc55k5s6HT04qDb8mLF8NLLx3K/v3hhQaXLl1NYyP85jeuWKIXBN+3r5KRI90D9bjj3CACwvr1bgC6//7saz/5pPubJ9RLRYUbkPyzpc47z80W89pqarLb4uBZDG+95QbaTZvC73vOHCenK4Hvjtu71w3UYYPIBRfAxIlbGT06/Z2EFfCsrW1FtbDvvNjEGkgMwyg9/FNuvZlEkLvMRtR55s17o9PTiqNmZrW3w5Il7s3Zb+0MHNjS4fapqUnPoBo6dB9PPumq8Y4alXmu+np48cX0vjd4uQdw/mB9W1v6ASziHuBDh7rNyw8Ja4tDW5uzED77Wfcgj6rWvHOnG6iPPdZVND7jjPTMrjB+/Wv47/8ezPr1md9JmKux0O+82OR1bfU2InIk8K/AIao6O6rNMA50amq8PAWH56qpri7sweinsdEl1N17b7TbzHtwemXi/bWm+vd3A8O8ec6N5dw+E7NkdAwHnKWSRgGhf3845ph0a9DFE8UFF7i/zz7rBq5x41y+xrhxLk/DL6snTyLhHvB/+IN7aOeiosLpd+DA9IP8Zz9LP+yvuMLdd2NjejBYteolJk5M62DuXCeLX3e1tc4yOu+81cDHM67pH1Tuvhv27TucJ55Iy9/V77wzRFokIvLRXFuck4vI3SKyVUReCbRPE5HXRGSdiHwr1zlUdb2qfjlfm2GUA4W6nQqhM2tW5MtDifum6+ViBNm9G37xC5cg6Ll9zjprS1bOxKxZmQH/6mqX73H99WsZPx62b888r/+tv1+/zAU9qqpc8uD06W4AvPded5+NjS7moer+epn2QX29846T2SVJZpNIuKe9NyFg9254++3MTPWHH86dPOr/HYTlscTNbF+/Pluf/u88Kj+o2ORybf0gx/b9mOf/BTDN35BavncBMB0YB3wulS3/ERF5MLANKehuDKPEKaYLIjgoxVkAK0qeRYsyfUmFVhv23GxTp7qHeDLpHu4VFZkPtro66N+/NUNGz53kD/h/8YsuIXHSpG288krmIk8e3lv/z372fEZl3uZmlzz40ENpHT39dPgg+/nPR+tryxYXuwmWPqmubueqq9w5jz6ajvv0nzc4eAe/K//vIKosTZyHf5g+/fewePGoHnF55U1I7PIFROqBB1X1w6n9U4HvqurU1P51AKp6U57zLA+6scLafJ9dAVwBMHTo0BOWLVtmSXKWkNhruo6z7rqHP9ks1xupf/3uyy9fw4ABA7KS6qIWt8onT65qw7kS4PxyeQswnX32ZjZvrum4n29/+2iGDGnPkBHgoIN2M2vWu5HJf1EJeLnWq582bXOHjoCshaF27Ejm1Ffuxb/CF8Dyqhf7vztPJyIaupZ8MtnGkiWrs773OL/rMH2uXj0w1rrxQTqbkJg3RiIitbhckg+o6hUiMhY4WlUfzHdsBMOBt337m4CTc1x/EHAjcLyIXKeqN4W1BY9T1dtFpBE4p7Ky8oROymoYRWHJktWRD+YgixeP4uWXD2HRolFZGc2Q/XBbsWI4K1YMzxqU5s3LPjZanlYmTGjqkGfQoBb692+lpSVBMtlGS0uC2trWVBn53Pe6Y0eS6dPf6hgUnnnGlab37ufb336u42Hll9F7iIXJ7enk5z8fzTvvHM38+a91PGzvvPNPLF58TIZu9+9P0NJSwYoVwzt0BG7G1IIFz4cOGmHXDfveTjmlkXPO2cZ//Mc49uypoKWlgoqKdj72se00NNTw9tu1Hfca/K7SWfxuIPL/DhYtyv29+/EPrGH6TL8IDMpYjTLs91YU8mUs4tYhuRZ4JbVfA7wQJ9sx1b/eOza1fz5wp2//EuAncc/Xmc0y2+PLUyxM19l9cpWBV42frR5c2bC2VnXy5M0Z2ddx8Msj0p4lT2dW5PNYuXJl5P0kk62RxwRJJlsjs9/9xwV1O3u26uGHp0u/+zPjCyUss3/u3MzvJ1cJ+eB3NXz4bhVJn8/L5A/73qN0HZbZHszQv/JK970WsuxAZzPb49Ta+puqfixQa+tFVT02zkBVLNdWZ7BaW+baitunJ3Sdz+2Ub/EqP0H3UWdqbfnluf/+w9m5s3/ONd5z3VtYn+bmgaH3M2fOS4wc2S/WeTdu3M/ixcfwxz8OIcyV5NXeCq7r/swzA2lsrE71aaelJRHqcoqDX09f+crHQmtreYtcvf9+P5qbKzpqaM2b9waLFo3K+K6GDNnDKae81yHr5s3VDBjQGsuNmMsl6XfjXX31G1x//fhQd2GUrsPai+baAlpEpIZUHQIR+SAQslJAbJ4FxorIaOAd4EIgxooAhlHe5HOj5HInBdmxI8nMmQ0dD6OtW6u6JM/cuVuK/pIRdT+HHdYMZA8kYVx22Zmh8Q9QJk/e2uGqybUyottXmpqS/PznozPiNbkIi1X9+td/5Sc/+QBPP12X9dD3BoxEop32duHtt2sYOLAl5LtKdHz/3t9bbx0by40Y5moLc+N5rs7f/nZ1pLuwqOQzWYCzgT8D7wK/AjYAE+OYO8BSoBHYj4uFfDnV/klcOfp/AP8a51xd2cy1FV+eYmG6zi+Ph98lEeVOykd36TusoGEhug67n0J0vXz5UzpihCuGmHb7tKtI2lXjPy7o9stVGDIfwVUMPWbO3BTqovRWN8znmgy7z7huxDVrVPv1S583kVCdMqUhy30WLHAZpEddWyIiwAhgD67qrwBPq+q24g9pxcdcW+baitunN3Xtn32VL8gaRXfpO0y2ntb1HXcczwMPuOWQVOETn2hg8GCyZnYFZe7Xz7m0hg/fyzvv1BDlGgvOmss3oy1sltT8+WtjuyYL+a6Cfb/4xY+xYUN/BgzYz623vthh4Rx+eHvWjLSrr36jpMrIPxdnRCrlzSyS+PIUC9N1fnmKVQo+7vUKOT6XbD2t6+Db+oQJW3MeF+xfX599H7mC72vWZAbqa2pUhwxJl6LPdW/5JlTkOz6qby7rKkxH+SZGFNsiiVNr62kROTFGP8MwCqAzmeg9RSnJFrWMb5z+d9/t1psPsmdPdLLm7bfDu++mVzHcuxe2bk2Xos9FsA5WsTLK16zJXX+sJ5dhDiPOrK1XgaOAt4DdOPtQVfWYnAeWAObaMtdW3D69pevg7KvOure6Q9833XQ0jz46jH792mltze8uiStPT+o66G7y8j0OPXQ/e/ZUxkw+zKRfvzaWLs1OHoxLZ11bl176Md56q3/HZ/X1u7nnnr8VTddh7cV0bY0K2+KYO6WymWsrvjzFwnSdXx7VzgfX48jU1eNHj3auk/PP71weSaGfFUvXYfkU+dxN3nH+oHVNjXOLBXNRfvvbpyID8XHojGtL1a0yOX686r33ur91dfnP11OurW4vkdKbmEViFkncPuWs62Jcz398vmBzKeu6qSnJV75yAk1NyY61zuOUjfGm+g4btpfHHhvWYSGOGrWbt97q3xG4d8QL2kfRlWB7oX1KxiLpC5tZJPHlKRam6/zyFJNi6jssc94fmC5VXXdl8oJnYYwenWkh1tWl97011YcP3x2pmzh01iLpTJ+eskhKfj0SwzB6ls5UEe5tgmux+Mk1QSB4nLcc8N13uyD7ggWuz89+lu7zzjvphebLQTc9ga2QaBhGFt01+6i7iFqlEdzAGFX+Ps7stGCfREIZOxYef7w8dNMTWIykAMxvHx/TdfnGSIrRtzt1vXHjfn7wgxM7Zkx5fW69dSwrVhzh6ykkEsqZZ24NrVfmEWfmXLFm14XJnwuLkZTQZjGS+PIUC9N1fnmKSU/qu7d17ZUoCRTLR2YAABD1SURBVJZI8WbAzZ7t4heJRLyZVXFmzvn7nHvupk7PrguTPxcWIzEMo2jEWTu9r5OOZ7jihN765MnkBJqb00l4553nVhr0r5eeC3/y3oIF+fvMm/cGEycOL5r8Pb2+endgMRLDKAOKuURvueLFKvxru198MSxdujqjX29neQeX1fWIkr8UKhl0FRtIDKOEKXTt9L6MN5vMv7b7wQdTcGZ5dxM16EfJH7QwowaiUsaC7QVgAeD4mK6Lo+vuqCgbRrkE28MWa7r22tUlMbEhXyInhMsfTI70V1y+/PI1Fmwvlc2C7fHlKRam6/zyxKXYFWXDKKdge6lObAhbC2Xs2OxkxajjO5tQWQrBdnNtGUYJEebWKLecjnKns66lujo3GWLPnnTbG2/kzmPxE5bTMnnylrKIodhAYhglRJh/vbeDxwcaXZnYMGUKjB3rZmIBVFTED6iHVRSorW0ti1l6NpAYRglgQfXepxjfwUMPwVlnQUuLGwxUCyuhErQ+d+xIdu5mepiyGEhE5EgRuUtElvvaPi0id4jI70VkSm/KZxhdpZQWkjpQKdZ30BVXZKGLeJUK3Z6QKCJ3AzOArar6YV/7NOBHQAVwp6reHHUOVV0PfNk/kKjq74DfichhwPeBR7vpFgyj2ymnQolNTUnOOKPvJUcW6zuIk+DY1+gJi+QXwDR/g4hUAAuA6cA44HMiMk5EPiIiDwa2IXnO/2+pcxlGWVMuQfXFi0f12eTIcvkOSo1ut0hU9S8iUh9oPglYl7I0EJFlwLmqehPOesmLiAhwM/Cwqj5fPIkNo3co9TfZvlziw6PUv4NSpUcSElMDyYOea0tEZgPTVPWy1P4lwMmq+tWI4wcBNwJn49xgN4nI14AvAM8CL6jqbYFjrgCuABg6dOgJy5YtsyQ5S0jsk7ouxvXiHJ9OjhxEc3NlZHJkvvOZrksz+TOsPW5CYm8VbcxeqxIiRzRVbQKuDLT9GPhxjmNuF5FG4JzKysoTOiuoYRiOQYNa6N+/lZaWCpLJNlpaEtTWtpZciRKjF4iTtdjVDagHXvHtnwo84tu/Driuu65vme3x5SkWpuv88hSTntL3rFmujHqukuv5zme6jn98T/6uw9qJmdneW66tSuB14CzgHZx76iJVLepcN6u1Za6tuH3KWdfFuF6pultM1+Xh2ur2WVsishT4K3C0iGwSkS+raivwVeAR4O/Ar4s9iBiGYfQWTU1Jvv7149i+vTwSCrtMHLOl3DdzbcWXp1iYrvPLU0z6qrulXHU9d25xCmyaa6sEMNeWubbi9ilnXRfjeqXqbik3XccpJV+qug5rtzLyZpGYRVJAn3LWdTGuV6pvyeWm62Ap+dpa1YsvziwlX6q6DmvHLBKzSMwiid+nnHVdjOuV6ltyOeraW5iqX7929u9PcM45DVx99Ruxjy/kWvn6mEViFklZv7mZrvPLU0zMIuk58l1v1iw3NTpqinSp6jqsnZgWSW8lJBqGYfRJDsQyK+baKgBzt8THdF1a7pZiHm+67pu6Dms315a5tsy1VUCfctZ1Ma5XSu6WhgbV0093AWrTdf6+peDaKouFrQzDOHDoylK3Ru9gMRLDMEoCl4OR3ndl6if2qTL1fRWLkRSA+e3jY7o2v32hut64cT+LFx/Dk08Oprm5gqqqNk45pZGvfW1jRoVh03X8PhYjsRhJWfvtTdf55SkmfcVvf+WVrrRIdbX7O3PmprzHm66j+1iMxDCMA47gUrc7dhwgRQ/LHIuRGIZRMgRzMFatWgtM7C1xjJiYRWIYhmF0CQu2F4AFgONjurYAsOm668dbsL2ENgu2x5enWJiu88tTTPpqANh0nb+vBdsNwzCMsscGEsMwDKNL2EBiGIZhdImSH0hE5EgRuUtElvvaPiQit4nIchGZ25vyGYZhHOh060AiIneLyFYReSXQPk1EXhORdSLyrVznUNX1qvrlQNvfVfVK4AIg/4wCwzAMo9vobovkF8A0f4OIVAALgOnAOOBzIjJORD4iIg8GtiFRJxaRmcCTwB+7T3zDMAwjH92a2a6qfxGR+kDzScA6VV0PICLLgHNV9SZgRgHnXgGsEJH/ApYUR2LDMAyjULo9ITE1kDyoqh9O7c8GpqnqZan9S4CTVfWrEccPAm4EzgbuVNWbRGQicB5QBbykqlkLWorIFcAVAEOHDj1h2bJlliRnCYl9UtfFuF6pJsmZrssjIbE3am1JSFvkaKaqTcCVgbZVwKpcF1HV20WkETinsrLyhMLFNAzDMOLQG7O2NgEjffsjgIZekMMwDMMoBnHS37uyAfXAK779SmA9MBpIAi8C47tTBiuREl+eYmG6zi9PMemrZTtM1/n7lkKJlG6NkYjIUlwN6MHAFuA7qnqXiHwS+CFQAdytqjd20/WtaKPFSGL1KWddF+N6peq3N12XR4yk1wsq9sRmFkl8eYqF6Tq/PMWkr74lF0vXDQ2qp5+u2tiYt2te+qquw9opBYuktzGLxCySuH3KWdfFuF6pviUXS9e33jqWBx44gnPOaeDqq9/I2TcffVXXYe1mkZhFYhZJAX3KWdfFuF6pviV3VdfV1aqQvVVXRx6Sl76q67B2zCIxi8Qskvh9ylnXxbheqb4ld1XXTU1JFi78IE8+OZjm5gqqqtqYMGEbc+f+g4EDW3LeQxR9Vddh7WaRmEViFkkBfcpZ18W4Xqm+JRdD11deqZpIOCskkVCdOzdn97z0VV2HtWMLWxmGYcCWLXDllfD00+7v5s29LVHfw1xbBWDulviYrs21Zbru+vHm2iqhzVxb8eUpFqbr/PIUk77qbjFd5+9rri3DMAyj7LGBxDAMw+gSFiMpAPPbx8d0bX5703XXj7cYSQltFiOJL0+xMF3nl6eY9FW/vek6f1+LkRiGYRhljw0khmEYRpewgcQwDMPoEhZsLwALAMfHdG0BYNN114+3YHsJbRZsjy9PsTBd55enmPTVALDpOn9fC7YbhmEYZY8NJIZhGEaXsIHEMAzD6BIlP5CIyJEicpeILA+09xeR50RkRm/JZhiGYXTzQCIid4vIVhF5JdA+TUReE5F1IvKtXOdQ1fWq+uWQj74J/LqY8hqGYRiFU9nN5/8F8FNgsdcgIhXAAuBsYBPwrIisACqAmwLHf0lVtwZPKiKTgVeB6u4R2zAMw4hLt+eRiEg98KCqfji1fyrwXVWdmtq/DkBVg4NI8DzLVXV26v8bgf7AOGAvMEtV2wP9rwCuABg6dOgJy5Yts9wGyyPpk7ouxvVKNbfBdG15JKQGqXrgFd/+bOBO3/4lwE9zHD8IuA34B3Bd4LNLgRn5ZLA8kvjyFAvTdX55iklfzW0wXefvWwp5JL1hkZwPTFXVy1L7lwAnqeq/dMO1LbPdLJJYfcpZ18W4Xqm+JZuuy8Mi6Y1ZW5uAkb79EUBDL8hhGIZhFIM4ZktXNrJdW5XAemA0kAReBMZ3pwzm2oovT7EwXeeXp5j0VXfLgaTrhgbV009XbWws7Fp93rUlIkuBicBgYAvwHVW9S0Q+CfwQN1PrblW9sZuub64tc23F6lPOui7G9UrV3XIg6frWW8fywANHcM45DVx99Ruxr1UKrq1ut0hKYTOLJL48xcJ0nV+eYmIWSc9RbF1XV6tC9lZdbRZJSWAWiVkkcfuUs66LcT2zSOJTbF03NSVZuPCDPPnkYJqbK6iqamPChG3MnfsPksntZpGUymYWSXx5ioXpOr88xcQskp6jO3R95ZWqiYSzQhIJ1blz41/LLJJuxiwSs0ji9ilnXRfjemaRxKc7dH399eMZOLCFGTMaePDBI9i+Pcn8+WstRlJKm1kk8eUpFqbr/PIUE7NIeo6+quuwdmxhK8MwDKMnMNdWAZi7JT6m6/J3t3Slr+m6OMeba6uENnNtxZenWJiu88tTTPqqu8V0nb+vubYMwzCMssdcWwVg7pb4mK7N3WK67vrx5toqoc1cW/HlKRam6/zyFJO+6m4xXefvWwqurT5tkXiIyLvAW8AhwD+7cKpCj4/TP1efqM/C2uO0DQa25ZGnWJiue07XYdfvzuNN131T12HtY1X1kHxC9rq10JMbcHtPHh+nf64+UZ+FtcdpI+bbhem6vHTd0/o2XfdNXUfoNpasB1qw/YEePj5O/1x9oj4La4/b1lOYrnuWntS36brnju9JXYe1x5L1gHBtGQ4R+ZvGCZwZXcZ03XOYrnufA80iOdC5vbcFOIAwXfccputexiwSwzAMo0uYRWIYhmF0CRtIDMMwjC5hA4lhGIbRJWwgMQAQkQ+JyG0islxE5va2PH0ZEfm0iNwhIr8XkSm9LU9fRkSOFJG7RGR5b8vSl7GBpA8gIneLyFYReSXQPk1EXhORdSLyrVznUNW/q+qVwAWATaWMoEi6/p2qXg5cCny2G8Uta4qk6/Wq+uXuldSwWVt9ABE5HdgFLFbVD6faKoDXgbOBTcCzwOeACuCmwCm+pKpbRWQm8C3gp6q6pKfkLyeKpevUcT8AfqWqz/eQ+GVFkXW9XFVn95TsBxqVvS2A0XVU9S8iUh9oPglYp6rrAURkGXCuqt4EzIg4zwpghYj8F2ADSQjF0LWICHAz8LANItEU63dtdD/m2uq7DAfe9u1vSrWFIiITReTHIvJz4KHuFq6PUZCugX8BJgOzReTK7hSsD1Lo73qQiNwGHC8i13W3cAcqZpH0XSSkLdKPqaqrgFXdJUwfp1Bd/xj4cfeJ06cpVNdNgA3W3YxZJH2XTcBI3/4IoKGXZOnrmK57DtN1CWIDSd/lWWCsiIwWkSRwIbCil2Xqq5iuew7TdQliA0kfQESWAn8FjhaRTSLyZVVtBb4KPAL8Hfi1qq7tTTn7AqbrnsN0XT7Y9F/DMAyjS5hFYhiGYXQJG0gMwzCMLmEDiWEYhtElbCAxDMMwuoQNJIZhGEaXsIHEMAzD6BI2kBhGJxGRXQX2/7SIXN9NslwqIj9N/f9VEflid1zHMMKwgcQweo5rgZ/F7Zwqmd4Z7ga+1sljDaNgbCAxjBiIyO9E5DkRWSsiV/jafyAiz4vIH0Xk8FTb10TkVRF5KVXmHBE5CmhW1W2p/Q+KyNMi8qyIzPesm1QV5pUisgR4Oc+1vygir4vIn4FPeO2qugfYICIndb9mDMMGEsOIy5dU9QTc6pFfE5FBQH/geVX9KPBn4Dupvt8CjlfVY0hXnv0E4F975EfAj1T1RLKLDp4E/Kuqjou6tojUAf87dd6zgXGBc/wNmNClOzaMmNhAYhjx+JqIvAg8jas+OxZoB+5Nff5L4LTU/y8BvxKRzwOtqbY64F3f+U4FfpP6P7iI2DOq+maea58MrFLVd1W1xSeHx1bgiILv0jA6gQ0khpEHEZmIW4jqVFU9FlgDVId09QrXfQpYAJwAPCcilcDeiGPC2B3z2rkK5VWnrmkY3Y4NJIaRn0OAHaq6R0T+B3BKqj0BeOuAXwQ8KSIJYKSqrsQF1w8FBuAq1Y7xnfNp4DOp/y/sxLVXAxNTbq5+wPmB444CXinwPg2jU9hAYhj5+QNQKSIvATfgBgFwlsN4EXkOOBOYD1QAvxSRl3HWw62q+h7wF9xyr94Kf/OAb4jIMzi31z8LubaqNgLfxZVZf5zM+Au42MnjXblpw4iLlZE3jB5CRH4EPKCqj4tILbBXVVVELgQ+p6rnFuk6xwPfUNVLinE+w8iHrdluGD3H93BBcnDxk5+mLJT3gC8V8TqDgX8v4vkMIydmkRiGYRhdwmIk/397dSwAAAAAMMjfehB7SyIAFpEAsIgEgEUkACwiAWARCQBLs0QbX9tczocAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "."
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEaCAYAAADUo7pxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXmYVOWV8H+nC3oDUVHAZgm0Cvk+TAJojEtE6UjAOLiAqAgxOokxwmSik0m+0UwSE5hMMhkzmSyAmrhhFEVCDDoYtwHBBDCKRsUNbTfsBhTahaW7oft8f7z3dt26XbcWqqq7qzi/56mn7n5P3aq65573bKKqGIZhGEYulHW3AIZhGEbxY8rEMAzDyBlTJoZhGEbOmDIxDMMwcsaUiWEYhpEzpkwMwzCMnDFlYmSNiLwhIhO96e+IyG97gEwqIkfn+Zg/EJHf5fOYxv4hIreKyL+lWL9TRI7sSpmMREyZlBgiMkNE1ovILhHZ5k3PEREpxPlU9d9V9bJcjyMiIzyF0CsfcvU0RKRGRJaLSIP3OUeE1leIyM0i8qGIbBGRbwbWnSgiD4vIDhF5V0TuEZGaJOcoF5GXRGRzhAyXeOfOx/fV8UCxn/t/S0QaReR9EVkpIlW5yKOqfVW1PpdjGLlhyqSEEJF/Bn4B/CdwBDAIuAL4LFAesU+sywQ8sGkH/gScF7H+B8BIYDhQB/w/ETnDW3cocCMwwlv/EXBLkmN8G9iW7OAicihwDbBxv6TPIyLyf4B/AyYBhwM/xF0fo5hRVXuVwAs4GNgFnJdmu1uBhcAKb/uJwN8BTwMfAm8DPwjtczHwJrAd+FfgDWCit+4HwO8C254I/AV4H/gbMCGwbhUwD/gz7ob4EHC4t+4tQIGd3uukJLLHgO8Ar3n7PwUM89YpTnFuApqA+YAE9v0y8KK37kFgeGDdMcDDwA5gK/Cd8GcDegOLgd8D5Tl8T708WUeElr8DTArMzwPuijjGscBHoWW13uf7ArA5yT7XA3O87+CyiOPWAc8F5h8BngjMPw6cC9yOu/nv8b6r/wdUAr/zfiPvA38FBkWc52jvt9cvi+t2q/cZHva++8dC36ECR3vTkb/nVHLi/kM3AY3e9/FvQCwg82PAB8B7wN3d/Z/vaa9uF8Beefoi4QxgH9ArzXa3en+Iz+Is00pgAvBJb/5T3g31XG/70d4N41SgAvgv7zydlAkwxPuTnukd6/Pe/ABv/SqcIhgFVHnzP/HWjfBuCJHy4568nwM+DggwBjjMW6fA/cAhwMeAd4EzvHXnAq8C/xd3M/8u8Bdv3UHezeOfvWtxEHBC8LN5sv6Pd+1iOX5PnZQJzvJQAjdfYDqBG3voGFcB60LL7gemet/l5tC6zwBPet/JKqKVSSVOQRzuybkFaPCuSZW3zr/eb/i/AW/+a8B9QDVO6R9HhLIA+gGv45R6RYbX7VacEvF/h78AHg+sDyqTCUT/niPlBO4FbgD6AAOBJ4CveesW4x6k/P/MKd39n+9pLxvmKh0OB95T1X3+AhH5izcmvUdETg1s+0dV/bOqtqtqs6quUtXnvPlncX+c07xtpwP3q+pqVW0Bvkf0kMQXgRWqusI71sO4m9iZgW1uUdVXVHUPsAQYm8VnvAz4rqq+rI6/qer2wPqfqOr7qvoWsDJw7K8BP1bVF73r8+/AWBEZDkwBtqjqz7xr8ZGqrg8csx9ueOo14O9VtS0LeTOlr/f+QWDZB7ibeAIi8ing+zjF6i+bilPCf0iyfQxYAPyjqqYcSlLVZtz3dSrwaeBZnDXyWZzFuSl0vYPsBQ7D3dDbVPUpVf0wYtsluGG7V4F7RaTCk/UOEfnHFCL+T+B3+K/ASSIyLMnnSPV7TiqniAzCWXVXqeouVd0G/ByYEdhvODDY+508nkLOAxJTJqXDduDwoANbVU9W1UO8dcHv+u3gjiJygucEfVdEPsANFx3urR4c3F5Vd3nHS8Zw4HxPgb0vIu8DpwBBZ/GWwPRu4jfSTBiGu6lHEXXs4cAvAjLtwFk2QzI45om4p9ufqPeIGkZEPuZFE+0UkZ2ZfZQE/H36BZb1wz2JB89zNPAAcKWqrvGW9QF+CkTdhOcAz6rq2gxleQz3ZH+qN70KdyM+zZuP4nacpXGXF2TwUxHpHd5IRD6OG077b0/mJpxCqQJOAB5NcY7g73An7nscnOQcqX7PUXIOxw1lNgZ+JzfgLBRwQ3kCPCEiG0XkyynkPCAxZVI6rAVagHMy2DZ8U7wTWI7zPxyMG5v2o78acTdcAESkGvdkl4y3gdtV9ZDAq4+q/mQ/ZIo6/lEZbJdsv6+F5KpS1b9kcMyHgB8Dj3pPr51Q1bfURRP1VdVslKO/fxPuOo8JLB5DwFnuWVGPAPNU9fbAdiNxQ4RrRGQLsAyo8SLCRgCnA1O9+S3AycDPROTXEeKElcljJFcmCd+Xqu5V1R+q6mjvHFOALyU5fi+cZdvmWUqXePPPAE+r6gsRckHi77Av0B83DBcm8vecQs63cf+fwwO/kX6qeoy33xZV/aqqDsZZugvyHYpe7JgyKRFU9X1cVMwCEZkuIn1FpExExuLGgFNxELBDVZtF5DPAzMC6pcAUETlFRMqBuUT/bn4HnCUik0UkJiKVIjJBRIZm8BHexd1UUuUK/BaYJyIjxfEpEYlSbEGuB64RkWMARORgETnfW3c/cISIXOWF5x4kIicEd1bVn+JuUI+KyOHsJyJSiRvvB6jw5n0WAd8VkUO9aKev4vwEiMgQ4H+B+ap6feiwz+NusmO912U4H8FY3A3yUpyvyF//JO538q8RYv4F55P6DM75vhH31H4CsDqw3VYC35WI1InIJ71htQ9xw0LJhgRfwgVJLBCRg3HWwEM4P1qbSMoQ9jMDv8N5wHpVfTvJdpG/5yg5VbXRk+NnItLP++8cJSKnefudH/gdN+GUaSGGPIsWUyYlhHfT+ybOJN+G+8PfAPwL7iYRxRxgroh8hBuPXxI45kbgH3A300bcHylpHoP3xz4HF3H1Lu5m9m0y+J2p6m7gR8CfvWGGE5Ns9l+ebA/hbgQ34RzD6Y79B+A/cEMbH+JuwF/w1n2ECxQ4CzdMtgk3DBM+xjycg/YREemf7pwR+NFP4G6qewLrrsUNt72JswD+U1X/5K27DHfjvjY8nKaq+7yn5i2qugU39NPuzbd5PqTg+lbgQ1UN+meCn3MXsAHYqKqt3uK1wJueH8Hnxzjl976IfAsXir4U97286H2GTgmfns9pCi5Q4jXc9T4e5zA/FhdBFcWd3nXagXOcz4rYLvL3nEbOL+FC6F/A/c6XEh+iPR5Y71335bihxtdTyHrAIRHDwIZhGIaRMWaZGIZhGDljysQwDMPIGVMmhmEYRs6YMjEMwzByxpSJYRiGkTMlWe47zCGHHKJHHx2dX7Rr1y769OmTclmqeX866j0b0u2Ti6yZyNydsqabLpSsUetKWdZ0y3KVM2p9umXF+L8qJlmz/a0+9dRT76nqgIwE1x5QIKzQr1GjRmkqVq5cmXZZqnl/Ouo9G9Ltk4usmcicDfmWNd10oWSNWlfKsqZblgn5+P6jZEkmV0/6rYbni0XWbH+rwJNqhR4NwzCMrsKUiWEYhpEzpkwMwzCMnDFlYhiGYeSMKZMipbERTjsNtmxJv61hGEahMWVSpMybB48/DnPndrckhmEYB0ieSSkxefJ4Wlvj8wsXwsKFE6ishD17ovczDMMoJGaZFBl33rmemTOhutrNV1fDxIlbed06KxiG0Y2YMikyDjuslX79oLkZKivde3X1Po44orslMwzjQMaUSRp8R/eOHeXdLUoHW7fCFVfAunXuvamp58hmGMaBiflM0uA7ug85ZDjTpnW3NI5ly+LT8+fDqlUbgQndJY5hGIYpkyjCju7ly4cggjm6DcMwkmDDXBGEHd0VFW3MmoU5ug3DMJJgyiSCsKO7tbWMfv0wR7dhGEYSTJmkIOjoPvvsBss2NwzDiMB8JikIOrqvumoTEyYM6T5hDMMwejBmmRiGYRg5Y8rEMAzDyBlTJoZhGEbOmDI5gGlshCuvHGuBBYZh5IwpkxJhf/qbzJsHzz13sJWxNwwjZ0yZlAjZ9DepqoK6ugksXAiqwsKFbr6qyq23xluGYWSLKZMiJ6gY2tvppBiSUV8Pp5++NbKMvTXeMgwjW0yZFDnpFEMyamqgT599NDdDeXlbRxn72loQIUEx+fXIzFIxDCMVRalMRKRMRH4kIr8SkUu6W57uJKgYUvU3CTvbm5rKueIKmD9/Q0cZ+/p6OjXemjULZswwS8UwjNR0uTIRkZtFZJuIPB9afoaIvCwir4rI1WkOcw4wBNgLbC6UrPmgK/wPvmJI1d8k7GyfO3cj8+fD0UfvYv58N19TQ0I9st274Y474LbbEi2VVENohmEcmHSHZXIrcEZwgYjEgPnAF4DRwEUiMlpEPiki94deA4GPA2tV9ZvA7C6WPyty8T9kqoh8xTBmDB2KwSeZs13EldgPnse3WoL1yC65BIYO7WypWOVkwzDCdLkyUdXVwI7Q4s8Ar6pqvaq2AncB56jqc6o6JfTahrNGmrx927pO+sxJ5hjP9qk+H47wZD6VWbNg8eL1CefxrZZly+hQTLfeClOmJLYItsrJhmEkQ1S1608qMgK4X1U/4c1PB85Q1cu8+YuBE1T16xH7VwO/AnYDL6nq/CTbXA5cDjBgwIDjlixZEinPzp076du3b8plqeb96eB7S0t/fvWrj7FuXQ0tLTEqKtoYP/49Zs9+jf79W4li586dnHfeF2htjXVaV17exu9//0DWsv7Hf4zgwQeH06tXO/v2lXHWWQ189atPpzzPgw+uAeD73z+G/v1bmTKlgfvvH8yOHeUdlo//OefOHc21177Q6XNle13TTYffsyHVPlHrSlnWdMtylTNqfbplmVzLfF/TTGXN9h7QE2XN9rdaV1f3lKp+OiPBVbXLX8AI4PnA/PnAbwPzFwO/ytf5Ro0apalYuXJl2mWp5v3p8PvZZ2/WsjLVykrVsjLV2bNTitGxb0OD6syZqtXVquDeZ81SbWzcP1nHj9+mc+ao/uY3T+icOapTp8bPc/rpW5KeJ4qGBtVTT3XbLF36Z62pURVJ/tmylTXddPg9G1LtE7WulGVNtywT0m2fyfcfJUsyuQp1TTORK918scia7W8VeFIzvM/2lGiuzcCwwPxQoKGbZMkbYcd4pk74sCM81+GlsLPdL62fLEQ43Xn8obehQ2H69JNpbARVc84bxoFOT+ln8ldgpIjUAu8AM4CZ3StS7sydu5EJEyYAzg+RDb4j/PLL4cYbnZO8EPgKb9y4DTz99PGR56mqckotHevXp9/GMIzSo8uViYgsBiYAh4vIZuBaVb1JRL4OPAjEgJtVdWOKw5Q8wcZc2SqibPAV3qpVu7jssujt6uvhW9+Ce+91IcOxGLS1KSAJ251xBmzYYE56wzjQ6HJloqoXRSxfAazoYnGMLFi5EvbsiQ+9JaOx0Q2fVVbCAw90rXyGYXQfPcVnYvRQ/FyXa65x06NHOx+QCIStkiDNzYm5LIZhlDY9xWdi9FCGDYO2Nli92s1v3Ahjx0JFBZxyylbWrh3E7t2J+/TqBRdeCNOmrQdO7nKZDcPoeswyMZJSVeWsj7YkKaGzZsEbb8QjwcoCv6JYDPbtc1FhqfJpfItnx47OpV8Mwyg+TJkYSQkXfQSnNETi4cN+JNiGDVBb615PPeXKsCxZklpR+CHGt902HLCuj4ZR7Ngwl5GUYK5LWZkrCXPeeTBgQDxMORj6XF8f37e6GpqanKKYNi3xuJMnj6c1YLAsXz4EEWfRtLe7ki4LFhT2sxmGkX/MMjEi8XNdNmyAOXPc8FUw6TGMPzTm1yPzFUVZWTxh884719OvX3AvV86nrS26EGWY7dvLOwpgWldIw+gZmGViRJJtrks4FwWUfv2Ejz5yxSoXLoTODvnEiLDqapg6FerqNnDaacdz992dz3PjjbWsXu2KUX7hC/FimGbRGEb3YZaJkTdqauDuuwlEdwkffhgvt+KILizau3e8pMvy5YNZswaOPTbue5k8eTwi8NBDNQBs29a510rQCvJJ5o8xi8Yw8ospEyOvTJoEI0dCecj3XlUF06ZBeXl75L5798Y6FMPy5UNQdTf98847maoqNwyWitpap7iuDrVWCzcG85dZ90jDyB+mTIy8smIFnH46npPdWSGxGLS0wMsvQ2tr9E9u4sStPPNMYqixj59xP2JE/Lhh/KZdt93mrBT/FW4MFu5zX1c3wQpUGkaOmDIx8kpVFVx/vT/nLIm2Nnfj3rgxviyOdoQcV1fvY8wY5zMJKoxYrJ1Zs+Cuu9bR0pK5LL17u2P5yqm62llHU6cmdo8cP/5dxo61IS/DyAVTJkZeCeen+Dfw2tr4NjGvH5d4euW882D27Hjv+g0bErdta5OOJMidO93yfv3gyCNTy3LuufDgg06RlZW109wMffu6cjDBGmNvv13FE090HvIKRo0ZhpEaUyZGXgnmp5SXt7F7t4sKC/aNb2tziuTpp+GccxrYtw+++114/PHDEfG3jVs1ZWXx4aiPPnLrPvwwMbclGffcEw8GaG8vo70dFi2K1xjzLaY33ujbqbVyYyN87WvHsWaN+VUMIxNMmRh5x89PmT9/A5dc4hpp+UNNvXu3MXKkK1U/ZgxcddUmli1zDnGAo4+mU7/6e+5Zy9NPw6BBexLOU1HhMu2HDs1exo0bYe9efy4+pDZtmnPiDx4M27dXJDT+ssKVhhGN5ZkYecfPT/F7pMyeHfej7N1bxsSJ8ZyQSZPGB27qwquv+tNKc7MQi8EPfzial1+GlpZEL3lLC1x0UXy4bP+JHyAqIRNg8WIrXGkYUZhlYhSURIc8gIuq8qOn6uq2AXE/SizmQouvu+4ZrrgC1qxxYb0XXuiGzcK0tbnM/K7AD1E2DKMzpkyMghJ2yFdUtDFrlhtKCiYg+tWJ29pg4kT4znc+xYIFzn+iKixaBK2tTuOIROeqRKP06QMf/3gySyY6kTLI+PHvJvh+IJ4Q+be/WRVk48DGlIlRUIIO+cpKl2fSr59TEjNnOuUCrgfKCSe8x6WXuuipO+9c3ykqbMCAZi69FE499T0geT6Kryiqq6FPHzftJ1Du2uVyXbST7ogaJ0vc8NVX+3RqR+wnRF5wgev5csMNtRjGgYgpE6Pg+A75devg7LMb2LIlrmRaW8uorHRRVYMGtXDLLc5vcdhhrQlRYc3NcNJJ27nlFmhvF+bMcSHE1dUunwScIvEVxe7d0L+/K1D5xBMwefKWDuWyvzQ2ViPiHP9+aRc/IfKVV9w2Dz1UY85644DEHPBGwQk6ta+6ahMTJgwBnJI5++wGfvjDIdx4Izz3XOIQka+Exo3bwNNPH9+x3i99X1WV2Is+bHH06uUKVLrtalJIqCS3TpJbLK2tsHTpepYtO5k774w4olf6xc9VuftuOlk1hlFKmDIxuo1ly2DVqk2MGTOE+fNh1aqNwISE9RCPCguvr6+HIUOSDVs5Xn+dDkvilFPe5fHHB0RIkn042PTpmUV1LVo0vKNg5YYNplCM0sWGuYyipabGRX5FUVbmStS3tpJCkewvToMNHx6f9pf7w2kiiQUra2qwaDCjZDFlYhQlfiMu31eRjPZ2eOCBaMslN5w18+ab8Wnf+d+/P4wc+VHS/Jfm5niGvZVqMUoJUyZGURIOOe7VJQO2qbWSU1rC22/Diy/28+qRxffp1ctl9L/+erwE/r/8iykVozQwn4lRlHQOOYYvfcklMEY5xaOJcsCHydy3oipe7TC3TyzmZLvrLrjjjvh2ixa595oaZ62YT8UoVswyMYqWYMjxFVfARx8R6i8fJJVVkXM9lrT4BSv79oVDD02+TU2NGyqzxEejGDHLxChakvWonzbNL1sf3jqVwsjUMsltn+nTXWHKdJx//olJ5DeMno1ZJkZJsWwZvP22G/pKpL3Dv5KM2tp4fbDMyN6ayUSRgCuXH9XPHsx5b/RMTJkYJUdtbWIyo6OMPXuSbQ0gvPFGMmsmTEHCwpKe56CD3FSyXirWv97oiZgyMUqOCy90735obq9erq7XgAFw8MHJ9/ELT15wQdCqCSuPwvtW/PN89BEJvVTKypxcdXUTEvrX+828ColZQkYmmDIxSgY/98SPkPLzS/btc42utm2DDz7ovJ/fY76hwZU9iVs1XaU80qMKBx0EVVUdzV86moeFKxnnG7OEjEwoSmUiIh8TkeUicrOIXN3d8hg9Az/3xH9Sj8XgzDPh0kvh+ON3JOSlBBMK29qEXr3iYbnnn++vyWVYK/9DYu+9B3v29O6Y373bhRkfcUS8Blg+S+H7yrmrLSGjOOlyZeIpgG0i8nxo+Rki8rKIvJqBghgF/I+qfhkYXTBhjaLCzz3xfSNtba7cyS23wE9+8lxCXopqYoju6tXxm+c99/hLc7FMusaq6dcPHnkEZsw4kdWrXQ2w1avh8suPy3lYKpwY2lWWkFGcdIdlcitwRnCBiMSA+cAXcMrhIhEZLSKfFJH7Q6+BwNPADBH5X2BlF8tv9FA6d3Wko6vj9u3l3HOPS2xs93prNTX5Wwmvv57Mad/z+fBD+PznYd8+91f2P9v27RUdtcB8qyVb5RJODG1udvOWWGkkQ7QwhYtSn1RkBHC/qn7Cmz8J+IGqTvbmrwFQ1R9H7P8t4AlVXS0iS1V1epJtLgcuBxgwYMBxS1LEZe7cuZO+ffumXJZq3p+Oes+GdPvkImsmMnenrOmm08m6fXs5CxcexeOPH05LS4yKijbGj3+P2bNf47e/Hcyf/jScs85q4EtfepP//u+R/OUvh9Pe7lsQ+5NrUiy0A8Jhh7Xy85+vZtiw3h3XcPv2cq699uPMnfsy5eU7Ol3X73//GPr3b2XKlAbuv38wO3aUM3fuRiA/3394WTH+r4pJ1mz/V3V1dU+p6qczElxVu/wFjACeD8xPB34bmL8Y+HWK/T8BLAWuB65Ld75Ro0ZpKlauXJl2Wap5fzrqPRvS7ZOLrJnInA35ljXddCayXnGFalmZamWle4/FVN2gVuIrcXm7gurIkcmXl9qrsjJ+DWfPVhVp19mzs/8N5OP7Dy8rxv9VMtl6qqzZ/q+AJzXD+3pPyYBP9kgYaTKp6vM4BWQYCfglVi6/HG680Y3vH3oo/P73bbS0xKiuhqlTYfHi4F7u57dpU/hopWmpNDdDXd1pgSXCwoWwcOEEKitJkY9jGNH0lGiuzcCwwPxQoKGbZDGKmGXLXGmVMWPc+4oVie2B/XH/zZsTI7+qqpyPYNiwYKSXdrQELh2UkSPdZzz66MTPP3HiVnOuG/tNT1EmfwVGikitiJQDM4Dl3SyTUSL47YH9gpC+I3rlSv8pXNmzB8491+VyqMZLq+zdG3XUYkXYtMlVNX711cTPX1bWztSpcNJJlqBoZE+XD3OJyGJc79XDRWQzcK2q3iQiXwceBGLAzaq6satlM0qTcHtgcFnx8fIpzhRZuDC+j1tXmsNcnXGf86GHajqWzJ0LCxZ0lzxGMdLllomqXqSqNaraW1WHqupN3vIVqjpKVY9S1R91tVxGaZCs9EdjI1x55Vi2bInnkiSrwzViRGLCY+rEw66PguxKLEHRyJaeMsxlGHkhWemPefPguecOZu5cl4hXFvGrf+MNN+xTWRlV9DGoQErfaqmogPXru1sKo1gwZWKUBFGlP/xlqi5iafDgeGJfMi691DXbqq2Fioo2LrgAhgxxvpQ+ffYm7euemuK1YFpaXCBDWRkcd5x7mT/FiMKUiVESJCv9MW2aCwMOLkulDC65xJVeGTPGHe9Pf3qcu+92kV8ffginn/5uR/HIzCl+C0YVNmxwr3XrOhd8tKrCBpgyMUqEZKU/Bg1yr+ZmKC9vo7kZRo5MtrfSp49TGMnwrZ7ly4dkIImG3ksP3+qbNGk8YFWFDYcpE6NkCPeE37IlvkxVaG+HV17pvN+IEbuYNCmxDXCQ+npn4YhkoiAk9F667N1bZlWFjQ56Sga8YeRMsp7wPp/73DqWLTuZe+91pdt79YJJk2DgQHjttT3Mn9+X005z/UzChQxrauDll+P5J845H6zlVcp1vVLR+TMfeigMHep8K9/+du5l8I3iwSwT44DBT1KsrHRP0n55+rlzN0YO1fhDXC+8ACARUV4HoiJJTlMTPPecsw5/+cujzJdyAGHKxDggWLRoOI2NMHp04jBYVVXqVrhhx36cdAqkdH0mmfLYY4NYvdpZdqZQSh9TJkZJE3aeb9wIY8fCzTe7YbH6ejj99K2RDaCCjv3ycmeWuCrdQWURNW34+L1VjNLFlIlRMiQLUfUti4oKpwiSKYs+ffalbADlO/Hnz9/AnDmd81SqqvYxZIjLx+jVSxkwoMAftEjxqxVHVScwihtTJkbJkMzv4VsW4arBQWXR1FTeKQosiF+J+OijdyU49ocOde/79gk7d8L06a6f/MCBwb3NUgkzdCg8+6ybDlYnMMVS3JgyMYqeqOx3f1glqmqwz9y5GxPK1keFCE+ePB4RFw0GwubNbvnevb344ANYssSFIG/c6NY7zDmfiAtiGDMmeXWCZ589OOt8FUua7BmYMjGKnmTZ78GhrGXL4KqrNqVVFum48871CT1QjEIgHQ8DkyePz2gPS5rsGZgyMYqeZNnv4aGsfHDYYa3EYvEeIHGSTdvwVq4sWLAh5fp0FqnRtZgyMUqCZNnvheD22/2p4PBVfLqioo3yhFw9Uyr7y7e//Sn+9rfOfpTt28s57TT3XaeySIP4+9hQWOEwZWKUBOF2vcmGsnIZW/fzUZLRp89ebrgBjjkG+vZt47DDXImWIZmU8jIiaWqqYOzYzn6URYuG8/jjcMMNmVuk/j42FFY4TJkYBwy5jK2H81GCFsfpp2/j8svh+edh6dK1NDTALbc8SXt76irFRqY4P0pd3YSOnCF/WOv6690WURZpMM/IhsIKS0plIiIxEfnPrhLGMApBuiz3THnmmUO8SC4IDm0tXz6Eqip45hmYMuWUjrDXhga45561HSHERm5UVe2lthZ8Re66YcLFF0dbpOnyjIz8kVKZqGobcJyIPV+k8zQdAAAgAElEQVQZxUu6LPdMmDfPjbt//OPwhS9ALOYyF6uq4JBDWlm/Hr74Rdi1K8bMmfH9DjuslSlTnPIKdnj0b4RG5uzZ09v7ztztyK+Tdttt0Q8HmeQZGfkhk6rBTwN/FJF7gF3+QlXdzwBLw+haMslyj6Kqym3vEF5+2VUQBqGy0kV27dnTmzFj4tts3Bj3ryxdWs4997jGWzt3wtKlTqm0tbkbYPbNtoxkHHSQG8JMhp9n9PWvD+FLX3LtmY38k4nPpD+wHfgccJb3mlJIoQwj36TLco8iWQ7L0KFOIQSVTJgjjtjD3/7mHL9NTU4ptbXBnDnw4INOwdXVuWOblZI7H33krmeytsJ+ntGyZU6xjBiR3bH9wI0dO6ykfipSWiYiEgOeVdWfd5E8hlEQ5s7dyIQJE4DOvU5SES702Nwc46yz4PTT17Js2cnceWfy/bZsqfKsFRfStXChW15RAX/4g7vhnXuus0za2oJ9UuDA7Y+SGzt2uIeFwYMT66c563JCx/zChe7lW5bp8AM3DjlkONOm5V/uUiETn8nZXSSLYfRIgoUefavmsMNa6dfPj9Zq77RPr15tTJ2a6PgVgZYW96SrGo9GisXgqaegttY/nimSXFCN+6i2bNl/n5lfPscP3Fi+fIhFgqUgk2Guv4jIr0VkvIgc678KLplh9BDChR79iKGtW2H2bDjttPcCWzsnyJlnbmHQIGhpKeuo55XMP1JWBps3u2ikP/8ZTjgBDjmkufOGRtaowrHHurYDq1YNYPduZxlm6jPzy+f4Sqiios0iwVKQiQP+ZO89GJ2vOB+KYRyw+Erl1FOF2lo4/njYtm0bb745iKamcmIxl7z4xht9I49x8cXxm9q8efDEE1BZuT/pXzY0lozGRvj858G/NmefDQMGuOXp8K1PP3CjpaXMIsFSkFaZqGpdVwhiGMVK0B+zatWLTJgwiIqK/rS2AkQrklGj4MMPwxFjsHt3KkdvlNIwRZIad33uucfNVVYmrvXLrdx9d/JeNpdfDtde28CWLVbWIIq0j0AiMkhEbhKRB7z50SLylcKLZhjFiz9E4vtMwhFbxxzjXsuWwdq1ZNFQy5RGPhg7NjHqK6rcSrBMjx8Rli3hMj6lWjI/E3v6VuBBYLA3/wpwVaEEMoyeTKZhov4QSWur+4vFI7UcGzfCAw+46RtvhHffDa615JNCs26di9QTya7cyv4ognAZn0zK+hSjwslEmRyuqkvwQlZUdR/QlnoXwyhN/BvBbbcNT7utnyz38MMwcmTcOvGjifyoIz9sODNM0RSG+HWtqID16ztXGt6+vZzjjoM1a+KKINXDRVQZn6iyPsFjFWOPlkwc8LtE5DC8qy0iJwIfFFQqw+hhhHMV/DDRysq4hRFm2TJYtWoTEyYM4fTT4bXXEjPwX38dvvUtuPdeF+1VXQ39+7uaXuE+80EGD4YPPoB9+1yocXLMIZ8d8WvV0uKGtcrKTuyICNuxA1paTu7YxuWqTHB7ClRW1vKLX8CVV5bT2Ahz5oxj9Giorn6XDRsGdNR069vXWal79rjve+pUuO46t27ePFi9GlavDp8n85yY7iQTZfJNYDlwlIj8GRgATC+oVIbRw6ivh4sv3sratYO8ENM2pk+Pcd118NJL0fv5T7d9+sQduTfe6J5Cw0299uxxN5obbniSFSuOZ/ny+PCYCHzuc9Cr1xaqq2t44IFUigRMkeROe7sbuEkX+aUKDz1UA8Djj5/I174GL77Yz1t7eMK2O3fGp/2HitraxACMIGGF05NJO8ylqhuA03Ahwl8DjlHVZwstmI+IHOkFACwNLOsjIreJyG9EZFZXyWIcuITre7W2ZhYm6jt2R4xI3m8l2NRr5Eh347rnnqEMGpToZ1F1ZVyuvvplli1zyi2xGnGmw182TFZI2tvLvGFLX5l3VurDhjlL99BDXZ2wtWsTo8v8IqIi7gEjFnPVEvxSMT210VdGAe2quk9VN6rq86q6N9ODi8jNIrJNRJ4PLT9DRF4WkVdF5Oo0565X1XD02DRgqap+FcvQN7qIYH2vs89uSPln9rOn0zl2ly2Dm2920UWvvOKWPfRQTUefjrPOgqOOctOrV8f3q6mBKVNc0mN2tb3MYiks6ZX12287JbF9uxsiHTcu0TJpa3O3ZVUYPRpWrnQ+nHXr4Oqr3QPKmjVu+K0nKZRCN8e6FTgjuMCr9zUf+AIwGrjICzf+pIjcH3oNjDjuUOBtb9qCAYwuYe7cjRmHiYZDg8MlPILROqn8I/fd53wt4Patq5tAZaXb94Yb3L7OgjEl0TPI7/ewcaNTPj633eYeUFTjQ6U9pbyLaIFrYIvICOB+Vf2EN38S8ANVnezNXwOgqj9Oc5ylqjrdm74YaFLV+0XkLlWdkWT7y4HLAQYMGHDckiVLIo+9c+dO+vbtm3JZqnl/Ouo9G9Ltk4usmcjcnbKmmy6UrFHrcpX1N78Zx333DaZ373b27i3jrLMa+Kd/2gTAz38+kvvuG8xZZzXQ3FzGQw/542XRN6OKijbGjdvKiy8exgcflHPaadvo1QvWrDmc1tYYcad7+D8toXWmeLoO/7sIfy/5/Q5ElKVL19K/fyuQv3tAXV3dU6r66YyEUNWkL+DYVK+o/ZIcZwTwfGB+OvDbwPzFwK9T7H8YcD3wGnCNt6wPcAuwEJiVToZRo0ZpKlauXJl2Wap5fzrqPRvS7ZOLrJnInA35ljXddKFkjVqXq6xTp6qec85mfeYZ1TlzVKdOVa2sVHUDGNm9ystTry8raw8tC8/bq2tf7aH3QhxbNRZTHTlSVaRdZ8/u/FttaFA99VTV3//+zyl/q1H/A+BJ1czu9amiuX6WSgex/7W5kqlkjTyR6nbgitCyXcDf7+f5DaNL8EODx4wZ0lH2vrExMRzY93f4ZegHDoTGRiX8N3GlWZITi8GnPtXE00/3Dyw166N7iXbA55O2Nti0yZ0nGEbsh6t3Zfn8SGWihavJtRkYFpgfCjQU6FyG0SNobIQZM1ztp2A4sO94dRFiLsIndShqoqKprXV5KYmKxChtkiuoqiqYNs2FEQ8fPj7hAcTPiyovH58mpHz/yaQ2V7WIfFdEbvTmR4pILp0W/wqMFJFaESkHZuDyWAyjZAlmNAfDgWtr3cvvANnUBOec47LmJek9o/PC1193/ToSHbFRxr6mWW8UJ8qePfHS+lHl8xcvXl8wCTKJ5roFaCVein4z8G+ZHFxEFgNrgY+LyGYR+Yq6cixfx9X7ehFYoqobs5bcMIqAcIOlhQtdp8Wbb3ZRYfX17uXnnzQ0uEixiRPhi19MfsxYDD72MZevMHZsPAcmswxpp4xqa3fRp0/ePqbR7bjvdeFCZ6GEy+f7eVG+g74QZKJMjlLVnwJ7AVR1DxkOBKrqRapao6q9VXWoqt7kLV+hqqNU9ShV/dF+S28YPZzwE2KmXf7AZUsfc0yihSKiqMLf/R289Zbzy1RVuWGMRJL/RYcO3c2wYfDGG33YtWv/PpPRM6mqSvxtBS3gdHlR+SATZdIqIlXQUZvrKKBAo26GUVqEnxAz7fIHTlGMGuW6OQ72anb379/S0TrYx29LG41S5v3TN2+u5u23QTX182BZoTPQjBxQykN1JcvK2mlpSfxt5aN8fjZkUpvrWuBPwDARuQP4LHBpIYUyjFIi2GDJr8uVKQ88kJgdvX17JQsWJJbf8Ie5/HLqyZIgzzvPWUV3372P5ubw375z9FiqREqju5FO0X2qwuzZ2f228k1KZSIiAryEK19yIu4Xd6WqvpdqP8Mw4gSfCP0Q4Uypr08MJQ4WmAzS1FTO7NmwbRssXQq+gjj/fGhtbeD3vx/iKYhkf3lfkXRWKkbPxo/gmjZtLdOmnZx+hwKS0pj1klbuVdXtqvo/qnq/KRLD6DrClYVbW8uIxeDCCxOHuvxSL7t2uX2uu+4Z5sxxfpfXX+/Dhg2u+mzqKC5TJMVELEbH0FYhHeuZksnI6DoROb7gkhiGkZSwI3XNmujGSSNGuO1Xrx7I/Plu/rnnDuaGG1wvjTgaMW10P6m/j0MPbWHUKFefLew/604y8ZnUAV8TkTeBXXhFZlT1UwWVzDAMID5M5hp0xaO2ghnP7e3Jk9Qckqabo1kkPYvo70MExo9/jz/+MTF6b9WqAouUAZlYJl8AjsKVTzkLmOK9G4bRhfhRW34JllgsHgoaDkEuL29jxIhkFWWjy3yImNe9J3PqqS5Sa+vWioz3SdVWON9k0hzrzWSvgktmGEYCRx4Jjz46qKNpVlsb3HGHy6APhyDv3VtGdbUbU+/dO7MuDenChY3uZfVqpxwGDUrMzGhshCuvHJt0uMuvvHDbbcMLLp9FkxtGkVBfDwMGNNPLG5zu1ct1W4xKUmtqcvMLFmzg4IP9o3Qup1JWpowc6awZiOe0pMd8Ld2BP4TpW53z5jm/WNCHFq684O8zefL4gsllysQwioSaGjjppO20t/t+EteJMSpJraHBlW356leP54MP/KMkWh9+XsqmTdDa6rRUQ8ZlV9NZMsmUjSmgXCkvd3W2VOlQGKqS0M2zp9bmMgyjhxBsHZxJJI/vZ/FvKrFYO2eeCQMHNlNbCw89BIce2kpNTbwrZP6y35MpGxtKy5XW1hh33eUs0qhSPT21NpdhGF1MsK1vkGDr4PnzSVsiw8+O928q7e3C8OFw993rqa93+7//fjmHHupuOMHMeqO7iFtvgwYRGKJ0in7o0N1MmpSYg1Re3tapVE9PrM1lGEYXEyxZnytBa8a/qQTH1FWFF15w7+3t7gZUXu5uYuEaUMmoqrJaXvlmwABXFfrkk+Gii9z19RX9scc2sWKFm/YVxvz5GzpZql1dm8t+AobRg6iqgrq6CQkl64PO1v0haM34N5X58zcwYED8uNXVMHHiVt58092AWlrcTWzfvrii8EOSw+zZk00tL/OZpEd49114+22nEIIWhut5E9fwvsI4+uhdGVmqhcSUiWH0IMI+jmxK1mfDffcN5t13nSLwh0iqq/clVDP2b2IbNsCcOXD22c45X1kZL4vvF5dMzK438oWIK/b53e/CN74B3/ueezjoiWSSAW8YRhcR9nFkU7I+EyZP9jPl4xnUra0xYrHEJ16ILlB56aWu+nHv3m20tjpzZefOTCUwB3w2zJrl2vDOnRsf9rzggu6WKjlmmRhGDyNVxNb27eVJHfOpCO7jh4z6kVv+8NbmzZk/8QbH6S+5hBQdG21Ia/9w100E7rrLPWAEhz3r6ibkNOxZKMwyMYwexty5G5kwYQLQuWT9okXDO55QFyzI7HjBfS64wIWM+pFbweGtl15y2zc2wowZcPfdyS0i32KpqDi2U1+NRMwK2V8GD4Zzz3XDm4ceGm9BUF0NJ5+8ldtvH9TdInbCLBPDKAKqqtyT6vLlQzJ2zCfbp65uAjfe6KK6kjl0Aa6+2pXuuPrq+LJkocq+leM75svKXATYZz8Ll1zisvWlIPqkmCyezGXt1w8eeQTOOaeBE05wDxIrVtCpU2fYt9VTMGViGD2U4A28vp6E4SmAkSNTO+bD+/hDWu+846K6/FwVf3jLDxdetMjtf9ttcYWVLFTZT4xTjYetzpzptrv1VpetrxqOAsuHIigNiyesaD/8ECZOhAceOCLBX5UqmqsnYcrEMHoowRt4TY0bdmppid+ZN21yy6OsEz+pLdmQVjKiCj02NxMZqhy+0QUtl6amcubMgaeecsUoa2vhP//zbyl8LIWk51kzAwa4BwJfEftVoBcvXp/wIBHMFwkq/56GKRPD6GGEi/T5N3BVGDJkd6ebTyrrZOvWzkNaUVVmFy9ex9FHJy6rrXUdGqNClcM3umXLnEV14onO8f+977l19fXu9elPv8/FF4eljGrUFS5KmYtC6C5rJvl5Kyrcd3PiiU5ZV1S479cveZLPpNWuwpSJYfQwwkX6/Bv4O+/Accc10drqnmb9m0+q8fNlyzoPaSWrMgtu2GrfPjcdzHwfNIisQpXnzYP16+GFF/olvRlu3QqjRrlp/z2OJJmO7sFSrLS0uAeE229382ef7ZT9DTfkP2m1q7BoLsPoYYSL9AVv4H7Y8OWXu1yPxsbMj+s6NU7w5qSjU2N5+XhavBYZ48bBmWcmHt8fykp3Tnf84BJJ6Aa5Z08wz8Xxyituu+JHyeVz3HOPe6+ogFNO2cpf/jKIPXvcNZ02zeWa9HRMmRhGDyTqBp4qbDgd9fVw8cVbWbt2UEeY6dSpMG3aeuBkIDpRMbjMH8+/8spER3B9PfzDP8Dy5XQ08IrF4Jxz4sfavwZc8Rt1r14waNBu3nmnej+OU0j8z+XLGn5Pz8CB8PDD8L3v7WPPHrdsz578Jq0WEhvmMoweSDJfRK4Es+uDVWZTlSVPFhIc1b2vpsYNibV1BJwpbW1umX8zXLx4Hf36hc+SqS9E2bePgigSkf3xxyTbJ/uhOb/22bZtcOyxLpQ7yMKFNsxlGEYPwx8mGzduA08/fXzaYbKgI/iWWxKHsfzuff4QVmOjG64ZNgxOOgm2bdvGm28O6lBEbhjs5CRncTfcgw+GDz5I9iQfvjHHtykrc/6dU05xORr7y6mnbmPEiEE8+ihs3pzpXvkZngsWyWwLdVj2rcdiGOYyy8QwDiD8CsLpqsz6CY9BR3CiPyTevc+P7Jo3D5qaYMoUF8Z87bUvUl/vLKvTTnPhw6ec8m6kbK4bZLIbdDiaK75Ne7uT67XXXKLkZz+bwUVIwmOPDeSllwhl9OfLWnH4BTL79KGj9XIs5q61b3lUVcXbJxeiNlshMWViGEYn6uvpFCYcpqXFdfyrre2seIL9xn3r5r/+C158sdMYF7GYezSvqoKKin2BZL7ON+Z+/VzP+jCvv+6SLNety+pjAs66mTx5C+vXu6GmY47x12RjeSgHHwzf+95GT1EkKsBhw5xss2e7/BK/9bKqu34tLXELr7W1jGOOybybZk/BlIlhGAlUVbnaUK++mrhcJJ7NXlkZ7/jnZ9qHQ5lVJUHJLFoE27dXAK7y7QUXwEEHQVubUFnpbqhHHNGMduiKzv6HDz903SKTUVnpLKDqahg+POkmSWlvhwcfrOmY39iRE5idZdLcDGPHfsjQoW7eD6+urYW33or7v8aNC2e0u2sbt/yEjRth7Fi4+ebu7VGSDaZMDMNIwFcOQcUxciQMGRIvndLaGu/4F2wfGxyaWbx4HTNnJj/HkiUu6mviRFeLyr+x7tzZmzlzXFTTkCG7O3VwdPPJb/LNzfC//+ue7tetI0lyZGfi+TSpnOmZILS0wPnnn8i4ce4zPfGEex87lpQZ7Q0NrhFWVD/3YqEolImIHCkiN4nI0sCyc0XkNyLyRxGZ1J3yGUaxkiwb3lcOQcUxcSIcf3x0jahkZVX8fBmRxLa+wRtlOKly6dK1zJ/vznfccU2eleJu9GVlbjhs0qQtzJwZ3SrYVQpwPVbiQ1bJiftIMlEc8SGrSy6Bz3xme6ct2tvL+MMfXH2tYGfLdBnt6fq5FwMFVyYicrOIbBOR50PLzxCRl0XkVRG5Omp/AFWtV9WvhJbdq6pfBS4FLsy74IZxABCVDZ9MOaSqERUVyrx1q/MTTJvmb6lJcyf8J/cdO+IKqqmpnNGj3fShh8KYMe9zxRWwZ08vYrHUrYLb2+EPf4AXX3Q3/k98oinNlVB69+68LMzHPuZKwf/kJ3DIIU4TOR+P27a8vK2jvhZEl8ZJFuqbqp97MdAVocG3Ar8GFvkLRCQGzAc+D2wG/ioiy4EY8OPQ/l9W1W0pjv9d71iGYWRIYia6n6k+ocMJnC55MVP840yb5qyEF16A0aM73yj9J/dDDhnOSSf5Q2oDOuRraoKmpv68+CI88MBGvvzlCW6NV7Msit69XQXj2tpOmqKDWMyF5O7d23ldWZkLRHj7bXdd3nrLnXPuXHjuuUMA5xOpr3fbt7aW0atXPHdn/vwNfOc7x7Nzp9s/Vaivf61WrdrFZZdFf6aeimiqbyJfJxEZAdyvqp/w5k8CfqCqk735awBUNaxIwsdZqqrTvWkBfgI8rKqdIsxF5HLgcoABAwYct2TJksjj7ty5k76hJtbhZanm/emo92xIt08usmYic3fKmm66ULJGrStWWbdvL+faaz/O3Lkv079/a1L53nprL4sWfYrHHz+clpYYFRVtnHhiI9/4xlspkxgzlTO43imuWKf1fghssnWg1NTsYseOqg75xo3byvr1NVlm0UdloCu9erWzb19ZxHp/X1Ksj6Z37zYeemgNP/3pCB54wEUDlJe3s3dvGWed1cA//dOmyH170j2grq7uKVX9dEYfWlUL/gJGAM8H5qcDvw3MXwz8OsX+hwHXA68B13jLvgE85S2/ItX5R40apalYuXJl2mWp5v3pqPdsSLdPLrJmInM25FvWdNOFkjVqXbHKOnu2qki7zp6dWr4rrlAtK1MtL9+nZWWqZ5+9OW9yBtc3NKjOnKlaVaUK7n3WLNXGRrfu9NO3aHW1W5f81a4iqiNGfKQibj562/irTx/VZ55x566o2KegWl2tOnKkqoi7Tg0NqlOnZn7MsFzZ7hOLufPl47qmWpavewDwpGZ4n++uDPhUmUmdV6huB64ILfsl8Ms8y2UYRUtiocXkhRyD+GP0fjb8c88VpumS71x29aaUPXskwWfil3ipqICWFqVXL2HfPpdP0revCwdWhTfe8J+aO98+ysriPhR/euBA58Pxe7qA83ds8owC//pEOfJzYfz4bRx++EBWrGijpSWWMLxVTE71bOiuaK7NwLDA/FCgoZtkMYySICrfw3cGh/Gd5n42fCGaLjU2upv19df7S5wiCNabamoq5+KLnU9l6NDdHWXw29udIgkrj7IylyAI8W6FQWe8P/3mm/ESL5MmbeHhh12Isx/y7L+77TN5vo3P9+4NAwc2U1eXfJ9DD93LoEGJjcliMbjwwuJzrGdKdymTvwIjRaRWRMqBGcDybpLFMEqCqPDSTH0ghWDePPd+9NHRORRz526kuhqeeQba2soS8kwSb/za0USqf3+YMweeftpFaw0dGj8+OKXxzjvwjW/A9u3Q3FzGxIlw+unx+lfhOlidiaoR5pz127ZVsXKlv8QpkZNOeo85c5yCDDcmW7Om+BpeZUPBh7lEZDEwAThcRDYD16rqTSLydeBBXATXzaraM3tRGkYRER66yqbfST5J7J0SzKZXmpvjw1zh7Robq1iwwGV+T5rUxP33V1Ne3kZra4wRI3Zx7719ufbaBsrKhnREmd16q6t1FVQOfktjn8ceG4SIs2QGDXJW0OrVsG+fHxGWWz8Sf9+1aw9HFb797U1MmzaAVas2ceKJQxLqmoV7vJQKBbdMVPUiVa1R1d6qOlRVb/KWr1DVUap6lKr+qNByGMaBQHjoqrtKcdTXw+mnb+2wFmIxZy1cd90zCTkU4e2CxSP9Csfz529gzhwYNmxPQiJgkEmTEvupR6EK777rzt/eHm+XW129N0meSae9M/jkwrp1LhM+eC2KPbs9E6wEvWEYeSfYOyWYRX/ccR/wz/8cvV1LS1mH1eI3AvPzLlat2ogb5OjMTTfBccfFCyY2Nztrxfe/BEOE29tdvgvEc0t2706rScjGcmlvL0MkHvwQ1TmzlCiKciqGYRQfvmURrn4bbrgV3O7ssxvYsqVzmZdkZV+CzJvnthk92h1nzpygIoEoRRB33Cd3wMdiLoEy6LuRDHRKWZkybVo8+CFZRYFSwywTwzAKQrIWw6tWJdapWrAgcburrtrEhAlDmDMnXublggsSy75ccEH8HOG+83613crKuBUQDBuG+Hy/fk7h7N7thtc+9rFYR9iwz+DBztppa3OJiG1tsZQZ974F1N7ufDN+8EO+Kgr0ZEyZGIbRJYSd7cnyYKLKvITnfed1fT1861tw77109LVvbk5UMOEaXv68Czt2tLSUdVIkILz9dnzu5JPf45hjBvH66/CnPyUv41JWBtOnw5o1zWzZUgS9dvOIDXMZhtElhJ3tyfJg7rxzfYKzuqoKBg3a05GTUl0NEydu7XBeJyt//8UvJjq8e/WCgQNd2FRZWTy/xQ8KeOQR1xyrpiY+nJWMxx4bxIIFsHKlG/pKdMgrs2bBPfes5e673ecolj4k+cKUiWGUKMkq8XYnYWd7sjwYv2y9nyvT0gJVVe7dz52prt6X4LwO+yM++iiuYMANZW3b5jRIe3s8HFeVjtyTq69+mW3bUueeiDg/yOuvux4kiX4W4Y474KKLTgBcfbSedO27AlMmhlGi+L6J227Lou1ggYlyygcJl2L/6KPeCfPBPiqQvPy9f4yHH452mIsknn/SJNefPW6dJI5jqTo/yBFHwObNMGBAc6CXeztDh8atrEWLhve4a19ozGdiGCVG2DexfPkQRJw18MAD3ScXRDvlg4RLsZ9//tqMQoQbG2HGDLj77kSH9xe/CLffHg8N7tXLlTUJ18lascL1XrnxxrjllIjz2dxyi7NuTjppO/ffP8TbVmhrgxkzTvDCjYcAide+lBIUk2GWiWGUGKkSAUuZcDdDf5jvvfdgxIhdHT3s9+1Lnufh1/H60pec5TR58paEMi3h6xi0skaM2EVjI9TVbWPmTLdtsn1KGVMmhlFidE4YLCvJJDmfqiqSdjMcNswplxEjXPb87Nnw1FMuByXZ8Nq8edDU5I43Zozzo0yZQuR1nDt3Izfd5EKR/YrGDz1Uw513uuiwA+HaBzFlYhglSLJEwFIlXK7Ep60trlzWrBnAzTd3bisMTnnU1U3opIwmTx6f4NxPdh0v7GgYHu9TP3Sos2oOhGsfxHwmhlGCJEsELFXC4cEtLa5K8TvvxHNPTj55K7ffPijp/vX1cPHFW1m7dlDH9lOnwrRp65k27eSO7YLXMeyXipdqKWPzZti2bWBHHbFSvvZBzDIxDKPoCVoQs2e7mlvBEORwOHGQTJuOJLgAAAr0SURBVEKWw/h+qaqOvMREyySqhwx0LidTKpgyMQyj6AmHB48bl5h7Eg4nDpNJyHIQXwG1tCTW7VIVzjrLKaKoemLhQIFSwYa5DMMoOcK1sFJVHIbMQpbD+AropZegvn4XRx7Zl4MOamDLFjesFawntmBB5zpipdbXxCwTwzCM/WDu3I3Mnw+PPgq33PIkjz7qfCQPPBB36KtKh0NftbT7mpgyMQzDyCNRNcjeeKO0+5qYMjEMw8gjQYe+X0/MVxql3NfEfCaGYRh5xvenjBu3gaefPp7GRre8lPuamDIxDMPIM+GWwwcCNsxlGIaRhnRtgw1TJoZhGGkJhvkayTFlYhiGEUGwiGQwzLfqwOrImxGmTAzDMCIIF5HMJjfkQBsaM2ViGIYRQbCIZDjMNx0H2tCYRXMZhmGkwM8NCYf5RpFYUVhKrmxKFKZMDMMwUhBuI5yOqJL2111XWDm7GxvmMgzDyCOpMuBLGbNMDMMw8kxUBnwpY8rEMAwjz1gGvGEYhmHsBz1emYjIkSJyk4gsDS3vIyJPiciU7pLNMAzDcBRUmYjIzSKyTUSeDy0/Q0ReFpFXReTqVMdQ1XpV/UqSVf8CLMmnvIZhGMb+UWifya3Ar4FF/gIRiQHzgc8Dm4G/ishyIAb8OLT/l1V1W/igIjIReAGoLIzYhmEYRjaIqhb2BCIjgPtV9RPe/EnAD1R1sjd/DYCqhhVJ+DhLVXW6N/0joA8wGtgDTFXV9tD2lwOXAwwYMOC4JUuijZidO3fSt2/flMtSzfvTUe/ZkG6fXGTNRObulDXddKFkjVpXyrKmW5arnFHr0y0rxv9VMcma7W+1rq7uKVX9dEaCq2pBX8AI4PnA/HTgt4H5i4Ffp9j/MOB64DXgmtC6S4Ep6WQYNWqUpmLlypVpl6Wa96ej3rMh3T65yJqJzNmQb1nTTRdK1qh1pSxrumWZkI/vP0qWZHL1pN9qeL5YZM32two8qRne67sjNFiSLIs0j1R1O3BFxLpb8ySTYRiGkQPdEc21GRgWmB8KNHSDHIZhGEae6A5l8ldgpIjUikg5MANY3g1yGIZhGHmi0KHBi4G1wMdFZLOIfEVV9wFfBx4EXgSWqOrGQsphGIZhFJaC+kxU9aKI5SuAFYU8t2EYhtF19PgMeMMwDKPnY8rEMAzDyBlTJoZhGEbOmDIxDMMwcsaUiWEYRpHQ2AhXXjmWLVu6W5LOmDIxDMMoEubNg+eeO5i5c7tbks6YMjEMw+jhVFWBCCxcCKrCwoVuvqqquyWLY8rEMAyjh1NfDzNnQnW1m6+uhlmz4PXXu1euIKZMDMMwejg1NdCvHzQ3Q3l5G83Nbv6II7pbsjjdUTXYMAzDyJKtW+GKK2DcuA08/fTxNDZ2t0SJmDIxDMMoApYtc++rVu3issu6V5Zk2DCXYRiGkTOmTAzDMIycMWViGIZh5IwpE8MwDCNnTJkYhmEYOWPKxDAMw8gZUdXulqHgiMgHwKYUmxwMfJBmWap5fzr8fjjwXpbiJpMlX7KmkrknyJpuulCyRq0rZVmTLctW1nx8/9nI2pN+q8Uqa7a/1eGqOiAjqVW15F/AjdmuDy9LNe9PJ3l/sifJmkbmbpc13XShZI1aV8qyRizLStZ8fP9ZXsse81stVlnz/VsNvg6UYa779mN9eFmq+fsi3veHQsqaSub9Id+yppsulKxR60pZ1qj12ZCP7z+VLMXyvwpO93RZ8/1b7eCAGObqLkTkSVX9dHfLkQkma2EwWfNPscgJB5asB4pl0l3c2N0CZIHJWhhM1vxTLHLCASSrWSaGYRhGzphlYhiGYeSMKRPDMAwjZ0yZGIZhGDljyqSbEJEyEfmRiPxKRC7pbnlSISITRGSNiFwvIhO6W550iEgfEXlKRKZ0tyxRiMj/9a7nUhGZ3d3ypEJEzhWR34jIH0VkUnfLkwoROVJEbhKRpd0tSzK83+Zt3vWc1d3ypCLba2nKZD8QkZtFZJuIPB9afoaIvCwir4rI1WkOcw4wBNgLbO7hsiqwE6gsAlkB/gVYUhgp8yOnqr6oqlcAFwAFCx3Nk6z3qupXgUuBC3u4rPWq+pVCyZiMLOWeBiz1rufZXSlntrJmfS1zyXg8UF/AqcCxwPOBZTHgNeBIoBz4GzAa+CRwf+g1ELga+Jq379IeLmuZt98g4I4eLutEYAbuxjelp8rp7XM28BdgZk++poH9fgYcWySyFuw/laPc1wBjvW3u7CoZ90fWbK+lte3dD1R1tYiMCC3+DPCqqtYDiMhdwDmq+mOg03CLiGwGWr3Ztp4sa4AmoKIQckLermsd0Af3x90jIitUtb2nyekdZzmwXET+B7gznzLmU1YREeAnwAOquqEQcuZL1u4gG7lxlv1Q4Bm6YWQoS1lfyObYNsyVP4YAbwfmN3vLolgGTBaRXwGrCylYErKSVUSmicgNwO3ArwssW5isZFXVf1XVq3A359/kW5GkINtrOkFEfuld1xWFFi5Etr/Vf8RZfNNF5IpCCpaEbK/rYSJyPTBORK4ptHApiJJ7GXCeiCwkT2VM8kBSWbO9lmaZ5A9JsiwyI1RVdwNdOrYbIFtZl+H+BN1BVrJ2bKB6a/5FSUm213QVsKpQwqQhW1l/CfyycOKkJFtZtwNdrfCSkVRuVd0F/H1XC5OGKFmzupZmmeSPzcCwwPxQoKGbZEmHyZp/ikVOMFm7gmKSOy+ymjLJH38FRopIrYiU45zAy7tZpihM1vxTLHKCydoVFJPc+ZG1q6MJSuEFLAYaiYf1fsVbfibwCi4y4l+7W06T9cCW02Q1ubtSViv0aBiGYeSMDXMZhmEYOWPKxDAMw8gZUyaGYRhGzpgyMQzDMHLGlIlhGIaRM6ZMDMMwjJwxZWIY+4mI7Mxy+3NF5PsFkuVSEfm1N/11EelpJTuMEseUiWF0Hf8PWJDpxiIS28/z3Ax8Yz/3NYz9wpSJYWSAiNwrrnvjRhG5PLD8ZyKyQUQeFZEB3rJviMgLIvKsV84bERkFtKjqe978USKyTkT+KiJzfSvHqya8UkTuBJ5Lc+6/F5FXROQx4LP+cnVFRN8Qkc8U/soYhsOUiWFkxpdV9ThcV8RviMhhuL4pG1T1WOAx4Fpv26uBcar6KeJVVz8LBHuB/AL4haoeT+eiep/BlbQYHXVuEakBfugd9/O4/i1BngTG5/SJDSMLTJkYRmZ8Q0T+BqzDVVgdCbQDd3vrfwec4k0/C9whIl8E9nnLaoB3A8c7CbjHmw43xnpCVV9Pc+4TgFWq+q6qtgbk8NkGDM76UxrGfmLKxDDSICITcM2hTlLVMcDTQGWSTf1Cd38HzAeOA54SkV7Anoh9krErw3OnKqxX6Z3TMLoEUyaGkZ6DgSZV3S0i/wc40VteBkz3pmcCj4tIGTBMVVfiHO6HAH2BF4GjA8dcB5znTc/Yj3OvByZ4Q169gfND+40Cns/ycxrGfmPKxDDS8yegl4g8C8zDKQJwFsQxIvIU8DlgLhADficiz+GsiJ+r6vu41szjvH7qAFcB3xSRJ3BDYB9kc25VbQR+AKwFHiHRHwPOl/JILh/aMLLBStAbRhchIr8A7lPVR0SkGtijqioiM4CLVPWcPJ1nHPBNVb04H8czjEywHvCG0XX8O85xDs6f8mvPUnkf+HIez3M48L08Hs8w0mKWiWEYhpEz5jMxDMMwcsaUiWEYhpEzpkwMwzCMnDFlYhiGYeSMKRPDMAwjZ0yZGIZhGDnz/wFan5ldd+rZgwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "F\n",
      "======================================================================\n",
      "FAIL: test_gradient_wt_bias (__main__.TestGradient)\n",
      "Gradient check on all weights and biases in the network.\n",
      "----------------------------------------------------------------------\n",
      "Traceback (most recent call last):\n",
      "  File \"<ipython-input-10-df3fb507f5c7>\", line 42, in test_gradient_wt_bias\n",
      "    self.assertLess(results[0][2],tol)\n",
      "AssertionError: 9.641852316385773e-06 not less than 1e-07\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "Ran 2 tests in 25.064s\n",
      "\n",
      "FAILED (failures=1)\n"
     ]
    }
   ],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "class TestGradient(unittest.TestCase):\n",
    "    \"\"\"A test case for gradient checking.\n",
    "    \"\"\"\n",
    "    \n",
    "    def setUp(self):\n",
    "        \"\"\"Configure an Lstm network. Set up input vectors, execute a forward prop and a back prop \n",
    "        through the network to determine cell states, hidden states and gradients.\n",
    "        \"\"\"\n",
    "        self.verbose=False\n",
    "        length=3\n",
    "        ns=[32,16,8]\n",
    "        nio=[16,32,16,4]\n",
    "        m=3#batch size\n",
    "        \n",
    "        self.net=Network(ns,nio,length)\n",
    "                \n",
    "        self.c=[np.random.randn(ns[i],m) for i in range(len(ns))]\n",
    "        self.h=[np.random.randn(ns[i],m) for i in range(len(ns))]\n",
    "        self.x=[np.random.randn(nio[0],m) for i in range(length)]\n",
    "        self.dc=[np.random.randn(ns[i],m) for i in range(len(ns))]\n",
    "        self.dh=[np.random.randn(ns[i],m) for i in range(len(ns))]\n",
    "        self.dx=[np.random.randn(nio[-1],m) for i in range(length)]\n",
    "\n",
    "        _,_,_=self.net.fwd_prop(self.c,self.h,self.x)\n",
    "        self.dc_,self.dh_,self.dx_=self.net.back_prop(self.dc,self.dh,self.dx)\n",
    "            \n",
    "    def test_gradient_wt_bias(self):\n",
    "        \"\"\"Gradient check on all weights and biases in the network.\n",
    "        \"\"\"\n",
    "        tol=1e-7\n",
    "        results=self.gradient_check([item for wt in self.net.wts for item in [wt.W,wt.b]]+\n",
    "                                    [self.net.wts[-1].Why,self.net.wts[-1].by],\n",
    "                                    [item for wt in self.net.wts for item in [wt.dW,wt.db]]+\n",
    "                                   [self.net.wts[-1].dWhy,self.net.wts[-1].dby])\n",
    "        if self.verbose:\n",
    "            for result in results:\n",
    "                print('wt&bias {: e} {: e} {: e}'.format(*result))\n",
    "        self.__class__.plot([abs(result[0]) for result in results],[result[2] for result in results],\n",
    "                            'Gradient check - {} wts & biases'.format(len(results)))\n",
    "        self.assertLess(results[0][2],tol)\n",
    "        \n",
    "    def test_gradient_input(self):\n",
    "        \"\"\"Gradient check on all inputs to the network.\n",
    "        \"\"\"\n",
    "        tol=1e-7\n",
    "        results=self.gradient_check(self.c+self.h+self.x,self.dc_+self.dh_+self.dx_)\n",
    "        if self.verbose:\n",
    "            for result in results:\n",
    "                print('input   {: e} {: e} {: e}'.format(*result))\n",
    "        self.__class__.plot([abs(result[0]) for result in results],[result[2] for result in results],\n",
    "                            'Gradient check - {} inputs'.format(len(results)))\n",
    "        self.assertLess(results[0][2],tol)\n",
    "        \n",
    "    def gradient_check(self,par,dpar,delta=1e-5):\n",
    "        \"\"\"Gradient check on specified parameters.\n",
    "        \"\"\"\n",
    "        results=[]\n",
    "        for param,dparam in zip(par,dpar):\n",
    "            self.assertEqual(param.shape,dparam.shape)\n",
    "            for k in range(param.size):\n",
    "                old=param.flat[k]\n",
    "                param.flat[k]=old+delta\n",
    "                c_plus,h_plus,x_plus=self.net.fwd_prop(self.c,self.h,self.x)\n",
    "                param.flat[k]=old-delta\n",
    "                c_minus,h_minus,x_minus=self.net.fwd_prop(self.c,self.h,self.x)\n",
    "                param.flat[k]=old\n",
    "                diff=np.concatenate(c_plus+h_plus+x_plus,axis=0)-np.concatenate(c_minus+h_minus+x_minus,axis=0)\n",
    "                #grad_numeric=np.dot(np.concatenate(self.dc+self.dh+self.dx,axis=0).T,diff).item()/\n",
    "                #(2*delta)\n",
    "                grad_numeric=np.dot(np.ravel(np.concatenate(self.dc+self.dh+self.dx,axis=0)),np.ravel(diff))/(2*delta)#**WARNING\n",
    "                grad_analytic=dparam.flat[k]\n",
    "                rel_error = abs(grad_analytic-grad_numeric)/abs(grad_numeric+grad_analytic)\n",
    "                results.append((grad_analytic,grad_numeric,rel_error))                \n",
    "        return sorted(results,key=lambda result:result[2],reverse=True)\n",
    "     \n",
    "    def plot(x,y,title):\n",
    "        \"\"\"Plot y=f(x). Static method.\n",
    "        \"\"\"\n",
    "        plt.loglog(x,y,basex=10,basey=10,color='blue',marker='*',linewidth=0)\n",
    "        plt.grid(True,which='both',axis='both')\n",
    "        plt.title(title)\n",
    "        plt.xlabel('abs(grad)')\n",
    "        plt.ylabel('rel err')\n",
    "        plt.show()        \n",
    "                \n",
    "    def tearDown(self):\n",
    "        \"\"\"\n",
    "        \"\"\"\n",
    "        pass\n",
    "\n",
    "if __name__=='__main__':\n",
    "    #unittest.main(argv=['first-arg-is-ignored'], exit=False)\n",
    "    loader=unittest.TestLoader()\n",
    "    classes=[TestGradient]\n",
    "    suite=unittest.TestSuite([loader.loadTestsFromTestCase(cls) for cls in classes])\n",
    "    unittest.TextTestRunner().run(suite)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vBe4dAMqUdM4"
   },
   "source": [
    "---\n",
    "## Training the network on the Penn Tree bank (PTB) dataset\n",
    "A non-regularized LSTM network, as in [Zaremba et al. (2015)][8], is configured and trained on the [Penn Tree Bank dataset][7]. Training the network for 13 epochs takes about 10 hours in the absence of any GPU acceleration.\n",
    "\n",
    "[7]: http://www.fit.vutbr.cz/~imikolov/rnnlm/simple-examples.tgz\n",
    "[8]: https://arxiv.org/abs/1409.2329"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 885
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 36074443,
     "status": "ok",
     "timestamp": 1529300278199,
     "user": {
      "displayName": "Thomas Mathai",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "102416209494325007955"
     },
     "user_tz": 420
    },
    "id": "fI-a4634UdM7",
    "outputId": "ff9af7e6-154d-44a1-caf8-0419dc97bbac"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:['aer', 'banknote', 'berlitz', 'calloway', 'centrust', 'cluett', 'fromstein', 'gitano', 'guterman', 'hydro-quebec']\n",
      "INFO:root:['consumers', 'may', 'want', 'to', 'move', 'their', 'telephones', 'a', 'little', 'closer']\n",
      "INFO:root:['no', 'it', 'was', \"n't\", 'black', 'monday', '<eos>', 'but', 'while', 'the']\n",
      "INFO:root:size_train 929589, size_valid 73760, size_test 82430\n",
      "INFO:root:train vocab 10000, valid vocab 6022, test vocab 6049\n",
      "INFO:root:training: 1 epoch=2323 steps, used=929200 words, unused=389 words\n",
      "INFO:root:validation: 1 epoch=184 steps, used=73600 words, unused=160 words\n",
      "INFO:root:test: 1 epoch=206 steps, used=82400 words, unused=30 words\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                TRAINING\n",
      "time,s    epoch#    step#    epochs         loss               perplexity\n",
      "                                       train   valid       train         valid\n",
      "======    ======    =====    ======    =====   =====       =====         =====\n",
      "0:47:08        0     2322      1.00    5.30    5.26        200.47        192.82\n",
      "1:33:52        1     2322      2.00    4.91    5.14        135.03        170.94\n",
      "2:21:18        2     2322      3.00    4.68    5.02        107.52        152.13\n",
      "3:07:51        3     2322      4.00    4.70    5.05        109.51        156.80\n",
      "3:53:36        4     2322      5.00    4.34    4.84         76.65        126.85\n",
      "4:40:46        5     2322      6.00    4.04    4.80         56.72        121.38\n",
      "5:26:41        6     2322      7.00    3.83    4.80         45.84        121.19\n",
      "6:12:40        7     2322      8.00    3.70    4.81         40.37        122.38\n",
      "6:59:15        8     2322      9.00    3.62    4.81         37.45        123.18\n",
      "7:46:09        9     2322     10.00    3.59    4.82         36.09        123.40\n",
      "8:31:44       10     2322     11.00    3.56    4.81         35.31        123.06\n",
      "9:15:58       11     2322     12.00    3.55    4.81         34.90        122.49\n",
      "9:59:52       12     2322     13.00    3.55    4.80         34.69        122.08\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:test set perplexity after 13.0 epochs - 116.42378985085216\n",
      "INFO:root:execution time - 10:01:09s\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfgAAAFnCAYAAABKGFvpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd4VFX6wPHvnZZeSEhoCggrTZqI\nIChNLCA/FRSVRbErKiAoUpVdVlGUpnQRFVksIAgaUcrSQSGKIKJ0BWmpQAopJJmZ3x9JJmVKZpK5\nmczM+3mePMzcdt4bJvPec+655yhms9mMEEIIIXyKxtMBCCGEEML9JMELIYQQPkgSvBBCCOGDJMEL\nIYQQPkgSvBBCCOGDJMELIYQQPkjn6QCE8FbNmzenYcOGaLVazGYzoaGhvPLKK3Tp0kXVco8fP86I\nESMAuHz5MpcvX6Zu3boADBgwgKFDh7qlnObNm7N9+3bLsavq1ltvZdq0aXTs2NHhdpmZmbz77rvs\n2rULRVHQarUMHDiQJ554AkVR3BKLEP5AErwQVbBs2TJLAvzll194/vnnWb9+PVFRUaqVee2117J+\n/XoAVq9eTVxcHJ988olq5VUnk8nEM888Q9OmTfn2228JCAggMTGRYcOGkZ6ezksvveTpEIXwGpLg\nhXCTG264gYYNG7J//3569+7Npk2bmD17NtnZ2TRq1IgZM2YQFRXF3LlzuXTpEklJSRw5coRatWqx\nYMECYmNjGTJkCLfeeisbN27k7Nmz3HjjjcycOdOlmmt8fDzvvvsuderUQafTMXPmTLux5OXlMW3a\nNHbu3El+fj4PPvggzz33nOVYa9eu5ZtvviEzM5NnnnmGhx9+GID58+cTFxeH0WikadOmTJ8+nfDw\ncIfnVtrKlSv58ssvWbZsGYGBgZblO3bsICkpiWXLlqHX6wGoW7cu7777LmlpaQCcP3+eSZMmcfbs\nWfR6PU8//TT9+/fn7NmzDBo0iLvuuotDhw7x9ttvc8899/DCCy/w9ddfk5aWxuTJk7ntttuYO3cu\niYmJvPnmmwBl3q9bt4758+djNBrR6XS89tprdO7cuXIfCiE8SO7BC+FGBQUFGAwGzpw5w9ixY5k5\ncyabN2+mc+fOTJ482bLd+vXrmThxIps2bSI6OpqvvvrKsm7Lli0sWbKEDRs2sGfPHvbt2+dyHIcO\nHWLQoEHMnDnTYSyLFy/mxIkTfPvtt6xdu5YNGzawdetWy3HOnz/Pt99+y0cffcQ777zDxYsX+f33\n3/nss8/46quv2LhxI3l5eXz66adOnRvA3r17Wbx4MQsXLiyT3AF++uknbr75ZktyL9awYUPatm0L\nwKRJk+jUqRMbNmxg0aJFTJkyhbNnzwKQlpZGy5YtLfFkZWWhKApr165l2rRpvPbaaxQUFDj83f3n\nP/9h0aJFrFu3jn//+99s2bLFhd+8EDWHJHgh3GT79u2kpqbSoUMHduzYQadOnWjWrBkAgwYNYsuW\nLRiNRgA6duxIgwYNUBSFli1bkpCQYDlOnz59CAwMJDg4mMaNG5dZ56zAwEBLXwBHsWzdupXBgwdj\nMBgIDg7m3nvvZePGjZbj9O/fH4CmTZvSpEkTfv/9d1q3bs22bdsIDQ1Fo9Fw/fXXc+bMGcs+js4t\nISGBCRMmMHfuXGrXrm0Vd3p6OtHR0XbPKz8/nx9//JHBgwcD0KBBAzp37syePXss62+//fYy+wwc\nOBCArl27UlBQwN9//+3wdxcdHc3y5cs5d+4cHTt2ZMKECQ63F6KmkiZ6IapgyJAhlk52DRo0YPHi\nxYSEhJCZmcnevXvp06ePZdvQ0FBLM3NYWJhluVartST+4u3srXNWRESE5bWjWDIzM5k6dSqzZs0C\nIC8vz1JTBqhVq5bldVhYGBkZGeTk5DB16lTi4+OBwqTcs2fPMtvZi/+tt95Co9HYTeK1atUiOTnZ\n7nmlpaVhNpvLlBEeHs7Fixct5ZX+/SmKUuZ3ER4eTnp6ut3jAyxcuJCFCxdy3333Ua9ePSZOnEin\nTp0c7iNETSQJXogqKN3JrrTY2Fi6du3KnDlzPBCV87HExsby5JNP0qtXL5v7pqenc/XVV1teR0RE\nsHTpUk6dOsXq1asJCQnh3XffJSkpyalYxowZw8mTJ/nXv/7FggULrNZ37tyZ8ePHk5ubW6b5/vTp\n02zevJkhQ4ag0WgssUBh0rd3wWA2m7l06ZLlQqV4P41Gg8lkKnOexRo2bMjUqVMxmUx8/fXXjB49\nmp07dzp1fkLUJNJEL4QKbrnlFvbu3Wtpuv7tt9+YMmVKjYuld+/erFy5EqPRiNlsZsGCBezYscOy\n79q1awH4888/OX36NG3atOHChQs0adKEkJAQzp07x/bt28nOznYqloYNGzJixAhOnz7NmjVrbMba\npEkTxo4dy+XLlwFITExk1KhRFBQUoNPpuOWWW1ixYgVQmPj37t1L165d7ZZZfA67du0iMDCQa665\nhtjYWI4dO4bJZOLixYuWc7548SJPPPEEly9fRqPR0K5dO3k0T3gtqcELoYLY2FjeeOMNhg0bRn5+\nPiEhIUycOLHGxTJ48GDOnj1Lv379MJvNtG7dmscee8yyb4MGDbj33nvJyMjg1VdfJTIykkGDBvHi\niy9y55130rx5c8aPH8+IESOcflTPYDDw9ttv8/TTT9OlS5cyLSCKovD+++/z7rvv0r9/f3Q6HUFB\nQTz88MOWe+n/+c9/eO2111i9ejV6vZ4pU6ZQr149S0e70rRaLfn5+fTr14/09HSmTJmCRqOhT58+\nxMXFcdttt9GkSRP69OnDhQsXiIqKolu3btx///1otVr0er2lp70Q3kaR+eCFEL7o7Nmz3HHHHRw6\ndMjToQjhEdJEL4QQQvgg1Zro4+PjGTlyJNdeey0AzZo14+mnn2bs2LEYjUZiYmKYPn06BoOBuLg4\nli5dikaj4cEHH+SBBx5QKywhhBDCL6jWRB8fH89nn31WpufuhAkT6N69O3379mXWrFnUrVuX/v37\nM2DAAFatWoVer2fgwIF8+umnREZGqhGWEEII4ReqtYk+Pj6e3r17A9CrVy92797NgQMHaNOmDWFh\nYQQGBtKhQ4dKjdwlhBBCiBKq9qI/ceIEzz33HOnp6QwfPpycnBwMBgNQOFpUSkoKqampZSbmiIqK\nIiUlRc2whBBCCJ+nWoJv3Lgxw4cPp2/fvpw5c4ZHH320zIhW9u4MOHPHwGw2o2jsND5cfTWcOQPy\ncIAQQgg/plqCr1OnDnfddRdQOLhF7dq1OXjwoGWEqqSkJGJjY4mNjSU1NdWyX3JyMu3bt3d4bEcD\nTxjNoAVSUjLdch41UUxMmE+fX0X8+fz9+dxBzl/O33/PPyYmrOKNylHtHnxcXBwfffQRACkpKVy4\ncIH77ruPDRs2ALBx40a6detGu3btOHjwIBkZGWRlZbFv3z46duxY9QCkBi+EEMKPqVaDv/XWW3nl\nlVfYvHkz+fn5TJ48mZYtWzJu3DhWrFhB/fr16d+/P3q9ntGjR/PUU0+hKArDhg0rM5GEEEIIIVzn\nvSPZ2WmmN151NdqzZ0hJSre7jbfz52Yq8O/z9+dzBzl/OX//Pf8a1UTvcXl5no5ACCGE8BjfS/BF\nDRJBn3zo4UCEEEIIz/G5BK89VzSjVG6uZwMRQgghPMjnEnwxxUu7FgghhBDu4LMJXgghhNi2bbNT\n282ePZPz58/ZXT9+/MvuCqnaSIIXQgjhkxISzrNp0wanth05cjT16zewu/7tt2e5K6xqo+pY9B4l\nTfRCCOHXZs16h8OH/6Bbtxu5446+JCSc5733FjB16uukpCSTk5PDk08+y803d2P48Gd5+eWxbN26\nmaysy5w+/Tfnzp3lxRdH06XLzfTr15vvvtvM8OHPcuONndm3by9paWm888671K5dm9dfn0RiYgJt\n2rRly5ZNrFnzvadP34dr8JLghRDCr/3zn0No374Djz/+NAUF+SxY8CFZWZfp1Okm5s37gNdfn8pH\nHy2y2i85OYkZM+YwcuQrxMWttlofEhLC7NkLuemmruzYsYU9e34kL+8KH3zwCR063Ehqas2YMM13\na/BCCCFqlFrdO6M7crhKx4gp9bqgRUsu7Yh3ar+WLa8DICwsnMOH/yAubjWKoiEjI91q27ZtC+dD\niY2N5fLly1br27W73rI+PT2dv/8+SZs27QDo0uVmtFqtK6ekGt9N8FKDF0KIGsXZZGxPVUay0+v1\nAPzvf+vJyMhg/vwPycjI4Omnh1htWzpB2xrstfx6s9mMRlO4TFEUhxOiVSffbaIXQgjh1zQaTZlp\nygHS0tKoV68+Go2G7du3kJ+fX+VyGjS4iqNHDwHw0097rMr0FEnwQgghfFKjRtdw9OgRsrJKmtl7\n9ryVH3/cyciRzxMUFERsbCxLliyuUjldu3YjKyuL559/igMH9hMeHlHV0N3C5yabKZY17lWyR4+r\npmCqlz9PuAD+ff7+fO4g5y/nXzPPPyMjnX379tKzZ29SUpIZOfJ5Pv/8K7eWUZnJZuQevBBCCFEF\nwcEhbNmyic8/X4bZbGLEiJoxKI7vJnghhBCiGuh0Ol5/faqnw7Diu/fgpQYvhBDCj/lughdCCCH8\nmCR4IYQQwgf5boKXJnohhBB+TBK8EEIIvzZw4N1kZ2ezbNkn/P77b2XWZWdnM3Dg3Q73L56S9vvv\nv2X79q2qxekq303wgJJ2Cf1W5+YCFkII4Vlr1ujo0SOYevVC6dEjmDVrqvdBryFDHqd167Yu7VN6\nStq77rqbHj16qRFapfjuY3JmM8Fz3iV43nukJGd4OhohhBAOrFmjY+jQIMv7w4e1Re9zGDCgoFLH\nfPLJh3nrrZnUrVuXxMQEJkwYTUxMLDk5OeTm5vLSS2No1aq1Zfs335xMz569ad/+el59dSx5eXmW\niWcANm5cx6pVK9BqNTRu3JRx4161TEm7ZMliTCYTkZGR3H//QyxYMJuDBw9QUGDk/vsfpE+ffjan\nmq1bt26lf2cV8ekavBBCCO/w3nsGm8tnz7a93Bndu/fihx92ALBz53a6d+/F//1ff+bOXcRzzw3n\ns8+W2txvw4Z1NGnSlAULPuTaa5tZlufk5DBz5lwWLvyY06dP8eefJyxT0j7xxDOW7X79dR9//fUn\nCxd+zJw57/Pxxx+QnZ0FWE81qybfrcHXkNl8hBBCFOrePZgjR1ybSvXQIS2xsaWHaS153aKFkR07\nsh2U14t5897j/vsfZNeu7Qwf/hLLly/jiy+WkZ+fT2BgoM39Tp36i/btbwDg+utvsCwPDw9nwoTR\nAPz990nS09Ns7n/kyCHat+8AQFBQEI0bN+HMmTOA9VSzavLdBC+d7IQQokZxlIx79Ajm8GHr5N+q\nlZFt2wr3c3Us+iZNmnLhQgpJSYlkZmayc+c2ateOZdKkNzhy5BDz5r1ncz+zGTSawkqiyVSYS/Lz\n85k1axqffPI50dG1GTt2lN1yFUUpk4IKCvItx6toKlp38uEmeknwQgjhLUaNyrO5fORI28ud1aXL\nLXzwwQK6detBenoaDRpcBcD27VspKLB9b79hw0YcOXIYgH379gKQnZ2FVqslOro2SUmJHDlymIKC\nAptT0rZocR379/9StF82586d5aqrGlbpPCrDZxO87uhRaaYXQggvMWBAAYsW5dCqlRGdzkyrVkYW\nLap8B7tiPXr0YtOmDfTs2Zs+ffqxYsVnvPTSMK67rjUXLlzgu+/irPbp06cff/xxkJEjn+fMmb9R\nFIWIiEhuvLEzTz/9KEuWLGbw4CHMmTPLMiXtnDkzLfu3a9ee5s1bMGzYM7z00jCee244QUFBVuWo\nzWeni81vfz353XoSPPddn+tFX1OnTKwu/nz+/nzuIOcv5++/51+Z6WJ9tgYvtXchhBD+TBK8EEII\n4YN8N8GbzZLkhRBC+C2vTfBZo17xdAhCCCFEjeW1CT7/pq6ON5DauxBCCD/mtQm+QpLghRBC+DHf\nTfCUJPiY2HAPxiGEEEJUP99N8IoitXghhBB+y2sTvDkiwtMhCCGEEDWW1yZ4owfG9RVCCCG8hdcm\n+Ap56Qi8QgghhDv4boIHzOXuwdduVAfD+u89FI0QQghRfXw2wet/+dlqmZKTg65o6j8hhBDCl/ls\nghdCCCH8mU8neP1vv3o6BCGEEMIjvDfBa7UVbqKkp1dDIEIIIUTN47UJXp6DF0IIIezz2gQvhBBC\nCPskwQshhBA+SBK8EEII4YMkwQshhBA+SBK8EEII4YMkwQshhBA+SNUEn5uby2233cbq1atJSEhg\nyJAhDB48mJEjR5KXlwdAXFwc999/Pw888AArV65UMxwhhBDCb6ia4BcuXEhE0fPqc+bMYfDgwXz+\n+ec0atSIVatWkZ2dzfz58/nkk09YtmwZS5cuJS0tTc2QhBBCCL+gWoL/888/OXHiBD179gQgPj6e\n3r17A9CrVy92797NgQMHaNOmDWFhYQQGBtKhQwf27dvnXAEaF0M3mQDQ/7THtf2EEEIIL6Ragn/n\nnXcYP3685X1OTg4GgwGA6OhoUlJSSE1NJSoqyrJNVFQUKSkpzhXgxFC1lJouNuiDBQAYftzl3PGF\nEEIIL6ZT46Bff/017du35+qrr7a53mw2u7TclpiYsAq30etLLgJC439wad+azhfOoSr8+fz9+dxB\nzl/O37/P3xWqJPht27Zx5swZtm3bRmJiIgaDgeDgYHJzcwkMDCQpKYnY2FhiY2NJTU217JecnEz7\n9u2dKiMlJZOYijbavbvk9XffldnXm8XEhHn9OVSFP5+/P587yPnL+fvv+VfmwkaVBP/ee+9ZXs+d\nO5cGDRqwf/9+NmzYwL333svGjRvp1q0b7dq147XXXiMjIwOtVsu+ffuYOHGiGiEJIYQQfkWVBG/L\niBEjGDduHCtWrKB+/fr0798fvV7P6NGjeeqpp1AUhWHDhhEWJs0vQgghRFUpZldufNcgKSmZxMSG\nV27f5Aw3R1O9/LmZCvz7/P353EHOX87ff8+/Mk30MpKdEEII4YMkwQshhBA+SBK8EEII4YMkwQsh\nhBA+SBK8EEII4YMkwQshhBA+SBK8EEII4YMkwRdRkpLQ/n7Q02EIIYQQbuHVCT5nyBNuO1b40CeI\nuvVmtx1PCCGE8CSvTvCVlp0N5QbwU/LyPBSMEEII4X5+meBjGtclYMXnng5DCCGEUI1fJngA7em/\nPR2CEEIIoRq/TfBWvHPOHSGEEMImSfBCCCGED5IEL4QQQvggSfBCCCGED5IEL4QQQvggSfAW0slO\nCCGE75AEX0x60QshhPAhkuCFEEIIHyQJXgghhPBB/pvgFcXTEQghhBCq8d8EX57cgxdCCOFDJMEL\nIYQQPkgSvBBCCOGDJMELIYQQPkgSfDG5By+EEMKHSIIvJvldCCGED5EEL4QQQvggSfDlRP7fHZ4O\nQQghhKgySfDl6H/a43C9kpyMcuFCNUUjhBBCVI4k+GJOdrKL7tSOyH63qRyMEEIIUTX+m+ArOVSt\nkp2FJiXFzcEIIYQQ7uW/CV4IIYTwYV6d4M2hoaocN2jRfDCZVDm2EEIIUR28OsFnjXtVleOGTpqA\nkpqqyrGFEEKI6uDVCZ7gYEwhbqrFy0h2QgghfIh3J3h3ciXBy8WAEEKIGs7vE7x+yyZiYsM9HYYQ\nQgjhVl6f4C+/M7PS+4aOH03koPvcGI0QQghRM3h9gr/y4D8rva9+1w7La0Wa3YUQQvgQr0/wlRUy\n7S2UzExPhyGEEEKowm8TPIA24bz9lY5q9JUcBU8IIYSoLn6d4CtNmvOFEELUcJLgi+j+OOjpEIQQ\nQgi3kQQvhBBC+CBJ8EIIIYQPkgQvhBBC+CBJ8HYELf3I7jrN5UyUixeqMRohhBDCNZLg7dDH73a4\nXnPxYjVFIoQQQrhOErw98iicEEIIL6ZT68A5OTmMHz+eCxcucOXKFV544QVatGjB2LFjMRqNxMTE\nMH36dAwGA3FxcSxduhSNRsODDz7IAw88oFZYTtPt34eSnAwBBswRkYQ/Ogi0qv26hBBCCLdSLWNt\n3bqV1q1b88wzz3Du3DmefPJJOnTowODBg+nbty+zZs1i1apV9O/fn/nz57Nq1Sr0ej0DBw7k9ttv\nJzIyUq3QnKLJukx0u+YUtG5L2v+2E7D+e8xarUdjEkIIIZylWhP9XXfdxTPPPANAQkICderUIT4+\nnt69ewPQq1cvdu/ezYEDB2jTpg1hYWEEBgbSoUMH9u3bp1ZYLlGMRjRJiZ4OQwghhHCZ6m3OgwYN\nIjExkffff58nnngCg8EAQHR0NCkpKaSmphIVFWXZPioqipSUlAqPGxMTplrMpWk1iqWs0iPQR0WF\nQDXFYEt1nX9N5c/n78/nDnL+cv7+ff6uUD3BL1++nMOHDzNmzBjMpTqume10YrO3vLyUlJKZ4GKq\nFqJDRpOZiymZxABmSpL8xYtZGFM8MxtdTExYmfP3N/58/v587iDnL+fvv+dfmQsb1Zrof//9dxIS\nEgBo2bIlRqORkJAQcnNzAUhKSiI2NpbY2FhSU1Mt+yUnJxMbG6tWWK4rPXOczCInhBDCS6iW4Pfu\n3cvHH38MQGpqKtnZ2XTt2pUNGzYAsHHjRrp160a7du04ePAgGRkZZGVlsW/fPjp27KhWWNUu+J03\nqdW5vafDEEII4WdUa6IfNGgQr776KoMHDyY3N5d//etftG7dmnHjxrFixQrq169P//790ev1jB49\nmqeeegpFURg2bBhhYTX0HkslavCGXTvQnfxLhWCEEEII+1RL8IGBgcycOdNq+ZIlS6yW9enThz59\n+qgVStVIE70QQggvJCPZCSGEED5IErwKDJs2gNHo6TCEEEL4MacS/BdffMHly5fVjqVG0p4/Z3mt\n5OU5tU/E4AfQ/xyvVkhCCCFEhZxK8EePHuWee+5h3Lhx7N27V+2YhBBCCFFFTnWymzx5MiaTifj4\neOLi4pgxYwa9e/fmwQcfJCIiQu0YhRBCCOEip+/BazQaGjZsSN26dcnLy+OPP/7g4YcfZtOmTWrG\n5/1k2lkhhBAe4FQNfs2aNaxevZq0tDQeeOABlixZQkREBBkZGTzyyCPcdtttasfpUF7PWzFs2+K5\nAK5cQcnJxhxZq2SZSok9sm9veOoJGPiIKscXQgjhG5xK8D/88AMjR44sM8Jcbm4u4eHhPPbYY6oF\n5yxzqGcHxgl7eQSBK5eTkpxhvdLNz87rf/kZYqIlwQshhHDIqSb6S5cuWQ0f+/DDDwNw//33uz+q\nGkZJu+Rwvfb033bX6eN3O1dGRjpKRrpLcQkhhBD2OKzBx8XFMX/+fBISEujZs6dleX5+PrVr11Y7\nthoj8MsvbK8wmdCkJLuljMjbe6CYTFz8+Te3HE8IIYR/c5jg77nnHvr168err77KiBEjLMs1Gk3N\nmvHNQwKXf0bYqGHkd+5S5WPJePVCCCHcyWET/aFDh9Bqtdx7772cPn3a8nPq1Cl++umn6oqxYh4a\nI165cMH+Shud7EJfGo5y0cE+QgghhJs4rMF//fXXtGrVigULFlitUxSFLl2qXnP1CU72mA/67L/k\n3fV/5N1eQyfWEUII4TMcJviJEycCsGjRIoKDg8usS0pKUi8qb5Cfb3mp/2lP9ZYtz9YLIYSogFO9\n6AcOHFhmiNpvvvmGRx7x78e0onrcJNPHCiGEqLGceg5+3rx5vP766zRv3pyEhAT0ej3Lly9XO7Ya\nL/T1SVbLtH/8XvjCXi1bat9CCCGqgVM1+CZNmvDiiy+ybt06jh8/zosvvkh0dLTasTkt+8WXPB2C\nRcisaZXazyytAUIIIdzIqRr8pEmTOHXqFJ9++ilpaWm89NJL3H777Tz//PNqx+eUgnbXq1tANde6\ntcePYby2mf0N5GJACCFEBZyqwTdt2pT//ve/NGzYkLZt2/r1/PAVKtX5rlKys4m6uWPF2wkhhBAO\nOJXgH3/8cbZv386nn34KFPagf+WVV1QNzFsFrP/O8QZ2GgOU4lYCk6niQuQ+vhBCiAo4leCnT5/O\nqlWrWL16NQDffvstU6ZMUTWwmkT/w07ntvtxV8kbScJCCCE8yKkE//PPPzNv3jxCQkIAGDZsGH/8\n8YeqgdUkmkuOJ5spFtn/LpUjEUIIIZzjVIIPCAgACkevAzAajRiNRvWi8kOWXvTSgU4IIYQbONWL\nvkOHDkyYMIHk5GSWLFnCxo0b6dSpk9qx1RgyfrwQQghv41SCf+mll1i/fj2BgYEkJibyxBNPcMcd\nd6gdW42hO37M9Z3kHrwQQggPcpjgz5w5Y3l93XXXcd1115VZd/XVV6sXmb+SJnohhBBu4DDBP/bY\nYyiKgtlGbVRRFDZv3qxaYL5GSU0FIGLIQ6QkZ3g4GmtKairm4GAoN6mQEEII7+QwwW/ZsqW64vB5\n0W0djExXA9Ru1YQrd/cn46P/ejoUIYQQbuDUPfgTJ04wZ84cTpw4gaIoNG/enBdffJHGjRurHJ6X\nKzWqnVJQ4HhbRfH4fXtNwnmPli+EEMJ9nHpMbvz48XTv3p158+YxZ84cbrrpJsaOHat2bC659L/t\nng7BSuiYUa7vJPfghRBCuIFTNfigoCAGDhxoed+0aVM2bNigWlCVofqEM5WgO3Hc+Y2l170QQgg3\ncqoGf9NNN7Fp0yZycnLIyspi8+bNXH/99ZjNZkzOjJ0uvIO0HgghhM9wqga/YMECmyPXzZs3D0VR\nOHz4sNsD8wku1MqLJ5tRsrPdelxP0yQmYA4NxRwa5ulQhBDCrziV4A8ePIhG41RlX1SCJinR8rp2\ny2s8GIn7RbdtzpU7+pDx6ZeeDkUIIfyKU1n7scceUzsOtyho1drTIZRwpZadl+fWog3rv6/cjio1\n0WtSU1Q5rhBCCPucqsG3bNmS2bNnc/3116PX6y3Lu3TpolpglZH18hginq5BFyMeakqPeHQQKecv\ngs6p/94Scg9eCCF8hlMZoPge+969ey3LFEWpcQneHBPr6RCqhwuJWPvXCTCbMTa9VsWAhBBC1DRO\nJfhly5YBYDabLVPGCic4U4PPzkZzXr0BZmr16IJy5Ypnh8f1ok6BQgjhK5y6B3/kyBHuu+8++vbt\nC8D8+fM5cOCAqoF5PbMZJTu5Q1kXAAAgAElEQVSrws1C/zWRWnerMDNfcVItNZqefsv/HCZbs1y8\nCSGEz3Aqwb/++uu89dZbxMTEAHDXXXcxdepUVQPzemYzusOHKtxMU5m55p2pEdvYJnLQ/Shpl1wv\nrxTt8WNlLhqEEELUTE4leJ1OR4sWLSzvr7nmGnSuduAS1cPGeAVOc6IGH3VzR4I+WlT5MoQQQlQL\npxP8mTNnLPfft2/fbnMKWY+rQTE5/WiYvaRayXMJfXlE2f1daXZ3clslq+JbD0IIITzLqWr4uHHj\neOGFFzh58iQ33HADDRo0YNq0aWrH5tWC351epf0NG9eTd2dfl/fTHfyt8IW9C4QadBEkhBBCPQ4T\n/OXLl5k/fz4nT57k3nvv5b777sNgMBAaGlpd8Xkt3fFjTm1nr2Ob5txZd4bjNN2v+6h1R0/P9roX\nQghRZQ6b6CdPnoyiKDz00EP8+eefLFu2TJK7G8TEhqM9UjR+f2V6rm/caHdV8Zj2lW2i1x476no8\nFXFjq0F0s4Zw1jMXP0II4U0c1uDPnTvHjBkzAOjevTuPP/54dcTkFzSJCRhbtLS/QU1uSvdgbJq0\nNDh6FNp28lgMQgjhDRzW4Ev3lNdqtaoHU1XGuvU8HYLzLDVsuxu45/huPqwQQgjv4DDBlx+1rqaP\nYmdq0tTTITivgib0gLVxVss0p07a3Db8scHU6t7ZueM7qn07+/+rwucg8s6e6LdtcftxhRDCXzls\not+/fz89e/a0vL9w4QI9e/a0DFm7bds2lcPzbcrlTLvzvxt+2Gm1LOTtN8q8D/jyCwJXfI5h5/aS\nY2Zmlt3JleZ0D17A6ffvw7B1M/k9b/VYDEII4UscJvj169dXVxx+yEztJg1c2kPJvVLmfcDab8ok\ndwDt6VNFh7ed2AOXf0bO8JEulWvF0/0DPF2+EEJ4AYcJvkED1xKQUJkLiU3BXHi7vahWHrRwHgCh\nr09ynOBVqMXLwDhCCFH9VB1vdtq0afzyyy8UFBQwdOhQ2rRpw9ixYzEajcTExDB9+nQMBgNxcXEs\nXboUjUbDgw8+yAMPPKBmWDWCUolaqHLZ9eZ3pWjc+KAPFjhTgssxOcPZMQFkPnohhHAf1RL8nj17\nOH78OCtWrODSpUsMGDCALl26MHjwYPr27cusWbNYtWoV/fv3Z/78+axatQq9Xs/AgQO5/fbbiYyM\nVCs0rxT5f3eg/2lPpfdX8vLcGI2HyYWAEEJUyKmx6CvjxhtvZPbs2QCEh4eTk5NDfHw8vXv3BqBX\nr17s3r2bAwcO0KZNG8LCwggMDKRDhw7s27dPrbBqjKCF8yveyGSyTB7jcnIvV7vXpCRXar9iMbHh\nBHz1pWsxlJaVRfiQhxxvI4lbCCHcRrUavFarJTg4GIBVq1bRvXt3du3ahcFgACA6OpqUlBRSU1OJ\nioqy7BcVFUVKSsUTtcTEhKkTeDUx7Nha4TYxQx+Dv/6C336zXhcTBnqt9bIitaNDIcL278jqd7d3\nb2FMAToM4UG2twHCTxY2tYeEBBDi4u8/JusCbFjn8P8tOCSAYGeOazZ7/f9/VfjzuYOcv5y/f5+/\nK1Sf83XTpk2sWrWKjz/+mDvuuMOy3N5sdM7OUpeSkmlzeYzrITptOQ/xFhM5RCtacYiJvMUgVqhW\nnmnXLjQXLpB64gy1y61LSckkPK+AgHLLis8/NTUTc57G5u+j/O8uYO8BwgHjsRNc2fsrwTa2iQGy\ns/MIBvLWbyT9+ZecOofi8i9ezCLKxnHLHz/Lzvryx7N3HF8XExPmt+cOcv5y/v57/pW5sFE1we/c\nuZP333+fDz/8kLCwMIKDg8nNzSUwMJCkpCRiY2OJjY0lNTXVsk9ycjLt27dXM6xKWc5D/JPllvcH\naWt5r1aS11y4AID2rz9tb+DoYsjJC6Ww558msKjpXXv6FMFzZlW4j2H3D04d22WKUrkx9IUQQlhR\n7R58ZmYm06ZNY9GiRZYOc127dmXDhg0AbNy4kW7dutGuXTsOHjxIRkYGWVlZ7Nu3j44dO6oVVqW9\nxUSby6cyoZojKWI0oly+bH+9kwleX+45+irLyiJk4phK7x6weiUxdSLcGJAQQvgn1Wrw33//PZcu\nXWLUqFGWZW+//TavvfYaK1asoH79+vTv3x+9Xs/o0aN56qmnUBSFYcOGERZW8+6xHKKVS8vVFvbC\n0xXWpENHDXNvoU5cNOiOHCL4w0VkvTXd9eMrClpnHqmT2r0QQlRItQT/0EMP8dBD1r2mlyxZYrWs\nT58+9OnTR61Q3KIVhzhIW6vl4aRzgLa0w7ojnJr0+35xvIHZTNDnyyo8jjY5yU0RuYGiEOhEzDKS\nnRBCVEy1JnpfM5G3bC6/lS304zu68gPLeITcMt3e3MOw+X9Wy5T0NLeXU6H8KjxL72StW5uYUPky\nhBBCWEiCd9IgVvAFg2jLAXTk05YDfMEgVvIQp2jMWKbxKY9wNWcYwzT+pInbyg6ZZn1xoUmzTvCG\n9d+XvDGbMVehKVt79Ij1svPnbW4bcW9fQkcXDX9rr8wq1roj+92O5vy5Kh1DCCH8ic8l+Lybu6l2\n7EGs4ADtycfAAdpbes/rMNKfb9hAH36kK2YUbmIPfVjHN9xDAdoKjuweEY8OKnlT1VbsoiFuSwtY\n+43NTQ27f8CwaUMVC8ThBYn+53h0B36tchlCCOEvfC7BG69xX825Mq7lBDMYw2kaMpjPeZvxXMNJ\n3uA1Eqjr0dhcoT131rUdNIUfpeAZb1e6TGcG/xFCCOEcn0vwNUUQuTzKMnbTlW+5m7NcRSsO8QBf\nsoVeVa5gV8hsrlJv88BPP6lwG82Z02j/PA6UXBAEbNpY6TIr6jioPX600scWQgh/Iwm+GrTnAIt4\njlM0pifbeJE5tOQw7zGSS6g0qY7ZjGIyqXPsIpH9bieqyw0u7RO0YG6lywudMrnS+wohhL+RBF+N\nIshgGAs4SBs+4Fni6cw1nORJPuJn3Du4j373rqodwJmpaK/kOlxv2Liu1MbF89JXPsELIYRwniR4\nD1CA7uzkCwZzlOZcy3EeYCU38hMf8wTZBFW5jIDv11Y90OxscDTNbAUXARGPlIyDoN+5rerxCCGE\ncJokeA+rQzITeJs/acpkJrOa+7iaM4ziXY7QvApHrvpd/ugOrQh/+rEqHwcg5K037K7TJNh+/E4I\nIUTlSYKvIbSY6Mf3rOVufuEGgsmmJ9u4lc2sZCD56FjOQ2Wew1+Og/nVqzram9mM5uJFAtZ/V7Xj\nlFeu45/2xHFCx7/i3jKEEEKoP12scF1j/uYtXrXU6OcxnGdZRBpRlm2qYzY7RzSn/7Y52I7d7S/b\nnuIxqqtrnfQAGapWCCGcIDX4GsxAPoNYwXZ6EkOKzW0qNZtdVlYVI4Pojm2qfIxK27nTc2ULIYSX\nkATvJf6iqc3lB2lDHHeTh77M8oC4r+0eKyBuTcUFVqKWrPvl5wq30bhjDP037N/PF0IIUUgSvJdo\nxSGby+tznhm8Qj0SeJZFbKMHJhQUo9HuscJHvgBATGy4/QIrMUhOrb69S97YuUBQcnJcPq4QQgjX\n+WyCN4VHeDoEt7I3m90MXmEHPdjP9fyDE4xkNg05zRimsZ/29vvSqzwITujEMaoe31NiYsOhoMDT\nYQghRIV8LsHnPPksAAVtrOdu92b2ZrMr7mDXkDOMZToHaM96+mAgj/tYTSsOMYVXrWe3q6gmXcWO\nbLr9FcxX781UvjgSQgh38LkEb2ztwc5fKrM3m115rfmDN3mNv2jCRzxFInXpwm5uYjdzGEESsWjP\nnnFcmPRUF0IIr+ZzCd6iChOt+AoF6Mpu5jGCczRgMpP5mRtpzlEGjvoHS3mUDMI8HabTSo8D0KNH\nMGvWyFOeQghhj+8meFGGngL6sIFlPMp56vNwn0TLqHkPsoKvuZcrGGzuG/HQgGqO1tpyHuKfLOcg\nbTGi4/BhLUOHBlklef3O7Y47D7qDXDwKIbyAJHg/FEwO9994im/oz1804TY28R6jqM95nuEDttIT\nk6kkiRm2bvZInPn5kJCgcPCghgl2OhnOnl32okR38LfqCE0IIWo8aeP0V0WP0UVzkWdZzLMs5gxX\nsZxBvMwskn+4mkEs5WE+43r2o1BYi36LiRyiFa04xETestsPwNbc7mYgkzD++kshJUXDpfgTZDGU\nZGJJJpYk6lheJzYL5fJliIoyU7u2mb9pbLOcY8c8cI0qNXghhBfw3QRvo5OYWadDkUec7Lqas4xh\nBmOYwYHrn2Rl/DUMZBUG8mjDb6ziQcu2xUPlGtHQmy1WCbr4dellycSixUjth4KpXdtMg19OUocO\nxJJMM47RjZ3EkkwdkjB8+CFht7RBk5MFISH0jj3JQayfjKhTx4zJBJriPF8++RqNhSvdmZQlwQsh\nvIDvJngbMt+bT/jwoZ4Oo2aoIEm1DDnNFD7mDSYRT2f+D9vTzz7CZ8SQYknMpf/9Bycsr4t/Qsgm\n5ecMAGJi+9stPz0/Ae1Huwh9dRwpyRlM5C3L2Pul6XRw113B/Oc/V+jc2Xpwn5h6tcicNZfcR9wz\nK54QQngLv0rw6PzrdB3RnjrpeIOiFhAFuIl40oi0uZmOApKp4+boCoW+Oq4whrRLllsBU5nAIVrR\nrJWGkSPz6H9LEhuve43nnvsvHToYebNJJOUflNQeO1phWZpTJzE1auxc7Vxq8EIIL+BXnezMwSGe\nDqHGCHt5hOMNyg3mYm+oXHvLHdH9uq/ijUrdYtGkpgJlxwHYti2bAQMK0CWd5xE+44ddl2nd2kSP\nDx5nDNNITy91LCcScnSndui3b0XJzEBz+m9XT0kIIWocv0nwuQMfIu/Ovp4Ow2sYtm8t897eULkT\nmOrysWvd0dOl7c0aBx/TouQd/dG7vPRSHj8PX0wakXTpEsJHH+nJR+d0jVvJziZs1HD7M+XJ4D9C\nCC/iNwne2KSpNK1WQUVD5bpK+8fvzm/sRIIvfjyuTng2i3mWlStzWLdORxsOsu7v65zLzYqC4sIc\n90IIUZP5TYIXVefsULnO0Caed3rbiMf+aX9lUYIP/GY1AKGTJgBw3XUmVq7MYRYv8+quuxk4MIjf\nf6/4427Yua3igORCUQjhBXw6waceKdWRTL6UvZbusP37/PrdP9hdpyhwF+s4mN6Qfv0KeOihIEaN\nCiApyc5nwVFLAUgTvRDCq/hkgr+0bjOZCxZjjor2dChCRZpzZwkbP7rC7fQU8OST+ezedomoKDPd\nu4cwc6aB7OxyG8o1oBDCh/hkgi+44UZM9eqXWZbf5eYK90tJzlArJKECw7YtTm+r37KJptfV5l//\nymPjxiyOHNHQtWsIX36pc37215pQgz9a8SN/QggBPprgy8t6eQz5XW/xdBiiFHMFt0yC3l/g1vK0\n589ZXjdqZGbx4lw++CCHJUsM3HlnMNvp7vRtnOjrmro1NmdpjxyGFi08UrYQwvv4RYIX3sewY2uF\n2wT+92OXjxu4tGSfTp1MfP99Ni+8kMdjLOWR97pynH/YjmfzRksNvvi5/OqmXMn1SLlCCO/kFwne\ndFVDT4cgytH/srdqB8jNRb/fiQFzihg2riv8t1yzvqLAgAEFHKYlNzS9SBd28zIzKf+0XMQ/B6Jc\nulS1mIUQohr5fIJPOZlA7sOPejoMUU7IdNcHyCktpmGsS9sbNm10uD6IXEb1P84hWpFDEF27hvDB\nB3ry8qoSpRBCeI7PJ3hCQuQROVEyi6DDjnIKsaSwkBdYvTqHLVt0dO8ewrp1OszAmnWhloF+evQI\nZs2aap7bQD7HQggXyOwrwmfo9/xYdkF+foX7hD82mNyBRdPglkqgLVqYWL48hy1btEyeHMDb/MHh\nV2KBwpaDw4dh6NAgIIcBA6ppCuKa0ItfCOE1fL8GL/xG5D19yryv3aS+nS1LBKxbS8RTRbdwbNSQ\nb73VyJYt2WQQbnP/6dMNZGW5HqvwLmvW6OjRI5h69UI90npTXL5Oh0fL99T5i8qRBC98lnLlivVC\nR7VgO03gOh0kUtfmuhMnNLRsGco//hHKLbcE88Ad+Yzs9SdTphj48EM9336r4+efNZw9q1Tpfv6a\nNTpuefYGj90eWLNGR88boV5skN8luDVrdAwdGsThw1qMRoXDh7UMHRpUbTGULR8Pl1/9518cg1zg\nuM47oqxGF7f+SFSvrp4OQ6hEU+p5+PKC58+2u64VhzhIW+vlrUxs3ZpNejokJmpIf2U2qb/+zfF7\nF3HihIafJn7H6Za3k5geTEqKQkSEmTp1zNSta6ZePZPldd26JurVK1xXu7YZrbakjOIv2GLVfXug\n5pWvtVu+yQRGIxQUlPxbUKBgNJZ+D0ajYtmm/Hbll7/xRoDNuP7znwBycwuvGc1mpehf+z/F8Tne\nTrFsW7xs8WK9zfInTQrg7FkNimJGoykcaVlRsLwG62WFr80oSsly6/WUWT9liu3znzo1gHr1zGg0\nheVrtYU/xccqfG8u997x+tLbFV9vu/L/rwZPl18VitnsnTf2UlIyK7VfTKztplYoGcnO0TbCO6Qk\nZ9j8fyxo1ZpL20ru1dv7vy4/quHm2Gf4J8uttlu0qOwfefgTjxDwXVyZz1L6R8vIu/tejEa4cEEh\nMbH4R0NiokJSkkJCgsayPD1doXZtc1HCN7Fnj5ZLl6wb2xo2NDFjRq7ly7j8D9h6bba7va19AZ5+\nOoi//rIuv3HjwvLz8yn6KWylKP86L68wmdp+r1j2t35f+ProUQ1Xrli3rmg0ZoKDyyZus1lBpzOj\n0xUmicJ/zZbXJcvNlkRS0fKNG7WYzdblK4qZBx8sKPV7M9v9/ZdPnI62K3+suXMNNsvXaMw8/3x+\nmYuG4tcmU9nXULxMsVpfet/y+wNs2KCze/433mjEZFIsF1ZGY8m+ha8VyzLrf5Uy78seQ7FcuBTG\nZF2+VmumVq2SC4TSP6UvWoovQKyXl7+oMdtcvnevlqws6/JbtTKybVv58a7VExMT5vI+UoMXwgnF\nM+dNZQKHaEWzVhpGjswrTO75+aAvqmU56Omu1UJsrJnYWDNt2wIYbW6XlwfJySUXARs22P4zPXNG\nYe5cA2C7tmirZmh7nf0ap9kMf/1l+5z+/lth9mwDOh0YDKDXmzEYKHpvRq/H8lP8PigI9HpT0faF\n+xSut/W+8HWfPsE2y9do4MCBy6USeckXszv16BHM4cNaq+UtW5qYO1f9wYf+9z+dzfJbtDDx73/b\nuA3lZo7Of+3aHFXKLP4MGo3QsGEoRht/KooC27Zll7kgKf1Tslyxs7z0T8mFT/n1//xnkHXhwLFj\nNf8Ot98m+PzOXdDH7y583f569L/u93BEoqYbxApLok/ZVlLDj2kQTfrSL8jr2w/tIRfmubfDYICr\nrjJz1VVmwETz5ia7X7BffaXOF2xpjr7gq6N8e+ffrJmJMNcrNS4bNSqvTBNtsZEjq2eQBH8sv3Sr\nR7Nm9v//Y2OdaYCuWiO1o89fTVfzL0FUkvbtBsvrgg4dPRiJ8Gq5hTU47d+FUxPr/vrT7UWMGmX7\ni7Q6v+D9ufwBAwpYtCiHNiF/oiOfVq2MVrdmqqP8Vq2M6HR4uHxztZfv6f9/T5dfFX5bgy+W3+EG\nl7Y3NrgK7bmzKkUj3CWqo3WHODUELv8MAO2fDhK72fkrfeXiBTCZMdeubVlW+EWaw5x3jBz9K6Ds\n7YFqUFz+3H9nciSxlsfKnzcmicMZDaq9/OIYnprcHW3W+TKtN9VZ/uMr7sFw+3WkvPqGR8ofcGcG\nBARQpgdoNZUNOcyebeDYMS3Nmhk98vkrLF9Ds2amav/8VZbf1eBz+99X5r0p1vbjT8Wu9OlX5n1+\ntx5uj0m4n/b0qSofQ79jG9h61K60opuDQUs/srtJ0JIPnS6z1p29iOrawWr5gAEF7Fz8K/kY2PH5\n8Wr/chkwoIAfR/6XfAxs25btkfJ/um2Mx8oHPD7QkGHLJvjqK4+VH9O4LsHT3/JI2QMGFLD3xqHk\nHzvpsc/ftm3ZnD9/2XOfv0rwuwRfVaZw6WHv1Vz4ko4ceA+BK617zhcL+OJTwia8UrLg8mWb22nP\nnYWCgsLauR3RTRpATg6apEQ05We6KafWzTc6DtxXeecDPz5Fq8ItKGcFLf0Ivv3WY+V7I0nwFZHx\nv/2byX7zevmhcWPKjZynpBXOPmdWFIJnvE3tFtfYPZbmciZKpnOPfmqybF9ICJXVhAuMmhCD8Br+\nl+BtJWxXkriNbY1161UhIFGddIf/AMCwNs7xeAfFTfPFD1nboFTwZRvw1UoANMnJaB0MsCOc5OmL\nbUmunv8/EC7xvwRfjjnY9jOOFjY+0HldbynzPmf4SHeGJFSm/3EXut8PONxGySjsSBU27mUi+91m\nvT7tUoVf+MXN91LjFsJN5ALDJaom+GPHjnHbbbfx6aefApCQkMCQIUMYPHgwI0eOJK9ocO64uDju\nv/9+HnjgAVauXKlmSDYoGBtc7dIe6Wu+K/M+59kXuLjrZ3cGJVQU2f+uijcqlbz1+/dZra7drBEB\nX33pdJmW3vbHjjq9T2UEzZlF6NiXVC3DX1XUYuMXPJ1gPV2+l1EtwWdnZ/PGG2/QpUsXy7I5c+Yw\nePBgPv/8cxo1asSqVavIzs5m/vz5fPLJJyxbtoylS5eSVkEnI3fLeX44qcdP215p64/aVjN9s+Zu\njkrUdIqdpntHDP/bUPFGdpip+Mst6IOFBH1S0qM/ePpUbA4DViny5SokwXsT1RK8wWBg8eLFxMbG\nWpbFx8fTu3dvAHr16sXu3bs5cOAAbdq0ISwsjMDAQDp06MC+fdY1JrUYGzUCrRZzRGSF2xZc20we\nkxNeJWT6VDQpyZ4OwzdIDV54GdUSvE6nIzAwsMyynJwcDIbCsbOjo6NJSUkhNTWVqKgoyzZRUVGk\npKSoFVYZqX+eJXvMRKvlBa1aA2CqVavM8ks/7CXvjr7VEpvwLO3Jv9Q5cEVJwtb6rCwi7+yJYm/I\nzdI1dFs1HDcnJs0ZO61dPq8GJHhPX2R4ugbt6fK9jMdGsrM3iZ2zk9tVZmYdACaMg+vbFe5v7xgz\npsFdd6ExGgl49ilYt9ZhmZWORXhMSEigw/W17r6jSse395kIDQ0gtPS67Gx4+mkAatcOtb3/xfOw\nfx+1aoWUXX/kCMydCwsWQHo6hIdDUqLV/tHHDkJbN9xCCiv8nUU//yTEx1f9eK4KLKwcePrvzZ/L\nDwwyEOjH5+9tqjXBBwcHk5ubS2BgIElJScTGxhIbG0tqaqplm+TkZNq3b1/hsSo7XSyNW8BzLcDB\n/ob0bCKAtCWfkX9zD2JslBlTavvi5bWaNFVlLHKhgjfUHe4zJSWzzGek2OXMXHJKf45KPap3IeEi\nURTe5Sz9WdNeuEwUcPFSNsVtXSkpmdS65150x48BcPHXwxhbtrL6rMYAGQmpXKns30spgZm5hAH5\nBUbS7B3PaCyZJcTNwnLzCaQKf/tVFG02o/Fg+ba+h6q7/NwrBWR68vwVxWPn72mVubCp1sfkunbt\nyoYNhZ2MNm7cSLdu3WjXrh0HDx4kIyODrKws9u3bR8eOnp38xRQeAbg+LK2pYSM1whE+RJN43u66\n6OtboVQ0NG5ppVu7bHWkK1qvOBisx91qdb2BsOefqrbyqpWnm8eFNNG7SLUa/O+//84777zDuXPn\n0Ol0bNiwgRkzZjB+/HhWrFhB/fr16d+/P3q9ntGjR/PUU0+hKArDhg0jrDrmgHSg4MbOpB487tEY\nhHcLfudN28sXv0/Wm9MqdcyQt+23OoRMe5Osf5ddHzR/TuELsxklJQUl6zKmxvZH03MH3cm/KvV0\ngSuUjHTMRRfhoppJgvUqqiX41q1bs2zZMqvlS5YssVrWp08f+vTpo1YolWKuU8fyOufhRwn67L+q\nl1l6jnrh3UJmvuP2YwY4eMQuYP33BKz/vsyy0NcnFb4wmYh4eCD6X/eTkmw9E5qSkU7tf1xNSnIG\nmsQEzCEhmMOcm3NBd2A/YUOf5NKe/TbXB894m+xXxjt1LGcFz5pO1uQpNtdpjx7B1KAB5lAVKgk1\noQZfE2LwJLnAcInfj2TnjPxburvlOMW98+0pPUe98GH5+e45jrNf9iaTw3HulQslk+BEt21O+DOP\nOx2Cfs+P1v1OiuLSnD1DyLS3CjsSupODWw5R3ToR8u/X3FteseJbHpcuqnN8b+DpBOvp8r2MJHhn\nuOmq2VSrFvntrne8jRo1D1GjhPy78NFM/bYt1VOgyYTuzxP215f7fCulOr2WXWH95Ro6aYL1ZtlZ\nhS+Kmuo11ZwQLeWrRL9rp911Yc89hebUSfUKT0hQ79jO8HSC9XT5XkYSvBtdLnVvNa97r0od4+Lu\nyg3yY5QOfl5D+/cpQt74N5EP9nduBxsXmNHX1HectEsJG/eyK+FV+CVaPGVo6Bg7Q+IWx1v0r1mn\nd618F2jOnMawYZ3N8jXnzjqeUMhVZqsXVgJXr8SwdbP7yrRBSUqyuy7yjh4oGemqlV085LI9+t0/\nqFY2UGHrlybBfidWfyQJ3hl2avAF17Xh4taSKUON1zYreX11yfj2GYs+LnzhxNVn6Xv/tuR162lz\nual27QqPLWoO3b69VdrfrRPYlP9cVvAxLZ6vPmhpyZC42j+Po0ksV7ss/rvR6yAnp2SGvqoq9fcY\nOmkCEUMesrlZ4Mrl7inPVcUXGEmJhUMFu5lidjCF8a/70Zz20EBE2dlE3qvyQGDDhtldpfs5nuh2\nLdQt38tIgneGnQRvrFcP43W276ubSiVqc0jRACVuaF66clc/q2UXDh4jffnqKh9bVCMnPgsxseHo\n9/xoNe98VWgP/eGwPAAl3fUaYFSXG4go1yIRfVPR7Shz4b3xiMEDXT5uGZbfWcnfY8D335astvQl\nKLpX7u45LWx8D4RMmkD4ww/Y3C705RGEqJDgS1MuXYTL5S72iso3fPctEQ8NULV8W+UW0x08gH7H\ntmorvnw/E+2xo2jOnjnlQVQAACAASURBVKm28msiSfDVwdbkNPUbVOpQpjrWc8+b6tTFHFnLxtai\nJtJkWPdktyfynj6EveK+6Yg15ZpvNWfPUOu2sp1IdZUcpld35HDhMS9dKrvCaER7+m90v/9WqeOW\nF1g0i19kubg1588VvihONO5+/t9Ggg/8dGnJ0w2W9YX/OnrqwV2i27UgotwFRvGQxhFPPKz67YKI\n+/6PgG+KKhflvudq9e5G5MB7VC0/+J03S1qOypUfdcuNRHe4TtXyazpJ8M6oVCc7WzU0xXIsY5Om\nVQpJeK/qeBRSv3O77RXlnlGP7nAdmkzHFxzaE8eJvPWWSsei5OYANhJ/8fGPHCa6ZeHz+bpf96Fc\ndjxSWXFNXf/br5ZltRvGErRkcdkN3f1IWdHxiucpCFowt8ytEkuiq0ZKbi6G4vveOYW/5+p8lM6w\naweG4lYUD3SAC5n5Doa133is/JpOErwzKvUHU2qfog+eKTa2zJcSgFmnc9hBzhQdXYmyK+fStxsr\nvW/ezd3cGIkfUPnLKPL+u22vqMRnWfdzPPrffyNkRklzs90LCBu0FQzfrDuwH01R0q51R0+Cpzlu\n1rY1L7uSm4v+l7L9GoIXzS98YTQSExtO4JIPnY7ZkeILldDJr1qWBaxcjuZMUXOw2gm26PjlOxBa\nxj2oJkFz3wMgcM1XAASs/65ay9cUzbsQ8P3aMu9FCUnwKko9eqrwRdGXubn07HpFyy5PeccyNK6n\nFXS+qdL75ldhX39kcCFBulUVLiw0pR6fs3sBYUNFTwuEj3iu7IKCyo0ToEkp7F2uZGaiJJdMkav/\ncRcAhv+tr9RxLRw0/YcPe9ZSgy9/AaJJOE/oK6PcN/6BDdH/uFq9GRDtCH3jX5bXwe9OJ8TGI5Nq\n0R45THSbwk7Nhl070MXvsf4cCUnwajLXiiq7oNSXq7lo2lzXD6pu7SB7qP1eqgAZC61rQfam3RX+\ny9GjXPbUblDYWuXMULdKmo3m/it5AOjj96DYeMpA99sBm8fS/nWCwI8+qLjMor+94IVz7Wxg++Ip\n8NOlBP33YwJLPXVQFbY6QWoy0kseESv3HRH4+bLC2r47LzDKlREy9Q3Qam1uqjl1Eu0fv7uvbEAp\nd1vJavbHUnMzKGmXSm5f+BlJ8M5wV1It9QWQPfIVy7Ez3/+ItFVxNnfJHm7nWWOVZL3huHnUeE0T\nq2XmsHC7f9yicgJWr1TluJH3/R+G9d/DlSuEuTBinavs1iYd/C0pRQko6BPbibD0M876nTscB1Dq\nb614P22y7YuO4FnTCZtQ+PeoJCfbjbH0ADqaMzYeRSsuMjsb3f5fSpYXJRslyz0D8ER172y7A2Fx\nAi8Xf8DqVQDofqnao5nFDOu/t/07sjODYK07ehDVq6v7Oj1mZYHJ+e/k2s0aEdPI8ePHvkoSfBUY\nmzl45tIQYHlptnzZlLrCDwkudZzm5HfvafMwOcNetF74nAebovx9LOxqEv6cejOyaY8dIbJ/XwKd\n7RRWiWb9WvfcaXtFXl7Ja6ORCBeemzYUNbUDBMStcbxxqWQTPnyow00Dv/wCAO3hQ9Ru/Q/0WzdV\nGIvN1oCiMhWzGe3RIyXLixKbYftWm8cKWL2S4Lder7DMMjHb6E+gO1EyQVbg5yXzgGiPHCqMy8EA\nPa4IeXMyAUW/s9K0584WvjCbCy8CihSPm+Cu746Qt99A/9Meh9tU+PnwE5LgnVB6AJtiKaeTyZr0\nH5vbX4rbQPbzI6xXuKtj1cKFlpeZs+w0F9pgioysetkqJviLm3dVvJFwAwXdwYofWdP9HO/+oktP\na5uTU9ID3AHD2jhq9bq5zDJbFyeKsWho3MwMghYvtFpfkagehf1IdIcOVbitYqufgFL4dWou93ce\n8t4MoPBesS3hzz1l2Ub7l3OjE2qLHwm0xWxGt6+kBUFzsbADo81e/mYzYUOfcKkDou7okTKDHNk6\nZshbNr4b7dwi0CScd2m+At2Rw2Xu/9sSPvRJp4/nyyTBO6GgYyfrWbgCA202S+d1uZmCGztBUJDV\nutLN22YHw4WZg0PsrjOVGiEva/Q4ch95zFHoFSpoXtgKYQ4IqGDLQsary/b4N8a6selLr96QpqIU\njQaldE3aBiUjnVr9bif8ySFuHWinMheIhi3/Q/fHwQq3U0olieBFC1wup3R5FXE0IU/wgjlo/z7l\nernfryXqpg4o6RUP0ONwwpu8fNCUfL8U92kIstHPQHv8GIFrviLkjX8Xvj/h5DTZpS/UyjObLRc7\npQW/P89qmX7bFqLbtSBsrPO3Iu21hFTEVj+AgJXLiYkNx7DW9i1SbycJ3s3Sv1lXJvFnvTyGgk43\nkXIygZznh1ttX9Cps9WynKcLmxSN9eqXWZ6SnEFB+w4lC1xsEch+cbTVsoyFHzk8VuY7s8ou0JWd\nYdjYyvmBJMxFCdxUy86gPE6cT0WT9YiK6UvfH7anKCkErP2GgLiv3Va2NrHUWOGefm75yhUCVnzu\nxgMWDVF76VKlpguOeHww4FySDfp0qd11te6+A91B2x0KrRTVqjWXMwmaM4uorjc4tZv+V9vTAwPU\nvjoGbAynq1y0vijRFk3Moz12BN1P8W6ZOyB0tI3bmoDWxqh2hi2Ft2N0Rw+jPXKYYBWmefYkSfAq\nyx4/qXBu6pAQ0GjIGjOB3AH3W2r4BW3aWe1jLrqXd/HAEat1jmTZmXf7yu2F90NtjldvNpPX9Rby\n7PQBMLZsZbV9ZWWPLLzAKGhp56LAmS/8yj59ICwCvnUiYRtLvqA1FQw844qoLjegPfgbyuVMIvvf\n5dxOKl0I6H/c5dSjVUFzZhH+5JCKj+emTmzuuA3mdCylx/WfMrnK5UJhi0HxqIbOlq//dT+1/u92\nt5QftOwTO4HZLz/knTeJ6t6ZkHfedEsMNYUk+GqWPWYCmYuWYPzHtVzYY/8q2Cnlvvjy+lqPUw+Q\n+b7jx3PSv/6ejE+/tLnOXP42hIMvn5zHKugYVhyvjWMYGzbG2PQfZL433/Ex7PTUFe5l6zEztx07\nJ4foVk3RH6ji578ynOnJXu7vKnDF5wQUj5bmRobvvrVdYy3396E9cRxNJZr8K5SdjSa5+gaHsdVE\nX52ddoPfnV5tZdUU8m3pQSYnh6vN69Xb5WNfOHis5I2NGlD2i85NIVrQsVPZBSYTebfeZnvbNm0d\nHsscUDjQj62OVcZGjUCrJXew45rSlT7WFzH2Wi5E5SkmB/dYq0qjoOTmqnd8B2KuqVd4/9hsRmtn\nalGl/DPTKrUg2B2bv1zOi+p6A7Xu7On28sPGjCJy0P0Vbhf89hsujVzorNBxLxM23vq2YXmak39Z\nDbFcGeVbNUImjiFQpcdRawpJ8DVRue8TY8PGdrYru2Fxx72Ucxcw1alr6Rhoq0OfsVHjoqIquIIu\n/+Wm09mfua6Cq3F7FwauKP/YoLFRY7LHTrTuBCmqpHgYUjWUHhHPHiUlhfD/b+/c45uo0j7+myQt\n0EIpLS1YbnItgrIF5I5yEeQisCwsF2tRWH1dRRRFKVBh0ZelUFSWi66gBS+FShF1QUWpsNQt2oIC\nW15BvICiItA2TZtekjSZ5P1jkskkmVybZJrk+X4+fj7NZOac59QyzznPec7vWZTBVdTzV+jbgsmE\nxF5d0OZJcVGnqK9P+bc/Ab7uMctE9q99QhDBkHkoRhS7+YWArH5bepj/kDgsDS3f9I/MsJCY3J0e\n3ZeUHOd58mEzgxx8M0Q/bCSMScnWC84cp7OVhV02uqgT92FVojxVBpNAG79x+EiPIwEAgFYtRS83\nPPRXaB50fVbZGY2j73R/E+E1rfa+HbC2294/3+09UV+fQouPucxm2S9X/G6DuwI7gaZ15lNOE/Fa\ne7Cq9ZWk7oJqlM7eKyLRFWdH/LzFZmxehOediR95S5SPGfhRnx/nIgkhBjn4Zoh+/AQoz3t2HtYG\nJ06bX8ELv/fCwZvMmfPGm7vbXG9Ylgl9Ex1sxZUbqM9+AY1Tp/nWgIwU9MKRKIGojT+T/ABbRTx3\nxE+7G4xW59f+AdcOK8pPZXVdkZQch+jiItHvAjnBaLWbq/jXautLjlshZqK+KHa4pvj+O7/0Hz/n\nj1x7Lk6SMGbdACFtVj2D+OlOxJuaMeTgQwDdtBl8Jry3VB07AbRu7XBdO3suAMDooo68sS0njMP2\n6u11vzW5b/Hyu7XZm7iLYpMKEb0AIVVfWv8hihbloaS7sEQoC+tvPMmIV5w9jbiMuYg6VQpG6X5L\nwSvqPExgNJkCsvftjlYCFbxAILtxHa3Xi4uEAUB0YROLArkhqvRLtJs0zun3Cie6+QHNSwkQ9HYM\nAfRjx0O9VyQZxINVOCtMfDOH7k3R0UDLlqi4poKxS1fR5zQLH4TyB7PetqdbBIL79HeOhck8sWD7\n38Z97ULcxxnCLH7tvHu5HyZZJzumaBLHCUeEK3i/t+1B9n70kcNoYXY0Mj+fKIj1YD+bqVYh6oti\nr6r2eYqnGflMeTlaO8lTaBJuVOtkFhEflgWjdFxNNxVft3xklZWAzv/RnECicH8L0VzRp9mJvjjZ\n4wYA1cefwTBwMJQjRsFk0cl3UiCm8vsrtmp63uYAWL4zr671I0bx14xt46GbOg0ylcqj+tHGjtY9\nw/r15kjA4cO87SYXEYhg0DhmnM/KWoRzFJd82KLyK4ET4YnZ/g+398RPu9tvYWl7Eoe4Pu0CcApv\nrV59OSDbBe7O25sYBrKfLqPVnrc8+l15i6hkrx1MTTVa5e50iKAwNTUwJSc7ear5QQ4+RBHLGmd7\n9obya3FJT8MQTjHPaKeOJ0ZTnaZ626swxbV1nEAwDJTf/QwwDKI/OuiRg7eE8J3q6HuRS9A4fgKv\nXOUrxrbxkAmkRE0xsah7dq3LkCMRevCFUyQiUM7dU+IeezhgbbsVWmIYJA5LC1z/nx1xe0/73uKR\nzVCDQvRhhrFrN/c3eYuXIXrd/PsAhoEpyk51Tibj/mMYNE6fCdXHzjW/TXI51EKBHmcJt144+PqV\nq11+r3nQ/Uut+kPHl4Pm8adQ8Uu5x3YQzR9Pj3ARAUBqCWNX2f1S2+YltIIn3NKwfJX4OVA3f+xs\nal+oDnEOsfr9jxz2+y1RBXsqbtS4D/8DYDt3sYb/PcDgorxv/bLlaFieBWNcHGL/8aLT+4w33WTz\nWT9qNDdpael8e4QgCM8J5BFNT4g6/ZWk/fsTWsETbtHNnI0GX9TiGAaG4SMAgDtO5+ns18P7qk6V\nQT9yNP9ZO2sO/3PNLpGXREyM07YMAwYCcjkaVrkuQ2mp7c0/17efkxuDg8nFmAiC8J7YDeukNsFv\n0Aqe8B07R6y/cwwaxzg/fuIruslTbYV/nCGodNc4dTrYrjdD/svPbh/zSgUviNrZnlD97kG0u8c/\nRToIgggvaAVP+IR+0GAYUm+xucb27I2ad/1flEP99j7UvbTNesGTFb5cjgaR8rz2eCtxa4pvB91d\nVofqi0ZAk3jjDYdLylNlUL/smewmQRBNIMR0N0LLWqLZUP3pcZjEys9KiXl1rfrMfLTFPBGoz8yy\nuU2z0E3VO1fIZGhYlsl/NKZ08vhRQ89evvdrQSAVzBlggvHm7tCPuqPpbfuA6qh/JEwJIiQIrRw7\ncvBECGMfLjd/NvzBVh+gYVkm6tbb6n57EvI3KRTQTbUKjegtlfXME4eKK54V6+AJQAYu25dLHDR2\n6uz3tj3BMCANVV98bZP/EEyqnRU+IohAEGJZ9OTgifDB2f64TAbN/zxq/Ww0ge3SxX17RiPUO3fz\nH2veese2Hzcyu/ao8/Y5/U43eSrq1v7dq/YAwNRK+iQ7tncfGG51L54SCPTjJ6D2HyJ1xoOE1FUM\nqw8FVtbVHbpJUyTtP+hQiJ4gAo9m0UM2SXUAHB28yGy7+uAnqM9ag5r8A1D+91uXfTROmQa04FT/\n9AMHwZSU5PJ+SwleAND+eZ7j9z25/fq6Z9faXK/Jfxfq3XugeewJKM+5ETjp39/2c3S0+H1BRj9q\ntPubAoT2vvuhHzxEsv6F9RKCjX74SLCdPZisBgj1284nrcGg0iKnTYhCDp4ISepyNjs48PoVz9qI\n4xjbOzpk/YhRMCUkwpSQ6HT/vOGxpVCe+w7qN/bw10ytBXW8nYXpBPOL2n++bu1z6HCb2xhBTe6K\nayo0TpjET1aE0ryi9OiB2g2clrkUq0dnofigJxvaYeiTKlnfUo+96sx56TpnGFRc9rw6n78xtY1H\n1RdfB60/mb8LDwUYcvBE2GDs3gM6gQNqnDYDld/+5HU79WvXOTpamcCpOw3TcR7e4ni189IBALUv\nbLFtqrLC+sFJPQCX+HhUr3bjS06/0yx6yKM2hGp/mnszfLIjEGiWPCm1CZGLSLXKYML27hO0vkxM\naLnM0LKWILyBYWCyzzr3FYFPNbUVKVsr+gz3EHtLP5vVtiVyYFPQRwSDk5Uh46OD187lqvHZbx8Y\n+t+GuvWboCosctuGYcgw6O6ZAQCo2/pPgVFBSj66/XarLanOlQmDgTHW6tiMCQnB7791m6D3KcQ+\nMhVsNAsWBr9TSrIjiPBC9WEhardZnRnbs7fo3p9+yDAYevS0XnDiiNnOXVD3t3UOmf32GG9KsWnP\nmxd6xfVqx4vmyEPD09ZjfurtO6A6/gWgUMCQNgjV73/kcR9C3E1WAMDgYqXl8T7yV1/BGMdNsFTF\np6zXg/Ti1Q+2TjDqtgiS++wUDoOBZvHjQe9TiHb+fdL2n74g+J1Skh1BhBeGYcMd9utNbR2r29W+\n8hpUgoQro8hKv2b3Huhm/AmaJUuhve9+l/3WbXwJDU8tBwDopkyD8qJ5u8HFCp5N6cTticpk/OSg\n5s186AcOspbYNYcZG8eOh868jcCPKy4OPmH34qvd5Fjms+GJZbw9Fgzde0CTvgBVZ85DWXbRw74c\nnTnrD40BD6j+5N9cf126QvfHWfx1zV8CV33NhmvXAHC5EN7IR5u8PPHhDMsEsD4zyysHq531Z7/0\nj/37OTv2vQ+DF4mVmgWL/NK9MbGZaX+4gRw8QfgLS7U8M/Wrn4fyVJnNLY3TZrjUxAcA5dkLqLiq\nBNu7D++ADf1v5TPmG+8YC/2wEeIPm0z8nihjXlU2Tp2G6iNF/IkAC6ww2iB83ox2xp/E+3CxWjaY\nCwxpzWJCltW28LnGqdO49v88D9WFRajb8gp3700pqN283WnbPGJ5CwKb9INvR/2y5Y623cKdQBCe\nYmgcMw7ambNQcVWJqhLfs+EbBJUKnUUjeB0FEezzNJzSsaP5B9tJnrL0rMvHWDeCTPrb/uC2a1NM\nLFdTAoAxIdHmb73u7xtd999d5G/NS3RTpwNzuBwbo93v2NDvVpfP6m9v+imLuufWA7HuI1XNCXLw\nBBEoYmJgvLm7148ZO3UGoqJsLwpepuwt/UTL1tZu3o66Dc4r4fEoFKj87mfU/e8Gkc6toWb9+Am2\nX7VrB8B1xjrbqw9fYAgAIJdBefobqD76DGxfW2lj/eDbHSMhBoN7+2XOExPrVzyL6vc/RkPmswAA\n3eR7+O9M5t+pRROhZk8Bava+i9rX3gSiosD27I1GF4qANg7aSRRFN30m1Lvz+CiFRrDKZTtzYkTq\nV3O57zIeQFXpGVRcVUL7wF/cnoqwWYXa9W85wmlMSED1oU+5iA3ACzWx/W4F2/VmhzZr8gqgOnIc\n1ceKoTz9jcv+1YJtKvv+DYIJQlXpGf5ny0kWyxFS+1yF2pzNUH1YiIpytWjUR0jdOsHfK8vafKf9\n02z+Z7HtKcMwLl9Ad/dk2+dmzoKqsAgV5Wro0wY6PCekSQqYEkEOniCaOaaoKOgH3e72Pm3GA/zq\nGABq9r0H1af/trmn6sRXMHbtBlO7BNEz9EInoL03g0+8q8l9C8rvrgAAGjKzUPFLucOzmgcfhuaB\nv9jaHhMLY5euMAwdBmOHjvx11eGj0GYsdBwEa3Xw9ln/JvPKvX7Fs6h/eoXjszA7lFatrBMihkHl\nhcuovHAZmofNYkfmCIqhVx+H3wHjYoJhMpcEZrt0hWFAmug9DY88BkPaIP7/A9ujF7QzZ6Hmjb1Q\nfP89APAnPepe2AK2Ry/HyZwTjO2tCaOmdraO0hTFjUN58Wfoh49E9eFjnD2Pcvv0uomT+MJLlslH\nRbkajZOmwDBwMNe+XTlnB4SrVyenPyrK1WB79OI1JpRfnePavikFACCrqkLjuLv4e7WLHuKdr9aN\nA7XZPrJz8Hz/P10DZDJ+MmCZbJjMCZHRnx+33ntNhdrX3oQhjZsMVRd+7rJ/d5G35ghVkyOIZk7l\nVaVPz7E9HPelWTfnxU2Jiag8fwlMfR1X7tf88jMlCE4jyGSA2dkJsY8eVJWchqmlde/X2Mb6gjY4\nCVezvayJeNr7F6HNyqf5z7U7diHOfF2MquJTNnvxDX9djMaJk/maCSZB1rvT1bLAcVSVnkHC8EHW\n78y5C1UlZ0STrfRDhzsc2TLcehs0TzwFAIg6VQrFhW8AhuHObrs5IqmbPBUtPj1svWCOXCjPnHfc\nC7afrFnyLeLi+LHqd7+GqP+eReOkKajZU+CybzEsuRuqDwthsFvtGu0mHHzOSmwsKs9fshGJUr+a\nixYfHfK6f8vvXL19h0M0yGSRnjZPQizCR2yPXlCe/C8/uTT07Yfarf/kar77ckQ1xCAHTxCEDaak\nJAfVPqNIUqE7LMp9PDExbsPQ+jHjUJ+Zxa3EFQpUXK9GUsd4LnHQzXlr1u7YXP06231h/e1DoZs8\n1WUb9SueRfycP6Jx7HibCZIm4wE0WIoWOVEPrP6o0Oaz/ViFSofOzm6zHW+C/DqXSKd+ex+Skq2T\nIv3wkQAc958BADKZQ3+VFy7bFITiTzrI5Wi8273EbOWPv6J9L2tflr8Jy4rbxu6+t3CrZwFV/zlp\n8xzfTkKi00makJq8ArRdYD3SaZJz7so+MRTgMvp1E63hd8PgIVAdPgqA08fgiY4G268/2H797Zuw\nwZiUDENqX0SfCO1iSozJ1MwKXHtIRUWt1CZIRlJSGxp/hI5firEzqiqHkHAwSUqOQ8WvFUCLFkEZ\nv6LsLNjOXWFKTIT8/84h4a7R/lENNJmAhga3iVptZ05FXc5msKl90fqpJWi1922+/6aMnykvh0xZ\nCfaWfk7vkV2/hsQBqVCevQBjp878BKPy/36AqUMHn/q1oPiKc/iGIcOc3hN14j+InzUNFTdqAIbh\n+1cVFsGQNqhJ429RkM/VTXCx3dU68ym0enMXKsrVYGrVaN+Ty5uoPvQpP8GSiqQk73UPyMGHIJHs\n4IDIHn8kjl04wQj2+Bl1DeIeXoQaiarWRRV/jvjZ0/3i4H0hZlM2Yl/cKFlRnbYzpyLqZAkqr6kA\nBGn8Wi2/BdU+JQFs6i2cVoTEkIOPECLxJS8kkscfyWMHaPxBHz/LgtE0wCSRah5TVwvoGnlFyqBP\n8G7cAKKjJI1gWfDFwdMePEEQBCGOXC6ZcwfA9S2h1H1TtyWkho7JEQRBEEQYQg6eIAiCIMIQcvAE\nQRAEEYY0mz347OxslJWVgWEYZGVlYcCAAVKbRBAEQRAhS7Nw8KdOncKVK1dQUFCAS5cuISsrCwUF\n3istEQRBEATB0SxC9CUlJZgwgSts0bNnT9TU1KCurk5iqwiCIAgidGkWDr6yshLtzJWqACAhIQEV\nFRUSWkQQBEEQoU2zCNHb44n2ji+H/sMJGn/kjj+Sxw7Q+Gn8kT1+b2gWK/jk5GRUVlbyn8vLy5Fk\nV6CAIAiCIAjPaRYOftSoUThy5AgA4Pz580hOTkZrN5WjCIIgCIJwTrMI0Q8aNAj9+/fH/PnzwTAM\n1q5dK7VJBEEQBBHShGyxGYIgCIIgnNMsQvQEQRAEQfgXcvAEQRAEEYaEnIPPzs7GvHnzMH/+fJw7\nd05qc4LOpk2bMG/ePMyePRuFhYVSmxN0tFotJkyYgPfff19qU4LOoUOHMGPGDMyaNQtFRUVSmxNU\n6uvrsWTJEixYsADz589HcXGx1CYFhe+//x4TJkzAnj17AADXrl3DggULkJ6ejqVLl6KxsVFiCwOL\n2PgXLlyIjIwMLFy4MOz1UuzHb6G4uBipqalunw8pBy+UtF2/fj3Wr18vtUlBpbS0FD/88AMKCgqQ\nm5uL7OxsqU0KOq+++iratm0rtRlBR6VS4ZVXXkF+fj527NiBY8eOSW1SUPnggw/QvXt35OXlYevW\nrRHxb7+hoQHr1q3DiBEj+Gvbtm1Deno68vPz0a1bNxw4cEBCCwOL2Pi3bNmCuXPnYs+ePZg4cSLe\neOMNCS0MLGLjBwCdTofXXnvNo6PkIeXgI13SdsiQIdi6dSsAIC4uDhqNBizLSmxV8Lh06RJ+/PFH\njB07VmpTgk5JSQlGjBiB1q1bIzk5GevWrZPapKDSrl07VFdXAwDUarWN8mW4Eh0djddffx3Jycn8\ntZMnT+Kuu+4CAIwbNw4lJSVSmRdwxMa/du1aTJo0CYDt30Q4IjZ+ANixYwfS09MRHR3tto2QcvCR\nLmkrl8sRExMDADhw4ADuvPNOyOVyia0KHjk5OVi5cqXUZkjCb7/9Bq1Wi0ceeQTp6elh/WIX4557\n7sHvv/+OiRMnIiMjAytWrJDapICjUCjQsmVLm2sajYZ/sScmJob1+09s/DExMZDL5WBZFvn5+Zg+\nfbpE1gUesfH/9NNPuHjxIqZMmeJZG4EwLFhE6gm/o0eP4sCBA9i9e7fUpgSNf/3rX0hLS0OXLl2k\nNkUyqqur8fLLL+P333/H/fffj+PHj4NhGKnNCgoHDx5ESkoKdu3ahYsXLyIrKysi8zCEROr7j2VZ\nZGZmYvjw4Q7h63Bnw4YNWL16tcf3h5SDJ0lbLrlix44dyM3NRZs2kaPJXFRUhF9//RVFRUW4fv06\noqOj0bFjR4wcOVJq04JCYmIiBg4cCIVCga5duyI2NhZVVVVITEyU2rSgcObMGYwePRoA0LdvX5SX\nl4Nl2YiKYAHcqYkhPQAABEtJREFUClar1aJly5a4ceOGQ/g2Eli1ahW6deuGJUuWSG1KULlx4wYu\nX76MZ555BgDn/zIyMhwS8ISEVIg+0iVta2trsWnTJuzcuRPx8fFSmxNUtmzZgvfeew/79+/HnDlz\nsHjx4ohx7gAwevRolJaWwmg0QqVSoaGhISL2oS1069YNZWVlAICrV68iNjY24pw7AIwcOZJ/BxYW\nFuKOO+6Q2KLgcujQIURFReGJJ56Q2pSg06FDBxw9ehT79+/H/v37kZyc7NK5AyG2go90SdvDhw9D\npVLhySef5K/l5OQgJSVFQquIYNChQwdMmjQJc+fOBQCsXr0aMllIzc+bxLx585CVlYWMjAwYDAY8\n99xzUpsUcL755hvk5OTg6tWrUCgUOHLkCF588UWsXLkSBQUFSElJwcyZM6U2M2CIjV+pVKJFixZY\nsGABAC7ZOlz/FsTGv337dq8WdyRVSxAEQRBhSOQsAQiCIAgigiAHTxAEQRBhCDl4giAIgghDyMET\nBEEQRBhCDp4gCIIgwpCQOiZHEITv/Pbbb5g8eTIGDhxoc33MmDF46KGHmtz+yZMnsWXLFrzzzjtN\nbosgiKZDDp4gIoiEhATk5eVJbQZBEEGAHDxBEOjXrx8WL16MkydPor6+Hhs3bkSfPn1QVlaGjRs3\nQqFQgGEY/O1vf0OvXr3w888/Y82aNTAajWjRogU2bNgAADAajVi7di2+/fZbREdHY+fOnQCAp59+\nGmq1GgaDAePGjcOjjz4q5XAJIiKgPXiCIMCyLHr37o28vDzce++92LZtGwAgMzMTq1atQl5eHhYt\nWoTnn38eAFe288EHH8TevXsxe/ZsfPLJJwC4kr6PP/449u/fD4VCgRMnTuDLL7+EwWBAfn4+9u3b\nh5iYGBiNRsnGShCRAq3gCSKCqKqq4mU+LSxfvhwA+GIugwYNwq5du6BWq6FUKjFgwAAAwNChQ7Fs\n2TIAwLlz5zB06FAAXClXgNuD79GjB9q3bw8A6NixI9RqNcaPH49t27Zh6dKlGDNmDObMmRNRMrsE\nIRXk4AkignC1By9UrWYYxqEUrb2qtdgqXKwATGJiIg4ePIizZ8/i2LFjmD17Nj744AOHWtcEQfgX\nmkYTBAEAKC0tBQCcPn0aqampaNOmDZKSkvgqbiUlJUhLSwPArfKLi4sBcEWQNm/e7LTdEydOoKio\nCIMHD0ZmZiZiYmKgVCoDPBqCIGgFTxARhFiIvnPnzgCACxcu4J133kFNTQ1ycnIAcNUKN27cCLlc\nDplMxlfuWrNmDdasWYP8/HwoFApkZ2fjl19+Ee2ze/fuWLlyJXJzcyGXyzF69Gh06tQpcIMkCAIA\nVZMjCAJAamoqzp8/D4WC5vwEES5QiJ4gCIIgwhBawRMEQRBEGEIreIIgCIIIQ8jBEwRBEEQYQg6e\nIAiCIMIQcvAEQRAEEYaQgycIgiCIMIQcPEEQBEGEIf8PYtryWduIPCYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8b0e42ed30>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "import logging\n",
    "import time\n",
    "import math\n",
    "import datetime\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "start_time=time.time()\n",
    "\n",
    "def inference_loss(ids,batch_size,length,embedding,cprev,hprev,net):\n",
    "    \"\"\"Run inference over an epoch.\n",
    "    \n",
    "        Arguments:\n",
    "        ids -- list or sequence of word id's\n",
    "        batch_size -- batch size\n",
    "        length -- length of the unrolled network\n",
    "        embedding -- word embeddings; numpy array of shape (vocab_size, hidden size)\n",
    "        cprev -- cell states; list of numpy arrays of shape (hidden size, batch size)\n",
    "        hprev -- hidden states; list of numpy arrays of shape (hidden size, batch size)\n",
    "        net -- trained network\n",
    "        \n",
    "        Returns:\n",
    "        loss -- loss per word\n",
    "    \"\"\"\n",
    "    steps=(len(ids)-1)//(batch_size*length)\n",
    "    use=steps*length*batch_size\n",
    "    losses=[]\n",
    "    \n",
    "    for step in range(steps):\n",
    "        loss=0\n",
    "        x_indx=[[i for i in range(step*length+j,use,steps*length)] for j in range(length)]\n",
    "        x_id=[[ids[i] for i in Lst] for Lst in x_indx]\n",
    "        x_embed=[np.transpose(embedding.lookup(Lst)) for Lst in x_id]\n",
    "        \n",
    "        cprev,hprev,y=net.fwd_prop(cprev,hprev,x_embed)\n",
    "        yhat=[softmax(z) for z in y]\n",
    "\n",
    "        for i in range(batch_size):\n",
    "            for j in range(length):\n",
    "                y_indx=x_indx[j][i]+1\n",
    "                loss-=math.log(yhat[j][ids[y_indx],i])\n",
    "                #yhat[j][ids[y_indx],i]-=1#dy\n",
    "        losses.append(loss)\n",
    "    return sum(losses)/use\n",
    "\n",
    "# ------------------ input data - training, validation & test sets -----------------------------------\n",
    "with open('ptb.train.txt','r') as f1,open('ptb.valid.txt','r') as f2,open('ptb.test.txt','r') as f3:\n",
    "    seq_train=f1.read().replace('\\n','<eos>').split(' ')\n",
    "    seq_valid=f2.read().replace('\\n','<eos>').split(' ')\n",
    "    seq_test=f3.read().replace('\\n','<eos>').split(' ')\n",
    "    \n",
    "seq_train=list(filter(None,seq_train))\n",
    "seq_valid=list(filter(None,seq_valid))\n",
    "seq_test=list(filter(None,seq_test))\n",
    "\n",
    "logging.info(seq_train[:10])\n",
    "logging.info(seq_valid[:10])\n",
    "logging.info(seq_test[:10])\n",
    "\n",
    "size_train=len(seq_train)\n",
    "size_valid=len(seq_valid)\n",
    "size_test=len(seq_test)\n",
    "logging.info('size_train {}, size_valid {}, size_test {}'.format(size_train,size_valid,size_test))\n",
    "\n",
    "vocab_train=set(seq_train)\n",
    "vocab_valid=set(seq_valid)\n",
    "vocab_test=set(seq_test)\n",
    "\n",
    "assert vocab_valid.issubset(vocab_train)\n",
    "assert vocab_test.issubset(vocab_train)\n",
    "size_vocab=len(vocab_train)\n",
    "logging.info('train vocab {}, valid vocab {}, test vocab {}'.format(len(vocab_train),len(vocab_valid),len(vocab_test)))\n",
    "\n",
    "vocab_train=sorted(vocab_train)#must have deterministic ordering for word2id dictionary to be reproducible\n",
    "word2id={w:i for i,w in enumerate(vocab_train)}\n",
    "id2word={i:w for i,w in enumerate(vocab_train)}\n",
    "\n",
    "ids_train=[word2id[word] for word in seq_train]\n",
    "ids_valid=[word2id[word] for word in seq_valid]\n",
    "ids_test=[word2id[word] for word in seq_test]\n",
    "\n",
    "# --------------- hyper parameters -----------------------------------\n",
    "size=200#hidden size\n",
    "nio=[size,size,size_vocab]\n",
    "ns=[size]*2\n",
    "length=20#unrolled length\n",
    "epochs=13\n",
    "m=20#batch size\n",
    "rate=1#learning rate\n",
    "decay=2#learning rate decay\n",
    "epochs_no_decay=4\n",
    "seed=0#random\n",
    "order=2#norm\n",
    "max_norm=5#gradient clipping\n",
    "\n",
    "steps_train=(size_train-1)//(m*length)\n",
    "steps_valid=(size_valid-1)//(m*length)\n",
    "steps_test=(size_test-1)//(m*length)\n",
    "\n",
    "use_train=steps_train*length*m\n",
    "use_valid=steps_valid*length*m\n",
    "use_test=steps_test*length*m\n",
    "\n",
    "logging.info(\"training: 1 epoch={} steps, used={} words, unused={} words\".format(steps_train,use_train,size_train-use_train))\n",
    "logging.info(\"validation: 1 epoch={} steps, used={} words, unused={} words\".format(steps_valid,use_valid,size_valid-use_valid))\n",
    "logging.info(\"test: 1 epoch={} steps, used={} words, unused={} words\".format(steps_test,use_test,size_test-use_test))\n",
    "\n",
    "# ---------------------------------- initialize ---------------------------------------\n",
    "np.random.seed(seed)\n",
    "net=Network(ns,nio,length)\n",
    "cprev_init=[np.zeros((size,m)) for size in ns]\n",
    "hprev_init=[np.zeros((size,m)) for size in ns]\n",
    "dcprev=[np.zeros((size,m)) for size in ns]\n",
    "dhprev=[np.zeros((size,m)) for size in ns]\n",
    "embedding=Embedding(size_vocab,size)\n",
    "perplex_train=[]\n",
    "perplex_valid=[]\n",
    "verbose=False#True\n",
    "freq=5\n",
    "\n",
    "# --------- train for 'epoch' epochs, validate after each epoch, & test at the end -------------------\n",
    "print(' '*32+'TRAINING'+'\\n'+\n",
    "      'time,s'+' '*4+\n",
    "      'epoch#'+' '*4+\n",
    "      'step#'+' '*4+\n",
    "      'epochs'+' '*9+\n",
    "      'loss'+' '*15+\n",
    "      'perplexity'+'\\n'+\n",
    "      ' '*39+\n",
    "      'train'+' '*3+\n",
    "      'valid'+' '*7+\n",
    "      'train'+' '*9+\n",
    "      'valid'+'\\n'+\n",
    "      '======'+' '*4+\n",
    "      '======'+' '*4+\n",
    "      '====='+' '*4+\n",
    "      '======'+' '*4+\n",
    "      '====='+' '*3+\n",
    "      '====='+' '*7+\n",
    "      '====='+' '*9+\n",
    "      '=====')\n",
    "    \n",
    "for epoch in range(epochs):\n",
    "    #train\n",
    "    cprev=cprev_init\n",
    "    hprev=hprev_init\n",
    "    for step in range(steps_train):\n",
    "        loss=0\n",
    "        x_indx=[[i for i in range(step*length+j,use_train,steps_train*length)] for j in range(length)]\n",
    "        x_id=[[ids_train[i] for i in Lst] for Lst in x_indx]\n",
    "        x_embed=[np.transpose(embedding.lookup(Lst)) for Lst in x_id]\n",
    "        \n",
    "        cprev,hprev,y=net.fwd_prop(cprev,hprev,x_embed)\n",
    "        yhat=[softmax(z) for z in y]\n",
    "\n",
    "        for i in range(m):\n",
    "            for j in range(length):\n",
    "                y_indx=x_indx[j][i]+1\n",
    "                loss-=math.log(yhat[j][ids_train[y_indx],i])\n",
    "                yhat[j][ids_train[y_indx],i]-=1#dy\n",
    "        \n",
    "        _,_,dx=net.back_prop(dcprev,dhprev,yhat)\n",
    "        \n",
    "        norm=np.linalg.norm(np.array([np.linalg.norm(grad,order) for grad in dx]+[net.get_gradient_norm(order)]),order)\n",
    "        if norm>max_norm:\n",
    "            shrink=max_norm/norm\n",
    "            net.shrink_gradient(shrink)\n",
    "            for grad in dx:\n",
    "                grad*=shrink\n",
    "                \n",
    "        net.update(alpha=rate)\n",
    "        for i in range(length):\n",
    "            embedding.update(x_id[i],dx[i],alpha=rate)\n",
    "            \n",
    "        loss_avg=loss/(m*length)\n",
    "        perplexity=math.exp(loss_avg)\n",
    "        perplex_train.append((epoch+(step+1)/steps_train,perplexity))\n",
    "\n",
    "        if verbose and step%(steps_train//freq)==0:\n",
    "            print('{:}'.format(datetime.timedelta(seconds=round(time.time()-start_time))),\n",
    "                  '{:5n}'.format(epoch),\n",
    "                  '{:5n}'.format(step),\n",
    "                  '{:6.2f}'.format(epoch+(step+1)/steps_train),\n",
    "                  '{:4.2f}'.format(loss_avg),\n",
    "                  ' -  ',\n",
    "                  '{:10.2f}'.format(perplexity),\n",
    "                  '    -',sep=' '*4)\n",
    "    \n",
    "    if epoch>=epochs_no_decay-1:\n",
    "        rate/=decay\n",
    "        \n",
    "    #validate\n",
    "    loss_avg_valid=inference_loss(ids_valid,m,length,embedding,cprev_init,hprev_init,net)\n",
    "    perplexity_valid=math.exp(loss_avg_valid)\n",
    "    perplex_valid.append((epoch+(step+1)/steps_train,perplexity_valid))\n",
    "    print('{:}'.format(datetime.timedelta(seconds=round(time.time()-start_time))),\n",
    "          '{:5n}'.format(epoch),\n",
    "          '{:5n}'.format(step),\n",
    "          '{:6.2f}'.format(epoch+(step+1)/steps_train),\n",
    "          '{:4.2f}'.format(loss_avg),\n",
    "          '{:4.2f}'.format(loss_avg_valid),\n",
    "          '{:10.2f}'.format(perplexity),\n",
    "          '{:10.2f}'.format(perplexity_valid),sep=' '*4)\n",
    "        \n",
    "#test\n",
    "logging.info(\"test set perplexity after {} epochs - {}\".format(epoch+(step+1)/steps_train,math.exp(inference_loss(ids_test,m,length,embedding,cprev_init,hprev_init,net))))\n",
    "logging.info(\"execution time - {}s\".format(datetime.timedelta(seconds=round(time.time()-start_time))))\n",
    "\n",
    "# ----------------------------------- plot ------------------------------------\n",
    "plt.plot([loss[0] for loss in perplex_train],[loss[1] for loss in perplex_train], linewidth=1,color='red',label='training')\n",
    "plt.plot([loss[0] for loss in perplex_valid],[loss[1] for loss in perplex_valid], linewidth=1,color='blue',label='validation', marker='o')\n",
    "plt.grid(True,which='both',axis='both')\n",
    "plt.title('Penn Treebank Corpus')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Perplexity')\n",
    "plt.axis([0,14,0,500])\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "# ----------------------------------- save -------------------------------------------------------\n",
    "net.savetxt()\n",
    "embedding.savetxt('.txt')\n",
    "def write_list(fname,Lst):\n",
    "    with open(fname, 'w') as f:\n",
    "        for e,p in Lst:\n",
    "            f.write('{:}, {:}'.format(e,p))\n",
    "write_list('train.txt',perplex_train)\n",
    "write_list('valid.txt',perplex_valid)   \n",
    "\n",
    "#---------------------------------- end --------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Final word-level perplexity\n",
    "\n",
    "The trained network is evaluated by computing its word-level perplexities on the training, validation and test sets. They are compared in Table 1 with those reported in [Zaremba et al. (2015)][8] and [TensorFlow tutorial][9]. The present results are close to those previously reported. The small differences could be due to the random initialization.\n",
    "\n",
    "[8]: https://arxiv.org/abs/1409.2329\n",
    "[9]: https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:['aer', 'banknote', 'berlitz', 'calloway', 'centrust', 'cluett', 'fromstein', 'gitano', 'guterman', 'hydro-quebec']\n",
      "INFO:root:['consumers', 'may', 'want', 'to', 'move', 'their', 'telephones', 'a', 'little', 'closer']\n",
      "INFO:root:['no', 'it', 'was', \"n't\", 'black', 'monday', '<eos>', 'but', 'while', 'the']\n",
      "INFO:root:size_train 929589, size_valid 73760, size_test 82430\n",
      "INFO:root:train vocab 10000, valid vocab 6022, test vocab 6049\n",
      "INFO:root:training: 1 epoch=929588 steps, used=929588 words, unused=1 words\n",
      "INFO:root:validation: 1 epoch=73759 steps, used=73759 words, unused=1 words\n",
      "INFO:root:test: 1 epoch=82429 steps, used=82429 words, unused=1 words\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Perplexity of trained LSTM network on PTB training set = 39.66173153000068\n",
      "Perplexity of trained LSTM network on PTB validation set = 121.71207615129721\n",
      "Perplexity of trained LSTM network on PTB test set = 116.30045062872492\n"
     ]
    }
   ],
   "source": [
    "#MIT License - Copyright (c) 2018 tmatha\n",
    "\n",
    "import logging\n",
    "import time\n",
    "import math\n",
    "import datetime\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "start_time=time.time()\n",
    "\n",
    "def inference_loss(ids,batch_size,length,embedding,cprev,hprev,net):\n",
    "    \"\"\"Run inference over an epoch.\n",
    "    \n",
    "        Arguments:\n",
    "        ids -- list or sequence of word id's\n",
    "        batch_size -- batch size\n",
    "        length -- length of the unrolled network\n",
    "        embedding -- word embeddings; numpy array of shape (vocab_size, hidden size)\n",
    "        cprev -- cell states; list of numpy arrays of shape (hidden size, batch size)\n",
    "        hprev -- hidden states; list of numpy arrays of shape (hidden size, batch size)\n",
    "        net -- trained network\n",
    "        \n",
    "        Returns:\n",
    "        loss -- loss per word\n",
    "    \"\"\"\n",
    "    steps=(len(ids)-1)//(batch_size*length)\n",
    "    use=steps*length*batch_size\n",
    "    losses=[]\n",
    "    \n",
    "    for step in range(steps):\n",
    "        loss=0\n",
    "        x_indx=[[i for i in range(step*length+j,use,steps*length)] for j in range(length)]\n",
    "        x_id=[[ids[i] for i in Lst] for Lst in x_indx]\n",
    "        x_embed=[np.transpose(embedding.lookup(Lst)) for Lst in x_id]\n",
    "        \n",
    "        cprev,hprev,y=net.fwd_prop(cprev,hprev,x_embed)\n",
    "        yhat=[softmax(z) for z in y]\n",
    "\n",
    "        for i in range(batch_size):\n",
    "            for j in range(length):\n",
    "                y_indx=x_indx[j][i]+1\n",
    "                loss-=math.log(yhat[j][ids[y_indx],i])\n",
    "                #yhat[j][ids[y_indx],i]-=1#dy\n",
    "        losses.append(loss)\n",
    "    return sum(losses)/use\n",
    "\n",
    "# ------------------ data - training, validation & test set -------------------------------------\n",
    "with open('ptb.train.txt','r') as f1,open('ptb.valid.txt','r') as f2,open('ptb.test.txt','r') as f3:\n",
    "    seq_train=f1.read().replace('\\n','<eos>').split(' ')\n",
    "    seq_valid=f2.read().replace('\\n','<eos>').split(' ')\n",
    "    seq_test=f3.read().replace('\\n','<eos>').split(' ')\n",
    "    \n",
    "seq_train=list(filter(None,seq_train))\n",
    "seq_valid=list(filter(None,seq_valid))\n",
    "seq_test=list(filter(None,seq_test))\n",
    "\n",
    "logging.info(seq_train[:10])\n",
    "logging.info(seq_valid[:10])\n",
    "logging.info(seq_test[:10])\n",
    "\n",
    "size_train=len(seq_train)\n",
    "size_valid=len(seq_valid)\n",
    "size_test=len(seq_test)\n",
    "logging.info('size_train {}, size_valid {}, size_test {}'.format(size_train,size_valid,size_test))\n",
    "\n",
    "vocab_train=set(seq_train)\n",
    "vocab_valid=set(seq_valid)\n",
    "vocab_test=set(seq_test)\n",
    "\n",
    "assert vocab_valid.issubset(vocab_train)\n",
    "assert vocab_test.issubset(vocab_train)\n",
    "size_vocab=len(vocab_train)\n",
    "logging.info('train vocab {}, valid vocab {}, test vocab {}'.format(len(vocab_train),len(vocab_valid),len(vocab_test)))\n",
    "\n",
    "vocab_train=sorted(vocab_train)#must have deterministic ordering, so word2id dictionary is reproducible\n",
    "word2id={w:i for i,w in enumerate(vocab_train)}\n",
    "id2word={i:w for i,w in enumerate(vocab_train)}\n",
    "\n",
    "ids_train=[word2id[word] for word in seq_train]\n",
    "ids_valid=[word2id[word] for word in seq_valid]\n",
    "ids_test=[word2id[word] for word in seq_test]\n",
    "\n",
    "# --------------- hyper parameters -----------------------------------\n",
    "size=200#hidden size\n",
    "nio=[size,size,size_vocab]\n",
    "ns=[size]*2\n",
    "length=1#unrolled length\n",
    "m=1#batch size\n",
    "\n",
    "steps_train=(size_train-1)//(m*length)\n",
    "steps_valid=(size_valid-1)//(m*length)\n",
    "steps_test=(size_test-1)//(m*length)\n",
    "\n",
    "use_train=steps_train*length*m\n",
    "use_valid=steps_valid*length*m\n",
    "use_test=steps_test*length*m\n",
    "\n",
    "logging.info(\"training: 1 epoch={} steps, used={} words, unused={} words\".format(steps_train,use_train,size_train-use_train))\n",
    "logging.info(\"validation: 1 epoch={} steps, used={} words, unused={} words\".format(steps_valid,use_valid,size_valid-use_valid))\n",
    "logging.info(\"test: 1 epoch={} steps, used={} words, unused={} words\".format(steps_test,use_test,size_test-use_test))\n",
    "\n",
    "# ---------------------- initialize ---------------------------------------\n",
    "net=Network(ns,nio,length)\n",
    "net.loadtxt()\n",
    "embedding=Embedding(size_vocab,size)\n",
    "embedding.loadtxt('.txt')\n",
    "hprev=[np.zeros((size,m)) for size in ns]\n",
    "cprev=[np.zeros((size,m)) for size in ns]\n",
    "\n",
    "#---------------- inference ------------------------------------------------------------\n",
    "perplex_train=math.exp(inference_loss(ids_train,m,length,embedding,cprev,hprev,net))\n",
    "perplex_valid=math.exp(inference_loss(ids_valid,m,length,embedding,cprev,hprev,net))\n",
    "perplex_test=math.exp(inference_loss(ids_test,m,length,embedding,cprev,hprev,net))\n",
    "print('Perplexity of trained LSTM network on PTB training set = {}'.format(perplex_train))\n",
    "print('Perplexity of trained LSTM network on PTB validation set = {}'.format(perplex_valid))\n",
    "print('Perplexity of trained LSTM network on PTB test set = {}'.format(perplex_test))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HCzx0-VhUdNO"
   },
   "source": [
    "<center>Table 1 Final word-level perplexity</center>\n",
    "\n",
    "\n",
    "|         -            |training set|validation set|test set|\n",
    "|----------------------|------------|--------------|--------|\n",
    "|Present result        | 39.66      | 121.71       | 116.30 |\n",
    "|Zaremba, et al. (2015)| ?          | 120.7        | 114.5  |\n",
    "|TensorFlow tutorial   | 37.99      | 121.39       | 115.91 |\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "lstm22.ipynb",
   "provenance": [
    {
     "file_id": "1ItBQlv9QBlVo7SbQ14XkKfFnSvPZXPKu",
     "timestamp": 1527092152165
    }
   ],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}