All questions2020/11/19 HW9 - Colaboratory...

Question

All questions2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 1/20 Before you turn this problem in, make sure everything runs as expected. First, restart the kernel (in the menubar, select Kernel Restart) and then run all cells (in the menubar, select Cell Run All). Make sure you �ll in any place that says YOUR CODE HERE  or "YOUR ANSWER HERE", as well as your name and collaborators below: → → NAME = "Yuxuan Guo" COLLABORATORS = "" o Copyright Luca de Alfaro, 2019-2020. License: CC-BY-NC-ND. Homework 9: From Expressions to Optimization and Machine Learning Please submit to this Google Form. Deadline: Thursday November 19, 11pm (check on Canvas for updated information). Submission The test contains 5 questions, for a total of 70 points. Test Format Optimization, and machine learning, are intimately connected. At a very coarse level, ML works as follows. First, you come up somehow with a very complicated model , which computes an output  as a function of an input  and of a vector of parameters . In general, , , and  are vectors, as the model has multiple inputs, multiple outputs, and several parameters. The model   needs to be complicated, because only complicated models can represent complicated phenomena; for instance,  can be a multi-layer neural net with parameters , where  is the number of parameters of the model. Second, you come up with a notion of loss , that is, how badly the model is doing. For instance, if you have a list of inputs , and a set of desired outputs , you can use as ML in a nutshell = ?( , )? ⃗  �⃗�  �⃗�  ? ⃗  �⃗�  �⃗�  �⃗�  ? ⃗  �⃗�  ? ? = [ , … , ]�⃗�  ?1 ?? ? ? , … ,�⃗� 1 �⃗� ? , … ,? ⃗ 1 ? ⃗ ? ⾃动保存失败。此⽂件是远程更新的或在其他标签⻚中更新的。 显示差异 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://docs.google.com/forms/d/e/1FAIpQLSdYG6YhWxV5MIgd3yINEOcr2n4wsRv5F1UF2n6PVAIW6J-r9Q/viewform?usp=sf_link javascript:void(0) 2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 2/20 loss: Here, we wrote  because, once the inputs  and the desired outputs  are chosen, the loss  depends on . Once the loss is chosen, you decrease it, by computing its gradient with respect to . Remembering that , The gradient is a vector that indicates how to tweak  to decrease the loss. You then choose a small step size , and you update  via . This makes the loss a little bit smaller, and the model a little bit better. If you repeat this step many times, the model will hopefully get (a good bit) better. Autogradient The key to pleasant ML is to focus on building the model  in a way that is su�ciently expressive, and on choosing a loss  that is helpful in guiding the optimization. The computation of the gradient is done automatically for you. This capability, called autogradient, is implemented in ML frameworks such as Tensor�ow, Keras, and PyTorch. It is possible to use these advanced ML libraries without ever knowing what is under the hood, and how autogradient works. Here, we will insted dive in, and implement autogradient. Building a model  corresponds to building an expression with inputs , . We will provide a representaton for expressions that enables both the calculation of the expression value, and the differentiation with respect to any of the inputs. This will enable us to implement autogradient. On the basis of this, we will be able to implement a simple ML framework. We say we, but we mean you. You will implement it; we will just provide guidance. ?( ) = || − ?( , )|| .�⃗�  ∑ ?=1 ? ? ⃗ ? �⃗� ? �⃗�   ?( )�⃗�  , … ,�⃗� 1 �⃗� ? , … ,? ⃗ 1 ? ⃗ ? ? �⃗�  �⃗�  = [ , … , ]�⃗�  ?1 ?? ? = [ , … , ] .∇?⃗  ∂? ∂?1 ∂? ∂?? �⃗�  ? �⃗�  := − ? ?�⃗�  �⃗�  ∇ ?⃗  ? ? ? �⃗�  �⃗�  Our main task will be to implement a class Expr  that represents expressions with autogradient. Implementing expressions We will have Expr  be the abstract class of a generic expression, and Plus , Multiply , and so on, be derived classes representing expression with given top-level operators. The constructor takes the children node. The code for the constructor, and the code to create addition expressions, is as follows. Expressions with autogradient class Expr(object):c ⾃动保存失败。此⽂件是远程更新的或在其他标签⻚中更新的。 显示差异 https://www.tensorflow.com/ https://keras.io/ https://pytorch.org/ javascript:void(0) 2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 3/20       def __init__(self, *args):         """Initializes an expression node, with a given list of children         expressions."""         self.children = args         self.value = None # The value of the expression.         self.values = None # The values of the child expressions.       def __add__(self, other):         """Constructs the sum of two expressions."""         return Plus(self, other)   The code for the Plus  class, initially, is empty; no Expr  methods are over-ridden. class Plus(Expr):     """An addition expression."""       pass   c To construct expressions, we need one more thing. So far, if we write things like 2 + 3 , Python will just consider these as expressions involving numbers, and compute their value. To write symbolic expressions, we need symbols, or variables. A variable is a type of expression that just contains a value as child, and that has an assign  method to assign a value to the variable. The assign  method can be used to modify the variable's content (without assign , our variables would be constants!). class V(Expr):     """This class represents a variable.  The derivative rule corresponds     to d/dx x = 1, but note that it will not be called, since the children     of a variable are just numbers."""       def assign(self, v):         """Assigns a value to the variable."""         self.children = [v]   c This su�ces for creating expressions. Let's create one. e = V(3) + 4 e   e  Computing the value of expressions ⾃动保存失败。此⽂件是远程更新的或在其他标签⻚中更新的。 显示差异 javascript:void(0) 2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 4/20 We now have our �rst expression. To compute the expression value, we endow each expression with a method op , whose task is to compute the value self.value  of the expression from the list of values self.values  of the children. Let's implement the compute  method for an expression. This method will: 1. Loop over the children, and computes the list self.values  of children values as follows: If the child is an expression (an instance of Expr ), obtain its value by calling compute  on it. If the child is not an instance of Expr , then the child must be a number, and we can use its value directly. 2. Call the method op  of the expression, to compute self.value  from self.values . 3. Return self.value . class Expr(object):       def __init__(self, *args):         """Initializes an expression node, with a given list of children         expressions."""         self.children = args         self.value = None # The value of the expression.         self.values = None # The values of the child expressions.         self.gradient = 0 # The value of the gradient.       def op(self):         """This operator must be implemented in subclasses; it should         compute self.value from self.values, thus implementing the         operator at the expression node."""         raise NotImplementedError()       def compute(self):         """This method computes the value of the expression.         It first computes the value of the children expressions,         and then uses self.op to compute the value of the expression."""         self.values = [e.compute() if isinstance(e, Expr) else e                         for e in self.children]         # This computes self.value         self.op()         return self.value       def __repr__(self):         return ("%s:%r %r (g: %r)" % (             self.__class__.__name__, self.children, self.value, self.gradient))       # Expression constructors       def __add__(self, other):         return Plus(self, other)   c ⾃动保存失败。此⽂件是远程更新的或在其他标签⻚中更新的。 显示差异 javascript:void(0) 2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 5/20     def __radd__(self, other):         return Plus(self, other)       def __sub__(self, other):         return Minus(self, other)       def __rsub__(self, other):         return Minus(other, self)       def __mul__(self, other):         return Multiply(self, other)       def __rmul__(self, other):         return Multiply(other, self)       def __truediv__(self, other):         return Divide(self, other)       def __rtruediv__(self, other):         return Divide(other, self)       def __pow__(self, other):         return Power(self, other)       def __rpow__(self, other):         return Power(other, self)       def __neg__(self):         return Negative(self)   Let us give op  for Plus , Multiply , and for variables via V , so you can see how it works. class V(Expr):     """This class represents a variable."""       def assign(self, v):         """Assigns a value to the variable.  Used to fit a model, so we         can assign the various input values to the variable."""         self.children = [v]       def op(self):         self.value = self.values[0]       def __repr__(self):         return "Variable: " + str(self.children[0])     class Plus(Expr):       def op(self):         self.value = self.values[0] + self.values[1]   class Multiply(Expr): ⾃动保存失败。此⽂件是远程更新的或在其他标签⻚中更新的。 显示差异 javascript:void(0) 2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 6/20 class Multiply(Expr):       def op(self):         self.value = self.values[0] * self.values[1]   c Let's de�ne testing helpers.#@title Let's define testing helpers.   def check_equal(x, y, msg=None):     if x != y:         if msg is None:             print("Error:")         else:             print("Error in", msg, ":")         print("    Your answer was:", x)         print("    Correct answer: ", y)     assert x == y, "%r and %r are different" % (x, y)   def check_almost_equal(x, y, tolerance=0.001, msg=None):     if abs(x - y) > tolerance:         if msg is None:             print("Error:")         else:             print("Error in", msg, ":")         print("    Your answer was:", x)         print("    Correct answer: ", y)     assert abs(x - y)

Sandeep Kumar · Accepted Answer

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c6ITdjT8wyGx"
   },
   "source": [
    "Before you turn this problem in, make sure everything runs as expected. First, **restart the kernel** (in the menubar, select Kernel$\rightarrow$Restart) and then **run all cells** (in the menubar, select Cell$\rightarrow$Run All).
",
    "
",
    "Make sure you fill in any place that says `YOUR CODE HERE` or "YOUR ANSWER HERE", as well as your name and collaborators below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "id": "0G9m-EaXwyGx"
   },
   "outputs": [],
   "source": [
    "NAME = "Yuxuan Guo"
",
    "COLLABORATORS = """
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zo5reyZAwyGx"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "XMYdXcXEwyGx",
    "nbgrader": {
     "checksum": "63ab1fa9b6548ab3e3083741a0cdcddf",
     "grade": false,
     "grade_id": "cell-c2546ad8aa448250",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# Homework 9: From Expressions to Optimization and Machine Learning
",
    "
",
    "Copyright Luca de Alfaro, 2019-2020. 
",
    "License: [CC-BY-NC-ND](https://creativecommons.org/licenses/by-nc-nd/4.0/).
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "OrrPaQgBwyGx",
    "nbgrader": {
     "checksum": "91626c449f859e681933f439fd6f4f1f",
     "grade": false,
     "grade_id": "cell-5f3c38b7a6291dac",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Submission
",
    "
",
    "[Please submit to this Google Form](https://docs.google.com/forms/d/e/1FAIpQLSdYG6YhWxV5MIgd3yINEOcr2n4wsRv5F1UF2n6PVAIW6J-r9Q/viewform?usp=sf_link).
",
    "
",
    "Deadline: Thursday November 19, 11pm (check on Canvas for updated information)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "RwH9WK5hwyGx",
    "nbgrader": {
     "checksum": "18f1a1706e1fc95532f0f0681a4720f5",
     "grade": false,
     "grade_id": "cell-dca15f51d39533b6",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Test Format
",
    "
",
    "The test contains 5 questions, for a total of 70 points. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "7B4SOFgTwyGx",
    "nbgrader": {
     "checksum": "a63b30f8d6cde2dad94f9a1d461fdd9e",
     "grade": false,
     "grade_id": "cell-9b6e16513f2274bb",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## ML in a nutshell
",
    "
",
    "Optimization, and machine learning, are intimately connected.  At a very coarse level, ML works as follows. 
",
    "
",
    "First, you come up somehow with a very complicated model $\vec{y} = M(\vec{x}, \vec{\theta})$, which computes an output $\vec{y}$ as a function of an input $\vec{x}$ and of a vector of parameters $\vec{\theta}$.   In general, $\vec{x}$, $\vec{y}$, and $\vec{\theta}$ are vectors, as the model has multiple inputs, multiple outputs, and several parameters.  The model $M$ needs to be complicated, because only complicated models can represent complicated phenomena; for instance, $M$ can be a multi-layer neural net with parameters $\vec{\theta} = [\theta_1, \ldots, \theta_k]$, where $k$ is the number of parameters of the model. 
",
    "
",
    "Second, you come up with a notion of _loss_ $L$, that is, how badly the model is doing.  For instance, if you have a list of inputs $\vec{x}_1, \ldots, \vec{x}_n$, and a set of desired outputs $\vec{y}_1, \ldots, \vec{y}_m$, you can use as loss: 
",
    "
",
    "$$
",
    "L(\vec{\theta}) = \sum_{i=1}^n |\!|\vec{y}_i - M(\vec{x}_i, \vec{\theta})|\!| \; .
",
    "$$
",
    "
",
    "Here, we wrote $L(\vec{\theta})$ because, once the inputs $\vec{x}_1, \ldots, \vec{x}_n$ and the desired outputs $\vec{y}_1, \ldots, \vec{y}_n$ are chosen, the loss $L$ depends on $\vec{\theta}$. 
",
    "
",
    "Once the loss is chosen, you decrease it, by computing its _gradient_ with respect to $\vec{\theta}$.  Remembering that $\vec{\theta} = [\theta_1, \ldots, \theta_k]$,
",
    "
",
    "$$
",
    "\nabla_\vec{\theta} L = \left[ \frac{\partial L}{\partial \theta_1}, \ldots, 
",
    "    \frac{\partial L}{\partial \theta_k} \right] \; .
",
    "$$
",
    "
",
    "The gradient is a vector that indicates how to tweak $\vec{\theta}$ to decrease the loss.  You then choose a small _step size_ $\delta$, and you update $\vec{\theta}$ via $\vec{\theta} := \vec{\theta} - \delta \nabla_\vec{\theta} L$.  This makes the loss a little bit smaller, and the model a little bit better.  If you repeat this step many times, the model will hopefully get (a good bit) better. 
",
    "
",
    "## Autogradient
",
    "
",
    "The key to _pleasant_ ML is to focus on building the model $M$ in a way that is sufficiently expressive, and on choosing a loss $L$ that is helpful in guiding the optimization.  The computation of the gradient is done automatically for you.  This capability, called _autogradient_, is implemented in ML frameworks such as [Tensorflow](https://www.tensorflow.com), [Keras](https://keras.io), and [PyTorch](https://pytorch.org).  
",
    "
",
    "It is possible to use these advanced ML libraries without ever knowing what is under the hood, and how autogradient works.  Here, we will insted dive in, and implement autogradient.  
",
    "
",
    "Building a model $M$ corresponds to building an expression with inputs $\vec{x}$, $\vec{\theta}$.  We will provide a representaton for expressions that enables both the calculation of the expression value, and the differentiation with respect to any of the inputs.  This will enable us to implement autogradient.  On the basis of this, we will be able to implement a simple ML framework. 
",
    "
",
    "We say we, but we mean you.  _You_ will implement it; we will just provide guidance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "wuxU79AuwyGx",
    "nbgrader": {
     "checksum": "3cc78b9f5abc2f78826438d7f57a22f7",
     "grade": false,
     "grade_id": "cell-2a72086b83ae1784",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Expressions with autogradient
",
    "
",
    "Our main task will be to implement a class `Expr` that represents expressions with autogradient.  
",
    "
",
    "### Implementing expressions
",
    "
",
    "We will have `Expr` be the abstract class of a generic expression, and `Plus`, `Multiply`, and so on, be derived classes representing expression with given top-level operators.  The constructor takes the children node.  The code for the constructor, and the code to create addition expressions, is as follows. 
",
    "
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "n96_CWoewyGy",
    "nbgrader": {
     "checksum": "56716e3515f384375c85c413a6276bc1",
     "grade": false,
     "grade_id": "cell-64a6e1036e3a8a8d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "class Expr(object):
",
    "
",
    "    def __init__(self, *args):
",
    "        """Initializes an expression node, with a given list of children
",
    "        expressions."""
",
    "        self.children = args
",
    "        self.value = None # The value of the expression.
",
    "        self.values = None # The values of the child expressions.
",
    "
",
    "    def __add__(self, other):
",
    "        """Constructs the sum of two expressions."""
",
    "        return Plus(self, other)
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "f2F8BRX6wyGy",
    "nbgrader": {
     "checksum": "96695aebd0b3794e8953533ba6f59042",
     "grade": false,
     "grade_id": "cell-d90eb0fa9962559c",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "The code for the `Plus` class, initially, is empty; no `Expr` methods are over-ridden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "J3eVYoYxwyGy",
    "nbgrader": {
     "checksum": "b9965ff93dc4824f9a9ce11d1e532aa8",
     "grade": false,
     "grade_id": "cell-80283830e855ede9",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "class Plus(Expr):
",
    "    """An addition expression."""
",
    "
",
    "    pass
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "TBO0wZg5wyGy",
    "nbgrader": {
     "checksum": "1af38fcf3d243ce198cc784bd8a9ffcb",
     "grade": false,
     "grade_id": "cell-9bb6b7656beaec0d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "To construct expressions, we need one more thing.  So far, if we write things like `2 + 3`, Python will just consider these as expressions involving numbers, and compute their value.  To write _symbolic_ expressions, we need symbols, or variables.  A variable is a type of expression that just contains a value as child, and that has an `assign` method to assign a value to the variable.  The `assign` method can be used to modify the variable's content (without `assign`, our variables would be constants!). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "eYt7EcHmwyGy",
    "nbgrader": {
     "checksum": "77f8abba50f1e93af8a73de7bd4ce946",
     "grade": false,
     "grade_id": "cell-96c9be270a7105fd",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "class V(Expr):
",
    "    """This class represents a variable.  The derivative rule corresponds
",
    "    to d/dx x = 1, but note that it will not be called, since the children
",
    "    of a variable are just numbers."""
",
    "
",
    "    def assign(self, v):
",
    "        """Assigns a value to the variable."""
",
    "        self.children = [v]
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "uztQHKWVwyGy",
    "nbgrader": {
     "checksum": "ad85ededb82da77030dbc0e4d60f57ed",
     "grade": false,
     "grade_id": "cell-47c1ead7e575e743",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "This suffices for creating expressions.  Let's create one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "S3cdQSfnwyGy",
    "nbgrader": {
     "checksum": "9d611721b84b2da96ae0825eebdab99c",
     "grade": false,
     "grade_id": "cell-2c74422d022963de",
     "locked": true,
     "schema_version": 1,
     "solution": false
    },
    "outputId": "beafb542-7c70-4ac8-dddc-c89a83b61298"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       ""
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e = V(3) + 4
",
    "e
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "sNSIJofWwyGy",
    "nbgrader": {
     "checksum": "f0025404593b37c016edd0487024821b",
     "grade": false,
     "grade_id": "cell-804c6ca38cd4f41e",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Computing the value of expressions
",
    "
",
    "We now have our first expression.  To compute the expression value, we endow each expression with a method `op`, whose task is to compute the value `self.value` of the expression from the list of values `self.values` of the children.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "z7Vm_mB5wyGy",
    "nbgrader": {
     "checksum": "8bd87d4a757b18bfb06f5f2f054d6b58",
     "grade": false,
     "grade_id": "cell-b6aaba31539c892c",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Let's implement the `compute` method for an expression.  This method will: 
",
    "
",
    "1. Loop over the children, and computes the list `self.values` of children values as follows: 
",
    "    * If the child is an expression (an instance of `Expr`), obtain its value by calling `compute` on it. 
",
    "    * If the child is not an instance of `Expr`, then the child must be a number, and we can use its value directly. 
",
    "2. Call the method `op` of the expression, to compute `self.value` from `self.values`. 
",
    "3. Return `self.value`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "tzL62aVKwyGy",
    "nbgrader": {
     "checksum": "445dae83548d8dbfbc688778c08c25c8",
     "grade": false,
     "grade_id": "cell-cd6e1cd62e23ccd6",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "class Expr(object):
",
    "
",
    "    def __init__(self, *args):
",
    "        """Initializes an expression node, with a given list of children
",
    "        expressions."""
",
    "        self.children = args
",
    "        self.value = None # The value of the expression.
",
    "        self.values = None # The values of the child expressions.
",
    "        self.gradient = 0 # The value of the gradient.
",
    "
",
    "    def op(self):
",
    "        """This operator must be implemented in subclasses; it should
",
    "        compute self.value from self.values, thus implementing the
",
    "        operator at the expression node."""
",
    "        raise NotImplementedError()
",
    "
",
    "    def compute(self):
",
    "        """This method computes the value of the expression.
",
    "        It first computes the value of the children expressions,
",
    "        and then uses self.op to compute the value of the expression."""
",
    "        self.values = [e.compute() if isinstance(e, Expr) else e
",
    "                        for e in self.children]
",
    "        # This computes self.value
",
    "        self.op()
",
    "        return self.value
",
    "
",
    "    def __repr__(self):
",
    "        return ("%s:%r %r (g: %r)" % (
",
    "            self.__class__.__name__, self.children, self.value, self.gradient))
",
    "
",
    "    # Expression constructors
",
    "
",
    "    def __add__(self, other):
",
    "        return Plus(self, other)
",
    "
",
    "    def __radd__(self, other):
",
    "        return Plus(self, other)
",
    "
",
    "    def __sub__(self, other):
",
    "        return Minus(self, other)
",
    "
",
    "    def __rsub__(self, other):
",
    "        return Minus(other, self)
",
    "
",
    "    def __mul__(self, other):
",
    "        return Multiply(self, other)
",
    "
",
    "    def __rmul__(self, other):
",
    "        return Multiply(other, self)
",
    "
",
    "    def __truediv__(self, other):
",
    "        return Divide(self, other)
",
    "
",
    "    def __rtruediv__(self, other):
",
    "        return Divide(other, self)
",
    "
",
    "    def __pow__(self, other):
",
    "        return Power(self, other)
",
    "
",
    "    def __rpow__(self, other):
",
    "        return Power(other, self)
",
    "
",
    "    def __neg__(self):
",
    "        return Negative(self)
"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "ZDhYKChqwyGy",
    "nbgrader": {
     "checksum": "3abcdb3ca5f0df789c6427e08a56383d",
     "grade": false,
     "grade_id": "cell-97de33b3461556ef",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Let us give `op` for `Plus`, `Multiply`, and for variables via `V`, so you can see how it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "RKHbN_mgwyGy",
    "nbgrader": {
     "checksum": "113dbe6e85da910fe6131731eb9fd7ff",
     "grade": false,
     "grade_id": "cell-1a1ed551215fcd72",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "class V(Expr):
",
    "    """This class represents a variable."""
",
    "
",
    "    def assign(self, v):
",
    "        """Assigns a value to the variable.  Used to fit a model, so we
",
    "        can assign the various input values to the variable."""
",
    "        self.children = [v]
",
    "
",
    "    def op(self):
",
    "        self.value = self.values[0]
",
    "
",
    "    def __repr__(self):
",
    "        return "Variable: " + str(self.children[0])
",
    "
",
    "
",
    "class Plus(Expr):
",
    "
",
    "    def op(self):
",
    "        self.value = self.values[0] + self.values[1]
",
    "
",
    "class Multiply(Expr):
",
    "
",
    "    def op(self):
",
    "        self.value = self.values[0] * self.values[1]
"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "0b2HUY3NwyGy",
    "nbgrader":

2020/11/19 HW9 - Colaboratory https://colab.research.google.com/drive/1YXdEmkpt-iGdf0qCKdpw9Ne01SZOJ3SE?authuser=1#scrollTo=0G9m-EaXwyGx&printMode=true 1/20 Before you turn this problem in, make sure...

Answer To: 2020/11/19 HW9 - Colaboratory...

Answer To This Question Is Available To Download

Related Questions & Answers

Submit New Assignment