Revision c423091432d55fa88c2799181b9d3575213530c8 authored by Huaxin Gao on 17 October 2017, 19:50:41 UTC, committed by gatorsmile on 17 October 2017, 19:53:14 UTC
## What changes were proposed in this pull request?

In Average.scala, it has
```
  override lazy val evaluateExpression = child.dataType match {
    case DecimalType.Fixed(p, s) =>
      // increase the precision and scale to prevent precision loss
      val dt = DecimalType.bounded(p + 14, s + 4)
      Cast(Cast(sum, dt) / Cast(count, dt), resultType)
    case _ =>
      Cast(sum, resultType) / Cast(count, resultType)
  }

  def setChild (newchild: Expression) = {
    child = newchild
  }

```
It is possible that  Cast(count, dt), resultType) will make the precision of the decimal number bigger than 38, and this causes over flow.  Since count is an integer and doesn't need a scale, I will cast it using DecimalType.bounded(38,0)
## How was this patch tested?
In DataFrameSuite, I will add a test case.

Please review http://spark.apache.org/contributing.html before opening a pull request.

Author: Huaxin Gao <huaxing@us.ibm.com>

Closes #19496 from huaxingao/spark-22271.

(cherry picked from commit 28f9f3f22511e9f2f900764d9bd5b90d2eeee773)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>

# Conflicts:
#	sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
1 parent 71d1cb6
History
File Mode Size
LICENSE-AnchorJS.txt -rw-r--r-- 1.1 KB
LICENSE-DPark.txt -rw-r--r-- 1.5 KB
LICENSE-Mockito.txt -rw-r--r-- 1.1 KB
LICENSE-SnapTree.txt -rw-r--r-- 1.7 KB
LICENSE-antlr.txt -rw-r--r-- 1.5 KB
LICENSE-boto.txt -rw-r--r-- 1.0 KB
LICENSE-cloudpickle.txt -rw-r--r-- 1.6 KB
LICENSE-d3.min.js.txt -rw-r--r-- 1.4 KB
LICENSE-dagre-d3.txt -rw-r--r-- 1.0 KB
LICENSE-f2j.txt -rw-r--r-- 1.5 KB
LICENSE-graphlib-dot.txt -rw-r--r-- 1.0 KB
LICENSE-heapq.txt -rw-r--r-- 14.2 KB
LICENSE-javolution.txt -rw-r--r-- 1.4 KB
LICENSE-jbcrypt.txt -rw-r--r-- 836 bytes
LICENSE-jline.txt -rw-r--r-- 1.5 KB
LICENSE-jpmml-model.txt -rw-r--r-- 1.5 KB
LICENSE-jquery.txt -rw-r--r-- 1.1 KB
LICENSE-junit-interface.txt -rw-r--r-- 1.3 KB
LICENSE-kryo.txt -rw-r--r-- 1.5 KB
LICENSE-minlog.txt -rw-r--r-- 1.5 KB
LICENSE-modernizr.txt -rw-r--r-- 1.1 KB
LICENSE-netlib.txt -rw-r--r-- 2.2 KB
LICENSE-paranamer.txt -rw-r--r-- 1.6 KB
LICENSE-postgresql.txt -rw-r--r-- 1.2 KB
LICENSE-protobuf.txt -rw-r--r-- 2.1 KB
LICENSE-py4j.txt -rw-r--r-- 1.4 KB
LICENSE-pyrolite.txt -rw-r--r-- 1.3 KB
LICENSE-reflectasm.txt -rw-r--r-- 1.5 KB
LICENSE-sbt-launch-lib.txt -rw-r--r-- 1.5 KB
LICENSE-scala.txt -rw-r--r-- 1.5 KB
LICENSE-scalacheck.txt -rw-r--r-- 1.5 KB
LICENSE-scopt.txt -rw-r--r-- 1.1 KB
LICENSE-slf4j.txt -rw-r--r-- 1.1 KB
LICENSE-sorttable.js.txt -rw-r--r-- 937 bytes
LICENSE-spire.txt -rw-r--r-- 1.0 KB
LICENSE-xmlenc.txt -rw-r--r-- 1.5 KB

back to top