Revision 287440df6816b5c9f2be2aee949a4c20ab165180 authored by jerryshao on 31 May 2017, 03:24:43 UTC, committed by Wenchen Fan on 31 May 2017, 03:24:50 UTC
## What changes were proposed in this pull request?

Current HistoryServer will display completed date of in-progress application as `1969-12-31 23:59:59`, which is not so meaningful. Instead of unnecessarily showing this incorrect completed date, here propose to make this column invisible for in-progress applications.

The purpose of only making this column invisible rather than deleting this field is that: this data is fetched through REST API, and in the REST API  the format is like below shows, in which `endTime` matches `endTimeEpoch`. So instead of changing REST API to break backward compatibility, here choosing a simple solution to only make this column invisible.

```
[ {
  "id" : "local-1491805439678",
  "name" : "Spark shell",
  "attempts" : [ {
    "startTime" : "2017-04-10T06:23:57.574GMT",
    "endTime" : "1969-12-31T23:59:59.999GMT",
    "lastUpdated" : "2017-04-10T06:23:57.574GMT",
    "duration" : 0,
    "sparkUser" : "",
    "completed" : false,
    "startTimeEpoch" : 1491805437574,
    "endTimeEpoch" : -1,
    "lastUpdatedEpoch" : 1491805437574
  } ]
} ]%
```

Here is UI before changed:

<img width="1317" alt="screen shot 2017-04-10 at 3 45 57 pm" src="https://cloud.githubusercontent.com/assets/850797/24851938/17d46cc0-1e08-11e7-84c7-90120e171b41.png">

And after:

<img width="1281" alt="screen shot 2017-04-10 at 4 02 35 pm" src="https://cloud.githubusercontent.com/assets/850797/24851945/1fe9da58-1e08-11e7-8d0d-9262324f9074.png">

## How was this patch tested?

Manual verification.

Author: jerryshao <sshao@hortonworks.com>

Closes #17588 from jerryshao/SPARK-20275.

(cherry picked from commit 52ed9b289d169219f7257795cbedc56565a39c71)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 5fdc7d8
History
File Mode Size
docs
lib
pyspark
test_support
.gitignore -rw-r--r-- 49 bytes
MANIFEST.in -rw-r--r-- 1.0 KB
README.md -rw-r--r-- 1.8 KB
pylintrc -rw-r--r-- 13.7 KB
run-tests -rwxr-xr-x 899 bytes
run-tests.py -rwxr-xr-x 8.3 KB
setup.cfg -rw-r--r-- 854 bytes
setup.py -rw-r--r-- 9.4 KB

README.md

back to top