Revision 9c1678534e9087859aaa33e83d996c68685af0c0 authored by Dongjoon Hyun on 16 February 2023, 05:52:17 UTC, committed by Dongjoon Hyun on 16 February 2023, 05:52:54 UTC
### What changes were proposed in this pull request?

This is found during Apache Spark 3.3.2 docker image publishing. It's not an Apache Spark but important for `docker-image-tool.sh` to provide backward compatibility during cross-building. This PR targets for all **future releases**, Apache Spark 3.4.0/3.3.3/3.2.4.

### Why are the changes needed?

Docker `buildx` v0.10.0 publishes OCI Manifests by default which is not supported by `docker manifest` command like the following.
https://github.com/docker/buildx/issues/1509
```
$ docker manifest inspect apache/spark:v3.3.2
no such manifest: docker.io/apache/spark:v3.3.2
```

Note that the published images are working on both AMD64/ARM64 machines, but `docker manifest` cannot be used. For example, we cannot create `latest` tag.

### Does this PR introduce _any_ user-facing change?

This will fix the regression of Docker `buildx`.

### How was this patch tested?

Manually builds the multi-arch image and check `manifest`.

```
$ docker manifest inspect apache/spark:v3.3.2
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3444,
         "digest": "sha256:30ae5023fc384ae3b68d2fb83adde44b1ece05f926cfceecac44204cdc9e79cb",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3444,
         "digest": "sha256:aac13b5b5a681aefa91036d2acae91d30a743c2e78087c6df79af4de46a16e1b",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}
```

Closes #40051 from dongjoon-hyun/SPARK-42462.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 2ac70ae5381333aa899d82f6cd4c3bbae524e1c2)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 07b7199
History

README.md

back to top