Revision 3f70709c1566f710b607d6a896e7f421c079654a authored by Oliver Woodman on 01 December 2018, 04:13:09 UTC, committed by Oliver Woodman on 01 December 2018, 04:13:09 UTC
1 parent 6fc9037
guide-v1.md
---
layout: default
title: Developer guide (1.x)
exclude_from_menu: true
---
{% include info-box.html content="This guide is for ExoPlayer 1.x, which is now deprecated.
The latest developer guide can be found [here](guide.html)." %}
{% include info-box.html content="The class reference for ExoPlayer 1.x can be found
[here](doc/reference-v1)." %}
<div id="table-of-contents">
<div id="table-of-contents-header">Contents</div>
<div markdown="1">
* TOC
{:toc}
</div>
</div>
Playing videos and music is a popular activity on Android devices. The Android framework provides
[`MediaPlayer`][] as a quick solution for playing media with minimal code. It also provides low
level media APIs such as [`MediaCodec`][], [`AudioTrack`][] and [`MediaDrm`][], which can be used to
build custom media player solutions.
ExoPlayer is an open source, application level media player built on top of Android's low level
media APIs. The open source project contains both the ExoPlayer library and a demo app:
* [ExoPlayer library](https://github.com/google/ExoPlayer/tree/release-v1/library) – This part of
the project contains the core library classes.
* [Demo app](https://github.com/google/ExoPlayer/tree/release-v1/demo) – This part of the project
demonstrates usage of ExoPlayer.
This guide describes the ExoPlayer library and its use. It refers to code in the demo app throughout
in order to provide concrete examples. The guide touches on the pros and cons of using ExoPlayer. It
shows how to use ExoPlayer to play DASH, SmoothStreaming and HLS adaptive streams, as well as
formats such as FMP4, MP4, M4A, MKV, WebM, MP3, AAC, MPEG-TS, MPEG-PS, OGG, FLV and WAV. It also
discusses ExoPlayer events, messages, customization and DRM support.
## Pros and cons ##
ExoPlayer has a number of advantages over Android's built in MediaPlayer:
* Support for Dynamic Adaptive Streaming over HTTP (DASH) and SmoothStreaming, neither of which are
are supported by MediaPlayer (it also supports HTTP Live Streaming (HLS), FMP4, MP4, M4A, MKV,
WebM, MP3, AAC, MPEG-TS, MPEG-PS, OGG, FLV and WAV).
* Support for advanced HLS features, such as correct handling of `#EXT-X-DISCONTINUITY` tags.
* The ability to customize and extend the player to suit your use case. ExoPlayer is designed
specifically with this in mind, and allows many components to be replaced with custom
implementations.
* Easily update the player along with your application. Because ExoPlayer is a library that you
include in your application apk, you have control over which version you use and you can easily
update to a newer version as part of a regular application update.
* Fewer device specific issues.
It's important to note that there are also some disadvantages:
* **ExoPlayer's standard audio and video components rely on Android's `MediaCodec` API, which was
released in Android 4.1 (API level 16). Hence they do not work on earlier versions of Android.**
## Library overview ##
At the core of the ExoPlayer library is the `ExoPlayer` class. This class maintains the player’s
global state, but makes few assumptions about the nature of the media being played, such as how the
media data is obtained, how it is buffered or its format. You inject this functionality through
ExoPlayer’s `prepare` method in the form of `TrackRenderer` objects.
ExoPlayer provides default audio and video renderers, which make use of the `MediaCodec` and
`AudioTrack` classes in the Android framework. Both require an injected `SampleSource` object,
from which they obtain individual media samples for playback.
Injection of components is a theme present throughout the ExoPlayer library. Figure 1 shows the
high level object model for an ExoPlayer configured to play MP4 streams. Default audio and video
renderers are injected into the `ExoPlayer` instance. An instance of a class called
`ExtractorSampleSource` is injected into the renderers to provide them with media samples.
`DataSource` and `Extractor` instances are injected into the `ExtractorSampleSource` to allow it
load the media stream and extract samples from the loaded data. In this case `DefaultUriDataSource`
and `Mp4Extractor` are used to play MP4 streams loaded from their URIs.
{% include figure.html url="/images/standard-model.png" index="1" caption="Object model for MP4 playbacks using ExoPlayer" %}
In summary, ExoPlayer instances are built by injecting components that provide the functionality
that the developer requires. This model makes it easy to build players for specific use cases, and
to inject custom components. The following sections outline three of the most important interfaces
in this model: `TrackRenderer`, `SampleSource` and `DataSource`.
### TrackRenderer ###
A `TrackRenderer` plays a specific type of media, such as video, audio or text. The ExoPlayer class
invokes methods on its `TrackRenderer` instances from a single playback thread, and by doing so
causes each type of media be rendered as the global playback position is advanced. The ExoPlayer
library provides `MediaCodecVideoTrackRenderer` as the default implementation for rendering video,
and `MediaCodecAudioTrackRenderer` for audio. Both implementations make use of Android's MediaCodec
class to decode individual media samples. They can handle all audio and video formats supported by a
given Android device (see
[Supported Media Formats](https://developer.android.com/guide/appendix/media-formats.html) for
details). The ExoPlayer library also provides an implementation for rendering text, called
`TextTrackRenderer`.
The code below is an example of the main steps required to instantiate an ExoPlayer to play video
and audio using the standard `TrackRenderer` implementations.
{% highlight java %}
// 1. Instantiate the player.
player = ExoPlayer.Factory.newInstance(RENDERER_COUNT);
// 2. Construct renderers.
MediaCodecVideoTrackRenderer videoRenderer = ...
MediaCodecAudioTrackRenderer audioRenderer = ...
// 3. Inject the renderers through prepare.
player.prepare(videoRenderer, audioRenderer);
// 4. Pass the surface to the video renderer.
player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface);
// 5. Start playback.
player.setPlayWhenReady(true);
...
player.release(); // Don’t forget to release when done!
{% endhighlight %}
For a complete example, see `PlayerActivity` and `DemoPlayer` in the ExoPlayer demo app. Between
them these classes correctly manage an ExoPlayer instance with respect to both the `Activity` and
`Surface` lifecycles.
### SampleSource ###
The standard `TrackRenderer` implementations provided by the library require `SampleSource`
instances to be injected into their constructors. A `SampleSource` object provides format
information and media samples to be rendered. The ExoPlayer library provides several concrete
`SampleSource` implementations for different use cases:
* `ExtractorSampleSource` – For formats such as FMP4, MP4, M4A, MKV, WebM, MP3, AAC, MPEG-TS,
MPEG-PS, OGG, FLV and WAV.
* `ChunkSampleSource` – For DASH and SmoothStreaming playbacks.
* `HlsSampleSource` – For HLS playbacks.
The use of these implementations is outlined in more detail later in this guide.
### DataSource ###
The standard `SampleSource` implementations provided by the library make use of `DataSource`
instances for loading media data. Various implementations can be found in the `upstream` package.
The most commonly used implementations are:
* `DefaultUriDataSource` – For playing media that can be either local or loaded over the
network.
* `AssetDataSource` – For playing media stored in the `assets` folder of the application's
apk.
## Traditional media playbacks ##
The Exoplayer library provides `ExtractorSampleSource` to play traditional media formats, including
FMP4, MP4, M4A, MKV, WebM, MP3, AAC, MPEG-TS, MPEG-PS, OGG, FLV and WAV. The diagram in Figure 1
shows the object model for an ExoPlayer built to play MP4 streams. The following code shows how the
`TrackRenderer` instances are constructed.
{% highlight java %}
Allocator allocator = new DefaultAllocator(BUFFER_SEGMENT_SIZE);
DataSource dataSource = new DefaultUriDataSource(context, null, userAgent);
ExtractorSampleSource sampleSource = new ExtractorSampleSource(
uri, dataSource, allocator, BUFFER_SEGMENT_COUNT * BUFFER_SEGMENT_SIZE);
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(
context, sampleSource, MediaCodecSelector.DEFAULT, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT);
MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(
sampleSource, MediaCodecSelector.DEFAULT);
{% endhighlight %}
`ExtractorSampleSource` will automatically load the correct `Extractor` for the media being played.
If there is no `Extractor` for the media you wish to play, it's possible to implement and inject
your own.
The ExoPlayer demo app provides a complete implementation of this code in
`ExtractorRendererBuilder`. The `PlayerActivity` class uses it to play some of the videos available
in the demo app.
## Adaptive media playbacks ##
ExoPlayer supports adaptive streaming, which allows the quality of the media data to be adjusted
during playback based on the network conditions. DASH, SmoothStreaming and HLS are examples of
adaptive streaming technologies. In all three, media is loaded in small chunks (typically 2 to 10
seconds in duration). Whenever a chunk of media is requested, the client selects from a number of
possible formats. For example, a client may select a high quality format if network conditions are
good, or a low quality format if network conditions are bad. In both techniques, video and audio are
streamed separately.
### DASH and SmoothStreaming ###
ExoPlayer supports DASH and SmoothStreaming adaptive playbacks through use of `ChunkSampleSource`,
which loads chunks of media data from which individual samples can be extracted. Each
`ChunkSampleSource` requires a `ChunkSource` object to be injected through its constructor, which is
responsible for providing media chunks from which to load and read samples. The `DashChunkSource`
class provides DASH playback using the FMP4 and WebM container formats. The
`SmoothStreamingChunkSource` class provides SmoothStreaming playback using the FMP4 container
format.
Both of the standard `ChunkSource` implementations require a `FormatEvaluator` and a `DataSource` to
be injected through their constructors. The `FormatEvaluator` objects select from the available
formats before each chunk is loaded, and the `DataSource` provides the means to actually load the
data. Finally, the `ChunkSampleSource` requires a `LoadControl` object that controls the chunk
buffering policy.
The typical object model for an ExoPlayer configured for DASH adaptive playbacks is shown in Figure
2. The video quality is varied at runtime using the adaptive implementation of `FormatEvaluator`,
while audio is played at a fixed quality level.
{% include figure.html url="/images/dash-model.png" index="2" caption="Object model for DASH adaptive playbacks using ExoPlayer" %}
The following code example outlines how the video and audio renderers are constructed.
{% highlight java %}
LoadControl loadControl = new DefaultLoadControl(new DefaultAllocator(BUFFER_SEGMENT_SIZE));
DefaultBandwidthMeter bandwidthMeter = new DefaultBandwidthMeter();
// Build the video renderer.
DataSource videoDataSource = new DefaultUriDataSource(context, bandwidthMeter, userAgent);
ChunkSource videoChunkSource = new DashChunkSource(manifestFetcher,
DefaultDashTrackSelector.newVideoInstance(context, true, false), videoDataSource,
new AdaptiveEvaluator(bandwidthMeter), LIVE_EDGE_LATENCY_MS, elapsedRealtimeOffset, null, null,
DemoPlayer.TYPE_VIDEO);
ChunkSampleSource videoSampleSource = new ChunkSampleSource(videoChunkSource, loadControl,
VIDEO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE);
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(context,
videoSampleSource, MediaCodecSelector.DEFAULT,
MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT);
// Build the audio renderer.
DataSource audioDataSource = new DefaultUriDataSource(context, bandwidthMeter, userAgent);
ChunkSource audioChunkSource = new DashChunkSource(manifestFetcher,
DefaultDashTrackSelector.newAudioInstance(), audioDataSource, null, LIVE_EDGE_LATENCY_MS,
elapsedRealtimeOffset, null, null, DemoPlayer.TYPE_AUDIO);
ChunkSampleSource audioSampleSource = new ChunkSampleSource(audioChunkSource,
loadControl, AUDIO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE);
MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(
audioSampleSource, MediaCodecSelector.DEFAULT);
{% endhighlight %}
In this code, `manifestFetcher` is an object responsible for loading the DASH manifest that defines
the media. The `videoAdaptationSetIndex` and `audioAdaptationSetIndex` variables index components
of the initially loaded manifest that correspond to video and audio respectively.
The ExoPlayer demo app provides a complete implementation of this code in `DashRendererBuilder`. The
`PlayerActivity` class uses this builder to construct renderers for playing DASH sample videos in
the demo app. For an equivalent SmoothStreaming example, see the `SmoothStreamingRendererBuilder`
class in the demo app.
### HLS ###
ExoPlayer supports HLS adaptive playbacks through use of `HlsSampleSource`, which loads chunks of
media data from which individual samples can be extracted. A `HlsSampleSource` requires a
`HlsChunkSource` to be injected through its constructor, which is responsible for providing media
chunks from which to load and read samples. A `HlsChunkSource` requires a `DataSource` to be
injected through its constructor, through which the media data can be loaded.
A typical object model for an ExoPlayer configured for HLS adaptive playbacks is shown in Figure
3. The video quality is varied at runtime using the adaptive implementation of 'FormatEvaluator',
while audio is played at a fixed quality level.
{% include figure.html url="/images/hls-model.png" index="3" caption="Object model for HLS playbacks using ExoPlayer" %}
The following code example outlines how the video and audio renderers are constructed.
{% highlight java %}
LoadControl loadControl = new DefaultLoadControl(new DefaultAllocator(BUFFER_SEGMENT_SIZE));
DefaultBandwidthMeter bandwidthMeter = new DefaultBandwidthMeter();
PtsTimestampAdjusterProvider timestampAdjusterProvider = new PtsTimestampAdjusterProvider();
DataSource dataSource = new DefaultUriDataSource(context, bandwidthMeter, userAgent);
HlsChunkSource chunkSource = new HlsChunkSource(true /* isMaster */, dataSource, url, manifest,
DefaultHlsTrackSelector.newDefaultInstance(context), bandwidthMeter, timestampAdjusterProvider,
HlsChunkSource.ADAPTIVE_MODE_SPLICE);
HlsSampleSource sampleSource = new HlsSampleSource(chunkSource, loadControl,
MAIN_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE);
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(context, sampleSource,
MediaCodecSelector.DEFAULT, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT);
MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(sampleSource,
MediaCodecSelector.DEFAULT);
{% endhighlight %}
The ExoPlayer demo app provides complete implementation of this code in `HlsRendererBuilder`. The
`PlayerActivity` class uses this builder to construct renderers for playing HLS sample videos in
the demo app.
## Player events ##
During playback, your app can listen for events generated by ExoPlayer that indicate the overall
state of the player. These events are useful as triggers for updating the app user interface such as
playback controls. Many ExoPlayer components also report their own component specific low level
events, which can be useful for performance monitoring.
### High level events ###
ExoPlayer allows instances of `ExoPlayer.Listener` to be added and removed using its `addListener()`
and `removeListener()` methods. Registered listeners are notified of changes in playback state, as
well as when errors occur that cause playback to fail. For more information about the valid playback
states and the possible transitions between them, see the ExoPlayer source code.
Developers who implement custom playback controls should register a listener and use it to update
their controls as the player’s state changes. An app should also show an appropriate error to the
user if playback fails.
### Low level events ###
In addition to high level listeners, many of the individual components provided by the ExoPlayer
library allow their own event listeners. For example, `MediaCodecVideoTrackRenderer` has
constructors that take a `MediaCodecVideoTrackRenderer.EventListener`. In the ExoPlayer demo app,
`DemoPlayer` acts as the listener to multiple individual components, forwarding events to
`PlayerActivity`. This approach allows `PlayerActivity` to adjust the dimensions of the target
surface to have the correct height and width ratio for the video being played:
{% highlight java %}
@Override
public void onVideoSizeChanged(int width, int height, int unappliedRotationDegrees,
float pixelWidthHeightRatio) {
surfaceView.setVideoWidthHeightRatio(
height == 0 ? 1 : (width * pixelWidthAspectRatio) / height);
}
{% endhighlight %}
The `RendererBuilder` classes in the ExoPlayer demo app inject the `DemoPlayer` as the listener to
each component.
Note that you must pass a `Handler` object to the renderer, which determines the thread on which
the listener’s methods are invoked. In most cases, you should use a `Handler` associated with the
app’s main thread, as is the case in this example.
Listening to individual components can be useful for adjusting UI based on player events, as in the
example above. Listening to component events can also be helpful for logging performance metrics.
For example, `MediaCodecVideoTrackRenderer` notifies its listener of dropped video frames. A
developer may wish to log such metrics to track playback performance in their app.
Many components also notify their listeners when errors occur. Such errors may or may not cause
playback to fail. If an error does not cause playback to fail, it may still result in degraded
performance, and so you may wish to log all errors in order to track playback performance. Note that
an ExoPlayer instance always notifies its high level listeners of errors that cause playback to
fail, in addition to the listener of the individual component from which the error originated.
Hence, you should display error messages to users only from high level listeners. Within individual
component listeners, you should use error notifications only for informational purposes.
## Sending messages to components ##
Some ExoPlayer components allow changes in configuration during playback. By convention, you
make these changes by passing asynchronous messages through the ExoPlayer to the component. This
approach ensures both thread safety and that the configuration change is executed in order with any
other operations being performed on the player.
The most common use of messaging is passing a target surface to`MediaCodecVideoTrackRenderer`:
{% highlight java %}
player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface);
{% endhighlight %}
Note that if the surface needs to be cleared because `SurfaceHolder.Callback.surfaceDestroyed()` has
been invoked, then you must send this message using the blocking variant of `sendMessage()`:
{% highlight java %}
player.blockingSendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, null);
{% endhighlight %}
You must use a blocking message because the contract of `surfaceDestroyed()` requires that the app
does not attempt to access the surface after the method returns.
## Customization ##
One of the main benefits of ExoPlayer over Android's `MediaPlayer` is the ability to customize and
extend the player to better suit the developer’s use case. The ExoPlayer library is designed
specifically with this in mind, defining a number of abstract base classes and interfaces that make
it possible for app developers to easily replace the default implementations provided by the
library. Here are some use cases for building custom components:
* `TrackRenderer` – You may want to implement a custom `TrackRenderer` to handle media types
other than audio and video. The `TextTrackRenderer` class within the ExoPlayer library is an
example of how to implement a custom renderer. You could use the approach it demonstrates to
render custom overlays or annotations. Implementing this kind of functionality as a
`TrackRenderer` makes it easy to keep the overlays or annotations in sync with the other media
being played.
* `Extractor` – If you need to support a container format not currently supported by the
ExoPlayer library, consider implementing a custom `Extractor` class, which can then be used to
together with `ExtractorSampleSource` to play media of that type.
* `SampleSource` – Implementing a custom `SampleSource` class may be appropriate if you wish
to obtain media samples to feed to renderers in a custom way.
* `FormatEvaluator` – For DASH and SmoothStreaming playbacks, the ExoPlayer library provides
`FormatEvaluator.AdaptiveEvaluator` as a simple reference implementation that switches between
different quality formats based on the available bandwidth. App developers are encouraged to
develop their own adaptive `FormatEvaluator` implementations, which can be designed to suit their
use specific needs.
* `DataSource` – ExoPlayer’s upstream package already contains a number of `DataSource`
implementations for different use cases. You may want to implement you own `DataSource` class to
load data in another way, such as over a custom protocol, using a custom HTTP stack, or through
a persistent cache.
### Customization guidelines ###
* If a custom component needs to report events back to the app, we recommend that you do so using
the same model as existing ExoPlayer components, where an event listener is passed together with a
`Handler` to the constructor of the component.
* We recommended that custom components use the same model as existing ExoPlayer components to allow
reconfiguration by the app during playback, as described in
[Sending messages to components](#sending-messages-to-components). To do this, you should
implement an `ExoPlayerComponent` and receive configuration changes in its `handleMessage()`
method. Your app should pass configuration changes by calling ExoPlayer’s `sendMessage()` and
`blockingSendMessage()` methods.
## Digital Rights Management ##
On Android 4.4 (API level 19) and higher, ExoPlayer supports Digital Rights Managment (DRM)
protected playback. In order to play DRM protected content with ExoPlayer, your app must inject a
`DrmSessionManager` into the `MediaCodecVideoTrackRenderer` and `MediaCodecAudioTrackRenderer`
constructors. A `DrmSessionManager` object is responsible for providing the `MediaCrypto` object
required for decryption, as well as ensuring that the required decryption keys are available to the
underlying DRM module being used.
The ExoPlayer library provides a default implementation of `DrmSessionManager`,
called `StreamingDrmSessionManager`, which uses `MediaDrm`. The session manager supports any DRM
scheme for which a modular DRM component exists on the device. All Android devices are required to
support Widevine modular DRM (with L3 security, although many devices also support L1). Some devices
may support additional schemes such as PlayReady.
The `StreamingDrmSessionManager` class requires a `MediaDrmCallback` to be injected into its
constructor, which is responsible for actually making provisioning and key requests. You should
implement this interface to make network requests to your license server and obtain the required
keys. The `WidevineTestMediaDrmCallback` class in the ExoPlayer demo app sends requests to a
Widevine test server.
[`MediaPlayer`]: {{ site.android_sdk }}/android/media/MediaPlayer.html
[`MediaCodec`]: {{ site.android_sdk }}/android/media/MediaCodec.html
[`AudioTrack`]: {{ site.android_sdk }}/android/media/AudioTrack.html
[`MediaDrm`]: {{ site.android_sdk }}/android/media/MediaDrm.html
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...