[FFmpeg-devel] [PATCH v2 01/12] avcodec/mediacodec: fix incorrect crop info

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed

* [FFmpeg-devel] [PATCH v2 01/12] avcodec/mediacodec: fix incorrect crop info
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 02/12] avcodec/mediacodecdec: don't break out if both input and output port return try again Zhao Zhili
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Aman Karmani, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

The crop info is optional, but used unconditionally.

Co-authored-by: Aman Karmani <ffmpeg@tmm1.net>
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/mediacodecdec_common.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c
index 2a605e7f5b..f430cfed31 100644
--- a/libavcodec/mediacodecdec_common.c
+++ b/libavcodec/mediacodecdec_common.c
@@ -487,8 +487,20 @@ static int mediacodec_dec_parse_format(AVCodecContext *avctx, MediaCodecDecConte
     AMEDIAFORMAT_GET_INT32(s->crop_left,   "crop-left",   0);
     AMEDIAFORMAT_GET_INT32(s->crop_right,  "crop-right",  0);
 
-    width = s->crop_right + 1 - s->crop_left;
-    height = s->crop_bottom + 1 - s->crop_top;
+    if (s->crop_right && s->crop_bottom) {
+        width = s->crop_right + 1 - s->crop_left;
+        height = s->crop_bottom + 1 - s->crop_top;
+    } else {
+        /* TODO: NDK MediaFormat should try getRect() first.
+         * Try crop-width/crop-height, it works on NVIDIA Shield.
+         */
+        AMEDIAFORMAT_GET_INT32(width,  "crop-width",  0);
+        AMEDIAFORMAT_GET_INT32(height, "crop-height", 0);
+    }
+    if (!width || !height) {
+        width = s->width;
+        height = s->height;
+    }
 
     AMEDIAFORMAT_GET_INT32(s->display_width,  "display-width",  0);
     AMEDIAFORMAT_GET_INT32(s->display_height, "display-height", 0);
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 02/12] avcodec/mediacodecdec: don't break out if both input and output port return try again
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 01/12] avcodec/mediacodec: fix incorrect crop info Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 03/12] avcodec/mediacodecdec_common: fix misuse av_free/av_freep Zhao Zhili
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

At the beginning of decoding, if we feed mediacodec too fast, the
input port will return try again. It takes some time for mediacodec
to consume bitstream and output frame. So the output port also return
try again. It possible that mediacodec_receive_frame doesn't consume
any AVPacket and no AVFrame is output. Then both avcodec_send_packet()
and avcodec_receive_frame() return EAGAIN, which shouldn't happen.

This bug can be produced with decoding benchmark on Pixel 3.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/mediacodecdec.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c
index 322b448d27..2c66f38541 100644
--- a/libavcodec/mediacodecdec.c
+++ b/libavcodec/mediacodecdec.c
@@ -444,7 +444,16 @@ static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame)
             index = ff_AMediaCodec_dequeueInputBuffer(s->ctx->codec, 0);
             if (index < 0) {
                 /* no space, block for an output frame to appear */
-                return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true);
+                ret = ff_mediacodec_dec_receive(avctx, s->ctx, frame, true);
+                /* Try again if both input port and output port return EAGAIN.
+                 * If no data is consumed and no frame in output, it can make
+                 * both avcodec_send_packet() and avcodec_receive_frame()
+                 * return EAGAIN, which violate the design.
+                 */
+                if (ff_AMediaCodec_infoTryAgainLater(s->ctx->codec, index) &&
+                    ret == AVERROR(EAGAIN))
+                    continue;
+                return ret;
             }
             s->ctx->current_input_buffer = index;
         }
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 03/12] avcodec/mediacodecdec_common: fix misuse av_free/av_freep
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 01/12] avcodec/mediacodec: fix incorrect crop info Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 02/12] avcodec/mediacodecdec: don't break out if both input and output port return try again Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 04/12] avcodec/mediacodecdec_common: fix useless av_buffer_unref Zhao Zhili
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/mediacodecdec_common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c
index f430cfed31..4c48b9142e 100644
--- a/libavcodec/mediacodecdec_common.c
+++ b/libavcodec/mediacodecdec_common.c
@@ -334,7 +334,7 @@ static int mediacodec_wrap_hw_buffer(AVCodecContext *avctx,
 
     return 0;
 fail:
-    av_freep(buffer);
+    av_freep(&buffer);
     av_buffer_unref(&frame->buf[0]);
     status = ff_AMediaCodec_releaseOutputBuffer(s->codec, index, 0);
     if (status < 0) {
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 04/12] avcodec/mediacodecdec_common: fix useless av_buffer_unref
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (2 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 03/12] avcodec/mediacodecdec_common: fix misuse av_free/av_freep Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 05/12] avcodec/mediacodec_wrapper: separate implementation from interface Zhao Zhili
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

Since frame->buf[0] is always NULL in this case, av_buffer_unref
has no effect. If it's not NULL, double-free will happen.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/mediacodecdec_common.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c
index 4c48b9142e..69a462ec48 100644
--- a/libavcodec/mediacodecdec_common.c
+++ b/libavcodec/mediacodecdec_common.c
@@ -335,7 +335,6 @@ static int mediacodec_wrap_hw_buffer(AVCodecContext *avctx,
     return 0;
 fail:
     av_freep(&buffer);
-    av_buffer_unref(&frame->buf[0]);
     status = ff_AMediaCodec_releaseOutputBuffer(s->codec, index, 0);
     if (status < 0) {
         av_log(avctx, AV_LOG_ERROR, "Failed to release output buffer\n");
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 05/12] avcodec/mediacodec_wrapper: separate implementation from interface
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (3 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 04/12] avcodec/mediacodecdec_common: fix useless av_buffer_unref Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 06/12] avcodec/mediacodec: add NDK media codec wrapper Zhao Zhili
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

This is in preparation for NDK media codec wrapper.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/mediacodec_wrapper.c | 244 +++++++++++++++++++++---------
 libavcodec/mediacodec_wrapper.h | 255 +++++++++++++++++++++++++++-----
 2 files changed, 394 insertions(+), 105 deletions(-)

diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c
index 8ffc58e1d8..e0c614680e 100644
--- a/libavcodec/mediacodec_wrapper.c
+++ b/libavcodec/mediacodec_wrapper.c
@@ -160,12 +160,14 @@ static const AVClass amediaformat_class = {
     .version    = LIBAVUTIL_VERSION_INT,
 };
 
-struct FFAMediaFormat {
+typedef struct FFAMediaFormatJni {
+    FFAMediaFormat api;
 
-    const AVClass *class;
     struct JNIAMediaFormatFields jfields;
     jobject object;
-};
+} FFAMediaFormatJni;
+
+static const FFAMediaFormat media_format_jni;
 
 struct JNIAMediaCodecFields {
 
@@ -272,9 +274,8 @@ static const AVClass amediacodec_class = {
     .version    = LIBAVUTIL_VERSION_INT,
 };
 
-struct FFAMediaCodec {
-
-    const AVClass *class;
+typedef struct FFAMediaCodecJni {
+    FFAMediaCodec api;
 
     struct JNIAMediaCodecFields jfields;
 
@@ -295,7 +296,9 @@ struct FFAMediaCodec {
     int CONFIGURE_FLAG_ENCODE;
 
     int has_get_i_o_buffer;
-};
+} FFAMediaCodecJni;
+
+static const FFAMediaCodec media_codec_jni;
 
 #define JNI_GET_ENV_OR_RETURN(env, log_ctx, ret) do {              \
     (env) = ff_jni_get_env(log_ctx);                               \
@@ -622,17 +625,17 @@ done:
     return name;
 }
 
-FFAMediaFormat *ff_AMediaFormat_new(void)
+static FFAMediaFormat *mediaformat_jni_new(void)
 {
     JNIEnv *env = NULL;
-    FFAMediaFormat *format = NULL;
+    FFAMediaFormatJni *format = NULL;
     jobject object = NULL;
 
-    format = av_mallocz(sizeof(FFAMediaFormat));
+    format = av_mallocz(sizeof(*format));
     if (!format) {
         return NULL;
     }
-    format->class = &amediaformat_class;
+    format->api = media_format_jni;
 
     env = ff_jni_get_env(format);
     if (!env) {
@@ -664,19 +667,19 @@ fail:
         av_freep(&format);
     }
 
-    return format;
+    return (FFAMediaFormat *)format;
 }
 
-static FFAMediaFormat *ff_AMediaFormat_newFromObject(void *object)
+static FFAMediaFormat *mediaformat_jni_newFromObject(void *object)
 {
     JNIEnv *env = NULL;
-    FFAMediaFormat *format = NULL;
+    FFAMediaFormatJni *format = NULL;
 
-    format = av_mallocz(sizeof(FFAMediaFormat));
+    format = av_mallocz(sizeof(*format));
     if (!format) {
         return NULL;
     }
-    format->class = &amediaformat_class;
+    format->api = media_format_jni;
 
     env = ff_jni_get_env(format);
     if (!env) {
@@ -693,7 +696,7 @@ static FFAMediaFormat *ff_AMediaFormat_newFromObject(void *object)
         goto fail;
     }
 
-    return format;
+    return (FFAMediaFormat *)format;
 fail:
     ff_jni_reset_jfields(env, &format->jfields, jni_amediaformat_mapping, 1, format);
 
@@ -702,10 +705,10 @@ fail:
     return NULL;
 }
 
-int ff_AMediaFormat_delete(FFAMediaFormat* format)
+static int mediaformat_jni_delete(FFAMediaFormat* ctx)
 {
     int ret = 0;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
 
     if (!format) {
@@ -724,10 +727,10 @@ int ff_AMediaFormat_delete(FFAMediaFormat* format)
     return ret;
 }
 
-char* ff_AMediaFormat_toString(FFAMediaFormat* format)
+static char* mediaformat_jni_toString(FFAMediaFormat* ctx)
 {
     char *ret = NULL;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
     jstring description = NULL;
 
@@ -749,10 +752,10 @@ fail:
     return ret;
 }
 
-int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out)
+static int mediaformat_jni_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out)
 {
     int ret = 1;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
     jstring key = NULL;
     jboolean contains_key;
@@ -788,10 +791,10 @@ fail:
     return ret;
 }
 
-int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out)
+static int mediaformat_jni_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out)
 {
     int ret = 1;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
     jstring key = NULL;
     jboolean contains_key;
@@ -827,10 +830,10 @@ fail:
     return ret;
 }
 
-int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out)
+static int mediaformat_jni_getFloat(FFAMediaFormat* ctx, const char *name, float *out)
 {
     int ret = 1;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
     jstring key = NULL;
     jboolean contains_key;
@@ -866,10 +869,10 @@ fail:
     return ret;
 }
 
-int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size)
+static int mediaformat_jni_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size)
 {
     int ret = 1;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
     jstring key = NULL;
     jboolean contains_key;
@@ -924,10 +927,10 @@ fail:
     return ret;
 }
 
-int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out)
+static int mediaformat_jni_getString(FFAMediaFormat* ctx, const char *name, const char **out)
 {
     int ret = 1;
-
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
     JNIEnv *env = NULL;
     jstring key = NULL;
     jboolean contains_key;
@@ -974,10 +977,11 @@ fail:
     return ret;
 }
 
-void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value)
+static void mediaformat_jni_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value)
 {
     JNIEnv *env = NULL;
     jstring key = NULL;
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
 
     av_assert0(format != NULL);
 
@@ -999,10 +1003,11 @@ fail:
     }
 }
 
-void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value)
+static void mediaformat_jni_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value)
 {
     JNIEnv *env = NULL;
     jstring key = NULL;
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
 
     av_assert0(format != NULL);
 
@@ -1024,10 +1029,11 @@ fail:
     }
 }
 
-void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value)
+static void mediaformat_jni_setFloat(FFAMediaFormat* ctx, const char* name, float value)
 {
     JNIEnv *env = NULL;
     jstring key = NULL;
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
 
     av_assert0(format != NULL);
 
@@ -1049,11 +1055,12 @@ fail:
     }
 }
 
-void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value)
+static void mediaformat_jni_setString(FFAMediaFormat* ctx, const char* name, const char* value)
 {
     JNIEnv *env = NULL;
     jstring key = NULL;
     jstring string = NULL;
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
 
     av_assert0(format != NULL);
 
@@ -1084,12 +1091,13 @@ fail:
     }
 }
 
-void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size)
+static void mediaformat_jni_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size)
 {
     JNIEnv *env = NULL;
     jstring key = NULL;
     jobject buffer = NULL;
     void *buffer_data = NULL;
+    FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx;
 
     av_assert0(format != NULL);
 
@@ -1131,7 +1139,7 @@ fail:
     }
 }
 
-static int codec_init_static_fields(FFAMediaCodec *codec)
+static int codec_init_static_fields(FFAMediaCodecJni *codec)
 {
     int ret = 0;
     JNIEnv *env = NULL;
@@ -1193,17 +1201,17 @@ static inline FFAMediaCodec *codec_create(int method, const char *arg)
 {
     int ret = -1;
     JNIEnv *env = NULL;
-    FFAMediaCodec *codec = NULL;
+    FFAMediaCodecJni *codec = NULL;
     jstring jarg = NULL;
     jobject object = NULL;
     jobject buffer_info = NULL;
     jmethodID create_id = NULL;
 
-    codec = av_mallocz(sizeof(FFAMediaCodec));
+    codec = av_mallocz(sizeof(*codec));
     if (!codec) {
         return NULL;
     }
-    codec->class = &amediacodec_class;
+    codec->api = media_codec_jni;
 
     env = ff_jni_get_env(codec);
     if (!env) {
@@ -1286,11 +1294,11 @@ fail:
         av_freep(&codec);
     }
 
-    return codec;
+    return (FFAMediaCodec *)codec;
 }
 
 #define DECLARE_FF_AMEDIACODEC_CREATE_FUNC(name, method) \
-FFAMediaCodec *ff_AMediaCodec_##name(const char *arg)    \
+static FFAMediaCodec *mediacodec_jni_##name(const char *arg)    \
 {                                                        \
     return codec_create(method, arg);                    \
 }                                                        \
@@ -1299,10 +1307,10 @@ DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createCodecByName,   CREATE_CODEC_BY_NAME)
 DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE)
 DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE)
 
-int ff_AMediaCodec_delete(FFAMediaCodec* codec)
+static int mediacodec_jni_delete(FFAMediaCodec* ctx)
 {
     int ret = 0;
-
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     JNIEnv *env = NULL;
 
     if (!codec) {
@@ -1335,11 +1343,12 @@ int ff_AMediaCodec_delete(FFAMediaCodec* codec)
     return ret;
 }
 
-char *ff_AMediaCodec_getName(FFAMediaCodec *codec)
+static char *mediacodec_jni_getName(FFAMediaCodec *ctx)
 {
     char *ret = NULL;
     JNIEnv *env = NULL;
     jobject *name = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, NULL);
 
@@ -1358,10 +1367,12 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags)
+static int mediacodec_jni_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
+    const FFAMediaFormatJni *format = (FFAMediaFormatJni *)format_ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1375,10 +1386,11 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_start(FFAMediaCodec* codec)
+static int mediacodec_jni_start(FFAMediaCodec* ctx)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1392,10 +1404,11 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_stop(FFAMediaCodec* codec)
+static int mediacodec_jni_stop(FFAMediaCodec* ctx)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1409,10 +1422,11 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_flush(FFAMediaCodec* codec)
+static int mediacodec_jni_flush(FFAMediaCodec* ctx)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1426,10 +1440,11 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render)
+static int mediacodec_jni_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1443,10 +1458,11 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs)
+static int mediacodec_jni_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1460,10 +1476,11 @@ fail:
     return ret;
 }
 
-ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs)
+static ssize_t mediacodec_jni_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1477,10 +1494,11 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags)
+static int mediacodec_jni_queueInputBuffer(FFAMediaCodec* ctx, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1494,10 +1512,11 @@ fail:
     return ret;
 }
 
-ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs)
+static ssize_t mediacodec_jni_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs)
 {
     int ret = 0;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -1529,11 +1548,11 @@ ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBu
     return ret;
 }
 
-uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size)
+static uint8_t* mediacodec_jni_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size)
 {
     uint8_t *ret = NULL;
     JNIEnv *env = NULL;
-
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     jobject buffer = NULL;
     jobject input_buffers = NULL;
 
@@ -1577,11 +1596,11 @@ fail:
     return ret;
 }
 
-uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size)
+static uint8_t* mediacodec_jni_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size)
 {
     uint8_t *ret = NULL;
     JNIEnv *env = NULL;
-
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     jobject buffer = NULL;
     jobject output_buffers = NULL;
 
@@ -1625,10 +1644,11 @@ fail:
     return ret;
 }
 
-FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec)
+static FFAMediaFormat* mediacodec_jni_getOutputFormat(FFAMediaCodec* ctx)
 {
     FFAMediaFormat *ret = NULL;
     JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     jobject mediaformat = NULL;
 
@@ -1639,7 +1659,7 @@ FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec)
         goto fail;
     }
 
-    ret = ff_AMediaFormat_newFromObject(mediaformat);
+    ret = mediaformat_jni_newFromObject(mediaformat);
 fail:
     if (mediaformat) {
         (*env)->DeleteLocalRef(env, mediaformat);
@@ -1648,44 +1668,52 @@ fail:
     return ret;
 }
 
-int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx)
+static int mediacodec_jni_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return idx == codec->INFO_TRY_AGAIN_LATER;
 }
 
-int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx)
+static int mediacodec_jni_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return idx == codec->INFO_OUTPUT_BUFFERS_CHANGED;
 }
 
-int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t idx)
+static int mediacodec_jni_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return idx == codec->INFO_OUTPUT_FORMAT_CHANGED;
 }
 
-int ff_AMediaCodec_getBufferFlagCodecConfig(FFAMediaCodec *codec)
+static int mediacodec_jni_getBufferFlagCodecConfig(FFAMediaCodec *ctx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return codec->BUFFER_FLAG_CODEC_CONFIG;
 }
 
-int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec)
+static int mediacodec_jni_getBufferFlagEndOfStream(FFAMediaCodec *ctx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return codec->BUFFER_FLAG_END_OF_STREAM;
 }
 
-int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec)
+static int mediacodec_jni_getBufferFlagKeyFrame(FFAMediaCodec *ctx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return codec->BUFFER_FLAG_KEY_FRAME;
 }
 
-int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec)
+static int mediacodec_jni_getConfigureFlagEncode(FFAMediaCodec *ctx)
 {
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     return codec->CONFIGURE_FLAG_ENCODE;
 }
 
-int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec)
+static int mediacodec_jni_cleanOutputBuffers(FFAMediaCodec *ctx)
 {
     int ret = 0;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
 
     if (!codec->has_get_i_o_buffer) {
         if (codec->output_buffers) {
@@ -1706,6 +1734,86 @@ fail:
     return ret;
 }
 
+static const FFAMediaFormat media_format_jni = {
+    .class = &amediaformat_class,
+
+    .create = mediaformat_jni_new,
+    .delete = mediaformat_jni_delete,
+
+    .toString = mediaformat_jni_toString,
+
+    .getInt32 = mediaformat_jni_getInt32,
+    .getInt64 = mediaformat_jni_getInt64,
+    .getFloat = mediaformat_jni_getFloat,
+    .getBuffer = mediaformat_jni_getBuffer,
+    .getString = mediaformat_jni_getString,
+
+    .setInt32 = mediaformat_jni_setInt32,
+    .setInt64 = mediaformat_jni_setInt64,
+    .setFloat = mediaformat_jni_setFloat,
+    .setString = mediaformat_jni_setString,
+    .setBuffer = mediaformat_jni_setBuffer,
+};
+
+static const FFAMediaCodec media_codec_jni = {
+    .class = &amediacodec_class,
+
+    .getName = mediacodec_jni_getName,
+
+    .createCodecByName = mediacodec_jni_createCodecByName,
+    .createDecoderByType = mediacodec_jni_createDecoderByType,
+    .createEncoderByType = mediacodec_jni_createEncoderByType,
+    .delete = mediacodec_jni_delete,
+
+    .configure = mediacodec_jni_configure,
+    .start = mediacodec_jni_start,
+    .stop = mediacodec_jni_stop,
+    .flush = mediacodec_jni_flush,
+
+    .getInputBuffer = mediacodec_jni_getInputBuffer,
+    .getOutputBuffer = mediacodec_jni_getOutputBuffer,
+
+    .dequeueInputBuffer = mediacodec_jni_dequeueInputBuffer,
+    .queueInputBuffer = mediacodec_jni_queueInputBuffer,
+
+    .dequeueOutputBuffer = mediacodec_jni_dequeueOutputBuffer,
+    .getOutputFormat = mediacodec_jni_getOutputFormat,
+
+    .releaseOutputBuffer = mediacodec_jni_releaseOutputBuffer,
+    .releaseOutputBufferAtTime = mediacodec_jni_releaseOutputBufferAtTime,
+
+    .infoTryAgainLater = mediacodec_jni_infoTryAgainLater,
+    .infoOutputBuffersChanged = mediacodec_jni_infoOutputBuffersChanged,
+    .infoOutputFormatChanged = mediacodec_jni_infoOutputFormatChanged,
+
+    .getBufferFlagCodecConfig = mediacodec_jni_getBufferFlagCodecConfig,
+    .getBufferFlagEndOfStream = mediacodec_jni_getBufferFlagEndOfStream,
+    .getBufferFlagKeyFrame = mediacodec_jni_getBufferFlagKeyFrame,
+
+    .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode,
+    .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers,
+};
+
+FFAMediaFormat *ff_AMediaFormat_new(void)
+{
+    return media_format_jni.create();
+}
+
+FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name)
+{
+    return media_codec_jni.createCodecByName(name);
+}
+
+FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type)
+{
+    return media_codec_jni.createDecoderByType(mime_type);
+}
+
+FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type)
+{
+    return media_codec_jni.createEncoderByType(mime_type);
+}
+
 int ff_Build_SDK_INT(AVCodecContext *avctx)
 {
     int ret = -1;
diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h
index b106ff315a..606fdbede5 100644
--- a/libavcodec/mediacodec_wrapper.h
+++ b/libavcodec/mediacodec_wrapper.h
@@ -58,28 +58,90 @@ int ff_AMediaCodecProfile_getProfileFromAVCodecContext(AVCodecContext *avctx);
 
 char *ff_AMediaCodecList_getCodecNameByType(const char *mime, int profile, int encoder, void *log_ctx);
 
-struct FFAMediaFormat;
 typedef struct FFAMediaFormat FFAMediaFormat;
+struct FFAMediaFormat {
+    const AVClass *class;
+
+    FFAMediaFormat *(*create)(void);
+    int (*delete)(FFAMediaFormat *);
+
+    char* (*toString)(FFAMediaFormat* format);
+
+    int (*getInt32)(FFAMediaFormat* format, const char *name, int32_t *out);
+    int (*getInt64)(FFAMediaFormat* format, const char *name, int64_t *out);
+    int (*getFloat)(FFAMediaFormat* format, const char *name, float *out);
+    int (*getBuffer)(FFAMediaFormat* format, const char *name, void** data, size_t *size);
+    int (*getString)(FFAMediaFormat* format, const char *name, const char **out);
+
+    void (*setInt32)(FFAMediaFormat* format, const char* name, int32_t value);
+    void (*setInt64)(FFAMediaFormat* format, const char* name, int64_t value);
+    void (*setFloat)(FFAMediaFormat* format, const char* name, float value);
+    void (*setString)(FFAMediaFormat* format, const char* name, const char* value);
+    void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size);
+};
 
 FFAMediaFormat *ff_AMediaFormat_new(void);
-int ff_AMediaFormat_delete(FFAMediaFormat* format);
 
-char* ff_AMediaFormat_toString(FFAMediaFormat* format);
+static inline int ff_AMediaFormat_delete(FFAMediaFormat* format)
+{
+    return format->delete(format);
+}
 
-int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out);
-int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out);
-int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out);
-int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size);
-int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out);
+static inline char* ff_AMediaFormat_toString(FFAMediaFormat* format)
+{
+    return format->toString(format);
+}
 
-void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value);
-void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value);
-void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value);
-void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value);
-void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size);
+static inline int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out)
+{
+    return format->getInt32(format, name, out);
+}
+
+static inline int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out)
+{
+    return format->getInt64(format, name, out);
+}
+
+static inline int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out)
+{
+    return format->getFloat(format, name, out);
+}
+
+static inline int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size)
+{
+    return format->getBuffer(format, name, data, size);
+}
+
+static inline int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out)
+{
+    return format->getString(format, name, out);
+}
+
+static inline void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value)
+{
+    format->setInt32(format, name, value);
+}
+
+static inline void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value)
+{
+    format->setInt64(format, name, value);
+}
+
+static inline void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value)
+{
+    format->setFloat(format, name, value);
+}
+
+static inline void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value)
+{
+    format->setString(format, name, value);
+}
+
+static inline void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size)
+{
+    format->setBuffer(format, name, data, size);
+}
 
-struct FFAMediaCodec;
-typedef struct FFAMediaCodec FFAMediaCodec;
 typedef struct FFAMediaCodecCryptoInfo FFAMediaCodecCryptoInfo;
 
 struct FFAMediaCodecBufferInfo {
@@ -90,41 +152,160 @@ struct FFAMediaCodecBufferInfo {
 };
 typedef struct FFAMediaCodecBufferInfo FFAMediaCodecBufferInfo;
 
-char *ff_AMediaCodec_getName(FFAMediaCodec *codec);
+typedef struct FFAMediaCodec FFAMediaCodec;
+struct FFAMediaCodec {
+    const AVClass *class;
+
+    char *(*getName)(FFAMediaCodec *codec);
+
+    FFAMediaCodec* (*createCodecByName)(const char *name);
+    FFAMediaCodec* (*createDecoderByType)(const char *mime_type);
+    FFAMediaCodec* (*createEncoderByType)(const char *mime_type);
+    int (*delete)(FFAMediaCodec* codec);
+
+    int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags);
+    int (*start)(FFAMediaCodec* codec);
+    int (*stop)(FFAMediaCodec* codec);
+    int (*flush)(FFAMediaCodec* codec);
+
+    uint8_t* (*getInputBuffer)(FFAMediaCodec* codec, size_t idx, size_t *out_size);
+    uint8_t* (*getOutputBuffer)(FFAMediaCodec* codec, size_t idx, size_t *out_size);
+
+    ssize_t (*dequeueInputBuffer)(FFAMediaCodec* codec, int64_t timeoutUs);
+    int (*queueInputBuffer)(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags);
+
+    ssize_t (*dequeueOutputBuffer)(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs);
+    FFAMediaFormat* (*getOutputFormat)(FFAMediaCodec* codec);
+
+    int (*releaseOutputBuffer)(FFAMediaCodec* codec, size_t idx, int render);
+    int (*releaseOutputBufferAtTime)(FFAMediaCodec *codec, size_t idx, int64_t timestampNs);
+
+    int (*infoTryAgainLater)(FFAMediaCodec *codec, ssize_t idx);
+    int (*infoOutputBuffersChanged)(FFAMediaCodec *codec, ssize_t idx);
+    int (*infoOutputFormatChanged)(FFAMediaCodec *codec, ssize_t indx);
+
+    int (*getBufferFlagCodecConfig)(FFAMediaCodec *codec);
+    int (*getBufferFlagEndOfStream)(FFAMediaCodec *codec);
+    int (*getBufferFlagKeyFrame)(FFAMediaCodec *codec);
+
+    int (*getConfigureFlagEncode)(FFAMediaCodec *codec);
+
+    int (*cleanOutputBuffers)(FFAMediaCodec *codec);
+};
+
+static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec)
+{
+    return codec->getName(codec);
+}
 
 FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name);
 FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type);
 FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type);
 
-int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags);
-int ff_AMediaCodec_start(FFAMediaCodec* codec);
-int ff_AMediaCodec_stop(FFAMediaCodec* codec);
-int ff_AMediaCodec_flush(FFAMediaCodec* codec);
-int ff_AMediaCodec_delete(FFAMediaCodec* codec);
+static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags)
+{
+    return codec->configure(codec, format, surface, crypto, flags);
+}
+
+static inline int ff_AMediaCodec_start(FFAMediaCodec* codec)
+{
+    return codec->start(codec);
+}
+
+static inline int ff_AMediaCodec_stop(FFAMediaCodec* codec)
+{
+    return codec->stop(codec);
+}
+
+static inline int ff_AMediaCodec_flush(FFAMediaCodec* codec)
+{
+    return codec->flush(codec);
+}
+
+static inline int ff_AMediaCodec_delete(FFAMediaCodec* codec)
+{
+    return codec->delete(codec);
+}
+
+static inline uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size)
+{
+    return codec->getInputBuffer(codec, idx, out_size);
+}
+
+static inline uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size)
+{
+    return codec->getOutputBuffer(codec, idx, out_size);
+}
+
+static inline ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs)
+{
+    return codec->dequeueInputBuffer(codec, timeoutUs);
+}
+
+static inline int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec *codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags)
+{
+    return codec->queueInputBuffer(codec, idx, offset, size, time, flags);
+}
+
+static inline ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs)
+{
+    return codec->dequeueOutputBuffer(codec, info, timeoutUs);
+}
+
+static inline FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec)
+{
+    return codec->getOutputFormat(codec);
+}
+
+static inline int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render)
+{
+    return codec->releaseOutputBuffer(codec, idx, render);
+}
+
+static inline int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs)
+{
+    return codec->releaseOutputBufferAtTime(codec, idx, timestampNs);
+}
 
-uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size);
-uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size);
+static inline int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx)
+{
+    return codec->infoTryAgainLater(codec, idx);
+}
 
-ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs);
-int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags);
+static inline int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx)
+{
+    return codec->infoOutputBuffersChanged(codec, idx);
+}
 
-ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs);
-FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec);
+static inline int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t idx)
+{
+    return codec->infoOutputFormatChanged(codec, idx);
+}
 
-int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render);
-int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs);
+static inline int ff_AMediaCodec_getBufferFlagCodecConfig(FFAMediaCodec *codec)
+{
+    return codec->getBufferFlagCodecConfig(codec);
+}
 
-int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx);
-int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx);
-int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t indx);
+static inline int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec)
+{
+    return codec->getBufferFlagEndOfStream(codec);
+}
 
-int ff_AMediaCodec_getBufferFlagCodecConfig (FFAMediaCodec *codec);
-int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec);
-int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec);
+static inline int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec)
+{
+    return codec->getBufferFlagKeyFrame(codec);
+}
 
-int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec);
+static inline int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec)
+{
+    return codec->getConfigureFlagEncode(codec);
+}
 
-int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec);
+static inline int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec)
+{
+    return codec->cleanOutputBuffers(codec);
+}
 
 int ff_Build_SDK_INT(AVCodecContext *avctx);
 
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 06/12] avcodec/mediacodec: add NDK media codec wrapper
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (4 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 05/12] avcodec/mediacodec_wrapper: separate implementation from interface Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 07/12] avcodec/mediacodecdec: enable NDK mediacodec Zhao Zhili
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 configure                         |   2 +
 libavcodec/mediacodec_wrapper.c   | 596 +++++++++++++++++++++++++++++-
 libavcodec/mediacodec_wrapper.h   |   8 +-
 libavcodec/mediacodecdec.c        |   2 +-
 libavcodec/mediacodecdec_common.c |   2 +-
 5 files changed, 600 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index e54649fa48..9da5a36072 100755
--- a/configure
+++ b/configure
@@ -3181,6 +3181,7 @@ h264_crystalhd_decoder_select="crystalhd h264_mp4toannexb_bsf h264_parser"
 h264_cuvid_decoder_deps="cuvid"
 h264_cuvid_decoder_select="h264_mp4toannexb_bsf"
 h264_mediacodec_decoder_deps="mediacodec"
+h264_mediacodec_decoder_extralibs="-landroid"
 h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser"
 h264_mf_encoder_deps="mediafoundation"
 h264_mmal_decoder_deps="mmal"
@@ -3199,6 +3200,7 @@ hevc_amf_encoder_deps="amf"
 hevc_cuvid_decoder_deps="cuvid"
 hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf"
 hevc_mediacodec_decoder_deps="mediacodec"
+hevc_mediacodec_decoder_extralibs="-landroid"
 hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser"
 hevc_mf_encoder_deps="mediafoundation"
 hevc_nvenc_encoder_deps="nvenc"
diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c
index e0c614680e..b12aced711 100644
--- a/libavcodec/mediacodec_wrapper.c
+++ b/libavcodec/mediacodec_wrapper.c
@@ -20,7 +20,11 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include <dlfcn.h>
 #include <jni.h>
+#include <media/NdkMediaFormat.h>
+#include <media/NdkMediaCodec.h>
+#include <android/native_window_jni.h>
 
 #include "libavutil/avassert.h"
 #include "libavutil/mem.h"
@@ -1794,23 +1798,607 @@ static const FFAMediaCodec media_codec_jni = {
     .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers,
 };
 
-FFAMediaFormat *ff_AMediaFormat_new(void)
+typedef struct FFAMediaFormatNdk {
+    FFAMediaFormat api;
+
+    void *libmedia;
+    AMediaFormat *impl;
+
+    AMediaFormat *(*new)(void);
+    media_status_t (*delete)(AMediaFormat*);
+
+    const char* (*toString)(AMediaFormat*);
+
+    bool (*getInt32)(AMediaFormat*, const char *name, int32_t *out);
+    bool (*getInt64)(AMediaFormat*, const char *name, int64_t *out);
+    bool (*getFloat)(AMediaFormat*, const char *name, float *out);
+    bool (*getSize)(AMediaFormat*, const char *name, size_t *out);
+    bool (*getBuffer)(AMediaFormat*, const char *name, void** data, size_t *size);
+    bool (*getString)(AMediaFormat*, const char *name, const char **out);
+
+    void (*setInt32)(AMediaFormat*, const char* name, int32_t value);
+    void (*setInt64)(AMediaFormat*, const char* name, int64_t value);
+    void (*setFloat)(AMediaFormat*, const char* name, float value);
+    void (*setString)(AMediaFormat*, const char* name, const char* value);
+    void (*setBuffer)(AMediaFormat*, const char* name, const void* data, size_t size);
+} FFAMediaFormatNdk;
+
+typedef struct FFAMediaCodecNdk {
+    FFAMediaCodec api;
+
+    void *libmedia;
+    AMediaCodec *impl;
+    ANativeWindow *window;
+
+    AMediaCodec* (*createCodecByName)(const char *name);
+    AMediaCodec* (*createDecoderByType)(const char *mime_type);
+    AMediaCodec* (*createEncoderByType)(const char *mime_type);
+    media_status_t (*delete)(AMediaCodec*);
+
+    media_status_t (*configure)(AMediaCodec *,
+                                const AMediaFormat *format,
+                                ANativeWindow *surface,
+                                AMediaCrypto *crypto,
+                                uint32_t flags);
+    media_status_t (*start)(AMediaCodec*);
+    media_status_t (*stop)(AMediaCodec*);
+    media_status_t (*flush)(AMediaCodec*);
+
+    uint8_t* (*getInputBuffer)(AMediaCodec*, size_t idx, size_t *out_size);
+    uint8_t* (*getOutputBuffer)(AMediaCodec*, size_t idx, size_t *out_size);
+
+    ssize_t (*dequeueInputBuffer)(AMediaCodec*, int64_t timeoutUs);
+    media_status_t (*queueInputBuffer)(AMediaCodec*, size_t idx,
+                                       long offset, size_t size,
+                                       uint64_t time, uint32_t flags);
+
+    ssize_t (*dequeueOutputBuffer)(AMediaCodec*, AMediaCodecBufferInfo *info, int64_t timeoutUs);
+    AMediaFormat* (*getOutputFormat)(AMediaCodec*);
+
+    media_status_t (*releaseOutputBuffer)(AMediaCodec*, size_t idx, bool render);
+    media_status_t (*releaseOutputBufferAtTime)(AMediaCodec *mData, size_t idx, int64_t timestampNs);
+
+    // Available since API level 28.
+    media_status_t (*getName)(AMediaCodec*, char** out_name);
+    void (*releaseName)(AMediaCodec*, char* name);
+} FFAMediaCodecNdk;
+
+static const FFAMediaFormat media_format_ndk;
+static const FFAMediaCodec media_codec_ndk;
+
+static const AVClass amediaformat_ndk_class = {
+    .class_name = "amediaformat_ndk",
+    .item_name  = av_default_item_name,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+static const AVClass amediacodec_ndk_class = {
+    .class_name = "amediacodec_ndk",
+    .item_name  = av_default_item_name,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+static FFAMediaFormat *mediaformat_ndk_create(AMediaFormat *impl)
+{
+    FFAMediaFormatNdk *format = av_mallocz(sizeof(*format));
+    if (!format)
+        return NULL;
+
+    format->api = media_format_ndk;
+
+    format->libmedia = dlopen("libmediandk.so", RTLD_NOW);
+    if (!format->libmedia)
+        goto error;
+
+#define GET_SYMBOL(sym) \
+    format->sym = dlsym(format->libmedia, "AMediaFormat_" #sym);    \
+    if (!format->sym)                                               \
+        goto error;
+
+    GET_SYMBOL(new)
+    GET_SYMBOL(delete)
+
+    GET_SYMBOL(toString)
+
+    GET_SYMBOL(getInt32)
+    GET_SYMBOL(getInt64)
+    GET_SYMBOL(getFloat)
+    GET_SYMBOL(getSize)
+    GET_SYMBOL(getBuffer)
+    GET_SYMBOL(getString)
+
+    GET_SYMBOL(setInt32)
+    GET_SYMBOL(setInt64)
+    GET_SYMBOL(setFloat)
+    GET_SYMBOL(setString)
+    GET_SYMBOL(setBuffer)
+
+#undef GET_SYMBOL
+
+    if (impl) {
+        format->impl = impl;
+    } else {
+        format->impl = format->new();
+        if (!format->impl)
+            goto error;
+    }
+
+    return (FFAMediaFormat *)format;
+
+error:
+    if (format->libmedia)
+        dlclose(format->libmedia);
+    av_freep(&format);
+    return NULL;
+}
+
+static FFAMediaFormat *mediaformat_ndk_new(void)
+{
+    return mediaformat_ndk_create(NULL);
+}
+
+static int mediaformat_ndk_delete(FFAMediaFormat* ctx)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    int ret = 0;
+    if (!format)
+        return 0;
+
+    av_assert0(format->api.class == &amediaformat_ndk_class);
+
+    if (format->impl && (format->delete(format->impl) != AMEDIA_OK))
+            ret = AVERROR_EXTERNAL;
+    if (format->libmedia)
+        dlclose(format->libmedia);
+    av_free(format);
+
+    return ret;
+}
+
+static char* mediaformat_ndk_toString(FFAMediaFormat* ctx)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    const char *str = format->toString(format->impl);
+    return av_strdup(str);
+}
+
+static int mediaformat_ndk_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    return format->getInt32(format->impl, name, out);
+}
+
+static int mediaformat_ndk_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    return format->getInt64(format->impl, name, out);
+}
+
+static int mediaformat_ndk_getFloat(FFAMediaFormat* ctx, const char *name, float *out)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    return format->getFloat(format->impl, name, out);
+}
+
+static int mediaformat_ndk_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    return format->getBuffer(format->impl, name, data, size);
+}
+
+static int mediaformat_ndk_getString(FFAMediaFormat* ctx, const char *name, const char **out)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    const char *tmp = NULL;
+    int ret = format->getString(format->impl, name, &tmp);
+
+    if (tmp)
+        *out = av_strdup(tmp);
+    return ret;
+}
+
+static void mediaformat_ndk_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    format->setInt32(format->impl, name, value);
+}
+
+static void mediaformat_ndk_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    format->setInt64(format->impl, name, value);
+}
+
+static void mediaformat_ndk_setFloat(FFAMediaFormat* ctx, const char* name, float value)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    format->setFloat(format->impl, name, value);
+}
+
+static void mediaformat_ndk_setString(FFAMediaFormat* ctx, const char* name, const char* value)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    format->setString(format->impl, name, value);
+}
+
+static void mediaformat_ndk_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size)
+{
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx;
+    format->setBuffer(format->impl, name, data, size);
+}
+
+static char *mediacodec_ndk_getName(FFAMediaCodec *ctx)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    char *ret = NULL;
+    char *name = NULL;
+
+    if (!codec->getName || !codec->releaseName) {
+        av_log(ctx, AV_LOG_DEBUG, "getName() unavailable\n");
+        return ret;
+    }
+
+    codec->getName(codec->impl, &name);
+    if (name) {
+        ret = av_strdup(name);
+        codec->releaseName(codec->impl, name);
+    }
+
+    return ret;
+}
+
+static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) {
+    FFAMediaCodecNdk *codec = av_mallocz(sizeof(*codec));
+    const char *lib_name = "libmediandk.so";
+
+    if (!codec)
+        return NULL;
+
+    codec->api = media_codec_ndk;
+    codec->libmedia = dlopen(lib_name, RTLD_NOW);
+    if (!codec->libmedia)
+        goto error;
+
+#define GET_SYMBOL(sym, required)                                   \
+    codec->sym = dlsym(codec->libmedia, "AMediaCodec_" #sym);       \
+    if (!codec->sym) {                                              \
+        av_log(codec, required ? AV_LOG_ERROR : AV_LOG_INFO,        \
+               #sym "() unavailable from %s\n", lib_name);          \
+        if (required)                                               \
+            goto error;                                             \
+    }
+
+    GET_SYMBOL(createCodecByName, 1)
+    GET_SYMBOL(createDecoderByType, 1)
+    GET_SYMBOL(createEncoderByType, 1)
+    GET_SYMBOL(delete, 1)
+
+    GET_SYMBOL(configure, 1)
+    GET_SYMBOL(start, 1)
+    GET_SYMBOL(stop, 1)
+    GET_SYMBOL(flush, 1)
+
+    GET_SYMBOL(getInputBuffer, 1)
+    GET_SYMBOL(getOutputBuffer, 1)
+
+    GET_SYMBOL(dequeueInputBuffer, 1)
+    GET_SYMBOL(queueInputBuffer, 1)
+
+    GET_SYMBOL(dequeueOutputBuffer, 1)
+    GET_SYMBOL(getOutputFormat, 1)
+
+    GET_SYMBOL(releaseOutputBuffer, 1)
+    GET_SYMBOL(releaseOutputBufferAtTime, 1)
+
+    GET_SYMBOL(getName, 0)
+    GET_SYMBOL(releaseName, 0)
+
+#undef GET_SYMBOL
+
+    switch (method) {
+    case CREATE_CODEC_BY_NAME:
+        codec->impl = codec->createCodecByName(arg);
+        break;
+    case CREATE_DECODER_BY_TYPE:
+        codec->impl = codec->createDecoderByType(arg);
+        break;
+    case CREATE_ENCODER_BY_TYPE:
+        codec->impl = codec->createEncoderByType(arg);
+        break;
+    default:
+        av_assert0(0);
+    }
+    if (!codec->impl)
+        goto error;
+
+    return (FFAMediaCodec *)codec;
+
+error:
+    if (codec->libmedia)
+        dlclose(codec->libmedia);
+    av_freep(&codec);
+    return NULL;
+}
+
+#define DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(name, method)       \
+static FFAMediaCodec *mediacodec_ndk_##name(const char *arg)    \
+{                                                               \
+    return ndk_codec_create(method, arg);                       \
+}                                                               \
+
+DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createCodecByName,   CREATE_CODEC_BY_NAME)
+DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE)
+DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE)
+
+static int mediacodec_ndk_delete(FFAMediaCodec* ctx)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    int ret = 0;
+
+    if (!codec)
+        return 0;
+
+    av_assert0(codec->api.class == &amediacodec_ndk_class);
+
+    if (codec->impl && (codec->delete(codec->impl) != AMEDIA_OK))
+        ret = AVERROR_EXTERNAL;
+    if (codec->window)
+        ANativeWindow_release(codec->window);
+    if (codec->libmedia)
+        dlclose(codec->libmedia);
+    av_free(codec);
+
+    return ret;
+}
+
+static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)format_ctx;
+    media_status_t status;
+
+    if (surface) {
+        JNIEnv *env = NULL;
+        JNI_GET_ENV_OR_RETURN(env, ctx, -1);
+        codec->window = ANativeWindow_fromSurface(env, surface);
+    }
+
+    if (format_ctx->class != &amediaformat_ndk_class) {
+        av_log(ctx, AV_LOG_ERROR, "invalid media format\n");
+        return AVERROR(EINVAL);
+    }
+
+    status = codec->configure(codec->impl, format->impl, codec->window, NULL, flags);
+    if (status != AMEDIA_OK) {
+        av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status);
+        return AVERROR_EXTERNAL;
+    }
+
+    return 0;
+}
+
+#define MEDIACODEC_NDK_WRAPPER(method)                                   \
+static int mediacodec_ndk_ ## method(FFAMediaCodec* ctx)                 \
+{                                                                        \
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;                   \
+    media_status_t status = codec->method(codec->impl);                  \
+                                                                         \
+    if (status != AMEDIA_OK) {                                           \
+        av_log(codec, AV_LOG_ERROR, #method " failed, %d\n", status);    \
+        return AVERROR_EXTERNAL;                                         \
+    }                                                                    \
+                                                                         \
+    return 0;                                                            \
+}                                                                        \
+
+MEDIACODEC_NDK_WRAPPER(start)
+MEDIACODEC_NDK_WRAPPER(stop)
+MEDIACODEC_NDK_WRAPPER(flush)
+
+static uint8_t* mediacodec_ndk_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    return codec->getInputBuffer(codec->impl, idx, out_size);
+}
+
+static uint8_t* mediacodec_ndk_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    return codec->getOutputBuffer(codec->impl, idx, out_size);
+}
+
+static ssize_t mediacodec_ndk_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    return codec->dequeueInputBuffer(codec->impl, timeoutUs);
+}
+
+static int mediacodec_ndk_queueInputBuffer(FFAMediaCodec *ctx, size_t idx,
+                                           off_t offset, size_t size,
+                                           uint64_t time, uint32_t flags)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    return codec->queueInputBuffer(codec->impl, idx, offset, size, time, flags);
+}
+
+static ssize_t mediacodec_ndk_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    AMediaCodecBufferInfo buf_info = {0};
+    ssize_t ret;
+
+    ret = codec->dequeueOutputBuffer(codec->impl, &buf_info, timeoutUs);
+    info->offset = buf_info.offset;
+    info->size = buf_info.size;
+    info->presentationTimeUs = buf_info.presentationTimeUs;
+    info->flags = buf_info.flags;
+
+    return ret;
+}
+
+static FFAMediaFormat* mediacodec_ndk_getOutputFormat(FFAMediaCodec* ctx)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    AMediaFormat *format = codec->getOutputFormat(codec->impl);
+
+    if (!format)
+        return NULL;
+    return mediaformat_ndk_create(format);
+}
+
+static int mediacodec_ndk_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    media_status_t status;
+
+    status = codec->releaseOutputBuffer(codec->impl, idx, render);
+    if (status != AMEDIA_OK) {
+        av_log(codec, AV_LOG_ERROR, "release output buffer failed, %d\n", status);
+        return AVERROR_EXTERNAL;
+    }
+
+    return 0;
+}
+
+static int mediacodec_ndk_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    media_status_t status;
+
+    status = codec->releaseOutputBufferAtTime(codec->impl, idx, timestampNs);
+    if (status != AMEDIA_OK) {
+        av_log(codec, AV_LOG_ERROR, "releaseOutputBufferAtTime failed, %d\n", status);
+        return AVERROR_EXTERNAL;
+    }
+
+    return 0;
+}
+
+static int mediacodec_ndk_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx)
+{
+    return idx == AMEDIACODEC_INFO_TRY_AGAIN_LATER;
+}
+
+static int mediacodec_ndk_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx)
+{
+    return idx == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED;
+}
+
+static int mediacodec_ndk_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx)
+{
+    return idx == AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED;
+}
+
+static int mediacodec_ndk_getBufferFlagCodecConfig(FFAMediaCodec *ctx)
+{
+    return AMEDIACODEC_BUFFER_FLAG_CODEC_CONFIG;
+}
+
+static int mediacodec_ndk_getBufferFlagEndOfStream(FFAMediaCodec *ctx)
+{
+    return AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM;
+}
+
+static int mediacodec_ndk_getBufferFlagKeyFrame(FFAMediaCodec *ctx)
+{
+    return 1;
+}
+
+static int mediacodec_ndk_getConfigureFlagEncode(FFAMediaCodec *ctx)
+{
+    return AMEDIACODEC_CONFIGURE_FLAG_ENCODE;
+}
+
+static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx)
+{
+    return 0;
+}
+
+static const FFAMediaFormat media_format_ndk = {
+    .class = &amediaformat_ndk_class,
+
+    .create = mediaformat_ndk_new,
+    .delete = mediaformat_ndk_delete,
+
+    .toString = mediaformat_ndk_toString,
+
+    .getInt32 = mediaformat_ndk_getInt32,
+    .getInt64 = mediaformat_ndk_getInt64,
+    .getFloat = mediaformat_ndk_getFloat,
+    .getBuffer = mediaformat_ndk_getBuffer,
+    .getString = mediaformat_ndk_getString,
+
+    .setInt32 = mediaformat_ndk_setInt32,
+    .setInt64 = mediaformat_ndk_setInt64,
+    .setFloat = mediaformat_ndk_setFloat,
+    .setString = mediaformat_ndk_setString,
+    .setBuffer = mediaformat_ndk_setBuffer,
+};
+
+static const FFAMediaCodec media_codec_ndk = {
+    .class = &amediacodec_ndk_class,
+
+    .getName = mediacodec_ndk_getName,
+
+    .createCodecByName = mediacodec_ndk_createCodecByName,
+    .createDecoderByType = mediacodec_ndk_createDecoderByType,
+    .createEncoderByType = mediacodec_ndk_createEncoderByType,
+    .delete = mediacodec_ndk_delete,
+
+    .configure = mediacodec_ndk_configure,
+    .start = mediacodec_ndk_start,
+    .stop = mediacodec_ndk_stop,
+    .flush = mediacodec_ndk_flush,
+
+    .getInputBuffer = mediacodec_ndk_getInputBuffer,
+    .getOutputBuffer = mediacodec_ndk_getOutputBuffer,
+
+    .dequeueInputBuffer = mediacodec_ndk_dequeueInputBuffer,
+    .queueInputBuffer = mediacodec_ndk_queueInputBuffer,
+
+    .dequeueOutputBuffer = mediacodec_ndk_dequeueOutputBuffer,
+    .getOutputFormat = mediacodec_ndk_getOutputFormat,
+
+    .releaseOutputBuffer = mediacodec_ndk_releaseOutputBuffer,
+    .releaseOutputBufferAtTime = mediacodec_ndk_releaseOutputBufferAtTime,
+
+    .infoTryAgainLater = mediacodec_ndk_infoTryAgainLater,
+    .infoOutputBuffersChanged = mediacodec_ndk_infoOutputBuffersChanged,
+    .infoOutputFormatChanged = mediacodec_ndk_infoOutputFormatChanged,
+
+    .getBufferFlagCodecConfig = mediacodec_ndk_getBufferFlagCodecConfig,
+    .getBufferFlagEndOfStream = mediacodec_ndk_getBufferFlagEndOfStream,
+    .getBufferFlagKeyFrame = mediacodec_ndk_getBufferFlagKeyFrame,
+
+    .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode,
+    .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers,
+};
+
+FFAMediaFormat *ff_AMediaFormat_new(int ndk)
 {
+    if (ndk)
+        return media_format_ndk.create();
     return media_format_jni.create();
 }
 
-FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name)
+FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk)
 {
+    if (ndk)
+        return media_codec_ndk.createCodecByName(name);
     return media_codec_jni.createCodecByName(name);
 }
 
-FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type)
+FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk)
 {
+   if (ndk)
+        return media_codec_ndk.createDecoderByType(mime_type);
     return media_codec_jni.createDecoderByType(mime_type);
 }
 
-FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type)
+FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk)
 {
+    if (ndk)
+        return media_codec_ndk.createEncoderByType(mime_type);
     return media_codec_jni.createEncoderByType(mime_type);
 }
 
diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h
index 606fdbede5..b9b882f243 100644
--- a/libavcodec/mediacodec_wrapper.h
+++ b/libavcodec/mediacodec_wrapper.h
@@ -80,7 +80,7 @@ struct FFAMediaFormat {
     void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size);
 };
 
-FFAMediaFormat *ff_AMediaFormat_new(void);
+FFAMediaFormat *ff_AMediaFormat_new(int ndk);
 
 static inline int ff_AMediaFormat_delete(FFAMediaFormat* format)
 {
@@ -198,9 +198,9 @@ static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec)
     return codec->getName(codec);
 }
 
-FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name);
-FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type);
-FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type);
+FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk);
+FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk);
+FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk);
 
 static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags)
 {
diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c
index 2c66f38541..2e07548b77 100644
--- a/libavcodec/mediacodecdec.c
+++ b/libavcodec/mediacodecdec.c
@@ -310,7 +310,7 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx)
     FFAMediaFormat *format = NULL;
     MediaCodecH264DecContext *s = avctx->priv_data;
 
-    format = ff_AMediaFormat_new();
+    format = ff_AMediaFormat_new(0);
     if (!format) {
         av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n");
         ret = AVERROR_EXTERNAL;
diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c
index 69a462ec48..4a9e50b0df 100644
--- a/libavcodec/mediacodecdec_common.c
+++ b/libavcodec/mediacodecdec_common.c
@@ -613,7 +613,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s,
     }
 
     av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name);
-    s->codec = ff_AMediaCodec_createCodecByName(s->codec_name);
+    s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, 0);
     if (!s->codec) {
         av_log(avctx, AV_LOG_ERROR, "Failed to create media decoder for type %s and name %s\n", mime, s->codec_name);
         ret = AVERROR_EXTERNAL;
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 07/12] avcodec/mediacodecdec: enable NDK mediacodec
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (5 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 06/12] avcodec/mediacodec: add NDK media codec wrapper Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 08/12] avutil/hwcontext_mediacodec: add ANativeWindow support Zhao Zhili
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 Changelog                         |  1 +
 libavcodec/mediacodecdec.c        | 10 +++++++++-
 libavcodec/mediacodecdec_common.c | 23 +++++++++++++++++++----
 libavcodec/mediacodecdec_common.h |  1 +
 libavcodec/version.h              |  4 ++--
 5 files changed, 32 insertions(+), 7 deletions(-)

diff --git a/Changelog b/Changelog
index 166aa8b5e7..eb8eb72127 100644
--- a/Changelog
+++ b/Changelog
@@ -20,6 +20,7 @@ version <next>:
 - ViewQuest VQC decoder
 - backgroundkey filter
 - nvenc AV1 encoding support
+- MediaCodec decoder via NDKMediaCodec
 
 
 version 5.1:
diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c
index 2e07548b77..2c231d1a34 100644
--- a/libavcodec/mediacodecdec.c
+++ b/libavcodec/mediacodecdec.c
@@ -40,6 +40,7 @@
 #include "hevc_parse.h"
 #include "hwconfig.h"
 #include "internal.h"
+#include "jni.h"
 #include "mediacodec_wrapper.h"
 #include "mediacodecdec_common.h"
 
@@ -54,6 +55,7 @@ typedef struct MediaCodecH264DecContext {
     int delay_flush;
     int amlogic_mpeg2_api23_workaround;
 
+    int use_ndk_codec;
 } MediaCodecH264DecContext;
 
 static av_cold int mediacodec_decode_close(AVCodecContext *avctx)
@@ -310,7 +312,10 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx)
     FFAMediaFormat *format = NULL;
     MediaCodecH264DecContext *s = avctx->priv_data;
 
-    format = ff_AMediaFormat_new(0);
+    if (s->use_ndk_codec < 0)
+        s->use_ndk_codec = !av_jni_get_java_vm(avctx);
+
+    format = ff_AMediaFormat_new(s->use_ndk_codec);
     if (!format) {
         av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n");
         ret = AVERROR_EXTERNAL;
@@ -388,6 +393,7 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx)
     }
 
     s->ctx->delay_flush = s->delay_flush;
+    s->ctx->use_ndk_codec = s->use_ndk_codec;
 
     if ((ret = ff_mediacodec_dec_init(avctx, s->ctx, codec_mime, format)) < 0) {
         s->ctx = NULL;
@@ -528,6 +534,8 @@ static const AVCodecHWConfigInternal *const mediacodec_hw_configs[] = {
 static const AVOption ff_mediacodec_vdec_options[] = {
     { "delay_flush", "Delay flush until hw output buffers are returned to the decoder",
                      OFFSET(delay_flush), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, VD },
+    { "ndk_codec", "Use MediaCodec from NDK",
+                   OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VD },
     { NULL }
 };
 
diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c
index 4a9e50b0df..ae82efaa0a 100644
--- a/libavcodec/mediacodecdec_common.c
+++ b/libavcodec/mediacodecdec_common.c
@@ -608,12 +608,27 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s,
 
     s->codec_name = ff_AMediaCodecList_getCodecNameByType(mime, profile, 0, avctx);
     if (!s->codec_name) {
-        ret = AVERROR_EXTERNAL;
-        goto fail;
+        // getCodecNameByType() can fail due to missing JVM, while NDK
+        // mediacodec can be used without JVM.
+        if (!s->use_ndk_codec) {
+            ret = AVERROR_EXTERNAL;
+            goto fail;
+        }
+        av_log(avctx, AV_LOG_INFO, "Failed to getCodecNameByType\n");
+    } else {
+        av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name);
     }
 
-    av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name);
-    s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, 0);
+    if (s->codec_name)
+        s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, s->use_ndk_codec);
+    else {
+        s->codec = ff_AMediaCodec_createDecoderByType(mime, s->use_ndk_codec);
+        if (s->codec) {
+            s->codec_name = ff_AMediaCodec_getName(s->codec);
+            if (!s->codec_name)
+                s->codec_name = av_strdup(mime);
+        }
+    }
     if (!s->codec) {
         av_log(avctx, AV_LOG_ERROR, "Failed to create media decoder for type %s and name %s\n", mime, s->codec_name);
         ret = AVERROR_EXTERNAL;
diff --git a/libavcodec/mediacodecdec_common.h b/libavcodec/mediacodecdec_common.h
index 0b21129fee..0ab29036bd 100644
--- a/libavcodec/mediacodecdec_common.h
+++ b/libavcodec/mediacodecdec_common.h
@@ -70,6 +70,7 @@ typedef struct MediaCodecDecContext {
     bool delay_flush;
     atomic_int serial;
 
+    bool use_ndk_codec;
 } MediaCodecDecContext;
 
 int ff_mediacodec_dec_init(AVCodecContext *avctx,
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 480dbd3b67..fcf589f61f 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,8 +29,8 @@
 
 #include "version_major.h"
 
-#define LIBAVCODEC_VERSION_MINOR  52
-#define LIBAVCODEC_VERSION_MICRO 102
+#define LIBAVCODEC_VERSION_MINOR  53
+#define LIBAVCODEC_VERSION_MICRO 100
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
                                                LIBAVCODEC_VERSION_MINOR, \
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 08/12] avutil/hwcontext_mediacodec: add ANativeWindow support
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (6 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 07/12] avcodec/mediacodecdec: enable NDK mediacodec Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 09/12] avcodec/mediacodec: " Zhao Zhili
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavutil/hwcontext_mediacodec.c | 56 +++++++++++++++++++++++++++++++-
 libavutil/hwcontext_mediacodec.h | 11 +++++++
 libavutil/version.h              |  2 +-
 3 files changed, 67 insertions(+), 2 deletions(-)

diff --git a/libavutil/hwcontext_mediacodec.c b/libavutil/hwcontext_mediacodec.c
index b0d8993e15..bb1779d34d 100644
--- a/libavutil/hwcontext_mediacodec.c
+++ b/libavutil/hwcontext_mediacodec.c
@@ -18,12 +18,24 @@
 
 #include "config.h"
 
+#include <android/native_window.h>
+#include <dlfcn.h>
+#include <media/NdkMediaCodec.h>
+
 #include "buffer.h"
 #include "common.h"
 #include "hwcontext.h"
 #include "hwcontext_internal.h"
 #include "hwcontext_mediacodec.h"
 
+typedef struct MediaCodecDeviceContext {
+    AVMediaCodecDeviceContext ctx;
+
+    void *libmedia;
+    media_status_t (*create_surface)(ANativeWindow **surface);
+} MediaCodecDeviceContext;
+
+
 static int mc_device_create(AVHWDeviceContext *ctx, const char *device,
                             AVDictionary *opts, int flags)
 {
@@ -35,13 +47,55 @@ static int mc_device_create(AVHWDeviceContext *ctx, const char *device,
     return 0;
 }
 
+static int mc_device_init(AVHWDeviceContext *ctx)
+{
+    MediaCodecDeviceContext *s = ctx->hwctx;
+    AVMediaCodecDeviceContext *dev = (AVMediaCodecDeviceContext *)s;
+    ANativeWindow *native_window = NULL;
+
+    if (dev->surface)
+        return 0;
+
+    if (dev->native_window)
+        return 0;
+
+    s->libmedia = dlopen("libmediandk.so", RTLD_NOW);
+    if (!s->libmedia)
+        return AVERROR_UNKNOWN;
+
+    s->create_surface = dlsym(s->libmedia, "AMediaCodec_createPersistentInputSurface");
+    if (!s->create_surface)
+        return AVERROR_UNKNOWN;
+
+    s->create_surface(&native_window);
+    dev->native_window = native_window;
+    return 0;
+}
+
+static void mc_device_uninit(AVHWDeviceContext *ctx)
+{
+    MediaCodecDeviceContext *s = ctx->hwctx;
+    AVMediaCodecDeviceContext *dev = ctx->hwctx;
+    if (!s->libmedia)
+        return;
+
+    if (dev->native_window) {
+        ANativeWindow_release(dev->native_window);
+        dev->native_window = NULL;
+    }
+    dlclose(s->libmedia);
+    s->libmedia = NULL;
+}
+
 const HWContextType ff_hwcontext_type_mediacodec = {
     .type                 = AV_HWDEVICE_TYPE_MEDIACODEC,
     .name                 = "mediacodec",
 
-    .device_hwctx_size    = sizeof(AVMediaCodecDeviceContext),
+    .device_hwctx_size    = sizeof(MediaCodecDeviceContext),
 
     .device_create        = mc_device_create,
+    .device_init          = mc_device_init,
+    .device_uninit        = mc_device_uninit,
 
     .pix_fmts = (const enum AVPixelFormat[]){
         AV_PIX_FMT_MEDIACODEC,
diff --git a/libavutil/hwcontext_mediacodec.h b/libavutil/hwcontext_mediacodec.h
index 101a9806d5..920e17764f 100644
--- a/libavutil/hwcontext_mediacodec.h
+++ b/libavutil/hwcontext_mediacodec.h
@@ -31,6 +31,17 @@ typedef struct AVMediaCodecDeviceContext {
      * This is the default surface used by decoders on this device.
      */
     void *surface;
+
+    /**
+     * Pointer to ANativeWindow.
+     *
+     * It both surface and native_window is NULL, try to create it
+     * automatically if OS support.
+     *
+     * It can be used as output surface for decoder and input surface for
+     * encoder.
+     */
+    void *native_window;
 } AVMediaCodecDeviceContext;
 
 #endif /* AVUTIL_HWCONTEXT_MEDIACODEC_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 9b9eea2946..3b616ea489 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,7 +79,7 @@
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  57
-#define LIBAVUTIL_VERSION_MINOR  42
+#define LIBAVUTIL_VERSION_MINOR  43
 #define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 09/12] avcodec/mediacodec: add ANativeWindow support
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (7 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 08/12] avutil/hwcontext_mediacodec: add ANativeWindow support Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 10/12] avcodec: add MediaCodec encoder Zhao Zhili
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

ANativeWindow can be used without JVM.
---
 libavcodec/mediacodec_surface.c   | 46 ++++++++++++++++++++++++-------
 libavcodec/mediacodec_surface.h   |  8 ++++--
 libavcodec/mediacodec_wrapper.c   | 30 +++++++++++++++-----
 libavcodec/mediacodec_wrapper.h   |  8 ++++--
 libavcodec/mediacodecdec_common.c |  4 +--
 5 files changed, 72 insertions(+), 24 deletions(-)

diff --git a/libavcodec/mediacodec_surface.c b/libavcodec/mediacodec_surface.c
index 09a42295d2..ef41cdafa7 100644
--- a/libavcodec/mediacodec_surface.c
+++ b/libavcodec/mediacodec_surface.c
@@ -20,33 +20,59 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include <android/native_window.h>
 #include <jni.h>
 
+#include "libavutil/mem.h"
 #include "ffjni.h"
 #include "mediacodec_surface.h"
 
-FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *log_ctx)
+FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *native_window, void *log_ctx)
 {
-    JNIEnv *env = NULL;
+    FFANativeWindow *ret;
 
-    env = ff_jni_get_env(log_ctx);
-    if (!env) {
+    ret = av_mallocz(sizeof(*ret));
+    if (!ret)
         return NULL;
+
+    if (surface) {
+        JNIEnv *env = NULL;
+
+        env = ff_jni_get_env(log_ctx);
+        if (env)
+            ret->surface = (*env)->NewGlobalRef(env, surface);
+    }
+
+    if (native_window) {
+        ANativeWindow_acquire(native_window);
+        ret->native_window = native_window;
     }
 
-    return (*env)->NewGlobalRef(env, surface);
+    if (!ret->surface && !ret->native_window) {
+        av_log(log_ctx, AV_LOG_ERROR, "Both surface and native_window are NULL\n");
+        av_freep(&ret);
+    }
+
+    return ret;
 }
 
 int ff_mediacodec_surface_unref(FFANativeWindow *window, void *log_ctx)
 {
-    JNIEnv *env = NULL;
+    if (!window)
+        return 0;
 
-    env = ff_jni_get_env(log_ctx);
-    if (!env) {
-        return AVERROR_EXTERNAL;
+    if (window->surface) {
+        JNIEnv *env = NULL;
+
+        env = ff_jni_get_env(log_ctx);
+        if (env)
+            (*env)->DeleteGlobalRef(env, window->surface);
     }
 
-    (*env)->DeleteGlobalRef(env, window);
+    if (window->native_window)
+        ANativeWindow_release(window->native_window);
+
+    av_free(window);
 
     return 0;
 }
diff --git a/libavcodec/mediacodec_surface.h b/libavcodec/mediacodec_surface.h
index 933dc2bf51..e2ac1c9057 100644
--- a/libavcodec/mediacodec_surface.h
+++ b/libavcodec/mediacodec_surface.h
@@ -25,10 +25,12 @@
 
 #include "libavcodec/avcodec.h"
 
-struct FFANativeWindow;
-typedef struct FFANativeWindow FFANativeWindow;
+typedef struct FFANativeWindow {
+    void *surface;
+    void *native_window;
+} FFANativeWindow;
 
-FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *log_ctx);
+FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *native_window, void *log_ctx);
 int ff_mediacodec_surface_unref(FFANativeWindow *window, void *log_ctx);
 
 #endif /* AVCODEC_MEDIACODEC_SURFACE_H */
diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c
index b12aced711..284d615980 100644
--- a/libavcodec/mediacodec_wrapper.c
+++ b/libavcodec/mediacodec_wrapper.c
@@ -1371,12 +1371,17 @@ fail:
     return ret;
 }
 
-static int mediacodec_jni_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags)
+static int mediacodec_jni_configure(FFAMediaCodec *ctx,
+                                    const FFAMediaFormat* format_ctx,
+                                    FFANativeWindow* window,
+                                    void *crypto,
+                                    uint32_t flags)
 {
     int ret = 0;
     JNIEnv *env = NULL;
     FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
     const FFAMediaFormatJni *format = (FFAMediaFormatJni *)format_ctx;
+    jobject *surface = window ? window->surface : NULL;
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
@@ -2151,16 +2156,27 @@ static int mediacodec_ndk_delete(FFAMediaCodec* ctx)
     return ret;
 }
 
-static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags)
+static int mediacodec_ndk_configure(FFAMediaCodec* ctx,
+                                    const FFAMediaFormat* format_ctx,
+                                    FFANativeWindow* window,
+                                    void *crypto,
+                                    uint32_t flags)
 {
     FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
     FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)format_ctx;
     media_status_t status;
+    ANativeWindow *native_window = NULL;
 
-    if (surface) {
-        JNIEnv *env = NULL;
-        JNI_GET_ENV_OR_RETURN(env, ctx, -1);
-        codec->window = ANativeWindow_fromSurface(env, surface);
+    if (window) {
+        if (window->surface) {
+            JNIEnv *env = NULL;
+            JNI_GET_ENV_OR_RETURN(env, ctx, -1);
+            native_window = ANativeWindow_fromSurface(env, window->surface);
+            // Save for release
+            codec->window = native_window;
+        } else if (window->native_window) {
+            native_window = window->native_window;
+        }
     }
 
     if (format_ctx->class != &amediaformat_ndk_class) {
@@ -2168,7 +2184,7 @@ static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* fo
         return AVERROR(EINVAL);
     }
 
-    status = codec->configure(codec->impl, format->impl, codec->window, NULL, flags);
+    status = codec->configure(codec->impl, format->impl, native_window, NULL, flags);
     if (status != AMEDIA_OK) {
         av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status);
         return AVERROR_EXTERNAL;
diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h
index b9b882f243..7cf3f4aecd 100644
--- a/libavcodec/mediacodec_wrapper.h
+++ b/libavcodec/mediacodec_wrapper.h
@@ -27,6 +27,7 @@
 #include <sys/types.h>
 
 #include "avcodec.h"
+#include "mediacodec_surface.h"
 
 /**
  * The following API around MediaCodec and MediaFormat is based on the
@@ -163,7 +164,7 @@ struct FFAMediaCodec {
     FFAMediaCodec* (*createEncoderByType)(const char *mime_type);
     int (*delete)(FFAMediaCodec* codec);
 
-    int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags);
+    int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, FFANativeWindow* surface, void *crypto, uint32_t flags);
     int (*start)(FFAMediaCodec* codec);
     int (*stop)(FFAMediaCodec* codec);
     int (*flush)(FFAMediaCodec* codec);
@@ -202,7 +203,10 @@ FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk);
 FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk);
 FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk);
 
-static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags)
+static inline int ff_AMediaCodec_configure(FFAMediaCodec *codec,
+                                           const FFAMediaFormat *format,
+                                           FFANativeWindow *surface,
+                                           void *crypto, uint32_t flags)
 {
     return codec->configure(codec, format, surface, crypto, flags);
 }
diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c
index ae82efaa0a..80089439ea 100644
--- a/libavcodec/mediacodecdec_common.c
+++ b/libavcodec/mediacodecdec_common.c
@@ -589,14 +589,14 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s,
             if (device_ctx->type == AV_HWDEVICE_TYPE_MEDIACODEC) {
                 if (device_ctx->hwctx) {
                     AVMediaCodecDeviceContext *mediacodec_ctx = (AVMediaCodecDeviceContext *)device_ctx->hwctx;
-                    s->surface = ff_mediacodec_surface_ref(mediacodec_ctx->surface, avctx);
+                    s->surface = ff_mediacodec_surface_ref(mediacodec_ctx->surface, mediacodec_ctx->native_window, avctx);
                     av_log(avctx, AV_LOG_INFO, "Using surface %p\n", s->surface);
                 }
             }
         }
 
         if (!s->surface && user_ctx && user_ctx->surface) {
-            s->surface = ff_mediacodec_surface_ref(user_ctx->surface, avctx);
+            s->surface = ff_mediacodec_surface_ref(user_ctx->surface, NULL, avctx);
             av_log(avctx, AV_LOG_INFO, "Using surface %p\n", s->surface);
         }
     }
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 10/12] avcodec: add MediaCodec encoder
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (8 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 09/12] avcodec/mediacodec: " Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 11/12] avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 12/12] fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec Zhao Zhili
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 Changelog                       |   1 +
 configure                       |   4 +
 libavcodec/Makefile             |   2 +
 libavcodec/allcodecs.c          |   2 +
 libavcodec/mediacodec_wrapper.c | 101 ++++++-
 libavcodec/mediacodec_wrapper.h |   8 +
 libavcodec/mediacodecenc.c      | 504 ++++++++++++++++++++++++++++++++
 libavcodec/version.h            |   2 +-
 8 files changed, 618 insertions(+), 6 deletions(-)
 create mode 100644 libavcodec/mediacodecenc.c

diff --git a/Changelog b/Changelog
index eb8eb72127..9310706b72 100644
--- a/Changelog
+++ b/Changelog
@@ -21,6 +21,7 @@ version <next>:
 - backgroundkey filter
 - nvenc AV1 encoding support
 - MediaCodec decoder via NDKMediaCodec
+- MediaCodec encoder
 
 
 version 5.1:
diff --git a/configure b/configure
index 9da5a36072..03004826fa 100755
--- a/configure
+++ b/configure
@@ -3183,6 +3183,8 @@ h264_cuvid_decoder_select="h264_mp4toannexb_bsf"
 h264_mediacodec_decoder_deps="mediacodec"
 h264_mediacodec_decoder_extralibs="-landroid"
 h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser"
+h264_mediacodec_encoder_deps="mediacodec"
+h264_mediacodec_encoder_extralibs="-landroid"
 h264_mf_encoder_deps="mediafoundation"
 h264_mmal_decoder_deps="mmal"
 h264_nvenc_encoder_deps="nvenc"
@@ -3202,6 +3204,8 @@ hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf"
 hevc_mediacodec_decoder_deps="mediacodec"
 hevc_mediacodec_decoder_extralibs="-landroid"
 hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser"
+hevc_mediacodec_encoder_deps="mediacodec"
+hevc_mediacodec_encoder_extralibs="-landroid"
 hevc_mf_encoder_deps="mediafoundation"
 hevc_nvenc_encoder_deps="nvenc"
 hevc_nvenc_encoder_select="atsc_a53"
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 94dc75a1b2..0bc7943d16 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -396,6 +396,7 @@ OBJS-$(CONFIG_H264_DECODER)            += h264dec.o h264_cabac.o h264_cavlc.o \
 OBJS-$(CONFIG_H264_AMF_ENCODER)        += amfenc_h264.o
 OBJS-$(CONFIG_H264_CUVID_DECODER)      += cuviddec.o
 OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o
+OBJS-$(CONFIG_H264_MEDIACODEC_ENCODER) += mediacodecenc.o
 OBJS-$(CONFIG_H264_MF_ENCODER)         += mfenc.o mf_utils.o
 OBJS-$(CONFIG_H264_MMAL_DECODER)       += mmaldec.o
 OBJS-$(CONFIG_H264_NVENC_ENCODER)      += nvenc_h264.o nvenc.o
@@ -420,6 +421,7 @@ OBJS-$(CONFIG_HEVC_DECODER)            += hevcdec.o hevc_mvs.o \
 OBJS-$(CONFIG_HEVC_AMF_ENCODER)        += amfenc_hevc.o
 OBJS-$(CONFIG_HEVC_CUVID_DECODER)      += cuviddec.o
 OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o
+OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o
 OBJS-$(CONFIG_HEVC_MF_ENCODER)         += mfenc.o mf_utils.o
 OBJS-$(CONFIG_HEVC_NVENC_ENCODER)      += nvenc_hevc.o nvenc.o
 OBJS-$(CONFIG_HEVC_QSV_DECODER)        += qsvdec.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index f5ec3bc6e1..d5a6c427e1 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -154,6 +154,7 @@ extern const FFCodec ff_h264_decoder;
 extern const FFCodec ff_h264_crystalhd_decoder;
 extern const FFCodec ff_h264_v4l2m2m_decoder;
 extern const FFCodec ff_h264_mediacodec_decoder;
+extern const FFCodec ff_h264_mediacodec_encoder;
 extern const FFCodec ff_h264_mmal_decoder;
 extern const FFCodec ff_h264_qsv_decoder;
 extern const FFCodec ff_h264_rkmpp_decoder;
@@ -844,6 +845,7 @@ extern const FFCodec ff_h264_videotoolbox_encoder;
 extern const FFCodec ff_hevc_amf_encoder;
 extern const FFCodec ff_hevc_cuvid_decoder;
 extern const FFCodec ff_hevc_mediacodec_decoder;
+extern const FFCodec ff_hevc_mediacodec_encoder;
 extern const FFCodec ff_hevc_mf_encoder;
 extern const FFCodec ff_hevc_nvenc_encoder;
 extern const FFCodec ff_hevc_qsv_encoder;
diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c
index 284d615980..555058e907 100644
--- a/libavcodec/mediacodec_wrapper.c
+++ b/libavcodec/mediacodec_wrapper.c
@@ -212,6 +212,9 @@ struct JNIAMediaCodecFields {
     jmethodID release_output_buffer_id;
     jmethodID release_output_buffer_at_time_id;
 
+    jmethodID set_input_surface_id;
+    jmethodID signal_end_of_input_stream_id;
+
     jclass mediainfo_class;
 
     jmethodID init_id;
@@ -261,6 +264,9 @@ static const struct FFJniField jni_amediacodec_mapping[] = {
         { "android/media/MediaCodec", "releaseOutputBuffer", "(IZ)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, release_output_buffer_id), 1 },
         { "android/media/MediaCodec", "releaseOutputBuffer", "(IJ)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, release_output_buffer_at_time_id), 0 },
 
+        { "android/media/MediaCodec", "setInputSurface", "(Landroid/view/Surface;)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, set_input_surface_id), 0 },
+        { "android/media/MediaCodec", "signalEndOfInputStream", "()V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, signal_end_of_input_stream_id), 0 },
+
     { "android/media/MediaCodec$BufferInfo", NULL, NULL, FF_JNI_CLASS, offsetof(struct JNIAMediaCodecFields, mediainfo_class), 1 },
 
         { "android/media/MediaCodec.BufferInfo", "<init>", "()V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, init_id), 1 },
@@ -1385,7 +1391,26 @@ static int mediacodec_jni_configure(FFAMediaCodec *ctx,
 
     JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
 
-    (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, surface, NULL, flags);
+    if (flags & codec->CONFIGURE_FLAG_ENCODE) {
+        if (surface && !codec->jfields.set_input_surface_id) {
+            av_log(ctx, AV_LOG_ERROR, "System doesn't support setInputSurface\n");
+            return AVERROR_EXTERNAL;
+        }
+
+        (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, NULL, NULL, flags);
+        if (ff_jni_exception_check(env, 1, codec) < 0)
+            return AVERROR_EXTERNAL;
+
+        if (!surface)
+            return 0;
+
+        (*env)->CallVoidMethod(env, codec->object, codec->jfields.set_input_surface_id, surface);
+        if (ff_jni_exception_check(env, 1, codec) < 0)
+            return AVERROR_EXTERNAL;
+        return 0;
+    } else {
+        (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, surface, NULL, flags);
+    }
     if (ff_jni_exception_check(env, 1, codec) < 0) {
         ret = AVERROR_EXTERNAL;
         goto fail;
@@ -1743,6 +1768,21 @@ fail:
     return ret;
 }
 
+static int mediacodec_jni_signalEndOfInputStream(FFAMediaCodec *ctx)
+{
+    JNIEnv *env = NULL;
+    FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx;
+
+    JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL);
+
+    (*env)->CallVoidMethod(env, codec->object, codec->jfields.signal_end_of_input_stream_id);
+    if (ff_jni_exception_check(env, 1, codec) < 0) {
+        return AVERROR_EXTERNAL;
+    }
+
+    return 0;
+}
+
 static const FFAMediaFormat media_format_jni = {
     .class = &amediaformat_class,
 
@@ -1801,6 +1841,7 @@ static const FFAMediaCodec media_codec_jni = {
 
     .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode,
     .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers,
+    .signalEndOfInputStream = mediacodec_jni_signalEndOfInputStream,
 };
 
 typedef struct FFAMediaFormatNdk {
@@ -1866,6 +1907,10 @@ typedef struct FFAMediaCodecNdk {
     // Available since API level 28.
     media_status_t (*getName)(AMediaCodec*, char** out_name);
     void (*releaseName)(AMediaCodec*, char* name);
+
+    // Available since API level 26.
+    media_status_t (*setInputSurface)(AMediaCodec*, ANativeWindow *);
+    media_status_t (*signalEndOfInputStream)(AMediaCodec *);
 } FFAMediaCodecNdk;
 
 static const FFAMediaFormat media_format_ndk;
@@ -2098,6 +2143,9 @@ static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) {
     GET_SYMBOL(getName, 0)
     GET_SYMBOL(releaseName, 0)
 
+    GET_SYMBOL(setInputSurface, 0)
+    GET_SYMBOL(signalEndOfInputStream, 0)
+
 #undef GET_SYMBOL
 
     switch (method) {
@@ -2184,10 +2232,32 @@ static int mediacodec_ndk_configure(FFAMediaCodec* ctx,
         return AVERROR(EINVAL);
     }
 
-    status = codec->configure(codec->impl, format->impl, native_window, NULL, flags);
-    if (status != AMEDIA_OK) {
-        av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status);
-        return AVERROR_EXTERNAL;
+    if (flags & AMEDIACODEC_CONFIGURE_FLAG_ENCODE) {
+        if (native_window && !codec->setInputSurface) {
+            av_log(ctx, AV_LOG_ERROR, "System doesn't support setInputSurface\n");
+            return AVERROR_EXTERNAL;
+        }
+
+        status = codec->configure(codec->impl, format->impl, NULL, NULL, flags);
+        if (status != AMEDIA_OK) {
+            av_log(codec, AV_LOG_ERROR, "Encoder configure failed, %d\n", status);
+            return AVERROR_EXTERNAL;
+        }
+
+        if (!native_window)
+            return 0;
+
+        status = codec->setInputSurface(codec->impl, native_window);
+        if (status != AMEDIA_OK) {
+            av_log(codec, AV_LOG_ERROR, "Encoder set input surface failed, %d\n", status);
+            return AVERROR_EXTERNAL;
+        }
+    } else {
+        status = codec->configure(codec->impl, format->impl, native_window, NULL, flags);
+        if (status != AMEDIA_OK) {
+            av_log(codec, AV_LOG_ERROR, "Decoder configure failed, %d\n", status);
+            return AVERROR_EXTERNAL;
+        }
     }
 
     return 0;
@@ -2330,6 +2400,26 @@ static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx)
     return 0;
 }
 
+static int mediacodec_ndk_signalEndOfInputStream(FFAMediaCodec *ctx)
+{
+    FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx;
+    media_status_t status;
+
+    if (!codec->signalEndOfInputStream) {
+        av_log(codec, AV_LOG_ERROR, "signalEndOfInputStream unavailable\n");
+        return AVERROR_EXTERNAL;
+    }
+
+    status = codec->signalEndOfInputStream(codec->impl);
+    if (status != AMEDIA_OK) {
+        av_log(codec, AV_LOG_ERROR, "signalEndOfInputStream failed, %d\n", status);
+        return AVERROR_EXTERNAL;
+    }
+    av_log(codec, AV_LOG_DEBUG, "signalEndOfInputStream success\n");
+
+    return 0;
+}
+
 static const FFAMediaFormat media_format_ndk = {
     .class = &amediaformat_ndk_class,
 
@@ -2388,6 +2478,7 @@ static const FFAMediaCodec media_codec_ndk = {
 
     .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode,
     .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers,
+    .signalEndOfInputStream = mediacodec_ndk_signalEndOfInputStream,
 };
 
 FFAMediaFormat *ff_AMediaFormat_new(int ndk)
diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h
index 7cf3f4aecd..f15ad66d83 100644
--- a/libavcodec/mediacodec_wrapper.h
+++ b/libavcodec/mediacodec_wrapper.h
@@ -192,6 +192,9 @@ struct FFAMediaCodec {
     int (*getConfigureFlagEncode)(FFAMediaCodec *codec);
 
     int (*cleanOutputBuffers)(FFAMediaCodec *codec);
+
+    // For encoder with FFANativeWindow as input.
+    int (*signalEndOfInputStream)(FFAMediaCodec *);
 };
 
 static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec)
@@ -311,6 +314,11 @@ static inline int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec)
     return codec->cleanOutputBuffers(codec);
 }
 
+static inline int ff_AMediaCodec_signalEndOfInputStream(FFAMediaCodec *codec)
+{
+    return codec->signalEndOfInputStream(codec);
+}
+
 int ff_Build_SDK_INT(AVCodecContext *avctx);
 
 #endif /* AVCODEC_MEDIACODEC_WRAPPER_H */
diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c
new file mode 100644
index 0000000000..7cdde59945
--- /dev/null
+++ b/libavcodec/mediacodecenc.c
@@ -0,0 +1,504 @@
+/*
+ * Android MediaCodec encoders
+ *
+ * Copyright (c) 2022 Zhao Zhili <zhilizhao@tencent.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "config_components.h"
+
+#include "libavutil/avassert.h"
+#include "libavutil/hwcontext_mediacodec.h"
+#include "libavutil/imgutils.h"
+#include "libavutil/opt.h"
+
+#include "avcodec.h"
+#include "codec_internal.h"
+#include "encode.h"
+#include "hwconfig.h"
+#include "jni.h"
+#include "mediacodec.h"
+#include "mediacodec_wrapper.h"
+#include "mediacodecdec_common.h"
+
+#define INPUT_DEQUEUE_TIMEOUT_US 8000
+#define OUTPUT_DEQUEUE_TIMEOUT_US 8000
+
+typedef struct MediaCodecEncContext {
+    AVClass *avclass;
+    FFAMediaCodec *codec;
+    int use_ndk_codec;
+    FFANativeWindow *window;
+
+    int fps;
+    int width;
+    int height;
+
+    uint8_t *extradata;
+    int extradata_size;
+
+    // Since MediaCodec doesn't output DTS, use a timestamp queue to save pts
+    // of AVFrame and generate DTS for AVPacket.
+    //
+    // This doesn't work when use Surface as input, in that case frames can be
+    // sent to encoder without our notice. One exception is frames come from
+    // our MediaCodec decoder wrapper, since we can control it's render by
+    // av_mediacodec_release_buffer.
+    int64_t timestamps[32];
+    int ts_head;
+    int ts_tail;
+
+    int eof_sent;
+
+    AVFrame *frame;
+} MediaCodecEncContext;
+
+enum {
+    COLOR_FormatYUV420Planar                              = 0x13,
+    COLOR_FormatYUV420SemiPlanar                          = 0x15,
+    COLOR_FormatSurface                                   = 0x7F000789,
+};
+
+static const struct {
+    int color_format;
+    enum AVPixelFormat pix_fmt;
+} color_formats[] = {
+    { COLOR_FormatYUV420Planar,         AV_PIX_FMT_YUV420P },
+    { COLOR_FormatYUV420SemiPlanar,     AV_PIX_FMT_NV12    },
+    { COLOR_FormatSurface,              AV_PIX_FMT_MEDIACODEC },
+};
+
+static const enum AVPixelFormat avc_pix_fmts[] = {
+    AV_PIX_FMT_MEDIACODEC,
+    AV_PIX_FMT_YUV420P,
+    AV_PIX_FMT_NV12,
+    AV_PIX_FMT_NONE
+};
+
+static void mediacodec_output_format(AVCodecContext *avctx)
+{
+    MediaCodecEncContext *s = avctx->priv_data;
+    char *name = ff_AMediaCodec_getName(s->codec);
+    FFAMediaFormat *out_format = ff_AMediaCodec_getOutputFormat(s->codec);
+    char *str = ff_AMediaFormat_toString(out_format);
+
+    av_log(avctx, AV_LOG_DEBUG, "MediaCodec encoder %s output format %s\n",
+           name ? name : "unknown", str);
+    av_free(name);
+    av_free(str);
+    ff_AMediaFormat_delete(out_format);
+}
+
+static av_cold int mediacodec_init(AVCodecContext *avctx)
+{
+    const char *codec_mime = NULL;
+    MediaCodecEncContext *s = avctx->priv_data;
+    FFAMediaFormat *format = NULL;
+    int ret;
+    int gop;
+
+    if (s->use_ndk_codec < 0)
+        s->use_ndk_codec = !av_jni_get_java_vm(avctx);
+
+    switch (avctx->codec_id) {
+    case AV_CODEC_ID_H264:
+        codec_mime = "video/avc";
+        break;
+    case AV_CODEC_ID_HEVC:
+        codec_mime = "video/hevc";
+        break;
+    default:
+        av_assert0(0);
+    }
+
+    s->codec = ff_AMediaCodec_createEncoderByType(codec_mime, s->use_ndk_codec);
+    if (!s->codec) {
+        av_log(avctx, AV_LOG_ERROR, "Failed to create encoder for type %s\n",
+               codec_mime);
+        return AVERROR_EXTERNAL;
+    }
+
+    format = ff_AMediaFormat_new(s->use_ndk_codec);
+    if (!format) {
+        av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n");
+        return AVERROR_EXTERNAL;
+    }
+
+    ff_AMediaFormat_setString(format, "mime", codec_mime);
+    s->width = FFALIGN(avctx->width, 16);
+    s->height = avctx->height;
+    ff_AMediaFormat_setInt32(format, "width", s->width);
+    ff_AMediaFormat_setInt32(format, "height", s->height);
+
+    if (avctx->pix_fmt == AV_PIX_FMT_MEDIACODEC) {
+        AVMediaCodecContext *user_ctx = avctx->hwaccel_context;
+        if (avctx->hw_device_ctx) {
+            AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)(avctx->hw_device_ctx->data);
+            AVMediaCodecDeviceContext *dev_ctx;
+
+            if (device_ctx->type != AV_HWDEVICE_TYPE_MEDIACODEC || !device_ctx->hwctx) {
+                ret = AVERROR(EINVAL);
+                goto bailout;
+            }
+            dev_ctx = device_ctx->hwctx;
+            s->window = ff_mediacodec_surface_ref(dev_ctx->surface, dev_ctx->native_window, avctx);
+        }
+
+        if (!s->window && user_ctx && user_ctx->surface)
+            s->window = ff_mediacodec_surface_ref(user_ctx->surface, NULL, avctx);
+
+        if (!s->window) {
+            ret = AVERROR(EINVAL);
+            av_log(avctx, AV_LOG_ERROR, "Missing hw_device_ctx or hwaccel_context for AV_PIX_FMT_MEDIACODEC\n");
+            goto bailout;
+        }
+    }
+
+    for (int i = 0; i < FF_ARRAY_ELEMS(color_formats); i++) {
+        if (avctx->pix_fmt == color_formats[i].pix_fmt) {
+            ff_AMediaFormat_setInt32(format, "color-format",
+                                     color_formats[i].color_format);
+            break;
+        }
+    }
+
+    if (avctx->bit_rate)
+        ff_AMediaFormat_setInt32(format, "bitrate", avctx->bit_rate);
+    // frame-rate and i-frame-interval are required to configure codec
+    if (avctx->framerate.num >= avctx->framerate.den && avctx->framerate.den > 0) {
+        s->fps = avctx->framerate.num / avctx->framerate.den;
+    } else {
+        s->fps = 30;
+        av_log(avctx, AV_LOG_INFO, "Use %d as the default MediaFormat frame-rate\n", s->fps);
+    }
+    gop = round(avctx->gop_size / s->fps);
+    if (gop == 0) {
+        gop = 1;
+        av_log(avctx, AV_LOG_INFO,
+                "Use %d as the default MediaFormat i-frame-interval, "
+                "please set gop_size properly (>= fps)\n", gop);
+    } else {
+        av_log(avctx, AV_LOG_DEBUG, "Set i-frame-interval to %d\n", gop);
+    }
+
+    ff_AMediaFormat_setInt32(format, "frame-rate", s->fps);
+    ff_AMediaFormat_setInt32(format, "i-frame-interval", gop);
+
+
+    ret = ff_AMediaCodec_getConfigureFlagEncode(s->codec);
+    ret = ff_AMediaCodec_configure(s->codec, format, s->window, NULL, ret);
+    if (ret) {
+        av_log(avctx, AV_LOG_ERROR, "MediaCodec configure failed, %s\n", av_err2str(ret));
+        goto bailout;
+    }
+
+    ret = ff_AMediaCodec_start(s->codec);
+    if (ret) {
+        av_log(avctx, AV_LOG_ERROR, "MediaCodec failed to start, %s\n", av_err2str(ret));
+        goto bailout;
+    }
+
+    mediacodec_output_format(avctx);
+
+    s->frame = av_frame_alloc();
+    if (!s->frame)
+        ret = AVERROR(ENOMEM);
+
+bailout:
+    if (format)
+        ff_AMediaFormat_delete(format);
+    return ret;
+}
+
+static int mediacodec_receive(AVCodecContext *avctx,
+                               AVPacket *pkt,
+                               int *got_packet)
+{
+    MediaCodecEncContext *s = avctx->priv_data;
+    FFAMediaCodec *codec = s->codec;
+    FFAMediaCodecBufferInfo out_info = {0};
+    uint8_t *out_buf;
+    size_t out_size = 0;
+    int ret;
+    int extradata_size = 0;
+    int64_t timeout_us = s->eof_sent ? OUTPUT_DEQUEUE_TIMEOUT_US : 0;
+    ssize_t index = ff_AMediaCodec_dequeueOutputBuffer(codec, &out_info, timeout_us);
+
+    if (ff_AMediaCodec_infoTryAgainLater(codec, index))
+        return AVERROR(EAGAIN);
+
+    if (ff_AMediaCodec_infoOutputFormatChanged(codec, index)) {
+        mediacodec_output_format(avctx);
+        return AVERROR(EAGAIN);
+    }
+
+    if (ff_AMediaCodec_infoOutputBuffersChanged(codec, index)) {
+        ff_AMediaCodec_cleanOutputBuffers(codec);
+        return AVERROR(EAGAIN);
+    }
+
+    if (index < 0)
+        return AVERROR_EXTERNAL;
+
+    if (out_info.flags & ff_AMediaCodec_getBufferFlagEndOfStream(codec))
+        return AVERROR_EOF;
+
+    out_buf = ff_AMediaCodec_getOutputBuffer(codec, index, &out_size);
+    if (!out_buf) {
+        ret = AVERROR_EXTERNAL;
+        goto bailout;
+    }
+
+    if (out_info.flags & ff_AMediaCodec_getBufferFlagCodecConfig(codec)) {
+        ret = av_reallocp(&s->extradata, out_info.size);
+        if (ret)
+            goto bailout;
+
+        s->extradata_size = out_info.size;
+        memcpy(s->extradata, out_buf + out_info.offset, out_info.size);
+        ff_AMediaCodec_releaseOutputBuffer(codec, index, false);
+        // try immediately
+        return mediacodec_receive(avctx, pkt, got_packet);
+    }
+
+    ret = ff_get_encode_buffer(avctx, pkt, out_info.size + s->extradata_size, 0);
+    if (ret < 0)
+      goto bailout;
+
+    if (s->extradata_size) {
+        extradata_size = s->extradata_size;
+        s->extradata_size = 0;
+        memcpy(pkt->data, s->extradata, extradata_size);
+    }
+    memcpy(pkt->data + extradata_size, out_buf + out_info.offset, out_info.size);
+    pkt->pts = av_rescale_q(out_info.presentationTimeUs, AV_TIME_BASE_Q, avctx->time_base);
+    if (s->ts_tail != s->ts_head) {
+        pkt->dts = s->timestamps[s->ts_tail];
+        s->ts_tail = (s->ts_tail + 1) % FF_ARRAY_ELEMS(s->timestamps);
+    }
+
+    if (out_info.flags & ff_AMediaCodec_getBufferFlagKeyFrame(codec))
+        pkt->flags |= AV_PKT_FLAG_KEY;
+    ret = 0;
+    *got_packet = 1;
+
+    av_log(avctx, AV_LOG_TRACE, "receive packet pts %" PRId64 " dts %" PRId64
+           " flags %d extradata %d\n",
+           pkt->pts, pkt->dts, pkt->flags, extradata_size);
+
+bailout:
+    ff_AMediaCodec_releaseOutputBuffer(codec, index, false);
+    return ret;
+}
+
+static void copy_frame_to_buffer(AVCodecContext *avctx, const AVFrame *frame, uint8_t *dst, size_t size)
+{
+    MediaCodecEncContext *s = avctx->priv_data;
+    uint8_t *dst_data[4] = {};
+    int dst_linesize[4] = {};
+    const uint8_t *src_data[4] = {
+            frame->data[0], frame->data[1], frame->data[2], frame->data[3]
+    };
+
+    if (avctx->pix_fmt == AV_PIX_FMT_YUV420P) {
+        dst_data[0] = dst;
+        dst_data[1] = dst + s->width * s->height;
+        dst_data[2] = dst_data[1] + s->width * s->height / 4;
+
+        dst_linesize[0] = s->width;
+        dst_linesize[1] = dst_linesize[2] = s->width / 2;
+    } else if (avctx->pix_fmt == AV_PIX_FMT_NV12) {
+        dst_data[0] = dst;
+        dst_data[1] = dst + s->width * s->height;
+
+        dst_linesize[0] = s->width;
+        dst_linesize[1] = s->width;
+    } else {
+        av_assert0(0);
+    }
+
+    av_image_copy(dst_data, dst_linesize, src_data, frame->linesize,
+                  avctx->pix_fmt, avctx->width, avctx->height);
+}
+
+static int mediacodec_send(AVCodecContext *avctx,
+                           const AVFrame *frame) {
+    MediaCodecEncContext *s = avctx->priv_data;
+    FFAMediaCodec *codec = s->codec;
+    ssize_t index;
+    uint8_t *input_buf = NULL;
+    size_t input_size = 0;
+    int64_t pts = 0;
+    uint32_t flags = 0;
+    int64_t timeout_us;
+
+    if (s->eof_sent)
+        return 0;
+
+    if (s->window) {
+        if (!frame) {
+            s->eof_sent = 1;
+            return ff_AMediaCodec_signalEndOfInputStream(codec);
+        }
+
+
+        if (frame->data[3]) {
+            pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q);
+            s->timestamps[s->ts_head] = frame->pts;
+            s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps);
+
+            av_mediacodec_release_buffer((AVMediaCodecBuffer *)frame->data[3], 1);
+        }
+        return 0;
+    }
+
+    timeout_us = INPUT_DEQUEUE_TIMEOUT_US;
+    index = ff_AMediaCodec_dequeueInputBuffer(codec, timeout_us);
+    if (ff_AMediaCodec_infoTryAgainLater(codec, index))
+        return AVERROR(EAGAIN);
+
+    if (index < 0) {
+        av_log(avctx, AV_LOG_ERROR, "dequeue input buffer failed, %zd", index);
+        return AVERROR_EXTERNAL;
+    }
+
+    if (frame) {
+        input_buf = ff_AMediaCodec_getInputBuffer(codec, index, &input_size);
+        copy_frame_to_buffer(avctx, frame, input_buf, input_size);
+
+        pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q);
+
+        s->timestamps[s->ts_head] = frame->pts;
+        s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps);
+    } else {
+        flags |= ff_AMediaCodec_getBufferFlagEndOfStream(codec);
+        s->eof_sent = 1;
+    }
+
+    ff_AMediaCodec_queueInputBuffer(codec, index, 0, input_size, pts, flags);
+    return 0;
+}
+
+static int mediacodec_encode(AVCodecContext *avctx, AVPacket *pkt)
+{
+    MediaCodecEncContext *s = avctx->priv_data;
+    int ret;
+    int got_packet = 0;
+
+    // Return on three case:
+    // 1. Serious error
+    // 2. Got a packet success
+    // 3. No AVFrame is available yet (don't return if get_frame return EOF)
+    while (1) {
+        ret = mediacodec_receive(avctx, pkt, &got_packet);
+        if (!ret)
+            return 0;
+        else if (ret != AVERROR(EAGAIN))
+            return ret;
+
+        if (!s->frame->buf[0]) {
+            ret = ff_encode_get_frame(avctx, s->frame);
+            if (ret && ret != AVERROR_EOF)
+                return ret;
+        }
+
+        ret = mediacodec_send(avctx, s->frame->buf[0] ? s->frame : NULL);
+        if (!ret)
+            av_frame_unref(s->frame);
+        else if (ret != AVERROR(EAGAIN))
+            return ret;
+    }
+
+    return 0;
+}
+
+static av_cold int mediacodec_close(AVCodecContext *avctx)
+{
+    MediaCodecEncContext *s = avctx->priv_data;
+    if (s->codec) {
+        ff_AMediaCodec_stop(s->codec);
+        ff_AMediaCodec_delete(s->codec);
+        s->codec = NULL;
+    }
+
+    if (s->window) {
+        ff_mediacodec_surface_unref(s->window, avctx);
+        s->window = NULL;
+    }
+
+    av_frame_free(&s->frame);
+
+    return 0;
+}
+
+static const AVCodecHWConfigInternal *const mediacodec_hw_configs[] = {
+    &(const AVCodecHWConfigInternal) {
+        .public          = {
+            .pix_fmt     = AV_PIX_FMT_MEDIACODEC,
+            .methods     = AV_CODEC_HW_CONFIG_METHOD_AD_HOC |
+                           AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX,
+            .device_type = AV_HWDEVICE_TYPE_MEDIACODEC,
+        },
+        .hwaccel         = NULL,
+    },
+    NULL
+};
+
+#define OFFSET(x) offsetof(MediaCodecEncContext, x)
+#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM
+static const AVOption common_options[] = {
+    { "ndk_codec", "Use MediaCodec from NDK",
+                    OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE },
+    { NULL },
+};
+
+#define MEDIACODEC_ENCODER_CLASS(name)              \
+static const AVClass name ## _mediacodec_class = {  \
+    .class_name = #name "_mediacodec",              \
+    .item_name  = av_default_item_name,             \
+    .option     = common_options,                   \
+    .version    = LIBAVUTIL_VERSION_INT,            \
+};                                                  \
+
+#define DECLARE_MEDIACODEC_ENCODER(short_name, long_name, codec_id)     \
+MEDIACODEC_ENCODER_CLASS(short_name)                                    \
+const FFCodec ff_ ## short_name ## _mediacodec_encoder = {              \
+    .p.name           = #short_name "_mediacodec",                      \
+    CODEC_LONG_NAME(long_name " Android MediaCodec encoder"),           \
+    .p.type           = AVMEDIA_TYPE_VIDEO,                             \
+    .p.id             = codec_id,                                       \
+    .p.capabilities   = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY           \
+                        | AV_CODEC_CAP_HARDWARE,                        \
+    .priv_data_size   = sizeof(MediaCodecEncContext),                   \
+    .p.pix_fmts       = avc_pix_fmts,                                   \
+    .init             = mediacodec_init,                                \
+    FF_CODEC_RECEIVE_PACKET_CB(mediacodec_encode),                      \
+    .close            = mediacodec_close,                               \
+    .p.priv_class     = &short_name ## _mediacodec_class,               \
+    .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,                      \
+    .p.wrapper_name = "mediacodec",                                     \
+    .hw_configs     = mediacodec_hw_configs,                            \
+};                                                                      \
+
+#if CONFIG_H264_MEDIACODEC_ENCODER
+DECLARE_MEDIACODEC_ENCODER(h264, "H.264", AV_CODEC_ID_H264)
+#endif
+
+#if CONFIG_HEVC_MEDIACODEC_ENCODER
+DECLARE_MEDIACODEC_ENCODER(hevc, "H.265", AV_CODEC_ID_HEVC)
+#endif
diff --git a/libavcodec/version.h b/libavcodec/version.h
index fcf589f61f..d149bc6c46 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,7 +29,7 @@
 
 #include "version_major.h"
 
-#define LIBAVCODEC_VERSION_MINOR  53
+#define LIBAVCODEC_VERSION_MINOR  54
 #define LIBAVCODEC_VERSION_MICRO 100
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 11/12] avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (9 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 10/12] avcodec: add MediaCodec encoder Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 12/12] fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec Zhao Zhili
  11 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

---
 libavutil/hwcontext.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
index ab9ad3703e..3396598269 100644
--- a/libavutil/hwcontext.c
+++ b/libavutil/hwcontext.c
@@ -397,10 +397,14 @@ int av_hwframe_transfer_get_formats(AVBufferRef *hwframe_ref,
 
 static int transfer_data_alloc(AVFrame *dst, const AVFrame *src, int flags)
 {
-    AVHWFramesContext *ctx = (AVHWFramesContext*)src->hw_frames_ctx->data;
+    AVHWFramesContext *ctx;
     AVFrame *frame_tmp;
     int ret = 0;
 
+    if (!src->hw_frames_ctx)
+        return AVERROR(EINVAL);
+    ctx = (AVHWFramesContext*)src->hw_frames_ctx->data;
+
     frame_tmp = av_frame_alloc();
     if (!frame_tmp)
         return AVERROR(ENOMEM);
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v2 12/12] fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec
       [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
                   ` (10 preceding siblings ...)
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 11/12] avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc Zhao Zhili
@ 2022-11-20  6:49 ` Zhao Zhili
  2022-11-20  6:56   ` [FFmpeg-devel] [PATCH v3 12/12] fftools/ffmpeg_demux: " Zhao Zhili
  11 siblings, 1 reply; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:49 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: matthieu.bouron, Zhao Zhili, aman

From: Zhao Zhili <zhilizhao@tencent.com>

---
 fftools/ffmpeg_demux.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c
index 76778d774d..d2a6d39703 100644
--- a/fftools/ffmpeg_demux.c
+++ b/fftools/ffmpeg_demux.c
@@ -625,6 +625,10 @@ static void add_input_streams(const OptionsContext *o, AVFormatContext *ic)
                     "with old commandlines. This behaviour is DEPRECATED and will be removed "
                     "in the future. Please explicitly set \"-hwaccel_output_format qsv\".\n");
                 ist->hwaccel_output_format = AV_PIX_FMT_QSV;
+            } else if (!hwaccel_output_format && hwaccel && !strcmp(hwaccel, "mediacodec")) {
+                // There is no real AVHWFrameContext implementation. Set
+                // hwaccel_output_format to avoid av_hwframe_transfer_data error.
+                ist->hwaccel_output_format = AV_PIX_FMT_MEDIACODEC;
             } else if (hwaccel_output_format) {
                 ist->hwaccel_output_format = av_get_pix_fmt(hwaccel_output_format);
                 if (ist->hwaccel_output_format == AV_PIX_FMT_NONE) {
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [FFmpeg-devel] [PATCH v3 12/12] fftools/ffmpeg_demux: set default hwaccel_output_format for mediacodec
  2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 12/12] fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec Zhao Zhili
@ 2022-11-20  6:56   ` Zhao Zhili
  0 siblings, 0 replies; 13+ messages in thread
From: Zhao Zhili @ 2022-11-20  6:56 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Zhao Zhili

From: Zhao Zhili <zhilizhao@tencent.com>

---
Fix commit subject ffmpeg_opt -> ffmpeg_demux

 fftools/ffmpeg_demux.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c
index 76778d774d..d2a6d39703 100644
--- a/fftools/ffmpeg_demux.c
+++ b/fftools/ffmpeg_demux.c
@@ -625,6 +625,10 @@ static void add_input_streams(const OptionsContext *o, AVFormatContext *ic)
                     "with old commandlines. This behaviour is DEPRECATED and will be removed "
                     "in the future. Please explicitly set \"-hwaccel_output_format qsv\".\n");
                 ist->hwaccel_output_format = AV_PIX_FMT_QSV;
+            } else if (!hwaccel_output_format && hwaccel && !strcmp(hwaccel, "mediacodec")) {
+                // There is no real AVHWFrameContext implementation. Set
+                // hwaccel_output_format to avoid av_hwframe_transfer_data error.
+                ist->hwaccel_output_format = AV_PIX_FMT_MEDIACODEC;
             } else if (hwaccel_output_format) {
                 ist->hwaccel_output_format = av_get_pix_fmt(hwaccel_output_format);
                 if (ist->hwaccel_output_format == AV_PIX_FMT_NONE) {
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2022-11-20  6:57 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20221120064947.94540-1-quinkblack@foxmail.com>
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 01/12] avcodec/mediacodec: fix incorrect crop info Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 02/12] avcodec/mediacodecdec: don't break out if both input and output port return try again Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 03/12] avcodec/mediacodecdec_common: fix misuse av_free/av_freep Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 04/12] avcodec/mediacodecdec_common: fix useless av_buffer_unref Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 05/12] avcodec/mediacodec_wrapper: separate implementation from interface Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 06/12] avcodec/mediacodec: add NDK media codec wrapper Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 07/12] avcodec/mediacodecdec: enable NDK mediacodec Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 08/12] avutil/hwcontext_mediacodec: add ANativeWindow support Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 09/12] avcodec/mediacodec: " Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 10/12] avcodec: add MediaCodec encoder Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 11/12] avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc Zhao Zhili
2022-11-20  6:49 ` [FFmpeg-devel] [PATCH v2 12/12] fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec Zhao Zhili
2022-11-20  6:56   ` [FFmpeg-devel] [PATCH v3 12/12] fftools/ffmpeg_demux: " Zhao Zhili

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git