[FFmpeg-devel] [RFC PATCH 0/3] Propagate PRFT side data

[FFmpeg-devel] [RFC PATCH 0/3] Propagate PRFT side data

[Clément Péron]

Dear FFMpeg contributors,

I'm new to the FFMpeg code base and audio/video sync world, so forgive me in
advance If my questions are a bit dumb.

I have a project where I need to synchronize multiple RTSP cameras with other
network sensors (sync with NTP or PTP).
In my case, I used Ffmpeg to decode the RTSP stream and then output the rawvideo
to the stdout pipe.

After looking in the RTPdec, I found multiple timestamps PST, DTS and also the
PRFT (Producer Reference Timestamp). In my case the PRFT seems the correct one.

After several tests and diggs, I found that the AV_PKT_DATA_PRFT produce by the
RTSP Demux doesn't seems to be forwarded to the encoder/decoder, nor to the
final Muxer.

So I have multiple question:

Is the forward of the AV_PKT_DATA_PRFT the correct solution?

I saw also that Dashenc and Movenc use this Side data but how do they get it?

Actually I have a dirty hack to output PRFT on the stdout, is there something
"more standard" to communicate between Ffmpeg and a python script?

Thanks for your help,
Clément

Clément Péron (3):
  frame: decode: propagate PRFT side data packet to frame
  avcodec: rawenc: Forward PRFT frame data to packet
  HACK: avformat: rawenc: allow to output a raw PRFT

 libavcodec/decode.c      |   1 +
 libavcodec/rawenc.c      |  12 ++++
 libavfilter/f_sidedata.c |   1 +
 libavformat/rawenc.c     | 122 +++++++++++++++++++++++++++++++++++++++
 libavutil/frame.c        |   1 +
 libavutil/frame.h        |   4 ++
 6 files changed, 141 insertions(+)

-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [RFC PATCH 1/3] frame: decode: propagate PRFT side data packet to frame

[Clément Péron]

Introduce a new AV_FRAME_DATA_PRFT to propagate Producer Reference Timestamp.

Signed-off-by: Clément Péron <peron.clem@gmail.com>
---
 libavcodec/decode.c      | 1 +
 libavfilter/f_sidedata.c | 1 +
 libavutil/frame.c        | 1 +
 libavutil/frame.h        | 4 ++++
 4 files changed, 7 insertions(+)

diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 169ee79acd..223bb2e35e 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -1429,6 +1429,7 @@ int ff_decode_frame_props_from_pkt(const AVCodecContext *avctx,
         { AV_PKT_DATA_S12M_TIMECODE,              AV_FRAME_DATA_S12M_TIMECODE },
         { AV_PKT_DATA_DYNAMIC_HDR10_PLUS,         AV_FRAME_DATA_DYNAMIC_HDR_PLUS },
         { AV_PKT_DATA_SKIP_SAMPLES,               AV_FRAME_DATA_SKIP_SAMPLES },
+        { AV_PKT_DATA_PRFT,                       AV_FRAME_DATA_PRFT },
     };
 
     frame->pts          = pkt->pts;
diff --git a/libavfilter/f_sidedata.c b/libavfilter/f_sidedata.c
index c86190b062..becc5cfa3e 100644
--- a/libavfilter/f_sidedata.c
+++ b/libavfilter/f_sidedata.c
@@ -73,6 +73,7 @@ static const AVOption filt_name##_options[] = { \
     {   "REGIONS_OF_INTEREST",        "", 0,             AV_OPT_TYPE_CONST,  {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST        }, 0, 0, FLAGS, "type" }, \
     {   "DETECTION_BOUNDING_BOXES",   "", 0,             AV_OPT_TYPE_CONST,  {.i64 = AV_FRAME_DATA_DETECTION_BBOXES           }, 0, 0, FLAGS, "type" }, \
     {   "SEI_UNREGISTERED",           "", 0,             AV_OPT_TYPE_CONST,  {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED           }, 0, 0, FLAGS, "type" }, \
+    {   "PRFT",                       "", 0,             AV_OPT_TYPE_CONST,  {.i64 = AV_FRAME_DATA_PRFT                       }, 0, 0, FLAGS, "type" }, \
     { NULL } \
 }
 
diff --git a/libavutil/frame.c b/libavutil/frame.c
index a3f07ca089..d379e57814 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -958,6 +958,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type)
     case AV_FRAME_DATA_DOVI_RPU_BUFFER:             return "Dolby Vision RPU Data";
     case AV_FRAME_DATA_DOVI_METADATA:               return "Dolby Vision Metadata";
     case AV_FRAME_DATA_AMBIENT_VIEWING_ENVIRONMENT: return "Ambient viewing environment";
+    case AV_FRAME_DATA_PRFT                       : return "Producer time reference";
     }
     return NULL;
 }
diff --git a/libavutil/frame.h b/libavutil/frame.h
index c0c1b23db7..8bd87aa545 100644
--- a/libavutil/frame.h
+++ b/libavutil/frame.h
@@ -224,6 +224,10 @@ enum AVFrameSideDataType {
      * encoding.
      */
     AV_FRAME_DATA_VIDEO_HINT,
+    /**
+     * Producer time reference data in the form of the AVProducerReferenceTime struct.
+     */
+    AV_FRAME_DATA_PRFT,
 };
 
 enum AVActiveFormatDescription {
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [RFC PATCH 2/3] avcodec: rawenc: Forward PRFT frame data to packet

[Clément Péron]

Producer Reference time could be used in the muxing, propagate it.

Signed-off-by: Clément Péron <peron.clem@gmail.com>
---
 libavcodec/rawenc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/libavcodec/rawenc.c b/libavcodec/rawenc.c
index 8c577006d9..f238c8e165 100644
--- a/libavcodec/rawenc.c
+++ b/libavcodec/rawenc.c
@@ -49,6 +49,8 @@ static av_cold int raw_encode_init(AVCodecContext *avctx)
 static int raw_encode(AVCodecContext *avctx, AVPacket *pkt,
                       const AVFrame *frame, int *got_packet)
 {
+    AVFrameSideData *side_data;
+
     int ret = av_image_get_buffer_size(frame->format,
                                        frame->width, frame->height, 1);
 
@@ -78,6 +80,16 @@ static int raw_encode(AVCodecContext *avctx, AVPacket *pkt,
         }
     }
     *got_packet = 1;
+
+    // Forward the PRFT to Mux
+    side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_PRFT);
+    if (side_data && side_data->size) {
+        uint8_t *buf = av_packet_new_side_data(pkt, AV_PKT_DATA_PRFT, side_data->size);
+        if (!buf)
+            return AVERROR(ENOMEM);
+        memcpy(buf, side_data->data, side_data->size);
+    }
+
     return 0;
 }
 
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Output the producer reference time to a dirty raw output.

Signed-off-by: Clément Péron <peron.clem@gmail.com>
---
 libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)

diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c
index f916db13a2..2953f07ec6 100644
--- a/libavformat/rawenc.c
+++ b/libavformat/rawenc.c
@@ -23,13 +23,49 @@
 #include "config_components.h"
 
 #include "libavutil/intreadwrite.h"
+#include "libavutil/opt.h"
 
 #include "avformat.h"
 #include "rawenc.h"
 #include "mux.h"
 
+typedef struct RawVideoContext {
+    const AVClass *class;
+
+    int write_prft;
+} RawVideoContext;
+
+// We want to have access to the timestamp with rawvideo
+static int ff_raw_write_prft(AVFormatContext *s, AVPacket *pkt)
+{
+    size_t prft_size = 0;
+    AVProducerReferenceTime *prft =
+        (AVProducerReferenceTime *)av_packet_get_side_data(pkt, AV_PKT_DATA_PRFT, &prft_size);
+
+    double frame_ts;
+
+    avio_write(s->pb, "TIMESTAMP_MAGIC", 16);
+    if (prft && prft_size == sizeof(AVProducerReferenceTime)) {
+        // Save the frame_ts as a double
+        frame_ts = (prft->wallclock) / (double)AV_TIME_BASE;
+        av_log(s, AV_LOG_DEBUG, "ff_raw_write_packet: frame_ts %f.\n", frame_ts);
+        avio_write(s->pb, (void *)&frame_ts, sizeof(frame_ts));
+    } else {
+        av_log(s, AV_LOG_DEBUG, "ff_raw_write_packet: No Timestamp.\n");
+        avio_write(s->pb, "\0\0\0\0\0\0\0", 8);
+    }
+
+    avio_write(s->pb, "FRAMEDATA_MAGIC", 16);
+    return 0;
+}
+
 int ff_raw_write_packet(AVFormatContext *s, AVPacket *pkt)
 {
+    RawVideoContext *raw = s->priv_data;
+
+    if (raw->write_prft)
+        ff_raw_write_prft(s, pkt);
+
     avio_write(s->pb, pkt->data, pkt->size);
     return 0;
 }
@@ -56,12 +92,27 @@ static int force_one_stream(AVFormatContext *s)
     return 0;
 }
 
+#undef OFFSET
+#define OFFSET(x) offsetof(RawVideoContext, x)
+static const AVOption raw_options[] = {
+    { "write_prft", "Output the Producer Reference Time", OFFSET(write_prft), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AV_OPT_FLAG_DECODING_PARAM },
+    { NULL }
+};
+
+static const AVClass raw_muxer_class = {
+    .class_name = "RAW muxer",
+    .item_name  = av_default_item_name,
+    .option     = raw_options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
 /* Note: Do not forget to add new entries to the Makefile as well. */
 
 #if CONFIG_AC3_MUXER
 const FFOutputFormat ff_ac3_muxer = {
     .p.name            = "ac3",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw AC-3"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/x-ac3",
     .p.extensions      = "ac3",
     .p.audio_codec     = AV_CODEC_ID_AC3,
@@ -69,6 +120,7 @@ const FFOutputFormat ff_ac3_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -95,6 +147,7 @@ static int adx_write_trailer(AVFormatContext *s)
 const FFOutputFormat ff_adx_muxer = {
     .p.name            = "adx",
     .p.long_name       = NULL_IF_CONFIG_SMALL("CRI ADX"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "adx",
     .p.audio_codec     = AV_CODEC_ID_ADPCM_ADX,
     .p.video_codec     = AV_CODEC_ID_NONE,
@@ -102,6 +155,7 @@ const FFOutputFormat ff_adx_muxer = {
     .write_packet      = ff_raw_write_packet,
     .write_trailer     = adx_write_trailer,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -109,12 +163,14 @@ const FFOutputFormat ff_adx_muxer = {
 const FFOutputFormat ff_aptx_muxer = {
     .p.name            = "aptx",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw aptX (Audio Processing Technology for Bluetooth)"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "aptx",
     .p.audio_codec     = AV_CODEC_ID_APTX,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -122,12 +178,14 @@ const FFOutputFormat ff_aptx_muxer = {
 const FFOutputFormat ff_aptx_hd_muxer = {
     .p.name            = "aptx_hd",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw aptX HD (Audio Processing Technology for Bluetooth)"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "aptxhd",
     .p.audio_codec     = AV_CODEC_ID_APTX_HD,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -135,12 +193,14 @@ const FFOutputFormat ff_aptx_hd_muxer = {
 const FFOutputFormat ff_avs2_muxer = {
     .p.name            = "avs2",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw AVS2-P2/IEEE1857.4 video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "avs,avs2",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_AVS2,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -148,12 +208,14 @@ const FFOutputFormat ff_avs2_muxer = {
 const FFOutputFormat ff_avs3_muxer = {
     .p.name            = "avs3",
     .p.long_name       = NULL_IF_CONFIG_SMALL("AVS3-P2/IEEE1857.10"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "avs3",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_AVS3,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -162,12 +224,14 @@ const FFOutputFormat ff_avs3_muxer = {
 const FFOutputFormat ff_cavsvideo_muxer = {
     .p.name            = "cavsvideo",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw Chinese AVS (Audio Video Standard) video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "cavs",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_CAVS,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -175,11 +239,13 @@ const FFOutputFormat ff_cavsvideo_muxer = {
 const FFOutputFormat ff_codec2raw_muxer = {
     .p.name            = "codec2raw",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw codec2 muxer"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.audio_codec     = AV_CODEC_ID_CODEC2,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -188,9 +254,11 @@ const FFOutputFormat ff_codec2raw_muxer = {
 const FFOutputFormat ff_data_muxer = {
     .p.name            = "data",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw data"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -198,12 +266,14 @@ const FFOutputFormat ff_data_muxer = {
 const FFOutputFormat ff_dfpwm_muxer = {
     .p.name            = "dfpwm",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw DFPWM1a"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "dfpwm",
     .p.audio_codec     = AV_CODEC_ID_DFPWM,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -211,12 +281,14 @@ const FFOutputFormat ff_dfpwm_muxer = {
 const FFOutputFormat ff_dirac_muxer = {
     .p.name            = "dirac",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw Dirac"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "drc,vc2",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_DIRAC,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -224,12 +296,14 @@ const FFOutputFormat ff_dirac_muxer = {
 const FFOutputFormat ff_dnxhd_muxer = {
     .p.name            = "dnxhd",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw DNxHD (SMPTE VC-3)"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "dnxhd,dnxhr",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_DNXHD,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -237,6 +311,7 @@ const FFOutputFormat ff_dnxhd_muxer = {
 const FFOutputFormat ff_dts_muxer = {
     .p.name            = "dts",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw DTS"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/x-dca",
     .p.extensions      = "dts",
     .p.audio_codec     = AV_CODEC_ID_DTS,
@@ -244,6 +319,7 @@ const FFOutputFormat ff_dts_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -251,6 +327,7 @@ const FFOutputFormat ff_dts_muxer = {
 const FFOutputFormat ff_eac3_muxer = {
     .p.name            = "eac3",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw E-AC-3"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/x-eac3",
     .p.extensions      = "eac3,ec3",
     .p.audio_codec     = AV_CODEC_ID_EAC3,
@@ -258,6 +335,7 @@ const FFOutputFormat ff_eac3_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -265,6 +343,7 @@ const FFOutputFormat ff_eac3_muxer = {
 const FFOutputFormat ff_g722_muxer = {
     .p.name            = "g722",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw G.722"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/G722",
     .p.extensions      = "g722",
     .p.audio_codec     = AV_CODEC_ID_ADPCM_G722,
@@ -272,6 +351,7 @@ const FFOutputFormat ff_g722_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -279,6 +359,7 @@ const FFOutputFormat ff_g722_muxer = {
 const FFOutputFormat ff_g723_1_muxer = {
     .p.name            = "g723_1",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw G.723.1"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/g723",
     .p.extensions      = "tco,rco",
     .p.audio_codec     = AV_CODEC_ID_G723_1,
@@ -286,6 +367,7 @@ const FFOutputFormat ff_g723_1_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -293,11 +375,13 @@ const FFOutputFormat ff_g723_1_muxer = {
 const FFOutputFormat ff_g726_muxer = {
     .p.name            = "g726",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw big-endian G.726 (\"left-justified\")"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.audio_codec     = AV_CODEC_ID_ADPCM_G726,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -305,11 +389,13 @@ const FFOutputFormat ff_g726_muxer = {
 const FFOutputFormat ff_g726le_muxer = {
     .p.name            = "g726le",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw little-endian G.726 (\"right-justified\")"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.audio_codec     = AV_CODEC_ID_ADPCM_G726LE,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -317,6 +403,7 @@ const FFOutputFormat ff_g726le_muxer = {
 const FFOutputFormat ff_gsm_muxer = {
     .p.name            = "gsm",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw GSM"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/x-gsm",
     .p.extensions      = "gsm",
     .p.audio_codec     = AV_CODEC_ID_GSM,
@@ -324,6 +411,7 @@ const FFOutputFormat ff_gsm_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -331,6 +419,7 @@ const FFOutputFormat ff_gsm_muxer = {
 const FFOutputFormat ff_h261_muxer = {
     .p.name            = "h261",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw H.261"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "video/x-h261",
     .p.extensions      = "h261",
     .p.audio_codec     = AV_CODEC_ID_NONE,
@@ -338,6 +427,7 @@ const FFOutputFormat ff_h261_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -345,6 +435,7 @@ const FFOutputFormat ff_h261_muxer = {
 const FFOutputFormat ff_h263_muxer = {
     .p.name            = "h263",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw H.263"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "video/x-h263",
     .p.extensions      = "h263",
     .p.audio_codec     = AV_CODEC_ID_NONE,
@@ -352,6 +443,7 @@ const FFOutputFormat ff_h263_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -368,6 +460,7 @@ static int h264_check_bitstream(AVFormatContext *s, AVStream *st,
 const FFOutputFormat ff_h264_muxer = {
     .p.name            = "h264",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw H.264 video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "h264,264",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_H264,
@@ -375,6 +468,7 @@ const FFOutputFormat ff_h264_muxer = {
     .write_packet      = ff_raw_write_packet,
     .check_bitstream   = h264_check_bitstream,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -391,6 +485,7 @@ static int vvc_check_bitstream(AVFormatContext *s, AVStream *st,
 const FFOutputFormat ff_vvc_muxer = {
     .p.name            = "vvc",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw H.266/VVC video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "vvc,h266,266",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_VVC,
@@ -398,6 +493,7 @@ const FFOutputFormat ff_vvc_muxer = {
     .write_packet      = ff_raw_write_packet,
     .check_bitstream   = vvc_check_bitstream,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -414,6 +510,7 @@ static int hevc_check_bitstream(AVFormatContext *s, AVStream *st,
 const FFOutputFormat ff_hevc_muxer = {
     .p.name            = "hevc",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw HEVC video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "hevc,h265,265",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_HEVC,
@@ -421,6 +518,7 @@ const FFOutputFormat ff_hevc_muxer = {
     .write_packet      = ff_raw_write_packet,
     .check_bitstream   = hevc_check_bitstream,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -428,12 +526,14 @@ const FFOutputFormat ff_hevc_muxer = {
 const FFOutputFormat ff_evc_muxer = {
     .p.name            = "evc",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw EVC video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "evc",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_EVC,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -441,12 +541,14 @@ const FFOutputFormat ff_evc_muxer = {
 const FFOutputFormat ff_m4v_muxer = {
     .p.name            = "m4v",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw MPEG-4 video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "m4v",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_MPEG4,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -454,6 +556,7 @@ const FFOutputFormat ff_m4v_muxer = {
 const FFOutputFormat ff_mjpeg_muxer = {
     .p.name            = "mjpeg",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw MJPEG video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "video/x-mjpeg",
     .p.extensions      = "mjpg,mjpeg",
     .p.audio_codec     = AV_CODEC_ID_NONE,
@@ -461,6 +564,7 @@ const FFOutputFormat ff_mjpeg_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -468,12 +572,14 @@ const FFOutputFormat ff_mjpeg_muxer = {
 const FFOutputFormat ff_mlp_muxer = {
     .p.name            = "mlp",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw MLP"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "mlp",
     .p.audio_codec     = AV_CODEC_ID_MLP,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -481,6 +587,7 @@ const FFOutputFormat ff_mlp_muxer = {
 const FFOutputFormat ff_mp2_muxer = {
     .p.name            = "mp2",
     .p.long_name       = NULL_IF_CONFIG_SMALL("MP2 (MPEG audio layer 2)"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/mpeg",
     .p.extensions      = "mp2,m2a,mpa",
     .p.audio_codec     = AV_CODEC_ID_MP2,
@@ -488,6 +595,7 @@ const FFOutputFormat ff_mp2_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -495,6 +603,7 @@ const FFOutputFormat ff_mp2_muxer = {
 const FFOutputFormat ff_mpeg1video_muxer = {
     .p.name            = "mpeg1video",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw MPEG-1 video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "video/mpeg",
     .p.extensions      = "mpg,mpeg,m1v",
     .p.audio_codec     = AV_CODEC_ID_NONE,
@@ -502,6 +611,7 @@ const FFOutputFormat ff_mpeg1video_muxer = {
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -509,12 +619,14 @@ const FFOutputFormat ff_mpeg1video_muxer = {
 const FFOutputFormat ff_mpeg2video_muxer = {
     .p.name            = "mpeg2video",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw MPEG-2 video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "m2v",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_MPEG2VIDEO,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -528,6 +640,7 @@ static int obu_check_bitstream(AVFormatContext *s, AVStream *st,
 const FFOutputFormat ff_obu_muxer = {
     .p.name            = "obu",
     .p.long_name       = NULL_IF_CONFIG_SMALL("AV1 low overhead OBU"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "obu",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_AV1,
@@ -535,6 +648,7 @@ const FFOutputFormat ff_obu_muxer = {
     .write_packet      = ff_raw_write_packet,
     .check_bitstream   = obu_check_bitstream,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -542,11 +656,13 @@ const FFOutputFormat ff_obu_muxer = {
 const FFOutputFormat ff_rawvideo_muxer = {
     .p.name            = "rawvideo",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "yuv,rgb",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_RAWVIDEO,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -554,12 +670,14 @@ const FFOutputFormat ff_rawvideo_muxer = {
 const FFOutputFormat ff_sbc_muxer = {
     .p.name            = "sbc",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw SBC"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.mime_type       = "audio/x-sbc",
     .p.extensions      = "sbc,msbc",
     .p.audio_codec     = AV_CODEC_ID_SBC,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -567,12 +685,14 @@ const FFOutputFormat ff_sbc_muxer = {
 const FFOutputFormat ff_truehd_muxer = {
     .p.name            = "truehd",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw TrueHD"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "thd",
     .p.audio_codec     = AV_CODEC_ID_TRUEHD,
     .p.video_codec     = AV_CODEC_ID_NONE,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
 
@@ -580,11 +700,13 @@ const FFOutputFormat ff_truehd_muxer = {
 const FFOutputFormat ff_vc1_muxer = {
     .p.name            = "vc1",
     .p.long_name       = NULL_IF_CONFIG_SMALL("raw VC-1 video"),
+    .priv_data_size    = sizeof(RawVideoContext),
     .p.extensions      = "vc1",
     .p.audio_codec     = AV_CODEC_ID_NONE,
     .p.video_codec     = AV_CODEC_ID_VC1,
     .init              = force_one_stream,
     .write_packet      = ff_raw_write_packet,
     .p.flags           = AVFMT_NOTIMESTAMPS,
+    .p.priv_class      = &raw_muxer_class,
 };
 #endif
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 0/3] Propagate PRFT side data

[Kieran Kunhya]

 On Thu, 21 Sept 2023, 13:17 Clément Péron, <peron.clem@gmail.com> wrote:

> 4I have a project where I need to synchronize multiple RTSP cameras with
> other
> network sensors (sync with NTP or PTP).
>

Just be aware the clock of the vast majority of cameras have no relation to
NTP or PTP so you will have drift and need to handle that (e.g by dropping
or duplicating frames).

Kieran

>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 0/3] Propagate PRFT side data

[Clément Péron]

Hi Kieran,

On Thu, 21 Sept 2023 at 15:13, Kieran Kunhya <kierank@obe.tv> wrote:
>
>  On Thu, 21 Sept 2023, 13:17 Clément Péron, <peron.clem@gmail.com> wrote:
>>
>> 4I have a project where I need to synchronize multiple RTSP cameras with other
>> network sensors (sync with NTP or PTP).
>
>
> Just be aware the clock of the vast majority of cameras have no relation to NTP or PTP so you will have drift and need to handle that (e.g by dropping or duplicating frames).

Thanks for pointing this out, and yes I consider each of my sensors
running on a free clock and I recreate a "virtual frame" that is not
correlated to the FPS of each sensor.

Thanks,
Clement

>
> Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 780 bytes --]

On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> Output the producer reference time to a dirty raw output.
> 
> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> ---
>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 122 insertions(+)

this breaks fate-filter-volume and others
(Segmentation fault)

I can rerun it with debug symbols and provide peoper gdb output
but i suspect given this has "HACK" in the title you are aware of this

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Homeopathy is like voting while filling the ballot out with transparent ink.
Sometimes the outcome one wanted occurs. Rarely its worse than filling out
a ballot properly.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Andreas Rheinhardt]

Michael Niedermayer:
> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
>> Output the producer reference time to a dirty raw output.
>>
>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
>> ---
>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 122 insertions(+)
> 
> this breaks fate-filter-volume and others
> (Segmentation fault)
> 
> I can rerun it with debug symbols and provide peoper gdb output
> but i suspect given this has "HACK" in the title you are aware of this
> 

The reason is that this patch presumes that every user of
ff_raw_write_packet() uses the newly added RawVideoContext as context,
but that is just not true.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Michael, Andreas,

On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
<andreas.rheinhardt@outlook.com> wrote:
>
> Michael Niedermayer:
> > On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> >> Output the producer reference time to a dirty raw output.
> >>
> >> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> >> ---
> >>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
> >>  1 file changed, 122 insertions(+)
> >
> > this breaks fate-filter-volume and others
> > (Segmentation fault)
> >
> > I can rerun it with debug symbols and provide peoper gdb output
> > but i suspect given this has "HACK" in the title you are aware of this

The "HACK" tag meaning was not supposed to be: "it's ok if it
segfaults", but more to trigger a discussion is it possible to
properly support an output timestamp in the raw video demux, and if
yes how to do it :)

I consider this patch dirty and not upstreamable like this.

I thought about it during the night and would it be ok to add a
"raw_fmt" option.

-raw_fmt="prft:packet"
or
-raw_fmt="packet:prft"
or
-raw_fmt="prft" to only output the prft packet

If you have another idea I would be glad to hear it,
thanks

> >
>
> The reason is that this patch presumes that every user of
> ff_raw_write_packet() uses the newly added RawVideoContext as context,
> but that is just not true.

Thanks for the explanation, I will fix it in v2,

Clement

>
> - Andreas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Andreas Rheinhardt]

Clément Péron:
> Hi Michael, Andreas,
> 
> On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> <andreas.rheinhardt@outlook.com> wrote:
>>
>> Michael Niedermayer:
>>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
>>>> Output the producer reference time to a dirty raw output.
>>>>
>>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
>>>> ---
>>>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
>>>>  1 file changed, 122 insertions(+)
>>>
>>> this breaks fate-filter-volume and others
>>> (Segmentation fault)
>>>
>>> I can rerun it with debug symbols and provide peoper gdb output
>>> but i suspect given this has "HACK" in the title you are aware of this
> 
> The "HACK" tag meaning was not supposed to be: "it's ok if it
> segfaults", but more to trigger a discussion is it possible to
> properly support an output timestamp in the raw video demux, and if
> yes how to do it :)

If you need a timestamp for raw video, then use a proper container and
not raw video. In fact, this patch basically creates new formats
different from all the raw formats.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Andreas,

On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
<andreas.rheinhardt@outlook.com> wrote:
>
> Clément Péron:
> > Hi Michael, Andreas,
> >
> > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > <andreas.rheinhardt@outlook.com> wrote:
> >>
> >> Michael Niedermayer:
> >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> >>>> Output the producer reference time to a dirty raw output.
> >>>>
> >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> >>>> ---
> >>>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
> >>>>  1 file changed, 122 insertions(+)
> >>>
> >>> this breaks fate-filter-volume and others
> >>> (Segmentation fault)
> >>>
> >>> I can rerun it with debug symbols and provide peoper gdb output
> >>> but i suspect given this has "HACK" in the title you are aware of this
> >
> > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > segfaults", but more to trigger a discussion is it possible to
> > properly support an output timestamp in the raw video demux, and if
> > yes how to do it :)
>
> If you need a timestamp for raw video, then use a proper container and
> not raw video. In fact, this patch basically creates new formats
> different from all the raw formats.

Yes I agree, but I do not want to add too much overhead nor
computation processing or memory copy to my pipeline just to mux and
demux between ffmpeg and my python script.

The idea is to have a very light structure to easily pipe it.

I'm not familiar with audio/video container but it seems to me that
parsing containers are not very light no?

Thanks,
Clement


>
> - Andreas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Paul B Mahol]

On Fri, Sep 22, 2023 at 10:38 AM Clément Péron <peron.clem@gmail.com> wrote:

> Hi Andreas,
>
> On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> <andreas.rheinhardt@outlook.com> wrote:
> >
> > Clément Péron:
> > > Hi Michael, Andreas,
> > >
> > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > > <andreas.rheinhardt@outlook.com> wrote:
> > >>
> > >> Michael Niedermayer:
> > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> > >>>> Output the producer reference time to a dirty raw output.
> > >>>>
> > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > >>>> ---
> > >>>>  libavformat/rawenc.c | 122
> +++++++++++++++++++++++++++++++++++++++++++
> > >>>>  1 file changed, 122 insertions(+)
> > >>>
> > >>> this breaks fate-filter-volume and others
> > >>> (Segmentation fault)
> > >>>
> > >>> I can rerun it with debug symbols and provide peoper gdb output
> > >>> but i suspect given this has "HACK" in the title you are aware of
> this
> > >
> > > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > > segfaults", but more to trigger a discussion is it possible to
> > > properly support an output timestamp in the raw video demux, and if
> > > yes how to do it :)
> >
> > If you need a timestamp for raw video, then use a proper container and
> > not raw video. In fact, this patch basically creates new formats
> > different from all the raw formats.
>
> Yes I agree, but I do not want to add too much overhead nor
> computation processing or memory copy to my pipeline just to mux and
> demux between ffmpeg and my python script.
>
> The idea is to have a very light structure to easily pipe it.
>
> I'm not familiar with audio/video container but it seems to me that
> parsing containers are not very light no?
>
>
Containers range from almost no processing like .y4m to complex monsters
like .mxf

This patch is hack and approach/solution it tries is flawed.


> Thanks,
> Clement
>
>
> >
> > - Andreas
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Andreas Rheinhardt]

Clément Péron:
> Hi Andreas,
> 
> On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> <andreas.rheinhardt@outlook.com> wrote:
>>
>> Clément Péron:
>>> Hi Michael, Andreas,
>>>
>>> On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
>>> <andreas.rheinhardt@outlook.com> wrote:
>>>>
>>>> Michael Niedermayer:
>>>>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
>>>>>> Output the producer reference time to a dirty raw output.
>>>>>>
>>>>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
>>>>>> ---
>>>>>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
>>>>>>  1 file changed, 122 insertions(+)
>>>>>
>>>>> this breaks fate-filter-volume and others
>>>>> (Segmentation fault)
>>>>>
>>>>> I can rerun it with debug symbols and provide peoper gdb output
>>>>> but i suspect given this has "HACK" in the title you are aware of this
>>>
>>> The "HACK" tag meaning was not supposed to be: "it's ok if it
>>> segfaults", but more to trigger a discussion is it possible to
>>> properly support an output timestamp in the raw video demux, and if
>>> yes how to do it :)
>>
>> If you need a timestamp for raw video, then use a proper container and
>> not raw video. In fact, this patch basically creates new formats
>> different from all the raw formats.
> 
> Yes I agree, but I do not want to add too much overhead nor
> computation processing or memory copy to my pipeline just to mux and
> demux between ffmpeg and my python script.
> 
> The idea is to have a very light structure to easily pipe it.
> 

Our libraries are meant to be used by API users and are designed for
that. The ffmpeg command line tool is just one user among many and
adding code to a library to circumvent a limitation of ffmpeg (or
another user of the libraries) is not appropriate. We would end up with
a ton of hacks.

> I'm not familiar with audio/video container but it seems to me that
> parsing containers are not very light no?

For certain formats the overhead of parsing containers can be negative
when compared to the raw format, because the containers provide a length
field for the packet whereas one has to search for the packet boundaries
in case of a truely raw stream. But this is not true for raw video which
is fixed size.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Paul

On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda@gmail.com> wrote:
>
> On Fri, Sep 22, 2023 at 10:38 AM Clément Péron <peron.clem@gmail.com> wrote:
>
> > Hi Andreas,
> >
> > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> > <andreas.rheinhardt@outlook.com> wrote:
> > >
> > > Clément Péron:
> > > > Hi Michael, Andreas,
> > > >
> > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > > > <andreas.rheinhardt@outlook.com> wrote:
> > > >>
> > > >> Michael Niedermayer:
> > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> > > >>>> Output the producer reference time to a dirty raw output.
> > > >>>>
> > > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > > >>>> ---
> > > >>>>  libavformat/rawenc.c | 122
> > +++++++++++++++++++++++++++++++++++++++++++
> > > >>>>  1 file changed, 122 insertions(+)
> > > >>>
> > > >>> this breaks fate-filter-volume and others
> > > >>> (Segmentation fault)
> > > >>>
> > > >>> I can rerun it with debug symbols and provide peoper gdb output
> > > >>> but i suspect given this has "HACK" in the title you are aware of
> > this
> > > >
> > > > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > > > segfaults", but more to trigger a discussion is it possible to
> > > > properly support an output timestamp in the raw video demux, and if
> > > > yes how to do it :)
> > >
> > > If you need a timestamp for raw video, then use a proper container and
> > > not raw video. In fact, this patch basically creates new formats
> > > different from all the raw formats.
> >
> > Yes I agree, but I do not want to add too much overhead nor
> > computation processing or memory copy to my pipeline just to mux and
> > demux between ffmpeg and my python script.
> >
> > The idea is to have a very light structure to easily pipe it.
> >
> > I'm not familiar with audio/video container but it seems to me that
> > parsing containers are not very light no?
> >
> >
> Containers range from almost no processing like .y4m to complex monsters
> like .mxf

.y4m doesn't contain a timestamp either, and I don't want to use a
complex container :),

>
> This patch is hack and approach/solution it tries is flawed.

100% agree with you that's why I prefix the patch with "HACK:",

Regards,
Clement


>
>
> > Thanks,
> > Clement
> >
> >
> > >
> > > - Andreas
> > >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Andreas,

On Fri, 22 Sept 2023 at 12:01, Andreas Rheinhardt
<andreas.rheinhardt@outlook.com> wrote:
>
> Clément Péron:
> > Hi Andreas,
> >
> > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> > <andreas.rheinhardt@outlook.com> wrote:
> >>
> >> Clément Péron:
> >>> Hi Michael, Andreas,
> >>>
> >>> On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> >>> <andreas.rheinhardt@outlook.com> wrote:
> >>>>
> >>>> Michael Niedermayer:
> >>>>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> >>>>>> Output the producer reference time to a dirty raw output.
> >>>>>>
> >>>>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> >>>>>> ---
> >>>>>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
> >>>>>>  1 file changed, 122 insertions(+)
> >>>>>
> >>>>> this breaks fate-filter-volume and others
> >>>>> (Segmentation fault)
> >>>>>
> >>>>> I can rerun it with debug symbols and provide peoper gdb output
> >>>>> but i suspect given this has "HACK" in the title you are aware of this
> >>>
> >>> The "HACK" tag meaning was not supposed to be: "it's ok if it
> >>> segfaults", but more to trigger a discussion is it possible to
> >>> properly support an output timestamp in the raw video demux, and if
> >>> yes how to do it :)
> >>
> >> If you need a timestamp for raw video, then use a proper container and
> >> not raw video. In fact, this patch basically creates new formats
> >> different from all the raw formats.
> >
> > Yes I agree, but I do not want to add too much overhead nor
> > computation processing or memory copy to my pipeline just to mux and
> > demux between ffmpeg and my python script.
> >
> > The idea is to have a very light structure to easily pipe it.
> >
>
> Our libraries are meant to be used by API users and are designed for
> that. The ffmpeg command line tool is just one user among many and
> adding code to a library to circumvent a limitation of ffmpeg (or
> another user of the libraries) is not appropriate. We would end up with
> a ton of hacks.

Yes I agree and maybe the final solution for this is "keep a fork of
FFMpeg with your patch on your side",

But my idea is could introducing a "raw-format user-defined" would be
acceptable or will it be considered a hack?

like we pass the pix_fmt why not passing a raw_fmt to specify the raw
output format? It will default to only "packet" but a user could add
other metadata if wanted.


>
> > I'm not familiar with audio/video container but it seems to me that
> > parsing containers are not very light no?
>
> For certain formats the overhead of parsing containers can be negative
> when compared to the raw format, because the containers provide a length
> field for the packet whereas one has to search for the packet boundaries
> in case of a truely raw stream. But this is not true for raw video which
> is fixed size.

Yes, good point, in my case I have a fixed size raw video.

Thanks for your comments,
Regards,
Clement

>
> - Andreas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Andreas Rheinhardt]

Clément Péron:
> Hi Andreas,
> 
> On Fri, 22 Sept 2023 at 12:01, Andreas Rheinhardt
> <andreas.rheinhardt@outlook.com> wrote:
>>
>> Clément Péron:
>>> Hi Andreas,
>>>
>>> On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
>>> <andreas.rheinhardt@outlook.com> wrote:
>>>>
>>>> Clément Péron:
>>>>> Hi Michael, Andreas,
>>>>>
>>>>> On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
>>>>> <andreas.rheinhardt@outlook.com> wrote:
>>>>>>
>>>>>> Michael Niedermayer:
>>>>>>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
>>>>>>>> Output the producer reference time to a dirty raw output.
>>>>>>>>
>>>>>>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
>>>>>>>> ---
>>>>>>>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>  1 file changed, 122 insertions(+)
>>>>>>>
>>>>>>> this breaks fate-filter-volume and others
>>>>>>> (Segmentation fault)
>>>>>>>
>>>>>>> I can rerun it with debug symbols and provide peoper gdb output
>>>>>>> but i suspect given this has "HACK" in the title you are aware of this
>>>>>
>>>>> The "HACK" tag meaning was not supposed to be: "it's ok if it
>>>>> segfaults", but more to trigger a discussion is it possible to
>>>>> properly support an output timestamp in the raw video demux, and if
>>>>> yes how to do it :)
>>>>
>>>> If you need a timestamp for raw video, then use a proper container and
>>>> not raw video. In fact, this patch basically creates new formats
>>>> different from all the raw formats.
>>>
>>> Yes I agree, but I do not want to add too much overhead nor
>>> computation processing or memory copy to my pipeline just to mux and
>>> demux between ffmpeg and my python script.
>>>
>>> The idea is to have a very light structure to easily pipe it.
>>>
>>
>> Our libraries are meant to be used by API users and are designed for
>> that. The ffmpeg command line tool is just one user among many and
>> adding code to a library to circumvent a limitation of ffmpeg (or
>> another user of the libraries) is not appropriate. We would end up with
>> a ton of hacks.
> 
> Yes I agree and maybe the final solution for this is "keep a fork of
> FFMpeg with your patch on your side",
> 

Yes, that is the likely outcome.

> But my idea is could introducing a "raw-format user-defined" would be
> acceptable or will it be considered a hack?
> 
> like we pass the pix_fmt why not passing a raw_fmt to specify the raw
> output format? It will default to only "packet" but a user could add
> other metadata if wanted.

This would require us to add hacks and introduce new container formats
instead of you simply using one of the already existing containers.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Paul B Mahol]

On Fri, Sep 22, 2023 at 12:06 PM Clément Péron <peron.clem@gmail.com> wrote:

> Hi Paul
>
> On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda@gmail.com> wrote:
> >
> > On Fri, Sep 22, 2023 at 10:38 AM Clément Péron <peron.clem@gmail.com>
> wrote:
> >
> > > Hi Andreas,
> > >
> > > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> > > <andreas.rheinhardt@outlook.com> wrote:
> > > >
> > > > Clément Péron:
> > > > > Hi Michael, Andreas,
> > > > >
> > > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > >>
> > > > >> Michael Niedermayer:
> > > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> > > > >>>> Output the producer reference time to a dirty raw output.
> > > > >>>>
> > > > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > > > >>>> ---
> > > > >>>>  libavformat/rawenc.c | 122
> > > +++++++++++++++++++++++++++++++++++++++++++
> > > > >>>>  1 file changed, 122 insertions(+)
> > > > >>>
> > > > >>> this breaks fate-filter-volume and others
> > > > >>> (Segmentation fault)
> > > > >>>
> > > > >>> I can rerun it with debug symbols and provide peoper gdb output
> > > > >>> but i suspect given this has "HACK" in the title you are aware of
> > > this
> > > > >
> > > > > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > > > > segfaults", but more to trigger a discussion is it possible to
> > > > > properly support an output timestamp in the raw video demux, and if
> > > > > yes how to do it :)
> > > >
> > > > If you need a timestamp for raw video, then use a proper container
> and
> > > > not raw video. In fact, this patch basically creates new formats
> > > > different from all the raw formats.
> > >
> > > Yes I agree, but I do not want to add too much overhead nor
> > > computation processing or memory copy to my pipeline just to mux and
> > > demux between ffmpeg and my python script.
> > >
> > > The idea is to have a very light structure to easily pipe it.
> > >
> > > I'm not familiar with audio/video container but it seems to me that
> > > parsing containers are not very light no?
> > >
> > >
> > Containers range from almost no processing like .y4m to complex monsters
> > like .mxf
>
> .y4m doesn't contain a timestamp either, and I don't want to use a
> complex container :),
>

I doubt storing clock time in container for each frame is correct approach.
Is this variable frame rate video?

One can always add another, trivial container with just one field having
whatever you want and with optional magic string in header.

Or can try/explore NUT container in FFmpeg.


>
> >
> > This patch is hack and approach/solution it tries is flawed.
>
> 100% agree with you that's why I prefix the patch with "HACK:",
>
> Regards,
> Clement
>
>
> >
> >
> > > Thanks,
> > > Clement
> > >
> > >
> > > >
> > > > - Andreas
> > > >
> > > > _______________________________________________
> > > > ffmpeg-devel mailing list
> > > > ffmpeg-devel@ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Andreas,

On Fri, 22 Sept 2023 at 13:33, Andreas Rheinhardt
<andreas.rheinhardt@outlook.com> wrote:
>
> Clément Péron:
> > Hi Andreas,
> >
> > On Fri, 22 Sept 2023 at 12:01, Andreas Rheinhardt
> > <andreas.rheinhardt@outlook.com> wrote:
> >>
> >> Clément Péron:
> >>> Hi Andreas,
> >>>
> >>> On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> >>> <andreas.rheinhardt@outlook.com> wrote:
> >>>>
> >>>> Clément Péron:
> >>>>> Hi Michael, Andreas,
> >>>>>
> >>>>> On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> >>>>> <andreas.rheinhardt@outlook.com> wrote:
> >>>>>>
> >>>>>> Michael Niedermayer:
> >>>>>>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> >>>>>>>> Output the producer reference time to a dirty raw output.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> >>>>>>>> ---
> >>>>>>>>  libavformat/rawenc.c | 122 +++++++++++++++++++++++++++++++++++++++++++
> >>>>>>>>  1 file changed, 122 insertions(+)
> >>>>>>>
> >>>>>>> this breaks fate-filter-volume and others
> >>>>>>> (Segmentation fault)
> >>>>>>>
> >>>>>>> I can rerun it with debug symbols and provide peoper gdb output
> >>>>>>> but i suspect given this has "HACK" in the title you are aware of this
> >>>>>
> >>>>> The "HACK" tag meaning was not supposed to be: "it's ok if it
> >>>>> segfaults", but more to trigger a discussion is it possible to
> >>>>> properly support an output timestamp in the raw video demux, and if
> >>>>> yes how to do it :)
> >>>>
> >>>> If you need a timestamp for raw video, then use a proper container and
> >>>> not raw video. In fact, this patch basically creates new formats
> >>>> different from all the raw formats.
> >>>
> >>> Yes I agree, but I do not want to add too much overhead nor
> >>> computation processing or memory copy to my pipeline just to mux and
> >>> demux between ffmpeg and my python script.
> >>>
> >>> The idea is to have a very light structure to easily pipe it.
> >>>
> >>
> >> Our libraries are meant to be used by API users and are designed for
> >> that. The ffmpeg command line tool is just one user among many and
> >> adding code to a library to circumvent a limitation of ffmpeg (or
> >> another user of the libraries) is not appropriate. We would end up with
> >> a ton of hacks.
> >
> > Yes I agree and maybe the final solution for this is "keep a fork of
> > FFMpeg with your patch on your side",
> >
>
> Yes, that is the likely outcome.
>
> > But my idea is could introducing a "raw-format user-defined" would be
> > acceptable or will it be considered a hack?
> >
> > like we pass the pix_fmt why not passing a raw_fmt to specify the raw
> > output format? It will default to only "packet" but a user could add
> > other metadata if wanted.
>
> This would require us to add hacks and introduce new container formats
> instead of you simply using one of the already existing containers.

I could propose it, if it's an acceptable solution and could also help
other users.

Regards,

>
> - Andreas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Paul,

On Fri, 22 Sept 2023 at 13:41, Paul B Mahol <onemda@gmail.com> wrote:
>
> On Fri, Sep 22, 2023 at 12:06 PM Clément Péron <peron.clem@gmail.com> wrote:
>
> > Hi Paul
> >
> > On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda@gmail.com> wrote:
> > >
> > > On Fri, Sep 22, 2023 at 10:38 AM Clément Péron <peron.clem@gmail.com>
> > wrote:
> > >
> > > > Hi Andreas,
> > > >
> > > > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > >
> > > > > Clément Péron:
> > > > > > Hi Michael, Andreas,
> > > > > >
> > > > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > > > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > > >>
> > > > > >> Michael Niedermayer:
> > > > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron wrote:
> > > > > >>>> Output the producer reference time to a dirty raw output.
> > > > > >>>>
> > > > > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > > > > >>>> ---
> > > > > >>>>  libavformat/rawenc.c | 122
> > > > +++++++++++++++++++++++++++++++++++++++++++
> > > > > >>>>  1 file changed, 122 insertions(+)
> > > > > >>>
> > > > > >>> this breaks fate-filter-volume and others
> > > > > >>> (Segmentation fault)
> > > > > >>>
> > > > > >>> I can rerun it with debug symbols and provide peoper gdb output
> > > > > >>> but i suspect given this has "HACK" in the title you are aware of
> > > > this
> > > > > >
> > > > > > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > > > > > segfaults", but more to trigger a discussion is it possible to
> > > > > > properly support an output timestamp in the raw video demux, and if
> > > > > > yes how to do it :)
> > > > >
> > > > > If you need a timestamp for raw video, then use a proper container
> > and
> > > > > not raw video. In fact, this patch basically creates new formats
> > > > > different from all the raw formats.
> > > >
> > > > Yes I agree, but I do not want to add too much overhead nor
> > > > computation processing or memory copy to my pipeline just to mux and
> > > > demux between ffmpeg and my python script.
> > > >
> > > > The idea is to have a very light structure to easily pipe it.
> > > >
> > > > I'm not familiar with audio/video container but it seems to me that
> > > > parsing containers are not very light no?
> > > >
> > > >
> > > Containers range from almost no processing like .y4m to complex monsters
> > > like .mxf
> >
> > .y4m doesn't contain a timestamp either, and I don't want to use a
> > complex container :),
> >
>
> I doubt storing clock time in container for each frame is correct approach.
> Is this variable frame rate video?

Why would it not? The python takes as an input a list of frames.
It then runs an inference model on it and could discard some frames if needed.
Taking the absolute timestamp of each frame avoids relying on a
supposed frame rate (based on a free running clock) that is not
guaranteed.
Also the result gives inference results which are properly timestamped
with the correct data acquisition time and I can easily resynchronize
with other sensor acquisition.

>
> One can always add another, trivial container with just one field having
> whatever you want and with optional magic string in header.

Yes, if this is acceptable by FFMpeg community I could propose a
trivial container where the format is user defined.
Else I can keep this dirty patch downstream, but I'm not a big fan.

Regards,

>
> Or can try/explore NUT container in FFmpeg.
>
>
> >
> > >
> > > This patch is hack and approach/solution it tries is flawed.
> >
> > 100% agree with you that's why I prefix the patch with "HACK:",
> >
> > Regards,
> > Clement
> >
> >
> > >
> > >
> > > > Thanks,
> > > > Clement
> > > >
> > > >
> > > > >
> > > > > - Andreas
> > > > >
> > > > > _______________________________________________
> > > > > ffmpeg-devel mailing list
> > > > > ffmpeg-devel@ffmpeg.org
> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > >
> > > > > To unsubscribe, visit link above, or email
> > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > _______________________________________________
> > > > ffmpeg-devel mailing list
> > > > ffmpeg-devel@ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Paul B Mahol]

On Fri, Sep 22, 2023 at 2:39 PM Clément Péron <peron.clem@gmail.com> wrote:

> Hi Paul,
>
> On Fri, 22 Sept 2023 at 13:41, Paul B Mahol <onemda@gmail.com> wrote:
> >
> > On Fri, Sep 22, 2023 at 12:06 PM Clément Péron <peron.clem@gmail.com>
> wrote:
> >
> > > Hi Paul
> > >
> > > On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda@gmail.com> wrote:
> > > >
> > > > On Fri, Sep 22, 2023 at 10:38 AM Clément Péron <peron.clem@gmail.com
> >
> > > wrote:
> > > >
> > > > > Hi Andreas,
> > > > >
> > > > > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> > > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > > >
> > > > > > Clément Péron:
> > > > > > > Hi Michael, Andreas,
> > > > > > >
> > > > > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > > > > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > > > >>
> > > > > > >> Michael Niedermayer:
> > > > > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron
> wrote:
> > > > > > >>>> Output the producer reference time to a dirty raw output.
> > > > > > >>>>
> > > > > > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > > > > > >>>> ---
> > > > > > >>>>  libavformat/rawenc.c | 122
> > > > > +++++++++++++++++++++++++++++++++++++++++++
> > > > > > >>>>  1 file changed, 122 insertions(+)
> > > > > > >>>
> > > > > > >>> this breaks fate-filter-volume and others
> > > > > > >>> (Segmentation fault)
> > > > > > >>>
> > > > > > >>> I can rerun it with debug symbols and provide peoper gdb
> output
> > > > > > >>> but i suspect given this has "HACK" in the title you are
> aware of
> > > > > this
> > > > > > >
> > > > > > > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > > > > > > segfaults", but more to trigger a discussion is it possible to
> > > > > > > properly support an output timestamp in the raw video demux,
> and if
> > > > > > > yes how to do it :)
> > > > > >
> > > > > > If you need a timestamp for raw video, then use a proper
> container
> > > and
> > > > > > not raw video. In fact, this patch basically creates new formats
> > > > > > different from all the raw formats.
> > > > >
> > > > > Yes I agree, but I do not want to add too much overhead nor
> > > > > computation processing or memory copy to my pipeline just to mux
> and
> > > > > demux between ffmpeg and my python script.
> > > > >
> > > > > The idea is to have a very light structure to easily pipe it.
> > > > >
> > > > > I'm not familiar with audio/video container but it seems to me that
> > > > > parsing containers are not very light no?
> > > > >
> > > > >
> > > > Containers range from almost no processing like .y4m to complex
> monsters
> > > > like .mxf
> > >
> > > .y4m doesn't contain a timestamp either, and I don't want to use a
> > > complex container :),
> > >
> >
> > I doubt storing clock time in container for each frame is correct
> approach.
> > Is this variable frame rate video?
>
> Why would it not? The python takes as an input a list of frames.
> It then runs an inference model on it and could discard some frames if
> needed.
> Taking the absolute timestamp of each frame avoids relying on a
> supposed frame rate (based on a free running clock) that is not
> guaranteed.
> Also the result gives inference results which are properly timestamped
> with the correct data acquisition time and I can easily resynchronize
> with other sensor acquisition.
>
> >
> > One can always add another, trivial container with just one field having
> > whatever you want and with optional magic string in header.
>
> Yes, if this is acceptable by FFMpeg community I could propose a
> trivial container where the format is user defined.
> Else I can keep this dirty patch downstream, but I'm not a big fan.
>

FFmpeg usually supports files found in wild, and not inventing own formats
unless there is very good reason for it.


>
> Regards,
>
> >
> > Or can try/explore NUT container in FFmpeg.
> >
> >
> > >
> > > >
> > > > This patch is hack and approach/solution it tries is flawed.
> > >
> > > 100% agree with you that's why I prefix the patch with "HACK:",
> > >
> > > Regards,
> > > Clement
> > >
> > >
> > > >
> > > >
> > > > > Thanks,
> > > > > Clement
> > > > >
> > > > >
> > > > > >
> > > > > > - Andreas
> > > > > >
> > > > > > _______________________________________________
> > > > > > ffmpeg-devel mailing list
> > > > > > ffmpeg-devel@ffmpeg.org
> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > > >
> > > > > > To unsubscribe, visit link above, or email
> > > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > > _______________________________________________
> > > > > ffmpeg-devel mailing list
> > > > > ffmpeg-devel@ffmpeg.org
> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > >
> > > > > To unsubscribe, visit link above, or email
> > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > >
> > > > _______________________________________________
> > > > ffmpeg-devel mailing list
> > > > ffmpeg-devel@ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Clément Péron]

Hi Paul,

On Fri, 22 Sept 2023 at 15:17, Paul B Mahol <onemda@gmail.com> wrote:
>
> On Fri, Sep 22, 2023 at 2:39 PM Clément Péron <peron.clem@gmail.com> wrote:
>
> > Hi Paul,
> >
> > On Fri, 22 Sept 2023 at 13:41, Paul B Mahol <onemda@gmail.com> wrote:
> > >
> > > On Fri, Sep 22, 2023 at 12:06 PM Clément Péron <peron.clem@gmail.com>
> > wrote:
> > >
> > > > Hi Paul
> > > >
> > > > On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda@gmail.com> wrote:
> > > > >
> > > > > On Fri, Sep 22, 2023 at 10:38 AM Clément Péron <peron.clem@gmail.com
> > >
> > > > wrote:
> > > > >
> > > > > > Hi Andreas,
> > > > > >
> > > > > > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
> > > > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > > > >
> > > > > > > Clément Péron:
> > > > > > > > Hi Michael, Andreas,
> > > > > > > >
> > > > > > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
> > > > > > > > <andreas.rheinhardt@outlook.com> wrote:
> > > > > > > >>
> > > > > > > >> Michael Niedermayer:
> > > > > > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron
> > wrote:
> > > > > > > >>>> Output the producer reference time to a dirty raw output.
> > > > > > > >>>>
> > > > > > > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > > > > > > >>>> ---
> > > > > > > >>>>  libavformat/rawenc.c | 122
> > > > > > +++++++++++++++++++++++++++++++++++++++++++
> > > > > > > >>>>  1 file changed, 122 insertions(+)
> > > > > > > >>>
> > > > > > > >>> this breaks fate-filter-volume and others
> > > > > > > >>> (Segmentation fault)
> > > > > > > >>>
> > > > > > > >>> I can rerun it with debug symbols and provide peoper gdb
> > output
> > > > > > > >>> but i suspect given this has "HACK" in the title you are
> > aware of
> > > > > > this
> > > > > > > >
> > > > > > > > The "HACK" tag meaning was not supposed to be: "it's ok if it
> > > > > > > > segfaults", but more to trigger a discussion is it possible to
> > > > > > > > properly support an output timestamp in the raw video demux,
> > and if
> > > > > > > > yes how to do it :)
> > > > > > >
> > > > > > > If you need a timestamp for raw video, then use a proper
> > container
> > > > and
> > > > > > > not raw video. In fact, this patch basically creates new formats
> > > > > > > different from all the raw formats.
> > > > > >
> > > > > > Yes I agree, but I do not want to add too much overhead nor
> > > > > > computation processing or memory copy to my pipeline just to mux
> > and
> > > > > > demux between ffmpeg and my python script.
> > > > > >
> > > > > > The idea is to have a very light structure to easily pipe it.
> > > > > >
> > > > > > I'm not familiar with audio/video container but it seems to me that
> > > > > > parsing containers are not very light no?
> > > > > >
> > > > > >
> > > > > Containers range from almost no processing like .y4m to complex
> > monsters
> > > > > like .mxf
> > > >
> > > > .y4m doesn't contain a timestamp either, and I don't want to use a
> > > > complex container :),
> > > >
> > >
> > > I doubt storing clock time in container for each frame is correct
> > approach.
> > > Is this variable frame rate video?
> >
> > Why would it not? The python takes as an input a list of frames.
> > It then runs an inference model on it and could discard some frames if
> > needed.
> > Taking the absolute timestamp of each frame avoids relying on a
> > supposed frame rate (based on a free running clock) that is not
> > guaranteed.
> > Also the result gives inference results which are properly timestamped
> > with the correct data acquisition time and I can easily resynchronize
> > with other sensor acquisition.
> >
> > >
> > > One can always add another, trivial container with just one field having
> > > whatever you want and with optional magic string in header.
> >
> > Yes, if this is acceptable by FFMpeg community I could propose a
> > trivial container where the format is user defined.
> > Else I can keep this dirty patch downstream, but I'm not a big fan.
> >
>
> FFmpeg usually supports files found in wild, and not inventing own formats
> unless there is very good reason for it.

Indeed, It will be hard to find a file in the wild as it's supposed to
be used when you read from a pipe.

So if I propose a patch to support a new demux where the format is
user defined is there a chance to be accepted?

Thanks,
>
>
> >
> > Regards,
> >
> > >
> > > Or can try/explore NUT container in FFmpeg.
> > >
> > >
> > > >
> > > > >
> > > > > This patch is hack and approach/solution it tries is flawed.
> > > >
> > > > 100% agree with you that's why I prefix the patch with "HACK:",
> > > >
> > > > Regards,
> > > > Clement
> > > >
> > > >
> > > > >
> > > > >
> > > > > > Thanks,
> > > > > > Clement
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > - Andreas
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > ffmpeg-devel mailing list
> > > > > > > ffmpeg-devel@ffmpeg.org
> > > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > > > >
> > > > > > > To unsubscribe, visit link above, or email
> > > > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > > > _______________________________________________
> > > > > > ffmpeg-devel mailing list
> > > > > > ffmpeg-devel@ffmpeg.org
> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > > >
> > > > > > To unsubscribe, visit link above, or email
> > > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > > >
> > > > > _______________________________________________
> > > > > ffmpeg-devel mailing list
> > > > > ffmpeg-devel@ffmpeg.org
> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > >
> > > > > To unsubscribe, visit link above, or email
> > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > _______________________________________________
> > > > ffmpeg-devel mailing list
> > > > ffmpeg-devel@ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Paul B Mahol]

On 9/22/23, Clément Péron <peron.clem@gmail.com> wrote:
> Hi Paul,
>
> On Fri, 22 Sept 2023 at 15:17, Paul B Mahol <onemda@gmail.com> wrote:
>>
>> On Fri, Sep 22, 2023 at 2:39 PM Clément Péron <peron.clem@gmail.com>
>> wrote:
>>
>> > Hi Paul,
>> >
>> > On Fri, 22 Sept 2023 at 13:41, Paul B Mahol <onemda@gmail.com> wrote:
>> > >
>> > > On Fri, Sep 22, 2023 at 12:06 PM Clément Péron <peron.clem@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Paul
>> > > >
>> > > > On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > > On Fri, Sep 22, 2023 at 10:38 AM Clément Péron
>> > > > > <peron.clem@gmail.com
>> > >
>> > > > wrote:
>> > > > >
>> > > > > > Hi Andreas,
>> > > > > >
>> > > > > > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
>> > > > > > <andreas.rheinhardt@outlook.com> wrote:
>> > > > > > >
>> > > > > > > Clément Péron:
>> > > > > > > > Hi Michael, Andreas,
>> > > > > > > >
>> > > > > > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
>> > > > > > > > <andreas.rheinhardt@outlook.com> wrote:
>> > > > > > > >>
>> > > > > > > >> Michael Niedermayer:
>> > > > > > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron
>> > wrote:
>> > > > > > > >>>> Output the producer reference time to a dirty raw output.
>> > > > > > > >>>>
>> > > > > > > >>>> Signed-off-by: Clément Péron <peron.clem@gmail.com>
>> > > > > > > >>>> ---
>> > > > > > > >>>>  libavformat/rawenc.c | 122
>> > > > > > +++++++++++++++++++++++++++++++++++++++++++
>> > > > > > > >>>>  1 file changed, 122 insertions(+)
>> > > > > > > >>>
>> > > > > > > >>> this breaks fate-filter-volume and others
>> > > > > > > >>> (Segmentation fault)
>> > > > > > > >>>
>> > > > > > > >>> I can rerun it with debug symbols and provide peoper gdb
>> > output
>> > > > > > > >>> but i suspect given this has "HACK" in the title you are
>> > aware of
>> > > > > > this
>> > > > > > > >
>> > > > > > > > The "HACK" tag meaning was not supposed to be: "it's ok if
>> > > > > > > > it
>> > > > > > > > segfaults", but more to trigger a discussion is it possible
>> > > > > > > > to
>> > > > > > > > properly support an output timestamp in the raw video demux,
>> > and if
>> > > > > > > > yes how to do it :)
>> > > > > > >
>> > > > > > > If you need a timestamp for raw video, then use a proper
>> > container
>> > > > and
>> > > > > > > not raw video. In fact, this patch basically creates new
>> > > > > > > formats
>> > > > > > > different from all the raw formats.
>> > > > > >
>> > > > > > Yes I agree, but I do not want to add too much overhead nor
>> > > > > > computation processing or memory copy to my pipeline just to mux
>> > and
>> > > > > > demux between ffmpeg and my python script.
>> > > > > >
>> > > > > > The idea is to have a very light structure to easily pipe it.
>> > > > > >
>> > > > > > I'm not familiar with audio/video container but it seems to me
>> > > > > > that
>> > > > > > parsing containers are not very light no?
>> > > > > >
>> > > > > >
>> > > > > Containers range from almost no processing like .y4m to complex
>> > monsters
>> > > > > like .mxf
>> > > >
>> > > > .y4m doesn't contain a timestamp either, and I don't want to use a
>> > > > complex container :),
>> > > >
>> > >
>> > > I doubt storing clock time in container for each frame is correct
>> > approach.
>> > > Is this variable frame rate video?
>> >
>> > Why would it not? The python takes as an input a list of frames.
>> > It then runs an inference model on it and could discard some frames if
>> > needed.
>> > Taking the absolute timestamp of each frame avoids relying on a
>> > supposed frame rate (based on a free running clock) that is not
>> > guaranteed.
>> > Also the result gives inference results which are properly timestamped
>> > with the correct data acquisition time and I can easily resynchronize
>> > with other sensor acquisition.
>> >
>> > >
>> > > One can always add another, trivial container with just one field
>> > > having
>> > > whatever you want and with optional magic string in header.
>> >
>> > Yes, if this is acceptable by FFMpeg community I could propose a
>> > trivial container where the format is user defined.
>> > Else I can keep this dirty patch downstream, but I'm not a big fan.
>> >
>>
>> FFmpeg usually supports files found in wild, and not inventing own formats
>> unless there is very good reason for it.
>
> Indeed, It will be hard to find a file in the wild as it's supposed to
> be used when you read from a pipe.
>
> So if I propose a patch to support a new demux where the format is
> user defined is there a chance to be accepted?

If it passes code review and is not blocked by other means, yes.

>
> Thanks,
>>
>>
>> >
>> > Regards,
>> >
>> > >
>> > > Or can try/explore NUT container in FFmpeg.
>> > >
>> > >
>> > > >
>> > > > >
>> > > > > This patch is hack and approach/solution it tries is flawed.
>> > > >
>> > > > 100% agree with you that's why I prefix the patch with "HACK:",
>> > > >
>> > > > Regards,
>> > > > Clement
>> > > >
>> > > >
>> > > > >
>> > > > >
>> > > > > > Thanks,
>> > > > > > Clement
>> > > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > - Andreas
>> > > > > > >
>> > > > > > > _______________________________________________
>> > > > > > > ffmpeg-devel mailing list
>> > > > > > > ffmpeg-devel@ffmpeg.org
>> > > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > > > > >
>> > > > > > > To unsubscribe, visit link above, or email
>> > > > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> > > > > > _______________________________________________
>> > > > > > ffmpeg-devel mailing list
>> > > > > > ffmpeg-devel@ffmpeg.org
>> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > > > >
>> > > > > > To unsubscribe, visit link above, or email
>> > > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> > > > > >
>> > > > > _______________________________________________
>> > > > > ffmpeg-devel mailing list
>> > > > > ffmpeg-devel@ffmpeg.org
>> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > > >
>> > > > > To unsubscribe, visit link above, or email
>> > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> > > > _______________________________________________
>> > > > ffmpeg-devel mailing list
>> > > > ffmpeg-devel@ffmpeg.org
>> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > >
>> > > > To unsubscribe, visit link above, or email
>> > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> > > >
>> > > _______________________________________________
>> > > ffmpeg-devel mailing list
>> > > ffmpeg-devel@ffmpeg.org
>> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > >
>> > > To unsubscribe, visit link above, or email
>> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> > _______________________________________________
>> > ffmpeg-devel mailing list
>> > ffmpeg-devel@ffmpeg.org
>> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> >
>> > To unsubscribe, visit link above, or email
>> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> >
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 0/3] Propagate PRFT side data

[Clément Péron]

Hi,

I plan to resent this series without the latest patch.

Regarding Patch 1 and 2 do you have any comment?

One thing is that unlike the decode.c which has a common
ff_decode_frame_props_from_pkt() there is no such thing for the encode
part. Or maybe I missed it ?

I noticed that the propagation of this data doesn't work when I enable
the hardware Nvidia encoder.

Does it make sense to introduce a ff_encode_packet_props_from_frame()?

Thanks,


On Thu, 21 Sept 2023 at 17:41, Clément Péron <peron.clem@gmail.com> wrote:
>
> Hi Kieran,
>
> On Thu, 21 Sept 2023 at 15:13, Kieran Kunhya <kierank@obe.tv> wrote:
> >
> >  On Thu, 21 Sept 2023, 13:17 Clément Péron, <peron.clem@gmail.com> wrote:
> >>
> >> 4I have a project where I need to synchronize multiple RTSP cameras with other
> >> network sensors (sync with NTP or PTP).
> >
> >
> > Just be aware the clock of the vast majority of cameras have no relation to NTP or PTP so you will have drift and need to handle that (e.g by dropping or duplicating frames).
>
> Thanks for pointing this out, and yes I consider each of my sensors
> running on a free clock and I recreate a "virtual frame" that is not
> correlated to the FPS of each sensor.
>
> Thanks,
> Clement
>
> >
> > Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC PATCH 0/3] Propagate PRFT side data

[Clément Péron]

Hi,

On Sun, 24 Sept 2023 at 11:12, Clément Péron <peron.clem@gmail.com> wrote:
>
> Hi,
>
> I plan to resent this series without the latest patch.
>
> Regarding Patch 1 and 2 do you have any comment?
>
> One thing is that unlike the decode.c which has a common
> ff_decode_frame_props_from_pkt() there is no such thing for the encode
> part. Or maybe I missed it ?
>
> I noticed that the propagation of this data doesn't work when I enable
> the hardware Nvidia encoder.
>
> Does it make sense to introduce a ff_encode_packet_props_from_frame()?

So I investigate this and understand that the cuvid packet has its own
format and it's not capable of forwarding metadata.

So I'm not sure I'm going in the right direction by forwarding the
PRFT all along FFMpeg.

Does it make sense to have the PTS to be an absolute timestamp?

If I look at the RTPdec it seems that everybody expects the ts to be relative.

https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/rtpdec.c#L669-L694

Thanks,



>
> Thanks,
>
>
> On Thu, 21 Sept 2023 at 17:41, Clément Péron <peron.clem@gmail.com> wrote:
> >
> > Hi Kieran,
> >
> > On Thu, 21 Sept 2023 at 15:13, Kieran Kunhya <kierank@obe.tv> wrote:
> > >
> > >  On Thu, 21 Sept 2023, 13:17 Clément Péron, <peron.clem@gmail.com> wrote:
> > >>
> > >> 4I have a project where I need to synchronize multiple RTSP cameras with other
> > >> network sensors (sync with NTP or PTP).
> > >
> > >
> > > Just be aware the clock of the vast majority of cameras have no relation to NTP or PTP so you will have drift and need to handle that (e.g by dropping or duplicating frames).
> >
> > Thanks for pointing this out, and yes I consider each of my sensors
> > running on a free clock and I recreate a "virtual frame" that is not
> > correlated to the FPS of each sensor.
> >
> > Thanks,
> > Clement
> >
> > >
> > > Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".