From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 601B340516 for ; Sat, 26 Mar 2022 14:52:40 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4200A68B1E1; Sat, 26 Mar 2022 16:52:37 +0200 (EET) Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2EF3768A4CD for ; Sat, 26 Mar 2022 16:52:31 +0200 (EET) Received: by mail-pg1-f169.google.com with SMTP id t4so5710272pgc.1 for ; Sat, 26 Mar 2022 07:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=4sBncwzMUQH/JcPu86Oh1oV4u4GQbfvBYitDn2MQZY4=; b=CjPDLtVe3trObWPWVNoohuhnQ1oAouKmyINJDxKNGUc3w1sUH2RQOs5bjhQK6L6iE6 D+Za6SXzP1dpwhWwm/VukSpn/nekGldIA8pcuVedDuxa66DSm4wdJanRR3tWtgvBkGE1 yjf5UGCt65hWQqxVHX0JtMIPjloOyVNyQm0n9wXfvkt210RqoU9s/CfO9lw5yTWDHMvE tQO37uCaetEKvsQ0amiitEoFwcQE6Jh1a29vYwBz96gJz1x9tX3TCb+pKqK9geiqld75 TWNxWOllqv+g8Jk2c6djbzvFIDNVm6BhsSf1BwNpmwi9NIqwGRlmwlF3hoRlDKvJJ0P/ EHkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4sBncwzMUQH/JcPu86Oh1oV4u4GQbfvBYitDn2MQZY4=; b=v5kEMb32teQ13fxTmGfQF1KwVVGlkIGw3dkk/zQ9TiFr4OPC1cMEvBFf4GiJonakXJ HOPI0luKSz2uqMz4pfc4zZ66lkvrTLeCwuxBDWovMUjHeuDUxQnLSLSCAzgcGod0yAwv f9vPPLwnncShrHL1RcISNHlJDgCDcLYBIZZ2xSJd2khju0/r6xiHVgy/WVTwPC8YU+3E ddSLixox5CE1cC0k+wuxhSWpIz6Ra2DnR+eIkvmFdKEnLrvw/juyR9GeVABTzI+LtuMi 71lqQhy1zVsj0EgNK8rJ37SOvNd5m4BrFJLTuwDI2kmr89tsMbaUofQsXaNYwHfLkWR2 uyhw== X-Gm-Message-State: AOAM533Wmfa0e5/Idjq4yyrCdFCvIwt/Vl6cXRmtEiyTYkHNxv4czYw7 hjT9F4FulI0DVCx0hj5S5Wt6mBQVXExJiMIMBx9tbmCZ0fW8hQ== X-Google-Smtp-Source: ABdhPJzwgVhzsAkLpxNpWm+BWkLRKrbK84mrsgMCGfIYUcoeRkdj2saehji/fwtXj5+ZVrb7Vh1JPHDT0vY4LlFxqDM= X-Received: by 2002:aa7:81c6:0:b0:4fa:70a2:2063 with SMTP id c6-20020aa781c6000000b004fa70a22063mr14986675pfn.71.1648306348526; Sat, 26 Mar 2022 07:52:28 -0700 (PDT) MIME-Version: 1.0 From: =?UTF-8?Q?R=C5=ABdolfs_Bundulis?= Date: Sat, 26 Mar 2022 16:52:17 +0200 Message-ID: To: ffmpeg-devel@ffmpeg.org X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [FFmpeg-devel] Suggestions on best way to add capture timestamp functionality over RTSP X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Hi, I was looking for some advice on how to best add capture timestamp support to ffmpeg (namely RTP muxer) so that it would be accepted as a patch. I'll try to explain the rationale and what I see as an mvp and would like to get some feedback. Currently, a lot of live streaming stacks use rtmp onFI messages to signal capture timestamp mapping to pts, but with stuff like OPUS / AV1 / ... becoming more and more popular, using rtmp becomes harder since none of these codecs are supported in rtmp. Some vendors add extensions to rtmp to be able to ingest these codecs, but rtsp already has muxers for most of them, but it sadly lacks a good mechanism to send capture timestamps that would help to estimate a/v desync and end to end latency. So, there is this rtp header extension ( https://github.com/webrtc/webrtc-org/blob/gh-pages/experiments/rtp-hdrext/abs-capture-time/index.md) and I just found that ffmpeg now supports AV_PKT_DATA_S12M_TIMECODE side data, that would allow passing the capture time allthrough from the capture device to the muxer (which was the main obstacle why I did not evaluate this earlier). So, what I'd like to do and get that accepted is: 1) Add a feature to the rtpenc.c, controlled by a flag which would be off by default, which generates the abs-capture-time header if AV_PKT_DATA_S12M_TIMECODE is present. The spec says that one should not send the header on each packet which makes sense so I assume I need and on/off flag and a frequency parameter, or a single parameter which just does nothing if defaulted to -1 ? In this case, what would be a good unit of frequency of the header (since potentially this could be used with any kind of media) - milliseconds? Or maybe leave the frequency up to the caputre device? 2) Add a way to actually embed this sidedata into a captured stream. As far as I can see decklink is the only device which currently generates this kind of side data, but unfortunately only from the smpte coming from the source. I have access to some decklink cards, but I certainly am not able to set up a signal chain with embedded smpte. So I see a couple of options here: a) add a "wallclock" parameter to the decklink "timecode_format" option, which would generate the side data value from wallclock b) add a filter which just adds AV_PKT_DATA_S12M_TIMECODE data to any packet going through (possibly, with a configurable frequency). The latter of course suffers from the fact that all latencies up to the filter are lost, but then again it could be then combined with any type of input device. It would be great to get some feedback on this and understand what is a good patch functionality wise, that would actually get accepted. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".