From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 36731435C9 for ; Fri, 17 Jun 2022 16:22:00 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 67DB668B92A; Fri, 17 Jun 2022 19:21:58 +0300 (EEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 41E4368B875 for ; Fri, 17 Jun 2022 19:21:52 +0300 (EEST) Received: from localhost (213-47-68-29.cable.dynamic.surfer.at [213.47.68.29]) (Authenticated sender: michael@niedermayer.cc) by mail.gandi.net (Postfix) with ESMTPSA id 250FB240007 for ; Fri, 17 Jun 2022 16:21:50 +0000 (UTC) Date: Fri, 17 Jun 2022 18:21:50 +0200 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20220617162150.GW396728@pb2> References: <20220316140045.369016-1-philip-dylan.gleonec@savoirfairelinux.com> <20220316140045.369016-3-philip-dylan.gleonec@savoirfairelinux.com> MIME-Version: 1.0 In-Reply-To: <20220316140045.369016-3-philip-dylan.gleonec@savoirfairelinux.com> Subject: Re: [FFmpeg-devel] [PATCH 2/2] avcodec/libopusdec: Enable FEC/PLC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============0769244376839209672==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --===============0769244376839209672== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="glUAVOaDCVqE0op2" Content-Disposition: inline --glUAVOaDCVqE0op2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 16, 2022 at 03:00:45PM +0100, Philip-Dylan Gleonec wrote: > Adds FEC/PLC support to libopus. The lost packets are detected as a > discontinuity in the audio stream. When a discontinuity is used, this > patch tries to decode the FEC data. If FEC data is present in the > packet, it is decoded, otherwise audio is re-created through PLC. >=20 > This patch is based on Steinar H. Gunderson contribution, and corrects > the pts computation: all pts are expressed in samples instead of time. > This patch also adds an option "decode_fec" which enables or disables > FEC decoding. This option is disabled by default to keep consistent > behaviour with former versions. >=20 > A number of checks are made to ensure compatibility with different > containers. Indeed, video containers seem to have a pts expressed in ms > while it is expressed in samples for audio containers. It also manages > the cases where pkt->duration is 0, in some RTP streams. This patch > ignores data it can not reconstruct, i.e. packets received twice and > packets with a length that is not a multiple of 2.5ms. >=20 > Signed-off-by: Philip-Dylan Gleonec > Co-developed-by: Steinar H. Gunderson > --- > libavcodec/libopusdec.c | 105 +++++++++++++++++++++++++++++++++++----- > 1 file changed, 94 insertions(+), 11 deletions(-) >=20 > diff --git a/libavcodec/libopusdec.c b/libavcodec/libopusdec.c > index 86ef715205..66134300d2 100644 > --- a/libavcodec/libopusdec.c > +++ b/libavcodec/libopusdec.c > @@ -43,10 +43,15 @@ struct libopus_context { > #ifdef OPUS_SET_PHASE_INVERSION_DISABLED_REQUEST > int apply_phase_inv; > #endif > + int decode_fec; > + int64_t expected_next_pts; > }; > =20 > #define OPUS_HEAD_SIZE 19 > =20 > +// Sample rate is constant as libopus always output at 48kHz > +const AVRational opus_timebase =3D { 1, 48000 }; static const=20 > + > static av_cold int libopus_decode_init(AVCodecContext *avc) > { > struct libopus_context *opus =3D avc->priv_data; > @@ -134,6 +139,8 @@ static av_cold int libopus_decode_init(AVCodecContext= *avc) > /* Decoder delay (in samples) at 48kHz */ > avc->delay =3D avc->internal->skip_samples =3D opus->pre_skip; > =20 > + opus->expected_next_pts =3D AV_NOPTS_VALUE; > + > return 0; > } > =20 > @@ -155,25 +162,100 @@ static int libopus_decode(AVCodecContext *avc, voi= d *data, > { > struct libopus_context *opus =3D avc->priv_data; > AVFrame *frame =3D data; > - int ret, nb_samples; > + uint8_t *outptr; > + int ret, nb_samples =3D 0, nb_lost_samples =3D 0, nb_samples_left; > + > + // If FEC is enabled, calculate number of lost samples > + if (opus->decode_fec && > + opus->expected_next_pts !=3D AV_NOPTS_VALUE && > + pkt->pts !=3D AV_NOPTS_VALUE && > + pkt->pts !=3D opus->expected_next_pts) { > + // Cap at recovering 120 ms of lost audio. > + nb_lost_samples =3D pkt->pts - opus->expected_next_pts; > + nb_lost_samples =3D FFMIN(nb_lost_samples, MAX_FRAME_SIZE); > + // pts is expressed in ms for some containers (e.g. mkv) > + // FEC only works for SILK frames (> 10ms) > + // Detect if nb_lost_samples is in ms, and convert in samples if= it is > + if (nb_lost_samples > 0) { > + if (avc->pkt_timebase.den !=3D 48000) { > + nb_lost_samples =3D av_rescale_q(nb_lost_samples, avc->p= kt_timebase, opus_timebase); > + } > + // For FEC/PLC, frame_size has to be to have a multiple of 2= =2E5 ms > + if (nb_lost_samples % (int)(2.5 / 1000 * opus_timebase.den))= { > + nb_lost_samples -=3D nb_lost_samples % (int)(2.5 / 1000 = * opus_timebase.den); something like this nb_lost_samples % (5LL * opus_timebase.den / 2000) would avoid the float also if noone reacts to your patch keep pinging it thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Into a blind darkness they enter who follow after the Ignorance, they as if into a greater darkness enter who devote themselves to the Knowledge alone. -- Isha Upanishad --glUAVOaDCVqE0op2 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEIAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCYqyqGgAKCRBhHseHBAsP q13wAJ9b1hp7a2JlyVocbE57l5EBOSM3ugCfQXd3jvu5QsWAeFXPxKYNJNntSFg= =ADeU -----END PGP SIGNATURE----- --glUAVOaDCVqE0op2-- --===============0769244376839209672== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============0769244376839209672==--