From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 5429444C4B for ; Mon, 14 Nov 2022 11:10:35 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3600168BDB2; Mon, 14 Nov 2022 13:10:33 +0200 (EET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11olkn2010.outbound.protection.outlook.com [40.92.20.10]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AF8C16883DD for ; Mon, 14 Nov 2022 13:10:26 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FE/mlSGd22VeZthjeYPaePs13YVmpW4XM2puG7I1+WRRnsFEAELDDRgWMAW6uObfJxO+dULFK51AO6gYEW6t36T4hrWodtQSoWlQK1yIXSGZL8YWDckGyZhnDMEiCp959eapHS+y8s9j9gwjuacQrCtpOy9R9/qzONSfSNocZsjetxcqAHlzuZsA7jMD17dSvdDojRO07DavrrMJ/8k2qJhizrjY+36DBIvbTZXqFnw9cNuhXfRAsWCBZDurhC93M8pTH1aPGHDi5B9K60dYdsdQpRbGjk1K2RaLLBffwA0EvC8a4vIC0X2wFUkc2a4KYo6BBkZA8WH53qeCnslbvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OlSdrrH2gdwpA4YWqgXxGJA+DlO8plw56g3/0IqSuFU=; b=YFJDbXMfFTtsrvlT0zkzknKGqvleAX6EHYBjLP192cDu2orX/4CwHqgRIZ0AhDxQS0rhZ4gVumuw8riEoQm+ympVpFzLLMRuYJtmZZnveJb/D/U0grpT76QDtyMSGZcKDzAtQHdnQK/tUJOW4p464XbAy5GyHiiJfa8QhWsjv6CMi2uD+hFkjDOohul27wNWkyhaAVYXP/ND3M3XfpM3q6hkSpg5ccYiByjIwsowdd9lU2yOoGC1SJMaoURa+dH4yNmrs4/ZE3nc1T7xK2jCeUt6QBOBE0Oh82SA8cK+nlQGfL991Yd1SSQ/MQuvILbRUJ/fMZoDKMGQiXu82Z6kpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OlSdrrH2gdwpA4YWqgXxGJA+DlO8plw56g3/0IqSuFU=; b=I9rAuorpgcImpUaFNbns9txl2NUPmkKsIxSehoZP7Q+/UzZLl4nBSCSrtWVURKf9B7UrUynVdm89DB03WSI5k19WU11HFAU+GhR5Fv8qKMJx7oi/DxI8uF1PYu9P6YrxWaGo6Pj6e+wI8HBbNn0ObeQZi6VFi9i3+IfR6Na3vNF3obIGaNbc1QTw3ba2aqZvnU0+4M2xiBDg1grrAaiMdJWFXXXBQcLQjMWrUWETYKpigCLV75VFl+hzDKuXJmFQmJlnZ5rNWxP/wd/XrI7zof1VVjUkjNEGfbCK8ZXjlVunAJcnCQJx2slzPBsTYWdiNsqh38KOaGh4RNql0uZXGg== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by MN0P223MB0533.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:3c7::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.17; Mon, 14 Nov 2022 11:10:23 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::45ea:be80:31df:38c0]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::45ea:be80:31df:38c0%9]) with mapi id 15.20.5813.017; Mon, 14 Nov 2022 11:10:23 +0000 From: Soft Works To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 Thread-Index: AQHYiHoQ/s8zpruhyEis6n+SHo1FCK2qkhHggA7hQ4CAABDRAIABrZcAgAAnFYCABrBT6IAGmxHkgAAVbHCAdmb/kA== Date: Mon, 14 Nov 2022 11:10:23 +0000 Message-ID: References: =?utf-8?q??= =?utf-8?q?=3C=3CDM8P223MB0365D91C08ED6BA00BCDE342BA649=40DM8P223MB0365=2ENA?= =?utf-8?q?MP223=2EPROD=2EOUTLOOK=2ECOM=3E=3CCAPYw7P7T=3DQd6mujTqsQbK=5FkBHk?= =?utf-8?q?3mDW8MfoVv=3DAf=5FpXmxcitvqg=40mail=2Egmail=2Ecom=3E=3Cac8af460-b?= =?utf-8?q?f9d-4c7f-aaa4-9c9a664c6037=40beta=2Efastmail=2Ecom=3E=3E?= <166117073498.25402.3952806733544526391@lain.khirnov.net> <08928116-7643-4c02-afa5-b1b38d388a88@www.fastmail.com> =?utf-8?q?=3CDM8P22?= =?utf-8?q?3MB0365254229D2A9B9871D0913BA739=40DM8P223MB0365=2ENAMP223=2EPROD?= =?utf-8?q?=2EOUTLOOK=2ECOM=3E?= <166154682732.3205.14632132553698015937@lain.khirnov.net> =?utf-8?q?=3CDM8P?= =?utf-8?q?223MB0365F56CC9655A58D833C2ADBA759=40DM8P223MB0365=2ENAMP223=2EPR?= =?utf-8?q?OD=2EOUTLOOK=2ECOM=3E?= <166190998678.25402.5822807528009867156@lain.khirnov.net> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [a3f2I12oNZcai0YiBSF3/ApUG+Nuuldz] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM8P223MB0365:EE_|MN0P223MB0533:EE_ x-ms-office365-filtering-correlation-id: 339d09c1-f589-46c3-e8b9-08dac630d362 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7iwktKUuy1xVvXDojQ8jS/FYrOxIgGVd/IXkCOuFUcQXfZAtrSbROK80kMYKio3tTQRBHrZpBADq+z5j1R9MvGxzrsDbkOkEP2/CvmRoN+AjsTnU4KuhAmH8SLXt07H7BLfFF6OxdoGFA9GQs+xNjx6wPu1g0uPFsZECy633rXgU6WKy6jCzW9vQI8chL9Ptgr/5548euekZmQ7DZ1y+FoFxqN5ZQoOo8HbmUArfwzXwN2vg6YBV+KVbPG4OcGOrDBiytpz+2GfJxVgrn7f3qNRhNvXqBZrCsKSA+C0I7S0G4thqsuYUbpcqu2hzaUZS1sSIUquvlDLt7y6KBBh+ZYsLtiWJr9Tr1uB41ZJPeRMo67bmIJSTjS/7+nQBm580n1h3TjMls1DIFdXjCp0G9HQnKlC5I7LFdWS4WO20Z6sshxoI4/W0ZIZh9YPyKP4Nr267zvrSjQghfByRE4+j/QpeMFSBQV5p04qSSv93RiqY5+LU9Ig1d2srRBZW7InqMMURu6YbZ7NUIr1YaDNXirQfjQaxyMb7BIUosms1VKKtiYVkHolkx9mayGS/B/URLfUqFZaKX/GJQqtrrkhvwe5PQcyRhNoAZti101/WomdgI1y3PVg19TLyikRjq9jdK7Hb7Xt3n+ZDgVY7twrD8GIBWRTL6K7OCvnGh38HCsCE7B3sw+wiukT9DvyPC6KS x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?VUExeThMbHlsamhBSFQ2ZVRzT3ZZS2FuV2pScHBaN2tLa3FrWVY1cnZubVdF?= =?utf-8?B?RURwalVvYmFNeExTYmlUdE9UclNzcUxzeTBoanRWV1c3aFdkaW9uNFNQMDJh?= =?utf-8?B?a014aHdyZHVyWHhoaXgyeGhvTFRMdVZJV3dmSThuZEN1d2xrL3ltL1pEN1dS?= =?utf-8?B?ZG14Z2xIQWNYV0x2NUI2aFRTSDB2Sit0Y1Q1c2FFTW5zMktCblBWK3hIYkhk?= =?utf-8?B?LzRmUzFzdWg2TUxDRUh2Y3VpNVhRSE9JZENaMjY3dVRuRU1sTlpLM2hTSjRD?= =?utf-8?B?b3lkSTE5WTFNTXlQVS9ubDZRcG9DN1FCUExtOUc4TnRQQlM4OThIVmMrLzZN?= =?utf-8?B?MytGNVFPRnFKV0VXQStmZ0sxL1NOK1JETkRvNDU0dFhvOGwvZ0o5dG10NnF5?= =?utf-8?B?UFdCQ3pxanpWSUIrYU11QlAvRklDT1RSOXk4UlRKdVQ0a2ZNNXRkMkFPa2JF?= =?utf-8?B?ZDVnMzR1cCtEVk01a1FyRng3d0JEN1VpNzdYcFo3c2trSDdOWVJ6L3N5SXVz?= =?utf-8?B?UC9OOElzZUs4YUlFMjFoL1VUMkJFakFwRUZ1aExXcGV1NG44bGRhYTRXYzNp?= =?utf-8?B?T0oreXZhdmhEYlBzb0tNNGNMUHJ0ZzB6RnhFMVlqUWxuNE1uYzlCRHVyVXZB?= =?utf-8?B?dmg5V0F4Y3RLSTAwRGNFMWJ2Ykx3WUhDZ3dJU1Ivdm9BS2lHcGpUdHNObk8x?= =?utf-8?B?OVlWUExIVm1hbzM5MTBpRWk3MklUYTRIUlNrNVlKQ3MwUkdCUVQ5ZFp5cU9w?= =?utf-8?B?YUQzK2xzZmN5RWY1T3IvU05tdENrSWtwYTZNUGZxeTMrVHZyQ1RYM2NTNWN2?= =?utf-8?B?MkNoL1RVZEx3WjlWR01xYk1xTUlOVkFFc2E1aVZGem9iNmdzdkZGdlkwT1M3?= =?utf-8?B?ZWpkVnhzMnlmT3lwSndEdkdsWDI0Q2R2bVhwK2F2V0dmQlIvN3JsenZQMnpN?= =?utf-8?B?S1FqYmJ2NnNrUllPRWhOQjRMTFBaK2w4K3RRSS9aSnZoeU1HZ1VzZ25vRnJG?= =?utf-8?B?M1VDQU1wM25iT0FRMWg5aW96RXc1MVQyM0hUTmxoSjJ6S2RhVHowZWpNS051?= =?utf-8?B?dVUvZWtYN2FGaVZnQW9VRzJKcUp0cWtmSllSVk1zVy9mbDZzeDNFMzdOb28y?= =?utf-8?B?ZnVpT3l0M21tYjZLaUZqTGhzckxVRlYxbm9uWWRzL2pQOGdGazdpeGFSWGQw?= =?utf-8?B?bjlIRkg1M3ZzMjV5cmVkNFRiT3FuaTdUTnc3ampEcncrZWdWYUFhQlJJVmto?= =?utf-8?B?eUNzMFZ0TE1hcHMxZWYxL25sR2dlUVRrWUpQRkJaQWtlNXZ3bUtsL3ppdW9V?= =?utf-8?B?Ymg3ZlpSUW01dzR0OFhvdnprOHF4RkRjdjRZd3BXbnpEZHFpMDU5MlY2K1gw?= =?utf-8?B?ZFljanpiSDh3UktJYmJlVFlsNG05UkZOLzg3K1NSblYycXYzcXArUHZQYi9s?= =?utf-8?B?VzhlR3B2bVcxcEh2MU5uU3Y3b2lCWjVTMUVlYzVtYU5rYmVtSmN4eVBLY2FN?= =?utf-8?B?Y0l2Z01OdDRscnR4akpXNzU4L2Q3c0RnVGlNMHZyc0wrYkJUUjVUK0doT1pj?= =?utf-8?B?V3lndjhSeG5HMGRMUEVUUjZ2UGw4bHlnVlhvODF4azh0cm5uRVo1T1IxMjAr?= =?utf-8?B?MVQwZ2oyR3lHb0crRy8rakVrOUxQMFFTQUk2UXU5MTlDQUNBUEt6WkpnUnVz?= =?utf-8?B?cHlUZ1ZzL0xjZ2pDN0FUQ1crZTJwWlpOT2Iya0Nac1NzanljdWtwWVpBPT0=?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 339d09c1-f589-46c3-e8b9-08dac630d362 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Nov 2022 11:10:23.2926 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0P223MB0533 Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Wednesday, August 31, 2022 6:02 AM > To: FFmpeg development discussions and patches devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > > -----Original Message----- > > From: ffmpeg-devel On Behalf Of > > Anton Khirnov > > Sent: Wednesday, August 31, 2022 3:40 AM > > To: FFmpeg development discussions and patches > devel@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > Quoting Soft Works (2022-08-27 00:48:01) > > > 2. "There's no reason why this cannot be handled using the buffer > > > and data fields" > > > > > > I had explained the reasons and in conversation on IRC, Lynne was > > > no longer insisting on this AFAIR. > > > > I did not see this explanation, can you restate it here? > You had asked me to restate the explanation. I did that (below) but you never responded. Thanks, softworkz > Sure. Let's look at the AVSubtitleArea struct first: > > > typedef struct AVSubtitleArea { > > enum AVSubtitleType type; > int flags; > > int x; ///< top left corner of area. > int y; ///< top left corner of area. > int w; ///< width of area. > int h; ///< height of area. > int nb_colors; ///< number of colors in bitmap palette (@ref > pal). > > /** > * Buffers and line sizes for the bitmap of this subtitle. > * > * @{ > */ > AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; > int linesize[AV_NUM_BUFFER_POINTERS]; > /** > * @} > */ > > uint32_t pal[256]; ///< RGBA palette for the bitmap. > > char *text; ///< 0-terminated plain UTF-8 text > char *ass; ///< 0-terminated ASS/SSA compatible event > line. > > } AVSubtitleArea; > > > > These structs are stored in AVFrame like this: > > > /** > * Number of items in the @ref subtitle_areas array. > */ > unsigned num_subtitle_areas; > > /** > * Array of subtitle areas, may be empty. > */ > AVSubtitleArea **subtitle_areas; > > > > The question was "why this cannot be handled using the buffer > and data fields?" - there are multiple reasons: > > 1. Area Count > > In ASS subtitles it's not uncommon that animations are defined > through subtitle events (regular ass events). This can go as > far as that dust particles are flying around on the screen and > each particle has its own subtitle event/line which defines > how it flies from x to y and how fast, etc. > Not yet, but in a future update, the ass decoder should be > improved in a way that it doesn't emit an individual AVFrame > for each event line but bundles subsequent events together > when these would have the same start time and duration. > As a result, we can have AVFrames with dozens or even a hundred > AVSubtitleArea items in extreme cases. > > The buffer and data fields are an array of fixed size (currently > 8). Increasing to 16 or 32 would not help: there will still be > cases where this does not suffice. > > 2. What exactly should be stored in frame->buf[]? > > Should we store the AVSubtitleArea structs as AVBuffer there? > > Or should we only store data in those buffers? If yes, which > data? The subtitle bitmap probably. How about the subtitle > text - maybe and area has even both. And should we also > store the palette in some frame->buf[n]? > If yes, how to keep track of which data is stored in which > buffer? > Further, there's a documented convention that requires that > non-zero buffers are contiguous, i.e. there must not be > an empty buffer pointer between two other a non-empty buffer > pointers. This would require to re-arrange the buffers to > close any gap when some subtitle area data is removed, zeroed > out or has been converted to another format. > Closing such gap also require to update any mapping information > ("which buffer is related to which area?") > > That's a lot of tedious work and every API user (or codec/filter > developer) would need to do it in the right way. > > > > 3. AVFrame Code Logic Mistmatch > > A subtitle frame can have 0 to N areas, which means that a > frame without any area is still valid. Opposed to that, existing > (code all over the place in ffmpeg) is treating an AVFrame > as invalid when the first data buffer is empty, > i.e. frame->buf[0] != NULL > > To accommodate to this situation, subtitle frames are always > creating a 1-byte buffer for buf[0]. > > When we would want subtitle data to be stored in those buffers, > every developer would need to be aware about the requirement > to have that dummy buffer in the first array element. When something > is to be stored, the dummy buffer would need to be freed first > before storing the actual data. > And when the last buffer gets deleted, API users would need to > know to create a new dummy buffer. > > > That fixed size array might be a good fit for image data, but > it's not for subtitle data. > > > The approach in my patchset is simple, intuitive and everybody > can easily work with it without needing any special knowledge: > > > unsigned num_subtitle_areas; > AVSubtitleArea **subtitle_areas; > > IMHO, this is the most suitable pattern for this situation. > > > > > If you claim the other points were addressed I will look at the > > patches > > again. > > Oh cool, that would be great! > > Thanks, > softworkz > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".