From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 154074974D for ; Thu, 18 Apr 2024 23:19:39 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4250768D2EA; Fri, 19 Apr 2024 02:19:37 +0300 (EEST) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E1C8E680122 for ; Fri, 19 Apr 2024 02:19:30 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 3C04820004 for ; Thu, 18 Apr 2024 23:19:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1713482370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Or6slpx6nt6y+/xJNo8K1Xcu/4Jtf+DP0t+OjP2M38E=; b=gz8vRFvetvlMI212o5x7WdG6n3Zbnof2Jl/gElBVvtCyexafpTGeUKJWAWLybtbc/x1+O6 9NUrTeuiD2OBFmlIO3tUR+Xiy9NrPe3VxP1h/4FHyiW0Hh2p0GFHAgONOkRtlvQr9VK9b9 tlMta1fSSqauJcqYgldzVo+SA7MACrj1m4YSZpC70VFPkYO81xUvHxc4fH2VhpTg5azDpp 2redwQfIcOw05ye2FVwk8inzwBbjk3Ipyh0LoVuEmYFOV4iaCQFkWSeVSbiOBmCGn4Y0rN 8UnceYIX+E/efZ0aZttAYvh8iHDyNE8RRodhniRVlJ09IEx25sqIj+BwyrsD5w== Date: Fri, 19 Apr 2024 01:19:29 +0200 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20240418231929.GU6420@pb2> References: <20240417135832.GJ6420@pb2> <20240418160207.GB45500@haasn.xyz> <20240418205351.GS6420@pb2> <5f454591-fe4c-4fc1-98b0-d3d21b207b12@gmail.com> MIME-Version: 1.0 In-Reply-To: <5f454591-fe4c-4fc1-98b0-d3d21b207b12@gmail.com> X-GND-Sasl: michael@niedermayer.cc Subject: Re: [FFmpeg-devel] [RFC] 5 year plan & Inovation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============3372631089175711426==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --===============3372631089175711426== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qIWqZABUZ74qI4tu" Content-Disposition: inline --qIWqZABUZ74qI4tu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 18, 2024 at 06:13:40PM -0300, James Almer wrote: > On 4/18/2024 5:53 PM, Michael Niedermayer wrote: > > On Thu, Apr 18, 2024 at 04:02:07PM +0200, Niklas Haas wrote: > > > On Wed, 17 Apr 2024 15:58:32 +0200 Michael Niedermayer wrote: > > > > Hi all > > > >=20 > > > > The pace of inovation in FFmpeg has been slowing down. > > > > Most work is concentarted nowadays on code refactoring, and adding > > > > support for new codecs and formats. > > > >=20 > > > > Should we > > > > * make a list of longer term goals > > > > * vote on them > > > > * and then together work towards implementing them > > > > ? > > > >=20 > > > > (The idea here is to increase the success of larger efforts > > > > than adding codecs and refactoring code) > > > > It would then also not be possible for individuals to object > > > > to a previously agreed goal. > > > > And it would add ideas for which we can try to get funding/grants f= or > > > >=20 > > > > (larger scale changes need consensus first that we as a whole want > > > > them before we would be able to ask for funding/grants for them) > > > >=20 > > > > Some ideas and why they would help FFmpeg: > > > >=20 > > > > * Switch to a plugin architecture > > > > (Increase the number of developers willing to contribute and r= educe > > > > friction as the team and community grows) > > >=20 > > > This would almost surely hurt productivity > >=20 > > i dont agree, ill elaborae below > >=20 > >=20 > > > as it will require exposing, > > > versioning, documenting and maintaining a vast number of internal APIs > >=20 > > yes to some extend that is needed. It can be more or less depending > > on how things are implemented > >=20 > >=20 > > > which we are currently at the liberty of modifying freely. > >=20 > > we are modifying these APIs for decades. That modification of APIs > > their documentation and all code using them costs time. >=20 > The AVCodec hooks being internal allowed us to add autoinserted bsfs and = to > painlessly rewrite the decouple I/O callbacks to work as a pure pull based > interface. Were said internals public, that wouldn't have been possible. A decoder API needs packet in, frame out. AVPacket and AVFrame are public. (and a bunch of key-value data like width / height / timebase / pixelformat= / aspect / ... teh struct for that is also public since a long time) I dont see the problem. you have the decoder API facing the user, that causes no problems, i dont agree that having a decoder API (or encoder, muxer, demuxer, ...) facing a plugin would be a bigger problem than the APIs we already have public sure there are details, sure there are things that need one to think about and sleep over and all that but these are from a high level point of view simple and also the same interfaces to what we already have public >=20 > >=20 > > More so we have issues with patch-management. And i claim this is > > not the mailing list but a lack of time and in some cases a lack of > > interrest in many areas. > >=20 > > A plugin system moves this patch-management to people who actually > > care, that is the authors of the codecs and (de)muxers. > >=20 > > Our productivity as is, is not good, many patches are ignored. >=20 > A lot of patches fall through the cracks rather than being ignored. > There are people that send patchsets unthreaded (Because of misconfigured > git send-email i suppose), and it makes browsing your mailbox hard. well i can say that i dont review many patches anymore because i just dont = have the time, its not that i cant keep track of what i wanted to review. either i make a note in a TODO list or i keep the patch marked as NEW in my mail user agent. trac in some sense or patchwork are just more public TODO lists that can be shared between people so if one doesnt do it another developer sees it and can do it. >=20 > > The people caring about these patches are their Authors and yet they > > are powerless as they must sometimes wait many months for reviews > > Besides that, developers are leaving for various reasons and they > > are forced to setup full forks not being able to maintain their > > code in any other way. > > IMO A plugin system would improve productivity as everyone could work > > on their own terms. >=20 > You say the ML is not the problem, but it sort of is. An MR based > development would greatly improve this problem. People historically where very opposed to merge requests But, having a public git repo (that people already have) asking for it to be merged. You get a merge commit and someone will probably feel offended by that. (thats not what you meant i guess) but i would 100% support doing git merge requests. (there are good arguments from people much smarter than me why merging is better than rebasing) >=20 > > No week or month long delays and weekly pinging patches > > No risk about patches being rejected or ignored > > No need to read every other discussion on the ML. One can just > > simply work on their own plugin looking just at the API documentation >=20 > I don't personally have a strong opinion one way or another on this subje= ct > at this moment, but any such approach would need to be carefully thought = and > designed, to prevent all the potential problems people have expressed so > far. of course this would require carefull design as any public API would. [...] > > >=20 > > > > * AI / neural network filters and codecs > > > > The future seems to be AI based. Future Filters and Codecs wil= l use > > > > neural networks. FFmpeg can be at the forefront, developing th= ese > > >=20 > > > We already have TensorFlow support, no? (vf_sr etc.) What more is > > > needed? > >=20 > > more of that AND more convenience > >=20 > > lets pick a comparission > > to run fate > > you write "make fate" > > if you want to do it with the samples its > > make fate-rsync ; make fate > >=20 > > if you want to use vf_sr, its reading the docs, looking at some scripts= reading their docs > > and i presume selecting a training set ? creating a model ? .... >=20 > By no means we could ever ship a custom AI model for the sake of a "git > fetch, compile, and run" scenario. This was already a problem when > tensorflow was first committed. So this can't be avoided. I am not sure i understand you, but training a model on lets say the fate samples or some public domain images. Why would we not be able to ship that in a similar way to the fate samples ? or why not ship a standarized script to build such a model from user data ? (i mean by standarized, the same for every filter, like ffmpeg is the same for every format not link to different papers and random off site scripts) >=20 > >=20 > > how many people do that ? >=20 > Every external dependency has its documented requirements... vf_sr doesnt have a example one can copy and paste that would work on a fresh checkout of ffmpeg. That alone is a fail IMHO thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Concerning the gods, I have no means of knowing whether they exist or not or of what sort they may be, because of the obscurity of the subject, and the brevity of human life -- Protagoras --qIWqZABUZ74qI4tu Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEIAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCZiGqfgAKCRBhHseHBAsP qyXrAJ979O7IQ94HAdN//MTNtVBnSLMOVgCglNjCPEbz+OGCobPhCDo8tirBReg= =8/M3 -----END PGP SIGNATURE----- --qIWqZABUZ74qI4tu-- --===============3372631089175711426== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============3372631089175711426==--