From: "Martin Storsjö" <martin@martin.st>
To: ffmpeg-devel@ffmpeg.org
Subject: [FFmpeg-devel] [PATCH 00/13] [RFC] Reduce unnecessary recompilation
Date: Wed, 23 Feb 2022 16:29:38 +0200
Message-ID: <20220223142951.40366-1-martin@martin.st> (raw)
Hi,
When updating the ffmpeg source, one quite often ends up in a situation
where practically all of the codebase (or all of a library) gets rebuilt,
due to updates to headers that are included in most files.
In some cases, full rebuilds are warranted of course, but they could also
be avoided in many cases - e.g. such as if the minor/micro version of
a library has been bumped, or if a new component (codec, demuxer, filter
etc) has been enabled (or removed if reconfiguring with an older source
version). Very few source files are affected by exactly what the full
library version is, or by the full list of enabled components.
To avoid such rebuilds, I've got a proof of concept patchset that
splits headers, so that most source files avoid including the bits that
change often and that shouldn't affect how they are built.
- The version.h headers are split into a separate version_major.h which
contains only the major library version, and accompanying FF_API_*
defines. The main library headers only include version_major.h, and
files that need the exact version (e.g. LIB<name>_VERSION* or
LIB<name>_IDENT) can include version.h explicitly. This is a minor
break of the public API though, as definitions that used to be available
no longer are.
This works mostly fine for most libraries, but there's little point in
splitting libavutil/version.h, because LIBAVUTIL_VERSION_INT is used
in every source file that defines an AVClass.
By splitting version.h, and update to the minor/micro version numbers
of all libraries except avutil now would require recompiling 30
files instead of 1653 before.
(This change also should lower the barrier to and reduce the risk of
forgetting to bump the version numbers, which one otherwise often
postpones while working on a patch, as it forces rebuilds.)
- config.h is split into a separate config_components.h that includes the
list of enabled/disabled components (corresponding to $ALL_COMPONENTS
in configure). One could consider splitting up config.h even more, but
that probably gives less benefit compared to the amount of churn.
Surprisingly, a nontrivial number of source files do depend on the
state of specific encoders/decoders/components, so quite a few files
do end up requiring including config_components.h. (Also this change
can possibly break compilation of source files that require external
dependencies that I haven't tested.)
In practice, this reduces the number of rebuilt source files from
1979 to 193, if there's a change to the list of enabled components
but not to the rest of config.h.
What do you think - is it worth the slight churn to avoid pointless
rebuilds?
Martin Storsjö (13):
libavutil: Remove leftover uses of version.h
libavcodec: Remove unnecessary includes of version.h
libavformat: Remove unnecessary includes of version.h
libavdevice: Remove unnecessary includes of version.h
libavcodec: Split version.h
libavformat: Split version.h
libavdevice: Split version.h
libpostproc: Split version.h
libswresample: Split version.h
libswscale: Split version.h
libavfilter: Split version.h
doc: Add an entry to APIchanges about no longer implicitly including
version.h
configure: Use a separate config_components.h header for
$ALL_COMPONENTS
configure | 17 +++++++--
doc/APIchanges | 6 ++++
fftools/cmdutils.c | 7 ++++
fftools/ffmpeg.c | 1 +
fftools/ffplay.c | 1 +
fftools/ffprobe.c | 7 ++++
libavcodec/8svx.c | 1 +
libavcodec/Makefile | 1 +
libavcodec/a64multienc.c | 1 +
libavcodec/aac_ac3_parser.c | 1 +
libavcodec/aacenc.c | 1 +
libavcodec/aactab.c | 1 +
libavcodec/aarch64/h264cmc_neon.S | 1 +
libavcodec/ac3_parser.c | 1 +
libavcodec/ac3dec.c | 1 +
libavcodec/ac3enc.c | 1 +
libavcodec/ac3enc_template.c | 1 +
libavcodec/adpcm.c | 1 +
libavcodec/adpcmenc.c | 1 +
libavcodec/allcodecs.c | 1 +
libavcodec/aptxdec.c | 1 +
libavcodec/aptxenc.c | 1 +
libavcodec/arm/flacdsp_init_arm.c | 1 +
libavcodec/arm/h264cmc_neon.S | 1 +
libavcodec/assdec.c | 1 +
libavcodec/assenc.c | 1 +
libavcodec/asvdec.c | 1 +
libavcodec/asvenc.c | 1 +
libavcodec/audiotoolboxdec.c | 1 +
libavcodec/av1dec.c | 1 +
libavcodec/avcodec.c | 1 +
libavcodec/avcodec.h | 2 +-
libavcodec/binkaudio.c | 1 +
libavcodec/bintext.c | 1 +
libavcodec/bsf.c | 1 +
libavcodec/codec.h | 2 +-
libavcodec/cyuv.c | 1 +
libavcodec/dpxenc.c | 1 +
libavcodec/flashsv.c | 1 +
libavcodec/g726.c | 1 +
libavcodec/gsmdec.c | 1 +
libavcodec/h263dec.c | 1 +
libavcodec/h264_slice.c | 1 +
libavcodec/h264dec.c | 2 ++
libavcodec/hevcdec.c | 1 +
libavcodec/huffyuvdec.c | 1 +
libavcodec/huffyuvenc.c | 1 +
libavcodec/idctdsp.c | 1 +
libavcodec/iff.c | 1 +
libavcodec/imc.c | 1 +
libavcodec/ituh263dec.c | 1 +
libavcodec/ituh263enc.c | 1 +
libavcodec/j2kenc.c | 1 +
libavcodec/lcldec.c | 1 +
libavcodec/libgsmdec.c | 1 +
libavcodec/libgsmenc.c | 1 +
libavcodec/libopencore-amr.c | 1 +
libavcodec/libvorbisenc.c | 1 +
libavcodec/libvpx.c | 1 +
libavcodec/libvpxdec.c | 1 +
libavcodec/libvpxenc.c | 1 +
libavcodec/libx264.c | 1 +
libavcodec/me_cmp.c | 1 +
libavcodec/mediacodec.c | 1 -
libavcodec/mediacodec_wrapper.c | 1 -
libavcodec/metasound_data.c | 1 +
libavcodec/mjpegdec.c | 1 +
libavcodec/mjpegenc.c | 1 +
libavcodec/mjpegenc_common.c | 1 +
libavcodec/mlpdec.c | 1 +
libavcodec/mlpenc.c | 1 +
libavcodec/mpeg12dec.c | 1 +
libavcodec/mpeg12enc.c | 1 +
libavcodec/mpeg4videodec.c | 1 +
libavcodec/mpeg4videoenc.c | 1 +
libavcodec/mpegaudiodec_fixed.c | 1 +
libavcodec/mpegaudiodec_float.c | 1 +
libavcodec/mpegvideo.c | 1 +
libavcodec/mpegvideo_enc.c | 1 +
libavcodec/mpegvideo_motion.c | 1 +
libavcodec/msmpeg4.c | 1 +
libavcodec/msmpeg4dec.c | 1 +
libavcodec/mvcdec.c | 1 +
libavcodec/options.c | 1 +
libavcodec/options_table.h | 2 +-
libavcodec/opus_pvq.c | 1 +
libavcodec/packet.h | 2 +-
libavcodec/pcm.c | 1 +
libavcodec/pngdec.c | 1 +
libavcodec/pnmdec.c | 1 +
libavcodec/pnmenc.c | 1 +
libavcodec/proresdec2.c | 1 +
libavcodec/pthread_frame.c | 2 +-
libavcodec/qpeldsp.c | 1 +
libavcodec/r210dec.c | 1 +
libavcodec/r210enc.c | 1 +
libavcodec/rv34_parser.c | 1 +
libavcodec/sonic.c | 1 +
libavcodec/sp5xdec.c | 1 +
libavcodec/speedhq.c | 1 +
libavcodec/speedhqenc.c | 1 +
libavcodec/srtdec.c | 1 +
libavcodec/srtenc.c | 1 +
libavcodec/textdec.c | 1 +
libavcodec/tiffenc.c | 1 +
libavcodec/v408dec.c | 1 +
libavcodec/v408enc.c | 1 +
libavcodec/vaapi_encode_h264.c | 1 +
libavcodec/vc1dec.c | 1 +
libavcodec/vc1dsp.c | 1 +
libavcodec/version.h | 28 ++-------------
libavcodec/version_major.h | 55 ++++++++++++++++++++++++++++++
libavcodec/videotoolbox.c | 1 +
libavcodec/vorbis_parser.c | 1 +
libavcodec/vp3.c | 1 +
libavcodec/vp56dsp.c | 1 +
libavcodec/vp8.c | 1 +
libavcodec/vp8dsp.c | 1 +
libavcodec/vp9.c | 1 +
libavcodec/wmadec.c | 1 +
libavcodec/wmaenc.c | 1 +
libavcodec/x86/blockdsp_init.c | 1 -
libavcodec/x86/flacdsp_init.c | 1 +
libavcodec/x86/hpeldsp_init.c | 1 +
libavcodec/xvmc.h | 1 -
libavdevice/Makefile | 1 +
libavdevice/android_camera.c | 2 --
libavdevice/avdevice.c | 1 +
libavdevice/avdevice.h | 2 +-
libavdevice/pulse_audio_dec.c | 1 +
libavdevice/pulse_audio_enc.c | 1 +
libavdevice/version.h | 10 ++----
libavdevice/version_major.h | 37 ++++++++++++++++++++
libavfilter/Makefile | 1 +
libavfilter/aeval.c | 1 +
libavfilter/af_afade.c | 1 +
libavfilter/af_agate.c | 1 +
libavfilter/af_biquads.c | 1 +
libavfilter/af_sidechaincompress.c | 1 +
libavfilter/avf_showspectrum.c | 1 +
libavfilter/avf_showwaves.c | 1 +
libavfilter/avfilter.c | 1 +
libavfilter/avfilter.h | 2 +-
libavfilter/f_bench.c | 1 +
libavfilter/f_cue.c | 1 +
libavfilter/f_drawgraph.c | 1 +
libavfilter/f_graphmonitor.c | 1 +
libavfilter/f_interleave.c | 1 +
libavfilter/f_latency.c | 1 +
libavfilter/f_loop.c | 1 +
libavfilter/f_metadata.c | 1 +
libavfilter/f_perms.c | 1 +
libavfilter/f_realtime.c | 1 +
libavfilter/f_reverse.c | 1 +
libavfilter/f_segment.c | 1 +
libavfilter/f_select.c | 1 +
libavfilter/f_sendcmd.c | 1 +
libavfilter/f_sidedata.c | 1 +
libavfilter/internal.h | 1 -
libavfilter/lut3d.h | 1 +
libavfilter/setpts.c | 1 +
libavfilter/settb.c | 1 +
libavfilter/src_movie.c | 1 +
libavfilter/trim.c | 1 +
libavfilter/version.h | 13 ++-----
libavfilter/version_major.h | 42 +++++++++++++++++++++++
libavfilter/vf_aspect.c | 1 +
libavfilter/vf_blend.c | 1 +
libavfilter/vf_colorconstancy.c | 1 +
libavfilter/vf_colorkey.c | 1 +
libavfilter/vf_convolution.c | 1 +
libavfilter/vf_convolve.c | 1 +
libavfilter/vf_drawbox.c | 1 +
libavfilter/vf_extractplanes.c | 1 +
libavfilter/vf_format.c | 1 +
libavfilter/vf_histogram.c | 1 +
libavfilter/vf_identity.c | 1 +
libavfilter/vf_lut.c | 1 +
libavfilter/vf_lut2.c | 1 +
libavfilter/vf_mix.c | 1 +
libavfilter/vf_neighbor.c | 1 +
libavfilter/vf_premultiply.c | 1 +
libavfilter/vf_setparams.c | 1 +
libavfilter/vf_stack.c | 1 +
libavfilter/vf_xmedian.c | 1 +
libavfilter/vsrc_testsrc.c | 1 +
libavformat/Makefile | 1 +
libavformat/ac3dec.c | 1 +
libavformat/alp.c | 1 +
libavformat/amr.c | 1 +
libavformat/apm.c | 1 +
libavformat/aptxdec.c | 1 +
libavformat/argo_asf.c | 1 +
libavformat/argo_cvg.c | 1 +
libavformat/asfenc.c | 1 +
libavformat/au.c | 1 +
libavformat/av1dec.c | 1 +
libavformat/avformat.h | 2 +-
libavformat/avidec.c | 1 +
libavformat/avienc.c | 1 +
libavformat/avio.h | 2 +-
libavformat/bintext.c | 1 +
libavformat/bit.c | 1 +
libavformat/codec2.c | 1 +
libavformat/concat.c | 1 +
libavformat/dashenc.c | 1 +
libavformat/demux.c | 1 +
libavformat/file.c | 1 +
libavformat/flacenc.c | 1 +
libavformat/format.c | 1 +
libavformat/framehash.c | 1 +
libavformat/g726.c | 1 +
libavformat/gopher.c | 1 +
libavformat/hashenc.c | 1 +
libavformat/hls.c | 1 +
libavformat/hlsenc.c | 1 +
libavformat/http.c | 1 +
libavformat/ilbc.c | 1 +
libavformat/img2dec.c | 1 +
libavformat/img2enc.c | 1 +
libavformat/kvag.c | 1 +
libavformat/matroskadec.c | 1 +
libavformat/matroskaenc.c | 2 ++
libavformat/mj2kdec.c | 1 +
libavformat/mlpdec.c | 1 +
libavformat/mmf.c | 2 ++
libavformat/mov.c | 1 +
libavformat/movenc.c | 3 ++
libavformat/mpeg.c | 1 +
libavformat/mpegenc.c | 1 +
libavformat/mpegts.c | 1 +
libavformat/mux.c | 1 +
libavformat/mxfenc.c | 1 +
libavformat/network.c | 1 +
libavformat/nutenc.c | 1 +
libavformat/oggenc.c | 2 ++
libavformat/pcmdec.c | 1 +
libavformat/pcmenc.c | 1 +
libavformat/rawdec.c | 1 +
libavformat/rawenc.c | 1 +
libavformat/rawvideodec.c | 1 +
libavformat/riff.c | 1 +
libavformat/rtmpproto.c | 2 ++
libavformat/rtsp.c | 2 ++
libavformat/rtspdec.c | 2 ++
libavformat/sdp.c | 1 +
libavformat/segment.c | 1 +
libavformat/swfenc.c | 1 +
libavformat/url.h | 1 -
libavformat/utils.c | 1 +
libavformat/version.h | 23 ++-----------
libavformat/version_major.h | 52 ++++++++++++++++++++++++++++
libavformat/voc.c | 1 +
libavformat/wavdec.c | 1 +
libavformat/wavenc.c | 1 +
libavutil/common.h | 1 -
libavutil/internal.h | 1 -
libpostproc/Makefile | 1 +
libpostproc/postprocess.c | 1 +
libpostproc/postprocess.h | 2 +-
libpostproc/version.h | 3 +-
libpostproc/version_major.h | 31 +++++++++++++++++
libswresample/Makefile | 1 +
libswresample/swresample.c | 1 +
libswresample/swresample.h | 2 +-
libswresample/version.h | 3 +-
libswresample/version_major.h | 31 +++++++++++++++++
libswscale/Makefile | 1 +
libswscale/swscale.h | 2 +-
libswscale/swscale_internal.h | 1 -
libswscale/utils.c | 1 +
libswscale/version.h | 9 ++---
libswscale/version_major.h | 35 +++++++++++++++++++
273 files changed, 586 insertions(+), 100 deletions(-)
create mode 100644 libavcodec/version_major.h
create mode 100644 libavdevice/version_major.h
create mode 100644 libavfilter/version_major.h
create mode 100644 libavformat/version_major.h
create mode 100644 libpostproc/version_major.h
create mode 100644 libswresample/version_major.h
create mode 100644 libswscale/version_major.h
--
2.32.0 (Apple Git-132)
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
next reply other threads:[~2022-02-23 14:30 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-23 14:29 Martin Storsjö [this message]
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 01/13] libavutil: Remove leftover uses of version.h Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 02/13] libavcodec: Remove unnecessary includes " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 03/13] libavformat: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 04/13] libavdevice: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 05/13] libavcodec: Split version.h Martin Storsjö
2022-02-25 14:34 ` Michael Niedermayer
2022-02-25 14:38 ` Martin Storsjö
2022-02-25 20:23 ` Andreas Rheinhardt
2022-02-25 20:44 ` Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 06/13] libavformat: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 07/13] libavdevice: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 08/13] libpostproc: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 09/13] libswresample: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 10/13] libswscale: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 11/13] libavfilter: " Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 12/13] doc: Add an entry to APIchanges about no longer implicitly including version.h Martin Storsjö
2022-02-23 14:29 ` [FFmpeg-devel] [PATCH 13/13] configure: Use a separate config_components.h header for $ALL_COMPONENTS Martin Storsjö
2022-02-24 21:22 ` Michael Niedermayer
2022-02-24 21:35 ` Martin Storsjö
2022-03-11 12:17 ` [FFmpeg-devel] [PATCH 00/13] [RFC] Reduce unnecessary recompilation Martin Storsjö
2022-03-13 21:04 ` Paul B Mahol
2022-03-14 14:23 ` Michael Niedermayer
2022-03-16 12:16 ` Martin Storsjö
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220223142951.40366-1-martin@martin.st \
--to=martin@martin.st \
--cc=ffmpeg-devel@ffmpeg.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git