* [FFmpeg-devel] [PATCH v7 0/3] ffbuild/commonmak: Fix rebuild check with implicit rule chains @ 2025-06-23 17:27 ffmpegagent 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 1/3] fftools/resources: Fix double-build by disabling .d file generation softworkz ` (2 more replies) 0 siblings, 3 replies; 4+ messages in thread From: ffmpegagent @ 2025-06-23 17:27 UTC (permalink / raw) To: ffmpeg-devel; +Cc: softworkz V2 == * Fix MSVC build (use the universal command pattern) V3 == * Skip dependency generation by clearing CC_DEPS instead (as suggested by Ramiro - thanks!) V4 == * Always keep .ptx files (as suggested by Timo - thanks) Tested all scenarios: * .ptx.c and .ptx.gz still get deleted (as intermediates) * repeated make shows "up-to-date" * removing a .ptx file does not cause a rebuild (it's still an intermediate, but an "intermediate to keep") * but changing a .ptx does (in case of dev/debugging) * changed .cu files always rebuild of course V5 == * First patch remains unchanged * Added second patch to clean up and consolidate the rules around compression V6 == * Rebased * Confirmed that it also resolves MSVC-CLang compilation (as reported by Kasper Michalow - thanks!) V7 == * As the log line about intermediate file deletion ("RM ....") didn't find much love, this version uses the workaround via the .SECONDARY special make target to prevent intermediate file deletion . softworkz (3): fftools/resources: Fix double-build by disabling .d file generation ffbuild/commonmak: Consolidate pattern rules for compression fftools/resources: Update .gitignore ffbuild/common.mak | 49 ++++++++++++++---------------------- fftools/Makefile | 1 + fftools/resources/.gitignore | 5 ++-- fftools/resources/Makefile | 12 ++++++--- 4 files changed, 31 insertions(+), 36 deletions(-) base-commit: e6298e0759430f64e9bd9298775de92597be8a50 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-80%2Fsoftworkz%2Fsubmit_commonmak-v7 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-80/softworkz/submit_commonmak-v7 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/80 Range-diff vs v6: 1: a8da0e9d17 ! 1: 391217966f ffbuild/commonmak: Fix rebuild check with implicit rule chains @@ Metadata Author: softworkz <softworkz@hotmail.com> ## Commit message ## - ffbuild/commonmak: Fix rebuild check with implicit rule chains - - When there's a chain of implicit rules, make treats files generated - inside that chain as intermediate files. Those intermediate files are - removed after completion of make. When make is run again, it normally - determines the need for a rebuild by comparing the timestamps of the - original source file and the final output of the chain and if still - up-to-date, it doesn't rebuild, even when the intermediate files are - not present. That makes sense of course - why would it delete them - otherwise, it would end up in builds being never up-to-date. - But this original by-the-book logic appeared to be broken and has been - worked around by adding all intermediate files to the .SECONDARY - special target, which required extra logic and issues with make clean. - - What broke the up-to-date checking is the dependency file generation. - For the .c files generated by BIN2C the compile target created a .d - file which indicated that the .ptx.o file has a dependency on the - ptx.c file. And that dependency broke the normal make behavior with - intermediate files. - - This patch compiles the BIN2C generated .c files without generating - .d files. In turn the files do not longer need to be added to the - .SECONDARY target in common.mak. - - When interested in those files for debugging, a line can be added to - the corresponding Makefile like - - .SECONDARY: %.ptx.c %.ptx.gz + fftools/resources: Fix double-build by disabling .d file generation Signed-off-by: softworkz <softworkz@hotmail.com> ## ffbuild/common.mak ## -@@ ffbuild/common.mak: else - $(BIN2C) $(patsubst $(SRC_PATH)/%,$(SRC_LINK)/%,$<) $@ $(subst .,_,$(basename $(notdir $@))) - endif - -+%.ptx.o: CCDEP = -+%.ptx.o: CC_DEPFLAGS = -+ -+ - # 1) Preprocess CSS to a minified version - %.css.min: TAG = SED - %.css.min: %.css -@@ ffbuild/common.mak: else # NO COMPRESSION - $(BIN2C) $< $@ $(subst .,_,$(basename $(notdir $@))) - endif - -+%.html.o %.css.o: CCDEP = -+%.html.o %.css.o: CC_DEPFLAGS = -+ -+ - clean:: - $(RM) $(BIN2CEXE) $(CLEANSUFFIXES:%=ffbuild/%) - @@ ffbuild/common.mak: SKIPHEADERS += $(ARCH_HEADERS:%=$(ARCH)/%) $(SKIPHEADERS-) SKIPHEADERS := $(SKIPHEADERS:%=$(SUBDIR)%) HOBJS = $(filter-out $(SKIPHEADERS:.h=.h.o),$(ALLHEADERS:.h=.h.o)) @@ ffbuild/common.mak: SKIPHEADERS += $(ARCH_HEADERS:%=$(ARCH)/%) $(SKIPHEADERS-) $(HOBJS): CCFLAGS += $(CFLAGS_HEADERS) checkheaders: $(HOBJS) -.SECONDARY: $(HOBJS:.o=.c) $(PTXOBJS:.o=.c) $(PTXOBJS:.o=.gz) $(PTXOBJS:.o=) $(RESOURCEOBJS:.o=.c) $(RESOURCEOBJS:%.css.o=%.css.min) $(RESOURCEOBJS:%.css.o=%.css.min.gz) $(RESOURCEOBJS:%.html.o=%.html.gz) $(RESOURCEOBJS:.o=) -+.SECONDARY: $(HOBJS:.o=.c) $(PTXOBJS:.o=) - +- ++.SECONDARY: $(HOBJS:.o=.c) $(PTXOBJS:.o=.c) $(PTXOBJS:.o=.gz) $(PTXOBJS:.o=) alltools: $(TOOLS) + $(HOSTOBJS): %.o: %.c +@@ ffbuild/common.mak: $(TOOLOBJS): | tools + + OUTDIRS := $(OUTDIRS) $(dir $(OBJS) $(HOBJS) $(HOSTOBJS) $(SHLIBOBJS) $(STLIBOBJS) $(TESTOBJS)) + +-CLEANSUFFIXES = *.d *.gcda *.gcno *.h.c *.ho *.map *.o *.objs *.pc *.ptx *.ptx.gz *.ptx.c *.ver *.version *.html.gz *.html.c *.css.gz *.css.c *$(DEFAULT_X86ASMD).asm *~ *.ilk *.pdb ++CLEANSUFFIXES = *.d *.gcda *.gcno *.h.c *.ho *.map *.o *.objs *.pc *.ptx *.ptx.gz *.ptx.c *.ver *.version *.html.gz *.html.c *.css.min.gz *.css.min *.css.c *$(DEFAULT_X86ASMD).asm *~ *.ilk *.pdb + LIBSUFFIXES = *.a *.lib *.so *.so.* *.dylib *.dll *.def *.dll.a + + define RULES + + ## fftools/Makefile ## +@@ fftools/Makefile: OBJS-ffmpeg += \ + fftools/textformat/tw_buffer.o \ + fftools/textformat/tw_stdout.o \ + $(OBJS-resman) \ ++ $(RESOBJS) \ + + OBJS-ffprobe += \ + fftools/textformat/avtextformat.o \ + + ## fftools/resources/Makefile ## +@@ fftools/resources/Makefile: clean:: + vpath %.html $(SRC_PATH) + vpath %.css $(SRC_PATH) + +-# Uncomment to prevent deletion during build +-#.PRECIOUS: %.css.c %.css.min %.css.gz %.css.min.gz %.html.gz %.html.c +- + OBJS-resman += \ + fftools/resources/resman.o \ ++ ++ ++RESOBJS += \ + fftools/resources/graph.html.o \ + fftools/resources/graph.css.o \ ++ ++ ++$(RESOBJS): CCDEP = ++$(RESOBJS): CC_DEPFLAGS = ++ ++.SECONDARY: $(RESOBJS:.o=.gz) $(RESOBJS:.o=.c) $(RESOBJS:%.css.o=%.css.min) $(RESOBJS:%.css.o=%.css.min.gz) $(RESOBJS:%.html.o=%.html.gz) $(RESOBJS:.o=) 2: 1781c5fc16 ! 2: 17b0af91ec ffbuild/commonmak: Consolidate pattern rules for compression @@ ffbuild/common.mak: $(BIN2CEXE): ffbuild/bin2c_host.o + $(RUN_BIN2C) endif - %.ptx.o: CCDEP = - %.ptx.o: CC_DEPFLAGS = - - -# 1) Preprocess CSS to a minified version -%.css.min: TAG = SED %.css.min: %.css @@ ffbuild/common.mak: $(BIN2CEXE): ffbuild/bin2c_host.o + $(RUN_BIN2C) endif - %.html.o %.css.o: CCDEP = + clean:: -: ---------- > 3: dde90b288f fftools/resources: Update .gitignore -- ffmpeg-codebot _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* [FFmpeg-devel] [PATCH v7 1/3] fftools/resources: Fix double-build by disabling .d file generation 2025-06-23 17:27 [FFmpeg-devel] [PATCH v7 0/3] ffbuild/commonmak: Fix rebuild check with implicit rule chains ffmpegagent @ 2025-06-23 17:27 ` softworkz 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 2/3] ffbuild/commonmak: Consolidate pattern rules for compression softworkz 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 3/3] fftools/resources: Update .gitignore softworkz 2 siblings, 0 replies; 4+ messages in thread From: softworkz @ 2025-06-23 17:27 UTC (permalink / raw) To: ffmpeg-devel; +Cc: softworkz From: softworkz <softworkz@hotmail.com> Signed-off-by: softworkz <softworkz@hotmail.com> --- ffbuild/common.mak | 6 ++---- fftools/Makefile | 1 + fftools/resources/Makefile | 12 +++++++++--- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/ffbuild/common.mak b/ffbuild/common.mak index ddf48923ea..81e8a46d0c 100644 --- a/ffbuild/common.mak +++ b/ffbuild/common.mak @@ -229,11 +229,9 @@ SKIPHEADERS += $(ARCH_HEADERS:%=$(ARCH)/%) $(SKIPHEADERS-) SKIPHEADERS := $(SKIPHEADERS:%=$(SUBDIR)%) HOBJS = $(filter-out $(SKIPHEADERS:.h=.h.o),$(ALLHEADERS:.h=.h.o)) PTXOBJS = $(filter %.ptx.o,$(OBJS)) -RESOURCEOBJS = $(filter %.css.o %.html.o,$(OBJS)) $(HOBJS): CCFLAGS += $(CFLAGS_HEADERS) checkheaders: $(HOBJS) -.SECONDARY: $(HOBJS:.o=.c) $(PTXOBJS:.o=.c) $(PTXOBJS:.o=.gz) $(PTXOBJS:.o=) $(RESOURCEOBJS:.o=.c) $(RESOURCEOBJS:%.css.o=%.css.min) $(RESOURCEOBJS:%.css.o=%.css.min.gz) $(RESOURCEOBJS:%.html.o=%.html.gz) $(RESOURCEOBJS:.o=) - +.SECONDARY: $(HOBJS:.o=.c) $(PTXOBJS:.o=.c) $(PTXOBJS:.o=.gz) $(PTXOBJS:.o=) alltools: $(TOOLS) $(HOSTOBJS): %.o: %.c @@ -252,7 +250,7 @@ $(TOOLOBJS): | tools OUTDIRS := $(OUTDIRS) $(dir $(OBJS) $(HOBJS) $(HOSTOBJS) $(SHLIBOBJS) $(STLIBOBJS) $(TESTOBJS)) -CLEANSUFFIXES = *.d *.gcda *.gcno *.h.c *.ho *.map *.o *.objs *.pc *.ptx *.ptx.gz *.ptx.c *.ver *.version *.html.gz *.html.c *.css.gz *.css.c *$(DEFAULT_X86ASMD).asm *~ *.ilk *.pdb +CLEANSUFFIXES = *.d *.gcda *.gcno *.h.c *.ho *.map *.o *.objs *.pc *.ptx *.ptx.gz *.ptx.c *.ver *.version *.html.gz *.html.c *.css.min.gz *.css.min *.css.c *$(DEFAULT_X86ASMD).asm *~ *.ilk *.pdb LIBSUFFIXES = *.a *.lib *.so *.so.* *.dylib *.dll *.def *.dll.a define RULES diff --git a/fftools/Makefile b/fftools/Makefile index b3c08ae5a0..bdb44fc5ce 100644 --- a/fftools/Makefile +++ b/fftools/Makefile @@ -36,6 +36,7 @@ OBJS-ffmpeg += \ fftools/textformat/tw_buffer.o \ fftools/textformat/tw_stdout.o \ $(OBJS-resman) \ + $(RESOBJS) \ OBJS-ffprobe += \ fftools/textformat/avtextformat.o \ diff --git a/fftools/resources/Makefile b/fftools/resources/Makefile index 8579a52678..3c936484d1 100644 --- a/fftools/resources/Makefile +++ b/fftools/resources/Makefile @@ -4,10 +4,16 @@ clean:: vpath %.html $(SRC_PATH) vpath %.css $(SRC_PATH) -# Uncomment to prevent deletion during build -#.PRECIOUS: %.css.c %.css.min %.css.gz %.css.min.gz %.html.gz %.html.c - OBJS-resman += \ fftools/resources/resman.o \ + + +RESOBJS += \ fftools/resources/graph.html.o \ fftools/resources/graph.css.o \ + + +$(RESOBJS): CCDEP = +$(RESOBJS): CC_DEPFLAGS = + +.SECONDARY: $(RESOBJS:.o=.gz) $(RESOBJS:.o=.c) $(RESOBJS:%.css.o=%.css.min) $(RESOBJS:%.css.o=%.css.min.gz) $(RESOBJS:%.html.o=%.html.gz) $(RESOBJS:.o=) -- ffmpeg-codebot _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* [FFmpeg-devel] [PATCH v7 2/3] ffbuild/commonmak: Consolidate pattern rules for compression 2025-06-23 17:27 [FFmpeg-devel] [PATCH v7 0/3] ffbuild/commonmak: Fix rebuild check with implicit rule chains ffmpegagent 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 1/3] fftools/resources: Fix double-build by disabling .d file generation softworkz @ 2025-06-23 17:27 ` softworkz 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 3/3] fftools/resources: Update .gitignore softworkz 2 siblings, 0 replies; 4+ messages in thread From: softworkz @ 2025-06-23 17:27 UTC (permalink / raw) To: ffmpeg-devel; +Cc: softworkz From: softworkz <softworkz@hotmail.com> This commit simplifies and consolidates all the rules around ptx and resource file compression. Signed-off-by: softworkz <softworkz@hotmail.com> --- ffbuild/common.mak | 43 +++++++++++++++++-------------------------- 1 file changed, 17 insertions(+), 26 deletions(-) diff --git a/ffbuild/common.mak b/ffbuild/common.mak index 81e8a46d0c..0a60d01623 100644 --- a/ffbuild/common.mak +++ b/ffbuild/common.mak @@ -115,6 +115,12 @@ COMPILE_LASX = $(call COMPILE,CC,LASXFLAGS) $(BIN2CEXE): ffbuild/bin2c_host.o $(HOSTLD) $(HOSTLDFLAGS) $(HOSTLD_O) $^ $(HOSTEXTRALIBS) +RUN_BIN2C = $(BIN2C) $(patsubst $(SRC_PATH)/%,$(SRC_LINK)/%,$<) $@ $(subst .,_,$(basename $(notdir $@))) +RUN_GZIP = $(M)gzip -nc9 $(patsubst $(SRC_PATH)/%,$(SRC_LINK)/%,$<) >$@ +RUN_MINIFY = $(M)sed 's!/\\*.*\\*/!!g' $< | tr '\n' ' ' | tr -s ' ' | sed 's/^ //; s/ $$//' > $@ +%.gz: TAG = GZIP +%.min: TAG = MINIFY + %.metal.air: %.metal $(METALCC) $< -o $@ @@ -122,61 +128,46 @@ $(BIN2CEXE): ffbuild/bin2c_host.o $(METALLIB) --split-module-without-linking $< -o $@ %.metallib.c: %.metallib $(BIN2CEXE) - $(BIN2C) $< $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) %.ptx: %.cu $(SRC_PATH)/compat/cuda/cuda_runtime.h $(COMPILE_NVCC) ifdef CONFIG_PTX_COMPRESSION -%.ptx.gz: TAG = GZIP %.ptx.gz: %.ptx - $(M)gzip -nc9 $(patsubst $(SRC_PATH)/%,$(SRC_LINK)/%,$<) >$@ + $(RUN_GZIP) %.ptx.c: %.ptx.gz $(BIN2CEXE) - $(BIN2C) $(patsubst $(SRC_PATH)/%,$(SRC_LINK)/%,$<) $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) else %.ptx.c: %.ptx $(BIN2CEXE) - $(BIN2C) $(patsubst $(SRC_PATH)/%,$(SRC_LINK)/%,$<) $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) endif -# 1) Preprocess CSS to a minified version -%.css.min: TAG = SED %.css.min: %.css - $(M)sed 's!/\\*.*\\*/!!g' $< \ - | tr '\n' ' ' \ - | tr -s ' ' \ - | sed 's/^ //; s/ $$//' \ - > $@ + $(RUN_MINIFY) ifdef CONFIG_RESOURCE_COMPRESSION -# 2) Gzip the minified CSS -%.css.min.gz: TAG = GZIP %.css.min.gz: %.css.min - $(M)gzip -nc9 $< > $@ + $(RUN_GZIP) -# 3) Convert the gzipped CSS to a .c array %.css.c: %.css.min.gz $(BIN2CEXE) - $(BIN2C) $< $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) -# 4) Gzip the HTML file (no minification needed) -%.html.gz: TAG = GZIP %.html.gz: %.html - $(M)gzip -nc9 $< > $@ + $(RUN_GZIP) -# 5) Convert the gzipped HTML to a .c array %.html.c: %.html.gz $(BIN2CEXE) - $(BIN2C) $< $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) else # NO COMPRESSION -# 2) Convert the minified CSS to a .c array %.css.c: %.css.min $(BIN2CEXE) - $(BIN2C) $< $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) -# 3) Convert the plain HTML to a .c array %.html.c: %.html $(BIN2CEXE) - $(BIN2C) $< $@ $(subst .,_,$(basename $(notdir $@))) + $(RUN_BIN2C) endif clean:: -- ffmpeg-codebot _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* [FFmpeg-devel] [PATCH v7 3/3] fftools/resources: Update .gitignore 2025-06-23 17:27 [FFmpeg-devel] [PATCH v7 0/3] ffbuild/commonmak: Fix rebuild check with implicit rule chains ffmpegagent 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 1/3] fftools/resources: Fix double-build by disabling .d file generation softworkz 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 2/3] ffbuild/commonmak: Consolidate pattern rules for compression softworkz @ 2025-06-23 17:27 ` softworkz 2 siblings, 0 replies; 4+ messages in thread From: softworkz @ 2025-06-23 17:27 UTC (permalink / raw) To: ffmpeg-devel; +Cc: softworkz From: softworkz <softworkz@hotmail.com> Signed-off-by: softworkz <softworkz@hotmail.com> --- fftools/resources/.gitignore | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fftools/resources/.gitignore b/fftools/resources/.gitignore index bda2c59a1c..cc742817b5 100644 --- a/fftools/resources/.gitignore +++ b/fftools/resources/.gitignore @@ -1,6 +1,5 @@ *.html.c *.css.c *.html.gz -*.css.gz -*.min -*.min.gz +*.css.min +*.css.min.gz -- ffmpeg-codebot _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-06-23 17:34 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-06-23 17:27 [FFmpeg-devel] [PATCH v7 0/3] ffbuild/commonmak: Fix rebuild check with implicit rule chains ffmpegagent 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 1/3] fftools/resources: Fix double-build by disabling .d file generation softworkz 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 2/3] ffbuild/commonmak: Consolidate pattern rules for compression softworkz 2025-06-23 17:27 ` [FFmpeg-devel] [PATCH v7 3/3] fftools/resources: Update .gitignore softworkz
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git