[FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[Gyan Doshi]

This is to not break linking with toolchains that don't support reading
args from a 'response file'.
---
I've assumed that ld on a system will have same support as ar.

 configure           | 7 +++++++
 ffbuild/library.mak | 8 ++++++++
 2 files changed, 15 insertions(+)

diff --git a/configure b/configure
index f6964c4ee1..d84e32196d 100755
--- a/configure
+++ b/configure
@@ -5230,6 +5230,12 @@ else
     ar_o='$@'
 fi
 
+if $ar 2>&1 | grep -qi  "@.*file"; then
+    ar_objs="true"
+else
+    ar_objs=""
+fi
+
 add_cflags $extra_cflags
 add_cxxflags $extra_cxxflags
 add_objcflags $extra_objcflags
@@ -8133,6 +8139,7 @@ DEPX86ASM=$x86asmexe
 DEPX86ASMFLAGS=\$(X86ASMFLAGS)
 AR=$ar
 ARFLAGS=$arflags
+AR_OBJS=$ar_objs
 AR_O=$ar_o
 AR_CMD=$ar
 NM_CMD=$nm
diff --git a/ffbuild/library.mak b/ffbuild/library.mak
index 72e3872157..7e1871b74c 100644
--- a/ffbuild/library.mak
+++ b/ffbuild/library.mak
@@ -35,8 +35,12 @@ OBJS += $(SHLIBOBJS)
 endif
 $(SUBDIR)$(LIBNAME): $(OBJS) $(STLIBOBJS)
 	$(RM) $@
+ifeq ($(AR_OBJS),true)
 	$(Q)echo $^ > $@.objs
 	$(AR) $(ARFLAGS) $(AR_O) @$@.objs
+else
+	$(AR) $(ARFLAGS) $(AR_O) $^
+endif
 	$(RANLIB) $@
 	-$(RM) $@.objs
 
@@ -68,8 +72,12 @@ $(SUBDIR)$(SLIBNAME): $(SUBDIR)$(SLIBNAME_WITH_MAJOR)
 
 $(SUBDIR)$(SLIBNAME_WITH_MAJOR): $(OBJS) $(SHLIBOBJS) $(SLIBOBJS) $(SUBDIR)lib$(NAME).ver
 	$(SLIB_CREATE_DEF_CMD)
+ifeq ($(AR_OBJS),true)
 	$(Q)echo $$(filter %.o,$$^) > $$@.objs
 	$$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) @$$@.objs $(FFEXTRALIBS)
+else
+	$$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) $$(filter %.o,$$^) $(FFEXTRALIBS)
+endif
 	$(SLIB_EXTRA_CMD)
 	-$(RM) $$@.objs
 
-- 
2.46.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[Zhao Zhili]

> On Mar 17, 2025, at 23:16, Gyan Doshi <ffmpeg@gyani.pro> wrote:
> 
> This is to not break linking with toolchains that don't support reading
> args from a 'response file'.
> ---
> I've assumed that ld on a system will have same support as ar.
> 
> configure           | 7 +++++++
> ffbuild/library.mak | 8 ++++++++
> 2 files changed, 15 insertions(+)
> 
> diff --git a/configure b/configure
> index f6964c4ee1..d84e32196d 100755
> --- a/configure
> +++ b/configure
> @@ -5230,6 +5230,12 @@ else
>     ar_o='$@'
> fi
> 
> +if $ar 2>&1 | grep -qi  "@.*file"; then
> +    ar_objs="true"
> +else
> +    ar_objs=""
> +fi

Works for me.

> +
> add_cflags $extra_cflags
> add_cxxflags $extra_cxxflags
> add_objcflags $extra_objcflags
> @@ -8133,6 +8139,7 @@ DEPX86ASM=$x86asmexe
> DEPX86ASMFLAGS=\$(X86ASMFLAGS)
> AR=$ar
> ARFLAGS=$arflags
> +AR_OBJS=$ar_objs
> AR_O=$ar_o
> AR_CMD=$ar
> NM_CMD=$nm
> diff --git a/ffbuild/library.mak b/ffbuild/library.mak
> index 72e3872157..7e1871b74c 100644
> --- a/ffbuild/library.mak
> +++ b/ffbuild/library.mak
> @@ -35,8 +35,12 @@ OBJS += $(SHLIBOBJS)
> endif
> $(SUBDIR)$(LIBNAME): $(OBJS) $(STLIBOBJS)
> 	$(RM) $@
> +ifeq ($(AR_OBJS),true)
> 	$(Q)echo $^ > $@.objs
> 	$(AR) $(ARFLAGS) $(AR_O) @$@.objs
> +else
> +	$(AR) $(ARFLAGS) $(AR_O) $^
> +endif
> 	$(RANLIB) $@
> 	-$(RM) $@.objs
> 
> @@ -68,8 +72,12 @@ $(SUBDIR)$(SLIBNAME): $(SUBDIR)$(SLIBNAME_WITH_MAJOR)
> 
> $(SUBDIR)$(SLIBNAME_WITH_MAJOR): $(OBJS) $(SHLIBOBJS) $(SLIBOBJS) $(SUBDIR)lib$(NAME).ver
> 	$(SLIB_CREATE_DEF_CMD)
> +ifeq ($(AR_OBJS),true)
> 	$(Q)echo $$(filter %.o,$$^) > $$@.objs
> 	$$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) @$$@.objs $(FFEXTRALIBS)
> +else
> +	$$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) $$(filter %.o,$$^) $(FFEXTRALIBS)
> +endif
> 	$(SLIB_EXTRA_CMD)
> 	-$(RM) $$@.objs
> 
> -- 
> 2.46.1
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[Gyan Doshi]

On 2025-03-17 09:44 pm, Zhao Zhili wrote:
>
>> On Mar 17, 2025, at 23:16, Gyan Doshi <ffmpeg@gyani.pro> wrote:
>>
>> This is to not break linking with toolchains that don't support reading
>> args from a 'response file'.
>> ---
>> I've assumed that ld on a system will have same support as ar.
>>
>> configure           | 7 +++++++
>> ffbuild/library.mak | 8 ++++++++
>> 2 files changed, 15 insertions(+)
>>
>> diff --git a/configure b/configure
>> index f6964c4ee1..d84e32196d 100755
>> --- a/configure
>> +++ b/configure
>> @@ -5230,6 +5230,12 @@ else
>>      ar_o='$@'
>> fi
>>
>> +if $ar 2>&1 | grep -qi  "@.*file"; then
>> +    ar_objs="true"
>> +else
>> +    ar_objs=""
>> +fi
> Works for me.

Good. Let's wait for another report.

Regards,
Gyan



>
>> +
>> add_cflags $extra_cflags
>> add_cxxflags $extra_cxxflags
>> add_objcflags $extra_objcflags
>> @@ -8133,6 +8139,7 @@ DEPX86ASM=$x86asmexe
>> DEPX86ASMFLAGS=\$(X86ASMFLAGS)
>> AR=$ar
>> ARFLAGS=$arflags
>> +AR_OBJS=$ar_objs
>> AR_O=$ar_o
>> AR_CMD=$ar
>> NM_CMD=$nm
>> diff --git a/ffbuild/library.mak b/ffbuild/library.mak
>> index 72e3872157..7e1871b74c 100644
>> --- a/ffbuild/library.mak
>> +++ b/ffbuild/library.mak
>> @@ -35,8 +35,12 @@ OBJS += $(SHLIBOBJS)
>> endif
>> $(SUBDIR)$(LIBNAME): $(OBJS) $(STLIBOBJS)
>> 	$(RM) $@
>> +ifeq ($(AR_OBJS),true)
>> 	$(Q)echo $^ > $@.objs
>> 	$(AR) $(ARFLAGS) $(AR_O) @$@.objs
>> +else
>> +	$(AR) $(ARFLAGS) $(AR_O) $^
>> +endif
>> 	$(RANLIB) $@
>> 	-$(RM) $@.objs
>>
>> @@ -68,8 +72,12 @@ $(SUBDIR)$(SLIBNAME): $(SUBDIR)$(SLIBNAME_WITH_MAJOR)
>>
>> $(SUBDIR)$(SLIBNAME_WITH_MAJOR): $(OBJS) $(SHLIBOBJS) $(SLIBOBJS) $(SUBDIR)lib$(NAME).ver
>> 	$(SLIB_CREATE_DEF_CMD)
>> +ifeq ($(AR_OBJS),true)
>> 	$(Q)echo $$(filter %.o,$$^) > $$@.objs
>> 	$$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) @$$@.objs $(FFEXTRALIBS)
>> +else
>> +	$$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) $$(filter %.o,$$^) $(FFEXTRALIBS)
>> +endif
>> 	$(SLIB_EXTRA_CMD)
>> 	-$(RM) $$@.objs
>>
>> -- 
>> 2.46.1
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[Martin Storsjö]

On Mon, 17 Mar 2025, Gyan Doshi wrote:

>
>
> On 2025-03-17 09:44 pm, Zhao Zhili wrote:
>> 
>>> On Mar 17, 2025, at 23:16, Gyan Doshi <ffmpeg@gyani.pro> wrote:
>>> 
>>> This is to not break linking with toolchains that don't support reading
>>> args from a 'response file'.
>>> ---
>>> I've assumed that ld on a system will have same support as ar.
>>> 
>>> configure           | 7 +++++++
>>> ffbuild/library.mak | 8 ++++++++
>>> 2 files changed, 15 insertions(+)
>>> 
>>> diff --git a/configure b/configure
>>> index f6964c4ee1..d84e32196d 100755
>>> --- a/configure
>>> +++ b/configure
>>> @@ -5230,6 +5230,12 @@ else
>>>      ar_o='$@'
>>> fi
>>> 
>>> +if $ar 2>&1 | grep -qi  "@.*file"; then
>>> +    ar_objs="true"
>>> +else
>>> +    ar_objs=""
>>> +fi
>> Works for me.
>
> Good. Let's wait for another report.

This unbreaks the build for me on macOS too.

As this change did break a fairly major platform, I don't think we should 
wait too long to either revert the change, or push the fix.

As for the fix - I generally find it more robust to actually _try_ doing 
the requested thing (use a response file), than to look for specific 
strings in help output.

I see that the regex, "@.*file", should be matched by both GNU ar and 
llvm-ar, so that's good. But e.g. MS lib.exe also supports response files, 
and it doesn't include this pattern in the output. Also MS lib.exe would 
certainly be a case where we'd want to use response files to overcome the 
command line length limit.

So TL;DR, please push this to unbreak platforms like macOS and BSD, or 
revert the original change. But I don't find the test entirely ideal.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[Gyan Doshi]

On 2025-03-18 03:27 pm, Martin Storsjö wrote:
> On Mon, 17 Mar 2025, Gyan Doshi wrote:
>
>>
>>
>> On 2025-03-17 09:44 pm, Zhao Zhili wrote:
>>>
>>>> On Mar 17, 2025, at 23:16, Gyan Doshi <ffmpeg@gyani.pro> wrote:
>>>>
>>>> This is to not break linking with toolchains that don't support 
>>>> reading
>>>> args from a 'response file'.
>>>> ---
>>>> I've assumed that ld on a system will have same support as ar.
>>>>
>>>> configure           | 7 +++++++
>>>> ffbuild/library.mak | 8 ++++++++
>>>> 2 files changed, 15 insertions(+)
>>>>
>>>> diff --git a/configure b/configure
>>>> index f6964c4ee1..d84e32196d 100755
>>>> --- a/configure
>>>> +++ b/configure
>>>> @@ -5230,6 +5230,12 @@ else
>>>>      ar_o='$@'
>>>> fi
>>>>
>>>> +if $ar 2>&1 | grep -qi  "@.*file"; then
>>>> +    ar_objs="true"
>>>> +else
>>>> +    ar_objs=""
>>>> +fi
>>> Works for me.
>>
>> Good. Let's wait for another report.
>
> This unbreaks the build for me on macOS too.
>
> As this change did break a fairly major platform, I don't think we 
> should wait too long to either revert the change, or push the fix.
>
> As for the fix - I generally find it more robust to actually _try_ 
> doing the requested thing (use a response file), than to look for 
> specific strings in help output.
>
> I see that the regex, "@.*file", should be matched by both GNU ar and 
> llvm-ar, so that's good. But e.g. MS lib.exe also supports response 
> files, and it doesn't include this pattern in the output. Also MS 
> lib.exe would certainly be a case where we'd want to use response 
> files to overcome the command line length limit.
>
> So TL;DR, please push this to unbreak platforms like macOS and BSD, or 
> revert the original change. But I don't find the test entirely ideal.

I'll push this now, and then work on a robust check.

Regards,
Gyan

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] ffbuild: use response files only if ar accepts them

[Gyan Doshi]

On 2025-03-18 03:34 pm, Gyan Doshi wrote:
>
>
> On 2025-03-18 03:27 pm, Martin Storsjö wrote:
>> On Mon, 17 Mar 2025, Gyan Doshi wrote:
>>
>>>
>>>
>>> On 2025-03-17 09:44 pm, Zhao Zhili wrote:
>>>>
>>>>> On Mar 17, 2025, at 23:16, Gyan Doshi <ffmpeg@gyani.pro> wrote:
>>>>>
>>>>> This is to not break linking with toolchains that don't support 
>>>>> reading
>>>>> args from a 'response file'.
>>>>> ---
>>>>> I've assumed that ld on a system will have same support as ar.
>>>>>
>>>>> configure           | 7 +++++++
>>>>> ffbuild/library.mak | 8 ++++++++
>>>>> 2 files changed, 15 insertions(+)
>>>>>
>>>>> diff --git a/configure b/configure
>>>>> index f6964c4ee1..d84e32196d 100755
>>>>> --- a/configure
>>>>> +++ b/configure
>>>>> @@ -5230,6 +5230,12 @@ else
>>>>>      ar_o='$@'
>>>>> fi
>>>>>
>>>>> +if $ar 2>&1 | grep -qi  "@.*file"; then
>>>>> +    ar_objs="true"
>>>>> +else
>>>>> +    ar_objs=""
>>>>> +fi
>>>> Works for me.
>>>
>>> Good. Let's wait for another report.
>>
>> This unbreaks the build for me on macOS too.
>>
>> As this change did break a fairly major platform, I don't think we 
>> should wait too long to either revert the change, or push the fix.
>>
>> As for the fix - I generally find it more robust to actually _try_ 
>> doing the requested thing (use a response file), than to look for 
>> specific strings in help output.
>>
>> I see that the regex, "@.*file", should be matched by both GNU ar and 
>> llvm-ar, so that's good. But e.g. MS lib.exe also supports response 
>> files, and it doesn't include this pattern in the output. Also MS 
>> lib.exe would certainly be a case where we'd want to use response 
>> files to overcome the command line length limit.
>>
>> So TL;DR, please push this to unbreak platforms like macOS and BSD, 
>> or revert the original change. But I don't find the test entirely ideal.
>
> I'll push this now, and then work on a robust check.

Pushed as 740d4009656abeb006e2d43001b59d875c1a7a6a

Regards,
Gyan

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".