Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Wu Jianhua]

> 29 Jan 2022, 13:34 by toqsxw at outlook.com:

>> Ping.
>>
>>> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com at ffmpeg.org>
>>> Sent: 2022年1月21日 19:42
>>> To: ffmpeg-devel at ffmpeg.org<mailto:ffmpeg-devel at ffmpeg.org>
>>> Cc: Wu, Jianhua<mailto:jianhua.wu at intel.com>
>>> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option
>>>
>>> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5] avfilter:add shader_vulkan filter [PATCH 3/5] avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5] avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5] avfilter/scale_vulkan: use RET for checking return value
>>>
>>> Patches attached.
>>>
>>
>> Hi there,
>>
>> Any update?
>>
>
> Sorry, haven't forgotten, but been busy with FFTs lately.
> Will try to review and test the patches soon.

Hi there,

I'm sorry for bothering you. If there is any update on this
thread, please do let me know. 

Thanks,
Jianhua

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Lynne]

18 Feb 2022, 16:24 by toqsxw@outlook.com:

>> 29 Jan 2022, 13:34 by toqsxw at outlook.com:
>>
>>> Ping.
>>>
>>>> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com at ffmpeg.org>
>>>> Sent: 2022年1月21日 19:42
>>>> To: ffmpeg-devel at ffmpeg.org<mailto:ffmpeg-devel at ffmpeg.org>
>>>> Cc: Wu, Jianhua<mailto:jianhua.wu at intel.com>
>>>> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option
>>>>
>>>> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5] avfilter:add shader_vulkan filter [PATCH 3/5] avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5] avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5] avfilter/scale_vulkan: use RET for checking return value
>>>>
>>>> Patches attached.
>>>>
>>>
>>> Hi there,
>>>
>>> Any update?
>>>
>>
>> Sorry, haven't forgotten, but been busy with FFTs lately.
>> Will try to review and test the patches soon.
>>
>
> Hi there,
>
> I'm sorry for bothering you. If there is any update on this
> thread, please do let me know. 
>

Pushed all except the strlen() in a loop condition and the shader filter.
I pushed a different, smaller version for the strlen patch.

As for a shader filter, I'd like something that's a lot less minimal.
You should expose the frame number, framerate (with an avoption to set it),
pixel format to the shader. Keep in mind the API will be fixed, so
we need to get this right the first time hopefully.
Also, correct the name style. We don't use camelcase for variables,
and we use "av_" instead of "ff_" for public API, which a shader sort of is IMO.
You should expose alpha planes as well.
Finally, could you implement N-inputs and M-outputs, configurable via
avoptions? That way, someone could make a custom blend filter without
a separate avfilter which takes multiple inputs. Or a separator filter.
Or a simple source filter that just produces an image pattern.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Wu, Jianhua]

Lynne:
> Sent: Tuesday, February 22, 2022 1:38 PM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV
> option
> 
> 18 Feb 2022, 16:24 by toqsxw@outlook.com:
> 
> >> 29 Jan 2022, 13:34 by toqsxw at outlook.com:
> >>
> >>> Ping.
> >>>
> >>>> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com at ffmpeg.org>
> >>>> Sent: 2022年1月21日 19:42
> >>>> To: ffmpeg-devel at ffmpeg.org<mailto:ffmpeg-devel at ffmpeg.org>
> >>>> Cc: Wu, Jianhua<mailto:jianhua.wu at intel.com>
> >>>> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add
> >>>> sizeV option
> >>>>
> >>>> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5]
> >>>> avfilter:add shader_vulkan filter [PATCH 3/5]
> >>>> avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5]
> >>>> avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5]
> >>>> avfilter/scale_vulkan: use RET for checking return value
> >>>>
> >>>> Patches attached.
> >>>>
> >>>
> >>> Hi there,
> >>>
> >>> Any update?
> >>>
> >>
> >> Sorry, haven't forgotten, but been busy with FFTs lately.
> >> Will try to review and test the patches soon.
> >>
> >
> > Hi there,
> >
> > I'm sorry for bothering you. If there is any update on this thread,
> > please do let me know.
> >
> 
> Pushed all except the strlen() in a loop condition and the shader filter.
> I pushed a different, smaller version for the strlen patch.
> 
Maybe you don't need to use strlen() at all. That patch could be applied
separately if you preferred to apply the shader_vulkan filter in the future.

> As for a shader filter, I'd like something that's a lot less minimal.
> You should expose the frame number, framerate (with an avoption to set it),
> pixel format to the shader. Keep in mind the API will be fixed, so we need to
> get this right the first time hopefully.

Frame number and framerate are okay to set if I can get them from FFmpeg API,
but pixel format may not be ideal to expose for there is a lot of pixel formats in
FFmpeg. Exposing a pixel format means we need to expose all values related to
pixel formats. Instead, we could expose two functions like

vec4 pixel = av_read_pixel(intput_images, av_position) 
av_write_pixel(ouput_images, pixel, positions)

so the user shader could only concentrate on the vector4 pixel variable and don't
need to care about what pixel format is. Or simply expose the subsampling scheme,
444, 420, or 422, and color space, YUV, and RGB.

> Also, correct the name style. We don't use camelcase for variables, and we
> use "av_" instead of "ff_" for public API, which a shader sort of is IMO.

Got it.

> You should expose alpha planes as well.
Could you elaborate it further?

> Finally, could you implement N-inputs and M-outputs, configurable via
> avoptions? That way, someone could make a custom blend filter without a
> separate avfilter which takes multiple inputs. Or a separator filter.
> Or a simple source filter that just produces an image pattern.
> 
Sounds great! However, at the present, I've no idea about how this could be done.
I think the current filter is already useful already for I could make some great video
effects just like Shadertoy. More extensions need further contributions.

Thanks,
Jianhua

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Lynne]

22 Feb 2022, 07:27 by jianhua.wu-at-intel.com@ffmpeg.org:

> Lynne:
>
>> Sent: Tuesday, February 22, 2022 1:38 PM
>> To: FFmpeg development discussions and patches <ffmpeg-
>> devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV
>> option
>>
>> 18 Feb 2022, 16:24 by toqsxw@outlook.com:
>>
>> >> 29 Jan 2022, 13:34 by toqsxw at outlook.com:
>> >>
>> >>> Ping.
>> >>>
>> >>>> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com at ffmpeg.org>
>> >>>> Sent: 2022年1月21日 19:42
>> >>>> To: ffmpeg-devel at ffmpeg.org<mailto:ffmpeg-devel at ffmpeg.org>
>> >>>> Cc: Wu, Jianhua<mailto:jianhua.wu at intel.com>
>> >>>> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add
>> >>>> sizeV option
>> >>>>
>> >>>> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5]
>> >>>> avfilter:add shader_vulkan filter [PATCH 3/5]
>> >>>> avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5]
>> >>>> avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5]
>> >>>> avfilter/scale_vulkan: use RET for checking return value
>> >>>>
>> >>>> Patches attached.
>> >>>>
>> >>>
>> >>> Hi there,
>> >>>
>> >>> Any update?
>> >>>
>> >>
>> >> Sorry, haven't forgotten, but been busy with FFTs lately.
>> >> Will try to review and test the patches soon.
>> >>
>> >
>> > Hi there,
>> >
>> > I'm sorry for bothering you. If there is any update on this thread,
>> > please do let me know.
>> >
>>
>> Pushed all except the strlen() in a loop condition and the shader filter.
>> I pushed a different, smaller version for the strlen patch.
>>
> Maybe you don't need to use strlen() at all. That patch could be applied
> separately if you preferred to apply the shader_vulkan filter in the future.
>
>> As for a shader filter, I'd like something that's a lot less minimal.
>> You should expose the frame number, framerate (with an avoption to set it),
>> pixel format to the shader. Keep in mind the API will be fixed, so we need to
>> get this right the first time hopefully.
>>
>
> Frame number and framerate are okay to set if I can get them from FFmpeg API,
> but pixel format may not be ideal to expose for there is a lot of pixel formats in
> FFmpeg. Exposing a pixel format means we need to expose all values related to
> pixel formats. Instead, we could expose two functions like
>

Wrapper functions won't help, shaders would still need to know what colorspace to work in.
I think if you just expose the entire pixdesc flags as-is, so shaders know if it's RGB or YUV, and the raw colorspace/transfer/matrix integers, it'll be good enough.


>> You should expose alpha planes as well.
>>
> Could you elaborate it further?
>

Yes, currently for YUVA images, the alpha channel remains untouched (the size of all image arrays is [3]) and unexposed to the shader.


>> Finally, could you implement N-inputs and M-outputs, configurable via
>> avoptions? That way, someone could make a custom blend filter without a
>> separate avfilter which takes multiple inputs. Or a separator filter.
>> Or a simple source filter that just produces an image pattern.
>>
> Sounds great! However, at the present, I've no idea about how this could be done.
> I think the current filter is already useful already for I could make some great video
> effects just like Shadertoy. More extensions need further contributions.
>

Just set AVFilter.inputs and AVFilter.outputs to NULL. Then in the init function,
copy what e.g. af_amix does to set inputs and what e.g. af_channelsplit does for
outputs. There are probably better examples out there if you look.
I think that this should be up to the shader to initialize, rather than the filter user,
so perhaps, in the init function, you compile the shader, run an init function in
the shader which tells you what it requires and does checking, then return,
and set it up for operation.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Wu, Jianhua]

Lynne:
> Sent: Tuesday, February 22, 2022 5:36 PM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV
> option
> 
> 22 Feb 2022, 07:27 by jianhua.wu-at-intel.com@ffmpeg.org:
> 
> > Lynne:
> >
> >> Sent: Tuesday, February 22, 2022 1:38 PM
> >> To: FFmpeg development discussions and patches <ffmpeg-
> >> devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add
> >> sizeV option
> >>
> >> 18 Feb 2022, 16:24 by toqsxw@outlook.com:
> >>
> >> >> 29 Jan 2022, 13:34 by toqsxw at outlook.com:
> >> >>
> >> >>> Ping.
> >> >>>
> >> >>>> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com at ffmpeg.org>
> >> >>>> Sent: 2022年1月21日 19:42
> >> >>>> To: ffmpeg-devel at ffmpeg.org<mailto:ffmpeg-devel at
> >> >>>> ffmpeg.org>
> >> >>>> Cc: Wu, Jianhua<mailto:jianhua.wu at intel.com>
> >> >>>> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add
> >> >>>> sizeV option
> >> >>>>
> >> >>>> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH
> >> >>>> 2/5] avfilter:add shader_vulkan filter [PATCH 3/5]
> >> >>>> avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5]
> >> >>>> avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5]
> >> >>>> avfilter/scale_vulkan: use RET for checking return value
> >> >>>>
> >> >>>> Patches attached.
> >> >>>>
> >> >>>
> >> >>> Hi there,
> >> >>>
> >> >>> Any update?
> >> >>>
> >> >>
> >> >> Sorry, haven't forgotten, but been busy with FFTs lately.
> >> >> Will try to review and test the patches soon.
> >> >>
> >> >
> >> > Hi there,
> >> >
> >> > I'm sorry for bothering you. If there is any update on this thread,
> >> > please do let me know.
> >> >
> >>
> >> Pushed all except the strlen() in a loop condition and the shader filter.
> >> I pushed a different, smaller version for the strlen patch.
> >>
> > Maybe you don't need to use strlen() at all. That patch could be
> > applied separately if you preferred to apply the shader_vulkan filter in the
> future.
> >
> >> As for a shader filter, I'd like something that's a lot less minimal.
> >> You should expose the frame number, framerate (with an avoption to
> >> set it), pixel format to the shader. Keep in mind the API will be
> >> fixed, so we need to get this right the first time hopefully.
> >>
> >
> > Frame number and framerate are okay to set if I can get them from
> > FFmpeg API, but pixel format may not be ideal to expose for there is a
> > lot of pixel formats in FFmpeg. Exposing a pixel format means we need
> > to expose all values related to pixel formats. Instead, we could
> > expose two functions like
> >
> 
> Wrapper functions won't help, shaders would still need to know what
> colorspace to work in.
> I think if you just expose the entire pixdesc flags as-is, so shaders know if it's
> RGB or YUV, and the raw colorspace/transfer/matrix integers, it'll be good
> enough.
> 
> 
> >> You should expose alpha planes as well.
> >>
> > Could you elaborate it further?
> >
> 
> Yes, currently for YUVA images, the alpha channel remains untouched (the
> size of all image arrays is [3]) and unexposed to the shader.
> 
> 
> >> Finally, could you implement N-inputs and M-outputs, configurable via
> >> avoptions? That way, someone could make a custom blend filter without
> >> a separate avfilter which takes multiple inputs. Or a separator filter.
> >> Or a simple source filter that just produces an image pattern.
> >>
> > Sounds great! However, at the present, I've no idea about how this could
> be done.
> > I think the current filter is already useful already for I could make
> > some great video effects just like Shadertoy. More extensions need
> further contributions.
> >
> 
> Just set AVFilter.inputs and AVFilter.outputs to NULL. Then in the init
> function, copy what e.g. af_amix does to set inputs and what e.g.
> af_channelsplit does for outputs. There are probably better examples out
> there if you look.
> I think that this should be up to the shader to initialize, rather than the filter
> user, so perhaps, in the init function, you compile the shader, run an init
> function in the shader which tells you what it requires and does checking,
> then return, and set it up for operation.
> 

I suppose you mean there is a init function in the shader source file and execute
it to read back some configuration data, don't you? If so, looks like a great feature.
And I think, isn't these characteristics should be only added when there is someone
who has the relative requirements?

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Wu, Jianhua]

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

[PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5] avfilter:add shader_vulkan filter [PATCH 3/5] avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5] avutil/vulkan: don't use strlen as loop condition [PATCH 5/5] avfilter/scale_vulkan: use RET for checking return value

Patches attached.

[-- Attachment #2: 0003-avfilter-vf_blend_vulkan-add-multiply-blend-mode.patch --]
[-- Type: application/octet-stream, Size: 2964 bytes --]

[-- Attachment #3: 0004-avutil-vulkan-don-t-use-strlen-as-loop-condition.patch --]
[-- Type: application/octet-stream, Size: 1531 bytes --]

[-- Attachment #4: 0005-avfilter-scale_vulkan-use-RET-for-checking-return-va.patch --]
[-- Type: application/octet-stream, Size: 2241 bytes --]

[-- Attachment #5: 0001-avfilter-vf_gblur_vulkan-add-sizeV-option.patch --]
[-- Type: application/octet-stream, Size: 19908 bytes --]

[-- Attachment #6: 0002-avfilter-add-shader_vulkan-filter.patch --]
[-- Type: application/octet-stream, Size: 14963 bytes --]

[-- Attachment #7: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Wu Jianhua]

Ping.

> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com@ffmpeg.org>
> Sent: 2022年1月21日 19:42
> To: ffmpeg-devel@ffmpeg.org<mailto:ffmpeg-devel@ffmpeg.org>
> Cc: Wu, Jianhua<mailto:jianhua.wu@intel.com>
> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option
>
> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5] avfilter:add shader_vulkan filter [PATCH 3/5] avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5] avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5] avfilter/scale_vulkan: use RET for checking return value
>
> Patches attached.

Hi there,

Any update?

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option

[Lynne]

29 Jan 2022, 13:34 by toqsxw@outlook.com:

> Ping.
>
>> From: Wu, Jianhua<mailto:jianhua.wu-at-intel.com@ffmpeg.org>
>> Sent: 2022年1月21日 19:42
>> To: ffmpeg-devel@ffmpeg.org<mailto:ffmpeg-devel@ffmpeg.org>
>> Cc: Wu, Jianhua<mailto:jianhua.wu@intel.com>
>> Subject: [FFmpeg-devel] [PATCH v1] avfilter/vf_gblur_vulkan: add sizeV option
>>
>> [PATCH 1/5] avfilter/vf_gblur_vulkan: add sizeV option [PATCH 2/5] avfilter:add shader_vulkan filter [PATCH 3/5] avfilter/vf_blend_vulkan: add multiply blend mode [PATCH 4/5] avutil/vulkan: don't use strlen as loop >condition [PATCH 5/5] avfilter/scale_vulkan: use RET for checking return value
>>
>> Patches attached.
>>
>
> Hi there,
>
> Any update?
>

Sorry, haven't forgotten, but been busy with FFTs lately.
Will try to review and test the patches soon.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".