[FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Peter B.]

Hi everyone :)

This is merely a question of interest. Not a request, complaint or 
trolling of any kind :)
I'm happy about any answer. I'm curious.

Would it possibly have a significant impact on coding speed of FFV1's 
slicecrc option (Where a CRC is calculated for each frame slice), to use 
a faster algorithm instead (if one exists)?
I'm wondering if, for example something like "xxHash" may warrant a try?
(Haven't found CRC vs xxHash benchmarks yet, but I'm still looking)



Anyhow,
Thanks for any of your time :D





_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Andreas Rheinhardt]

Peter B.:
> Hi everyone :)
> 
> This is merely a question of interest. Not a request, complaint or
> trolling of any kind :)
> I'm happy about any answer. I'm curious.
> 
> Would it possibly have a significant impact on coding speed of FFV1's
> slicecrc option (Where a CRC is calculated for each frame slice), to use
> a faster algorithm instead (if one exists)?
> I'm wondering if, for example something like "xxHash" may warrant a try?
> (Haven't found CRC vs xxHash benchmarks yet, but I'm still looking)
> 

The format of the checksum is determined by the FFV1 specification and
not by the decoder/encoder implementation.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Kieran Kunhya]

On Tue, 25 Oct 2022, 21:32 Peter B., <pb@das-werkstatt.com> wrote:

> Hi everyone :)
>
> Would it possibly have a significant impact on coding speed of FFV1's
> slicecrc option (Where a CRC is calculated for each frame slice), to use
> a faster algorithm instead (if one exists)?
> I'm wondering if, for example something like "xxHash" may warrant a try?
> (Haven't found CRC vs xxHash benchmarks yet, but I'm still looking)
>
> Anyhow,
> Thanks for any of your time :D


I haven't benchmarked the overhead of the CRC in FFv1 vs the encode or
decode process.
But FFmpeg's CRC could be optimised using multiple unrolled tables or SIMD
or other approaches.

Regards,
Kieran Kunhya
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Paul B Mahol]

On 10/26/22, Kieran Kunhya <kierank@obe.tv> wrote:
> On Tue, 25 Oct 2022, 21:32 Peter B., <pb@das-werkstatt.com> wrote:
>
>> Hi everyone :)
>>
>> Would it possibly have a significant impact on coding speed of FFV1's
>> slicecrc option (Where a CRC is calculated for each frame slice), to use
>> a faster algorithm instead (if one exists)?
>> I'm wondering if, for example something like "xxHash" may warrant a try?
>> (Haven't found CRC vs xxHash benchmarks yet, but I'm still looking)
>>
>> Anyhow,
>> Thanks for any of your time :D
>
>
> I haven't benchmarked the overhead of the CRC in FFv1 vs the encode or
> decode process.
> But FFmpeg's CRC could be optimised using multiple unrolled tables or SIMD
> or other approaches.

It is already using multiple tables.

>
> Regards,
> Kieran Kunhya
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1378 bytes --]

On Tue, Oct 25, 2022 at 10:31:48PM +0200, Peter B. wrote:
> Hi everyone :)
> 
> This is merely a question of interest. Not a request, complaint or trolling
> of any kind :)
> I'm happy about any answer. I'm curious.
> 
> Would it possibly have a significant impact on coding speed of FFV1's
> slicecrc option (Where a CRC is calculated for each frame slice), to use a
> faster algorithm instead (if one exists)?
> I'm wondering if, for example something like "xxHash" may warrant a try?
> (Haven't found CRC vs xxHash benchmarks yet, but I'm still looking)

CRC is not a random pick for error detection, CRC has specfic properties
like for example that whole classes of errors are guranteed to be detected
a random hash will not gurantee that.
for example a properly designed 8bit CRC will detect every single 1 byte change
always. a 8 bit hash like taking 8 bits from SHA256 will detect only 255 out of
256 1 byte changes

also CRC can be done in parallel if you have cores to spare, you can split a
chunk in as many chunks as you want do the CRC for each and combine them with a
bit of math

and CRC is in the specification so using somehing else would require
changing the spec 

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

He who knows, does not speak. He who speaks, does not know. -- Lao Tsu

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Leo Izen]

On 10/26/22 15:13, Michael Niedermayer wrote:
> 
> CRC is not a random pick for error detection, CRC has specfic properties
> like for example that whole classes of errors are guranteed to be detected
> a random hash will not gurantee that.
> for example a properly designed 8bit CRC will detect every single 1 byte change
> always. a 8 bit hash like taking 8 bits from SHA256 will detect only 255 out of
> 256 1 byte changes

I'm a bit confused here, aren't there only 255 one-byte changes in the 
first place? The 256th is when the byte remains unchanged.

That aside I don't believe xxhash is designed to be an error-correcting 
hash, but rather one that's used for fast database lookup.

- Leo Izen (thebombzen)
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Nicolas George]

Leo Izen (12022-11-03):
> I'm a bit confused here, aren't there only 255 one-byte changes in the first
> place? The 256th is when the byte remains unchanged.

Most data is made of more than one byte.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Leo Izen]

On 11/3/22 09:19, Nicolas George wrote:
> Leo Izen (12022-11-03):
>> I'm a bit confused here, aren't there only 255 one-byte changes in the first
>> place? The 256th is when the byte remains unchanged.
> 
> Most data is made of more than one byte.
> 

Ah, so it's on average 255 out of every 256 one-byte changes across the 
entire bitstream, got it. Thanks.

- Leo Izen (thebombzen)
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] FFV1 slicecrc: "hashxx" instead of "CRC" for speed?

[Peter B.]

Thanks for all your replies!

Now I can refer to this, when being asked "Why CRC and not ...?" :)



Kind regards,
Peter
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".