[FFmpeg-devel] [bug?] The curious case of the newline in a filename

[FFmpeg-devel] [bug?] The curious case of the newline in a filename

[Fredrick R. Brennan]

[-- Attachment #1: Type: text/plain, Size: 4229 bytes --]

Consider the following, if you dare:

> ffmpeg -f concat -safe 0 -i <(find . -iname '*.flac' -and -type f -print0 | parallel --null printf '"file %q"' '{=1 my $pwd=qx(pwd); $pwd =~ s/^\s+|\s+$//g; $_=($pwd . "/" . q()) =}{1}' \&\& echo) -b:a 128k -c:a aac -y o.aac

1.
>  find . -iname '*.flac' -and -type f -print0    Finds all files in the current directory and its subdirectories with 
a case-insensitive file extension .flac. It prints the results separated 
by a null character (ASCII 0).

2. >  parallel --null printf '"file %q"' '{=1 my $pwd=qx(pwd); $pwd =~ 
s/^\s+|\s+$//g; $_=($pwd . "/" . q()) =}{1}' \&\& echo   GNU parallel 
reads the null-separated input from the find command and processes them 
in parallel. For each input file, it does the following:

3.
    a. > `{=1 my $pwd=qx(pwd); $pwd =~ s/^\s+|\s+$//g; $_=($pwd . "/" . 
q()) =}`
    Gets the current working directory and removes any leading or
    trailing whitespace. It then concatenates the current directory path
    with the input file name.

    b.
> `printf '"file %q"'    This part formats the output, escaping any special characters in the
    filename, and prefixing it with "file ".

    c.
> \&\& echo    Just adds \n to all lines.

4.
> ffmpeg -f concat -safe 0 -i <(...) -b:a 128k -c:a aac -y o.aac
    Finally, ffmpeg reads the list of escaped file paths (generated @ №2)
    using process substitution <(...). The concat demuxer is used to
    concatenate the input files, -safe0 is specified to allow unsafe file
    paths, and the output is encoded with an AAC codec at 128 kbps
    bitrate, overwriting the output file o.aac if it exists.

Or, ffmpeg *should* do that. It can't:

> [concat @ 0x559da1fd1cc0] Impossible to open '/home/fred/Workspace/gcloud-synthesize-voice/./[af-ZA] Ek hoop dit is nie die geval nie.$\nHoop in die een hand en kak in die ander..flac'
> /proc/self/fd/11: No such file or directory

I should not have to rename my files, ffmpeg is what's broken here. The 
raw output being:

> fred@デブ狸~/Workspace/gcloud-synthesize-voice%  find . -iname '*.flac' -and -type f -print0 | parallel --null printf '"file %q"' '{=1 my $pwd=qx(pwd); $pwd =~ s/^\s+|\s+$//g; $_=($pwd . "/" . q()) =}{1}' \&\& echo
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[af-ZA\]\ Ek\ hoop\ dit\ is\ nie\ die\ geval\ nie.$'\n'Hoop\ in\ die\ een\ hand\ en\ kak\ in\ die\ ander..flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[ar-XA\]\ آمل\ أن\ لا\ يكون\ الأمر\ كذلك.$'\n'املئ\ يدك\ الواحدة\ بالأمل\ والأخرى\ بالسوء..flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[bg-BG\]\ Надявам\ се,\ че\ това\ не\ е\ така.$'\n'Надявай\ се\ в\ едната\ ръка\ и\ срах\ в\ другата..flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[bn-IN\]\ আমি\ আশা\ করি\ এটা\ সত্যি\ না।$'\n'একটা\ হাতে\ আশা\ আর\ অপরে\ বিষম\ চিন্তা।.flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[ca-ES\]\ Espero\ que\ no\ sigui\ el\ cas.$'\n'Esperança\ a\ una\ mà\ i\ merda\ a\ l\'altra..flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[cmn-CN\]\ 我希望不是这样。$'\n'一手抓希望，一手抓屎。.flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[cmn-TW\]\ 我希望不是這樣。$'\n'一手抓希望，一手抓屎。.flac
> file /home/fred/Workspace/gcloud-synthesize-voice/./\[cs-CZ\]\ Doufám,\ že\ to\ není\ pravda.$'\n'Nadej\ se\ v\ jedné\ ruce\ a\ hovno\ v\ druhé..flac


And so on.

This is surely a bug, right? (Unfortunately GNU Parallel's weird NUL 
handling does not help clarify the question, but xargs mangles things 
worse.)

-- 
------------------------------------------------------------------------

Best,
Fred Brennan
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

GnuPG: |98F2 8F76 7470 129F BE3B 054C E215 4DD1 A1C7 7B8B|
Twitter: @fr_brennan
Personal website <https://copypaste.wtf/>

[-- Attachment #2: broken.txt --]
[-- Type: text/plain, Size: 10015 bytes --]

file /home/fred/Workspace/gcloud-synthesize-voice/./\[af-ZA\]\ Ek\ hoop\ dit\ is\ nie\ die\ geval\ nie.$'\n'Hoop\ in\ die\ een\ hand\ en\ kak\ in\ die\ ander..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ar-XA\]\ آمل\ أن\ لا\ يكون\ الأمر\ كذلك.$'\n'املئ\ يدك\ الواحدة\ بالأمل\ والأخرى\ بالسوء..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[bg-BG\]\ Надявам\ се,\ че\ това\ не\ е\ така.$'\n'Надявай\ се\ в\ едната\ ръка\ и\ срах\ в\ другата..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[bn-IN\]\ আমি\ আশা\ করি\ এটা\ সত্যি\ না।$'\n'একটা\ হাতে\ আশা\ আর\ অপরে\ বিষম\ চিন্তা।.flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ca-ES\]\ Espero\ que\ no\ sigui\ el\ cas.$'\n'Esperança\ a\ una\ mà\ i\ merda\ a\ l\'altra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[cmn-CN\]\ 我希望不是这样。$'\n'一手抓希望，一手抓屎。.flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[cmn-TW\]\ 我希望不是這樣。$'\n'一手抓希望，一手抓屎。.flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[cs-CZ\]\ Doufám,\ že\ to\ není\ pravda.$'\n'Nadej\ se\ v\ jedné\ ruce\ a\ hovno\ v\ druhé..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[da-DK\]\ Jeg\ håber,\ det\ ikke\ er\ tilfældet.$'\n'Håb\ i\ den\ ene\ hånd\ og\ lort\ i\ den\ anden..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[de-DE\]\ Ich\ hoffe,\ das\ ist\ nicht\ der\ Fall.$'\n'Hoffnung\ in\ der\ einen\ Hand\ und\ Scheiße\ in\ der\ anderen..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[el-GR\]\ Ελπίζω\ να\ μην\ είναι\ αυτή\ η\ περίπτωση.$'\n'Ελπίδα\ στο\ ένα\ χέρι\ και\ σκατά\ στο\ άλλο..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[en-AU\]\ I\ hope\ that\'s\ not\ the\ case.$'\n'Hope\ in\ one\ hand\ and\ shit\ in\ the\ other..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[en-GB\]\ I\ three\ hope\ that\ is\ not\ the\ case.$'\n'You,\ three,\ hope\ in\ one\ hand\ and\ shit\ in\ the\ other..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[en-IN\]\ I\ four\ hope\ that\ is\ not\ the\ case.$'\n'You,\ four,\ hope\ in\ one\ hand\ and\ shit\ in\ the\ other..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[en-US\]\ I,\ too,\ hope\ that\'s\ not\ the\ case.$'\n'You,\ too,\ hope\ in\ one\ hand\ and\ shit\ in\ the\ other..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[es-ES\]\ También\ espero\ que\ no\ sea\ el\ caso.$'\n'Y\ a\ ti\ también\ digo:\ Del\ dicho\ al\ hecho\ hay\ mucho\ trecho..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[es-US\]\ Espero\ que\ no\ sea\ el\ caso.$'\n'Esperanza\ en\ una\ mano\ y\ mierda\ en\ la\ otra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[eu-ES\]\ Espero\ ez\ dela\ hala.$'\n'Esperantza\ batetik\ eta\ kaka\ bestetik..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[fi-FI\]\ Toivottavasti\ näin\ ei\ ole.$'\n'Toivo\ toisessa\ kädessä\ ja\ paska\ toisessa..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[fil-PH\]\ Sana\ hindi\ ganoon\ ang\ sitwasyon.$'\n'Pag-asa\ sa\ isang\ kamay\ at\ tae\ sa\ kabilang\ kamay..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[fr-CA\]\ Moi\ aussi,\ j\'espère\ que\ ce\ n\'est\ pas\ le\ cas.$'\n'Autant\ pisser\ dans\ un\ violon\ !.flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[fr-FR\]\ J\'espère\ que\ ce\ n\'est\ pas\ le\ cas.$'\n'L\'espoir\ dans\ une\ main\ et\ la\ merde\ dans\ l\'autre..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[gl-ES\]\ Espero\ que\ non\ sexa\ o\ caso.$'\n'Esperanza\ nunha\ man\ e\ merda\ na\ outra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[gu-IN\]\ હું\ આશા\ કરું\ છું\ કે\ તેમ\ ન\ હોય.$'\n'બાજુએ\ આશા\ અને\ બીજી\ બાજુ\ મળજલ..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[he-IL\]\ אני\ מקווה\ שזה\ לא\ המקרה.$'\n'תקווה\ ביד\ אחת\ וחרא\ בשניה..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[hi-IN\]\ मैं\ उम्मीद\ करता\ हूँ\ कि\ ऐसा\ नहीं\ हो।$'\n'एक\ हाथ\ में\ आशा\ और\ दूसरे\ में\ गंदगी।.flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[hu-HU\]\ Remélem,\ nem\ ez\ a\ helyzet.$'\n'Remény\ az\ egyik\ kézben,\ szar\ a\ másikban..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[id-ID\]\ Saya\ harap\ itu\ bukan\ masalahnya.$'\n'Harapan\ di\ satu\ tangan\ dan\ kotoran\ di\ tangan\ lainnya..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[is-IS\]\ Ég\ vona\ að\ þetta\ sé\ ekki\ svo.$'\n'Von\ í\ annarri\ hendi\ og\ skítur\ í\ hinni..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[it-IT\]\ Spero\ che\ non\ sia\ il\ caso.$'\n'Speranza\ in\ una\ mano\ e\ merda\ nell\'altra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ja-JP\]\ そうではないことを願います。$'\n'片手に希望、もう片手にくそ。.flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[kn-IN\]\ ನಾನು\ ಅದು\ ಹಾಗಿರದೆಂದು\ ಆಶಿಸುತ್ತೇನೆ.$'\n'ಒಂದು\ ಕೈಯಲ್ಲಿ\ ನಿರೀಕ್ಷೆ,\ ಇನ್ನೊಂದು\ ಕೈಯಲ್ಲಿ\ ಮಳೆ..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ko-KR\]\ 그렇지\ 않길\ 바랍니다.$'\n'한\ 손엔\ 희망,\ 다른\ 손엔\ 똥..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[lt-LT\]\ Tikiuosi,\ kad\ taip\ nėra.$'\n'Viltis\ vienoje\ rankoje,\ šūdas\ kitoje..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[lv-LV\]\ Ceru,\ ka\ tas\ nav\ tā.$'\n'Cerība\ vienā\ rokā\ un\ sūds\ otrā..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[mr-IN\]\ माझी\ आशा\ असं\ नसावं.$'\n'एका\ हातात\ आशा\ आणि\ दुसऱ्या\ हातात\ गलिच्छ..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ms-MY\]\ Saya\ harap\ itu\ bukan\ kesnya.$'\n'Harapan\ di\ satu\ tangan\ dan\ kotoran\ di\ tangan\ yang\ lain..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[nb-NO\]\ Jeg\ håper\ det\ ikke\ er\ tilfellet.$'\n'Håp\ i\ én\ hånd\ og\ dritt\ i\ den\ andre..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[nl-BE\]\ Ik\ hoop\ ook\ dat\ dat\ niet\ het\ geval\ is.$'\n'Jij\ hoopt\ ook\ in\ de\ ene\ hand\ en\ schijt\ in\ de\ andere..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[pa-IN\]\ ਮੈਂ\ ਉਮੀਦ\ ਕਰਦਾ\ ਹਾਂ\ ਕਿ\ ਇਹ\ ਕੇਸ\ ਨਾ\ ਹੋਵੇ.$'\n'ਇੱਕ\ ਹੱਥ\ ਵਿਚ\ ਉਮੀਦ\ ਅਤੇ\ ਦੂਜੇ\ ਹੱਥ\ ਵਿਚ\ ਮੈਲ..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[pl-PL\]\ Mam\ nadzieję,\ że\ tak\ nie\ jest.$'\n'Nadzieja\ w\ jednej\ ręce,\ gówno\ w\ drugiej..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[pt-BR\]\ Espero\ que\ não\ seja\ o\ caso.$'\n'Esperança\ numa\ mão\ e\ merda\ na\ outra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[pt-PT\]\ Espero\ também\ que\ não\ seja\ o\ caso.$'\n'Você\ também\ tem\ esperança\ numa\ mão\ e\ merda\ na\ outra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ro-RO\]\ Sper\ că\ nu\ e\ așa.$'\n'Speranță\ într-o\ mână\ și\ rahat\ în\ cealaltă..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ru-RU\]\ Я\ надеюсь,\ что\ это\ не\ так.$'\n'Надежда\ в\ одной\ руке,\ а\ дерьмо\ в\ другой..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[sk-SK\]\ Dúfam,\ že\ to\ tak\ nie\ je.$'\n'Nádej\ v\ jednej\ ruke\ a\ hovno\ v\ druhej..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[sr-RS\]\ Надам\ се\ да\ то\ није\ случај.$'\n'Надај\ се\ у\ једној\ руци\ и\ говно\ у\ другој..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[sv-SE\]\ Jag\ hoppas\ att\ det\ inte\ är\ fallet.$'\n'Hopp\ i\ en\ hand\ och\ skit\ i\ den\ andra..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[ta-IN\]\ அது\ அப்படி\ இல்லை\ என்று\ நான்\ நம்புகிறேன்.$'\n'ஒரு\ கையில்\ நம்பிக்கை,\ மற்றொரு\ கையில்\ அசுத்தம்..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[te-IN\]\ నేను\ అది\ అలా\ ఉండదు\ అనుకుంటున్నాను.$'\n'ఒక\ చేతిలో\ ఆశ,\ మరో\ చేతిలో\ మలినం..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[th-TH\]\ ฉันหวังว่ามันไม่ใช่กรณีนั้น.$'\n'ความหวังอยู่ในมือหนึ่งและขี้อยู่ในมืออีกอัน..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[tr-TR\]\ Umarım\ öyle\ değildir.$'\n'Bir\ elde\ umut,\ diğerinde\ bok..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[uk-UA\]\ Сподіваюся,\ що\ це\ не\ так.$'\n'Надія\ в\ одній\ руці,\ а\ лайно\ в\ іншій..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[vi-VN\]\ Tôi\ hy\ vọng\ điều\ đó\ không\ phải\ là\ trường\ hợp.$'\n'Hy\ vọng\ trong\ một\ tay\ và\ cặn\ bã\ trong\ tay\ kia..flac
file /home/fred/Workspace/gcloud-synthesize-voice/./\[yue-HK\]\ 希望唔係咁嘅。$'\n'一隻手攞希望，一隻手攞屎。.flac

[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".