From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 968D34AC82 for ; Mon, 17 Jun 2024 19:21:18 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D241E68D7AA; Mon, 17 Jun 2024 22:20:46 +0300 (EEST) Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 18AA068D711 for ; Mon, 17 Jun 2024 22:20:36 +0300 (EEST) Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-356c4e926a3so4433731f8f.1 for ; Mon, 17 Jun 2024 12:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718652035; x=1719256835; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=zWOWjZ5f74HFO6osESaqpNjmIOWO031L/UmmQ4NxUII=; b=b3N7IF/YtATiZt9e+OBdnX3hOVzty9IMEYwwGSq4yjGnqKBebfELz9B74k+ZeYL/rt 6/1GZr246Lu8VNxhkQ9cjlZJPUkKcrV1O7Vac5ChuPWmsa32KxGYh2IlmmgL3w7tkCdf klaqYDAEm3ewn5JC6u9mefGxNLztxilLEvCwnplNW9Wr1eQCV8YN/v4ljC0fu4QafZ8k po63AbWup8LRIXVCGkzBxqPLkgcxgPUjaQACZ2JPkJurM+XzgmYn+jMSrc1a0UNcK8O0 GsubaM0PSC+RVLsGX56G4lBWrqF2Dpw2Sckd5c1NeYbfuPV3gKCUSx9i4ZFszfbPLaO6 D1lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718652035; x=1719256835; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zWOWjZ5f74HFO6osESaqpNjmIOWO031L/UmmQ4NxUII=; b=Vh+dTekTj9bjeCJyhfaa/71ycVPh5gKuhPICrQJuNCzk7vZyTFCYLrUd4NVBqFt6BS CCCQ3rSxEAZw+Z8KMuOhH1O8DmPOpGxwW+btwHLLB7Ro3slhP4g3heglUHupIVzfGema qmlFfpmOIh5dIKzDphu3KWvrI5qc4TtEsEl7NhlTBLWFdxtviWh7Gyo/JOYKKzlSugPs 205vN31sD/Mntug0OQPwRsGP80Be98rNsrHEUuQzlROCflGoSuLzRE6CLd/vqzeY7fZZ rtuwNoME3fv3tSo3BKXmNJ+UwKOip30TThTaM+qYa4eODRO8WoHVo/PSTnX0OMHYv67k gDGw== X-Gm-Message-State: AOJu0YxB/OlG8MZzqg0OAv6ATYv8YnFhbDiD8lcIb2H5FpivoXe9Fyes KBTbCWVmADGmyK9xnDjYDJIBf0ug9FqNA5/XrM/RJp3XGD0YDvdyNzkVhQ== X-Google-Smtp-Source: AGHT+IHofH6ezuJiRdPzMO354Ass37KI58cj4naD68azjCN0Ux659ZbY8yKyQMYTpWCuImmmQbA2Zg== X-Received: by 2002:adf:f003:0:b0:360:9cd3:c496 with SMTP id ffacd0b85a97d-3609cd3c612mr1455450f8f.54.1718652034323; Mon, 17 Jun 2024 12:20:34 -0700 (PDT) Received: from localhost.localdomain (0540dce9.skybroadband.com. [5.64.220.233]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3607509353csm12531797f8f.22.2024.06.17.12.20.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 12:20:33 -0700 (PDT) From: Derek Buitenhuis To: ffmpeg-devel@ffmpeg.org Date: Mon, 17 Jun 2024 20:20:18 +0100 Message-ID: <20240617192019.512769-5-derek.buitenhuis@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240617192019.512769-1-derek.buitenhuis@gmail.com> References: <20240617192019.512769-1-derek.buitenhuis@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 4/5] avformat/mov: Add support for exporting Video Extension Usage info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: This box is provided by files created by the Apple Vision Pro, as well as the iPhone 15+ when capture for Vision Pro is enabled. The boxes are a mix of things documented by Apple in some PDFs, their API docs, and reverse engineering. Ideally we will have a real spec one day. Links: * https://developer.apple.com/av-foundation/Stereo-Video-ISOBMFF-Extensions.pdf * https://developer.apple.com/documentation/videotoolbox/kvtcompressionpropertykey_horizontaldisparityadjustment * https://developer.apple.com/documentation/videotoolbox/kvtcompressionpropertykey_stereocamerabaseline * https://developer.apple.com/documentation/videotoolbox/kvtcompressionpropertykey_heroeye Signed-off-by: Derek Buitenhuis --- libavformat/mov.c | 283 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 283 insertions(+) diff --git a/libavformat/mov.c b/libavformat/mov.c index 9016cd5ad0..5724b4ef93 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -6477,6 +6477,288 @@ static int mov_read_sv3d(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } +static int mov_read_vexu_proj(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + AVStream *st; + MOVStreamContext *sc; + int size; + uint32_t tag; + enum AVSphericalProjection projection; + + if (c->fc->nb_streams < 1) + return 0; + + st = c->fc->streams[c->fc->nb_streams - 1]; + sc = st->priv_data; + + if (atom.size != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size for proj box: %"PRIu64"\n", atom.size); + return AVERROR_INVALIDDATA; + } + + size = avio_rb32(pb); + if (size != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size for prji box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + tag = avio_rl32(pb); + if (tag != MKTAG('p','r','j','i')) { + av_log(c->fc, AV_LOG_ERROR, "Invalid child box of proj box: 0x%08X\n", tag); + return AVERROR_INVALIDDATA; + } + + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + tag = avio_rl32(pb); + switch (tag) { + case MKTAG('r','e','c','t'): + projection = AV_SPHERICAL_RECTILINEAR; + break; + case MKTAG('e','q','u','i'): + projection = AV_SPHERICAL_EQUIRECTANGULAR; + break; + case MKTAG('h','e','q','u'): + projection = AV_SPHERICAL_HALF_EQUIRECTANGULAR; + break; + case MKTAG('f','i','s','h'): + projection = AV_SPHERICAL_FISHEYE; + break; + default: + av_log(c->fc, AV_LOG_ERROR, "Invalid projection type in prji box: 0x%08X\n", tag); + return AVERROR_INVALIDDATA; + } + + sc->spherical = av_spherical_alloc(&sc->spherical_size); + if (!sc->spherical) + return AVERROR(ENOMEM); + + sc->spherical->projection = projection; + + return 0; +} + +static int mov_read_eyes(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + AVStream *st; + MOVStreamContext *sc; + int size, flags = 0; + int64_t remaining; + uint32_t tag, baseline = 0; + enum AVStereo3DView view = AV_STEREO3D_VIEW_PACKED; + enum AVStereo3DPrimaryEye primary_eye = AV_PRIMARY_EYE_NONE; + AVRational horizontal_disparity_adjustment = { 0, 0 }; + + if (c->fc->nb_streams < 1) + return 0; + + st = c->fc->streams[c->fc->nb_streams - 1]; + sc = st->priv_data; + + remaining = atom.size; + while (remaining > 0) { + size = avio_rb32(pb); + if (size < 8 || size > remaining ) { + av_log(c->fc, AV_LOG_ERROR, "Invalid child size in eyes box\n"); + return AVERROR_INVALIDDATA; + } + + tag = avio_rl32(pb); + switch (tag) { + case MKTAG('s','t','r','i'): { + int has_right, has_left; + uint8_t tmp; + if (size != 13) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of stri box: %d\n", size); + return AVERROR_INVALIDDATA; + } + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + tmp = avio_r8(pb); + + // eye_views_reversed + if (tmp & 8) { + flags |= AV_STEREO3D_FLAG_INVERT; + } + // has_additional_views + if (tmp & 4) { + // skip... + } + + has_right = tmp & 2; // has_right_eye_view + has_left = tmp & 1; // has_left_eye_view + + if (has_left && has_right) + view = AV_STEREO3D_VIEW_PACKED; + else if (has_left) + view = AV_STEREO3D_VIEW_LEFT; + else if (has_right) + view = AV_STEREO3D_VIEW_RIGHT; + break; + } + case MKTAG('h','e','r','o'): { + int tmp; + if (size != 13) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of hero box: %d\n", size); + return AVERROR_INVALIDDATA; + } + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + tmp = avio_r8(pb); + if (tmp == 0) + primary_eye = AV_PRIMARY_EYE_NONE; + else if (tmp == 1) + primary_eye = AV_PRIMARY_EYE_LEFT; + else if (tmp == 2) + primary_eye = AV_PRIMARY_EYE_RIGHT; + else + av_log(c->fc, AV_LOG_WARNING, "Unknown hero eye type: %d\n", tmp); + + break; + } + case MKTAG('c','a','m','s'): { + uint32_t subtag; + int subsize; + if (size != 24) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of cams box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subsize = avio_rb32(pb); + if (subsize != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of blin box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subtag = avio_rl32(pb); + if (subtag != MKTAG('b','l','i','n')) { + av_log(c->fc, AV_LOG_ERROR, "Expected blin box, got 0x%08X\n", subtag); + return AVERROR_INVALIDDATA; + } + + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + baseline = avio_rb32(pb); + + break; + } + case MKTAG('c','m','f','y'): { + uint32_t subtag; + int subsize; + int32_t adjustment; + if (size != 24) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of cmfy box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subsize = avio_rb32(pb); + if (subsize != 16) { + av_log(c->fc, AV_LOG_ERROR, "Invalid size of dadj box: %d\n", size); + return AVERROR_INVALIDDATA; + } + + subtag = avio_rl32(pb); + if (subtag != MKTAG('d','a','d','j')) { + av_log(c->fc, AV_LOG_ERROR, "Expected dadj box, got 0x%08X\n", subtag); + return AVERROR_INVALIDDATA; + } + + avio_skip(pb, 1); // version + avio_skip(pb, 3); // flags + + adjustment = (int32_t) avio_rb32(pb); + + horizontal_disparity_adjustment.num = (int) adjustment; + horizontal_disparity_adjustment.den = 10000; + + break; + } + default: + av_log(c->fc, AV_LOG_WARNING, "Unknown tag in eyes: 0x%08X\n", tag); + avio_skip(pb, size - 8); + break; + } + remaining -= size; + } + + if (remaining != 0) { + av_log(c->fc, AV_LOG_ERROR, "Broken eyes box\n"); + return AVERROR_INVALIDDATA; + } + + if (!sc->stereo3d) { + sc->stereo3d = av_stereo3d_alloc(); + if (!sc->stereo3d) + return AVERROR(ENOMEM); + } + + sc->stereo3d->flags = flags; + sc->stereo3d->view = view; + sc->stereo3d->primary_eye = primary_eye; + sc->stereo3d->baseline = baseline; + sc->stereo3d->horizontal_disparity_adjustment = horizontal_disparity_adjustment; + + return 0; +} + +static int mov_read_vexu(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + int size; + int64_t remaining; + uint32_t tag; + + if (c->fc->nb_streams < 1) + return 0; + + if (atom.size < 8) { + av_log(c->fc, AV_LOG_ERROR, "Empty video extension usage box\n"); + return AVERROR_INVALIDDATA; + } + + remaining = atom.size; + while (remaining > 0) { + size = avio_rb32(pb); + if (size < 8 || size > remaining ) { + av_log(c->fc, AV_LOG_ERROR, "Invalid child size in vexu box\n"); + return AVERROR_INVALIDDATA; + } + + tag = avio_rl32(pb); + switch (tag) { + case MKTAG('p','r','o','j'): { + MOVAtom proj = { tag, size - 8 }; + int ret = mov_read_vexu_proj(c, pb, proj); + if (ret < 0) + return ret; + break; + } + case MKTAG('e','y','e','s'): { + MOVAtom eyes = { tag, size - 8 }; + int ret = mov_read_eyes(c, pb, eyes); + if (ret < 0) + return ret; + break; + } + default: + av_log(c->fc, AV_LOG_WARNING, "Unknown tag in vexu: 0x%08X\n", tag); + avio_skip(pb, size - 8); + break; + } + remaining -= size; + } + + if (remaining != 0) { + av_log(c->fc, AV_LOG_ERROR, "Broken vexu box\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + static int mov_parse_uuid_spherical(MOVStreamContext *sc, AVIOContext *pb, size_t len) { int ret = 0; @@ -8595,6 +8877,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('d','f','L','a'), mov_read_dfla }, { MKTAG('s','t','3','d'), mov_read_st3d }, /* stereoscopic 3D video box */ { MKTAG('s','v','3','d'), mov_read_sv3d }, /* spherical video box */ +{ MKTAG('v','e','x','u'), mov_read_vexu }, /* video extension usage */ { MKTAG('d','O','p','s'), mov_read_dops }, { MKTAG('d','m','l','p'), mov_read_dmlp }, { MKTAG('S','m','D','m'), mov_read_smdm }, -- 2.43.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".