Commit Graph

245 Commits

Author SHA1 Message Date
Zhao Zhili
69c79bb32a tests/fate-run: Fix pixdesc failure
-u and -q doesn't work together for diff on macOS.

Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-10-31 08:23:44 +01:00
James Almer
f67c76aaee fate/filter-video: make fate-filter-pixdesc compare the hashed output with and without pixdesctest filtering
This helps detecting inconsistencies in swscale input/output code and
a av_read_image_line()/av_write_image_line() pass.

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-10-22 08:53:05 +02:00
Niklas Haas
cadf46878e tests/fate: force MPEG range for rawvideo tests
The input file is MPEG range, so we should also encode to MPEG range
before comparing against it. This bug was avoided in the past because
YUVJ inputs were automatically converted back to limited range when converting
to a different pixfmt (in the absence of tagging). However, with proper YUV
negotiation in place, the default behavior is to preserve the YUV range
wherever possible. Since `rawvideo` does not signal or care about the YUV
range, we need to explicitly request the desired output range to force a
conversion.

Affects all rawvideo tests, in particular vsynth and owdenoise.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-09-13 08:39:49 +02:00
J. Dekker
f872413b51 configure, etc: switch to shebang without space
Note that the config.sh file is left without a shebang, this file is
supposed to be sourced into the current environment.

This commit is purely cosmetic.

Signed-off-by: J. Dekker <jdek@itanimul.li>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-04-10 01:21:07 +02:00
Nicolas Gaullier
0c62ecc102 fate: fix generating references when sh=dash
Regression since 0b98f28c46a7e3e914c51debc461

Signed-off-by: Nicolas Gaullier <nicolas.gaullier@cji.paris>
Signed-off-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-03-17 19:46:42 +01:00
James Almer
38e8eb284c fate/flvenc: force the av1 decoder in all steps for fate-enhanced-flv-av1
Tested-by: Steven Liu <lingjiujianke@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-03-17 19:46:35 +01:00
James Almer
abe38f96a8 fate/mov: print stream group information for avif/heic tests
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-03-17 19:41:02 +01:00
Marton Balint
d9796bd422 fate: never generate auto-generated references
GEN=1 is used to generate reference files in the source tree, not
auto-generated reference samples.

Without this patch GEN=1 could overwrite the auto generated reference files
in each test where they are used causing failures.

Signed-off-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-03-03 00:39:00 +01:00
Andreas Rheinhardt
eabfb18449 tests/fate-run: Do not ignore errors from intermediate commands
Otherwise the test may pass while ignoring errors from sanitizers.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-02-24 15:27:44 +01:00
Andreas Rheinhardt
f87f5aa83c tests/fate-run: Sanitize test argument
test -z is a binary operator.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2024-02-11 12:27:31 +01:00
Andreas Rheinhardt
8d5db120c2 tests/fate-run: Ensure that THREADS=random is actually random
From the documentation of GNU awk [1]:
"In most awk implementations, including gawk, rand() starts generating
numbers from the same starting number, or seed, each time you run awk.45
Thus, a program generates the same results each time you run it. The
numbers are random within one awk run but predictable from run to run.
This is convenient for debugging, but if you want a program to do
different things each time it is used, you must change the seed to a
value that is different in each run. To do this, use srand()."

This commit does exactly this.

[1]: https://www.gnu.org/software/gawk/manual/html_node/Numeric-Functions.html#index-rand_0028_0029-function

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2023-09-25 22:02:24 +02:00
James Almer
fb548fba04 avcodec/av1dec: export pixel format even if no hardware decoder is present
And remove the AVOID_PROBING flag, given it's the last av1 decoder to be tested
either way.
This fixes a regression introduced in 1652f2492f,
where even if forcing the native av1 decoder, if another decoder was present,
like libdav1d or libaom-av1, they'd be used for probing and some fate tests
would have different results.

Signed-off-by: James Almer <jamrial@gmail.com>
2023-09-08 09:42:25 -03:00
Anton Khirnov
1b45405d44 tests/fate-run: add testing with a random number of threads
Useful for discovering bugs that depend on a specific thread count.

Use like THREADS=randomX for a random thread count from 1 to X, with
X=16 when not specified. Note that the thread count is different for
every test.
2023-07-11 19:19:32 +02:00
Anton Khirnov
2953ebe7b6 tests/fate: add a simple test for libx264
Since this is an external encoder not under our control, we cannot test
the encoded output exactly as is done for internal encoders. We can
still test however that the output is decodable and produces the
expected number of frames with expected dimensions, pixel formats, and
timestamps.
2023-05-15 10:39:02 +02:00
Andreas Rheinhardt
a5ab4be081 tests/fate-run: Allow to set input options for encoding pass
This will be useful in the next commit.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-09-10 01:38:07 +02:00
Andreas Rheinhardt
4fb8741c46 tests/fate-run: Allow to skip file checksums for lavf_image
The output file (even the filesize) of the recently added
EXR tests depends on the endianness; therefore checksums
of these files must not be part of the ref file. Therefore
this commit adds an option (unused for now) to disable these
checksums on a per-test basis.

In order to avoid having to check twice, the checksum and
the filesize info are moved to immediately follow one another;
this results into updates to the ref files of all lavf-image tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-02 02:27:05 +02:00
Anton Khirnov
60f948dd55 tests/fate/vcodec: add tests for ffv1 2pass mode 2022-05-24 13:35:16 +02:00
Anton Khirnov
83560e48f6 tests/fate-run: give consistent names to enc_dec() arguments
enc_dec() performs two ffmpeg runs - the first one encoding a source
file into a specified output format, the second one decoding previously
encoded file.

The arguments to this function currently have confusing names - e.g.
dec_opt contains _output_ (i.e. encoding) options for the second
(decoding) ffmpeg invocation. It is also possible to supply _input_
(i.e. decoding) options for the second ffmpeg run, but the argument
is currently unnamed and referred to by number.

Add an _in/_out suffix to argument names to make it clear what they are
used for. Give a name to input options for the decoding ffmpeg run.
2022-05-24 13:35:16 +02:00
Andreas Rheinhardt
06f1e87281 tests/fate-run: Make AVCONV-tests honour -thread_type
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-09 21:23:24 +02:00
Andreas Rheinhardt
aa9ac1ce49 tests/fate-run: Remove temporary files from pixfmt conversions
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-06 05:39:25 +02:00
Andreas Rheinhardt
e4563c2caf tests/fate-run: Remove temporary fate-lavf files if possible
The temporary fate-lavf files can easily be removed
if they are not needed as inputs for other tests (mainly
fate-seek-tests). This commit implements this.
The size of the remaining files decreases from 260890083B
to 79481793B.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-06 05:39:25 +02:00
Andreas Rheinhardt
95cbd97cce tests/Makefile: Redo how to keep intermediate FATE-files
Extend the ordinary mechanism to signal KEEP for this.
This also allows to remove the keep-parameter from enc_dec,
transcode and stream_remux, so that several empty parameters
'""' could be removed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-06 05:39:25 +02:00
Andreas Rheinhardt
c9e7050a3e tests/fate-run: Allow to set dec opts for generated file in transcode
This can be necessary to e.g. force a fixed-point audio codec.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-06 05:39:25 +02:00
Andreas Rheinhardt
bf1337f99c tests/fate-run: Don't create unnecessary CRC-files, forward errors
These CRC-only files (the output of the CRC-muxer) are only used once,
so they need not be preserved. Furthermore, errors from ffmpeg (used
for creating the CRC) are no longer ignored with this patch.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-06 05:39:25 +02:00
Andreas Rheinhardt
bf8411c495 fate/lavf-audio: Disable CRC for lavf-peak_only.wav test
The output of this test is just a file containing the positions
of peaks; it is not a wave file and trying to demux it just
returns AVERROR_INVALIDDATA; said error has just been ignored
as the return value from do_avconv_crc is the return value from echo.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-06 05:39:10 +02:00
Andreas Rheinhardt
0a7939d706 tests/fate-run: Always use -bitexact for ffprobe
Makes no difference currently.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-03 04:18:06 +02:00
Andreas Rheinhardt
26e51325d4 fate/vbn, tests/fate-run: Delete intermediate VBN-files after tests
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-05-03 04:18:05 +02:00
Andreas Rheinhardt
1f34e8963f tests/fate-run: Remove intermediate lavf_container_fate files
They are not used lateron.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-04-28 02:39:37 +02:00
Marton Balint
7ac559b819 fate: add fate tests for VBN encoder and decoder
Signed-off-by: Marton Balint <cus@passwd.hu>
2022-04-10 20:12:23 +02:00
Marton Balint
80e997b081 fate/filter-refcmp-*: make refcmp_metadata fail on empty or truncated input
On empty input the awk script was always successful which caused the
filter-refcmp tests to always succeed.

Also fix the command lines for refcmp_metadata compare function because it
needs auto conversion filters, and update reference of test
filter-refcmp-psnr-rgb because it was missed in
a7fc78c1a6 but was never noticed due to the
original issue...

Signed-off-by: Marton Balint <cus@passwd.hu>
2022-04-10 20:12:23 +02:00
Boris Baracaldo
80831e742b lavfilter: Add SITI filter
Calculate Spatial Info (SI) and Temporal Info (TI) scores for a video, as defined
in ITU-T P.910: Subjective video quality assessment methods for multimedia
applications.
2022-04-01 20:27:43 +02:00
Andreas Rheinhardt
6d72126e9b tests/fate-run: Use DEC_OPTS for additional input, too
Should fix failures in the cover-art-flac-remux test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-10-08 12:06:00 +02:00
Andreas Rheinhardt
0a0204c48a tests/fate-run: Allow multiple inputs for transcode()
This allows nicer tests by having a greater range of inputs available
(without requiring adding further samples to the fate-suite).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-10-07 21:54:30 +02:00
Andreas Rheinhardt
2f4dfe861d tests/fate-run: Set bitexact flag for output, too
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-10-07 11:52:10 +02:00
Anton Khirnov
bdc1bdf3f5 FATE: allow multithreaded filtering 2021-08-29 18:44:38 +02:00
James Almer
44d5e12c8f ffprobe: remove references to frame->pkt_pts
The field was removed during the last major bump.

Signed-off-by: James Almer <jamrial@gmail.com>
2021-08-07 10:16:07 -03:00
Anton Khirnov
af607a8838 fate-run.sh: print stream side data in probeaudiostream()
Adds a test for reading AUDIO_SERVICE_TYPE in fate-hls-fmp4_ac3
2021-05-09 11:07:20 +02:00
Anton Khirnov
b8041fb838 fate-oggopus-demux: convert to ffprobe
It can print side data cleanly.
2021-05-09 10:59:21 +02:00
Andreas Rheinhardt
8de5d196d1 tests/fate-run.sh: Don't overlook errors from md5 tests
The md5 test up until now ignored errors from ffmpeg (the cli) and just
md5'ed whatever ffmpeg has output; while testing scenarios in which
ffmpeg fails has its merits, errors should not be overlooked by default;
doing so also reduces the effectiveness of sanitizers as errors from
them are ignored. This has happened with a memleak in the AV1 decoder.

Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-02-16 22:50:36 +01:00
Andreas Rheinhardt
e575d59afe tests/fate-run: Always overwrite output files for md5 tests
Otherwise the result of such tests will not accurately reflect the
current state.

Reviewed-by: Jan Ekström <jeebjp@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-27 09:51:31 +02:00
Nicolas George
f08e024ac7 fate: disable automatic conversions on many tests.
Explicitly insert the scale or aresample filter where it would
have been inserted by the negotiation.
Re-enable conversions if it cannot be done easily.

If a conversion is needed in a test, we want to know about it.
If the negotiation changes and makes new conversion necessary,
we want to know about it even more.
2020-09-08 14:16:08 +02:00
Nicolas George
7969551a22 fate: remove "-v 0" from ffprobe tests. 2020-08-24 14:30:18 +02:00
Steven Liu
9ee010e734 tests/fate/hlsenc: add testcase of ac3 surround sound input in hlsenc
add probeaudiostream for get audio stream's codec_name,codec_time_base,
sample_fmt,channels and channel_layout.

Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
2020-07-07 14:32:07 +08:00
Anton Khirnov
ea980d4162 fate: add tests for h264 and vp9 video enc parameters export 2020-05-25 11:59:45 +02:00
Andreas Rheinhardt
277306f54b fate: Allow to test temporary files with ffprobe
This is primarily intended to test that muxers correctly write chapters
or metadata; but given that it does this by having our demuxers read the
generated files, it also tests demuxers. And of course it may prove
useful for encoders, too.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-19 00:13:40 +02:00
Andreas Rheinhardt
c4c96d5942 fate: Don't use depreceated keepside option
The tests for concat use this option which is scheduled for removal and
does nothing any more. So remove it; otherwise, these tests would fail
at the next major version bump.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2020-01-10 12:28:18 -03:00
Martin Storsjö
e9bb1410e4 fate: Fix use of target_path/target_samples
Signed-off-by: Martin Storsjö <martin@martin.st>
2019-12-12 11:28:21 +02:00
Andrey Semashev
6d9d053edb tests: Fix bash errors in lavf_container tests.
Because the lavf_container is sometimes called with only 2 arguments,
fate tests produce bash errors like this:

  tests/fate-run.sh: 299: test: =: unexpected operator

This commit fixes this.

Reviewed-by: Limin Wang <lance.lmwang@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-09-09 18:16:06 +02:00
Carl Eugen Hoyos
d0ca749adb tests/fate-run: New variable hostexecsuf for local fate tools.
Allows running fate in-tree on wsl with msvc.
2019-04-24 01:42:56 +02:00
Carl Eugen Hoyos
96fc0cbfde tests: Add EXESUF to program calls.
Fixes fate in Windows subsystem for Linux.
2019-04-19 01:11:39 +02:00