Linux Test Project Ltp Versions Save

Linux Test Project (mailing list: https://lists.linux.it/listinfo/ltp)

20240129

3 months ago

NOTABLE CHANGES

New tests
- splice07, accept03, readahead01:
  
  Tests that feeds the syscall all kinds of invalid file descriptors and checks that the syscall fails properly.
- proc_sched_rt01 a regression test for:
  
  c1fc6484e1fb ("sched/rt: sysctl_sched_rr_timeslice show default timeslice after reset") 079be8fc6309 ("sched/rt: Disallow writing invalid values to sched_rt_period_us")
- ksm07 a test for KSM smart-scan.
- pathconf02 negative testcases for pathconf()
- pipe15 A regresson test for:
  
  46c4c9d1beb7 ("pipe: increase minimum default pipe size to 2 pages")
- nft02 A regression test for:
  
  515ad530795c ("netfilter: nf_tables: do not ignore genmask when looking up chain by id") aka CVE-2023-31248
- sched_setscheduler04 a test for SCHED_RESET_ON_FORK
- setsockopt10 A regression test for: 2c02d41d71f9 ("net/ulp: prevent ULP without clone op from entering the LISTEN status") aka CVE-2023-0461
- gettid02 A basic gettid02 test.
- splice06 A test that splicing from a /proc files works.
Increased coverage
- ioctl02 cover more modern termios ioctls() as well.
- fanotify13 More test coverage for overlayfs.
- input* test now check UI_GET_NAME ioctl() too.
Removed tests
- fork12 Which was a naive fork bomb test, we do have other tests (setrlimit01.c, ...) that better cover this scenario.
- mongo test framework
  
  Which was test reiser, ext2 and jfs unmaintained for a decade.
- simple_tracer.c Tracing test from 2009 that does not compile anymore.
- runtest/fsx-linux This is consolidation of runtest files, since these test duplicated in ltp-aiodio.part3.
- testscripts/autofs{1,4}.sh, testscripts/sysfs.sh, testscripts/ltp-scsi_debug.sh, testscripts/ltpdmmapper.sh
  
  These were unmaintained and unused scripts.
- scsi/* A testsuite with a custom kernel module for kernel 2.5 that didn't compile cleanly for decades.
- load_stress_all_kernel_modules.sh Broken for quite some time.
The LTP library now has a functions to iterate over different file descriptors. That allows us to easily implement more comprehensive tests that feed various syscalls all possible file descriptors and check that the syscall fails properly with invalid combinations. Most notable use of this library is the newly introduced splice07.c test.
The minimal size of the device for a few filesystem tests was increased to 1GB because modern filesystems, most notably Btrfs does not work properly on smaller devices.
LTP now tests bcachefs if kernel support and bcache.mkfs is present
30 testcases were converted to the new test library

The usual amount of fixes and cleanups

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

The performance tests in net.features can be now skipped by setting LTP_NET_FEATURES_IGNORE_PERFORMANCE_FAILURE=1 which is useful when testing is executed on VMs on a overloaded host or if we are running tests with a background load.

KIRK (previously RUNLTP-NG)

Kirk was updated to v1.2

This version brings the following updates:

show both stdout and stderr when executing tests on host
support for external commands on different SUTs
warning message when SUT doesn't support parallel execution
more stable epoll() communication with LTX
minor fixes
updated documentation

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20240129

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/Build-System

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20230929..

147  Petr Vorel <[email protected]>
25  Yang Xu <[email protected]>
21  Andrea Cervesato <[email protected]>
18  Martin Doucha <[email protected]>
12  Cyril Hrubis <[email protected]>
11  Marius Kittler <[email protected]>
 8  Wei Gao <[email protected]>
 7  Amir Goldstein <[email protected]>
 7  Richard Palethorpe <[email protected]>
 6  Avinesh Kumar <[email protected]>
 6  Edward Liaw <[email protected]>
 6  Jan Kara <[email protected]>
 5  Jan Stancek <[email protected]>
 4  Hongchen Zhang <[email protected]>
 4  Kevin Brodsky <[email protected]>
 4  Xiao Yang <[email protected]>
 3  Shiyang Ruan <[email protected]>
 2  Li Wang <[email protected]>
 2  Li Zhijian <[email protected]>
 2  Pengfei Xu <[email protected]>
 2  Stefan Roesch <[email protected]>
 1  Alexander Kanavin <[email protected]>
 1  Brennan Ashton <[email protected]>
 1  Korobeynikov Gleb <[email protected]>
 1  Marcos Paulo de Souza <[email protected]>
 1  Mengchi Cheng <[email protected]>
 1  Murphy Zhou <[email protected]>
 1  Shizhao Chen <[email protected]>
 1  Shoukui Zhang <[email protected]>
 1  Subramanya Swamy <[email protected]>
 1  haopengxiang <[email protected]>
 1  nietingting <[email protected]>
 1  wangxuewen <[email protected]>
 1  ybonatakis <[email protected]>

And also thanks to patch reviewers:

111 Petr Vorel <[email protected]>
 70 Cyril Hrubis <[email protected]>
 47 Li Wang <[email protected]>
 35 Richard Palethorpe <[email protected]>
 14 Martin Doucha <[email protected]>
  8 Amir Goldstein <[email protected]>
  7 Xiao Yang <[email protected]>
  7 Marius Kittler <[email protected]>
  6 Jan Kara <[email protected]>
  4 Avinesh Kumar <[email protected]>
  3 Andrea Cervesato <[email protected]>
  3 Jan Stancek <[email protected]>
  2 Yang Xu <[email protected]>
  1 Petr Vorel <[email protected]>
  1 Christian Brauner <[email protected]>

20230929

7 months ago

NOTABLE CHANGES

New tests
- epoll_wait07 for EPOLLONESHOT
- epoll_wait06 for EPOLLET
- epoll_wait05 for EPOLLRDHUP
- process_madvise01 for MADV_PAGEOUT
- faccessat201 basic faccessat2() test
- faccessat202 checks that the syscall fails correctly
- tcindex01 aka CVE-2023-1829
- mremap06 reproducer for 7e7757876f25 ("mm/mremap: fix vm_pgoff in vma_merge() case 3")
- starvation a simple test case for sched starvation https://lwn.net/ml/linux-kernel/[email protected]/
- pipe14 for end-of-file (read() returns 0) in a case the write end is closed
- statx12 basic test for STATX_ATTR_MOUNT_ROOT
- kvm_svm03 KVM test for CPU lockup through malicous SVM guest
- kvm_svm02 test that KVM correctly intercepts VMSAVE and VMLOAD instructions in a nested virtual machine aka CVE-2021-3656
- statx11 a basic test for STATX_DIOALIGN on block device
- statx10 a basic test for STATX_DIOALIGN on regular file
Increased coverage
- fanotify13 added test variant for overalfs upper fs
- mmap05 increased coverage for EINVAL
- mmap06 increased coverage for EACCES
- bind03 now includes regression test for c0c3b8d380a8 ("unix_bind_bsd(): unlink if we fail after successful mknod")
- stack_clash added test for mmap() minding gap
ebizzy benchmark counter increment races and overflows were fixed
A few more testcases were converted to guarded buffers that is data passed by pointer to kernel are immediatelly following a PROT_NONE page and followed by canaries to catch off-by-one errors
Tests that modprobe kernel modules are now skipped when secure boot is enabled on x86 and ppc64le
ioprio tests now use IOPRIO_PRIO_NUM instead of hardcoded value to check for prio range
35 testcases were converted to the new test library
- we finally got rid of the old and messy libclone library

The usual amount of fixes and cleanups

KIRK (previously RUNLTP-NG)

The new runltp-ng version is now called kirk
- The name was changed since kirk now supports different testsuites as well
- Apart from the name changes the internals were rewritten and cleaned up
- Prototype for parallel testruns is included as well
- https://github.com/linux-test-project/kirk/#readme
There is now experimental LTX support implemented
- Beware this is VERY EXPERIMENTAL at the moment
- LTX is tiny binary that runs on system under test
- LTX communicates with kirk over serial using messages packed in msgpack
- LTX is going to be locked in memory and executed with realtime priority
- LTX allows advanced features such as running tests in parallel
- https://github.com/linux-test-project/ltx

REMOVED TESTS

mmap07 (included in mmap06)
setgroups04 (included in setgroups03)

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/YYYYMMDD

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/Build-System

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

$ git shortlog -s -e -n 20230516..
    71	Petr Vorel <[email protected]>
    32	Yang Xu <[email protected]>
    26	Avinesh Kumar <[email protected]>
    22	Andrea Cervesato <[email protected]>
    22	Martin Doucha <[email protected]>
    13	Cyril Hrubis <[email protected]>
    13	Li Wang <[email protected]>
     6	Wei Gao <[email protected]>
     5	Souta Kawahara <[email protected]>
     4	Amir Goldstein <[email protected]>
     4	Marius Kittler <[email protected]>
     3	Richard Palethorpe <[email protected]>
     3	Ci Zhou <[email protected]>
     2	Damien Le Moal <[email protected]>
     2	Jan Kara <[email protected]>
     2	Konstantin Khorenko <[email protected]>
     2	Min Li <[email protected]>
     2	Murphy Zhou <[email protected]>
     2	Vishal Chourasia <[email protected]>
     1	Ashwin Dayanand Kamat <[email protected]>
     1	Duncan.chu <[email protected]>
     1	Dylan Jhong <[email protected]>
     1	Edward Liaw <[email protected]>
     1	Hongchen Zhang <[email protected]>
     1	Jeff Layton <[email protected]>
     1	Leo Yu-Chi Liang <[email protected]>
     1	Nageswara R Sastry <[email protected]>
     1	Randolph <[email protected]>
     1	Rick Edgecombe <[email protected]>
     1	Shizhao Chen <[email protected]>
     1	Takuya Wakazono <[email protected]>
     1	Zachary Leaf <[email protected]>
     1	Li Qiang <[email protected]>
     1	Ioannis Bonatakis <[email protected]>

And also thanks to patch reviewers:

$ git log 20230516.. | grep -Ei '(reviewed|acked)-by:' | sed 's/.*by: //' | sort | uniq -c | sort -n -r
    104 Cyril Hrubis <[email protected]>
     66 Petr Vorel <[email protected]>
     46 Li Wang <[email protected]>
     15 Martin Doucha <[email protected]>
     14 Avinesh Kumar <[email protected]>
     14 Richard Palethorpe <[email protected]>
      8 Xiao Yang <[email protected]>
      3 Niklas Cassel <[email protected]>
      3 Marius Kittler <[email protected]>
      3 Damien Le Moal <[email protected]>
      2 Srikar Dronamraju <[email protected]>
      2 Shrikanth Hegde <[email protected]>
      2 Linus Walleij <[email protected]>
      2 Amir Goldstein <[email protected]>
      1 Yang Xu <[email protected]>
      1 Vlastimil Babka <[email protected]>
      1 Murphy Zhou <[email protected]>
      1 Jeff Moyer <[email protected]>
      1 Andrea Cervesato <[email protected]>

20230516

1 year ago

NOTABLE CHANGES

New tests
- kvm_svm01 aka CVE-2021-3653
- cgroup_core03 test case for the cgroup kill
- hugetlb32 regression test for:
  - ba9c1201beaa (mm/hugetlb: clear compound_nr before freeing gigantic pages)
  - a01f43901cfb (hugetlb: be sure to free demoted CMA pages to CMA)
- mmap20 test for mmap() with MAP_SHARED_VALIDATE
- mqnotify03 aka CVE-2021-38604
- mprotect05 regression test for:
  - 2fcd07b7ccd5 (mm/mprotect: Fix successful vma_merge() of next in do_mprotect_pkey())
- fsconfig03 aka CVE-2022-0185
- madvise11 regression test for:
  - d4ae9916ea29 (mm: soft-offline: close the race against page allocation)
- mount07 test case for MS_NOSYMFOLLOW
Increased coverage
- NFS tests now run on btrfs, ext4 and xfs
- setpgid were rewritten and increased coverage in a few corner cases
- ioctl01 added more termios coverage
fs_fill test was fixed on 256+ CPUs
loongarch support was added

The usual amount of fixes and cleanups

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

NFS tests now run on btrfs, ext4, xfs (might be extended to more filesystems later)
nfs08.sh a new test for NFS cache invalidation

RUNLTP-NG

The new runltp-ng was finally included in the previous release tarball
- https://github.com/linux-test-project/runltp-ng/#readme
- https://www.youtube.com/watch?v=JMeJBt3S7B0
There were no changes done in this release, however parallel text execution is being worked on, possibly there would be preview ready for the next release

REMOVED TESTS

tomoyo testsuite which is maintained elsewhere by tomoyo devs
execltp script which was unmaintained for years

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20230516

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/Build-System

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20230127..

58  Petr Vorel <[email protected]>
20  Andrea Cervesato via ltp <[email protected]>
19  Yang Xu <[email protected]>
15  Martin Doucha <[email protected]>
11  Andrea Cervesato <[email protected]>
11  Edward Liaw <[email protected]>
10  Wei Gao <[email protected]>
 8  Avinesh Kumar <[email protected]>
 5  Cyril Hrubis <[email protected]>
 5  Teo Couprie Diaz <[email protected]>
 4  Li Wang <[email protected]>
 3  Hao Ge <[email protected]>
 3  Ping Fang <[email protected]>
 3  Richard Palethorpe <[email protected]>
 3  Tarun Sahu <[email protected]>
 2  Hao Zeng <[email protected]>
 2  Leo Yu-Chi Liang <[email protected]>
 2  Sowmya Indranna <[email protected]>
 1  Andrei Gherzan <[email protected]>
 1  Ashwin Dayanand Kamat via ltp <[email protected]>
 1  David Hildenbrand <[email protected]>
 1  Enze Li <[email protected]>
 1  Fabrice Fontaine <[email protected]>
 1  Frank He <[email protected]>
 1  Hui Min Mina Chou <[email protected]>
 1  Jan Stancek <[email protected]>
 1  Liam R. Howlett <[email protected]>
 1  Mahesh Kumar G <[email protected]>
 1  Nikita Yushchenko <[email protected]>
 1  Paulson Raja L <[email protected]>
 1  Seth Forshee (DigitalOcean) <[email protected]>
 1  Vignesh Raman <[email protected]>
 1  William Roche <[email protected]>

And also thanks to patch reviewers:

 68 Cyril Hrubis <[email protected]>
 60 Petr Vorel <[email protected]>
 36 Richard Palethorpe <[email protected]>
 35 Li Wang <[email protected]>
  7 Jan Stancek <[email protected]>
  4 Avinesh Kumar <[email protected]>
  3 Yang Xu <[email protected]>
  2 Andrea Cervesato <[email protected]>
  1 Tetsuo Handa <[email protected]>

20230127

1 year ago

NOTABLE CHANGES

New tests
- 31 hugetlb tests were cleaned up and ported from libhugetlbfs
- test for cve-2022-4378 /proc/sys/ files overflow on writing
- cgroup_core02 regression test for e57457641613 aka CVE-2021-4197
- dirtyc0w_shmem aka CVE-2022-2590
- openat04 regression test for ac6800e279a2
- nice05 basic functional test for nice()
- madvise03 new test for MADV_DONTNEED
- prctl10 basic test for PR_SET/GET_TSC
Increased coverage
- mount03 add tests for MS_STRICTATIME, MS_NODIRATIME validate f_flags
- getitimer01 add check for nonzero timer
- setitimer01 add interval timer test
- getitimer02 add ITIMER_VIRTUAL timer error check
- madvise01 add tests for MADV_COLD and MADV_PAGEOUT
The minimal supported kernel version was raised to 3.10
- LTP is no longer tested on older kernels
- various checks for old kernels were removed from the test code
The AIO test cleanup was finished with rewrite of aiocp and aio-stress
LTP is now compiled with explicit -std=gnu99 which forces reasonably modern C even on oldest distributions we still have to support
LTP build system now forces LC_{COLLATE, NUMERIC}=C in order to have reproducible builds
31 tests were cleaned up and converted for the new test API

The usual amount of fixes and cleanups

RUNLTP-NG

The new runltp-ng is finally included in the released tarball
- https://github.com/linux-test-project/runltp-ng/#readme
- https://www.youtube.com/watch?v=JMeJBt3S7B0

REMOVED TESTS

fs-bench which was broken beyond repair and not worth of keeping

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20230127

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/BuildSystem

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20220930..

50  Petr Vorel <[email protected]>
34  Yang Xu <[email protected]>
33  Tarun Sahu <[email protected]>
28  Avinesh Kumar <[email protected]>
21  Richard Palethorpe <[email protected]>
13  Martin Doucha <[email protected]>
15  Andrea Cervesato <[email protected]>
 9  Li Wang <[email protected]>
 9  Zhao Gongyi <[email protected]>
 6  Akihiko Odaki <[email protected]>
 6  David Hildenbrand <[email protected]>
 6  Jan Stancek <[email protected]>
 6  Tudor Cretu <[email protected]>
 6  Luo xiaoyu <[email protected]>
 4  Wei Gao <[email protected]>
 3  Edward Liaw via ltp <[email protected]>
 3  Jan Kara <[email protected]>
 2  Alessandro Carminati <[email protected]>
 2  Alex <[email protected]>
 2  Cyril Hrubis <[email protected]>
 2  Pengfei Xu <[email protected]>
 2  Rong Tao <[email protected]>
 2  Teo Couprie Diaz <[email protected]>     1  He Zhe <[email protected]>
 1  Huangjie Liao <[email protected]>
 1  Ilya Kurdyukov <[email protected]>
 1  Jie2x Zhou <[email protected]>
 1  Joerg Vehlow <[email protected]>
 1  Khem Raj <[email protected]>
 1  Murphy Zhou <[email protected]>
 1  Nikolaus Voss <[email protected]>
 1  Randolph <[email protected]>
 1  Sven Schnelle <[email protected]>
 1  Vitaly Chikunov <[email protected]>
 1  [email protected] <[email protected]>
 1  wangkaiyuan <[email protected]>
 1  zijunzhao <[email protected]>

And also thanks to patch reviewers:

106 Richard Palethorpe <[email protected]>
 70 Petr Vorel <[email protected]>
 47 Cyril Hrubis <[email protected]>
 45 Li Wang <[email protected]>
 14 Yang Xu <[email protected]>
  7 Jan Stancek <[email protected]>
  8 Martin Doucha <[email protected]>
  3 Avinesh Kumar <[email protected]>
  2 Pengfei Xu <[email protected]>
  1 Richard Purdie <[email protected]>
  1 Khem Raj <[email protected]>
  1 Christian Brauner (Microsoft) <[email protected]>
  1 Chang S. Bae <[email protected]>
  1 Andrea Cervesato <[email protected]>
  1 Akihiko Odaki <[email protected]>

20220930

1 year ago

NOTABLE CHANGES

New tests
- bpf_prog07 aka CVE-2022-23222
- bpf_prog06 aka CVE-2021-4204
- dirtypipe aka CVE-2022-0847
- execve06 test for argv[0] handling aka CVE-2021-4034
Increased coverage
- creat09 added test for another variant of the CVE-2018-13405
- fanotify10 test for FAN_MARK_IGNORE with FAN_ONDIR
- fanotify10 test for FAN_MARK_IGNORE with FAN_EVENT_ON_CHILD
- fanotify09, fanotify14 add test cases for FAN_MARK_IGNORE
- fanotify14 add test cases for events not allowed on non-dir
- fanotify09 add test cases for merge of ignore mask
- fanotify23 test for FAN_MARK_EVICTABLE
- fanotify12 test for inotify mask flags IN_ONESHOT and IN_EXCL_UNLINK
- futex_waitv01 test for EAGIN/ETIMEDOUT
- mount_setattr01 basic functional test
Minimal filesystem size was increased from 256MB to 300MB in order to be able to format it with XFS (mkfs.xfs 5.19)
More testcaseses were made runtime aware + runtime fixes
Improvements in the shell test library
- filesystem detection have been improved
- added $TST_SKIP_FILESYSTEMS support to match C API
- added $TST_ALL_FILESYSTEMS support to match C API
Improvements in open posix testsuite build and installation
Shell cgroup testcases were rewritten and cleaned up
46 tests were rewritten to the new library

The usual amount of fixes and cleanups

WHAT IS IN THE QUEUE

Tracking of minimal device sizes per filesystem, which will allow to run test with smaller device sizes for a subset of filesystems

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

Removed old RPC tests: rup01.sh, rusers01.sh
Moved rpc01.sh and rpcinfo01.sh to runtest/net.rpc_tests and deleted runtest/net.rpc
The tst_net.sh library was fixed for cases when IPv6 was disabled
netns tests were rewritten to use tst_net.sh

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20220930

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/BuildSystem

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20220527..

97  Petr Vorel <[email protected]>
50  Avinesh Kumar <[email protected]>
26  Martin Doucha <[email protected]>
22  Amir Goldstein <[email protected]>
19  Luke Nowakowski-Krijger <[email protected]>
12  Cyril Hrubis <[email protected]>
12  Yang Xu <[email protected]>
10  Andrea Cervesato <[email protected]>
10  Li Wang <[email protected]>
 9  Richard Palethorpe <[email protected]>
 7  Jan Stancek <[email protected]>
 6  Andrea Cervesato via ltp <[email protected]>
 6  Edward Liaw <[email protected]>
 5  Joerg Vehlow <[email protected]>
 5  Tudor Cretu <[email protected]>
 5  Zhao Gongyi <[email protected]>
 3  Dylan Jhong <[email protected]>
 2  Andrea Cervesato <[email protected]>
 2  Chen Hanxiao <[email protected]>
 2  Hongchen Zhang <[email protected]>
 2  Rong Tao <[email protected]>
 1  Bogdan Lezhepekov <[email protected]>
 1  Cristian Marussi <[email protected]>
 1  Huangjie Liao <[email protected]>
 1  Jan Kara <[email protected]>
 1  Jun Huang <[email protected]>
 1  Khem Raj <[email protected]>
 1  Leo Yu-Chi Liang <[email protected]>
 1  Luo xiaoyu <[email protected]>
 1  Ross Burton <[email protected]>
 1  Sharan Turlapati <[email protected]>
 1  Xiao Liang <[email protected]>
 1  Xiao Yang <[email protected]>
 1  Zhao Gongyi via ltp <[email protected]>

And also thanks to patch reviewers:

118 Petr Vorel <[email protected]>
114 Cyril Hrubis <[email protected]>
 77 Li Wang <[email protected]>
 65 Richard Palethorpe <[email protected]>
 18 Jan Kara <[email protected]>
 13 Martin Doucha <[email protected]>
  6 Yang Xu <[email protected]>
  5 Matthew Bobrowski <[email protected]>
  4 Amir Goldstein <[email protected]>
  2 Jan Stancek <[email protected]>
  1 Thomas Petazzoni <[email protected]>
  1 Joerg Vehlow <[email protected]>
  1 Christian Brauner (Microsoft) <[email protected]>
  1 Avinesh Kumar <[email protected]>

20220527

1 year ago

NOTABLE CHANGES

New tests
- kvm_pagefault01: aka CVE-2021-38198
- fcntl39: test for DN_RENAME (dnotify)
- io_control01: first cgroup IO controller test
- keyctl09: test encrypted keys with provided decrypted data
- memcontrol03, memcontrol04: tests that cgroup memory is partitioned correctly under pressure
- setsockopt09: regression test for use-after-free in prb_retire_rx_blk_timer_expired()
- setsockopt09: regression test for double free of rx_owner_map aka CVE-2021-22600
- pty07: regression test for use-after-free in vt_ioctl()
- pty06: regression test for VT_DISALLOCATE freeing in-use virtual console
- pidfd_getfd01 basic functional test
- pidfd_getfd02 basic error test
- pidfd_open04 test with PIDFD_NONBLOCK flag
- futex_waitv01 error tests
- futex_waitv02 basic functional test with private memory
- futex_waitv03 basic functional test with shared memory
- inotify11: test opening files after receiving IN_DELETE regression test for a37d9a17f099 (fsnotify: invalidate dcache before IN_DELETE event)
- statx09 test for STATX_ATTR_VERITY
- wqueue09 tests WATCH_META_LOSS_NOTIFICATION event
- wqueue08 tests WATCH_META_REMOVAL_NOTIFICATION event
- wqueue07 tests NOTIFY_KEY_SETATTR event
- wqueue06 tests NOTIFY_KEY_CLEARED event
- wqueue05 tests NOTIFY_KEY_INVALIDATED event
- wqueue04 tests NOTIFY_KEY_LINKED event
- wqueue03 tests NOTIFY_KEY_REVOKED event
- wqueue02 tests NOTIFY_KEY_UNLINKED event
- wqueue01 tests NOTIFY_KEY_UPDATED event
Increased coverage
- fanotify14, fanotify16: Add tests for FAN_REPORT_TARGET_FID and FAN_RENAME
- futex_waitv01: Add test verifies EINVAL for invalid nr_futexes
- getcontext01: Test rewritten so that it actually jumps back
- fstat02: Validate st_nlink as well
- sched_get_priority_max01, sched_get_priority_min01: Add missing policies
KVM library was written for LTP
- a simple library that can spin up a VM and execute payload
- see also https://github.com/linux-test-project/ltp/wiki/KVM-Test-API
The concept of test runtime was introduced
- for details see https://people.kernel.org/metan/test-timeout-and-runtime
Removed tests
- syslog tests as these were testing mostly userspace syslog daemon and were broken for most of the syslog daemon implementations
The test library was fixed to terminate all leftover processes in the case that the main test process dies or exits
New sparse check for {} terminator for struct arrays in tst_test has been implemented
Shell library improvements
- added support for $TST_FORMAT_DEVICE (.format_device in C API)
- preparations for shell API get support for testing on all filesystems (.all_filesystems in C API)
- getopts cleanup and changes how is the library sources, which resulted in much simpler library code
Userns tests were rewritten into the new test API
Mountns tests were rewritten into the new test API
33 tests were rewritten to the new library

The usual amount of fixes and cleanups

WHAT IS IN THE QUEUE

In-flight patches include:

CGroup shell test rewrite that should fix many problems

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

Several fixes (e.g. important nfs_lib: Fix fsid randomisation)
Ongoing discussion where should be SCTP testsuite home

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20220527

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20220121..

72  Petr Vorel <[email protected]>
51  Cyril Hrubis <[email protected]>
42  Andrea Cervesato <[email protected]>
22  Li Wang <[email protected]>
21  Richard Palethorpe <[email protected]>
17  Martin Doucha <[email protected]>
15  Yang Xu <[email protected]>
10  Jan Stancek <[email protected]>
10  Zhao Gongyi <[email protected]>
 7  Amir Goldstein <[email protected]>
 6  Dai Shili <[email protected]>
 6  Bogdan Lezhepekov <[email protected]>
 4  tangmeng <[email protected]>
 2  Chunyu Hu <[email protected]>
 2  Kushal Chand <[email protected]>
 2  xiaoshoukui <[email protected]>
 1  Alexander Ivanov <[email protected]>
 1  Alexander Kanavin <[email protected]>
 1  Andrea Arcangeli <[email protected]>
 1  Edward Liaw via ltp <[email protected]>
 1  Fabrice Fontaine <[email protected]>
 1  Pavel Tikhomirov <[email protected]>
 1  Wang Kunfeng via ltp <[email protected]>
 1  Yael Tzur via ltp <[email protected]>
 1  ihsinme <[email protected]>

And also thanks to patch reviewers:

148 Cyril Hrubis <[email protected]>
105 Petr Vorel <[email protected]>
 73 Li Wang <[email protected]>
 21 Yang Xu <[email protected]>
 17 Richard Palethorpe <[email protected]>
 10 Martin Doucha <[email protected]>
  6 Jan Stancek <[email protected]>
  1 Waiman Long <[email protected]>
  1 Jan Kara <[email protected]>
  1 Gabriel Krisman Bertazi <[email protected]>
  1 Andrea Cervesato <[email protected]>
  1 Alexey Kodanev <[email protected]>

20220121

2 years ago

Since the last release 269 patches by 32 authors were merged.

Patch review is what most of the projects struggle with and LTP is no different. If you can spare some effort helping with the patch review is more than welcomed.

NOTABLE CHANGES

New tests
- quotactl{08,09} tests for quota info hidden in filesystem.
- fanotify22 test for FAN_FS_ERROR events by intentionally corrupting a fs image
- memcontrol02 test for cgroup memory.current and memory.stat
- memcontrol01 test for cgroup subtree_control and memory controller
- write06 test if write() works correctly with O_APPEND fd
- statx08 statx test for the stx_attributes_mask
- fanotify{20,21} tests for FAN_REPORT_PIDFD
- irqbalance01 test that tries to assert that IRQ are spread between CPUs reasonably
- dup207 tests that file offset is correct for both original and dup()ed fds
- dup206 tests special case where oldfd == newfd
- ima_conditionals.sh test uid, gid, fowner, and fgroup options added in kernel commit 40224c41661b ("ima: add gid support") from v5.16
Increased coverage
- quotactl* tests added support for quotactl_fd and hidden quota files
- quotaclt* tests also test with vfsv1 format
- statx01 added check for stx_mnt_id
- statx01 added check for nlink
- stat01 added check for nlink
- finit_module02 added ETXTBSY check
- epoll_create added two test variants (libc wrapper and raw syscall)
- creat09 runs on all filesystems now
- dup202 added check that file mode is same after it has been changed on dup()ed fd
The test library process, that watches over timeouts and carries on with the test cleanup, now adjusts it's own OOM score so that it's more likely that it outlives the child process that runs the test
First half of the ltp-aiodio testcases was rewritten and cleaned up
- the rewritten tests are free of runtest file hacks that were problematic for years
- the second half should get fixed during the next development cycle
- the rewritten tests so far are:
  - dio_truncate
  - dio_read (previously ltp-diorh)
  - dio_append
  - dio_sparse
  - aiodio_append
39 tests were rewritten to the new library
'make check' improvements
- added check if test.tags is properly NULL terminated
- added check if test global functions and variables are static
- fixed not to report tst_ prefixed static inline functions
The test library gained support for 'known-fail' tag
- to mark kernel bugs that are unfixed for a long time
- this is similar to the existing tags
- the first use of the interface are ustat() tests that are failing on Btrfs ever since the introduction of the filesystem
The test library gained support for version check when a tool is required to be installed on a system, e.g. .needs_cmds = {"mkfs.ext4 >= 1.43.0", NULL} will skip the test unless the mkfs.ext4 is present and the version is at least 1.43.0
The test library added .supported_archs field to the test_test structure which can be used to limit the test to a subset of CPU architectures
The test library added .skip_in_compat flag to skip tests in compat (-m32) mode
The shell library gained support boolean expressions on kernel .config variables
New environment variable KCONFIG_SKIP_CHECK was added that allows users to skip kernel config checks
New environment variable LTP_SINGLE_FS_TYPE was added that can be used to limit tests that run against all filesystem to a single one
The new library testcases now print environment variables in help (-h)
The test metadata are parsed during the build and installed by default now, which is another step towards making the test executor smarter.

The usual amount of fixes and cleanups

WHAT IS IN THE QUEUE

In-flight patchse include:

KVM test infrastructure that would allow us implement KVM regression tests
watchqueue (pipe based kernel notification) testsuite
futex_waitv tests
CGroup testcases and fixes
Test runtime API for the test library that would allow us to controll testrun runtime explicitly and fix a few tests that run into timeouts on older/slower hardware

Unfortunately the runltp-ng is still not included in LTP, we will try to work on it during the next development cycle as well, but we can't promise it will be rock stable to be included in the next release either.

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

NFS tests get support for emulating external server activity
fix: run exportfs at "server side" in LTP_NETNS case
new test: nfs07.sh - NFS directory listing regression

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20220121

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/BuildSystem

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release: git shortlog -s -e -n 20210927..

51  Petr Vorel <[email protected]>
47  Yang Xu <[email protected]>
31  Richard Palethorpe <[email protected]>
24  Joerg Vehlow <[email protected]>
17  Cyril Hrubis <[email protected]>
15  Andrea Cervesato <[email protected]>
14  tangmeng <[email protected]>
 9  Gabriel Krisman Bertazi <[email protected]>
 7  zhanglianjie <[email protected]>
 6  Dai Shili <[email protected]>
 6  Li Wang <[email protected]>
 6  Martin Doucha <[email protected]>
 4  Jan Stancek <[email protected]>
 3  Alex Henrie <[email protected]>
 3  Matthew Bobrowski <[email protected]>
 3  Zhao Gongyi <[email protected]>
 3  sujiaxun <[email protected]>
 2  Krzysztof Kozlowski <[email protected]>
 2  QI Fuli <[email protected]>
 2  Ralph Siemsen <[email protected]>
 2  YiLin.Li <[email protected]>
 2  wenyehai <[email protected]>
 1  Bogdan Lezhepekov <[email protected]>
 1  Egor Petrov <[email protected]>
 1  Fabian Vogt <[email protected]>
 1  Fabrice Fontaine <[email protected]>
 1  Masayoshi Mizuma <[email protected]>
 1  Nikita Yushchenko <[email protected]>
 1  Thadeu Lima de Souza Cascardo <[email protected]>
 1  Wang Kunfeng <[email protected]>
 1  chengweibin <[email protected]>
 1  sujiauxn <[email protected]>

And also thanks to patch reviewers:

154 Cyril Hrubis <[email protected]>
 71 Petr Vorel <[email protected]>
 29 Li Wang <[email protected]>
 24 Richard Palethorpe <[email protected]>
 12 Amir Goldstein <[email protected]>
 11 Yang Xu <[email protected]>
  9 Martin Doucha <[email protected]>
  3 Tim Bird <[email protected]>
  3 Matthew Bobrowski <[email protected]>
  3 Jan Stancek <[email protected]>
  3 Eric Biggers <[email protected]>
  3 Alexey Kodanev <[email protected]>
  2 Liu Xinpeng <[email protected]>
  2 Andrea Cervesato <[email protected]>
  1 Petr Vorel <[email protected]>
  1 Mimi Zohar <[email protected]>

20210927

2 years ago

Since the last release 483 patches by 40 authors were merged.

This is a record breaking release by the number of patches as well as by the amount of newly implemented functionality. Many thanks to all contributors!

And even more thanks to the reviewers!

Patch review is what most of the projects struggle with and LTP is no different. If you can spare some effort helping with the patch review is more than welcomed.

NOTABLE CHANGES

New tests
- epoll_ctl05 checks for ELOOP error for circular loop of epolls
- epoll_ctl04 checks for EINVAL when epoll nesting is too deep
- epoll_ctl02 checks for EPERM when epoll is not supported for fd
- epoll_create01 checks that epoll create returns a valid fd
- epoll_create1_02 checks for EINVAL when epoll_create1() is passed invalid flags
- epoll_pwait02 test if events are reported properly
- epoll_pwait03 tests if epoll_pwait/epoll_pwait2 timeouts correctly
- epoll_pwait04 adds tests for zero timeout and invalid timespec values
- shmget05 test for /proc/sys/kernel/shm_next_id
- shmget06 test when shm_next_id is already in use
- io_getevents02 checks for EINVAL errors via libaio
- io_cancel02 checks for EFAULT errors via libaio
- io_submit03 checks for invalid input via syscall
- io_submit02 checks if correct operation via syscall
- io_setup02 checks for invalid inputs via syscall
- io_destroy02 checks for invalid inputs via syscall
- cfs_bandwidth01 a multi-level cgroup cpu controller test
New regression tests
- creat09 privilege escalation using setgid directory aka CVE-2018-13405
- can_bcm01 race condition vulnerability in CAN BCM aka CVE-2021-3609
- setsockopt08 out-of-bounds write corrupting aka CVE-2021-22555
- perf_event_open03 kernel memory leak aka CVE-2020-25704
- squashfs01 regression test for broken sanity checks
- set_mempolicy05 information leak aka CVE-2017-7616
- userns08 broken id mappings in nested namespaces CVE-2018-18955
- tc01.sh regression test for tc qdisc command
- fanotify19 added regression test for: a8b98c808eab ("fanotify: fix permission model of unprivileged group")
- kill13 INT_MIN negation reproducer aka CVE-2018-10124
- wait403 INT_MIN negation reproducer
- vsock01 race condition reproducer aka CVE-2021-26708
- icmp_rate_limit01 DNS cache poisoning attack aka CVE-2020-25705
Increased coverage
- epoll_create02 checks for more EINVAL failures
- epoll_ctl03 checks that the syscall works with all valid flags combinations
- mbind01 more tests for MPOL_LOCAL
- epoll_pwait01 adds test variants for epoll_pwait2()
75 tests were rewritten to the new library
Removed tests
- sched/stress as it was unmaintained and unused for years
- stress_cd and stress_floppy since these scripts were broken and unmaintained for years
- disktest as it was unmaintained and unused for years
The CI was moved from Travis to GitHub actions
'make check' was introduced to the LTP build system
- which is a tool for developers that runs various sanity checks on the source code
- this includes:
  - checkpatch.pl, custom set of sparse rules for C
  - checkbashism.pl for shell
- please make sure you run 'make check' or 'make check-testname' and fix the warnings before submitting patches
Subset of our library tests i.e. tests for the test library is now executed CI in github actions, this is ongoing effort to make sure that the test library used by the testcases is working as expected
The test library was fixed to print all messages to stderr
Testcases were fixed not to expect certain users and groups to be present on the system
- some tests were expecting bin, daemon and similar groups and users to exist on the system
- now LTP depends only on user 'nobody' and group 'nogroup'
New environment variable LTP_VIRT_OVERRIDE was added
- this is meant as a quick workaround for cases where the test library cannot properly detect if system is virtualized or not
- see https://github.com/linux-test-project/ltp/wiki/User-Guidelines
Fixed (again) timeouts in shell test library
- turns out you cannot make timeouts in shell race-free so the timeout logic is now implemented in a small C helper binary
Finally after the years we are slowly making tests limits-aware
- cgropup tests and a few others are now aware of pid limits imposed by the system
- next step would be doing rough estimates on the number of processes we can run based on the free system memory
There is ongoing effort to fix LTP cgroup tests, this release includes many small fixes and improvements in the cgroup tests
The aarch64 system call table was fixed not to include 32bit only syscall numbers
The TST_EXP_*() macros were improved and a few bugs were fixed
The test library developer documentation and tutorials were reorganized and improved

The more that the usual amount of fixes and cleanups

WHAT IS IN THE QUEUE

Next release will hopefully include runltp-ng which is new and improved LTP testrunner. The current runltp script will be maintained alongside with the new solution for some time but removed after a few releases.

You are more than welcome to try it and make sure that it works for you before it becomes the official LTP testrunner.

See: https://github.com/metan-ucw/runltp-ng

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

C network library
- Added device management helpers
- And documentation: https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API
Shell network library
- Allow to skip initialization with TST_NET_SKIP_VARIABLE_INIT=1
- Disable IPv6 Duplicate Address Detection on link-local addresses
Tests rewritten to the new library
- getaddrinfo01
- ssh-stress
Removed tests
- broken_ip-totlen

More busybox and general fixes

DOWNLOAD AND LINKS

The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20210927

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/BuildSystem

Patches, new tests, bugs, comments or questions should go to to our mailing list at [email protected].

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20210927..

87  Petr Vorel <[email protected]>
51  Xie Ziyao <[email protected]>
51  Richard Palethorpe <[email protected]>
50  Martin Doucha <[email protected]>
30  Krzysztof Kozlowski <[email protected]>
29  Cyril Hrubis <[email protected]>
28  Alexey Kodanev <[email protected]>
27  Yang Xu <[email protected]>
20  Li Wang <[email protected]>
18  zhanglianjie <[email protected]>
17  Joerg Vehlow <[email protected]>
12  Vinay Kumar <[email protected]>
 8  Xie Ziyao <[email protected]>
 7  sujiaxun <[email protected]>
 5  Bogdan Lezhepekov <[email protected]>
 5  Dai Shili <[email protected]>
 5  Jan Stancek <[email protected]>
 5  QI Fuli <[email protected]>
 4  Leo Liang <[email protected]>
 2  Alexander Egorenkov <[email protected]>
 2  Amir Goldstein <[email protected]>
 2  Shiyang Ruan <[email protected]>
 2  Thadeu Lima de Souza Cascardo <[email protected]>
 2  Zhao Gongyi <[email protected]>
 1  Alessio Balsini <[email protected]>
 1  Bogdan Lezhepekov via ltp <[email protected]>
 1  Cao jin <[email protected]>
 1  Egor Petrov <[email protected]>
 1  Ian May <[email protected]>
 1  John Stultz <[email protected]>
 1  Leo Yu-Chi Liang <[email protected]>
 1  Marius Hillenbrand <[email protected]>
 1  Naresh Kamboju <[email protected]>
 1  Radoslav Kolev <[email protected]>
 1  Wang Xin <[email protected]>
 1  Xiao Yang <[email protected]>
 1  Zou Wei <[email protected]>
 1  dongshijiang <[email protected]>

And also thanks to patch reviewers:

289 Cyril Hrubis <[email protected]>
113 Petr Vorel <[email protected]>
 75 Li Wang <[email protected]>
 25 Richard Palethorpe <[email protected]>
 16 Yang Xu <[email protected]>
 12 Alexey Kodanev <[email protected]>
  7 Jan Stancek <[email protected]>
  6 Richard Palethorpe <[email protected]>
  5 Xiao Yang <[email protected]>
  5 Martin Doucha <[email protected]>
  3 Joerg Vehlow <[email protected]>
  3 Enji Cooper <[email protected]>
  2 Krzysztof Kozlowski <[email protected]>
  1 Yang Xu <[email protected]
  1 Xie Ziyao <[email protected]>
  1 Stephen Kitt <[email protected]>
  1 Petr Vorel <[email protected]>
  1 Leo Yu-Chi Liang <[email protected]>
  1 Joerg Vehlow <[email protected]>

20210524

2 years ago

Since the last release 364 patches by 50 authors were merged.

NOTABLE CHANGES

New tests
- fanotify17 - tests for fanotify() user limits
- fanotify18 - tests for validating user permissions
- fanotify19 - tests for unpriviledged events
- brk02 - test that removes two VMAs on shrink
- close_range{01,02} - basic close_range() tests
- rtc02 - /dev/rtc timer tests
New regression tests
- bpf_prog05 truncation on 32bit div/mod by zero aka CVE-2021-3444
- writev03 regression test for d4690f1e1cda 'fix iov_iter_fault_in_readable()'
- io_uring02 regression test for unsafe handling of the root directory aka CVE-2020-29373
Increased coverage
- tmpfs was added for tests that loop over filesystems supported by the SUT
- fanotify05 added more queue overflow tests and test for unlimited queue
52 tests were rewritten to the new library
- with that we reached important milestone where new library testcases outnumber the old library tests for the first time ever
- Converted tests:
  - chdir01
  - chmod05
  - chown01, chown02, chown03
  - close01, close02
  - get_mempolicy01
  - getgid01, getgid03
  - getpid01, getpid02
  - getppid01, getppid02
  - io_cancel01, io_getevents01
  - mallopt01
  - mmap19
  - netns_netlink
  - pec
  - pidns13
  - reboot01, reboot02
  - sched_getparam01, sched_getparam02, sched_getparam03
  - semctl01, semctl02, semctl03, semctl04, semctl05, semctl07
  - sendfile03, sendfile04, sendfile05, sendfile07
  - setgid01, setgid02, setgid03
  - swapoff01, swapoff02, swapoff03
  - swapon01, swapon02, swapon03
  - swapoff01, swapoff02, swapoff03
  - swapon01, swapon02, swapon03
  - time01
  - times01
  - tkill01, tkill02
  - unlinkat01
  - unshare01, unshare02
  - wait01, wait02
  - CAN tests were cleaned up and converted to the new library
Integrity Measurement Architecture (IMA)
- new SELinux measurement tests
- fixes
Added new library for managing network interfaces over netlink
Added new library for cgroup discovery and management
- this is a big leap forward for cgroup testing
- greatly simplifies testscases as the library gracefully handles v1/v2 split configurations that are unfortunatelly still common on recent systems
- first few tests were converted to use the library
- more tests are in the queue
Fixed timeouts in shell test library
- ensure that all child processes are killed on timeout as well
- runs a cleanup after timeout
The fuzzy sync library was fixed to yield() on single CPU systems
- which fixed test timeouts on single CPU systems
- self tests for the library where implemented where we check that races we crafted into the code are triggered correctly by the library even on single CPU systems
Fixed kernel module detection on BusyBox
Open Posix Testsuite now use $TMPDIR instead of previously hardcoded /tmp
LTP should now produce reproducible build after a few fixes to the buildsystem and libs
The documentation generated from the test sources was improved, you can check out the generated documentation pages for this release at:

http://linux-test-project.github.io/metadata/metadata.asciidoctor.html

Also have a look at a few nice test documentation examples:

http://linux-test-project.github.io/metadata/metadata.asciidoctor.html#acct02 http://linux-test-project.github.io/metadata/metadata.asciidoctor.html#bpf_prog05 http://linux-test-project.github.io/metadata/metadata.asciidoctor.html#semctl09
We also wrote down a document about the oldest supported kernel and libc, see: https://github.com/linux-test-project/ltp/wiki/Supported-kernel,-libc,-toolchain-versions

The usual amount of fixes and cleanups

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

New tests: UDP encapsulation tests (FOU and GUE)
Tests converted to new API: CAN tests, multicast stress tests, route-change
Reduce false positive on tests using tst_netload

Other fixes

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20210121..

84  Petr Vorel <[email protected]>
35  Zhao Gongyi <[email protected]>
35  Richard Palethorpe <[email protected]>
26  Xie Ziyao <[email protected]>
22  Martin Doucha <[email protected]>
20  Cyril Hrubis <[email protected]>
15  Joerg Vehlow <[email protected]>
13  Li Wang <[email protected]>
10  Xiao Yang <[email protected]>
 8  Amir Goldstein <[email protected]>
 7  QI Fuli <[email protected]>
 7  Yang Xu <[email protected]>
 6  Feiyu Zhu <[email protected]>
 5  Alexey Kodanev <[email protected]>
 5  Richard Purdie <[email protected]>
 5  Yang Xu <[email protected]>
 4  Jan Stancek <[email protected]>
 4  Shiyang Ruan <[email protected]>
 4  zhanglianjie <[email protected]>
 4  zhao gongyi <[email protected]>
 3  Cixi Geng <[email protected]>
 3  Kory Maincent <[email protected]>
 3  Lakshmi Ramasubramanian <[email protected]>
 3  Shiyang Ruan <[email protected]>
 3  Wang Xin <[email protected]>
 2  Palmer Dabbelt via ltp <[email protected]>
 2  Petr Vorel <[email protected]>
 2  Thadeu Lima de Souza Cascardo <[email protected]>
 2  Xiao Yang <[email protected]>
 2  Zenghui Gao <[email protected]>
 1  Alexander Egorenkov <[email protected]>
 1  Ankit Jain <[email protected]>
 1  Bogdan Lezhepekov <[email protected]>
 1  Colin Ian King <[email protected]>
 1  Cristian Marussi <[email protected]>
 1  He Zhe <[email protected]>
 1  Jinghui Huang <[email protected]>
 1  Jinhui Huang <[email protected]>
 1  Konstantin Aladyshev <[email protected]>
 1  Leo Yu-Chi Liang <[email protected]>
 1  Liam Howlett <[email protected]>
 1  QI Fuli <[email protected]>
 1  Sharan Turlapati <[email protected]>
 1  Tim Gates <[email protected]>
 1  Yann Sionneau via ltp <[email protected]>
 1  dongshijiang <[email protected]>
 1  ouyangciyan <[email protected]>
 1  sujiaxun <[email protected]>
 1  tianyu <[email protected]>
 1  Tree Davies <[email protected]>

And also thanks to patch reviewers:

151 Cyril Hrubis <[email protected]>
130 Petr Vorel <[email protected]>
 70 Li Wang <[email protected]>
  7 Yang Xu <[email protected]>
  7 Xiao Yang <[email protected]>
  6 Oliver Hartkopp <[email protected]>
  5 Alexey Kodanev <[email protected]>
  4 Yang Xu <[email protected]>
  4 Martin Doucha <[email protected]>
  4 Lakshmi Ramasubramanian <[email protected]>
  4 Jan Stancek <[email protected]>
  3 Xiao Yang <[email protected]>
  2 Viresh Kumar <[email protected]>
  2 Richard Palethorpe <[email protected]>
  2 Leo Yu-Chi Liang <[email protected]>
  2 Jan Kara <[email protected]>
  1 Liam R. Howlett <[email protected]>
  1 Jiri Slaby <[email protected]>
  1 Eric Biggers <[email protected]>

20210121

3 years ago

Since the last release 303 patches by 35 authors were merged.

NOTABLE CHANGES

New tests
- finit_module()
- init_module()
- delete_module()
- semctl09 for SEM_STAT_ANY flag
- msgctl06 for MSG_STAT_ANY flag
- shmctl04 for SHM_STAT_ANY flag
- sendmmsg() failure tests
- recvmmsg() failure tests
- open_by_handle_at()
- name_to_handle_at()
- ptrace11 to check if we can trace PID 1
- select04 failure tests
- select04 to check if rfds/wfds flags are clered on empty/full fd
New regression tests
- inotify10 added test for fecc4559780d (fsnotify: fix events reported to watching parent and child)
- fanotify09 for 7372e79c9eb9 (fanotify: fix logic of reporting name info with watched parent)
- ptrace10 bd14406b78e6 (perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set)
Increased coverage
- fanotify09 added case with FAN_CLOSE_NOWRITE on a sibling non-dir child
Removed tests
- sync01, sync02 these tests were useless so there was no point in keeping them
The metadata extraction patchset was merged. LTP now produces metadata.json with a metadata for new library testcases and also html test catalogue build from the extracted metadata.

https://github.com/linux-test-project/ltp/blob/master/docparse/README.md
Kernel .config parser was rewritten to support proper boolean expressions
LTP now requires pkg-config > 0.23 (working version 0.24 was released in 2010)
Error handling in test library (mostly SAFE_MACROS()) was unified
High level test library overview was written: https://github.com/linux-test-project/ltp/blob/master/lib/README.md
IMA/EVM
- fixed ima_tpm.sh for TPM 2.0 and for TPM 1.2 on kernel >= v5.8 (commit 6f1a1d103b48)
- rewrote ima_boot_aggregate.c to new API
16 testcases were converted to the new test library

The usual amount of fixes and cleanups.

NOTABLE CHANGES IN NETWORK TESTS

brought to you by Petr Vorel

New tests
- wireguard tests
Fixes
- fix various false-positive failures in tests using tst_netload (tests in net.features and net_stress.ipsec_* runtest files)
- if-mtu-change.sh on s390x: fix max packet size for IPv4
Compatibility fixes:
- ping and traceroute from BusyBox
- MUSL (netstress)
- traceroute6 from iputils
- various other fixes and cleanup
Rewrite into new API
- host01
- netstat01.sh

CREDITS

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20200930..

88  Petr Vorel <[email protected]>
52  Cyril Hrubis <[email protected]>
25  Martin Doucha <[email protected]>
19  Yang Xu <[email protected]>
18  Viresh Kumar <[email protected]>
10  Xiao Yang <[email protected]>
 8  Alexey Kodanev <[email protected]>
 8  Amir Goldstein <[email protected]>
 8  Feiyu Zhu <[email protected]>
 8  Kory Maincent <[email protected]>
 7  Radoslav Kolev <[email protected]>
 5  Richard Palethorpe <[email protected]>
 4  Cixi Geng <[email protected]>
 4  Krzysztof Dynowski <[email protected]>
 4  Li Wang <[email protected]>
 4  Po-Hsu Lin <[email protected]>
 4  Joerg Vehlow <[email protected]>
 3  Alexander Egorenkov <[email protected]>
 2  Khem Raj <[email protected]>
 2  Li Zhijian <[email protected]>
 2  Pengfei Xu <[email protected]>
 2  Tree Davies <[email protected]>
 2  Yang Xu <[email protected]>
 2  Bogdan Lezhepekov <[email protected]>
 2  Johannes Nixdorf <[email protected]>
 1  Alexander Egorenkov <[email protected]>
 1  Deepak Rawat <[email protected]>
 1  Filip Bozuta <[email protected]>
 1  Jan Stancek <[email protected]>
 1  Peter Bee <[email protected]>
 1  Petr Cervinka via ltp <[email protected]>
 1  Punit Agrawal <[email protected]>
 1  Radoslav Kolev via ltp <[email protected]>
 1  Xinpeng Liu <[email protected]>
 1  bhargavdas <[email protected]>

And also thanks to patch reviewers:

114 Cyril Hrubis <[email protected]>
 58 Petr Vorel <[email protected]>
 51 Li Wang <[email protected]>
 13 Yang Xu <[email protected]>
 11 Amir Goldstein <[email protected]>
 10 Alexey Kodanev <[email protected]>
  9 Jan Stancek <[email protected]>
  5 Xiao Yang <[email protected]>
  5 Jan Kara <[email protected]>
  3 Xiao Yang <[email protected]>
  5 Richard Palethorpe <[email protected]>
  3 Martin Doucha <[email protected]>
  2 Mimi Zohar <[email protected]>
  1 Kory Maincent <[email protected]>
  1 Joerg Vehlow <[email protected]>