forge/linux-uconsole
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions
linux-uconsole/fs
History
Sean Purcell 690646e829 UPSTREAM: squashfs: Add zstd support 
Add zstd compression and decompression support to SquashFS. zstd is a
great fit for SquashFS because it can compress at ratios approaching xz,
while decompressing twice as fast as zlib. For SquashFS in particular,
it can decompress as fast as lzo and lz4. It also has the flexibility
to turn down the compression ratio for faster compression times.

The compression benchmark is run on the file tree from the SquashFS archive
found in ubuntu-16.10-desktop-amd64.iso [1]. It uses `mksquashfs` with the
default block size (128 KB) and and various compression algorithms/levels.
xz and zstd are also benchmarked with 256 KB blocks. The decompression
benchmark times how long it takes to `tar` the file tree into `/dev/null`.
See the benchmark file in the upstream zstd source repository located under
`contrib/linux-kernel/squashfs-benchmark.sh` [2] for details.

I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD.

| Method         | Ratio | Compression MB/s | Decompression MB/s |
|----------------|-------|------------------|--------------------|
| gzip           |  2.92 |               15 |                128 |
| lzo            |  2.64 |              9.5 |                217 |
| lz4            |  2.12 |               94 |                218 |
| xz             |  3.43 |              5.5 |                 35 |
| xz 256 KB      |  3.53 |              5.4 |                 40 |
| zstd 1         |  2.71 |               96 |                210 |
| zstd 5         |  2.93 |               69 |                198 |
| zstd 10        |  3.01 |               41 |                225 |
| zstd 15        |  3.13 |             11.4 |                224 |
| zstd 16 256 KB |  3.24 |              8.1 |                210 |

This patch was written by Sean Purcell <me@seanp.xyz>, but I will be
taking over the submission process.

[1] http://releases.ubuntu.com/16.10/
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/squashfs-benchmark.sh

zstd source repository: https://github.com/facebook/zstd

Change-Id: I6d6d6c1a7f565d7ad28eb01d052604ec7b588a0b
Signed-off-by: Sean Purcell <me@seanp.xyz>
Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>
Acked-by: Phillip Lougher <phillip@squashfs.org.uk>
Signed-off-by: Ziyuan Xu <xzy.xu@rock-chips.com>
(cherry-picked from commit 87bf54bb43)
2018-04-25 16:21:55 +08:00
..
9p fs/9p: Compare qid.path in v9fs_test_inode 2017-11-30 08:37:22 +00:00
adfs
affs affs: fix remount failure when there are no options changed 2016-06-07 18:14:32 -07:00
afs afs: Fix afs_kill_pages() 2017-12-20 10:04:56 +01:00
autofs4 autofs: fix careless error in recent commit 2017-12-20 10:04:51 +01:00
befs
bfs
btrfs Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-03-26 02:32:06 +00:00
cachefiles FS-Cache: Add missing initialization of ret in cachefiles_write_page() 2015-11-16 20:38:43 -05:00
ceph ceph: drop negative child dentries before try pruning inode's alias 2017-12-20 10:04:52 +01:00
cifs cifs: small underflow in cnvrtDosUnixTm() 2018-03-24 10:58:46 +01:00
coda coda: fix 'kernel memory exposure attempt' in fsync 2017-11-24 08:32:25 +01:00
configfs configfs: Fix race between create_link and configfs_rmdir 2017-06-26 07:13:08 +02:00
cramfs
crypto fscrypt: updates on 4.15-rc4 2018-01-22 13:16:20 +05:30
debugfs LSK 17.08 v4.4-android 2017-09-01 18:56:00 +08:00
devpts devpts: clean up interface to pty drivers 2016-08-16 09:30:49 +02:00
dlm dlm: avoid double-free on error path in dlm_device_{register,unregister} 2017-09-13 14:09:45 -07:00
ecryptfs eCryptfs: use after free in ecryptfs_release_messaging() 2017-11-30 08:37:20 +00:00
efivarfs efi: Make efivarfs entries immutable by default 2016-03-03 15:07:09 -08:00
efs
exofs osd fs: __r4w_get_page rely on PageUptodate for uptodate 2015-12-12 10:15:34 -08:00
exportfs
ext2 Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-02-01 12:02:38 +08:00
ext4 Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-03-19 12:01:32 +08:00
f2fs f2fs: relax node version check for victim data in gc 2018-03-22 09:23:22 +01:00
fat fat: fix using uninitialized fields of fat_inode/fsinfo_inode 2017-03-15 09:57:15 +08:00
freevxfs
fscache FS-Cache: fix dereference of NULL user_key_payload 2017-10-27 10:23:18 +02:00
fuse Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-11-20 20:53:19 +05:30
gfs2 GFS2: Take inode off order_write list when setting jdata flag 2017-12-20 10:04:59 +01:00
hfs hfs: fix B-tree corruption after insertion at position 0 2015-09-10 13:29:01 -07:00
hfsplus posix_acl: Clear SGID bit when setting file permissions 2016-10-31 04:13:58 -06:00
hostfs hostfs: Freeing an ERR_PTR in hostfs_fill_sb_common() 2016-09-30 10:18:39 +02:00
hpfs hpfs: implement the show_options method 2016-06-01 12:15:54 -07:00
hugetlbfs mm: larger stack guard gap, between vmas 2017-06-26 07:13:11 +02:00
isofs isofs: fix timestamps beyond 2027 2017-11-30 08:37:20 +00:00
jbd2 jbd2: don't leak modified metadata buffers on an aborted journal 2017-03-12 06:37:26 +01:00
jffs2 posix_acl: Clear SGID bit when setting file permissions 2016-10-31 04:13:58 -06:00
jfs fs: add i_blocksize() 2017-06-14 13:16:24 +02:00
kernfs kernfs: fix regression in kernfs_fop_write caused by wrong type 2018-02-16 20:09:42 +01:00
lockd Mainly smaller bugfixes and cleanup. We're still finding some bugs from 2015-11-11 20:11:28 -08:00
logfs mm, fs: introduce mapping_gfp_constraint() 2015-11-06 17:50:42 -08:00
minix
ncpfs staging: ncpfs: memory corruption in ncp_read_kernel() 2018-03-28 18:40:15 +02:00
nfs NFS: Fix missing pg_cleanup after nfs_pageio_cond_complete() 2018-03-24 10:58:42 +01:00
nfs_common lockd: fix "list_add double add" caused by legacy signal interface 2018-02-03 17:04:28 +01:00
nfsd nfsd4: permit layoutget of executable-only files 2018-03-24 10:58:48 +01:00
nilfs2 nilfs2: fix race condition that causes file system corruption 2017-11-30 08:37:20 +00:00
nls
notify Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-08-11 19:28:33 +05:30
ntfs mm, fs: introduce mapping_gfp_constraint() 2015-11-06 17:50:42 -08:00
ocfs2 Revert "ocfs2: should wait dio before inode lock in ocfs2_setattr()" 2017-12-09 18:42:43 +01:00
omfs
openpromfs
overlayfs ovl: fix failure to fsync lower dir 2018-02-16 20:09:47 +01:00
proc Merge branch 'linux-linaro-lsk-v4.4-android' of git://git.linaro.org/kernel/linux-linaro-stable.git 2018-01-26 19:26:47 +08:00
pstore pstore/console: ignore log level 2017-11-15 14:48:58 +08:00
qnx4
qnx6
quota quota: Check for register_shrinker() failure. 2018-02-03 17:04:28 +01:00
ramfs mm, fs: obey gfp_mapping for add_to_page_cache() 2015-10-16 11:42:28 -07:00
reiserfs reiserfs: Make cancel_old_flush() reliable 2018-03-22 09:23:24 +01:00
romfs romfs: use different way to generate fsid for BLOCK or MTD 2017-06-17 06:39:38 +02:00
sdcardfs ANDROID: sdcardfs: Set num in extension_details during make_item 2018-03-05 22:01:27 +05:30
squashfs UPSTREAM: squashfs: Add zstd support 2018-04-25 16:21:55 +08:00
sysfs sysfs: be careful of error returns from ops->show() 2017-04-12 12:38:33 +02:00
sysv fix sysvfs symlinks 2015-11-23 21:11:08 -05:00
tracefs tracefs: Fix refcount imbalance in start_creating() 2015-11-04 22:13:45 -05:00
ubifs ubifs: Fix journal replay wrt. xattr nodes 2017-01-26 08:23:48 +01:00
udf udf: Avoid overflow when session starts at large offset 2017-12-20 10:05:01 +01:00
ufs ufs_getfrag_block(): we only grab ->truncate_mutex on block creation path 2017-06-14 13:16:24 +02:00
xfs xfs: quota: check result of register_shrinker() 2018-03-03 10:19:44 +01:00
aio.c fs/aio: Use RCU accessors for kioctx_table->table[] 2018-03-22 09:23:31 +01:00
anon_inodes.c
attr.c ANDROID: vfs: user permission2 in notify_change2 2017-04-10 13:12:16 +05:30
bad_inode.c
binfmt_aout.c
binfmt_elf.c UPSTREAM: ELF: Also pass any interpreter's file header to `arch_check_elf' 2018-03-05 21:56:13 +05:30
binfmt_elf_fdpic.c libnvdimm for 4.4: 2015-11-10 12:07:22 -08:00
binfmt_em86.c
binfmt_flat.c
binfmt_misc.c
binfmt_script.c
block_dev.c fs/block_dev: always invalidate cleancache in invalidate_bdev() 2017-05-20 14:27:01 +02:00
buffer.c fs: add i_blocksize() 2017-06-14 13:16:24 +02:00
char_dev.c
compat.c
compat_binfmt_elf.c binfmt_elf: compat: avoid unused function warning 2018-02-25 11:03:51 +01:00
compat_ioctl.c i2c-dev: Fix typo in ioctl name reference 2015-10-23 23:26:43 +02:00
coredump.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-07-18 14:31:54 +08:00
dax.c dax: disable pmd mappings 2015-11-16 23:54:45 -08:00
dcache.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-03-23 02:33:06 +00:00
dcookies.c
direct-io.c direct-io: Prevent NULL pointer access in submit_page_section 2017-10-18 09:20:42 +02:00
drop_caches.c
eventfd.c
eventpoll.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-09-08 12:02:15 +08:00
exec.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-07-22 12:02:06 +08:00
fcntl.c fs/fcntl: f_setown, avoid undefined behaviour 2018-01-31 12:06:11 +01:00
fhandle.c fs/coredump: prevent fsuid=0 dumps into user-controlled directories 2016-04-12 09:08:58 -07:00
file.c vfs: clear remainder of 'full_fds_bits' in dup_fd() 2015-11-05 23:05:32 -08:00
file_table.c
filesystems.c
fs-writeback.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-12-21 12:06:53 +08:00
fs_pin.c
fs_struct.c ANDROID: fs: Export free_fs_struct and set_fs_pwd 2017-02-03 15:05:18 +05:30
inode.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-03-05 20:20:17 +05:30
internal.h ANDROID: vfs: Allow filesystems to access their private mount data 2017-02-03 14:16:53 +05:30
ioctl.c
Kconfig f2fs: backport from (4c1fad64 - Merge tag 'for-f2fs-4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs) 2017-10-15 23:28:17 +05:30
Kconfig.binfmt
libfs.c
locks.c locks: don't check for race with close when setting OFD lock 2018-01-17 09:35:27 +01:00
Makefile f2fs: backport from (4c1fad64 - Merge tag 'for-f2fs-4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs) 2017-10-15 23:28:17 +05:30
mbcache.c
mbcache2.c BACKPORT: [UPSTREAM] mbcache2: reimplement mbcache 2017-05-01 15:05:27 +05:30
mount.h mnt: In propgate_umount handle visiting mounts in any order 2017-07-21 07:44:57 +02:00
mpage.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-07-11 16:22:22 +08:00
namei.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-03-23 02:33:06 +00:00
namespace.c ANDROID: mnt: Fix freeing of mount data 2017-09-18 21:14:36 +01:00
no-block.c
nsfs.c nsfs: mark dentry with DCACHE_RCUACCESS 2018-02-16 20:09:43 +01:00
open.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-07-18 14:31:54 +08:00
pipe.c pipe: avoid round_pipe_size() nr_pages overflow on 32-bit 2018-01-23 19:50:15 +01:00
pnode.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-07-22 12:02:06 +08:00
pnode.h Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-05-04 12:01:39 +08:00
posix_acl.c tmpfs: clear S_ISGID when setting posix ACLs 2017-01-26 08:23:47 +01:00
proc_namespace.c ANDROID: vfs: Allow filesystems to access their private mount data 2017-02-03 14:16:53 +05:30
read_write.c vfs: Return -ENXIO for negative SEEK_HOLE / SEEK_DATA offsets 2017-10-05 09:41:45 +02:00
readdir.c
select.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-02-01 12:02:38 +08:00
seq_file.c Make file credentials available to the seqfile interfaces 2017-08-06 19:19:42 -07:00
signalfd.c
splice.c vfs: fix uninitialized flags in splice_to_pipe() 2017-02-23 17:43:09 +01:00
stack.c
stat.c ufs: restore maintaining ->i_blocks 2017-06-14 13:16:24 +02:00
statfs.c
super.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2018-03-05 20:20:17 +05:30
sync.c ANDROID: sched: add a counter to track fsync 2017-04-10 13:12:16 +05:30
timerfd.c timerfd: Protect the might cancel mechanism proper 2017-05-08 07:46:01 +02:00
userfaultfd.c Merge branch 'linux-linaro-lsk-v4.4' into linux-linaro-lsk-v4.4-android 2017-12-21 12:06:53 +08:00
utimes.c ANDROID: vfs: Add setattr2 for filesystems with per mount permissions 2017-02-03 15:04:29 +05:30
xattr.c ANDROID: xattr: Pass EOPNOTSUPP to permission2 2018-03-05 21:51:20 +05:30