pixz是一个并行xz压缩工具它支持并行压缩和解压。从存储库 https://github.com/vasi/pixz 下载源代码解压缩到/par/pixz-1.0.7。configure提示缺少libarchive包。root6ae32a5ffcde:/par/pixz-1.0.7# ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /usr/bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether make supports the include directive... yes (GNU style) checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... gcc3 checking for gcc option to accept ISO C99... none needed checking for gcc option to accept ISO Standard C... (cached) none needed checking for src/pixz.1... yes checking for ceil in -lm... yes checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking if compiler needs -Werror to reject unknown flags... no checking for the pthreads library -lpthreads... no checking whether pthreads work without any flags... yes checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE checking if more special flags are required for pthreads... no checking for PTHREAD_PRIO_INHERIT... yes checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for LIBARCHIVE... no configure: error: Package requirements (libarchive) were not met: Package libarchive, required by virtual:world, not found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables LIBARCHIVE_CFLAGS and LIBARCHIVE_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details.希望不用apt安装libarchive而从源代码编译。从https://libarchive.org/downloads/下载源代码解压缩到/par/libarchive-3.8.9。使用cmake .和make -j8命令编译libarchive库编译好的库文件和源代码都在par/libarchive-3.8.9/libarchive目录下。按照错误提示在configure命令行中添加LIBARCHIVE_CFLAGS和LIBARCHIVE_LIBS环境变量root6ae32a5ffcde:/par/pixz-1.0.7# ./configure LIBARCHIVE_CFLAGS/par/libarchive-3.8.9/libarchive LIBARCHIVE_LIBS/par/libarchive-3.8.9/libarchive ... config.status: creating Makefile config.status: creating src/Makefile config.status: creating test/Makefile config.status: creating config.h config.status: executing depfiles commands root6ae32a5ffcde:/par/pixz-1.0.7# make -j8 make all-recursive make[1]: Entering directory /par/pixz-1.0.7 Making all in src make[2]: Entering directory /par/pixz-1.0.7/src gcc -DHAVE_CONFIG_H -I. -I.. /par/libarchive-3.8.9/libarchive -Wall -Wno-unknown-pragmas -g -O2 -MT pixz-common.o -MD -MP -MF .deps/pixz-common.Tpo -c -o pixz-common.o test -f common.c || echo ./common.c ... gcc: warning: /par/libarchive-3.8.9/libarchive: linker input file unused because linking not done mv -f .deps/pixz-common.Tpo .deps/pixz-common.Po gcc -Wall -Wno-unknown-pragmas -g -O2 -o pixz pixz-common.o pixz-cpu.o pixz-endian.o pixz-list.o pixz-pixz.o pixz-read.o pixz-write.o -lm /par/libarchive-3.8.9/libarchive -llzma -lm /usr/bin/ld: cannot find /par/libarchive-3.8.9/libarchive: file format not recognized collect2: error: ld returned 1 exit status make[2]: *** [Makefile:412: pixz] Error 1 make[2]: Leaving directory /par/pixz-1.0.7/src make[1]: *** [Makefile:377: all-recursive] Error 1 make[1]: Leaving directory /par/pixz-1.0.7 make: *** [Makefile:318: all] Error 2生成makefile成功编译出错因为LIBARCHIVE_LIBS需要是动态库文件名而不是搜索目录, 如果写成静态链接库名称则报错如下root6ae32a5ffcde:/par/pixz-1.0.7# ./configure LIBARCHIVE_CFLAGS/par/libarchive-3.8.9/libarchive LIBARCHIVE_LIBS/par/libarchive-3.8.9/libarchive/libarchive.a root6ae32a5ffcde:/par/pixz-1.0.7# make -j8 make all-recursive make[1]: Entering directory /par/pixz-1.0.7 Making all in src make[2]: Entering directory /par/pixz-1.0.7/src gcc -Wall -Wno-unknown-pragmas -g -O2 -o pixz pixz-common.o pixz-cpu.o pixz-endian.o pixz-list.o pixz-pixz.o pixz-read.o pixz-write.o -lm /par/libarchive-3.8.9/libarchive/libarchive.a -llzma -lm /usr/bin/ld: /par/libarchive-3.8.9/libarchive/libarchive.a(archive_string.c.o): in function get_sconv_object: archive_string.c:(.text.get_sconv_object0x279): undefined reference to libiconv_open /usr/bin/ld: archive_string.c:(.text.get_sconv_object0x2cb): undefined reference to libiconv_open /usr/bin/ld: archive_string.c:(.text.get_sconv_object0x3e5): undefined reference to libiconv_close ... collect2: error: ld returned 1 exit status make[2]: *** [Makefile:412: pixz] Error 1 make[2]: Leaving directory /par/pixz-1.0.7/src make[1]: *** [Makefile:377: all-recursive] Error 1 make[1]: Leaving directory /par/pixz-1.0.7 make: *** [Makefile:318: all] Error 2LIBARCHIVE_LIBS改成动态库文件名就可以了。root6ae32a5ffcde:/par/pixz-1.0.7# ./configure LIBARCHIVE_CFLAGS/par/libarchive-3.8.9/libarchive LIBARCHIVE_LIBS/par/libarchive-3.8.9/libarchive/libarchive.so ... config.status: creating Makefile config.status: creating src/Makefile config.status: creating test/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands root6ae32a5ffcde:/par/pixz-1.0.7# make -j8 make all-recursive make[1]: Entering directory /par/pixz-1.0.7 Making all in src make[2]: Entering directory /par/pixz-1.0.7/src gcc -Wall -Wno-unknown-pragmas -g -O2 -o pixz pixz-common.o pixz-cpu.o pixz-endian.o pixz-list.o pixz-pixz.o pixz-read.o pixz-write.o -lm /par/libarchive-3.8.9/libarchive/libarchive.so -llzma -lm make[2]: Leaving directory /par/pixz-1.0.7/src Making all in test make[2]: Entering directory /par/pixz-1.0.7/test make[2]: Nothing to be done for all. make[2]: Leaving directory /par/pixz-1.0.7/test make[2]: Entering directory /par/pixz-1.0.7 make[2]: Leaving directory /par/pixz-1.0.7 make[1]: Leaving directory /par/pixz-1.0.7编好的二进制文件在pixz-1.0.7/src目录下。root6ae32a5ffcde:/par/pixz-1.0.7# src/pixz Refusing to output to a TTY pixz: Parallel Indexing XZ compression, fully compatible with XZ Basic usage: pixz input output.pxz # Compress a file in parallel pixz -d input.pxz output # Decompress Tarballs: pixz input.tar output.tpxz # Compress and index a tarball pixz -d input.tpxz output.tar # Decompress pixz -l input.tpxz # List tarball contents very fast pixz -x path/to/file input.tpxz | tar x # Extract one file very fast tar -Ipixz -cf output.tpxz dir # Make tar use pixz automatically Input and output: pixz input output.pxz # Same as pixz input output.pxz pixz -i input -o output.pxz # Ditto pixz [-d] input # Automatically choose output filename Other flags: -0, -1 ... -9 Set compression level, from fastest to strongest -p NUM Use a maximum of NUM CPU-intensive threads -t Dont assume input is in tar format -k Keep original input (do not remove it) -c ignored -h Print this help pixz 1.0.7 (C) 2009-2012 Dave Vasilevsky davevasilevsky.ca https://github.com/vasi/pixz You may use this software under the FreeBSD License用它来测试压缩和解压。并和xz命令对比。root6ae32a5ffcde:/par/pixz-1.0.7# cd .. root6ae32a5ffcde:/par# pixz-1.0.7/src/pixz 1y.csv 1y.pxz root6ae32a5ffcde:/par# pixz-1.0.7/src/pixz -l 1y.pxz 268837 / 16777216 159479 / 16777216 ... 265727 / 16777216 519429 / 16777216 146829 / 16473668 root6ae32a5ffcde:/par# root6ae32a5ffcde:/par# time xz -1 1y.csv -k -f real 0m25.174s user 0m12.151s sys 0m1.057s root6ae32a5ffcde:/par# time pixz-1.0.7/src/pixz -1 1y.csv 1y1.pxz real 0m26.298s user 0m12.014s sys 0m2.234s root6ae32a5ffcde:/par# ls -l 1y* -rwxrwxrwx 1 root root 888888900 Sep 20 04:13 1y.csv -rwxrwxrwx 1 root root 400017165 Sep 20 04:13 1y.csv.lz4 -rwxrwxrwx 1 root root 22061280 Sep 20 04:13 1y.csv.xz --------- -rwxrwxrwx 1 root root 18418873 Sep 24 09:38 1y.kanzi -rwxrwxrwx 1 root root 400331957 Sep 20 09:16 1y.parquet -rwxrwxrwx 1 root root 15995932 Sep 26 10:21 1y.pxz -------默认级别 -rwxrwxrwx 1 root root 22042116 Sep 26 10:35 1y1.pxz --------级别1 最快 -rwxrwxrwx 1 root root 101690581 Sep 20 09:16 1yzst.parquet root6ae32a5ffcde:/par# root6ae32a5ffcde:/par# time pixz-1.0.7/src/pixz -d 1y1.pxz 1y1.csv real 0m3.822s user 0m2.377s sys 0m0.297s root6ae32a5ffcde:/par# time xz -d 1y.csv.xz -f real 0m15.329s user 0m2.596s sys 0m0.987s同样的压缩级别压缩时间和压缩率和xz一样解压快了好几倍。