注目の記事

s3fsよりも高速に使えるgoofysを試してみた

2016.01.27

西澤です。S3バケットを直接マウントしてファイルシステムのように使いたいケースがありますが、s3fsはややパフォーマンスに難があります。Goで書かれていてs3fsよりも高速に動作することを売りにした"goofys"というツールを見つけたので、早速試してみることにしました。

前提パッケージのインストール

今回はAmazon Linux(Amazon Linux AMI 2015.09.1 (HVM), SSD Volume Type)環境で検証を行いました。golangとfuseパッケージが前提として必要となりますので、下記のようにインストールします。

$ sudo yum install golang fuse -y
Loaded plugins: priorities, update-motd, upgrade-helper
Resolving Dependencies
--> Running transaction check
---> Package fuse.x86_64 0:2.9.4-1.17.amzn1 will be installed
---> Package golang.x86_64 0:1.4.2-3.19.amzn1 will be installed
--> Processing Dependency: golang-src = 1.4.2-3.19.amzn1 for package: golang-1.4.2-3.19.amzn1.x86_64
--> Processing Dependency: golang-bin for package: golang-1.4.2-3.19.amzn1.x86_64
--> Processing Dependency: bzr for package: golang-1.4.2-3.19.amzn1.x86_64
--> Processing Dependency: subversion for package: golang-1.4.2-3.19.amzn1.x86_64
--> Processing Dependency: mercurial for package: golang-1.4.2-3.19.amzn1.x86_64
--> Processing Dependency: git for package: golang-1.4.2-3.19.amzn1.x86_64
--> Running transaction check
---> Package bzr-python27.x86_64 0:2.6.0-8.17.amzn1 will be installed
--> Processing Dependency: bzr-common = 2.6.0-8.17.amzn1 for package: bzr-python27-2.6.0-8.17.amzn1.x86_64
---> Package git.x86_64 0:2.4.3-7.42.amzn1 will be installed
--> Processing Dependency: perl-Git = 2.4.3-7.42.amzn1 for package: git-2.4.3-7.42.amzn1.x86_64
--> Processing Dependency: perl(Term::ReadKey) for package: git-2.4.3-7.42.amzn1.x86_64
--> Processing Dependency: perl(Git) for package: git-2.4.3-7.42.amzn1.x86_64
--> Processing Dependency: perl(Error) for package: git-2.4.3-7.42.amzn1.x86_64
---> Package golang-pkg-bin-linux-amd64.x86_64 0:1.4.2-3.19.amzn1 will be installed
--> Processing Dependency: golang-pkg-linux-amd64 = 1.4.2-3.19.amzn1 for package: golang-pkg-bin-linux-amd64-1.4.2-3.19.amzn1.x86_64
--> Processing Dependency: gcc for package: golang-pkg-bin-linux-amd64-1.4.2-3.19.amzn1.x86_64
---> Package golang-src.noarch 0:1.4.2-3.19.amzn1 will be installed
---> Package mercurial-python27.x86_64 0:2.6.3-1.25.amzn1 will be installed
--> Processing Dependency: mercurial-common(x86-64) = 2.6.3-1.25.amzn1 for package: mercurial-python27-2.6.3-1.25.amzn1.x86_64
---> Package subversion.x86_64 0:1.8.13-7.52.amzn1 will be installed
--> Processing Dependency: subversion-libs(x86-64) = 1.8.13-7.52.amzn1 for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_fs_base-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libserf-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libaprutil-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_fs_util-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_fs-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_fs_fs-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_ra_serf-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_client-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libapr-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_diff-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_wc-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_subr-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_delta-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_ra_local-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_ra-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_ra_svn-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Processing Dependency: libsvn_repos-1.so.0()(64bit) for package: subversion-1.8.13-7.52.amzn1.x86_64
--> Running transaction check
---> Package apr.x86_64 0:1.5.1-1.12.amzn1 will be installed
---> Package apr-util.x86_64 0:1.4.1-4.17.amzn1 will be installed
---> Package bzr-common.noarch 0:2.6.0-8.17.amzn1 will be installed
---> Package gcc48.x86_64 0:4.8.3-9.109.amzn1 will be installed
--> Processing Dependency: cpp48(x86-64) = 4.8.3-9.109.amzn1 for package: gcc48-4.8.3-9.109.amzn1.x86_64
--> Processing Dependency: libgomp(x86-64) >= 4.8.3-9.109.amzn1 for package: gcc48-4.8.3-9.109.amzn1.x86_64
--> Processing Dependency: glibc-devel(x86-64) >= 2.2.90-12 for package: gcc48-4.8.3-9.109.amzn1.x86_64
--> Processing Dependency: libmpc.so.3()(64bit) for package: gcc48-4.8.3-9.109.amzn1.x86_64
--> Processing Dependency: libmpfr.so.4()(64bit) for package: gcc48-4.8.3-9.109.amzn1.x86_64
---> Package golang-pkg-linux-amd64.noarch 0:1.4.2-3.19.amzn1 will be installed
---> Package libserf.x86_64 0:1.3.7-1.7.amzn1 will be installed
---> Package mercurial-common.x86_64 0:2.6.3-1.25.amzn1 will be installed
---> Package perl-Error.noarch 1:0.17020-2.9.amzn1 will be installed
---> Package perl-Git.noarch 0:2.4.3-7.42.amzn1 will be installed
---> Package perl-TermReadKey.x86_64 0:2.30-20.9.amzn1 will be installed
---> Package subversion-libs.x86_64 0:1.8.13-7.52.amzn1 will be installed
--> Running transaction check
---> Package cpp48.x86_64 0:4.8.3-9.109.amzn1 will be installed
---> Package glibc-devel.x86_64 0:2.17-106.163.amzn1 will be installed
--> Processing Dependency: glibc-headers = 2.17-106.163.amzn1 for package: glibc-devel-2.17-106.163.amzn1.x86_64
--> Processing Dependency: glibc-headers for package: glibc-devel-2.17-106.163.amzn1.x86_64
---> Package libgomp.x86_64 0:4.8.3-9.109.amzn1 will be installed
---> Package libmpc.x86_64 0:1.0.1-3.3.amzn1 will be installed
---> Package mpfr.x86_64 0:3.1.1-4.14.amzn1 will be installed
--> Running transaction check
---> Package glibc-headers.x86_64 0:2.17-106.163.amzn1 will be installed
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers-2.17-106.163.amzn1.x86_64
--> Processing Dependency: kernel-headers for package: glibc-headers-2.17-106.163.amzn1.x86_64
--> Running transaction check
---> Package kernel-headers.x86_64 0:4.1.13-19.31.amzn1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================
Package Arch Version Repository Size
========================================================================================================================
Installing:
fuse x86_64 2.9.4-1.17.amzn1 amzn-main 88 k
golang x86_64 1.4.2-3.19.amzn1 amzn-main 3.3 M
Installing for dependencies:
apr x86_64 1.5.1-1.12.amzn1 amzn-updates 116 k
apr-util x86_64 1.4.1-4.17.amzn1 amzn-main 87 k
bzr-common noarch 2.6.0-8.17.amzn1 amzn-main 624 k
bzr-python27 x86_64 2.6.0-8.17.amzn1 amzn-main 6.9 M
cpp48 x86_64 4.8.3-9.109.amzn1 amzn-main 6.7 M
gcc48 x86_64 4.8.3-9.109.amzn1 amzn-main 17 M
git x86_64 2.4.3-7.42.amzn1 amzn-updates 9.8 M
glibc-devel x86_64 2.17-106.163.amzn1 amzn-updates 1.1 M
glibc-headers x86_64 2.17-106.163.amzn1 amzn-updates 734 k
golang-pkg-bin-linux-amd64 x86_64 1.4.2-3.19.amzn1 amzn-main 18 M
golang-pkg-linux-amd64 noarch 1.4.2-3.19.amzn1 amzn-main 9.5 M
golang-src noarch 1.4.2-3.19.amzn1 amzn-main 6.8 M
kernel-headers x86_64 4.1.13-19.31.amzn1 amzn-updates 1.0 M
libgomp x86_64 4.8.3-9.109.amzn1 amzn-main 175 k
libmpc x86_64 1.0.1-3.3.amzn1 amzn-main 53 k
libserf x86_64 1.3.7-1.7.amzn1 amzn-main 64 k
mercurial-common x86_64 2.6.3-1.25.amzn1 amzn-main 1.4 M
mercurial-python27 x86_64 2.6.3-1.25.amzn1 amzn-main 2.3 M
mpfr x86_64 3.1.1-4.14.amzn1 amzn-main 237 k
perl-Error noarch 1:0.17020-2.9.amzn1 amzn-main 33 k
perl-Git noarch 2.4.3-7.42.amzn1 amzn-updates 61 k
perl-TermReadKey x86_64 2.30-20.9.amzn1 amzn-main 33 k
subversion x86_64 1.8.13-7.52.amzn1 amzn-main 1.6 M
subversion-libs x86_64 1.8.13-7.52.amzn1 amzn-main 1.2 M

Transaction Summary
========================================================================================================================
Install 2 Packages (+24 Dependent packages)

Total download size: 89 M
Installed size: 261 M
Downloading packages:
(1/26): apr-1.5.1-1.12.amzn1.x86_64.rpm | 116 kB 00:00
(2/26): apr-util-1.4.1-4.17.amzn1.x86_64.rpm | 87 kB 00:00
(3/26): bzr-common-2.6.0-8.17.amzn1.noarch.rpm | 624 kB 00:00
(4/26): bzr-python27-2.6.0-8.17.amzn1.x86_64.rpm | 6.9 MB 00:00
(5/26): cpp48-4.8.3-9.109.amzn1.x86_64.rpm | 6.7 MB 00:00
(6/26): fuse-2.9.4-1.17.amzn1.x86_64.rpm | 88 kB 00:00
(7/26): gcc48-4.8.3-9.109.amzn1.x86_64.rpm | 17 MB 00:00
(8/26): git-2.4.3-7.42.amzn1.x86_64.rpm | 9.8 MB 00:00
(9/26): glibc-devel-2.17-106.163.amzn1.x86_64.rpm | 1.1 MB 00:00
(10/26): glibc-headers-2.17-106.163.amzn1.x86_64.rpm | 734 kB 00:00
(11/26): golang-1.4.2-3.19.amzn1.x86_64.rpm | 3.3 MB 00:00
(12/26): golang-pkg-bin-linux-amd64-1.4.2-3.19.amzn1.x86_64.rpm | 18 MB 00:00
(13/26): golang-pkg-linux-amd64-1.4.2-3.19.amzn1.noarch.rpm | 9.5 MB 00:00
(14/26): golang-src-1.4.2-3.19.amzn1.noarch.rpm | 6.8 MB 00:00
(15/26): kernel-headers-4.1.13-19.31.amzn1.x86_64.rpm | 1.0 MB 00:00
(16/26): libgomp-4.8.3-9.109.amzn1.x86_64.rpm | 175 kB 00:00
(17/26): libmpc-1.0.1-3.3.amzn1.x86_64.rpm | 53 kB 00:00
(18/26): libserf-1.3.7-1.7.amzn1.x86_64.rpm | 64 kB 00:00
(19/26): mercurial-common-2.6.3-1.25.amzn1.x86_64.rpm | 1.4 MB 00:00
(20/26): mercurial-python27-2.6.3-1.25.amzn1.x86_64.rpm | 2.3 MB 00:00
(21/26): mpfr-3.1.1-4.14.amzn1.x86_64.rpm | 237 kB 00:00
(22/26): perl-Error-0.17020-2.9.amzn1.noarch.rpm | 33 kB 00:00
(23/26): perl-Git-2.4.3-7.42.amzn1.noarch.rpm | 61 kB 00:00
(24/26): perl-TermReadKey-2.30-20.9.amzn1.x86_64.rpm | 33 kB 00:00
(25/26): subversion-1.8.13-7.52.amzn1.x86_64.rpm | 1.6 MB 00:00
(26/26): subversion-libs-1.8.13-7.52.amzn1.x86_64.rpm | 1.2 MB 00:00
------------------------------------------------------------------------------------------------------------------------
Total 19 MB/s | 89 MB 00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : apr-1.5.1-1.12.amzn1.x86_64 1/26
Installing : apr-util-1.4.1-4.17.amzn1.x86_64 2/26
Installing : mpfr-3.1.1-4.14.amzn1.x86_64 3/26
Installing : libmpc-1.0.1-3.3.amzn1.x86_64 4/26
Installing : libserf-1.3.7-1.7.amzn1.x86_64 5/26
Installing : 1:perl-Error-0.17020-2.9.amzn1.noarch 6/26
Installing : subversion-libs-1.8.13-7.52.amzn1.x86_64 7/26
Installing : subversion-1.8.13-7.52.amzn1.x86_64 8/26
Installing : cpp48-4.8.3-9.109.amzn1.x86_64 9/26
Installing : bzr-python27-2.6.0-8.17.amzn1.x86_64 10/26
Installing : bzr-common-2.6.0-8.17.amzn1.noarch 11/26
Installing : kernel-headers-4.1.13-19.31.amzn1.x86_64 12/26
Installing : glibc-headers-2.17-106.163.amzn1.x86_64 13/26
Installing : glibc-devel-2.17-106.163.amzn1.x86_64 14/26
Installing : libgomp-4.8.3-9.109.amzn1.x86_64 15/26
Installing : gcc48-4.8.3-9.109.amzn1.x86_64 16/26
Installing : golang-src-1.4.2-3.19.amzn1.noarch 17/26
Installing : mercurial-common-2.6.3-1.25.amzn1.x86_64 18/26
Installing : mercurial-python27-2.6.3-1.25.amzn1.x86_64 19/26
Installing : perl-TermReadKey-2.30-20.9.amzn1.x86_64 20/26
Installing : perl-Git-2.4.3-7.42.amzn1.noarch 21/26
Installing : git-2.4.3-7.42.amzn1.x86_64 22/26
Installing : golang-1.4.2-3.19.amzn1.x86_64 23/26
Installing : golang-pkg-linux-amd64-1.4.2-3.19.amzn1.noarch 24/26
Installing : golang-pkg-bin-linux-amd64-1.4.2-3.19.amzn1.x86_64 25/26
Installing : fuse-2.9.4-1.17.amzn1.x86_64 26/26
Verifying : golang-pkg-linux-amd64-1.4.2-3.19.amzn1.noarch 1/26
Verifying : perl-TermReadKey-2.30-20.9.amzn1.x86_64 2/26
Verifying : mercurial-python27-2.6.3-1.25.amzn1.x86_64 3/26
Verifying : mercurial-common-2.6.3-1.25.amzn1.x86_64 4/26
Verifying : apr-util-1.4.1-4.17.amzn1.x86_64 5/26
Verifying : mpfr-3.1.1-4.14.amzn1.x86_64 6/26
Verifying : golang-src-1.4.2-3.19.amzn1.noarch 7/26
Verifying : libgomp-4.8.3-9.109.amzn1.x86_64 8/26
Verifying : apr-1.5.1-1.12.amzn1.x86_64 9/26
Verifying : libserf-1.3.7-1.7.amzn1.x86_64 10/26
Verifying : golang-pkg-bin-linux-amd64-1.4.2-3.19.amzn1.x86_64 11/26
Verifying : golang-1.4.2-3.19.amzn1.x86_64 12/26
Verifying : bzr-common-2.6.0-8.17.amzn1.noarch 13/26
Verifying : libmpc-1.0.1-3.3.amzn1.x86_64 14/26
Verifying : git-2.4.3-7.42.amzn1.x86_64 15/26
Verifying : 1:perl-Error-0.17020-2.9.amzn1.noarch 16/26
Verifying : glibc-headers-2.17-106.163.amzn1.x86_64 17/26
Verifying : cpp48-4.8.3-9.109.amzn1.x86_64 18/26
Verifying : kernel-headers-4.1.13-19.31.amzn1.x86_64 19/26
Verifying : gcc48-4.8.3-9.109.amzn1.x86_64 20/26
Verifying : perl-Git-2.4.3-7.42.amzn1.noarch 21/26
Verifying : bzr-python27-2.6.0-8.17.amzn1.x86_64 22/26
Verifying : subversion-1.8.13-7.52.amzn1.x86_64 23/26
Verifying : fuse-2.9.4-1.17.amzn1.x86_64 24/26
Verifying : subversion-libs-1.8.13-7.52.amzn1.x86_64 25/26
Verifying : glibc-devel-2.17-106.163.amzn1.x86_64 26/26

Installed:
fuse.x86_64 0:2.9.4-1.17.amzn1 golang.x86_64 0:1.4.2-3.19.amzn1

Dependency Installed:
apr.x86_64 0:1.5.1-1.12.amzn1 apr-util.x86_64 0:1.4.1-4.17.amzn1
bzr-common.noarch 0:2.6.0-8.17.amzn1 bzr-python27.x86_64 0:2.6.0-8.17.amzn1
cpp48.x86_64 0:4.8.3-9.109.amzn1 gcc48.x86_64 0:4.8.3-9.109.amzn1
git.x86_64 0:2.4.3-7.42.amzn1 glibc-devel.x86_64 0:2.17-106.163.amzn1
glibc-headers.x86_64 0:2.17-106.163.amzn1 golang-pkg-bin-linux-amd64.x86_64 0:1.4.2-3.19.amzn1
golang-pkg-linux-amd64.noarch 0:1.4.2-3.19.amzn1 golang-src.noarch 0:1.4.2-3.19.amzn1
kernel-headers.x86_64 0:4.1.13-19.31.amzn1 libgomp.x86_64 0:4.8.3-9.109.amzn1
libmpc.x86_64 0:1.0.1-3.3.amzn1 libserf.x86_64 0:1.3.7-1.7.amzn1
mercurial-common.x86_64 0:2.6.3-1.25.amzn1 mercurial-python27.x86_64 0:2.6.3-1.25.amzn1
mpfr.x86_64 0:3.1.1-4.14.amzn1 perl-Error.noarch 1:0.17020-2.9.amzn1
perl-Git.noarch 0:2.4.3-7.42.amzn1 perl-TermReadKey.x86_64 0:2.30-20.9.amzn1
subversion.x86_64 0:1.8.13-7.52.amzn1 subversion-libs.x86_64 0:1.8.13-7.52.amzn1

Complete!

goofysのインストール

READMEに記載の手順通りに実行するだけで、簡単にコンパイルまでできました。

$ export GOPATH=$HOME/go
$ go get github.com/kahing/goofys
$ go install github.com/kahing/goofys
$ $GOPATH/bin/goofys -h
Error: goofys takes exactly two arguments.

NAME:
goofys - Mount an S3 bucket locally

USAGE:
goofys [global options] bucket mountpoint

VERSION:
0.0.3

GLOBAL OPTIONS:
--help, -h Print this help text and exit successfuly.
-o [-o option -o option] Additional system-specific mount options. Be careful!
--dir-mode "493" Permissions bits for directories. (default: 0755)
--file-mode "420" Permission bits for files (default: 0644)
--uid "500" UID owner of all inodes.
--gid "500" GID owner of all inodes.
--endpoint The non-AWS endpoint to connect to. Possible values: http://127.0.0.1:8081/
--storage-class "STANDARD" The type of storage to use when writing objects. Possible values: REDUCED_REDUNDANCY, STANDARD (default), STANDARD_IA.
--use-path-request Use a path-style request instead of virtual host-style. (deprecated, always on)
--profile Use a named profile from $HOME/.aws/credentials instead of "default"
--stat-cache-ttl "1m0s" How long to cache StatObject results and inode attributes.
--type-cache-ttl "1m0s" How long to cache name -> file/dir mappings in directory inodes.
--debug_fuse Enable fuse-related debugging output.
--debug_s3 Enable S3-related debugging output.
-f Run goofys in foreground.
--version, -v print the version

S3バケットのマウント

S3バケットにアクセスする権限は、EC2に割り当てたIAMロールに付与しておけば、作業不要です。個別に設定する場合も、~/.aws/credentialsに記載をすれば良いようですので、AWS CLIを利用してaws configureコマンドで設定をしておけば、その認証情報を利用できます。今回はIAMロール(インスタンスプロファイル)に権限を付与してあるので、特に作業は不要でした。

$ mkdir ~/goofys
$ $GOPATH/bin/goofys test-bucket ~/goofys
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.8G 1.5G 6.2G 20% /
devtmpfs 489M 56K 489M 1% /dev
tmpfs 498M 0 498M 0% /dev/shm
test-bucket 1.0P 0 1.0P 0% /home/ec2-user/goofys

特別なオプション指定もなしでマウントできました。goofysからは1PBで見えるようになっているようですね。

s3fsとgoofysの簡単な性能比較

今回は比較の為、s3fsも同一のマシンにインストールしました。コマンドの実行結果等の詳細は割愛します。

$ sudo yum install automake fuse-devel gcc-c++ git libcurl-devel libxml2-devel openssl-devel -y
$ git clone https://github.com/s3fs-fuse/s3fs-fuse.git
$ cd s3fs-fuse
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install

goofysでマウントしたS3バケットと同じバケットをs3fsからもマウントしました。サイズは異なって見えていますが、実体は同じものです。

$ mkdir ~/s3fs
$ s3fs test-bucket ~/s3fs -o iam_role=EC2Role
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.8G 1.6G 6.1G 21% /
devtmpfs 489M 56K 489M 1% /dev
tmpfs 498M 0 498M 0% /dev/shm
test-bucket 1.0P 0 1.0P 0% /home/ec2-user/goofys
s3fs 256T 0 256T 0% /home/ec2-user/s3fs

それでは簡単な試験を行ってみましょう。ファイルをいくつか書き込んでみます。

$ cd ~/s3fs/
$ i=1; time while [ $i -le 500 ]
> do
> touch s3fs_test_$i > /dev/null 2>&1
> i=$((i+1))
> done

real 1m1.115s
user 0m0.036s
sys 0m0.080s
$ cd ~/goofys/
$ i=1; time while [ $i -le 500 ]
> do
> touch goofys_test_$i > /dev/null 2>&1
> i=$((i+1))
> done

real 0m12.409s
user 0m0.036s
sys 0m0.072s

けっこう差が出ました。想像以上に速いです。ではこの状態でリストしてみます。

$ time ls -laR ~/s3fs > /dev/null

real 0m2.923s
user 0m0.012s
sys 0m0.016s
$ time ls -laR ~/goofys > /dev/null

real 0m0.276s
user 0m0.004s
sys 0m0.004s

こちらは10倍程度の性能差が出ました。1万オブジェクト程度を配置したバケットでlsコマンドを実行しても、人間が待てるレベルの性能だったので、個人的にはかなり使えるのではないか、という印象を持ちました。

まとめ

試しに触ってみたgoofysでしたが、s3fsよりも想像以上に速くて驚きました。個人的にはs3fsはマウントしてlsコマンドを叩くだけで(遅すぎて)使うのを諦めていたのですが、goofysは用途次第では利用に耐え得る性能ではないかと感じました。s3fsとの細かい機能の違いや信頼性等も調べてみようと思います。

S3本来の使い方ではありませんが、ローカルディスク容量を気にせず、これまで通りのファイルシステムとして利用できるのは魅力的ですね。興味が湧いた方はぜひ触ってみてください。