Snapshot取得後に圧縮をする意味はあるのか
こんにちは、のんピ(@non____97)です。
皆さんはAmazon FSx for NetApp ONTAP(以降FSxN)でSnapshot取得後にInactive data compressionを実行しても物理使用量が減少するのか気になったことはありますか? 私はあります。
以下記事でカスケードSnapMirrorの中間ボリュームでInactive data compression実行後にSnapMirrorをすると、中間ボリュームのInactive data compressionによるデータ削減効果がSnapMirrorの転送先でも維持されていることを確認しました。
一方、重複排除の場合はSnapshotを取得してから重複排除を実行しても重複しているデータブロックは即座に解放されません。解放されるにはSnapshotを削除する必要があります。
もしかすると、Inactive data compressionのようなaggregateレイヤーで動作するデータ削減効果はSnapshotの影響を無視するのでしょうか。
実際に確認してみました。
いきなりまとめ
- Snapshot取得後にInactive data compressionを実行しても即座に物理データ使用量が削減される
- AFSではなく、Snapshotしかデータが存在しなくても削減される
やってみた
ファイルの書き込み
早速やってみます。
ファイル書き込み前のStorage Efficiency、ボリューム、aggregate、Snapshotの情報は以下のとおりです。
::> set diag
Warning: These diagnostic commands are for use by NetApp personnel only.
Do you want to continue? {y|n}: y
::*> volume efficiency show -volume vol1 -fields state, policy, storage-efficiency-mode, inline-compression, inline-dedupe, compression, data-compaction, auto-adaptive-compression-savings, auto-adaptive-compression-existing-volume, using-auto-adaptive-compression
vserver volume state policy compression inline-compression storage-efficiency-mode inline-dedupe data-compaction auto-adaptive-compression-savings using-auto-adaptive-compression auto-adaptive-compression-existing-volume
------- ------ -------- ------ ----------- ------------------ ----------------------- ------------- --------------- --------------------------------- ------------------------------- -----------------------------------------
svm vol1 Disabled auto false false efficient false false true false false
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ -------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Disabled Idle for 00:28:38 Sat Jan 20 04:02:39 2024 Sat Jan 20 04:02:39 2024 0B 0% 0B 316KB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ----- --------- --------------- ------- ----- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 128GB 121.6GB 128GB 121.6GB 316KB 0% 0B 0% 0B 316KB 0% 316KB 0% - 316KB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 316KB 0%
Footprint in Performance Tier 2.14MB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 107.5MB 0%
Delayed Frees 1.83MB 0%
File Operation Metadata 4KB 0%
Total Footprint 109.6MB 0%
Effective Total Footprint 109.6MB 0%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId012f5aba611482f32-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 460KB
Total Physical Used: 432KB
Total Storage Efficiency Ratio: 1.06:1
Total Data Reduction Logical Used Without Snapshots: 152KB
Total Data Reduction Physical Used Without Snapshots: 300KB
Total Data Reduction Efficiency Ratio Without Snapshots: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones: 152KB
Total Data Reduction Physical Used without snapshots and flexclones: 300KB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 960KB
Total Physical Used in FabricPool Performance Tier: 7.75MB
Total FabricPool Performance Tier Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 652KB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 7.62MB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Logical Space Used for All Volumes: 152KB
Physical Space Used for All Volumes: 152KB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 432KB
Physical Space Used by the Aggregate: 432KB
Space Saved by Aggregate Data Reduction: 0B
Aggregate Data Reduction SE Ratio: 1.00:1
Logical Size Used by Snapshot Copies: 308KB
Physical Size Used by Snapshot Copies: 132KB
Snapshot Volume Data Reduction Ratio: 2.33:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 2.33:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 1
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used, sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 860.6GB 861.8GB 1.12GB 83.83MB 0% 0B 0% 0B 0B 0B 0% 0B -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 1.12GB 0%
Aggregate Metadata 6.93MB 0%
Snapshot Reserve 45.36GB 5%
Total Used 46.48GB 5%
Total Physical Used 83.83MB 0%
Total Provisioned Space 129GB 14%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.
::*> snapshot show -volume vol1
There are no entries matching your query.圧縮が効くような48GiBのファイルを作成します。
$ sudo mount -t nfs svm-0a7c58a5f3a47283c.fs-012f5aba611482f32.fsx.us-east-1.amazonaws.com:/vol1 /mnt/fsxn/vol1
$ df -hT -t nfs4
Filesystem Type Size Used Avail Use% Mounted on
svm-0a7c58a5f3a47283c.fs-012f5aba611482f32.fsx.us-east-1.amazonaws.com:/vol1 nfs4 122G 320K 122G 1% /mnt/fsxn/vol1
$ yes \
$(base64 /dev/urandom -w 0 \
| head -c 1K
) \
| tr -d '\n' \
| sudo dd of=/mnt/fsxn/vol1/1KB_random_pattern_text_block_48GiB bs=4M count=12288 iflag=fullblock
12288+0 records in
12288+0 records out
51539607552 bytes (52 GB, 48 GiB) copied, 346.536 s, 149 MB/sファイル書き込み後のStorage Efficiency、ボリューム、aggregateの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ -------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Disabled Idle for 00:36:14 Sat Jan 20 04:02:39 2024 Sat Jan 20 04:02:39 2024 0B 0% 0B 48.20GB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ----- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 128GB 73.40GB 128GB 121.6GB 48.20GB 39% 0B 0% 0B 48.20GB 38% 48.20GB 40% - 48.20GB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.20GB 5%
Footprint in Performance Tier 48.23GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 27.50MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.54GB 5%
Effective Total Footprint 48.54GB 5%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId012f5aba611482f32-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 48.19GB
Total Physical Used: 48.19GB
Total Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used Without Snapshots: 48.19GB
Total Data Reduction Physical Used Without Snapshots: 48.19GB
Total Data Reduction Efficiency Ratio Without Snapshots: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones: 48.19GB
Total Data Reduction Physical Used without snapshots and flexclones: 48.19GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 48.20GB
Total Physical Used in FabricPool Performance Tier: 48.34GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.20GB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.34GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Logical Space Used for All Volumes: 48.19GB
Physical Space Used for All Volumes: 48.19GB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 48.19GB
Physical Space Used by the Aggregate: 48.19GB
Space Saved by Aggregate Data Reduction: 0B
Aggregate Data Reduction SE Ratio: 1.00:1
Logical Size Used by Snapshot Copies: 308KB
Physical Size Used by Snapshot Copies: 132KB
Snapshot Volume Data Reduction Ratio: 2.33:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 2.33:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 1
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used, sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 812.2GB 861.8GB 49.60GB 48.53GB 5% 0B 0% 0B 0B 0B 0% 0B -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.56GB 5%
Aggregate Metadata 44.20MB 0%
Snapshot Reserve 45.36GB 5%
Total Used 94.96GB 10%
Total Physical Used 48.53GB 5%
Total Provisioned Space 129GB 14%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.Snapshotの取得
Snapshotを取得します。
::*> snapshot create -vserver svm -volume vol1 -snapshot test.2024-01-20_0439 -snapmirror-label test
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-20_0439 152KB 0% 0%
::*> snapshot show -volume vol1 -instance
Vserver: svm
Volume: vol1
Snapshot: test.2024-01-20_0439
Snapshot Data Set ID: 4294968322
Snapshot Master Data Set ID: 6452728471
Creation Time: Sat Jan 20 04:39:43 2024
Snapshot Busy: false
List of Owners: -
Snapshot Size: 152KB
Percentage of Total Blocks: 0%
Percentage of Used Blocks: 0%
Consistency Point Count: 165
Comment: -
File System Version: 9.13
File System Block Format: 64-bit
Physical Snap ID: 1
Logical Snap ID: 1
Database Record Owner: -
Snapshot Tags: SM_Label=test
Instance UUID: 09b46587-f567-4a9e-abc1-05e7ee340b32
Version UUID: 09b46587-f567-4a9e-abc1-05e7ee340b32
7-Mode Snapshot: false
Label for SnapMirror Operations: test
Snapshot State: -
Constituent Snapshot: false
Node: FsxId012f5aba611482f32-01
AFS Size from Snapshot: 48.20GB
Compression Savings from Snapshot: 0B
Dedup Savings from Snapshot: 0B
VBN Zero Savings from Snapshot: 0B
Reserved (holes and overwrites) in Snapshot: 0B
Snapshot Logical Used: 48.20GB
Performance Metadata from Snapshot: 1.52MB
Snapshot Inofile Version: 4
Expiry Time: -
Compression Type: none
SnapLock Expiry Time: -
Application IO Size: -
Is Qtree Caching Support Enabled: false
Compression Algorithm: lzopro
Snapshot Created for Conversion: falseInactive data compressionの実行
Inactive data compressionの実行をします。
::*> volume efficiency on -vserver svm -volume vol1
Efficiency for volume "vol1" of Vserver "svm" is enabled.
::*> volume efficiency modify -vserver svm -volume vol1 -compression true
::*> volume efficiency inactive-data-compression modify -vserver svm -volume vol1 -is-enabled true
::*> volume efficiency inactive-data-compression start -volume vol1 -inactive-days 0
Inactive data compression scan started on volume "vol1" in Vserver "svm"
::*> volume efficiency inactive-data-compression show -instance
Volume: vol1
Vserver: svm
Is Enabled: true
Scan Mode: default
Progress: RUNNING
Status: SUCCESS
Compression Algorithm: lzopro
Failure Reason: -
Total Blocks: -
Total blocks Processed: -
Percentage: 0%
Phase1 L1s Processed: 2454
Phase1 Lns Skipped:
L1: 0
L2: 0
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
Phase2 Total Blocks: 0
Phase2 Blocks Processed: 0
Number of Cold Blocks Encountered: 625632
Number of Repacked Blocks: 0
Number of Compression Done Blocks: 622088
Number of Vol-Overwrites: 0
Time since Last Inactive Data Compression Scan started(sec): 0
Time since Last Inactive Data Compression Scan ended(sec): 0
Time since Last Successful Inactive Data Compression Scan started(sec): -
Time since Last Successful Inactive Data Compression Scan ended(sec): 0
Average time for Cold Data Compression(sec): 0
Tuning Enabled: true
Threshold: 14
Threshold Upper Limit: 21
Threshold Lower Limit: 14
Client Read history window: 14
Incompressible Data Percentage: 0%
::*> volume efficiency inactive-data-compression show -instance
Volume: vol1
Vserver: svm
Is Enabled: true
Scan Mode: default
Progress: RUNNING
Status: SUCCESS
Compression Algorithm: lzopro
Failure Reason: -
Total Blocks: -
Total blocks Processed: -
Percentage: 73%
Phase1 L1s Processed: 48924
Phase1 Lns Skipped:
L1: 0
L2: 0
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
Phase2 Total Blocks: 16506912
Phase2 Blocks Processed: 12089718
Number of Cold Blocks Encountered: 12630880
Number of Repacked Blocks: 0
Number of Compression Done Blocks: 12589968
Number of Vol-Overwrites: 0
Time since Last Inactive Data Compression Scan started(sec): 0
Time since Last Inactive Data Compression Scan ended(sec): 0
Time since Last Successful Inactive Data Compression Scan started(sec): -
Time since Last Successful Inactive Data Compression Scan ended(sec): 0
Average time for Cold Data Compression(sec): 0
Tuning Enabled: true
Threshold: 14
Threshold Upper Limit: 21
Threshold Lower Limit: 14
Client Read history window: 14
Incompressible Data Percentage: 0%
::*> volume efficiency inactive-data-compression show -instance
Volume: vol1
Vserver: svm
Is Enabled: true
Scan Mode: -
Progress: IDLE
Status: SUCCESS
Compression Algorithm: lzopro
Failure Reason: -
Total Blocks: -
Total blocks Processed: -
Percentage: -
Phase1 L1s Processed: -
Phase1 Lns Skipped: -
Phase2 Total Blocks: -
Phase2 Blocks Processed: -
Number of Cold Blocks Encountered: 12630880
Number of Repacked Blocks: 0
Number of Compression Done Blocks: 12589968
Number of Vol-Overwrites: 0
Time since Last Inactive Data Compression Scan started(sec): 14
Time since Last Inactive Data Compression Scan ended(sec): 1
Time since Last Successful Inactive Data Compression Scan started(sec): -
Time since Last Successful Inactive Data Compression Scan ended(sec): 1
Average time for Cold Data Compression(sec): 13
Tuning Enabled: true
Threshold: 14
Threshold Upper Limit: 21
Threshold Lower Limit: 14
Client Read history window: 14
Incompressible Data Percentage: 0%Inactive data compressionの実行が完了して5分後のStorage Efficiency、ボリューム、aggregate、Snapshotの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ ------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Enabled Idle for 00:48:30 Sat Jan 20 04:02:39 2024 Sat Jan 20 04:02:39 2024 0B 0% 0B 48.21GB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ----- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 128GB 73.39GB 128GB 121.6GB 48.21GB 39% 0B 0% 0B 48.35GB 38% 48.21GB 40% - 48.21GB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.35GB 5%
Footprint in Performance Tier 48.40GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 52.98MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.72GB 5%
Footprint Data Reduction 46.36GB 5%
Auto Adaptive Compression 46.36GB 5%
Effective Total Footprint 2.36GB 0%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId012f5aba611482f32-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 96.06GB
Total Physical Used: 9.80GB
Total Storage Efficiency Ratio: 9.80:1
Total Data Reduction Logical Used Without Snapshots: 47.86GB
Total Data Reduction Physical Used Without Snapshots: 9.78GB
Total Data Reduction Efficiency Ratio Without Snapshots: 4.90:1
Total Data Reduction Logical Used without snapshots and flexclones: 47.86GB
Total Data Reduction Physical Used without snapshots and flexclones: 9.78GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 4.90:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 96.41GB
Total Physical Used in FabricPool Performance Tier: 10.30GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 9.36:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.21GB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 10.27GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 4.69:1
Logical Space Used for All Volumes: 47.86GB
Physical Space Used for All Volumes: 47.86GB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 55.74GB
Physical Space Used by the Aggregate: 9.80GB
Space Saved by Aggregate Data Reduction: 45.94GB
Aggregate Data Reduction SE Ratio: 5.69:1
Logical Size Used by Snapshot Copies: 48.20GB
Physical Size Used by Snapshot Copies: 147.8MB
Snapshot Volume Data Reduction Ratio: 334.09:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 334.09:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 0
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used, sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 847.9GB 861.8GB 13.90GB 12.67GB 1% 45.94GB 77% 2.05GB 0B 45.94GB 77% 2.05GB -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.73GB 5%
Aggregate Metadata 10.12GB 1%
Snapshot Reserve 45.36GB 5%
Total Used 59.26GB 7%
Total Physical Used 12.67GB 1%
Total Provisioned Space 129GB 14%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-20_0439 147.6MB 0% 0%
::*> snapshot show -volume vol1 -instance
Vserver: svm
Volume: vol1
Snapshot: test.2024-01-20_0439
Snapshot Data Set ID: 4294968322
Snapshot Master Data Set ID: 6452728471
Creation Time: Sat Jan 20 04:39:43 2024
Snapshot Busy: false
List of Owners: -
Snapshot Size: 147.6MB
Percentage of Total Blocks: 0%
Percentage of Used Blocks: 0%
Consistency Point Count: 165
Comment: -
File System Version: 9.13
File System Block Format: 64-bit
Physical Snap ID: 1
Logical Snap ID: 1
Database Record Owner: -
Snapshot Tags: SM_Label=test
Instance UUID: 09b46587-f567-4a9e-abc1-05e7ee340b32
Version UUID: 09b46587-f567-4a9e-abc1-05e7ee340b32
7-Mode Snapshot: false
Label for SnapMirror Operations: test
Snapshot State: -
Constituent Snapshot: false
Node: FsxId012f5aba611482f32-01
AFS Size from Snapshot: 48.20GB
Compression Savings from Snapshot: 0B
Dedup Savings from Snapshot: 0B
VBN Zero Savings from Snapshot: 0B
Reserved (holes and overwrites) in Snapshot: 0B
Snapshot Logical Used: 48.20GB
Performance Metadata from Snapshot: 1.52MB
Snapshot Inofile Version: 4
Expiry Time: -
Compression Type: none
SnapLock Expiry Time: -
Application IO Size: -
Is Qtree Caching Support Enabled: false
Compression Algorithm: lzopro
Snapshot Created for Conversion: false注目すべきポイントは以下のとおりです。
aggr show-efficiencyのTotal Physical Used: 48.19GB -> 9.80GBaggr show-efficiencyのPhysical Size Used by Snapshot Copies: 48.19GB -> 147.8MBaggr showのphysical-used: 48.53GB -> 12.67GB
いずれも大幅に削減されています。
また、約140MBほどSnapshotのサイズが増加しています。
CloudWatchメトリクスからもStorage Efficiencyによるデータ削減量StorageEfficiencySavingsとSSDとキャパシティプールストレージの物理データ使用量であるStorageUsedの値を確認します。
Inactive data compressionを実行したタイミングでストレージの物理使用量が減り、データ削減量が増えていることが分かります。
つまりはSnapshot取得後のInactive data compressionでも効果があることが分かります。
もし、Snapshot取得後にInactive data compressionによるデータ削減効果が得られないとするならば残念だったのですが、これは嬉しいですね。
Snapshotはデフォルトのポリシーでも毎時、日次、週次で取得しています。そしてInactive data compressionは日次で動作します。仮にInactive data compressionがSnapshotで取得しているデータブロックに対して効果がないのであれば、デフォルトのSnapshotポリシーで動作している環境でInactive data compressionを有効化する効果は薄いと考えます。
Snapshotを削除した際の物理使用量の変化
Snapshotを削除した場合、物理使用量が変化するのか確認します。
Snapshotを削除します。
::*> snapshot delete -vserver svm -volume vol1 -snapshot test.2024-01-20_0439
Warning: Deleting a Snapshot copy permanently removes data that is stored only in that Snapshot copy. Are you sure you want to delete Snapshot copy "test.2024-01-20_0439" for
volume "vol1" in Vserver "svm" ? {y|n}: y
::*> snapshot show -volume vol1
There are no entries matching your query.Storage Efficiency、ボリューム、aggregateの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ ------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Enabled Idle for 01:04:37 Sat Jan 20 04:02:39 2024 Sat Jan 20 04:02:39 2024 0B 0% 0B 48.21GB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ----- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 128GB 73.39GB 128GB 121.6GB 48.21GB 39% 0B 0% 0B 48.21GB 38% 48.21GB 40% - 48.21GB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.21GB 5%
Footprint in Performance Tier 48.41GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 200.9MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.72GB 5%
Footprint Data Reduction 46.36GB 5%
Auto Adaptive Compression 46.36GB 5%
Effective Total Footprint 2.36GB 0%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId012f5aba611482f32-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 47.86GB
Total Physical Used: 9.79GB
Total Storage Efficiency Ratio: 4.89:1
Total Data Reduction Logical Used Without Snapshots: 47.86GB
Total Data Reduction Physical Used Without Snapshots: 9.79GB
Total Data Reduction Efficiency Ratio Without Snapshots: 4.89:1
Total Data Reduction Logical Used without snapshots and flexclones: 47.86GB
Total Data Reduction Physical Used without snapshots and flexclones: 9.79GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 4.89:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 48.21GB
Total Physical Used in FabricPool Performance Tier: 10.29GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 4.69:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.21GB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 10.29GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 4.69:1
Logical Space Used for All Volumes: 47.86GB
Physical Space Used for All Volumes: 47.86GB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 55.73GB
Physical Space Used by the Aggregate: 9.79GB
Space Saved by Aggregate Data Reduction: 45.94GB
Aggregate Data Reduction SE Ratio: 5.69:1
Logical Size Used by Snapshot Copies: 660KB
Physical Size Used by Snapshot Copies: 272KB
Snapshot Volume Data Reduction Ratio: 2.43:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 2.43:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 0
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used, sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 847.8GB 861.8GB 13.94GB 12.72GB 1% 45.94GB 77% 2.05GB 0B 45.94GB 77% 2.05GB -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.73GB 5%
Aggregate Metadata 10.16GB 1%
Snapshot Reserve 45.36GB 5%
Total Used 59.30GB 7%
Total Physical Used 12.72GB 1%
Total Provisioned Space 129GB 14%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.aggr show-efficiencyのTotal Physical Usedが9.80GBから9.79GBに減少したなど微々たるもので大きな変化はありませんでした。
Snapshot取得後にファイルを削除した状態でInactive data compressionを実行した場合の物理使用量の変化
ファイルの書き込み
ふと、「Snapshot取得後にファイルを削除した状態でInactive data compressionを実行した場合はどんな挙動になるのだろう」と気になりました。
要するに圧縮対象のデータがAFSではなく、Snapshot上にしかない場合です。
実際に試してみます。
新しくFSxNファイルシステムを作り直しました。
再度ファイルの書き込みを行います。
$ sudo mount -t nfs svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1 /mnt/fsxn/vol1
$ yes \
$(base64 /dev/urandom -w 0 \
| head -c 1K
) \
| tr -d '\n' \
| sudo dd of=/mnt/fsxn/vol1/1KB_random_pattern_text_block_48GiB bs=4M count=12288 iflag=fullblock
12288+0 records in
12288+0 records out
51539607552 bytes (52 GB, 48 GiB) copied, 346.775 s, 149 MB/s
$ ls -lh /mnt/fsxn/vol1
total 49G
-rw-r--r--. 1 root root 48G Jan 22 06:49 1KB_random_pattern_text_block_48GiBファイル書き込み後のStorage Efficiency、ボリューム、aggregateの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ -------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Disabled Idle for 00:19:03 Mon Jan 22 06:33:53 2024 Mon Jan 22 06:33:53 2024 0B 0% 0B 48.20GB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ---- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 64GB 12.60GB 64GB 60.80GB 48.20GB 79% 0B 0% 0B 48.20GB 75% 48.20GB 79% - 48.20GB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.20GB 5%
Footprint in Performance Tier 48.23GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 29.33MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.55GB 5%
Effective Total Footprint 48.55GB 5%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId0451902d5d3fc71e2-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 48.19GB
Total Physical Used: 48.19GB
Total Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used Without Snapshots: 48.19GB
Total Data Reduction Physical Used Without Snapshots: 48.19GB
Total Data Reduction Efficiency Ratio Without Snapshots: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones: 48.19GB
Total Data Reduction Physical Used without snapshots and flexclones: 48.19GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 48.20GB
Total Physical Used in FabricPool Performance Tier: 48.34GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.20GB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.34GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Logical Space Used for All Volumes: 48.19GB
Physical Space Used for All Volumes: 48.19GB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 48.19GB
Physical Space Used by the Aggregate: 48.19GB
Space Saved by Aggregate Data Reduction: 0B
Aggregate Data Reduction SE Ratio: 1.00:1
Logical Size Used by Snapshot Copies: 0B
Physical Size Used by Snapshot Copies: 0B
Snapshot Volume Data Reduction Ratio: 1.00:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 1.00:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 1
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used,sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 812.2GB 861.8GB 49.60GB 48.49GB 5% 0B 0% 0B 0B 0B 0% 0B -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.56GB 5%
Aggregate Metadata 44.08MB 0%
Snapshot Reserve 45.36GB 5%
Total Used 94.96GB 10%
Total Physical Used 48.49GB 5%
Total Provisioned Space 65GB 7%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.Snapshotの取得
Snapshotを取得します。
::*> snapshot create -vserver svm -volume vol1 -snapshot test.2024-01-22_0654 -snapmirror-label test
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-22_0654 144KB 0% 0%
::*> snapshot show -volume vol1 -instance
Vserver: svm
Volume: vol1
Snapshot: test.2024-01-22_0654
Snapshot Data Set ID: 4294968322
Snapshot Master Data Set ID: 6451744734
Creation Time: Mon Jan 22 06:54:03 2024
Snapshot Busy: false
List of Owners: -
Snapshot Size: 144KB
Percentage of Total Blocks: 0%
Percentage of Used Blocks: 0%
Consistency Point Count: 166
Comment: -
File System Version: 9.13
File System Block Format: 64-bit
Physical Snap ID: 1
Logical Snap ID: 1
Database Record Owner: -
Snapshot Tags: SM_Label=test
Instance UUID: 39be6539-c91a-45e0-bd50-b9d60aba5c1f
Version UUID: 39be6539-c91a-45e0-bd50-b9d60aba5c1f
7-Mode Snapshot: false
Label for SnapMirror Operations: test
Snapshot State: -
Constituent Snapshot: false
Node: FsxId0451902d5d3fc71e2-01
AFS Size from Snapshot: 48.20GB
Compression Savings from Snapshot: 0B
Dedup Savings from Snapshot: 0B
VBN Zero Savings from Snapshot: 0B
Reserved (holes and overwrites) in Snapshot: 0B
Snapshot Logical Used: 48.20GB
Performance Metadata from Snapshot: 1.52MB
Snapshot Inofile Version: 4
Expiry Time: -
Compression Type: none
SnapLock Expiry Time: -
Application IO Size: -
Is Qtree Caching Support Enabled: false
Compression Algorithm: lzopro
Snapshot Created for Conversion: falseファイルの削除
ボリューム上のファイルを削除します。
$ sudo rm /mnt/fsxn/vol1/1KB_random_pattern_text_block_48GiB
$ ls -lh /mnt/fsxn/vol1
total 0
$ df -hT -t nfs4
Filesystem Type Size Used Avail Use% Mounted on
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1 nfs4 61G 46G 16G 75% /mnt/fsxn/vol1
$ ls -lh /mnt/fsxn/vol1/.snapshot
total 4.0K
drwxr-xr-x. 2 root root 4.0K Jan 22 06:43 test.2024-01-22_0654
$ ls -lh /mnt/fsxn/vol1/.snapshot/test.2024-01-22_0654
total 49G
-rw-r--r--. 1 root root 48G Jan 22 06:49 1KB_random_pattern_text_block_48GiB
$ df -hT -t nfs4
Filesystem Type Size Used Avail Use% Mounted on
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1 nfs4 61G 46G 16G 75% /mnt/fsxn/vol1
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1/.snapshot/test.2024-01-22_0654 nfs4 3.3G 3.3G 0 100% /mnt/fsxn/vol1/.snapshot/test.2024-01-22_0654Snapshotにのみしか存在しないため、Snapshot領域の使用量が大きくなっておりSnapshotオーバーフローが発生しています。
ファイル削除後のStorage Efficiency、ボリューム、aggregate、Snapshotの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ -------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Disabled Idle for 00:28:02 Mon Jan 22 06:33:53 2024 Mon Jan 22 06:33:53 2024 0B 0% 0B 217.7MB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ---- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 64GB 15.79GB 64GB 60.80GB 45.01GB 74% 0B 0% 0B 48.21GB 75% 45.01GB 74% - 217.7MB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.21GB 5%
Footprint in Performance Tier 48.25GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 47.47MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.57GB 5%
Effective Total Footprint 48.57GB 5%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId0451902d5d3fc71e2-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 48.40GB
Total Physical Used: 48.19GB
Total Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used Without Snapshots: 201.5MB
Total Data Reduction Physical Used Without Snapshots: 201.6MB
Total Data Reduction Efficiency Ratio Without Snapshots: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones: 201.5MB
Total Data Reduction Physical Used without snapshots and flexclones: 201.6MB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 48.42GB
Total Physical Used in FabricPool Performance Tier: 48.35GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 218.0MB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 359.4MB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Logical Space Used for All Volumes: 201.5MB
Physical Space Used for All Volumes: 201.5MB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 48.19GB
Physical Space Used by the Aggregate: 48.19GB
Space Saved by Aggregate Data Reduction: 0B
Aggregate Data Reduction SE Ratio: 1.00:1
Logical Size Used by Snapshot Copies: 48.20GB
Physical Size Used by Snapshot Copies: 48.00GB
Snapshot Volume Data Reduction Ratio: 1.00:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 1.00:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 1
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used,sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 812.1GB 861.8GB 49.62GB 48.53GB 5% 0B 0% 0B 0B 0B 0% 0B -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.58GB 5%
Aggregate Metadata 44.54MB 0%
Snapshot Reserve 45.36GB 5%
Total Used 94.98GB 10%
Total Physical Used 48.53GB 5%
Total Provisioned Space 65GB 7%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-22_0654 48.00GB 75% 100%
::*> snapshot show -volume vol1 -instance
Vserver: svm
Volume: vol1
Snapshot: test.2024-01-22_0654
Snapshot Data Set ID: 4294968322
Snapshot Master Data Set ID: 6451744734
Creation Time: Mon Jan 22 06:54:03 2024
Snapshot Busy: false
List of Owners: -
Snapshot Size: 48.00GB
Percentage of Total Blocks: 75%
Percentage of Used Blocks: 100%
Consistency Point Count: 166
Comment: -
File System Version: 9.13
File System Block Format: 64-bit
Physical Snap ID: 1
Logical Snap ID: 1
Database Record Owner: -
Snapshot Tags: SM_Label=test
Instance UUID: 39be6539-c91a-45e0-bd50-b9d60aba5c1f
Version UUID: 39be6539-c91a-45e0-bd50-b9d60aba5c1f
7-Mode Snapshot: false
Label for SnapMirror Operations: test
Snapshot State: -
Constituent Snapshot: false
Node: FsxId0451902d5d3fc71e2-01
AFS Size from Snapshot: 48.20GB
Compression Savings from Snapshot: 0B
Dedup Savings from Snapshot: 0B
VBN Zero Savings from Snapshot: 0B
Reserved (holes and overwrites) in Snapshot: 0B
Snapshot Logical Used: 48.20GB
Performance Metadata from Snapshot: 1.52MB
Snapshot Inofile Version: 4
Expiry Time: -
Compression Type: none
SnapLock Expiry Time: -
Application IO Size: -
Is Qtree Caching Support Enabled: false
Compression Algorithm: lzopro
Snapshot Created for Conversion: falseSnapshotのサイズが48.00GBと非常に大きくなりました。
Inactive data compressionの実行
Inactive data compressionを実行します。
::*> volume efficiency on -vserver svm -volume vol1
Efficiency for volume "vol1" of Vserver "svm" is enabled.
::*> volume efficiency modify -vserver svm -volume vol1 -compression true
::*> volume efficiency inactive-data-compression modify -vserver svm -volume vol1 -is-enabled true
::*> volume efficiency inactive-data-compression start -volume vol1 -inactive-days 0
Inactive data compression scan started on volume "vol1" in Vserver "svm"
::*> volume efficiency inactive-data-compression show -instance
Volume: vol1
Vserver: svm
Is Enabled: true
Scan Mode: default
Progress: RUNNING
Status: SUCCESS
Compression Algorithm: lzopro
Failure Reason: -
Total Blocks: -
Total blocks Processed: -
Percentage: 1%
Phase1 L1s Processed: 0
Phase1 Lns Skipped:
L1: 0
L2: 0
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
Phase2 Total Blocks: 16506912
Phase2 Blocks Processed: 194048
Number of Cold Blocks Encountered: 193152
Number of Repacked Blocks: 0
Number of Compression Done Blocks: 193136
Number of Vol-Overwrites: 0
Time since Last Inactive Data Compression Scan started(sec): 0
Time since Last Inactive Data Compression Scan ended(sec): 0
Time since Last Successful Inactive Data Compression Scan started(sec): -
Time since Last Successful Inactive Data Compression Scan ended(sec): 0
Average time for Cold Data Compression(sec): 0
Tuning Enabled: true
Threshold: 14
Threshold Upper Limit: 21
Threshold Lower Limit: 14
Client Read history window: 14
Incompressible Data Percentage: 0%
::*> volume efficiency inactive-data-compression show -instance
Volume: vol1
Vserver: svm
Is Enabled: true
Scan Mode: -
Progress: IDLE
Status: SUCCESS
Compression Algorithm: lzopro
Failure Reason: -
Total Blocks: -
Total blocks Processed: -
Percentage: -
Phase1 L1s Processed: -
Phase1 Lns Skipped: -
Phase2 Total Blocks: -
Phase2 Blocks Processed: -
Number of Cold Blocks Encountered: 12582728
Number of Repacked Blocks: 0
Number of Compression Done Blocks: 12581224
Number of Vol-Overwrites: 0
Time since Last Inactive Data Compression Scan started(sec): 462
Time since Last Inactive Data Compression Scan ended(sec): 77
Time since Last Successful Inactive Data Compression Scan started(sec): -
Time since Last Successful Inactive Data Compression Scan ended(sec): 77
Average time for Cold Data Compression(sec): 384
Tuning Enabled: true
Threshold: 14
Threshold Upper Limit: 21
Threshold Lower Limit: 14
Client Read history window: 14
Incompressible Data Percentage: 0%Inactive data compressionの実行が完了してから15分後のStorage Efficiency、ボリューム、aggregate、Snapshotの情報は以下のとおりです。
::*> date show -fields date
(cluster date show)
node date
------------------------- -------------------------
FsxId0451902d5d3fc71e2-01 1/22/2024 07:32:12 +00:00
FsxId0451902d5d3fc71e2-02 1/22/2024 07:32:12 +00:00
2 entries were displayed.
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ ------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Enabled Idle for 00:58:19 Mon Jan 22 06:33:53 2024 Mon Jan 22 06:33:53 2024 0B 0% 0B 219.0MB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size, dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ---- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 64GB 15.79GB 64GB 60.80GB 45.01GB 74% 0B 0% 0B 48.21GB 75% 45.01GB 74% - 219.0MB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.21GB 5%
Footprint in Performance Tier 48.28GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 69.23MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.59GB 5%
Footprint Data Reduction 46.20GB 5%
Auto Adaptive Compression 46.20GB 5%
Effective Total Footprint 2.39GB 0%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId0451902d5d3fc71e2-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 48.21GB
Total Physical Used: 9.83GB
Total Storage Efficiency Ratio: 4.90:1
Total Data Reduction Logical Used Without Snapshots: 6.12MB
Total Data Reduction Physical Used Without Snapshots: 1.37GB
Total Data Reduction Efficiency Ratio Without Snapshots: 1.00:1
Total Data Reduction Logical Used without snapshots and flexclones: 6.12MB
Total Data Reduction Physical Used without snapshots and flexclones: 1.37GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 48.42GB
Total Physical Used in FabricPool Performance Tier: 10.18GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 4.76:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 219.3MB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 1.72GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Logical Space Used for All Volumes: 6.12MB
Physical Space Used for All Volumes: 6.12MB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 55.77GB
Physical Space Used by the Aggregate: 9.83GB
Space Saved by Aggregate Data Reduction: 45.94GB
Aggregate Data Reduction SE Ratio: 5.67:1
Logical Size Used by Snapshot Copies: 48.20GB
Physical Size Used by Snapshot Copies: 48.00GB
Snapshot Volume Data Reduction Ratio: 1.00:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 1.00:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 0
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used,sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-saved sis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- --------------- ----------------------- ---------------- ---------------------------------------
aggr1 848.0GB 861.8GB 13.77GB 12.54GB 1% 45.94GB 77% 2.06GB 0B 45.94GB 77% 2.06GB -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.60GB 5%
Aggregate Metadata 10.11GB 1%
Snapshot Reserve 45.36GB 5%
Total Used 59.13GB 7%
Total Physical Used 12.54GB 1%
Total Provisioned Space 65GB 7%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-22_0654 48.00GB 75% 100%Physical Size Used by Snapshot Copiesは特に変わりありませんが、Total Physical Usedとphysical-usedは先ほどの検証と同様に約35GB以上減少しています。
CloudWatchメトリクスからStorageUsedとStorageEfficiencySavingsを確認します。
Inactive data compressionを実行したタイミングでストレージの物理使用量が減っていますが、データ削減量は全く変わっていません。謎です。
SnapRestore後の物理データ使用量の変化
SnapRestore後の物理データ使用量の変化を確認します。
SnapRestoreをします。
::*> snapshot restore -vserver svm -volume vol1 -snapshot test.2024-01-22_0654
Warning: Quota rules currently enforced on volume "vol1" might change during this operation. If the currently enforced quota rules are different from those in Snapshot copy "test.2024-01-22_0654", you
might have to resize or reinitialize quotas on this volume after this operation.
Do you want to continue? {y|n}: y
Warning: Export policies currently enforced on the qtrees of volume "vol1" will not change during this operation. If the currently enforced export policies are different from those in Snapshot copy
"test.2024-01-22_0654", reassign the export policies of the qtrees on this volume after this operation.
Do you want to continue? {y|n}: ySnapRestore後、NFSクライアントからファイルがリストアされたことを確認します。
$ ls -lh /mnt/fsxn/vol1/
total 49G
-rw-r--r--. 1 root root 48G Jan 22 06:49 1KB_random_pattern_text_block_48GiB
$ df -hT -t nfs4
Filesystem Type Size Used Avail Use% Mounted on
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1 nfs4 61G 49G 13G 80% /mnt/fsxn/vol1
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1/.snapshot/test.2024-01-22_0654 nfs4 3.3G 194M 3.1G 6% /mnt/fsxn/vol1/.snapshot/test.2024-01-22_0654Snapshot領域の使用量が大幅に減っていますね。
Storage Efficiency、ボリューム、aggregate、Snapshotの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ -------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Disabled Idle for 01:11:00 Mon Jan 22 06:33:53 2024 Mon Jan 22 06:33:53 2024 0B 0% 0B 48.21GB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size,dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ---- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 64GB 12.59GB 64GB 60.80GB 48.21GB 79% 0B 0% 0B 48.40GB 76% 48.21GB 79% - 48.21GB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.40GB 5%
Footprint in Performance Tier 48.45GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 53.50MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.76GB 5%
Footprint Data Reduction 46.36GB 5%
Auto Adaptive Compression 46.36GB 5%
Effective Total Footprint 2.40GB 0%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId0451902d5d3fc71e2-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 96.19GB
Total Physical Used: 10.00GB
Total Storage Efficiency Ratio: 9.62:1
Total Data Reduction Logical Used Without Snapshots: 47.98GB
Total Data Reduction Physical Used Without Snapshots: 9.97GB
Total Data Reduction Efficiency Ratio Without Snapshots: 4.81:1
Total Data Reduction Logical Used without snapshots and flexclones: 47.98GB
Total Data Reduction Physical Used without snapshots and flexclones: 9.97GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 4.81:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 96.41GB
Total Physical Used in FabricPool Performance Tier: 10.37GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 9.30:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 48.21GB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 10.33GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 4.67:1
Logical Space Used for All Volumes: 47.98GB
Physical Space Used for All Volumes: 47.98GB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 55.94GB
Physical Space Used by the Aggregate: 10.00GB
Space Saved by Aggregate Data Reduction: 45.94GB
Aggregate Data Reduction SE Ratio: 5.59:1
Logical Size Used by Snapshot Copies: 48.20GB
Physical Size Used by Snapshot Copies: 193.8MB
Snapshot Volume Data Reduction Ratio: 254.69:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 254.69:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 1
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used, sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-savedsis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- -------------------------------------- ---------------- ---------------------------------------
aggr1 847.8GB 861.8GB 14.00GB 12.77GB 1% 45.94GB 77% 2.06GB 0B 45.94GB77% 2.06GB -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.77GB 5%
Aggregate Metadata 10.16GB 1%
Snapshot Reserve 45.36GB 5%
Total Used 59.35GB 7%
Total Physical Used 12.77GB 1%
Total Provisioned Space 65GB 7%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-22_0654 193.7MB 0% 0%Physical Size Used by Snapshot CopiesとSnapshotのサイズがどちらも193.8MBと小さくなりましたね。
CloudWatchメトリクスからStorageUsedとStorageEfficiencySavings、ボリュームのSnapshotの使用量を確認します。
SnapRestore前後でストレージの物理使用量は変わりありませんが、データ削減量が増加し、Snapshotの使用量が減少しました。
再度ファイルを削除
再度ファイルを削除するとどうなるでしょうか。データ削減量はまた0になるのでしょうか。
試しにファイルを削除します。
$ sudo rm /mnt/fsxn/vol1/1KB_random_pattern_text_block_48GiB
$ ls -lh /mnt/fsxn/vol1/.
total 0
$ df -hT -t nfs4
Filesystem Type Size Used Avail Use% Mounted on
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1 nfs4 61G 49G 13G 80% /mnt/fsxn/vol1
svm-073883c3dde85c8e0.fs-0451902d5d3fc71e2.fsx.us-east-1.amazonaws.com:/vol1/.snapshot/test.2024-01-22_0654 nfs4 3.3G 3.3G 0 100% /mnt/fsxn/vol1/.snapshot/test.2024-01-22_0654ファイル削除後のStorage Efficiency、ボリューム、aggregate、Snapshotの情報は以下のとおりです。
::*> volume efficiency show -volume vol1 -fields changelog-usage, changelog-size, logical-data-size, state, progress, last-op-size, last-op-begin, last-op-end
vserver volume state progress last-op-begin last-op-end last-op-size changelog-usage changelog-size logical-data-size
------- ------ -------- ----------------- ------------------------ ------------------------ ------------ --------------- -------------- -----------------
svm vol1 Disabled Idle for 01:22:28 Mon Jan 22 06:33:53 2024 Mon Jan 22 06:33:53 2024 0B 0% 0B 16.91MB
::*> volume show -volume vol1 -fields available, filesystem-size, total, used, percent-used, performance-tier-inactive-user-data, performance-tier-inactive-user-data-percent, size,dedupe-space-saved, dedupe-space-saved-percent, dedupe-space-shared,logical-used, logical-used-percent,logical-used-by-afs, logical-available, physical-used, physical-used-percent
vserver volume size available filesystem-size total used percent-used dedupe-space-saved dedupe-space-saved-percent dedupe-space-shared physical-used physical-used-percent logical-used logical-used-percent logical-available logical-used-by-afs performance-tier-inactive-user-data performance-tier-inactive-user-data-percent
------- ------ ---- --------- --------------- ------- ------- ------------ ------------------ -------------------------- ------------------- ------------- --------------------- ------------ -------------------- ----------------- ------------------- ----------------------------------- -------------------------------------------
svm vol1 64GB 15.79GB 64GB 60.80GB 45.01GB 74% 0B 0% 0B 48.21GB 75% 45.01GB 74% - 21.59MB 0B 0%
::*> volume show-footprint -volume vol1
Vserver : svm
Volume : vol1
Feature Used Used%
-------------------------------- ---------- -----
Volume Data Footprint 48.21GB 5%
Footprint in Performance Tier 48.48GB 100%
Footprint in FSxFabricpoolObjectStore 0B 0%
Volume Guarantee 0B 0%
Flexible Volume Metadata 322.4MB 0%
Delayed Frees 274.8MB 0%
File Operation Metadata 4KB 0%
Total Footprint 48.79GB 5%
Footprint Data Reduction 46.39GB 5%
Auto Adaptive Compression 46.39GB 5%
Effective Total Footprint 2.40GB 0%
::*> aggr show-efficiency -instance
Name of the Aggregate: aggr1
Node where Aggregate Resides: FsxId0451902d5d3fc71e2-01
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 48.00GB
Total Physical Used: 10.00GB
Total Storage Efficiency Ratio: 4.80:1
Total Data Reduction Logical Used Without Snapshots: 48.00GB
Total Data Reduction Physical Used Without Snapshots: 1.38GB
Total Data Reduction Efficiency Ratio Without Snapshots: 34.68:1
Total Data Reduction Logical Used without snapshots and flexclones: 48.00GB
Total Data Reduction Physical Used without snapshots and flexclones: 1.38GB
Total Data Reduction Efficiency Ratio without snapshots and flexclones: 34.68:1
Total Logical Size Used by Volumes, Clones, Snapshot Copies in the FabricPool Performance Tier: 48.22GB
Total Physical Used in FabricPool Performance Tier: 10.36GB
Total FabricPool Performance Tier Storage Efficiency Ratio: 4.65:1
Total Data Reduction Logical Used without snapshots and flexclones in the FabricPool Performance Tier: 17.23MB
Total Data Reduction Physical Used without snapshots and flexclones in the FabricPool Performance Tier: 1.75GB
Total FabricPool Performance Tier Data Reduction Efficiency Ratio without snapshots and flexclones: 1.00:1
Logical Space Used for All Volumes: 17.23MB
Physical Space Used for All Volumes: 17.23MB
Space Saved by Volume Deduplication: 0B
Space Saved by Volume Deduplication and pattern detection: 0B
Volume Deduplication Savings ratio: 1.00:1
Space Saved by Volume Compression: 0B
Volume Compression Savings ratio: 1.00:1
Space Saved by Inline Zero Pattern Detection: 0B
Volume Data Reduction SE Ratio: 1.00:1
Logical Space Used by the Aggregate: 55.93GB
Physical Space Used by the Aggregate: 10.00GB
Space Saved by Aggregate Data Reduction: 45.94GB
Aggregate Data Reduction SE Ratio: 5.59:1
Logical Size Used by Snapshot Copies: 48.20GB
Physical Size Used by Snapshot Copies: 48.19GB
Snapshot Volume Data Reduction Ratio: 1.00:1
Logical Size Used by FlexClone Volumes: 0B
Physical Sized Used by FlexClone Volumes: 0B
FlexClone Volume Data Reduction Ratio: 1.00:1
Snapshot And FlexClone Volume Data Reduction SE Ratio: 1.00:1
Number of Volumes Offline: 0
Number of SIS Disabled Volumes: 1
Number of SIS Change Log Disabled Volumes: 1
::*> aggr show -fields availsize, usedsize, size, physical-used, physical-used-percent, data-compaction-space-saved, data-compaction-space-saved-percent, data-compacted-count, composite-capacity-tier-used, sis-space-saved, sis-space-saved-percent, sis-shared-count, inactive-data-reporting-start-timestamp
aggregate availsize size usedsize physical-used physical-used-percent data-compaction-space-saved data-compaction-space-saved-percent data-compacted-count composite-capacity-tier-used sis-space-savedsis-space-saved-percent sis-shared-count inactive-data-reporting-start-timestamp
--------- --------- ------- -------- ------------- --------------------- --------------------------- ----------------------------------- -------------------- ---------------------------- -------------------------------------- ---------------- ---------------------------------------
aggr1 847.7GB 861.8GB 14.06GB 12.84GB 1% 45.94GB 77% 2.06GB 0B 45.94GB77% 2.06GB -
::*> aggr show-space
Aggregate : aggr1
Performance Tier
Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 49.80GB 5%
Aggregate Metadata 10.20GB 1%
Snapshot Reserve 45.36GB 5%
Total Used 59.42GB 7%
Total Physical Used 12.84GB 1%
Total Provisioned Space 65GB 7%
Aggregate : aggr1
Object Store: FSxFabricpoolObjectStore
Feature Used Used%
-------------------------------- ---------- ------
Logical Used 0B -
Logical Referenced Capacity 0B -
Logical Unreferenced Capacity 0B -
Total Physical Used 0B -
2 entries were displayed.
::*> snapshot show -volume vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- -------- ------------------------------------- -------- ------ -----
svm vol1
test.2024-01-22_0654 48.19GB 75% 100%Physical Size Used by Snapshot CopiesとSnapshotのサイズがどちらも48.19Gと大きくなりましたね。
CloudWatchメトリクスを確認します。
ファイルを削除しても、データ削減量は減りませんでした。むしろ増えています。謎です。
StorageEfficiencySavingsは参考値として捉えるのが良さそうです。
SnapshotはInactive data compressionを実行する際の障壁にならない
Snapshot取得後にInactive data compressionを実行しても物理使用量が減少することを確認しました。
SnapshotがInactive data compressionを実行する際の障壁にならないことが分かって安心しました。
Inactive data compressionを実行したことによるパフォーマンスへの影響が少ないワークロードではInactive data compressionを有効にして運用したいですね。
この記事が誰かの助けになれば幸いです。
以上、AWS事業本部 コンサルティング部の のんピ(@non____97)でした!
3
