Amazon FSx for NetApp ONTAPにおけるSnapMirrorの同時転送数の上限を確認してみた

最大同時転送数やボリューム数、SnapMirrorによる負荷を考慮した上で実行間隔を決定しよう
2023.12.30

SnapMirrorで同時にいくつ転送できるか気になる

こんにちは、のんピ(@non____97)です。

皆さんはAmazon FSx for NetApp ONTAP(以降FSxN)におけるSnapMirrorの最大同時転送数が気になったことはありますか? 私はあります。

SnapMirrorの最大同時転送数が少なければ、大量のボリュームの移行やバックアップをするときに時間がかかりそうです。

SnapMirrorの同時転送数の上限はプラットフォームの種類やONTAPのバージョンによって異なるようです。具体的な値は以下NetAppのTRに記載がありました。

同時レプリケーション処理数

同時に実行できるSnapMirror処理数には制限があります。この制限はノードごとに設定され、プラットフォームの種類およびData ONTAPのバージョンによって異なります。ノードモデルごとに許可される同時SnapMirror処理数については、「NetApp Hardware Universe」を参照してください。

ONTAPのバージョン別およびコントローラモデル別のノードあたりの同時SnapMirror転送の最大数

ONTAP 9向けSnapMirrorの構成およびベストプラクティス ガイド

ただし、プラットフォームの種類に Amazon FSx for NetApp ONTAP はありません。

風の噂によると32らしいです。

本当に32なのか自分の目で確認してみます。

いきなりまとめ

  • 検証した限り、2023/12/30時点のAmazon FSx for NetApp ONTAP(ONTAP 9.13.1P5)におけるSnapMirrorの同時転送数の上限は100
  • 同時転送数の上限以上のリクエストは拒否されるのではなくキューに入れられ、転送を待機する
  • SnapMirror relationshipは105以上作成可能

やってみた

検証環境

検証環境は以下のとおりです。

Amazon FSx for NetApp ONTAPにおけるSnapMirrorの同時転送数の上限を確認してみた検証環境構成図

どちらのFSxNファイルシステムもONTAP 9.13.1P5です。

::> version
NetApp Release 9.13.1P5: Thu Nov 02 20:37:09 UTC 2023

クラスターピアリングの作成

まず、クラスターピアリングをします。

その前にFSxNファイルシステムのLIFのIPアドレスを確認します。

FSxN 1

::> cluster identity show

          Cluster UUID: f92234ac-a056-11ee-981e-bdd56ead09c8
          Cluster Name: FsxId0ab6f9b00824a187c
 Cluster Serial Number: 1-80-000011
      Cluster Location:
       Cluster Contact:

FsxId0ab6f9b00824a187c::> network interface show -service-policy default-intercluster
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
FsxId0ab6f9b00824a187c
            inter_1      up/up    10.0.8.123/24      FsxId0ab6f9b00824a187c-01
                                                                   e0e     true
            inter_2      up/up    10.0.8.84/24       FsxId0ab6f9b00824a187c-02
                                                                   e0e     true
2 entries were displayed.

FSxN 2

::> cluster identity show

          Cluster UUID: a98388b8-a587-11ee-aa97-d98765698200
          Cluster Name: FsxId058fb0d9773b79e09
 Cluster Serial Number: 1-80-000011
      Cluster Location:
       Cluster Contact:

::> network interface show -service-policy default-intercluster
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
FsxId058fb0d9773b79e09
            inter_1      up/up    10.0.8.85/24       FsxId058fb0d9773b79e09-01
                                                                   e0e     true
            inter_2      up/up    10.0.8.93/24       FsxId058fb0d9773b79e09-02
                                                                   e0e     true
2 entries were displayed.

FSxN 2からクラスターピアリングを作成します。

::> cluster peer create -peer-addrs 10.0.8.123 10.0.8.84

Notice: Use a generated passphrase or choose a passphrase of 8 or more characters. To ensure the authenticity of the peering relationship, use a phrase or
        sequence of characters that would be hard to guess.

Enter the passphrase:
Confirm the passphrase:

Notice: Now use the same passphrase in the "cluster peer create" command in the other cluster.

FSxN 1からもクラスターピアリングを作成します。

FsxId0ab6f9b00824a187c::> cluster peer create -peer-addrs 10.0.8.85 10.0.8.93

Notice: Use a generated passphrase or choose a passphrase of 8 or more characters. To ensure the authenticity of the peering relationship, use a phrase or
        sequence of characters that would be hard to guess.

Enter the passphrase:
Confirm the passphrase:

FsxId0ab6f9b00824a187c::> cluster peer show
Peer Cluster Name         Cluster Serial Number Availability   Authentication
------------------------- --------------------- -------------- --------------
FsxId058fb0d9773b79e09    1-80-000011           Pending        ok

FsxId0ab6f9b00824a187c::> cluster peer show
Peer Cluster Name         Cluster Serial Number Availability   Authentication
------------------------- --------------------- -------------- --------------
FsxId058fb0d9773b79e09    1-80-000011           Available      ok

クラスターピアリングできましたね。

SVMピアリングの作成

続いて、SVMピアリングを行います。

FSxN 1からSVMピアリングを作成します。

::> vserver peer create -vserver svm -peer-vserver svm_fsxn2 -applications snapmirror -peer-cluster FsxId058fb0d9773b79e09

Info: [Job 158] 'vserver peer create' job queued

FSxN 2側でSVMピアリングを承認します。

::> vserver peer show-all
            Peer        Peer                           Peering        Remote
Vserver     Vserver     State        Peer Cluster      Applications   Vserver
----------- ----------- ------------ ----------------- -------------- ---------
svm_fsxn2   svm         pending      FsxId0ab6f9b00824a187c
                                                       snapmirror     svm

::> vserver peer accept -vserver svm_fsxn2 -peer-vserver svm

Info: [Job 45] 'vserver peer accept' job queued

::> vserver peer show-all
            Peer        Peer                           Peering        Remote
Vserver     Vserver     State        Peer Cluster      Applications   Vserver
----------- ----------- ------------ ----------------- -------------- ---------
svm_fsxn2   svm         peered       FsxId0ab6f9b00824a187c
                                                       snapmirror     svm

40個同時にSnapMirror Initialize

それではSnapMirrorを行います。

噂によるとFSxNのSnapMirrorの最大同時転送は32らしいので、それを超える40で試します。

全SnapMirrorの転送元ボリュームはFSxN 1のsvm上のvol1とします。

snapmirror protectを40回入力するのも大変なので、シェルスクリプトで実行するコマンドを書き出します。

$ for i in {01..40}; do
  echo "snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst${i}"
done

snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst01
snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst02
snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst03
.
.
(中略)
.
.
snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst38
snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst39
snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst40

FSxNから書き出したコマンドを実行します。

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst01
[Job 46] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst02
[Job 47] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst03
[Job 48] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".
.
.
(中略)
.
.
::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst38
[Job 83] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst39
[Job 84] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst40
[Job 85] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

SnapMirror relationshipの状態を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated
source-path destination-path     state         status       progress-last-updated
----------- -------------------- ------------- ------------ ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 12/28 15:37:27
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 12/28 15:37:29
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 12/28 15:37:29
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 12/28 15:37:27
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 12/28 15:37:33
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 12/28 15:37:33
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 12/28 15:37:27
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 12/28 15:37:29
8 entries were displayed.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress
source-path destination-path     state         status       total-progress progress-last-updated
----------- -------------------- ------------- ------------ -------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 3.91GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 2.20GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 2.20GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 3.47GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 1.95GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 1.98GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 3.61GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 2.17GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst14 Uninitialized Transferring 420.7MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst16 Uninitialized Transferring 452.6MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst15 Uninitialized Transferring 798.6MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst13 Uninitialized Transferring 711.0MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst09 Uninitialized Transferring 857.9MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst11 Uninitialized Transferring 491.8MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst10 Uninitialized Transferring 585.0MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst12 Uninitialized Transferring 1.25GB         12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst20 Uninitialized Transferring 498.6MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst21 Uninitialized Transferring 360.2MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst23 Uninitialized Transferring 375.2MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst22 Uninitialized Transferring 344.8MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst25 Uninitialized Transferring 253.3MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst26 Uninitialized Transferring 49.77MB        12/28 15:38:30
svm:vol1    svm_fsxn2:vol1_dst17 Uninitialized Transferring 0B             12/28 15:38:31
23 entries were displayed.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress
source-path destination-path     state         status       total-progress progress-last-updated
----------- -------------------- ------------- ------------ -------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 4.38GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 2.46GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 2.45GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 3.97GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 2.20GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 2.20GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 4.17GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 2.43GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst14 Uninitialized Transferring 651.9MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst16 Uninitialized Transferring 714.4MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst15 Uninitialized Transferring 1.03GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst13 Uninitialized Transferring 970.1MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst09 Uninitialized Transferring 1.09GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst11 Uninitialized Transferring 751.2MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst10 Uninitialized Transferring 849.0MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst12 Uninitialized Transferring 1.50GB         12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst20 Uninitialized Transferring 759.8MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst21 Uninitialized Transferring 618.9MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst23 Uninitialized Transferring 630.9MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst22 Uninitialized Transferring 603.1MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst25 Uninitialized Transferring 510.1MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst26 Uninitialized Transferring 304.3MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst24 Uninitialized Transferring 0B             12/28 15:38:55
svm:vol1    svm_fsxn2:vol1_dst18 Uninitialized Transferring 0B             12/28 15:38:46
svm:vol1    svm_fsxn2:vol1_dst27 Uninitialized Transferring 0B             12/28 15:38:49
svm:vol1    svm_fsxn2:vol1_dst17 Uninitialized Transferring 225.0MB        12/28 15:38:45
svm:vol1    svm_fsxn2:vol1_dst19 Uninitialized Transferring 0B             12/28 15:38:49
svm:vol1    svm_fsxn2:vol1_dst29 Uninitialized Transferring 0B             12/28 15:38:55
svm:vol1    svm_fsxn2:vol1_dst31 Uninitialized Transferring 0B             12/28 15:38:48
svm:vol1    svm_fsxn2:vol1_dst30 Uninitialized Transferring 0B             12/28 15:38:55
svm:vol1    svm_fsxn2:vol1_dst32 Uninitialized Transferring 0B             12/28 15:38:48
svm:vol1    svm_fsxn2:vol1_dst37 Uninitialized Transferring 0B             12/28 15:38:56
svm:vol1    svm_fsxn2:vol1_dst39 Uninitialized Transferring 0B             12/28 15:38:56
svm:vol1    svm_fsxn2:vol1_dst38 Uninitialized Transferring 0B             12/28 15:38:56
svm:vol1    svm_fsxn2:vol1_dst33 Uninitialized Transferring 0B             12/28 15:38:56
svm:vol1    svm_fsxn2:vol1_dst28 Uninitialized Transferring 0B             12/28 15:38:56
svm:vol1    svm_fsxn2:vol1_dst34 Uninitialized Transferring 0B             12/28 15:38:55
svm:vol1    svm_fsxn2:vol1_dst40 Uninitialized Transferring 0B             12/28 15:38:56
38 entries were displayed.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress
source-path destination-path     state         status       total-progress progress-last-updated
----------- -------------------- ------------- ------------ -------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 4.67GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 2.59GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 2.59GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 4.28GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 2.35GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 2.34GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 4.40GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 2.57GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst14 Uninitialized Transferring 796.7MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst16 Uninitialized Transferring 857.5MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst15 Uninitialized Transferring 1.17GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst13 Uninitialized Transferring 1.09GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst09 Uninitialized Transferring 1.23GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst11 Uninitialized Transferring 900.5MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst10 Uninitialized Transferring 999.8MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst12 Uninitialized Transferring 1.65GB         12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst20 Uninitialized Transferring 905.4MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst21 Uninitialized Transferring 766.8MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst23 Uninitialized Transferring 781.6MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst22 Uninitialized Transferring 745.4MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst25 Uninitialized Transferring 655.7MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst26 Uninitialized Transferring 450.7MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst24 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst18 Uninitialized Transferring 79.51MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst27 Uninitialized Transferring 100.4MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst17 Uninitialized Transferring 370.3MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst19 Uninitialized Transferring 81.45MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst29 Uninitialized Transferring 0B             12/28 15:39:00
svm:vol1    svm_fsxn2:vol1_dst31 Uninitialized Transferring 83.73MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst30 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst32 Uninitialized Transferring 62.49MB        12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst37 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst39 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst38 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst33 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst28 Uninitialized Transferring 0B             12/28 15:39:01
svm:vol1    svm_fsxn2:vol1_dst34 Uninitialized Transferring 0B             12/28 15:38:55
svm:vol1    svm_fsxn2:vol1_dst40 Uninitialized Transferring 0B             12/28 15:38:56
svm:vol1    svm_fsxn2:vol1_dst35 Uninitialized Transferring 0B             12/28 15:39:03
svm:vol1    svm_fsxn2:vol1_dst36 Uninitialized Transferring 0B             12/28 15:39:01
40 entries were displayed.

じわじわとSnapMirror relationshipが作成され、転送が開始されています。

このままだと初めの方に転送を開始したものが転送終了してしまいそうです。FSxN 1のSnapMirrorrのグローバルスロットリングを調整して、転送速度を落とします。

::> options

FsxId0ab6f9b00824a187c
    replication.throttle.incoming.max_kbs
                                      -                    -
    replication.throttle.outgoing.max_kbs
                                      -                    -
2 entries were displayed.

FsxId0ab6f9b00824a187c::> options -option-name replication.throttle.outgoing.max_kbs 4096
1 entry was modified.

FsxId0ab6f9b00824a187c::> options

FsxId0ab6f9b00824a187c
    replication.throttle.incoming.max_kbs
                                      -                    -
    replication.throttle.outgoing.max_kbs
                                      4096                 -
2 entries were displayed.

しばらくSnapMirrorの様子を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress
source-path destination-path     state         status       total-progress progress-last-updated
----------- -------------------- ------------- ------------ -------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 4.70GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 2.63GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 2.63GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 4.32GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 2.38GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 2.37GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 4.43GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 2.60GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst14 Uninitialized Transferring 829.1MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst16 Uninitialized Transferring 890.3MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst15 Uninitialized Transferring 1.21GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst13 Uninitialized Transferring 1.13GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst09 Uninitialized Transferring 1.26GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst11 Uninitialized Transferring 933.9MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst10 Uninitialized Transferring 1.00GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst12 Uninitialized Transferring 1.68GB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst20 Uninitialized Transferring 938.8MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst21 Uninitialized Transferring 801.4MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst23 Uninitialized Transferring 815.5MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst22 Uninitialized Transferring 779.4MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst25 Uninitialized Transferring 690.3MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst26 Uninitialized Transferring 484.3MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst24 Uninitialized Transferring 10.96MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst18 Uninitialized Transferring 113.4MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst27 Uninitialized Transferring 132.1MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst17 Uninitialized Transferring 403.9MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst19 Uninitialized Transferring 115.7MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst29 Uninitialized Transferring 8.43MB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst31 Uninitialized Transferring 118.3MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst30 Uninitialized Transferring 25.65MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst32 Uninitialized Transferring 96.13MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst37 Uninitialized Transferring 25.65MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst39 Uninitialized Transferring 9.70MB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst38 Uninitialized Transferring 25.65MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst33 Uninitialized Transferring 9.95MB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst28 Uninitialized Transferring 25.65MB        12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst34 Uninitialized Transferring 7.67MB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst40 Uninitialized Transferring 7.67MB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst35 Uninitialized Transferring 8.94MB         12/28 15:43:40
svm:vol1    svm_fsxn2:vol1_dst36 Uninitialized Transferring 25.65MB        12/28 15:43:40
40 entries were displayed.

40個のSnapMirror全て同時に転送されていそうです。

同時転送数の上限32個の噂は気のせいだったかもしれません。

65個同時にSnapMirror Initialize

次に制約があるとしたら64でしょうか。

65同時にSnapMirror Initializeしてみて、確認しましょう。

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst41
[Job 129] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst42
[Job 130] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst43
[Job 131] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".
.
.
(中略)
.
.
::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst63
[Job 152] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst64
[Job 153] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

::> snapmirror protect -path-list svm:vol1 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none -destination-volume-suffix _dst65
[Job 154] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol1".

SnapMirror relationshipの状態を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress
source-path destination-path     state         status       total-progress progress-last-updated
----------- -------------------- ------------- ------------ -------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 4.73GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 2.65GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 2.65GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 4.34GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 2.40GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 2.40GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 4.45GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 2.62GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst14 Uninitialized Transferring 852.0MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst16 Uninitialized Transferring 910.6MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst15 Uninitialized Transferring 1.23GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst13 Uninitialized Transferring 1.15GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst09 Uninitialized Transferring 1.28GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst11 Uninitialized Transferring 955.7MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst10 Uninitialized Transferring 1.03GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst12 Uninitialized Transferring 1.70GB         12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst20 Uninitialized Transferring 961.9MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst21 Uninitialized Transferring 824.1MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst23 Uninitialized Transferring 838.1MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst22 Uninitialized Transferring 803.5MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst25 Uninitialized Transferring 713.6MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst26 Uninitialized Transferring 508.3MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst24 Uninitialized Transferring 33.56MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst18 Uninitialized Transferring 137.5MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst27 Uninitialized Transferring 151.4MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst17 Uninitialized Transferring 427.0MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst19 Uninitialized Transferring 139.6MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst29 Uninitialized Transferring 32.55MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst31 Uninitialized Transferring 141.9MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst30 Uninitialized Transferring 37.61MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst32 Uninitialized Transferring 120.8MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst37 Uninitialized Transferring 35.33MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst39 Uninitialized Transferring 33.31MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst38 Uninitialized Transferring 36.09MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst33 Uninitialized Transferring 33.31MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst28 Uninitialized Transferring 36.85MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst34 Uninitialized Transferring 32.04MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst40 Uninitialized Transferring 32.04MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst35 Uninitialized Transferring 32.80MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst36 Uninitialized Transferring 34.58MB        12/28 15:47:17
svm:vol1    svm_fsxn2:vol1_dst41 Uninitialized Transferring 0B             12/28 15:47:18
svm:vol1    svm_fsxn2:vol1_dst42 Uninitialized Transferring 0B             12/28 15:47:22
svm:vol1    svm_fsxn2:vol1_dst43 Uninitialized Transferring 0B             12/28 15:47:22
svm:vol1    svm_fsxn2:vol1_dst45 Uninitialized Transferring 0B             12/28 15:47:22
svm:vol1    svm_fsxn2:vol1_dst44 Uninitialized Transferring 0B             12/28 15:47:26

source-path destination-path     state         status       total-progress progress-last-updated
----------- -------------------- ------------- ------------ -------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst46 Uninitialized Transferring 0B             12/28 15:47:28
svm:vol1    svm_fsxn2:vol1_dst48 Uninitialized Transferring 0B             12/28 15:47:27
svm:vol1    svm_fsxn2:vol1_dst49 Uninitialized Transferring 0B             12/28 15:47:28
svm:vol1    svm_fsxn2:vol1_dst47 Uninitialized Transferring 0B             12/28 15:47:28
49 entries were displayed.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type
source-path destination-path     state         status       total-progress current-transfer-type progress-last-updated
----------- -------------------- ------------- ------------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst07 Uninitialized Transferring 4.75GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst06 Uninitialized Transferring 2.67GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst05 Uninitialized Transferring 2.67GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst08 Uninitialized Transferring 4.36GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst01 Uninitialized Transferring 2.43GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst03 Uninitialized Transferring 2.42GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst04 Uninitialized Transferring 4.47GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst02 Uninitialized Transferring 2.65GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst14 Uninitialized Transferring 876.9MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst16 Uninitialized Transferring 935.6MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst15 Uninitialized Transferring 1.25GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst13 Uninitialized Transferring 1.17GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst09 Uninitialized Transferring 1.31GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst11 Uninitialized Transferring 980.8MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst10 Uninitialized Transferring 1.05GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst12 Uninitialized Transferring 1.73GB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst20 Uninitialized Transferring 987.1MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst21 Uninitialized Transferring 849.0MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst23 Uninitialized Transferring 863.4MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst22 Uninitialized Transferring 828.8MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst25 Uninitialized Transferring 738.7MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst26 Uninitialized Transferring 533.5MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst24 Uninitialized Transferring 58.70MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst18 Uninitialized Transferring 162.7MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst27 Uninitialized Transferring 176.6MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst17 Uninitialized Transferring 452.0MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst19 Uninitialized Transferring 164.6MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst29 Uninitialized Transferring 57.68MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst31 Uninitialized Transferring 167.1MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst30 Uninitialized Transferring 62.49MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst32 Uninitialized Transferring 146.0MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst37 Uninitialized Transferring 60.21MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst39 Uninitialized Transferring 58.19MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst38 Uninitialized Transferring 60.97MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst33 Uninitialized Transferring 58.44MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst28 Uninitialized Transferring 61.73MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst34 Uninitialized Transferring 57.18MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst40 Uninitialized Transferring 57.18MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst35 Uninitialized Transferring 57.94MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst36 Uninitialized Transferring 59.71MB        initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst41 Uninitialized Transferring 3.87MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst42 Uninitialized Transferring 3.62MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst43 Uninitialized Transferring 3.62MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst45 Uninitialized Transferring 3.62MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst44 Uninitialized Transferring 3.11MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst46 Uninitialized Transferring 3.11MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst48 Uninitialized Transferring 3.11MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst49 Uninitialized Transferring 3.11MB         initialize            12/28 15:51:40

source-path destination-path     state         status       total-progress current-transfer-type progress-last-updated
----------- -------------------- ------------- ------------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst47 Uninitialized Transferring 2.86MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst51 Uninitialized Transferring 2.61MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst50 Uninitialized Transferring 2.61MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst53 Uninitialized Transferring 1.85MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst52 Uninitialized Transferring 2.14MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst57 Uninitialized Transferring 2.13MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst56 Uninitialized Transferring 1.85MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst54 Uninitialized Transferring 1.85MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst55 Uninitialized Transferring 1.59MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst58 Uninitialized Transferring 1.07MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst60 Uninitialized Transferring 1.59MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst61 Uninitialized Transferring 1.33MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst59 Uninitialized Transferring 1.07MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst62 Uninitialized Transferring 1.07MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst64 Uninitialized Transferring 1.07MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst65 Uninitialized Transferring 1.07MB         initialize            12/28 15:51:40
svm:vol1    svm_fsxn2:vol1_dst63 Uninitialized Transferring 841.5KB        initialize            12/28 15:51:40
65 entries were displayed.

65個同時に転送できています。

同時転送数64の壁も突破しました。

さらに40個の転送元ボリュームを追加してSnapMirror Initialize

もしかして転送元ボリューム数が1つであるのがよくないのでしょうか。同時転送数の上限32というのは、同時に転送できる転送元のボリューム数の上限が32というのを指しているのかもしれません。

試しにSnapMirrorの転送元ボリュームとして40個のボリュームを追加します。

ボリューム作成用のコマンドをシェルスクリプトで書き出します。

$ for i in {01..40}; do
  echo "volume create -vserver svm -volume vol${i} -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol${i}"
done

volume create -vserver svm -volume vol01 -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol01
volume create -vserver svm -volume vol02 -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol02
volume create -vserver svm -volume vol03 -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol03
.
.
(中略)
.
.
volume create -vserver svm -volume vol38 -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol38
volume create -vserver svm -volume vol39 -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol39
volume create -vserver svm -volume vol40 -aggregate aggr1 -size 4GB -state online -policy default -tiering-policy none -snapshot-policy none -junction-path /vol40

書き出したコマンドをFSxN 1で実行します。実行後ボリュームが40個追加されていることを確認します。

::> volume show -vserver svm
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
svm       svm_root     aggr1        online     RW          1GB    960.9MB    1%
svm       vol01        aggr1        online     RW          4GB     3.80GB    0%
svm       vol02        aggr1        online     RW          4GB     3.80GB    0%
svm       vol03        aggr1        online     RW          4GB     3.80GB    0%
svm       vol04        aggr1        online     RW          4GB     3.80GB    0%
svm       vol05        aggr1        online     RW          4GB     3.80GB    0%
svm       vol06        aggr1        online     RW          4GB     3.80GB    0%
svm       vol07        aggr1        online     RW          4GB     3.80GB    0%
svm       vol08        aggr1        online     RW          4GB     3.80GB    0%
svm       vol09        aggr1        online     RW          4GB     3.80GB    0%
svm       vol1         aggr1        online     RW         16GB     5.83GB   61%
svm       vol10        aggr1        online     RW          4GB     3.80GB    0%
svm       vol11        aggr1        online     RW          4GB     3.80GB    0%
svm       vol12        aggr1        online     RW          4GB     3.80GB    0%
svm       vol13        aggr1        online     RW          4GB     3.80GB    0%
svm       vol14        aggr1        online     RW          4GB     3.80GB    0%
svm       vol15        aggr1        online     RW          4GB     3.80GB    0%
svm       vol16        aggr1        online     RW          4GB     3.80GB    0%
svm       vol17        aggr1        online     RW          4GB     3.80GB    0%
svm       vol18        aggr1        online     RW          4GB     3.80GB    0%
svm       vol19        aggr1        online     RW          4GB     3.80GB    0%
svm       vol20        aggr1        online     RW          4GB     3.80GB    0%
svm       vol21        aggr1        online     RW          4GB     3.80GB    0%
svm       vol22        aggr1        online     RW          4GB     3.80GB    0%
svm       vol23        aggr1        online     RW          4GB     3.80GB    0%
svm       vol24        aggr1        online     RW          4GB     3.80GB    0%
svm       vol25        aggr1        online     RW          4GB     3.80GB    0%
svm       vol26        aggr1        online     RW          4GB     3.80GB    0%
svm       vol27        aggr1        online     RW          4GB     3.80GB    0%
svm       vol28        aggr1        online     RW          4GB     3.80GB    0%
svm       vol29        aggr1        online     RW          4GB     3.80GB    0%
svm       vol30        aggr1        online     RW          4GB     3.80GB    0%
svm       vol31        aggr1        online     RW          4GB     3.80GB    0%
svm       vol32        aggr1        online     RW          4GB     3.80GB    0%
svm       vol33        aggr1        online     RW          4GB     3.80GB    0%
svm       vol34        aggr1        online     RW          4GB     3.80GB    0%
svm       vol35        aggr1        online     RW          4GB     3.80GB    0%
svm       vol36        aggr1        online     RW          4GB     3.80GB    0%
svm       vol37        aggr1        online     RW          4GB     3.80GB    0%
svm       vol38        aggr1        online     RW          4GB     3.80GB    0%
svm       vol39        aggr1        online     RW          4GB     3.80GB    0%
svm       vol40        aggr1        online     RW          4GB     3.80GB    0%
42 entries were displayed.

NFSクライアントで追加したボリュームをマウントして、各ボリュームに1GiBのファイルを作成します。

$ for i in {01..40}; do
  sudo mkdir -p "/mnt/fsxn/vol${i}"
  sudo mount -t nfs4 "svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol${i}" "/mnt/fsxn/vol${i}"
  sudo dd if=/dev/urandom of=/mnt/fsxn/vol${i}/urandom_block_file bs=1M count=1024
done

$ df -hT -t nfs4
Filesystem                                                                    Type  Size  Used Avail Use% Mounted on
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol01 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol01
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol02 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol02
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol03 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol03
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol04 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol04
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol05 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol05
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol06 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol06
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol07 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol07
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol08 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol08
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol09 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol09
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol10 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol10
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol11 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol11
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol12 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol12
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol13 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol13
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol14 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol14
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol15 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol15
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol16 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol16
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol17 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol17
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol18 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol18
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol19 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol19
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol20 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol20
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol21 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol21
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol22 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol22
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol23 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol23
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol24 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol24
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol25 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol25
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol26 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol26
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol27 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol27
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol28 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol28
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol29 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol29
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol30 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol30
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol31 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol31
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol32 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol32
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol33 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol33
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol34 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol34
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol35 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol35
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol36 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol36
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol37 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol37
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol38 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol38
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol39 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol39
svm-0058ae83d258ab2e3.fs-0ab6f9b00824a187c.fsx.us-east-1.amazonaws.com:/vol40 nfs4  3.9G  1.1G  2.8G  27% /mnt/fsxn/vol40

追加したボリュームをSnapMirrorの転送元ボリュームとするようにsnapmirror protectのコマンドを書き出します。

$ for i in {01..40}; do
  echo "snapmirror protect -path-list svm:vol${i} -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none"
done
snapmirror protect -path-list svm:vol01 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
snapmirror protect -path-list svm:vol02 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
snapmirror protect -path-list svm:vol03 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
.
.
(中略)
.
.
snapmirror protect -path-list svm:vol38 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
snapmirror protect -path-list svm:vol39 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
snapmirror protect -path-list svm:vol40 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none

書き出したコマンドを実行します。

::> snapmirror protect -path-list svm:vol01 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
[Job 180] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol01".

::> snapmirror protect -path-list svm:vol02 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
[Job 181] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol02".

::> snapmirror protect -path-list svm:vol03 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
[Job 182] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol03".
.
.
(中略)
.
.
::> snapmirror protect -path-list svm:vol38 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
[Job 218] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol38".

::> snapmirror protect -path-list svm:vol39 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
[Job 219] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol39".

::> snapmirror protect -path-list svm:vol40 -destination-vserver svm_fsxn2 -policy MirrorAllSnapshots -auto-initialize true -support-tiering true -tiering-policy none
[Job 220] Job is queued: snapmirror protect for list of source endpoints beginning with "svm:vol40".

SnapMirror relationshipの状態を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type
source-path destination-path    state         status       total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------- ------------ -------------- --------------------- ---------------------
svm:vol01   svm_fsxn2:vol01_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol02   svm_fsxn2:vol02_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol03   svm_fsxn2:vol03_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol04   svm_fsxn2:vol04_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol05   svm_fsxn2:vol05_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol06   svm_fsxn2:vol06_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol07   svm_fsxn2:vol07_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol08   svm_fsxn2:vol08_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol09   svm_fsxn2:vol09_dst Uninitialized Transferring 0B             initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst07
                                Uninitialized Transferring 4.84GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst06
                                Uninitialized Transferring 2.76GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst05
                                Uninitialized Transferring 2.76GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst08
                                Uninitialized Transferring 4.45GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst01
                                Uninitialized Transferring 2.51GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst03
                                Uninitialized Transferring 2.51GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst04
                                Uninitialized Transferring 4.56GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst02
                                Uninitialized Transferring 2.74GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst14
                                Uninitialized Transferring 965.0MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst16
                                Uninitialized Transferring 1023MB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst15
                                Uninitialized Transferring 1.34GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst13
                                Uninitialized Transferring 1.26GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst09
                                Uninitialized Transferring 1.39GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst11
                                Uninitialized Transferring 1.04GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst10
                                Uninitialized Transferring 1.14GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst12
                                Uninitialized Transferring 1.81GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst20
                                Uninitialized Transferring 1.05GB         initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst21
                                Uninitialized Transferring 937.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst23
                                Uninitialized Transferring 951.6MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst22
                                Uninitialized Transferring 917.0MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst25
                                Uninitialized Transferring 826.7MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst26
                                Uninitialized Transferring 621.8MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst24
                                Uninitialized Transferring 146.6MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst18
                                Uninitialized Transferring 250.6MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst27
                                Uninitialized Transferring 264.6MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst17
                                Uninitialized Transferring 540.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst19
                                Uninitialized Transferring 252.7MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst29
                                Uninitialized Transferring 145.8MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst31
                                Uninitialized Transferring 255.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst30
                                Uninitialized Transferring 150.7MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst32
                                Uninitialized Transferring 233.8MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst37
                                Uninitialized Transferring 148.4MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst39
                                Uninitialized Transferring 146.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst38
                                Uninitialized Transferring 149.1MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst33
                                Uninitialized Transferring 146.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst28
                                Uninitialized Transferring 149.9MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst34
                                Uninitialized Transferring 145.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst40
                                Uninitialized Transferring 145.3MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst35
                                Uninitialized Transferring 146.0MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst36
                                Uninitialized Transferring 147.6MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst41
                                Uninitialized Transferring 91.88MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst42
                                Uninitialized Transferring 91.62MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst43
                                Uninitialized Transferring 91.36MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst45
                                Uninitialized Transferring 91.36MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst44
                                Uninitialized Transferring 91.09MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst46
                                Uninitialized Transferring 90.82MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst48
                                Uninitialized Transferring 90.83MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst49
                                Uninitialized Transferring 90.81MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst47
                                Uninitialized Transferring 90.81MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst51
                                Uninitialized Transferring 90.54MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst50
                                Uninitialized Transferring 90.54MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst53
                                Uninitialized Transferring 90.01MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst52
                                Uninitialized Transferring 90.27MB        initialize            12/28 16:15:26
svm:vol1    svm_fsxn2:vol1_dst57
                                Uninitialized Transferring 91.07MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst56
                                Uninitialized Transferring 91.07MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst54
                                Uninitialized Transferring 91.07MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst55
                                Uninitialized Transferring 90.81MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst58
                                Uninitialized Transferring 90.30MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst60
                                Uninitialized Transferring 90.81MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst61
                                Uninitialized Transferring 90.56MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst59
                                Uninitialized Transferring 90.30MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst62
                                Uninitialized Transferring 90.30MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst64
                                Uninitialized Transferring 90.05MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst65
                                Uninitialized Transferring 90.30MB        initialize            12/28 16:15:41
svm:vol1    svm_fsxn2:vol1_dst63
                                Uninitialized Transferring 90.05MB        initialize            12/28 16:15:41
svm:vol10   svm_fsxn2:vol10_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol11   svm_fsxn2:vol11_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol12   svm_fsxn2:vol12_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol13   svm_fsxn2:vol13_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol14   svm_fsxn2:vol14_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol15   svm_fsxn2:vol15_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol16   svm_fsxn2:vol16_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol17   svm_fsxn2:vol17_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol18   svm_fsxn2:vol18_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol19   svm_fsxn2:vol19_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol20   svm_fsxn2:vol20_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol21   svm_fsxn2:vol21_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol22   svm_fsxn2:vol22_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol23   svm_fsxn2:vol23_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol24   svm_fsxn2:vol24_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol25   svm_fsxn2:vol25_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol26   svm_fsxn2:vol26_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol27   svm_fsxn2:vol27_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol28   svm_fsxn2:vol28_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol29   svm_fsxn2:vol29_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol30   svm_fsxn2:vol30_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol31   svm_fsxn2:vol31_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol32   svm_fsxn2:vol32_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol33   svm_fsxn2:vol33_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol34   svm_fsxn2:vol34_dst Uninitialized Queued       -              update                -
svm:vol35   svm_fsxn2:vol35_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol36   svm_fsxn2:vol36_dst Uninitialized Transferring 0B             initialize            12/28 16:15:41
svm:vol37   svm_fsxn2:vol37_dst Uninitialized Queued       -              update                -
svm:vol38   svm_fsxn2:vol38_dst Uninitialized Queued       -              update                -
svm:vol39   svm_fsxn2:vol39_dst Uninitialized Queued       -              update                -
svm:vol40   svm_fsxn2:vol40_dst Uninitialized Queued       -              update                -
105 entries were displayed.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type -status queued
source-path destination-path    state         status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------- ------ -------------- --------------------- ---------------------
svm:vol34   svm_fsxn2:vol34_dst Uninitialized Queued -              update                -
svm:vol37   svm_fsxn2:vol37_dst Uninitialized Queued -              update                -
svm:vol38   svm_fsxn2:vol38_dst Uninitialized Queued -              update                -
svm:vol39   svm_fsxn2:vol39_dst Uninitialized Queued -              update                -
svm:vol40   svm_fsxn2:vol40_dst Uninitialized Queued -              update                -
5 entries were displayed.

5つのSnapMirror relationshipの状態がQueuedとなっています。

追加したSnapMirror relationshipのうち35個はTransferringとなっているため、「同時に転送できる転送元のボリューム数の上限が32」という仮説は否定されました。

キューに溜まっている状態から転送を開始する様子も確認してみます。

FSxN 1のグローバルスロットリングでunlimitedを指定して、転送速度を早めます。

::> options -option-name replication.throttle.outgoing.max_kbs unlimited
1 entry was modified.

FsxId0ab6f9b00824a187c::> options

FsxId0ab6f9b00824a187c
    replication.throttle.incoming.max_kbs
                                      -                    -
    replication.throttle.outgoing.max_kbs
                                      unlimited            -
2 entries were displayed.

SnapMirror relationshipの状態を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type -status queued
There are no entries matching your query.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type
source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol01   svm_fsxn2:vol01_dst Snapmirrored Idle   -              -                     -
svm:vol02   svm_fsxn2:vol02_dst Snapmirrored Idle   -              -                     -
svm:vol03   svm_fsxn2:vol03_dst Snapmirrored Idle   -              -                     -
svm:vol04   svm_fsxn2:vol04_dst Snapmirrored Idle   -              -                     -
svm:vol05   svm_fsxn2:vol05_dst Snapmirrored Idle   -              -                     -
svm:vol06   svm_fsxn2:vol06_dst Snapmirrored Idle   -              -                     -
svm:vol07   svm_fsxn2:vol07_dst Snapmirrored Idle   -              -                     -
svm:vol08   svm_fsxn2:vol08_dst Snapmirrored Idle   -              -                     -
svm:vol09   svm_fsxn2:vol09_dst Snapmirrored Idle   -              -                     -
svm:vol1    svm_fsxn2:vol1_dst07
                                Snapmirrored Transferring
                                                    6.55KB         update                12/28 16:27:34
svm:vol1    svm_fsxn2:vol1_dst06
                                Uninitialized
                                             Transferring
                                                    3.94GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst05
                                Uninitialized
                                             Transferring
                                                    3.94GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst08
                                Snapmirrored Transferring
                                                    6.55KB         update                12/28 16:27:38
svm:vol1    svm_fsxn2:vol1_dst01
                                Uninitialized
                                             Transferring
                                                    3.69GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst03
                                Uninitialized
                                             Transferring
                                                    3.68GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst04
                                Snapmirrored Transferring
                                                    6.55KB         update                12/28 16:27:40
svm:vol1    svm_fsxn2:vol1_dst02
                                Uninitialized
                                             Transferring
                                                    3.92GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst14
                                Uninitialized
                                             Transferring
                                                    2.12GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst16
                                Uninitialized
                                             Transferring
                                                    2.17GB         initialize            12/28 16:27:37

source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst15
                                Uninitialized
                                             Transferring
                                                    2.52GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst13
                                Uninitialized
                                             Transferring
                                                    2.43GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst09
                                Uninitialized
                                             Transferring
                                                    2.57GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst11
                                Uninitialized
                                             Transferring
                                                    2.22GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst10
                                Uninitialized
                                             Transferring
                                                    2.32GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst12
                                Uninitialized
                                             Transferring
                                                    2.99GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst20
                                Uninitialized
                                             Transferring
                                                    2.23GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst21
                                Uninitialized
                                             Transferring
                                                    2.09GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst23
                                Uninitialized
                                             Transferring
                                                    2.10GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst22
                                Uninitialized
                                             Transferring
                                                    2.07GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst25
                                Uninitialized
                                             Transferring
                                                    1.99GB         initialize            12/28 16:27:37

source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst26
                                Uninitialized
                                             Transferring
                                                    1.78GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst24
                                Uninitialized
                                             Transferring
                                                    1.32GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst18
                                Uninitialized
                                             Transferring
                                                    1.42GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst27
                                Uninitialized
                                             Transferring
                                                    1.43GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst17
                                Uninitialized
                                             Transferring
                                                    1.70GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst19
                                Uninitialized
                                             Transferring
                                                    1.42GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst29
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst31
                                Uninitialized
                                             Transferring
                                                    1.42GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst30
                                Uninitialized
                                             Transferring
                                                    1.32GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst32
                                Uninitialized
                                             Transferring
                                                    1.40GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst37
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37

source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst39
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst38
                                Uninitialized
                                             Transferring
                                                    1.32GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst33
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst28
                                Uninitialized
                                             Transferring
                                                    1.32GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst34
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst40
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst35
                                Uninitialized
                                             Transferring
                                                    1.32GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst36
                                Uninitialized
                                             Transferring
                                                    1.31GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst41
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst42
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst43
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37

source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst45
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst44
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst46
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst48
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst49
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst47
                                Uninitialized
                                             Transferring
                                                    1.27GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst51
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst50
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst53
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst52
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst57
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37

source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst56
                                Uninitialized
                                             Transferring
                                                    1.27GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst54
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst55
                                Uninitialized
                                             Transferring
                                                    1.27GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst58
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst60
                                Uninitialized
                                             Transferring
                                                    1.27GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst61
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst59
                                Uninitialized
                                             Transferring
                                                    1.27GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst62
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst64
                                Uninitialized
                                             Transferring
                                                    1.28GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst65
                                Uninitialized
                                             Transferring
                                                    1.26GB         initialize            12/28 16:27:37
svm:vol1    svm_fsxn2:vol1_dst63
                                Uninitialized
                                             Transferring
                                                    1.27GB         initialize            12/28 16:27:37

source-path destination-path    state        status total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------ -------------- --------------------- ---------------------
svm:vol10   svm_fsxn2:vol10_dst Snapmirrored Idle   -              -                     -
svm:vol11   svm_fsxn2:vol11_dst Snapmirrored Idle   -              -                     -
svm:vol12   svm_fsxn2:vol12_dst Snapmirrored Idle   -              -                     -
svm:vol13   svm_fsxn2:vol13_dst Snapmirrored Idle   -              -                     -
svm:vol14   svm_fsxn2:vol14_dst Snapmirrored Idle   -              -                     -
svm:vol15   svm_fsxn2:vol15_dst Snapmirrored Idle   -              -                     -
svm:vol16   svm_fsxn2:vol16_dst Snapmirrored Idle   -              -                     -
svm:vol17   svm_fsxn2:vol17_dst Snapmirrored Idle   -              -                     -
svm:vol18   svm_fsxn2:vol18_dst Snapmirrored Idle   -              -                     -
svm:vol19   svm_fsxn2:vol19_dst Snapmirrored Idle   -              -                     -
svm:vol20   svm_fsxn2:vol20_dst Snapmirrored Idle   -              -                     -
svm:vol21   svm_fsxn2:vol21_dst Snapmirrored Idle   -              -                     -
svm:vol22   svm_fsxn2:vol22_dst Snapmirrored Idle   -              -                     -
svm:vol23   svm_fsxn2:vol23_dst Snapmirrored Idle   -              -                     -
svm:vol24   svm_fsxn2:vol24_dst Snapmirrored Idle   -              -                     -
svm:vol25   svm_fsxn2:vol25_dst Snapmirrored Idle   -              -                     -
svm:vol26   svm_fsxn2:vol26_dst Snapmirrored Idle   -              -                     -
svm:vol27   svm_fsxn2:vol27_dst Snapmirrored Idle   -              -                     -
svm:vol28   svm_fsxn2:vol28_dst Snapmirrored Idle   -              -                     -
svm:vol29   svm_fsxn2:vol29_dst Snapmirrored Idle   -              -                     -
svm:vol30   svm_fsxn2:vol30_dst Snapmirrored Idle   -              -                     -
svm:vol31   svm_fsxn2:vol31_dst Snapmirrored Idle   -              -                     -
svm:vol32   svm_fsxn2:vol32_dst Snapmirrored Idle   -              -                     -
svm:vol33   svm_fsxn2:vol33_dst Snapmirrored Idle   -              -                     -
svm:vol34   svm_fsxn2:vol34_dst Uninitialized
                                             Transferring
                                                    142.3MB        initialize            12/28 16:27:38
svm:vol35   svm_fsxn2:vol35_dst Snapmirrored Idle   -              -                     -
svm:vol36   svm_fsxn2:vol36_dst Snapmirrored Idle   -              -                     -
svm:vol37   svm_fsxn2:vol37_dst Uninitialized
                                             Transferring
                                                    130.1MB        initialize            12/28 16:27:38
svm:vol38   svm_fsxn2:vol38_dst Uninitialized
                                             Transferring
                                                    115.2MB        initialize            12/28 16:27:38
svm:vol39   svm_fsxn2:vol39_dst Uninitialized
                                             Transferring
                                                    127.3MB        initialize            12/28 16:27:38
svm:vol40   svm_fsxn2:vol40_dst Uninitialized
                                             Transferring
                                                    130.8MB        initialize            12/28 16:27:38
105 entries were displayed.

一部のSnapMirrorの転送が完了したからか、キューに溜まっていたsvm:vol34svm:vol37の転送が開始されましたね。

「同時に転送できる転送元のボリューム数の上限が35」なのか、「最大同時転送数が100」なのかの切り分け

先の検証で「同時に転送できる転送元のボリューム数の上限が35」なのか? というところが気になるところです。

しかし、35という値はあまりにもキリが悪いです。

それよりも、105個あるSnapMirror relationshipのうち5つ転送できないのであれば、最大同時転送数は100であると捉えるのが自然です。

切り分けるために一度SnapMirrorを流し切ります。

65個のSnapMirror relationshipの転送元ボリュームはTiering PolicyがAllであり、ほとんどのデータブロックがキャパシティプールストレージに存在しています。そのため転送速度が非常に低速です。

Tiering PolicyをNone、Cloud retrievalポリシーをPromoteに変更し、データをSSDに書き戻すことで読み込み速度を向上させます。

::*> volume modify -vserver svm -volume vol1 -cloud-retrieval-policy promote -tiering-policy none

Warning: The "promote" cloud retrieve policy retrieves all of the cloud data for the specified volume. If the tiering policy is "snapshot-only" then only AFS data
         is retrieved. If the tiering policy is "none" then all data is retrieved. Volume "vol1" in Vserver "svm" is on a FabricPool, and there are approximately
         9925271552 bytes tiered to the cloud that will be retrieved. Cloud retrieval may take a significant amount of time, and may degrade performance during
         that time. The cloud retrieve operation may also result in data charges by your object store provider.
Do you want to continue? {y|n}: y
Volume modify successful on volume vol1 of Vserver svm.

::*> volume show-footprint -volume vol1


      Vserver : svm
      Volume  : vol1

      Feature                                         Used       Used%
      --------------------------------             ----------    -----
      Volume Data Footprint                            9.40GB       1%
             Footprint in Performance Tier            649.2MB       7%
             Footprint in FSxFabricpoolObjectStore
                                                       8.92GB      93%
      Volume Guarantee                                     0B       0%
      Flexible Volume Metadata                        92.66MB       0%
      Deduplication Metadata                          12.04MB       0%
           Deduplication                              12.04MB       0%
      Delayed Frees                                   155.8MB       0%
      File Operation Metadata                             4KB       0%

      Total Footprint                                  9.66GB       1%

      Footprint Data Reduction in capacity tier        3.75GB        -
      Effective Total Footprint                        5.91GB       1%

SnapMirror relationshipの状態を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type -status Transferring, Finalizing
source-path destination-path     state        status     total-progress current-transfer-type progress-last-updated
----------- -------------------- ------------ ---------- -------------- --------------------- ---------------------
svm:vol1    svm_fsxn2:vol1_dst40 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:34
svm:vol1    svm_fsxn2:vol1_dst35 Snapmirrored Transferring
                                                         4.09GB         update                12/28 17:07:36
svm:vol1    svm_fsxn2:vol1_dst41 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst42 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:34
svm:vol1    svm_fsxn2:vol1_dst44 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst46 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst49 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:34
svm:vol1    svm_fsxn2:vol1_dst47 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:30
svm:vol1    svm_fsxn2:vol1_dst51 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:26
svm:vol1    svm_fsxn2:vol1_dst50 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst57 Snapmirrored Transferring
                                                         4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst56 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:32
svm:vol1    svm_fsxn2:vol1_dst54 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst55 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst58 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:34
svm:vol1    svm_fsxn2:vol1_dst61 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:35
svm:vol1    svm_fsxn2:vol1_dst62 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:30
svm:vol1    svm_fsxn2:vol1_dst64 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:34
svm:vol1    svm_fsxn2:vol1_dst63 Snapmirrored Finalizing 4.09GB         update                12/28 17:07:28
19 entries were displayed.

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type -status Transferring, Finalizing
There are no entries matching your query.

流しきり終えました。

SnapMirrorの転送がすぐに完了しないように、FSxN 1のグローバルスロットリングで転送速度を4MBに制限します。

::> options -option-name replication.throttle.outgoing.max_kbs 4096
1 entry was modified.

FsxId0ab6f9b00824a187c::> options

FsxId0ab6f9b00824a187c
    replication.throttle.incoming.max_kbs
                                      -                    -
    replication.throttle.outgoing.max_kbs
                                      4096                 -
2 entries were displayed

40個のボリューム分のsnapmirror updateコマンドを書き出します。

$ for i in {01..40}; do
  echo "snapmirror update -destination-path svm_fsxn2:vol${i}_dst"
done

snapmirror update -destination-path svm_fsxn2:vol01_dst
snapmirror update -destination-path svm_fsxn2:vol02_dst
snapmirror update -destination-path svm_fsxn2:vol03_dst
.
.
(中略)
.
.
snapmirror update -destination-path svm_fsxn2:vol38_dst
snapmirror update -destination-path svm_fsxn2:vol39_dst
snapmirror update -destination-path svm_fsxn2:vol40_dst

書き出したコマンドを実行します。

実行後、SnapMirror relationshipの状態を確認します。

::> snapmirror show -fields source-path, destination-path, state, status, progress-last-updated, total-progress, current-transfer-type -status Transferring, Finalizing
source-path destination-path    state        status       total-progress current-transfer-type progress-last-updated
----------- ------------------- ------------ ------------ -------------- --------------------- ---------------------
svm:vol01   svm_fsxn2:vol01_dst Snapmirrored Transferring 60.33MB        update                12/28 17:21:19
svm:vol02   svm_fsxn2:vol02_dst Snapmirrored Transferring 42.67MB        update                12/28 17:21:19
svm:vol03   svm_fsxn2:vol03_dst Snapmirrored Transferring 42.16MB        update                12/28 17:21:19
svm:vol04   svm_fsxn2:vol04_dst Snapmirrored Transferring 39.38MB        update                12/28 17:21:19
svm:vol05   svm_fsxn2:vol05_dst Snapmirrored Transferring 45.20MB        update                12/28 17:21:19
svm:vol06   svm_fsxn2:vol06_dst Snapmirrored Transferring 44.19MB        update                12/28 17:21:19
svm:vol07   svm_fsxn2:vol07_dst Snapmirrored Transferring 31.78MB        update                12/28 17:21:19
svm:vol08   svm_fsxn2:vol08_dst Snapmirrored Transferring 36.59MB        update                12/28 17:21:19
svm:vol09   svm_fsxn2:vol09_dst Snapmirrored Transferring 37.35MB        update                12/28 17:21:19
svm:vol10   svm_fsxn2:vol10_dst Snapmirrored Transferring 35.83MB        update                12/28 17:21:19
svm:vol11   svm_fsxn2:vol11_dst Snapmirrored Transferring 36.34MB        update                12/28 17:21:19
svm:vol12   svm_fsxn2:vol12_dst Snapmirrored Transferring 27.92MB        update                12/28 17:21:19
svm:vol13   svm_fsxn2:vol13_dst Snapmirrored Transferring 28.42MB        update                12/28 17:21:19
svm:vol14   svm_fsxn2:vol14_dst Snapmirrored Transferring 28.93MB        update                12/28 17:21:19
svm:vol15   svm_fsxn2:vol15_dst Snapmirrored Transferring 29.44MB        update                12/28 17:21:19
svm:vol16   svm_fsxn2:vol16_dst Snapmirrored Transferring 26.65MB        update                12/28 17:21:19
svm:vol17   svm_fsxn2:vol17_dst Snapmirrored Transferring 27.66MB        update                12/28 17:21:19
svm:vol18   svm_fsxn2:vol18_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol19   svm_fsxn2:vol19_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol20   svm_fsxn2:vol20_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol21   svm_fsxn2:vol21_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol22   svm_fsxn2:vol22_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol23   svm_fsxn2:vol23_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol24   svm_fsxn2:vol24_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol25   svm_fsxn2:vol25_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol26   svm_fsxn2:vol26_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol27   svm_fsxn2:vol27_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol28   svm_fsxn2:vol28_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol29   svm_fsxn2:vol29_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol30   svm_fsxn2:vol30_dst Snapmirrored Transferring 23.87MB        update                12/28 17:21:19
svm:vol31   svm_fsxn2:vol31_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol32   svm_fsxn2:vol32_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol33   svm_fsxn2:vol33_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol34   svm_fsxn2:vol34_dst Snapmirrored Transferring 20.07MB        update                12/28 17:21:19
svm:vol35   svm_fsxn2:vol35_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol36   svm_fsxn2:vol36_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol37   svm_fsxn2:vol37_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol38   svm_fsxn2:vol38_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol39   svm_fsxn2:vol39_dst Snapmirrored Transferring 25.64MB        update                12/28 17:21:19
svm:vol40   svm_fsxn2:vol40_dst Snapmirrored Transferring 20.07MB        update                12/28 17:21:19
40 entries were displayed.

転送元ボリュームが40個あったとしても、SnapMirrorを同時に流すことができました。

そのため、SnapMirrorの同時転送数の上限は100であると考えます。多くのモデルでも同時転送数の上限は100で設定されているので違和感は特にありません。

最大同時転送数やボリューム数、SnapMirrorによる負荷を考慮した上で実行間隔を決定しよう

Amazon FSx for NetApp ONTAPにおけるSnapMirrorの同時転送数の上限を確認してみました。

大量のボリュームが存在する場合はこちらの上限の影響を受けるか意識しましょう。上限を超過するものはキューに溜まって転送されるといっても安心してはいけません。たとえば、5分間隔で100個超のボリュームをSnapMirrorで転送するとなると、キューに溜まっているものを捌き切ることができずトラブルの元になると考えます。

また、SnapMirrorによる負荷も見逃せません。システムに問題を与えない範囲で実行間隔を決定することをお勧めします。

この記事が誰かの助けになれば幸いです。

以上、AWS事業本部 コンサルティング部の のんピ(@non____97)でした!