[Mac] How to Shrink Raspberry Pi SD Card Image Size Using a Docker Container with PiShrink Installed

(この記事には日本語版があります)

You can use the dd command to backup the entire Raspberry Pi SD card on Mac. If you simply read the entire SD card with dd, a huge image file for the entire capacity of the SD card will be created, including the unused area. This squeezes your storage capacity and takes a long time to restore to an SD card. So I tried to reduce the size of the image file using a Docker container with a tool called PiShrink. This article explains the procedure.

Environments

• Mac: 10.14.6
• Raspberry Pi 3 Model B+
• Docker for Mac(docker desktop): 2.1.0.4
• Docker container with PiShrink installed
  • I used turee/pishrink-docker this time. Several similar containers have been published on the Docker Hub, but unverified.

Deleting unnecessary applications, software packages

Before creating an SD card image, delete unnecessary applications and software packages. Insert the SD card you want to create an image file into your Raspberry Pi and power it.

Deleting unnecessary applications

sudo apt-get purge libreoffice wolfram-engine sonic-pi scratch

Be careful not to accidentally delete the apps you are using!

Deleting unnecessary software packages

sudo apt-get clean
sudo apt-get autoremove

Creating an SD card image file

Shut down the Raspberry Pi and remove the SD card. Connect the SD card to your Mac. Then, run diskutil list command on your command line to find the location of the SD card device file.

$ diskutil list
 /dev/disk0 (internal):
    #:                       TYPE NAME                    SIZE       IDENTIFIER
    0:      GUID_partition_scheme                         251.0 GB   disk0
    1:                        EFI EFI                     314.6 MB   disk0s1
    2:                 Apple_APFS Container disk1         250.7 GB   disk0s2
~ ~ ~ snip ~ ~ ~ 
 /dev/disk3 (external, physical):
    #:                       TYPE NAME                    SIZE       IDENTIFIER
    0:     FDisk_partition_scheme                        *32.0 GB    disk3
    1:             Windows_FAT_32 boot                    46.0 MB    disk3s1
    2:                      Linux                         32.0 GB    disk3s2

In the above example, it's OK to use the path /dev/disk3 where the SD card is mounted on. On the other hand, You can use another device file named /dev/rdisk3 which is also created for the SD card. Reading and writing via /dev/rdiskX is faster than via /dev/diskX since /dev/rdiskX is not buffered, but /dev/diskX is buffred on the other hand("r" for "rdisk" means "raw").

$ ls /dev/rdisk*
 /dev/rdisk0   /dev/rdisk0s2 /dev/rdisk1s1 /dev/rdisk1s3 /dev/rdisk2   /dev/rdisk2s2 /dev/rdisk3s1
 /dev/rdisk0s1 /dev/rdisk1   /dev/rdisk1s2 /dev/rdisk1s4 /dev/rdisk2s1 /dev/rdisk3   /dev/rdisk3s2
$ sudo dd if=/dev/rdisk3 of=./XXXX.img bs=1m

The dd command installed on Mac doesn't show any progress while it is running, but typing Ctrl + T on the terminal will show the number of bytes processed at that time.

Compressing the SD card image file

Let's compress the image file using a tool called PiShrink. Install and start Docker Desktop for Mac before proceeding. There are several Docker containers with PiShrink installed in the Docker Hub. I used turee/pishrink-docker this time.

$ docker pull turee/pishrink-docker
 Using default tag: latest
 latest: Pulling from turee/pishrink-docker
 Digest: sha256:31e16c7d382b7a96e28cd92458a0cc1bf1e9b88bad116b68fd7dd4ca95fbf6b8
 Status: Image is up to date for turee/pishrink-docker:latest
 docker.io/turee/pishrink-docker:latest

It is supported to mount a volume of the host machine to the Docker container when executing docker run, but an error occurred when PiShrink tried to run with the directory of the backed-up SD card image file which is in the host filesystem.

$ docker run --rm --privileged=true -v (pwd):/workdir turee/pishrink-docker pishrink XXXX.img Warning: Unable to open /workdir/XXXX.img read-write (Permission denied).  /workdir/XXXX.img has been opened read-only.
 Warning: Unable to open /workdir/XXXX.img read-write (Permission denied).  /workdir/XXXX.img has been opened read-only.
 mount: /tmp/tmp.nDyP6lWYv6: WARNING: device write-protected, mounted read-only.
 Creating new /etc/rc.local
~ ~ ~ snip ~ ~ ~
  Couldn't find valid filesystem superblock.
 resize2fs: Operation not permitted while trying to open /dev/loop0
 ERROR: resize2fs failed...
 mount: /tmp/tmp.nDyP6lWYv6: can't read superblock on /dev/loop0.
 mv: cannot stat '/tmp/tmp.nDyP6lWYv6/etc/rc.local.bak': No such file or directory
 umount: /tmp/tmp.nDyP6lWYv6: not mounted.
 losetup: /dev/loop0: detach failed: No such device or address

To avoid this error, we need to copy the original image file to the container and compress it, then copy the compressed file to the host side again. If only one argument (the path of the original source image file) is specified for pishrink command, the original file is overwritten with the compressed file.

$ docker cp XXXX.dmg 32f4176aa68a:/root/XXXX.img
$ docker exec -it 32f4176aa68a bash
 root@32f4176aa68a:/workdir# cd /root/
 root@32f4176aa68a:~# pishrink XXXX.img 
 Creating new /etc/rc.local
 rootfs: 114911/950272 files (0.5% non-contiguous), 938002/3793920 blocks
 resize2fs 1.44.1 (24-Mar-2018)
~ ~ ~ snip ~ ~ ~ 
 Begin pass 4 (max = 12048)
 Updating inode references     XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
 The filesystem on /dev/loop0 is now 937412 (4k) blocks long.

 Shrunk XXXX.img from 30G to 3.7G
 root@32f4176aa68a:~# exit
 exit
$ docker cp 32f4176aa68a:/root/XXXX.img XXXX-shrink.img

Finally, let's compress the pishrinked-image file withxz command. Although it depends on the contents of the SD card image, the pishrinked-file that was 3.7GB was compressed to about 1.1GB.

$ xz XXXX.img 
 load: 2.88  cmd: xz 30200 running 58.35u 0.42s

As with the dd command, typing Ctrl + T while the xz command is running will display the progress. If only the compression source file name is specified in the xz command arguments, the original file will be replaced with the compressed file. The name of the file will be the same as the original file but the .xz extension will be added to its tail.

Writing compressed SD card image file

The .xz file can be directly written to an SD card using Etcher.

Booting Raspberry Pi with compressed SD card image

The partition will be expanded to the full free space on the SD card at the first boot.

Conclusion

The procedure to reduce the size of the SD card image file of Raspberry Pi using a tool called PiShrink installed in Docker container have explained. Smaller image files save storage space and require less writing time, so I recommend that you compress after creating a backup from a large SD card.

References

• How do you clean up unused files in memory sd card? - Raspberry Pi Stack Exchange
• sd card - Why is “/dev/rdisk” about 20 times faster than “/dev/disk” in Mac OS X - Super User
• Docker Desktop for Mac
• PiShrink
• Docker containers with PiShrink installed on Docker Hub
• turee/pishrink-docker
• SDカードの作成 - FaBo DonkeyCar Docs
• balenaEtcher