Mudkip Mud Sport

Mudkip's Mud Sport Journal

Minisforum V3 AMD Tablet

Quest for the Ideal Portable Linux Device

After my 2016 MacBook Pro experienced multiple hardware issues and was sent for recycling, I no longer have a personal laptop. Until recently, if I wish to do some programming in a coffee shop or during a family trip, my only options are the corporate-provisioned MacBook or my iPad Pro 2020, neither of which is ideal. I had hoped the iPad Pro could be more productive and tried various setups involving code-server and remote development, but I encountered many limitations. Furthermore, the fact that Apple has restricted iPadOS, preventing me from installing any software I want, has become increasingly uncomfortable for me in light of the recent Digital Markets Act and antitrust discussions.

I adore the form factor of the iPad Pro as a travel computer, and the concept of a MacPad is indeed extraordinary. However, I’m hesitant to invest significant time and money into a custom product that might not have the bless of its manufacturer. Additionally, memories of the Microsoft Surface have resurfaced. I owned a Surface 3 many years ago but sold it due to its performance not meeting my needs, eventually replacing it with a Hades Canyon mini PC.

As I’ve become more immersed in Linux, both through my Homelab VMs and my desktop PC, the idea of owning a portable Linux device appeals to me. However, I must confess that I’m not fond of the design of most mainstream non-Apple laptops, and Linux-friendly laptop brands like Framework, System76, and Starlabs are not yet available in my region. I’ve experimented with Asahi Linux on my iMac, and while it offers a generally great experience, the ARM Linux ecosystem, especially with 16K pages, doesn’t quite meet my daily needs.

Ultimately, I’m in search of a Surface-like device that offers good performance, runs Linux well, and is ultraportable. Although the Surface Pro 10 was released just as I began my search, Microsoft’s pricing strategy—essentially doubling the price of the whole device for adequate RAM and SSD—is unacceptable to me. It seems Microsoft has taken the wrong lessons from Apple in this regard, and I believe such pricing strategies should be considered unethical.

Discovering the V3 Tablet

During my search, I discovered an intriguing product from Minisforum, a manufacturer renowned for creating NUC-like mini PCs, which has gained popularity within the Homelab community. The tablet, named “V3,” has been in development for over a year. Initially, it was developed by another manufacturer with an AMD Ryzen 7 6800U processor. Later, Minisforum took over the project, replacing the processor first with the 7840U, and ultimately with the 8840U in the final product.

The tablet features a 14" 2.5K 165Hz P3 color touch screen, a 50.82Wh battery, four speakers, front and rear cameras, and face and fingerprint unlocking, along with two USB4 ports. Surprisingly, it also features a USB-C port labeled “VLink” or “DP-in,” enabling the tablet to function as an external display for other devices. This is a feature I’ve long believed should be standard on all screens and regret its absence in recent iMacs. My curiosity was piqued about how Linux would perform on this device.

The tablet was released on March 29 locally, and its price was set at 6999 CNY (~968 USD) for the configuration with 32GB DDR5 RAM and a 1TB SSD—a price point I consider quite reasonable. Early buyers were offered a keyboard case at no extra cost and had the opportunity to purchase the MPP 2.0 stylus for just 200 CNY (~28 USD). Moreover, it was mentioned that Linux runs on this tablet, with support for Ubuntu 22.04 and 23.10, albeit with “only a few minor issues” that are currently being addressed. I placed my order on day one and have been playing with it over the past few weeks.

Initial Experience

My first impressions of the V3 tablet were less than ideal, primarily due to Minisforum’s approach to selling and shipping the initial units. Although the kickstand and keyboard case were offered for free to early buyers, there was a catch: they were shipped after leaving a review on the online retailer JD. The absence of both the kickstand and the keyboard significantly detracted from the user experience. Additionally, a bug in Windows 11 prevented me from creating a PIN code to complete the setup, a problem that could only be resolved by connecting a USB keyboard. Placing the tablet on a table without the kickstand led to the air opening on the back being covered, which is far from ideal for airflow.

I believe that shipping the free kickstand and keyboard with the tablet would lead to more positive reviews by significantly improving initial customer satisfaction. I hope global customers won’t have to wait for the kickstand, keyboard, and stylus to be shipped separately, as international shipping could add to the frustration.

The V3 tablet comes with Windows 11 Pro, a detail I find quite welcoming, even I primarily intend to use Linux desktop on this device, as most laptops in this price range only come with the Home edition of Windows. The Pro edition grants users more control over BitLocker encryption, which is crucial for a portable device. Additionally, Hyper-V is important for those who need to install VMs on Windows.

The screen’s color and brightness are exceptional. Initially, I didn’t anticipate the 165Hz refresh rate to be a significant feature for this tablet, assuming that most games would struggle to run at 2K on the integrated GPU and that I didn’t have any videos exceeding 60Hz to watch. However, the difference in UI smoothness between 60Hz and 165Hz is stark, making it hard for me to revert to the lower refresh rate.

It’s somewhat disappointing that the tablet’s resolution is only 2.5K; at this screen size, I believe 2.8K or 3K would be optimal for 200% integer scaling. Nonetheless, given that KDE and Wayland now adequately support fractional scaling at 175% and the AMD GPU has good Wayland compatibility, this resolution is acceptable. Consequently, content on the V3 tablet appears as sharp as on MacBooks other than 14 and 16 inch MacBook Pro, which has 254 pixels per inch.

Linux on the Device

Unlike my desktop PC, which runs EndeavourOS, I chose Fedora KDE for this tablet. My preference for Fedora stems from its relatively newer kernel and desktop environment, as well as its out-of-the-box support for secure boot. The latter is particularly crucial for a portable computer, allowing me to enable full disk encryption and set up automatic decryption via TPM with systemd-cryptsetup. My choice to stick with KDE was primarily due to its excellent support for scaling Xwayland applications. Although many GTK4 apps appeal to me—and I can still use them within KDE—GNOME has continued to refuse to address the issue of Xwayland apps appearing blurry on HiDPI displays.

I was concerned about hardware support on Linux, but my experience turned out to be reassuring. The touchscreen, keyboard, touchpad, Wi-Fi, USB, Bluetooth and suspend/sleep all work without any issues. It’s surprising that the fingerprint reader on the power button works right out of the box. After adding my fingerprint in KDE settings, not only can I use it on the lock screen, but I can also use my fingerprint to unlock sudo commands in Konsole with no extra setup. Both the front-facing and back cameras work well. I didn’t try face unlocking because I actually prefer using touch over face recognition.

Most apps I installed on V3 are similar to those on my desktop PC, only I use dnf instead of pacman. I don’t need to configure Wayland for Chromium and Electron apps, as Xwayland performs well with an AMD GPU. I couldn’t get Timeshift working with a btrfs filesystem inside a LUKS partition, so I switched to btrfs-assistant and snapper for automatically creating snapshots.

Enabling hardware acceleration for video playback in Fedora is somewhat weird. I need to add the RPMFusion repository and switch from mesa-va-drivers to mesa-va-drivers-freeworld to enable H.264 and HEVC hardware decoding, due to some absurd patent law issues.

Using the V3 as a Linux Tablet

KDE Tablet Mode and Handwritten Notes in Xournal++

Using Linux with a touchscreen is as good as using Windows. I’ve used the desktop mode on the Steam Deck, and all the gestures are familiar. KDE has a “tablet mode” that enlarges the taskbar icons and many UI elements when I detach the keyboard, making it easier to tap.

The stylus is well-supported on Linux. Hover and pen pressure functionalities work great in apps like Krita and Xournal++. It’s a pleasure to take handwritten notes and annotate PDFs on this tablet, especially with the 165Hz refresh rate. Using the stylus makes selecting text and navigating the UI easier than using my finger.

The on-screen keyboard is a bit tricky here. Both Fcitx5 and Maliit use the virtual keyboard protocol on KDE Wayland. When the keyboard is attached, I use Fcitx5 for inputting Chinese or Japanese texts, and when detached, I switch to Maliit for the on-screen keyboard. I haven’t found a way to automatically switch between these two, but I believe it’s possible. Maliit supports multiple languages and can be configured with gsettings set org.maliit.keyboard.maliit enabled-languages "['en', 'zh-hans', 'ja', 'emoji']".

I use Waydroid to run Android apps on the V3 tablet, with support for Google Play and arm translation. Some apps, like Google Play Books and Pocket Casts, work great. However, the performance and experience don’t compare to a native Android tablet. I considered using Microsoft OneNote as a cross-platform tool for my handwritten notes, but there’s a major issue with OneNote for Android on Waydroid—the handwriting doesn’t appear on the screen until I lift the stylus, touchpad, or finger. Another issue is video playback; while I have hardware acceleration in Linux apps like Firefox, mpv, and VLC, Android apps can only decode videos with the CPU, leading to worse battery life. Cameras do not work on Waydroid either.

One functionality missing in Linux on the V3 is automatic rotation. I have iio-sensor-proxy installed, and KDE Wayland should support automatic rotation out of the box. However, monitor-sensors reports there is no accelerometer. I hope Minisforum can address this issue with a driver or firmware update, or at least provide a DKMS module.

Another minor issue is that the volume keys on the side of the tablet only work when the keyboard is attached in Linux. I suspect this is a userland problem.

Performance and Battery Life

Watching Sound! Euphonium

The performance of the AMD Ryzen 7 8840U is outstanding. It outperforms my M1 iMac in tasks such as building Android apps and the 7z b benchmark. Its single-core performance surpasses that of my Core i7 12700 homelab PC, though the 12700 only performs better in multi-core tasks because it has four more E-cores. I haven’t conducted extensive benchmarks, and I haven’t yet managed to get Handbrake working with AMD VCN on Linux to test its hardware-accelerated video encoding speed. However, the KDE UI and most applications feel as fast as my Dragon Canyon desktop PC equipped with an RTX 3080. In some cases, it feels even snappier, due to the ongoing issues between NVIDIA and Wayland, which are expected to be resolved very soon.

During my normal usage in Linux, which includes web browsing with Firefox, video playback with Haruna Media Player (with hwdec set to auto), writing in Obsidian, and lightweight coding in Visual Studio Code and Android Studio, the tablet lasts for 6 hours. The battery life is not as long as that of Apple Silicon MacBooks but it is satisfactory to me and better than what I remember from using Intel MacBooks. The battery life might vary with Windows, potentially achieving longer duration.

Gaming on this tablet delivers performance in line with what you would expect from the Radeon 780M integrated GPU. I’ve tested it with two recent games I’m playing. I can play Persona 3 Reload at high settings (though with Ray-Traced Reflections disabled) at 1200p and 60fps. For Like a Dragon: Infinite Wealth, I achieve 55 to 60 fps at high settings (with FSR set to optimize for frame rate) in 1200p. Both games perform much better than on the Steam Deck. The frame rate only drops slightly when reducing the TDP from 28W to 15W, showcasing the efficiency of the GPU under varied power conditions.

The tablet is marketed as an “AI PC,” due to the AMD Ryzen 7 8840U processor, which features an XDNA NPU, and a keyboard equipped with a Copilot key. While local Large Language Models like Llama 2 perform rapidly and effectively using platforms like ollama and LM Studio, I believe these currently run on the CPU. As of now, I am not aware of any AI products specifically utilizing the NPU, but AMD does provide some tutorials which might hint at future developments that could leverage the NPU for enhanced AI performance.

Working as an Portable Display

The external display feature of the V3 tablet, which makes it a “3 in 1” device, is unique. Although this feature is still in its early stages based on my testing, I believe it represents a promising start.

The VLink USB-C port only operates when the computer part of the tablet is shut down. Unfortunately, there appear to be compatibility issues, as it doesn’t work with my M1 Macs using a USB-C cable. I tried two cables that function with my Mac and the LG UL850 display, but neither could activate the V3’s display. However, it worked fine when connected to my desktop PC. I suspect that the issue might be resolved by using a USB-C to DisplayPort dongle and a DisplayPort to USB-C cable, but this solution is too cumbersome. I hope a firmware update can resolve this issue.

When used as an external display, the keyboard case can also function as an external USB keyboard. This is particularly useful in setups where I want to configure a NAS or mini PC for eventual headless operation. However, it seems the V3 only supports a 60Hz refresh rate when used as an external display, and I’m not sure if this is due to compatibility or cable issues.

Another feature I hope to see improved in the future is the ability to turn on the Nintendo Switch’s TV mode with just one USB-C cable. I understand that Nintendo doesn’t use standard display protocols, making this a challenging feature to implement. However, I’m aware of portable displays that support this feature. It would be incredibly valuable for travel with the Nintendo Switch, and I would be willing to pay extra for this capability.

Other Hardware Details

The form factor of the V3 tablet is similar to a Microsoft Surface Pro, especially when the keyboard is attached. However, the kickstand is separate from the tablet, which makes the entire setup slightly heavier. I prefer the integrated kickstand design of the Microsoft Surface and Nintendo Switch OLED, but the separate kickstand does provide a wider range of angles.

The tablet is completely sealed with no visible screws for opening, which raises concerns about long-term repairability, especially if the cooling system degrades over time. While a 1TB SSD is sufficient for my needs with this device, it would be beneficial for users who require more storage if it can replace the NVMe drive.

The speaker is loud and good, but there seems to be an issue with volume control in Linux. The global volume control can only mute or unmute the speaker, while the actual volume is controlled at the application level. I suspect there might be some driver issues, as when I use headphones, both the global and application level volume controls work well.

The keyboard case is quite pleasant to use. While I might still prefer the feel of the Magic Keyboard, this one comes close. The key layout feels just right. I particularly appreciate that the keyboard includes dedicated Home, Pg Up, Pg Dn, and End keys on the right side. This is especially useful to me, as I’m primarily familiar with macOS/Emacs shortcuts for these commands, which are not supported in Linux except within the terminal.

The fan noise is a mixed bag for me. During the day, even under heavy workloads, I can barely hear the fan. However, at midnight, when my room is completely silent, I can often hear a strange, faint buzzing noise coming from the tablet. This noise disappears when I run benchmarks or play games that maximize the fan speed. It’s not like the typical fan noise but resembles the sound of an HDD, though at a much lower volume. It seems this noise might be caused by the fan frequently changing speeds or being affected by the magnets in the tablet. I’d be interested to hear if others have experienced this issue, and if it turns out to be unique to my unit or an early production issue, I’ll replace it under warranty.

Conclusion

After two weeks of use, I consider the Minisforum V3 AMD Tablet to be a nearly perfect portable Linux device. It offers a promising blend of performance, versatility, and Linux compatibility at an acceptable price point. This tablet is an excellent start for tablets featuring AMD APUs and those that triple as external displays. However, I hope that the minor issues I’ve mentioned can be addressed, either through firmware updates or in the next iteration of the product. I also wish for some level of repairability, such as the ability for users to open the case to clean the fans or replace the NVMe drive. Looking ahead, I plan to bring the V3 tablet along for travel and weekends in coffee shops.

KDE Plasma 6

I have been using EndeavourOS on my Dragon Canyon NUC for months. It is my chosen Linux distribution because it offers an easy-to-install Arch Linux environment, and I am familiar with Arch Linux. I need to stay on the cutting edge for desktop environments, display protocols, and graphics drivers due to issues with NVIDIA and the Linux desktop (especially Wayland). Many developments are happening in real-time to improve them. With Arch Linux, I can easily install an AUR package from a merge request, as well as switch to beta drivers when they become available.

Wayland is important to me because I’m using a 27" 4K display. For the best visual size, it should be set to 150% fractional scaling. KDE Plasma supports fractional scaling under Wayland and has an option to “let Xwayland applications scale themselves” to prevent them from being blurry. That’s the reason I chose KDE as my desktop environment. Moreover, KDE Plasma 6 offers a nice experience both functionally and aesthetically.

Here is a note in Logseq that I use to track how I install and configure EndeavourOS. I’m interested in a declarative way to build operating systems and am slowly learning NixOS. However, in the meantime, I am very satisfied with my current EndeavourOS setup.


  • Configure the Windows Registry to use UTC hardware time:
    1
    reg add "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation" /v RealTimeIsUniversal /d 1 /t REG_DWORD /f
  • Disable Secure Boot and Fast Boot in the BIOS. If Fast Boot is enabled, accessing the BIOS by pressing F2 is difficult. Instead, I have to use Settings > System > Recovery > Advanced startup in Windows, and after rebooting, choose Troubleshoot > Advanced options > UEFI Firmware Settings to access the BIOS.
  • Flash the EndeavourOS ISO to a USB disk using Rufus, contrary to the documentation on the EndeavourOS website. I had to choose the GPT partition and ISO mode to make my PC recognize the boot drive.
  • Change network settings in KDE settings: manually set the IP address and DNS and gateway to 192.168.1.2.
  • Install EndeavourOS using the online installer, KDE desktop environment, erasing the Samsung 970 EVO disk with btrfs and without swap.
  • Eject the USB disk and reboot into EndeavourOS.
  • Install nvidia-inst using yay and run nvidia-inst to install the GPU driver, then reboot again.
  • Disable the Dummy HDMI Plug in Display Settings and set the scale to 150%; turn off Adaptive Sync.
  • Fix the suspend issue by preserving video memory as described here:
    • sudo nano /etc/modprobe.d/nvidia-power-management.conf
      • options nvidia NVreg_PreserveVideoMemoryAllocations=1 NVreg_TemporaryFilePath=/var/tmp
    • sudo systemctl enable nvidia-suspend.service
    • sudo systemctl enable nvidia-hibernate.service
    • sudo reinstall-kernels
  • Install the xorg-xwayland-explicit-sync-git and egl-wayland-git AUR packages using yay. This will make Xwayland issues with NVIDIA more manageable.
  • Install fcitx5-im and fcitx5-chinese-addons using yay, and select Fcitx 5 in System Settings > Keyboard > Virtual Keyboard; add Pinyin as an Input Method.
  • Copy .ssh/id_rsa, .ssh/config, and .ssh/id_rsa.pub from Windows on the same device.
  • Enable mdns and kdeconnect in the Firewall configuration.
  • Configure Firefox:
    • Extensions:
      • Mainichi
      • uBlock Origin
      • Bitwarden
      • floccus
    • Always show the bookmark bar.
    • Disable Recommend extensions and features as you browse.
    • Enable Proxy DNS when using SOCKS v5.
    • Disable DNS over HTTPS.
    • Disable Ask to save passwords.
    • Enable “Tell websites not to sell or share my data” and “Send websites a Do Not Track (DNT) request”.
    • Uncheck all checkboxes in Firefox Data Collection and Use.
    • Uncheck “Block dangerous and deceptive content”.
    • Toggle identity.fxaccounts.enabled to false in about:config.
    • Toggle extensions.pocket.enabled to false in about:config.
    • Set the homepage to “Blank Page”.
  • Disable “Show small window previews when hovering over tasks” in task manager options.
  • Arch packages installed with pacman:
    • flatpak
    • thunderbird
    • obsidian
      • Create .config/obsidian/user-flags.conf with --ozone-platform-hint=wayland --enable-wayland-ime.
    • bitwarden
      • Create .config/electron28-flags.conf with the following content:
        1
        2
        --ozone-platform-hint=wayland
        --enable-wayland-ime
    • fastfetch
    • docker
      • Add the current user to the docker group: sudo usermod -aG docker $USER.
    • cuda
    • libreoffice-fresh
      • Create .config/environment.d/env-vars.conf with SAL_USE_VCLPLUGIN=gtk3. This can fix text and icon sizes.
    • btop
    • htop
    • bottom
    • nvtop
    • gst-plugins-qsv
    • gst-plugins-bad
    • libva-utils
    • intel-media-driver
    • intel-media-sdk
    • intel-gpu-tools
    • handbrake
    • gimp
    • krita
    • thunderbird
    • kdenlive
    • remmina
    • freerdp
    • discover
    • kid3
    • merkuro
    • zanshin
    • tokodon
    • kasts
    • telegram-desktop
    • ttf-jetbrains-mono
    • zeal
    • kubectl
    • kubectx
    • zsh
    • rustup
    • nextcloud-client
    • neovim
    • tmux
    • inkscape
    • pastel
    • yt-dlp
    • k3b
    • bat
    • dust
    • git-delta
    • eza
    • eva
    • fd
    • gitui
    • glow
    • gron
    • xh
    • pipe-rename
    • procs
    • yazi
    • ttf-hack-nerd
    • rclone
    • ripgrep
    • silicon
    • skim
    • tokei
    • viu
    • zellij
    • timeshift
      • Setup daily btrfs snapshots.
      • systemctl enable cronie.
    • nvidia-container-toolkit
    • speech-dispatcher
    • cmatrix
  • AUR packages installed with yay:
    • logseq-desktop-bin
      • Note: Wayland has not been enabled for Logseq due to a bug causing the window to shrink with each start and occasional crashes on close.
    • visual-studio-code-bin
      • Create .config/code-flags.conf with the following content:
        1
        2
        --ozone-platform-hint=wayland
        --enable-wayland-ime
    • vlc-git
    • xone-dkms
    • spotify
      • Create .config/spotify-flags.conf with the following content:
        1
        2
        3
        --enable-features=UseOzonePlatform 
        --ozone-platform=wayland
        --enable-wayland-ime
    • drawio-desktop
    • kopia-ui-bin
      • Set up daily backups of the home directory.
    • plex-mpv-shim
      • Open TCP 3000 and UDP 32410-32414 ports in firewall settings.
      • Set up auto-start in ~/.config/systemd/user/plex-mpv-shim.service:
        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
        [Unit]
        Description=Plex MPV Shim Service
        After=network.target

        [Service]
        Type=simple
        ExecStart=/usr/bin/plex-mpv-shim
        Restart=always
        RestartSec=3
        Environment=GDK_BACKEND=x11

        [Install]
        WantedBy=default.target
        1
        2
        3
        systemctl --user daemon-reload
        systemctl --user enable plex-mpv-shim.service
        systemctl --user start plex-mpv-shim.service
  • Flatpak apps:
    • Ungoogled Chromium
      • Create ~/.var/app/com.github.Eloston.UngoogledChromium/config/chromium-flags.conf with --ozone-platform-hint=wayland --enable-wayland-ime.
    • Android Studio
    • OpenLens
    • DBeaver Community
    • Zotero
    • Discord
    • Plexamp
    • Flatseal
    • Gear Lever
  • AppImage apps:
    • Insomnia
  • Configure mpv for hardware decoding: create ~/.config/mpv.conf and ~/.config/plex-mpv-shim/mpv.conf with hwdec=auto-copy. Additionally, add the command set hwdec auto-copy in Haruna Media Player.
  • Install prezto:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    zsh

    git clone --recursive https://github.com/sorin-ionescu/prezto.git "${ZDOTDIR:-$HOME}/.zprezto"

    setopt EXTENDED_GLOB
    for rcfile in "${ZDOTDIR:-$HOME}"/.zprezto/runcoms/^README.md(.N); do
    ln -s "$rcfile" "${ZDOTDIR:-$HOME}/.${rcfile:t}"
    done

    chsh -s /bin/zsh
    • Configure Konsole command to /bin/zsh
    • Add pmodules in .zpreztorc: git, syntax-highlighting, ssh, tmux, fasd before prompt
  • Install goenv:
    1
    2
    3
    4
    5
    git clone https://github.com/go-nv/goenv.git ~/.goenv
    echo 'export GOENV_ROOT="$HOME/.goenv"' >> ~/.zshenv
    echo 'export PATH="$GOENV_ROOT/bin:$PATH"' >> ~/.zshenv
    echo 'eval "$(goenv init -)"' >> ~/.zshrc
    goenv install 1.22
  • Install fnm:
    1
    2
    curl -fsSL https://fnm.vercel.app/install | bash
    fnm install 20
  • Set up the rust toolchain and install some cargo applications:
    1
    2
    3
    4
    5
    rustup install stable
    cargo install cargo-update
    cargo install dtool
    cargo install --locked csview
    cargo install fclones
    • Append export PATH="$HOME/.local/bin:$HOME/.cargo/bin:$PATH" to .zshrc.
  • Append aliases to .zshrc:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    alias comet-core="ssh comet-core.lan -t 'tmux new-session -A -s main'"
    alias porygon2="ssh porygon2.lan -t 'tmux new-session -A -s main'"

    alias vim="nvim"
    alias vi="nvim"
    alias vimdiff="nvim -d"

    export EDITOR="nvim"
    export GIT_EDITOR="nvim"
  • Check that the Intel iGPU works for video encoding: LIBVA_DRIVER_NAME=iHD vainfo --display drm --device /dev/dri/renderD129.

Lenovo Legion Y700

The Google Nexus 7, released in 2012, is the first “small tablet computer” I can recall. It featured a pleasing 800p display, a powerful processor for its time, and shipped with a “stock Android” system. I used a Nexus 7 for a few years alongside an iPad Mini, which was subsequently replaced by a larger iPad Air 2. Although the device has not been with me for many years, the memories of reading manga on a perfectly sized device and the hackable Android experience, including running a bash shell and emulators, always lead me to seek the spiritual successor to the Google Nexus 7.

For the past two years, I have been using the Lenovo Legion Y700 (2022) as my primary device for content consumption. It comes equipped with a Snapdragon 870 processor, 256GB of storage, 12GB of RAM, and an 8.8-inch 120Hz 2K display. Similar to the Google Nexus 7, there is an improved “2023” version featuring a Snapdragon 8 Gen 1 processor. However, the older 2022 model suffices for my needs, and I prefer its advanced 3.5mm audio port.

While the hardware performance is excellent and the freedom to install apps from various sources—such as Google Play, F-Droid, GitHub, or directly from the developer’s website—feels right, I have never felt good about its ZUI operating system or any Android OEM system. I appreciate that Lenovo doesn’t include too much bloatware, doesn’t require me to log in to its account, and natively supports Google Play Services, but there are always moments when I encounter buggy or peculiar issues with some apps, or something just doesn’t seem right with the OS.

On the other side, for the past two years, a Google Pixel 6a installed with GrapheneOS has been my secondary phone. While the hardware is decent, it kind of underperforms when compared to an iPhone or a Snapdragon 8-equipped Android device. However, the open-source operating system is more than just great. It offers many secure features, and even Google Play Services can be run in a sandbox to protect privacy and battery life. I can disable network access for Google Camera, Photos, and Gboard while still keeping their on-device ML features. Additionally, I can place some apps that I don’t trust as much in a Work Profile with Shelter, making other apps and data invisible to these apps.

In the end, I learned that the bootloader of the Lenovo tablet can be unlocked, and I discovered Project Treble and the Generic System Image (GSI), which enable the availability of custom Android distributions on most devices without excessive difficulties. Weeks ago, I followed this video and installed crDroid on my Lenovo Legion Y700.

crDroid, a fork of LineageOS, offers a pure and customizable Android experience. After experimenting with it for a few hours, I deeply regret not replacing the OS on this tablet sooner. Using an open-source stock-like Android system is incredibly refreshing. The UI is smooth, and I can truly appreciate the 120fps screen while navigating. It brings back memories from more than a decade ago with the Google Nexus 7, making me realize that I’m holding what could be considered the spiritual successor to the Google Nexus 7.

System Experience

As is common with most custom images lacking official support, there are a few caveats when using a GSI ROM on this tablet, but I can say that I have fewer issues with crDroid than with the stock ZUI.

The settings and issues worth noting are:

  • 120fps can be enabled under Settings > Phh Treble Settings > Misc features > Force FPS, by setting it to [email protected].
  • Lock screen rotation can be activated by going to Settings, searching for “rotation,” and selecting “Rotate lock screen.” I also enabled all rotation modes here.
  • The 3.5mm audio port does not work initially, but it can be fixed under Settings > Phh Treble Settings > Qualcomm features > “Use alternate audio policy.”
  • The double-tap to wake feature is not operational yet, but, surprisingly, the Lenovo screen cover lock/unlock feature does work.
  • Adaptive brightness does not work.
  • Changing Wallpaper crashes initially, which is fixed by installing Fossify Gallery.

The crDroid launcher is a lightweight fork of the Google Pixel launcher. Initially, I preferred it as it is similar to the launcher used in GrapheneOS, so I was already accustomed to it. However, there is an issue where the background of the app drawer remains dark even when using the light theme, which may be due to some compatibility issues with the tablet size or the enabling of the work profile. As a result, I switched to Niagara Launcher, and it looks perfect.

I also learned a valuable lesson: Unlike Pixel phones, Lenovo does not allow the re-locking of the bootloader once a non-stock OS is installed. The first action I took after installing crDroid_gsi was attempting to re-lock the bootloader. Subsequently, the tablet became bricked, displaying a message that the system was corrupted and could not boot. Thankfully, the tablet could be unbricked by following this guide.

Privacy and Security

In terms of security, I’m content with the basic Android security model, but it’s important to note that the tablet is less secure than a Pixel equipped with GrapheneOS.

Keeping the bootloader unlocked is clearly a security risk, as theoretically, a thief with hardware access could install a hacked ROM to decrypt data on the device. Additionally, the face recognition feature on this device is less secure than Apple’s Face ID or the face unlock feature on the Pixel 8, meaning a thief could theoretically mimic my face to unlock this device.

I do not consider a tech-savvy thief attempting to access my data to be within my threat model. Nonetheless, I choose not to store sensitive data on this tablet, and I also disabled biometric unlock in Bitwarden after setting up most of my apps.

While crDroid does not come with sandboxed Google Play Services like GrapheneOS does, disabling the advertising ID, personalization, autofill, and location settings is sufficient for my privacy needs regarding Google.

One theoretical privacy threat when using mobile devices is the possibility of manufacturers or popular apps harvesting the list of installed apps on the device and potentially sharing this information with authorities, which could result in phone calls from “anti-fraud offices” warning about the risks of certain apps. While this is all theoretical and without neither any evidence nor my own experience, using an open-source system can definitely mitigate such threats. Similar to my setup on my Pixel phone, I use Shelter to isolate certain apps within a Work Profile.

Apps

Despite it being over a decade since the launch of Nexus 7, there are still not many Android apps optimized for tablet size. However, since Android does not restrict non-optimized apps to run in a centered window like the iPad, most Android apps function adequately on this smaller tablet. Additionally, the operating system supports basic multitasking features, such as split-screen.

I prefer to install apps from Neo Store when possible, which is a frontend for F-Droid and also supports IzzyOnDroid by default. For apps that are only available through GitHub releases, I can update them automatically using Obtainium.

Here is a list of apps worth mentioning on this tablet:

  • Cromite: The default browser, featuring built-in ad blocking and privacy enhancements.
  • DAVx5 and Etar: Used for syncing calendars with Nextcloud.
  • FocusReader: An RSS client connecting to Miniflux.
  • GitNex: A client for Forgejo.
  • Material Files: A file manager supporting SMB.
  • Mihon: A manga reader and Komga client.
  • Moe Memos: A note-taking app developed by me.
  • MX Player Pro: Used for watching local videos not in my Plex library.
  • NewPipe: Considered the best YouTube client.
  • Play Books: Used for reading ePub books, chosen for its page-turn animation.
  • Plex and Plexamp: My primary media players.
  • Squawker: Used to subscribe to voice actors from the former birdsite.
  • Stealth: A Reddit client.
  • Tasks.org: For to-do lists syncing with Nextcloud.
  • Termux: A powerful terminal emulator.
  • Transdroid: Used to manage my qBittorrent server.
  • Tusky: A Mastodon client.
  • Zoo for Zotero: For reading from my PDF library.

Gaming

The Lenovo Legion Y700 is marketed as a “gaming tablet.” Although I am not a mobile gamer—since most modern mobile games adopt a freemium, casino-like business model that I feel detracts from the concept of video games as works of art—Android remains an excellent platform for emulation and game streaming. This tablet has the capability to emulate PS2 and Wii games at full speed with upscaled textures, as well as stream PC games via Moonlight, or PS5 games using PSPlay or Chiaki, all with negligible latency.

Gaming on Android devices represents a larger story for me, but it’s not particularly tied to this tablet or the Pixel phone. I’m keeping it a secret for now and may discuss it in more detail in the future.

Conclusion

The journey to find the spiritual successor to the Google Nexus 7 led me to the Lenovo Legion Y700, a device that, while initially not perfect due to its ZUI operating system, transformed into a nearly ideal tablet experience after installing crDroid. This shift back to a stock-like Android experience reminiscent of the Nexus 7 days has been both nostalgic and enlightening, proving that the essence of a great tablet is not just in its hardware but in the freedom and flexibility of its software. The Lenovo Legion Y700, with its powerful specs and the newfound joy of a pure Android experience, serves as a testament to the enduring appeal of customizable, user-centric technology. It reminds us that sometimes, the true successor to a beloved piece of technology is not about finding an identical replacement, but about recapturing the spirit of innovation and openness that made the original so special.

Homelab, Illustrated By DALL·E

Homelab is a place where you can store all your family’s data, self-host applications and services, locally stream media, and experiment with various technologies.

A Homelab can start with low-power devices such as a Raspberry Pi, or with hacking your router to install open-source firmware and run custom applications. It can also involve easy-to-use consumer NAS solutions like Synology. Alternatively, you might find yourself overwhelmed by used data center hardware.

Throughout 2023, I have been tinkering with my home network and computing setup, which has been both fun and rewarding.

Hardware

Various devices operate continuously at my home. Some were shelved last year, but there are a few newcomers worth mentioning.

The router “Rotom,” which I use for my home’s internet, is an iKoolCore R1. This compact device is powered by a Celeron N5105 processor and is equipped with 8GB of RAM, a 128GB SSD, and four 2.5GbE Ethernet ports. It runs Proxmox VE, hosting an OpenWRT VM as my primary firewall, and a Debian VM for several networking applications.

This summer, I noticed that the SSD became read-only on several occasions due to overheating. Fortunately, the manufacturer offers an R1 Pro upgrade kit. This kit replaces the case with a heatsink that directly contacts the NVMe SSD, effectively mitigating the overheating issue. Additionally, it includes a PWM fan, which keeps this compact computer silent in my closet.

My Network Cabinet in Augest 2023

I have a 15U network cabinet situated in the corner of my living room. At the bottom lies a Santak TG-BOX 850 UPS, safeguarding the devices within the cabinet. Positioned in the middle is my NAS, acquired in 2020 – a Synology DS1621+ equipped with six 8TB HGST drives and 12GB of ECC RAM. It’s set up in a RAID-6 configuration, providing a total storage capacity of 32TB.

At the top of the cabinet is the Homelab server “Porygon,” which I assembled in 2023. Inspired by Tao of Mac, it’s built around an ASRock DeskMeet B660 box. The configuration includes an Intel Core i7 12700 CPU, 128GB of DDR4 RAM, two NVMe SSDs (a WD SN770 2TB and an older Samsung 970 Evo Plus 1TB), and a Geforce RTX 3060 GPU.

The DeskMeet B660 also runs Proxmox VE. The integrated Iris Xe GPU is dedicated to a Fedora Workstation VM for transcoding videos and offering a remote desktop. The RTX 3060 switches between a Windows VM, which I use for game streaming, and a Debian VM for doing AI experiments.

However, I’ve encountered heat issues with the “Porygon” server inside the network cabinet. The CPU, cooled by a Noctua NH-L9i-17xx, can quickly reach 100°C during moderately intensive tasks. The SSD, even with a heatsink, often heats up to 70°C. To address this, I implemented two solutions. Firstly, I replaced the PSU fan with a Noctua NF-A12, which reduced the CPU temperature by 15°C, both at idle and under load, at the cost of injuring both my hands. Secondly, I installed a dedicated cabinet fan controlled by a Xiaomi Smart Plug. When the SSD temperature hits 70°C, Home Assistant activates the fan, which turns off once the temperature drops below 65°C. Ultimately, I’m quite content with the new Homelab server. It’s usually quiet and energy-efficient, yet it can be exceptionally powerful when needed.

Applications and Services

In my Homelab, I run a wide array of applications, catering to various needs. This includes network applications that ensure connectivity for my devices at home and allow me to access my home network remotely. I also use note-taking apps, read-later services, home automation systems, media management services, file synchronization programs, RSS readers, as well as a git server complemented by CI/CD pipelines. Moreover, I have dashboards in place to navigate and monitor all these applications efficiently. It’s fair to say that a significant portion of my digital life depends on these machines, which are entirely under my control at home.

Homepage Dashboard and my Applications

On my router “Rotom”

On the compact yet capable machine, I have set up two virtual machines and one LXC container to manage various services and applications.

One VM operates as my primary router OS, running OpenWRT. I’ve configured it with passthrough for three NIC ports. The WAN port connects to the modem provided by my ISP, while the LAN ports, in conjunction with the VM net port, establish a bridged LAN network. The majority of OpenWRT’s configuration revolves around DHCP settings and firewall rules.

In addition to the routing functions, “Rotom” hosts several other programs:

  • Jellyfin: This serves as my secondary media center. I’m contemplating a switch from Plex to Jellyfin since it is fully open-source and doesn’t depend on a remote server.
  • frpc: This is for exposing certain services, like an SSH service that enables me to connect back to my home network. It’s useful because I have a CG-NAT IPv4 network. If I encounter a lack of IPv6 support in an external Wi-Fi network, I rely on frp to access my home network.
  • cloudflared: This acts as an alternative method for exposing services via Cloudflare Tunnels.
  • cloudflare-ddns: A dynamic DNS service that maintains an AAAA record, facilitating direct IPv6 connections back to my home.
  • traefik: This reverse proxy manages services not running on “Porygon.” I’ll detail the services running on the Homelab PC separately.

Jellyfin operates within an LXC container, with the Intel iGPU passed through for video transcoding. The remaining services are housed in Docker containers running on a Debian VM. The Debian VM also functions as a side router for my Homelab PC and devices such as the Apple TV and Sonos One.

On my Synology NAS “Uxie”

As my storage center, I need my NAS to be as robust as possible, so I haven’t installed too many applications on it.

In addition to standard NAS services like SMB, NFS, and File Station, the applications I use include:

  • Synology Photos: Backs up the photos of my family. I’m considering switching to an open-source solution like Immich, but the last time I tried, Immich didn’t integrate well with the iCloud Photo Library.
  • qBittorrent and VueTorrent: Mostly used to download Linux ISOs.
  • aria2: Another tool for downloading Linux ISOs.
  • Forgejo: A self-hosted Git server; it’s lightweight and a community fork of Gitea.
  • Sonatype Nexus Repository: My private package manager for Docker and npm, and also a cache proxy for docker.io and ghcr.io to ensure all my applications can start without an internet connection.
  • MinIO: An S3-like object storage service for my other applications.
  • Navidrome: A lightweight music manager compatible with the Subsonic API.
  • Sonarr: An automatic media manager for anime and TV shows.
  • Radarr: An automatic media manager for movies.
  • Bazarr: A subtitle downloader for content in Sonarr and Radarr.

Apart from first-party Synology apps, all third-party programs are running in Docker containers.

A Home Kubernetes Cluster in “Porygon”

I’ve migrated the k3s control plane and worker VMs from Frost Canyon NUC “Comet” to my new Homelab PC. Most of my applications are running in this k3s cluster because it’s easy to maintain, upgrade, and recover. Building this has also been a great way to learn.

Here’s a list of applications running in this cluster:

  • Memos: My primary note-taking app for capturing my thoughts and ideas. I also developed the iOS and Android client titled “Moe Memos.”
  • Miniflux: My RSS reader. It supports the Fever and Google Reader API and can be connected to various RSS clients.
  • Vaultwarden: An unofficial server written in Rust for the open-source password manager Bitwarden. It’s more lightweight than the official server.
  • Wallabag: My link manager and “read later” app.
  • MicroBin: A text and file-sharing application for when I’m transferring things without Universal Clipboard.
  • NocoDB: An AirTable-like smart spreadsheet. I use it for personal project management and to track games and movies.
  • Komga: My manga library. It supports OPDS clients like Panels and Mihon.
  • Paperless-ngx: A document management system for organizing scanned documents.
  • Stirling PDF: An all-in-one PDF toolkit.
  • Audiobookshelf: I use it as an automatic podcast downloader.
  • Photoview: I use it to manage collected pictures not in my photo library, such as CD and Blu-ray booklets.
  • Prowlarr backed by FlareSolverr: An indexer manager for Sonarr/Radarr.
  • YoutubeDL-Material: Downloads videos from YouTube via yt-dlp.
  • draw.io: The most powerful diagramming tool.
  • Kimai: A self-hosted time tracker.
  • Firefly III: A self-hosted finance manager. I haven’t used it much yet.
  • Node-RED and n8n: Powerful automation tools, but I haven’t used them much yet.
  • Woodpecker CI: A Kubernetes native continuous integration solution.

The manifest of my Kubernetes cluster is managed in a Git repository and is automatically deployed via a GitOps tool named Flux CD. When I push changes to the repository, such as adding a new application or upgrading Docker images, the deployment occurs within a few minutes.

Recently, I discovered a powerful tool named Renovate Bot. It scans my Kubernetes manifest repository every few hours to identify which Helm releases or container image tags require upgrades and then creates pull requests in Forgejo, similar to how Dependabot operates on GitHub.

In addition to the applications I’m running, my setup includes the Ingress NGINX Controller, MetalLB, and cert-manager to facilitate service access and manage certificates. The data for these applications is stored on Longhorn volumes, which are replicated across all three worker VMs and two SSDs. I’ve also integrated Kube Prometheus Stack and Grafana Loki to monitor the cluster and review pod logs.

Another tool I use in parallel with kubectl for managing resources in the cluster is Portainer. I’ve also configured Portainer environments to manage Docker Compose stacks on my NAS, router, and several cloud servers. For convenient navigation through all the apps in my Homelab, I utilize a highly customizable dashboard named Homepage. It seamlessly integrates with Kubernetes and is capable of discovering services based on Ingress annotations.

All k3s nodes operate under Ubuntu 22.04, and I’ve configured the system-upgrade-controller to automatically update k3s and apt.

Remote Development and Desktop

Like my Kubernetes cluster, The Fedora Workstation VM “comet-core” is also migrated from the Frost Canyon NUC. The Intel Iris Xe iGPU is passed through it. I use it mostly as a remote development server, and when I need GUI applications, I can access the GNOME desktop with RDP via xrdp and xorgxrdp.

Notable applications running in this VM contains:

  • Plex: My primary media library. I have lifetime Plex Pass and I still prefer Plex for a variety of unique features. But I’m kinda worried about Plex’s future and its troublesome handling of privacy. Maybe I’ll switch to Jellyfin some day.
  • code-server: A web development environment. However, I often develop on this VM via the Remote Development feature in Visual Studio Code.
  • calibre and calibre-web: My book library. I also use calibre to DeDRM and convert digital books I bought.
  • Handbrake: I use it to transcode videos from my Blu-ray collection to H.265 with Intel Quick Sync.

Gaming and AI Experiments

Having a dedicated GPU has significantly justified upgrading my Homelab PC. I’ve set up a Windows VM “Porygon-Z” and a Debian VM “Porygon2,” both configured to utilize the RTX 3060 GPU. However, only one VM can be powered on at a time since GPU passthrough is restricted to a single VM.

In the Debian VM, I’ve been experimenting with a few smaller LLM apps such as Ollama and ChatGLM-6B, as well as creating art with the Stable Diffusion web UI. I’ve also explored vector databases such as Chroma.

On the Windows VM, I’ve installed Sunshine, primarily to stream games from PC Game Pass, and titles that are either unsupported or don’t perform well on the Steam Deck, to my handheld console with Moonlight. The ability to play games at 1080p 60fps with the highest settings on a handheld device was a dream I never thought would be possible until now.

Other VMs and LXC Containers

Last but not least, I’d like to highlight more applications.

Nextcloud serves as my file synchronization server. It used to perform poorly, particularly when syncing a large number of small files. However, with recent upgrades to both Nextcloud versions and my hardware, its performance has significantly improved, offering super-fast file transfers. I’ve set up Nextcloud with Nextcloud AIO in an LXC container on “Porygon.” Beyond file syncing, Nextcloud also manages my calendar and tasks through CalDAV and syncs browser bookmarks using Floccus.

Home Assistant OS orchestrates all my IoT devices and bridges them to HomeKit. For devices that don’t natively support HomeKit, I can access them through hass-xiaomi-miot. Home Assistant also automates various tasks, such as controlling the network cabinet fan I mentioned earlier and activating the Aqara M1S night light when someone enters the living room at night.

Backups

I loosely adhere to the 3-2-1 backup strategy for safeguarding my data. My NAS serves as the primary backup target for various devices, including my family’s iMac via Time Machine and two Proxmox VE devices using a Proxmox Backup Server VM. Additionally, my older Synology DS916+ NAS, which has been operational for 8 years, secures the data from my main NAS through rsync and Hyper Backup.

For offsite backup, I utilize Backblaze B2 and OneDrive. Backblaze B2 houses backups from Hyper Backup, restic, Longhorn, and Arq, while selected folders from my NAS are one-way synced to OneDrive using Cloud Sync. I’ve opted not to have a remote backup for my media library due to the storage costs, considering that most of the content can be reacquired by re-ripping physical media or re-downloading.

Homelab in 2024

That’s all for my 2023 Homelab setup. I’m very satisfied with my current setup, and there are a few areas I’d like to explore.

Currently, I only have Gigabit Ethernet, even though I have a few 2.5GbE devices. Perhaps I should add a few 2.5GbE switches to access my data faster.

I’d like to make the most of my devices. I’m considering reconfiguring my Frost Canyon NUC as a backup device in case one or two pieces of hardware in my Homelab fail. It happened before with the Noctua CPU fan, and I lost access to “Porygon” for a few days. I also have a spare Raspberry Pi 4 and a Zero 2 W, but I haven’t figured out how to utilize them effectively.

I’m also interested in open-source NAS solutions such as mergerfs, SnapRAID, and openmediavault, but I’m not yet motivated to build one. I’m looking into VLAN setup as well, although I don’t think I really need it yet.

Overall, I have enjoyed my Homelab journey and am looking forward to more building and learning. I hope you enjoyed reading this post, and if you have any suggestions or ideas, please reach out to me on Mastodon.

Background

Hundreds of thousands of image files are stored on 52Poké Wiki. Previously, we used AWS S3 to store all these images, AWS Lambda for image processing (generating thumbnails and converting to WebP format), and two layers of cache (a container on Linode and Cloudflare CDN) for image distribution.

Our monthly AWS bill increased gradually due to the growing number of images and traffic each month. Even with two layers of cache, more than 280GB of traffic defaulted to AWS S3, incurring a $20 expense every month.

Fortunately, there are many S3-compatible services available at much more competitive prices. We considered switching to Cloudflare R2, Backblaze B2, or Linode Object Storage.

Choice

Currently, we have 150GB of files. It costs $2.25 every month to store on Cloudflare R2, $0.9 for Backblaze B2, and $5 for Linode Object Storage. Egress traffic charges are zero for all services: Backblaze B2 doesn’t charge for 3x the monthly data stored, Linode uses a traffic pool that sufficiently covers our usage, and Cloudflare never charges for egress bandwidth. If the data storage increases to 250GB, it will cost $3.75 on Cloudflare, $1.5 on Backblaze, and remain $5 on Linode.

However, since the main services of 52Poké are located in Linode’s Tokyo datacenter, there is significant latency when proxying from the other side of the Pacific Ocean. Linode recently announced Object Storage availability in their Osaka datacenter, and with a latency of only 8ms between Tokyo and Osaka, this is an optimal choice performance-wise. We can also eliminate one cache layer (20GB) backed by block storage, saving $2 monthly. Ultimately, by migrating from AWS S3 to Linode Object Storage, we can save $17 every month.

Steps

Creating a Bucket on Linode and Setting Up the Bucket Policy

We can create an Object Storage bucket in Akamai Cloud Manager with the bucket name media.52poke.com.

Previously, we configured a bucket policy on S3 to allow access solely from the IP addresses of 52Poké’s cloud servers. While Akamai Cloud Manager doesn’t support direct bucket policy configurations, they can be set up using s3cmd as per Linode’s documentation.

The bucket policy looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::media.52poke.com/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"..."
]
}
}
}
]
}

Full Migration

To minimize disruption to 52Poké Wiki during this migration, we’ll implement a two-step mechanism. First, we’ll migrate all files from the AWS S3 bucket and allow users to upload or update files. Later, we’ll conduct an incremental migration, which should take considerably less time.

A Kubernetes job will be used to run rclone sync to sync all files to the new Linode Object Storage bucket. For the access key ID and secret access key of Linode Object Storage, we’ll use SOPS to encrypt and store them in the Git repository. These can then only be decrypted with the private key in the 52Poké Kubernetes cluster and deployed automatically.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
apiVersion: batch/v1
kind: Job
metadata:
name: migrate-media
namespace: default
spec:
ttlSecondsAfterFinished: 100
template:
spec:
containers:
- command:
- /bin/sh
- -c
- rclone sync awss3:media.52poke.com los:media.52poke.com
env:
- name: RCLONE_CONFIG_AWSS3_TYPE
value: s3
- name: RCLONE_CONFIG_AWSS3_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: accessKeyID
name: aws-s3
- name: RCLONE_CONFIG_AWSS3_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secretAccessKey
name: aws-s3
- name: RCLONE_CONFIG_LOS_TYPE
value: s3
- name: RCLONE_CONFIG_LOS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: accessKeyID
name: linode-object-storage
- name: RCLONE_CONFIG_LOS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secretAccessKey
name: linode-object-storage
- name: RCLONE_CONFIG_LOS_ENDPOINT
value: jp-osa-1.linodeobjects.com
image: rclone/rclone:latest
imagePullPolicy: Always
name: migrate-media
restartPolicy: Never

Disabling Uploads

In the second step, we must disable uploads on MediaWiki to prevent data loss. This can be configured via LocalSettings.php.

1
$wgEnableUploads = false;

Incremental Migration

We’ll now run the Kubernetes job again to sync updates from the old AWS S3 bucket to the new one. Additionally, we’ll manually check that recently uploaded files from 52Poké Wiki have been synced to the new bucket.

Updating Malasada

Malasada is an AWS Lambda Serverless function designed to generate image thumbnails and convert images to the WebP format to conserve bandwidth.

Previously, we used an IAM policy to grant the Serverless function access to the S3 bucket. Now, with the shift from AWS S3, we’ll need to manually provide the access key ID, secret access key, and the endpoint of Linode Object Storage to the Serverless function using environment variables.

1
2
3
4
5
6
7
8
const s3 = new S3({
credentials: {
accessKeyId: process.env.S3_ACCESS_KEY_ID,
secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
},
endpoint: process.env.S3_ENDPOINT,
region: process.env.S3_REGION,
});

Configuring Nginx

The Nginx configuration for the domains media.52poke.com, s0.52poke.wiki, and s1.52poke.wiki must be updated to replace the AWS S3 domain with that of Linode Object Storage. We also removed the persistent volume claim backed by block storage from the nginx deployment.

1
proxy_pass http://media.52poke.com.jp-osa-1.linodeobjects.com$request_uri;

After updating the nginx configuration, we noticed all requests were failing due to permission issues. However, there was no issue when requesting the upstream Object Storage URL with curl in the nginx container.

Upon further investigation, we determined that Linode Object Storage was blocking requests containing “X-Forwarded-For” and “X-Real-IP” headers because these headers didn’t align with the bucket policy. Removing these headers from the request resolved the issue.

1
2
proxy_set_header X-Real-IP '';
proxy_set_header X-Forwarded-For '';

Configuring MediaWiki

We now need to adjust the AWS S3 MediaWiki extension to use Linode Object Storage instead of AWS S3. This adjustment can be quickly made via the endpoint parameter. Afterward, we can re-enable file uploading on MediaWiki.

1
2
$wgFileBackends['s3']['endpoint'] = 'http://jp-osa-1.linodeobjects.com';
$wgEnableUploads = true;

Configuring WordPress

The 52Poké homepage uses the WP Offload Media Lite plugin to upload files to S3. While the plugin’s admin page doesn’t offer functionality to change the object storage endpoint, our investigation of the source code revealed hooks that allow modifying the S3 connection configuration. This can be adjusted in wp-config.php.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
add_filter( 'as3cf_aws_s3_client_args', function ( $args ) {
$args['endpoint'] = 'https://jp-osa-1.linodeobjects.com';
$args['region'] = 'jp-osa-1';
$args['use_path_style_endpoint'] = true;
return $args;
} );
add_filter( 'as3cf_aws_get_regions', function ( $regions ) {
$regions = array(
'jp-osa-1' => 'Osaka',
);
return $regions;
} );
add_filter( 'as3cf_aws_s3_bucket_in_path', '__return_true' );
add_filter( 'as3cf_aws_s3_domain', function ( $domain ) {
return 'linodeobjects.com';
} );

Backup

As the migration nears completion, it’s important to remember that we have a cron job set up to back up files from AWS S3 to Backblaze B2. We’ll need to update the backup source to Linode Object Storage, which can be achieved through the environment variables of rclone.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
env:
- name: RCLONE_CONFIG_MEDIA_TYPE
value: s3
- name: RCLONE_CONFIG_MEDIA_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: accessKeyID
name: linode-object-storage
- name: RCLONE_CONFIG_MEDIA_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: secretAccessKey
name: linode-object-storage
- name: RCLONE_CONFIG_MEDIA_ENDPOINT
value: jp-osa-1.linodeobjects.com

Conclusion

52Poké Wiki successfully transitioned from AWS S3 to Linode Object Storage for image storage. Despite the availability of lower-cost options like Cloudflare R2 and Backblaze B2, Linode Object Storage was the optimal choice given its low latency benefits and the site’s primary services being located in Linode’s Tokyo datacenter. With this change, we anticipates a monthly savings of $17 and enhanced performance. Through a careful migration process, including addressing challenges with permissions and configurations, the transition was executed smoothly. This move underscores the importance of regularly evaluating and optimizing infrastructure choices to ensure both cost-effectiveness and performance for online communities.

0%