# Building software for the NAS

The NAS has no compiler. You build on your own Linux computer, or on the NAS inside Bucket & Code Server IDE's
**environments**, and check the result against the target NAS before it runs there. The glibc rules further down
hold for both ways.

## On your own computer

What follows was measured on Ubuntu 24.04 (glibc 2.39) on an x86_64 computer.

- For an x86_64 NAS, build with the computer's own `gcc` and `g++`. For a 64-bit ARM NAS (aarch64, QDK's
  `arm_64`), Ubuntu's cross toolchain: `apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu`, whose
  programs carry the prefix `aarch64-linux-gnu-` (`aarch64-linux-gnu-gcc`).
- Link every dependency statically and glibc dynamically. Link C++ programs with `-static-libgcc
  -static-libstdc++`. Build the dependencies into a sysroot of your own and point pkg-config only at it:
  `PKG_CONFIG="pkg-config --static"` and `PKG_CONFIG_LIBDIR=<sysroot>/lib/pkgconfig`, so that no library of the
  computer ends up in the program.
- For Go, `CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build` (`GOARCH=arm64` for ARM) gives a static program that
  needs nothing from the NAS, users and DNS included. It is the simplest thing to ship.

Check every binary before you ship it (use the computer's own `objdump` for an x86_64 build):

```sh
readelf -h f | grep Machine     # matches the target: AArch64, or Advanced Micro Devices X86-64
readelf -d f | grep NEEDED      # only libc, libm, libpthread, libdl, librt, libresolv, ld-linux, libgcc_s
aarch64-linux-gnu-objdump -T f | grep -oE 'GLIBC_[0-9.]+' | sort -uV | tail -1   # the newest glibc it needs
```

The last line must not name a version newer than the target NAS's glibc: read that on the NAS (below). An aarch64
binary can be started on the computer with `qemu-aarch64-static -L /usr/aarch64-linux-gnu ./f` (package
`qemu-user-static`). That run uses the cross toolchain's glibc, not the NAS's, so it does not replace the check.

To package the result, see [qpkg.md](qpkg.md).

## In Bucket & Code Server IDE: environments

Bucket & Code Server IDE builds on the NAS inside environments, Linux systems unpacked from a root file system
into `envs/<name>` of the package's data root. `bvs guide` opens its full user guide.

| Root file system (catalog id) | glibc | Good for |
|---|---|---|
| `ubuntu-24.04-<arch>` | 2.39 | the default builder: dynamic programs for every NAS the package runs on (glibc 2.39, the floor); the App Builder, `bvs qbuild`, the Clang, Fortran, Kotlin, .NET and kernel builder presets |
| `debian-12-<arch>` | 2.36 | dynamic programs for a NAS with glibc 2.36 or newer; the mail server and web server example projects |
| `ubuntu-22.04-<arch>` | 2.35 | dynamic programs for a NAS with glibc 2.35 or newer; older toolchains |
| `debian-13-<arch>` | 2.41 | programs that stay in the environment, or static ones: a dynamic build needs glibc 2.41 on the NAS |
| `alpine-3.22-<arch>` | musl | static programs only: the NAS has no musl loader |

`<arch>` is `x86_64` or `aarch64`, the NAS's own processor.

```sh
bvs env list
bvs env create deb12 --from debian-12-x86_64 --package build-essential   # an admin's job
bvs env shell deb12                      # a root shell inside, in the current folder
bvs env run deb12 -- make                # one command inside, in the current folder
```

- The Examples gallery's presets make ready environments: `ubuntu2404-builder` (the default builder: the App
  Builder's build tools, patchelf, rsync), `debian12-builder` (the same on Debian 12), `clang-llvm`, `fortran`,
  `kernel-builder`, `dotnet8`, `kotlin`.
- `bvs env shell` and `bvs env run` start in the folder you run them from. A folder under the data root's
  `projects/` is the same folder under `/projects` inside; any other folder (a CI checkout) is bound at `/work`.
- Inside, `apt-get install` works and what you install stays in that environment. Nothing of it reaches the NAS.
- Everything in an environment runs as root, sees every process of the NAS and uses its network. It is not a
  security boundary.

## Runtimes and cross toolchains in Bucket & Code Server IDE

`bvs tc install <id>` (an admin) installs from the signed catalog; `eval "$(bvs use <id>)"` puts it on `PATH` in the
current shell:

- Runtimes for x86_64 and aarch64: `python-3.13-*`, `go-1.27-*`, `node-22-*`, `node-24-*`, `zig-0.15-*`, `rust-1.98-*`,
  `cmake-4-*`, `ninja-1.13-*`, `jdk-25-*`.
- Cross toolchains (Bootlin, glibc 2.39 or musl): `bootlin-aarch64-glibc-stable`, `bootlin-armv7-eabihf-glibc-stable`,
  `bootlin-x86-64-glibc-stable` and their `-musl-` twins. They run on an x86_64 NAS only.
- Go from `go-1.27-*` builds the same static program as on your own computer (above).

## glibc: what the NAS can load

A dynamic program runs on a NAS only when that NAS's glibc has every `GLIBC_` version the program asks for. The
glibc of the system you build on (your computer, or an environment) is the newest the programs built there may ask
for, so build on one whose glibc is not newer than the target's, or check with `objdump -T` (above) that the program
asks for nothing newer. Do not count on any other library the program loads from the build system: the firmware
carries some under the same names, in versions that change between releases (from QuTS hero h6.0.1 to h6.1.0,
OpenSSL 3.0.20 became 3.5.7, zlib 1.2.8 became 1.3.2 and libxml2 2.9 became `libxml2.so.16`), and ncurses 5 where
the environments have 6. Bundle them or link static.

In Bucket & Code Server IDE, 2.39 is the package's glibc floor: the glibc measured in QuTS hero h6.0.1's root
filesystem, and Ubuntu 24.04's. The package's build refuses any program of its own that needs more, its service
script does not start where those programs do not run, and the App Builder checks every app for other NAS models
against it. QTS 6.1.0 and QuTS hero h6.1.0 measured 2.43. QTS 6.0 has no public release to measure and earlier
h6.0.0 builds were not measured, so for a NAS you do not know, read its glibc.

To read the target's glibc:

- On the NAS where Bucket & Code Server IDE runs, `bvs ldd` does it for you (below).
- On any other NAS, and from your own computer in an SSH login, `ldd`, `getconf` and `readelf` are not there, but
  BusyBox `strings` is:

  ```sh
  strings /lib/libc.so.6 | grep '^GLIBC_2\.'   # the versions this NAS's glibc defines
  ```

  A program runs there only if every `GLIBC_` version it needs is in that list.

On your own computer, check a binary with `readelf` and `objdump` ([On your own computer](#on-your-own-computer)).
In Bucket & Code Server IDE, the analyzer reads the file and the loader data of the NAS it runs on (or of an
environment) and never runs the program:

```sh
bvs ldd ./mytool                          # against this NAS; exit 3 when a finding stops the program
bvs ldd ./mytool --env ubuntu2404-builder   # against an environment instead
```

**"`GLIBC_2.xx' not found`"** means the program asks for a newer glibc than the system that loaded it has. It
has two causes:

- **The target NAS is older than the build system.** An Ubuntu 24.04 build (2.39) that fails with `GLIBC_2.38`
  only tells you the target's glibc is below 2.38. Read it, then: 2.36 or newer, build on Debian 12 (the
  `debian-12` environment); 2.35, on Ubuntu 22.04 (`ubuntu-22.04`); older than that, or not known, link static (Go
  with `CGO_ENABLED=0`, or the App Builder's `static` mode). A NAS still on QTS 5.x can have a much older glibc
  (2.21 has been reported on QTS 5.2). The analyzer's `version_missing` finding offers the same two fixes: an older
  environment (`older_env`) or a static link.
- **The program runs inside an older environment** (Bucket & Code Server IDE). A program built in
  `ubuntu2404-builder` and run inside `debian12-builder` (2.36) fails the same way. `bvs ldd ./mytool --env
  debian12-builder` shows it; build in the environment the program runs in.

Never copy a newer `libc.so.6` next to the program.

The analyzer's other findings and what fixes them: `missing_library` (add the recipe that has it, bundle it, or link
static), `wrong_machine` (build for the NAS's processor), `interpreter_missing` (the program asks for a loader the
NAS lacks: Alpine's musl, for example), `runpath_*` (a RUNPATH into a build folder or outside the app).

## Static glibc programs and NSS

A statically linked glibc program still loads the NAS's NSS modules when it looks up a user, a group or a host
through any source other than `files` and `dns`. QTS's `/etc/nsswitch.conf` names `compat` (and `wins` for hosts).
A static program built with another glibc that loads the NAS's `libnss_compat` can crash: `id`, `ls -l` and
`curl --version` did. Options:

- Go with `CGO_ENABLED=0`, which has its own resolver and user lookup.
- The App Builder's `static` link mode, which links a small pin into every program that does such lookups so that
  only `files` and `dns` are used.
- glibc linked dynamically and needing no `GLIBC_` version the NAS lacks: static dependencies with a dynamic glibc
  on your own computer, or a dynamic build in an environment.

A static glibc program also finds no locale data on QTS and runs in the C locale.

## App Builder (Bucket & Code Server IDE)

The App Builder compiles recipes (153 of them: bash, curl, openssl, nginx, tmux, sqlite …) into an app prefix,
`/usr/app/<name>` by default, inside a Debian or Ubuntu environment, `ubuntu2404-builder` unless you choose another. The link mode says how the programs find their
libraries:

| Mode | Needs on the NAS |
|---|---|
| `dynamic` | the tree at the prefix path (absolute RUNPATH) and the NAS's glibc |
| `relocatable` | the NAS's glibc; the tree finds its libraries from any folder (`$ORIGIN`) |
| `bundle` | the NAS's glibc only: the libraries the programs load from the environment are copied into `lib/` |
| `static` | nothing of the NAS; recipes that cannot link static are refused unless the app builds them as bundle |

- An app is built for every NAS the package runs on (the default: no file may need a glibc above 2.39) or for this
  NAS only (its files may need this NAS's glibc; such an app is never exported as a QPKG). The wizard asks
  ("Runs on"); the API field is `glibc_target`.
- Every build ends with a final check: every file's `GLIBC_` needs against that ceiling (`glibc_above_target`, also
  for a cross build), then the analyzer over the new tree as it will sit at the prefix on this NAS. A finding of
  severity `error` fails the build before anything changes. `bvs tree-deps <prefix> --glibc-max 2.39` runs the same
  glibc check on any tree.
- The Patch tool changes the RUNPATH or the interpreter of chosen files of the newest build, checked the same way.
  A patch lasts until the next build.
- Activation links the prefix to the app's newest build. The root file system is a RAM disk, so the package makes
  the link again at every start. `bvs deps <app>` checks an app's files at any time.
- Export as QPKG wraps the newest build in a QPKG (unsigned: see [qpkg.md](qpkg.md)). It refuses an app for this
  NAS only and a build with a file above the floor.

A project can carry recipes of its own in `recipes/<name>/`, in the library's format: they replace the library's recipe
of the same name in the project's apps, and `bvs recipe lint` checks them. How to write one, the lint, and static and
aarch64 builds: [recipes.md](recipes.md).

## Projects and CI in Bucket & Code Server IDE

**New project** in the panel (command palette, "New project") makes a Forgejo repository, a first commit with a
template and a CI workflow `.forgejo/workflows/build.yml`, and the folder `projects/<name>` the editor opens. The
templates: C, C++ with CMake, Go for x86_64 and aarch64, Rust, Node, Python, Java, a kernel module, the ch341
driver, a QPKG package and an App Builder app. Every project also gets this skill in `.claude/skills/qnap-dev/` and
an `AGENTS.md` that points to it.

A workflow with `runs-on: bucketvs` runs on the NAS's own runner:

- the steps run as root in bash on the NAS, with the package's tools and Node on `PATH`;
- `bvs env run <env> -- <command>` builds the job's checkout inside an environment (the checkout is `/work`);
- `bvs use`, `bvs qbuild`, `bvs repo publish` and `bvs kernel make --build-only` work in a step.

```yaml
name: build
on: [push]
jobs:
  build:
    runs-on: bucketvs
    steps:
      - uses: actions/checkout@v4
      - run: bvs env run ubuntu2404-builder -- make test
```

## Ports

The package's own ports, the firmware's, and how to check and bind one: [networking-ports.md](networking-ports.md).

## Sources

The section on your own computer was measured while building QPKGs with QDK 2.5.0 on Ubuntu 24.04 for QTS 6.1 and
QuTS hero 6.1 units. Bucket & Code Server IDE's user guide (`bvs guide`: "Toolchains and environments", "App
Builder", "New project", "CI runners") and its `bvs` command; what the NAS lacks (`ldd`, `getconf`, `readelf`) from
QNAP Club Poland's [linux-gap.md](https://forum.qnap.net.pl/llms/linux-gap.md).
