# Writing App Builder recipes in a project

A project can carry its own App Builder recipes, in the same format as Bucket & Code Server IDE's recipe library. Write
one when the recipe library has nothing for a library or a program the project needs, or when the project needs
another version of one of its recipes.

## Where a recipe lives

- A recipe is a folder `recipes/<name>/` in the project's folder. It holds `recipes/<name>/recipe.json`, `build.sh`
  and, when the recipe has one, the vendor file its `recipe.json` names. Any other file, or a subfolder, leaves the
  recipe out: a patch or a note goes into `build.sh` (below).
- The folders are flat and hold regular files only. bvsd reads them without following a link, and leaves out a recipe
  folder that is a link, holds a link, or holds more than 8 entries.
- The limits: `recipe.json` up to 256 KiB, `build.sh` up to 1 MiB, a vendor file up to 4 MiB; a file over its limit
  leaves its recipe out. Recipes past the 256th are left out together. Past 1024 entries in `recipes/`, or 64 MiB
  read for the whole project, bvsd reads no recipe of it. Each time it says why.
- The name follows the library's rule, `^[a-z0-9][a-z0-9+._-]*$`, and is the folder's name. `lib` is not a recipe
  name: it is the builder's own folder in the library.
- The recipes serve the apps that name the project. In those apps, a project recipe replaces the library's recipe of
  the same name. A project recipe that is left out (a broken `recipe.json`, for example) also holds back the library's
  recipe of its name, so an app never builds the library's version by mistake.

## recipe.json

Exactly these keys (and optionally `env_packages_for`, `license_files`, `vendor` and `watch`); the lint refuses any
other key or a wrong type:

| Key | Type | Meaning |
|---|---|---|
| `name` | string | the folder's name |
| `version` | string | the upstream version of the source; in a project's recipe `latest`, or the key left out, takes the newest release its `watch` lists at each build |
| `summary` | string | one line for the package picker |
| `category` | string | `library`, `shell`, `core`, `tool`, `network`, `editor`, `server` or `language` |
| `license` | string | an SPDX expression where one exists |
| `homepage` | string | the upstream home page |
| `source` | object | `{url}`: the https address of the source archive; `{version}`, `{major}`, `{minor}`, `{patch}` and `{nodots}` in it stand for the version. A library recipe also pins `sha256` |
| `deps` | string[] | recipes built into the same prefix first, from the project or the library |
| `env_packages` | string[] | Debian packages the build needs beyond the builder's own: build tools only, never a library's `-dev` package (`libc6-dev` and `linux-libc-dev` apart) |
| `env_packages_for` | object, optional | `{"<release>": [packages]}`: the list that replaces `env_packages` in one distribution release (`debian-12`, `ubuntu-24.04`), for a package that release names otherwise |
| `static` | boolean | whether the recipe supports the static link mode; a static app refuses a recipe that says `false` unless it builds such recipes as a bundle |
| `nss` | boolean | whether its code looks up users, groups, hosts or services through glibc |
| `check` | string | a command run in `$PREFIX` after the install that proves the result works |
| `vendor` | string, optional | the vendor file: further sources (below) |
| `watch` | object, optional | where the newest release is listed: `{"kind": "github-release", "repo": "owner/name"}` (or `github-tag`), `{"kind": "pypi", "package": …}`, `{"kind": "npm", "package": …}`, or `{"kind": "index", "url": …, "regex": …}`; a project recipe whose version is `latest` needs it, and the library's automatic updates read it |
| `license_files` | array, optional | files under the source archive's root that hold the licence when no `COPYING`, `LICENSE`, `COPYRIGHT` or `NOTICE` file does (lua keeps it in `doc/readme.html`); Export as QPKG copies them into the package's `licenses/` |

A project's recipe needs no sha256. The first build of a version downloads the archive, computes its sha256 and
writes it into `recipes/<name>/recipe.lock`, one line per version and address; a later build of that version from the
same address takes only the same file, and stops when the file there changed. Commit `recipe.lock` with the recipe.
When the program's authors released a file again under the same version, delete its line, and the next build records
the new file. The project wizard's starter recipe names a placeholder address on example.org, so its first Build fails
until the address is real.

## build.sh

`build.sh` defines `do_configure`, `do_build` and `do_install`, and runs nothing when it is loaded.

- Each phase runs in its own subshell with `set -eo pipefail`, starting in the unpacked source folder `$S`. A
  variable set in one phase is gone in the next; put shared values at the top of the file.
- The builder sets `$PREFIX` (the app's prefix), `$HOST` (the cross target, empty for a native build), `$JOBS`,
  `$STATIC` (1 in a static build), `$LDFLAGS`, `$PKG_CONFIG` (which sees only the prefix's `.pc` files) and `$V` (the
  version).
- Helpers: `configure_std [ARGS]` for an autoconf `./configure`, `cmake_std [ARGS]` for CMake (Ninja, the prefix,
  `lib`, Release and the cross settings), `meson_std [ARGS]` for Meson, `make_j [ARGS]` for `make -j$JOBS`.
- A patch goes into `build.sh` as a here-document (`patch -p1 <<'PATCH'`), so the recipe's key covers it.
- `CONFFILES=(etc/...)` on one line at the top names the configuration files a rebuild keeps.
- A phase must not leave a process running; the next build of the app would wait for it.

## Static programs and aarch64

- A static recipe (`"static": true`) may depend only on static recipes.
- A tool that runs during the build comes from the environment, never from a recipe: a recipe is built for the target,
  which in a cross build may not run on the machine that builds. Every build installs the builder's tools in its
  environment when they are missing (build-essential, autoconf, automake, libtool, pkg-config, cmake, meson,
  ninja-build, bison, flex, gettext, perl, python3 and others); `env_packages` names a tool beyond them, such as a
  code generator.
- An aarch64 app built on an x86_64 machine is a cross build (`$HOST` set). Before the checks, the builder compiles a
  small program for `$HOST` and tries to run it. That works where qemu-user is registered in binfmt_misc, with the
  cross libc of `crossbuild-essential-arm64`, which an aarch64 app installs at its first build; the checks then run
  under emulation. Where it does not, each check is recorded as
  `not run: this machine cannot execute <HOST> programs`, and the build claims nothing about it.
- CMake in a static app: when CMake finds a dependency as a file (FindCurses gives `libncurses.a`), it ends the
  link line with `-Wl,-Bdynamic`, and the link then asks for a shared libc ("attempted static link of dynamic
  object libc.so.6"). Configure with
  `if [ "$STATIC" = 1 ]; then cmake_std -DCMAKE_LINK_SEARCH_END_STATIC=ON; else cmake_std; fi`, as the Package
  from source starter does (D312).
- A recipe that cannot cross-build says so in `do_configure` and stops there, for example
  `[ -z "$HOST" ] || { echo "this recipe cannot cross-build" >&2; exit 1; }`.

## Sources from a package registry

A program that installs packages from npm, crates.io or Go's proxy during its build gets them from a vendor list
instead: nothing is downloaded during the build.

- `"vendor": "vendor.txt"` names a file of `<sha256> <https url> <dest>` lines. The download step fetches and checks
  every line before the build.
- A build adds the line `VENDOR_SHA256=<the vendor file's sha256>` to its copy of a project recipe's `build.sh`, so a
  changed vendor file rebuilds the recipe; a library recipe has that line in `build.sh` itself. `build.sh` then checks
  each file again and unpacks it.
- For npm, make the list on the NAS with the package's tool and the Python of the Toolchains page (install
  `python-3.13` there first):

  ```bash
  eval "$(bvs use python-3.13-$(uname -m))"
  python3 /usr/bin/bucketvs/share/bucketvs/tools/npm-vendor.py package-lock.json "$BVS_DATA/tmp/npm-vendor" \
    --dev --roots esbuild --platform linux-x64-glibc --platform linux-arm64-glibc > vendor.txt
  ```

  `--roots` keeps only what the named packages need. Build tools are usually dev entries, so they need `--dev`.
  `go-vendor.py` and `cargo-vendor.py` sit beside it.

## Check and build

- `bvs recipe lint` in the project's folder checks the recipes as bvsd reads them. Its exit status is 1 when it finds
  an error. The editor runs the same lint when a file under `recipes/<name>/` is saved, and shows the findings in the
  Problems view.
- `bvs deps <app>` lists the libraries each file of the app's newest build needs.
- In a project whose Makefile is an App Builder Makefile (Package from source makes one), `make app build` in the
  editor's terminal, or Build in the Project view, makes the project's apps and builds them, and `make qpkg` packs the
  package from the apps' newest successful builds.

## A worked example: libnatpmp

`recipes/libnatpmp/recipe.json` of the Transmission example, which pins its sha256 as every example Bucket & Code
ships does; a recipe of your own leaves `sha256` out:

```json
{
  "name": "libnatpmp",
  "version": "20230423",
  "summary": "libnatpmp: the NAT-PMP client library Transmission maps its peer port with",
  "category": "library",
  "license": "BSD-3-Clause",
  "homepage": "https://miniupnp.tuxfamily.org/libnatpmp.html",
  "source": {
    "url": "https://miniupnp.tuxfamily.org/files/libnatpmp-20230423.tar.gz",
    "sha256": "0684ed2c8406437e7519a1bd20ea83780db871b3a3a5d752311ba3e889dbfc70"
  },
  "deps": [],
  "env_packages": [],
  "static": true,
  "nss": false,
  "check": "test -s lib/libnatpmp.a && ! ls lib/libnatpmp.so* > /dev/null 2>&1 && test -s include/natpmp_declspec.h && CHECK_CPPFLAGS=-DENABLE_STRNATPMPERR STATIC=1 check_lib libnatpmp 'natpmp.h arpa/inet.h' 'natpmp_t n; int r = initnatpmp(&n, 1, inet_addr(\"127.0.0.1\")); if (r) { printf(\"initnatpmp: %d (%s)\\n\", r, strnatpmperr(r)); return 1; } r = sendpublicaddressrequest(&n); closenatpmp(&n); if (r != 2) { printf(\"sendpublicaddressrequest: %d (%s)\\n\", r, strnatpmperr(r)); return 1; } printf(\"libnatpmp 20230423: a %d-byte public address request sent to 127.0.0.1:%d, linked from the static archive\\n\", r, NATPMP_PORT)'"
}
```

and its `build.sh` (its header left out):

```bash
# libnatpmp: a static library only, in every link mode; the apps link it into their programs. Its CMakeLists.txt
# builds libnatpmp.a, the natpmpc client and a test program, and has no install rule, so do_build makes the library
# alone and do_install copies it and its two headers itself, and writes lib/pkgconfig/libnatpmp.pc, the module
# Transmission's FindNATPMP asks pkg-config for (upstream ships none). The check compiles and runs a program against
# the archive (check_lib with STATIC=1: there is no shared library to look for), with ENABLE_STRNATPMPERR, which
# natpmp.h needs to declare strnatpmperr.
do_configure() {
  cmake_std -DCMAKE_POSITION_INDEPENDENT_CODE=ON
}
do_build() {
  cmake --build _build --target natpmp -j"$JOBS"
}
do_install() {
  install -d "$PREFIX/lib" "$PREFIX/include"
  install -m 644 _build/libnatpmp.a "$PREFIX/lib/"
  install -m 644 natpmp.h natpmp_declspec.h "$PREFIX/include/"
  install -d "$PREFIX/lib/pkgconfig"
  printf '%s\n' "prefix=$PREFIX" 'libdir=${prefix}/lib' 'includedir=${prefix}/include' '' 'Name: libnatpmp' \
    'Description: the NAT-PMP client library' "Version: $V" 'Libs: -L${libdir} -lnatpmp' 'Cflags: -I${includedir}' \
    >"$PREFIX/lib/pkgconfig/libnatpmp.pc"
}
```

## Sources

Bucket & Code Server IDE's recipe library and the notes on its format in the package's sources (the keys, the builder's
variables and helpers, checks, cross builds, vendor lists), its user guide (`bvs guide`: "Your own recipes", "App
Builder") and its `bvs recipe lint` command; the Transmission example's recipes.
