Hardware realities, resource limits and deep performance monitoring

When a QNAP NAS feels slow, the investigation starts with what its hardware can do. Then it looks at what the NAS spends its resources on, and only then at the deeper queues: processes stuck on I/O and virtual machines fighting over the processor.

1. Expectationswhat this hardware is for
2. Baselinestop the extras, watch RAM and SWAP
3. D-statesprocesses stuck on I/O
4. vCPUsthreads, oversubscription, steal
5. GraphsBlackBox and Pulse, read right

01Hardware expectations and reality

QNAP servers are built with parts that match their price. QNAP still ships firmware updates for units more than ten years old, but that does not mean an old or entry-level NAS can run today's whole range of apps without compromise. Software grows heavier as it develops; the hardware stays what it was.

A budget NAS is designed for smooth file serving. Expecting it to run virtual machines or heavy databases without trouble misreads what it is. Check what your processor and memory can really do, and own the choice of hardware. The NAS's core job is working with files; everything else is an add-on that runs within the resources left over.

02Branch 1: the baseline, by isolating services

When the NAS slows down, above all when its memory runs out and it starts to use SWAP (which slows every operation down sharply), the first diagnostic step is to isolate the services.

1. Sign in to QTS or QuTS hero.
2. Open App Center and stop the non-essential apps and services one by one. The Workbench does not stop apps for you.
3. Watch the system in QTS's Resource Monitor or in the Workbench's Pulse (the Utilities desk, Start menu → Monitoring → Pulse; administrators only). Its cockpit refreshes every two seconds: CPU busy with one bar per logical CPU, RAM as used against total with the page cache in a band of its own, a SWAP bar when the NAS has swap, the 1-minute load and the run queue. Watch the CPU load and the SWAP in use come down.
4. Find the baseline: keep stopping services until the NAS's core function, smooth file serving, is fully back.

That is your hardware's baseline. From there, start the other apps one at a time, strictly within the resources the hardware has left.

03Branch 2: D-states and hidden bottlenecks

Sometimes the problem is not an overloaded CPU or a lack of RAM. The system can stall with the processor far from full when processes sit in uninterruptible sleep, the D state. In the Workbench's own words, a process in state D is stuck waiting on kernel I/O (disk, a network file system, a hung driver) and cannot even be killed until that I/O completes. Many processes in D means they are queuing, usually behind slow disk I/O, and the device freezes.

This queuing hurts most when the NAS hosts virtual machines (Virtualization Station, KVM). A poor allocation of resources there shows up as dropped network packets and long I/O delays.

• Pulse raises Processes in D-state (I/O) as soon as one process is in D, and marks it critical above two. On its Processes tab, D + Z shows only the stuck (D) and zombie (Z) processes and their parents.
• BlackBox records the NAS every 20 seconds from the moment the package runs; there is nothing to switch on. It keeps CPU busy, iowait and per-core clock, the load, the run queue and Blocked (D); memory used and available, page cache, swap and OOM kills; per interface the traffic, drops and TCP retransmits; per disk the await, queue depth and utilisation; the temperatures and fans; and per VM its CPU, vCPUs, memory and disk I/O. It keeps every 20-second sample for a day, then summaries by the minute for a week, by five minutes for a month and by the hour for a year.
• Two different D counts. BlackBox's Blocked (D) is the kernel's count of blocked tasks; Pulse counts the processes it sees in state D. They need not match.
• iowait, not PSI. QNAP's kernels are built without pressure stall information, so the Workbench measures I/O delay as iowait.

04CPU allocation: pCPU, lCPU and vCPU

Before you give a VM its resources, know the topology:

• pCPU (physical CPU): the physical cores of the NAS processor.
• lCPU (logical CPU): the cores the operating system sees, Hyper-Threading included. Hyper-Threading does not double performance; as a rule of thumb it adds about 30% throughput.
• vCPU (virtual CPU): the virtual cores given to the VMs.

Pulse counts logical CPUs: where it says 12 cores on a six-core, twelve-thread Xeon W-1250, those are threads.

The oversubscription trap. Giving 128 vCPUs to VMs on a 32-core processor assumes they never all get busy at once. When they do, the hypervisor cannot schedule the vCPUs fast enough, and the VMs stall waiting for CPU time, which inside a guest shows as steal time. Pulse reports it from the host side:

• vCPU over-subscription (information) when the running VMs have more vCPUs than the NAS has logical CPUs;
• VMs contending for CPU (warning) when, on top of that, the host is over 75% busy or its run queue is longer than its thread count. Its note on this NAS: Over-subscribed (17 vCPUs / 12 cores) AND the host is busy (84%, queue 11) — your VMs are likely contending for physical cores NOW. This is CPU theft, host-side.
• Steal time is measured inside the guest. On the NAS it reads zero by design; the Workbench's own note says you cannot see a guest's steal from the host. Judge the contention on the NAS by the vCPUs against the threads, the busy percentage and the run queue, and read steal inside the VM (the st column of top or vmstat in a Linux guest).

05Watching the virtual machines

The Workbench has no agent to install in a VM. It measures the NAS it runs on, and that includes the virtual machines the NAS hosts:

1. Open Pulse, the VM tab: every running VM with its vCPUs, CPU%, memory, RSS, state and process ID, and the title line Hosting 4 VM(s), 17 vCPUs.
2. Open BlackBox (Start menu → Monitoring), switch to Heat map and turn on the Virtual machines lane: each VM's CPU over the day, beside the temperatures and the disks' await. Its charts also keep each VM's vCPUs, memory and disk I/O, and the start and stop of every VM.
3. For a VM's network reach, add its address in BlackBox under Settings → Monitored hosts: BlackBox then records its round trip and packet loss.
4. Inside the VM, watch its steal time with the guest's own tools.

The Virtual Machines · LOCAL window (Local NAS → Virtual Machines) shows each VM's configuration from Virtualization Station: its vCPUs, topology, CPU set, CPU model, disks and network cards. Opening it runs QNAP's own diagnostic dumps into the Workbench's cache; it changes no setting.

06Reading the graphs: the critical states

• High D-state and I/O delay. Processes are waiting on the disks. BlackBox's weekly report names it Processes waiting on storage (iowait at the 95th percentile of 20% or more is a warning, 40% or more critical) and names the busiest disk; Slow array member flags a disk whose await is twice its siblings' and at least 50 ms (critical at 200 ms), and Disk with no headroom one that is 90% busy or more for two hours. The remedy is the author's advice: move the VM to an SSD volume, check the disks' health (the performance guide), and check that no other process saturates the RAID.
• High CPU queueing. The NAS's processor is oversubscribed: Pulse says VMs contending for CPU, and the weekly report's Virtual machine hogging the host flags a VM that uses 150% of one core or more (300% critical). Give the VMs fewer vCPUs: start with one or two per VM and add more only when the guest needs them.
• Thermal throttling. A high CPU temperature, then a lower clock and longer delays, means the processor is overheating. The weekly report calls it CPU thermal throttling: a core running at 60% or less of its own usual clock while the CPU is at least 60% busy. On the heat map, 65 °C is amber and 80 °C red. The Workbench's own advice: clean the dust from the heatsink, the fans and the intakes, check that the fans spin, and improve the airflow or lower the room temperature; a higher fan speed in QTS helps too.

07KVM good practice: the CPU model

• Host / passthrough (recommended): passes the NAS processor itself to the VM, with every instruction set it has (AVX, for example). It gives the best performance and the least overhead on a single NAS.
• An emulated model (a generic Intel or AMD CPU): hides the host's processor. It is useful in a cluster, to move running VMs between physically different hosts; on a single NAS it only costs performance.

The Virtual Machines · LOCAL window shows the CPU model each VM is given (for example Westmere) and its topology; the Workbench does not judge the choice.

08Elementary knowledge: resource overcommit

How not to drown VM performance at the limit of what a NAS can do. The ratio of vCPUs to physical cores (pCPU) is the first thing worth counting:

General rule. Overcommit is safe under a light, uneven load with active monitoring, and dangerous at peaks with no headroom. Keep a reserve.