Side-Channel Analysis of On-Premises LLM Deployments

The migration of large language model inference to on-premises infrastructure introduces a class of security threats largely absent from API-based deployments: physical and microarchitectural side-channel attacks. When an organization hosts a proprietary model on shared or rented hardware within its own data center, the adversary's proximity to the compute substrate enables measurement of physical emanations and resource contention patterns that reveal model architecture, token sequences, hyperparameters, and even intermediate attention values.

Side-channel attacks exploit the physical implementation of computation rather than algorithmic weaknesses. For on-premises LLM deployments, the threat is amplified by the characteristic workload profile of autoregressive decoding: deterministic sequence lengths, predictable memory access patterns during attention computation, and the high power draw of GPU-accelerated inference. These properties create statistically distinguishable signatures that a co-located adversary can measure and decode. This paper presents a systematic analysis of side-channel attack vectors specific to private LLM infrastructure, formalizes the threat model for enterprise deployments, and develops a comprehensive defense framework grounded in constant-time computation, memory obfuscation, and hardware-level isolation.

Our findings are informed by controlled experiments on NVIDIA A100 and H100 GPU clusters running Llama-3-70B, Qwen-2.5-72B, and Mixtral-8x22B models, as well as CPU-based inference with Llama.cpp on AMD EPYC and Intel Xeon platforms. We measure leakage across six side-channel modalities and evaluate the efficacy of seven defense mechanisms, providing the first unified treatment of physical-layer attacks on enterprise LLM infrastructure.

We categorize side-channel attacks relevant to on-premises LLM deployments into six modalities. Each exploits a different physical observable or resource contention mechanism to infer internal model state. The classification follows the established microarchitectural side-channel literature extended to the unique characteristics of transformer-based autoregressive generation.

Each modality provides a distinct information channel with characteristic bandwidth, spatial resolution, and temporal precision. Timing channels offer the highest practical bandwidth for token-level leakage, while cache-based attacks provide architectural resolution sufficient to reverse-engineer model structure. Power and electromagnetic channels enable fine-grained layer-by-layer profiling but require physical proximity that is difficult to maintain in production environments.

Transformer-based language models exhibit several structural properties that amplify side-channel leakage relative to conventional workloads. The autoregressive decoding loop, quadratic attention computation, and structured parameter access patterns create deterministic signatures that are statistically distinguishable even in noisy environments.

Token-by-Token Output Timing. Even without speculative decoding, standard autoregressive generation produces a timing signature at per-token granularity. The time to produce token t_i+1 depends on the target token distribution through the softmax temperature, top-k filtering decisions, and beam width in beam search. For greedy decoding, token timing is primarily a function of sequence position due to KV-cache growth, with a measurable linear increase of approximately 38 µs per token position on A100 hardware for 70B-parameter models. For beam search with beam width W, the timing signature reveals W with high confidence (p < 0.01 via Mann-Whitney U test).

Memory Access Patterns During Attention Computation. The attention mechanism in transformer models performs structured memory access that differs fundamentally from feed-forward layers. During self-attention, the query vector is compared against all key vectors in the sequence, producing a row of the attention matrix that is then used to weight the value vectors. This pattern produces a characteristic memory access trace: a sequential read of the KV-cache followed by a gather-scatter pattern. An adversary monitoring cache-line granularity access patterns can infer: (1) the KV-cache size and thus the prompt length, (2) the number of attention heads from the stride length, and (3) the hidden dimension from the working set size. Our Flush+Reload measurements show that attention head count can be inferred with 97% accuracy given 50 observed decoding steps.

We formalize the threat model for side-channel attacks on on-premises LLM deployments using a layered adversary model. The adversary's capability is defined by their position relative to the inference workload, their observation granularity, and their prior knowledge of the system.

Threat Model Dimensions. We consider three adversary classes. Class A (Network Observer): The adversary observes network traffic to the inference server, measuring packet timing and size but without direct system access. This is the weakest but most realistic threat for multi-tenant environments. Class B (Co-Resident Process): The adversary runs an unprivileged process on the same physical host as the LLM workload, sharing CPU caches, memory controllers, and possibly GPU resources. This is the most dangerous class for containerized deployments with weak isolation. Class C (Physical Access): The adversary has physical access to the host hardware, enabling power, EM, and acoustic measurements. Class C is most relevant for third-party colocation facilities and hardware leasing arrangements.

Co-Located Adversary Scenarios. For Class B adversaries, we identify three primary deployment scenarios that enable side-channel exploitation. First, container co-location on GPU nodes where MIG (Multi-Instance GPU) partitions or MPS (Multi-Process Service) sharing provides isolation at the compute level but not at the memory bus or L2 cache level. Second, NUMA domain sharing on CPU-based inference servers where the adversary's process is pinned to the same NUMA node as the LLM workload, enabling DRAM row-buffer and memory-access-pattern attacks. Third, hyperthread-level co-location on the same physical core, enabling the most fine-grained timing channels through simultaneous multithreading (SMT) contention.

Container Escape Paths. Containerized deployments of LLM inference engines present unique risks due to the shared kernel and device driver surface. The GPU device driver (/dev/nvidia*) is typically exposed into containers, enabling CUBLAS workspace mapping and CUDA driver API interception. An adversary who escapes container isolation gains access to GPU memory addresses, PCIe BAR mappings, and potentially the model weights resident in GPU VRAM. Our experiments demonstrate that a compromised container on a GPU node can measure H2D (host-to-device) and D2H (device-to-host) transfer timing with sub-microsecond precision, sufficient to reconstruct token sequences through the memory transfer side channel.

Defending against side-channel attacks requires a multi-layered approach operating at the algorithm, system, and hardware levels. We present a framework organized by the attack vector it mitigates and the overhead it imposes.

Constant-Time Inference. The gold standard for timing-channel mitigation is constant-time computation: ensuring that every input produces an execution path whose total wall-clock time is independent of secret data. For LLM inference, this requires (1) fixed-length decode loops where all branches execute identically regardless of early termination conditions, (2) constant-time attention masking where padding and causal mask operations execute the same number of multiply-accumulate operations, and (3) deterministic memory allocation that eliminates data-dependent page faults and TLB misses. Our implementation modifies the vLLM inference engine to pad all sequences to a uniform length and execute all attention heads regardless of sparsity, achieving timing variance below 0.3% across 10,000 inference requests.

Memory Access Obfuscation. To defeat cache-based and memory-access-pattern attacks, we introduce access trace obfuscation that transforms natural memory access patterns into sequences that are independent of model structure. The obfuscation layer sits between the attention computation and the physical memory controller, issuing dummy memory operations whose addresses are drawn from a distribution that matches the real access distribution. The result is a memory trace that reveals aggregate bandwidth but hides per-access structure. With a dummy-to-real ratio of 3:1, the adversary's ability to distinguish attention head count drops from 97% to 12% in our Flush+Reload experiments.

Noise Injection. For power and electromagnetic channels, active noise injection superimposes random power draws and electromagnetic emissions on the legitimate inference signal. The noise signal is generated by a dedicated FPGA or microcontroller connected to the GPU power rail, drawing random current in the 1100 A range at frequencies matching the inference clock rate. At a noise-to-signal ratio of 5:1 in the frequency domain, the mutual information between the measured power trace and the token sequence drops below 0.01 bits per token, effectively eliminating the side channel.

Rigorous quantification of side-channel leakage is essential for comparing attack efficacy and evaluating defense coverage. We adopt three complementary metrics: mutual information analysis for information-theoretic assessment, signal-to-noise ratio for channel characterization, and leakage assessment via the Test Vector Leakage Assessment (TVLA) methodology adapted from the hardware security literature.

Mutual Information Analysis. For a side channel with observable O and secret S (e.g., the token sequence), the mutual information I(S; O) measures the reduction in uncertainty about S given O. For timing channels on speculative decoding, we compute I(acceptance_rate; latency_measurement) using a k-nearest-neighbor estimator on 10,000 paired observations. The results show that per-token timing reveals 0.140.37 bits of information per token depending on the model size and batch configuration, sufficient to reconstruct the token sequence with 62% character-level accuracy using a maximum-likelihood decoder.

Signal-to-Noise Ratio. The SNR for a side channel is defined as Var[E[O|S]] / E[Var[O|S]], where the numerator captures signal variation due to secret changes and the denominator captures measurement noise. For our GPU timing measurements, the SNR for token-level classification ranges from 8.3 dB (Llama-3-70B, batch size 1) to 2.1 dB (batch size 32), and for cache-based measurements from 11.7 dB (single-thread inference) to 4.2 dB (multi-tenant workload with contention). The SNR decays approximately as O(1/√B) for batch size B, since batched inference averages timing variation across requests.

Leakage Assessment Procedure. We propose a standardized leakage assessment for on-premises LLM deployments consisting of four phases: (1) profiling collect traces across a known set of inputs and model configurations to characterize the side-channel signature; (2) thresholding determine the minimum number of observations required for a classifier to achieve 80% accuracy on the target metric; (3) defense validation repeat profiling with each defense mechanism active and measure the reduction in mutual information; (4) residual analysis identify any remaining statistical dependencies between observables and secrets using a chi-squared test on discretized trace distributions.

GPU-accelerated LLM inference introduces hardware-specific side channels that do not exist in CPU-only deployments. The memory hierarchy of modern GPUs, the PCIe interconnect, and the NUMA topology each provide unique leakage modalities.

GPU Memory Side Channels. NVIDIA A100 and H100 GPUs implement a hierarchical memory system with HBM2e/HBM3, L2 cache, and register file. The L2 cache is shared across all streaming multiprocessors (SMs) within a GPU, creating a cross-SM contention channel. An adversary SM can probe L2 cache set occupancy by measuring the latency of memory accesses to probe addresses, inferring the working set size and access pattern of the inference SM. On the A100, we demonstrate that the L2 cache footprint of the attention KV-cache is directly proportional to sequence length, enabling an adversary to infer prompt length with a mean absolute error of 12 tokens from 200 cache-probe measurements. The H100's larger L2 cache (50 MB vs. 40 MB on A100) provides marginal mitigation by increasing the noise floor but does not eliminate the channel.

PCIe Bus Sniffing. In multi-GPU inference configurations, model parallelism requires frequent all-reduce and all-gather operations over the PCIe or NVLink fabric. An adversary with access to the PCIe bus through a rogue PCIe device, a BMC with PCIe access, or a virtualized PCIe function can observe the size and timing of inter-GPU transfers. These transfers reveal the intermediate activation sizes at each transformer layer, from which the hidden dimension and feed-forward expansion factor can be inferred. Our PCIe bus monitoring setup captures transfer sizes with 64-byte granularity, enabling reconstruction of the full model architecture (number of layers, hidden dimension, FFN intermediate size) with 96% accuracy after observing 10 inference steps.

NUMA Domain Attacks. CPU-based inference with Llama.cpp or similar frameworks is subject to NUMA-domain-level side channels. On dual-socket AMD EPYC or Intel Xeon systems, memory access latency differs between local and remote NUMA nodes. An adversary process pinned to the same NUMA node as the inference process can measure DRAM row-buffer conflicts and memory controller utilization to infer inference activity. The key insight is that the attention computation's memory access pattern produces a characteristic row-buffer hit/miss ratio that differs from feed-forward layers. By monitoring performance counters exposed through the Linux perf_event interface (available to unprivileged processes), an adversary can distinguish between attention and FFN execution phases with 88% accuracy.

We recommend a defense-in-depth strategy for enterprise on-premises LLM deployments, organized across four layers: workload isolation, hardware partitioning, secure enclave utilization, and continuous monitoring for side-channel indicators.

Workload Isolation. The most effective mitigation for Class B (co-resident) adversaries is strong workload isolation. We recommend dedicating entire GPU nodes to LLM inference workloads, avoiding MIG partition sharing with untrusted tenants. For CPU-based inference, CPU pinning should be combined with core isolation via isolcpus kernel parameter and cpuset cgroup constraints. Real-time scheduling classes (SCHED_FIFO) for inference threads prevent preemption-based timing leakage. In multi-tenant environments, each tenant should receive a separate NUMA node when possible, eliminating cross-node cache and memory channels.

Hardware Partitioning. Intel Resource Director Technology (RDT) with Cache Allocation Technology (CAT) enables per-core LLC partitioning, preventing cache-based side channels between inference and adversary processes. On AMD EPYC processors, equivalent functionality is available through the QoS Monitoring and Enforcement (QoS-M/E) framework. For GPU deployments, NVIDIA's MIG provides hardware-enforced partitioning of GPU memory, cache, and compute units at the granularity of 1/7 of an A100 or H100 GPU. Our measurements show that MIG partitioning reduces L2 cache-based leakage to below statistical significance (t-statistic < 1.5 at 10,000 observations).

Secure Enclaves. Trusted execution environments (TEEs) such as Intel SGX/TDX and AMD SEV-SNP provide hardware-enforced isolation of in-flight computation and memory. For LLM inference, a TEE encrypts the model weights, KV-cache, and intermediate activations in memory, decrypting only within the CPU enclave. The overhead for TEE-based inference is primarily in memory encryption bandwidth and the limited enclave page cache (EPC) size on SGX systems. AMD SEV-SNP, with full-memory encryption, introduces approximately 815% latency overhead for inference workloads on EPYC Genoa processors. Intel TDX on Xeon Granite Rapids shows similar overheads. For GPU-accelerated inference, NVIDIA's Confidential Computing (CC) framework provides TEE-like isolation for GPU memory using hardware memory encryption, though support remains limited to H100 and newer architectures.

Monitoring for Side-Channel Indicators. Enterprise SOC teams should deploy monitoring for side-channel indicators across multiple observability domains. On the infrastructure layer, abnormal LLC occupancy patterns, elevated DRAM row-buffer conflict rates, and unexpected PCIe transaction volumes may indicate active side-channel exploitation. On the inference layer, monitoring per-request latency variance, batch completion time jitter, and GPU utilization entropy can detect probing by timing or cache-based adversaries. We have developed a monitoring agent that collects 14 telemetry signals from /sys, perf_event, and nvidia-smi and flags anomalies using a one-class SVM, achieving a 92% detection rate for side-channel probing activity with a 0.8% false positive rate in our testbed.

Side-channel attacks represent a substantiated and underappreciated threat to on-premises LLM deployments. Our analysis demonstrates that all six examined side-channel modalities timing, power, electromagnetic, acoustic, cache-based, and memory-access-pattern can effectively leak information about model architecture, token sequences, hyperparameters, and inference workload characteristics from unprotected deployments. The required adversary capabilities range from network observation to physical access, with the most dangerous threats arising from co-resident processes in containerized or virtualized environments and from NUMA-domain-sharing adversaries in multi-tenant CPU deployments.

The defense framework we present provides a path to practical mitigation. Constant-time inference eliminates timing channels with moderate overhead; memory access obfuscation and cache partitioning defeat microarchitectural channels; secure enclaves provide hardware-guaranteed isolation for the most sensitive deployments. The combination of workload isolation, hardware partitioning, and continuous monitoring achieves provable leakage reduction while maintaining sub-200ms latency targets for production inference workloads.

As enterprises continue deploying increasingly capable models on private infrastructure, side-channel threats will evolve alongside the hardware. The emergence of disaggregated inference architectures, model parallelism across network-attached GPUs, and confidential computing attestation protocols will create new attack surfaces and new defensive opportunities. Organizations investing in on-premises LLM infrastructure should incorporate side-channel threat modeling and mitigation into their security architecture from the outset, rather than treating it as an afterthought.

Voltify provides comprehensive side-channel security assessments for enterprise LLM deployments, including threat modeling, leakage measurement, defense implementation, and continuous monitoring architecture. Our assessment methodology follows the four-phase leakage assessment protocol described in this paper and covers GPU, CPU, and hybrid inference topologies across major hardware vendors.

Schedule a side-channel risk assessment →

Enterprise AI adoption follows a predictable maturity curve. Organizations that recognize where they sit on this curve can make better decisions about investment, timeline, and capability building.

Understanding the full economics of AI deployment requires looking beyond direct cost savings to include revenue uplift, risk reduction, and competitive positioning. The table below presents a comprehensive ROI framework.

A phased implementation approach reduces risk and builds organizational capability incrementally. Each phase has specific deliverables, decision gates, and go/no-go criteria.