LUN, Thin Provisioning and Snapshot Recovery
A thin-provisioned LUN’s logical block order lives in the pool’s allocation metadata, not in the raw bytes of the pool. Without that map, a raw scan of the pool usually cannot restore the complete logical order of a volume: fragments, file signatures and partial structures may still be found, but that is not a reconstruction of the LUN. Recovery therefore depends on preserving the maps, the reference metadata and the extents together.
- Without the allocation map, a raw pool scan usually recovers fragments and partial structures, not the LUN’s complete logical block order.
- A pool configuration backup is not a backup of block mappings; in Linux LVM, vgcfgbackup does not back up thin pool metadata.
- Zeros read from a thin LUN do not by themselves show what a block held before or why it is empty.
- A snapshot timestamp is not evidence of application consistency; only the method used to take the snapshot can show that.
- Deleting snapshots or repairing pool metadata on the original to free space can destroy the references recovery needs; preserve first.
A LUN is a logical unit, not a disk
iSCSI and Fibre Channel both distinguish an initiator, a target and a logical unit (LUN) as separate concepts. A host can lose access to a LUN for reasons entirely outside the backing storage, such as target or masking configuration, zoning, multipath or controller failover, or a lost target-to-LUN mapping. “The host no longer sees the disk” is not evidence that the storage pool or the LUN’s data was destroyed. Working out which layer stopped presenting the LUN is covered in SAN Storage Stack Reconstruction; this page is about what happens inside the pool: block maps, references and consistency.
Preserve LUN identity before anything else
- LUN ID and LUN UUID/WWN — the logical unit’s own identity.
- Target IQN and initiator IQNs, or FC WWPNs — which endpoints were talking to which.
- Host groups, LUN masking and mapping — who was allowed to see the LUN.
- Controller owner and multipath identifiers — which controller and path served the LUN.
- Host-side disk signatures, partition GUIDs, and the VMFS UUID or Windows disk ID — how the host itself recognized the volume.
These identifiers are what let a later recovery attempt map application, host disk, LUN, storage object and pool extents back onto each other.
Thick vs. thin changes what “the data” means
A thick-provisioned volume has its full capacity reserved when it is created; a thin-provisioned one takes space from the pool only as data is written. Reserved does not mean simple, though. Depending on the implementation, a thick volume can still sit on RAID, be built from extents that are not physically contiguous, pass through deduplication or compression, or be one more mapped object inside the same pool, so “thick” does not by itself promise sequential physical sectors or the absence of lower mapping layers. Check what the specific array or volume manager actually does.
Thin volumes add a further dependency. Their virtual size can exceed the physical storage behind them: the Linux lvmthin(7) documentation describes a thin LV’s virtual size as potentially much larger than the available physical storage, and NAS and SAN products offer similar over-provisioning. The consequence for recovery is direct: in thin storage, the logical-to-physical allocation map is part of the data. Logical blocks that were never written have no physical backing at all, and written blocks sit wherever the pool happened to place them.
Configuration, maps, extents and references are separate
“Pool metadata” covers several different things, and a backup of one does not cover the others. Losing any of them leaves a different gap.
Scroll horizontally to see the full table →
| Part | What it records | What is missing without it |
|---|---|---|
| Pool and volume configuration | Which pools, volumes, LUNs and snapshots exist, their sizes, names, chunk or block size and placement on the underlying devices | Which objects to look for and their declared geometry; the data can be intact yet unassignable to a named LUN |
| Block allocation metadata | For each thin volume or snapshot, which logical block maps to which physical block in the data area, plus free-space accounting | The logical order of the volume; a raw scan sees blocks but not which LUN or offset they belong to |
| Data extents | The stored blocks themselves | The content; a perfect map pointing at overwritten or unreadable extents recovers nothing for those blocks |
| Snapshot and reference metadata | Which blocks are shared between a volume, its snapshots and clones, reference counts, lineage and, where used, deduplication references | Which generation a block belongs to and whether it is still held by an older state; blocks can be misattributed between generations |
The Linux device-mapper thin target is one example of this split, not a model of every NAS or SAN. Its documentation keeps metadata on a separate device from data (kernel thin-provisioning documentation, Introduction), and in LVM the pool configuration lives in the volume group metadata while the block mappings live in a hidden thin pool metadata LV. That is why an ordinary LVM configuration backup is not a backup of the mappings: lvmthin(7) states that vgcfgbackup does not back up thin pool metadata, and vgcfgrestore(8) warns that when a volume group contains thin pools, changes to thin metadata cannot be reverted and restoring metadata that does not match the pool’s kernel metadata can lose data. For proprietary arrays, which parts exist, where they live and what a configuration export contains has to be confirmed from that vendor’s documentation.
What a logical block can be
At any point in time a logical block of a thin LUN is in one of several states, and each implies something different for recovery. The array may not expose all of them, and the terms vary.
Scroll horizontally to see the full table →
| State | Meaning | Recovery implication |
|---|---|---|
| Allocated | Mapped to a physical block that only this volume uses | Recoverable if the map entry and the extent both survive |
| Never allocated | No physical block was ever assigned | There is nothing to recover; the block was never written in this volume |
| Mapped shared | Points at a physical block also referenced by a snapshot, clone or, with deduplication, another volume | The content may be intact, but assigning it to the right volume and generation needs the reference metadata |
| Discarded or reclaimed | The mapping was removed after a discard; the physical block may return to free space if nothing else references it | Old content may survive until reuse, or may already be gone; without the old map it cannot be placed back reliably |
| Reallocated | A previously freed physical block has been assigned again, possibly to a different volume or LUN | The old content was likely overwritten by the new owner’s data; a block found there may belong to another object entirely |
Reading zeros from a thin LUN does not, on its own, distinguish between these states. A block can return zeros because it was never allocated, because a discard removed its mapping, because the pool zeroes newly provisioned blocks, or because zeros were actually written. Zeros alone say nothing about what the block held before or why its content is absent; only the allocation history, snapshots or other preserved metadata can.
Reclamation: a TRIM analogy, not the same mechanism
SCSI UNMAP and similar discard requests let a thin pool reclaim space the host has released. The effect on recoverability resembles what TRIM, deallocate and garbage collection do on an SSD, but the mechanism is different: here a mapping is removed at the array’s allocation layer, and what follows depends on the array. A discard may be ignored, processed only inside the pool, or also passed down to the underlying devices; in LVM, for example, these are the ignore, nopassdown and passdown modes documented in lvmthin(7). Blocks still referenced by a snapshot are not freed: lvmthin(7) notes that thin snapshots keep pool chunks from being freed.
A discard is therefore not automatically an instant physical loss. Whether old content survives depends on the array’s discard handling, on snapshot references, on how the volume was provisioned and on what was written afterwards. The real risk is reuse: a reclaimed extent can be reallocated to a completely different volume or LUN, and a later write there overwrites data that belonged to another logical object.
Deduplication adds a layer of indirection
Where deduplication is in use, a LUN block, another LUN block and a snapshot can all point through a shared dedup reference at one physical content block. Recovering that content block without the dedup index and reference metadata that explain which logical object it belonged to is not enough — losing the reference metadata can be as damaging as losing the content itself, even when the physical blocks are intact.
A snapshot is a mapping, not automatically a second copy
A storage snapshot commonly preserves an older mapping, copy-on-write or redirect-on-write references, and blocks shared with the live volume — it is not necessarily an independent physical duplicate. In LVM thin pools, for example, the data blocks common to a thin LV and its snapshots are shared (lvmthin(7)). Deleting snapshot metadata can free the historical extents it was holding onto; clone lineage matters; flattening or consolidating snapshots can itself involve heavy writes. Keep array snapshots, LUN snapshots, VM snapshots and file-system snapshots conceptually separate — they are different mechanisms with different failure modes. Hypervisor snapshot chains are covered in Virtual Disk Snapshot Chain Recovery.
Pool trouble or file-system damage inside the LUN?
A problem in the pool’s metadata or capacity and a damaged file system inside one LUN can look alike from the host, but they call for different handling. Signs that point at the pool rather than at one file system include:
- pool data or metadata usage at or near 100%, or a capacity alert from the array;
- writes that hang and then fail, often on several thin LUNs from the same pool at once;
- the pool switching to read-only, refusing to activate, or reporting that its metadata needs checking or repair;
- snapshots failing to be created, or being removed automatically, around the time of the incident.
Signs that point at the file system inside a LUN include a LUN that is presented and readable end to end, a healthy pool, other LUNs in the same pool unaffected, and errors limited to that LUN’s file system or application. The two can be connected: the kernel documentation describes the device-mapper pool erroring I/O when metadata space runs out, and lvmthin(7) warns that metadata exhaustion can lead to inconsistent pool metadata and inconsistent file systems. Check the pool before treating guest file-system errors as the root cause.
When a pool is full or its metadata is suspect, do not delete, consolidate or flatten snapshots, or run metadata repair on the original, simply to free space or get the volume back online, unless there is a plan for preserving what those actions change. Snapshot deletion releases historical extents; repair writes a new version of the maps. Preserved original metadata and matching copies of the pool devices retain an earlier candidate state for analysis or reconstruction on copies. They do not guarantee that a repair performed on the live pool can be reversed: later writes, reclamation and changes to data extents may make that impossible, so old metadata should not be restored over a live pool that has since changed. lvmthin(7), for example, advises keeping the original, unmodified metadata LV if repair fails, for support or more advanced recovery methods.
Training scenario: thin pool full, snapshots offered as the fix
Several LUNs from one thin pool stopped accepting writes at the same time. The array shows the pool at full capacity and suggests deleting old snapshots. One host now reports file-system errors on its LUN.
The common failure across LUNs points at pool capacity, not at independent file-system damage. The file-system errors on one host may be a consequence of failed writes. The old snapshots may be the only remaining copy of blocks the live volumes have since changed.
Stop further writes, record the pool, volume and snapshot state and the array logs, and preserve the pool metadata and, where possible, the underlying devices before anything is deleted or repaired. Then decide on space recovery with that copy in hand.
Whether snapshots hold unique data, and what deleting them would free, depends on the array. If the array was set to remove snapshots automatically when space runs low, part of the history may already be gone.
Crash, file-system and application consistency
A snapshot can be crash-consistent (the state a sudden power loss would leave), file-system-consistent (the file system was quiesced) or application-consistent (applications or VMs flushed their data first). Products differ in how they achieve the stronger levels. In QNAP’s implementation, for example, an ordinary LUN snapshot is crash-consistent, while its Snapshot Agent first tells a Windows server to flush memory, logs and pending I/O to the LUN, or has VMware take a VM snapshot, before the LUN snapshot is taken (QNAP FAQ). A snapshot’s timestamp is not evidence of which level it achieved; only the record of how it was taken is. That difference can determine whether a database or VM recovered from it is directly usable or needs further repair.
What the recovery result should be
The useful output is not a heap of recovered blocks but a defined logical object, worked on copies:
- A logical LUN image or export for one chosen state: the live volume or a named snapshot, built from one identified generation of the allocation maps.
- An explained point in time — which snapshot or metadata generation it represents and what consistency level that state can claim.
- Known logical states — ranges that the confirmed mapping state shows as never allocated or unmapped by a discard; these normally read back as zeros, which is expected and not a recovery defect.
- Actual gaps — unreadable, missing or uncertain ranges from acquisition or reconstruction, listed separately, with any placeholder or zero-fill and its reason recorded rather than silently mixed in with real zeros.
- Geometry — logical size, block size and sector size, so the host-side partition table and file system line up.
- Verification on the copy — the file system and, where it matters, the application or database checked on the exported image, not on the original LUN (see How to Validate Recovered Data).
Do not combine data extents from one generation with maps from another without checking that they match: a map from before a reclaim or reallocation can point at blocks that now hold another volume’s data, and the result can look plausible while being wrong. Choosing between competing metadata generations is discussed in Storage Metadata Generations and Consistency.
Preserve the highest intact layer that still knows where the logical blocks belong, together with the extents it points to. An allocation map or a dedup index that survives is often worth more than another raw image of pool sectors.
Technical references: Linux kernel: device-mapper thin provisioning — metadata on a separate device from data, shared data between volumes and recursive snapshots, pool erroring I/O on metadata exhaustion, discard and zeroing options · lvmthin(7) — virtual size larger than physical storage, data blocks shared between thin LVs and snapshots, vgcfgbackup not backing up thin pool metadata, discard modes, metadata exhaustion and repair, keeping the original metadata LV · vgcfgrestore(8) — warning that thin metadata changes cannot be reverted and mismatched restores can lose data · QNAP: crash-consistent vs. application-consistent LUN snapshots — product-specific example of how application consistency is coordinated. The kernel and LVM sources describe Linux device-mapper and LVM only, and the QNAP page describes QNAP products only; other NAS and SAN arrays, including proprietary snapshot and deduplication schemes, need their own vendor documentation.
Related: SAN Storage Stack Reconstruction · Sparse, Thin-Provisioned and Virtual Storage Recovery · Reconstructing NAS Storage Stacks · Virtual Disk Snapshot Chain Recovery · TRIM, Deallocate and Garbage Collection · Storage Metadata Generations and Consistency · SAN (Storage Area Network) Failure · NAS Device Failure