I'm trying to use xmlstarlet sel to list the disk partitions I need to create from an xml file that list them in ascending block position on disk (example : https://github.com/finley/SystemImager/blob/initrd-from-imageserver-and-dont-package-initrd/doc/examples/disk-layout-complex.xml) This file is generated by dumping an installed system that has to be replicated. User can then replace size with "*" for partition he wants to be adapted to new disk.
Right now I'm doing the following:
local IFS=;
DISK_DEV=sda
DISKS_LAYOUT_FILE=/tmp/disk-layout-complex.xml
cd /tmp
wget https://raw.githubusercontent.com/finley/SystemImager/initrd-from-imageserver-and-dont-package-initrd/doc/examples/disk-layout-complex.xml
xmlstarlet sel -t -m "config/disk[@dev=\"${DISK_DEV}\"]/part" -v "concat(@num,';',@size,';',@p_type,';',@id,';',@p_name,';',@flags,';',@lvm_group,';',@raid_dev)" -n ${DISKS_LAYOUT_FILE} | sed '/^\s*$/d' |\
while read P_NUM P_SIZE P_TYPE P_ID P_NAME P_FLAGS P_LVM_GROUP P_RAID_DEV
do
# process partitions creation.
echo "Creating partition $P_NUM of size $P_SIZE tpye $P_TYPE"
done
The above xmlstarlet will produce the following output that is then processed by the while read loop:
1;500;primary;;;boot;;
3;4096;primary;;;;;;
4;*;extended;;;;;;
7;4096;logical;;;;;;
5;*;logical;;;;;;
6;2048;logical;;;;;;
2;1024;primary;;;swap;;
After line 3 (partition #4) is processed, there is no space left on disk, The loop will process line 4 (partition #7) and will fail with no space left on disk.
The problem is for variable size partition (use 100% ("*" in the file)). If one is listed before other remaining ones (part 4 in the above case), then, it is created with full remaining space, leaving no space on disk to process the last ones. Thus, for example, it is not possible to put a primary swap partition at the end of a disk with a / partition that has a variable size.
Q: Is there a clever way to use xmlstarlet sel to list partitions in the following order:
list all primary and extended partition in same order as written in the xml file until a partition with size "*" is seen;
For all partition, add a field stating if it was listed in order or reverse order so I can know if I have to create partition relative to beginning of free space or relative to end of free space. (variable partition would be tagged as normal order as they would be created starting at beginning of free space)
For the listed example (disk-layout-complex.xml), this would list partitions to create in the following order: (below, the output of xmlstarlet that would then be processed by a while read loop similar to the above code but with one more preceding read argument OFFSET_CREATE that would read the normal/reverse value)
normal;1;500;primary;;;boot;;
normal;3;4096;primary;;;;;;
reverse;2;1024;primary;;;swap;;
normal;4;*;extended;;;;;;
normal;7;4096;logical;;;;;;
reverse;6;2048;logical;;;;;;
normal;5;*;logical;;;;;;
Processing the above xmlstarlet output would never trigger a situation were there are some partition to create while disk has no space left because a partition was created with 100% remainiong space.
I'm processing this within a specially crafted initrd, so I only have access to most common utils like sed/grep/bash2/xmlstarlet/awk. no perl, no python, no language that needs libraries in general.
I'm pretty convinced that there is a solution that does most of the job if not all, but I'm far not enough skilled to even evaluate if it can be done this way. I think I can achieve that in pure bash, but this will be far less elegant.
The final answer is the following.
<!— Sample cut, relevant for this question —>
<config>
<disk dev="/dev/sda" label_type="msdos" unit_of_measurement="MiB">
<part num="1" size="500" p_type="primary" flags="boot" />
<part num="3" size="4096" p_type="primary" />
<part num="4" size="*" p_type="extended" />
<part num="7" size="4096" p_type="logical" />
<part num="5" size="*" p_type="logical" />
<part num="6" size="2048" p_type="logical" />
<part num="2" size="1024" p_type="primary" flags="swap" />
</disk>
</config>
We want the following output in order to list partition in an order such as we can create partitions in order of apearance on the disk. The partition with a size of '*' will occupy all remaining space. So it must be created in the end.
Thus we need to create the following partition in sequence. (beginning and end are reference telling if we need to create the partition relatinve to beginning of available space or relative to the end of available space)
/dev/sda;end;2;1024;MiB;primary;;;swap;;
/dev/sda;beginning;1;500;MiB;primary;;;boot;;
/dev/sda;beginning;3;4096;MiB;primary;;;;;
/dev/sda;beginning;4;*;MiB;extended;;;;;
/dev/sda;end;6;2048;MiB;logical;;;;;
/dev/sda;beginning;7;4096;MiB;logical;;;;;
/dev/sda;beginning;5;*;MiB;logical;;;;;
It is obtained by running the sxl tranformation file with:
xmlstarlet tr do_part.xsl ./disk-layout-complex.xml
And now the code:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Call me with:
xmlstarlet tr do_part.xsl disk-layout.xml
Output: List of partitions to create in order.
Each line list the following values separated by semicolons:
- disk device
- creation reference
- partition number
- partition size
- partition size unit
- partition type
- partition id
- partition name
- partition flags
- lvm group it belongs to
- raid device it belongs to
Author: Olivier LAHAYE (c) 2019
Licence: GPLv2
-->
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:exslt="http://exslt.org/common" version="1.0" extension-element-prefixes="exslt">
<xsl:output method="text" omit-xml-declaration="yes" indent="no"/>
<xsl:strip-space elements="*"/>
<xsl:template match="/config/disk"> <!-- We are loking for disk informations only -->
<!-- For each disk block -->
<xsl:call-template name="PrintPartition"> <!-- Compute primary or extended partitions to create -->
<xsl:with-param name="index"><xsl:value-of select="count(part)"/></xsl:with-param>
<xsl:with-param name="reference">end</xsl:with-param>
<xsl:with-param name="type">primary|extended</xsl:with-param>
</xsl:call-template>
<xsl:call-template name="PrintPartition"> <!-- then, compute logical partitions to create -->
<xsl:with-param name="index"><xsl:value-of select="count(part)"/></xsl:with-param>
<xsl:with-param name="reference">end</xsl:with-param>
<xsl:with-param name="type">logical</xsl:with-param>
</xsl:call-template>
</xsl:template> <!-- We're done -->
<!-- Main recursive template that will dump partitions to create for the matched disk -->
<xsl:template name="PrintPartition">
<xsl:param name="index"/> <!-- partition node number within disk item-->
<xsl:param name="reference"/> <!-- beginning or end: should we create partition relative from beginning or from end of free space -->
<xsl:param name="type"/> <!-- type of partitions -->
<xsl:choose>
<xsl:when test="$index=1">
<xsl:if test="contains($type,part[position()=$index]/@p_type)">
<xsl:value-of select="concat(@dev,';',$reference,';',part[position()=$index]/@num,';',part[position()=$index]/@size,';',@unit_of_measurement,';',part[position()=$index]/@p_type,';',part[position()=$index]/@id,';',part[position()=$index]/@p_name,';',part[position()=$index]/@flags,';',part[position()=$index]/@lvm_group,';',part[position()=$index]/@raid_dev,' ')"/> <!-- write partition information -->
</xsl:if>
</xsl:when>
<xsl:when test="contains($type,part[position()=$index]/@p_type) and part[position()=$index]/@size!='*'">
<xsl:if test="$reference='end'">
<xsl:value-of select="concat(@dev,';',$reference,';',part[position()=$index]/@num,';',part[position()=$index]/@size,';',@unit_of_measurement,';',part[position()=$index]/@p_type,';',part[position()=$index]/@id,';',part[position()=$index]/@p_name,';',part[position()=$index]/@flags,';',part[position()=$index]/@lvm_group,';',part[position()=$index]/@raid_dev,' ')"/> <!-- write partition information -->
</xsl:if>
<xsl:call-template name="PrintPartition">
<xsl:with-param name="index"><xsl:value-of select="number($index)-1"/></xsl:with-param>
<xsl:with-param name="reference"><xsl:value-of select="$reference"/></xsl:with-param>
<xsl:with-param name="type"><xsl:value-of select="$type"/></xsl:with-param>
</xsl:call-template>
<xsl:if test="$reference='beginning'">
<xsl:value-of select="concat(@dev,';',$reference,';',part[position()=$index]/@num,';',part[position()=$index]/@size,';',@unit_of_measurement,';',part[position()=$index]/@p_type,';',part[position()=$index]/@id,';',part[position()=$index]/@p_name,';',part[position()=$index]/@flags,';',part[position()=$index]/@lvm_group,';',part[position()=$index]/@raid_dev,' ')"/> <!-- write partition information -->
</xsl:if>
</xsl:when>
<xsl:when test="contains($type,part[position()=$index]/@p_type) and part[position()=$index]/@size='*'">
<xsl:call-template name="PrintPartition">
<xsl:with-param name="index"><xsl:value-of select="number($index)-1"/></xsl:with-param>
<xsl:with-param name="reference">beginning</xsl:with-param>
<xsl:with-param name="type"><xsl:value-of select="$type"/></xsl:with-param>
</xsl:call-template>
<xsl:value-of select="concat(@dev,';','beginning;',part[position()=$index]/@num,';',part[position()=$index]/@size,';',@unit_of_measurement,';',part[position()=$index]/@p_type,';',part[position()=$index]/@id,';',part[position()=$index]/@p_name,';',part[position()=$index]/@flags,';',part[position()=$index]/@lvm_group,';',part[position()=$index]/@raid_dev,' ')"/> <!-- write partition information -->
</xsl:when>
<xsl:otherwise>
<xsl:call-template name="PrintPartition">
<xsl:with-param name="index"><xsl:value-of select="number($index)-1"/></xsl:with-param>
<xsl:with-param name="reference"><xsl:value-of select="$reference"/></xsl:with-param>
<xsl:with-param name="type"><xsl:value-of select="$type"/></xsl:with-param>
</xsl:call-template>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
</xsl:stylesheet>
And voilà: it works perfectly.
There are certainly better or more elegant solutions. (feel free to comment)
(code available here (with few changes): https://github.com/finley/SystemImager/blob/initrd-from-imageserver-and-dont-package-initrd/lib/dracut/modules.d/51systemimager/do_partitions.xsl)