Scaling an SVG without affecting paths

I'm looking for some pointers on scaling SVG containers without affecting paths inside. I'm working with Greensock.js and would like the SVG containers to scale with the width of the letters themselves (without changing the aspect ratio of the paths inside). The end goal is that the surrounding letters shuffle with the scaling letter. See attached gif for expected result.

Scaling Letters Codepen Final_Result

See the Codepen, but here is how I've set up my HTML:

<div class="letters">
    <svg class="lcontainer" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 324 406">
        <rect class="bottom" x="46" y="353" width="278" height="53" />
        <rect width="53" height="406" />
    </svg>
    <svg class="econtainer" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 324 406">
        <rect class="outer" x="46" width="278" height="53" />
        <rect class="inner" x="46" y="247" width="240" height="53" />
        <rect class="outer" x="46" y="353" width="278" height="53" />
        <rect width="54" height="406" />
    </svg>
</div>

Solution

Thanks for the responses, as enxaneta suggested I ended up putting all paths in one SVG and working with them in there.

My final answer is having them in groups which let me use EventListeners on both individual parts of the letters and their containers, while keeping them in the same viewBox.

Here is my Codepen for the solution: https://codepen.io/leecranny/full/MLdBra

<div class="leah">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 982 408" height="400">
    <g class="hbox">
        <rect  x="797.125" width="53.55" height="408" />
        <rect id="h_middle" x="806.05" y="104.55" width="158.1" height="53.55" />
        <rect id="h_side" x="928.45" width="53.55" height="408" />
    </g>
    <g class="abox">
        <path id="a_short" d="M750.75169,405.5H698.34861V106.05385c0-29.082-18.01843-53.65077-39.34861-53.65077s-39.34861,24.56881-39.34861,53.65077V405.5H567.24831V106.05385C567.24831,47.57558,608.40754,0,659,0s91.75169,47.57558,91.75169,106.05385Z" transform="translate(0 1)"/>
        <path id="a_wide" d="M821.75169,405.5H769.34861V146.05385C769.34861,87,716,52.40308,659,52.40308S549.65139,91,549.65139,146.05385V405.5H497.24831V136.05385C497.24831,61,568,0,659,0S821.75169,55,821.75169,136.05385Z" transform="translate(0 1)"/>
        <rect id="a_middle" x="590.37692" y="299.5908" width="137.24615" height="52.40308"/>
    </g>
    <g class="ebox">
        <rect id="e_inner" x="243.03125" y="248" width="240" height="53" />
        <rect id="e_outer" x="243.03125" y="354" width="278" height="53" />
        <rect id="e_outer" x="243.03125" y="1" width="278" height="53" />
        <rect id="e_left" x="197.03125" y="1" width="54" height="406" />
    </g>
    <g class="lbox">
        <rect id="l_short" x="46" y="354" width="105" height="53" />
        <rect id="l_left" y="1" width="54" height="406" />
    </g>
</svg>