MonoGame - proper SpriteFont scaling

I'm making a GUI for my game and now I'm stuck on animations. I need to scale a font up when the mouse hovers over it and scale it down when it's not. Here's my code:

// Update()
if (!IsDisabled)
{
    elapsedSecondsFast = (float)gameTime.ElapsedGameTime.TotalSeconds * 3;
    if (Size.Contains(InputManager.MouseRect))
    {
        scale += elapsedSecondsFast;
        if (scale >= 1.05f) scale = 1.05f;
    }
    else
    {
        scale -= elapsedSecondsFast;
        if (scale <= 1.0f) scale = 1.0f;
    }
}
// Draw()
if ((PrimaryFont != null) && (SecondaryFont != null) && (!string.IsNullOrEmpty(Text)))
{
    if (IsHovered) TextOutliner.DrawBorderedText(spriteBatch, SecondaryFont, Text, new Vector2(TextRectangle.X, TextRectangle.Y), ForeColor, 0.0f, new Vector2((SecondaryFont.MeasureString(Text).X / 2 * scale - SecondaryFont.MeasureString(Text).X / 2), (SecondaryFont.MeasureString(Text).Y / 2 * scale - SecondaryFont.MeasureString(Text).Y / 2)), scale);
    else TextOutliner.DrawBorderedText(spriteBatch, PrimaryFont, Text, new Vector2(TextRectangle.X, TextRectangle.Y), ForeColor, 0.0f, new Vector2(PrimaryFont.MeasureString(Text).X / 2 * scale - PrimaryFont.MeasureString(Text).X / 2, (PrimaryFont.MeasureString(Text).Y / 2 * scale - PrimaryFont.MeasureString(Text).Y / 2)), scale);
}

The above is a GUIElement class which is inherited by my Button class. Let me explain my code briefly:

PrimaryFont and SecondaryFont are 2 SpriteFonts that use the same font but a different size. This gives me the scale up/down animation I need without blurring my PrimaryFont.
TextRectangle and Size are 2 different Rectangles. Since my button has a texture and text I decided not to draw text on the texture file but have the game position my text over the texture to "fake" the effect. So TextRectangle is the size and location of button text and Size is size and location of button texture. TextRectangle has its center point in the center of the Button texture. So far I have been using magic numbers to achieve this. This is the core of the problem here.
You can see my origin, I passed it to the DrawBorderedText method of my TextOutliner class. The attributes are in the same order as if it were a spriteBatch.DrawString() call, only without SpriteEffects and layerDepth.

The Problem

Since I'm scaling the font (origin = center I think) it will no longer be in the center of the button. And since I have been using magic numbers to position the un-scaled text over the center of the button texture, I don't want to be forced to do the same thing for scaled text. I'm looking for an algorithm that would always position the text in the middle of my 270x72 texture, no matter if the text is scaled or not, while keeping the scale animation shown above, for each instance of the Button class. Preferably to have its origin point in the center.

Edit

So should I draw like this:

if (IsHovered) TextOutliner.DrawBorderedText(spriteBatch, SecondaryFont, Text, new Vector2(TextRectangle.X, TextRectangle.Y), ForeColor, 0.0f, new Vector2((SecondaryFont.MeasureString(Text).X / 2), (SecondaryFont.MeasureString(Text).Y / 2)), scale);
else TextOutliner.DrawBorderedText(spriteBatch, PrimaryFont, Text, new Vector2(TextRectangle.X, TextRectangle.Y), ForeColor, 0.0f, new Vector2(PrimaryFont.MeasureString(Text).X / 2, (PrimaryFont.MeasureString(Text).Y / 2)), scale);

and then draw the button's text at btn.Size.Width / 2, btn.Size.Height / 2, (int)MainGame.GameFontLarge.MeasureString("Play").X / 2, (int)MainGame.GameFontLarge.MeasureString("Play").Y / 2

Solution

So I eventually found the algorithm by myself and finally eliminated the need of using magic numbers for position of the text. Here's my technique:

Vector2 m = Font.MeasureString(Text);

TextOutliner.DrawBorderedText(
    spriteBatch,
    Font,
    Text,
    new Vector2(
        Size.X + (Size.Width - m.X) / 2,
        Size.Y + (Size.Height - m.Y) / 2
    ),
    ForeColor,
    0.0f,
    new Vector2(
        m.X / 2 * (scale - 1),
        m.Y / 2 * (scale - 1)
    ),
    scale
);

I couldn't take the scale out of the origin equation as @LibertyLocked suggested, because the font was scaling from top-left point upon Mouse Hover and not the center as I want it to.