How to hide StackLayout in Xamarin Forms

Question

I'm trying to hide StackLayout called InfoStackLayout after my config is downloaded, displaying information label for 5s, but after hiding Infostack in code, my Grid in ContentPage doesn't refresh the view and leaves empty space. (the InfoStackLayout is invisible but still reserves the space) Even worse, buttons in ButtonsStack doesn't respond correctly to clicked events. After locking and unlocking phone everything works as it should.

<Grid x:Name="RootGrid" VerticalOptions="Fill" RowSpacing="0" ColumnSpacing="0">
  <Grid.RowDefinitions>
    <RowDefinition Height="Auto" />
    <RowDefinition Height="Auto" />
  </Grid.RowDefinitions>
  <StackLayout x:Name="InfoStackLayout" Grid.Row="0" Orientation="Horizontal">
    <Label x:Name="InfoLabel" TextColor="White" HorizontalOptions="StartAndExpand" Font="18" />
    <Image x:Name="GetConfig" HorizontalOptions="EndAndExpand" Source="loading.png" HeightRequest="32" WidthRequest="32">
      <Image.GestureRecognizers>
        <TapGestureRecognizer Tapped="ReloadConfig_Tapped" NumberOfTapsRequired="1"/>
      </Image.GestureRecognizers>
    </Image>
  </StackLayout>
  <StackLayout x:Name="ButtonsStack" Grid.Row="1">
  </StackLayout>
</Grid>

InfoLabel.Text = "Download Config";
Device.StartTimer(TimeSpan.FromSeconds(5), () =>
{
    Task.Factory.StartNew(async () =>
    {
        await InfoStackLayout.FadeTo(0);
        InfoStackLayout.IsVisible = false;
        InfoStackLayout.HeightRequest = 0;
    });
    return false;
});

Answer 1

@Shan answer was the best solution. Just replacing Task.Factory.StartNew with Device.BeginInvokeOnMainThread do the trick.