Plotting p-value/significance symbol in between group

Apart from the p-value that I got from running general ANOVA (p=0.034*), which is significant and manually annotated in my bar chart. I would also like to find and include the p-values/ significance symbols within the factors ("Normal" and "High light+Chilled" from growth_condition) at each independent time points (0, 0.5, 8, 24, 48).
However it appeared that all of them were non-significant as shown in the attached image.
May I know which step have I done wrongly?
p.s. examples of my desired p-value/significance plotting

(http://www.sthda.com/english/sthda-upload/figures/ggpubr/010-add-p-values-to-ggplots-compare-means-interaction-3.png) (http://www.sthda.com/english/sthda-upload/figures/ggpubr/010-add-p-values-to-ggplots-compare-means-interaction-2.png)

My plot

My data frame:

Exp1_fv<-summarySE(data=Exp1, measurevar="fv", groupvars=c("growth_condition","time"))

data.frame(Exp1_fv)

growth_condition time N        fv          sd          se         ci 
Normal           0.0 3 0.8066667 0.005773503 0.003333333 0.01434218             
Normal           0.5 3 0.8033333 0.011547005 0.006666667 0.02868435
Normal           8.0 3 0.8100000 0.010000000 0.005773503 0.02484138
Normal          24.0 3 0.8166667 0.005773503 0.003333333 0.01434218
Normal          48.0 3 0.8100000 0.010000000 0.005773503 0.02484138
High light+Chilled  0.0 3 0.8133333 0.005773503 0.003333333 0.01434218
High light+Chilled  0.5 3 0.6766667 0.068068593 0.039299420 0.16909176
High light+Chilled  8.0 3 0.4433333 0.095043850 0.054873592 0.23610201
High light+Chilled 24.0 3 0.3900000 0.045825757 0.026457513 0.11383749
High light+Chilled 48.0 3 0.6966667 0.035118846 0.020275875 0.08724005

My scripts:

ggplot(data=Exp1_fv, mapping = aes(x = factor(time), y = fv, fill= growth_condition))+

  geom_bar(stat = "identity", position = "dodge")+

  labs(title="Fv/Fm", x= "Time (hours)",  y="Fv/Fm", fill= "Growth condition")+

  ylim(0,1.2)+

  geom_errorbar(aes(ymin=fv-se, ymax=fv+se), width=.2, position=position_dodge(width= 0.9))+

  annotate(geom="text", x=1, y=1.1, label="p=0.034")+

  annotate(geom="text", x=1, y=1.0, label="n=3")+

  stat_compare_means(aes(group=growth_condition), label="p.signif")

Solution

Apparently, "stat_compare_means" will require the full data of the groupings in order to provide any meaningful statistical test (credit to Adam Quek for the advice). Hence, I will have to refer my raw data (annotated as "Exp1") in the "stat_compare_means" function, rather than using the summarized data (annotated as "Exp1_fv" which I got from the "summarySE" function).

Modified script: stat_compare_means(data=Exp1, label="p.signif", label.y= 0.85, method="anova")

Modified image