I am using Glassfish server. It seems that the @LocalBean is getting initialized at server startup. For other beans they are correctly initialized on look up. Is this the correct behaviour for LocalBean ?
There is no rule saying that @LocalBean should be eagerly initialized and others should not. It's left to the container provider to decide when particular bean should be initialized.
The only case you have control over when bean is initialized is to use @Singleton EJB with @Startup annotation. This will force container provider to create an instance of the singleton bean during server startup. This is a good place to put your initialization logic within.
The behavior you've observed might be correct in case of Glassfish but I would not relay on it because other container providers might choose different approach.