My question is about adding an application server into a SharePoint 2013 farm which was, initially, a 2 servers environment dedicated for development. It had one server working as WFE / App server and one server dedicated for SQL Server (using an instance).
The WFE / app server began to struggle with all the services running and all the application pools required for all the services to running. Unfortunately, as it's a DEV and QAS environment, we had to create separate web applications and application pools to be able to develop the custom solutions, resulting to 14 applications pools that have to run all the time (very bad for the performance).
As stated, we decided to add an application server so we could move all the services to the second server (Search, Excel Calculation, Business Connectivity, Central Admin, ...). The installation of SharePoint 2013 SP1 was ok after some struggle on the Language Packs. To add the new server in the farm, we followed the steps in PowerShell located here Add web or application servers into farm in SharePoint 2013 but the service "Microsoft SharePoint Foundation Web Application" was started automatically (which is not the intention, we don't want that server to be a web front end, it is also causing issues with some third-party licences). We moved all the services to the new server and the performances globally increased. However, we saw that issues started to appear everywhere, Search became corrupted, a health rule was asking all the time for a psconfig to run (which was failing all the time), nearly impossible to start / stop services on the new server (was taking forever - and was mandatory to kill the service using powershell).
Because of all these issues and lack of solutions, we decided to move back all the services to the WFE server, remove the app server from the farm then put it back in the farm step by step. After following the same steps as the first time, we can see that the psconfig is failing directly now, saying there is an error but doesn't show any error in the upgrade log. I guess something went wrong again or was not cleaned at the removal of the server from the farm.
The question I have is what can we do so :
Thank you in advance for your help, if needed, I can provide more information depending on the questions.
Best Regards, Kevin
It's a good choice to split your farm into multiple server roles to have a better performance and failover improvements. However we must take care about the following situations :)
About the search problem - it's required to change the search topology (index partitions, query, content and crawling components, ..) before you stop the services on current server. Probably you had a corrupted index and topology afected during the services on servers switch. In such situations like that I recommend the following steps:
Take a look at those links:
https://learn.microsoft.com/en-us/sharepoint/search/manage-the-search-topology https://learn.microsoft.com/en-us/sharepoint/search/manage-search-components
If I'm not wrong the 'Microsoft SharePoint Foundation Web Application' is started by default, but you may stop it without any problem (please make sure you don't have a load balance rule that's forwading web requests for this node). It must not cause any problem into your environment.
As you said you had a rule asking 'psconfig to run' and it's already ran when you started 'SharePoint Products Configuration Wizard' to joing the new app server, BUT if there's a cumulative update or product installation pending that required psconfig to run earlier, that may be a problem, for example - Reporting Services, Project Server, or any other update that may require this to run.
Probably you had a pending a PSConfig to run on new or old server by any reason, so it may mislead this setup. The only way to know what's going on is checking on upgrade log file and fix all required stuff and finally.. run the PSConfig ;)
Answer your last answer - We can clean the new server from the feature it copied from the previous trial (before the removal from the farm)?
It's not a good choice to copy or clone SharePoint servers, because you have a bunch of references locally and on SharePoint Databases (hi SharePoint_Config!).
At this point I recommend you to make sure both of servers are at the same build, considering Operating System, .NetFramework, KBs and SharePoint Service Pack and related build.
After all I suggest you to garantee that your new application server is not joined at the farm anymore (you can do it on manage servers on CA), to make sure you don't have old references, and them running Config Wizard at the first one web front end, to ensure you don't have pending changes to be applied at the farm level, and for last step, join the brand new app server to the farm with all same configuration. You may use powershell or wizard, no problems ;)
Good luck!