I have list of shape file generated from ArcGIS residing in a directory hosted server, note the server is not ArcGIS server and the shape files are not published.
Is it possible to load this shape files as a layer(Feature layer or anything) using the ArcGIS JS API as its possible to load KML files?
Is it neccessary to publish the maps to ArcGIS server or do I need to convert the *.shp files to KML?
No, you cannot directly consume a shapefile with the API.
Since you are already apparently using ArcGIS to create the shapefile, the easiest approach would be to just convert that data to KML. There are stock tools for this in the conversion toolbox.
Note that the KML files will need to be public hosted for them to work with KMLLayer (https://developers.arcgis.com/javascript/jsapi/kmllayer-amd.html).