Let's say I have a simple CSS layout and my menu is a column on the side. This menu is going to be on every web page (About 10 web pages). If I have items on this menu in the form of links and such, how can I make it so that if I add a link on one page, it will add it to all pages? Would you just make it a PHP page and in that <div> element include a PHP file called menu or something, and just edit that PHP file (so I only have to edit that file and not every web page)?
If this is raw PHP (no frameworks) then you simply include it.
include 'sidebar.php';
Make sure that you can access the file from where you are including it though. If you have files in a folder called foo for example, and accessed via example.com/foo/somefile.php you will have to change the include statement to include('../sidebar.php'); assuming sidebar.php is in the root.