I'm a mathematician and am currently taking a course in developing parallel programs.
There are a lot of technical terms about computers where I'm not that familiar with ( like cache, processors, shared-address-space ... ), so I am looking for a good book about the technical part of the working of a computer.
I've been searching myself but I only seem to find books about logic or programming, not the technical aspect.
Let me propose a simple next step, that may help you start your great journey into an Alice in The Wonderlands domain of computing toys.
You could try a first taste with Victor Eijkhout HPC-book, where you may get views from both sides of the cake -- hardware and performance -- in context.
Nevertheless, let me also mention this. Get ready to be disappointed on the amount of technology related tricks in this domain, as it is hard to compress more than about 80 years of forefront experience in computing into any single book.
Yes, just remember Richard FEYNMAN's team has started first [CONCURRENT] processing in Los Alamos, on Project Manhattan, to the great surprise of both theirs chiefs and IBM ( who manufactured their punched
cards machine without yet realising there is a performance bonus if tasks are being run in a [CONCURRENT] manner )
So we all learn on the fly. 80+ years...