Given two convex polygons P, Q separated by a line, how can I find their common tangents? There should be 4 total.
Geometry isn't my strong side so any help will be appreciated :)
Any tangent will be generated from (at least) one point on each polygon. The tangent forms a half-space for each polygon, containing that entire polygon; different tangents are generated by requiring that each polygon be above or below the tangent line.
Whenever you have a problem involving two convex polygons, the algorithm probably involves "pick a point on each one, then iterate." This algorithm is no different.
What you do, basically, is start with a random guess, and refine. Pick a vertex on each polygon and calculate the line through the two vertices. Look at polygon A and see whether either of the two neighboring vertices is on the wrong side of the line. If one is, replace the current vertex with that vertex. Then do the same check on polygon B. Then, if you updated either vertex, repeat. Eventually the lines will converge to tangents with each vertex the extremal one in the desired direction.