I'm trying to design a program that draws graphs given a set of points (x, y), and it also should recognize the curve (straight line, hyperbole, parabola), with only the help of the points. Is there an algorithm to do that?
You'll need five points if the curve could be a straight line or a conic (hyperbola, parabola, ellipse, circle).
If the five points are collinear, you have a straight line. (Or a degenerate conic? But if you're expecting straight lines, this should indicate a straight line.)
If four are collinear, you have a degenerate conic, given by the line through the four collinear points and any line through the fifth point that is not parallel to the first line.
If three are collinear, you have a degenerate conic, given by the line through the three collinear points and the line through the two other points. (Unless these two lines are parallel, in which case this isn't a conic.)
If no three points are collinear, you have a unique, non-degenerate conic.
To find the equation for this conic (Ax^2 + Bxy + Cy^2 + Dx + Ey + F = 0), take a look at this page, specifically the formula in the DETAILS section. Put in your five x and y values, calculate the determinant of the matrix in terms of x and y, and this will give you the formula of your conic. Then see here to figure out what kind of conic you have based on the values of A, B and C.
If you have more than 5 points, pick five points (preferably so no three are collinear), find the conic, and then check that the remaining points lie on the conic.