On a circular billiard-table, the billiard-ball collides with the boundary of that table with some velocity v1. This collision is detected as follows:
double s = sqrt( (p.x-a)*(p.x-a) + (p.y-b)*(p.y-b) );
if (s<r) // point lies inside circle
// do nothing
else if (s==r) // point lies on circle
// calculate new velocity
else if (s>r) // point lies outside circle
// move point back onto circle (I already have that part)
// calculate new velocity
Now how can the new velocity v2 after the collision be calculated, such that angle of incidence = angle of reflection (elastic collision)?
PS: The billiard-ball is represented by a point p(x,y) with a velocity-vector v(x,y). The simulation is without friction.
Assuming you're making some simple (game-like) billiards simulation you could use something like:
v_new = coeff*(v_old - 2*dot(v_old, boundary_normal)*boundary_normal);
Here v_old is your current velocity vector and boundary_normal is the inward pointing normal of your circular billiards table at the point of impact. If you know the center c of your circular table and you have the point of impact p then the normal is simply normalize(c-p). That is, the normalized vector you obtain when subtracting p from c.
Now I have taken coeff to be a fudge factor between 0 (no velocity at all anymore after impact) and 1 (same velocity after impact). You can make this more physically plausible by determining a correct coefficient of restitution.
In the end all the formula above is, is simple reflection as you might have seen in a basic ray tracer for example. As said, it's a fairly crude abstraction from an accurate physics simulation, but will most likely do the job.