How much effort do you have to put in to get gains from using SSE?

Case One

Say you have a little class:

class Point3D
{
private:
  float x,y,z;
public:
  operator+=()
  
  ...etc
};

Point3D &Point3D::operator+=(Point3D &other)
{
  this->x += other.x;
  this->y += other.y;
  this->z += other.z;
}

A naive use of SSE would simply replace these function bodies with using a few intrinsics. But would we expect this to make much difference? MMX used to involve costly state cahnges IIRC, does SSE or are they just like other instructions? And even if there's no direct "use SSE" overhead, would moving the values into SSE registers and back out again really make it any faster?

Case Two

Instead, you're working with a less OO-based code base. Rather than an array/vector of Point3D objects, you simply have a big array of floats:

float coordinateData[NUM_POINTS*3];

void add(int i,int j) //yes it's unsafe, no overlap check... example only
{
  for (int x=0;x<3;++x)
  {
    coordinateData[i*3+x] += coordinateData[j*3+x];
  }
}

What about use of SSE here? Any better?

In conclusion

Is trying to optimise single vector operations using SSE actually worthwhile, or is it really only valuable when doing bulk operations?

Solution

In general you will need to take additional steps to get the best out of SSE (or any other SIMD architecture):

data needs to be 16 byte aligned (ideally)
data needs to be contiguous
you need enough data to make the SIMD operation worthwhile
you need to coalesce as many operations as you can to mitigate the costs of loads/stores
you need to be aware of the cache/memory hierarchy and its performance impact (e.g. use strip-mining/tiling)