What does it mean to write an algebraic relational expression for a referential integrity constraint in a database design?
Short answer: it can't be expressed, in general.
But perhaps you have in mind a less-general situation?
Longer answer: There's many variants of relational algebra. Maybe some are designed to express constraints; but the more common ones (especially Codd 1972) aren't.
You'd need a construct to return a Boolean (yes/no this database content conforms to the constraint(s)). Whereas most algebras are aimed at returning relation values.
Or you'd need a convention that (say) an empty result means 'valid' -- presuming an entirely empty database is valid; but some constraints are such that an entirely empty database is not valid; then you'd have to concoct a non-empty result, which would make your query 'domain dependent', which Codd took pains to avoid.