I wrote a test some time ago that tests an integration I wrote between my code and a third-party API. The test makes sure that the integration works properly and that we get back the expected results.
The formal build failed today because the test received a 500 error while trying to connect to the third-party API.
Does it make sense to test a situation like this?
In my opinion integration tests are okay to fail if the 3rd party (db, webservice etc.) isn't available. If you don't want to test the integration itself and just the plain functionality you could mock the result of your API and test against them. In that scenario your test are not depending on the 3rd party availability anymore.
I generally mark unit tests depending on a 3rd party availability with a group attribute like "Integration" and exclude them from the continous integration process. Instead I let them run in a nightly build. Integration tests are generally expensive in time and the whole point of continuous integration is to provide rapid feedback.