Scenario 1: I've been playing around lately with a CI tool Jenkins, to integrate my Selenium WebDriver tests using Maven. I understand continuous integration (with respect to testing) is a concept where tests are triggered automatically, as soon as some build (let's call it Build#666) is checked in by the developers. This helps us identify bugs at the early stages.
Scenario 2: Now suppose I run the same tests without using Jenkins. Say I don't even use Maven, I just right-click on my TestNG suite and run it against Build#666 (meaning I'm manually initiating my automated tests to run as soon as the code for Build#666 is checked in).
Apart from the fact that Jenkins allows the tests to be run automatically (without manual intervention like in Scenario 2) and has features to schedule tests at a desired time, what is the biggest difference?
→ Can I consider also Scenario 2 to be continuous integration, as I am deploying the same tests against the same build, but I don't use a CI tool, but do it manually?
There are many reasons to use a CI server (Jenkins is only one of many options) instead of building and testing locally:
A CI server provides a known environment for running tests. Developer machines are usually configured somewhat manually, so test results can be different due to different versions of software installed, environment configuration, etc.
A CI server can make the build faster by running on better or more hardware than a development machine.
CI servers provide features such as dependencies between different builds which you would otherwise have to implement (script) yourself.
A CI server is a repository for build artifacts: both compiled and/or packaged software, and analysis results like code coverage and static analysis results.
Unlike an engineer, a CI server can run the build
Using a CI server is cheaper than having an engineer do the same work.
The second-to-last point means that no, if you're not running your builds automatically, you're not doing CI. Automation puts the "C" in CI.