How do keyword research/analysis software work [theoratically]?

I have seen a lot of keyword research/analysis applications, such as Market Samurai: Keyword Analysis Tool, and SEMRush keyword tool.

My question is how can they get stats about those keywords ? are they using google api to achieve that ?

I fail to see how can a software not connected to google search database get information about monthly searches, competition ...etc.

Thanks.

Solution

For Search Volume, Paid Competition, CPC data, most of these tools get it in one of three ways.

They can get it directly from Google via the AdWords API (requires "Standard Access" and must meet RMF requirements).
Another way is to get it from a third-party who can pull data from Google updated monthly such as GrepWords.
Using their own models with various data sources from third parties, mixed possibly Google's statistics and other click stream data, and applying machine learning algorithms to make predictions that can even rival Google's own data.

For Keyword Difficulty (KD) or Organic Competition scores, all tools provide an estimate of how difficult it might be to rank high organically for a specific keyword. Tools will typically use a combination of techniques. Below is a short list of what they may include:

Search Engine Result Pages (SERP) analysis
Each keywords' SERP density analysis
Analysis of competitors for each keyword
Word difficulty and frequency
Backlink and domain authority analysis of competitors
and many other indicators

A few tools and where they get their Search Volume and CPC data:

SEMRush uses an algorithm to estimate their traffic (source: spoke with them at a conference in 2016).
ahrefs uses a third party to get click stream data and pairs it with data from Google
MOZ uses a third party to get their Google data and click stream data (source: spoke with team).
KWFinder reports that their data is the same as Google Keyword Planner.
Twinword Ideas actually gets their data directly from Google (source: I work there).