I'd like to know the list of chars that \w passes, is it just [a-zA-Z0-9_] or are there more chars that it might cover?
I'm asking this question, because based on this, \d is different with [0-9] and is less efficient.
\wvs[a-zA-Z0-9_]: which one might be faster in large scale?
[ This answer is Perl-specific. The information within may not apply to PCRE or the engine used by the other languages tagged. ]
/\w/aa (the actual equivalent of /[a-zA-Z0-9_]/) is usually faster, but not always. That said, the difference is so minimal (less than 1 nanosecond per check) that it shouldn't be a concern. To put it in to context, it takes far, far longer to call a sub or start the regex engine.
What follows covers this in detail.
First of all, \w isn't the same as [a-zA-Z0-9_] by default. \w matches every
alphabetic, numeric, mark and connector punctuation Unicode Code Point. There are 119,821 of these![1] Determining which is the fastest of non-equivalent code makes no sense.
However, using \w with /aa ensures that \w only matches [a-zA-Z0-9_]. So that's what we're going to be using for our benchmarks. (Actually, we'll use both.)
(Note that each test performs 10 million checks, so a rate of 10.0/s actually means 10.0 million checks per second.)
ASCII-only positive match
Rate [a-zA-Z0-9_] (?u:\w) (?aa:\w)
[a-zA-Z0-9_] 39.1/s -- -26% -36%
(?u:\w) 52.9/s 35% -- -13%
(?aa:\w) 60.9/s 56% 15% --
When finding a match in ASCII characters, ASCII-only \w and Unicode \w both beat the explicit class.
/\w/aa is ( 1/39.1 - 1/60.9 ) / 10,000,000 = 0.000,000,000,916 s faster on my machine
ASCII-only negative match
Rate (?u:\w) (?aa:\w) [a-zA-Z0-9_]
(?u:\w) 27.2/s -- -0% -12%
(?aa:\w) 27.2/s 0% -- -12%
[a-zA-Z0-9_] 31.1/s 14% 14% --
When failing to find a match in ASCII characters, the explicit class beats ASCII-only \w.
/[a-zA-Z0-9_]/ is ( 1/27.2 - 1/31.1 ) / 10,000,000 = 0.000,000,000,461 s faster on my machine
Non-ASCII positive match
Rate (?u:\w) [a-zA-Z0-9_] (?aa:\w)
(?u:\w) 2.97/s -- -100% -100%
[a-zA-Z0-9_] 3349/s 112641% -- -9%
(?aa:\w) 3664/s 123268% 9% --
Whoa. This tests appears to be running into some optimization. That said, running the test multiple times yields extremely consistent results. (Same goes for the other tests.)
When finding a match in non-ASCII characters, ASCII-only \w beats the explicit class.
/\w/aa is ( 1/3349 - 1/3664 ) / 10,000,000 = 0.000,000,000,002,57 s faster on my machine
Non-ASCII negative match
Rate (?u:\w) [a-zA-Z0-9_] (?aa:\w)
(?u:\w) 2.66/s -- -9% -71%
[a-zA-Z0-9_] 2.91/s 10% -- -68%
(?aa:\w) 9.09/s 242% 212% --
When failing to find a match in non-ASCII characters, ASCII-only \w beats the explicit class.
/[a-zA-Z0-9_]/ is ( 1/2.91 - 1/9.09 ) / 10,000,000 = 0.000,000,002,34 s faster on my machine
Conclusions
/\w/aa and /[a-zA-Z0-9_]/./\w/aa is faster; in others, /[a-zA-Z0-9_]/./\w/aa and /[a-zA-Z0-9_]/ is very minimal (less than 1 nanosecond)./\w/aa and /\w/u is quite small despite the latter matching 4 orders of magnitude more characters than the former.use strict;
use warnings;
use feature qw( say );
use Benchmarks qw( cmpthese );
my %pos_tests = (
'(?u:\\w)' => '/^\\w*\\z/u',
'(?aa:\\w)' => '/^\\w*\\z/aa',
'[a-zA-Z0-9_]' => '/^[a-zA-Z0-9_]*\\z/',
);
my %neg_tests = (
'(?u:\\w)' => '/\\w/u',
'(?aa:\\w)' => '/\\w/aa',
'[a-zA-Z0-9_]' => '/[a-zA-Z0-9_]/',
);
$_ = sprintf( 'use strict; use warnings; our $s; for (1..1000) { $s =~ %s }', $_)
for
values(%pos_tests),
values(%neg_tests);
local our $s;
say "ASCII-only positive match";
$s = "J" x 10_000;
cmpthese(-3, \%pos_tests);
say "";
say "ASCII-only negative match";
$s = "!" x 10_000;
cmpthese(-3, \%neg_tests);
say "";
say "Non-ASCII positive match";
$s = "\N{U+0100}" x 10_000;
cmpthese(-3, \%pos_tests);
say "";
say "Non-ASCII negative match";
$s = "\N{U+2660}" x 10_000;
cmpthese(-3, \%neg_tests);