Tarantool fiber behavior with fiber.yield() and fiber.testcancel()

Question

I ran into an unexpected behavior while building Tarantool app based on fibers.

Simple reproducer of my code looks like this:

local log = require('log')
local fiber = require('fiber')

box.cfg{}

local func = function()
    for i = 1, 100000 do
        if pcall(fiber.testcancel) ~= true then
            return 1
        end

        fiber.yield()
    end

    return 0
end

local wrapfunc = function()
    local ok, resp = pcall(func)
    log.info(ok)
    log.info(resp)
end

for _ = 1, 100 do
    local myfiber = fiber.create(wrapfunc)
    fiber.sleep(0.02)
    fiber.kill(myfiber)
end

and it prints to log false, fiber is cancelled. Moreover, if I use the following func:

local func = function()
    for i = 1, 100000 do
        if pcall(fiber.testcancel) ~= true then
            return 1
        end

        pcall(fiber.yield)
    end

    return 0
end

it prints to log true, 1, and if I use

local func = function()
    for i = 1, 100000 do
        if pcall(fiber.testcancel) ~= true then
            return 1
        end

        if pcall(fiber.yield) ~= true then
            return 2
        end
    end

    return 0
end

it prints to log true, 2.

I expected that after yielding from running myfiber, if control returns to the external fiber and it calls fiber.kill(myfiber), the next time control returns to the cancelled myfiber we will be in the end of cycle iteration and on the next iteration code will successfully return 1. However, the work of func ends with throwing error fiber is cancelled, not with return. So how the real life cycle of yielding fiber works?

Answer 1

Actually, there is no unexpected behaviour here. I believe it mostly documentation issue. Let me explain. I've simplified your example a bit:

#!/usr/bin/env tarantool

local fiber = require('fiber')

local f1 = function() fiber.yield() end
local f2 = function() pcall(fiber.yield) end

local func = function(fn)
    fn()
    if not pcall(fiber.testcancel) then
        return 'fiber.testcancel() failed'
    end
end

local fiber1 = fiber.create(function() print(pcall(func, f1)) end)
fiber.kill(fiber1)
local fiber2 = fiber.create(function() print(pcall(func, f2)) end)
fiber.kill(fiber2)

The output would be:

false   fiber is cancelled
true    fiber.testcancel() failed

When you call fiber.kill, fiber.yield() or fiber.sleep() just raises an error, so your fiber is not able to reach fiber.testcancel and just dies. When you do pcall(fiber.yield), you basically suppress this error and proceed. Then fiber.testcancel checks its fiber status and reraise an exception. But this is a silly example.

Now, with bigger chunks of code, when lots of function invocations involved, you usually want to catch those errors during yield, do some finalisation work and call fiber.testcancel() to promote error upwards (imagine multiple checks of this kind in different parts of big stacktrace). I believe it is the basic use case, fiber.testcancel was introduced for, besides discussions if its design is usable or not.

P.s. And yes, occasional exceptions of such yield calls are not documented. At least I could not find anything in the fiber page