Fitting the pipeline causing exception in Ml.net. The Fit is not an awaitable and I'm confused why this happens. Any help is appreciated
var model = pipeline.Fit(dataView);
Stack trace
at System.Threading.Channels.AsyncOperation.ThrowIncompleteOperationException() at System.Threading.Channels.AsyncOperation
1.GetResult(Int16 token) at Microsoft.ML.Transforms.RowShufflingTransformer.Cursor.MoveNextCore() at Microsoft.ML.Data.RootCursorBase.MoveNext() at Microsoft.ML.Trainers.TrainingCursorBase.MoveNext() at Microsoft.ML.Trainers.SdcaTrainerBase3.TrainCore(IChannel ch, RoleMappedData data, LinearModelParameters predictor, Int32 weightSetCount) at Microsoft.ML.Trainers.StochasticTrainerBase2.TrainModelCore(TrainContext context) at Microsoft.ML.Trainers.TrainerEstimatorBase2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) at Microsoft.ML.Trainers.TrainerEstimatorBase2.Fit(IDataView input) at Microsoft.ML.Data.EstimatorChain1.Fit(IDataView input) at ML.DetectFakeJobPosts.Analyzer.Train() in D:\Sources\code-everything\CodeItHere\ML.DetectFakeJobPosts\Program.cs:line 75 at ML.DetectFakeJobPosts.Program.Main(String[] args) in D:\Sources\code-everything\CodeItHere\ML.DetectFakeJobPosts\Program.cs:line 13
Pipeline
var pipeline = _context.Transforms.Categorical.OneHotEncoding("ec_title", "title")
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_location", "location"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_department", "department"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_salary_range", "salary_range"))
.Append(_context.Transforms.Text.FeaturizeText("ec_company_profile", "company_profile"))
.Append(_context.Transforms.Text.FeaturizeText("ec_description", "description"))
.Append(_context.Transforms.Text.FeaturizeText("ec_requirements", "requirements"))
.Append(_context.Transforms.Text.FeaturizeText("ec_benefits", "benefits"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_employment_type", "employment_type"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_required_experience", "required_experience"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_required_education", "required_education"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_industry", "industry"))
.Append(_context.Transforms.Categorical.OneHotEncoding("ec_function", "function"))
//drop unnecessary columns from view
.Append(_context.Transforms.DropColumns("title", "location", "department", "salary_range", "company_profile", "description", "requirements", "benefits", "employment_type", "required_experience", "required_education", "industry", "function", "telecommuting", "has_company_logo", "has_questions"))
//concate features
.Append(_context.Transforms.Concatenate("Features", "ec_title", "ec_location", "ec_department", "ec_salary_range", "ec_company_profile", "ec_description", "ec_requirements", "ec_benefits", "ec_employment_type", "ec_required_experience", "ec_required_education", "ec_industry", "ec_function"))
//set label/prediction column
.Append(_context.Transforms.Conversion.ConvertType("Label", "fraudulent", DataKind.Boolean))
//select a trainer
.Append(_context.BinaryClassification.Trainers.SdcaLogisticRegression());
Versions
Ml.net : 1.5.1
Core : 3.1
The issue for this is from a PR I made to update part of the code to use Channels. It turns out that it's not always guaranteed to finish the async call before we check the result of it which is causing the error.
There's a fix for this currently in the daily NuGet feed to get the latest daily build until the 1.5.2 version gets officially released.