Today I am using Jpmml in order to load pmml models in my code. but the'evaluate' method takes long time. Here is the working code today :
String modelPath = "....";
ModelEvaluatorFactory factory = ModelEvaluatorFactory.newInstance();
InputStream in = new ByteArrayInputStream(modelPath.getBytes("UTF-8"));
PMML pmmlModel = JAXBUtil.unmarshalPMML(new StreamSource(in));
ModelEvaluator<?> evaluator = factory.newModelManager(pmmlModel);
List<FieldName> activeFields = evaluator.getActiveFields();
Map<FieldName, FieldValue> defaultFeatures = new HashMap<>();
//after filling the 'defaultFeatures' the line below takes long time
Map<FieldName, ?> results = evaluator.evaluate(defaultFeatures);
PMML example :
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<PMML xmlns="http://www.dmg.org/PMML-4_2" version="4.2">
<Header>
<Application name="JPMML-SkLearn" version="1.0-SNAPSHOT"/>
<Timestamp>2017-01-22T14:18:05Z</Timestamp>
</Header>
<DataDictionary>
<DataField name="GENDER" optype="categorical" dataType="string">
<Value value="0"/>
<Value value="1"/>
</DataField>
<DataField name="1GA_" optype="continuous" dataType="double"/>
//67000 rows of datafield
</DataDictionary>
<TransformationDictionary>
<DefineFunction name="logit" optype="continuous" dataType="double">
<ParameterField name="value" optype="continuous" dataType="double"/>
<Apply function="/">
<Constant dataType="double">1</Constant>
<Apply function="+">
<Constant dataType="double">1</Constant>
<Apply function="exp">
<Apply function="*">
<Constant dataType="double">-1</Constant>
<FieldRef field="value"/>
</Apply>
</Apply>
</Apply>
</Apply>
</DefineFunction>
</TransformationDictionary>
<MiningModel functionName="classification">
<MiningSchema>
<MiningField name="GENDER" usageType="target"/>
<MiningField name="1GA_"/>
//67000 rows of MiningField
</MiningSchema>
<Output>
<OutputField name="probability_0" feature="probability" value="0"/>
<OutputField name="probability_1" feature="probability" value="1"/>
</Output>
<LocalTransformations>
<DerivedField name="x1" optype="continuous" dataType="double">
<FieldRef field="1GA_"/>
</DerivedField>
//100000 rows
</LocalTransformations>
<Segmentation multipleModelMethod="modelChain">
<Segment id="1">
<True/>
<RegressionModel functionName="regression">
<MiningSchema>
<MiningField name="1GA_"/>
</MiningSchema>
<Output>
<OutputField name="decisionFunction_1" feature="predictedValue"/>
<OutputField name="logitDecisionFunction_1" optype="continuous" dataType="double" feature="transformedValue">
<Apply function="logit">
<FieldRef field="decisionFunction_1"/>
</Apply>
</OutputField>
</Output>
<RegressionTable intercept="-5.303370169392045">
<NumericPredictor name="x1" coefficient="0.18476274186559316"/>
//100000 rows of NumericPredictor
</RegressionTable>
</RegressionModel>
</Segment>
<Segment id="2">
<True/>
<RegressionModel functionName="regression">
<MiningSchema>
<MiningField name="logitDecisionFunction_1"/>
</MiningSchema>
<Output>
<OutputField name="logitDecisionFunction_0"
feature="predictedValue"/>
</Output>
<RegressionTable intercept="1.0">
<NumericPredictor name="logitDecisionFunction_1"
coefficient="-1.0"/>
</RegressionTable>
</RegressionModel>
</Segment>
<Segment id="3">
<True/>
<RegressionModel functionName="classification">
<MiningSchema>
<MiningField name="GENDER" usageType="target"/>
<MiningField name="logitDecisionFunction_1"/>
<MiningField name="logitDecisionFunction_0"/>
</MiningSchema>
<RegressionTable intercept="0.0" targetCategory="1">
<NumericPredictor name="logitDecisionFunction_1"
coefficient="1.0"/>
</RegressionTable>
<RegressionTable intercept="0.0" targetCategory="0">
<NumericPredictor name="logitDecisionFunction_0"
coefficient="1.0"/>
</RegressionTable>
</RegressionModel>
</Segment>
</Segmentation>
</MiningModel>
</PMML>
There is a thought of trying to use MLlib instead of Jpmml. Any ideas? Thanks
What do you mean by "load"? Is it "parse PMML document into an in-memory data structure" or "execute PMML document"?
Your code appears to be aim for the latter. But it will definitely fail, because the JAXBUtil#unmarshalPMML(Source) method is invoked with a byte array, which doesn't contain a valid PMML document (no XML parser will accept "....".getBytes("UTF-8")).
Also, what do you mean by "takes long time"? The JAXB framework has one-time initialization cost of around ~1 second. After that it can unmarshal ~200 to 500 MB (that's megabytes) of PMML content per second. How much more do you need?