heri-eval

NAME

heri-eval - evaluate classification algorithm

SYNOPSIS

heri-eval [ OPTIONS ] dataset [-- SVM_TRAIN_OPTIONS ]

DESCRIPTION

heri-eval runs training algorithm on dataset and then evaluate it using testing set, specified by option -e. If option -n was applied, cross-validation is used for evaluation, training and testing on different folds are run in parallel, thus utilizing available CPUs. If -r is used, the dataset is splitted into training and testing datasets randomly with the specified ratio, and then holdout is run.

OPTIONS

-h, --help

Display help information.

	-f		Enable output of per-fold statistics. See -Mf.
	-n N		Enable T*N-fold cross-validation mode and set the number of folds to N.

-r ratio

Split the dataset into training and testing parts with the specified ratio of their sizes (in percents).

-t T

Enable T*N-fold cross-validation mode and set the number of runs to T which 1 by default.

-e testing_dataset

Enable hold-out mode and set the testing dataset.

-T threshold

Set the minimum threshold for making a classification decision. If this flag is applied, micro-average precision, recall, and F1 are calculated instead of accuracy.

-o filename

Save predictions from testing sets to the specified file.

Format: outcome_class prediction_class [score]

-O filename

Save incorrectly classified objects to the specified file.

Format: #object_number: outcome_class prediction_class [score])

-m filename

Save confusion matrix to the specified file.

Format: frequency : outcome_class prediction_class

-p opts

Pass the specified opts to heri-stat(1).

-s opts

Pass the specified opts to heri-split(1).

-M chars

Sets the output mode where chars are: t -- output total statistics, f -- output per-fold statistics, c -- output cross-fold statistics. The default is "-M tc".

-S seed

Pass the specified seed to heri-split(1).

	-K		Keep temporary directory after exiting.
	-D		Turn on the debugging mode, implies -K.

EXAMPLES

heri-eval -e testing_set.libsvm training_set.libsvm -- -s 0 -t 0
export SVM_TRAIN_CMD='liblinear-train'
export SVM_PREDICT_CMD='liblinear-predict'
heri-eval -p '-mr' -n 5 training_set.libsvm -- -s 4 -q
heri-eval -p '-mr' -n 5 training_set.libsvm -- -s 4 -q
export SVM_TRAIN_CMD='scikit_rf-train --estimators=400'
export SVM_PREDICT_CMD='scikit_rf-predict'
heri-eval -p '-c' -Mt -t 50 -r 70 dataset.libsvm

ENVIRONMENT

SVM_TRAIN_CMD

Training utility, e.g., liblinear-train (the default is svm-train).

SVM_PREDICT_CMD

Predicting utility, e.g., liblinear-predict (the default is svm-predict).

SVM_HERI_STAT_CMD

Utility for calculating statistics (the default is heri-stat(1)).

SVM_HERI_STAT_ADDONS_CMD

Utility for calculating additional statistics (the default is heri-stat-addons(1)).

SVM_HERI_SPLIT_CMD

Utility for splitting the dataset (the default is heri-split(1)).

TMPDIR

Temporary directory (the default is /tmp).

HOME

<http://github.com/cheusov/herisvm>

SEE ALSO

heri-split(1) heri-stat(1)