CPerformanceMeasures Class Reference


Detailed Description

Class to implement various performance measures.

based on: Fawcett, T: March 2004, ROC Graphs: Notes and Practical Considerations for Researchers and input from Sonnenburg, S: Feburary 2008, various discussions

Definition at line 45 of file PerformanceMeasures.h.

Inheritance diagram for CPerformanceMeasures:
Inheritance graph
[legend]

List of all members.

Public Member Functions

 CPerformanceMeasures ()
 CPerformanceMeasures (CLabels *true_labels, CLabels *output)
virtual ~CPerformanceMeasures ()
void init (CLabels *true_labels, CLabels *output)
bool set_true_labels (CLabels *true_labels)
CLabelsget_true_labels () const
bool set_output (CLabels *output)
CLabelsget_output () const
int32_t get_num_labels () const
void get_ROC (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_auROC ()
float64_t get_aoROC ()
void get_PRC (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_auPRC ()
float64_t get_aoPRC ()
void get_DET (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_auDET ()
float64_t get_aoDET ()
void get_all_accuracy (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_accuracy (float64_t threshold=0)
void get_all_error (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_error (float64_t threshold=0)
void get_all_fmeasure (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_fmeasure (float64_t threshold=0)
void get_all_CC (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_CC (float64_t threshold=0)
void get_all_WRAcc (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_WRAcc (float64_t threshold=0)
void get_all_BAL (float64_t **result, int32_t *num, int32_t *dim)
float64_t get_BAL (float64_t threshold=0)
virtual const char * get_name () const

Protected Member Functions

void init_nolabels ()
float64_t trapezoid_area (float64_t x1, float64_t x2, float64_t y1, float64_t y2)
void create_sortedROC ()
void compute_ROC (float64_t **result)
void compute_accuracy (float64_t **result, int32_t *num, int32_t *dim, bool do_error=false)
void compute_PRC (float64_t **result)
void compute_DET (float64_t **result)
void compute_confusion_matrix (float64_t threshold, int32_t *tp, int32_t *fp, int32_t *fn, int32_t *tn)

Protected Attributes

CLabelsm_true_labels
CLabelsm_output
int32_t m_num_labels
int32_t m_all_true
int32_t m_all_false
int32_t * m_sortedROC
float64_t m_auROC
float64_t m_auPRC
float64_t m_auDET

Constructor & Destructor Documentation

default constructor

Definition at line 18 of file PerformanceMeasures.cpp.

CPerformanceMeasures ( CLabels true_labels,
CLabels output 
)

constructor

Parameters:
true_labels true labels as seen in real world
output output labels/hypothesis from a classifier

Definition at line 24 of file PerformanceMeasures.cpp.

~CPerformanceMeasures (  )  [virtual]

Definition at line 31 of file PerformanceMeasures.cpp.


Member Function Documentation

void compute_accuracy ( float64_t **  result,
int32_t *  num,
int32_t *  dim,
bool  do_error = false 
) [protected]

compute ROC accuracy/error

Parameters:
result where the result will be stored
num number of points
dim dimension == 2 (output, accuracy/error)
do_error if error instead of accuracy shall be computed

Definition at line 381 of file PerformanceMeasures.cpp.

void compute_confusion_matrix ( float64_t  threshold,
int32_t *  tp,
int32_t *  fp,
int32_t *  fn,
int32_t *  tn 
) [protected]

compute confusion matrix

caller has to delete[]

Parameters:
threshold threshold to compute against
tp storage of true positives or NULL if unused
fp storage of false positives or NULL if unused
fn storage of false negatives or NULL if unused
tn storage of true negatives or NULL if unused

Definition at line 144 of file PerformanceMeasures.cpp.

void compute_DET ( float64_t **  result  )  [protected]

compute DET points and auDET

Parameters:
result where the result will be stored

Definition at line 328 of file PerformanceMeasures.cpp.

void compute_PRC ( float64_t **  result  )  [protected]

compute PRC points and auPRC

Parameters:
result where the result will be stored

Definition at line 278 of file PerformanceMeasures.cpp.

void compute_ROC ( float64_t **  result  )  [protected]

compute ROC points and auROC

Definition at line 204 of file PerformanceMeasures.cpp.

void create_sortedROC (  )  [protected]

create index for ROC sorting

Definition at line 115 of file PerformanceMeasures.cpp.

float64_t get_accuracy ( float64_t  threshold = 0  ) 

get classifier's accuracy at given threshold

Parameters:
threshold all values below are considered negative examples (default 0)
Returns:
classifer's accuracy at threshold

Definition at line 369 of file PerformanceMeasures.cpp.

void get_all_accuracy ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get classifier's accuracies (swig compatible)

accuracy = (true positives + true negatives) / all labels

caller has to free

Parameters:
result storage of accuracies in 2 dim array: (output, accuracy), sorted by output
num number of accuracy points
dim dimension == 2 (output, accuracy)

Definition at line 409 of file PerformanceMeasures.cpp.

void get_all_BAL ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get classifier's Balanced Error (swig compatible)

BAL = 0.5 * ( true positives / all true labels + true negatives / all false labels )

caller has to free

Parameters:
result storage of BAL in 2 dim array: (output, BAL), sorted by output
num number of BAL points
dim dimension == 2 (output, BAL)

Definition at line 581 of file PerformanceMeasures.cpp.

void get_all_CC ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get classifier's Cross Correlation coefficients (swig compatible)

CC = ( true positives * true negatives

  • false positives * false negatives ) / sqrt( (true positives + false positives) * (true positives + false negatives) * (true negatives + false positives) * (true negatives + false negatives) )

also check http://en.wikipedia.org/wiki/Correlation

caller has to free

Parameters:
result storage of CCs in 2 dim array: (output, CC), sorted by output
num number of CC points
dim dimension == 2 (output, CC)

Definition at line 490 of file PerformanceMeasures.cpp.

void get_all_error ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get classifier's error rates (swig compatible)

value is 1 - accuracy

caller has to free

Parameters:
result storage of errors in 2 dim array: (output, error), sorted by output
num number of accuracy points
dim dimension == 2 (output, error)

Definition at line 415 of file PerformanceMeasures.cpp.

void get_all_fmeasure ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get classifier's F-measure (swig compatible)

F-measure = 2 / (1 / precision + 1 / recall)

caller has to free

Parameters:
result storage of F-measure in 2 dim array (output, fmeasure), sorted by output
num number of accuracy points
dim dimension == 2 (output, fmeasure)

Definition at line 452 of file PerformanceMeasures.cpp.

void get_all_WRAcc ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get classifier's Weighted Relative Accuracy (swig compatible)

WRAcc = ( true positives / (true positives + false negatives) )

  • ( false positives / (false positives + true negatives) )

caller has to free

Parameters:
result storage of WRAccs in 2 dim array: (output, WRAcc), sorted by output
num number of WRAcc points
dim dimension == 2 (output, WRAcc)

Definition at line 537 of file PerformanceMeasures.cpp.

float64_t get_aoDET (  ) 

return area over Detection Error Tradeoff curve

value is 1 - auDET

Returns:
area over DET curve

Definition at line 230 of file PerformanceMeasures.h.

float64_t get_aoPRC (  ) 

return area over Precision Recall Curve

value is 1 - auPRC

Returns:
area over PRC

Definition at line 188 of file PerformanceMeasures.h.

float64_t get_aoROC (  ) 

return area over Reveiver Operating Curve

value is 1 - auROC

Returns:
area over ROC

Definition at line 146 of file PerformanceMeasures.h.

float64_t get_auDET (  ) 

return area under Detection Error Tradeoff curve

calculated by adding trapezoids

Returns:
area under DET curve

Definition at line 213 of file PerformanceMeasures.h.

float64_t get_auPRC (  ) 

return area under Precision Recall Curve

calculated by adding trapezoids

Returns:
area under PRC

Definition at line 171 of file PerformanceMeasures.h.

float64_t get_auROC (  ) 

return area under Receiver Operating Curve

calculated by adding trapezoids

Returns:
area under ROC

Definition at line 129 of file PerformanceMeasures.h.

float64_t get_BAL ( float64_t  threshold = 0  ) 

get classifier's Balanced Error at threshold 0

Returns:
classifer's BAL at threshold 0

Definition at line 565 of file PerformanceMeasures.cpp.

float64_t get_CC ( float64_t  threshold = 0  ) 

get classifier's Cross Correlation coefficient at threshold

Returns:
classifer's CC at threshold

Definition at line 476 of file PerformanceMeasures.cpp.

void get_DET ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get Detection Error Tradeoff curve for previously given labels (swig compatible)

DET point = false positives / all false labels, false negatives / all false labels

caller has to free

Parameters:
result where computed DET values will be stored
num number of labels/examples
dim dimension == 2 (false positive rate, false negative rate)

Definition at line 319 of file PerformanceMeasures.cpp.

float64_t get_error ( float64_t  threshold = 0  ) 

get classifier's error at threshold

value is 1 - accuracy0

Parameters:
threshold all values below are considered negative examples (default 0)
Returns:
classifer's error at threshold

Definition at line 277 of file PerformanceMeasures.h.

float64_t get_fmeasure ( float64_t  threshold = 0  ) 

get classifier's F-measure at threshold 0

Returns:
classifer's F-measure at threshold 0

Definition at line 423 of file PerformanceMeasures.cpp.

virtual const char* get_name (  )  const [virtual]

get the name of tghe object

Returns:
name of object

Implements CSGObject.

Definition at line 390 of file PerformanceMeasures.h.

int32_t get_num_labels (  )  const

get number of labels in output/true labels

Returns:
number of labels in output/true labels

Definition at line 107 of file PerformanceMeasures.h.

CLabels* get_output (  )  const

get classifier's output labels/hypothesis

Returns:
output labels

Definition at line 101 of file PerformanceMeasures.h.

void get_PRC ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get Precision Recall Curve for previously given labels (swig compatible)

PRC point = true positives / all true labels, true positives / (true positives + false positives)

caller has to free

Parameters:
result where computed ROC values will be stored
num number of labels/examples
dim dimension == 2 (recall, precision)

Definition at line 269 of file PerformanceMeasures.cpp.

void get_ROC ( float64_t **  result,
int32_t *  num,
int32_t *  dim 
)

get Receiver Operating Curve for previously given labels (swig compatible)

ROC point = false positives / all false labels, true positives / all true labels

caller has to free

Parameters:
result where computed ROC values will be stored
num number of labels/examples
dim dimensions == 2 (false positive rate, true positive rate)

Definition at line 196 of file PerformanceMeasures.cpp.

CLabels* get_true_labels (  )  const

get true labels as seen in real world

Returns:
true labels as seen in real world

Definition at line 83 of file PerformanceMeasures.h.

float64_t get_WRAcc ( float64_t  threshold = 0  ) 

get classifier's Weighted Relative Accuracy at threshold 0

Returns:
classifer's WRAcc at threshold 0

Definition at line 518 of file PerformanceMeasures.cpp.

void init ( CLabels true_labels,
CLabels output 
)

initialise performance measures

Parameters:
true_labels true labels as seen in real world
output output labels/hypothesis from a classifier

Definition at line 55 of file PerformanceMeasures.cpp.

void init_nolabels (  )  [protected]

initialise values independent from true labels/output

Definition at line 45 of file PerformanceMeasures.cpp.

bool set_output ( CLabels output  ) 

set output labels/hypothesis from a classifier

Parameters:
output output labels
Returns:
if setting was successful

Definition at line 90 of file PerformanceMeasures.h.

bool set_true_labels ( CLabels true_labels  ) 

set true labels as seen in real world

Parameters:
true_labels true labels
Returns:
if setting was successful

Definition at line 72 of file PerformanceMeasures.h.

float64_t trapezoid_area ( float64_t  x1,
float64_t  x2,
float64_t  y1,
float64_t  y2 
) [protected]

calculate trapezoid area for auROC

Parameters:
x1 x coordinate of point 1
x2 x coordinate of point 2
y1 y coordinate of point 1
y2 y coordinate of point 2
Returns:
trapezoid area for auROC

Definition at line 132 of file PerformanceMeasures.cpp.


Member Data Documentation

int32_t m_all_false [protected]

number of negative examples in true_labels

Definition at line 463 of file PerformanceMeasures.h.

int32_t m_all_true [protected]

number of positive examples in true_labels

Definition at line 461 of file PerformanceMeasures.h.

float64_t m_auDET [protected]

area under DET; 1 - area over DET

Definition at line 473 of file PerformanceMeasures.h.

float64_t m_auPRC [protected]

area under PRC; 1 - area over PRC

Definition at line 471 of file PerformanceMeasures.h.

float64_t m_auROC [protected]

area under ROC; 1 - area over ROC

Definition at line 469 of file PerformanceMeasures.h.

int32_t m_num_labels [protected]

number of true labels/outputs/accuracies/ROC points

Definition at line 458 of file PerformanceMeasures.h.

CLabels* m_output [protected]

output labels/hypothesis from a classifier

Definition at line 456 of file PerformanceMeasures.h.

int32_t* m_sortedROC [protected]

array of size num_labels with indices of true_labels/output sorted to fit ROC algorithm

Definition at line 467 of file PerformanceMeasures.h.

CLabels* m_true_labels [protected]

true labels/examples as seen in real world

Definition at line 454 of file PerformanceMeasures.h.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation