Optimizing a training set

When setting up an analytics index, the option to optimize the training set will perform the following operations:

1. remove conceptually irrelevant documents.

2. remove documents which are too long or too short to serve as good examples.

3. remove spreadsheets or documents which consist of predominately numeric data.

4. remove log files.

5. remove documents with text resulting from processing errors.

Word count, word uniqueness, punctuation, and words with a high character count are evaluated to determine what must be removed.