Optimal Feature Selection Visualization
This page contains a guide for visualizing Optimal Feature Selection learners.
Plotting Grid Search Results
sparsity is the only parameter in the grid search, it is possible to automatically visualize the results of the search. After loading Plots.jl, you can call
plot on the
GridSearch object. The
plot function supports two values for the
Validation Score Against Sparsity
:validation shows the validation score against sparsity as a line plot:
using Plots plot(grid, type=:validation)
This plot is particularly useful in selecting the sparsity level. Even though the grid search returns the optimal sparsity level, the validation scores can be close for many of the parameters. As a result, it is important to explore the tradeoff between the complexity of the model and the quality of predictions, which is what this plot shows.
Variable Importance Heatmap
:importance shows the variable importance at various levels of sparsity as a heatmap. The importance is normalized so the most important variable has a value of 1 at each sparsity. The variables are incrementally included from the bottom as they become selected under higher sparsity.
We see that
Disp is the first variable to be selected at sparsity of 1. However, as we increase the sparsity,
WT followed by
HP becomes consistently selected with high importance, indicating potentially more consistent and higher importance of
Combining Multiple Plots
You can also combine these plots by combining the type inputs. For example, you can plot them side by side (this is the default if
type is not specified):
plot(grid, type=[:importance :validation])
plot(grid, type=[:validation; :importance])
write_questionnaire function allows you to create an interactive questionnaire from an Optimal Feature Selection learner.
lnr = IAI.get_learner(grid) IAI.write_questionnaire("questions.html", lnr)
You can open a browser window with the questionnaire for any Optimal Feature Selection learner using
You can also make multi-learner interactive visualizations or visualize grid search results. For example, to show all the Optimal Feature Selection learners searched in a grid search as a single questionnaire, you can do the following: