AlphaMaven

Company Logo'

Managed Futures

Commodity Pool

Back

Managed Futures

Cornell University Research Paper - The Best Way To Select Features? Comparing MDA, LIME And SHAP

Xin Man And Ernest Chen Of QTS Capital Management

posted by Commodity Pool
3yrs ago 197

Feature selection in machine learning is subject to the intrinsic randomness of the feature selection algorithms (for example, random permutations during MDA). Stability of selected features with respect to such randomness is essential to the human interpretability of a machine learning algorithm. We proposes a rank based stability metric called instability index to compare the stabilities of three feature selection algorithms MDA, LIME, and SHAP as applied to random forests. Typically, features are selected by averaging many random iterations of a selection algorithm. Though we find that the variability of the selected features does decrease as the number of iterations increases, it does not go to zero, and the features selected by the three algorithms do not necessarily converge to the same set. We find LIME and SHAP to be more stable than MDA, and LIME is at least as stable as SHAP for the top ranked features. Hence overall LIME is best suited for human interpretability. However, the selected set of features from all three algorithms significantly improves various predictive metrics out of sample, and their predictive performances do not differ significantly. Experiments were conducted on synthetic datasets, two public benchmark datasets, and on proprietary data from an active investment strategy.


Read full research paper here.


ALL ALPHAMAVEN CONTENT IS FOR INFORMATIONAL PURPOSES ONLY. CONTENT POSTED BY MEMBERS DOES NOT NECESSARILY REFLECT THE OPINION OR BELIEFS OF ALPHAMAVEN AND HAS NOT ALWAYS BEEN INDEPENDENTLY VERIFIED BY ALPHAMAVEN. PAST PERFORMANCE IS NOT INDICATIVE OF FUTURE RESULTS. THIS IS NOT A SOLICITATION FOR INVESTMENT. THE MATERIAL PROVIDED HEREIN IS FOR INFORMATIONAL PURPOSES ONLY. IT DOES NOT CONSTITUTE AN OFFER TO SELL OR A SOLICITATION OF AN OFFER TO BUY ANY INTERESTS OF ANY FUND OR ANY OTHER SECURITIES. ANY SUCH OFFERINGS CAN BE MADE ONLY IN ACCORDANCE WITH THE TERMS AND CONDITIONS SET FORTH IN THE INVESTMENT'S PRIVATE PLACEMENT MEMORANDUM. PRIOR TO INVESTING, INVESTORS ARE STRONGLY URGED TO REVIEW CAREFULLY THE PRIVATE PLACEMENT MEMORANDUM (INCLUDING THE RISK FACTORS DESCRIBED THEREIN), THE LIMITED PARTNERSHIP AGREEMENT AND THE SUBSCRIPTION DOCUMENTS, TO ASK SUCH QUESTIONS OF THE INVESTMENT MANAGER AS THEY DEEM APPROPRIATE, AND TO DISCUSS ANY PROSPECTIVE INVESTMENT IN THE FUND WITH THEIR LEGAL AND TAX ADVISERS IN ORDER TO MAKE AN INDEPENDENT DETERMINATION OF THE SUITABILITY AND CONSEQUENCES OF AN INVESTMENT.
ALL ALPHAMAVEN CONTENT IS FOR INFORMATIONAL PURPOSES ONLY. CONTENT POSTED BY MEMBERS DOES NOT NECESSARILY REFLECT THE OPINION OR BELIEFS OF ALPHAMAVEN AND HAS NOT ALWAYS BEEN INDEPENDENTLY VERIFIED BY ALPHAMAVEN. PAST PERFORMANCE IS NOT INDICATIVE OF FUTURE RESULTS. THIS IS NOT A SOLICITATION FOR INVESTMENT. THE MATERIAL PROVIDED HEREIN IS FOR INFORMATIONAL PURPOSES ONLY. IT DOES NOT CONSTITUTE AN OFFER TO SELL OR A SOLICITATION OF AN OFFER TO BUY ANY INTERESTS OF ANY FUND OR ANY OTHER SECURITIES. ANY SUCH OFFERINGS CAN BE MADE ONLY IN ACCORDANCE WITH THE TERMS AND CONDITIONS SET FORTH IN THE INVESTMENT'S PRIVATE PLACEMENT MEMORANDUM. PRIOR TO INVESTING, INVESTORS ARE STRONGLY URGED TO REVIEW CAREFULLY THE PRIVATE PLACEMENT MEMORANDUM (INCLUDING THE RISK FACTORS DESCRIBED THEREIN), THE LIMITED PARTNERSHIP AGREEMENT AND THE SUBSCRIPTION DOCUMENTS, TO ASK SUCH QUESTIONS OF THE INVESTMENT MANAGER AS THEY DEEM APPROPRIATE, AND TO DISCUSS ANY PROSPECTIVE INVESTMENT IN THE FUND WITH THEIR LEGAL AND TAX ADVISERS IN ORDER TO MAKE AN INDEPENDENT DETERMINATION OF THE SUITABILITY AND CONSEQUENCES OF AN INVESTMENT.