If a malware detector relies heavily on a feature that is obfuscated in a given malware sample, then the detector will likely fail to correctly classify the malware. In this research, we obfuscate selected features of known Android malware samples and determine whether these obfuscated samples can still be reliably detected. Using this approach, we discover which features are most significant for various sets of Android malware detectors, in effect, performing a black box analysis of these detectors. We find that there is a surprisingly high degree of variability among the key features used by popular malware detectors.