← DARPA APAC
Section 2

Introduction

Comprehension-Driven Program Analysis (CPA) for Malware Detection in Android Phones

Contents

2. Introduction

This work was performed by Iowa State University (ISU) and was issued by the Air Force Research Laboratory (AFRL) under Cooperative Agreement No. FA8750-12-2-0126, Comprehension-Driven Program Analysis (CPA) for Malware Detection in Android Phones. The Defense Advanced Research Project Agency (DARPA) program manager was Tim Fraser and the AFRL Program Manager was Mark Williams. The PI was Dr. Suresh Kothari from ISU and the subcontractors were ENSOFT. Corp and North Carolina State University (NCSU participated only during the early part of Phase I).

As described by DARPA [1] this program aims to address the following challenges.

The Automated Program Analysis for Cybersecurity (APAC) program aims to address the challenge of timely and robust security validation of mobile apps by first defining security properties to be measured against and then developing automated tools to perform the measuring. APAC will draw heavily from the field of formal-methods program analysis (theorem proving, logic and machine proofing) to keep malicious code out of DoD Android-based application marketplaces. APAC will apply recent research breakthroughs in this field in an attempt to scale DoD’s program analysis capability to a level never before achieved with an automated solution. .. The second challenge APAC aims to address is producing practical, automated tools to demonstrate the cybersecurity properties identified. Successful tools would minimize false alarms, missed detections and the need for human filtering of results to prove properties.

2.1 Team

Our team is composed of three sub teams, each of which brought a unique capability to this project.

Suresh Kothari, the principal investigator, and his team at Iowa State University have a track record of innovative advancements in applications of program analysis.

Jeremías Sauceda, the co-principal investigator and his team at ENSOFT have experience in building world-class easy-to-use engineering tools that apply sophisticated algorithms.

In the first year of Phase I, Xuxian Jiang and his team at North Carolina State University provided their expertise in Android malware in the wild.

2.2 Goals and Progress Towards Goals

The Automated Program Analysis for Cybersecurity (APAC) program was designed to find malware in Android phones. The goals as set in the program BAA [2] are listed below.

  1. Develop a practical program analysis tool to keep malicious applications written in Java out of the DoD Android-based mobile marketplaces.
  2. Conduct novel research that leverages advances in static program analysis to develop innovative capabilities, far beyond existing practices, to detect highly sophisticated malware attacks that the DoD needs to be prepared for.
  3. To meet the DoD needs for practical deployment, the proposed tool will surpass the following minimum performance requirements:
    1. False alarms less than 30% in Phase I and less than 5% in Phase II
    2. Missed detection less than 30% in Phase I and less than 5% in Phase II
    3. Manual labor required per mobile app less 160 hours in Phase I and less than 80 hours in Phase II
    4. An analyst with basic knowledge of software development and malicious techniques should be able to operate the tool effectively

When evaluating our success of goal 1, we consider that we were the top performing Blue team in Phase I and among the top 3 performers in Phase II, but on the other hand, a missed detection rate of even 5% may have dire consequences when dealing with DoD level software audits. The Red team has shown that even under somewhat ideal conditions, the problem of detecting malware is far from solved. Additionally even though we performed well within our analysis time goals, the APAC performers received feedback from DoD analysts during PI meetings that human analysis time may need to be decreased further to meet current analysis demands. That being said, our average analysis time was significantly under the proposed analysis time goals proposed in the BAA.

In response to goal 2, our team has published peer-reviewed papers on novel program analysis techniques, which are detailed in the Publications section at the end of this report.

With regard to goal 3A, our team found unintended malware (malicious behavior not purposely crafted by the Red teams, nonetheless found to exist in a challenge app). In Phase I, we found 6 unintended malwares and 35 unintended malwares in Phase II. Our human-in-the-loop process did not produce false alarms in Phase I or II.

Pertaining to goal 3B, we exceeded our Phase I goal of limiting missed detections to 30 % with a missed detection rate of 6.49 %. In Phase II, as a result of the changing nature of the challenge applications our missed detection rate increased to 25% and so we did not meet our Phase II goal of 5 %. The Red team has clearly shown that malware detection is still an unsolved problem and more research is needed. The APAC program has brought to the surface some of the hardest problems of static analysis and the need for new technology to address those problems. Specifically, there is a critical need for new static analysis based program comprehension techniques to enable humans to better hypothesize the candidates for potential malware. In almost all challenge apps in Phase II where we did not find the planted malware, our failure was because of our inability to come up with the right hypothesis. In many of those case we hypothesized and found malware but it was not the one that was planted by the Red Team. Thus, in phase II although we found a lot more unintended malware (35 instances of unintended malware in Phase II as compared to 6 instances in Phase I), our missed detection rate increased was worse in Phase II (25% missed detection rate in Phase II compared to 6.49% in Phase I).

Our 3C goal of analyzing applications under 160 hours per application in Phase I and under 80 hours in Phase II was achieved. In Phase I we averaged 1.13 hours per application and an average of 9.19 hours per application in Phase II. Our analysis time increased between Phase I and Phase II, the bulk of the increase is due to the time that analysts needed for hypothesizing the malware. The time for automated analysis was not the issue. In fact the apps in Phase II were typically much smaller than the apps in Phase I and the scalability of automated analysis was not the issue.

As for goal 3D, we believe we have met this goal. Throughout APAC Phase I and II, ISU has employed undergraduate students to provide feedback and assist with development tasks of the Security Toolbox. And an undergraduate course in software engineering that used Atlas for homework on program analysis showed that users with limited background in software analysis and malware could operate the tool successfully. The Red team continually praised our tool for its usability and maturity in the field.

A further discussion of these results can be found in the Results and Discussions section.