Automatic hybrid analysis technique to improve botnet code coverage using fake server

Seong Bae, Soo Han Kim, Eul Gyu Im

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The number of newly found malware and malware variants keeps increasing every year, and malware can be analyzed through static analysis or dynamic analysis. Malware developers use various packing techniques to avoid or to hinder static analysis and dynamic analysis needs to be used to analyze packed malware. In addition, malware that uses network communications to perform malicious actions that makes more difficult to be analyzed. Malware often hides malicious behaviors which are only triggered when certain conditions are satisfied, and this trigger-based malware usually communicates with the C2(Command and Control) server. With the increasing number of daily submitted malware, various malware analysis tools have also been developed steadily. In this paper, we propose a HyImCoCo (Hybrid Improve Code Coverage) which is a detection tool of trigger-based behaviors by satisfying the triggering conditions using hybrid analysis. IDApython is utilized to extract network-based features, and network-related instructions are modified to connect to our fake server called BKserver. The server can support TCP, UDP, IRC and HTTP protocols which are commonly used protocols by malware. HyImCoCo includes five modules which is Find Path module, Patch module, BKserver module, and CFG(Control Flow Graph) module. Experiments have shown that the proposed method can improve code coverage. With this tool, HyImCoCo helps to analyze the branch after recv by attracting malicious code to BKserver. We argue that our proposed tool, HyImCoCo can contribute to reduce the overheads of dynamic analysis through triggering malicious hidden behaviors in malware and increase to code coverage this helps analyze various paths.

Original languageEnglish
Title of host publicationProceedings of the 2019 Research in Adaptive and Convergent Systems, RACS 2019
PublisherAssociation for Computing Machinery, Inc
Pages276-282
Number of pages7
ISBN (Electronic)9781450368438
DOIs
Publication statusPublished - 2019 Sep 24
Event2019 Conference on Research in Adaptive and Convergent Systems, RACS 2019 - Chongqing, China
Duration: 2019 Sep 242019 Sep 27

Publication series

NameProceedings of the 2019 Research in Adaptive and Convergent Systems, RACS 2019

Conference

Conference2019 Conference on Research in Adaptive and Convergent Systems, RACS 2019
CountryChina
CityChongqing
Period19/09/2419/09/27

    Fingerprint

Keywords

  • Botnet
  • Code coverage
  • Command and control
  • Malware analysis
  • Reverse engineering

Cite this

Bae, S., Kim, S. H., & Im, E. G. (2019). Automatic hybrid analysis technique to improve botnet code coverage using fake server. In Proceedings of the 2019 Research in Adaptive and Convergent Systems, RACS 2019 (pp. 276-282). (Proceedings of the 2019 Research in Adaptive and Convergent Systems, RACS 2019). Association for Computing Machinery, Inc. https://doi.org/10.1145/3338840.3355670