A Backpressure Mitigation Scheme in Distributed Stream Processing Engines

Muhammad Hanif, Hyeongdeok Yoon, Choonhwa Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

There is a significant rise in the adaptation of streaming applications in the past decade by individuals researchers and organizations in both industry and academia. These applications are all based on the modern data stream processing systems that implement resource allocation and management in order to provide an uninterrupted track of queries over incoming input distributed data streams. More than a few stream processing engines exists to handle these distributed streaming applications. These distributed applications have open challenges like backpressure. In this paper, we introduce a backpressure mitigation mechanism for the distributed stream processing systems. The proposed backpressure mitigation technique is a generic one and is feasible to be implemented on top of a number of popular streaming frameworks. We use Flink as a testbed for this work and use its available APIs.

Original languageEnglish
Title of host publication34th International Conference on Information Networking, ICOIN 2020
PublisherIEEE Computer Society
Pages713-716
Number of pages4
ISBN (Electronic)9781728141985
DOIs
StatePublished - 2020 Jan
Event34th International Conference on Information Networking, ICOIN 2020 - Barcelona, Spain
Duration: 2020 Jan 72020 Jan 10

Publication series

NameInternational Conference on Information Networking
Volume2020-January
ISSN (Print)1976-7684

Conference

Conference34th International Conference on Information Networking, ICOIN 2020
CountrySpain
CityBarcelona
Period20/01/720/01/10

    Fingerprint

Keywords

  • backpressure
  • Cloud Computing
  • Distributed Computing
  • SLA
  • Streaming

Cite this

Hanif, M., Yoon, H., & Lee, C. (2020). A Backpressure Mitigation Scheme in Distributed Stream Processing Engines. In 34th International Conference on Information Networking, ICOIN 2020 (pp. 713-716). [9016513] (International Conference on Information Networking; Vol. 2020-January). IEEE Computer Society. https://doi.org/10.1109/ICOIN48656.2020.9016513