Filter Large-scale Engine Data using Apache Spark

Steven Begg, Andrew Fish, Donato Pirozzi, Guillaume De Sercey, Vittorio Scarano, Andrew Harvey

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBNpeer-review


This paper introduces a minimum viable softwareproduct to filter large datasets of engine data recorded duringlaboratory experiments of combustion engines. The aim is to supportanalysts in the identification and analysis of specific physicalphenomenon within hours of recorded engine experimental data.Specifically, the tool has been designed considering the use case ofidentifying Low Speed Pre-Ignition events. This work describesthe tool's graphical user interface and its scalable architecturebased on mainstream web and big-data technologies as wellas the practical application to pre-ignition events identification.The paper provides details on the architecture's performance,providing evidence of its scalability by increasing the number ofavailable computing workers.
Original languageEnglish
Title of host publicationIEEE-INDIN 2016 14th international conference on industrial informatics
Place of PublicationPoitiers, France
Number of pages6
Publication statusPublished - 18 Jul 2016
EventIEEE-INDIN 2016 14th international conference on industrial informatics - University of Poitiers, France, 18-21 July 2016
Duration: 18 Jul 2016 → …


ConferenceIEEE-INDIN 2016 14th international conference on industrial informatics
Period18/07/16 → …

Bibliographical note

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.


Dive into the research topics of 'Filter Large-scale Engine Data using Apache Spark'. Together they form a unique fingerprint.

Cite this