Filter Large-scale Engine Data using Apache Spark

Steven Begg; Andrew Fish; Donato Pirozzi; Guillaume De Sercey; Vittorio Scarano; Andrew Harvey

Filter Large-scale Engine Data using Apache Spark

Steven Begg, Andrew Fish, Donato Pirozzi, Guillaume De Sercey, Vittorio Scarano, Andrew Harvey

Research output: Chapter in Book/Conference proceeding with ISSN or ISBN › Conference contribution with ISSN or ISBN › peer-review

Abstract

This paper introduces a minimum viable softwareproduct to filter large datasets of engine data recorded duringlaboratory experiments of combustion engines. The aim is to supportanalysts in the identification and analysis of specific physicalphenomenon within hours of recorded engine experimental data.Specifically, the tool has been designed considering the use case ofidentifying Low Speed Pre-Ignition events. This work describesthe tool's graphical user interface and its scalable architecturebased on mainstream web and big-data technologies as wellas the practical application to pre-ignition events identification.The paper provides details on the architecture's performance,providing evidence of its scalability by increasing the number ofavailable computing workers.

Original language	English
Title of host publication	IEEE-INDIN 2016 14th international conference on industrial informatics
Place of Publication	Poitiers, France
Pages	1300-1305
Number of pages	6
Publication status	Published - 18 Jul 2016
Event	IEEE-INDIN 2016 14th international conference on industrial informatics - University of Poitiers, France, 18-21 July 2016 Duration: 18 Jul 2016 → …

Conference

Conference	IEEE-INDIN 2016 14th international conference on industrial informatics
Period	18/07/16 → …

Bibliographical note

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Access to Document

INDIN16_CameraReady.pdfAccepted author manuscript, 1.1 MBLicence: Unspecified

Steven Begg
- S.M.Beggbrighton.acuk
- School of Arch, Tech and Eng - Reader
- Centre for Precision Health and Translational Medicine
- Advanced Engineering Centre - Director
Person: Academic

Cite this

@inproceedings{adf6a6385e084ee5875dfc4a3952a1e3,

title = "Filter Large-scale Engine Data using Apache Spark",

abstract = "This paper introduces a minimum viable softwareproduct to filter large datasets of engine data recorded duringlaboratory experiments of combustion engines. The aim is to supportanalysts in the identification and analysis of specific physicalphenomenon within hours of recorded engine experimental data.Specifically, the tool has been designed considering the use case ofidentifying Low Speed Pre-Ignition events. This work describesthe tool's graphical user interface and its scalable architecturebased on mainstream web and big-data technologies as wellas the practical application to pre-ignition events identification.The paper provides details on the architecture's performance,providing evidence of its scalability by increasing the number ofavailable computing workers.",

author = "Steven Begg and Andrew Fish and Donato Pirozzi and {De Sercey}, Guillaume and Vittorio Scarano and Andrew Harvey",

note = "{\textcopyright} 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.; IEEE-INDIN 2016 14th international conference on industrial informatics ; Conference date: 18-07-2016",

year = "2016",

month = jul,

day = "18",

language = "English",

pages = "1300--1305",

booktitle = "IEEE-INDIN 2016 14th international conference on industrial informatics",

}

TY - GEN

T1 - Filter Large-scale Engine Data using Apache Spark

AU - Begg, Steven

AU - Fish, Andrew

AU - Pirozzi, Donato

AU - De Sercey, Guillaume

AU - Scarano, Vittorio

AU - Harvey, Andrew

N1 - © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2016/7/18

Y1 - 2016/7/18

N2 - This paper introduces a minimum viable softwareproduct to filter large datasets of engine data recorded duringlaboratory experiments of combustion engines. The aim is to supportanalysts in the identification and analysis of specific physicalphenomenon within hours of recorded engine experimental data.Specifically, the tool has been designed considering the use case ofidentifying Low Speed Pre-Ignition events. This work describesthe tool's graphical user interface and its scalable architecturebased on mainstream web and big-data technologies as wellas the practical application to pre-ignition events identification.The paper provides details on the architecture's performance,providing evidence of its scalability by increasing the number ofavailable computing workers.

AB - This paper introduces a minimum viable softwareproduct to filter large datasets of engine data recorded duringlaboratory experiments of combustion engines. The aim is to supportanalysts in the identification and analysis of specific physicalphenomenon within hours of recorded engine experimental data.Specifically, the tool has been designed considering the use case ofidentifying Low Speed Pre-Ignition events. This work describesthe tool's graphical user interface and its scalable architecturebased on mainstream web and big-data technologies as wellas the practical application to pre-ignition events identification.The paper provides details on the architecture's performance,providing evidence of its scalability by increasing the number ofavailable computing workers.

M3 - Conference contribution with ISSN or ISBN

SP - 1300

EP - 1305

BT - IEEE-INDIN 2016 14th international conference on industrial informatics

CY - Poitiers, France

T2 - IEEE-INDIN 2016 14th international conference on industrial informatics

Y2 - 18 July 2016

ER -

Filter Large-scale Engine Data using Apache Spark

Abstract

Conference

Bibliographical note

Access to Document

Fingerprint

Profiles

Steven Begg

Cite this