Reliability Test based on a Binomial Experiment for Probabilistic Worst-Case Execution Times

Arcaro, Luis Fernando; Silva, Karila Palma; Oliveira, Rômulo Silva de; Almeida, Luís

http://hdl.handle.net/10400.22/17845

Use this identifier to reference this record.

Name:	Description:	Size:	Format:
COM_CISTER_Arcaro_2020.pdf		818.59 KB	Adobe PDF	Download

Send Feedback

Authors

Arcaro, Luis Fernando

Silva, Karila Palma

Oliveira, Rômulo Silva de

Almeida, Luís

Abstract(s)

Measurement-Based Probabilistic Timing Analysis (MBPTA) produces Probabilistic Worst-Case Execution Times (pWCETs), i.e., WCET estimates associated with known low exceedance probabilities. Despite applicability and goodness-of-fit tests being used within MBPTA, any method based on the sampling of a population is subject to a degree of uncertainty. The acceptance of MBPTA in industrial engineering processes depends on obtaining enough evidence that the produced pWCETs are indeed reliable. In this paper we propose a statistical hypothesis test to check the reliability of pWCET estimates, done at a specified significance level. We assume as null hypothesis that the pWCET estimate is reliable, and as alternative hypothesis that it is optimistic. Both Type I and Type II errors are considered. The reliability test is based on a binomial experiment and it is complementary to applicability and goodness-of-fit tests. We evaluated the test using multiple synthetic and real-hardware execution time samples, and applied it on 20 pWCET estimates generated for each of them. The combined use of the proposed reliability test with applicability and goodness-of-fit tests could detect most of the knowingly unreliable estimates on synthetic samples. Similar behaviour was observed for real-hardware samples, evidencing the test’s usefulness for selecting pWCET estimates with increased confidence.

Keywords

Real-time systems Ttiming analysis Worst-case execution time Embedded software

URI

http://hdl.handle.net/10400.22/17845

Citation

L. F. Arcaro, K. P. Silva, R. S. de Oliveira and L. Almeida, "Reliability Test based on a Binomial Experiment for Probabilistic Worst-Case Execution Times," 2020 IEEE Real-Time Systems Symposium (RTSS), Houston, TX, USA, 2020, pp. 51-62, doi: 10.1109/RTSS49844.2020.00016.