Efektivitas Requirements Traceability Matrix (RTM) sebagai Scaffolding terhadap Deteksi Defect pada Pengujian Perangkat Lunak Mahasiswa

Abstract

The Requirements Traceability Matrix (RTM) is widely used as scaffolding to guide students through thorough software testing, yet a rarely tested assumption holds that completing the procedure equates to detecting defects. This study analyzes planted-bug detection by defect category under uniform RTM completion, a condition not previously examined. A descriptive multiple-case study was applied to nine Information Systems student groups (24 students) testing five web applications containing 33 identically planted defects across seven a priori categories. Detection was assessed through strict matching between reported and planted defects, and analyzed using Wilson confidence intervals and Fisher's exact test. Although all groups completed the RTM at 100%, the overall detection rate was only 55.7% (95% CI 43.3-67.5%), ranging from 14.3% to 87.5% across groups. Detection declined sharply by category; observable defects were detected significantly more often than non-observable ones (67.4% vs 27.8%; odds ratio 5.39; p = 0.010). These findings confirm a dissociation between procedural completeness and detection effectiveness. The study contributes category-level empirical evidence that procedural scaffolding ensures completeness but not detection acuity, and recommends enriching scaffolding with explicit emphasis on negative-scenario and non-functional testing.