Availability of open source build systems like OpenEmbedded can simplify the usage of embedded Linux by allowing creation of highly customizable Linux distributions in time efficient fashion. Testing matrix becomes huge if build system supports various host operating systems and target platforms. In this paper we present a comprehensive model for executing unattended tests covering both the host side build system testing and target side runtime testing including benchmarking of an embedded Linux. This model facilitates testing for “Agile distributed development” in continuous Integration environment. Utilizing this model enabled us to identify some critical issues that would have gone undetected otherwise.