About the Project

TbUIS (Testbed University Information System) is a set of software products (parts) which together serve primarily as a complete solution that can be used for controlled experiments evaluating newly developed testing methods and approaches.

The main idea behind this system was based on the fact that newly developed testing methods are usually evaluated on really simple and unrealistic (trivial) applications. Moreover, artificially injected defects into these applications usually are not very realistic either—they often demonstrate only one particular type of defect. This is because existing software tools for injection of these bugs perform only a minor syntactic modification of the source code (or byte-code)—code mutation techniques and similar other techniques.

The main idea behind TbUIS is that there is already one defect-free application (SUT—System Under Test) which can serve as an etalon. Moreover, several clones of this application (another SUTs) are prepared with different injected defects (e.g. defect-clone). The defect injection is ensured by a configurable compilation of the system version from a number of modules which allow simulation of more complex software defects.

Parts of TbUIS

• UIS (University Information System)—an application which fulfills the following requirements:
  • Semi-realistic and easy to understand—the application comes from a university environment where teachers (one type of an actor) teach several subjects, and students (another type of an actor) enroll in these subjects and obtain credits after passing them;
  • Complex three-layered web application with ORM layer—because there is a massive support for testing such types of applications;
  • Uses database for data storage;
  • Preset with realistic (e.g. sufficient amount and quality) data in plenty of mutual combinations—this allows to test it immediately without worrying about initialization;
  • Possibility of initialization with another data set via import of an external XML file (opposite activity, e.g. export is implemented too);
  • Defect-free—application was massively tested (see below) so it can be regarded as verified;
  • Well documented—use cases, requirements, specifications, source code;
  • Prepared for functional testing—each web element has own ID, etc.
• Set of tests are divided into the following categories:
  • Unit tests used in source code;
  • Independent functional tests—Selenium WebDriver and JUnit framework were used;
  • Independent acceptance tests—Selenium WebDriver and Robot Framework.
• ErrorSeeder—a sophisticated tool for injecting defects into the source code:
  • Able to inject almost any defect of any level (misunderstanding the specification, misspelling in source code, malfunction of DB, etc.);
  • Anybody can prepare their own modules with specifically needed defects;
  • Stand-alone application with GUI environment;
  • Prepares .war files.
• Defect-clones—UIS (e.g. the same functionality) with injected defects:
  • Each defect-clone can have one or more injected defects which can be categorized according to their severity (Critical, High, Medium, Low);
  • Defect-clones are independent on each other.

Designed architecture and selected technologies mean that the whole system is ready for further expansion and will serve as a good basis for future development. Available as open source software, hosted on GitLab.

About us

TbUIS has been developed and maintained at the research group ReliSA (Reliable Software Architectures) at Department of Computer Science and Engineering at University of West Bohemia, Czech Republic.

Team:

• Pavel Herout—project leader herout@kiv.zcu.cz (2017—)
  • An idea of the TbUIS, functional testing of UIS, web pages content
• Jiří Matyáš—developer (2017—2018)
  • UIS (version up to 1.2.0) and ErrorSeeder
• Jakub Šmaus—developer (2018—2019)
  • Updating UIS (version up to 1.5.2), maintenance of functional tests, preparing new defect-clones, standalone test runner
• Jan Čarnogurský—web developer (2018—2019)
  • Web pages design
• Jitka Poubová—developer (2018—2021)
  • Generating automated tests—the very first experiment with TbUIS
  • Generating of acceptance tests—the second experiment with TbUIS
• Radek Vais—developer (2019—2020)
  • Updating UIS (version since 1.6.0), acceptance test with Robot Framework
• David Bárta—tester (2019—2020)
  • Load and stress tests