Research Problems for MS/Ph.D. Theses


Software Testing Research and Development

Comparison of Population Models and Program Flowgraphs: Population models depict a view of a software system from a completely external point of view. Program flowgraphs, on the other hand, create a very similar graph structure except from a completely internal (i.e., based on the source code) point of view. This project would require empirical study of each model of a given product to determine if a relationship between the two models exists and can be derived. If there is a congruity between the operational modes (which define the population models) and the internal data architecture of the program, then it may be possible to automatically extract the population model from program flowgraphs and documentation.

Specializing Markov Chain Population Models for Real-time Systems: Currently, state spaces for Markov models are determined using operational modes that are based on criteria adequate for user interface intensive systems. However, these same criteria may distinguish very few states in many real-time systems. This project will be to study various real-time systems in order to determine what criteria help distinguish operational modes in such systems. Similar projects on other specific classes of software, e.g., client-server, parallel/distributed systems, and so forth are also of interest.

Stochastic Models for Software Applications: The current model of choice for representing software input populations is the finite-state, discrete-parameter Markov chain. However, experience with these models shows that they have the following weaknesses:

Other stochastic models have been proposed including the stochastic push-down automaton (S-PDA), probabilistic neural networks and context sensitive grammars. This project involves the development of a new method for constructing population models based on one or more of these structures.

Automated Model Construction using Constraint Reasoning: Currently a Markov chain is constructed by discovering an application's operational modes and manually combining these modes to determine the state space. Next, arcs are drawn between states to establish legal orderings for test cases. We propose that after operational modes are discovered that the tester then write constraints that specify:

This would allow automatic generation of the Markov chain. This project consists of a research component to recast Markov chain construction as a constraint reasoning problem and then a development component to automate the new process.

Intelligent Methods for Test Case Generation: Currently test cases are generated without regard to input domain coverage or the likelihood of finding bugs. Intelligent test case generation algorithms would allow dynamic adjustment of the transition probabilities based on either

Such algorithms must be cognizant of the test traversal history and the location of past bugs. Since these are already encoded in the sample model, we propose using this as the guide for dynamic adjustment of the parameters of the population model as testing progresses. This project will involve study of various AI techniques for learning which paths are more productive, including reinforcement learning, data mining and heuristics.

Test Case Verification Using Functional Inversion: Hardware diagnostic procedures have been developed that involve model-generated system traversals to find broken components. Each time a component is visited, its function is inverted by treating its output as input for the inverse of the component's specified function. A mismatch between the original input domain and the output of the inverted function indicates a malfunction.

The parallels between this process and testing from the population model are intriguing. This project involves recasting the hardware diagnostic procedure on software. It is likely that we will require a more formal specification (i.e., a black box function) for candidate software than is otherwise necessary. Also, we may have to build population models with more direct similarity to program control flow.

Stochastic Models for Sample Analysis: Since the population model is represented by a stochastic process, it seems reasonable to construct a model of the sample using a similar stochastic process. In fact, this research is already in progress and valuable results have been derived, including the ability to construct sample models and compute quality measures and stopping criteria . This project is to extend the existing model by introducing any of the following:

Human-Computer Interaction

Stochastic Models for Usability Analysis (Pursued by Mike Terry): Population models allow valuable information about interface complexity to be extracted from a proposed or existing design. This project attempts to quantify the complexity of an interface in terms of its usability (or lack thereof), i.e., how difficult it is for user to understand and use an interface, and what interface designs are the most usable.

We have constructed an interface that will be installed on a frequently used (wed-based) library retrieval system. We will instrument the interface with data collection mechanisms that will dynamically construct a stochastic model of user interaction with the interface. We will then use this model to derive measures of usage complexity and compare and contrast different interface design strategies. We hope to generalize a set of metrics to assess interface complexity and also invent new interface designs that will be easier to use by certain classes of users.

Stochastic Models for Interface Certification: Human-computer interfaces are a special class of software systems that can fail in two different ways. First, like other types of software, they can fail because of buggy code. Second, they can fail because of poor design decisions that create frustrating (or even hazardous) situations for users. For example, incorrect default values on a button menu and misleading toolbar descriptors are not bugs, but they would cause extra work and/or confusion on the part of the user.

This project will consist of a systematic study of what determines an interface flaw and then classify these flaws according to severity. Next, it should be possible to fit this problem of interface certification as a software reliability problem which would enable us to compute reliability metrics for a user interface.

Process Estimation

Stochastic Models for Software Process Estimation (Pursued by Tom Gorsuch): Estimation models for cost/duration of software development projects based on stochastic process theory have been proposed but no results have been generated. This project consists of defining a method to construct stochastic models of software process execution and the derivation of measures and metrics to assess the process before, during and after execution.