Testing the migration of Graphical User Interfaces

Technologies and frameworks are not immortal, it’s a fact. For a software editor like us, it is regularly a necessity to migrate a framework to another. When you consider, large software like e-sedit for example, migrating the entire software in one shot is almost impossible. Many companies are migrating their software systems. And so, we are! Here we relate to the migration of User Interface frameworks. We are migrating applications from the GWT GUI framework to the Angular GUI framework. This migration includes a framework switch (GWT to Angular) and a programming language switch (from Java to TypeScript). To do so, we developed an approach based on meta-models that semi-automatically migrates the GUI of an application. The approach can be split into 3 steps: source code extraction, GUI extraction, exportation. The source code extraction creates a model of the source code that we can request and navigate. The GUI extraction retrieves in the source code model the application widgets, their attributes, and their composition (to build a DOM). Finally, the exportation recreates the GUI in the target language (in our context Angular). Here we present our last experiment to assist the e-sedit team in the migration from the front-end framework Google Web Toolkit (GWT) to Angular. During this process of migration, we asked ourselves:

How can we evaluate the success or failure of our migration process?

And how to ensure that the new UI will look like the old one?

More precisely, in the case of a GUI migration: How to ensure that the new UI will look like the old one? In the migration research study, two UI are often considered similar if they have similar DOM. So, we decided to try on experimenting with this strategy and it was a disaster. Indeed, two UI may have the same DOM and have two completely different visual aspects. We must find another way to validate the migration of the GUI.

Layout validation and Image comparison

The layout can be defined at multiple levels of granularity, at the pixel level, the widget level, or at the panel level. The last one is particularly interesting because it validates that the migrated page looks like the original one but does not require pages to be perfectly identical. It is expected behavior when migrating GUI because the migration is also the moment to improve the look and feel of its application without disrupting the final user).
The idea is to compare the layout of the original GUI and the layout of the migrated GUI. Comparing layout is not a novel idea. Indeed, other research fields already applied such a strategy to classify the document into different categories. For instance, a scientific paper does not have the same layout as a blog post.

First sketch of Solution

The approach to validate the migration of an application is divided into 5 steps :

  1. Detecting pages to validate: we have to collect the pages of the original application and the corresponding migrated page.
  2. Browsing original and migrated pages: the tool browses the pages using a browser (Chrome, Firefox, Safari, …).
  3. Creating blocks: we highlight the structural layout element (panel, fieldset, …) and hide the other element (button, text, etc.)
  4. Taking a screenshot: the tool takes a screenshot of both original and migrated pages with the blocks.
  5. Comparing: for each couple of pages, we compare the position of the blocks.

Challenges in comparing layout

From our experiment we identified 6 challenges for the layout validation:

  • Structural layout elements: Our approach is based on the highlight of “Structural layout elements”. Thus, it is essential to be able to identify those structures. We considered that the fieldsets are the structural layout elements of all pages. However, layout also exists in pages where the DOM does not contain any fieldsets.
  • Ajax-based architecture: The approach is also based on the ability to browse each couple of pages in the source and target applications. It is important to note that the actions needed to access a page may differ between the two applications. For example, the source application might be an HTML/CSS application where a page is accessed using its URL, whereas the migrated application is written using recent web framework (such as Angular, Read, etc.), and the access of a page is made through executing actions on the web page (access the main page by its URL, and then click on a button that will navigate to the page to browse).
  • Successive shifting: the shifting of one block may cause shifting of other blocks and so on. This situation may lead to two completely different images whereas the layout differences are minimal.
  • Dynamic content: The size of widgets may also depend on the data they present. For example, a table displays information that come from a database. If the data of the database evolve, the data present in the table will also change, and the size of the table will change. Thus, the screenshots between the source and the migrated application will be different whereas the layout stay unchanged.
  • Interactive widget: Some widgets are interactive. It is the case of the expandable panel, a panel that can be opened or closed by the user. If the panel is opened by default in the source application, and closed in the migrated application, the resulted screenshots may differ. However, the layout is unchanged. So, we must ensure that all interactive widgets keep the same state (open/close) before the screenshot step.
  • Overlap: A user interface is composed of structural layout element nested the one inside the others. So, an element can be on top of another one, or under another one. The position of the elements is commonly named z-index, and it is part of the layout. Although, our approach allows one to determine the elements inside other, it does not distinguish if an element is above or under another one.

Validation helping features

Additionally, from analyzing the previously defined challenges we recognize some of the features that would be needed to develop elegant and powerful solutions for overcoming these challenges.

  • Block identification: In our approach, we compare screenshots of images where you previously highlight the blocks. However, we did not identify the blocks in the image, we “only” apply a new CSS style that create the blocks. Really identifying the blocks would allow one to perform more precise analysis. For example, one can count the number of blocks or compare the pixels of a source block with the rest of the migrated UI.
  • Traceability: If we can identify blocks, one helpful feature will be to trace the blocks, i.e., which block in the source application corresponds to which one in the migrated application. Thus, one will be able to compare a block of the source application with the corresponding generated block in the migrated application.
  • Block relationship: Another nice feature would be the capability of computing block relationship. Instead of comparing the block position in a page, it compares the position of the blocks the one compare to the other. Achieving this would be the final step in layout migration validation.


We have conducted a first experiment on the validation of the GUI migration from GWT to Angular. It appears that the validation of layout migration is essential but often ignored. Thus, we proposed a first sketch of a solution and defined the main challenges that one must deal with to validate the layout migration. We also identified future challenges in layout migration validation.

More ...

Retour en haut