GIMP USABILITY
Usability Test of GIMP 2.0



GIMP 2.0 Usability Test - May 2004
Authors: Ellen Reitmayr and Roman Joost

wilber
   

0. Summary


The test is based on four subjects showing different levels of imaging manipulation experience.

The GIMP 1.2 user (who had attended two GIMP workshops) had no problems completing the task and determined various improvements with respect to functionality and usability. Nevertheless, he was also faced some problems, especially with respect to selection modes, visual feedback, and the journal.

The two Photoshop users completed most tasks, but were partially faced severe problems while interacting with the software. Mostly, that was because they did not find the appropriate options, as they had the Photoshop concepts in mind and were searching for the corresponding labels, words and icons. Also, they were used to different keyboard shortcuts.

The image manipulation beginner did not know the underlying concepts. Nevertheless, she completed the easy tasks successfully.


0.1 Tasks

The tasks were given in german language and were intended to simulate typical use cases when working with the GIMP.

  1. Create a rectengular selection and subtract the edge of the created selection.
  2. Create a selection with fixed width and height and a fixed aspect ratio.
  3. Selection by color, fill an area with a pattern and cancel the selection.
  4. Measure how long a distance between some image objects is.
  5. Clone an image object with the clone tool.
  6. Transform image objects (scaling, shearing etc)
  7. Use the color picker to get the color info with sample merged enabled.
  8. Using the Undo History.
  9. Add an image grid and change the distance of the grid points.
  10. Use the path tool to create a cloud and stroke it with some patterns.
  11. Try to dock some tabs into the docks.




1. General Aspects

1.1 Issue: Providing hints

It was shown that especially users that were new to the GIMP did not find important functionalities as they were hidden from the user. There were no hints, and in the case of tooltip texts, the information was often not sufficient.

Suggestions:

  1. Provide some extra space in the status bar where context sensitive hints are given.
  2. Wherever possible, provide information about shortcuts in the tooltip texts.

1.2 Issue: Determining the current layer

All users had problems to determine the current layer, as it is neither marked in the image, nor in the layer dialog if it does not have the window focus. This caused errors and confusion while manipulating the images.

Suggestions:

  1. Mark the current layer in the layer dialog even if it does not have the window focus.




2. Zooming

2.1 Issue: Handling of the Zoom-Function

Both, the menu 'View  Zoom' and the keyboard shortcuts are intuitive and easy to use.

One testperson tried to use the mouse wheel and expected to zoom in and out by that means. As this did not work, he used the zoom function in the toolbox. He tried the keyboard shortcut CTRL which worked fine, and stated that there used to be a problem with this in the GIMP 1.2. Only, it did not become clear what 'Threshold' meant in that context.

Suggestions:

  1. Provide a short tooltip text to explain the 'Threshold' option.
  2. Provide a context sensitive help (F1) to explain the function of the 'Threshold' option in that context.




3. Selections

3.1 Issue: Saving the settings

One user (Photoshop expert) had severe problems to make a free selection as the fixed ratio option was still enabled from the last session (as all settings, the selection modes are saved after a session).

As the user did not know that the fixed ratio (1:1) option was preselected, he thought that the square selection was the default setting. Therefore, he tried various shortcuts to override the 'default', but did not succeed. Then he viewed the tool options, but still had no idea what to do. He searched for approximately five minutes, until he finally found the appropriate pulldown menu. The reason why he did not find the option was a wording problem as he was searching for the german notion for 'proportional selection' instead of 'fixed aspect ratio'.

While the wording problems were caused by the fact that the person was used to Photoshop notions, the problem caused by saving modes among sessions might happen to everyone who is not too much involved in working with the GIMP. While it is helpful to save settings like units, user-defined colors, and other options that are likely to be re-used, it is not helpful to save operation modes. When interacting with an application, operation modes are changed several times, and it is unlikely that a user remembers the mode he lastly utilised in his last session.

Another example for problems caused by saving operation modes also occurred in the scope of the selection tool: In three cases out of four, when starting with the second task, the selection tool was still in the subtraction mode. Then, it is not possible to draw a selection (until a selection in the 'replace' or 'addition' mode is made). All users were puzzled that their selection was not drawn. While the GIMP 1.2 user recognised quite soon why that did not work, the other two users had severe problems to find the error.

Suggestions:

  1. Do not save operation modes between sessions.

3.2 Issue: Keyboard Shortcuts

For Photoshop users, it is not convinient to use the 'ctrl' key as a shortcut. Therefore, one of the users had some trouble realising that he could use that key.

The shortcuts are not consistent among the selection modes. For example, holding the key pressed when drawing a selection causes a centered 1:1 selection in the 'addition' ('shift') and the 'intersection' mode ('ctrl' + 'shift'). In the 'subtraction' mode ('ctrl'), it causes a centered free selection.

The GIMP 1.2 user had learned the shortcuts in a workshop and therefore had only little problems to utilise them. The two Photoshop users instead did not completely understand the shortcuts. They were annoyed that they could not turn off the centered selection, and therefore always had to use the tool options dialog to change between the modes (none of the users appreciated the centered selection). Moving the mouse to the options dialog and pushing the corresponding buttons decreases working speed significantly compared to utilising keyboard shortcuts.

Also, the users stated that hints concerned with the shortcuts were missing. In Photoshop, they are provided in the status bar.

Suggestions:

  1. Provide the keyboard shortcuts in the selection modes' tooltips to facilitate their usage for new users.
  2. Consider carefully if 'centered selection' should be the default when using shortcuts. According to this test, a non-centered selection is more convenient (but as the number of subjects was very low it is not possibl to make a general suggestion).
  3. However, if non-centered selection becomes the default, provide both, a checkbox in the options dialog, and a shortcut (e.g. pressing 'c' concurrently) for centered selection.
  4. Provide consistent behaviour: When keeping the key pressed while dragging the mouse, make the selections squared in all modes.

3.3 Issue: Moving a selection

All users first ran into trouble when trying to change the position of the current selection. For example, when selecting the DBA logo in Task 2, they realised that their selection was not placed in the center of the original logo. Instead of starting all over again and make a new selection, they tried to move the frame of the current selection to the center of the logo. When pressing the left mouse button inside the selection and moving the mouse, the whole selection is moved (corresponds to cut and paste). In order to change the position of the selection (that means the selection frame is moved  which is what the users wanted), the 'alt' key has to be pressed concurrently.

While the GIMP 1.2 user quickly found the corresponding shortcut and from then on applied it during the whole test, the Photoshop users did not get used to this. In Photoshop, the selection's position is changed by simply dragging it.


Suggestions:

  1. Leave the shortcuts as they are, but when the pointer is above the selection, provide a hint in the status bar ('Press 'alt' and drag the selection to change its position').

3.4 Issue: Manipulating the size of a selection

The GIMP 1.2 user repeatedly tried to manipulate the size of a given selection. He tried several keyboard shortcuts, moved the pointer onto the edges of the selection and was looking for an opportunity to increase its size (this is possible in the 'crop' tool).

He was quite annoyed that it did not work, because he had to cancel the current selection and start all over again.

Suggestions:

  1. Provide the opportunity to manipulate the size of a selection: When the mouse pointer reaches the edges of a selection, the user can increase and decrease the size by simply dragging the edge. Provide visual feedback next to the mouse pointer (corresponding to the 'crop' tool). In case of elliptic selection, the size is changed at the edges of the reference rectangle.

3.5 Issue: Fixed Aspect Ratio

One user could not determine the difference between 'fixed aspect ratio' and 'fixed size', even after he tried both options.

Two out of four users did not understand the handling of the input field for the 'fixed aspect ratio'. Both specified something like 15:15 or 30:30, but did not recognize, that such input types would not change the behaviour of the selection tool. They were confused by the 'unit' specification field (pulldown menu) next to the input fields, as it invoked the impression that absolute sizes were specified instead of ratios. Nevertheless, one user had no problems with this, and the GIMP 1.2 user utilised keyboard shortcuts instead of the entry field.

Another issue is that the user-defined aspect ratio is overwritten by the size of the recent selection as soon as the 'fixed aspect ratio' mode is left. As the two Photoshop users often changed the modes while they explored the GIMP's functionality, they were really annoyed by that behaviour. Other than the mode itself, this option is not an operation mode, but a user-defined setting. Therefore, it should be saved.

Suggestions:

  1. Provide another input dialog for the 'Fixed aspect ratio': Use radio buttons for the most common selections (1:1, 2:1, 1:2), and additionally a field 'other: [ ]:[ ]'.
  2. Remember the selection and the entry in the 'other' field, even if the 'fixed aspect ratio' mode is left.

3.6 Issue: Subtracting a selection

At any point while completing Task 1 or Task 2, each of the three image manipulation experts used the 'subtraction' mode. They had severe problems with the handling of the tool, especially when they used the dialog buttons instead of shortcuts. Then, the starting point of the selection that shall be subtracted must be outside the original selection, otherwise the original selection is moved. It is only possible to set the starting point inside the selection if using shortcuts instead of the dialog buttons. Also, if nothing is selected, it is not possible to set the basic selection when the 'subtraction' mode is active.

The Gimp 1.2 user and one of the Photoshop users tried to remove the lower right edge from their selection. In order to do this, both selected the subtraction mode by clicking the corresponding button in the tool options dialog. Then they tried to set the start point inside the basic selection  which did not work. Neither the GIMP 1.2 user nor the Photoshop user understood the reason for that, so the GIMP user decided to complete the task by utilising channels, and the Photoshop user utilised 'addition'.

The other Photoshop user tried to cut a hole into his basic selection. Again, the dialog buttons where utilised instead of shortcuts  but it is impossible to cut a hole into a selection without using shortcuts. Also, repeatedly, he tried to set the basic selection in 'subtraction' mode. Finally, after 25 minutes, the Photoshop user completed the task by creating several layers. Nevertheless, he was very frustrated.

Suggestions:

  1. In 'addition', 'subtraction', and 'intersection' mode, the mode should persist when the mouse pointer is moved over the selection. Only in 'replace' mode, the selection can be moved.

3.7 Issue: Floating selections

The Photshop users and the image manipulation beginner had severe problems to understand the concept of floating selections. For them, it was not clear that a floating selection forms a layer on its own. Two of them did not understand why they could not create a selection in the background after a floating selection is created.

The three non-GIMP users repeatedly tried to remove the floating selection by clicking elsewhere onto the image. When that did not work, they mostly searched the 'Selection' menu and were confused because most of the menu items were inactive (e.g. 'Selection  None'). They did not search the 'Layer' menu, as they did not know that the GIMP handles floating selections as special layers.

Mostly, that problem occurred after transforming a selection (e.g. make a rectangular selection, scale the selection). In order to remove the selection, they either tried the 'Selection' menu, or they once more chose the 'rectangular selection' tool. In doing so, the anchor symbol was shown next to the mouse pointer, so the users thought they might anchor the selection by simply clicking the left mouse button - but that did not work. Instead, the 'move' tool or the menu item 'Layer  Anchor Layer' had to be selected. In this case, the visual feedback was misleading.

Suggestions:

  1. When the anchor is shown next to the mouse pointer, it must always be possible to anchor the selection.
  2. Do not change the menu structure with respect to the anchoring. But provide a hint in the status bar 'To remove the selection, you have to anchor it by selecting 'Layer  Anchor Layer''.




4. Select regions by colour

4.1 Issue: Setting the threshold

The GIMP 1.2 user missed the former dialog which showed the colour channels and provided a slider to set the threshold of the selection. The dialog provided some kind of preview of the actual selection, as in a complex image, it is quite hard to determine which parts are selected and which are not.

In the GIMP 2.0, the threshold slider is in the tool options dialog. While the GIMP 1.2 user had no problems to fine tune his selection with the help of the threshold slider, one of the Photoshop users was confused by this option. He expected to find mutliple sliders, e.g. one for each of the RGB values. The label 'threshold' does not tell the user, which threshold is specified. Also, the tooltip text 'Maximum Color Difference' provides little information on how this difference is calculated.

Suggestions:

  1. Re-implement the colour channel dialog.
  2. Provide more detailed information on the threshold's calculation method.
  3. If possible, provide additional sliders, e.g. for brightness or RGB values.

4.2 Issue: Removing the selection

Removing the selection was difficult for the image manipulation beginner. She repeatedly clicked into and outside of the image, but that did not work.

Suggestions:

  1. Provide a hint in the status bar 'To remove the selection, click 'Selection  None''.




5. Fill and stroke selections

5.1 Issue: Finding the tool

In order to complete Task 6, all non-GIMP users first scanned the toolbox for a tool to draw a circle. When they did not find it, they understood immediately that they would have to use the elliptic selection tool. But then, all users had problems to find the menu item 'Edit  Stroke selection' and 'Edit  Fill with xxx Colour', respectively.

The GIMP 1.2 user only found the 'fill' option after a hint. Instead, he used the 'bucket' tool to fill his selection. He stated that would have rather expected that function in the 'Selection' menu as it applied to the selection (which also manifested in his actions as he often scanned the 'Selection' menu).

One of the Photoshop users found the 'fill' option, but did not find 'stroke'. That was merely caused by the German label 'Auswahl nachziehen'.

Suggestions:

  1. Move the 'Fill' and 'Stroke' functions to the 'Selection' menu.
  2. Maybe provide a tool to draw basic shapes such as rectangles and circles.




5.2 Issue: Setting the stroke style

Especially the image manipulation beginner was confused by the numerous options in the stroke style dialog.

None of the other users understood the use of the 'Miter-Limit' option. Even when they tried it, it did not become clear.

The concept of the dialog to set a user-defined line pattern is very good and intuitive. Only, it is a bit confusing that the pattern and the preview are displayed in the same field, and that the pattern preview is split up by the area were settings are made.

One user wondered why the pulldown menu to select pre-defined line styles does only name the styles, but not show what they look like.

Suggestions:

  1. Where necessary, add explanatory tooltip texts to the option labels ('Miter').
  2. To define a line style, split the 'setting' field (which may be manipulated) from the 'preview'. If twelfe units is the maximum number of user-defined patterns, do not show more than that in the setting field. The preview should be double or three-times the size of the setting field.
  3. In the pulldown menu, rather provide previews of the line style than labels, or provide both.




6. Measure tool

6.1 Issue: Handling the tool

In Task 4, the users had to measure the width of a rectangular. None had severe problems, but the handling could be more comfortable with respect to some points.

As the default setting, the info dialog is not shown. Three out of four users did not see the info in the status line, and therefore first had to search where the information is displayed. However, they found the checkbox to open the dialog quite quickly.

Two users stated that it is quite hard to determine if the measure line is straight or if it is not. They missed an opportunity to specify a certain angle.

Two users missed an opportunity to set the unit of the distance. Even if pixel is probably the most common unit, sometimes it might be useful to determine distances other than pixel or zoll.

Suggestions:

  1. As the default setting, the info dialog should be displayed.
  2. Communicate the different modes and the corresponding shortcuts. E.g. Introduce mode buttons (straight line, addition mode etc.) and provide the shortcuts in the tooltip texts.
  3. In the tool options dialog, provide an input field to determine the measure angle.
  4. In the toll options dialog, provide a field to set the measure unit.




7. Scaling a selection or a layer

7.1 Issue: Finding the tool

Except for the image manipulation beginner, none of the users had problems to find the tool in the toolbox. The reason why the image manipulation beginner did not find it is due to a wording problem in the German translation, which will be discussed in the 'German Translations' section.

7.2 Issue: Handling the tool

When selecting the 'scale' tool in the toolbox, the mouse pointer does not change its shape until it is moved over the current selection. Also, the info dialog only shows up if the user clicks the image. That behaviour confused the image manipulation beginner, so she waited three minutes for a dialog to show up. As nothing happened, she clicked into the image and could finally start her task.

All users liked to adjust the selection/layer size by dragging the grid. Only the GIMP 1.2 user said that he would prefer real preview, as he could not see the actual layer size. Also, he would appreciate if the dialog would stay open after confirmation, in order to facilitate further manipulations or a 'reset'.

Also, the GIMP 1.2 user missed shortcuts to perform a centered and a squared scaling.

Suggestions:

  1. When selecting the 'scale' tool, immediately open the dialog.
  2. If there are shortcuts to perform a centered scaling, communicate them in tooltips or in the status line.
  3. Keep the dialog open after confirmation. This applies to all transformation tools (including 'crop' and the like).




8. Text tool

8.1 Issue: Modifying the text size

Two users first tried to modify the text size in the editor by clicking the icon for vertical alignment. When they realised that this was not correct, they found the font size in the tool options dialog.

Two users realised that it is possible to adjust the font size dynamically, or even later when selecting the corresponding layer. The third user did not see that option, and created a new text layer with an adequate text size. The image manipulation beginner used the 'scale' tool to increase the text size.

Suggestions:

  1. If possible include two buttons into the text editor: One to increase the font size by five pixel, and one to decrease it.

8.2 Issue: Moving a text field

All users had severe problems to move a text field. Even if they realised that they had to select the corresponding layer and the 'move' tool, it is very hard to find the right location to grap the text. It happened very often that users moved the background instead of the text.

Suggestions:

  1. Facilitate the grapping and moving of text fields.

8.3 Issue: Changing the text colour

To change the text colour, the user can either use the 'Color' field in the text tool options, or he can change the colour in the toolbox as soon as the text tool is selected. If the user first changes the colour in the toolbox, and selects the text tool afterwards, the colour in the toolbox is overwritten by the lastly utilised text colour. Due to that, two users lost the colour they had selected manually.

Suggestions:

  1. When selecting the text tool, apply the toolbox's foreground colour to the text tool.
  2. The button to change the colour in the text tool options does not make it clear that it is clickable. It rather looks like a status field. Better provide a status field showing the selected colour, and a clearly marked button to change the colour beside it.




9 Journal

9.1 Issue: Finding the Journal

The location of the journal next to the undo/redo menu items supported the users in finding the function immediately.

9.2 Issue: Handling the Journal

Unfortunately, none of the user could handle the journal successfully. Either, the steps in question were not displayed any more, as an insufficient number of steps was remembered. Or, the steps were reported, but the users did not understand the functionality of the journal.

Actually, the journal provides a list of snapshots documenting the manipulation history, and the user can return to a certain state of his image. What confused the users most were the 'undo' and 'redo' buttons underneath the list. They invoked the impression, that single steps might be deleted and redone, respectively. One user was even searching for a trash bin to delete single steps. However, deleting single steps is not possible, the user can only move back and forward in the manipulation history.

Suggestions:

  1. Remove the 'redo' and 'undo' buttons below the journal list. Then, the user will navigate through the list by clicking the single steps, which is way more intuitive than the buttons.





10. Clone tool

10.1 Issue: Defining a source

The GIMP 1.2 user knew the tool and therefore had no problems to use it. The Photoshop users instead got confused at first by the stop symbol that was displayed next to the mouse pointer. In order to define a source, the user first has to press the 'ctrl' key  which was unknown to the Photoshop users.

Therefore, they tried several techniques in order to define a source. They tried to mark an area by dragging the mouse around or above it, or tried to mark the edges. One user finally used tried the 'ctrl' key, while the did not find it without a hint. Even after the hint that user was confused and tried to mark a whole area.

Suggestions:

  1. Provide a hint in the status bar 'Create a reference point by pressing CTRL'.





11. Path tool

11.1 Issue: Deleting points

Only the image manipulation experts had to perform this task. All the three of them had problems to erase points once they had drawn them.

In order to delete points, the user first has to change from the default 'design' mode to the 'edit' mode. Then, he has to press the 'shift' key and klick the point that should be removed.

All users first tried to utilise the 'delete' key in 'design' mode, and found the correct way to remove points only after extensive exploration.

Suggestions:

  1. Re-organise the three modes 'Design', 'Edit', 'Move': 'Design' stays the same (adds points, forms the shapes), 'Edit' is renamed to 'Remove' (removes points, corresponds to the delete key or 'shift'), 'Move' stays the same (corresponds to the 'alt' key). Apply buttons for each mode, and add tooltips that communicate the shortcuts.
  2. Allow the delete key to delete points.

11.2 Issue: Connecting start and end points

To connect start and end points, the 'ctrl' key has to be pressed in 'design' mode. Then, the 'intersection' symbol is shown next to the mouse pointer.

While the GIMP 1.2 and one Photoshop user had no problems to find the shortcut, the other user had severe problems. He was sure, that it was not possible to connect start and end points with the GIMP.

Suggestions:

  1. Provide a hint in the status bar 'To connect start and end point, press CTRL'.

11.3 Issue: Shaping the path

To form the path's shape was rather easy for the users. Both, dragging the curves themselves and the handling of the guides was quite intuitive  except for one user, who could not properly navigate with the guides. For him, they were too sensitive, so he turned them several times around their own axis.


11.4 Issue: Stroking the path

After setting the path properly, the users stroke the path. While two of them found the option immediately  either in the 'Edit' menu or in the tool option dialog itself, one user had severe problems.

In his case, other image elements were selected, so he tried to use 'stroke selection' rather than 'stroke path'. After looking a while through the menus, he found 'stroke path' in the tool options. He didn't try a right click on the path dialog, and for some reason did not see the menu item 'Edit  Stroke path'.

One user stated that 'Selection from Path' is missing in the tool option dialog.

Suggestions:

  1. Add a'Selection from Path' button to the tool option dialog.




12. Docks and dialogs

12.1 Issue: Arranging the docks

None of the users had problems to rearrange docks and dialogs. All users appreciated the concept of containers in order to facilitate the control of dialogs and to save screen space.

Interestingly, the docking showed that it is very important to provide multiple ways to complete one goal. While some users used drag and drop to rearrange the dialogs, others did not even think of that opportunity. They used the tab menus instead and would not have been able to add new tabs to a dialog if there would have been no menu.

The drag and drop users instead did not even realise the small buttons to open the menu and close the tab. This might be due to their shape and position: They rather look as if they belonged to the tab panel than to the header of the tab.

Suggestion:

  1. Move the tab's 'menu' and 'close' buttons a bit further to the top right edge of the tab, and closer to each other. So it will become more obvious that they refer to the tab and not to its contents.




13. Other Issues

13.1 Issue: Marking selections

The GIMP 1.2 user repeatedly stated that he had problems to identify the current selection, especially in case of colour or composed selections. To make it more significant, he suggested to colour the selection in a certain way, or to shape the non-selected parts in a light grey. However, if that should be implemented, it must be taken care that the colour does not decrease the visibility of the image.

13.2 Issue: Previews

Windows users often expected a preview of what they did. E.g. when making a rectangular selection and changing the fixed aspect ratio, they expected that the newly entered attributes were applied to the current selection simply by clicking return. They did not understand that the attributes would just be applied to a new selection.

The same happened when one of the Phtotshop users wanted to select a color. He first used the 'fuzzy selection' tool to select a color, and then chose 'Select by color' in hope, the function would apply his selected color to the whole image.




14. Issues of the German Translation

14.1 Selection Tool

Suggestions:

  1. 'Auswahl schneiden' was not clear. Better: 'Schnittmenge bilden'.

14.2 Selection Menu

Suggestions:

  1. 'Auswahl  Nichts' was not clear. Better: 'Auswahl  Aufheben'.

14.3 Scaling

The image manipulation newbie was looking for a tool labeld 'Vergroessern' ('resize') or 'Groesse veraendern' ('modifying the size'). As she did not find anything in the menus, she scanned the tooltips in the toolbox. There, the zoom tool is unfortunately labeled as 'Vergr–ssern/Verkleinern' ('increase/decrease size') in the German version. The user selected that tool and did not understand why it did not increase the size of the current selection, but of the whole image.

Suggestions:

  1. Tooltip for the zoom tool should be 'Ansicht vergroessern/verkleinern' or 'Hineinzoomen/Herauszoomen'.
  2. Tooltip for the scale tool should stay the same.


overview


0 Summary
1 General aspects
2 Zooming
3 Selections
4 Selection by colour
5 Fill and stroke
6 Measure tool
7 Scaling
8 Text tool
9 Journal
10 Clone tool
11 Path tool
12 Docks and dialogs
13 Other issues
14 German translation

mailto: ellen reitmayr or roman joost