Adaptive User Interfaces have been a subject of research since 1985. No existing research has attempted to draw the user's attention towards changes in the adaptive elements. An adaptive toolbar was implemented in a simulated text editor application. Feedback was used to inform the participant of updates in the adaptive toolbar. The effect of this feedback was tested using an eye-gaze tracker. It was found that feedback affected both the toolbar used by the participants to solve tasks, and in which toolbar the participants first sought the solution.